Lack of evidence for interventions offered in UK fertility centresBMJ 2016; 355 doi: https://doi.org/10.1136/bmj.i6295 (Published 28 November 2016) Cite this as: BMJ 2016;355:i6295
All rapid responses
To The Editor, BMJ
Dear Dr Godlee
Balen et al have responded to our papers. [1,2] In our BMJ open paper we systematically identified 276 claims of benefit relating to 41 different fertility interventions and 16 published references were cited 21 times on 13 of the 74 websites we searched. In our BMJ analysis paper, we systematically examined the evidence for ‘38 additional fertility interventions’ and found evidence of improvements in live birth rates for only five interventions. We used standard critical appraisal techniques (as detailed in our paper) to explore the quality of these studies. We identified that for all five of these interventions the studies had methodological problems, raising uncertainty about the significance of the results. This is a serious issue for patients, public health, public trust, and regulators.
Balen et al suggest that our evidence review included - and found evidence lacking for - things which are not “add-ons”. We classified ‘add-ons’ in the BMJ analysis article in response to peer review where the issue was discussed (see online peer review comments).
Peer review comment: ‘The methodology has led them to five interventions for which there may be some evidence of improvement in live birth rates. One of these is intrauterine insemination in a natural cycle. This is an alternative treatment, rather than an “add-on”, and usually not one applied to the population who require IVF, and certainly not one intended to increase live birth rate.’
Our response: ‘We have also added a further table 1: Classifications of IVF Interventions: This is the only reference in the HFEA we could find on the words ‘add-on’ which was only discussed at the Feb 2016 meeting: 'The committee discussed which techniques are more common or established and may be incorporated into other sections of the website. The committee agreed that the remaining techniques for discussion could be placed onto a page about ‘Treatment add-ons’. And we note there is no current definition of what is an ‘Add-on’ treatment as it very much depends on what [the] clinic offers.'
We, therefore, classified 27 of these as ‘Add-on’ interventions: six as alternative treatments to IVF and five treatments for the preservation of fertility (see Table 1 in the paper). We, therefore, followed the advice of peer review. By ‘we’ we mean two general practitioners adjudicated and all authors confirmed the classifcation. Of note, there were concerns that we are not fit to do this. However, the HFEA’s advice is that when it comes to IVF “you should talk to your GP to go through the options available”.
Also in response to peer review, we added to our conclusion the following: ‘Clinics vary significantly in what they offer and there is a lack of clarity on what actually constitutes an ‘Add-on’ intervention. Clear classification systems could aid understanding and reduce variation in what is offered to couples.’
I also draw your attention to the HFEA Press statement at the time that did not question our classification. HFEA statement on fertility treatment 'add-ons' 28 November 2016 (http://www.hfea.gov.uk/10541.html)
HFEA official response to BBC Panorama regarding fertility treatment 'add-ons'.
‘UK fertility clinics generally provide excellent patient care. However, an increasing number of patients tell us they feel unsure about IVF treatment add-ons – whether they work and are worth the extra cost. This puts pressure on patients to make difficult decisions at what is already a stressful time for them.
'We are concerned about the recent step change in the use of treatment add-ons. Unfortunately, we have limited powers to stop clinics offering them, nor to control pricing. Instead, we publish information directly for patients, so that they can have the facts at their fingertips before they go to a clinic and are able to make informed decisions and properly consent to treatment.
'With expert advice from our scientific committee, we have for some years been publishing information about add-ons such as reproductive immunology and PGS, making it clear whether or not they are proven to be effective. We are also working on producing information for our new website about a wider range of add-ons, introducing a simple labeling system to highlight their levels of effectiveness and potential harm.’
Balen et al, therefore, appear to disagree with our categorisation of the 27 interventions as add-ons (see Box 2 of the paper).  This is a matter for reasonable debate, but more importantly suggests a lack of outward clarity to many users of this information, notably patients. We stand by our methods (we searched systematically for a classification system at the outset), and to the best of our knowledge, The BMJ is the only journal to have a published classification system.
Regardless, of whether Balen et al prefer to categorise interventions as “add-ons” or part of core treatment, this does not affect our conclusion: there is little or no evidence supporting interventions available from UK fertility clinics to improve live birth rate. We note that this is not a conclusion that is directly challenged by Balen et al.
Balen et al criticise our research team on the grounds that we are evidence based medicine researchers, clinicians, and epidemiologists, not “specialists in reproductive medicine”. This would seemingly seem an unusual line of argument. Given the known concerns over potential conflicts of interests in the IVF industry, and the extensive list of conflicts highlighted at the end of Balen et al’s letter, it was paramount that this work was conducted by an independent group with no ties to the IVF industry.  That said, should the opportunity arise for further work assessing the quality of evidence for fertility interventions we would be happy to consider collaborating with co-signatories to improve the evidence-base.
Balen et al appear to object to our use of “live birth” as the core metric of success. Once again, this appears to be an unusual line of argument to take. We refer Balen et al to our recent paper in Trials Journal entitled “Why clinical trial outcomes fail to translate into benefits for patients”.  In this paper we highlight some of the issues with substantial amounts of clinical research utilising surrogate, composite and subjective endpoints and failing to take account of patients’ perspectives when designing research outcomes. We agree with Barnhart, who highlights the need for the reproductive industry to make live births as the correct, patient focused, outcome for trials evaluating therapies for infertility.  We were therefore pleased to see that live birth rates are being used as a key outcome by RCTs currently recruiting patients in the UK.
Balen et al say that the sector has been well regulated by the HFEA. We question the strength of their argument to support this. Following the publication of our paper, we were pleased to note that the HFEA themselves have committed to updating their evidence to ‘provide free, clear and impartial information to all affected by assisted reproduction,’ and to provide clear evidence-based information to patients and the public as stated in their 2017 innovation and regulation plan.  We also noted that the Director of Strategy at the HFEA, Juliet Tizzard, publicly stated, in reference to our work, that the HFEA “don't regulate this but do care deeply.” 
Balen et al also comment that we highlight the intervention “Intralipid”, which they suggest is not widely used. This suggestion is not in keeping with our systematic findings. Our analysis stemmed from identification of claims of benefit on fertility centre websites, and we found three such instances claiming benefits for Intralipid; however, Intralipid is offered more widely (four offer Intralipid without stating a claim benefit on the website), meaning approximately 1 in 10 of the clinics we searched offer this intervention. We highlighted our concern for the widespread advocacy of Intralipid, especially given the lack of evidence for this intervention, as noted by both the BFS and Royal College of Obstetricians and Gynaecologists. 
Balen et al state that ‘many of the items have a clearly defined role in specific situations; e.g. for a man with a physical blockage sperm has to be extracted surgically, frozen and used in an IVF cycle combined with intra-cytoplasmic sperm injection (ICSI).’ We agree and ICSI was not categorised as an ‘add-on’ treatment in our analysis; we show to whom these interventions apply according to NICE guidance in our data supplement.  Balen et al say that looking for improved outcomes is irrelevant for this group of patients because ‘without those interventions (which are clearly not “add-ons”) IVF cannot happen.’ NICE, however, in their research recommendations (45 guideline CG156) state the need to ‘determine the long term effects of IVF with or without ICSI in children in the UK.’ 
We were pleased to note that Balen et al recognise that the evidence for some interventions is poor. We note that since the publication of our paper the evidence continues to evolve: a new systematic review on time-lapse imaging reported that there is insufficient evidence to support time-lapse imaging compared to conventional methods and there is a need for ‘more well-designed RCTs’  ; a further systematic review, published in May 2017, on strategies to improve fertilisation rates with assisted conception, further stated that ‘adequately powered multicentre randomised trials are required to evaluate these techniques before considering clinical application.’ 
We note that Balen et al have said there are shortcomings in our assessments of the quality of evidence for these interventions improving patients’ chances. If the authors can now provide these, we will happily review these and engage in constructive discussion on the emerging findings.
Finally, the use of ad hominem comments provides little benefit to anyone in this debate. Although we recognise the sensitivities and emotions that our work may have evoked amongst the IVF community, and despite widespread concerns over conflicts of interests in the IVF field, we would urge Balen et al to refocus their concerns at the IVF industry itself, and take a lead in utilising their extensive resources to reassure end users, the HFEA and the wider scientific community that their industry’s work is genuinely patient focused and evidence based. We would further suggest that the first step towards this would be to engage with the media, patient groups, policymakers and regulators to address the systemic findings that many interventions that lack good evidence to back them up are being offered to patients.
Carl Heneghan, Ben Goldacre, Elizabeth A Spencer, Jack O’Sullivan, Kamal R Mahtani,
 Spencer EA, Mahtani KR, Goldacre B, et al Claims for fertility interventions: a systematic assessment of statements on UK fertility centre websites. BMJ Open 2016;6:e013940. doi: 10.1136/bmjopen-2016-013940.
 Heneghan Carl, Spencer E A, Bobrovitz N, Collins D R J, Nunan D, Plüddemann A et al. Lack of evidence for interventions offered in UK fertility centres BMJ 2016; 355 :i6295
 Don’t dismiss conflict-of-interest concerns in IVF, they have a basis
 Carl Heneghan, Ben Goldacre, Kamal R. Mahtani. Why clinical trial outcomes fail to translate into benefits for patients. Trials, 2017, Volume 18, Number 1, Page 1
 Kurt T. Barnhart. Live Birth is the Correct Outcome for Clinical Trials Evaluating Therapy for the Infertile Couple Fertil Steril. 2014 May; 101(5): 1205–1208.
 HFEA: Innovation in regulation Feb 2017. http://ifqtesting.blob.core.windows.net/umbraco-website/1796/innovation-...
 Fertility problems: assessment and treatment. Clinical guideline [CG156] Published date: February 2013 Last updated: August 2016 https://www.nice.org.uk/guidance/cg156
 Does time-lapse imaging have favorable results for embryo incubation and selection compared with conventional methods in clinical in vitro fertilization? A meta-analysis and systematic review of randomized controlled trials. Chen M, Wei S, Hu J, Yuan J, Liu F.PLoS One. 2017 Jun 1;12(6):e0178720.
 Strategies to improve fertilisation rates with assisted conception: a systematic review. Jeve YB, Potdar N, Blower JA, Gelbaya T. Hum Fertil (Camb). 2017 May 26:1-19. doi: 10.1080/14647273.2017.1324182.
Competing interests: We have read and understood BMJ policy on declaration of interests and declare the following: CH has received grant funding from WHO, the National Institute for Health Research (NIHR), and the NIHR School of Primary Care and occasionally receives expenses for teaching evidence based medicine. He jointly runs the EvidenceLive Conference with BMJ, is editor in chief of the BMJ Evidence-Based Medicine Journal and receives payment as an NHS GP in urgent care. BG has received research funding from the Laura and John Arnold Foundation, the Wellcome Trust, the NIHR, the Health Foundation, and WHO; he also receives personal income from speaking and writing for lay audiences on the misuse of science. KM has received funding from the NIHR and the RCGP for independent research projects. The views expressed here are those of the authors and not necessarily those of any of the affiliated institutions mentioned in the manuscript.
14 June 2017
Professor of EBM
Ben Goldacre, Elizabeth A Spencer, Jack O'Sullivan, Kamal R Mahtani
Centre For Evidence-Based Medicine, Nuffield Department of Primary Care Health Scienes, University of Oxford
Radcliffe Observatory Quarter, Woodstock Road, Oxford. OX2 6GG
Competing interests: Competing interests: We have read and understood BMJ policy on declaration of interests and declare the following: CH has received grant funding from WHO, the National Institute for Health Research (NIHR), and the NIHR School of Primary Care and occasionally receives expenses for teaching evidence based medicine. He jointly runs the EvidenceLive Conference with BMJ, is Editor in Chief of the BMJ Evidence-Based Medicine Journal and receives payment as an NHS GP in urgent care. BG has received research funding from the Laura and John Arnold Foundation, the Wellcome Trust, the NIHR, the Health Foundation, and WHO; he also receives personal income from speaking and writing for lay audiences on the misuse of science. KM has received funding from the NIHR and the RCGP for independent research projects. The views expressed here are those of the authors and not necessarily those of any of the affiliated institutions mentioned in the manuscript.
To The Editor, BMJ
Dear Dr Godlee
Re: Lack of evidence for interventions offered in UK fertility centres.
We are writing to express our concern regarding the papers by Heneghan et al, (2016) and Spencer et al (2016) owing to their lack of scientific robustness.
We would like to state at the outset that we oppose the provision of procedures or treatments that do not have a scientific basis and we welcome the initiative by the Human Fertilisation and Embryology Authority (HFEA), which has been endorsed by the British Fertility Society (BFS), to introduce a grading scheme for “add-ons”.
Spencer, Heneghan and colleagues have unfortunately obscured their important message by mixing various categories of treatment, not all of which come under the category of “add-ons”. Indeed, a number are accepted components of routine treatment. The papers have grouped together three categories of care: (i) necessary investigations (e.g. assessment of ovarian reserve, which is vital in determining correct dosage of ovarian stimulation drugs to optimise outcome and ensure patient safety), (ii) essential treatments (e.g. surgical sperm retrieval) and (iii) interventions that can be termed “add-ons” - namely an addition to a pathway of care, whether as an additional drug or therapeutic procedure. Many of the items identified have a clearly defined role in specific situations; e.g. for a man with a physical blockage sperm has to be extracted surgically, frozen and used in an IVF cycle combined with intra-cytoplasmic sperm injection (ICSI). For this group of patients, without those interventions (which are clearly not “add-ons”) IVF cannot happen and looking for improved outcomes is irrelevant. Therefore, the papers are inherently flawed as the basis upon which they are structured is clinically and scientifically unsound.
The BMJ disregarded key points made by one of the referees which should have alerted the authors to fundamental flaws in their enquiry: Quoting Dr Avery’s review “For this paper to have the intended impact, the authors need to decide whether to include all procedures for which centres are charging a fee, or whether as the title suggests, they are going to stick to add-ons, specifically those for which claims of increased live birth rates are made.……., in its current form it [the paper] could easily be dismissed by those it targets as having been written by those with no understanding of the field or the technology ….” It is clear that no specialists in reproductive medicine were involved which is why the authors misunderstood and misinterpreted their task. To quote one co-author this is a clear example of “Bad Science” (Goldacre, 2009).
We are fully aware there may be “add-ons” that are not evidence-based, either because the research has not yet been carried out, or because they have been found to be ineffective (Harper et al 2017). The use of intralipid is one such example. This has been clearly stated by the BFS (Nardo et al 2014) and Royal College of Obstetricians and Gynaecologists (RCOG 2016) and so it is perfectly legitimate to criticise a clinic for offering intralipid. However, when it is only 3 clinics (4%) offering this intervention on their websites, it should be clearly stated that this is not mainstream practice or representative of the whole sector.
Treatments that appear more frequently on websites, such as ICSI and blastocyst culture, and the “add-ons” such as embryoglue and endoscratch do have an evidence base. We can present an analysis of each of the 41 items which clarifies the evidence for many and against some. However, it is important to recognise that, although live birth rate is the key outcome measure recommended by the HFEA and something patients need to know, using live birth as the sole indicator of an evidence base oversimplifies a hugely complex process and fails to recognise the significant scientific research underlying decisions to bring treatments into clinical practice.
We are under no illusion that there are sections of the sector that are commercially driven and offering invalidated, costly “add-ons”. Clinics are inspected and called to account by the HFEA, resulting in greater regulation of the fertility sector than any other area of medicine. Furthermore, the HFEA inspectors examine all written information and consent forms that are given to patients, which will always be a truer reflection of clinical practice than the summaries on websites. In the conclusion of the publications, the authors’ own criteria were not fulfilled. The authors were asked “to carry out an independent review of the evidence for fertility treatments additional to IVF”, which they did not do. They only looked at websites and took no account of information provided to or understood by patients at consultations or in the clinics’ written literature. Whether patient literature should be referenced is an interesting concept, as not even that produced by NICE includes references. Nonetheless, we are absolutely clear that websites must provide accurate information that does not mislead patients.
Had the authors sought the input of specialists in reproductive medicine they would have been able to write scientifically sound papers and contributed to a debate (and associated work by the HFEA and BFS) already underway in the sector. To place an article in the public domain that is highly inaccurate and misrepresents the fertility sector is both misleading and deeply unhelpful to patients.
We strongly believe that these papers should not have been published in their current form. Whilst we welcome scrutiny that contributes accurate analysis and informs the mechanisms by which future advances should be brought into clinical practice, our concern is that this should be conducted in a scientifically robust manner that provides clarity to the topic. Given the rapid advances in reproductive medicine, such work is vital and contributes to the natural evolution of the sector’s regulatory machinery. It is important that we work together to promote ethical and innovative patient care.
The key objectives of the BFS are to promote high quality clinical practice and high quality research. To this end, we would be happy to collaborate with the authors to ensure that their research results in valid conclusions.
Adam Balen, MD, DSc, FRCOG.
Professor of Reproductive Medicine and Surgery, Leeds &
Chair of The British Fertility Society (BFS).
Lesley Regan, MD DSc FRCOG FACOG
President of the RCOG,
Professor& Head of Department of Obstetrics and Gynaecology, Imperial College & St Mary’s Hospital,London
Sue Avery PhD FRCPath PGDipLaw - referee of the original paper.
Consultant Embryologist & Director of Birmingham Women’s Fertility Centre.
Peter Braude OBE, FRCOG, FRSB, FMedSci
Emeritus Professor of Obstetrics and Gynaecology, King's College London
Ian Cooke FMedSci, FRCOG, FRANZCOG (Hon.)
Former Chairman, former President of the British Fertility Society and current Chairman of Trustees
Director of Education, International Federation of Fertility Societies, 2004-2011
Emeritus Professor, University of Sheffield
Grace Dugdale BSc, MA.
Reproductive biologist, London
Susan Seenan, Chief Executive, Fertility Network UK
(The UK support organisation for people with infertility)
Yacoub Khalaf MSc MD FRCOG
Consultant Gynaecologist & Sub-Specialist in Reproductive Medicine and Surgery,
Director of the Assisted Conception Unit & Centre for Pre-implantation Genetic Diagnosis, Guys & St Thomas’ Hospital, London
Member of The HFEA.
Anthony J Rutherford FRCOG
Consultant in Reproductive Medicine & Gynaecological Surgery, Leeds.
Past Chair of The British Fertility Society.
Member of The HFEA.
Peter R. Brinsden MB BS FRCOG
Past-President of the British Fertility Society
Former Medical Director Bourn Hall clinic, Cambridge.
Tim Child MA MD MRCOG
Associate Professor, University of Oxford
Medical Director, Oxford Fertility.
Henry Leese PhD, FRCOG
Professor Emeritus Reproductive Biology, Hull York Medical School
President of The British Fertility Society.
Alison Murdoch MD, FRCOG
Professor of Reproductive Medicine, Newcastle &
Past Chair of The British Fertility Society.
Allan Pacey MBE, PhD, FRCOG ,
Professor of Andrology, Sheffield University.
Past Chair of The British Fertility Society.
Mark Hamilton MD FRCOG
Consultant in Reproductive Medicine, Aberdeen Maternity Hospital.
Past Chair of The British Fertility Society.
David Adamson, MD, FRCSC, FACOG, FACS
Chair, International Committee Monitoring Assisted Reproduction Technology
Past President, American Society for Reproductive Medicine
Bart CJM Fauser, MD, PhD,
Professor of Reproductive Medicine & Gynecology, Utrecht, The Netherlands
Chair WHO Global Guidelines Taskforce for the Management of Infertility.
And (in alphabetical order)
Hossam Abdalla MD, FRCOG
Director & Person Responsible of the Lister Fertility Clinic, London
Past Member of The HFEA.
Ian Aird MB ChB, FRCOG
Consultant Gynaecologist and Person Responsible
Gateshead Fertility Unit, Queen Elizabeth Hospital Gateshead
Valentine Akande PhD MRCOG
Lead Clinician & Person Responsible
Bristol Centre for Reproductive Medicine
Richard A Anderson PhD, FRCOG
Elsie Inglis Professor of Clinical Reproductive Sciences
Head of Section, Obstetrics and Gynaecology, University of Edinburgh,
MRC Centre for Reproductive Health
Harish Bhandari MD MRCOG
Subspeciality Trainee in Reproductive Medicine, Newcastle
N. Ellissa Baskind MD MRCOG
Subspeciality Trainee in Reproductive Medicine, Leeds Teaching Hospitals
Joshua D Blazek PhD
CooperGenomics, Genesis Genetics
Virginia N Bolton MA, PhD
Consultant Embryologist, Honorary Senior Lecturer, KCL
Assisted Conception Unit, Guy's & St Thomas’ Hospital,
Treasurer of The British Fertility Society
Rachel Cutting MBE, MSc
Consultant Embryologist, Sheffield University Hospitals
Former Chair of The Association of Clinical Embryologists
Dr Melanie Davies MA MRCP FRCOG
Consultant in Reproductive Medicine, HFEA Person Responsible, University College London Hospitals
C Janine Elson MD, FRCOG,
Consultant Gynaecologist and Subspecialist in Reproductive Medicine and Surgery,
Associate Medical Director CARE Fertility Group
Professor Simon Fishel PhD (Cantab) FRSB
Founder, President and Head of R&D for CARE Fertility.
Darren K Griffin PhD, DSc, FRSA, FRSB, FRCPath
Professor of Genetics, School of Biosciences, University of Kent
President of the International Chromosome and Genome Society
Jamie Grifo MD PhD
Professor and Director New York University Fertility Center, New York, USA
Stephen Harbottle PhD
Consultant Clinical Embryologist,
HFEA Person Responsible, Cambridge University Fertility Centre
Chair of the Association of Clinical Embryologists
James Hopkisson MBBS MD MRCOG
Associate Professor, & Sub Specialist in Reproductive Medicine, Nottingham University Hospital; Person Responsible Nurture Fertility
Kanna Jayaprakasan MD MRCOG PhD
Hon. Associate Professor & Subspecialist in Reproductive Medicine,
HFEA Person Responsible, Derby Fertility Unit, Royal Derby Hospital, Derby.
Jason Kasraie PhD
Consultant Embryologist & Andrologist
HFEA Person Responsible, Shropshire & Mid-Wales Fertility Centre
Chair Elect of the Association of Clinical Embryologists
Charles Kingsland MD FRCOG
Professor of Reproductive Medicine
Liverpool Women's Hospital
Jackson C Kirkman-Brown MBE PhD,
Reader & NIHR/HEE Senior Clinical Fellow in Healthcare Science, College of Medical & Dental Sciences, The University of Birmingham.
Stuart Lavery MBBCh MSc MRCOG
Consultant Gynaecologist, Person Responsible IVF Unit, Hammersmith Hospital
Honorary Senior Lecturer, Imperial College, London.
Christine Leary BSc, PhD, FRCPath
The Hull IVF Unit, Hull Royal Infirmary
Sheena E. M. Lewis BSc PhD CBiol FRSB
Emeritus Professor Andrology,
Queen's University Belfast,
Chair British Andrology Society,
Member of HFEA SCAAC Committee
Gillian Lockwood FRCOG DPhil MA (Oxon)
Medical Director, Midland Fertility
Jane MacDougall FRCOG MD MEd
Consultant Reproductive Medicine, Cambridge University Hospitals,
Head of School, EOE Postgraduate School Obstetrics & Gynaecology, Director of Studies Clinical Medicine, Newnham College Cambridge
Kevin McEleny PhD, FRCS (Urol)
Consultant Urological Surgeon & Andrologist, Newcastle
Alison McTavish RGN, RN
Manager Aberdeen Fertility Centre,
Past Secretary of BRITISH FERTILITY SOCIETY and Chair of Senior Infertility Nurses Group
Enda McVeigh FRCOG
Consultant in Reproductive Medicine, Oxford
Nick S. Macklon MD, FRCOG
Professor of Obstetrics and Gynaecology
University of Southampton
Scientific Director, Complete Fertility Centre.
Steve Maguiness MD, FRCOG
Consultant in Reproductive Medicine, Hull Royal Infirmary
Abha Maheshwari MBBS, MD, FRCOG
Person Responsible & Clinical Lead Reproductive Medicine, Aberdeen Fertility Centre
Raj Mathur MD, FRCOG
Clinical Lead for Reproductive Medicine, Central Manchester NHS Foundation Trust;
Hon. Senior Lecturer, University of Manchester
Dimitrios Mavrelos MD, MRCOG
Consultant in Reproductive Medicine, University College London Hospital
James Nicopoullos BSc MBBS MD MRCOG DFFP
Consultant Gynaecologist, Subspecialist in Reproductive Medicine & Surgery, Lister Fertility Clinic, London
David Polson MD FRCOG
Medical Director, Manchester Fertility
Nick Raine-Fenning PhD, MRCOG
Consultant in Reproductive Medicine & Associate Professor, Nottingham.
PD Dr. med. habil. Andreas G. Schmutzler
Gynecologist, Specialist in Reproductive Medicine and Lawyer, Lecturer
Kiel University for Gynecology, Reproductive Medicine and Ethics
Jane Stewart, MD, FRCOG.
Consultant in Reproductive Medicine, Newcastle
Secretary of The British Fertility Society
Isabel Traynor BA(hons) RGN,
Nurse Representative British Fertility Society.
Geoffrey H Trew FRCOG.
Consultant in Reproductive Medicine & Surgery, Hammersmith Hospital, London.
Attila Vereczkey, MD, MA
Medical Director of Versys Clinics, Human Reproduction Institute, Budapest Hungary
President of the Hungarian Human Reproduction Society
President of International Society of Periconceptional Medicine
Simon Wood MD, FRCOG.
Consultant in Reproductive Medicine, Countess of Chester NHS Foundation Trust
Ephia Yasmin MD, MRCOG
Consultant Obstetrician and Gynaecologist. Sub specialist in Reproductive Medicine.
Clinical Lead, Reproductive Medicine Unit University College London Hospital, London
Goldacre B. Bad Science by, 2009, Harper Perennial ISBN: 9780007284870.
Harper, J, Jackson, E, Sermon, K, Aitken RJ, Harbottle, S, Mocanu, E, Hardarson, T, Mathur, R, Viville, S, Vail, A, Lundin, K Adjuncts in the IVF laboratory: where is the evidence for ‘add-on’ interventions? Human Reproduction (2017 in press).
Heneghan C, Spencer EA, Bobrovitz N, Collins D R J, Nunan D, Plüddemann A, Gbinigie OA, Onakpoya I, O’Sullivan J, Rollinson A, Tompson A, Goldacre B, Mahtani K R. Lack of evidence for interventions offered in UK fertility centres. BMJ 2016; 355 doi: http://dx.doi.org/10.1136/bmj.i6295)
Nardo LG, El-Toukhy T, Stewart J, Balen AH, Potdar N. Adjuvants in IVF: evidence for good clinical practice. On behalf of the British Fertility Society P&P Committee. Human Fertiity, 2014; DOI:10.3109/14647273.2015.985454.
RCOG. The role of natural killer cells in human fertility. RCOG Scientific Impact Paper, number 52, November 2016.
Spencer EA, Mahtani K R, Goldacre B, Heneghan C. Claims for fertility interventions: a systematic assessment of statements on UK fertility centre websites. BMJ 2016; 355:i6295 doi:10.1136/bmj.6295.
Competing interests: Adam Balen: NHS Consultant in Reproductive Medicine and Clinical lead for the Leeds Centre for Reproductive Medicine, which performs all fertility treatments funded by the NHS. Partner in Genesis LLP, the private arm of the Leeds Centre for Reproductive Medicine, which performs self-funded fertility treatments using identical protocols to the NHS. Chair, British Fertility Society. Chair, NHS England IVF Pricing Development Expert Advisory Group. Chair, World Health Organisation Expert Working Group on Global Infertility Guidelines: Management of PCOS. Consultant for ad hoc advisory boards for Ferring Pharmaceuticals, Astra Zeneca, Merck Serono, Gideon Richter, Uteron Pharma. Research funding received in the past--Pharmasure / IBSA (Key note lecture at ESHRE 2016 & hospitality to attend meetings); OvaScience (Member of international ethics committee);Clear Blue (National medical advisory board); IVI, UK (Chair, Clinical Board). Raj Mathur: I have an NHS and Private practice where I see patients undergoing assisted conception treatment. Anthony J Rutherford: Consultant in Reproductive Medicine, working in a combined NHS & Private IVF unit at Leeds Teaching Hospitals NHS Trust, and Medical Director & Personal Responsible of a private IVF unit, IVI Wimpole Street, London. • Clinical Member of the Human Fertilization & Embryology Authority • My wife works as a Clinic Development Manager for Pharmasure, a Pharmaceutical Company involved in Reproductive Medicine. Harish Bhandari: Junior Clinicians' representative on the British Fertility Society Executive Committee and a co-opted member on the British Fertility Society's Training Sub-Committee. Darren K Griffin: Treasurer of the Preimplantation Genetic Diagnosis International Society (PGDIS) and holder of 2 grants part sponsored by the London Women’s Clinic and Genesis Genetics to investigate PGS. James Hopkisson: Consultant in Reproductive Medicine, working in an NHS Hospital Trust, as a clinical academic. Medical Director & Person Responsible to the HFEA and CQC of a private IVF unit NURTURE Fertility that provides treatment to both NHS and self funding patients. I am a shareholder in NURTURE LLP and Burton Fertility. Susan Seenan: Employed as CE of patient charity Fertility Network UK. Ian Aird: Clinical member British Fertility Society. I have received sponsorship from pharmaceutical and reproductive science industry to cover travel and accommodation costs to attend scientific meetings and symposia. Geoffrey H Trew: Consultant in Reproductive Medicine Imperial College NHS Trust. Director TFP. Jason Kasraie: 1) Consultant Embryologist & Andrologist (Clinical Scientist) and HFEA Person Responsible at the Shropshire & Mid-Wales Fertility Centre, Shrewsbury & Telford Hospitals NHS Trust. 2) Chair of the Association of Clinical Embryologists. Allan Pacey: Principal applicant of research grant awards from the Medical Research Council and the National Institute for Health Research (with funds awarded to the University of Sheffield). Engaged as a consultant by the British Broadcasting Corporation, Merck Serono Limited, Swiss Precision Diagnostics Development Company, PZ Media, Purple Orchid Pharma Ltd., and the World Health Organisation (with all monies and expenses paid to the University of Sheffield). Unpaid roles as: (i) Editor in Chief of Human Fertility; (ii) Chairman of the Andrology Specialist Advisory Group of theUnited Kingdom National External Quality Assessment Service (UK NEQAS); and (iii) trustee of the Progress Educational Trust. Nick S Macklon: Non-executive director, Anecova. Jane Stewart: Full time NHS consultant in reproductive medicine. Our unit also treats self-funded patients. Honorary secretary, British Fertility Society. Simon Fishel: Shareholder in CARE Fertility. Ian Cooke: Consultant to the WHO. Chair of the Trustees of the British Fertility Society. Grace Dugdale: Temporary freelance work for the British Fertility Society; Developing educational resources Oct 2016-present. Sheena E M Lewis: Professor Emeritus of Reproductive Medicine, Queen’s University Belfast. Chair, British Andrology Society. National representative for UK, European Society Human Reproduction and Embryology. Recipient of funding from NIHR and Marie Curie Initial Training Network. Member of HFEA SCAAC. Ad Hoc speaker for Science Media Centre. Director of University spin out company, SpermComet Ltd. Ephia Yasmin: I am a Consultant Obstetrician and Gynaecologist at University College London Hospital. I work as a consultant at IVI-UK. I am on the scientific advisory board of Ferring Pharmaceuticals. Bart C J M Fauser: During the most recent 5 year period B.C.J.M.F. has received fees and grant support from (in alphabetic order); Actavis/Watson/Uteron, Controversies in Obstetrics & Gynecology (COGI), Dutch Heart Foundation, Dutch Medical Research Counsel (ZonMW), Euroscreen, Ferring, London Womens Clinic (LWC), Merck Serono (GFI), Netherland Genomic Initiative (NGI), OvaScience, Pantharei Bioscience, PregLem/Gedeon Richter/Finox, RBM Online, Roche, Teva, World Health Organisation (WHO). C Janine Elson: Paid member of Finox Advisory Board. Received travel and accommodation expenses from Pharmasure. Joshua D Blazek: I am employed by industry and am a member of PGDIS, COGEN, and ASRM. All other signatories: None declared.
Aiyenigba and Weeks [Aiyenigba rapid response] appropriately highlight the social and psychological aspects of fertility treatments, with particular reference to sub-Saharan Africa. The authors describe the need to involve patients first and foremost in determining health service provision, whilst providing evidence to help decision-making - the practice of evidence based healthcare. They underline our point that, whilst we found high variability and lack of evidence provision in fertility services offered in the UK, the situation is likely to be worse where there is less regulation  .
Bahadur and colleagues [ref Bahadur rapid response] draw attention to the current confusion in guidance and shortcomings in current NICE guidelines. We agree that guidelines should be based on the best evidence. NICE’s remit is to provide guidance and advice to improve health and social care; faced with the rapidly increasing array of fertility interventions offered, this task is becoming more difficult. NICE could help by prominently flagging evidence gaps, and setting out clearly how they could be addressed. Whilst HFEA may not have that same remit, they are a port of call for many patients seeking advice with assisted reproduction. We view the news that the HFEA are updating the information provided to patients via their website as an important step to maximising the use of evidence and guidance.
Griffin and colleagues’ support for evidence-based medicine in fertility services is to be welcomed [Griffin rapid response]. Researching interventions in assisted reproduction may be challenging, but is not impossible; the IDEAL collaboration has been grappling with similar problems in the field of surgery, and their framework has relevance to improving the evidence for fertility interventions.  Griffin et al suggest there is already sufficient evidence available on PGS: within the data, however, only one trial  reports on live birth, the key relevant outcome. Clearly more evidence is warranted on PGS, and results of ongoing trials will be important to determining its safety and applicability. As our research showed, for many other interventions the evidence is even poorer. There is a clear need for urgent action to reduce uncertainty for patients. As with any other area of healthcare, pragmatic trials should be embedded in practice such that research continues to provide evidence on the benefits and harms of interventions.
Fertility treatments may be the “canary in the cage” for wider future challenges around evidence based practice. If healthcare services continue to fracture from the NHS then patients could increasingly find themselves trying to make informed decisions in a context of highly variable services, lack of clear guidance on existing evidence, and conflict of interest from competing providers, all compounded by the additional administrative and regulatory barriers to effective multi-centre research that are likely with a patchwork of private providers. Fixing the shortcomings in evidence on fertility treatments now may help us prepare better for these future challenges.
Carl Heneghan, Elizabeth Spencer and Ben Goldacre.
1 Heneghan C, Spencer EA, Bobrovitz N, et al. Lack of evidence for interventions offered in UK fertility centres. BMJ 2016;355:i6295.
2 Spencer EA, Mahtani KR, Goldacre B, et al. Claims for fertility interventions: a systematic assessment of statements on UK fertility centre websites. BMJ Open 2016;6:e013940.
3 Scott RT Jr, Upham KM, Forman EJ, et al. Blastocyst biopsy with comprehensive chromosome screening and fresh embryo transfer significantly increases in vitro fertilization implantation and delivery rates: a randomized controlled trial. Fertil Steril 2013;100:697–703.
4 McCulloch P, Altman DG, Campbell WB, et al. No surgical innovation without evaluation: the IDEAL recommendations. Lancet 2009;374:1105–12.
Competing interests: EAS has no competing interests. CH has received expenses from the WHO and holds grant funding from the NIHR, the NIHR School of Primary Care Research and the WHO and is part of the IDEAL collbaration. BG has received research funding from the Wellcome Trust, the NIHR School of Primary Care Research, the Laura and John Arnold Foundation, NHS England and the Health Foundation; he receives personal income from speaking and writing for lay audiences on problems in science.
Continuing to deliver: The evidence base for PGS
We read the BMJ manuscript by Heneghan and co-workers  - a study commissioned by the BBC - and watched the subsequent Panorama programme made by BMJ associate editor Deborah Cohen, with considerable interest.
The purpose of this letter is to respond specifically to the comments made about preimplantation genetic screening (PGS) also known as preimplantation genetic for aneuploidy (PGD-A), one of the three so-called “add on” treatments highlighted in the programme and among the 41 in the manuscript. We would like to draw the reader to the current extensive evidence base that supports the efficacy of PGS/PGD-A. At time of writing at least 20 retrospective studies and 4 RCTs provide evidence that PGS, if performed to a high standard, can, and does, improve IVF success for some patient groups [2-5.] We accept that these studies are open to criticism (indeed this could be applied to any study) and we thus continue to support more advanced investigations, both randomised and retrospective. After careful consideration however, the CoGEN forum  has distributed guidelines for the sensible use of PGS based on the balance of the available evidence. The overwhelming opinion among the IVF community is acceptance of the evidence that, for certain patient groups, offering PGS is more beneficial than not offering it.
The Panorama programme mentions that only one add-on treatment has an evidence base behind it, but fails to mention that this is the case for PGS. Without this caveat, it gives the impression of viewing PGS as completely unsupported by published evidence, which is misleading. We also question the wisdom of only highlighting the opinion of one lab, known opponents of PGS, without providing balance by also presenting the evidence base in favour of PGS.
We are strong advocates of evidence based medicine (EBM) and thus subscribe to the view that medical practice should be supported by “well designed and well conducted studies” - the widely accepted definition of EBM. We point out however that the quality of study design is relatively easy to assess by reading a manuscript. Whether the study has been well conducted however is considerably more difficult. The 10 year old Mastenbroek study  (the only one cited in the Panorama programme) is a clear example: if one mines the evidence provided in the manuscript it gives a clear indication that it is their practice of cleavage stage embryo biopsy, not the screening for chromosome abnormalities per se, that led to reduced IVF success and pregnancy rates. What the Mastenbroek study and its follow-ups gave the world therefore was the evidence base to question the use of cleavage stage embryo biopsy for PGS, not to discontinue PGS completely. Indeed questions remain to this day about the extent to which the reduction in IVF caused by the use of cleavage stage biopsy was a generic issue or specific to the individuals performing the trial. In any event, PGS has moved on in the last 6 years to using trophectoderm biopsy – an improved procedure - and whole karyotype screening (array CGH, qPCR and next generation sequencing) – a totally different and more accurate chromosome assessment techniques, these are the ones that provide evidence for its benefit. Even today however PGS remains a complex procedure, not only requiring a robust genetic test, but, critically, the need for high quality embryological practice.
Where we would like to engage in dialogue with the Oxford group is how EBM should be viewed and interpreted in the context of reproductive medicine (and for PGS specifically). In a field where the outcome measure is the likelihood of achieving a live birth of a healthy child, we point out that there are countless individual components that can, in our collected experience, have a profound effect on IVF success. To assess each individually by randomised controlled trials (RCTs) would be prohibitively costly both in terms of time and money. A possible consequence might be that patients would be denied the opportunity to consider the highest quality treatment until the trial was published (and no doubt criticised further), which would be far too late for many. The ESTEEEM trial8 is a good example where this has yet to be published because of criticisms of its recruitment strategy, mixed skill variance and now out of date technology as the field has moved away from array CGH and towards sequencing.
In conclusion therefore, we would like to offer the hand of collaboration to the Oxford group in the hope of working together to consider the evidence base that supports IVF innovations in general (and PGS in particular) in its unique setting. Together we feel we can consider the relative value of single centre and retrospective studies and the possible pitfalls surrounding relying on RCTs alone. As a final point, we also ask the community to consider the implications of not implementing PGS, for instance the harm that could be caused to a patient who has an adverse outcome (e.g. trisomic conceptus), assuming that they could, and would, have chosen to avoid this, had PGS been offered.
We all want every patient entering an IVF clinic to be given the highest possible chance of a healthy live birth. The national average in the UK is ~28%, which is 50% lower than some clinics offering PGS and publishing their data on single embryo transfer only. We feel that with an open minded, pragmatic approach to EBM, we can elevate success rates further.
1. Heneghan C and colleagues. Lack of evidence for interventions offered in UK fertility centres BMJ 2016;355:i6295
2. Lee E, Illingworth P, Wilton L, Chambers GM. (2014). 'The clinical effectiveness of preimplantation genetic diagnosis for aneuploidy in all 24 chromosomes (PGD-A): systematic review'. Hum Reprod. 30(2):473-83.
3. Dahdouh EM, Balayla J, García-Velasco JA. (2015). 'Comprehensive chromosome screening improves embryo selection: a meta-analysis.'
Fertil Steril. 104(6):1503-12.
4. Chen M, Wei S, Hu J, Quan S. (2015). 'Can Comprehensive Chromosome Screening Technology Improve IVF/ICSI Outcomes? A Meta-Analysis.'
PLoS One. 15;10(10).
5. Chang J, Boulet SL, Jeng G, et al. (2016). Outcomes of in vitro fertilization with preimplantation genetic diagnosis: an analysis of the United States Assisted Reproductive Technology Surveillance Data, 2011–2012 Fertil Steril. 105(2):394-400.
7. Mastenbroek S, Twisk M, van Echten-Arends J, et al (2007). 'In vitro fertilization with preimplantation genetic screening'. N Engl J Med. 5;357(1):9-17 | 05 July 2007
Competing interests: The corresponding author does not have competing interests (as he is an academic researcher) other than being treasurer of the Preimplantation Genetic Diagnosis International Society (PGDIS) and a collaborator with clinics that perform PGS, including individuals in the other list. The remainder of the author list contains clinicians and PGS practitioners and well as members of laboratories who's business is to process PGS samples.
Non-evidence based fertility practices were important to highlight, but we disagree with the recommendation that NICE together with HFEA should provide fertility guidance on what is offered (1). HFEA have no legal jurisdiction in providing clinical guidance while NICE have seriously failed to provide evidence for first line treatment options (2). The original NICE guidelines stating IVF should be offered over intrauterine insemination (IUI) was controversial that an update review was initiated to take account of newer RCT leading to ask a specific question: 'What is the effectiveness of IUI in people with unexplained infertility, mild endometriosis or 'mild' male factor infertility?' (2). When the review occurred, the question was reworded to 'What is the evidence for IUI, with or without ovarian stimulation, compared with expectant management for people with unexplained infertility, mild endometriosis and mild male-factor infertility, and whether the 2013 recommendations should be updated?'. NICE continues to state in both reviews. 'The lack of high-quality evidence available’. The original recommendation was based on low dose clomid/ IUI yielding poor outcomes which NICE fail to qualify while omitting gonadotropin-stimulated cycles widely practiced yielding good IUI results. NICE therefore, fail to disclose the limitation of their recommendations (2) and more significantly have only earmarked the comparison of IUI with IVF for 2017, thereby NICE have issued definitive guidance before having considered the evidence.
It is not just the large number of non-evidenced based procedures on offer to the patients, the non-evidenced based NICE guidelines which persuade CCGs to pay prematurely for expensive and unnecessary IVF procedures remains contentious, and in the absence of a valid RCT comparing IUI with IVF is widely rejected (3). The 13-fold increased use of IVF procedures could not have been founded on evidenced based medicine (4), as would the unusually increased 2.9-fold classification of male factor infertility which would justify expensive ICSI procedures.
Fund holders have to pay a high price for unjustified IVF procedures while financial analysis show IUI is still a cost-effective treatment option. In one analysis, IVF was 43,375 Euro more expensive than IUI (5), with a possible annual cost saving of at least 20 million Euro in Europe is possible (6). Globally 74 million subfertile couples cannot access IVF procedures and IUI is their main option (4). For all these reasons clinicians and funding agencies should remain cautious of NICE fertility recommendation (2).
1. Spencer E, Mahtani K, Goldacre B, et al. Claims for fertility interventions: a systematic assessment of statements on UK fertility centre websites. BMJ Open 2016;6:e013940doi: 10.1136/bmjopen-2016-013940.
2. Fertility problems: assessment and treatment, NICE guidelines [CG156] Published date: February 2013 Last updated: August2016 https://www.nice.org.uk/guidance/cg156
3. Kim D, Child T, Farquhar Intrauterine insemination: a UK survey on the adherence to NICE clinical guidelines by fertility clinics. BMJ Open. 2015 May 15;5(5):e007588. doi: 10.1136/bmjopen-2015-007588.
4. Bahadur G, Homburg R, Muneer A, Racich P, Alangaden T, Al-Habib A, Okolo S. First line fertility treatment strategies regarding IUI and IVF require clinical evidence. Hum Reprod. 2016 Jun;31(6):1141-6. doi: 10.1093/humrep/dew075. Epub 2016 Apr 12.
5. Tjon-Kon-Fat RI, Bensdorp AJ, Bossuyt PM, Koks C, Oosterhuis GJ, Hoek A, Hompes P, Broekmans FJ, Verhoeve HR, de Bruin JP et al. Is IVF-served two different ways-more cost-effective than IUI with controlled ovarian hyperstimulation? Hum Reprod 2015; 30:2331–2339.
6. Haagen EC, Nelen WL, Adang EM, Grol RP, Hermens RP, Kremer JA. Guideline adherence is worth the effort: a cost-effectiveness analysis in intrauterine insemination care. Hum Reprod. 2013 Feb;28(2):357-66. doi: 10.1093/humrep/des408. Epub 2012 Nov 30.)
G. Bahadur, B Woodward, R. Homburg, A. Al-Habib, A Muneer
Reproductive Medicine Unit, North Middlesex University Hospital, Old Admin Block, Sterling Way, London N18 1QX; Homerton Fertility Centre, Homerton University Hospital, Homerton Row, London , E9 6SR; IVF Consultancy Services, Leicester, UK; University College London Hospital, 250 Euston Road, London NW1 2BU
Competing interests: No competing interests
Spencer et al (2016) highlight the lack of supportive evidence for claims made on fertility treatment websites in UK. The fact that these interventions are widely used and paid for at great expense reflects the high psychological stress of infertility and the intense drive of women, their partners and doctors to solve the problem.
The situation is exaggerated in sub-Saharan Africa, where infertility is an even more complex social problem with childbearing central to a woman’s sense of respect and relevance in the community. Whilst advances in infertility treatment have allowed many patients to experience the joy of parenthood, disappointed patients go through tremendous stress, and are vulnerable to stigma and abuse for being unable to conceive. Infertility therefore carries with it enormous stress and women commonly develop psychological morbidities. We recently carried out a cross-sectional study in a Nigerian hospital to determine the prevalence of psychological morbidities amongst 124 patients attending for infertility treatments. The 12 item General Health Questionnaire found that 46% of participants had a psychological morbidity, often linked with domestic violence in the home. The rate was related to the duration of infertility.
Although infertility care is a critical priority for many women in sub-Saharan Africa, it has been ignored to a large extent by international researchers and funders who focus instead on family planning and maternal health. Some even may see infertility in societies with high fertility rates as a convenient natural solution to over-population. But whose health is it? It is time to ask women what they think and focus on their priorities. The move to involving lay people in priority setting for research and funding has been championed by the James Lind Alliance – but now needs to be extended to low resource settings. This could result in an increased focus on medical and psychological treatments for couples with infertility. But this empowerment should also focus on providing couple with evidence-based information that would give them the knowledge to make appropriate decisions about their own priorities when seeking private care.
Spencer EA, Mahtani KR, Goldcare B, et al. Claims for fertility interventions: a systematic assessment of statements on UK fertility centre websites. BMJ Open 2016. BMJ 2016;355:i6295 .
Competing interests: No competing interests