You are here

  • Home
  • Online First
  • Opportunities, challenges and risks of using artificial intelligence for evidence synthesis

Article Text

Article menu
Download PDFPDF
Analysis
Opportunities, challenges and risks of using artificial intelligence for evidence synthesis
  1. Waldemar Siemens1,
  2. Erik von Elm2,3,
  3. Harald Binder4,
  4. Daniel Böhringer5,
  5. Angelika Eisele-Metzger1,2,
  6. Gerald Gartlehner6,7,
  7. Piet Hanegraaf8,
  8. Maria-Inti Metzendorf9,
  9. Jacob-Jan Mosselman8,
  10. Artur Nowak10,
  11. Riaz Qureshi11,
  12. James Thomas12,
  13. Siw Waffenschmidt13,
  14. Valérie Labonté1,2,
  15. Joerg J Meerpohl1,2
  1. 1Institute for Evidence in Medicine, Medical Center - University of Freiburg, Faculty of Medicine, Freiburg, Germany
  2. 2Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany
  3. 3Cochrane Switzerland, Lausanne, Switzerland
  4. 4Institute of Medical Biometry and Statistics (IMBI), Medical Center – University of Freiburg, Faculty of Medicine, Freiburg, Germany
  5. 5Eye Center, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany
  6. 6Department for Evidence-based Medicine and Evaluation, University for Continuing Education, Krems, Austria
  7. 7RTI International, Center for Public Health Methods, Durham, North Carolina, USA
  8. 8Pitts.ai, Zeist, Netherlands
  9. 9Cochrane Evidence Synthesis Unit Germany/UK, Institute of General Practice, Heinrich-Heine-Universitat, Düsseldorf, Germany
  10. 10Evidence Prime, Krakow, Poland
  11. 11Cochrane Eyes and Vision, Dept. of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA
  12. 12EPPI Centre, UCL Social Research Unit, University College London, London, UK
  13. 13Institute for Quality and Efficiency in Healthcare, Cologne, Germany
  1. Correspondence to Dr Waldemar Siemens; waldemar.siemens{at}uniklinik-freiburg.de

https://doi.org/10.1136/bmjebm-2024-113320

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Artificial intelligence (AI) is developing rapidly. In particular, in the area of machine learning (ML), significant progress has been made, and large language models (LLMs) have become widely available (for definitions see box 1). This sparks significant interest in the potential application of this technology across various sectors, including healthcare. One area where LLMs promise substantial benefits is evidence synthesis.1 This article discusses the opportunities, challenges and risks associated with using AI, and LLMs in particular, for this purpose, drawing on insights from the third Methods Forum of Cochrane Germany, held in Freiburg, Germany, on 14 June 2024.

    Box 1

    Definitions

    Artificial Intelligence (AI):

    Computational systems that are able to perform tasks such as problem-solving, decision-making and language processing, seemingly similar to human intelligence.19

    Machine learning (ML):

    Automated pattern recognition algorithms, many being from the field of AI, to learn from data and enhance performance on specific tasks over time without explicit programming, that can be categorized, eg into unsupervised, semi-supervised, self-supervised, and reinforcement learning.20

    Large language models (LLMs):

    LLMs employ sophisticated AI algorithms to generate human-like text by predicting the next word in a sequence. Trained on vast amounts of text data, often sourced from the internet, these models can perform various language tasks, including answering questions, summarizing content, translating text, and creating narratives. 19

    Opportunities with using LLMs

    Systematic reviews are essential for evidence-based healthcare because they provide comprehensive and unbiased syntheses of research data on specific clinical and public health questions. However, they are labour-intensive and time-consuming. Using AI applications, based on ML and in particular LLMs, offers review authors opportunities to streamline and enhance the production of systematic reviews, including but not limited to the following.

    Searching for studies

    LLMs can be used to generate Boolean search queries and assist with the development of search strategies by selecting suitable search terms or translating database syntax. However, the literature shows that using LLMs for …

    View Full Text

    Footnotes

    • X @mintimetz

    • Contributors WS: conceptualisation, writing the original draft and visualisation. EVE: conceptualisation, writing the review and editing, and supervision. HB, DB, AEM, GG, PH, MIM, JJM, AN, RQ, JT, SW and VL: writing the review and editing. JJM: conceptualisation, writing the review and editing, and supervision. We used ‘ChatGPT 4o’ by OpenAI to assist with refining the language of this manuscript. The final content was solely produced by the authors.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests PH and JJM are co-founders of the systematic review software PITTS. AN is a co-founder of Evidence Prime, the company behind the development of the systematic review software LASER AI. JT is involved in the development of the systematic review software EPPI-Reviewer. These relationships may be considered a potential conflict of interest. All authors have made every effort to ensure that the article is objective and based on sound scientific evidence, without any influence from commercial interests.

    • Provenance and peer review Not commissioned; externally peer reviewed.