You are here

  • Home
  • Online First
  • Incomplete reporting and spin in acupuncture randomised controlled trials: a cross-sectional meta-epidemiological study

Article Text

Article menu

Download PDFPDF

Original research
Incomplete reporting and spin in acupuncture randomised controlled trials: a cross-sectional meta-epidemiological study
  1. Yuting Duan1,2,3,
  2. Pinge Zhao4,
  3. Yuening Deng1,2,
  4. Zhirui Xu1,
  5. Siqi Wu1,
  6. Lin Shi1,
  7. Feng Jiang1,
  8. Shujuan Liu1,2,
  9. Xinyu Li5,
  10. Binbin Tang2,
  11. Jinjin Zhou3,
  12. Lin Yu1,2,3
  1. 1Evidence Based Medicine Center, The Affiliated Traditional Chinese Medicine Hospital, Guangzhou Medical University, Guangzhou, People's Republic of China
  2. 2Clinical School of Integrated Traditional Chinese and Western Medicine, Guangzhou Medical University, Guangzhou, Guangdong, China
  3. 3The Affiliated Guangzhou Hospital of TCM of Guangzhou University of Chinese Medicine, Guangzhou, China
  4. 4The Affiliated Brain Hospital, Guangzhou Medical University, Key Laboratory of Neurogenetics and Channelopathies of Guangdong Province and the Ministry of Education of China, Guangzhou Medical University, Guangzhou, People's Republic of China
  5. 5Hong Kong Baptist University, Hong Kong, Hong Kong
  1. Correspondence to Dr Yuting Duan; duanyt{at}gzhmu.edu.cn; Dr Lin Yu; yul{at}gzhmu.edu.cn

Abstract

Objectives To investigate the reporting, data sharing and spin (using reporting strategies to emphasise the benefit of non-significant results) in acupuncture randomised controlled trials (RCTs).

Design Cross-sectional meta-epidemiological study.

Data sources Eligible studies indexed in MEDLINE, Embase, CENTRAL, CBM, CNKI, Wanfang Data and VIP Database between 1 January 2014 and 1 May 2024.

Eligibility criteria Peer-reviewed acupuncture RCTs used traditional medicine (TM), published in English or Chinese, two parallel arms for humans.

Main outcome measures We assessed (1) the reporting of acupuncture RCTs by the Consolidated Standards for Reporting Trials (CONSORT) 2010 statement and STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) checklist; (2) the data sharing level by the International Committee of Medical Journal Editors (ICMJE) data sharing statement; (3) spin frequency and level by the prespecified spin strategies.

Results This study evaluated 476 eligible studies, of which 166 (34.9%) explored the specific efficacy or safety of acupuncture in the nervous system, 68 (14.3%) in the motor system and 61 (12.8%) in the digestive system. 244 (57.7%) studies used conventional acupuncture, 296 (62.2%) used multicentre study design and 369 (77.5%) were supported by institutional funding. 312 (65.5%) eligible studies were poorly reported. The sufficiently reporting scores of the CONSORT 2010 statement and the STRICTA checklist differed from 0.63% to 97.5%, and 32 (59.3%) items were less than 50%. For the data sharing level of acupuncture RCTs, only 66 (17.2%) studies followed the ICMJE data sharing statement, but 49 (14.5%) need to require authors to obtain data, and only 5 (1.5%) provided data by open access. Spins were identified in 408 (85.7%) studies (average spin frequencies: 2.94). 59 (37.2%) studies with non-significant primary outcomes had spin levels.

Conclusions This study found that the reporting of acupuncture RCTs was low compliance with the CONSORT 2010 statement, the STRICTA checklist and the ICMJE data sharing statement, and spin appeared frequently. Journal policies on using reporting guidelines, data sharing and equitable consideration of non-significant results might enhance the reporting of acupuncture RCTs.

Trial registration number This study was registered at the Open Science Framework (OSF): (https://doi.org/10.17605/OSF.IO/2WTE6, and https://doi.org/10.17605/OSF.IO/9XDN4,)

  • Acupuncture
  • Methods

Data availability statement

Data are available in a public, open access repository. The datasets used and/or analysed during the current study have been provided in the appendixes and available via Open Science Framework at: https://osf.io/gvpzh/?view_only=78e46fbd6327484e8e27d42aa907fc97

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjebm-2024-113364

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • Incomplete, non-transparent reporting and inappropriate interpretation of study results could undermine the study’s credibility.

    • Due to the limited evidence, the reporting and spin (using reporting strategies to emphasise the benefit of non-significant results) of acupuncture randomised controlled trials (RCTs) remain inconclusive.

    WHAT THIS STUDY ADDS

    • We found that published acupuncture RCTs generally suffer from poor reporting, low data availability and frequent spin.

    • There is an urgent need for acupuncture authors and stakeholders to take effective strategies to improve acupuncture reporting.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • Researchers and journal stakeholders should change the perception of non-significant results and emphasise the use of reporting guidelines to promote the objective and complete presentation of study outcomes.

    Introduction

    Acupuncture originated in China over 2000 years ago and is now accepted in 183 countries to prevent and treat over 100 diseases.1 2 More than 14 000 acupuncture randomised controlled trials (RCTs) have been published in the past half century,3 4 providing substantial evidence for clinical practice to promote the well-being of patients.5 Nevertheless, the reporting of acupuncture RCTs encounters challenges due to the complexity of varying underlying theories, acupoint prescriptions, needle manipulation techniques, combination therapies and individualised treatment regimens,6 which may negatively affect the transparency, openness, and reproducibility of acupuncture RCTs.

    The Transparency and Openness Promotion committee emphasises that any details of the design and implementation of methodology and the data availability should be adequate and transparently reported to increase the credibility of the study results.7 The Consolidated Standards for Reporting Trials (CONSORT) 2010 statement,8 the Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA),9 and the International Committee of Medical Journal Editors (ICMJE) data-sharing statement10 also provide the minimum list of information of the core components of RCTs. Adherence to these guidelines/statements in manuscript preparation facilitates the detailed documentation of the essential elements of the study and promotes the transparency and accuracy, which collectively enhance the validity, reliability and potential for replication of research findings.11 However, there is a lack of comprehensive review of acupuncture RCTs reporting.

    In addition, even with complete reporting, inappropriate result interpretation can significantly affect the internal authenticity and external validity of study result.12 Successful hypotheses in medical research seem to be more ‘attractive’ and ‘valuable’ to editors and readers.13 And researchers are highly susceptible to misreporting or distorting research results under the ‘publish or perish’ pressure.14 This kind of inappropriate result interpretation, termed ‘Spin’, or ‘science hype’,15 16 refers to the misrepresentation of research findings. It encompasses several manifestations: enhancement of non-significant results, preferential reporting of favourable outcomes and the construction of overly optimistic conclusions unsupported by the data. Further concerning practices include unexplained deviations from registered protocols and inadequate treatment of missing data in outcome analyses.17 Spin may influence the decision-making in research and even mislead clinical practice,18 and it was underexplored in acupuncture RCTs.

    The purpose of this study was to assess whether published acupuncture RCTs used appropriate reporting strategies and reported sufficient information to allow the studies to be accurately interpreted and replicated. Therefore, our study investigated the reporting of acupuncture RCTs by the CONSORT 2010 statement and STRICTA checklist, the data sharing level by the ICMJE data-sharing statement; and the spin frequency and spin level by the prespecified spin strategies.

    Methods

    We conducted a cross-sectional analysis for the reporting of acupuncture RCTs published from 2014 to 2024. Methods used in this study were prespecified in the protocols,19 20 and any inconsistencies with the protocol are available on the Open Science Framework (OSF): https://osf.io/gvpzh/?view_only=78e46fbd6327484e8e27d42aa907fc97. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology guidelines for cross-sectional studies and the Preferred Reporting Items for Systematic Review and Meta-Analysis 2020 statement.21 22

    Patient and public involvement

    Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans in this study.

    Identification and selection of articles

    We included studies with: (1) RCTs using acupuncture therapy only in the intervention group; (2) acupuncture therapy should be based on traditional medicine (TM), regardless of the difference in instrument size, stimulating spots and needling manipulation, such as autotomy and auricular acupuncture; (3) the specific type of the control group should be no intervention, placebo intervention, sham acupuncture, western medicine, waiting list and other interventions guided by modern medical theory; (4) studies with two parallel study groups applied to humans; (5) studies published in English and Chinese in peer-reviewed journals. And excluded studies with: (1) not for clinical therapeutic objectives (cost-effectiveness, diagnostic test, etc), (2) the acupuncture needle does not penetrate the skin, (3) full text unavailable, (4) duplication articles, (5) the sample size is ≤10 in each group and (6) authors of the trial are <3.

    We comprehensively searched for acupuncture RCTs between 1 January 2014 and 1 May 2024 in three English databases (through OVID) MEDLINE, Excerpta Medica Database (EMBASE), Cochrane Central Register of Controlled Trials (CENTRAL) and four Chinese databases including Chinese Biomedical Literature Service System (CBM), China National Knowledge Infrastructure (CNKI), Wanfang Data and VIP Chinese Medical Journal Database. The search strategy for all databases is available in online supplemental appendix 1.

    Supplemental material

    Results from all of the searched Chinese publications were imported into EndNote (V.X7.1) and English publications into Rayyan (https://www.rayyan.ai/) to remove duplicated studies. Then, two researchers (ZX and XL) independently screened all titles and abstracts, and two researchers (PZ and ZX) independently screened full texts. Any disagreements were resolved by discussion or two senior investigators (YDu and LY) if necessary.

    Data extraction

    The data collection form included four sections. The first section captured the general characteristics of acupuncture RCTs, the second section scored for study reporting by CONSORT 2010 and STRICTA items, the third section consisted of items describing data availability and the final section captured the spin strategies in acupuncture RCTs. Two authors collected data independently and in duplicate from all of the studies using a standardised form created in Microsoft Excel 2019 (Microsoft Excel 2019 MSO 2210 Build 16.0.15726.20188 32). Before data collection, three rounds of pilot tests were performed on a random sample of 10 studies to ensure the consistency and accuracy of data extraction.

    The general characteristics were extracted by XL and ZX independently, and any discrepancy in the data collected was resolved via discussion or adjudication by the senior reviewer (YDu). The basic characteristics included: (1) title, (2) authors, (3) language, (4) year, (5) publication journal, (6) Journal Impact Factor (JIF) (English journals cited in Web of Science - Journal Citation Reports 2023,23 Chinese journals cited in China Scientometrics and Bibliometrics Research Center - Journal Reports 202324), (7) type of study centre, (8) location of the study, (9) sample size, (10) source of funding, (11) conflict of interest, (12) registry and (13) outcomes type. We stratified funding into four categories: institutional funding (including governmental, non-profit and industry-derived funding), individual funding, mixed funding (a mix of institutional and individual funding) and none. Conflicts of interest were dichotomised into declared (including both declared conflicts and formal declarations of no conflicts) and not mentioned. Trial registration status was classified as prospective (registration predating participant recruitment), retrospective (registration subsequent to recruitment), or unregistered.

    The reporting scores were assessed by two researchers (PZ and YDe) independently, and any discrepancy in the data collected was resolved via discussion or adjudication by the senior reviewer (YDu).9 The CONSORT 2010 statement was used to assess the incomplete reporting scores of included acupuncture RCTs, and the STRICTA checklist was used to evaluate the reporting scores of specific acupuncture interventions. A total of 54 items were scored in four possibilities: ‘0’ for ‘Not applicable’, ‘1’ for ‘Sufficiently reported’, ‘2’ for ‘Insufficiently reported’, and ‘3’ for ‘Unreported’. The scoring details were made according to the CONSORT Explanation and Elaboration25 and shown in online supplemental appendix 2.

    Supplemental material

    The data availability of eligible studies was assessed by two researchers (YDe and BT) independently, and any discrepancy in the data collected was resolved via discussion or adjudication by the senior reviewer (LY). Specifically, we used the ICMJE Data-Sharing Statement (DSS) (seven items) to assess the data-sharing level for acupuncture RCTs. As ICMJE policy requires that manuscripts published after 1 July 2018 should contain a DSS, we only included studies published from July 2018 to May 2024.

    The spin assessment was extracted by two researchers (YDe and JZ) independently, and any discrepancy in the data collected was resolved via discussion or adjudication by the senior reviewer (PZ). We appraised the spin frequency and spin level in acupuncture RCTs regardless of the statistical significance of primary outcomes. This is distinct from previous studies,26 27 as we aimed to use spin to assess the inappropriate interpretation of study results. Spin was defined as the ‘use specific reporting strategies to emphasise benefit of non-significant results or distract from non-significant results’.15 We used a prespecified spin classification criteria to identify spin based on previous knowledge, which include: (1) the specific acknowledgement of the non-significant outcomes, (2) the exaggerated presentation of non-significant results and selectively highlight the benefits, (3) selectively emphasising certain outcomes or time points with a statistically significant difference when multiple outcomes or time points are available, (4) the use of favourable rhetoric to distort the interpretation of non-significant results, (5) any inconsistencies with the protocol (without explanation) and (6) based on the specific acknowledgement of the non-significant primary outcomes, we classified studies as none, low, moderate and high level of spin.15 We counted spin frequencies in the title, results, discussions and conclusions of RCTs of acupuncture and assessed spin levels. A total of 38 items were assessed and scored in two possibilities: ‘1’ for ‘Spin’, and ‘0’ for ‘non-Spin’ and ‘Not applicable’. The definition details and specific examples are shown in online supplemental appendix 2.

    Data analysis

    We summarised the general and reporting characteristics of the included acupuncture RCTs using descriptive statistics. Categorical items were presented in counts (n), percentages (%) and 95% CIs, continuous items with skewed distributions were represented as the median value (25th percentile, 75th percentile).

    The specificity 95% CI were calculated by Wilson Procedure (without correction for continuity).28 The median value (25th percentile, 75th percentile) of JIF (the 2023 JIF of journals published eligible acupuncture RCTs) and the median value of study sample size was calculated using the IBM SPSS Statistics software (V.23).

    For the reporting of acupuncture RCTs, we counted the reporting scores for each item of the CONSORT 2010 statement and STRICTA checklist and presented them in a bar chart. For the data-sharing, we calculated the reporting rate for each item of the ICMJE data-sharing statement and presented them in counts (n), percentages (%) and 95% CI in chart. For the spin assessment, we evaluated the frequency of each spin item and the spin level and presented them in counts (n), percentages (%) and 95% CI in chart.

    Results

    For this exploratory study, a total of 22 608 potentially relevant records were initially screened after the search. Among these, 8629 duplicated studies were eliminated, 13 534 were eliminated from the initial screening and 677 studies were eliminated from the full-text reading. Finally, 476 studies were included (see figure 1). Citations, reporting details and pilot tests of all included studies are available on the OSF (https://osf.io/gvpzh/?view_only=78e46fbd6327484e8e27d42aa907fc97).

    Figure 1
    Figure 1

    Identification of studies. TCM, traditional Chinese medicine.

    General characteristics of acupuncture RCTs

    Most studies were conducted in China, employing multicentre designs and conventional acupuncture interventions. Table 1 presents the general characteristics of the included studies (the final kappa coefficient of general characteristics pilot tests was 0.961, 95% CI 0.886~1.037). More details are available on the OSF (https://osf.io/gvpzh/?view_only=78e46fbd6327484e8e27d42aa907fc97).

    View this table:
    Table 1

    General characteristics of included studies

    Reporting scores of acupuncture RCTs

    Figure 2 presents the reporting scores of the CONSORT 2010 statement and the STRICTA checklist of the included studies (the final kappa coefficient of reporting scores on the pilot test was 0.960, 95% CI 0.929 to 0.992). The sufficiently reporting scores ranged from 0.63% to 97.5% across the different items in the CONSORT 2010 statement and the STRICTA checklist. 24 (64.9%) of CONSORT 2010 statement items (1b, 2b, 3a, 3b, 6a, 6b, 7a, 7b, 8b, 9, 10, 11a, 11b, 12b, 14a, 14b, 17a, 17b, 18, 19, 20, 23, 24, 25) and 8 (47.1%) of the STRICTA checklist items (1b, 1c, 2a, 2g, 4a, 4b, 5, 6a) sufficiently reporting scores are less than 50%. 312 (65.5%) studies were poorly reported (more than half of the recommended items were incompletely reported), details of the reporting scores for each trial are available on the OSF (https://osf.io/gvpzh/?view_only=78e46fbd6327484e8e27d42aa907fc97).

    Figure 2
    Figure 2

    The CONSORT reporting scores of acupuncture RCTs. CONSORT, Consolidated Standards for Reporting Trials; RCTs, randomised controlled trials; STRICTA, STandards for Reporting Interventions in Clinical Trials of Acupuncture.

    The average sufficiently reporting scores of included studies did not improve over years (see online supplemental appendix 4 table 1). Studies published in English exhibit higher average sufficiently reporting scores compared with Chinese publications (52.9–37.3%), with English publications demonstrating higher scores in the methods and results sections. Conversely, Chinese publications having higher sufficiently reporting scores in the acupuncture intervention delivery section (see online supplemental appendix 4 table 2). Registered studies significantly outperform unregistered studies in terms of sufficiently reporting scores (56.4–17.3%, online supplemental appendix 4 table 3). Additionally, multicentre studies show similar average sufficiently reporting scores to single-centre studies (43.0–46.8%, Online supplemental appendix 4 table 4).

    Supplemental material

    Data sharing in acupuncture RCTs

    Table 2 presents the data-sharing statement in each section of the included studies (the final kappa coefficient of data-sharing pilot tests was 0.960, 95% CI 0.929 to 0.992). A total of 339 eligible studies were assessed in this analysis (the ICMJE states that publications after July 2018 should contain DSS). Only 66 studies (17.2%) contain a data sharing statement, with 49 (14.5%) willing to share the individual participant data. Only 66 studies (17.2%) contain a data-sharing statement, with 49 (14.5%) willing to share the individual participant data. However, most studies (n=57, 16.8%) should require authors to obtain data, and only 5 (1.5%) provided data in the open assessment. More details are available on the OSF (https://osf.io/gvpzh/?view_only=78e46fbd6327484e8e27d42aa907fc97).

    View this table:
    Table 2

    Data sharing in acupuncture randomised controlled trials

    Spin assessment in acupuncture RCTs

    Table 3 presents the spin strategies in each section of the included studies (the final kappa coefficients of spin assessment pilot tests were 0.953, 95% CI 0.888 to 1.018). 408 (85.7%) studies have different frequencies of spin, 279 (84.3%) of studies with significant primary outcomes have spin, and 129 (89.0%) studies with non-significant primary outcomes have spin. For studies with non-significant primary outcomes, 57 (37.2%) have spin levels. Spin occurred more frequently in the abstracts than in the main text of included studies. Details of the spin assessment are available on the OSF (https://osf.io/gvpzh/?view_only=78e46fbd6327484e8e27d42aa907fc97).

    View this table:
    Table 3

    Spin assessment in acupuncture randomised controlled trials

    We found that studies with non-significant primary outcomes have more spin than studies with significant outcomes (average spin 3.92–2.51). The spin of studies with non-significant primary outcomes appeared mainly on inconsistency with the protocol and a selective emphasis on statistically significant results. In contrast, the spin of studies with significant primary outcomes is evident in the title and omitted from safety outcomes. More details are shown in online supplemental appendix 4 table 5.

    Discussion

    This study investigated the reporting, data sharing and spin of 476 eligible acupuncture RCTs. Notably, 312 (65.5 %) studies were suboptimally reported (sufficiently reporting items under 50%), especially in Chinese-language publications and unregistered studies. Data availability of acupuncture RCTs was low, with only 5 (1.5%) studies providing raw data openly. Regardless of whether the primary outcome was significant or not, spin frequently appeared in the vast majority of studies. Those issues raise concerns regarding the integrity of acupuncture RCTs, as incomplete reporting and inappropriate interpretation of studies could undermine the credibility of study outcomes. Therefore, pragmatic approaches are necessary to enhance the reporting quality of acupuncture RCTs.

    Reporting scores of included studies

    Research findings can be effectively shared or poorly communicated through publications. To prevent wasted research efforts and enable readers to fully understand clinical studies, reports must be comprehensive and transparent.29 However, our findings align with previous research that has identified significant gaps in crucial aspects of acupuncture RCTs, including how sample sizes were determined, how treatments were described and whether results can be applied more broadly can make mistakes.30 31 The low adherence rate of some critical items of the CONSORT 2010 statement and STRICTA checklist in acupuncture RCTs may be attributed to several possible reasons.

    First, unlike pharmacological interventions, acupuncture interventions are inherently more complex. Multiple factors can significantly influence acupuncture outcomes, including the selection of acupoints (which can be fixed, flexible and individualised prescription), the type of acupuncture implements used (such as electroacupuncture, small needle-knife), treatment parameters (duration, intensity, repetition intervals, etc), patient–practitioner interaction and participants’ expectations.6 In contrast to highly standardised pharmaceutical trials, acupuncture RCTs often lack precision in documenting these critical details, resulting in ambiguous and inadequate reporting of interventional specifics. Second, authors of acupuncture RCTs may still lack awareness of reporting guidelines, especially authors whose native language is not English. Although the CONSORT statement has been published for 28 years and the STRICTA checklist for 23 years,32 33 the Chinese-translated version did not appear until 2021.34 This may have contributed to the poor reporting of Chinese publications. Third, editors and peer reviewers may lack a recommendation for use or lack scrutiny of the details of reporting guidelines. Finally, journals may not recommend reporting guidelines strongly enough, as mandatory use rather than recommendations can improve adherence to reporting guidelines.35

    Data sharing in acupuncture RCTs

    Responsible sharing of data is crucial for maximising the value of clinical research.36 However, several barriers impede comprehensive data dissemination. First, researchers may lack motivation for sharing data, due to the absence of incentive policies and the concern about potential challenges, including questions of data ownership and integrity.37 Second, the absence of official harmonised guidelines creates inconsistencies across journals, with varying requirements that challenge reviewers and editors in establishing standardised data-sharing practices.38 Third, there is a lack of resource support. Data-sharing requires time and resources for data preparation (anonymisation, and standard data and metadata exchange, etc), annotation and uploading, which is often lacking in public health.39 Finally, technical limitations compound these challenges. While numerous data-sharing platforms exist, researchers encounter insufficient guidance on platform use and lack standardised data management templates that could specify essential elements such as content, sources, methodological approaches, data formats and variables.40

    Additionally, the real assessment of data in acupuncture RCTs may be much lower than claimed data-sharing statements by the authors. As 57 (86.4% of 66) studies must contact corresponding authors to obtain data, and previous studies found that 93% of authors declined or did not respond to access requests for raw data, despite the authors indicating that they will share data on request.37 38

    Spin assessment in acupuncture RCTs

    Significant results often receive disproportionate attention in clinical research.13 Current publication metrics and academic incentive structures tend to prioritise positive results, marginalising non-significant findings despite their potential scientific value.14 And for researchers, academic publications are viewed as proxies for scientific productivity and competence.41 Consequently, such perverse incentives may be the cause of spin in acupuncture studies.

    For the spin assessment, we found: (1) selective emphasis on meaningful endpoints or time points, use of the word ‘significant’ for within-group comparison and use of hyperbole to describe the conclusion were the most frequent spin of included acupuncture RCTs. This was related to the emphasise of the benefit to study results; (2) spin appeared more often in the abstracts than the main text of acupuncture RCTs. This may be related to the authors’ need to convey enough information to engage readers within a short, word-limited abstract, resulting in the selective emphasis on some meaningful outcomes41; (3) we found that only 169 included studies provided registry information (including prospective registration and retrospective registration), but 121 (71.6%) studies had discrepancies between registries and publications (without explanation), with 102 (60.3%) changes in outcomes, and 66 (39.1%) changes in participants (see online supplemental appendix 3), which may be related to favouring the presentation of statistically significant results in publications.42 (4) 204 (42.9%) of the included studies failed to report any safety outcomes, which may be related to the widespread perception that the use of acupuncture by qualified practitioners is safe, so some minor side effects (eg, pain, bleeding) are considered too trivial for acupuncturists/researchers to report.43

    Supplemental material

    Strengths and limitations of the study

    The strengths of this study are: (1) We searched several databases to identify eligible acupuncture RCTs, and we did not limit the diseases or journals to obtain a high level of representation. (2) We comprehensively and meticulously assessed the incomplete reporting and spin of acupuncture RCTs, which identified deficiencies in acupuncture RCT reporting and provided direction for improvement in acupuncture RCT reporting. (3) This study captured the adherence to the ICMJE data-sharing statement and further explored the types of data and the acquisition methods to avoid open data in appearance only.

    The limitations of this study are: (1) We only included acupuncture RCTs with two parallel designs (the CONSORT statement focused on the guidance for two parallel group designs8), which may not apply to other RCT designs. (2) We only included studies published in Chinese and English; study results may be unable to apply to other languages. (3) We may be unable to avoid subjective bias in spin assessment, even if we standardise interassessor agreement between reviewers, as spin assessment should be based on the interpretation of the results. Analysis of spin must be interpreted with caution. (4) We found 308 (64.7%) studies that did not provide registry information, which prevented us from identifying whether the primary outcomes of these studies were the same as the study design. The analysis of spin for significant or non-significant primary outcomes should be cautious.

    Implications for future research and policy

    To ensure scientific progress and publication integrity, we recommend the following guidelines:

    For Researchers: (1) Adhere to academic integrity, rigorous design and implementation of research; (2) Follow the ‘Guidance to consider when designing an acupuncture trial’5 during the study design and implementation stage to adequately consider and rigorously assess the characteristics of acupuncture treatment to facilitate the complete reporting of the study process and results; (3) Register studies in advance and provide detailed documentation; (4) Follow the CONSORT statement for manuscript preparation and follow the STRICTA checklist to report acupuncture intervention (or sham acupuncture control) details; (5) Present findings objectively (non-significant outcomes, safety outcomes, etc) and share the raw data during the manuscript submission.

    For journals and stakeholders: (1) Create dedicated sections for non-significant study results to mitigate publication bias; (2) Enhance reporting guidelines and data-sharing requirements, and provide technical support for data sharing, such as providing links to reporting guidelines, data-sharing platforms and detailed data sharing templates in the authors guidelines; (3) Editors and reviewers strengthen the review of using reporting guidelines, check for consistency between registrations and publications to avoid bias, and, if necessary, require explanations from the authors; (4) Editors and reviewers assess the findability, accessibility, interoperability and reusability of data,44 to avoid open data in appearance only (without any actual data sharing occurrence); (5) Develop incentives for data sharing, for example, protecting researchers’ ownership and copyright of their data (requiring their approval to use the data), recognition of data authorship, etc39; (6) Establish safeguards for data sharing, for example, the application of 5–10% of research funds by researchers to data collection, monitoring, and sharing, etc.45

    Conclusion

    The study found low compliance in reporting and data sharing of several recommended items in the CONSORT 2010 statement, the STRICTA checklist and the ICMJE data sharing statement, and spin appeared frequently in acupuncture RCTs. There is an urgent need for acupuncture authors and stakeholders to take effective strategies to improve acupuncture reporting.

    Data availability statement

    Data are available in a public, open access repository. The datasets used and/or analysed during the current study have been provided in the appendixes and available via Open Science Framework at: https://osf.io/gvpzh/?view_only=78e46fbd6327484e8e27d42aa907fc97

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    References

    1. ↵
      1. World Health Organization
      . Acupuncture: review and analysis of reports on controlled clinical trials. Available: https://drjohnbarrett.com/wp-content/uploads/2019/05/WHO-Acupuncture-Clinical-T rials.pdf [accessed 2 Sep 2024]
    2. ↵
      1. World Health Organization
      . WHO traditional medicine strategy 2014–2023. Available: https://www. who. int/publications/i/item/9789241506096 [accessed 2 Sep 2024]
    3. ↵
      1. Zhang Y-Q,
      2. Jiao R-M,
      3. Witt CM, et al
      . How to design high quality acupuncture trials-a consensus informed by evidence. BMJ 2022;376:e067476. doi:10.1136/bmj-2021-067476
    4. ↵
      1. Pang B,
      2. Yi S-W,
      3. Shan X-J, et al
      . Randomized controlled trials of acupuncture and moxibustion in China in past 45-year (1975-2019): hotspots and trends. Zhongguo Zhen Jiu 2021;41:1283–90. doi:10.13703/j.0255-2930.20200918-0004
      OpenUrlPubMed
    5. ↵
      1. Lu L,
      2. Zhang Y,
      3. Tang X, et al
      . Evidence on acupuncture therapies is underused in clinical practice and health policy. BMJ 2022;376:e067475. doi:10.1136/bmj-2021-067475
    6. ↵
      1. Fei Y-T,
      2. Cao H-J,
      3. Xia R-Y, et al
      . Methodological challenges in design and conduct of randomised controlled trials in acupuncture. BMJ 2022;376:e064345. doi:10.1136/bmj-2021-064345
    7. ↵
      1. Nosek BA,
      2. Alter G,
      3. Banks GC, et al
      . SCIENTIFIC STANDARDS. Promoting an open research culture. Science 2015;348:1422–5. doi:10.1126/science.aab2374
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Schulz KF,
      2. Altman DG,
      3. Moher D, et al
      . CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c332. doi:10.1136/bmj.c332
    9. ↵
      1. MacPherson H,
      2. Altman DG,
      3. Hammerschlag R, et al
      . Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): Extending the CONSORT statement. J Evid Based Med 2010;3:140–55. doi:10.1111/j.1756-5391.2010.01086.x
      OpenUrlCrossRefPubMed
    10. ↵
      1. Taichman DB,
      2. Sahni P,
      3. Pinborg A, et al
      . Data Sharing Statements for Clinical Trials - A Requirement of the International Committee of Medical Journal Editors. N Engl J Med 2017;376:2277–9. doi:10.1056/NEJMe1705439
      OpenUrlCrossRefPubMed
    11. ↵
      EQUATOR network: what we do and how we are organised. Available: https://www.equator-network.org/about-us/equator-network-what-we-do-and-how-we-are-organised/ [Accessed 20 Jun 2024].
    12. ↵
      1. Khan MS,
      2. Lateef N,
      3. Siddiqi TJ, et al
      . Level and Prevalence of Spin in Published Cardiovascular Randomized Clinical Trial Reports With Statistically Nonsignificant Primary Outcomes: A Systematic Review. JAMA Netw Open 2019;2:e192622. doi:10.1001/jamanetworkopen.2019.2622
    13. ↵
      1. Mlinarić A,
      2. Horvat M,
      3. Šupak Smolčić V
      . Dealing with the positive publication bias: Why you should really publish your negative results. Biochem Med (Zagreb) 2017;27:030201. doi:10.11613/BM.2017.030201
    14. ↵
      1. Grimes DR,
      2. Bauch CT,
      3. Ioannidis JPA
      . Modelling science trustworthiness under publish or perish pressure. R Soc Open Sci 2018;5:171511. doi:10.1098/rsos.171511
    15. ↵
      1. Boutron I,
      2. Dutton S,
      3. Ravaud P, et al
      . Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 2010;303:2058–64. doi:10.1001/jama.2010.651
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Chiu K,
      2. Grundy Q,
      3. Bero L
      . 'Spin' in published biomedical literature: A methodological systematic review. PLoS Biol 2017;15:e2002173. doi:10.1371/journal.pbio.2002173
    17. ↵
      1. Fletcher RH,
      2. Black B
      . 'Spin' in scientific writing: scientific mischief and legal jeopardy. Med Law 2007;26:511–25.
      OpenUrlPubMed
    18. ↵
      1. Rowhani-Farid A,
      2. Hong K,
      3. Grewal M, et al
      . Consistency between trials presented at conferences, their subsequent publications and press releases. BMJ Evid Based Med 2023;28:95–102. doi:10.1136/bmjebm-2022-111989
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Open Science Framework
      . Registries. reporting quality and data sharing for acupuncture randomized controlled trials. 2023. Available: https://doi.org/10.17605/OSF.IO/2WTE6
    20. ↵
      1. Open Science Framework
      . Registries. spin in acupuncture randomized controlled trials. 2023. Available: https://doi.org/10.17605/OSF.IO/9XDN4
    21. ↵
      STROBE statement—checklist of items that should be included in reports of cross-sectional studies. Available: https://view.officeapps.live.com/op/view.aspx?src=https%3A%2F%2Fwww.equator-network.org%2Fwp-content%2Fuploads%2F2015%2F10%2FSTROBE_checklist_cross-sectional.docx&wdOrigin=BROWSELINK [accessed 1 Sep 2023]
    22. ↵
      1. Page MJ,
      2. McKenzie JE,
      3. Bossuyt PM, et al
      . The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi:10.1136/bmj.n71
    23. ↵
    24. ↵
      1. China Scientometrics and Bibliometrics Research Center
      . Journal reports. Available: https://eval.cnki.net/News/ItemDetail?ID=6abe3e41dbd5462387559540834e81cd [Accessed 1 Sep 2023].
    25. ↵
      1. Moher D,
      2. Hopewell S,
      3. Schulz KF, et al
      . CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869. doi:10.1136/bmj.c869
    26. ↵
      1. Arunachalam L,
      2. Hunter IA,
      3. Killeen S
      . Reporting of Randomized Controlled Trials With Statistically Nonsignificant Primary Outcomes Published in High-impact Surgical Journals. Ann Surg 2017;265:1141–5. doi:10.1097/SLA.0000000000001795
      OpenUrlPubMed
    27. ↵
      1. Won J,
      2. Kim S,
      3. Bae I, et al
      . Trial registration as a safeguard against outcome reporting bias and spin? A case study of randomized controlled trials of acupuncture. PLoS One 2019;14:e0223305. doi:10.1371/journal.pone.0223305
    28. ↵
      1. Newcombe RG
      . Two-sided confidence intervals for the single proportion: comparison of seven methods. Statist Med 1998;17:857–72. doi:10.1002/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E
      OpenUrlCrossRef
    29. ↵
      1. Glasziou P,
      2. Altman DG,
      3. Bossuyt P, et al
      . Reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014;383:267–76. doi:10.1016/S0140-6736(13)62228-X
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Xiu W-C,
      2. Meng X,
      3. Hu X-Y, et al
      . Evaluation of the report quality of Chinese and English randomized controlled trials of acupuncture based on CONSORT statement and STRICTA checklist. Zhongguo Zhen Jiu 2023;43:355–61. doi:10.13703/j.0255-2930.20220328-k0005
      OpenUrlPubMed
    31. ↵
      1. Kim KH,
      2. Kang JW,
      3. Lee MS, et al
      . Assessment of the quality of reporting in randomised controlled trials of acupuncture in the Korean literature using the CONSORT statement and STRICTA guidelines. BMJ Open 2014;4:e005068. doi:10.1136/bmjopen-2014-005068
    32. ↵
      1. Begg C,
      2. Cho M,
      3. Eastwood S, et al
      . Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996;276:637–9. doi:10.1001/jama.276.8.637
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. MacPherson H,
      2. White A,
      3. Cummings M, et al
      . Standards for reporting interventions in controlled trials of acupuncture: the STRICTA recommendations. Complement Ther Med 2001;9:246–9. doi:10.1054/ctim.2001.0488
      OpenUrlCrossRefPubMedWeb of Science
    34. ↵
      1. Jiang Y,
      2. Liu Y,
      3. Shang HC, et al
      . CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Chin J Evid Based Med 2021;21:313–37.
      OpenUrl
    35. ↵
      1. Agha RA,
      2. Fowler AJ,
      3. Limb C, et al
      . Impact of the mandatory implementation of reporting guidelines on reporting quality in a surgical journal: A before and after study. Int J Surg 2016;30:169–72. doi:10.1016/j.ijsu.2016.04.032
      OpenUrlPubMed
    36. ↵
      1. Committee on Strategies for Responsible Sharing of Clinical Trial Data; Board on Health Sciences Policy; Institute of Medicine
      . Sharing clinical trial data: maximizing benefits, minimizing risk. Washington (DC): National Academies Press (US), 2015.
    37. ↵
      1. Watson H,
      2. Gallifant J,
      3. Lai Y, et al
      . Delivering on NIH data sharing requirements: avoiding Open Data in Appearance Only. BMJ Health Care Inform 2023;30:e100771. doi:10.1136/bmjhci-2023-100771
    38. ↵
      1. Anderson JM,
      2. Johnson A,
      3. Rauh S, et al
      . Perceptions and Opinions Towards Data-Sharing: A Survey of Addiction Journal Editorial Board Members. J Sci Pract Integr 2022;2022. doi:10.35122/001c.35597
    39. ↵
      1. van Panhuis WG,
      2. Paul P,
      3. Emerson C, et al
      . A systematic review of barriers to data sharing in public health. BMC Public Health 2014;14:1144. doi:10.1186/1471-2458-14-1144
    40. ↵
      1. Dey P,
      2. Ross JS,
      3. Ritchie JD, et al
      . Data Sharing and Cardiology: Platforms and Possibilities. J Am Coll Cardiol 2017;70:3018–25. doi:10.1016/j.jacc.2017.10.037
      OpenUrlFREE Full Text
    41. ↵
      1. Su N,
      2. van der Linden MW,
      3. Faggion CM, et al
      . Assessment of spin in the abstracts of randomized controlled trials in dental caries with statistically nonsignificant results for primary outcomes: A methodological study. Caries Res 2023;57:553–62. doi:10.1159/000531569
      OpenUrlPubMed
    42. ↵
      1. Mathieu S,
      2. Boutron I,
      3. Moher D, et al
      . Comparison of registered and published primary outcomes in randomized controlled trials. JAMA 2009;302:977–84. doi:10.1001/jama.2009.1242
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      1. Zhang J,
      2. Shang H,
      3. Gao X, et al
      . Acupuncture-related adverse events: a systematic review of the Chinese literature. Bull World Health Organ 2010;88:915–921C. doi:10.2471/BLT.10.076737
      OpenUrlCrossRefPubMedWeb of Science
    44. ↵
      1. Wilkinson MD,
      2. Dumontier M,
      3. Aalbersberg IJJ, et al
      . The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 2016;3:160018. doi:10.1038/sdata.2016.18
    45. ↵
      1. Chan M,
      2. Kazatchkine M,
      3. Lob-Levyt J, et al
      . Meeting the demand for results and accountability: a call for action on health data from eight global health agencies. PLoS Med 2010;7:e1000223. doi:10.1371/journal.pmed.1000223

    Footnotes

    • X @Duanyuting_Dawn

    • YD, PZ and YD contributed equally.

    • Contributors YDu and LY conceptualised and supervised the study and served as guarantors of the study. ZX and PZ developed the search strategies and made the assessment standards. PZ, YDe, XL, ZX, BT and JZ screened, extracted and analysed data. YDu and PZ drafted the manuscript, and LY revised the manuscript. ZX, SW, LS, XL, SL and YDe provided critical comments and substantially improved the quality of the manuscript. All authors provided detailed comments on earlier drafts and approved the final manuscript. The corresponding authors attest that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

    • Funding This study was supported by the Guangzhou Science and Technology Fund (2024A03J0791, 2024A03J0787), Guangzhou Key Lab of Acupuncture-Moxibustion (2023A03J0776), Plan on enhancing scientific research in GMU (GMUCR 2024-01018) and the Guangzhou Health Science and Technology project (20242A011005). The funding body had no role in any process of the study (ie, study design, analysis, interpretation of data, writing of the report and the decision to submit the article for publication). All authors had full access to all the data in the study and accepted the responsibility to submit it for publication.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.