You are here

  • Home
  • Online First
  • Identifying actionable statements in Chinese health guidelines: a cross-sectional study

Article Text

Article menu

Download PDFPDF

Original research
Identifying actionable statements in Chinese health guidelines: a cross-sectional study
  1. Xiangying Ren1,
  2. Tamara Lotfi2,3,
  3. Jiyu Chen4,
  4. Yuling Lei5,
  5. Chenyibei Zhou4,
  6. Wei Zhang4,
  7. Qiao Huang1,
  8. Yongbo Wang1,
  9. Siyu Yan1,
  10. Shichun Wang1,
  11. Siyuan Ruan1,
  12. Wanru Wang1,
  13. Qiyi Zhang1,
  14. Xiaomei Yao2,6,
  15. Yinghui Jin1,
  16. Holger J Schuenemann2,3,7
  1. 1Center for Evidence-Based and Translational Medicine, Zhongnan Hospital of Wuhan University, Wuhan, Hubei, China
  2. 2Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
  3. 3Michael G. DeGroote Cochrane Canada and GRADE Centre, McMaster University, Hamilton, Ontario, Canada
  4. 4School of Nursing, Wuhan University, Wuhan, Hubei, China
  5. 5School of Nursing, Hangzhou Normal University, Hangzhou, Zhejiang, China
  6. 6Center for Clinical Practice Guideline Conduction and Evaluation, Children’s Hospital of Fudan University, Shanghai, China
  7. 7Clinical Epidemiology and Research Center (CERC), Humanitas University and Research Hospital, Roberto Rocca Innovation Building, Via Rita Levi Montalcini 4-20090 Pieve Emanuele, Milan, Italy
  1. Correspondence to Yinghui Jin; jinyinghuiebm{at}163.com; Xiaomei Yao; yaoxia{at}mcmaster.ca

Abstract

Objective The purpose of this study is to validate the taxonomy and framework using Chinese guidelines and identify actionable statements.

Design and setting We searched five databases, to identify the health guidelines from 1 January 2020 to 1 May 2023. Five researchers categorised statements into six types: formal recommendations (Type I) with clear direction and strength, with explicit and direct evidence; good practice statements (GPS) (Type II), actionable in isolation with a significant benefit; remarks (Type III), an inseparable unit belonging to a formal recommendation or GPS that provides additional clarification; research only recommendations (Type IV) for specific populations; implementation considerations, tools and tips (Type V), that describe the how, who, where, what and when, in relation to implementing a recommendation and lacking a direct evidence link; and informal recommendations (Type VI), unrelated to evidence and not meeting GPS criteria.

Results We included 116 guidelines, including 74 Western medicine guidelines, 12 traditional Chinese medicine guidelines and 30 integrated Chinese and Western medicine guidelines. 99 guidelines (85.3%) used the Grading of Recommendations Assessment, Development and Evaluation criteria. Medical specialty societies developed the highest number of guidelines (53.4%). Of all the statements, 4422 statements were extracted from the guidelines. Among them, 2154 (48.7%) were formal recommendations, 197 (4.4%) were GPS, 394 (8.9%) were remarks, 16 (0.4%) were research only recommendations, 1106 (25.0%) were implementation considerations, tools and tips, and 555 (12.6%) were informal recommendations.

Conclusions Up to date, the Chinese guideline developers tend to overestimate the number of formal recommendations and underestimate the number of GPS, remarks, research only recommendations, implementation considerations, tools and tips, and informal recommendations. Thus the current quality of actionable statements in Chinese health guidelines requires further enhancement.

  • Methods

Data availability statement

No data are available.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjebm-2024-113050

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • A preliminary search of the China National Knowledge Infrastructure, Wanfang, VIP Chinese periodical service platform, SinoMed and PubMed databases was conducted on 1 December 2022. No studies that focused on the classification of health guideline recommendations in Chinese guidelines were identified, indicating the necessity for a study to explore this point further.

    WHAT THIS STUDY ADDS

    • Our research indicates that the methodological development of the current Chinese health guideline recommendation process needs further improvement, as 12.6% of statements were informal recommendations and thus, may lack evidence to support them. Guideline developers must adhere to this framework and correctly understand the distinction between formal recommendations and other types of statements.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • It is crucial for guideline developers to strive towards creating formal recommendations that are well-documented with clear evidence and a rigorous development process. This transparency enables users to evaluate the credibility and evidence base of the recommendations, reducing the potential risks associated with the indiscriminate use of informal recommendations by individuals who may struggle to differentiate between formal and informal guidance.

    Introduction

    Health guidelines provide recommendations for physicians and other healthcare professionals and establish standards for clinical practice. The American Institute of Medicine (IOM) updated the definition of guidelines in 2011, which now states that ‘Clinical practice guidelines are statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options’.1 As the most crucial guiding documents in medical practice, the recommendations are essential parts of health guidelines. The recommendations combine evidence with the experience of clinical experts and patients’ preferences to help medical workers make decisions in specific clinical situations, thereby improving medical quality and patient outcomes.

    The recommendations are the link between health research evidence and clinical and public health practice and transform the best research evidence into actionable behaviour statements to be applied in clinical practice. GuideLine Implementability Appraisal points out that most obstacles affecting the implementation of health guidelines lie in the statement and presentation of recommended opinions, such as executability and effect on the process of care.2 The Appraisal of Guidelines for Research and Evaluation II assessment tool, which is the most popularly accepted tool to assess an existing guideline’s quality, also has precise requirements for the recommended opinions of the guidelines.3 An important objective of the 2002 American guidelines Standardisation Conference was to exchange research among guideline developers, implementers and disseminators on preparing guidance statements to promote their implementation.4 Embodied in the value of the guidelines is the aim to develop recommendations that accurately reflect scientific evidence and to incorporate them into decision support systems. Guidelines also should effectively influence the behaviour of health decision-makers and eventually affect patients’ outcomes.

    Lotfi et al published a study in 2021, which classified guideline recommendations into six types, including formal recommendations; research recommendations; good practice statements (GPS); implementation considerations, tools and tips; remarks and informal recommendations. Medical staffs’ understanding and implementation of recommendations can be improved through the identification of these six different levels of recommendations.5

    Over the past few decades, the number of guidelines developed by global governments and private organisations has grown exponentially.6 The number of health guidelines issued in China has also increased significantly over the past few decades, with an average growth rate of 37.1%.7–9 A study analysing Chinese guidelines over 11 years revealed the following findings: only 25.29% of the guidelines clearly stated the grading criteria used within the text; 34.34% explicitly described the recommended criteria; only 26.62% indicated the evidence levels and recommendation strengths; and the methodological quality of 1127 guidelines needs improvement.9 Another study indicated that the current guidelines have low implementability due to unclear and non-specific recommendations and low operational feasibility.10 There is an urgent need to identify formal recommendations and standardise the presentation of recommendations by guideline developers.

    A preliminary search of five databases was conducted on 1 December 2022. During the search, no studies that focused on the classification of recommendations in the Chinese health guidelines were identified. Therefore, there is a need to explore this point further.

    This study aimed to verify six-type guideline’s statement in Chinese guidelines, including formal recommendations, GPS, remarks, research only recommendations, implementation considerations, tools and tips and informal recommendations. However, guideline developers should avoid making informal recommendations in the guidelines. While identifying informal recommendations, Chinese guideline developers are urged to standardise the statement of their guideline recommendations so that clinical practitioners can use recommendations more confidently in clinical practice.

    Methods

    Protocol

    The study protocol was designed prospectively but was not published.

    Data source and search strategy

    A comprehensive literature search was conducted using the following electronic academic databases for potentially relevant records published from 1 January 2020 to 1 May 2023, including China National Knowledge Infrastructure(CNKI), Wanfang Database, VIP Chinese periodical service platform(VIP), SinoMed and PubMed. The Chinese search terms included ‘指南’, ‘实践指南’, ‘临床指南’, ‘临床实践指南’, ‘推荐意见’. The English search terms included ‘guid*’ and ‘recommendation’ (see online supplemental table S1). Additionally, the guidelines in the Chinese Medical Association were manually searched. Any additional articles known to the review authors, even if not captured during the search, were also considered.

    Supplemental material

    Inclusion criteria

    Studies were selected based on the following inclusion criteria:

    1. The articles met the definition of a guideline proposed by IOM and were original guideline or rapid original guideline developed by Chinese scholars.11

    2. If several versions of one guideline existed, the version that included the clearest detail on the guideline development methodology and the updated version was considered.

    3. If one guideline had been published in several parts, they were merged into one complete guideline for the assessment.

    Exclusion criteria

    1. Consensus-based guidelines or adapted/adopted guidelines.11

    2. Guidelines without recommendations and those where the recommendations were not accompanied by the certainty of the evidence and strength of the recommendations.

    3. Guidelines not describe the methodology or with an incomplete description of the methodology (the methodology should include the description of the processes of how to conduct a systematic review and how to move on from evidence to recommendations).

    4. Systematic reviews, editorials, translations, compilations, adaptions, interpretations and short summaries of existing guidelines.

    5. Guidelines for which full texts of guidelines are unavailable.

    Guideline selection

    Articles were imported into EndNote X21 software and independently reviewed by two reviewers based on the eligibility criteria. First, title and abstract screening was independently performed by two reviewers (XR and JC). Second, for all the potentially relevant guidelines, the publications were retrieved and assessed for final inclusion by two reviewers (XR and JC) independently. Disagreement was resolved through discussion with a third author (YJ). Any discrepancies were discussed and resolved through consensus.

    Data extraction

    The data were extracted using a double-extraction method from each eligible study by four independent reviewers (JC, YL, CZ and WZ) who then proceeded to cross-check the results. The following information was extracted: guideline title, journal name, first author name, date of publication, sources of guidelines (National Health Commission of the People’s Republic of China, medical specialty societies, charities or others: those which only listed author information), the content of guidelines, the criteria for grading the quality of evidence and strength of recommendations and the number of recommendations. The reviewers also extracted the sentences where the author mentioned ‘recommend’, ‘suggest’ or other related terms regarding actions. When classifying a recommendation, the supporting evidence and references of the recommendation were reviewed and their relevance was determined by reviewing the title. In cases, where determining whether the evidence was directly or indirectly related to the recommendation based on the title alone was unfeasible, the full text of the evidence was reviewed. All reviewers are familiar with evidence-based medicine and guideline development methodology and are systematically trained in the extraction and classification of recommendations. Disagreements between the reviewers regarding the extracted data were resolved through discussion and consensus with a third reviewer (YJ). If the disagreement remains unresolved, further communication was made with the authors (TL and HJS) of the framework. To ensure the accuracy and consistency of the extracted data, we calculated the intraclass correlation coefficient using Fleiss’ Kappa statistics. A kappa value of ‘1’ for one guideline indicates complete agreement between the two results.12

    Identifying statements

    We identified actionable statements from the included guidelines using the six types proposed by Lotfi et al.5 In brief, a statement with a clear direction and strength to support one intervention and providing an explicit and direct link to systematically gathered evidence, is qualified as a formal recommendation (Type I). A statement can be called GPS (Type II) if it is actionable in isolation with an expected large net benefit, not graded for strength or the certainty of evidence or accompanied by a citation for supporting evidence; and the alternative of the stated statement was judged as illogical or did not conform with ethical norms.13 14 A remark (Type III) is an inseparable unit belonging to a formal recommendation or GPS that provides additional clarification. Research only recommendation (Type IV) is a recommendation that confines the use of intervention options in a specific population, to a research setting. Implementation considerations, tools and tips (Type V) often describe how, who, where, what and when related to implementing a recommendation and may not have a clear link to evidence. Informal recommendations (Type VI) do not directly link to the bodies of evidence assembled for the guideline and do not fulfil the rigorous set of logical rules identifying GPS. A highly credible evidence-based guideline should be able to completely avoid informal recommendations.5

    Data synthesis

    The data was analysed and summarised descriptively, with study characteristics presented in tabular and graphical forms and summarised in the text using a narrative approach. Actionable statements were identified through a comparative analysis of the study characteristics (For example, we used Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group criteria for assessing the appropriateness of issuing a GPS:13 (1) clear and actionable; (2) rationale necessitating the message for healthcare practice; (3) practicality of systematically searching for evidence; (4) likely net positive consequences from implementing the GPS and (5) clear link to the indirect evidence). The reviewer (XR) documented the characteristics, evaluation criteria and results of the included guidelines in an Excel file. The characteristics were aggregated and stratified by the year of guideline development, the content of guidelines and the source of guidelines. The classified characteristics were presented in terms of frequency and percentage and bar charts were created using Excel tables to display the data.

    Results

    Search results

    During the initial literature search, a total of 23 336 records were identified. After removing 9299 duplicates as well as 2231 guideline interpretations, translations and planning documents and excluding 11 136 studies through screening titles and abstracts, 670 studies were reviewed using full texts. Finally, 116 articles met the inclusion criteria and were analysed. Among these, 23 health guidelines were published in 2020, 53 in 2021, 39 in 2022 and 1 in 2023. (See figure 1).

    Figure 1
    Figure 1

    Flow diagram. Abbreviations: CNKI, China National Knowledge Infrastructure; VIP, VIP Chinese periodical service platform.

    Data synthesis

    We selected eight guidelines and had four independent reviewers extract and identify the actionable statements from these eight guidelines.15–22 We then conducted the kappa analysis to ensure the accuracy and consistency of the extracted data. The kappa values of the two guidelines ranged from 0.4 to 0.75, indicating moderate consistency,15 16 while the kappa values of five guidelines ranged from 0.75 to 1, indicating good consistency.17–21 Only one guideline had a kappa value between 0 and 0.4 which may be related to the small number of items in the recommendation statements.22

    General characteristics of guidelines

    Content of guidelines

    Among the 116 guidelines, 74 (63.8%) focused on Western medicine (15 in 2020, 36 in 2021 and 23 in 2022), 12 (10.3%) focused on traditional Chinese medicine (TCM) (5 in 2020, 3 in 2021, 3 in 2022 and 1 in 2023) and 30 (25.9%) guidelines focused on a combination of Western medicine and TCM (3 in 2020, 14 in 2021 and 13 in 2022).

    Grading of evidence and recommendation strength

    A total of 99 guidelines (85.3%) used the GRADE criteria for evidence levels and grading standards, while nine guidelines used modified criteria for evidence levels and recommendation strength.23–31 Two guidelines used different evidence levels and recommendation strength grading criteria (such as the evidence level is classified according to the American Society of Intravenous Nursing’s evidence-based grading system and the recommendation level is classified according to the GRADE in the same guideline).32 33 Three guidelines adopted the standards used by the American Academy of Orthopaedic Surgeons,22 34 35 one used the Oxford Centre for Evidence-Based Medicine criteria,36 one used the European Society of Cardiology criteria37 and one used ‘Suggestions for grading evidence in clinical research based on body of evidence’ which developed by Jianping Liu.38

    Source of guidelines

    Among the 116 guidelines, 27 (23.3%) belonged to government agencies (National Health Commission of the People’s Republic of China and National Administration of Traditional Chinese Medicine), 62 (53.4%) were owned by medical specialty societies (including their branches) and the 27 (23.3%) were developed by others (individual authors without support from the government agencies or medical specialty societies).

    Number of statements

    A total of 4422 recommendation statements were extracted from the guidelines (1021 in 2020, 2166 in 2021, 1184 in 2022 and 51 in 2023). The highest number of statements included in the guidelines was 126,39 whereas the lowest was four.40

    Types of statements

    Based on the definition of the six types of recommendation,5 among the 4422 recommendations, 2154 (48.7%) were identified as formal recommendations (Type I), 197 (4.4%) as GPS (Type II), 394 (8.9%) as remarks (Type III), 16 (0.4%) as research only recommendations (Type IV), 1106 (25.0%) as implementation considerations, tools and tips (Type V) and 555 (12.6%) as informal recommendations (Type VI). (Figure 2 shows the results by years). The detailed data is presented in online supplemental table S2).

    Figure 2
    Figure 2

    Number of statements types by years (2020–2023). Abbreviations: GPS, good practice statements.

    Some guidelines provided vague or poorly reported recommendations. These situations typically include: (1) clearly stating the evidence grade in the recommendation but failing to provide corresponding evidence content or references. (2) Recommendation statements not indicating the evidence grade and strength of recommendation. (3) Instances where guideline developers made errors in interpreting the evidence. These problems undoubtedly affect users confidence in the guidelines. We classified the recommendations that met any of the above three criteria as GPS or informal recommendations.

    Evaluation of total formal recommendations (Type I)

    The agreement between the judgement by the guideline developers (numerator) and our judgement according to the features of the formal recommendations (denominator) were reported below.

    For the source of the guidelines, the proportion of labelled as formal recommendations and qualified formal recommendations by the government agencies, the medical specialty societies and others were 566/429 (131.9%), 1961/1311 (149.6%) and 497/414 (120%), respectively. Overall, 2154 statements were qualified as formal recommendations by our judgement (429+1311+414=2154).

    For the contents of guidelines, the proportion of labelled as formal recommendations and qualified formal recommendations in the Western medicine guidelines, the TCM guidelines and the integrated Chinese and Western medicine guidelines were 2106/1397 (150.8%), 355/290 (122.4%) and 563/467 (120.6%), respectively, (1397+290+467=2154). The detailed data are presented in online supplemental tables S3,S4.

    Among 2154 statements that were qualified as formal recommendations, 61 (2.8%) were not labelled by developers. Among 3024 statements that were labelled as formal recommendations, 931 (30.8%) were misclassified as formal recommendations by the guideline developers (see online supplemental table S5).

    Evaluation of total GPS (Type II)

    For the source of the guidelines, the proportion of labelled as GPS and qualified GPS by the government agencies, the others and the medical specialty societies were 0/6, 21/52 (40.4%) and 58/139 (41.7%), respectively. Overall, 197 statements were qualified as GPS by our judgement (0+52+139=197).

    For the contents of guidelines, the proportion of labelled as GPS and qualified GPS in the Western medicine guidelines, the TCM guidelines and the integrated Chinese and Western medicine guidelines were 57/145 (39.3%), 22/49 (44.9%) and 0/3, respectively, (145+49+3=197). The detailed data is presented in online supplemental table S6,S7.

    Among 197 statements that were qualified as GPS, 123 (62.4%) were not labelled by developers. Among 79 statements that were labelled as formal recommendations, 57 (72.2%) were misclassified by the guideline developers (see online supplemental table S8).

    Evaluation of total remarks (Type III)

    For the source of the guidelines, the proportion of labelled as remarks and qualified remarks by the government agencies, the others and the medical specialty societies were 10/42 (23.8%), 6/87 (6.8%) and 95/265 (35.8%), respectively. Overall, 394 statements were qualified as remarks by our judgement (42+87+265=394).

    For the contents of guidelines, the proportion of labelled as remarks and qualified remarks in the Western medicine guidelines, the TCM guidelines and the integrated Chinese and Western medicine guidelines were 95/320 (29.7%), 6/22 (27.3%) and 10/52 (19.2%), respectively, (320+22+52=394). The detailed data is presented in online supplemental tables S9,S10.

    Among 394 statements that were qualified as remarks, 320 (81.2%) were not labelled by developers. Among 111 statements that were labelled as formal recommendations, 37 (33.3%) were misclassified by the guideline developers (see online supplemental table S11).

    Evaluation of total implementation considerations, tools and tips (Type V)

    For the source of the guidelines, the proportion of labelled as implementation considerations, tools and tips and qualified implementation considerations, tools and tips by the government agencies, the others and the medical specialty societies were 0/208, 1/134 (0.7%) and 17/764 (2.2%), respectively. Overall, 1106 statements were qualified as implementation considerations, tools and tips by our judgement (208+134+764=1106).

    For the contents of guidelines, the proportion of labelled as implementation considerations, tools and tips and qualified implementation considerations, tools and tips in the Western medicine guidelines, the TCM guidelines and the integrated Chinese and Western medicine guidelines were 18/636 (2.8%), 0/221 and 0/249, respectively, (636+221+249=1106). The detailed data is presented in online supplemental tables S12,S13.

    Among 1106 statements that were qualified as implementation considerations, tools and tips, 841 (76.0%) were not labelled by developers. Among 35 statements that were labelled as implementation considerations, tools and tips, 0 were misclassified by the guideline developers (see online supplemental table S14).

    Evaluation of research only recommendations (Type IV) and informal recommendations (Type VI)

    Although we identified 16 (0.4%) research only recommendations and 555 (12.6%) informal recommendations, none of these types were labelled by guideline developers.

    Discussion

    In this study, we analysed the general characteristics of information in the Chinese health guidelines, particularly, the types of recommendations or actionable statements they include. A total of 116 health guidelines were included, with 85.3% of guidelines which used the GRADE approach. More than 36% focused on TCM or a combination of Western medicine and TCM. Most of the guidelines (53.4%) were developed by medical specialty societies. Among 4422 recommendations identified, 2154 (48.7%) were formal recommendations (Type I), 197 (4.4%) were GPS (Type II), 394 (8.9%) were remarks (Type III), 16 (0.4%) were research only recommendations (Type IV), 1106 (25.0%) were implementation considerations, tools and tips (Type V) and 555 (12.6%) were informal recommendations (Type VI). We also found that guideline developers tend to overestimate the number of formal recommendations and underestimate the number of GPS, remarks, research only recommendations, implementation considerations, tools and tips and informal recommendations which is consistent with our expectations.

    Implications for guideline developers

    Our research indicates that guideline developers need to provide further detailed reporting on GPS, remarks, research only recommendations and implementation considerations, tools and tips during the guidelines development process. It is also crucial to note that not all recommendations are considered formal recommendations, they should be formally deliberated on and have explicit and direct links to the related bodies of evidence. Guideline developers also should use signalling questions to determine whether a GPS should be developed before proceeding with its development.12 Additionally, during the extraction and identification of recommendations, we discovered that some recommendations lacked evidence to support or misinterpretations of evidence by guideline developers which could potentially lead to misleading use of recommendations in clinical practice.

    Nations and organisations have dedicated considerable time, effort and financial resources to developing health guidelines to improve the quality of medical services, optimise the allocation of medical resources and bridge the gap between evidence and clinical practice. Nevertheless, the uptake and adherence to guideline recommendations among clinicians remain notably low. Studies have shown that the factors hindering the utilisation of the guidelines include such as ‘Lack of applicability, for example, lack of a clear, feasible and practical implementation method; or too simple to solve the patient’s practical problem’, ‘Ambiguity and lack of clarity’.41 In fact, the process of how to make recommendations from systematic reviews is often not reported transparently.42 Furthermore, when making recommendations, the guidelines’ makers usually do not consider the attitudes and preferences of patients and users and the factors that affect implementation and some recommendations also lack strong evidence to support recommendations.43–45 This will affect the confidence of end users in implementing the guidelines’ recommendations.46 47

    Use of evidence

    One of the key points for clinicians to rapidly assess the credibility of recommendation statements is to determine whether they are based on the most recent and comprehensive evidence, with the certainty of evidence supporting the recommendations.48 This point is also reflected in Lotfi et al’s framework, as it requires a clear and direct connection between the evidence and the formal recommendation statements.5 In this area, the Western medicine guidelines performed poorly compared with TCM guidelines and integrated Chinese and Western medicine guidelines. Specifically, among the 555 informal recommendations, the proportion of informal recommendations in Western medicine guidelines was 411 (74.1%), while those in TCM and integrated guidelines were 63 (11.4%) and 81 (14.6%), respectively. This suggests that guideline developers need to focus on the correlation between evidence and recommendations. Additionally, it is noteworthy that, due to the strict inclusion and exclusion criteria set in our study, several guidelines with lower methodological reporting quality (such as those lacking recommendation strength and grading criteria) were excluded during the literature screening phase. According to the classification of statements, the majority of recommendations within these excluded guidelines would be qualified as informal recommendations. Consequently, the actual proportion of informal recommendations should substantially surpass the reported 12.6%.

    Clear and specific recommendations

    Clear and specific recommendations are the core contents of evidence-based health guidelines, but implementation considerations, tools and tips often describe the how, who, where, what and when related to implementing a recommendation (Type V).5 Approximately half (42.5%) of the implementation considerations, tools and tips found in the current study were provided by TCM or integrated Chinese and Western medicine guidelines, although they together accounted for 42 out of 116 (36.2%) guidelines. This may be due to the unique features of TCM. In TCM guidelines, since the recommended drugs are usually formulations or decoctions, a single dose of Chinese medicine can potentially comprise up to 12 ingredients. Consequently, the drug composition, the dosage for each ingredient and usage instructions are provided below the recommendation.12 49 These implementation considerations and tips help to reduce the inappropriate use of Chinese herbal medicine or patent medicine while optimising the medicine’s effectiveness and minimising the risk of adverse reactions.50 51

    One study retrieved COVID-19 guidelines published between 1 March 2020 and 24 September 2021, identifying actionable statements and assessing the development and quality of GPS within these guidelines. The findings indicate that 64% of the statements met the GPS criteria, whereas only 1% were designated as GPS by the guideline developers.13 These findings deviate somewhat from ours and are potentially influenced by crisis-driven constraints on developers, necessitating a delicate balance between methodological rigour and expeditious guidance production.

    As the number of global guidelines continues to rise,6 this framework will help enhance the transparency and rigour in the development of recommendations in clinical practice guidelines. In the future, we will further identify and analyse the types of statements in global guidelines and compare the findings with the results from Chinese guidelines.

    Overall, guideline developers need to adhere to the framework and use more explicit and transparent methods to formulate statements in order to avoid informal recommendations. Meanwhile, developers also need to explicitly label and identify the types of statements.

    Strengths and limitations

    The strengths of this study include that this is the first systematic evaluation of a large sample of recent Chinese health guidelines irrespective of topics and publication sources. We recognise that the nature of the judgement depends on the expertise and knowledge of the evaluator which can exhibit some variability. To increase confidence, every evaluation was performed by trained reviewers and then verified by an expert in guideline development to validate the decisions soundly and accurately. However, our study has some limitations. We did not search grey literature. We only analysed the actionable statements of health guidelines from 2020 to 1 May 2023 which does not reflect the status of guidelines in China in other years.

    Conclusion

    Applying Lotfi et al’s framework to identify actionable statements in Chinese health guidelines allowed us to detect various issues in the formulation and presentation of recommendations in the guidelines. The quality of statements in Chinese health guidelines needs to be further improved, as 12.6% of statements were informal recommendations. When formulating recommendations, guideline developers should (1) provide clear evidence to support the recommendation and state the reasons for the recommendation underneath them; (2) provide transparent grading of evidence and strength of recommendation; (3) articulate clear statements of clinical problems and clear and actionable recommendations; and (4) ensure clear methodological content to ensure clarity, specificity and transparency in the guidance development process. Implementing these strategies can significantly elevate the quality and applicability of recommendations within Chinese health guidelines.

    Data availability statement

    No data are available.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    Acknowledgments

    We thank Jean Glover from Tianjin Golden Framework Consulting Company for English editing.

    References

    1. ↵
      1. Graham R,
      2. Mancher M,
      3. Miller Wolman D
      , eds. Institute of medicine committee on standards for developing trustworthy clinical practice guidelines. In: Clinical practice guidelines we can trust. Washington (DC): National Academies Press (US), 2011.
    2. ↵
      1. Shiffman RN,
      2. Dixon J,
      3. Brandt C, et al
      . The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak 2005;5:23. doi:10.1186/1472-6947-5-23
    3. ↵
      1. Brouwers MC,
      2. Kho ME,
      3. Browman GP, et al
      . AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ 2010;182:E839–42. doi:10.1503/cmaj.090449
      OpenUrlFREE Full Text
    4. ↵
      1. Shiffman RN,
      2. Shekelle P,
      3. Overhage JM, et al
      . Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization. Ann Intern Med 2003;139:493–8. doi:10.7326/0003-4819-139-6-200309160-00013
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Lotfi T,
      2. Hajizadeh A,
      3. Moja L, et al
      . A taxonomy and framework for identifying and developing actionable statements in guidelines suggests avoiding informal recommendations. J Clin Epidemiol 2022;141:161–71. doi:10.1016/j.jclinepi.2021.09.028
      OpenUrlCrossRefPubMed
    6. ↵
      1. Qaseem A,
      2. Forland F,
      3. Macbeth F, et al
      . Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med 2012;156:525–31. doi:10.7326/0003-4819-156-7-201204030-00009
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Zheng Z,
      2. Cui S,
      3. Lu X, et al
      . Analysis of the status of Chinese clinical practice guidelines development. BMC Health Serv Res 2012;12:218. doi:10.1186/1472-6963-12-218
    8. ↵
      1. Chen Y-L,
      2. Yao L,
      3. Xiao X-J, et al
      . Quality assessment of clinical guidelines in China: 1993 - 2010. Chin Med J (Engl) 2012;125:3660–4.
      OpenUrlCrossRefPubMed
    9. ↵
      1. Zhang R,
      2. Yan S-Y,
      3. Wang Y-Y, et al
      . Analysis of the Status and Trends of Chinese Clinical Practice Guideline Development Between 2010 and 2020: A Systematic Review. Front Med (Lausanne) 2021;8:758617. doi:10.3389/fmed.2021.758617
    10. ↵
      1. Wang D-Q,
      2. Huang Q,
      3. Huang X, et al
      . Knowledge of and Compliance With Guidelines in the Management of Non-Muscle-Invasive Bladder Cancer: A Survey of Chinese Urologists. Front Oncol 2021;11:735704. doi:10.3389/fonc.2021.735704
    11. ↵
      1. Yao X,
      2. Xia J,
      3. Jin Y, et al
      . Methodological approaches for developing, reporting, and assessing evidence-based clinical practice guidelines: a systematic survey. J Clin Epidemiol 2022;146:77–85. doi:10.1016/j.jclinepi.2022.02.015
      OpenUrlCrossRefPubMed
    12. ↵
      1. Qingquan L,
      2. Tengfei C,
      3. Guozhen Z, et al
      . Clinical practice guideline for the treatment of influenza by traditional chinese medicine(2021). J Tradit Chin Med 2022;63:85–98. doi:10.13288/j.11-2166/r.2022.01.016
      OpenUrl
    13. ↵
      1. Dewidar O,
      2. Lotfi T,
      3. Langendam M, et al
      . Which actionable statements qualify as good practice statements In Covid-19 guidelines? A systematic appraisal. BMJ Evid Based Med 2022;27:361–9. doi:10.1136/bmjebm-2021-111866
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Dewidar O,
      2. Lotfi T,
      3. Langendam MW, et al
      . Good or best practice statements: proposal for the operationalisation and implementation of GRADE guidance. BMJ Evid Based Med 2023;28:189–96. doi:10.1136/bmjebm-2022-111962
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Gong C,
      2. Jingying J,
      3. Yue T, et al
      . Evidence-based practice guidelines for diagnosis and treatment of biliary atresia. Chin J Evid Based Pediatr 2022;17:245–59.
      OpenUrl
    16. ↵
      1. Qinwei G,
      2. Lu B,
      3. Jianchao G, et al
      . An evidence-based clinical guideline for the diagnosis and treatment of lateral ankle avulsion fracture (2022 version). Chin J Trauma 2022;38:769–75. doi:10.3760/cma.j.cn501098-20220224-00135
      OpenUrl
    17. ↵
      1. Pediatric Osteopathic Group, Pediatric Surgery Branch of Chinese Medical Association, ea al
      . Chinese clinical guideline for diagnosis and treatment of anterior cruciate ligament injury in children and adolescents: Tibial spine avulsion fractures. Chin J Orthop 2022;42:129–40. doi:10.3760/cma.j.cn121113-20211109-00642
      OpenUrl
    18. ↵
      1. Wei Z,
      2. Jing Z,
      3. Huilian C, et al
      . Evidence-based guidelines for food allergy of children in China. Chin J Appl Clin Pediatr 2022;37:572–83. doi:10.3760/cma.j.cn101070-20220313-00257
      OpenUrl
    19. ↵
      1. The Subspecialty Group of Nephrology, the Society of Pediatrics, Chinese Medical Association the Editorial Board, Chinese Journal of Pediatrics
      . Evidence-based guideline on the diagnosis and management of growth stardation in pediatric chronic kidney disease. Chin J Pediatr 2022;60:869–76. doi:10.3760/cma.j.cn112140-20220715-00648
      OpenUrl
    20. ↵
      1. Standardization Project Team 'Clinical Application Guidelines for the treatment of superior diseases of proprietary Chinese medicine'.
      Guidelines for the clinical application of proprietary chinese medicine in the treatment of diarrhea in children (2021). CJIM 2022;42:915–21.
      OpenUrl
    21. ↵
      1. Jie H,
      2. Wanqing C,
      3. Ni L, et al
      . China guideline for the screening and early detection of prostate cancer (2022, Beijing). Chin J Oncol 2022;44:29–53.
      OpenUrl
    22. ↵
      1. Tao S,
      2. Jian C,
      3. Zhenfei H, et al
      . Clinical guideline for spinal reconstruction of osteoporotic thoracolumbar fracture in elderly patients (version 2022). Chin J Trauma 2022;38:1057–66. doi:10.3760/cma.j.cn501098-20220912-00605
      OpenUrl
    23. ↵
      1. Chinese Society of Nutritional Oncology CACA
      . Guidelines for clinical diagnosis and treatment of cancer cachexia (2020 Edition). Chin J Clin Oncol 2021;48:379–85. doi:10.3969/j.issn.1000-8179.2021.08.369
      OpenUrl
    24. ↵
      1. World Federation of Chinese Medicine Societies, China Association of Chinese Medicine
      . International traditional chinese medicine clinical practice guidelines for alzheimer’s disease (2019-10-11). World Chin Med 2021;16:1181–7. doi:10.3969/j.issn.1673-7202.2021.08.001
      OpenUrl
    25. ↵
      1. Professional Committee of Spine and Spinal Cord of Chinese Association of Rehabilitation Medicine, Chinese Orthopaedic Association of Orthopaedic Rehabilitation Group
      . Clinical guidelines for nonspecific low back pain in China. Chin J Spine and Spinal Cord 2022;32.
    26. ↵
      Expert group of 'guidelines for the evaluation, treatment and prevention of childhood obesity in China'. Chin J Woman Child Health Res 2021;32:1716–22.
      OpenUrl
    27. ↵
      1. Yi L,
      2. Lang Y,
      3. Ming L, et al
      . Guidelines for surgical treatment of extensive cerebral infarction. Natl Med J China 2021;101.
    28. ↵
      1. National Center of Gerontology, Chinese Society of Geriatrics, Diabetes Professional Committee of Chinese Aging Well Association
      . Guideline for the management of diabetes mellitus in the elderly in China (2021 edition). Chin J Diabetes Mellitus 2021;40:1–33. doi:10.3760/cma.j.issn.0254-9026.2021.01.001
      OpenUrl
    29. ↵
      1. Xudong T,
      2. Yandong W,
      3. Fengyun W, et al
      . Clinical practice guideline for hypochondriac pain. J Trad Chin Med 2020.
    30. ↵
      1. Chinese Society of Cardiology of Chinese Medical Association, Cardiovascular Disease revention and Rehabilitation Committee of Chinese Association of Rehabilitation Medicine, et al
      . Chinese guideline on the primary prevention of cardiovascular diseases. Chin J Cardiol 2020;48:1000–38.
      OpenUrl
    31. ↵
      1. Xudong T,
      2. Yandong W,
      3. Fengyun W, et al
      . Guidelines for clinical diagnosis and treatment of bradygia (gastroptosis). J Trad Chin Med 2020;61.
    32. ↵
      1. Intravenous Infusion Therapy Committee of Chinese Nursing Association
      . Clinical nursing practice guidelines for common complications of intravenous catheters. Chin J Mod Nurs 2022;28:2381–95.
      OpenUrl
    33. ↵
      1. Chinese Association of Integrated Traditional and Western Medicine obstetrics and Gynecology professional committee
      . Guidelines for diagnosis and treatment of early-onset ovarian dysfunction in combination with traditional chinese and western medicine. J Trad Chin Med 2022;63.
    34. ↵
      1. Spinal Trauma Commitee of Orthopedics Branch of Chinese Medical Doctor Association
      . Clinical evidence-based guideline for diagnosis and treatment of hyperextension injury of cervical spinal cord. Chin J Trauma 2021;37:586–92.
      OpenUrl
    35. ↵
      1. Bohua C,
      2. Qixin C,
      3. Liming C, et al
      . Clinical guideline for surgical treatment of symptomatic chronic osteoporotic vertebral fractures. Chin J Trauma 2020;36:577–86.
      OpenUrl
    36. ↵
      1. Chinese Thoracis Society, Chinese Medical Doctor Association Respiratory physician branch
      . Guidelines for diagnosis, treatment and prevention of COVID-19 in adults in China. Natl Med J China 2021;101:1293–356.
      OpenUrl
    37. ↵
      1. Chinese Society of Nutritional Oncology
      . Guidelines for clinical diagnosis and treatment of cancer-related sarcopenia. Electron J Metab Nutr Cancer 2022;9.
    38. ↵
      1. China Association of Chinese Medicine
      . Guidelines for clinical diagnosis and treatment of bacillary dysentery in traditional chinese medicine internal medicine. BJ J Trad Chin Med 2020;39.
    39. ↵
      1. Pulmonary Embolism and Pulmonary Vascular Disease Group of Chinese Thoracis Society, Pulmonary embolism and pulmonary vascular disease working group, et al
      . Guidelines for Diagnosis and Treatment of Pulmonary Hypertension (2021 edition). Natl Med J China 2021;101:11–51. doi:10.3760/cma.j.cn112137-20201008-02778
      OpenUrl
    40. ↵
      1. Long G,
      2. Xiao C,
      3. Qian Z, et al
      . Tuina for children with acute diarrhea: an evidence-based clinical guideline. CJEBM 2021;21:745–53. doi:10.7507/1672-2531.202102098
      OpenUrl
    41. ↵
      1. Jin Y,
      2. Li Z,
      3. Han F, et al
      . Barriers and enablers for the implementation of clinical practice guidelines in China: a mixed-method study. BMJ Open 2019;9:e026328. doi:10.1136/bmjopen-2018-026328
    42. ↵
      1. Guo Q,
      2. Jiang G,
      3. Zhao Q, et al
      . Rapid review: A review of methods and recommendations based on current evidence. J Evid Based Med 2024;17:434–53. doi:10.1111/jebm.12594
      OpenUrlCrossRefPubMed
    43. ↵
      1. Fang-yu X,
      2. Xiu-jie S,
      3. Xiu-yan L, et al
      . Current status of recommendation of low-quality evidence in chinese nursing practice guidelines. Chin J Nurs 2021;28:39–43. doi:10.16460/j.issn1008-9969.2021.20.039
      OpenUrl
    44. ↵
      1. Fanaroff AC,
      2. Califf RM,
      3. Windecker S, et al
      . Levels of Evidence Supporting American College of Cardiology/American Heart Association and European Society of Cardiology Guidelines, 2008-2018. JAMA 2019;321:1069–80. doi:10.1001/jama.2019.1122
      OpenUrlCrossRefPubMed
    45. ↵
      1. Miles KE,
      2. Rodriguez R,
      3. Gross AE, et al
      . Strength of Recommendation and Quality of Evidence for Recommendations in Current Infectious Diseases Society of America Guidelines. Open Forum Infect Dis 2021;8:ofab033. doi:10.1093/ofid/ofab033
    46. ↵
      1. VanWagner LB,
      2. Aghaulor B,
      3. Hussain T, et al
      . When evidence is lacking: a mixed-methods approach for the development of practice guidance in liver transplantation. Gastroenterol Rep (Oxf) 2021;9:22–30. doi:10.1093/gastro/goaa068
      OpenUrlCrossRefPubMed
    47. ↵
      1. Solà I,
      2. Carrasco JM,
      3. Díaz Del Campo P, et al
      . Attitudes and perceptions about clinical guidelines: a qualitative study with Spanish physicians. PLoS ONE 2014;9:e86065. doi:10.1371/journal.pone.0086065
    48. ↵
      1. Zhikang Y,
      2. Hui Y
      . The development, problems and application of clinical practice guidelines: How clinicians can read the essence of clinical guidelines from the surface to the inside. CGP 2023;26:1559–67.
      OpenUrl
    49. ↵
      1. Jin Y-H,
      2. Wang Y-P,
      3. Xie Y-L, et al
      . Research on the development methodology for clinical practice guidelines for organic integration of traditional Chinese and Western medicine. Mil Med Res 2023;10:45. doi:10.1186/s40779-023-00481-9
    50. ↵
      1. Yicheng G,
      2. Qianyun C,
      3. Minjing L, et al
      . Suggestions for developing and writing the acupuncture prescription in the recommendations of clinical practice guidelines. CJEBM 2023;23:1236–40.
      OpenUrl
    51. ↵
      1. Ziteng H,
      2. Ning L,
      3. Lijiao Y, et al
      . Methods for formulating clinical practice guidelines for proprietary chinese medicines (v): Retrieval and synthesis of key information on rational drug use. J Trad Chin Med 2024;65(02):167–71.
      OpenUrl

    Footnotes

    • Contributors Guarantor: YJ. Conceptualisation: XY, YJ. Data access, formal analysis and visualisation: XR, JC, YL, CZ, WZ, QZ, QH, YW, SY, SW, SR, WW. Methodology: XR, TL, XY, YJ, HJS. Writing—review and editing: All authors. Read and approved final version of manuscript: All authors. Critical comments provide: TL, XY, YJ, HJS. Supervision: XY, YJ. All authors have seen and approved the final version of this manuscript for publication.

    • Funding National Natural Science Foundation of China (82174230), Youth Interdisciplinary Special Fund of Zhongnan Hospital of Wuhan University (ZNQNTC2023006).

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.