Canadian guidelines for clinical practice: an analysis of their quality and relevance to the care of adults with comorbidity
BMC Family Practice volume 12, Article number: 74 (2011)
Clinical guidelines have been the subject of much criticism in primary care literature partly due to potential conflicts in their implementation among patients with multiple chronic conditions. We assessed the relevance of selected Canadian clinical guidelines on chronic diseases for patients with comorbidity and examined their quality.
We selected 16 chronic medical conditions according to their frequency of occurrence, complexity of treatment, and pertinence to primary care. Recent Canadian clinical guidelines (2004 - 2009) on these conditions, published in English or French, were retrieved. We assessed guideline relevance to the care of patients with comorbidity with a tool developed by Boyd and colleagues. Quality was assessed using the Appraisal of Guidelines Research and Evaluation (AGREE) instrument.
Regarding relevance, 56.2% of guidelines addressed treatment for patients with multiple chronic conditions and 18.8% addressed the issue for older patients. Fifteen guidelines (93.8%) included specific recommendations for patients with one concurrent condition; only three guidelines (18.8%) addressed specific recommendations for patients with two comorbid conditions and one for more than two concurrent comorbid conditions. Quality of the evaluated guidelines was good to very good in four out of the six domains measured using the AGREE instrument. The domains with lower mean scores were Stakeholder Involvement and Applicability.
The quality of the Canadian guidelines examined is generally good, yet their relevance for patients with two or more chronic conditions is very limited and there is room for improvement in this respect.
Chronic disease is currently the leading health problem worldwide and represents an important challenge for healthcare systems . The proportion of total direct medical care expenditures devoted to people with chronic diseases has reached $39 billion a year (2004) in Canada . All chronic diseases combined are estimated to account for over 53% of the total economic burden of illness in Canada .
The proportion of patients affected by multiple concurrent chronic medical conditions (multimorbidity) is on the rise . Data from different countries demonstrate that this situation results in a high burden for primary care practices where multimorbidity is the rule rather than the exception [3–7]. A Canadian study that included 980 adult patients consulting in a primary care setting found that nine out of 10 patients had more than one chronic condition, and approximately 50% had five or more .
Clinical guidelines aim to improve the quality of care provided to patients . Given that guidelines are mostly disease-oriented, they have been the subject of much criticism in primary care literature, partly due to potential conflicting recommendations when implementing those guidelines with patients presenting multiple concurrent chronic conditions [9–12]. This leads to the extended use of physician's own clinical experience and patients' views on treatment choice instead of national guidelines recommendations .
A tool recently developed by Boyd and colleagues  allowed us to evaluate the applicability of guidelines on chronic diseases for the treatment of subjects with comorbidity (the term comorbidity replaces multimorbidity when the focus is on one specific index disease) . This instrument assesses whether guidelines address treatment for people with several comorbid conditions, as well as patient-centered aspects such as patient preferences and quality of life. Canadian guidelines for chronic diseases have never been evaluated this way. We examined the quality and the relevance of Canadian clinical guidelines for patients with comorbidity for selected chronic diseases.
This is a descriptive study based on a literature review and the critical appraisal of clinical guidelines. We initially selected chronic medical conditions according to their prevalence, complexity of treatment, and pertinence to primary care. The selected conditions were: dyslipidemia, chronic impaired renal function, anxiety, heart failure, coagulopathy, obesity, atrial fibrillation, glaucoma, peripheral arterial disease, chronic obstructive pulmonary disease (COPD), asthma, osteoporosis, rheumatoid arthritis, type II diabetes, dementia, high blood pressure, hypothyroidism, and depression. Recent Canadian clinical guidelines (2004 - 2009) published in English or French were selected. Guidelines were identified through the Medline database using the Ovid search engine ("guideline" [publication type] or text word and "Canada" or "Canadian" as text word combined with the corresponding diseases with various spellings as MeSH terms). We also explored several websites known to refer to guidelines: Guidelines Advisory Committee (GAC), Canadian Medical Association InfoBase, Canadian Task Force on Preventive Health Care, Recommended Clinical Practice Guidelines, Canadian Clinical Practice Guideline, Canadian Thoracic Society, Canadian Cardiovascular Society, College of Physicians of Quebec, Canadian Ophthalmological Society and The Canadian Diabetes Association. When more than one guideline had been published by the same organization only the most recent version was retained.
The study was carried out at the Family Medicine Unit of the Centre de santé et de services sociaux de Chicoutimi and provided an opportunity for five family medicine residents to complete their mandatory research training under the guidance and supervision of the principal investigator (MF).
To assess the relevance of the guidelines for the care of patients with multiple chronic diseases, we adapted a 14-item tool developed by Boyd and colleagues  and expanded it to 17 by adding three items related to medication. This instrument, of relatively recent introduction, is the only one available for this purpose. Boyd et al.'s tool is a checklist in which each item is scored as "yes" or "no". The two principal measurement properties of this instrument of interest to us are its content validity and interrater reliability. Content validity is a qualitative assessment that concerns the extent to which the measure includes all relevant items. It was judged to be adequate by the developers of the tool  and by others having used it . We improved its comprehensiveness with the addition of items related to medication. Regarding interrater reliability, the agreement between two reviewers scoring the different items ranged from good to excellent . In this study consensus was reached among independent evaluators. The instrument assesses if guidelines address treatment for older people and for people with several comorbid conditions, as well as patient-centred aspects such as patient preferences and quality of life. A total score was determined for each guideline for comparison purposes.
Guidelines quality was assessed using the Appraisal of Guidelines Research and Evaluation (AGREE) instrument . This instrument comprises 23 items covering six domains. Agreement with the statement for each item is scored on a Likert scale from 1 (complete disagreement) to 4 (complete agreement). This instrument was validated and its reliability is good (Cronbach's alpha 0.64-0.88). A detailed description of the properties of the instrument (reliability and validity) was reported elsewhere . It should be scored by at least two evaluators. Domain scores can be calculated by summing up all the scores obtained for each individual item in a domain and by standardizing the total as a percentage of the maximum possible score for that domain.
The scoring of the two instruments was initially standardized in two steps by five evaluators. First, one guideline (anticoagulant therapy) was discussed and analyzed jointly and consensus was reached for each item. Then, two guidelines (hypertension and anxiety) were read and appraised by the five evaluators independently before results were compared. Differences in rating were resolved by consensus. Thereafter, pairs of evaluators were formed randomly to appraise each guide according to AGREE and Boyd and colleagues' modified instrument. Each evaluator appraised the guidelines independently. A consensus was reached after discussion on divergent items.
Sixteen Canadian guidelines were selected and appraised [18–33]. Hypothyroidism and depression were removed from the original list since we could not find guidelines for the timeframe of the study (2004 - 2009). Table 1 shows the relevance of Canadian clinical guidelines for the treatment of patients with multiple conditions. Nine guidelines (56.2%) addressed treatment for patients with multiple chronic conditions but only three (18.8%) addressed the issue for older patients. Fifteen guidelines (93.8%) included specific recommendations for patients with one comorbid condition; however, only three guidelines (18.8%) addressed specific recommendations for patients with two comorbid conditions and only one (chronic kidney disease) for more than two concurrent comorbid conditions. All guidelines discussed medication side effects, but only 10 (62.5%) discussed possible medication interactions related to comorbidities. Table 2 shows the performance of each Canadian guideline according to Boyd and colleagues' criteria. The guidelines on chronic impaired renal function, heart failure, diabetes mellitus and dementia had the best performance scores, whereas the guidelines on atrial fibrillation and rheumatoid arthritis had the lowest.
The individual standardized AGREE domain scores and mean scores are shown in Table 3. Scope and Purpose and Clarity and Presentation obtained the highest and second highest mean scores, respectively. Rigor of Development and Editorial Independence were good although the latter showed much variability. The Stakeholder Involvement and Applicability domains obtained the lowest scores. Five guidelines showed good quality (score above 60%) in all domains: Chronic Impaired Renal Function, Obesity, COPD, Diabetes Mellitus and Glaucoma, the last two getting the highest marks.
Although 56.2% of the guidelines addressed the treatment of patients with multiple comorbid conditions, only a minority (18.8%) provided specific treatment recommendations for patients with two other conditions and even less for those with more than two comorbid conditions. This does not reflect the reality of primary care where there is a very high prevalence of multimorbidity [3–5]. These results are similar to an Australian study reporting that 53% of 17 guidelines addressed treatment for patients with multiple comorbid conditions . One can argue that due to the diversity of chronic diseases seen in primary care, one cannot account for all possible combinations and adjust guidelines accordingly. Chances are that the evidence to support such recommendations would be weak or absent as randomized control trials often exclude patients with comorbidity or fail to describe the morbidity burden of their participants in published reports . Of particular concern is the fact that the issue of multiple comorbid conditions is rarely discussed for older patients in the Canadian guidelines despite the strong association between aging and the presence of multiple chronic conditions [3–5]. This situation is not surprising considering the reliance on randomized control trials (RCTs) that have excluded patients with comorbidities in the development of evidence-based guidelines . This calls for a dramatic shift in clinical research towards more pragmatic trials to generate evidence and a greater implication of the primary care sector in the research process. Research on new models of care for more complex patients and implying interdisciplinary collaboration is promising but thus far, publications are scarce . A relatively low coverage of burden of treatment and patient preferences is also an indication of the lack of focus on the patient's perspective in Canadian guidelines on chronic diseases. This is in line with the disease-oriented paradigm from which the guidelines arise. This is a concrete example of the misconceptualization of the primary care role from the guidelines developer perspective as reported by Starfield . All guidelines discussed medication side effects, with 62.5% of them discussing, to some extent, possible interactions related to comorbidity. However, in most cases, the discussion was superficial and limited to a few warnings. Both drug-drug and drug-disease interactions are possible in patients with comorbidities . Appropriate management of comorbidity should be included in guidelines for particular diseases . Most guidelines failed to discuss the burden of comprehensive treatment on patients or caregivers. This is especially of concern for older patients with multimorbidity who may find it very difficult to adopt the regimen issued from the many guidelines that would apply to them . Summing up the score for individual items for each guideline from Boyd and colleagues' instrument resulted in an overall score allowing for comparisons between guidelines. Guidelines for diabetes mellitus performed the best. This is reassuring as diabetes is associated with many chronic diseases and complications. However, a low score for atrial fibrillation is especially of concern, as the treatment of this condition will interfere with many other conditions.
The quality of the guidelines evaluated in this study was good to very good in four out of the six domains measured by the AGREE instrument. This evaluation is necessary to ensure that the guidelines are of good quality according to Boyd and colleagues' criteria. The domains with lower scores were: Stakeholder Involvement and Applicability. Guidelines devoted to improved quality of primary care would certainly benefit from a strong partnership with primary care providers in charge of applying the guidelines and this is especially of concern due to the high prevalence of chronic diseases in this setting. The low scores of those two criteria call for a more pragmatic way of elaborating and formulating guidelines in order to better fit the needs of targeted users. There is no use for a guideline if its implementation into practice is not facilitated and does not entail a shift of paradigm from the more disease-oriented guideline developers to the more patient-oriented users [38, 39]. The Scope and Purpose, and Clarity and Presentation domains obtained very good scores; this, along with the good scores of the Rigor of Development domain, speaks well to the scientific quality and validity of guidelines that is the hallmark of Canadian research.
This study has limitations. Although our selection criteria allowed us to include a variety of important medical conditions, our results cannot be generalized to other Canadian guidelines not included in this study. Another limitation is that Boyd and colleagues' instrument is a relatively new tool; this may raise questions about its use in evaluating guidelines. The AGREE instrument provides a well validated framework for the assessment of the quality of guidelines. However, ratings can be influenced by the reviewer's background knowledge of the guideline subject. We tried to reduce this bias by distributing guidelines randomly among the reviewers.
Our results show that the quality of Canadian guidelines is good, particularly in their scope and purpose, as well as in clarity and presentation. However, their relevance for patients with more than two comorbid conditions is very limited and there is room for improvement in this respect. It may be the reflection of some disconnection between guideline developers and primary care providers. Clinical judgment is still necessary to adjust guideline recommendations for patients with comorbidity.
Innovative Care for Chronic Conditions. Building blocks for action. [http://www.who.int/diabetesactiononline/about/icccglobalreport.pdf]
The cost of chronic disease in Canada. [http://www.gpiatlantic.org/pdf/health/chroniccanada.pdf]
Uijen AA, van de Lisdonk EH: Multimorbidity in primary care: prevalence and trend over the last 20 years. Eur J Gen Pract. 2008, 1: 28-32.
Fortin M, Bravo G, Hudon C, Vanasse A, Lapointe L: Prevalence of multimorbidity among adults seen in family practice. Ann Fam Med. 2005, 3: 223-228. 10.1370/afm.272.
van den Akker M, Buntinx F, Metsemakers JF, Roos S, Knottnerus JA: Multimorbidity in general practice: prevalence, incidence, and determinants of co-occurring chronic and recurrent diseases. J Clin Epidemiol. 1998, 51: 367-375. 10.1016/S0895-4356(97)00306-5.
Britt HC, Harrison CM, Miller GC, Knox SA: Prevalence and patterns of multimorbidity in Australia. Med J Aust. 2008, 189: 72-77.
Schram MT, Frijters D, van de Lisdonk EH, Ploemacher J, de Craen AJ, de Waal MW, van Rooij FJ, Heeringa J, Hofman A, Deeg DJ, Schellevis FG: Setting and registry characteristics affect the prevalence and nature of multimorbidity in the elderly. J Clin Epidemiol. 2008, 61: 1104-1112. 10.1016/j.jclinepi.2007.11.021.
Garber AM: Evidence-based guidelines as a foundation for performance incentives. Health Aff (Millwood). 2005, 24: 174-179. 10.1377/hlthaff.24.1.174.
Starfield B: Threads and yarns: weaving the tapestry of comorbidity. Ann Fam Med. 2006, 4: 101-103. 10.1370/afm.524.
van Weel C, Schellevis FG: Comorbidity and guidelines: conflicting interests. Lancet. 2006, 367: 550-551. 10.1016/S0140-6736(06)68198-1.
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, Rubin HR: Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999, 282: 1458-1465. 10.1001/jama.282.15.1458.
Braithwaite RS, Fiellin D, Justice AC: The payoff time: a flexible framework to help clinicians decide when patients with comorbid disease are not likely to benefit from practice guidelines. Med Care. 2009, 47: 610-617. 10.1097/MLR.0b013e31819748d5.
Ekerstad N, Löfmark R, Carlsson P: Elderly people with multi-morbidity and acute coronary syndrome: doctors' views on decision-making. Scand J Public Health. 2010, 38: 325-331. 10.1177/1403494809354359.
Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW: Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. Jama. 2005, 294: 716-724. 10.1001/jama.294.6.716.
Vitry AI, Zhang Y: Quality of Australian clinical guidelines and relevance to the care of older people with multiple comorbid conditions. Med J Aust. 2008, 189: 360-365.
AGREE (appraisal of guidelines research and evaluation) Collaboration. AGREE instrument. [http://www.agreecollaboration.org/instrument/]
AGREE Collaboration: Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003, 12: 18-23.
McPherson R, Frohlich J, Fodor G, Genest J, Canadian Cardiovascular Society: Canadian Cardiovascular Society position statement--recommendations for the diagnosis and treatment of dyslipidemia and prevention of cardiovascular disease. Can J Cardiol. 2006, 22: 913-927. 10.1016/S0828-282X(06)70310-5.
Levin A, Hemmelgarn B, Culleton B, Tobe S, McFarlane P, Ruzicka M, Burns K, Manns B, White C, Madore F, et al: Guidelines for the management of chronic kidney disease. CMAJ. 2008, 179: 1154-1162. 10.1503/cmaj.080351.
Arnold JM, Liu P, Demers C, Dorian P, Giannetti N, Haddad H, Heckman GA, Howlett JG, Ignaszewski A, Johnstone DE, et al: Canadian Cardiovascular Society consensus conference recommendations on heart failure 2006: diagnosis and management. Can J Cardiol. 2006, 22: 23-45. 10.1016/S0828-282X(06)70237-9.
Anticoagulothérapie en milieu ambulatoire. [http://www.umfrimouski.com/anticoag.pdf]
Lau DC, Douketis JD, Morrison KM, Hramiak IM, Sharma AM, Ur E, Obesity Canada Clinical Practice Guidelines Expert Panel: 2006 Canadian clinical practice guidelines on the management and prevention of obesity in adults and children [summary]. CMAJ. 2007, 176: S1-13.
2004 Canadian Cardiovascular Society Consensus Conference: Atrial Fibrillation. Can J Cardiol. 2005, 21: 9B-73B.
2005 Canadian cardiovascular society consensus conference. Peripheral arterial disease. [http://www.ccs.ca/download/consensus_conference/consensus_conference_archives/CCFinalPre_CJC_Pub.pdf]
O'Donnell DE, Aaron S, Bourbeau J, Hernandez P, Marciniuk D, Balter M, Ford G, Gervais A, Goldstein R, Hodder R, et al: Canadian Thoracic Society recommendations for management of chronic obstructive pulmonary disease--2003. Can Respir J. 2003, 10 (Suppl A): 11A-65A.
Brown JP, Fortier M: Conférence canadienne de consensus sur l'ostéoporose, mise à jour 2006. J Obstet Gynaecol Can. 2006, 28: S111-S131.
Rheumatoid Arthritis: Diagnosis and Management. [http://www.bcguidelines.ca/gpac/pdf/rheumatoid_arthritis.pdf]
Comité d'experts des Lignes directrices de pratique clinique de l'Association canadienne du diabète: Lignes directrices de pratique clinique 2008 de l'Association canadienne du diabète pour la prévention et le traitement du diabète au Canada. Can J Diabetes. 2008, 32: S1-S225.
Lemière C, Bai T, Balter M, Bayliff C, Becker A, Boulet LP, Bowie D, Cartier A, Cave A, Chapman K, et al: Adult Asthma Consensus Guidelines update 2003. Can Respir J. 2004, 11 (Suppl A): 9A-18A.
Troisième conférence canadienne de consensus sur le diagnostic et le traitement de la démence. [http://www.cccdtd.ca/pdfs/Recommandations_approuvees_CCCDTD_2007.pdf]
Canadian Ophthalmological Society Glaucoma Clinical Practice Guideline Expert Committee: Canadian Ophthalmological Society evidence-based clinical practice guidelines for the management of glaucoma in the adult eye. Can J Ophthalmol. 2009, 44: S7-S93.
Swinson RP, Antony MM, Bleau P, Chokka P, Craven M, Fallu A, Kjernisted K, Lanius R, Manassis K, et al: Guide de pratique clinique. Traitement des troubles anxieux. Rev can psychiatrie. 2006, 51: 7S-96S.
Les recommandations de 2009 du programme éducartif canadien sur l'hypertension. [http://www.hypertension.ca/downloads/chep/ids/docs/booklets/2009FullRecommendationsFrench.pdf]
Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L: Randomized clinical trials: Do they have external validity for patients with multiple comorbidities?. Ann Fam Med. 2006, 4: 104-108. 10.1370/afm.516.
Smith SM, Soubhi H, Fortin M, Hudon C, O'Dowd T: Interventions to improve outcomes in patients with multimorbidity in primary care and community settings: a systematic review. Fam Med. 2010, 42 (supp 2): Presentations Given At The 2009 NAPCRG Annual Meeting
Tinetti ME, Bogardus ST, Agostini JV: Potential pitfalls of disease-specific guidelines for patients with multiple conditions. N Engl J Med. 2004, 351: 2870-2874. 10.1056/NEJMsb042458.
Caughey GE, Roughead EE, Vitry AI, McDermott RA, Shakib S, Gilbert AL: Comorbidity in the elderly with diabetes: Identification of areas of potential treatment conflicts. Diabetes Res Clin Pract. 2010, 87: 385-393. 10.1016/j.diabres.2009.10.019.
Grumback K: Chronic illness, comorbidities, and the need for medical generalism. Ann Fam Med. 2003, 1: 4-7. 10.1370/afm.47.
Dawes M: Co-morbidity: we need a guideline for each patient not a guideline for each disease. Fam Pract. 2010, 27: 1-2. 10.1093/fampra/cmp106.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2296/12/74/prepub
Acknowledgements and Funding
We want to thank Drs. Claudie Leroux, Christelle Potvin and Malakamir Malakajoo for their participation in the project as collaborators. They were first-year residents in family medicine when the study was carried out. Martin Fortin is funded by the Canadian Institutes of Health Research and partners (Applied CIHR Chair on Health Services and Policy Research on chronic diseases in primary care/CIHR-IHSPR-CHSRF-CSSSC).
The authors declare that they have no competing interests.
MF and M-EP conceived and designed the study. They also supervised the data collection, analyzed the data and drafted the manuscript. EC and CS participated in the data collection and analysis and critically reviewed the manuscript. JA and CH participated in the data analysis, and helped draft the manuscript. All authors gave their final approval of the version of the manuscript submitted for publication. MF takes responsibility for the integrity of the work as a whole.
About this article
Cite this article
Fortin, M., Contant, E., Savard, C. et al. Canadian guidelines for clinical practice: an analysis of their quality and relevance to the care of adults with comorbidity. BMC Fam Pract 12, 74 (2011). https://doi.org/10.1186/1471-2296-12-74
- Chronic Obstructive Pulmonary Disease
- Comorbid Condition
- Chronic Medical Condition
- Stakeholder Involvement
- Multiple Chronic Condition