Practice development plans to improve the primary care management of acute asthma: randomised controlled trial

Background Our professional development plan aimed to improve the primary care management of acute asthma, which is known to be suboptimal. Methods We invited 59 general practices in Grampian, Scotland to participate. Consenting practices were randomised to early and delayed intervention groups. Practices undertook audits of their management of all acute attacks (excluding children under 5 years) occurring in the 3 months preceding baseline, 6-months and 12-months study time-points. The educational programme [including feedback of audit results, attendance at a multidisciplinary interactive workshop, and formulation of development plan by practice teams] was delivered to the early group at baseline and to the delayed group at 6 months. Primary outcome measure was recording of peak flow compared to best/predicted at 6 months. Analyses are presented both with, and without adjustment for clustering. Results 23 consenting practices were randomised: 11 to early intervention. Baseline practice demography was similar. Six early intervention practices withdraw before completing the baseline audit. There was no significant improvement in our primary outcome measure (the proportion with peak flow compared to best/predicted) at either the 6 or 12 month time points after adjustment for baseline and practice effects. However, the between group difference in the adjusted combined assessment score, whilst non-significant at 6 months (Early: 2.48 (SE 0.43) vs. Delayed 2.26 (SE 0.33) p = 0.69) reached significance at 12 m (Early:3.60 (SE 0.35) vs. Delayed 2.30 (SE 0.28) p = 0.02). Conclusion We demonstrated no significant benefit at the a priori 6-month assessment point, though improvement in the objective assessment of attacks was shown after 12 months. Our practice development programme, incorporating audit, feedback and a workshop, successfully engaged the healthcare team of participating practices, though future randomised trials of educational interventions need to recognise that effecting change in primary care practices takes time. Monitoring of the assessment of acute attacks proved to be a feasible and responsive indicator of quality care.


Background
National and international guidelines provide evidencebased advice about the management of acute asthma, emphasising the need for objective assessment, prompt treatment of the attack, and provision of self-management education as part of structured follow-up. [1,2] Despite early hopes about the potential of guidelines to improve practice, [3] and increasing emphasis on ensuring wide dissemination, [4,5] there is continuing concern that the care of acute asthma remains suboptimal. [6][7][8][9] This is of particular importance in primary care where 90% of acute asthma is managed. [6][7][8] Acute asthma is common, [10] with over 100,000 admissions and 1,500 deaths a year attributed to asthma in the UK. [11,12] Confidential enquiries into the cause of asthma deaths over the last three decades have consistently implicated failure to appreciate the severity of the attack resulting in delayed, inadequate emergency treatment as a potentially preventable factor. [13][14][15][16] Didactic lectures and written guidelines, even those including a brief summary of relevant points for clinicians, are known to be ineffective in promoting change in practice [17]. This has stimulated the development of less passive educational interventions. Audit and feedback can change professional practice, though the magnitude of effect varies and questions remain about the most appropriate supporting interventions [18]. Recent policy-driven changes in continuing medical education have led to a shift towards formal needs assessment and multi-professional practice-based learning in primary care through professional development plans [19]. Founded on these concepts a Professional Development Programme was developed by the General Practice Airways Group (a UK primary care interest group). The programme provides a framework for practices wishing to improve their management of acute asthma which incorporates audit, feedback and an interactive workshop. An early pilot study using the programme in three selected, asthma-interested practices, suggested it may have the potential for positively changing practice behaviour [20].
Our randomised controlled trial aimed to establish the effectiveness of the Acute Asthma Professional Development Programme to improve management (specifically the recording of objective assessment of severity) in general practices recruited from one region of Scotland.

Methods
The trial was undertaken during 2002 with approval from Grampian Research Ethics Committee.

Recruitment of practices
We invited, by letter, all 59 general practices in the Aberdeen, and Banff and Buchan regions of Grampian, Scotland to participate in the study. All non-responding practices were telephoned and a personal invitation issued. Participating practices gave their fully informed consent to taking part in all aspects of the development programme.

Randomisation
Using random number tables, consenting practices were centrally randomised to early or delayed intervention groups by a researcher not involved in the trial. It was not possible to blind the practices as the intervention was an active process. The researchers were aware of allocation, though the audit data were submitted to a different centre (Tayside Centre for General Practice, Dundee) to that of the research team organising the educational intervention in Aberdeen.

Procedure
Each enrolled practice undertook audits of their management of acute asthma at baseline, 6-months and 12months using previously piloted methodology [20]. We provided full instructions on how to undertake the audits and a study helpline was available for support. The educational intervention included feedback of audit data and attendance at a workshop during which a practice development plan was formulated by participants. The intervention was provided immediately after the baseline audit to the early intervention group, and immediately after the 6-month audit to the delayed group.

The Acute Asthma Professional Development Programme
The baseline audit Practices undertook a critical event analysis of acute attacks occurring over a 3-month period in adults and children aged 5 years and over. An attack was defined as "an acute deterioration of asthma for which the patient seeks urgent medical advice" [8]. Participants were advised to identify prospectively all acute attacks occurring during the audit period using computer databases, discharge letters, out-of-hours service slips, visit requests, prescriptions for courses of oral steroids and nebuliser use. Data about the management of the attacks was collected retrospectively by a member of the practice team from patients' written and computer records. The previously piloted critical event analysis form (figure 1) was designed to collect data on objective assessment of severity, treatment provided by health professionals and follow-up within 6 weeks post exacerbation [20]. Actions not recorded were assumed not to have been done.
Audit form for the critical event analysis Figure 1 Audit form for the critical event analysis.

Critical Event Analysis -Individual patient data sheet
An acute attack is defined as an acute deterioration in asthma for which the patient seeks urgent medical advice.
Identify attacks which have occurred during the last 3 months. Exclude patients under the age of 5yrs. (Attacks may be identified from discharge letters, out-of-hours service slips, visit requests, prescriptions for courses of oral steroids, nebuliser use) Collect the following data from the patient's written/computer records completing a data sheet for each episode. Data not recorded is assumed not to have been done.

Follow-up
Was the patient given specific advice on recognising subsequent deterioration? Yes / No Was the patient seen for routine follow up within 6 weeks?
Yes / No / DNA Were written asthma action plans issued or reinforced?

Yes / No
The educational intervention • Feedback of audit results: The results of the baseline audit were fed back to participating practices with anonymised comparative data from the other practices taking part in the programme, as well as the standards set by current asthma guidelines. Figure 2 provides an example of the feedback which was posted to practices prior to the workshop.
• Multi-disciplinary interactive workshop: A 3-hour workshop held in the University of Aberdeen, facilitated by two of the researchers (a general practitioner (HP) and respiratory nurse (GH)), was attended by representatives of the participating practice teams (normally a GP and practice nurse; the practice manager or senior receptionist was also invited). Amalgamated audit results from the group were discussed and used to tailor discussion towards specific aspects of acute asthma care identified as falling short of guideline standards. Case studies were used to facilitate discussion of practical aspects of acute asthma management highlighting key deficiencies reported in published literature [8]. Recording of objective assessment of attacks was emphasised.
• A list of suggested references (selected because of their relevance to primary care management of acute asthma), Example of 'post-intervention' feedback provided to practices Figure 2 Example of 'post-intervention' feedback provided to practices.

Assessment of severity in adults and older children:
Guideline recommendation The BTS Guideline advises that the severity of an acute asthma attack should be assessed by recording a peak flow and comparing the reading with the patient's best (or predicted).
Comparison with previous studies GPIAG National Asthma Attack Audit: Peak flow was recorded in 82% of episodes Canterbury & Thanet Acute Asthma Audit: Peak flow was recorded in 82% of episodes 'Post-intervention' feedback was provided at 6 months for the early intervention group and 12 months for the delayed intervention group. It includes the relevant recommendation from the BTS Guideline, [1] comparative data from published studies. [6] [8] The graphs illustrates the progress made between baseline and the first 'post-intervention' audits.

Baseline 6 months post-intervention
resources (e.g. guideline summary charts, web-sites of professional organisations, training organisations, equipment manufacturers) and practical materials (e.g. examples of asthma action plans) were provided to all participants and discussed in the workshops.
• Formulation of a development plan: Time was allocated at the end of the workshop for practice teams to formulate a practice-specific acute asthma development plan. Using an outline proforma, ( Figure 3) the practice teams reflected on their current performance, identified aspects of care they wished to improve and made practical plans to overcome barriers and institute change.

Follow-up audits
The audits completed at 6-months and 12-months were undertaken using the same methodology as at baseline and were fed back to all practices. The 6-month audit provided a review of progress for the early intervention group, and baseline assessment for the delayed intervention group. The time scale of 6 months had proved to be feasible in our pilot study [20].

Outcome measures
Our primary outcome measure was the proportion of acute episodes with a recording of a peak flow compared to the patients' best (or predicted if best was not known) at the 6-month audit. This measurement is recommended by current asthma guidelines as a basis for the classification of severity of acute attacks and for determining appropriate emergency treatment [1]. Our pilot study demonstrated an improvement in the proportion of patients with a peak flow compared to best/predicted from the baseline prevalence of 21% to 61% at 6 months [20].
Other outcome measures, reflecting the recommendations of the guideline in force at the beginning of the trial, [21] from the critical event analysis were considered in three domains: assessment (recording of peak flow, 'peak flow compared to best/predicted', respiratory rate, heart rate, ability to speak), management (provision of oxygen, bronchodilation. systemic steroids actually administered, steroids prescribed, inhaled steroids, referral to hospital) and follow-up (provision of advice, follow up consultation, self-management education). Combined scores for each domain were calculated by summing the questions which were answered 'yes' to give a score out of 5, 6 or 3 for assessment, management and follow-up respectively.

Sample size calculation and statistical analysis
In order to take account of the clustering by practice, the sample size calculation needed to include an estimate of the intracluster correlation coefficient (ICC). Our pilot work suggested that we could expect three acute episodes per full time general practitioner during each three month period and we estimate an average of 5 general practitioners per practice. Using a conservative estimate for the ICC of 0.05, to detect a change of 30% in the proportion with a peak flow compared to best/predicted from a baseline prevalence of 21% with 80% power and 5% two-sided significance level would require 5 practices per arm (ie. an estimated 25 general practitioners and 75 clinical episodes in each group). To allow for withdrawals we decided to recruit at least 20 practices in total.
We analysed associations between categorical data using the chi-square test. Differences in normally distributed combined scores between early and delayed intervention groups were examined with independent sample t-tests.
Because randomisation was at practice level, the statistical analysis had to adjust for the effect of clustering. Multilevel modelling was inappropriate since the outcomes were practice level data across time and not multiple measurements across time on the same individuals. Therefore, the primary analysis was based on the practice summary measures. Analysis of covariance was used to examine differences in outcomes between groups at 6months and 12-months after adjustment for baseline differences and practice effects. Practices who recorded no exacerbations during an audit cycle were excluded from the analysis at that time point.

Results
Of the 59 practices invited, 23 (39%) were recruited into the study: those who declined did so due to pressures of time and resources. Practice demography was similar for the two groups at baseline. (Table 1).

Withdrawals
The flow of practices through the trial is given in Figure 4. Five practices in the early intervention group withdrew before submission of the baseline audit data: the remaining six practices completed all aspects of the programme. All but one of the 12 practices in the delayed group submitted data at all three stages (baseline, 6-months and 12months), but only seven attended the workshop. Practice workload pressure (2), staffing problems (2) or a very low asthma attack rate (2) were the reasons given for withdrawal.

Critical event analysis
Data were returned from audits at baseline, 6-months  How may this be achieved?
What additional resources do you need?
What barriers will need to be overcome in order to allow change to happen?
identified a mean of 2.1 episodes in each of the 3-month audit periods compared to 2.0 per GP in the late intervention group There were no significant differences in the age and gender of the patients in the three audit phases, nor between groups (Table 2).

Early vs. delayed intervention groups
Primary outcome measure: proportion with peak flow compared to best/predicted The proportion of episodes in which the presenting peak flow had been compared to best (or predicted) was comparable in the early and delayed intervention groups at baseline (see Table 2). There was a trend to continuing improvement in the recording of peak flow compared to best/predicted in the early group practices throughout the year. Unadjusted data showed no difference at 6-months (p = 0.07) but a statistically significant improvement was observed at 12-months (p < 0.001). Although this trend remained, these differences did not reach significance at either time point after adjustment for baseline and practice effects.

Combined scores
The mean combined scores are given in Table 3. In the early intervention group, there was a consistent trend to gradual improvement in assessment and follow-up scores over the year of the study, with highly significant betweengroup differences in the unadjusted scores at 12-months. After adjustment for baseline difference and practice, the between-group difference in the combined assessment score remained significant at 12-months. There was no consistent change in the management score within or between groups.
The timescale of the changes in the combined assessment, management and follow-up scores in the individual practices are illustrated in Figure 5. The most consistent effect was the improvement in the combined assessment scores in early intervention practices.

Discussion
Our practice development programme incorporating audit, feedback and a workshop did not result in a significant improvement in our primary outcome measure (the proportion with peak flow compared to best/predicted) at either the 6 or 12 month time points after adjustment for baseline and practice effects. However, one of our secondary outcome measures (the recording of the objective assessment of attacks) showed a trend to improvement which was apparent at 6-months, and reached statistical significance at 12-months, suggesting that the hypothesised 6-month period was too short to enable practices fully to implement change. The marked trend to improvement in follow up arrangements lost its statistical significance after adjustment for practice effects and baseline differences. The latter, however, may be an over-adjustment as collecting and responding to the baseline audit data was a part of the educational intervention. For example: poor baseline performance should stimulate a practice to develop that aspect of their care. The true effect of this complex intervention is, therefore, likely to fall somewhere between our adjusted and unadjusted results.

Limitations of the study
The withdrawal of five practices from the early intervention group before submitting their baseline data was unfortunate. The remaining practices are likely to have been the most motivated making it easier to effect change in this group. By contrast, all but one of the delayed intervention group practices completed the programme, though five did not attend the educational intervention. If less motivated practices in the 'delayed' group had chosen Flow of participating practices through each stage of the trial Figure 4 Flow of participating practices through each stage of the trial. to 'stay the course' rather than withdraw this might have diluted any effect in this group. However, because of the considerable commitment required we believe it is unlikely that reluctant practices would have continued to provide data.

Withdrew (n=5)
We rejected the option of collecting baseline data before randomisation because the audit was an integral part of the educational intervention. Willingness to participate in all aspects of the programme (including the audit, workshop and development plan) was one of the factors potentially influencing the effectiveness of this 'complex intervention' [22] Randomising only those practices who submitted baseline data would have eliminated one of these important factors.
Practices submitted less than the projected three episodes/ GP in each three month audit period. This, combined with the withdrawal of five practices, meant that we did not achieve the required number of 75 acute episodes at baseline and 6-months in the early intervention group so we were slightly underpowered to detect a statisticallysignificant change in the primary outcome measure.
Although raising the possibility that some acute episodes had been overlooked, the rate of identification of acute attacks per GP was similar in both groups, and was consistent over the three audit time points. In addition, the demography of the patients suffering acute episodes was similar to that described in previous regional and national audits (allowing for the slightly different recruitment strategies). [6][7][8] The reduction in number of acute episodes may reflect the reported decline in asthma exacerbations in the UK [23]. The audits were carried out internally  Timescale of changes in combined scores within the early and delayed intervention groups Figure 5 Timescale of changes in combined scores within the early and delayed intervention groups. Combined assessment scores Heavy line is the mean score for the group, Light lines are the individual practices, Dotted lines are practices who contributed less than 5 acute episodes at 2 or more of the audit points by practice staff, which introduced the possibility of inaccurate, incomplete or biased data. Despite this concern, we considered it important that practices undertook their own data collection as part of the learning process. Financial constraints prevented quality checks of the accuracy of submitted data.
Of the 59 practices approached, just over a third agreed to participate, and just over a quarter completed the programme which may be interpreted as limiting generalisability. However, the recruited practices reflected the range of demography typical of the area. In addition, participation in a professional development plan is an active process which demands considerable application with significant workload implications for members of the team. The reasons given for non-participation and withdrawal suggest that the time and effort required was not possible for many practices during the study period. Practice development needs vary over time and it is likely that under other circumstances other practices would have wished to participate.

Main strengths of study
Our trial included a broad range of urban and rural practices, though to facilitate local workshops we limited recruitment to one Scottish region. The demography of the practices in the two study groups was similar. The audit tools were developed from a published survey, [8] and the proforma had been used successfully in an earlier pilot project [20]. The data submitted from the baseline audit was broadly similar to that observed in other published surveys [6][7][8]20] increasing confidence in the reliability of our outcomes.

Interpretation of findings in relation to previously published work
In common with the majority of studies of educational interventions, [24,25] our trial evaluates 'behaviour' -the third tier of Kirkpatrick's hierarchy of levels of evaluation [26]. Although this is one step removed from the ultimate goal (improved patient outcomes) our selected process outcomes are recommended by evidence based guidelines [21]. Importantly, poor performance in objectively assessing severity has been consistently linked with poor patient outcomes in a number of confidential enquiries. [13][14][15][16] In addition, keeping adequate records which allow retrospective assessment of severity is good clinical practice and a medico-legal requirement. [27,28] The development plans prepared by individual practices confirmed that objective assessment was a priority for five of the early intervention group practices. Referral data, a potential outcome measure which we included within the management domain, is difficult to interpret as rates could be influenced by management strategies or different severity of presenting attacks.
Using the structure of Professional Development Plans [19] our intervention not only involved the recognised formula of 'audit and feedback', [18] but also sought formally to engage the practice team in the process of identifying the challenges and obstacles relevant to their practice, and to plan how change may be effected. This is a recognised educational strategy, [25,29] but 'individualising' of the practice development plan, but may have diluted the effect on specific outcomes as some practices may have decided that certain aspects were not relevant to them. Whilst incorporating many of the key elements of the stepwise cycle of change described by Grol, [30] we may have underestimated the importance of the 6-month audit. Audit is associated with improved asthma care, [31] and any improvements at 6-months (albeit not significant) may have encouraged the 'early' practices to further develop their care in time for the 12-month audit. By contrast the delayed intervention group received no feedback until the 6-months and had little time to effect change.
There is no consensus on the ideal duration of educational intervention studies, [18] though six months is the median duration of trials reviewed by Wensing, [32] Ideal duration is a balance between the time needed to effect change, whilst avoiding the danger of deterioration if the time scale were too prolonged [18]. Our trial was designed to demonstrate a difference between the two groups at 6months, when the early intervention group had received the education component of the programme with the 'delayed' practices acting as controls. However, after adjustment for baseline differences and practice, statistically significant change only occurred at 12-months. Future randomised trials of educational interventions need to recognise that effecting change in practices takes time. Although the timeframe for change was implicit within the structure of the trial, the addition of an explicit timeline within the Practice Development Plan might have facilitated more rapid change.
In order to achieve improvement, the practices had to commit to the project for a year and participate in three audit cycles and a workshop. We offered no financial incentives to cover the cost of the work involved suggesting that the process of formulating a practice-specific acute asthma development plan in response to their baseline performance had sufficiently engaged practices to encourage an on-going focus on aspects of care they wished to improve.
Other factors which may have influenced outcomes include the publication of national guidelines in February 2002, [1] though this is unlikely to have significantly affected our results as the final audit was already underway at the time of the launch. The primary care management of acute asthma was not one of the key messages promoted by the publicity following the launch [4]. The lack of substantial change in the delayed intervention group confirms that this was unlikely to be a significant factor.
Interpretation of the management scores is difficult without objective independent assessment of asthma severity. It is possible that the more aggressive management of attacks at baseline reflected more severe attacks at this audit time point rather than inappropriate management.

Conclusion
Although our randomised trial of an acute asthma professional development programme for general practices did not demonstrate significant improvements at the a priori 6-month assessment point, there was an improvement compared to baseline in the objective assessment of severity 12 months into the trial. Monitoring of the assessment of acute attacks proved to be a feasible and responsive indicator of quality care. Our findings, especially the timescale needed to effect change, have implications for the design of future trials.
Publish with Bio Med Central and every scientist can read your work free of charge