Study setting and design
The University of Toronto Practice-based Research Network (UTOPIAN) consists of 14 family medicine clinics in south central Ontario affiliated with the Department of Family and Community Medicine of the University of Toronto. Two clinics had participated in a pilot study of the intervention and were excluded . Of the remaining 12 clinics invited to participate, family physicians and nurse practitioners from six clinics agreed to participate in a pragmatic controlled trial of a primary care provider-focused stewardship intervention. The intervention was delivered between September 2018 and December 2018, followed by a 5 month evaluation period from January 1st 2019 until May 31st 2019. Baseline prescribing rates for each clinic were calculated from prescribing data for the previous winter. Ethics approval was obtained from research ethics boards of the University of Toronto, Mount Sinai Hospital, Women’s College Hospital and North York General Hospital. The study was registered with clinicaltrials.gov on 08/05/2018 (registration number NCT03517215).
Randomization by minimization was conducted due to the small number of clinics . Using prescribing data collected for the winter before the intervention, prescribing rates, number of providers and the presence of trainees were selected as minimizing factors. Clinics were assigned a number (by SV) and randomly assigned (by WM) without awareness of clinic identities. However, delays in securing participation from all clinics led to five being initially randomized, and a sixth clinic was later allocated to balance provider numbers. As a result, imbalances in factors associated with antibiotic use at baseline persisted. Statistical methods were therefore utilized to adjust for these factors in estimating the effect of the intervention.
The intervention was a multi-faceted program of clinician education, clinical decision aids for prescribing decisions, patient information leaflets, audit and feedback of clinic prescription practices, local clinic support, and incentives. An initial one hour on-site education session was delivered by study staff at each clinic regarding antimicrobial resistance, stewardship, and interventions for reducing antibiotic prescribing. The clinic’s prescribing practices the previous winter were reviewed, and providers set prescribing goals. Providers were then sent electronic modules to complete over four months explaining the use of the decision aids for a given condition and optimal prescription practices. Additional on-site sessions were held during the winter to review antibiotic utilization and revise prescribing goals.
The modules addressed five infections: acute sinusitis, acute uncomplicated upper respiratory infections (URI), sore throat presentations (pharyngitis, tonsillitis), acute bronchitis and acute uncomplicated cystitis. These conditions account for approximately 50% of community antibiotic prescriptions in Canada . Modules took approximately 15 minutes to complete and were sent each month by email to intervention clinics.
Module topics included prescribing issues for each infection and a 1-page clinical prescribing decision aid. The aid addressed criteria for diagnosis, indications for antibiotics, recommended first line antibiotic choices, treatment durations, and ‘red flags’ for serious presentations. Validated clinical decision rules were incorporated into prescribing aides where available [19, 20]. Modules also included ‘communication’ scripts to engage patients in prescribing decisions , patient handouts, delayed prescription options,  and advice to give patients about when to seek medical care where antibiotics were not prescribed (‘safety-netting’) .
Clinicians at intervention sites received $200 compensation, pro-rated for the number of completed modules and education sessions. Continuing medical education credits and a free antibiotic prescribing formulary  were also provided. Control arm participants provided usual care but received the free antibiotic formulary upon trial completion.
Data from eligible visit encounters were abstracted from electronic medical record (EMR) systems at each clinic. Eligible visits were defined as those involving adults 18 years of age or older, seen by a consenting physician between January 1st 2018 and February 28th 2018 (baseline period) or between January 1st 2019 and May 31st 2019 (evaluation period), and with an eligible ICD-9 diagnosis code (International Classification of Diseases, Ninth Revision (ICD 9) identified from billing records. Urinary infections involving males or pregnant females were excluded, as were follow-up visits of previously treated infections.
To identify most eligible visits, a number of ICD-9 billing codes were selected. These included 460 (URI), 462 (pharyngitis), 463 (tonsillitis), 461(acute sinusitis), 466 (acute bronchitis), and (595) acute uncomplicated cystitis. In addition, visits coded 464 (laryngitis), 599 (other urinary-eg. hematuria, incontinence), 486 (pneumonia), and 487(influenza) were included to ensure other respiratory and urinary symptom visits had not been mislabeled. This was assessed by comparing a provider’s written diagnosis and the billing code. Two independent raters reviewed all visits using standardized coding rules developed for the study (Available upon request). The final visit diagnosis was adjusted if the written diagnosis indicated an eligible infection presentation. Non-infection diagnoses (e.g.’new patient visit’) and non-eligible infections (eg. pyleonephritis, chronic sinusitis) were excluded, as were presentations involving asthma or chronic lung disease. Raters agreed on the final diagnosis for 94% of visits. Disagreements were resolved through case review and agreement by both raters.
Data abstracted from each visit included patient age, sex, antibiotic allergies, clinic site, visit date, clinician type, billing ICD-9 code, clinician written diagnosis, selected vital signs, tests ordered, and antibiotic prescriptions. Prescription information included the antibiotic name, prescription duration, and if a patient was advised to delay filling the prescription (‘delayed prescription’).
The primary study outcome was total antibiotic prescriptions for the five selected conditions combined (URI, sore throat presentations, acute sinusitis, acute bronchitis, acute uncomplicated cystitis) in each arm. Secondary outcomes were the proportion of prescriptions issued as delayed antibiotic prescriptions, prescriptions for longer than 7 days duration, and total, delayed and long duration prescriptions for each infection individually. A post-hoc decision was made to also assess test utilization.
A 25% relative reduction in the total prescriptions was selected as the minimum important effect size, consistent with previously reported national goals . To detect a 25% relative difference with 90% power, assuming a similar 30% antibiotic prescribing rate as in the pilot study , a sample size of 834 cases in each study arm was estimated, unadjusted for clustering. Visit and clinic characteristics of each group were compared using unadjusted chi-square, Fisher’s exact test or t-tests as appropriate. The intervention effects were expressed as odds ratios, estimated from unadjusted and adjusted logistic generalized estimation equation (GEE) models that accounted for clustering by clinics in each study arm. Models were adjusted for differing baseline characteristics between control and intervention groups associated with antibiotic prescriptions, as well as baseline prescribing rates. All analyses were performed using R statistical software .