Skip to main content
  • Research article
  • Open access
  • Published:

Effects of multicomponent primary care-based intervention on immunization rates and missed opportunities to vaccinate adults

Abstract

Background

Adult immunization rates are below Healthy People 2020 targets. Our objective was to evaluate the effectiveness of a multicomponent intervention to improve adult immunization rates.

Methods

This prospective interventional before-and-after non-randomized study was conducted through the American Academy of Family Physicians National Research Network with 43 primary care physicians from a large multi-specialty healthcare organization (multicomponent intervention group n = 23; comparator group n = 20) in the United States. The multicomponent intervention included provider reminders, quarterly provider-level performance reports, provider education, patient visual aid materials, and standing orders on adult pneumococcal, influenza, and zoster immunizations. We assessed individual and comparative provider-level vaccination rates and missed opportunities detailing concordance with targets established by Healthy People 2020 for pneumococcal, influenza, and zoster immunizations.

Results

Vaccination rates increased after 12 months in intervention and comparator groups respectively for: a). influenza from 44.4 ± 16.7 to 51.3% ± 12.9% (by 6.9 percentage points, p = 0.001) and from 35.1 ± 19.1 to 41.3% ± 14.2%, (by 6.2 percentage points, p = 0.01); b). pneumococcal vaccinations in older adults from 62.8 ± 17.6 to 81.4% ± 16.6% (by 18.6 percentage points, for p < 0.0001) and from 55.9 ± 20.0 to 72.7% ± 18.4% (by 16.7 percentage points, p < 0.0001); and c). zoster from 37.1 ± 13.4 to 41.9% ± 13.1% (by 4.8 percentage points, p < 0.0001) and from 35.0 ± 18.7 to 42.3% ± 20.9% (7.3 percentage points, p = 0.001). Pneumococcal vaccinations in adults at risk did not change from baseline in intervention group (35.7 ± 19.6 to 34.5% ± 19.0%, p = 0.3) and improved slightly in comparator group (24.3 ± 20.1 to 28.2% ± 20.0%, p = 0.003). Missed opportunities reduced after 12 months, most noticeably, for: a). for influenza from 57.7 to 48.6% (by 9.1 percentage points, p < 0.0001) and from 69.7 to 59.6% (by 10.1 percentage points, p < 0.0001); b). pneumococcal vaccinations in older adults from 18.1 to 11.5% (by 6.6 percentage points p < 0.0001) and from 24.6 to 20.4% (by 4.3 percentage points, p < 0.0001) in intervention and comparator groups respectively.

Conclusions

Multicomponent interventions show promise in improving vaccination rates and reducing missed opportunities in older adults for pneumococcal and zoster vaccines and vaccination against influenza. Provider reminders remain the most effective strategy when delivered either as a component of these interventions or alone.

Peer Review reports

Background

Adult immunization rates remain well below Healthy People 2020 targets, [1] even though adults, on average, visit a primary care provider more than three times a year [2]. Primary care physicians provide more than 560 million office visits each year, putting them in the unique position to administer immunizations to patients of all ages [3]. Specialty care provides more than 230 million office visits, but, based on low immunization rates in specialty care, a large proportion of these visits are viewed as missed opportunities to provide adequate vaccination for adults [4].

We recently explored the missed opportunities in adult vaccinations in a meta-narrative literature review and concluded that missed opportunities are present in various health care settings [5]. Moreover, we reported two methodologies for quantifying missed opportunities: 1. Based on the number of healthcare encounters without appropriate vaccination services, defined as a number or percentage of visits per patient with no vaccination related services (missed opportunities per patient or patient-level); and 2. Based on vaccination status as “non-vaccinated” among a group of patients who had a health care encounter where the vaccination should/could have happened, presented as percentage of non-vaccinated patients who had an encounter with health care providers (missed opportunities per patient population or population-level) [5]. We suggested that missed opportunities could represent a more accurate measure of provider and health care organization performance as compared to population-wide immunization rates because missed opportunities are clearly associated with a patient encounter.

The low vaccination rates among adults, and especially among adults age 19–64 years, [6] are not surprising as the vaccination recommendations are complicated by revaccination recommendations and strategies, the varying target immunization ages and goals, the current immunization rates that differ for high-risk adults and elderly, and the annual changes to the vaccination schedules. Even though the recommendations are reviewed, endorsed, and promoted by several major medical professional organizations and CDC, [7] providers and patients encounter many barriers, and the efforts still result in low vaccination rates among adults [8, 9].

A comprehensive report by Task Force on Community Preventative Services (Task Force) recommends several interventions with strong evidence for their effectiveness in improving immunization rates. Of all interventions used alone, provider reminders have sufficient evidence of effectiveness to be currently recommended. The Task Force recommends that interventions implemented in combination (multicomponent interventions) should be considered for improving vaccination coverage among high-risk populations; specifically, the multicomponent interventions that combine one or more interventions from three categories: 1. enhancing access to care and reducing administrative barriers; 2. implementing provider and system-based interventions such as provider reminders; and 3. increasing vaccination demand among patients have been shown to increase immunizations rates [10].

The purpose of this paper is to present the results of a project that tested a multicomponent approach to increase immunization rates and reduce missed opportunities in adults. Based on our earlier work, we also aimed to test the missed opportunities as an outcome of the multicomponent intervention in addition to standard metrics for vaccination rates. The project aimed to test the generalizability of the Task Force recommendations to primary healthcare settings and different patient populations.

Methods

Study design

This was a prospective interventional before-and-after non-randomized study with an intervention arm and a comparator arm to evaluate the effectiveness of a multicomponent intervention to improve immunizations in a multispecialty medical group. The primary care providers willing to receive a multicomponent intervention were allocated to the intervention group, the rest of the primary care providers elected to participate in the comparator group. This was a pragmatic trial with no direct patient recruitment or enrollment. This study was conducted as quality improvement project between July, 2015, and August, 2016, where the intervention period encompassed at least one full influenza season. The intervention targeted the organization for system-level changes and individual providers for improving physician behaviors and patient care. The comparator group received a single provider-level intervention in the form of point-of-care reminders, and the intervention group received multicomponent intervention as described below. The American Academy of Family Physicians Institutional Review Board (AAFP IRB) approved the study.

Setting and participants

The study was conducted in one large health care organization in North Carolina, USA, and focused on general internal medicine and family medicine settings. The organization is a multi-specialty clinic (including pulmonology, infectious disease, family medicine, and internal medicine) with primary care providers integrated into the system. From ten sites 43 providers within the same organization were included in the study; 23 primary care providers participated in the intervention arm of the study while 20 providers were in the comparator group. The professional composition of the intervention and comparator groups respectively included: physician assistants (1 in each group); nurse practitioners (2 and 1); family practice physicians (4 and 12); and general internal medicine physicians (16 and 6).

Patient population overview

This study included all patients age 18 years and older receiving services at the participating providers throughout the year and during flu season from 2013 through 2015 who were eligible for pneumococcal and influenza vaccination. The patient cohorts were defined as number of unique eligible patients among all adults in each of the subgroups:

  • Pneumococcal:

    • all age 65 and older

    • age 19–64 with at least one risk factor (see Appendix 1 for a list of included conditions)

  • Influenza: All age 18 and older

  • Zoster: all age 60 and older

Eligible patients were included if they had at least one visit to any primary care provider from the list of participating providers within a timeframe of data analysis (year, season, or a specific time period) and further assigned to a provider based on the majority rule (i.e., patients assigned to the provider they saw the most for a given time period).

Key elements of intervention

Clinical decision support (CDS) provider reminders

The project team delivered this component to the providers in both intervention and comparator groups. The project team developed or updated algorithms for CDS Order Sets for provider reminders and standing orders for adult vaccinations according to the current detailed guidelines for vaccination and revaccination of adult patients. The final algorithms that supported the CDS systems developed by the study team with the input from the lead clinician are available from the authors.

Each provider in both the intervention and comparator groups was able to view a patient’s recommendation report at the time of the visit. The report included action items for the provider and staff based on current vaccination eligibility, recommendations, patient immunization history, and status at the time of the visit. The system was distributed for all primary care providers to use for the duration of the study.

The following components were delivered to providers in the intervention group only.

Standing orders

At the beginning of the study, the participant organization did not systematically use standing orders. For adult vaccines, three providers in the intervention group reported always having standing orders, nine providers reported sometimes having standing orders; while six providers reported occasionally or never having standing orders. Educational materials on standing orders were developed for the influenza and pneumococcal vaccinations to encourage the providers, practice staff, and leaders to consider adopting standing orders. At the beginning of the project, the organization received these practice education materials and implementation strategies (available from the authors).

Provider audit and feedback

The project team developed quarterly provider-level and overall reports using practice’s electronic health records data. Providers and staff met quarterly to review and discuss provider-level and overall reports. The providers were identified by the name in the reports and whenever necessary, peer pressure or alternative performance improvement strategies (ranking, positive reinforcement, competition, incentives etc.) were applied. Clear performance goals for reducing missed opportunities for vaccination at every visit were emphasized.

Improving documentation

This intervention component consisted of technical training for providers that focused on improving documentation in two areas: 1. Past immunization history – asking patients about their recent vaccinations and documenting in the EHR; and 2. Documenting current immunization status with focus on vaccinations offered at the visit (record of vaccination offered and given or declined by patients).

Provider education and communications

Providers received materials for patient and staff immunization education, talking points for communication with patients about immunizations, technical guidelines on vaccine storage and billing/coding recommendations, recommendations on how to address vaccine shortages, and guidance on expanding access to vaccination services. A monthly newsletter about the project was produced and shared with the participants.

Enhancing patient outreach

The project incorporated the following strategies in patient education and outreach to increase demand and reduce refusals: educating physicians and their practice staff about effective patient engagement; increasing patient awareness and acceptance of recommended vaccines; and patient education during the visits via visual aids. A set of patient-facing educational materials developed by the CDC and other sources was provided to the site.

Data collection and evaluation

Data sources

The data for the study were obtained from the EHR. The overall total number of visits was collected for each provider. For patients who visited more than one provider in a given time period, these patients were assigned to the provider they saw the most (which we called the majority-rule approach). For patients, we extracted demographics, current diagnoses, smoking status, and vaccines given, discussed, or refused with reasons for refusal whenever possible. Records on vaccines given elsewhere were obtained from historic records as documented in EHR. The data were collected every three months and summarized for immunization rates for provider feedback, and then used for longitudinal data analysis. Vaccination rates are presented here as percentage of vaccinated individuals over all eligible persons for a given vaccine within the timeframe reported. We established baseline immunization rates via EHR data pulls by reviewing immunization performance data for patients seen in the year or influenza season prior to the beginning of the intervention. The definitions for missed opportunities were based on operational methodologies developed by the project team and included population-based, patient/visit-based, and reminder-based approaches [5].

Briefly, in the population-based missed opportunity metric, we defined missed opportunity as the percentage of eligible unvaccinated patients with at least one scheduled visit where the vaccination should/could have happened over the observation period (one year or one season as applicable).

In the visit-based metric or missed opportunities per patient (patient-level) we defined missed opportunity based on the number of healthcare encounters without appropriate vaccination services, defined as a number of visits per patient with no vaccination related services. We counted the number of visits for all eligible patients (i.e., those matching cohort criteria, matching to one of the participating providers by majority-rule) as well as the number of visits where a provider is compliant (vaccine given or vaccine offered but refused). Further, we presented these results clustered by the patient’s primary provider using the majority-rule approach. A visit was considered a missed opportunity if a vaccination record or vaccine order that was refused by the patient did not appear either on or within 7 days after the date of visit to account for possible delays with data entry.

Statistical analyses

Discrete variables are described using frequency and percentage. Means and standard deviations (SD) are used to describe continuous variables. T-tests were used for comparing groups in case of continuous variables – for before and after the intervention period the paired samples t-test were used, and independent sample t-tests were used to assess unadjusted differences between the intervention and comparator groups. Hierarchical linear regressions were also used to examine the relationship between vaccination rates (dependent variable), and predictor variables: group assignment (intervention or comparator) and baseline vaccination rates. Pearson correlations between the vaccination rates and the missed opportunity including population-based and visit-based metric were also calculated. Statistically significant associations and differences were identified by p-values of less than 0.05. All analyses were conducted using SAS version 9.4 (copyright 2002–2012 by SAS Institute Inc., Cary, NC, USA).

Role of the funding source

This work was supported in part by a research grant from Investigator-Initiated Studies Program of Merck Sharp & Dohme Corp. Merck Sharp & Dohme Corp. was not involved in the study design, data collection, data analysis and interpretation, writing or reporting of this work, and did not have any involvement in the decision to submit this article for publication. The opinions expressed in this paper are those of the authors and do not necessarily represent those of Merck Sharp & Dohme Corp.

Results

Participant characteristics

Forty-four providers were initially included in the study. One provider left the practice shortly after enrollment. There were 23 clinicians in the intervention group (four family medicine physicians, sixteen internal medicine physicians, two nurse practitioners, and one physician assistant). The comparator group (n = 20) included twelve family medicine physicians, six internal medicine physicians, one nurse practitioner, and one physician assistant. Patient characteristics for all unique patients are presented in Table 1.

Table 1 Overview of included patient cohorts seen by participating study providers

Effects of intervention on vaccination rates

Baseline and end-of-study vaccination rates are presented in Table 2. Vaccination rates increased after 12 months in intervention and comparator groups respectively for: a). influenza from 44.4 ± 16.7 to 51.3% ± 12.9% (by 6.9 percentage points, p = 0.001) and from 35.1 ± 19.1 to 41.3% ± 14.2%, (by 6.2 percentage points, p = 0.01); b). pneumococcal vaccinations in older adults from 62.8 ± 17.6 to 81.4% ± 16.6% (by 18.6 percentage points, p < 0.0001) and from 55.9 ± 20.0 to 72.7% ± 18.4% (by 16.7 percentage points, p < 0.0001); and c). zoster from 37.1 ± 13.4 to 41.9% ± 13.1% (by 4.8 percentage points, p < 0.0001) and from 35.0 ± 18.7 to 42.3% ± 20.9% (by 7.3 percentage points, p = 0.001). Pneumococcal vaccinations in adults at risk did not change from baseline in intervention group (35.7 ± 19.6 to 34.5% ± 19.0%, p = 0.3) and improved slightly in comparator group (24.3 ± 20.1 to 28.2% ± 20.0%, 3.8 percentage point increase, p = 0.003). This difference for adults at risk was the only significant difference in percentage point changes between the intervention and comparator groups in (p = 0.001).

Table 2 Baseline and end of project vaccination rates

Regression models were used to examine the relationship between vaccination rates (dependent variable), and predictor variables: group assignment (intervention or comparator) and baseline vaccination rates (Table 3). Baseline vaccination rates had significant (p < 0.0001) zero-order associations with end of project vaccination rates for all cohorts. The intervention group assignment had statistically significant associations with improved vaccination rates over time for pneumococcal vaccinations in adults at risk (Group: p = 0.006; pneumococcal baseline vaccine rate: < 0.0001). The was no effect of intervention on rates for influenza vaccinations (Group: p = 0.080; influenza baseline vaccine rate: p = < 0.0001), pneumococcal vaccine in older adults (Group: p = 0.212; pneumococcal baseline vaccine rate: < 0.0001), and zoster vaccinations (Group: p = 0.174; zoster baseline vaccine rate: < 0.0001).

Table 3 Hierarchical Linear Regression Results

We did not conduct subgroup analyses for vaccination rates and changes over time among individual risk factor sub-groups of patients age 19–64 years with various risks for pneumococcal disease. This was due to a small sample size for several conditions and absence of main effects in overall change in vaccination rates in this group of patients.

Missed opportunities analyses

The Tables 4 and 5 show the rates of missed opportunities across the four study vaccination cohorts using two missed opportunity metrics for influenza, pneumococcal (age), pneumococcal (risk), and zoster.

Table 4 Missed Opportunities Before and After Intervention: Population-level Metric
Table 5 Missed Opportunities Before and After Intervention: Visit-Based Metric

Population-based missed opportunities

Missed opportunities reduced after 12 months for: a). influenza from 57.7 to 48.6% (by 9.1 percentage points, p < 0.0001) and from 69.7 to 59.6% (by 10.1 percentage points, p < 0.0001); b). pneumococcal vaccinations in older adults from 18.1 to 11.5% (by 6.6 percentage points p < 0.0001) and from 24.6 to 20.4% (by 4.3 percentage points, p < 0.0001) in intervention and comparator groups respectively. Missed opportunities for zoster vaccinations decreased in the intervention group from 55.3 to 49.7% (by 5.6 percentage points p < 0.0001) and did not change in the comparator group (53.6% vs. 52.7%, 0.9 percentage points difference, p = 0.3).

Missed opportunities in pneumococcal vaccinations in adults at risk did not change in either group (Table 4).

Patient- (visit-) based missed opportunities

The total numbers of visits provided by all providers in the study defined as vaccination eligible visits and visit-based missed opportunities are presented in Table 5. Visit-based missed opportunities were reduced significantly only for zoster vaccinations in the intervention group (55.3% vs. 49.7%, p < 0.0001). No other reductions in missed opportunity visits were observed in either group.

Correlations between vaccination rates and missed opportunities

As shown in Table 6, the missed opportunities metrics generated by two primary methods (population and patient/visit) correlated significantly with vaccination rates at both baseline and end of study. Lower vaccination rates were strongly associated with more missed opportunities that was consistently observed for both missed opportunity metrics.

Table 6 Correlations Between Vaccination Rates and Missed Opportunities

Discussion

Despite long-standing recommendations to vaccinate adults against vaccine-preventable diseases such as influenza, pneumococcal disease, and shingles, the rates of vaccination coverage among adults remain low. The vaccination rates in this study were comparable to the national benchmarks [6] at the beginning of the study, and afterward exceeded the national rates in both groups, on all vaccines. However, not every provider was able to reach vaccination targets set by the Healthy People 2020, which indicates that the likelihood of receiving recommended vaccination may be more provider-dependent than previously thought, and having all providers meet the Healthy People 2020 targets may require additional approaches and perhaps more time to see noticeable change [11].

This study demonstrated improvements in vaccination rates in both intervention group by 18% and the comparator group by 16% for pneumococcal vaccines in older adults and smaller improvements in influenza and zoster vaccination. The results for the comparator group that received only CDS provider reminders is comparable to rates reported by other studies for the same intervention when used alone [10]. The intervention groups received a combination of interventions, and overall demonstrated improvements moderately but not significantly beyond those seen in the comparator group. In the pneumococcal vaccinations for adults19–64 years old at risk, however, the multicomponent intervention showed no increase in vaccination rates or reductions in missed opportunities. Future studies need to consider the balance of impact vs. costs of multicomponent interventions compared to CDS when used alone.

The pneumococcal vaccine rates among the adults aged 19–64 years with risk factors did not change considerably and remained substantially below Healthy People 2020 targets in both groups. Why the rates did not change in response to intervention remains unclear. We expect that despite the National Vaccine Advisory Committee recommendations, [12] providers may be hesitant to make strong recommendations for needed vaccines for adults aged 19 to 64 years. Future studies need to focus on exploring barriers to vaccinate in this particularly challenging group and consider interventions to maximize the value of individual intervention components that are effective in this patient population.

Our study demonstrated improvements in both provider groups that received the CDS point-of-care reminders. Our study corroborates previous reports that provider reminders remain the most effective way to increase vaccination rates at the provider level [13]. Studies exploring impact of CDS in general practice, however, indicate substantial levels of provider “alert fatigue,” and future studies need to explore ways to keep providers engaged in continuous vaccination improvement efforts [14, 15]. Additionally, in order for the CDS systems to be most effective, they have to rely on accurate evidence-based clinical algorithms that need to be regularly updated and aligned among multiple systems, settings, and patient populations. Future efforts need to be taken in designing CDS systems or a combination of interventions that improve attention to the at-risk adult group, as the existing CDS or a combination of interventions seem to have low effectiveness in this group [16].

Significant missed opportunities have been identified in both groups using at least two metrics – population-based and patient/visit-based, suggesting need for further understanding and research on how to reduce missed opportunities. Similar to the vaccination rates, however, no changes in missed opportunities for pneumococcal vaccination were observed in the adults aged 19–64 years at risk. The reasons for low intervention effects on any outcomes related to this group need further explorations, as it seems that the CDS has modest effect on the rates of vaccination in this group among providers with low vaccination rates, and the additional components of the intervention did not improve the rate any further. The intervention has been effective in reducing population-based missed opportunities; however, the overwhelming majority of patients who were still not vaccinated by the end of study had at least one encounter with the providers over the study period. While the reasons for existing missed opportunities need to be explored in future studies, the high level of correlations among vaccination rates and missed opportunities suggest that low vaccination rates among those who visited the clinic/provider in the reporting period can, once again, be explained primarily by provider-related factors that result in missed opportunities.

The missed opportunities methodology needs further research, in particular the visit-based metric. As the numbers of visits to primary care providers increase every year, the methodology needs to account for total number of visits in the study period as well as the number of eligible individuals. While we believe the percentage approach to this metric is reliable, it does not demonstrate the actionable reduction in the actual number of missed opportunities visits due to intervention.

The study had several limitations, including non-randomized study design with existing baseline vaccination level differences between the groups. Although we accounted for these differences statistically through regression analyses, it is possible that the providers who volunteered to test the multicomponent intervention placed a higher priority on vaccinations and were more successful in vaccinating their adult patients than those who did not volunteer. We have not explored the effects of professional group compositions or the effects of office affiliation/location/culture, which may have contributed to the baseline and overall vaccination rate and missed opportunity differences. The 16.7% older adult pneumococcal vaccination rate increase in the comparator group, however, demonstrates that CDS as a single intervention may be as effective in the group of providers who did not volunteer to receive other invention components. We used a “majority rule approach” that retrospectively assigned patients to a primary care provider based on the highest proportion of patient visits during each observation period. While this method has general limitations [17], the majority of patents in our study saw the same provider at least 75% of the time, and the rest fluctuated at random. Due to the relatively short duration, the study was limited in its ability to capture uptake and impact of standing orders because upon the study enrollment only three providers in the intervention group reported always having standing orders for adult vaccines, and shortly after the standing orders educational materials were delivered to the practice, the organization decided to roll out standing orders as an organization-wide quality improvement initiative. Large organization-wide quality improvement initiatives typically take long time to implement, and the standing orders were not fully adopted by all primary care providers by the conclusion of the study. This observation also demonstrates the unique challenges of implementing efficacious interventions from clinical trials in real world clinical settings where factors beyond researchers’ control may influence the fidelity of research. Additionally, we did not determine to what extent the participating providers shared practice staff who may have a role in vaccinations or to what extent that could have affected the study outcomes. Although most components of the multicomponent intervention, including CDS, targeted providers only, it is possible that some educational materials or patient-facing visual aids were accessible for practice staff.

Conclusions

Primary care providers play a key role in delivering adult vaccinations. Despite multiple efforts to increase vaccination rates in adults, the vaccination coverage remains suboptimal, and significant missed opportunities still exist. Strong correlations between vaccination rates and missed opportunities to vaccinate patients at the time of the primary care visit suggest provider-related factors may be responsible.

Provider reminders remain the most effective intervention for vaccination rate improvements, but even they do not substantially reduce missed opportunities or facilitate substantial progress toward Healthy People 2020 adult immunization targets. While promising, various types and intensities of multicomponent intervention need to be further studied to maximize their impact on improving vaccination rates and reducing missed opportunities for vaccinating adults.

Availability of data and materials

The datasets generated or analyzed during the current study are not publicly available due to existing data sharing and use agreements and organizational policies in place but are available from the corresponding author on reasonable request.

Abbreviations

SD:

Standard deviation

References

  1. Ammerman AS, Haines PS, DeVellis RF, Strogatz DS, Keyserling TC, Simpson RJJ, Siscovick DS. A brief dietary assessment to guide cholesterol reduction in low-income individuals: design and validation. J Am Diet Assoc. 1991;91(11):1385–90.

    CAS  PubMed  Google Scholar 

  2. Nowalk MP, Zimmerman RK, Cleary SM, Bruehlman RD. Missed opportunities to vaccinate older adults in primary care. J Am Board Fam Pract. 2005;18(1):20–7.

    Article  Google Scholar 

  3. Campos-Outcalt D, Jeffcott-Pera M, Carter-Smith P, Schoof BK, Young HF. Vaccines provided by family physicians. Ann Fam Med. 2010;8(6):507–10.

    Article  Google Scholar 

  4. Daniels NA, Nguyen TT, Gildengorin G, Perez-Stable EJ. Adult immunization in university-based primary care and specialty practices. J Am Geriatr Soc. 2004;52(6):1007–12.

    Article  Google Scholar 

  5. Loskutova N, Smail C, Webster B, Ajayi K, Wood J, Carroll J. Missed opportunities for improving practice performance in adult immunizations: a meta-narrative review of the literature. BMC Fam Pract. 2017;18(1):108.

    Article  Google Scholar 

  6. Williams WW, Lu PJ, O'Halloran A, Kim DK, Grohskopf LA, Pilishvili T, Skoff TH, Nelson NP, Harpaz R, Markowitz LE, et al. Surveillance of vaccination coverage among adult populations - United States, 2015. Morb Mortal Wkly Rep. 2017;66(11):1–28.

    Google Scholar 

  7. Kim DK, Riley LE, Hunter P. Advisory committee on immunization P: recommended immunization schedule for adults aged 19 years or older, United States, 2018. Ann Intern Med. 2018;168(3):210–20.

    Article  Google Scholar 

  8. Hurley LP, Bridges CB, Harpaz R, Allison MA, O'Leary ST, Crane LA, Brtnikova M, Stokley S, Beaty BL, Jimenez-Zambrano A, et al. U.S. physicians' perspective of adult vaccine delivery. Ann Intern Med. 2014;160(3):161.

    Article  Google Scholar 

  9. Johnson DR, Nichol KL, Lipczynski K. Barriers to adult immunization. Am J Med. 2008;121(7 Suppl 2):S28–35.

    Article  Google Scholar 

  10. Ndiaye SM, Hopkins DP, Shefer AM, Hinman AR, Briss PA, Rodewald L, Willis B. Task force on community preventive S: interventions to improve influenza, pneumococcal polysaccharide, and hepatitis B vaccination coverage among high-risk adults: a systematic review. Am J Prev Med. 2005;28(5 Suppl):248–79.

    Article  Google Scholar 

  11. National Vaccine Advisory C. Strategies to achieve the healthy people 2020 Annual influenza vaccine coverage goal for health-care personnel: recommendations from the national vaccine advisory committee. Public Health Rep. 2013;128(1):7–25.

    Article  Google Scholar 

  12. National Vaccine Advisory C. Recommendations from the National Vaccine Advisory committee: standards for adult immunization practice. Public Health Rep. 2014;129(2):115–23.

    Article  Google Scholar 

  13. Bridges CB, Hurley LP, Williams WW, Ramakrishnan A, Dean AK, Groom AV. Meeting the challenges of immunizing adults. Vaccine. 2015;33(Suppl 4):D114–20.

    Article  Google Scholar 

  14. Green LA, Nease D Jr, Klinkman MS. Clinical reminders designed and implemented using cognitive and organizational science principles decrease reminder fatigue. JABFM. 2015;28(3):351–9.

    Article  Google Scholar 

  15. Holt TA, Thorogood M, Griffiths F. Changing clinical practice through patient specific reminders available at the time of the clinical encounter: systematic review and meta-analysis. J Gen Intern Med. 2012;27(8):974–84.

    Article  Google Scholar 

  16. Groom H, Hopkins DP, Pabst LJ, Murphy Morgan J, Patel M, Calonge N, Coyle R, Dombkowski K, Groom AV, Kurilo MB, et al. Immunization information systems to increase vaccination rates: a community guide systematic review. JPHMP. 2015;21(3):227–48.

    PubMed  Google Scholar 

  17. Higuera L, Carlin C. A comparison of retrospective attribution rules. Am J Manag Care. 2017;23(6):e180–5.

    PubMed  Google Scholar 

Download references

Acknowledgments

We thank all participants of this project. We thank the DARTNet Institute for data and CDS support. We would like to acknowledge the AAFP National Research Network for providing essential expertise, staff and support.

Funding

Supported in part by a research grant from Investigator-Initiated Studies Program of Merck Sharp & Dohme Corp. Merck Sharp & Dohme Corp. played no role in the study design, data collection, data analysis and interpretation, writing or reporting of this work, and did not have any involvement in the decision to submit this article for publication. The opinions expressed in this work are those of the authors and do not necessarily represent those of Merck Sharp & Dohme Corp.

Author information

Authors and Affiliations

Authors

Contributions

NYL and CS made substantial contributions to the conception and design of the work, acquisition, analysis, and interpretation of data and have drafted and substantively revised the manuscript. EWS made substantial contributions to the interpretation of data and substantively revised the work. NN, EC, BW and WDP were primarily involved in data collection, analysis, and interpretation of the results and final editing of the manuscript. All authors have approved the submitted version (and any substantially modified version that involves the author’s contribution to the study). All authors have agreed both to be personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.

Corresponding author

Correspondence to Natalia Y. Loskutova.

Ethics declarations

Ethics approval and consent to participate

The American Academy of Family Physicians Institutional Review Board (AAFP IRB) approved the study. The providers in the study provided signed informed consent; the patient informed consent was waived.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Appendix 1 Patient Population Overview.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Loskutova, N.Y., Smail, C., Callen, E. et al. Effects of multicomponent primary care-based intervention on immunization rates and missed opportunities to vaccinate adults. BMC Fam Pract 21, 46 (2020). https://doi.org/10.1186/s12875-020-01115-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12875-020-01115-y

Keywords