Skip to main content
Download PDF

What quantifies good primary care in the United States? A review of algorithms and metrics using real-world data

BMC Primary Care volume 24, Article number: 130 (2023) Cite this article

Abstract

Primary care physicians (PCPs) play an indispensable role in providing comprehensive care and referring patients for specialty care and other medical services. As the COVID-19 outbreak disrupts patient access to care, understanding the quality of primary care is critical at this unprecedented moment to support patients with complex medical needs in the primary care setting and inform policymakers to redesign our primary care system. The traditional way of collecting information from patient surveys is time-consuming and costly, and novel data collection and analysis methods are needed. In this review paper, we describe the existing algorithms and metrics that use the real-world data to qualify and quantify primary care, including the identification of an individual’s likely PCP (identification of plurality provider and major provider), assessment of process quality (for example, appropriate-care-model composite measures), and continuity and regularity of care index (including the interval index, variance index and relative variance index), and highlight the strength and limitation of real world data from electronic health records (EHRs) and claims data in determining the quality of PCP care. The EHR audits facilitate assessing the quality of the workflow process and clinical appropriateness of primary care practices. With extensive and diverse records, administrative claims data can provide reliable information as it assesses primary care quality through coded information from different providers or networks. The use of EHRs and administrative claims data may be a cost-effective analytic strategy for evaluating the quality of primary care.

Peer Review reports

Background

In the United States, primary care physicians (PCPs) have the essential responsibilities of referring their patients to appropriate specialists, ensuring the coordination of clinical care and other medical resources, and helping to detect, treat and prevent illness. Given the critical role of PCPs in healthcare, efficient quality assessment of primary care remains indispensable. The U.S. Agency for Healthcare Research and Quality (AHRQ) develops patient experience and satisfaction surveys to determine the quality of primary care [1]. However, the significant problems with these subjective, self-reported measurements are that they are time- and cost-consuming [2]. It is arduous to produce a representative yet standardized survey across the country to measure patients’ perspectives regarding primary care quality and allow for meaningful comparisons among peers. In contrast, big data in healthcare is an easier and faster data organization that could alleviate time and money consumption. Big data in healthcare includes, but is not limited to, electronic health records (EHRs), claims data collected by payor records, medical imaging, genomic sequencing, and eHealth data [3]. Objective metrics based on existing EHRs and administrative claims data demonstrate the tremendous possibility of examining primary care effectively.

Measuring the performance of PCP care continues to be an important yet challenging part of clinical research and quality improvement due to the lack of the “common vocabulary [4]”. In the early period of primary care establishment, Barbara Starfield emphasized the four different responsibilities of high-quality primary care services, which include “first-contact accessibility, continuity, comprehensiveness, and coordination [5].”

“First contact accessibility” describes the capacity of a primary care system to serve as the first contact person, gatekeeper to the health system, and coordinator of care. To promote first-contact accessibility and person-centeredness, it is important to focus efforts on where people currently seek primary care services and who is the most likely responsible PCP. “Continuity of Care,” as a key pillar of an effective healthcare system, is defined as seeing the same PCP over time [6]. “Comprehensiveness” overlaps the scope of practice, sites of care depth, and breadth of conditions managed by the PCP [7], including prevention and wellness and acute, chronic, and comorbid condition management. “Coordination” refers to the holistic organization of patient-physician interaction among multiple healthcare providers [8], for example, the specialists and PCPs. No consensus definition has fully evolved for the definition of “comprehensiveness” and “coordination” [9]; however, they are both based mainly on the “process” measures of primary care.

Type of data resources, and strength and weakness of these data sources

EHRs and administrative claims data represent two different sets of real-world healthcare information resources. The EHRs reflect the practice pattern by recording and managing patient care, while claims data track the patient’s utilization of services based on the billing and payment of health services. The use of EHRs and claims data to analyze patients’ PCP encounters requires comprehensive computerization of primary care practices. Along with national level efforts following the Health Information Technology for Economic and Clinical Health Act 2009 [52], EHR audits may have an immense potential for a large number of digital medical records to describe the workflow of PCP encounters quantitatively. Administrative claims data, based on large and diverse records from the integrated cloud and on-premises data warehouses, could include billing codes for patients' primary care services from different providers or networks and reduce the selection bias from a single provider or network. The use of administrative claims data can also provide a comprehensive view of a patient’s encounters with providers (e.g., place and date of service, diagnosis history and codes, treatment, and PCP types), beneficiary benefit coverage, pharmacy records, procedures, and performance of laboratory tests.

Measurements of PCP performance

A wide range of quality indicators and algorithms have been proposed to evaluate these aspects of primary care (Table 1): 1) identifying an individual’s likely responsible PCP, 2) process measures, and 3) the continuity and regularity of care. In this review paper, we examined and compared the existing algorithms and metrics and discussed the strengths and limitations of EHR and claims data, as well as specific case examples. The structural knowledge of the measurements can help researchers identify and make the most appropriate choices among different conceptions and methodological strategies.

Table 1 Algorithms and metrics as quality indicators of primary care
Full size table

Identifying an individual’s likely responsible PCP

Researchers suggested that care coordination would improve when patients visit the same PCP for most of their primary care visits and when specialist referrals are determined by the likely responsible PCP [4, 10]. The issue of having multiple PCPs included increased medical services expenditures due to more office visits, prescriptions, and a more significant number of specialists seen for disease-specific populations [11]. Previous health services studies recommended that every patient see the same PCP [12]. However, in reality, patients may not have any responsible PCP, or they may have more than one. The claims-based “majority” and “plurality” algorithms (Table 1) were developed to identify the likely responsible PCP. The plurality provider is the provider who billed the highest number of Evaluation & Management (E&M) visits in one year, whereas the majority provider is the provider who billed for the plurality (greater than 50%) of all visits.

Process measures

Several process measures have been documented in the literature, including medication intensification [13], lifestyle counseling documented in the notes of PCPs [14], the appropriate-care-model composite score for preventative care [15,16,17], time from patient discharge to the first visit with PCP [18, 19], peer review and identification of negligent adverse events, and the duration of PCP consultation [20](1). Rather than the manual extraction, peer review, and identifications from the PCP notes, the process of these face-to-face encounters can be efficiently extracted from appropriate billing codes. For instance, medication intensification, defined as the initiation of a new medication or an increase in the dose of an existing medication [21], can be identified from the pharmacy data. Morrison et al. [13] used the EHR data from Brigham and Women’s and Massachusetts General Hospitals to identify the acute encounters for acute pain and infection of patients based on the availability of appropriate billing codes and ICD-9 diagnosis codes. The data linkage to patients’ pathology test data for elevated A1C, lipid profile, or blood pressure [13] provided the possibility for identifying the preventative care services the patient received. The use of billing codes [22] may accompany great uncertainty and does not necessarily reflect every single patient's clinical information when the billing codes are created in a way more relevant to providers.

There was an ongoing debate regarding the optimal duration of the consultation. Orton and colleagues demonstrated that longer consultations had significant beneficial effects on patients [23]; however, the Cochrane systematic review by Wilson and Childs showed no incremental benefits from having longer consultations [24]. The controversy led to the development of new measures. The long to short consultation ratio [25, 26] was a method to measure the quality of care for general practices. The PCPs with the fastest times (1st quartile) were assigned as “faster” doctors; those in the last quartile with the longest times were described as “slower” doctors; and within each of the three doctor styles, the percentage distribution of consultation lengths was displayed and the ratio of long to short consultations calculated. The billing codes for claims data [27] also allowed for identifying the duration of the PCP consultations. Since 2003, the Centers for Medicare & Medicaid Services (CMS) collected claims data for Medicare and Medicaid patients across various categories and times, including Inpatient and Outpatient claims, Master Beneficiary Summary Files, and many other files. CMS continues to update a list of Current Procedural Terminology (CPT®) and Healthcare Common Procedure Coding System (HCPCS) codes, bill procedures, and services codes, which the American Medical Association primitively developed. In CPT codes, 99,211, 99,212, 99,213, 99,214, and 99,215 corresponded to 5 ~ , 10 ~ , 15 ~ , 25 ~ , and 40 ~ minute consultation, respectively [27]. Since COVID-19, telehealth has experienced a dramatic increase. Connection to PCP via telehealth was not novel but emerged during the COVID period as a promising model of healthcare. Some measures, for example, “no-shows” and “same-day” cancellations, “after-hours” care availability (i.e., care that is not Mon–Fri, 8 a.m. – 5 p.m.) from both EHR and claims data have evolved as quality-based metrics [28].

Distance to the medical offices and availability of primary care services in patients’ communities are known barriers to access to care, especially in rural areas [29]. The geographic access estimation, for example, number of PCPs/10,000 population and availability of PCPs within a distance of a 30-min drive time [30, 31], enables the assessment of the spatial distribution of primary care providers and helps inform the health planners of the possibility of optimizing the health resources in order to achieve efficient primary care [32, 33].

Continuity and regularity of care

Another core principle of primary care is the continuity of care [34]. It illustrates an ongoing relationship between the PCP and the patient beyond a single episode of illness. A couple of studies have found a strong correlation [35,36,37,38] of the continuity of PCP care with enhanced patient-physician relationships, which include trust-building, effective communications, a sense of responsibility over time, greater patient satisfaction and higher quality of care (e.g., better recognition of problems and diagnostic accuracy, effective management of patients with chronic conditions and maternity care outcomes), higher rates of compliance to medications, better performance of screening tests, timely receipt of preventive medicine services, less loss in follow-up visits, and a considerable reduction in hospitalizations, repeat hospitalizations, emergency department (ED) visits, and shorter length of hospital stay. There are multiple indices and algorithms to check the continuity of care from different dimensions (Table 1). From the review of existing evidence [39,40,41,42], we have found convergences and divergences in a considerable spectrum of continuity of care measurements with different considerations of what is essential to measure. For instance, they can be described as measures of concentration (the proportion of consultations with one specific provider), dispersion (the number of different professionals consulted), distribution (the distribution of consultations between providers, giving higher scores to people who consult fewer providers); or sequence (whether each consultation was with the same provider as the previous consultation) [43]. Some measures are based on attributing scores to individuals (“individual measures”), while others attribute a score to each consultation (“visit measures”). Some approaches have focused on visit patterns only, whereas others have defined an individual provider as the “primary” provider for each patient. Evidently, no single measurement approach could fully capture the whole concept. Therefore, more emphasis should be given to developing and applying direct measures from the patient’s perspective, focusing on information sharing and care consistency between various organizations, and broadening the inclusion of continuity measurement in the multi-disciplinary team level [41].

Continuity of care also has potential limitations. First, those measurements may not always have a straightforward meaning [43]; for example, the value between 0 (different doctors on every occasion) and 1 (all care from the same doctor) for the continuity of care (COC) index does not release a specific message by itself. Secondly, repeated contact with the same PCP may bring gain and loss. Having multiple PCPs [44] may allow confirmation of diagnoses and suggestion of additional directions for diagnoses after discussing with each other. Some British researchers shared their concerns that higher COC [45] might paradoxically impair the patients because familiarity may reduce the duration of each visit, prevent patients from discussing new problems, and lessen the PCP’s objectivity. It could frustrate PCPs [45] if their patients have insoluble issues, and the long-term continuity may lead to PCP burnout.

The temporal regularity of primary care visits has been a novel concept in recent years. This method of regularity score to assess the regularity of PCP was developed by Einarsdóttir et al. [46] as 1/[1 + Variance (\({\varnothing}i\))], where \({\varnothing}i\) was the time interval between the (i-l)th and ith PCP visits. It ranged from 0 to 1 (with 1 representing perfect regularity) and was divided into quartiles, each containing 25% of the study population. Any individual score did not represent a unique distribution of any set number of PCP visits; instead, an aggregate tendency toward even spread and lack of clustering. The two hypotheses [47] behind the regularity index was (1) that a more temporally regular pattern of visits reflected the higher quality of care at some sites, achieved through more effective efforts to manage patients proactively and conversely to prevent loss to follow-up, and (2) temporal regularity was capturing unmeasured patient-level variables that are associated with risk for poor outcomes, such as a patient’s propensity to attend scheduled appointments or to participate in other health-promoting activities. Researchers were generally positive about this new measurement because they have identified a high correlation between high temporal regularity of PCP visits and positive health outcomes in various study samples  [47,48,49,50,51].

Application: a case example for understanding the role of primary care in cancer survivorship

In this section, we present a case on operationalizing various metrics to guide the conceptualization and evaluation of emerging cancer survivorship care models involving PCPs [53, 54]. Foremost, the scope of the metrics can be compared against existing guidelines [55] or frameworks [56] to clearly define the role of PCPs. In the management of survivors of cancer, the relevant care domains include health surveillance, lifestyle modification, preventive care, management of physical, psychosocial issues, and chronic conditions, together with the use of survivorship care tools to ensure care continuity beyond the oncology to primary care setting, are highly compatible with the process and care continuity measures. It was postulated that a lack of regularity of interactions with the health system among cancer survivors compared to patients with other chronic conditions may have reduced opportunities for advocacy of self-management skills [57], a care gap that PCPs can address with existing skillsets [58]. Also, oncologists could conveniently apply algorithms to EHR data to identify the appropriate PCPs for survivors’ consideration, allaying reported concerns of disrupting existing doctor-patient relationships wherever possible [59, 60]. A clear delineation of the PCPs’ role could directly address existing divergent views reported in the literature [61] and demarcate roles and responsibilities [53] from tertiary providers. Furthermore, the clear links between these performance indicators and principles of primary care serve to reduce confidence-related hesitancy.

Besides helping to conceptualize PCPs’ roles, the library of measurements will allow a holistic and timely audit or evaluation of primary care involvement in new care models or programs using readily available EHR data in real time. By including metrics from different primary care aspects one can ensure comprehensive and robust outcomes to overcome the pitfalls of considering indicators in silo. Additionally, the granularity of the data from the metrics allows researchers to identify poor performing areas, facilitating the development of strategies to improve specific care aspects. Lastly, this case of cancer survivorship supported our recommendation on developing additional measures to assess information sharing and care consistency between various organizations. This endeavor will complement existing efforts by the American Society of Cancer Oncology to build oncology EHR for advancing and ensuring quality cancer care [62, 63].

Challenges and opportunities

We acknowledge that, in some cases, the available data may not be ideally suited for the objective measures described above. The implementation of these quality measures requires substantial investments in data collection, processing, and analysis. The digital transition of medical records to EHRs requires tremendous efforts, which may not be immediately achievable at the local, state, and national levels. Most delivery systems rely on technical support provided solely by a single EHR software vendor and only identify the care process for network PCPs, which results in a lack of interoperability across providers, networks, and vendors. Additionally, the adoption of “big data” in healthcare in the United States falls behind other industries under the regulatory environment of the Health Insurance Portability and Accountability Act, and the Health Information Technology for Economic and Clinical Health Act enacted to protect personal health information in the United States [64], compared to some other countries (e.g., the United Kingdom), where the detailed electronic primary care records, procedure registries, cause-specific hospitalization, mortality record, and census data are available at a national scale [65].

Moreover, there was a knowledge gap between the data-based algorithms versus the patient-reported quality of PCP care. Bentleret et al. [66] compared the claims data-based COC indices with self-reported National Health and Health Services Use questionnaires of 2,620 Medicare beneficiaries and found that most claims-based COC measures failed to coincide with patients’ subjective perceptions of continuous patient-provider relationships. Future studies may potentially gain insight into the performance of the EHR and claims-based metrics.

The COVID-19 pandemic significantly impacts patient care and the healthcare system, including primary care. The pandemic has dramatically changed how primary care is delivered. Providers defer elective and preventive visits, such as annual physical exams, and convert in-person visits to telemedicine/telehealth visits to prevent person-to-person transmission. Currently, CMS [67] allows more telehealth services for all clinicians by expanding the terms of CPT codes, including remote evaluations, virtual check-ins and e-visits, and remote patient monitoring, and temporarily waive Medicare and Medicaid’s requirements that physicians and non-physician practitioners be licensed in the state where they are providing services. All of these efforts could take effect but leave many uncertainties about the factual delivery of primary care. As more clinical data become collected and standardized across providers, we hope that integrating EHRs and administrative claims data will be a cost-effective way to assess the quality of primary care.

Conclusion

Population-level systematic assessment of the quality of primary care is challenging and costly. In the digital big data era, the integrated use of EHRs and administrative claims data sources may represent a promising cost-effective approach for comprehensive assessments of primary care quality. Migration to integrated universal health information infrastructure [83] and sharing and linkage of health data may support more systematic evaluation of primary care. However, significant efforts are needed to facilitate data integration in many countries. Bearing the limitation, machine learning and artificial intelligence have the potential to address some of the limitations mentioned in the context of population-level systematic assessment of primary care quality. Machine learning and artificial intelligence techniques can help in extracting valuable insights from the integrated use of EHRs and administrative claims data sources, which can support comprehensive assessments of primary care quality in a cost-effective way. However, there is a need for ongoing research and development to refine the existing measurements and develop new measures to assess all primary care domains effectively. Moreover, efforts are needed to facilitate data integration and ensure the security and privacy of health data.

Availability of data and materials

Not applicable.

Abbreviations

PCP:

Primary care physician

EHRs:

Electronic health records

E&M:

Evaluation & Management

CMS:

Centers for Medicare & Medicaid Services

CPT®:

Current Procedural Terminology

HCPCS:

Healthcare Common Procedure Coding System

ED:

Emergency department

COC:

Continuity of care

References

  1. Davis K. a. S. S. C. a. A. A. M. "A 2020 vision of patient-centered primary care,". J Gen Int Med. 2005;20(10):953–7. https://doi.org/10.1111/j.1525-1497.2005.0178.x.

  2. Prinz A, Menschner P, Leimeister JM. Electronic data capture in healthcare-NFC as easy way for self-reported health status information. Health Policy and Technology. 2012;1(3):137–44. https://doi.org/10.1016/j.hlpt.2012.07.001.

    Article  Google Scholar 

  3. Catalyst N. "Healthcare big data and the promise of value-based care,". NEJM Catalyst. 2018;4(1).

  4. O’Malley AS, Cunningham PJ. Patient Experiences with Coordination of Care: The Benefit of Continuity and Primary Care Physician as Referral Source. J Gen Intern Med. 2009;24(2):170–7. https://doi.org/10.1007/s11606-008-0885-5.

    Article  PubMed  Google Scholar 

  5. Starfield B. IS PRIMARY-CARE ESSENTIAL. Lancet. 1994;344(8930):1129–33. https://doi.org/10.1016/s0140-6736(94)90634-3.

    Article  CAS  PubMed  Google Scholar 

  6. IfCaE R. "2020–2023 Value Assessment Framework," ed. pp. 2020. https://icer.org/wp-content/uploads/2020/10/ICER_2020_2023_VAF_013120-4-1.pdf.

  7. O’Malley AS, Rich EC. Measuring Comprehensiveness of Primary Care: Challenges and Opportunities. J Gen Intern Med. 2015;30:568–75. https://doi.org/10.1007/s11606-015-3300-z.

    Article  PubMed Central  Google Scholar 

  8. Starfield BH, Simborg DW, Horn SD, Yourtee SA. Continuity and coordination in primary care - their achievement and utility. Med Care. 1976;14(7):625–36. https://doi.org/10.1097/00005650-197607000-00008.

    Article  CAS  PubMed  Google Scholar 

  9. Haggerty J, et al. "Operational definitions of attributes of primary health care: Consensus among Canadian experts,". Ann Fam Med. 2007;5(4):336–44.  https://doi.org/10.1370/afm.682.

  10. Pham HH, Schrag D, O’Malley AS, Wu BN, Bach PB. Care patterns in Medicare and their implications for pay for performance. N Engl J Med. 2007;356(11):1130–9. https://doi.org/10.1056/NEJMsa063979.

    Article  CAS  PubMed  Google Scholar 

  11. Raddish M, Horn SD, Sharkey PD. Continuity of care: Is it cost effective? Am J Managed Care. 1999;5(6):727–34.

    CAS  Google Scholar 

  12. DuGoff EH, Walden E, Ronk K, Palta M, Smith M. Can Claims Data Algorithms Identify the Physician of Record? Med Care. 2018;56(3):e16–20. https://doi.org/10.1097/mlr.0000000000000709.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Morrison F, Shubina M, Goldberg SI, Turchin A. Performance of primary care physicians and other providers on key process measures in the treatment of diabetes. Diabetes Care. 2013;36(5):1147–52. https://doi.org/10.2337/dc12-1382.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Morrison F, Shubina M, Turchin A. Lifestyle counseling in routine care and long-term glucose, blood pressure, and cholesterol control in patients with diabetes. Diabetes Care. 2012;35(2):334–41. https://doi.org/10.2337/dc11-1635.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Higgins A, Zeddies T, Pearson SD. Measuring the performance of individual physicians by collecting data from multiple health plans: the results of a two-state test. Health Aff. 2011;30(4):673–81. https://doi.org/10.1377/hlthaff.2011.0070.

    Article  Google Scholar 

  16. Scholle SH, et al. Benchmarking physician performance: reliability of individual and composite measures. Am J Manag Care. 2008;14(12):829–38.

    Google Scholar 

  17. Shwartz M, Restuccia JD, Rosen AK. Composite Measures of Health Care Provider Performance: A Description of Approaches. Milbank Q. 2015;93(4):788–825. https://doi.org/10.1111/1468-0009.12165.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Starfield B, Cassady C, Nanda J, Forrest CB, Berk R. Consumer experiences and provider perceptions of the quality of primary care: Implications for managed care. J Fam Pract. 1998;46(3):216–26.

    CAS  PubMed  Google Scholar 

  19. A. I. o. H. a. Welfare, "Transition between hospital and community care for patients with coronary heart disease: New South Wales and Victoria 2012–2015," AIHW, Canberra, 2018. [Online]. Available: https://www.aihw.gov.au/reports/heart-stroke-vascular-diseases/transition-hospital-community-care-heart-disease.

  20. Bener A, Almarri S, Ali B, Aljaber K. Do minutes count for health care? Consultation length in a tertiary care teaching hospital and in general practice. Middle East J Fam Med. 2007;5(1):3–8.

    Google Scholar 

  21. Heisler M, Hogan MM, Hofer TP, Schmittdiel JA, Pladevall M, Kerr EA. When More Is Not Better. Circulation. 2008;117(22):2884–92. https://doi.org/10.1161/CIRCULATIONAHA.107.724104.

    Article  PubMed  Google Scholar 

  22. Holmes C. "The problem list beyond meaningful use. Part 2: fixing the problem list,". J AHIMA. 2011;82(3):32–5; quiz 36, 2011.

  23. Orton PK, Gray DP. Factors influencing consultation length in general/family practice. Fam Pract. 2016;33(5):529–34. https://doi.org/10.1093/fampra/cmw056.

    Article  PubMed  Google Scholar 

  24. Wilson AD, Childs S. "Effects of interventions aimed at changing the length of primary care physicians' consultation,". Cochrane Database Syst Rev. 2006;(1). Art no. CD003540.pub2.  https://doi.org/10.1002/14651858.CD003540.pub2.

  25. Howie JGR, Porter AMD, Heaney DJ, Hopton JL. long to short consultation ratio - a proxy measure of quality of care for general-practice. Br J Gen Pract. 1991;41(343):48–54.

    CAS  PubMed  PubMed Central  Google Scholar 

  26. Heath I. LONG TO SHORT CONSULTATION RATIOS. Br J Gen Pract. 1991;41(345):169–169.

    Google Scholar 

  27. Heidelbaugh JJ, Riley M, Habetler JM. 10 billing & coding tips to boost your reimbursement. J Fam Pract. 2008;57(11):724–30.

    PubMed  Google Scholar 

  28. Hollander J. Neinstein A. "Maturation from adoption-based to quality-based telehealth metrics," NEJM Catalyst Innovations in Care Delivery. 2020;1(5).

  29. Guagliardo MF. "Spatial accessibility of primary care: concepts, methods and challenges,". Int J Health Geogr. 2004;3(1):3.

  30. Shah TI, Clark AF, Seabrook JA, Sibbald S, Gilliland JA. "Geographic accessibility to primary care providers: Comparing rural and urban areas in Southwestern Ontario,". Canadian Geographer-Geographe Canadien. 2020;64(1):65–78. https://doi.org/10.1111/cag.12557.

  31. Teach SJ, et al. Spatial accessibility of primary care pediatric services in an urban environment: Association with asthma management and outcome. Pediatrics. 2006;117(4):S78–85. https://doi.org/10.1542/peds.2005-2000E.

    Article  PubMed  Google Scholar 

  32. Luo W, Wang FH. Measures of spatial accessibility to health care in a GIS environment: synthesis and a case study in the Chicago region. Environ Plann B-Plann Des. 2003;30(6):865–84. https://doi.org/10.1068/b29120.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Wang FH, Luo W. Assessing spatial and nonspatial factors for healthcare access: towards an integrated approach to defining health professional shortage areas. Health Place. 2005;11(2):131–46. https://doi.org/10.1016/j.healthplace.2004.02.003.

    Article  PubMed  Google Scholar 

  34. Hall MN. "Continuity: A Central Principle of Primary Care,". J Grad Med Educ. 2016;8(4):615–6.

  35. Haggerty JL, Reid RJ, Freeman GK, Starfield BH, Adair CE, McKendry R. Continuity of care: a multidisciplinary review. BMJ. 2003;327(7425):1219–21. https://doi.org/10.1136/bmj.327.7425.1219.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Kringos DS, Boerma WGW,  Hutchinson A, van der Zee J. Groenewegen PP. Bmc Health Services Research. 2010;10. Art no. 65.  https://doi.org/10.1186/1472-6963-10-65.

  37. Nutting P. A, Goodwin M. A, Flocke S. A, Zyzanski S. J, Stange K. C. Continuity of primary care: to whom does it matter and when? Ann Fam Med. 2003;1(3):149–55. https://doi.org/10.1370/afm.63.

    Article  PubMed  PubMed Central  Google Scholar 

  38. van Walraven C, Oake N, Jennings A, Forster AJ. The association between continuity of care and outcomes: a systematic and critical review. J Eval Clin Pract. 2010;16(5):947–56. https://doi.org/10.1111/j.1365-2753.2009.01235.x.

    Article  PubMed  Google Scholar 

  39. Jee SH, Cabana MD. Indices for continuity of care: A systematic review of the literature. Med Care Res Rev. 2006;63(2):158–88. https://doi.org/10.1177/1077558705285294.

    Article  PubMed  Google Scholar 

  40. Saultz J. W, Lochner J. Interpersonal continuity of care and care outcomes: A critical review. Ann Fam Med. 2005;3(2):159–66. https://doi.org/10.1370/afm.285.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Saultz J. W. Defining and measuring interpersonal continuity of care. Ann Fam Med. 2003;1:134–43. https://doi.org/10.1370/afm.23.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Uijen AA, Schers HJ, Schellevis FG, van den Bosch W. How unique is continuity of care? A review of continuity and related concepts. Fam Pract. 2012;29(3):264–71. https://doi.org/10.1093/fampra/cmr104.

    Article  PubMed  Google Scholar 

  43. Salisbury C, Sampson F, Ridd M, Montgomery AA. How should continuity of care in primary health care be assessed? Br J Gen Pract. 2009;59(561):276–82. https://doi.org/10.3399/bjgp09X420257.

    Article  Google Scholar 

  44. Dreiher J, Comaneshter D. S, Rosenbluth Y, Battat E, Bitterman H, Cohen A. D. The association between continuity of care in the community and health outcomes: a population-based study. Isr J Health Policy Res. 2012;1:21. https://doi.org/10.1186/2045-4015-1-21.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Gray DP, et al. Towards a theory of continuity of care. J R Soc Med. 2003;96(4):160–6. https://doi.org/10.1258/jrsm.96.4.160.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Einarsdottir K, Preen DB, Emery JD, Holman CDJ. Regular primary care decreases the likelihood of mortality in older people with epilepsy. Med Care. 2010;48(5):472–6. https://doi.org/10.1097/MLR.0b013e3181d68994.

    Article  PubMed  Google Scholar 

  47. Rose AJ, Timbie JW, Setodji C, Friedberg MW, Malsberger R, Kahn KL. Primary Care Visit Regularity and Patient Outcomes: an Observational Study. J Gen Intern Med. 2019;34(1):82–9. https://doi.org/10.1007/s11606-018-4718-x.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Camacho F, Hwang W, Kern T, Anderson R. T. Receipt of Regular Primary Care and Early Cancer Detection in Appalachia. J Rural Health. 2015;31(3):269–81. https://doi.org/10.1111/jrh.12097.

    Article  PubMed  Google Scholar 

  49. Einarsdottir K, Preen DB, Emery JD, Holman CDJ. Regular primary care plays a significant role in secondary prevention of ischemic heart disease in a Western Australian cohort. J Gen Intern Med. 2011;26(10):1092–7. https://doi.org/10.1007/s11606-011-1665-1.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Gibson DAJ, Moorin RE, Preen D, Emery J, Holman CDJ. Enhanced primary care improves GP service regularity in older patients without impacting on service frequency. Aust J Prim Health. 2012;18(4):295–303. https://doi.org/10.1071/py11050.

    Article  PubMed  Google Scholar 

  51. Youens D, Harris M, Robinson S, Preen DB, Moorin RE. Regularity of contact with GPs: Measurement approaches to improve valid associations with hospitalization. Fam Pract. 2019;36(5):650–6. https://doi.org/10.1093/fampra/cmz002.

    Article  PubMed  Google Scholar 

  52. Chatterji M, et al. Development and Validation of a Health Information Technology Curriculum: Toward More Meaningful Use of Electronic Health Records. Pedagogy in Health Promotion. 2017;3(3):154–66.

    Article  Google Scholar 

  53. Chaput G, Sussman J. Integrating primary care providers through the seasons of survivorship. Curr Oncol. 2019;26(1):48–54. https://doi.org/10.3747/co.26.4687.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Nekhlyudov L, O’Malley DM, Hudson SV. Integrating primary care providers in the care of cancer survivors: gaps in evidence and future opportunities. Lancet Oncology. 2017;18(1):E30–8. https://doi.org/10.1016/s1470-2045(16)30570-8.

    Article  PubMed  Google Scholar 

  55. C. D. Runowicz et al., "American Cancer Society/American Society of Clinical Oncology Breast Cancer Survivorship Care Guideline," Journal of Clinical Oncology, vol. 34, no. 6, pp. 611-+, Feb 2016,  https://doi.org/10.1200/jco.2015.64.3809.

  56. L. Nekhlyudov, M. A. Mollica, P. B. Jacobsen, D. K. Mayer, L. N. Shulman, and A. M. Geiger, "Developing a Quality of Cancer Survivorship Care Framework: Implications for Clinical Care, Research, and Policy," Jnci-Journal of the National Cancer Institute, vol. 111, no. 11, pp. 1120–1130, Nov 2019, Art no. djz089, doi: https://doi.org/10.1093/jnci/djz089.

  57. B. C. Risendal, A. Dwyer, R. W. Seidel, K. Lorig, L. Coombs, and M. G. Ory, "Meeting the challenge of cancer survivorship in public health: results from the evaluation of the chronic disease self-management program for cancer survivors," Frontiers in public health, vol. 2, p. 214, 2014 2014, doi: https://doi.org/10.3389/fpubh.2014.00214.

  58. S. Dineen-Griffin, V. Garcia-Cardenas, K. Williams, and S. I. Benrimoj, "Helping patients help themselves: A systematic review of self-management support strategies in primary health care practice," Plos One, vol. 14, no. 8, Aug 2019, Art no. e0220116, doi: https://doi.org/10.1371/journal.pone.0220116.

  59. Chan A, et al. Practitioners’ perspectives on community-based breast cancer survivorship care in Singapore: A focus group study. Health Soc Care Community. 2018;26(3):404–11. https://doi.org/10.1111/hsc.12528.

    Article  PubMed  Google Scholar 

  60. Fok RWY, et al. Roles and recommendations from primary care physicians towards managing low-risk breast cancer survivors in a shared-care model with specialists in Singapore-a qualitative study. Fam Pract. 2020;37(4):547–53. https://doi.org/10.1093/fampra/cmaa009.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Crabtree B. F, et al. Cancer Survivorship Care Roles for Primary Care Physicians. Ann Fam Med. 2020;18(3):202–9. https://doi.org/10.1370/afm.2498.

    Article  PubMed  PubMed Central  Google Scholar 

  62. "Ensuring Continuity of Care Through Electronic Health Records," Journal of Oncology Practice, vol. 3, no. 3, pp. 137–142, 2007,  https://doi.org/10.1200/jop.0733501.

  63. P. Yu, D. Artz, and J. Warner, "Electronic health records (EHRs): supporting ASCO's vision of cancer care," American Society of Clinical Oncology educational book. American Society of Clinical Oncology. Annual Meeting, pp. 225–31, 2014 2014,  https://doi.org/10.14694/EdBook_AM.2014.34.225.

  64. Cohen IG, Mello MM. HIPAA and Protecting Health Information in the 21st Century. JAMA. 2018;320(3):231–2. https://doi.org/10.1001/jama.2018.5630.

    Article  PubMed  Google Scholar 

  65. Denaxas SC, et al. Data Resource Profile: Cardiovascular disease research using linked bespoke studies and electronic health records (CALIBER). Int J Epidemiol. 2012;41(6):1625–38. https://doi.org/10.1093/ije/dys188.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Bentler SE, Morgan RO, Virnig BA, Wolinsky FD. Do claims-based continuity of care measures reflect the patient perspective? Med Care Res Rev. 2014;71(2):156–73. https://doi.org/10.1177/1077558713505909.

    Article  PubMed  Google Scholar 

  67. Cohen B. H, Busis, N. A, Ciccarelli L. Coding in the World of COVID-19: Non-Face-to-Face Evaluation and Management Care. Continuum (Minneapolis, Minn). 2020;26(3):785–98. https://doi.org/10.1212/con.0000000000000874.

    Article  PubMed  Google Scholar 

  68. Phillips RL, Bartholomew LA, Dovey SM, Fryer GE, Miyoshi TJ, Green LA. Learning from malpractice claims about negligent, adverse events in primary care in the United States. Qual Saf Health Care. 2004;13(2):121–6. https://doi.org/10.1136/qshc.2003.008029.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Mira J. J, Nebot C, Lorenzo S, Perez-Jover V. Patient report on information given, consultation time and safety in primary care. Qual Saf Health Care. 2010;19(5):e33. https://doi.org/10.1136/qshc.2009.037978.

    Article  PubMed  Google Scholar 

  70. Pollack CE, Lemke KW, Roberts E, Weiner JP. Patient sharing and quality of care measuring outcomes of care coordination using claims data. Med Care. 2015;53(4):317–23.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Bice TW, Boxerman SB. Quantitative measure of continuity of CARE. Med Care. 1977;15(4):347–9. https://doi.org/10.1097/00005650-197704000-00010.

    Article  CAS  PubMed  Google Scholar 

  72. Lou WYW. The exact distribution of the continuity of care measure NOP. Statist Probab Lett. 2000;48(4):361–8. https://doi.org/10.1016/s0167-7152(00)00017-1.

    Article  Google Scholar 

  73. Reid RJ, Haggerty JL, McKendry R, Reid R, Haggerty J. "Defusing the Confusion: Concepts and Measures of Continuity of Healthcare,". 2002.

  74. Starfield B. Primary care: balancing health needs, services, and technology. Religion in America, 1998.

  75. Steinwachs DM. Measuring provider continuity in ambulatory care - assessment of alternative approaches. Med Care. 1979;17(6):551–65. https://doi.org/10.1097/00005650-197906000-00001.

    Article  CAS  PubMed  Google Scholar 

  76. Cooper S. Can primary care and continuity of care prevent asthma-related emergency department use and hospitalizations amongst children? McGill University (Canada), 2019.

  77. Lou W. A new measure for continuity of care: the Alpha index. Health Serv Outcomes Res Method. 2000;1(3):277–89.

    Article  Google Scholar 

  78. Sequist TD, Singh S, Pereira AG, Rusinak D, Pearson SD. Use of an electronic medical record to profile the continuity clinic experiences of primary care residents. Acad Med. 2005;80(4):390–4. https://doi.org/10.1097/00001888-200504000-00017.

    Article  PubMed  Google Scholar 

  79. Lee JH, Choi YJ, Lee SH, Sung NJ, Kim SY, Hong JY. Association of the length of doctor-patient relationship with primary care quality in seven family practices in Korea. J Korean Med Sci. 2013;28(4):508–15. https://doi.org/10.3346/jkms.2013.28.4.508.

    Article  PubMed  PubMed Central  Google Scholar 

  80. Reddy A, Pollack CE, Asch DA, Canamucio A, Werner RM. The Effect of Primary Care Provider Turnover on Patient Experience of Care and Ambulatory Quality of Care. JAMA Intern Med. 2015;175(7):1157–62. https://doi.org/10.1001/jamainternmed.2015.1853.

    Article  PubMed  PubMed Central  Google Scholar 

  81. Pollack CE, Hussey PS, Rudin RS, Fox DS, Lai J, Schneider EC. Measuring Care Continuity A Comparison of Claims-based Methods. Med Care. 2016;54(5):E30–4. https://doi.org/10.1097/mlr.0000000000000018.

    Article  PubMed  PubMed Central  Google Scholar 

  82. Ha N. T, Harris M, Preen D, Robinson S, Moorin R. A time-duration measure of continuity of care to optimise utilisation of primary health care: a threshold effects approach among people with diabetes. Bmc Health Serv Res. 2019;19:276. https://doi.org/10.1186/s12913-019-4099-9.

    Article  PubMed  PubMed Central  Google Scholar 

  83. Gamal A, Barakat S, Rezk A. Letter to Editor (Response from author): Toward a universal electronic health record system. J Biomed Inform. 2021;117:103770. https://doi.org/10.1016/j.jbi.2021.103770.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We are grateful for the support from Chapman University School of Pharmacy.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. School of Pharmacy, Chapman University, Irvine, US

    Yun Wang, Jianwei Zheng, Jerika Lam, Mary Gutierrez, Ivan Portillo, Yang Qu & Lawrence Brown

  2. Gehr Center for Health Systems Science and Innovation, Keck School of Medicine, University of Southern California, Los Angeles, US

    Todd Schneberk

  3. Department of Clinical Pharmacy Practice, School of Pharmacy & Pharmaceutical Sciences, University of California, Irvine, US

    Yu Ke & Alexandre Chan

  4. Department of Geography, Oklahoma State University, Stillwater, US

    Tao Hu

  5. Department of Clinical Research, Faculty of Infectious and Tropical Diseases, London, School of Hygiene and Tropical Medicine, London, UK

    Dan Wu

  6. Program in Occupational Therapy, Department of Medicine, and Department of Orthopaedic Surgery, Washington University School of Medicine, St. Louis, MO, US

    Chih-Hung Chang

  7. Sol Price School of Public Policy, University of Southern California, Los Angeles, US

    Michael B. Nichol

Authors
  1. Yun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianwei Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Todd Schneberk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alexandre Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jerika Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mary Gutierrez
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ivan Portillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Dan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Chih-Hung Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Yang Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Lawrence Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Michael B. Nichol
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YW contributed literature search, analysis and design of review strategy. YW conducted data analysis and drafted manuscript. All authors contributed to critically revised the work.

Corresponding author

Correspondence to Yun Wang.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

All the authors have no competing interests to declare that are relevant to the content of this articl.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Zheng, J., Schneberk, T. et al. What quantifies good primary care in the United States? A review of algorithms and metrics using real-world data. BMC Prim. Care 24, 130 (2023). https://doi.org/10.1186/s12875-023-02080-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12875-023-02080-y

Keywords

BMC Primary Care

ISSN: 2731-4553

Contact us