Skip to main content
Download PDF

Development of type 2 diabetes mellitus quality indicators in general practice by a modified Delphi method in Beijing, China

BMC Family Practice volume 21, Article number: 146 (2020) Cite this article

Abstract

Background

The service capacity of primary care has improved in China. General practice also takes growing responsibility in the management of type 2 diabetes mellitus, but there are concerns about the paucity of evidence of the quality of care delivered. And there is an absence of systematic quality indicators of type 2 diabetes mellitus in general practice in China. This study aimed to develop a set of type 2 diabetes mellitus quality indicators to facilitate quality measurement in general practice in China.

Methods

Preliminary quality indicators were generated and refined by literature review and an expert consultation meeting. Two rounds of email-based Delphi survey and a consensus meeting were carried out to identify quality indicators. Delphi questionnaires with 43 indicators were sent to 30 participants in the first round. There were 16 general practitioners and 10 community health service center leaders from primary care, 3 endocrinologists and a primary care researcher in the first round. And 27 out of the 30 participants participated in the second round. The consensus meeting was held among 9 participants to refine the indicators and a last round of rating was carried out in the meeting. The indicators were rated in terms of importance and feasibility. The agreement criteria were defined as median ≥ 7.0 and ≥ 85.0% of ratings in the 7–9 tertile for importance; median ≥ 7.0 and ≥ 65.0, 70.0, 75.0% of ratings in the 7–9 tertile for feasibility respectively in the three rounds of rating.

Results

After 2 rounds of Delphi survey and the consensus meeting, total 38 indicators achieved consensus for inclusion in the final set of indicators. The final set of indicators were grouped into 7 domains: access (5 indicators), monitoring (12 indicators), health counseling (7 indicators), records (2 indicators), health status (7 indicators), patient satisfaction (2 indicators) and self-management (3 indicators).

Conclusions

A set of 38 potential quality indicators of type 2 diabetes mellitus in general practice were identified by an iterative Delphi process in Beijing, China. Preliminary approach for measurement and data collection were described. However, the indicators still need to be validated by testing in a further study.

Peer Review reports

Background

General practice is an essential component of primary care providing high-quality care to individuals and families [1, 2]. In order to improve the accessibility and efficiency of the health care system, the government of China decided to strengthen primary care by establishing the community health service (CHS) system in 1997 [3]. Community health service institutions (CHSIs) are the main primary care institutions providing basic medical and public health services in China, which include community health service centers (CHSCs) and community health service stations (CHSSs) [4]. CHSCs usually comprise departments of general practice, traditional Chinese medicine, preventive care, rehabilitation, women health, laboratory tests, and pharmacy [4]. The CHSSs are affiliated institutions of CHSCs to cover areas distant away from CHSCs in the community [4]. CHSSs usually comprise departments of general practice and pharmacy. General practice is responsible for providing basic medical care in CHSIs. In China’s health care reform since 2009 [5], the government increased financial support to primary care institutions from 2.8 billion US dollars in 2008 to 20.3 billion US dollars in 2015 [6, 7]. The number of CHSIs increased from 22,656 in 2006 to 34,997 in 2018 [8]. There were 9352 CHSCs, 25,645 CHSSs and 156,800 registered general practitioners (GPs) in 2018 in China [8]. However, the choice of first contact of care is voluntary for the patients in China. General practices in CHSIs are often bypassed by the patients, who prefer to seek care from secondary and tertiary hospitals with better infrastructure and reputation [9]. The goal of the government is to establish a tiered health care system, in which primary care is the first contact of care collaborating with secondary and tertiary care hospitals [10]. Higher reimbursement rate and lower price of services were provided to incentivize patients to choose CHSIs as the first contact of care [10]. Visits to CHSIs increased from 484,516,000 in 2010 to 799,094,000 in 2018 [8]. Policy to develop general practice was also implemented, i.e. establishment of the nationwide general practice training system since 2011 [11]. GPs will be trained by a 3-year standard residency training program after medical school, with the goal of building up a workforce of 300,000 GPs nationwide till 2020 [11]. China has made remarkable progress in strengthening its primary care system. Nevertheless, the primary care system still faces challenges in workforce, incentive policies and quality of care (QOC) [12].

Chronic disease management is defined as a core task of general practice to tackle the prevalence of chronic diseases at the community level in China [11]. The prevalence of diabetes mellitus was 10.9% in China in 2013 [13]. It was the sixth leading cause of disability-adjusted life years (DALYs) in China in 2016 [14]. General practice has important roles in the management of type 2 diabetes mellitus (T2DM) [15, 16]. There has been a shift from hospital care to primary care in T2DM management in China [17]. General practice is responsible for the screening, long-term management and referral of T2DM patients in the community [18]. High quality care models e.g. patient centered medical home and chronic care model had been proven effective in improving diabetes care [19, 20]. China also launched a “family doctor contract service” model, in which the GPs will sign contracts with patients [21]. The model is intended to let patients have their personal doctors and improve the continuity of care [21]. A team approach is adopted in the model, with a GP, nurse and preventive care physician working together to provide comprehensive care for the patients [22, 23]. Electronic health record (EHR) systems are also developed in CHSIs. Basic modules in the EHR system comprise health profile, medical charts and check-up reports for T2DM management [24]. Innovations in information technologies such as electronic appointment system, distance monitoring and consultation were also piloted in CHSIs with better infrastructure [24, 25]. Despite introduction of the new model, there is little evidence on the quality of T2DM care in general practice in China [12]. One of the causes is the absence of systematic T2DM quality measures for general practice in China [12, 26].

QOC is an important area in primary care which draws international attention [27, 28]. Quality indicators (QIs) can be used to measure the quality of process, outcome and structure of health care [29, 30]. The QOC in general practice in China is mainly assessed by a pay for performance (PFP) scheme which contains 3 QIs on T2DM [31], including: (1) the ratio of T2DM patients being registered in the management of general practice; (2) the ratio of patients being followed up for at least 4 times per year; (3) the ratio of patients whose blood glucose is under control [31]. The GPs will be incentivized if they can achieve a predefined ratio for the QIs [31]. However, the indicators were not sufficient for reflecting the QOC in general practice [32]. Key QIs of clinical care are often unavailable, which impedes the generation of evidence of QOC in general practice [12]. Thus, scientific and feasible QIs are needed for the quality measurement in general practice in China. Many QIs are already available from different countries [33,34,35], however it should be prudent in the use of existing QIs because of the variation in professional culture and the complexity of clinical practice [36]. This study aimed to develop a set of QIs of T2DM in general practice by a modified Delphi method in Beijing to facilitate quality measurement in general practice in China.

Methods

The Delphi process

An iterative Delphi process was conducted to reach consensus on the indicators in this study. This technique was utilized because it had been shown to be an effective method for developing QIs in primary care research [37, 38]. The avoidance of face to face interaction among participants in Delphi survey can prevent individuals feeling intimidated and opinions can be expressed without pressure. However, the absence of face to face discussion prohibits the exchange of different perspectives [37]. Physical meeting can be held when reaching a consensus is difficult or to confirm the results of Delphi survey [39, 40].

The whole Delphi process in this study consisted of: (1) literature review to screen potential indicators; (2) an expert consultation meeting to refine the preliminary indicators; (3) two rounds of email-based Delphi survey; (4) a face to face consensus meeting. The meeting was held because we perceived the necessity to confirm the results of Delphi survey and to refine the description of indicators. The study lasted from October 2018 to March 2019. Please see Fig. 1 for the process of the Delphi study.

Fig. 1
figure 1

The process of the Delphi study. The Delphi process consisted of 2 rounds of survey and a consensus meeting. The list of indicators was modified after each round of rating. The consensus meeting involved discussion and the final rating process. 38 indicators were identified eventually

Full size image

Panel of participants

A list of 75 potential participants was generated from the faculty of general practice of the school and members of general practice academic association who had active roles in general practice training and research in Beijing. The participants were professionals with different roles in T2DM management, including GPs, administrative leaders of CHSCs, endocrinologists in tertiary hospitals and primary care researchers. The participants were chosen based on the following criteria: (1) GP: working as first-line GP for over 5 years, teacher of GP training, experience in research projects and publications; (2) administrative leader of CHSC: working in the administration of CHSC for over 10 years, experience in quality measurement of T2DM care in general practice, teacher of GP training, experience in research projects and publications; (3) endocrinologist: working as endocrinologist for over 10 years, teacher of GP training, experience in research projects and publications; (4) primary care researcher: over 10 years of experience as primary care researcher, familiar with T2DM management, experience in QI development research. The participants were asked for their willingness to take part in the study. Thirty participants from Beijing who met the criteria agreed to take part in the study in the first round. Geographical distribution was also considered in selecting the participants. There are 16 districts in Beijing and the 30 participants covered 8 districts of Beijing.

The response rate in the second round was 90.0% (27/30). Three participants dropped out of the study due to the lack of availability. In the consensus meeting, 10 participants (1/3) were invited to represent the panel based upon their expertise in T2DM management, experience in quality measurement and availability to attend the meeting. Health professionals are promoted within a title system in terms of their work experience and research achievement in China [41]. There are junior grade, middle grade, associate senior grade and senior grade titles. Participants with relatively longer work experience and higher titles (associate senior grade and senior grade titles) were invited in the consensus meeting. Nine (9/10) participants attended the meeting. The primary care researcher who was a professor of university was invited but unavailable for the meeting, so the final indicators were sent to the researcher by email for further comments.

Indicator selection and refinement

A preliminary list of indicators was constructed from three sources by literature review. Firstly, 2 published T2DM clinical guidelines from China and 4 guidelines from International Diabetes Federation (IDF), United States of America (USA), United Kingdom (UK) and Australia were reviewed [15, 18, 42,43,44,45]. Secondly, T2DM QIs produced by key organizations in China, USA, UK and Australia were reviewed [33, 34, 46, 47]. Thirdly, indicators were extracted from 73 published research papers on T2DM quality measurement in general practice which were identified by literature review. Please see Table 1 for the sources of clinical guidelines and indicators from key organizations.

Table 1 Sources of guidelines and indicators from key institutions reviewed
Full size table

Potential indicators were extracted from these sources and screened by a panel of 3 reviewers (JGH, WY, YLL) according to the following criteria: (1) the indicator was relevant to the management of T2DM in general practice; (2) the indicator had explicit standard or recommendation; (3) the indicator was measurable. When there were doubts about whether an indicator should be retained, the research team would discuss together to make a decision. Total 353 indicators were identified by the screening process. Duplicate indicators were deleted to form a preliminary list of 75 indicators. However, there were still indicators difficult to be measured and indicators needed to be integrated because of similar dimensions being measured. Thus, the indicators were discussed in detail one by one in a research team meeting. After further removal and integration, 49 indicators were left, which were categorized preliminarily into the clinical domains of T2DM management [35, 42].

Before constructing the Delphi questionnaire, we conducted a consultation meeting with six experts who had important roles in general practice training, quality measurement, and research in Beijing. There were 2 GPs, 1 director of CHSC and 3 primary care researchers. The GPs and director of CHSC were also teachers of general practice training in CHSCs and were experienced in quality measurement in general practice. The three primary care researchers were academic professors in the university with a lot of experience in general practice research. All the six experts also had active roles in general practice academic associations in China. The goals of the meeting were to: (1) refine the framework and description of the indicators; (2) recommend important indicators from the experts’ perspective; (3) identify indicators that should be further integrated; (4) decide the agreement criteria to be used in the Delphi rating process. After the meeting, a list of 43 indicators was finalized and categorized into nine domains.

Delphi questionnaire

The indicators were developed into a Delphi questionnaire. Importance and feasibility were utilized as two dimensions to rate the indicators on a 1–9 Likert scale. Importance was defined as the extent to which the indicator was considered important for providing high quality T2DM care in general practice. Feasibility was defined as the extent to which the indicator could be feasibly used in the general practice setting. Spaces were left for the participants to make comments to the indicators or recommend new indicators. The materials of Delphi survey were sent to the participants by email including, the research background, instructions, and rating form. In the consensus meeting, rating forms were completed by the participants after discussion. Please see supplement file for the first-round rating form [see Additional file 1].

Delphi survey

In the Delphi survey, the assumption was that most indicators would be considered as important, however the feasibility of indicators might be variable because of the variations in practice infrastructure and the complexity of real life practice. So, it was decided in the expert consultation meeting to set lower agreement level of ratings in feasibility than in importance and raise the agreement level between rounds, in case important indicators were deleted in early rounds. This approach has been adopted in previous Delphi research [38]. Please see Table 2 for the agreement criteria between rounds.

Table 2 Agreement criteria between Delphi rating rounds
Full size table

In the first round, agreement criteria were defined as median ≥ 7.0 and ≥ 85.0% of ratings in the 7–9 tertile for importance and ≥ 65.0% of ratings in the 7–9 tertile for feasibility. Indicators recommended to be deleted by more than 3 participants would also be removed. The indicators were clarified and modified based on the rating results and feedback from the participants in the first round. The modified indicators were sent to the participants again for the second-round rating. The modifications and ratings in the first round were elaborated in a document with the second-round questionnaire. The participants were able to see their responses as well as the responses of the other participants without knowing each other’s identity. The agreement criteria were defined as median ≥ 7.0 and ≥ 85.0% of ratings in the 7–9 tertile for importance and ≥ 70.0% of ratings in the 7–9 tertile for feasibility in the second round.

In the consensus meeting, we presented the results of the first and second rounds of Delphi survey. The indicators deleted in the first and second rounds were discussed in detail again. The description of each indicator as well as the assumed approach for measurement and data collection were also clarified in the meeting. The remaining indicators and the deleted ones were then rated by the participants for the last time to confirm the results. The agreement criteria were defined as median ≥ 7.0 and ≥ 85.0% of ratings in the 7–9 tertile for importance and ≥ 75.0% of ratings in the 7–9 tertile for feasibility.

Results

Demographics of the participants in Delphi survey

The participants were from 8 districts in Beijing in the first and second rounds and 4 districts in the consensus meeting. There were more female participants in both rounds (70.0 and 66.7%) and the consensus meeting (77.8%). Over 50.0% of the participants were between 40 ~ 49 years old in both rounds (50.0 and 51.9%) and the consensus meeting (66.7%). Over half of the participants were GPs in both rounds (53.4 and 55.6%) and the consensus meeting (55.6%). Most of the participants had been working for 11 ~ 20 years in both rounds (56.7 and 59.3%) and the consensus meeting (77.8%). In the first and second rounds, 46.7 and 48.2% of the participants had master’s degree. And 55.6% of the participants had master’s degree in the consensus meeting. In the first and second rounds, 33.3 and 29.6% of the participants were with senior grade title. In the consensus meeting, 55.6% of the participants were with senior grade title. Please see Table 3.

Table 3 Demographics of the participants
Full size table

Delphi survey and consensus meeting

In the first round, all the 43 indicators achieved 85.0% agreement except one indicator “admission days” (83.3%) in terms of importance. In terms of feasibility, one indicator “ankle-brachial index monitoring”(63.3%)failed to achieve 65.0% agreement. Four indicators were recommended to be removed by more than 3 participants, despite the agreement they had achieved. So, 6 indicators were deleted in the first round. Four indicators were integrated as two indicators, and 2 new indicators were added. There were 37 indicators left after the first round.

In the second round, all the 37 indicators achieved 85.0% agreement in terms of importance. In terms of feasibility, two indicators, including “psychological counseling” (59.3%) and “quality of life” (66.7%) failed to achieve 70.0% agreement and were deleted. There were 35 indicators left after the second round.

In the third round, 43 indicators including the deleted ones were rated after discussion. And 42 indicators achieved 85.0% agreement for importance. In terms of feasibility, 38 indicators achieved 75.0% agreement. Five indicators still failed to achieve agreement. Three indicators deleted in the first 2 rounds of Delphi survey were resurrected and modified in the consensus meeting. They were “psychological assessment or counseling”, “hypoglycemia episodes” and “T2DM related admissions to hospitals”. Please see Table 4 for the distribution of indicators by agreement level and Table 5 for the five deleted indicators. Modifications on the list of indicators in each round were presented in Fig. 1. The results of ratings in the Delphi survey and the consensus meeting were provided in an additional file [see Additional file 2].

Table 4 Distribution of indicators by agreement level
Full size table
Table 5 Five deleted indicators and ratings in the consensus meeting
Full size table

The final set of indicators

After 2 rounds of Delphi survey and the consensus meeting, a total of 38 indicators achieved consensus for inclusion in the final set of indicators. The 38 indicators were grouped into 7 domains: access (5 indicators), monitoring (12 indicators), health counseling (7 indicators), records (2 indicators), health status (7 indicators), patient satisfaction (2 indicators) and self-management (3 indicators). Potential data sources and measurement were also recommended by the panel of participants. Please see Table 6 for the final set of indicators.

Table 6 Description of the final set of indicators
Full size table

Discussion

QOC measurement of T2DM in general practice has been widely carried out in many countries both in national programs and individual studies [33,34,35]. The Quality and Outcomes Framework (QOF) is an extensive PFP program in general practice in the UK and there were 11 QIs of T2DM in the QOF (2018/2019) [33]. A systematic review showed the number of diabetes QIs in primary care research varied widely from 3 to 57, with a median of 14 indicators [35]. Despite extensive international experiences in QOC measurement of T2DM in general practice, little evidence on the QOC of T2DM in general practice is available in China [12, 26]. The current 3 QIs of T2DM in the government-funded PFP program in general practice has been used for nearly a decade [31], however important indicators related to risk factors such as glycosylated hemoglobin (HbA1c) and low-density lipoprotein cholesterol (LDL-C) are still missing. The EHR system was also under-utilized for quality measurement in the PFP program [12]. A systematic review in Asia and the Middle East showed the paucity of research from these areas on the quality of diabetes care in primary care [26]. The absence of systematic QIs might be one of the causes for the lack of QOC studies in general practice in China [12, 26].

Unlike recommendations in clinical guidelines, feasibility is a critical prerequisite for QIs [37, 48]. The translation of recommendations into operationalizable QIs in general practice requires rigorous process [35, 37]. Thus, the aim of this study was to identify indicators that were both important and feasible in the general practice context in China. A set of 38 potential indicators to reflect the QOC of T2DM in general practice were identified in this study based on an iterative process including literature review, consultation meeting, Delphi survey and consensus meeting. Seven domains of indicators were identified, including access, monitoring, health counseling, records, health status, patient satisfaction and self-management. Most indicators in monitoring, health counseling and health status were process and intermediate outcome indicators derived from clinical guidelines which were frequently used in previous T2DM quality measurement studies [35, 49]. Individualized targets had been recommended for intermediate outcome indicators including HbA1c, blood pressure (BP), LDL-C in both Chinese and international clinical guidelines [42, 45], however using individualized targets in quality measurement is still a complex problem to be solved [50]. Considering the feasibility of data collection and the recommended values in Chinese clinical guidelines [18, 42], we used different HbA1c, BP, LDL-C targets for different patient groups in this study instead of setting one value for all patient groups. Patient safety is an important aspect of T2DM care, which is emphasized by recent clinical guidelines [45]. However, few indicators related to patient safety are available in most QIs used previously [35, 51]. We made an attempt to identify indicators in this domain on the basis of literature review and the participants’ consensus. Three indicators including “hypoglycemia awareness counseling”, “medication safety counseling”, and “emergency help counseling” were identified as important patient safety indicators in the health counseling domain. And “hypoglycemia episodes” and “T2DM related admissions to hospital” in the health status domain were identified. These indicators have not been frequently used in previous studies [35, 51], however the participants agreed on the practical importance of the indicators in real life practice.

High quality T2DM care has shifted to a patient centered approach [15, 45]. There were calls for patient centered indicators as well [50, 52]. We tried to incorporate indicators to measure domains of patient centered care, i.e. access of care, self-management and patient satisfaction (patient experience) [52, 53]. Access of care is a fundamental feature of general practice [1, 53], however it is difficult to translate the concept into measurable indicators. The Primary Care Assessment Tool (PCAT) had been used for assessing dimensions of access in primary care in China [54]. In this study, we identified 5 indicators based on the access dimensions of PCAT and the participants’ opinions. There were having a personal doctor for T2DM patients, patients’ perception of a functional GP team, waiting time, access to seeking health advice, and referral access to necessary specialist care. These indicators were also relevant to the essential medical services provided in general practice of CHSIs in China [21]. Patient self-management is critical for the outcome of T2DM care and is proposed as a QI to assess how patients are doing in T2DM management [50]. We identified 3 self-management indicators, i.e. knowledge of self-management, adherence to medication and adherence to healthy behavior in this study. Quality measures in adherence may facilitate new efforts to improve adherence and patient outcomes [50], however controversy exists in using adherence indicators due to the difficulty in measurement. There are scales in Chinese version with proved validity and reliability to measure T2DM health literacy and adherence [55,56,57]. We proposed the Michigan Diabetes Knowledge Test (MDKT) for self-management knowledge [58], Morisky 8 Item Scale for medication adherence [59], and the Summary of Diabetes Self-Care Activities Measure (SDSCA) for healthy behavior adherence as preliminary tools for the measurement of the indicators [60]. However, the feasibility of this approach needs to be verified by testing the indicators. Patient satisfaction had been used as an indicator in previous T2DM quality measurement studies [61]. The patient’s satisfaction with treatment and communication were identified as patient experience indicators in this study. And the Diabetes Treatment Satisfaction Questionnaire (DTSQ) was proposed for measuring the patient’s satisfaction with treatment [61]. Five indicators failed to meet the agreement criteria, because the indicators were difficult to be defined or measured in the current general practice context of China. For example, the participants agreed that “rational use of medicines” was important, however it was difficult to achieve consensus on how the indicator could be properly defined or feasibly assessed via the current EHR system in China.

The indicators in this study were recommended to be measured on the accumulation of individual patient data by two methods: EHR review and patient survey. Indicators in monitoring, records and health status were recommended to be measured via the EHR system. EHR systems have been developed in CHSIs in the health care reform in China [21]. The growing use of EHR systems in CHSIs provides an opportunity to assess the comprehensive QOC of T2DM in general practice [25, 62]. However, there are still problems in the current EHR systems in CHSIs. Because different systems are being used among CHSIs [25, 62], the modules of clinical quality information among different systems might be fragmented, which would increase the difficulty of data extraction. Partial clinical indicators might even be unavailable in CHSIs with less-developed EHR systems [12]. A widely used and accurate EHR system for data collection needs to be developed for general practice in China. The clinical QIs can be embedded into the EHR system to improve the quality of clinical information, which might also be helpful for the decision making based on systematic and timely clinical information [12, 63]. Nurses in the GP team also have growing roles in terms of health education and routine data collection of T2DM follow up in CHSIs [25]. This might also facilitate the data extraction for quality measurement from the EHR system. Indicators in access, health counseling, patient satisfaction and self-management were recommended to be measured by patient survey. Self-report measures have been found to be feasible QIs in primary care settings [52]. Information technologies such as web-based survey methods can improve the efficiency of patient survey [64].

This Delphi study provided a basis for the QOC measurement of T2DM in general practice in China. However, the indicators still need to be tested in real life practice in a further study [37, 48]. Critical aspects need to be assessed in testing the indicators i.e. data availability in the current EHR system in different CHSIs, accuracy of data, work load of data collection, reliability and reproducibility of data extraction, acceptability of the indicators for GPs being measured and stakeholders in CHSIs, and responsiveness to the change of QOC of T2DM [37, 48]. Composite QIs had been widely adopted in quality measurement [65, 66]. Approaches can be used to combine indicators to assess the overall QOC of T2DM, which enables comparison or ranking among general practices [67]. However, there are problems in using composite indicators, e.g. availability of data for the indicators among different CHSIs, bias caused by missing values, appropriate weight for the indicators and uncertainty in the final score [50, 68]. Thus, it should be prudent to use the set of indicators as composite indicators. Scoring strategies should use weights based on clinical importance and balance the weights to the dynamic changes of QOC in general practice [50, 68]. Detailed individual indicator data should be provided for the clear instruction on data collection, missing data processing and the scoring protocol. Case-mix adjustments for the indicator might be performed with detailed patient level information [50, 68].

Limitation of the study

There are limitations of this study. The study was conducted in Beijing, all the participants were health professionals from Beijing. So, the application of the indicators in other regions should be based on the local health policy and available health resources. However, CHSIs are set up in accordance with standard specifications from the government [4], and over half of the participants were GPs and administrative leaders from CHSIs, which may improve the feasibility of the indicators in general practice setting. QOC in T2DM could encompass very broad dimensions. Some aspects of T2DM care might not be reflected in the indicators of this study, however, we attempted to identify the indicators considered both important and feasible in the current general practice context in China. Patients have more and more important roles in the development of QIs [52], but no T2DM patient was involved in this study. This might impede the opportunity to identify important indicators from the patient’s perspective. Further study should be done to explore the patient’s perspective on the QOC of T2DM in general practice in China.

Conclusions

In summary, we identified a set of 38 potential QIs on T2DM care in general practice by an iterative Delphi process in Beijing, China. Preliminary approach for measurement and data collection of the QIs were recommended. However, the indicators still need to be validated by testing in real life practice in a further study.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author (xq6518@163.com) on reasonable request.

Abbreviations

CHS:

Community health service

CHSI:

Community health service institution

CHSC:

Community health service center

CHSS:

Community health service station

GP:

General practitioner

QOC:

Quality of care

DALYs:

Disability-adjusted life years

T2DM:

Type 2 diabetes mellitus

EHR:

Electronic health record

QI:

Quality indicator

PFP:

Pay for performance

IDF:

International diabetes federation

USA:

United States of America

UK:

United Kingdom

HbA1c:

Glycosylated hemoglobin

BP:

Blood pressure

TC:

Total cholesterol

TG:

Triglyceride

LDL-C:

Low density lipoprotein cholesterol

HDL-C:

High density lipoprotein cholesterol

BMI:

Body mass index

ECG:

Electrocardiogram

CHD:

Coronary heart disease

ASCVD:

Arteriosclerotic cardiovascular disease

DTSQ:

Diabetes treatment satisfaction questionnaire

MDKT:

Michigan diabetes knowledge test

SDSCA:

Summary of diabetes self-care activities measure

QOF:

Quality and Outcomes Framework

References

  1. Kidd M. The contribution of family medicine to improving health systems: a guidebook from the World Organization of Family Doctors. 2nd ed. London: Radcliffe Publishing Ltd; 2013.

  2. Kringos DS, Boerma W, Hutchinson A, Der Zee JV, Groenewegen PP. The breadth of primary care: a systematic literature review of its core dimensions. BMC Health Serv Res. 2010;10(1):65.

    PubMed  PubMed Central  Google Scholar 

  3. The State Council of China. The decision on health care reform and development [in Chinese]. 1997. http://www.nhc.gov.cn/wjw/zcjd/201304/743ba60a223646cd9eb4441b6d5d29fa.shtml. Accessed 1 May 2020.

  4. Ministry of Health of China. Regulations on community health service institutions [in Chinese]. 2006. http://www.gov.cn/zwgk/2006-08/10/content_359147.htm. Accessed 1 May 2020.

  5. The State Council of China. Guidelines on deepening the health care reform [in Chinese]. 2009. http://www.gov.cn/gongbao/content/2009/content_1284372.htm. Accessed 1 May 2020.

  6. Ministry of Health of China. China health statistical yearbook 2009 [in Chinese]. Beijing: Peking Union Medical College Publishing House; 2010.

  7. National Health and Family Planning Commission of China. China health and family planning statistical yearbook 2016 [in Chinese]. Beijing: Peking Union Medical College Publishing House; 2017.

  8. National Health Commission of China. China health statistical yearbook 2019 [in Chinese]. Beijing: Peking Union Medical College Publishing House; 2019.

  9. Shao S, Zhao FF, Wang J, Feng L, Lu XQ, Du J, et al. The ecology of medical care in Beijing. Plos One. 2013;8(12): e82446.

  10. The State Council of China. Guidelines on facilitating the establishment of tiered health care system [in Chinese]. 2015. http://www.gov.cn/zhengce/content/2015-09/11/content_10158.htm. Accessed 1 May 2020.

  11. The State Council of China. Guidelines on establishing general practice system. [in Chinese] 2011. http://www.gov.cn/zwgk/2011-07/07/content_1901099.htm. Accessed 1 May 2020.

  12. Li X, Lu J, Hu S, Cheng KK, De Maeseneer J, Meng Q, et al. The primary health-care system in China. Lancet. 2017;390(10112):2584–94.

    PubMed  Google Scholar 

  13. Wang L, Gao P, Zhang M, Huang Z, Zhang D, Deng Q, et al. Prevalence and ethnic pattern of diabetes and prediabetes in China in 2013. JAMA. 2017;317(24):2515–23.

    PubMed  PubMed Central  Google Scholar 

  14. Liu M, Liu S, Wang L, Bai YM, Zeng X, Guo HB, et al. Burden of diabetes, hyperglycaemia in China from to 2016: findings from the 1990 to 2016, global burden of disease study. Diabetes Metab. 2019;45(3):286–93.

    CAS  PubMed  Google Scholar 

  15. Royal Australian College of General Practitioners. General practice management of type 2 diabetes 2016–18. 2016. https://www.racgp.org.au/clinical-resources/clinical-guidelines/key-racgp-guidelines/management-of-type-2-diabetes. Accessed 1 May 2020.

  16. John GC, Tor C, Anne KJ, Geir T, Marie FH, Wibeche I, et al. Quality of care for patients with type 2 diabetes in primary care in Norway is improving: results of cross-sectional surveys of 33 general practices in 1995 and 2005. Diabetes Care. 2009;32(1):81–3.

    Google Scholar 

  17. The State Council of China. Long term plan (2017–2025) for prevention and treatment of chronic diseases in China. 2017. http://www.gov.cn/xinwen/2017-02/14/content_5167942.htm. Accessed 1 May 2020.

  18. Chinese Diabetes Mellitus Association. National guidelines for the prevention and control of diabetes in primary care (2018) [in Chinese]. Chin J Intern Med. 2018;57(12):885–93.

    Google Scholar 

  19. Friedberg MW, Rosenthal MB, Werner RM, Volpp KG, Schneider EC. Effects of a medical home and shared savings intervention on quality and utilization of care. JAMA Intern Med. 2015;175(8):1362–8.

    PubMed  PubMed Central  Google Scholar 

  20. Stellefson M, Dipnarine K, Stopka C. The chronic care model and diabetes management in US primary care settings: a systematic review. Prev Chronic Dis. 2013;10:E26.

    PubMed  PubMed Central  Google Scholar 

  21. National Health and Family Planning Commission of China. Guidelines on facilitating the family doctor contract service [in Chinese]. 2016. http://www.gov.cn/xinwen/2016-06/06/content_5079984.htm. Accessed 1 May 2020.

  22. Wei XJ, Wu H, Yu HY, Cui SQ, Ge CY, Jia HY, et al. Role of family doctor service team in community-based integrated management of diabetes [in Chinese]. Chi Gen Prac. 2017;20(27):3349–52.

    Google Scholar 

  23. Chen SR, Wang L. Evaluation of the effect of family doctor responsibility contract service on community type 2 diabetes mellitus intervention [in Chinese]. Chi Prim Heal Care. 2018;32(9):30–2.

    Google Scholar 

  24. Wei XJ, Wu H, Yu HY, Ge CY, Jia HY, Wang L, et al. Practical effect of family doctor service on promoting hierarchical diagnosis and treatment in Fangzhuang community [in Chinese]. Chi Gen Prac. 2017;20(13):1558–63.

    Google Scholar 

  25. Wang Q, Li AL, Yi M, Dong T, Li QX, Liu XL, et al. Application of diabetes management module in electronic contracted family doctor services management platform:a survey in Wuhou District [in Chinese]. Chi Gen Prac. 2019;12(33):4105–09,15.

    Google Scholar 

  26. Shivashankar R, Kirk K, Kim WC, Rouse C, Tandon N, Narayan KM, et al. Quality of diabetes care in low- and middle-income Asian and middle eastern countries (1993-2012): 20-year systematic review. Diabetes Res Clin Pract. 2015;107(2):203–23.

    PubMed  Google Scholar 

  27. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: The National Academies Press; 2001.

    Google Scholar 

  28. World Organization of Family Doctors. Statement of principles of the WONCA Working Party on Quality and Safety in Family Medicine. 2013. https://www.globalfamilydoctor.com/groups/WorkingParties/QualitySafety.aspx. Accessed 1 May 2020.

  29. Nicolucci A, Greenfield S, Mattke S. Selecting indicators for the quality of diabetes care at the health systems level in OECD countries. Int J Qual Health Care. 2006:26–30.

  30. American Academy of Family Physicians. What are the types of measures?. 2020. https://www.aafp.org/practice-management/improvement/measures.html. Accessed 1 May 2020.

  31. Ministry of Health of China. Pay for performance in community health service institutions [in Chinese]. 2011. http://www.gov.cn/gzdt/2011-06/30/content_1896720.htm. Accessed 1 May 2020.

  32. Peng YC, Su N, Chen Q, He YJ, Liang WN. Evaluation of current performance appraisal system by community health service staff [in Chinese]. Chi Gen Prac. 2011;14(2a):361–3.

    Google Scholar 

  33. National Health Service. Quality and outcomes framework 2018/19. 2019. https://digital.nhs.uk/data-and-information/publications/statistical/quality-and-outcomes-framework-achievement-prevalence-and-exceptions-data/2018-19-pas. Accessed 1 May 2020.

  34. Centers for Medicare & Medicaid Services. Primary care measures. 2016. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/Core-Measures.html. Accessed 1 May 2020.

  35. Calsbeek H, Ketelaar NA, Faber MJ, Wensing M, Braspenning JC. Performance measurements in diabetes care: the complex task of selecting quality indicators. Int J Qual Health Care. 2013;25(6):704–9.

    PubMed  Google Scholar 

  36. Marshall M, Shekelle PG, Mcglynn EA, Campbell S, Brook RH, Roland M. Can health care quality indicators be transferred between countries. Qual Saf Health Care. 2003;12(1):8–12.

    CAS  PubMed  PubMed Central  Google Scholar 

  37. Campbell S, Braspenning JC, Hutchinson A, Marshall M. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. 2002;11(4):358–64.

    CAS  PubMed  PubMed Central  Google Scholar 

  38. Shield T, Campbell S, Rogers A, Worrall A, Chewgraham C, Gask L. Quality indicators for primary care mental health services. Qual Saf Health Care. 2003;12(2):100–6.

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. Plos One. 2011;6(6):e20476.

  40. Albarqouni L, Hoffmann T, Straus S, Olsen N, Young T, Ilic D, et al. Core competencies in evidence-based practice for health professionals: consensus statement based on a systematic review and Delphi survey. JAMA Netw Open. 2018;1(2):e180281.

    PubMed  Google Scholar 

  41. Ministry of Health of China. Guidelines on building up urban community health service workforce [in Chinese]. 2006. http://www.gov.cn/zwgk/2006-08/10/content_359177.htm. Accessed 1 May 2020.

  42. Chinese Diabetes Mellitus Association. Guidelines on the prevention and treatment of type 2 diabetes mellitus in China 2017 [in Chinese]. Chin J Diabetes Mellitus. 2018;10(1):4–66.

    Google Scholar 

  43. International Diabetes Federation. Global guideline for type 2 diabetes. 2012. https://www.idf.org/e-library/guidelines/79-global-guideline-for-type-2-diabetes.html. Accessed 1 May 2020.

  44. National Institute for Health and Care Excellence. Type 2 diabetes in adults: management. 2015. https://www.nice.org.uk/guidance/ng28. Accessed 1 May 2020.

  45. American Diabetes Association. Standards of medical care in diabetes-2018. Diabetes Care. 2018;41(Suppl 1):S1–153.

    Google Scholar 

  46. Community Health Association of China. Community health service quality standards [in Chinese]. 2016. http://www.chs.org.cn/news/show.php?itemid=60. Accessed 1 May 2020.

  47. The Royal Australian College of General Practitioners. Clinical indicators for Australian general practice. 2015. https://www.racgp.org.au/running-a-practice/practice-management/general-practice-governance/clinical-indicators. Accessed 1 May 2020.

  48. Campbell S, Kontopantelis E, Hannon K, Burke MJ, Barber A, Lester H. Framework and indicator testing protocol for developing and piloting quality indicators for the UK quality and outcomes framework. BMC Fam Pract. 2011;12(1):85.

    PubMed  PubMed Central  Google Scholar 

  49. Seddon ME, Marshall M, Campbell S, Roland M. Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand. BMJ Qual Saf. 2001;10(3):152–8.

    CAS  Google Scholar 

  50. Oconnor PJ, Bodkin NL, Fradkin JE, Glasgow RE, Greenfield S, Gregg EW, et al. Diabetes performance measures: current status and future directions. Diabetes Care. 2011;34(7):1651–9.

    Google Scholar 

  51. Rodriguezgutierrez R, Lipska KJ, Mccoy RG, Ospina NS, Ting HH, Montori VM. Hypoglycemia as an indicator of good diabetes care. BMJ. 2016;352:i1084.

    Google Scholar 

  52. Russell EG, Malinda P, Soren ES. Where is the patient in diabetes performance measures? Diabetes Care. 2008;34(7):1651–9.

    Google Scholar 

  53. Campbell S, Roland M, Buetow S. Defining quality of care. Soc Sci Med. 2000;51(11):1611–25.

    CAS  PubMed  Google Scholar 

  54. Yang H, Shi L, Lebrun LA, Zhou X, Liu J, Wang H. Development of the Chinese primary care assessment tool: data quality and measurement properties. Int J Qual Health Care. 2013;25(1):92–105.

    PubMed  Google Scholar 

  55. Wang J, Mo YZ, Bian RW. Evaluation of reliability and validity of application of the Chinese version of 8-item Morisky medication adherence scale in patients with type 2 diabetes [in Chinese]. Chin J Diabetes. 2013;21(12):1101–4.

    CAS  Google Scholar 

  56. Zhang XX, Wu SY, Wang FS, Mayinuer YSF, Sun KG, Hu K, et al. Association between social support and self-management behaviors among patients with diabetes in community [in Chinese]. J Peking Univ. 2017;49(3):455–61.

    Google Scholar 

  57. Fu AD, Lu GM, Yang J, Liu Q. Application and effect evaluation of clinical pathways for health education in diabetic patients [in Chinese]. Chi Gen Prac. 2011;14(28):3268–70.

    Google Scholar 

  58. Fitzgerald JT, Funnell MM, Anderson RM, Nwankwo R, Stansfield RB, Piatt GA. Validation of the revised brief diabetes knowledge test (DKT2). Diabetes Educ. 2016;42(2):178–87.

    PubMed  Google Scholar 

  59. Clifford S, Pereznieves M, Skalicky A, Reaney M, Coyne KS. A systematic literature review of methodologies used to assess medication adherence in patients with diabetes. Curr Med Res Opin. 2014;30(6):1071–85.

    CAS  PubMed  Google Scholar 

  60. Toobert DJ, Hampson SE, Glasgow RE. The summary of diabetes self-care activities measure: results from 7 studies and a revised scale. Diabetes Care. 2000;23(7):943–50.

    CAS  PubMed  Google Scholar 

  61. Saisho Y. Use of diabetes treatment satisfaction questionnaire in diabetes care: importance of patient-reported outcomes. Int J Environ Res Public Health. 2018;15:947.

    PubMed Central  Google Scholar 

  62. Yue Z, Quan C, Xin Y, Juan L, Hongpu H. Application status of information system functions in community health service settings and township health centers in China [in Chinese]. Chi Gen Prac. 2016;19(7):766–70.

    Google Scholar 

  63. Brown JS, Holmes JH, Shah K, Hall K, Lazarus R, Platt R. Distributed health data networks: a practical and preferred approach to multi-institutional evaluations of comparative effectiveness, safety, and quality of care. Med Care. 2010;48(Suppl):S45–51.

    PubMed  Google Scholar 

  64. Wu LY, Xie JF. Survey on knowledge, attitude and behavior in patients with chronic kidney disease [in Chinese]. Chin J Gen Pract. 2019;18(8):737–41.

    Google Scholar 

  65. Kaplan SH, Griffith JL, Price LL, Pawlson LG, Greenfield S. Improving the reliability of physician performance assessment: identifying the “physician effect” on quality and creating composite measures. Med Care. 2009;47:378–87.

    PubMed  Google Scholar 

  66. Scholle SH, Roski J, Adams JL, Dunn DL, Kerr EA, Dugan DP, et al. Benchmarking physician performance: reliability of individual and composite measures. Am J Manag Care. 2008;14:833–8.

    PubMed  PubMed Central  Google Scholar 

  67. Lovaglio PG. Benchmarking strategies for measuring the quality of healthcare: problems and prospects. Sci World J. 2012;606154.

  68. Barclay M, Dixonwoods M, Lyratzopoulos G. The problem with composite indicators. BMJ Qual Saf. 2019;28(4):338–44.

    PubMed  Google Scholar 

Download references

Acknowledgements

The authors want to acknowledge Dr. Hongyan Jia, Dr. Chunze Yan and Dr. Yue Jiang for their professional advice for this study.

Funding

This work was supported by the Beijing Key Researcher Development Funding Program (grant number: 2017000020124G286). The funder has no role in the design, data collection, analysis, and interpretation of this study or in writing the manuscript.

Author information

Authors and Affiliations

  1. Department of General Practice, School of General Practice and Continuing Education, Capital Medical University, Beijing, People’s Republic of China

    Guanghui Jin, Yun Wei, Feiyue Wang, Meirong Wang, Yali Zhao, Juan Du, Shuqi Cui & Xiaoqin Lu

  2. Department of General Practice, Beijing Tiantan Hospital, Capital Medical University, Beijing, People’s Republic of China

    Yanli Liu

Authors
  1. Guanghui Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yun Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanli Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feiyue Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Meirong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yali Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Juan Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shuqi Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Xiaoqin Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

GHJ and XQL jointly conceived and designed the study. GHJ, YW, YLL, FYW, MRW were responsible for the indicator generation and Delphi survey process and statistical analyses. GHJ and YLZ prepared the manuscript. XQL, JD, SQC revised the paper. All the authors had read and approved the final manuscript.

Corresponding author

Correspondence to Xiaoqin Lu.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Ethical Committee of Capital Medical University, Beijing, China. Written informed consents were achieved from all the participants in this study. The participants were assured that their information would only be used for analysis in this study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Rating form in the first round-translation

Additional file 2.

Results of ratings in the Delphi survey and consensus meeting

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jin, G., Wei, Y., Liu, Y. et al. Development of type 2 diabetes mellitus quality indicators in general practice by a modified Delphi method in Beijing, China. BMC Fam Pract 21, 146 (2020). https://doi.org/10.1186/s12875-020-01215-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12875-020-01215-9

Keywords

BMC Primary Care

ISSN: 2731-4553

Contact us