Skip to main content

Specialist wait time reporting using family physicians’ electronic medical record data: a mixed method study of feasibility and clinical utility

Abstract

Background

More than 50% of Canadian adult patients wait longer than four weeks to see a specialist after referral from primary care. Access to accurate wait time information may help primary care physicians choose the timeliest specialist to address a patient’s specific needs. We conducted a mixed-methods study to assess if primary to specialist care wait times can be extracted from electronic medical records (EMR), analyzed the wait time information, and used focus groups and interviews to assess the potential clinical utility of the wait time information.

Methods

Two family practices were recruited to examine primary care physician to specialist wait times between January 2016 and December 2017, using EMR data. The primary outcome was the median wait time from physician referral to specialist appointment for each specialty service. Secondary outcomes included the physician and patient characteristics associated with wait times as well as qualitative analyses of physician interviews about the resulting wait time reports.

Results

Wait time data can be extracted from the primary care EMR and converted to a report format for family physicians and specialists to review. After data cleaning, there were 7141 referrals included from 4967 unique patients. The 5 most common specialties referred to were Dermatology, Gastroenterology, Ear Nose and Throat, Obstetrics and Gynecology and Urology. Half of the patients were seen by a specialist within 42 days, 75% seen within 80 days and all patients within 760 days. There were significant differences in wait times by specialty, for younger patients, and those with urgently labelled medical situations. Overall, wait time reports were perceived by clinicians to be important since they could help family physicians decide how to triage referrals and might lead to system improvements.

Conclusions

Wait time information from primary to specialist care can aid in decision-making around specialist referrals, identify bottlenecks, and help with system planning. This mixed method study is a starting point to review the importance of providing wait time data for both family physicians, specialists and local health systems. Future work can be directed towards developing wait time reporting functionality and evaluating if wait time information will help increase system efficiency and/or improve provider and patient satisfaction.

Peer Review reports

Introduction

Canada has the longest wait times to see a specialist physician according to a recent Commonwealth Fund survey of eleven countries [1]. More than 50% of adult patients wait longer than four weeks to see a specialist, compared to the international average of 36% [1]. Median wait times to see specialists in Ontario range from 39 to 76 days for medical specialists and 33 to 66 days for surgical specialists [2]. Wait times from primary to specialty care in Canada have been increasing and are 155% longer than in 1993 [3]. Prolonged waiting for specialists may be associated with adverse consequences such as increased pain and suffering, inferior medical outcomes, and economic costs from lost wages and productivity [4, 5].

In 2007, Willcox et al. conducted a cross-national comparison of strategies used by countries to measure and reduce wait times [6]. Five countries used supply-side strategies, such as targeting funding toward increased hospital capacity and staff. Some countries used more complex initiatives that addressed demand side techniques, such as using explicit criteria to prioritize access to surgery. The authors recommended that policymakers consider extending the measurement of waiting times to include the point of referral to treatment in order to reflect patients’ actual experience. To date, there are few benchmarks in place targeting wait times from primary to specialty care [5, 7]. In addition, there are limited interventions found in the literature that have shown impact on reducing wait times from primary care to specialty care. Jaakkimainen et al. published a study that outlines patient and provider characteristics of wait times from primary to specialty care [2]. They found that patient factors and most physician factors do not seem to be consistently associated with wait times, except for family physician practice location and practice size.

In Canada, most referrals are directed to specific specialist physicians by primary care providers. In some cases, referrals are made to specialist clinics and the specialist is then selected for the referral by the specialty clinic. Specialist visits are covered by Canada’s universal health insurance plan, and primary care providers can refer patients to any specialist of their choosing. Primary care providers typically select the specialist, and fax or email a referral letter to the specialist’s office using a paper-based system or an electronic health record of their choosing. Historically, there has been little information available to primary care physicians about which specialists have long waits or are available in different regions. Physicians rely on an ad hoc system based on personal experience, word of mouth, electronic medical record (EMR) or internet searches to identify specialists. At times, physicians may refer a patient and find out months later the referral was never received or additional information is needed [8]. Neimanis et al. analyzed 770 referrals and found that 36.4% of referrals received no response within a 5 to 7 week period [8]. Access to accurate wait time information may help family physicians choose the most suitable specialist to address a patient’s specific needs.

Objective

This mixed methods study assessed the feasibility of creating primary to specialist care wait reports from primary care electronic medical records (EMRs). The primary outcome was the median wait time from physician referral to specialist appointment for each specialty service. The clinical utility of the resulting wait time data was evaluated through focus groups with family physicians and specialist physicians [9].

Methods

Study setting

A convenience sample of two primary care clinics from the University of Toronto Practice-based Research Network (UTOPIAN) [10] were recruited for this study: an academic family health team in downtown Toronto, Ontario (caring for approximately 12,000 patients of all ages), and a community primary care clinic in Vaughan, Ontario (caring for approximately 10,000 patients of all ages). Both are affiliated with Sinai Health and utilized the same EMR (Nightingale), at the time of the study. We included both sites to study community and downtown academic family practice perspectives and to include data from diverse referral networks. Ethics approval was obtained from the Sinai Health Research Ethics Board.

Data collection

Patients from the two study sites were included if they had one or more referral records between January 1st, 2016 and December 31st, 2017, and if they had at least 2 visits in 3 years prior to December 31st, 2017. Only patients who were not deceased by the end of the study period were included. Age was calculated at January 1st 2016. The unit of analysis is a “visit”, where multiple visits can be nested for a patient. For example, one patient might have referrals to both cardiology and endocrinology, possibly over different visits during the study window. The data set was created by entering the back end of the electronic health record and extracting fields from patient referral. Variables in the final dataset included: family practice location, age, sex, urgency of visit coded to 4 categories (as per availability, within 1 month, within 1 week and urgent), specialist name, specialty, specialist department or clinic, specialist address, referral date, referral status, patient postal code and appointment date. Wait times were defined as the number of days between referral date and appointment date. Routine referrals were considered those indicated by the referring physician “as per availability” and “within 1 month”. Urgent referrals included those marked “urgent” and “within 1 week”. Referral data was further cleaned by manual review by the lead study author (MN) to remove referrals with an unknown site, missing department/speciality or to a non-medical or surgical specialty (for example, referrals to physiotherapy or optometry) and referrals where the wait time was missing. Where referrals to a specific specialty were fewer than 50 patients, specialities were grouped into an “other” category. Wait times were calculated by taking the appointment date and subtracting the number of days from the referral date. To validate the accuracy and completeness of the wait times calculated, a random sample of 150 referrals was selected and a research assistant reviewed the charts to assess the agreement between wait times calculated through EMR data that was extracted, cleaned and coded, and wait times calculated through manual chart review (the gold standard).

Creation of wait time reports

The format of the wait time report was developed with input from the study investigators. A sample report is included in Appendix 1 and has been anonymized for purposes of publication.

Statistical analysis

Quantitative analysis

Wait times did not have a normal distribution therefore the analysis included the calculation of median wait times and inter-quartile ranges [2]. Median wait times were calculated and stratified by specialist/department and by practice location. Exploratory, bivariate analyses were conducted examining the median wait time including completeness of wait time data and patient or physician characteristics that may have contributed to wait times such as patient age or sex, patient income level before taxes [11], combined material and social deprivation index [12] or physician age, sex and location of practice. Area-level indices such as income quintiles, and the material and social deprivation index were derived using patients’ 6-digit postal codes. Non-parametric Wilcoxon rank-sum tests and Kruskal Wallis tests were used to assess the association between these patient/provider characteristics and median wait-times. We calculated the proportion of referrals that were seen within a benchmark of 18 weeks (a benchmark used in the United Kingdom where 90–95% of referrals are targeted to be booked within 18 weeks) [13]. All statistical analyses were conducted at an alpha of 0.05, using SAS version 9.4.

Qualitative analysis

A qualitative approach informed by grounded theory was used [14]. A review of the literature helped to inform the design of the qualitative aspect of the study [15, 16]. Focus groups were used to capture the subjective meaning and utility of the wait time reports that were created. Specialists from the top 7 specialties referred to by family physicians at both practice sites were invited to participate in a focus group. The aim was to have 6–8 participants in the specialist focus group, and in each of the Toronto and Vaughan family physician focus groups. Individual interviews were scheduled with participants who were unable to attend a focus group. Focus groups and interviews were conducted in-person by the qualitative research assistant and the Principal Investigator. At the start of the session, physicians received a paper wait time report to review that included median wait times of specialists (anonymized) by practice address for 7 specialty areas. Specialists received a similar paper report plus a more detailed confidential report with their personal wait times listed for each referral made to them specifically. A semi-structured interview guide was used for family physicians (Appendix 2) and specialists (Appendix 3) that asked about the relevance, clinical utility and acceptability of the wait time reports. Focus groups and interviews were conducted over 2 months from October to November, 2018. Interviews lasted 60 min and were audio-recorded and transcribed verbatim, anonymized and analyzed using techniques informed by grounded theory [14] including coding, interpretations of data patterns and constant comparison method [17].

Research team members performed line by line open coding of interview transcripts [18] using an inductive approach with principles of constant comparison [19] for analyzing data. Co-investigators (M.N. and T.M.) read the same transcripts independently and met with one another to develop a coding manual. The transcripts were subsequently coded line by line by the qualitative researcher (T.M.). Team members met to review and refine codes and any discrepancies were resolved through discussion. Inductive analysis was appropriate for use in this research since there were no previous studies dealing with the phenomenon under investigation [20]. NVivo 11 (NVivo qualitative data analysis Software; QSR International Pty Ltd. Version 11, 2016) was used for data management.

Results

Quantitative analysis

Figure 1 summarizes the wait time data extraction and cleaning process from the EMR. The data cleaning process required some manual review of spreadsheets to remove some referrals as noted in the Figure. It took approximately 2–3 h of time to identify and remove missing referrals, and specialties that were not applicable. Validation of wait time calculations from 150 charts demonstrated a wait time concordance rate of 100%. Referrals analyzed included referrals where appointments dates were pending or not recorded in the EMR. The final sample of specialist referrals where wait time information was available included 7141 referrals (4967 unique patients).

Fig. 1
figure 1

Wait time data extraction process

Table 1 outlines characteristics of the referrals. Wait times differed across age groups. We observed differences in median wait times in those > 65 years (46.0 days) versus those 0–19 years (40.0 days) (p = 0.0006). We also observed differences in median wait times in those > 65 years (46.0 days) versus those 20–44 years (42.0 days) (p = 0.0002). There was no difference in wait times by sex. Wait times for referrals marked as urgent had a shorter wait time compared to routine referrals: 13 days versus 43 days respectively (p < 0.01). There seems to be some cyclicality to wait times based on season referred, with a longer wait in the summer season, however, more robust data using more referrals and practice sites would be needed to examine this trend further. Analysis of income level and material and social deprivation level for patients with an available postal code did not show any meaningful difference in wait times by income quintile before taxes, or by material and social deprivation.

Table 1 Comparison of wait times by patient and referral characteristics (N = 7141)

Of the 4967 unique patients, 69% had one referral, 22% had 2 referrals, 6% had 3 referrals and 3% of patients had 4 or more referrals. There were 1357 unique specialist names in the database and 596 unique departments. The list was reduced to 33 unique specialities by grouping similar specialty services/departments (Table 2). The top 10 specialties consulted were Dermatology, Gastroenterology, Ear Nose and Throat, Obstetrics and Gynecology, Urology, Ophthalmology, Immunology, Orthopedics, General Surgery and Rheumatology.

Table 2 List of specialties referred to and their wait time distribution (N = 7141)

The median wait time of all referrals was 42 days, 75% of patients were seen within 80 days and all patients were seen within 760 days. Almost 90% of patients saw the specialist within an 18 week benchmark, and 99.53% of referrals were seen within the year (Fig. 2).

Fig. 2
figure 2

Wait times within 18 weeks and over 18 weeks by specialty

Qualitative analysis

We conducted three focus group sessions, two family physician-centred focus groups and one specialist-centered focus group, along with two one-on-one specialist interviews. In total, 6 specialists and 14 family physicians participated in the focus groups. Socio-demographic surveys were completed by 17 of the 20 (85%) participants (Table 3). Thirteen of the 17 respondents (76%) were based at academic type practices, with years in practice varying from 4.5 to 21 years. Table 4 summarizes the information obtained from the focus groups and interviews organized by topic area discussed in the semi-structured interviews.

Table 3 Sociodemographic information of focus group participants
Table 4 Emerging themes from primary care providers and specialists

General impressions of wait time reports by family physicians were that the data was viewed positively and seen to have value. Family physicians perceived that having wait time information had significant clinical utility when deciding upon which specialist to refer. Some physicians reported that in non-urgent cases, they would refer to their preferred specialist- this was often based on familiarity or a prior good working relationship. However, referring a patient to a specialist with a shorter wait, particularly when patient quality of life was of concern was seen as important by most, for example if the patient was in pain or had elevated anxiety. The following comment exemplifies this:

“A non-urgent referral doesn’t mean that it can wait a year to be seen. It just means they don’t have to be seen tomorrow. But if they are having significant sinus symptoms, and I’ve done everything in my arsenal to help them, do they need to be seen tomorrow for sinus symptoms? No. But I also don’t want them to wait 4 months to see somebody and suffer for 4 months needlessly if they could see somebody within 2 weeks. So that would be super helpful information in my opinion.” (Family Physician)

Wait time information could also provide a means for family physicians to learn of new specialists and manage patient expectations around waiting for the referral. Having easy access to data at the point of care was identified as important to family physicians (versus paper reports or an email with wait time information).

Wait time information was viewed as an important piece of information by both family physicians and specialists for local, regional and provincial planning.

“I think it’s amazing. It seems like this would be really useful to be out there both for like a [specialty named] association, for government, for LHINs [Local Health Integration Network]. I’ve never seen data like this. So I think it would be incredibly useful.” (Specialist)

General impressions of wait time reports by specialists were that seeing the wait time data was interesting. Some specialists had never seen their own wait time data or that of their colleagues. Specialists had variable perceptions about the clinical utility of wait time data. The data could reveal system inefficiencies and allow for redistribution of referrals for some specialties. However, even if aware, specialists acknowledge they may not be able to change their practice or appointment booking processes.

“When you’re looking at a process, it’s either inefficient or it’s a capacity problem. And at least from the [specialty named] side of things, there are some reasonable evidence based on a glance of the distribution that it’s an inefficiency problem. So certain people are holding up the line, and certain people are not. And they’re not being redistributed that way. But that’s the way referrals have been made in Ontario for the last 200 years. And so that’s why I think there’s a lot more push now to create programs like a rapid assessment clinic. A lot of groups now are sort of first-come, first-serve, depending on individual practitioner wait times.[…]So I guess the way I look at it is that it just confirms a lot of people’s suspicion that within this is probably not a capacity issue right now, it’s probably an inefficiency of distribution issue.” (Specialist)

Focus groups and interviews furthermore highlighted that specialists may be unaware of their own actual wait times and levels of triaging referrals by specialist offices can vary widely. Specialists stated that sub-specialist wait times would be another key piece of data for additional analysis to help determine where system bottlenecks may be. A perceived limitation of the wait time report was that the data does not reflect variance with regards to referrals. For example, factors influencing referrals and wait times include patient preference, primary care provider preference, type of problem or diagnosis and appropriateness of the referral.

“It could be perceived that Dr. X with a wait time of 190 days is bad. But it might be that, you know, they are the only person who does that. Or b) they have… they’re doing a really good job, and a lot of people want to go to them, and they don’t want to go anywhere else, and they want to wait.”

Having access to additional, more robust data was of interest to both specialists and family physicians.

There were mixed views about establishing wait time benchmarks. While benchmarks are seen by some specialists as arbitrary because they depend on sub-specialty or diagnosis, they were also seen as important for setting standards and improving healthcare system. Family physicians thought benchmarks may be unrealistic because of geography and location of specialists, and may set up unreasonable expectations. Benchmarks may also inherently place judgement, blame and stress on specialists. It may be better to look at similar healthcare systems doing better and find ways of mirroring or learning from them.

Discussion

We conducted a mixed-methods study to evaluate the feasibility of producing primary to specialist care wait time reports from the EMR and conducted focus groups and interviews to determine the clinical utility of the wait time information. Primary care EMRs house valuable specialist wait time data, which has been an untapped resource in most jurisdictions. Interviews with family physicians and specialists demonstrated the relevance and clinical utility of wait-time related information, especially for family physicians.

The 2 year wait time data extracted from the health records of patients in both clinics closely resembles broader trends in specialist referral patterns in the Province. Our top 10 list of specialist referrals varied slightly from another Ontario study published in 2017 that examined specialist referrals in Ontario [21]. Our list includes rheumatology in the top 10, whereas the other study includes cardiology and plastic surgery. Our top 10 list of specialties was used to form the list of specialists that were invited to the specialist physicians focus groups/interviews. On further evaluation of our wait time data, there was no meaningful difference detected in wait times when comparing low income to high income populations, or by least material and social deprivation to greatest material and social deprivation level. There was no association between patient sex and referral wait times, however there was an association between patient age and wait times, with older patients having shorter wait times compared to younger patients. A possible difference was detected by season referred, with summer months resulting in longer wait times, although more robust data involving more years and referrals would be needed to definitively conclude these trends. These results are similar to information found in a 2014 study examining specialist wait times in Ontario. Although slightly different measures of socioeconomic status were used, specialist wait times in this study did not vary by patient income or social and material deprivation scales [2]. It was reassuring to note that wait times were not impacted by a patient’s income level in Canada’s single payer system. In terms of the wait times themselves, the overall median wait time in this study was 42 days and the 75th percentile was 82 days. Our range is 34–75 days for the median wait time compared to the median range 33–76 days in 2014, and 75thpercentile of 62 to 112 days compared to 63 to 231.5 days in the 2014 study [2]. Interestingly, triaging does seem to happen on a system level with a median wait time that is 30 days less for urgent versus routine appointments. With approximately 94% of referrals being marked as “routine”, family physicians in these practices do not seem to be “overcalling” the urgency of the nature of the problem when making referrals to specialists, or do not use the option of marking the urgency of a referral.

Specialists and family physicians in this study highlighted that wait time reporting would enable primary care physicians to more comprehensively consider the factors when creating a referral. Such factors may include: (1) having the shortest wait time is most important to reduce morbidity experienced by patients while waiting; (2) that geography is more important to a patient and some may prefer a longer wait time if it meant they could see a specialist closer to where they live; or (3) it may be that seeing a specific specialist is most important, and a patient would be willing to wait longer for their consultation to see a particular specialist. Wait time data may also be useful for physicians newer to practice or those without a preferred specialist in mind to learn about specialists in the region. Additionally, combining referral data across practices/regions could lead to better local, regional and provincial specialist access information for human resource planning purposes. This could complement other initiatives that may contribute to shortening specialist wait times including telehealth consults [22] (electronic asynchronous consultations obviating the need for face to face appointments between patient and specialist) and e-referral systems (i.e. province wide electronic health referral system where wait times could be viewed and referral status could be tracked), pre-assessment in specialized clinics [23] (a model of triage and appointment allocation to reduce wait times), and central intake [24] (instead of having multiple-queues and multiple-servers to manage referrals, specific specialists in a given jurisdiction would have a single queue allowing each patient to see the first available specialist) [25, 26]. In 2008, the Edmonton North Primary Care Network (PCN) developed a provincial e-referral system, which includes a specialist database with information on specialist referral requirements, forms and protocols, and tracks wait times [27]. A trend analysis of the referral wait time (defined as the time from referral by a family physician to an appointment date with a specialist) from 2009 to 2011 using the program database (n= 33,281 referrals) for 22 specialties showed a decrease in the overall wait time year over year [25]. The province of Ontario is currently introducing eReferral which is an electronic platform, integrated within a growing number of electronic health management systems but also available as a web-based platform, which allows secure referrals to be sent and received between family physicians and specialists [28]. eReferral enhances communication between primary care providers and specialists, eliminates the need for fax-based methods of correspondence, and displays specialist wait time information [29].

The United Kingdom’s National Health Service (NHS) tracks wait times from the time a patient is referred to a specialist by the family doctor to the time the patient receives medical treatment from a specialist. The NHS has set a benchmark: 90–95% of patients should wait no longer than 18 weeks from the time at which they are referred to the time when they are treated [11]. In comparing our median wait time data with the NHS 18 week benchmarks it was interesting to note that about 90% of patients had a wait time within the 18 week target. Applying benchmarks could be another way to shift the needle on wait times. A first step is to be able to have access to the wait time information, which is presently not easily extractable and publicly available by most EMR vendors.

This study demonstrated that wait time information, valued by family physicians and specialists, lives within electronic health records, but is not easily extractable without manual manipulation. Aside from time required to create the wait time reports manually, limitations of the current study include a high degree of missing data such as postal codes and urgency of the referral, which were used to derive income quintiles, material deprivation indices, and urgency of referral. The degree of missing data highlights the need to have these fields entered in a codifiable format in the EMR. The study also employed a small sample size (both in terms of unique patients and numbers of referrals) and only a single EMR. The same level of detailed referral data may not be available in all EMRs. The study setting focused on the Greater Toronto Area, and is likely not representative of wait times for a broader geography. The study also did not consider the time to make the referral in the first place (wait time 0), before it was sent to the specialist and the appointment was booked. However, the primary objective of the study was not to analyze the wait times themselves, but to explore the feasibility of extracting wait time data from the EMR and the clinical utility of the information for family physicians and specialists. The heterogeneity with which referral information is captured in primary care EMRs is a hurdle to scaling up wait time reporting to provincial/national levels. However, this study highlights important potential uses of wait time information for providers, patients and health regulators, and next steps would include the development and expansion of systems, such as eReferral, to more accurately collect and report wait time data. Finally, the descriptive qualitative aspect of this research never intended to reach saturation of all ideas related to wait time reporting, but rather act as a starting point to obtain information about how family physicians and specialists’ view and value wait time information.

Conclusion

Wait time information is perceived as valuable information to family physicians and specialists. While there are challenges with scaling up the functionality of various EMRs to provide wait time data, this study demonstrates that having access to specialist wait time information could aid provider and patient decision-making regarding specialist referrals, and potentially help with human resource planning and reduction of system bottlenecks. The use of technologies such as machine learning to code existing EMR data, or eReferral systems that readily report wait times, may improve the efficiency of creating wait time reports or sharing wait time information. Future work can be directed towards expanding primary care to specialist wait time reporting functionality through referral modalities such as e-referral systems. This way, family physicians and patients can make a more informed choice when considering the multitude of factors that go into the decision of which specialist to refer to.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to confidentiality but are available from the corresponding author on reasonable request.

Abbreviations

EMR:

Electronic medical record

CPSO:

College of Physicians and Surgeons of Ontario

UTOPIAN:

University of Toronto Practice-based Research Network

PCN:

Primary Care Network

NHS:

National Health Service

References

  1. Schoen C, Osborn R. The commonwealth fund 2010 international health policy survey in eleven countries. London: Commonwealth Fund; 2010.

    Google Scholar 

  2. Jaakkimainen L, Glazier R, Barnsley J, Salkeld E, Lu H, Tu K. Waiting to see the specialist: patient and provider characteristics of wait times from primary to specialty care. BMC Fam Prac. 2014;15(1):1–3.

    Article  Google Scholar 

  3. Barua B, Ren F. Waiting your turn: wait times for health care in Canada, 2016 report. Fraser Institute; 2016. https://www.fraserinstitute.org/sites/default/files/waiting-your-turn-wait-times-for-health-care-in-canada-2016.pdf. Accessed 7 Oct 2019.

  4. Globerman S, Esmail N, Day B, Henderson D. Reducing wait times for health care: what Canada can learn from theory and international experience. Fraser Institute. 2013 Oct. https://www.fraserinstitute.org/sites/default/files/reducing-wait-times-for-health-care.pdf. Accessed 7 Oct 2019.

  5. Wait Time Alliance. Shedding light on Canadian’s total wait for care. In: Report card on wait times in Canada. 2015. (http://www.waittimealliance.ca/wta-reports/2015-wta-report-card/ Accessed 9 Jan 2017).

    Google Scholar 

  6. Willcox S, Seddon M, Dunn S, Edwards RT, Pearse J, Tu JV. Measuring and reducing waiting times: a cross-national comparison of strategies. Health Aff. 2007;26(4):1078–87.

    Article  Google Scholar 

  7. Ontario Ministry of Health and Long Term Care. Ontario Wait Time Strategy. 2015. (http://www.health.gov.on.ca/en/public/programs/waittimes/strategy.aspx Accessed 20 Mar 2017).

    Google Scholar 

  8. Neimanis I, Gaebel K, Dickson R, Levy R, Goebel C, Zizzo A, Woods A, Corsini J. Referral Processes and Wait Times in Primary Care. Can Fam Phys. 2017;63(8):619–24.

    Google Scholar 

  9. Sale JE, Lohfeld LH, Brazil K. Revisiting the quantitative-qualitative debate: Implications for mixed-methods research. Qual Quant. 2002;36(1):43–53.

    Article  Google Scholar 

  10. Department of Family and Community Medicine. About Utopian. University of Toronto. https://www.dfcm.utoronto.ca/about-utopian#UTOPIAN. Accessed 20 Feb 2020.

  11. Statistics Canada. Postal codeOM conversion file (PCCF), August 2015. Statistics Canada. 2016. https://www150.statcan.gc.ca/n1/en/catalogue/92-154-X

  12. Centre for Urban Health Solutions. Ontario marginalization index. Public Health Ontario. 2016. https://www.publichealthontario.ca/en/data-and-analysis/health-equity/ontario-marginalization-index.

  13. NHS. Guide to NHS waiting times in England. UK. https://www.nhs.uk/nhs-services/hospitals/guide-to-nhs-waiting-times-in-england/. Accessed 20 Mar 2017.

  14. Charmaz K. Constructing grounded theory. In: A practical guide through qualitative analysis. Thousand Oaks: Sage Publications; 2006.

    Google Scholar 

  15. Sadelowski M. Focus on Resarch Methods: Whatever Happened to Qualitative Description? Res Nurs Health. 2000;23(4):334–40.

    Article  Google Scholar 

  16. Sandelowski M. What’s in a Name? Qualitative Description Revisited. Res Nurs Health. 2010;33(1):77–84.

    Article  Google Scholar 

  17. Boeije H. A purposeful approach to the constant comparative method in the analysis of qualitative data. Qual Quant. 2002;36(4):391–409.

    Article  Google Scholar 

  18. Crabtree B, Miller W. Doing Qualitative Research. 2nd ed. London: Sage Publications; 1999.

    Google Scholar 

  19. Boeije H. A purposeful approach to the constant comparative method in the analysis of qualitative data. Qual Quant. 2002;36:391–409.

    Article  Google Scholar 

  20. Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15.

    Article  Google Scholar 

  21. Liddy C, Arbab-Tafti S, Moroz I, Keely E. Primary care physician referral patterns in Ontario, Canada: a descriptive analysis of self-reported referral data. BMC Fam Pract. 2017;18(1):18–81.

    Article  Google Scholar 

  22. Caffery LJ, Farjian M, Smith AC. Telehealth interventions for reducing waiting lists and waiting times for specialist outpatient services: A scoping review. J Telemed Telecare. 2016;22(8):504–12.

    Article  Google Scholar 

  23. Lewis AK, Taylor NF, Carney PW, Harding KE. Specific timely appointments for triage to reduce wait times in a medical outpatient clinic: protocol of a pre-post study with process evaluation. BMC Health Serv Res. 2019;19(1):1–8.

    Article  Google Scholar 

  24. Milakovic M, Corrado AM, Tadrous M, Nguyen ME, Vuong S, Ivers NM. Effects of a single-entry intake system on access to outpatient visits to specialist physicians and allied health professionals: a systematic review. CMAJ Open. 2021;9(2):E413.

    Article  Google Scholar 

  25. Milne V, Tebber J, Pendharkarh S. Four ways Canada can shorten wait times for specialists. Healthy Debate 2017 Feb 23. https://healthydebate.ca/2017/02/topic/wait-times-specialists. Accessed   11 Sept 2019

  26. Milakovic M, Corrado AM, Tadrous M, Nguyen NE, Vuong S, Ivers NM. Effects of a single-entry intake system on access to outpatient visits to specialist physicians and allied health professionals: a systematic review. CMAJ Open. 2021;9(2):e413–23.

    Article  Google Scholar 

  27. Thanh NX, Wanke M, McGeachy L. Wait time from primary to specialty care: a trend analysis from Edmonton, Canada. Healthcare Policy. 2013;8(4):35.

    PubMed  PubMed Central  Google Scholar 

  28. Think Research Corporation. eReferrals in Ontario [Internet]. Toronto: Think Research Corporation; 2021. (https://www.thinkresearch.com/ca/programs/electronic-referrals-ontario/ Cited 2021 Nov 16).

    Google Scholar 

  29. Wierts KW. End User Satisfaction with eReferral [Internet]. Ontario: eHealth Centre of Excellence; 2021. (https://ehealthce.ca/userContent/documents/Research%20and%20Evaluation/eReferral%20case%20studies/End%20User%20Satisfaction%20with%20eReferral_Jan%202021_Final.pdf Cited 2021 Nov 16).

    Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This work was supported by the UTOPIAN program at the University of Toronto. The funding body had no role in the design of the study, collection, analysis, and interpretation of the data, or in the writing of the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

MN designed study, formed study team, coordinated meetings of team members, attended focus groups, drafted manuscript. NI, WM, CM, LJ, CB helped design study, reviewed qualitative survey tool, reviewed manuscript. CM, RM, BA data extraction, data analysis, manuscript review. TM led focus groups, qualitative analysis. JP: study coordination, data management, prepared wait time reports, data analysis, manuscript preparation and review. SK manuscript preparation and submission. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Michelle S. Naimer.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained from the Sinai Health Research Ethics Board in Toronto, Ontario (MSH REB #17–0356-E). Participants gave their verbal informed consent to participate in the study as described in the methods section of this paper, as was approved by the ethics committee. This research was conducted in accordance with the relevant guidelines and regulations as stated in the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Naimer, M.S., Aliarzadeh, B., Bell, C.M. et al. Specialist wait time reporting using family physicians’ electronic medical record data: a mixed method study of feasibility and clinical utility. BMC Prim. Care 23, 72 (2022). https://doi.org/10.1186/s12875-022-01679-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12875-022-01679-x

Keywords