Joint teleconsultations – real-time outpatient appointments using video-conferencing equipment to connect patients visiting their GPs to remotely located consultants – were first implemented in Portugal in 1998 [21]. By 2011, more than 32,000 teleconsultations had been provided, the majority of which in the Alentejo region. Dermatology accounted for most teleconsultations performed, followed by cardiology and neurology. Other specialties included physical and rehabilitation medicine, respiratory medicine, urology and psychiatry. In Portugal, patients seeking an appointment with a consultant must first be referred by a GP. If the GP practice is part of the telemedicine network then the patient must first be referred to a teleconsultation before a subsequent face-to-face appointment can be arranged, if necessary. Different practices adopt different organisational approaches. Some appoint a GP coordinator who is present in every teleconsultation for a specific specialty; in others, the patient’s own GP is present. There are no financial incentives for GPs to use teleconsultations and while practices are reimbursed for teleconsultations, these add to GP workload so that there is no clear financial incentive to provide these appointments.
Experimental design and data collection
A DCE is a survey method based on the assumption that a good or a service (e.g., referral to a specialist) can be described by a set of characteristics or attributes (e.g., waiting time), and that the extent to which individuals (e.g., GPs) value that good or service is determined by the nature and levels of the characteristics [22]. Put simply, GPs’ decisions to refer are affected by a number of factors or attributes, and by observing their decisions it might be possible to determine how the factors and attributes affect their choices. The method has been used extensively to elicit patient and physician preferences for healthcare services [23]. Conducting a DCE in the context of GP referrals involves a number of steps [24]: identifying the relevant attributes affecting the decision to refer and their levels; selecting a sample of possible level combinations (i.e., referral choices) to include in a questionnaire; distributing the questionnaire to a sample of GPs; and analysing GPs’ responses using appropriate regression techniques.
We reviewed determinants of GP referrals and identified four categories of potentially influential factors [20,25,26]: patient characteristics (e.g., clinical need and anxiety for a referral), GP characteristics (e.g., years of experience and training) practice characteristics (e.g., size and location), and secondary care factors (e.g., waiting time and perceived quality). Previous evidence suggests that perceived clinical need for a referral is the most important factor in GPs’ decisions to refer. To capture this in our questionnaire, we restricted our analysis to four dermatological presentations, selected from a database of actual GP referrals in a major Portuguese hospital. Among the cases that were most commonly referred, we identified conditions which should be managed in primary care (chronic plaque psoriasis and seborrhoeic keratosis) and conditions which require urgent attention and referral (cutaneous malignant melanoma and melanocytic naevus), so as to cover the spectrum of clinical need and urgency. Textual descriptions were produced for each case with the help of a senior dermatologist and supplemented with photographs of the lesions. Dermatology was chosen for a number of reasons: variation in referral rates to dermatologists has been shown to exist [27], potentially signalling overreferral and/or underreferral; dermatological lesions tend to develop independently of other illnesses [27], so that the potential confounding effect of co-morbidities can be disregarded; and cases can be described in paper form using text and images with relatively little loss of information (as opposed to neurological cases, for example).
Based on the review of determinants of referrals, we further posited that decisions were affected by three other attributes for which levels were chosen so as to reflect the Portuguese context: average waiting time for a dermatology appointment (levels included 30, 60, 150 and 365 days); distance from GP practice to dermatologist (levels were 0, 30, 60 and 100 kilometres); and pressure from patients or families to be referred (levels were yes or no). Patient age and gender were excluded given previously mixed evidence on their impact.
The combination of all levels of attributes resulted in 4x4x4x2 = 128 choice situations. In other words, situations in which the GP needs to decide whether or not to refer, given a specific clinical presentation, waiting time, distance, and pressure from patients. Naturally, it would be infeasible to present 128 scenarios to each GP, so we created 8 questionnaire versions with 16 scenarios each, making sure that all levels and attributes appeared with equal frequency across versions (this is commonly done in DCEs and is referred to as blocking). To test for GP heterogeneity, one choice scenario was repeated across all versions, so that 7 versions had 17 scenarios (GP heterogeneity is not considered in this study but could be in future work). An example of a choice scenario – as presented to respondents – is provided in Figure 1 (the format is the same for all scenarios except the levels change). While we asked GPs what priority they would choose for referrals (see Figure 1), we do not use that information in this analysis (i.e., we aggregate all normal priority and urgent referrals).
On the first page of each questionnaire, before the choice scenarios, GPs were asked to provide information on their age, gender, years of experience, number of patients in their practice, region, distances to the closest private and public dermatologists, an assessment of their patients’ general health status, whether they had a special interest in dermatology, and finally whether they had participated in real-time teleconsultations. Responses to these questions provided further information to characterise referral preferences. Altogether, the questionnaire provided us with multiple variables from all four categories identified in the literature.
Around 600 self-complete questionnaires were distributed in two meetings of the Portuguese Association of General Practice, in early 2013. Questionnaires were included in the delegates’ welcome packages which were distributed on a first-come-first-served basis, so that respondents were randomly assigned to one of the 8 versions. Based on the pre-test, questionnaires took an average of 8 minutes to complete.
Model specification and data analysis
For each of the 16 scenarios in each questionnaire, respondents were asked whether or not they would refer the patient. The answer is thus a binary variable (yes or no, 1 or 0) and so generalised linear regression techniques must be used. We used a binary logistic regression model to predict the outcome of the dependent variable (modelled as Y = 1 if a referral was made and Y = 0 otherwise) based on the values of the independent, or explanatory, variables (e.g., clinical presentation, GP age, etc.). As well as a general error term capturing unobserved variation across GPs, we also included a GP-specific error term to account for the fact that there were multiple observations for each GP. Two models were estimated: model 1a with only attributes and model 1b with both attributes and covariates. This allowed us to test for changes in the sign, magnitude, and statistical significance of explanatory variables, which could indicate confounding.
After running the logistic model, marginal effects were used to determine how the probability of a referral being made is associated with changes to specific attributes or characteristics. To determine the association between participation in teleconsultations and the probability of a referral being made, we calculated marginal effects at representative values (i.e., all other explanatory variables were set to their sample means or modes) for each of the four clinical presentations. All statistical analyses were performed with Stata 12.
The study was sponsored by the Portuguese Fundação para a Ciência e a Tecnologia and conducted in Portugal. The sponsor does not require ethical approval for anonymous surveys (no identifiable data was collected) which do not involve patients, as was the case. Participation in the study was voluntary and no incentives were offered.