The results suggest that GPAQ-R, a development of previous versions of the General Practice Assessment Questionnaire, is suitable to use for revalidation of doctors in the National Health Service, meeting the requirements for survey development set out by the General Medical Council and with psychometric properties similar to those of the GMC’s own questionnaire . We recommend that 30 completed questionnaires should be obtained to give sufficiently reliable results for scoring doctor-patient communication for individual doctors. However, where this initial screen raises concern, a survey might be repeated with 50 returned questionnaires to give greater reliability, increasing the reliability coefficient from 0.7 to 0.8.
While these numbers give satisfactory levels of reliability for both items and the composite scale for doctor patient communication, they do not for the scale on trust and confidence or for two out of the three individual items on trust and confidence. In particular, for the item ‘Would you be completely happy to see this GP again?’ where over 99% of patients replied ‘Yes’, over 300 responses would be needed to achieve reliability of 0.7. Although taken from the GMC questionnaire, this item is unlikely to be discriminating as a screen for poor performance.
The strengths of this questionnaire compared to the GMC’s questionnaire are that it intentionally incorporates a range of practice characteristics to be assessed and is therefore suitable for a wider range of uses within the NHS than the GMC questionnaire which focuses solely on items relevant to revalidation. However, because of this, GPAQ-R is also considerably longer than the GMC questionnaire and this could affect response rate. It is important to note that the GMC’s recommended methodology (handing out questionnaires after a consultation) does not require response rates to be recorded. For GPs only wishing to use GPAQ-R for revalidation purposes, we have designed the survey so that the front page can be used on its own, which significantly shortens the questionnaire.
The relatively high non-completion rate for one item is of concern, namely the 12% of patients who did not provide valid responses to the question “Would you be completely happy to see this GP again?”, although some of these were data entry errors. We do not think this is due to wording of the question as the phrasing of this item is virtually identical to the GMC’s own questionnaire where lower non-complete rates have been reported. The high non-completion rate for this item may be in part due to the proximity of space for patients to make free comments about their experience with the GP. Thirty five of the blank items on this question had associated handwritten comments and we have now modified the instruction on the first page to include a comment on the importance of completing all questions. Patients may also be concerned that doctors would see the response to this question, and we note that GMC guidance is that patients should return questionnaires in a sealed envelope which may increase their confidence in that their answers will remain confidential, and we are not certain that this guidance was always followed in this study. Where patients choose to give a free text comment as an alternative to ticking a box, we believe that this is likely to indicate that the patient regards this as more valuable information, and we have adopted this approach in other research which we are carrying out. We therefore recommend that free text comments should form part of the feedback that doctors receive on their performance. However, if this is done, some comments need to be anonymised before being fed back which substantially increases the costs of processing the questionnaire data.
GPAQ-R, like the GMC questionnaire, takes an approach of asking about the quality of communication (e.g. ‘How good was the doctor at ….’), sometimes called evaluation questions. This contrasts with some other surveys which focus on whether particular questions were asked (e.g. ‘Did the doctor ask you about …’), sometimes called report questions which are sometimes regarded as less subjective and easier to interpret . A commonly cited cognitive model of how patients respond to questionnaire items was developed by Tourangeau  who suggests that completion of survey questions requires (1) comprehension of the question, (2) retrieval from memory of the relevant information, (3) use of the information to make a judgment if the question calls for one, and (4) selection and reporting of the response. Although report and evaluation approaches are sometimes contrasted, we believe that the difference between the two is modest provided very specific questions are asked, partly because ‘report’ items often have an evaluative component implied in their wording and in many circumstances both require a judgement to be made (stage 3 of Tourangeau’s model). Items in GPAQ-R ask for the patient’s evaluation of very specific aspects of care and do not include questions on general satisfaction.