Our study protocol was published previously and provides a detailed overview of the study’s methodology [13] in accordance with CONSORT guidelines for pilot studies [14].
Design
A randomised controlled design was used, whereby participating practices were assigned to either the intervention or control group by an external researcher using online randomisation software. Given the nature of the intervention, blinding of practice assignment among the researchers was not feasible.
Ethical approval
Ethical approval for the study was received from the Irish College of General Practitioners’ Research Ethics Committee (no reference number, approved January 2016) and the Office for Research Ethics Committees of Northern Ireland (16/NI/0008, approved February 2016).
Recruitment
Our sample size was pragmatic, aiming to include practices of diverse size and location, and from two different healthcare systems, in order to test the intervention’s feasibility in a range of settings. A purposeful sample of RoI practices was recruited through the Western Research Network (WestREN) [15], an Irish GP research network. Practices were stratified according to size (large (> 2 GP principals) or small (≤2 GP principals) and location (urban or rural). In the RoI, 13 practices were invited to participate, with 11 practices agreeing: two declined, due to the required time commitment. In NI, two practices of similar size and location (large; urban) were invited through the NI Clinical Research Network Primary Care Group [16]: both agreed to participate.
Invitation letters were sent to the principal GP(s) in the selected practices (see Fig. 1) and, with their consent, other staff were then invited to participate. In the RoI, GPs provide services for private patients who pay for each consultation, and for patients with medical cards, whose healthcare is publicly funded. In NI, GP services are provided free-of-charge through the National Health Service.
Setting and participants
Of the 11 practices who initially joined the study, 10 completed (8 RoI, 2 NI; 7 urban, 3 rural). A total of six of the practices were ‘large’. The mean number of managerial staff (GP principals, practice managers) was 4.8 (SD 1.5; range 3–7) and of non-managerial staff (non-principals, practice nurses, administrators) it was 7.5 (SD 2.9; range 4–11). Practices (N = 4) categorised as ‘small’ had a mean of 2 managerial staff (SD 0), and 4.3 (SD 1.7; range 2–6) non-managerial. In each practice, all staff provided personal written consent regarding their participation and all data were recorded anonymously.
Procedure
Intervention practices
The intervention consisted of two components: (1) safety climate (SC) measurement and feedback and (2) patient record review using a specialised trigger tool to identify instances of harm (i.e., the trigger review method (TRM) [17]).
SC was assessed using the GP-SafeQuest for primary care [18]. SC is described as a measurable snapshot of an underlying safety culture at a particular period of time [3, 10, 19]. The GP-SafeQuest is a valid and reliable [20] survey instrument, designed specifically to measure SC perception, in primary care settings, across five subscales (leadership, teamwork, communication, workload, and safety systems), using Likert scales. Paper copies and stamped addressed envelopes were delivered to practices at three time points (baseline, study midpoint, and study terminus; see Fig. 1).
Each intervention practice received individualised practice-level feedback (a written report) on their baseline and study midpoint SC surveys, within a month of survey completion. Simple descriptive statistics and illustrative diagrams allowed comparison of SC scores with other practices’ anonymised data. Additionally, a research team member presented and led discussion of the findings at a practice meeting where the primary care team also discussed PSIs identified through reviews of their patients’ records, as described below.
One GP from each practice was asked to conduct a patient record review using the TRM at 3 and 7 months (see Fig. 1) [17]. The GP who conducted the review attended a two-hour training workshop delivered one-to-one by a facilitator (CC) using previously developed materials [11]. It is important to indicate that the TRM was an intervention component and not an outcome measure: its purpose was to facilitate the identification of specific patient safety issues within each practice.
At each time-point, the reviewer was requested to apply the TRM [17, 21] to a minimum of 20 and maximum of 30 records from a high-risk group (aged > 75 years), randomly selected from patients who had attended the practice during the previous three months. Records were first reviewed in order to detect whether they contained a ‘trigger’, defined as flags, occurrences or prompts that alert reviewers to potential errors and previously undetected adverse events (e.g., more than three consultations in seven days, hospital admission, repeat medication stopped) [17]. If a trigger was found, the record was reviewed in greater detail to determine whether the patient experienced any harm. Based on the definition used by De Wet and Bowie [17], harm was defined as ‘anything that happens as a result of interaction with health services that you would not want to happen to you or your relatives’. If no harm was detected, or the reviewer was unsure whether harm had occurred, they were advised not to record the incident. If harm was detected, the reviewer classified its perceived severity, and whether it was preventable and originated in secondary or primary care, using previously developed rating scales [11].
Participants completed a Trigger Review Summary Report (TRSR) [11], a standardised form containing a summary of anonymised data on the number of detected triggers, details of PSIs, and actions that were or should be taken as a result of the review.
Each intervention practice received 1000 Euro for participating. All participants from control and intervention practices were entered into a draw for a 100 Euro voucher at study baseline and terminus.
Control practices
All staff in control group practices were invited to complete the GP-SafeQuest [18] at baseline and at the study’s terminus (see Fig. 1). Feedback on their SC was provided at the conclusion of the study, with access to the intervention materials.
Outcome measures
Safety climate
The GP-SafeQuest was used to evaluate the impact of the intervention, with SC measured at the beginning and end of the study in all practices (see Fig. 1).
Process evaluation
Process outcomes of interest were: willingness of practices to participate; response rates to questionnaires; retention of control and intervention practices; and the intervention group’s views on the feasibility, usefulness, and sustainability of the intervention. Semi-structured interviews were conducted (see ‘Interview Schedule for SAP-C’ in Additional files 1 and 2) with between one and three members of each intervention practice team regarding their perceptions of the intervention and an end-of-study questionnaire was circulated to all members of the intervention practices. Purposive sampling was used to select interview candidates, in order to obtain perspectives from a range of healthcare professionals and practices. We aimed to interview at least one GP in each practice and, where feasible, others with nursing or administrative responsibilities.
Interviews were conducted by a human factors psychologist (POC) and General Practitioner (MC). The interviews were either carried out by phone or face-to-face, digitally recorded and then transcribed.
Final feedback questionnaire
A feedback questionnaire was distributed for completion at the completion of the study (see ‘Safety in Primary Care (SAP-C) Feedback’ in Additional files 1 and 2).
Data analysis
Safety climate questionnaire
Descriptive analysis was used [12] to report means and standard deviations of the five subscale scores and total scores on the GP-SafeQuest for each group at baseline and the study terminus. The effect size, regarding differences in pre-test and post-test subscale and total GP-SafeQuest scores, was computed.
Patient record review using TRM
Descriptive analysis was carried out on the number of records reviewed, triggers identified, number and type of PSIs recorded, PSI severity, preventability and origin, and changes made following the review.
Feedback questionnaire
Descriptive analysis was used to examine feedback questionnaire responses.
Interviews
Interview transcripts were analysed using the Framework Method [22] of thematic analysis which provided a structured process to summarise and explain the data. After two researchers independently coded three of the transcripts, a set of codes was agreed upon and grouped into clearly defined categories to form a working analytical framework which was then applied to all interviews by one researcher.