Skip to main content

A pilot study of participatory and rapid implementation approaches to increase depression screening in primary care



Most individuals with depression go unidentified and untreated. In 2016 the US Preventive Services Task Force released guidelines recommending universal screening in primary care to identify patients with depression and to link them to treatment. Feasible, acceptable, and effective strategies to implement these guidelines are needed.


This three-phased study employed rapid participatory methods to design and test strategies to increase depression screening at Penn Medicine, a large health system with 90 primary care practices. First, researchers solicited ideas and barriers from stakeholders to increase screening using an innovation tournament—a crowdsourcing method that invites stakeholders to submit ideas to address a workplace challenge. Second, a panel of stakeholders and scientists deliberated over and ranked the tournament ideas. An instant runoff election was held to select the winning idea. Third, the research team piloted the winning idea in a primary care practice using rapid prototyping, an approach that quickly refines and iterates strategy designs.


The innovation tournament yielded 31 ideas and 32 barriers from diverse stakeholders (12 primary care physicians, 10 medical assistants, 4 nurse practitioners, 2 practice managers, and 4 patient support assistants). A panel of 6 stakeholders and scientists deliberated on the ideas and voted for patient self-report (i.e., through tablet computers, text message, or an online patient portal) as the winning idea. The research team rapid prototyped tablets in one primary care practice with one physician over 5 five-hour shifts to examine the feasibility, acceptability, and effectiveness of the strategy. Most patients, the physician, and medical assistants found the tablets acceptable and feasible. However, patient support assistants struggled to incorporate them in their workflow and expressed concerns about scaling up the process. Depression screening rates were higher using tablets compared to usual care; follow-up was comparable between tablets and usual care.


Rapid participatory methods engaged and amplified the voices of diverse stakeholders in primary care. These methods helped design an acceptable and feasible implementation strategy that showed promise for increasing depression screening in a primary care setting. The next step is to evaluate the strategy in a randomized controlled trial across primary care practices.

Peer Review reports

Depression is a leading cause of disability, affecting between 8 and 17% of the population [1,2,3,4,5,6]. Untreated depression is associated with significant medical comorbidity, functional impairment, low medication adherence, and increased risk of mortality [2, 7, 8]. Though a variety of mental health interventions reduce symptoms and other sequelae, only a fraction of people with depression receive treatment [9,10,11,12]. One major challenge is the identification of individuals with depression [13, 14]; as many as half of cases of depression go undiagnosed [15, 16].

Primary care is an optimal place to identify individuals with depression. First, due to the association between psychiatric disorders and physical diseases, primary care practices serve individuals with elevated rates of depression compared to the general population [17,18,19,20,21]. Second, patients overwhelmingly trust their primary care clinicians and may be more willing to seek treatment with their encouragement [22, 23]. Screening and subsequently treating patients for depression in primary care settings is effective at increasing response to treatment and remission, controlling physical disease, and reducing total healthcare costs [24,25,26,27].

In response to this mounting evidence, national organizations, payers, policymakers, and health systems have begun to transform their depression screening practices. The US Preventive Services Task Force now recommends that health systems implement universal depression screening protocols [28, 29]. These recommendations prompted the Centers for Medicare & Medicaid Services (CMS) to cover annual depression screening for Medicare beneficiaries in primary care and to financially incentivize health system universal screening practices [30]. The National Committee for Quality Assurance, the national accrediting body that approves quality of care performance metrics, approved the Patient Health Questionnaires (PHQ) as potential depression screeners. These include the PHQ-2, a well-validated 2-item tool shown to be sensitive and specific to Major Depressive Disorder, and the PHQ-9, the 9-item version of the questionnaire [31]. The PHQ-2 and PHQ-9 have become the depression assessments of choice for many health systems.

Despite payer incentives, there have been few specific guidelines to support health system implementation of universal depression screening in primary care. Many health systems have attempted to increase depression screening and have encountered several barriers [32]. Just before the 2016 universal screening guidelines were implemented, nationally representative studies found that depression screening occurred at rates of just 3-4% in primary care practices [33, 34]. Thus, acceptable and feasible implementation strategies to increase depression screening in primary care are needed.

In our study, we used participatory and rapid implementation methods that involved stakeholders across primary care at the University of Pennsylvania Health System (Penn Medicine) to design and pilot strategies to increase depression screening and follow-up. Stakeholder participation ensures the acceptability and feasibility of implementation efforts [35]. Large-scale health system initiatives to increase universal depression screening must incorporate the interests of all stakeholders involved in the process: leaders from Primary Care and Psychiatry, clinicians, staff, and patients. In addition to growing recognition that stakeholder input is crucial to implementation success, it is now acknowledged that the gap between research and practice is sometimes prolonged by traditional randomized controlled trial implementation studies, that, though rigorous, are resource intensive. Rapid implementation methods are an increasingly popular approach to design strategies [36,37,38,39]. Without sacrificing the systematicity of more rigorous approaches, rapid implementation methods accelerate the pace of data collection to immediately identify problems in system changes and are therefore designed to pilot and fine-tune implementation strategies. Rapid implementation allows health systems engaged in quality improvement projects and researchers interested in developing generalizable implementation strategies to “fail fast” and quickly refine their strategies before they scale up their approach. In other words, rapid implementation approaches are flexible methods that can be used by both health systems seeking to generate context-specific knowledge for quality improvement and by researchers seeking to design strategies that can be tested in other health service contexts. Our study provides health systems with a specific strategy to improve depression screening and assessment in primary care settings, as well as a general framework and set of methods to design implementation strategies.

General methods

Study context

In 2018, Penn Medicine’s Department of Psychiatry and Primary Care Service Line initiated a staged and adapted implementation of the evidence-based Collaborative Care Model—referred to as the Penn Integrated Care program (PIC) [40]—an approach for managing psychiatric disorders in primary care that combines mental and physical health services into a single setting [41, 42]. Healthcare professionals from multiple disciplines work together to coordinate care and treat patients. The Collaborative Care Model improves access to care and has shown promise in improving clinical outcomes for psychiatric and physical health conditions [43, 44].

Penn Medicine launched the PIC program in eight of its 90 primary care practices. These eight practices in Philadelphia serve over 90,000 patients and range in size from 4 to 26 primary care clinicians. The PIC program is modeled on the traditional Collaborative Care Model, though it also includes a “Resource Center” to assess and triage Penn Medicine primary care patients in need of any mental health services. We decided to conduct this study in PIC practices based on consistent feedback from primary care stakeholders that depression screening can only be ethically conducted if follow-up care is readily available.

When we began our study in February 2019, across all 90 of Penn Medicine’s primary care practices (including its PIC practices), depression screening was variably implemented. Around 40% of eligible patients were screened for depression annually, with eligibility defined by CMS as all adult patients with a primary care office visit who are not already diagnosed with a mood disorder [45]. The PHQ-2 was typically administered verbally by a medical assistant (MA) before the patient saw their primary care clinician as their vital signs were checked (i.e., temperature, blood pressure, etc.). Depending on the practice, the PHQ-2 was not always entered in the same location in the patient’s electronic health record, Epic©. If the patient screened positive on the PHQ-2 (a score of > 2), the MA would provide a paper-and-pencil version of the PHQ-9 for the patient to complete and hand-off to their primary care clinician when they entered the exam room. The primary care clinician would then be expected to manually enter the patient’s PHQ-9 scores in Epic© and follow-up if clinically indicated.

Overview of participants and procedures

Our study includes three distinct phases, all of which were conducted in PIC practices. In Phase 1, we conducted an innovation tournament to generate strategies to improve depression screening. Innovation tournaments are a novel participatory method in which stakeholders are invited to submit their ideas to address a specific challenge faced by a workplace, industry, or service system [46, 47]. We targeted leaders from Primary Care and Psychiatry, clinicians, and staff involved in depression screening (i.e., who interface with the PHQ-2 and PHQ-9) as our key stakeholders. We analyzed responses from the innovation tournament using a content coding approach to organize the ideas into themes. In Phase 2, we held a panel with expert stakeholders and scientists to discuss the ideas. After extended deliberation, the panel voted on a winning idea from the tournament. In Phase 3, we piloted the winning idea in a PIC practice with one physician over 5 five-hour shifts. When piloting, we used a mixed methods rapid implementation approach called rapid prototyping, which systematically tested and refined the strategy to ensure the implementation method was acceptable and feasible [38, 39, 48, 49]. All study procedures were approved by the University of Pennsylvania’s Institutional Review Board.

Phase 1 — the innovation tournament



In order to generate acceptable and feasible methods to increase depression screening in primary care, we deployed an innovation tournament. Innovation tournaments are designed to “democratize innovation” by increasing participation and engagement among stakeholders that have first-hand experience with a problem but are not typically consulted in workplace transformations [47]. Cash prizes incentivize tournament participation and creative idea generation [50]. Innovation tournaments have been shown to be effective in Penn Medicine and community settings for designing implementation strategies that empower stakeholders and increase investment in the suggested ideas [47, 51]. The innovation tournament was based on methods developed by the Penn Medicine Center for Health Care Innovation and prior work with clinicians [47, 51, 52].

Tournament platform

The Penn Medicine Center for Health Care Innovation hosts a web-based platform called “Your Big Idea” to run tournaments. Researchers post prompts about a healthcare challenge to crowdsource solutions. Participants can respond to these prompts, called “Idea Challenges,” with an idea, rate other participants’ ideas on a 1–5 “star” rating scale, and comment on other participants’ ideas.

Tournament prompts

The Idea Challenges were designed through a participatory process. The research team developed several different prompts that asked participants how they would improve depression screening. After consultation with leaders in the Penn Medicine Center for Health Care Innovation with experience conducting innovation tournaments, Idea Challenges were narrowed down to three options and a question was added about barriers to depression screening. The research team attended staff meetings (attended by practice leaders, clinicians, staff, and administrators) at two of the PIC practices and asked attendees to respond to each of the three Idea Challenges. Meeting attendees discussed the advantages and disadvantages of each prompt. At both meetings, the consensus was to focus the question on increasing screening rates. The final prompt was: “What’s your big idea for increasing depression screening rates in the primary care setting?” The barrier question was “What currently gets in the way of screening patients for depression in your clinic?”


First, we invited all stakeholders (leaders from Primary Care and Psychiatry, clinicians, and staff) in PIC practices by e-mail to participate in the innovation tournament called “Increasing Depression Screening Rates in Primary Care.” Three emails in total were sent to a total of 420 participants. In terms of the tournament response, 150 stakeholders (36%) opened the first email and 17 (4%) clicked on the “Your Big Idea” link; 122 (29%) opened the second email and 15 (4%) clicked the link; 118 (28%) opened the third email and 14 (3%) clicked the link. To enhance participation, we posted flyers (see Fig. 1) advertising the innovation tournament at seven of the eight PIC primary care clinics; one program did not respond to the request. The team also spent time in staff rooms in three of the clinics to recruit clinicians and staff. The landing page for the innovation tournament, which was live between March 12 and April 5, 2019, is available for viewing at

Fig. 1
figure 1

Flyer Advertising the Innovation Tournament in PIC practices


Ideas coding

When the innovation tournament closed, 31 ideas and 32 barriers were submitted through the Your Big Idea platform from 12 primary care physicians, 10 MAs, 4 nurse practitioners, 2 practice managers, and 4 Patient Support Assistants (PSAs). The 31 ideas received 48 ratings; eight ideas received 5 stars (out of a 5-star system) and these top-rated ideas all proposed that patients complete the PHQ-2/9 via self-report either on tablet computers in the waiting area, by text message before their appointments, or through MyPennMedicine (Penn Medicine’s confidential online patient portal) before their appointments.

The research team organized a meeting with a panel of expert stakeholders and scientists to deliberate and vote on the innovation tournament winning idea. To reduce the burden on the panel, the research team refined the list of 31 ideas into themes using a content coding approach [53]. The team coded the 31 ideas together and when there were disagreements, they resolved them by consensus. The ideas were organized into four themes: (1) patient self-report (e.g., patients complete the screener on tablet computers in the waiting room, through MyPennMedicine communications, or by text message; n = 12); (2) reframing (e.g., changing the wording of the screener, changing the way it is introduced; n = 8); (3) workflow changes (e.g., putting screening results in the “Vitals” or “Chief Complaint” sections of the electronic health record Epic©, create reminders in Epic©; n = 7); (4) patient education (e.g., an anti-stigma campaign, clinician-provided psychoeducation about depression, flyers providing education regarding signs and symptoms placed strategically in clinics; n = 4). See Table 1 for the ideas, their respective themes, and the barriers identified by stakeholders.

Table 1 Themes, ideas, and barriers from the innovation tournament

Phase 2 — the panel of expert stakeholders & scientists



We invited a panel of expert stakeholders and scientists to discuss the ideas from the innovation tournament and to decide on the winning idea. To ensure that the panel was representative of all stakeholders involved in depression screening as part of the PIC program (from PSAs who check patients in at the front desk, to primary care clinicians, to patients, to social workers receiving referrals, etc.) we invited stakeholders from each group. To recruit MAs and PSAs, we asked the eight PIC practice managers to invite their staff to our upcoming meeting. To ensure patient participation, a primary care leader invited a patient from Penn Medicine’s primary care patient advisory board who had participated in the PIC program. In addition, two members of the research team—one expert in implementation science involved in the PIC program (CBW) and one expert in applying behavioral science methods to health systems (AMB)—were invited to participate. In total, the panel included six voting members: one social worker providing mental health services in the PIC program, one primary care physician in the PIC program, one leader in psychiatry, one patient from a PIC practice, and the two scientists on our research team. It should be noted that aside from one person, all other panelists had not participated in the innovation tournament and were therefore naïve to the tournament responses when discussing and rating them.


The one-hour meeting took place over lunch. BSL summarized the innovation tournament results in the first five-minutes and guided the panel discussion with the following questions: “Which ideas are most surprising?”; “Are there any ideas you feel are missing from the list that would be important to test?”; “Which ideas can we immediately rule out?”; and “Which ideas seem most feasible, acceptable to clinicians and patients, and immediately actionable?”; and “Which ideas are your favorites?”

The panel discussed for 50 min and then anonymously rank-choice voted on their preferred idea theme (i.e., patient self-report, reframing, workflow changes, or patient education) using paper ballots. The discussion was transcribed verbatim.

The patient participating in the panel was compensated $100. Five innovation tournament participants with the most elaborated strategy to implement the winning idea were sent $100, and an additional five randomly selected innovation tournament participants were sent $100 to reward their participation.

Analysis plan

The discussion transcript was analyzed using content analysis to identify themes [53]. Two members of the research team (BSL and ACF) identified themes and repeating ideas. Disagreements were resolved by discussion and consensus. Rank-choice votes were analyzed using an instant runoff election method—a vote counting method for rank-choice elections— to select the winner of the innovation tournament [54]. The software used to analyze the election results was OpaVote©.


Discussion themes

Table 2 displays the ideas and themes from the panel discussion. BSL and ACF identified several repeating ideas from the panel discussion. The discussion was divided into two parts: a discussion of the barriers to depression screening and a discussion of the ideas submitted to the innovation tournament.

Table 2 Themes and ideas from the panel discussion

In terms of the barriers to screening, panelists were concerned about MAs administering the depression screener. The majority of panelists (67%) stated that MAs are not clinically trained to administer a sensitive mental health questionnaire. Most (67%) panelists voiced that the PHQ-2/9 were designed and validated for self-administration. Panelists (67%) also suggested that both clinicians, MAs, and patients may not fully understand the rationale for administering the depression screener. Finally, panelists (33%) indicated that there are several technological challenges related to the way depression screening results are integrated in Epic©.

Panelists discussed the ideas submitted to the innovation tournament. Most panelists (67%) focused on the fact that the “reframing” idea revealed that those submitting ideas to the innovation tournament were less focused on increasing screening rates, but rather improving the accuracy of screens. That is, tournament “reframing” ideas described ways to facilitate a deeper connection between providers and patients; they also recommended changing the wording of the PHQ-2/9 questions for the sake of clarity. The majority of panelists (83%) liked the idea of patient self-report of the depression screener, particularly the implementation of tablet computers in the patient waiting area. Panelists discussed safety and liability concerns with pre-check-in text messages through MyPennMedicine such as the need to ensure that the endorsement of suicidality could be quickly identified and acted upon if the patient is completing the PHQ-9 outside of the office. Most panelists (83%) also liked the idea of patient education and thought it was necessary, though one panelist voiced that patient education would unlikely increase rates; rather it would improve the accuracy of the depression assessment. Panelists felt that both ideas would also be feasible and acceptable to implement in the practices.

Election results

The instant runoff election winner to the rank-choice election was the patient self-report idea. The panel determined that the ideas suggesting patients complete the depression screener confidentially either on tablet computers, through MyPennMedicine ahead of the visit, or via text message were the most viable and potentially impactful strategies to improve depression screening.

Phase 3 — piloting the winning innovation tournament strategy

Overview of rapid prototyping

Rapid prototyping is the systematic testing of ideas in order to create and refine strategies quickly [55]. First employed in industrial design, this method has been extended to healthcare contexts where effective implementation strategies don’t yet exist and immediate feedback is necessary to optimize healthcare quality and safety [38, 39, 48, 49, 56, 57]. Rapid prototyping facilitates learning as quickly as possible whether a strategy works and allows researchers to make adjustments as needed. Identified problems are documented and the implementation plan is revised. A subsequent experiment is conducted to see if it resolves the problem and to identify any further problems. Rapid prototyping is done iteratively and cyclically, much like Plan-Do-Study-Act cycles frequently used in quality improvement studies [58]. See the analysis plan below for the process description.



To evaluate the feasibility and acceptability of the winning innovation tournament idea, we planned to first pilot in two PIC primary care practices before scaling the project up. The research team met with leadership in the Primary Care Service Line, including leaders from each PIC practice, and two practices agreed to participate in the pilot. Along with an informational technological consultant from the health system, two members of the research team met with the practice manager and the lead clinician of each practice to discuss the pilot workflow changes. Given that piloting the strategy would involve substantial changes to the workflow, one practice requested to initially pilot the tablets with one physician who already adhered to depression screening guidelines before scaling to the entire practice. Due to COVID-19 pandemic related physical distancing precautions, the pilot was halted after prototyping in one practice with one physician.


Study design

Data collection was conducted according to a withdrawal design method—i.e., a method in which the intervention is “withdrawn” systematically to allow for a comparison between changes in the outcome in baseline versus intervention periods [59]. The research team was present on specific “intervention” days to use tablets for depression screening. On alternating “baseline” days, depression screening was conducted as it was normally conducted in the practice. In the case of this specific PIC practice, usual care for depression screening involves the MA verbally administering the PHQ-2, and (if indicated, i.e., a score of > 2) the patient self-administering the PHQ-9 using paper-and-pencil. Once the primary care clinician arrives in the examination office, the patient hands off the completed PHQ-9 for the clinician to manually enter in Epic©.


Two tablet computers were purchased to conduct the rapid prototyping. To ensure patient safety and confidentiality, the health system encrypted each tablet and installed Epic Welcome©, the patient-facing application version of Epic©. Patients complete questionnaires and consent forms on Epic Welcome© and their responses sync in real-time with the clinician-facing version of Epic©. On Epic Welcome©, when patients complete the PHQ-2, the questionnaire automatically expands into the PHQ-9 if patients’ score on the PHQ-2 is greater than or equal to 1 (endorsing at least some symptoms on either of the two items). Notably, this is a more liberal cut-off than CMS and health system guidelines, which recommend follow-up at a PHQ-2 score greater than 2. This discrepancy provides further evidence that significant technological barriers prevent health system standardization of depression screening, as described by innovation tournament participants and panelists.

Rapid prototyping analysis plan

Rapid prototyping was conducted over 5 days (i.e., five 5-h shifts) in a PIC practice. The three steps of the rapid prototyping process are outlined below.

Step 1 — design

Before each 5-h shift, members of the research team (BSL and ACF) designed a plan to use tablet computers to screen for depression. This plan included decisions about the workflow, materials needed, the staff involved in the process, and the location of screening. To ensure rapid feedback, the research team collected field notes and interviewed stakeholders (PSAs, the practice manager, MAs, the physician, and patients) involved in the pilot. Field note templates and qualitative interview guides are provided in Additional files 1 and 2 respectively. Qualitative interviews were transcribed verbatim in real-time using shorthand or on a laptop to ensure accuracy. The research team also documented whether the PHQ-2 was administered.

Step 2— evaluate and review

Immediately after a 5-h shift of rapid prototyping, the research team met to review the findings. Field notes and interviews were read aloud together and synthesized to eliminate redundancies and ascertain discrepancies. Qualitative data were analyzed using a rapid immersion/crystallization approach [36, 60,61,62]. BSL and ACF, who had been extensively immersed in the experience, developed impressionistic summaries of what they learned. Researchers’ holistic impressions of the experience were crystallized through discussion and written documentation. To ensure systematicity, the research team also recorded key features of each rapid prototyping cycle: (1) a summary of the workflow design; (2) screening results; (3) workflow successes; (4) workflow challenges; and (5) a summary of the changes to be tested in the next cycle.

Step 3 — refine and iterate

After determining the necessary workflow changes, the research team planned to refine the tablet screening process. This process sometimes involved writing scripts for the PSAs presenting tablets to patients to ensure that patients received uniform rationale about the PHQ-2. Other times, this involved placing laminated sheets with screenshots to guide MAs and the clinician to find the PHQ-2 depression screening data in their version of the electronic health record. The research team communicated with the practice ahead of the shift to ensure the changes were acceptable. The new iteration was then tested in the subsequent shift. In order to be able to directly evaluate whether the specific iteration of the strategy was superior to the previous cycle’s, attributes were not modified if they did not present challenges. This process repeated for each cycle.


Qualitative data

Field notes and interview transcripts from each cycle of the rapid prototyping process were collected and analyzed to iteratively improve the pilot process.

Quantitative data

The primary outcomes for the pilot were PHQ-2 and PHQ-9 screening rates and PHQ-2 follow-up. According to CMS in 2019, follow-up to the PHQ-2 (if the score is greater than 2) is considered complete if one of the following actions is taken: (1) the patient completes a more extended depression questionnaire (i.e., PHQ-9) or suicide assessment; or (2) the primary care clinician refers the patient to a mental health clinician; or (3) the clinician prescribes depression medications; or (4) the clinician documents a depression follow-up plan; or (5) the clinician documents a depression diagnosis.

Depression screening data were extracted from Epic©. Patient eligibility for the screener, PHQ-2 scores, PHQ-9 scores, medication list, medical diagnoses, referrals, and patient notes were extracted from Epic© for intervention and baseline shifts. To ensure a fair comparison and avoid any potential confounds related to the timing of the visit, baseline shift data were collected from the same time window and same physician as the intervention shifts. Because tablets automatically triggered the PHQ-9 at a lower score (a PHQ-2 score > 1) than CMS requirements and the usual care practice (a PHQ-2 score > 2), PHQ-9 and follow-up rates were compared statistically based on CMS requirements (PHQ-2 score > 2).


Qualitative results

Table 3 displays the rapid prototyping process results from each day of piloting; Additional file 3 narratively describes these detailed results. Fig. 2 displays the modifications of each rapid prototyping cycle and tablet administration results.

Table 3 Rapid Prototyping Process for Tablets
Fig. 2
figure 2

Rapid Prototyping Changes and Tablet Administration. Note. Rapid prototyping cycle icons are present in the upper window of the figure if modifications were made to this aspect of the piloting strategy based on decisions from the previous cycle. It should be noted that during piloting, though not all patients were administered the questionnaire via tablet, all patients completed the PHQ-2 (with the exception of one patient during the first cycle)

Broadly, rapid prototyping revealed that tablet computers were acceptable and feasible to most stakeholders (the physician, the practice manager, MAs, and patients) involved in depression screening. However, PSAs found the additional responsibilities of checking patients in, handing off the tablets, and introducing the screener to patients to be somewhat disruptive to their workflow. In addition, anywhere between 1 and 3 patients during the first four five-hour shifts did not complete the questionnaire via tablet either due to the patient’s physical limitations, because the workflow was not yet mastered by stakeholders, or because there was not sufficient time to complete the depression screener via tablet. These patients completed the PHQ-2 verbally with the MA (i.e., through usual care). After several refinements were made during the rapid prototyping process, the process of depression screening via tablet improved and the PSAs eventually described that the workflow adjustments were “smooth sailing.” During the final shift, all patients were screened via tablet and stakeholders considered the pilot a “success.” Stakeholders made several recommendations for how to scale up the process across PennMedicine primary care practices, including transformations to Epic© and standardization of depression screening workflows across the health system.

Quantitative results

Overall, the rapid prototyping process resulted in comparable PHQ-2 screening rates to usual care (given that the practice participating in the pilot already screened all eligible patients). However, tablets significantly increased PHQ-9 screening rates. Follow-up between usual care and piloting days were comparable. Z test results for non-significant findings should be viewed in light of the fact that the pilot study was not adequately powered to detect a statistical difference.

On baseline days 30 out of 30 (100%) eligible patients were administered the PHQ-2. On intervention days 35 out of 36 (97%) of eligible patients were administered the PHQ-2. According to field notes, the patient whose PHQ-2 data were not recorded was given a tablet, indicating that responses were not synced with Epic©. Given that the practice participating in the pilot was highly compliant with PHQ-2 screening guidelines, differences between baseline and intervention days were not significant (z = 0.92, p = 0.36).

For those patients whose PHQ-2 score was positive (a score > 2), PHQ-9 screening was evaluated. On baseline days, 1 out of 3 (33%) of eligible patients were administered the PHQ-9. On intervention days, 6 out of 6 (100%) of eligible patients were administered the PHQ-9. Differences between baseline and intervention days were significant (z = − 2.27, p = 0.02).

Using the more inclusive definition of PHQ-2 follow-up per CMS guidelines (described above), follow-up rates were extracted from Epic©. On baseline days, 2 out of 3 patients (67%) received follow-up after a positive PHQ-2 screen. On intervention days, 6 out of 6 patients (100%) received follow-up after a positive PHQ-2 screen. Differences between baseline and intervention days were not significant (z = − 1.50, p = 0.13).


Major findings

Our pilot study employed participatory and rapid implementation methods to increase universal depression screening in Penn Medicine primary care practices. First, we employed an innovation tournament to gather ideas from stakeholders (leaders from Primary Care and Psychiatry, clinicians, and staff) about how to increase depression screening in primary care. Second, a panel of expert stakeholders and scientists deliberated and voted on the best innovation tournament idea to pilot. The panel determined that rather than the usual care practice of verbal PHQ-2 administration by MAs, the research team should pilot an electronic self-report method. Third, we piloted this winning idea in one primary care practice with one physician over 5 cycles. Using an innovative rapid implementation method called rapid prototyping, we designed and refined a strategy to screen patients for depression with tablet computers. Our pilot study found that using a tablet for patient self-administration of depression questionnaires was feasible and acceptable across stakeholder groups, though PSAs expressed concern about the additional responsibilities resulting from these workflow changes. Despite significant workflow changes, in our limited sample PHQ-2 screening rates using tablets were comparable to usual care screening rates given that the practice/physician we worked with already accomplished universal (i.e., 100%) screening rates. Moreover, PHQ-9 screening rates were significantly higher using the tablet. Follow-up rates for usual care and intervention days were comparable. PHQ-9 screening rates were likely higher because, unlike usual care at the practice that tasked MAs with following up on the PHQ-2, tablet computers automatically triggered and generated the PHQ-9 screener if a patient screened positive on the PHQ-2. Whereas usual care necessitated continuous and deliberate human intervention, the app on the tablet computer generated the PHQ-9 by default.

Strengths and weaknesses of participatory and rapid implementation approaches

This case study revealed the advantages and disadvantages of using participatory and rapid implementation approaches to strategy design. This study is unique for its inclusion of a variety of stakeholder perspectives across the three study phases. As a consequence of health system incentives for physicians to see more patients and meet more demands, more responsibilities—formerly within the sole purview of physicians—are being shifted onto non-physician and often non-clinical workers [63]. The few studies that have sought the perspectives of non-physicians have discovered that health system transformations rarely incorporate their often diverging concerns [64, 65].

Consistent with this work, the panel discussion of expert stakeholders and scientists revealed that different stakeholder groups have different knowledge and priorities about screening that need to be addressed by health systems [66,67,68]. For example, the panelists felt that the reframing ideas by the diverse innovation tournament participants showcased a misunderstanding about the rationale for screening. The panelists also discussed some of the drawbacks of task-shifting through the employment of non-clinical staff and MAs for screening. The panel suggested that clinicians, staff, and patients may require additional education to better understand the rationale for depression screening. They also discussed that not all clinicians feel equipped or comfortable with providing depression follow-up. These factors influence the accuracy of the depression screen and may indirectly impact screening rates [66, 69,70,71,72,73]. Field notes and qualitative interviews from the rapid prototyping phase again displayed conflicting stakeholder perspectives. Though most patients found depression screening on the tablet acceptable, a contingent could or would not use the tablets for various reasons (e.g., physical ability, digital literacy, concerns about the impersonality of the questionnaire). PSAs felt that they were being given more responsibilities that were disruptive to the workflow and one expressed not understanding why depression screening was a priority, whereas clinical staff found the tablet screening acceptable and feasible.

These diverging stakeholder views may reflect differences in responsibilities, interest, knowledge, and stigma related to mental health issues. Health systems must incorporate and address these different perspectives in future implementation efforts to ensure their success. For example, future efforts using tablets to screen for depression might consider developing a back-up plan for patients who dislike or cannot use a self-report electronic screener and consider reducing PSA burden. In addition, though the clinical staff in the rapid prototyping phase were eager to expand these strategies to other mental health conditions and to standardize the physician-administered suicide assessment protocol, based on our prior work in the PIC program, some clinicians prefer to hand off psychiatric and suicide assessments to the PIC mental health provider [40]. Implementation strategies for depression follow-up, including assessing suicide risk, that can assist staff and clinicians with varying levels of comfort treating psychiatric conditions may be needed as health systems integrate mental and physical health services.

Another strength of the rapid implementation approach was the ability to get feedback quickly from stakeholders who were invested in the quality improvement effort. The innovation tournament was an efficient method to reach engaged stakeholders in the PIC program and the panel meeting enabled the research team to promptly evaluate the feasibility of the innovation tournament ideas. Rapid prototyping allowed the research team to receive immediate feedback on whether the stakeholder-proposed changes were acceptable, feasible, and increased screening rates. In our pilot study, several of the stakeholder-proposed changes were successful, and others were not (e.g., conducting the PHQ-2 in the exam room). We were able to present the data to stakeholders and rapidly change course when there was agreement that the current iteration of the strategy was ineffective. Altogether, our work suggests that there are significant advantages for researchers to employ these methods. Rapid experimentation that engages the entire workforce involved in health system transformations is the necessary next step to realize the promise of a learning health system [39, 56].

There were also several weaknesses to these participatory and rapid implementation methods. Most significantly, the stakeholders engaged in the process (from the innovation tournament to the rapid prototyping phase) were heavily invested in system change, concerned about increasing depression screening, and motivated to voice their input in the research process. These stakeholders were not representative. The stakeholders that chose to participate in the tournament tended to provide highly elaborate responses, which accords with the literature suggesting that the most engaged stakeholders who are eager to share their ideas will submit ideas to the tournament [46, 47, 51, 74].

Stakeholder self-selection was perhaps most evident in the high practice screening rates at baseline. We worked with a practice/physician for the pilot that already had high PHQ-2 screening rates so the practice felt assured they would not risk compromising patient care during the pilot. However, similar to other health systems, depression screening rates across Penn Medicine primary care practices are far more variable, ranging from 10 to 90% [69, 72, 75, 76]. The level of engagement and motivation in the process is not surprising given our approach, which relies on innovators (i.e., those interested in and willing to engage in health system experimentation). These methods are likely best suited for the early stages of innovation diffusion [77]. Though the innovation tournament fielded ideas from stakeholders across PIC practices, it is important to test whether the tablet strategy will generalize to other primary care practices where depression screening and stakeholder investment may be lower. It is possible that at other practices where there is less buy-in or motivation, social incentives and peer comparison interventions would be needed to complement these efforts [78, 79]. Going forward, leveraging the leadership and embeddedness of implementation champions will be crucial to effecting large-scale change in the entire health system [80].

Study limitations and future directions

Beyond the limitations of the methods, our study also has several specific limitations. For one, the results of the innovation tournament were heavily influenced by the framing of the Big Idea prompt. The research team intended for respondents to generate innovative ways to increase PHQ-2/PHQ-9 use, yet many respondents focused on ways to improve depression screening that did not involve these specific measures. Though we learned much about stakeholders’ attitudes and knowledge about depression screening from the innovative tournament results, which informed our investigation, in future research we would likely focus the Big Idea prompt on the PHQ-2/PHQ-9 to generate more targeted responses. A related limitation was the absence of patient participants in the innovation tournament. Given how much we learned about diverging stakeholder perspectives throughout the pilot study, it is important to understand how patients would prefer to be screened especially because verbal administration of the PHQ-2 (the current screening protocol at many of the primary care practices) is not psychometrically valid. Moreover, patient-centered depression care is associated with improved treatment outcomes [81].

Third, we were not able to scale up the pilot beyond one physician in one practice due to the onset of the COVID-19 pandemic, which halted research. Though our study yielded important lessons on innovative participatory and rapid methods to improve implementation design, depressing screening rates from the pilot study should be appropriately contextualized in light of the small sample. Researchers are planning to conduct further examinations in the health system to scale up this experimentation when physical distancing guidelines are no longer in place. One potential benefit from the COVID-19 pandemic was the rapid switch to telehealth, during which health systems became well-versed in virtual mental health screening and risk assessment. Health systems may now be less hesitant to screen patients ahead of their primary care visits through confidential patient portals or via other methods due to liability issues regarding the endorsement of suicidality on the PHQ-9. Increasing the availability of options for patients to complete the PHQ-2 and PHQ-9, including ahead of their primary care visit, would likely improve depression screening rates and should be explored by health systems.

Another limitation of our study is that due to the preliminary nature of the investigation, permanent modifications to the electronic health record that would have affected the whole health system were not feasible. For example, through Epic© it is possible to automate the assignment of the depression questionnaire to avoid PSA burden. Yet due to the preliminary nature of our study, the research team was not able to execute this change. During the rapid prototyping process, PSAs found assigning the PHQ-2 to be disruptive and burdensome. This particular challenge may not generalize to a large-scale implementation initiative where such changes would be permitted.


Employing participatory and rapid implementation methods to increase depression screening in primary care is effective at engaging stakeholders, generating investment in the project, and improving the design of implementation strategies. Our findings reveal that involving all stakeholders impacted by these implementation efforts can provide important guidance for how to effect large-scale change in the health system. Health systems and payers must attend to the diversity of perspectives from all stakeholders affected by transformations in healthcare provision. In particular, while electronic depression screening is considered a priority for many stakeholders, our pilot study found that a contingent of patients could not complete this method and that PSAs were saddled with additional responsibilities that they perceived to be disruptive. To accurately capture depression rates, screening practices must allow for implementation flexibility. More broadly, our preliminary results suggest that these methods can improve universal depression screening in primary care practices. We were able to leverage participatory and rapid approaches to design implementation strategies to improve screening at relatively low cost, with sustained stakeholder engagement and buy-in, and without disrupting workflows permanently. Health systems committed to implementing evidence-based practices beyond depression screening stand to gain from these rapid, stakeholder-centered design methods.

Availability of data and materials

The datasets generated and analyzed during the current study are not publicly available due to the sensitive and identifiable nature of qualitative data and small pilot study data, but de-identified data are available from the corresponding author on reasonable request.



Penn Integrated Care


Medical Assistant


Patient Support Assistant


Patient Health Questionnaire


The Centers for Medicare & Medicaid Services


  1. Kessler RC, Chiu WT, Demler O, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):617–27.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Kessler RC. The costs of depression. Psychiatr Clin. 2012;35(1):1–14.

    Google Scholar 

  3. Brody DJ, Pratt LA, Hughes JP. Prevalence of depression among adults aged 20 and over: United States, 2013-2016; 2018.

    Google Scholar 

  4. Weinberger AH, Gbedemah M, Martinez AM, Nash D, Galea S, Goodwin RD. Trends in depression prevalence in the USA from 2005 to 2015: widening disparities in vulnerable groups. Psychol Med. 2018;48(8):1308.

    Article  CAS  PubMed  Google Scholar 

  5. Ettman CK, Abdalla SM, Cohen GH, Sampson L, Vivier PM, Galea S. Prevalence of depression symptoms in US adults before and during the COVID-19 pandemic. JAMA Netw Open. 2020;3(9):e2019686.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Lim GY, Tam WW, Lu Y, Ho CS, Zhang MW, Ho RC. Prevalence of depression in the community from 30 countries between 1994 and 2014. Sci Rep. 2018;8(1):1–10.

    Article  Google Scholar 

  7. DiMatteo MR, Lepper HS, Croghan TW. Depression is a risk factor for noncompliance with medical treatment: meta-analysis of the effects of anxiety and depression on patient adherence. Arch Intern Med. 2000;160(14):2101–7.

    Article  CAS  PubMed  Google Scholar 

  8. Katon WJ. Epidemiology and treatment of depression in patients with chronic medical illness. Dialogues Clin Neurosci. 2011;13(1):7.

    Article  PubMed  Google Scholar 

  9. Ahrnsbrak R, Bose J, Hedden SL, Lipari RN, Park-Lee E. Key substance use and mental health indicators in the United States: results from the 2016 National Survey on drug use and health. Rockville: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; 2017.

    Google Scholar 

  10. Cuijpers P, van Straten A, Andersson G, van Oppen P. Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. J Consult Clin Psychol. 2008;76(6):909.

    Article  PubMed  Google Scholar 

  11. Cuijpers P, Andersson G, Donker T, van Straten A. Psychological treatment of depression: results of a series of meta-analyses. Nordic J Psychiatry. 2011;65(6):354–64.

    Article  Google Scholar 

  12. Driessen E, Cuijpers P, de Maat SC, Abbass AA, de Jonghe F, Dekker JJ. The efficacy of short-term psychodynamic psychotherapy for depression: a meta-analysis. Clin Psychol Rev. 2010;30(1):25–36.

    Article  PubMed  Google Scholar 

  13. Chekroud AM, Foster D, Zheutlin AB, Gerhard DM, Roy B, Koutsouleris N, et al. Predicting barriers to treatment for depression in a US national sample: a cross-sectional, proof-of-concept study. Psychiatr Serv. 2018;69(8):927–34.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Simon GE, Fleck M, Lucas R, Bushnell DM, Group L. Prevalence and predictors of depression treatment in an international primary care study. Am J Psychiatr. 2004;161(9):1626–34.

    Article  PubMed  Google Scholar 

  15. Sheehan DV. Depression: underdiagnosed, undertreated, underappreciated. Manag Care. 2004;13(6 Suppl Depression):6–8.

    PubMed  Google Scholar 

  16. Waitzfelder B, Stewart C, Coleman KJ, Rossom R, Ahmedani BK, Beck A, et al. Treatment initiation for new episodes of depression in primary care settings. J Gen Intern Med. 2018;33(8):1283–91.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Rosmond R. Obesity and depression: same disease, different names? Med Hypotheses. 2004;62(6):976–9.

    Article  PubMed  Google Scholar 

  18. Carney RM, Freedland KE. Depression and coronary heart disease. Nat Rev Cardiol. 2017;14(3):145.

    Article  PubMed  Google Scholar 

  19. Arnow BA, Hunkeler EM, Blasey CM, Lee J, Constantino MJ, Fireman B, et al. Comorbid depression, chronic pain, and disability in primary care. Psychosom Med. 2006;68(2):262–8.

    Article  PubMed  Google Scholar 

  20. Roca M, Gili M, Garcia-Garcia M, Salva J, Vives M, Campayo JG, et al. Prevalence and comorbidity of common mental disorders in primary care. J Affect Disord. 2009;119(1–3):52–8.

    Article  CAS  PubMed  Google Scholar 

  21. Gaynes BN, Rush AJ, Trivedi MH, Wisniewski SR, Balasubramani GK, Spencer DC, et al. Major depression symptoms in primary care and psychiatric care settings: a cross-sectional analysis. Ann Fam Med. 2007;5(2):126–34.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Mainous AG, Baker R, Love MM, Gray DP, Gill JM. Continuity of care and trust in one’s physician: evidence from primary care in the United States and the United Kingdom. Fam Med. 2001;33(1):22–7.

    PubMed  Google Scholar 

  23. VanBuskirk KA, Wetherell JL. Motivational interviewing with primary care populations: a systematic review and meta-analysis. J Behav Med. 2014;37(4):768–80.

    Article  PubMed  Google Scholar 

  24. Unützer J, Park M. Strategies to improve the management of depression in primary care. Prim Care. 2012;39(2):415–31.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Katon WJ, Lin EH, Von Korff M, Ciechanowski P, Ludman EJ, Young B, et al. Collaborative care for patients with depression and chronic illnesses. N Engl J Med. 2010;363(27):2611–20.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Kessler R. Integrated behavioral health in primary care: step-by-step guidance for assessment and intervention. Fam Syst Health. 2010;28(4):389–90.

    Article  Google Scholar 

  27. Unützer J, Katon WJ, Fan M-Y, Schoenbaum MC, Lin EH, Della Penna RD, et al. Long-term cost effects of collaborative care for late-life depression. Am J Manag Care. 2008;14(2):95.

    PubMed  PubMed Central  Google Scholar 

  28. Siu AL, Bibbins-Domingo K, Grossman DC, Baumann LC, Davidson KW, Ebell M, et al. Screening for depression in adults: US preventive services task force recommendation statement. Jama. 2016;315(4):380–7.

    Article  CAS  PubMed  Google Scholar 

  29. Calonge N, Petitti DB, DeWitt TG, Dietrich AJ, Gordis L, Gregory KD, et al. Screening for depression in adults: US preventive services task force recommendation statement. Ann Intern Med. 2009;151(11):784–92.

    Article  Google Scholar 

  30. Jacques L, Jensen TS, Schafer J, Caplan S, Schott L. Decision memo for screening for depression in adults: The Centers for Medicare & Medicaid Services; 2011. [cited 2021 Feb 17]. Report No.: CAG00425N. Available from:

    Google Scholar 

  31. Löwe B, Kroenke K, Gräfe K. Detecting and monitoring depression with a two-item questionnaire (PHQ-2). J Psychosom Res. 2005;58(2):163–71.

    Article  PubMed  Google Scholar 

  32. Colligan EM, Cross-Barnet C, Lloyd JT, McNeely J. Barriers and facilitators to depression screening in older adults: a qualitative study. Aging Ment Health. 2020;24(2):341–8.

    Article  PubMed  Google Scholar 

  33. Akincigil A, Matthews EB. National rates and patterns of depression screening in primary care: results from 2012 and 2013. Psychiatr Serv. 2017;68(7):660–6.

    Article  PubMed  Google Scholar 

  34. Samples H, Stuart EA, Saloner B, Barry CL, Mojtabai R. The role of screening in depression diagnosis and treatment in a representative sample of US primary care visits. J Gen Intern Med. 2020;35(1):12–20.

    Article  PubMed  Google Scholar 

  35. Minkler M, Salvatore AL, Chang C. Participatory approaches for study design and analysis in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. 2nd ed. Oxford: Oxford University Press; 2018. p. 175–90.

    Google Scholar 

  36. Palinkas LA, Zatzick D. Rapid assessment procedure informed clinical ethnography (RAPICE) in pragmatic clinical trials of mental health services implementation: methods and applied case study. Adm Policy Ment Health Ment Health Serv Res. 2019;46(2):255–70.

    Article  Google Scholar 

  37. Smith J, Rapport F, O’Brien TA, Smith S, Tyrrell VJ, Mould EVA, et al. The rise of rapid implementation: a worked example of solving an existing problem with a new method by combining concept analysis with a systematic integrative review. BMC Health Serv Res. 2020;20(1):449.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Austrian J, Mendoza F, Szerencsy A, Fenelon L, Horwitz LI, Jones S, et al. Applying a/B testing to clinical decision support: rapid randomized controlled trials. J Med Internet Res. 2021;23(4):e16651.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Horwitz LI, Kuznetsova M, Jones SA. Creating a learning health system through rapid-cycle, randomized testing. N Engl J Med. 2019;381(12):1175–9.

    Article  PubMed  Google Scholar 

  40. Wolk CB, Last BS, Livesey C, Oquendo MA, Press MJ, Mandell DS, et al. Addressing common challenges in the implementation of collaborative care for mental health: the Penn integrated care program. Ann Fam Med. 2021;19(2):148–56.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Adaji A. Integrated care: a disruptive innovation for extending psychiatric expertise to primary care practices. Mayo Clin Proc Innov Qual Outc. 2018;2(2):99–102.

    Article  Google Scholar 

  42. Unützer J, Katon W, Callahan CM, Williams JW Jr, Hunkeler E, Harpole L, et al. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. Jama. 2002;288(22):2836–45.

    Article  PubMed  Google Scholar 

  43. Bower P, Gilbody S, Richards D, Fletcher J, Sutton A. Collaborative care for depression in primary care: making sense of a complex intervention: systematic review and meta-regression. Br J Psychiatry. 2006;189(6):484–93.

    Article  PubMed  Google Scholar 

  44. Muntingh AD, van der Feltz-Cornelis CM, van Marwijk HW, Spinhoven P, van Balkom AJ. Collaborative care for anxiety disorders in primary care: a systematic review and meta-analysis. BMC Fam Pract. 2016;17(1):1–15.

    Article  Google Scholar 

  45. Centers for Medicare and Medicaid Services. Preventive care and screening: screening for depression and follow-up plan. Baltimore: United States Department of Health and Human Services; 2019. [cited 2021 Aug 21]. (Prevention, Treatment, and Management of Mental Health). Report No.: 134 (NQF 041). Available from:

    Google Scholar 

  46. Terwiesch C, Ulrich KT. Innovation tournaments: creating and selecting exceptional opportunities: Harvard Business Press; 2009.

    Google Scholar 

  47. Terwiesch C, Mehta SJ, Volpp KG. Innovating in health delivery: the Penn medicine innovation tournament: Healthcare. Elsevier; 2013. p. 37–41.

    Google Scholar 

  48. Hirshberg ABM, Vandertuyn M, Mahraj K. Rapid-cycle innovation testing of text-based monitoring for management of postpartum hypertension. JCOM. 2017;24(2):77–85.

  49. Sriram V, Jenkinson C, Peters M. Using rapid cycle tests of change to develop the Carers assistive technology experience questionnaire: a cognitive interview study in the UK. BMJ Open. 2021;11(3):e042361.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Morgan J, Wang R. Tournaments for ideas. Calif Manag Rev. 2010;52(2):77–97.

    Article  Google Scholar 

  51. Stewart RE, Williams N, Byeon YV, Buttenheim A, Sridharan S, Zentgraf K, et al. The clinician crowdsourcing challenge: using participatory design to seed implementation strategies. Implement Sci. 2019;14(1):63.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Penn Medicine Center for Healthcare Innovation. Innovation tournaments. Philadelphia. Available from: Accessed 24 Feb 2021.

  53. Woike BA. Content coding of open-ended responses. In: Robins RW, Fraley CR, Kreuger RF, editors. Handbook of research methods in personality psychology. New York: Guilford Press; 2007. p. 292–307.

    Google Scholar 

  54. Gehrig M. Instant-runoff voting [internet]. St. Paul: Minnesota House of Representatives; 2007. (Information Brief). Available from:

    Google Scholar 

  55. Thomke S. Enlightened experimentation: the new imperative for innovation. Harv Bus Rev. 2001;79(2):66–75.

    CAS  PubMed  Google Scholar 

  56. Asch DA, Rosin R. Innovation as discipline, not fad. N Engl J Med. 2015;373(7):592–4.

    Article  CAS  PubMed  Google Scholar 

  57. Lemke DS, Fielder EK, Hsu DC, Doughty CB. Improved team performance during pediatric resuscitations after rapid cycle deliberate practice compared with traditional debriefing: a pilot study. Pediatr Emerg Care. 2019;35(7):480–6.

    PubMed  Google Scholar 

  58. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan–do–study–act method to improve quality in healthcare. BMJ Qual Saf. 2014;23(4):290.

    Article  PubMed  Google Scholar 

  59. Rusch FR, Kazdin AE. Toward a methodology of withdrawal designs for the assessment of response maintenance. J Appl Behav Anal. 1981;14(2):131–40.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Borkan J. Immersion/crystallization. In: Crabtree BF, Miller WL, editors. Doing qualitative research. 2nd ed. Thousand Oaks: Sage Publications, Inc; 1999. p. 179–94.

    Google Scholar 

  61. Miller WL, Crabtree BF. Primary care research: a multimethod typology and qualitative roadmap. In: Miller WL, Crabtree BF, editors. Doing qualitative research. 1st ed. Newbury Park: Sage Publications, Inc.; 1992. p. 3–28.

    Google Scholar 

  62. Miles MB, Huberman AM, editors. Qualitative data analysis: an expanded sourcebook. 2nd ed. Thousand Oaks: Sage; 1994.

    Google Scholar 

  63. Martínez-González NA, Tandjung R, Djalali S, Rosemann T. The impact of physician–nurse task shifting in primary care on the course of disease: a systematic review. Hum Resour Health. 2015;13(1):55.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Dlamini-Simelane T, Moyer E. Task shifting or shifting care practices? The impact of task shifting on patients’ experiences and health care arrangements in Swaziland. BMC Health Serv Res. 2017;17(1):20.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Padmanathan P, De Silva MJ. The acceptability and feasibility of task-sharing for mental healthcare in low and middle income countries: a systematic review. Soc Sci Med. 2013;97:82–6.

    Article  PubMed  Google Scholar 

  66. Pilipenko N, Vivar-Ramon C. Depression screening in a primary care setting: examining perceptions and experiences of English- and Spanish-speaking patients. J Clin Psychol Med Settings. 2020;28:584–95.

  67. Kahalnik F, Sanchez K, Faria A, et al. Improving the identification and treatment of depression in low income primary care clinics: a qualitative study of providers in the VitalSign6 program. Int J Qual Health Care. 2019;31(1):57–63.

  68. Poghosyan L, Norful AA, Ghaffari A, George M, Chhabra S, Olfson M. Mental health delivery in primary care: the perspectives of primary care providers. Arch Psychiatr Nurs. 2019;33(5):63–7.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Henry TL, Schmidt S, Lund MB, Haynes T, Ford D, Egwuogu H, et al. Improving depression screening in underserved populations in a large urban academic primary care center: a provider-centered analysis and approach. Am J Med Qual. 2020;35(4):315–22.

    Article  PubMed  Google Scholar 

  70. Thombs BD, Ziegelstein RC. Does depression screening improve depression outcomes in primary care? BMJ. 2014;348:g1253.

  71. Mojtabai R. Universal depression screening to improve depression outcomes in primary care: sounds good, but where is the evidence? Am Psychiatric Assoc; 2017.

    Google Scholar 

  72. Pfoh ER, Janmey I, Anand A, Martinez KA, Katzan I, Rothberg MB. The impact of systematic depression screening in primary care on depression identification and treatment in a large health care system: a cohort study. J Gen Intern Med. 2020;35:3141–7.

    Article  PubMed  PubMed Central  Google Scholar 

  73. O’Connor EA, Whitlock EP, Beil TL, Gaynes BN. Screening for depression in adult patients in primary care settings: a systematic evidence review. Ann Intern Med. 2009;151(11):793–803.

    Article  PubMed  Google Scholar 

  74. Darragh D. Response rate of crowdsourcing challenges through the your big idea platform; 2018.

    Google Scholar 

  75. Harder VS, Barry SE, French S, Consigli AB, Frankowski BL. Improving adolescent depression screening in pediatric primary care. Acad Pediatr. 2019;19(8):925–33.

    Article  PubMed  Google Scholar 

  76. Bose J, Zeno R, Warren B, Sinnott LT, Fitzgerald EA. Implementation of Universal Adolescent Depression Screening: Quality Improvement Outcomes. J Pediatr Health Care. 2021;35(3):270–7.

  77. Rogers EM. Diffusion of preventive innovations. Addict Behav. 2002;27(6):989–93.

    Article  PubMed  Google Scholar 

  78. Lagarde M, Blaauw D. Physicians’ responses to financial and social incentives: a medically framed real effort experiment. Soc Sci Med. 2017;179:147–59.

    Article  PubMed  Google Scholar 

  79. Last BS, Buttenheim AM, Timon CE, Mitra N, Beidas RS. Systematic review of clinician-directed nudges in healthcare contexts. BMJ Open. 2021;11(7):e048801.

    Article  PubMed  PubMed Central  Google Scholar 

  80. Bonawitz K, Wetmore M, Heisler M, Dalton VK, Damschroder LJ, Forman J, et al. Champions in context: which attributes matter for change efforts in healthcare? Implement Sci. 2020;15(1):1–10.

    Article  Google Scholar 

  81. Rossom RC, Solberg LI, Vazquez-Benitez G, Crain AL, Beck A, Whitebird R, et al. The effects of patient-centered depression care on patient satisfaction and depression remission. Fam Pract. 2016;33(6):649–55.

    Article  PubMed  PubMed Central  Google Scholar 

Download references


We are grateful for the support and partnership of the many leaders, clinicians, staff, and patients at PennMedicine who contributed to this research.


Funding for this study was provided by grants from the National Institute of Mental Health (P50 MH 113840, Beidas, Buttenheim, Mandell, MPI). BSL also receives funding support from the National Science Foundation Graduate Research Fellowship Program (DGE-1321851).

Author information

Authors and Affiliations



BSL conceived of and designed the research study; acquired and analyzed the data; interpreted the data; drafted the manuscript and substantially revised it. AMB analyzed the data; interpreted the data; and substantially revised the manuscript. ACF acquired and analyzed the data; interpreted the data; and substantially revised the manuscript. CL helped design the research study; acquired and analyzed the data; and substantially revised the manuscript. JJ helped design the research study; acquired the data; and substantially revised the manuscript. RES analyzed the data and substantially revised the manuscript. MR acquired the data and substantially revised the manuscript. MP substantially revised the manuscript. CBW helped conceive of and design the research study; analyzed the data; interpreted the data; and substantially revised the manuscript. RSB helped conceive of and design the research study; interpreted the data; and substantially revised the manuscript. All authors approved the submitted version; have agreed to be accountable for the contributions; and attest to the accuracy and integrity of the work, even aspects for which the authors were not personally involved.

Corresponding author

Correspondence to Briana S. Last.

Ethics declarations

Ethics approval and consent to participate

All procedures were reviewed and approved by the University of Pennsylvania Institutional Review Boards and are in accordance with the Declaration of Helsinki. The protocol (832617) was deemed exempt by the University of Pennsylvania Institutional Review Board for the minimal risk it posed to human subjects, and therefore the need for informed consent to participate was waived.

Consent for publication

Not applicable.

Competing interests

RSB reports royalties from Oxford University Press, has received consulting fees from the Camden Coalition of Healthcare Providers, currently consults for United Behavioral Health, and sits on the scientific advisory committee for Optum Behavioral Health. The other authors declare no financial or non-financial competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

 Field Notes Template.

Additional file 2.

 Rapid Qualitative Interview Template.

Additional file 3.

 Rapid Prototyping Results.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Last, B.S., Buttenheim, A.M., Futterer, A.C. et al. A pilot study of participatory and rapid implementation approaches to increase depression screening in primary care. BMC Fam Pract 22, 228 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: