Published on 10.09.18 in Vol 10, No 3 (2018): Jul-Sept
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/10655, first published Mar 30, 2018.
Phase I of the Detecting and Evaluating Childhood Anxiety and Depression Effectively in Subspecialties (DECADES) Study: Development of an Integrated Mental Health Care Model for Pediatric Gastroenterology
Background: Children with gastrointestinal symptoms have a very high rate of anxiety and depression. Rapid identification of comorbid anxiety and depression is essential for effective treatment of a wide variety of functional gastrointestinal disorders.
Objective: The objective of our study was to determine patient and parent attitudes toward depression, anxiety, and mental health screening during gastroenterology (GI) visits and to determine patient and parent preferences for communication of results and referral to mental health providers after a positive screen.
Methods: We augmented standard qualitative group session methods with patient-centered design methods to assess patient and parent preferences. We used a variety of specific design methods in these sessions, including card sorting, projective methods, experience mapping, and constructive methods.
Results: Overall, 11 families (11 patients and 14 parents) participated in 2 group sessions. Overall, patients and their parents found integrated mental health care to be acceptable in the subspecialty setting. Patients’ primary concerns were for the privacy and confidentiality of their screening results. Patients and their parents emphasized the importance of mental health services not interfering with the GI visit and collaboration between the GI physician, psychologist, and primary care provider.
Conclusions: Patients and their families are open to integrated mental health care in the pediatric subspecialty clinic. The next phase of the DECADES study will translate patient and parent preferences into an integrated mental health care system and test its efficacy in the pediatric GI office.
J Participat Med 2018;10(3):e10655
The Detecting and Evaluating Childhood Anxiety and Depression Effectively in Subspecialties (DECADES) study seeks to develop a model for integrated mental health care that is acceptable to pediatric gastroenterology patients and their families and to compare this model of care with standard care. The first phase of this study sought to develop an integrated mental health care model that is acceptable to both gastroenterology patients and their parents by exploring their attitudes and preferences using qualitative methods augmented by patient-centered design methods.
Depression and anxiety are 2 of the most common disorders occurring during childhood and adolescence [- ], but they frequently remain unrecognized or untreated [ - ]. Rates of depression and anxiety are significantly higher in children with chronic illnesses [ , ], including gastrointestinal disorders [ - ], than those in the general population. Furthermore, children with depression or anxiety are far more likely to have somatic complaints and greater utilization of subspecialty care, especially in gastroenterology [ - ]. Efforts to recognize and treat mental health problems in children with chronic medical illness, such as gastrointestinal disorders, have been shown to improve adherence to therapy and other clinical outcomes [ , ]. More importantly, improving these mental health concerns may improve the outcomes patients care about the most.
Validated tools exist to screen for anxiety and depression in children, including the Screen for Childhood Anxiety Related Emotional Disorders (SCARED)  and the Patient Health Questionnaire (PHQ) [ ]. Despite the established importance of depression and anxiety in the gastrointestinal health of children, few data-driven studies exist that describe the identification and management of anxiety and depression by pediatric gastroenterologists and how families or patients view the subspecialty office as the setting to detect or care for mental illness.
Patient engagement is a process by which patients, families, and health professionals work in partnership to improve health care , and it is a process for developing patient-centered care. When patients are engaged in the development of new models of care, it improves recruitment and retention to randomized clinical trials, and clinical care is more meaningful to patients and their families [ ].
This study describes the development of an integrated mental health care model for pediatric gastroenterology as part of the larger DECADES trial. We sought to develop this model of care using patient-centered design methods to augment qualitative methodology and by directly engaging patients and their parents in the design process. The goals of this study are to determine patient and parent attitudes toward depression, anxiety, and mental health screening during gastroenterology visits and their preferences for communication of results and referral to mental health providers after a positive screen.
This study involves a series of qualitative group sessions in which standard qualitative focus group methodology has been augmented by patient-centered design methods. At the end of these sessions, we sought to develop a set of specific, actionable recommendations that could then be used to improve patient-centeredness in a subsequent randomized trial. As noted in, the overall objective of the qualitative phase of the DECADES study was to develop a greater understanding of patient preferences related to mental health screening in a pediatric subspecialty office. This was accomplished by both individual interviews (the subject of a separate manuscript) and group sessions (the subject of this manuscript).
Patients were seeking care in a pediatric gastroenterology clinic and their parents were approached for enrollment in this qualitative study. Inclusion criteria were as follows: age 5-18 years, a parent or guardian who agreed to participate, and no diagnosed cognitive disabilities. Recruitment was conducted in the pediatric gastroenterology outpatient clinic at Riley Children’s Health, part of Indiana University Health, in Indianapolis, IN. The principal investigator or study coordinator recruited all eligible patients. Permission to approach the patient was obtained from the gastroenterologist of record. Both new and established patients were enrolled.
Overview of Group Sessions
Two group sessions were conducted with multiple families. Sessions were facilitated by design research specialists using patient-centered design research methods, which are enumerated below, and are based on established methodology in the design literature. Sessions lasted approximately 90 minutes and were audio recorded and transcribed for analysis. All families were compensated for travel to sessions and given a US $30 gift card.
Group sessions used generative design research activities to engage the patients and their parents in codesigning the integrated mental health process [- ]. Generative design activities tend to be open-ended and allow for a wide range of responses and response types to minimize bias and allow families to be as truthful as possible about their preferences. Sessions began with warm-up activities to encourage participation and collaboration [ ]. We then used the following 2 specific types of generative design: projective methods, which are specifically designed to encourage participants to express their thoughts and feelings, and constructive methods to help with concept development [ ].
Specific Patient-Centered Design Activities During Group Sessions
Question on the Board
The goal of this activity  was to establish participants’ baseline knowledge of anxiety and depression and understand how they express these concepts in their own words, informing how to present a screening tool to patients and their families. Participants were asked to answer the following questions on separate notecards: “What does depression mean to you?” and “What does anxiety mean to you?” Notecards were collected by study staff, and responses were not shared with the group.
The purpose of the card sorting activity  was to identify concerns with the screener to help improve how we delivered the screener to patients. For this activity, families were divided into 2 groups (parents and patients) in separate rooms. Both groups were presented with the same stack of cards. Each card listed an item from brief versions of the SCARED and PHQ, the five-item SCARED-5 [ ] or the four-item Patient Health Questionnaire [ ]. Parents were asked to divide cards into the following 2 piles: “I would be concerned if my child said ‘yes’” and “I would not be concerned if my child said ‘yes.’” Patients were also asked to divide the cards into the following 2 piles: “I would have a hard time answering honestly” and “I would not have a hard time answering honestly.” Next, parents and patients were asked to imagine that they or their child answered all cards in pile #1 affirmatively, and they were asked to write what they would be concerned about happening next and how they would want the results communicated to them. Then, the facilitator encouraged further discussion and elaboration.
The purpose of the sales pitch was to use projective methods  to inform the most acceptable sender, message, and environment for the mental health screener. Both patient and parent participants were asked to convince the person sitting next to them to take the anxiety and depression screeners. Then, they were asked to convince the person to be honest while taking the screener. After completing this exercise, the facilitator encouraged participants to discuss who they would like to explain the screener to them and where they would like to answer the screening questions. They also discussed what steps patients and their parents expected would occur if the patient’s responses yielded a positive screen as well as who would communicate the results of a positive screen.
The Struggle is Real
This projective technique was a cartoon completion test  used to define what “feeling better” means to patients and their families with regards to anxiety, depression, and gastrointestinal symptoms. Patients and parents sat at separate tables for this activity. Patients were presented with several recognizable memes and were asked to fill in the blanks and react to prompts, such as “My face when...” or “That feeling when...” Parents were presented with 3 different cartoon drawings with blank speech balloons. The first cartoon displayed a frowning child and neutral adult, the second showed a frowning child and happy adult, and the third showed a happy child and a happy adult. Parents were asked to fill in the speech balloons to describe a situation related to having and managing gastrointestinal disorders. After completing the activity, the facilitator encouraged participants to share their responses among the group and facilitated discussion.
Backward Experience Map
The backward experience map activity was intended to explore patients’ preferred experience from the time they submit the screening questions to symptom improvement a year later . Patients and parents completed this activity separately. There was a large sheet of paper on the wall with 7 equidistant points connected by an “s”-shaped curve. The beginning point and the last 3 points were subsequently identified (ie, “leaving,” “three months later,” and “one year later”). Participants were asked to identify steps toward getting “better” and fill in the appropriate points on the map. By identifying points that allow participants to get from point A to B, patterns begin to emerge. These patterns begin to uncover themes that establish patient preferences for the treatment experience, patient-centered outcomes, and what “better” means to them.
This activity used constructive methods  to define patient and parent preferences for the best possible sequence of events. Patients and parents completed this activity separately. Expanding upon the results of the backward experience map, participants mapped out their ideal integrated mental health clinic flow process. The facilitator initiated discussion by asking participants to determine what would be “even better” than the ideas that were generated during the backward experience map.
The results of all group session activities were analyzed and coded by the design research specialists who conducted the sessions. They synthesized data from pictures and written documentation (eg, note cards and maps), and they reviewed the audio recordings of the sessions. Data were organized into themes based on Ackoff’s theory , which uses a grounded theory approach to distinguish between 3 levels of sense-making (data, information, and knowledge). This study was approved by the Indiana University Institutional Review Board. All patients and family members who participated signed informed consent or assent documents prior to participation.
Overall, 11 families participated in the group sessions, which included 11 patients and 14 parents (); 5 families participated in the first group session and 6 families participated in the second group session. One family was present for both group sessions.
Depression and Anxiety
displays participants’ responses to the question on the board activity. With regard to depression, they described emotional feelings of sadness, negative thoughts (eg, worthlessness), and behaviors consistent with depression (eg, social isolation and withdrawal). Participants described symptoms of various anxiety subtypes (ie, generalized anxiety and social anxiety) as well as physiological symptoms of anxiety (ie, tachycardia, sweating, and nausea).
Mental Health Screening and Consultation in the Gastroenterology Clinic
It was important to patients and their parents that patients still receive the gastroenterology (GI) care they intended to receive. Many participants stated that they could be traveling quite a distance for their appointment, and they stressed the importance of keeping their regularly scheduled appointment. One patient explained, "You should still have the GI appointment because that’s what you were scheduled for and you still need that service."
They agreed that if the patient screened positive for depression or anxiety, they would like to discuss it with their doctor and consult with a psychologist, but it was important to them that this discussion did not interfere with their GI appointment.
Patient Comfort with Mental Health Screening
Patients indicated that their level of comfort with completing mental health screening was related to how the screener was presented. In turn, their comfort would impact how honestly they would answer the questions. If patients felt a sense of control, they would be more likely to respond honestly; for example, patients indicated that if their parents were worried or if the screener was presented unexpectedly with no explanation, they would be more anxious about completing the screener. As a result, patients felt that they may rush through the screening questions or select the most desirable responses. Patients were clear that they would like to be prepared, and they requested to know how many questions are on the screener, how long it would take them to complete, and what would happen after taking the screener.
There was disagreement between parents and patients regarding the privacy of patients’ screening results. When asked about whether parents should receive the results of the screener at the same time as their children, many parents acknowledged that their children would probably want privacy. However, because it is a health issue, parents wanted to be involved and aware of results. Most parents agreed that they had a right to their child’s protected health information; therefore, they screening results should be shared with them. On the other hand, they acknowledged that their children may be less likely to answer questions honestly if they knew their parents would see their results.
Some of the questions might be questionable. They may not want the parent to see. It’s their privacy.
Right, but kids don’t have that yet.
|Participants||Session 1||Session 2|
|Gender (female), n (%)||4 (80)||5 (83)|
|Age (years), mean (SD); range||15.8 (2.7); 11-17||13.8 (3.3); 9-17|
|Race, n (%)|
|Caucasian||5 (100)||5 (83)|
|African American||0 (0)||1 (17)|
|Asian||0 (0)||0 (0)|
|Ethnicity, n (%)|
|Hispanic||0 (0)||0 (0)|
|Non-Hispanic||5 (100)||6 (100)|
|Primary gastroenterology complaint, >n (%)|
|Irritable bowel syndrome||1 (20)||3 (50)|
|Abdominal pain||0 (0)||1 (17)|
|Vomiting||0 (0)||1 (17)|
|Crohn disease||1 (20)||0 (0)|
|Eosinophilic esophagitis||1 (20)||1 (17)|
|Peptic esophagitis||1 (20)||0 (0)|
|Celiac disease||1 (20)||0 (0)|
|Insurance type, n (%)|
|Commercial||1 (20)||2 (33)|
|Public||3 (60)||4 (67)|
|Self-pay||1 (20)||0 (0)|
|Gender (female), n (%)||5 (83)||7 (88)|
|Race, n (%)|
|Caucasian||6 (100)||4 (50)|
|African American||0 (0)||1 (13)|
|Asian||0 (0)||0 (0)|
|Unknown||0 (0)||3 (38)|
|Ethnicity, n (%)|
|Hispanic||0 (0)||0 (0)|
|Non-Hispanic||6 (100)||5 (63)|
|Unknown||0 (0)||3 (38)|
Most patients did not want their parents to be aware of their screening results unless there was a serious concern. They described being more comfortable discussing these issues with their doctor than with their parents. This was also observed during group sessions. When the parents and patients were together for discussion, patients generally did not express their opinions, but when they were separated from their parents, patients contributed their opinions and actively engaged in discussion. However, there were a few patients who noted that they would feel more comfortable discussing screening results with a parent in the room. One patient explained, "I just feel more comfortable with people I know around."
In both sessions, patients expressed that sharing screening results with their parents may make their parents anxious, and they did not want to worry them. One patient said, "I wouldn’t want her to have to deal with something unless it was like big or something...If I hadn’t told her about that, I wouldn’t necessarily want her to know."
They acknowledged that not being transparent with their parents about screening results may also cause parents to be distressed. Patients agreed that parental worry was a major concern for them. Overall, they expressed that they wanted a choice about whether their parent was in the room while discussing the screening results.
Communicating Results of Mental Health Screening
Patients agreed that they would not expect their screening results to suggest that they have anxiety or depression and taking a screener might cause some distress for them. Having conversations with their GI doctor and a psychologist would ease their worry about the screener. Most patients wanted their doctor(s) to talk to them without their parents right after taking the screener. All patients stated that they would like some kind of result and plan immediately after taking the screener. Although all patients said they would want some sort of feedback that day, only one said that they would like to get a diagnosis from the screener. Most patients wanted to talk to their GI doctor and a psychologist at the same time right after taking the screener.
The Best Clinic Experience
When asked what the order of events should be from the moment they completed the screener, patients and parents had little problem creating a process flow that was agreeable to the other participants within their own group but they had difficulty synthesizing a single agreed upon flow. The steps of the desired clinic flow for patients and their parents are included inSteps are numbered chronologically and “even better” steps are listed next to the initially desired step.
Recommendations for the DECADES Study
Based on these results, our design team formulated the following recommendations, all of which have been incorporated in the protocol for the randomized controlled trial portion of the DECADES study.
- Create a survey or worksheet for the parents to fill out while the patient is taking the screener to both educate and provide a parental distraction
- Provide a handout that describes depression and anxiety and how it relates to GI symptoms
- Use informational graphics to educate patients on the relationship between GI symptoms and mental health, as seen in
- Develop an introduction to mental health screening that includes how many questions will be included in the screener, how long it will take, and what will happen after patients complete the screener
- Use the following language to frame the screener:
- Regarding the relationship between mental health and physical health: “FACT: When your GI system is messed up, it can mess with your brain too, causing anxiety or depression. BONUS FACT: When you have anxiety or depression, it can mess with your GI system, causing all kinds of problems.”
- Regarding the brain-gut connection: “Your GI system and your brain—like everything else in your body—are connected. When one is irritated, often so is the other.”
- Regarding privacy: “The answers you give are CONFIDENTIAL. That means they can only be viewed by you and your doctor, unless YOU choose to share it.”
- Help patients develop a plan for care that addresses both their mental health concerns and their GI symptoms
- Talk to the patient separately from the parent and ask patients if they would like their parents to be involved in the discussion
- Have the patient meet with the GI physician and the psychologist at the same time initially; for example, the physician might say, “This is (psychologist). She’s going to talk to you about the results of that screener you took. We’ll work together to make a plan for treatment of your depression or anxiety and how they might affect your GI issue.”
|Desired flow||“Even better” flow||Desired flow||“Even better” flow|
|1||Immediate feedback from the screener with animation||N/Aa||Educational information about brain-gut connection, depression, and anxiety available in the waiting area and food provided||N/A|
|2||Discussion of results with GIb physician||N/A||GI physician gives the results of the screener||Parents and children receive screening results simultaneously; young children may even receive results from their parent|
|3||GI visit with physician||N/A||Provide additional educational materials||N/A|
|4||Choose whether or not parents are involved in conversation about screening results||Choose the therapist||Meet with the psychologist and GI physician together||Receive referral to a qualified psychologist close to home. Even better than that would be to be taught coping strategies to use until next appointment.|
|5||Reassurance and normalization of symptoms||N/A||The family, psychologist, and GI physician agree on a care plan.||Child is involved in care planning.|
|6||Patients, parents, and care team develop a treatment plan||N/A||Family and care team have a clear understanding of patient’s illness||N/A|
|7||GI symptom improvement||Being cured of GI symptoms||N/A||N/A|
|8||Self-management of symptoms with fewer visits to health care providers||N/A||N/A||N/A|
aN/A: not applicable.
We conducted a qualitative study of patient and parent preferences regarding integrated mental health care in the GI office using patient-centered design methods to augment typical group session methodology. Our results suggest that this screening process is highly acceptable to patients and their families with the caveat that confidentiality remains intact, message delivery be customized to the patient or family member, and mental health services do not interfere with their GI visit.
This represents the first attempt, to our knowledge, to develop a set of clear criteria for effective mental health screening in a pediatric subspecialty office. These criteria were developed, not by expert consensus of clinicians as is often the case in similar studies, but by directly engaging with patients and their families who are already visiting this clinic. We believe this will result in a far more effective screening process that is much more acceptable to families and that increases the efficacy of subsequent mental health interventions. We plan to test this in the next, randomized controlled trial phase of the DECADES study.
There are several important limitations to this study. First, owing to the relatively small sample size and few male participants, it is difficult to ascertain broad generalizability of these findings. However, we attempted to recruit patients of various ages, gastrointestinal complaints, and insurance types to increase generalizability to our larger clinic population. Second, the design methods used are novel in health-related research, but they have been well-established in service and product design. Third, results may be limited because adolescents were less likely to contribute to the patient-parent group discussions than their parents. However, patients engaged very well in discussion when they met as a separate adolescent group. Furthermore, the total number of subjects participating in the group sessions was low (11 families comprising 11 children and 14 adults), but these numbers are typical for this type of research, and larger groups tend to be less effective. Most of our subjects were female (both children and parents), which we believe reflects the higher rate of comorbid anxiety and depression in female adolescents as well as greater maternal engagement in child health. Finally, the subjects we recruited were a sample of convenience of nonconsecutive patients seen at our pediatric gastroenterology clinic, who were willing to participate in research, and may not represent a random sample of our patient population.
The next step in the DECADES project is to conduct a randomized comparative effectiveness trial. Patients in the gastroenterology clinic will complete depression and anxiety screening in accordance with the results and recommendations of this first phase of the study. Those who screen positive will be presented with their results and randomized to either standard care or consultation with a pediatric psychologist on the same day as the visit.
We thank the Agency for Healthcare Research and Quality for generous support for this study through the Infrastructure Development Program in Patient-Centered Outcomes Research R24HS022434 (AEC, SEW, and WEB Jr) and the Patient Engagement Core, including Gina Claxton and Lisa Parks. The Patient Engagement Core is supported by the Indiana Clinical and Translational Sciences Institute funded in part by National Institute for Advancing Translational Science TR001107 (SEW). We also thank the Indiana University Departments of Pediatrics and Psychiatry and Behavioral Sciences as well as the Indiana Clinical and Translational Sciences Institute.
Conflicts of Interest
- Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Arch Gen Psychiatry 2003 Aug;60(8):837-844. [CrossRef] [Medline]
- Kessler RC, Walters EE. Epidemiology of DSM-III-R major depression and minor depression among adolescents and young adults in the National Comorbidity Survey. Depress Anxiety 1998;7(1):3-14. [Medline]
- Polanczyk G, Salum G, Sugaya L, Caye A, Rohde L. Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. J Child Psychol Psychiatry 2015 Mar;56(3):345-365. [CrossRef] [Medline]
- Ani C, Bazargan M, Hindman D, Bell D, Farooq MA, Akhanjee L, et al. Depression symptomatology and diagnosis: discordance between patients and physicians in primary care settings. BMC Fam Pract 2008 Jan 03;9:1 [FREE Full text] [CrossRef] [Medline]
- Ferreiro F, Seoane G, Senra C. Gender-related risk and protective factors for depressive symptoms and disordered eating in adolescence: a 4-year longitudinal study. J Youth Adolesc 2012 May;41(5):607-622. [CrossRef] [Medline]
- S S, Nair C, L S R. A Study On Prevalence Of Anxiety Disorders Among Higher Secondary School Students. jemds 2015 Mar 27;4(26):4473-4478. [CrossRef]
- Pinquart M, Shen Y. Depressive symptoms in children and adolescents with chronic physical illness: an updated meta-analysis. J Pediatr Psychol 2011 May;36(4):375-384. [CrossRef] [Medline]
- Ferro MA, Boyle MH. The impact of chronic physical illness, maternal depressive symptoms, family functioning, and self-esteem on symptoms of anxiety and depression in children. J Abnorm Child Psychol 2015 Jan;43(1):177-187. [CrossRef] [Medline]
- Engström I. Mental health and psychological functioning in children and adolescents with inflammatory bowel disease: a comparison with children having other chronic illnesses and with healthy children. J Child Psychol Psychiatry 1992 Mar;33(3):563-582. [Medline]
- Hyams J, Burke G, Davis P, Rzepski B, Andrulonis P. Abdominal pain and irritable bowel syndrome in adolescents: a community-based study. J Pediatr 1996 Aug;129(2):220-226. [Medline]
- Fond G, Loundou A, Hamdani N, Boukouaci W, Dargel A, Oliveira J, et al. Anxiety and depression comorbidities in irritable bowel syndrome (IBS): a systematic review and meta-analysis. Eur Arch Psychiatry Clin Neurosci 2014 Dec;264(8):651-660. [CrossRef] [Medline]
- de BA, Sprangers M, Bartelsman J, de HH. Predictors of health care utilization in patients with inflammatory bowel disease: a longitudinal study. Eur J Gastroenterol Hepatol 1998 Sep;10(9):783-789. [Medline]
- Wong W, Lai K, Lam K, Hui W, Hu W, Lam C, et al. Prevalence, clinical spectrum and health care utilization of gastro-oesophageal reflux disease in a Chinese population: a population-based study. Aliment Pharmacol Ther 2003 Sep 15;18(6):595-604 [FREE Full text] [Medline]
- Cunningham NR, Jagpal A, Peugh J, Farrell MK, Cohen MB, Mezoff AG, et al. Risk Categorization Predicts Disability in Pain-associated Functional Gastrointestinal Disorders After 6 Months. J Pediatr Gastroenterol Nutr 2017 May;64(5):685-690 [FREE Full text] [CrossRef] [Medline]
- Mackner LM, Crandall WV. Oral medication adherence in pediatric inflammatory bowel disease. Inflamm Bowel Dis 2005 Nov;11(11):1006-1012. [Medline]
- Keerthy D, Youk A, Srinath AI, Malas N, Bujoreanu S, Bousvaros A, et al. Effect of Psychotherapy on Health Care Utilization in Children With Inflammatory Bowel Disease and Depression. J Pediatr Gastroenterol Nutr 2016 Dec;63(6):658-664 [FREE Full text] [CrossRef] [Medline]
- Birmaher B, Brent D, Chiappetta L, Bridge J, Monga S, Baugher M. Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): a replication study. J Am Acad Child Adolesc Psychiatry 1999 Oct;38(10):1230-1236. [CrossRef] [Medline]
- Kroenke K, Spitzer RL, Williams JBW, Löwe B. An ultra-brief screening scale for anxiety and depression: the PHQ-4. Psychosomatics 2009;50(6):613-621. [CrossRef] [Medline]
- Carman K, Dardess P, Maurer M, Sofaer S, Adams K, Bechtel C, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood) 2013;32(2):a-31.
- Domecq J, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res 2014 Feb 26;14:89 [FREE Full text] [CrossRef] [Medline]
- Martin B, Hanington B. Universal methods of designways to research complex problems, develop innovative ideas, and design effective solutions. In: Universal Methods of Design: 100 Ways to Research Complex Problems, Develop Innovative Ideas, and Design Effective Solutions. Beverly, MA: Rockport Publishers; 2012.
- Prahalad C, Ramaswamy V. The future of competition: Co-creating unique value with customers. Boston, MA: Harvard Business Press; 2004.
- Sanders E, Stappers P. Convivial toolbox: Generative research for the front end of design. Amsterdam, Netherlands: BIS; 2012.
- Michalko M. Thinkertoys: A handbook of creative-thinking techniques. Berkeley, CA: Ten Speed Press; 2010.
- Cataldo E, Johnson R, Kellstedt L, Milbrath L. Card sorting as a technique for survey interviewing. Public Opinion Quarterly 1970;34(2):202-215.
- Bellenger D, Bernhardt K, Goldstucker J. Qualitative research in marketing. Decatur, GA: Marketing Classics Press; 2011.
- Adaptive Path. Adaptive Path's Guide to Experience Mapping. San Francisco, CA: Blurb, Inc; 2015.
- Ackoff R. From data to wisdom. Journal of applied systems analysis 1989;16(1):3-9.
|DECADES: Detecting and Evaluating Childhood Anxiety and Depression Effectively in Subspecialties|
|PHQ: Patient Health Questionnaire|
|SCARED: Screen for Childhood Anxiety Related Emotional Disorders|
Edited by A Powell, T Graedon; submitted 30.03.18; peer-reviewed by S Stones, C Chen, P Sossai; comments to author 11.05.18; revised version received 19.06.18; accepted 22.06.18; published 10.09.18
©Stephanie E Hullmann, Stacy A Keller, Dustin O Lynch, Kelli Jenkins, Courtney Moore, Brandon Cockrum, Sarah E Wiehe, Aaron E Carroll, William E Bennett Jr. Originally published in Journal of Participatory Medicine (http://jopm.jmir.org), 10.09.2018.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in Journal of Participatory Medicine, is properly cited. The complete bibliographic information, a link to the original publication on http://jopm.jmir.org, as well as this copyright and license information must be included.