Ethics approval for the video evaluation was granted by the University of British Columbia Behavioral Research Ethics Board (H21-03767) on January 19, 2022. All participants received written information about the study and provided written consent before proceeding. Data in the study were treated confidentially and stored on a secure server in Canada. Participants were entered into a prize draw for 1 of 2 CAD $50 (approximately US $35) Visa gift cards. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Data were analyzed using SPSS (version 29; IBM Corp). Descriptive statistics were used to summarize demographics and feedback regarding video acceptability. Paired-sample t tests were used to compare summary scores on the eHEALS before and after viewing the video. The ordinal nature of mHealth literacy items warranted the use of a nonparametric, 2-sample paired sign test to assess video impacts. Significance was set at P=.05, and all analyses were 2-tailed. Effect sizes for paired-sample t tests were estimated using Cohen d, and effect sizes for nonparametric, 2-sample paired sign tests were estimated using Cliff δ, given the nonnormal distribution of the difference scores [51,52]. Sensitivity analyses (Multimedia Appendix 3) were conducted to evaluate the potential influence of key demographic and baseline variables on missing data, the impact of outliers, and the influence of missing data [53].

Of individuals who consented to the survey (n=77), suspected fraudulent responses (n=23) were removed based on indicators including duplicate IP addresses, email addresses that did not match provided names, infeasible completion times, and duplicate responses to open-ended survey items [54,55], leaving 54 valid entries. In total, 42 respondents completed the survey; their data were used for analyses of acceptability and changes in digital health literacy.

Demographics are summarized in Table 2. Survey completers were primarily women (n=29, 69%), White (n=31, 74%), and residing in North America (n=34, 81%), with a mean age of 38.6 (SD 12) years. Under half the sample self-reported a BD-II diagnosis (n=19, 45%), and most participants were receiving psychiatric treatment, including medication (n=38, 90%) and counseling (n=25, 60%). The majority of the sample had completed postsecondary education (n=34, 81%).

To provide some insights into whether data were missing in a systematic fashion (Multimedia Appendix 3), we compared those who dropped out prior to survey completion and those who completed the study using independent t tests for age and baseline eHEALS. Chi-square tests were used to assess for differences in survey completion rates related to gender and previous use of BD-related health apps, as this was found to be associated with digital health literacy in a previous analysis [14]. We did not assess for differences between BD-I and BD-II, as in the same previous analysis, when BD-I was used as the reference category in our regression model BD-II did not emerge as a significant predictor of eHEALS scores [14]. No significant differences were found between completers and noncompleters, suggesting that missing data were not associated with these demographic characteristics.

Perceptions of the content, length, and presentation of the video were overall positive (Figure 3). Ratings of video acceptability were collapsed to simplify the presentation (strongly agree or agree=agree and strongly disagree or disagree=disagree).

A paired-sample t test was used to assess the impacts of the video on eHEALS scores. No evidence of nonnormality was detected according to the Shapiro-Wilk test (W=0.96; P=.11) nor visual examination of the histogram and quantile-quantile plot. eHEALS scores of the survey completers were significantly higher after watching the video (mean 33.57, SD 4.67) than at baseline (mean 32.40, SD 4.87; t₄₁=–3.236; P=.002; d=–0.50). The influence of 2 potential outliers was evaluated via a paired-sample t test with outliers removed. As overall findings remained unchanged (Multimedia Appendix 3), these cases were retained.

For a conservative estimate of the impact of missing data [53,56], the paired-sample t test was repeated with posttest data for survey noncompleters imputed using the last observation carried forward. Results from this sensitivity analysis showed a significant improvement in eHEALS scores after viewing the video (Multimedia Appendix 3).

Responses of survey completers to mHealth literacy items before and after viewing the video are summarized in Table 3. A Shapiro-Wilk test showed that the distribution of the difference scores of evaluation items departed significantly from normality (question 1: W=0.74; P<.001; question 2: W=0.70; P<.001; question 3: W=0.87; P<.001; question 4: W=0.88; P<.001; question 5: W=0.92; P=.007; and question 6: W=0.77; P<.001). Distributions of the difference scores were found to be nonsymmetrical from visual inspection of the histograms.

Based on the skewed and nonnormal distribution of the differences, a nonparametric, 2-sample paired sign test was used to evaluate changes in participant responses to mHealth literacy items (Table 3). Positive differences indicate the number of cases where responses were higher after watching the video compared to before. Negative differences indicate the number of cases where responses were lower after watching the video than before. Ties indicate no change in ranking. After watching the video, survey respondents felt better able to determine which apps would protect their data (P=.004; δ=.417) and were more empowered to ask their health care provider for support in choosing an app (P<.001; δ=.253). The median response to these items changed from neither agree nor disagree to agree.

With the input of people living with BD, we developed a brief psychoeducational video designed to support individuals with this condition in selecting safe, effective, and engaging mental health apps. Preliminary evaluation data show that the video was largely perceived as acceptable, and viewing the video resulted in improvements to eHealth literacy. This study adds to a body of research showing that educational initiatives can improve digital health literacy for people with chronic health conditions. A previous scoping review identified 9 interventions aimed at improving digital health literacy that were grouped into 2 categories: those providing education and training and those providing social support, with education and training initiatives (including videos, workshops, and massive open online courses) showing greater benefits for digital health literacy [50]. We are only aware of 2 interventions developed to address digital health literacy in individuals with mental health conditions, including DOORS (developed to support individuals with psychosis to use smartphones and apps) [24] and video-based training to use a patient portal for people with chronic conditions (including depression and anxiety, among other physical health conditions) [57]. While these interventions reported positive effects for eHealth literacy measures, neither were developed with specific consideration of the app-related preferences and information needs of people living with BD, a gap addressed by our video-based intervention.

To complement the eHEALS, which is focused on digital health literacy more broadly, we also included some researcher-developed items to evaluate change in smartphone-specific competencies, such as searching for and evaluating apps. Positively, we observed improvements to some aspects of mHealth literacy, such as willingness to ask a health care provider for support and confidence in evaluating app privacy policies. We note that our previous web-based survey of health care providers found a common barrier to discussing or recommending smartphone apps to patients with BD was practitioner knowledge [18]—our findings therefore suggest that clinician education efforts are also needed in order for patients to receive the desired support from health care providers regarding app selection. Furthermore, in light of consensus that the presence of privacy and data security protections is of foundational importance in the decision of whether or not to use apps [39,40], and BD-specific literature showing control over information privacy or security ranks among the top 4 most important mental health app features [34], the finding that confidence evaluating privacy policies improved after the video is of particular note. As we included several strategies to support viewers in evaluating privacy policies (ie, key aspects of privacy policies, encouragement to seek the support of health care providers, and links to app libraries), future qualitative evaluations could explore which of these were most impactful from a viewer perspective, which could inform refinements to this and similar digital health literacy interventions.

It is important to acknowledge that not all aspects of mHealth literacy demonstrated improvements. Potentially, this may be indicative of some ceiling effects, given median baseline responses to items that did not demonstrate change were “agree” or “strongly agree.” We acknowledge the possibility that the use of web-based recruitment methods may have biased the participating sample to individuals with higher baseline digital health literacy as well as interest in app-based tools (described further in the Limitations section). However, it is also possible that the brief video-based intervention was not detailed enough to result in changes to self-perceived knowledge. Indeed, while video acceptability ratings were overall positive, some minor disagreement was observed regarding the appropriateness of the length of the video. Our own CREST.BD advisory groups offered similar reflections regarding the need to offer more in-depth learning opportunities for specific subgroups; the development of a suite of self-guided educational resources to address this feedback is detailed below.

Our project adds to a body of literature on the utility of CBPR frameworks for developing educational outputs that are well-received and impactful in the target population [58-60]. Input from peer researchers and advisory groups helped to ensure that the video focused on issues of primary importance to people with BD, that recommendations were feasible and practical, and that video delivery was engaging and accessible. Participatory research activities in this study also highlighted challenges in planning the timelines and scope of projects developing and evaluating interventions using CBPR frameworks. For example, discussion with peer researchers and advisory groups identified potential user groups whose needs may not be sufficiently met by the intervention as originally conceptualized (ie, a brief video). It was noted that specific subgroups, such as those impacted by the digital divide, may need guidance in basic phone features or additional resources to support the application of strategies. The informational needs of health care providers were also highlighted via consultation activities and a prior survey [18]. To address this feedback, coauthors EM, EEM, and SSK created a complementary suite of self-guided resources for people with BD and health care providers, structured around the video themes (ie, privacy, efficacy, and engagement) and levels of the APA app evaluation framework not covered in the video (ie, background information and data integration). Emerging information regarding the potential risks of apps in BD, such as the potential for mood monitoring to reinforce depressive symptoms in vulnerable individuals [61] and the limitations of using apps designed for the general population for BD concerns [11], was also detailed. These resources were hosted on an innovative learning platform, the Tapestry Tool [62], where hierarchical relationships between concepts are represented spatially similar to a mind map (Figure 4), and multimodal resources including text, videos, and web articles can be linked. Similar online courses to support digital health literacy have been shown to improve eHEALS scores in specific populations, such as people with type 1 and 2 diabetes [63]. Combining this brief video with a self-guided exploration of the Tapestry Tool educational module could therefore further enhance impacts on digital health literacy. However, as this Tapestry Tool educational module was developed in addition to the planned, funded activities (ie, development of the brief video), we did not have the resources to evaluate the impacts of these resources separately and in combination. This illustrates a common tension in CBPR research: extensive consultation with communities is needed to inform grant applications; yet, this can be difficult to resource before grant funding is available [64]. To avoid situations where there are not sufficient resources to fund research priorities identified by the community, we suggest a need for more funding opportunities specifically supporting CBPR during project conceptualization.

A number of limitations to this study should be noted. For context, we note that the grant provided to fund this project (Michael Smith Health Research BC REACH Grant) was specifically intended to cover costs associated with the development of the educational resources (including payment of peer researchers). For this grant, costs associated with research studies are noneligible expenses and were covered in kind by CREST.BD. This limited our ability to conduct a more fulsome randomized controlled trial, as we did not have sufficient funds to fairly compensate participants for their involvement in a study where they may not have received exposure to the intervention. In addition, it limited our ability to conduct more resource-intensive recruitment strategies, such as outreach into face-to-face settings. The implications of this for the study limitations are described in more detail below.

First and foremost, this was a nonrandomized pilot evaluation; findings should therefore be interpreted with caution. In the absence of a control group, spontaneous improvements due to expectancy effects, baseline sample characteristics, or other confounding variables cannot be ruled out. In addition, the small sample size limits generalizability. Removal of suspected fraudulent responses detected on review of the data (n=23) reduced the total valid survey entries (n=54). This finding emphasizes the importance of applying additional strategies to ensure sample validity, such as rigorous screening procedures, inclusion of questions to detect poor quality or inattentive responses, and restrictions on where and how surveys are advertised [65]. Although our sample was small, it is comparable to other evaluations of digital health literacy interventions in serious mental illness populations [24,66]. Unfortunately, this sample was too small to conduct additional subgroup analyses, including gender-based comparisons.

Our sample was predominantly White and had completed some form of postsecondary education; efforts are needed to ensure that digital health literacy interventions are accessible to those with limited English proficiency. A survey of established (living in Canada for >10 years) senior Punjabi and Chinese immigrants (n=896) found that only one-quarter of participants reported advanced reading and writing proficiencies in English, and lower levels of education were associated with poorer eHEALS scores. As 65% of participants expressed an interest in using a smartphone to improve their health [67], this group may benefit from support to develop digital health literacy. To support equitable access to intervention content in Canada, we have translated the video into Mandarin, Punjabi, and American Sign Language, although we note that the evaluation was only conducted in English, limiting ability to generalize findings to other language groups.

Funding restrictions and issues of feasibility influenced our choice of recruitment strategy: we used a web-based survey to increase the likelihood of reaching a target sample size, given the relatively low prevalence of BD [68]. It may be that the use of web-based recruitment methods biased our sample toward individuals with higher pre-existing levels of digital health literacy. Relatedly, one survey that used telephone, hard-copy, and online data collection methods to assess digital health literacy and digital engagement for people with severe mental illnesses (including BD) found that higher levels of digital health literacy were associated with having outstanding or good self-reported knowledge of the internet [25]. As such, future studies should consider evaluating the impact of this video-based resource using alternative dissemination methods, such as DVDs that can be played in mental health clinics, or one-on-one consultations with health care providers.

The eHEALS measures self-perceived digital health literacy and not necessarily the actual performance of these skills; it is therefore possible that participants may experience an increase in self-perceived competencies without a concordant improvement in the real-world application of their skills. Future studies may wish to use procedural assessments of digital health literacy competencies. Approaches to performance-based assessments of digital health literacy are highly heterogenous and include simulated behavioral tasks, knowledge assessments, and evaluation tasks [69]. For example, previous studies have provided participants with a list of both high- and low-quality health information websites [70,71]; the concordance of participants’ evaluation of these websites with researcher ratings (as based on a standardized framework) was used to evaluate eHealth literacy skills. A similar approach could be used in the future to compare participants’ evaluations of apps with expert ratings as a proxy for mHealth literacy skills. Alternatively, comparing eHEALS scores to skills-based assessments may improve confidence about the real-world implications of improvements on this measure. While some work has been conducted to demonstrate modest correlations between perceived and performed eHealth literacy [72], we acknowledge that additional external validation is required. Unfortunately, we are not aware of any validated measures of mHealth literacy (performance-based or self-assessment)—a clear priority for future research. Our own in-house items were developed, given the dearth of available instruments; however, the fact that they were not validated remains a limitation of this study.

Interventions are needed to help address the digital divide by promoting the skills and knowledge needed to take advantage of digital mental health tools and enhance the uptake of safe and effective mental health apps by people with BD. In this study, receiving only 4.5 minutes of psychoeducation about the risks and benefits of mental health apps for BD was found to improve self-perceived eHealth literacy and some aspects of mHealth literacy in individuals with this diagnosis. However, it must be noted that multiple aspects of mHealth literacy remained unchanged, and 19% (n=8) of the survey completers denied learning anything new as a result of the video. While findings remain preliminary due to the small sample size, nonrandomized design, and the use of nonvalidated mHealth literacy items, they are encouraging for future evaluations. To support the reach of the video and the accompanying web-based educational module, we have made these resources freely available for health care providers and patients [36,62].