Experiences and Challenges of Patients With Chronic Respiratory Disease During a Virtual Research Study: Qualitative Study

Introduction

The COVID-19 pandemic disrupted research and health care delivery, resulting in transitions to virtual platforms [1,2]. Health care professionals increasingly used eHealth tools, defined as the provision of health care services to patients through the use of technology [3]. Additionally, researchers, particularly qualitative researchers, have applied virtual data collection methods such as online surveys and virtual interviewing through video and telephone calls, emails, and SMS text messages [4]. These methods offer the benefits of synchronous or asynchronous communication [5]. The various approaches of virtual research have unique advantages and disadvantages for both researchers and participants.

Advantages of virtual research include cost and time efficiency, flexible scheduling, and more representative samples by reducing barriers to travel, enabling geographically dispersed and mobility-friendly participation [6-8]. Additionally, virtual methods often provide a higher degree of perceived and actual anonymity, which may encourage participation from vulnerable groups and facilitate the discussion of sensitive topics [5,6,8,9]. Consequently, studies show that virtual data collection methods can be as effective as traditional in-person approaches [10-17].

However, virtual research presents challenges such as technological barriers, lessened data familiarity, and shortened participant engagement. Researchers have limited control over participants’ technological issues such as internet connectivity, device compatibility, and familiarity with technology [4,18]. Additionally, sampling bias may occur, favoring participants with better technological skills [18,19]. Advances in technology have led to the utilization of automatic transcription in videoconferencing software; however, this may lead to researchers becoming less familiar with their data compared with manual transcription [4]. Both participants and researchers may also be susceptible to “Zoom fatigue” or distractions at home, such as interruptions from children and pets, and on-screen notifications leading to reduced engagement [4,20].

Privacy and data integrity concerns represent critical challenges to both the participant and the researcher in virtual research, especially when using international technology platforms. Platforms such as Zoom (Zoom Communications, Inc) and Microsoft Teams (Microsoft Corporation) offer Health Insurance Portability and Accountability Act–compliant versions with encryption, but they are potentially vulnerable to data breaches, cyberattacks, and third-party data-handling practices [21-24]. In Canada, the regulatory landscape governing health data privacy in research is complex, characterized by overlapping federal and provincial privacy laws without harmonized national standards. While not a specific aim of this research paper, it is important to follow Canadian privacy legislation which requires safeguards proportionate to data sensitivity. International data sharing is permitted, but researchers remain accountable for protecting participant privacy which is relevant when using cloud-based platforms hosted outside Canada [25-28]. Researchers should also be aware of the potential of fraudulent participants, who may have ulterior motives for participating in research, and consider incorporating ways to minimize and detect fraudulent participation [29-34].

While several studies discuss potential challenges and offer recommendations for virtual research, these are mostly commentaries focusing on researchers’ perspectives [5,35]. The lived experiences and challenges of participants, especially within specific demographic cohorts such as patients with chronic respiratory disease (CRD), are important. To address this gap and identify areas to improve participant experience and study quality, this paper explores the experiences of patient participants with CRD in a virtual research study using qualitative methods. It is part of a larger study validating a health literacy (HL) tool for patients with CRD through virtual or phone interviews [36]. The primary objective of this substudy was to understand the experiences of patients with CRD with virtual research methods participating in the HL measurement tool study.

Methods

Ethical Considerations

The study received approval from The University of British Columbia Office of Behavioral Research Ethics (H15-01954).

Participant Recruitment and Eligibility

Participants were recruited between August 2019 and June 2020. A research recruitment service was used where community patients were reached via random phone callouts in the Greater Vancouver Area. Patients were offered to receive a call back if they were interested in the study and met the eligibility criteria. Eligibility criteria included the following: (1) aged 19 years and older, (2) able to read and communicate in English, (3) physician-diagnosed asthma and/or chronic obstructive pulmonary disease, and (4) current use of a medication for asthma and/or chronic obstructive pulmonary disease.

Survey and Checklist Development

The study involved 2 stages: first, a 6-question open-ended survey (Multimedia Appendix 1) was developed to elicit participants’ challenges and experiences in virtual research. The survey focused on communication and technological challenges from the perspectives of the participant and the interviewer and was developed through research team discussions, a thorough literature review, and consultation with observation notes (ONs). During the second stage, participant and interviewer responses to the open-ended survey and interview ONs informed the development of an 18-item checklist (Multimedia Appendix 1). The checklist covered preinterview challenges (eg, recruitment- and communication-related) and interview difficulties (eg, questionnaire-, communication-, and technology-related) for participants and interviewers. Both tools underwent pilot testing among a group of participants and revisions were integrated.

Interview Procedures and Data Collection

Participants provided written informed consent through a digital consent form. They selected an interview date and received a confirmation email with study details and the HL questionnaire. Participants used a computer, tablet, or smartphone to view the questionnaire and provided responses to questions to the research assistant over the phone. Participants with no compatible device received a physical copy of the ethics protocol and questionnaire via courier. Telephone interviews lasted 40‐80 minutes. All participants were offered a CAD $30 (US $21) honorarium. After completing the questionnaire, participants were given the virtual research survey or checklist, with responses recorded verbatim. The research assistant took observational notes during the interviews for further data contextualization. All observational notes, interview recordings, and transcripts were deidentified and stored using password-protected files on institutional servers.

Data Analysis

Initially, 3 members of the research team (NT, AM, and YSC) thoroughly reviewed the survey or checklist and corresponding ONs. While doing so, each team member independently created a list of challenges inductively. Meetings were then held to develop a coding guide in which conceptually similar codes were combined, infrequent codes were removed, and definitions were developed.

The coding guide was then piloted with 2 coders (NT and AM), who separately coded a random 15% of the data with equal distribution of survey or checklist and initial or retest interviews. The goal was to achieve 80% interrater agreement during the pilot, a benchmark accepted by many researchers [37]. After the initial trial of coding, the desired agreement level was not reached. The coding framework was subsequently revised by NT and AM by once again removing unnecessary items, clarifying definitions, and combining similar items. The coders then coded a new, random 15% of the data (with equal distribution of survey or checklist and initial or retest interviews) and achieved the desired interrater agreement (≥80%).

Each coder then coded half of the remaining 85% of data. Subsequently, codes were reviewed by YSC, who identified incorrect or missed codes. The team then resolved issues through discussion. Any disagreements between the coder (either NT or AM) and the reviewer (YSC) were resolved by the other coder. Both direct quotes (DQs) and ONs were coded. Following coding, thematic analysis [38] took place. The codes from the framework were discussed and synthesized into larger overarching themes. Constant feedback on the emerging themes was elicited from team members. Themes not explicitly relating to virtual challenges were removed.

Results

Overview

From June to October 2020, participants completed a virtual phone interview where they completed the questionnaire with a research assistant. Some of these were considered initial interviews, where the participant filled out the questionnaire for the first time (n=185) and completed a postinterview checklist (n=36) or survey on virtual research (n=149). To test the HL tool’s reproducibility, 110 participants completed another interview, with the postinterview checklist (n=79) or survey (n=31), on their experience with virtual research challenges. Participants primarily identified as female (122/185, 65.95%), were older than 50 years (167/185, 90.27%), had asthma (108/185, 58.38%), and had a disease duration of less than 20 years (110/185, 59.46%). Additional participant characteristics can be found in Table 1.

The following sections describe the 6 overarching themes and 23 subthemes derived from analysis: theme 1: overall experience with virtual research, theme 2: clarity of instructions, theme 3: technology, theme 4: communication, theme 5: condition and cognition, and theme 6: logistics. Participants’ direct quotes are indicated by “DQ” and researchers’ observation notes with “ON.” Identifiers for each participant, denoted by the letters “HL” and followed by a number, are used to anonymously identify which interviews the quotes or notes came from. Textbox 1 contains additional DQs and ONs from the 6 themes.

Theme 1: Overall Experience With Virtual Research

Participation was generally satisfying and meaningful to participants.

Overall Telehealth Experience

Most participants found their first experience with virtual research positive. They appreciated learning about relevant self-management topics through the questionnaire. This sentiment was captured by one individual who mentioned that it was both “educational and informative” [HL94, DQ].

Negative Overall Telehealth Experience

Few participants expressed dissatisfaction, with the main complaints surrounding the questionnaire, with one participant “[refusing] to answer three questions because he thought they were vague” [HL21, ON].

Wants to Be Updated

Participants took pride in contributing to research that might benefit others with similar health conditions and many expressed interest in learning the study’s outcomes. One asked “Is it possible to be notified of the study results after everything is complete?” [HL65, DQ]

Theme 2: Clarity of Instructions

Instructions regarding the study’s requirements, delivered via email and phone, were sometimes a source of confusion.

Documents

For some participants, confusion arose from understanding which documents required signatures and the process of phone-based questionnaires. One asked “which [form] should be signed” [HL147, ON]. Additionally, some were unsure whether signed copies they received needed further action.

Email

Some participants struggled to discern the importance of the various emails they received. One participant thought that the content and documentation were excessive but assumed “it’s all necessary” [HL156, DQ].

Interview

Participants were occasionally caught off guard by the nature of the interview, especially those unfamiliar with the field of HL. One was confused over the method of data collection: “I thought I could fill out the questionnaire instead of verbally telling you my answers over the phone” [HL185, DQ]. The need for very clear expectations was expressed.

Purpose

Some participants questioned the purpose of the study and the tasks they were asked to perform, discussing the need for clarification and reiteration. One commented: “In [the] first interview, the purpose of the study was not clear” [HL116, DQ].

Recruitment

Despite eligibility screening during recruitment, some participants still had questions about their eligibility at the interview stage. For one participant, the research assistant noted: “although her doctor diagnosed her with COPD, she doesn’t have any symptoms and asked if she is still eligible to be part of this study” [HL175, ON].

Theme 3: Technology

Virtual study conduction necessitated technological engagement, leading to several challenges.

Device and Internet

Participants sometimes grappled with device-related challenges. Issues included tablet or phone incompatibilities with document formats, unexpected system behaviors (eg, start-ups or restarts and low battery), and distractions from app notifications. For instance, one participant’s phone screen could not adequately display the full document: “the screen wasn’t big enough to have the full [file] format because I was opening the document from my cell phone” [HL62, DQ]. Poor internet signal caused difficulties in accessing documents and signing online forms.

Knowledge

For many, unfamiliarity with technology caused challenges opening documents, adjusting view settings, and signing digital forms. There was a preference among some for in-person interviews or alternatives such as Zoom: “It would have been easier if it was face-to-face or in-person...maybe even by Zoom” [HL157, DQ].

Navigation

Scrolling through the digital questionnaire was troublesome for some. In the digital format, participants often had to scroll to reference material to answer questions, leading to disorientation in the document. One lamented the constant movement or scrolling, feeling like they were going “up and down like a yo-yo” [HL16, ON].

Document Opening

A minority of participants encountered difficulties in opening study documents due to the absence of instructions on how to access PDF or Microsoft Word attachments within emails. For instance, a participant mentioned that they, “had trouble getting the questionnaire to open at first as an attachment to the email” [HL50, ON].

Document Retrieval

Some participants had difficulty retrieving study documents from emails misrouted to the “junk” folder or lost in their inbox. For example, “the emails had initially gone to her junk mail so there was a bit of difficulty on finding the documents” [HL37, ON].

Consent Form

Participants experienced various issues with the online consent form. Some reported that the provided link was nonfunctional: “the link to the consent form...did not work” [HL05, ON]. Mobile and tablet users faced navigation and compatibility issues, and the signature process was cumbersome for many, with one describing it as “doing etch-a-sketch with the mouse” [HL61, ON]. A frequent concern was the lack of submission confirmation, leading participants to often seek verification from the research assistant.

Theme 4: Communication

Communication issues occurred between research staff and participants throughout the research process. These challenges are categorized as challenges with contacting the research team, interruptions during the interview, and other challenges.

Contact

On rare occasions, participants had trouble communicating with the research team postrecruitment and prequestionnaire or interview, which made scheduling difficult. One reported that they “didn’t know how to get a hold of [the research team] after [the] initial call” [HL26, ON].

Interruption

Telephone interviews, conducted at a location of participants’ convenience, led to interruptions from external sources, such as pets, family members, other calls, and phone connection issues. A notable instance was when a participant asked the researcher to hold the call for 6 minutes while answering another [HL08, ON].

Interview

Phone interviews sometimes had communication hitches from poor call quality, background disturbances, or participants’ swift responses. One research assistant reported: “[the] participant was reading out answers very quickly, I had to slow [them] down to keep up” [HL103, ON].

Theme 5: Condition and Cognition

Some participants had health and/or cognitive conditions that caused difficulties during the research study.

Anxiety or Worries

Despite no set time limit, some participants indicated that they felt pressure to answer swiftly, as evidenced by one individual who felt the strain of “analyzing the passage while on the spot” [HL110, ON]. Another cause of stress for participants was feeling as though they were being assessed.

Font and Formatting (Eyesight)

Several participants found the font size in the questionnaire too small and often attributed this to their poor eyesight, as one mentioned: “numbers are too small” [HL16, ON].

Condition (Noneyesight)

Some participants had hearing difficulties which complicated the phone interview, while others had breathing issues due to their respiratory conditions. In the case of one participant, “[they] coughed a lot and asked for [a] 10-minute break in the middle of the interview and took [their] inhaler” [HL 154, ON].

Theme 6: Logistics

There were also logistical and planning challenges that arose throughout the study.

Confidentiality

A few participants raised concerns about confidentiality, particularly regarding their initial contact (random callouts) and potential data sharing with third parties. Some individuals were also initially concerned with how their personal information might be used and how or where their information would be stored. One participant thought that “some of these studies share info with insurance and pharmaceutical companies [as] every year his list of allowed or covered medications [had] gone down” [HL138, ON].

Rescheduling and Delays

Interviews often faced rescheduling due to participants being unprepared, forgetful, falling ill, or having other commitments.

Team Error

Errors from the research team, although infrequent, did arise. Examples include incorrect email entries and honorarium transfer discrepancies, which were promptly addressed.

Discussion

Principal Findings

Our findings illustrate the experiences and challenges faced by participants in a virtual research study validating an online HL tool. Six themes emerged, showing that participants generally reported a positive experience but faced challenges with technology, mental fatigue, and data confidentiality concerns. Communication issues, such as distractions and unclear instructions, were also noted.

The virtual nature of this study allowed participants to engage at a convenient time and location, eliminating travel-related time and cost burdens. Individuals whose health limited their ability to attend in-person sessions were also able to participate, enhancing the diversity of the sample. These 2 advantages align with literature supporting virtual research as more cost-effective and time-efficient, allowing for broader geographical reach and greater sample diversity [4-6,8]. However, later we discuss how factors such as participant response times may not always make virtual research more time- and cost-effective than in-person research.

A common concern with virtual methods is the exclusion of individuals with lower technological skill, particularly older adults [39,40]. In this study, most participants were aged 50 years or older, with 43% (80/185) being aged 50‐70 years and 47% (87/185) being older than 70 years. Additionally, internet use among older adults in British Columbia is high, with 88% of adults aged 45-64 years and 56% of adults aged 65 years and older using the internet [41]. Despite high rates of internet use, the digital divide persists among older adults, particularly those with lower socioeconomic status, limited education, and rural residences [42-44]. Even for older adults with internet access, disparities exist in the breadth and depth of digital engagement, with many lacking the skills to perform health-related tasks such as accessing patient portals, booking appointments, or participating in telehealth [43-45]. Barriers extend beyond access to include physical and cognitive impairments, privacy concerns, lack of social support, and preference for face-to-face interactions [46-49]. Without interventions such as digital literacy training or infrastructure improvements, virtual methods can risk exacerbating health disparities by excluding older adults with the greatest health needs and fewest resources [44,50,51].

Challenges arose with communication about study expectations. Despite regular contact through email and phone, participants occasionally felt unclear about the study’s objectives, the data collection process, and documents requiring their signature before receiving further clarification. This highlights the need for clear, concise instructions and avoidance of overburdening participants, while also ensuring opportunities to ask questions. Research teams should create detailed and user-friendly instructional materials and provide multiple avenues for participants to seek clarification. These recommendations also apply to in-person research, where recruitment and interview coordination often also occur online [4].

Participation in virtual studies requires technology and internet access. In this study, participants used various devices including laptops, tablets, and smartphones. While this allowed a wide range of individuals to participate, technological skills varied, leading to challenges such as difficulty opening attachments, navigating documents, and phone users struggling to display documents in full. The research assistants supported these participants by asking them to explain the issue they were experiencing in their own words and then providing tangible steps at their technology literacy level. This contributed to varied amounts of time spent with each participant. Thus, virtual research may not always be more cost-effective or time-efficient than in-person research and depends on the individual participant [52]. Some virtual studies have reported additional time spent supporting participants with technology and higher participant cancellation and no-show rates [53]. This was also observed in our study as participants sometimes had to reschedule interviews due to being unprepared, forgetting, falling sick, or having multiple commitments at the same time. While rescheduling may happen more often in virtual studies, it can be seen as an advantage as the virtual environment allows for more flexible rescheduling instead of participant dropouts [4,7].

Throughout the study, communication issues were encountered. Since virtual studies can be conducted from any location, there were interruptions from external sources such as family members, pets, other calls, and connection issues. These interruptions can be detrimental to the quality of data collected, as distractions can lead to less contextually rich information provided, and poor call quality can result in missing information. Additionally, some participants also experienced mental fatigue and difficulty focusing, further contributing to decreased data quality [20]. These challenges are common in virtual research [5,18,52,54]. To avoid this, researchers can proactively ask participants to prepare a quiet environment to minimize distractions, ensure that questionnaires and interviews are not too lengthy, take breaks when necessary, and offer adequate incentives to maintain participant engagement.

This study generated several practical lessons for future virtual research design. In summary, participants benefited from simplified and consolidated communication. Sending fewer emails with clearly labeled attachments and step-by-step instructions can help reduce confusion. Electronic consent procedures should also be optimized for mobile devices and include confirmation messages indicating successful submission. Document design should minimize scrolling demands and allow participants to view passages and questions simultaneously where possible, particularly for older adults or those with visual impairments. Researchers should anticipate varying levels of digital literacy and allocate sufficient time for individualized technical support. Finally, participants should be encouraged to prepare a quiet environment and be informed in advance about interview expectations, duration, and required materials to minimize interruptions and anxiety during participation.

Documenting patient experiences in virtual research is fundamental to participatory medicine and patient-centered research. Participatory medicine is guided by the phrase “nothing about me without me” and incorporates patient perspectives at every step of the research process, from question selection and study design through data collection, analysis, and dissemination [55-58]. Participants in this study emphasized the importance of clear communication, opportunities for clarification, technical support, and understanding the purpose of research. Many participants also expressed a desire to receive study findings and remain informed about how their contributions would be used. These observations reinforce the principle that participants should be treated as active partners in research. By capturing the lived experiences of patients with CRD with virtual research methods, this study contributes practical insights for designing patient-centered research.

Strengths and Limitations

This study has several strengths. The virtual nature allowed for broader participation, likely leading to a more representative sample. Furthermore, the data analysis used investigator triangulation to maintain consistency between coders. However, limitations were identified, including potential sampling bias by excluding participants without access to reliable technology. To mitigate this, we mailed surveys to a couple of participants who requested it. Furthermore, inconsistent time was spent with each participant due to the varied technological skill level. However, this variety meant that perspectives were being collected from a diverse set of participants and did not favor participants with better technological skill. Future research should focus on comparing synchronous and asynchronous methods and explore participant engagement strategies. Additionally, addressing ethical considerations will help optimize the effectiveness of virtual research.

In conclusion, this study identified the experiences and challenges of patients with CRD with virtual research. Participants generally reported a positive experience and expressed satisfaction to participate in research that may benefit individuals with similar medical conditions. Challenges mainly encompassed technological barriers, the need for clear and concise instructions, mental fatigue, and concerns about data confidentiality. These findings emphasize the importance of providing comprehensive technical support, clear and concise instructions, and assurance of secure data storage methods to ensure effective, ethical, and inclusive virtual research. With these considerations, virtual research can remain a viable and valuable method, even among older adults. Future research should focus on enhancing participant engagement and improving digital technology accessibility post COVID-19 pandemic.