Background: The rise of major complex public health problems, such as vaccination hesitancy and access to vaccination, requires innovative, open, and transdisciplinary approaches. Yet, institutional silos and lack of participation on the part of nonacademic citizens in the design of solutions hamper efforts to meet these challenges. Against this background, new solutions have been explored, with participatory research, citizen science, hackathons, and challenge-based approaches being applied in the context of public health.
Objective: Our aim was to develop a program for creating citizen science and open innovation projects that address the contemporary challenges of vaccination in France and around the globe.
Methods: We designed and implemented Co-Immune, a program created to tackle the question of vaccination hesitancy and access to vaccination through an online and offline challenge-based open innovation approach. The program was run on the open science platform Just One Giant Lab.
Results: Over a 6-month period, the Co-Immune program gathered 234 participants of diverse backgrounds and 13 partners from the public and private sectors. The program comprised 10 events to facilitate the creation of 20 new projects, as well as the continuation of two existing projects, to address the issues of vaccination hesitancy and access, ranging from app development and data mining to analysis and game design. In an open framework, the projects made their data, code, and solutions publicly available.
Conclusions: Co-Immune highlights how open innovation approaches and online platforms can help to gather and coordinate noninstitutional communities in a rapid, distributed, and global way toward solving public health issues. Such initiatives can lead to the production and transfer of knowledge, creating novel solutions in the public health sector. The example of Co-Immune contributes to paving the way for organizations and individuals to collaboratively tackle future global challenges.
As the world faces a rise in the number of complex challenges that threaten the resilience of our economic, environmental, social, and health systems, we observe a shift toward more collaboration and openness in the way science and innovation is performed [- ], bringing governments, civil society, and the private sector closer. Examples of this include the efforts made to accelerate society’s progress toward the Sustainable Development Goals (SDGs) [ ] and the fight against pandemics, such as COVID-19 [ ]. Yet, access to vaccines and vaccination hesitancy remain as some of the complex challenges to be addressed in order to achieve universal health coverage [ ].
Immunization is one of the most cost-effective interventions to protect oneself and others from infectious diseases  and saves between 2 million and 3 million lives per year [ ].
Yet, the annual death toll for vaccine-preventable diseases stands at 1.5 million, and large gaps in coverage persist, not only between countries but also within their territories . In particular, the World Health Organization (WHO) listed vaccine hesitancy among the top 10 global health threats for 2019 [ ]. Continuing global efforts to leave no one behind may be a long-standing challenge [ ] when new information technologies and social media platforms are both part of the problem [ ] and the solution. More recently, the COVID-19 pandemic demonstrated the repertoire of logistical and administrative challenges to the deployment and administration of vaccines, especially in low-resource settings [ ].
In response, the WHO Global Vaccine Action Plan 2011-2020  committed 140 countries and 290 organizations to promoting and prioritizing greater collaboration between governments, nongovernmental organizations, the private sector, and all citizens to address outbreaks of vaccine-preventable diseases. Additionally, a number of new digital and open innovation initiatives have been launched: the WHO has developed the Vaccine Safety Net [ ], a network of websites about vaccination; health authorities in Canada have developed a school-based quiz to educate children about immunology and vaccines [ ]; Finland is testing a computer game to communicate the benefits of human papillomavirus vaccination [ ]; a project in India uses digital necklaces to record children’s immunization history [ ]; and the global Vaccination Acceptance Research Network has been established [ ].
Global health guidelines showcase the positive outcomes of social participation for universal health coverage , which include more meaningful dialogue, more sustainable solutions, and more trust from citizens in health system institutions or in the decisions that are made. Indeed, there is room for more initiatives that allow people to genuinely co-design solutions in a multidisciplinary manner during and following pandemics [ ]. Hence, the number and sustainability of these types of initiatives could be amplified by fostering increased collaboration with nonacademic citizens in the creation and development of solutions in an open innovation framework [ ]. This is the gap that Just One Giant Lab (JOGL) is proposing to fill with the Co-Immune program.
Citizen science is an emerging and highly diverse practice that can be broadly defined as the general public being involved in the process of doing research . Research has demonstrated that intensity and diversity of collaboration positively affect the quality [ ] and productivity [ ] of research, while positively impacting the knowledge integration from participants [ ]. Likewise, participant transdisciplinarity [ ] seems critical to generating innovative outcomes [ ] and dealing with complex real-world problems [ ]. Such mechanisms are often at play in the field of citizen science, promising to transform the knowledge generation landscape by tapping into networks of nonacademic citizens [ , ] in a new social contract for this kind of research [ ]. Citizen science has the potential to expand the number of individuals contributing knowledge and ideas, transform how hypotheses are generated, and transform how data sets are analyzed. Such approaches have already been applied to investigate individual diseases through patient-led research [ , ] and public health challenges, such as the epidemiology of cancer [ - ].
Other approaches to create and develop knowledge and solutions to complex challenges are slowly entering the mainstream. In particular, hackathons, challenge-based approaches, and the participation of citizens in science have been flourishing over the last two decades , especially within the natural sciences [ ] and, more recently, within medical sciences, public health, and population-health research [ , ].
Hackathons are short, intensive, and collaborative events that are designed to prototype solutions addressing a specific problem. They originated in the early 2000s in digital and tech fields and have been adapted to address more complex challenges in global health [- ]. Such initiatives are not without pitfalls: they suffer, by design, from the lack of paths to sustainability for the projects they launch [ ]. In response to such criticisms, there are increasing efforts, such as the “Make the Breast Pump not Suck” hackathon and “Trans*H4CK,” to improve hackathon methodology by working directly with affected communities [ ]. Several initiatives, such as a Massachusetts Institute of Technology collaborative design studio, provide insights into hackathon methods [ ] to facilitate better hackathons [ , ]. More recently, multiple entities have engaged in organizing hackathons to address the COVID-19 crisis [ , ].
Challenge-based approaches, which provide frameworks for learning while solving real-world issues, have also been on the rise in global health and have proven to be efficient for generating innovative solutions and for incentivizing mass community engagement . For example, the potential of participative models to address complex questions, along with the power of contests to offer a structure that catalyzes this work, has been exhibited by the Epidemium initiative on cancer epidemiology [ ].
Despite the numerous tools and technologies created to facilitate collaboration in citizen science projects, challenges remain. These include the issues of the complementarity, coherence, and diffusion of these initiatives  to efficiently address international policies and local needs, as the local adoption of hackathon solutions often remains low [ ].
Therefore, the promotion of transdisciplinarity and citizen science in an open innovation framework, coupled with methods such as hackathons, and a challenge-based approach represent an opportunity to address current complex challenges of vaccination that would overcome the limits of either solution alone. In this paper, we describe the design, implementation, and outputs of Co-Immune, a collaborative open innovation program that was run in 2019 to address vaccination hesitancy and access to vaccination.
Co-Immune’s aim was to develop an environment that favors the creation and development of citizen science and open innovation projects addressing the contemporary challenges of vaccination in France and around the globe. This program had four specific objectives: (1) to foster a collaborative, open, and transdisciplinary dynamic; (2) to promote the emergence of accessible knowledge and innovative solutions; (3) to support participants in the elaboration and development of their project; and (4) to disseminate the outputs and results in an open science framework. In this study, we describe the methodology of Co-Immune and its implementation, and we present its key outcomes.
The overall program duration was 10 months (March 2019 to January 2020), divided into 6 months of preparation and 4 months of rollout of activities that included offline and online events, support for the development of citizen science projects, and assessment and awards for projects participating in the challenge-based competition. The main outputs of the program were projects, categorized as leading to (1) knowledge production, if they performed data analysis or generated new knowledge, whether it was context specific, generic , or knowledge transfer [ ]; or (2) solutions, such as hardware, software, and interventions.
Co-Immune was coordinated online through the JOGL platform  and supported by 13 partners from the public and private sector (Table S1 in ). The challenge-based nature of the program was designed to be an incentive for teams and participants to continue developing their projects after hackathon events or to create their project on JOGL at any other time.
The governance of Co-Immune was designed to provide freedom for projects to develop innovative solutions while ensuring their compliance with local and international regulations and consideration of ethical and scientific integrity. To this end, we constituted the independent Committee for Ethics, Science and Impact (CESI), which issued an opinion on the rules of participation in the program and validated the strategic orientation of the program. Public health priorities were identified based on a literature review and divided between two main challenges to streamline participants’ work: vaccination coverage and vaccination hesitancy. They were then validated by the CESI. In addition, through a series of semistructured interviews, experts at the 7th Fondation Merieux Vaccine Acceptance conference  identified eight specific issues to address and potential room for solutions. The CESI also participated in the co-elaboration of the assessment grid, which was used as a base to grant nonmonetary prizes to projects in December 2019.
To be eligible for a prize, a project was required to have created a comprehensive description of their initiative on the JOGL platform and a video pitch. This material was provided to experts in charge of the assessment.
Participants were recruited through our network of partners from around the globe and social media communication. Participation was open to everyone above the age of 18 years, if they had agreed to follow the participation rules validated by the CESI. Participants could take the role of “project leader” or “contributor.”
Co-Immune participants used the JOGL platform to document their projects and recruit collaborators throughout the course of the program. JOGL is a decentralized mobilization platform designed for use in collaborative research and innovation (). Within the JOGL platform, users can create a profile and declare their skills. Once registered, they can create or join projects, follow the activity of other members, post on their project feed, and comment on other posts. They can also highlight needs for a project they are part of, specifying skills that can help to solve project problems. We compared the JOGL features to those of other online platforms for citizen science, social networking, and science and publishing through a cluster analysis (see as well as the supplementary method and Figure S1 in ), indicating that the platform is functionally similar to other platforms in the space and is suitable to hold a citizen science program such as Co-Immune.
The Co-Immune program was realized through an interrelated and interacting set of technological and social features (). Our coordination team implemented the larger program (ie, events, online platform, and contest approach) and helped to recruit a community of partners and participants who interacted with each other and were supported in their efforts through the high-level design features. With support from the governance structure of the Co-Immune program, the individual projects managed to provide outputs that included knowledge production and transfer as well as solutions, such as hardware, software, and interventions.
Building an Open Community
To build the community, we contacted organizations involved in a wide range of domains before the launch of the program, thereby creating a first pool of contributing professionals and students. We also recruited participants via the organization of events, typically in the evening, aimed at creating projects, fostering collaboration among participants to address project needs, and providing mentorship. To facilitate the coordination of the community, all participants were required to use the JOGL platform to describe their projects, form teams, list their needs, and initiate collaboration.
In order to create a supportive and collaborative environment for the participants, we reached out to various organizations to establish partnerships. Our intention was two-fold: (1) to facilitate the participation of the organizations’ students and employees as participants or mentors by involving their institution and (2) to enhance the sustainability of projects after the course of the program by connecting them with potential partners at the early stage of their development.
The 13 partners operated in the health, technology, and social sectors, and included research, innovation, and education organizations, as well as professional networks, incubators, and communication specialists (). The number of partners grew over the life span of the initiative and were often suggested by existing partners or through connections made during events.
We organized 10 offline and online events between October and December 2019 (). Participants for events were recruited through social media and mailing lists leveraging our network of partners. Among the four on-site events that were organized, two were hackathons aimed at motivating participants to join the program, while the other two were aimed at fostering collaboration around the most advanced projects. Their median duration was 2.25 (IQR 2) hours.
The facilitation of the hackathon-style events relied on the use of participatory and collective intelligence design and problem-solving techniques . In particular, participants were encouraged to form multidisciplinary teams including both professionals and students.
Three partners in Paris—Epitech, the Wild Code School, and the Center for Research and Interdisciplinarity (CRI)—co-organized and hosted events for their students, respectively, in their engineering, coding, and life science and education schools. Other partners—Kap Code, Excelya, and CorrelAid—mobilized their teams to act as mentors during these events. A total of 14 mentors attended events, and five came to more than one event.
In addition, we organized four 1-hour online events. The first was an opportunity to share information about Co-Immune with people around the globe. Another event discussed best practices to document open science projects. Finally, two events focused on the resolution of needs of single projects ([ , ]).
|Name||Mode; type; location||Duration (hours), n||Objective||Design; supporting partners (if applicable)||Participants, n|
|Launch||Offline; ceremony; CRIa, Paris||3||Gather the initial community||Presentation of the program design, features, timeline, and partners, as well as networking||60|
|OpenJOGLb; Co-Immune||Online||1||Q&Ac session on the program||Presentation of Co-Immune and questions and answers||3|
|Sprint; open data||Offline; hackathon; CRI, Paris||2.5||Build community, create projects, and create data repositories||Statement of the problem (videos of experts), team formation and effort, mentoring, and publication of results on the JOGL platform; supported by CRI and CorrelAid||25|
|OpenJOGL; Vaccination Awareness Escape Game ||Online||1||Foster collaboration around single projects||Pitch of the project and its needs, feedback from experts, and questions and answers||7|
|Sprint; project creation||Offline; hackathon; CRI, Paris||4||Build community and create multidisciplinary projects||Statement of the problem (videos of experts), ice breaker, multidisciplinary team formation and effort, mentoring, presentation of results, vote for the most promising projects, publication of results on the JOGL platform, and networking; supported by CRI, Epitech, Wild Code School, CorrelAid, and Excelya||22|
|Sprint; open data||Offline; hackathon; Wild Code School, Paris||3||Accelerate the development of projects related to data science||Selection of a project by participants among the two choices available, team formation and effort, mentoring, presentation of results, publication on the JOGL platform, and networking; supported by Wild Code School, CorrelAid, and Excelya||15|
|Sprint; open data||Offline; hackathon; Epitech, Paris||3||Build the community, create projects, and accelerate the development of one project using Twitter data||Statement of the problem, selection of a project by participants among the four choices available (including one already existing project), team formation and effort, mentoring, presentation of results, vote for the most promising project, publication of results on the JOGL platform, and networking; supported by Epitech, Kap Code, Excelya, and CorrelAid||35|
|OpenJOGL; HERAd: A Health Platform for Refugees ||Online||1||Foster collaboration around single projects||Pitch of the project and its needs, feedback from experts, and questions and answers||7|
|OpenJOGL; better documentation for better collaboration||Online||1||Help teams document their projects in the most open and reproducible way||Expert presentation on best practices for documenting open science projects, presentation of Co-Immune expectations for documentation, and questions and answers||13|
|Closing ceremony||Offline; ceremony; CRI, Paris||2||Close the Co-Immune program||Presentation of the main outputs of the program and awards for the best projects||70|
aCRI: Center for Research and Interdisciplinarity.
bJOGL: Just One Giant Lab.
cQ&A: question and answer.
dHERA: Health Recording App.
Co-Immune Experts: CESI Members, Mentors, and Interviewees
Individuals who were considered “experts” included all the CESI members as well as experienced professionals of a certain field who attended events and provided technical guidance to teams as “mentors.”
The CESI members were sought to represent the diversity of stakeholders involved in advancing access to vaccines and reducing vaccine hesitancy. By choosing interviewees who were researchers specializing in the challenges of access to vaccines and vaccination hesitancy, we aimed at benefiting from their expert understanding of the issues and of the priorities to be addressed to streamline the work of participants around particular problems. Finally, we grew the pool of mentors over the span of the program to best match their expertise with the needs of the projects in an agile manner.
Overall, the mentors’ domains of expertise ranged from biology to social sciences, design, technology, and data science (). One-third of them were working as health or public health professionals.
The CESI consisted of eight volunteer members and included virologists, pharmacists, health economists, experts in the digital sciences and ethics fields, and biologists; members were working at international, national, and local levels of the health system. All of them worked for public or nonprofit organizations. Interviewees were mostly researchers in social sciences and medical practitioners.
Co-Immune Project Assessment
The assessment of projects by experts was designed to be an opportunity for learning and growth. To be assessed, teams were asked to provide a video pitch summarizing their project and detailed documentation on their project page on the JOGL platform, including links to their open access data and code. Project assessment was performed through a grid that was codeveloped by JOGL and the CESI. In addition to grades, teams received detailed feedback on their projects.
The assessment grid was based on a literature review of project evaluation standards and consisted of 10 questions graded from 0 to 5 (). Three areas were assessed: the approach, the implementation strategy, and the impact. First, the assessment of the approach included the following: (1) clarity and relevance of the problem and alignment with the program scope, (2) fit between the approach and methodology and the problem statement, and (3) innovation potential (ie, the project introduces groundbreaking objectives, novel concepts, or approaches). Second, the implementation strategy was assessed using following the criteria: (1) state of progress toward set goal (ie, state of advancement), (2) clarity and relevance of the timeline and needs for the future (ie, major tasks and milestones), and (3) project actively engages and aligns with all relevant stakeholders. Finally, the assessment of the impact covered the following: (1) clarity and relevance of the criteria used to measure impact, (2) the extent to which the project considers its ecosystem (ie, ecological, environmental, ethical, and social considerations), (3) sustainability and scalability of the project in the long term, and (4) open and reproducible dissemination strategy. For each of these three categories, JOGL awarded a prize to the project with the best score based on the grades given by reviewers. Additionally, a grand prize was given to the project with the overall highest score. JOGL provided visibility, while two partners also provided an award to a project of their choosing.
JOGL Platform Data Collection and Analysis
Participants added their professional background, skills, and employment status to the JOGL platform. These data were used to evaluate the composition of the community. All users who joined JOGL during the span of the program were considered to be participants of Co-Immune, as it was the only ongoing program, and all outreach activities were related to it.
To better understand how skills were related across participants, we used a network approach to assess similarity between skills and to get further insights about the global diversity of the community. In this network approach, each declared skill was a node and the skills were considered linked if they co-occurred in a participant. Links were then weighted by the number of participants within which they co-occurred. Gephi 0.9.2 was used to represent the network shown in the skill map of the Co-Immune community, and the modularity algorithm was used with default parameters to compute communities representing the sets of skills that tend to co-occur more together than with other skills. Since these skills are linked through the participants who share them, they can be understood as "participant types" constitutive of the Co-Immune community.
We provide the data related to this study on Zenodo . These data include (1) the link, description, and assessment scores of projects; (2) the profiles of platform users; (3) the description of events; (4) the profiles of experts; and (5) the list and types of partners.
Community Growth Through Events
During the program, 234 participants signed up to the platform (). The participant growth was mostly linear over the life span of the program (July 10 to December 18, 2019), suggestive of the potential for continued growth if the program had continued. The growth rate outside of events, at around one per day (between 0.86 and 0.98 users/day), was consistent with the prekickoff growth rate (0.94 users/day). This highlights the importance of events (dashed lines in ) for driving participant enrollment, with the four offline events accounting for 45% of the growth. In total, offline events were responsible for the generation of 82% (18/22) of the projects. The rest consisted of 4 out of 22 (18%) projects created on the platform outside of events and 2 already-existing projects prior to the program.
Participant Skills and Backgrounds: A Transdisciplinary Community
Out of the 234 participants, 187 (79.9%) declared their job category. The community was composed of a mix of students (67/187, 35.8%) and workers (94/187, 50.3%), most of whom worked full time (81/94, 86%;). Other categories included “between jobs” (n=11), “nonprofit” (n=12), and “for profit” (n=3). Out of the 75 participants who declared their country in their JOGL profile, 57% (n=43) were based in France, with the rest coming from other regions, including the rest of Europe, the Americas, Africa, and Asia.
The 234 participants specified a total of 492 unique skills (median 3 [IQR 4.5] skills per participant). We observed a high representation of data science and coding alongside biology, which, altogether, related to the technical skills emphasized during the program (). The skill network shows that the community spanned a vast interdisciplinary landscape, from open science to open data and coding, and from project management to biology. The network exhibited the largest connected component of 416 interconnected skills (84.6% of all skills; ). The modularity maximization (see the Methods section) resulted in the identification of 12 modules corresponding to “participant types” constitutive of the Co-Immune community.
Co-Immune Project Description
A total of 22 projects were created by 20 project leads, with teams of up to 11 members ([ , , - ]). Among these, 15 (68%) projects proposed to develop software covering web technologies, mobile apps, algorithms, data lakes, data modeling and analysis, and visualization tools. The other 7 (32%) projects included hardware development and interventions involving biotechnologies, game design, behavioral sciences, education, and communication. Overall, one-third of the projects focused on knowledge transfer.
Among the 15 projects relying on software technology, 11 (73%) aimed at contributing to the production of knowledge by facilitating the analysis of publicly available data; they did this via the use of parsing tools and the creation of repositories (n=3), the analysis of open data (n=3), the development of machine learning tools to extract and analyze Twitter data related to vaccination hesitancy (n=2), and the production of data visualizations (n=3). In particular, more than 40 data sets were identified and collected by 4 projects that were created during the data-centered events. In addition, a database of 2464 tweets, in French, posted over a period of 7 years was made available by a partner, and another data set of 89,979 tweets was gathered by the project Qualitative Analysis of Tweets on Vaccination .
Out of the 15 projects above, 4 (27%) used software for knowledge transfer; for instance, the HERA (Health Recording App) project  provided educational content and health data storage through its mobile app to improve the monitoring of vaccination and perinatal health among Syrian refugees in Turkey. The Pass It On project [ ] focused on role-playing video games directed at health professionals as another method of knowledge transfer. The Neutralizing Information About Vaccines project [ ] implemented an algorithm for parsing web pages, helping citizens identify trustworthy content related to vaccines.
A total of 5 projects out of 22 (23%) focused on different interventions (), including raising awareness about vaccination through an escape game (ie, Vaccination Awareness Escape Game [ ]) and communication campaigns on social media (ie, Go Viral! [ ]). The HEROIC Santé project [ ] developed and tested a short questionnaire using engagement approaches from the social sciences to engage health care professionals and users around the question of flu vaccination. Finally, one team proposed applying synthetic biology methods to tuberculosis vaccines (ie, Project APRICOT [Antigen Presentation Using Crispr for TB] [ ]).
|Project name||Project status||Solution category||Summary description|
|HERAa: A Health Platform for Refugees ||A mobile health app designed for improving the monitoring of vaccination and perinatal health of Syrian refugees in Turkey; it provides recall of vaccines, storage of health data, health promotion (educational content), and financial incentives for immunization|
|Qualitative Analysis of Tweets on Vaccination ||A web-based platform providing real-time visualization and analysis of tweets related to vaccination and vaccination hesitancy; data analysis included sentiment analysis and network analysis; an area of development was the development of predictive models of epidemic occurrence based on Twitter data|
|Commit to Get Vacc & to Promote Vaccination – HEROIC Santé ||A short questionnaire (7 minutes) using engagement approaches from the human and social sciences, such as “the importance of the source,” “voluntary consent,” or “fear and danger management,” to engage health care professionals and users, not only to be vaccinated against the flu, but also to promote flu vaccination|
|Project APRICOTc ||Development of a synthetic biology–based methodology that addresses the evasion mechanisms adopted by the mycobacterium tuberculosis and induces the acceleration of lysosomal biogenesis to improve antigen presentation|
|Vaccination Awareness Escape Game ||An escape game to raise vaccination awareness among the general population|
|Harmonize Vaccination ||A tool for parsing various formats of vaccination coverage data sets and for visualizing them on a common platform|
|Pass It On: A Game About Vaccine Hesitancy ||A role-play video game aiming to improve the capacity of health professionals to respond to their patients’ hesitation to be vaccinated|
|Global Vaccination Risk Assessment ||A tool to create an overview of risk factors of “not getting vaccinated,” by country, while looking at the more comprehensive picture; the methodology of this project is based on fuzzy logic, multi-criterion analysis, and the risk triangle|
|Immuno ||A board game providing access to the general public’s understanding of medical sciences related to immunization|
|Vaccine DataDump ||A vaccination-related data repository and analysis tool for quick analysis of vaccine-related issues|
|Measuring Vaccination Hesitancy From Social Media ||Data analysis of social media (ie, Twitter) to examine whether negative sentiment related to vaccination precedes declaration of symptoms and to study the relationship between vaccination hesitancy and epidemiological outbreaks|
|Mortality According to Access to Vaccines ||Data analysis exploring the link between immunization coverage, mortality rate, and distance from health centers|
|The Health System Matrices ||Exploratory analysis of the various parameters influencing vaccination coverage over time|
|Meta Immune – Data Exploration of Existing DB ||A data lake on immunization data|
|Bilobae ||An intervention incentivizing people to increase vaccine uptake through vouchers, supporting the existing mobile app Biloba|
|Wakuchin Senshi ||An interactive role-play board game to increase awareness about vaccination among the general population|
|Neutralizing Information About Vaccines ||An algorithm for parsing web pages, identifying misinformation, and identifying trustworthy content to help users in their health decisions related to vaccines; this also aims to be used by search engines in their recommender systems|
|Go Viral! ||A communication campaign on social media using gamification methods to illustrate contagion among users and, thereby, increase awareness of the importance of vaccines|
|Make Vaccines Affordable ||A web-based portal with data related to population demand for care in order to negotiate prices of vaccines with suppliers|
|Identify Topics of Discussion in Vaccination Posts ||Analysis of discussion in vaccination-related posts on Twitter and their evolution over time|
|Detect Vaccine Administration in Social Media Patient Data ||A classifier able to detect vaccine administration in tweets related to vaccination|
|Detect Vaccine Hesitancy in Social Media Patient Data ||A classifier able to detect vaccine hesitancy in tweets related to vaccination|
aHERA: Health Recording App.
bThese were projects that were assessed by experts at the end of the program. To be assessed by a pool of experts, the project team needed to provide detailed documentation of their project, provide a short video pitch, and deposit their data and code on the Just One Giant Lab (JOGL) platform.
cAPRICOT: Antigen Presentation Using Crispr for TB.
dThese were projects that were not assessed by experts at the end of the program because they did not provide sufficient documentation.
eThe Biloba project, which was not part of Co-Immune, was used as a base to create the team’s own project, as the Biloba founder was a mentor during this event.
Co-Immune Project Assessment
Out of 22 projects, 7 (32%) provided sufficient documentation on JOGL to be assessed by the pool of independent experts. In total, 27 reviews were performed, yielding scores ranging from 18 to 32.8 out of a possible total of 45 across the different dimensions that were assessed (ie, approach, implementation strategy, and impact). The average score was 25.1 (SD 6.4).
HERA: A Health Platform for Refugees  was awarded with prizes, based on a total score of 15, for best approach (mean score 11.4, SD 2) and impact (mean score 14.6, SD 3.2). Commit to Get Vacc & to Promote Vaccination – HEROIC Santé [ ] was awarded the best implementation strategy prize (mean score 10.33, SD 2.5).
The projects were more successful, globally, in terms of approach, with a mean score of 9.37 (SD 1.79) out of 15 points. Out of 7 projects that were assessed, 4 (57%;) had a score higher than 4 out of 5 for clarity, relevance, and alignment of their problem statement with the program objectives. For 6 projects (86%), the fit between the methods and the projects’ objectives was scored highly by reviewers, with a score of at least 3 out of 5.
The implementation strategy score of projects was low, overall, given the early stage of the projects at the time of review. As such, only projects that existed prior to the program—HERA  and HEROIC Santé [ ]—got a score of at least 3 out of 5.
For winners in each category, JOGL awarded them physical space for showcasing their project during the 2020 ChangeNOW forum at the Grand Palais in Paris as well as tickets for the Maddy Keynote, a major innovation event in Paris. Two partners—Excelya and the Wild Code School—also provided awards to the projects of their choice. Additionally, the Qualitative Analysis of Tweets on Vaccination  project was chosen to be the focus of a hackathon by the Wild Code School, and Project APRICOT [ ] was offered technical support for data science and legal and regulatory affairs by Excelya.
The Co-Immune program was designed to foster the creation and development of citizen science and open innovation projects addressing the contemporary challenges of vaccination in France and around the globe by reaching four specific objectives: (1) to foster collaborative, open, and transdisciplinary dynamics; (2) to promote the emergence of accessible knowledge and innovative solutions; (3) to support participants in the elaboration and development of their projects; and (4) to disseminate the outputs and results in an open science framework. Below, we discuss to what extent Co-Immune reached these objectives and highlight the challenges and facilitators in implementing such a program.
First, the program succeeded in creating a collaborative and transdisciplinary environment through its three core features: the JOGL platform, the organization of events, and the contest approach. This led to forming partnerships with 13 different organizations and recruiting over 230 participants, who displayed 492 unique skills and were engaged in creating 22 projects. The use of on-site hackathons was beneficial in gathering nonacademic participants from various backgrounds. Our data show that in-person events and local outreach played a significant role in growing the community around Co-Immune. These offline events recruited 45% of the total community. Local enrollment was further strengthened by local partnerships, such as higher education organizations. However, the localization of our on-site events in Paris did not allow for the participation of people living in other parts of France or the rest of the world. Additionally, our online communication restricted the access of the online events to our realm of influence and to people with an internet connection. More inclusive participation geared toward people with diverse socioeconomic statuses and geographic situations is desirable in the future to give them agency over solving the problems that affect them. The development of new communities is usually a slow process in the absence of exogenous shocks, such as the surge in collaborative communities created by the COVID-19 pandemic . Tapping into existing projects and networks for events has proven to be fruitful in our case, allowing for a steady growth of the Co-Immune community up until the end of the program. However, we did not observe further growth of the community after the end of the program. This highlights that in order to build a sustainable community using open innovation to tackle global health challenges, one needs to facilitate the entry and exit of members, provide resources to support the current ones, focus on building on existing communities and projects, design inclusive environments for collaboration, and empower members to run their own activities.
Second, two design elements of the program converged to promote the emergence of knowledge and solutions to address aspects of access to vaccines and vaccination hesitancy: (1) the identification of challenges by experts in the field and (2) the alignment of the program strategy with national and international policies by frequent consultation with public health bodies and mobilization of members of public institutions in the CESI. Yet, greater representation of people affected by poor access to vaccines and people who are hesitant would be desirable to strengthen the alignment between the solutions developed and the most pressing needs at the local level.
Recently, online events have been used widely during the COVID-19 pandemic [- ], supporting our initial assumption that forming and animating a distributed online community for public health programs is a relevant approach.
Third, the use of the JOGL platform, the mentorship during events, the assessment and feedback from experts, and the connection with a wide range of partners supported participants in the elaboration of their project in an efficient way. The use of the JOGL platform enabled projects to gain visibility, list their needs to create interfaces for collaboration, and share open data sets, code, and tools. Indeed, online platforms can offer projects that started at hackathons a pathway to pursue their development, potentially alleviating one of the main drawbacks of such short temporal interventions . In this case, it also enabled the program coordinators to connect participants with project leaders based on a match between needs and skills. Yet, this approach was time-consuming, and scaling up our efforts proved to be challenging. The automation of such matchmaking tasks through a recommender system would help to minimize these efforts and increase the impact of projects through accelerated development [ ]. In addition, mentoring is a known strategy that is used by open, online communities [ , ] and was leveraged by the Co-Immune program. Given the diversity of backgrounds and level of expertise across the participants, it was necessary to engage a similar diversity among the mentors. In our context, the highly rated projects that eventually received awards did not originate or participate in hackathons, but rather benefited from Co-Immune as a platform for further growth. Several of these projects already existed before the start of Co-Immune and had a higher maturity level than the projects created during the short span of the program. In addition, these projects were launched and run by people outside the larger Paris region. Thus, we stress the potential of online platforms and open innovation to build on existing projects and to replicate, adapt, and scale their activities in other contexts. Additional support consisted of promoting visibility on social media by the organization team as well as opportunities for networking during events. Although no financial compensation was provided as part of this program, partners, through their own experts and co-organizing events, engaged in close relationships with JOGL and the individual projects. This was favorable for sustaining collaborations and projects after the end of the program. In the future, the sustainability of the newly created project efforts could potentially be improved by using incentives, such as microgrants or fellowship programs, for continuing projects in the postprogram period [ ]. While the short time frame and limited resources allocated to the program did not allow us to implement a strong monitoring and evaluation strategy, future implementations should ensure that they conduct a minimum of pre- and postprogram data collection for assessing the full impact of the program.
Finally, the open science environment of this program was not only an asset for disseminating the outputs and results of the projects developed, but it also enabled them to replicate initiatives and, thereby, accelerate the resolution of the global health challenges they address. An example of this was given by the team from the project HERA: A Health Platform for Refugees , who opened its code, enabling any individual to replicate it. However, the lack of a thorough evaluation strategy prevents us from reaching a more definitive conclusion on the effective replication of projects carried out in Co-Immune.
Co-Immune showcases that short, focused programs can be efficient at mobilizing diverse communities in a rapid manner and harvesting ideas from various domains to address global health challenges. Yet, more case studies and evaluation work on similar programs are necessary to assess the full relevance of their design and the impact of the projects that are developed within them.
Co-Immune highlights how open innovation approaches and online platforms can help to gather and coordinate noninstitutional communities in a rapid, distributed, and global way toward solving SDG-related issues. The Co-Immune program gathered participants and partners from various backgrounds in a newly formed community to facilitate the creation of new projects as well as the continuation of existing projects to address the issues of vaccination hesitancy and access. In an open framework, the projects made their data, code, and solutions publicly available.
Through hackathons and other contest approaches, such initiatives can lead to the production and transfer of knowledge, creating novel solutions in the public health sector. The example of Co-Immune contributes to paving the way for organizations and individuals to collaboratively tackle future global challenges.
First, we would like to thank all the Co-Immune participants that made the program possible by bringing their creativity, skills, and insights to address contemporary public health challenges. We thank Sanofi for funding this program, especially Diane Brément and Nansa Burlet for their assistance. We thank the Just One Giant Lab (JOGL) team for their work on coordinating the Co-Immune program, with special efforts from Lola Casamitjana and Marine Vouard. We thank the Center for Research and Interdisciplinarity (CRI), Paris; Epitech Paris; Sup Biotech; and the Wild Code School for their support in organizing events with their students; Kap Code, Excelya, Girls in Tech, CorrelAid, and Data for Good for their guidance and technical assistance to participants and projects; and S3Odéon, TUBÀ, ChangeNOW, and the Maddy Keynote for the visibility they provided to this program. We thank the interviewees at the 7th Fondation Merieux Vaccine Acceptance conference for highlighting the key issues to address and potential solutions on which participants could build. We thank the mentors for the support they provided to projects and participants throughout the duration of the Co-Immune program. We thank Enric Senabre Hidalgo for insightful comments during the final stages of writing this manuscript. Finally, we thank the members of the Committee for Ethics, Science and Impact (CESI)—Gilles Babinet, Jérôme Béranger, Anshu Bhardwaj, Liem Binh Luong Nguyen, Mélanie Heard, Ariel Lindner, Juliette Puret, and Olivier Rozaire—for their valuable input that allowed the creation and implementation of a framework for ethics, science, and impact for the Co-Immune program and the independent assessment of its projects.
TL and IV co-designed the theme and the scope of the program at the early stage of the initiative. CM and TL conceived the program. CM led the coordination team of the program. CM, BGT, TL, CLBG, and MS participated in the program implementation. BGT, RJ, and MS analyzed data. CM, BGT, GF, CLBG, and MS wrote the paper.
Conflicts of Interest
GF was paid by Just One Giant Lab (JOGL) to support the elaboration of the manuscript; provided consulting services to Vaccines Europe, a trade association based in Belgium; and is a volunteer board member of the Coalition for Life-Course Immunisation, a UK-based charity. CM worked as the Co-Immune program coordinator for JOGL. BGT was an unpaid advisor to JOGL. JOGL received funding from Sanofi to develop and implement the program, which included publication fees. Sanofi respected the strict independence of JOGL, which administers its platform and the Co-Immune page in complete autonomy. Similarly, the Committee for Ethics, Science and Impact (CESI) was independent of Sanofi and decided alone on the strategic and scientific orientations of the program and the best projects to be rewarded.
Co-Immune partners and supplementary method for comparing Just One Giant Lab (JOGL) with other platforms.PDF File (Adobe PDF File), 196 KB
Co-Immune project assessment grid.PDF File (Adobe PDF File), 213 KB
- Nosek B, Alter G, Banks G, Borsboom D, Bowman SD, Breckler SJ, et al. SCIENTIFIC STANDARDS. Promoting an open research culture. Science 2015 Jun 26;348(6242):1422-1425 [FREE Full text] [CrossRef] [Medline]
- Burgelman J, Pascu C, Szkuta K, Von Schomberg R, Karalopoulos A, Repanas K, et al. Open science, open data, and open scholarship: European policies to make science fit for the twenty-first century. Front Big Data 2019;2:43 [FREE Full text] [CrossRef] [Medline]
- Elliott K, Resnik D. Making open science work for science and society. Environ Health Perspect 2019 Jul;127(7):075002 [FREE Full text] [CrossRef]
- Fraisl D, Campbell J, See L, Wehn U, Wardlaw J, Gold M, et al. Mapping citizen science contributions to the UN sustainable development goals. Sustain Sci 2020 Jul 02;15(6):1735-1751 [FREE Full text] [CrossRef]
- Auffray C, Balling R, Blomberg N, Bonaldo MC, Boutron B, Brahmachari S, et al. COVID-19 and beyond: A call for action and audacious solidarity to all the citizens and nations, it is humanity’s fight. F1000Res 2020 Sep 14;9:1130 [FREE Full text] [CrossRef]
- United Nations Statistics Division. Global Indicator Framework for the Sustainable Development Goals and Targets of the 2030 Agenda for Sustainable Development. New York, NY: United Nations; 2017 Jul. URL: https://unstats.un.org/sdgs/indicators/Global%20Indicator%20Framework%20after%202021%20refinement_Eng.pdf [accessed 2021-12-21]
- Immunization, Vaccines and Biologicals Team. Global Vaccine Action Plan 2011-2020. Geneva, Switzerland: World Health Organization; 2013. URL: https://www.who.int/publications/i/item/global-vaccine-action-plan-2011-2020 [accessed 2021-12-21]
- Immunization coverage. World Health Organization. 2021 Jul 15. URL: https://www.who.int/news-room/fact-sheets/detail/immunization-coverage [accessed 2021-12-21]
- Ten threats to global health in 2019. World Health Organization. URL: https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019 [accessed 2020-10-08]
- Immunization Agenda 2030: A Global Strategy to Leave No One Behind. Geneva, Switzerland: World Health Organization; 2020. URL: https://tinyurl.com/4hhuzvru [accessed 2021-12-21]
- Chan M, Jamieson K, Albarracin D. Prospective associations of regional social media messages with attitudes and actual vaccination: A big data and survey study of the influenza vaccine in the United States. Vaccine 2020 Sep 11;38(40):6236-6247 [FREE Full text] [CrossRef] [Medline]
- Wouters OJ, Shadlen KC, Salcher-Konrad M, Pollard AJ, Larson HJ, Teerawattananon Y, et al. Challenges in ensuring global access to COVID-19 vaccines: production, affordability, allocation, and deployment. Lancet 2021 Feb 12:1023-1034 [FREE Full text] [CrossRef] [Medline]
- Vaccine Safety Net. Geneva, Switzerland: World Health Organization URL: https://www.vaccinesafetynet.org/ [accessed 2020-10-08]
- Boost Community. URL: https://boostcommunity.org/ [accessed 2020-10-08]
- Finnegan G. Has Finland found an antivax antidote? Vaccines Today. 2019 Jul 08. URL: https://www.vaccinestoday.eu/stories/has-finland-found-an-antivax-antidote/ [accessed 2020-10-09]
- Vaccine Acceptance & Demand. Washington, DC: Sabin Vaccine Institute URL: https://vaccineacceptance.org/ [accessed 2020-10-09]
- Voice, Agency, Empowerment - Handbook on Social Participation for Universal Health Coverage. Geneva, Switzerland: World Health Organization; 2021. URL: https://www.who.int/publications-detail-redirect/9789240027794 [accessed 2021-08-14]
- Marston C, Renedo A, Miles S. Community participation is crucial in a pandemic. Lancet 2020 May 30;395(10238):1676-1678 [FREE Full text] [CrossRef] [Medline]
- West J, Lakhani K. Getting clear about communities in open innovation. Ind Innov 2009 Sep;15(2):223-231 [FREE Full text] [CrossRef]
- Haklay M, Fraisl D, Greshake Tzovaras B, Hecker S, Gold M, Hager G, et al. Contours of citizen science: A vignette study. R Soc Open Sci 2021 Aug;8(8):202108 [FREE Full text] [CrossRef] [Medline]
- Hsiang Liao C. How to improve research quality? Examining the impacts of collaboration intensity and member diversity in collaboration networks. Scientometrics 2010 Nov 2;86(3):747-761 [FREE Full text] [CrossRef]
- Juan AA, Daradoumis T, Roca M, Grasman SE, Faulin J, editors. Collaborative and Distributed E-Research: Innovations in Technologies, Strategies and Applications. Hershey, PA: IGI Global; 2012.
- Polk M. Transdisciplinary co-production: Designing and testing a transdisciplinary research framework for societal problem solving. Futures 2015 Jan;65:110-122. [CrossRef]
- Leavy P. Essentials of Transdisciplinary Research: Using Problem-Centered Methodologies. Walnut Creek, CA: Left Coast Press, Inc; 2011.
- Stokols D. Toward a science of transdisciplinary action research. Am J Community Psychol 2006 Sep;38(1-2):63-77. [CrossRef] [Medline]
- Nielsen M. Reinventing Discovery: The New Era of Networked Science. Princeton, NJ: Princeton University Press; 2011.
- Franzoni C, Sauermann H. Crowd science: The organization of scientific research in open collaborative projects. Res Policy 2014 Feb;43(1):1-20 [FREE Full text] [CrossRef]
- Vayena E, Brownsword R, Edwards S, Greshake B, Kahn JP, Ladher N, et al. Research led by participants: A new social contract for a new kind of research. J Med Ethics 2016 Apr;42(4):216-219 [FREE Full text] [CrossRef] [Medline]
- Greshake Tzovaras B, Angrist M, Arvai K, Dulaney M, Estrada-Galiñanes V, Gunderson B, et al. Open Humans: A platform for participant-centered research and personal data exploration. Gigascience 2019 Jun 01;8(6):1-13 [FREE Full text] [CrossRef] [Medline]
- Lewis DM, Leibrand S, Street TJ, Phatak SS. Detecting insulin sensitivity changes for individuals with type 1 diabetes. Diabetes 2018 Jul;67(Supplement 1):79-LB [FREE Full text] [CrossRef]
- Afuah A, Tucci C. Crowdsourcing as a solution to distant search. Acad Manage Rev 2012 Jul;37(3):355-375. [CrossRef]
- Sitruk Y, Kazakçi A. Crowd-based data-driven hypothesis generation from data and the organisation of participative scientific process. In: Proceedings of the DESIGN 2018 15th International Design Conference. 2018 Presented at: DESIGN 2018 15th International Design Conference; May 21-24, 2018; Dubrovnik, Croatia p. 1673-1684 URL: https://doi.org/10.21278/idc.2018.0510 [CrossRef]
- Anderson C. The end of theory: The data deluge makes the scientific method obsolete. Wired. 2008 Jun 23. URL: https://www.wired.com/2008/06/pb-theory/ [accessed 2021-12-21]
- Bosman J, Bruno I, Chapman C, Greshake Tzovaras B, Jacobs N, Kramer B, et al. The Scholarly Commons - Principles and practices to guide research communication. OSF Preprints. 2017 Sep 15. URL: https://osf.io/6c2xt/ [accessed 2021-12-21]
- Hecker S, Haklay M, Bowser A, Makuch Z, Vogel J, Bonn A, editors. Citizen Science: Innovation in Open Science, Society and Policy. London, UK: UCL Press; Oct 15, 2018.
- Katapally T. The SMART framework: Integration of citizen science, community-based participatory research, and systems science for population health science in the digital age. JMIR Mhealth Uhealth 2019 Aug 30;7(8):e14056 [FREE Full text] [CrossRef] [Medline]
- Rowbotham S, McKinnon M, Leach J, Lamberts R, Hawe P. Does citizen science have the capacity to transform population health science? Crit Public Health 2017 Nov 09;29(1):118-128. [CrossRef]
- Ramatowski JW, Lee CX, Mantzavino A, Ribas J, Guerra W, Preston ND, et al. Planning an innovation marathon at an infectious disease conference with results from the International Meeting on Emerging Diseases and Surveillance 2016 Hackathon. Int J Infect Dis 2017 Dec;65:93-97 [FREE Full text] [CrossRef] [Medline]
- Angelidis P, Berman L, Casas-Perez MDLL, Celi LA, Dafoulas GE, Dagan A, et al. The hackathon model to spur innovation around global mHealth. J Med Eng Technol 2016;40(7-8):392-399 [FREE Full text] [CrossRef] [Medline]
- Kienzler H, Fontanesi C. Learning through inquiry: A Global Health Hackathon. Teach High Educ 2016 Sep 23;22(2):129-142 [FREE Full text] [CrossRef]
- D'Ignazio C, Klein LF. Data Feminism. Cambridge, MA: The MIT Press; Mar 2020.
- Costanza-Chock S. Design Justice: Community-Led Practices to Build the Worlds We Need. Cambridge, MA: The MIT Press; Mar 2020.
- DePasse J, Carroll R, Ippolito A, Yost A, Santorino D, Chu Z, et al. Less noise, more hacking: How to deploy principles from MIT's hacking medicine to accelerate health care. Int J Technol Assess Health Care 2014 Aug 06;30(3):260-264 [FREE Full text] [CrossRef]
- Day K, Humphrey G, Cockcroft S. How do the design features of health hackathons contribute to participatory medicine? Australas J Inf Syst 2017 Mar 08;21:1-20 [FREE Full text] [CrossRef]
- Pan S, Stein G, Bayus B, Tang W, Mathews A, Wang C, et al. Systematic review of innovation design contests for health: Spurring innovation and mass engagement. BMJ Innov 2017;3:227-237 [FREE Full text] [CrossRef] [Medline]
- Benchoufi M, Fournier M, Magrez D, Macaux G, Barué V, Mansilla Sanchez A, et al. Epidemium: A multidisciplinary community to tackle cancer using big and open data. In: Proceedings of the American Society of Clinical Oncology (ASCO) Annual Meeting. 2018 Presented at: American Society of Clinical Oncology (ASCO) Annual Meeting; June 1-5, 2018; Chicago, IL p. e13604 URL: https://ascopubs.org/doi/10.1200/JCO.2018.36.15_suppl.e13604 [CrossRef]
- Badger E. Are civic hackathons stupid? Bloomberg. 2013 Jul 05. URL: https://www.bloomberg.com/news/articles/2013-07-05/are-civic-hackathons-stupid [accessed 2021-12-21]
- Tress B, Tress G, Fry G, Opdam P, editors. From Landscape Research to Landscape Planning: Aspects of Integration, Education and Application. Dordrecht, The Netherlands: Springer; 2006.
- Argote L, Ingram P. Knowledge transfer: A basis for competitive advantage in firms. Organ Behav Hum Decis Process 2000 May;82(1):150-169. [CrossRef]
- Just One Giant Lab (JOGL). URL: https://app.jogl.io/ [accessed 2021-12-22]
- Attwell K, Betsch C, Dubé E, Sivelä J, Gagneur A, Suggs LS, et al. Increasing vaccine acceptance using evidence-based approaches and policies: Insights from research on behavioural and social determinants presented at the 7th Annual Vaccine Acceptance Meeting. Int J Infect Dis 2021 Apr;105:188-193 [FREE Full text] [CrossRef] [Medline]
- Peach K, Berditchevskaia A, Bass T. Collective Intelligence Design Playbook (beta). Nesta. URL: https://www.nesta.org.uk/toolkit/collective-intelligence-design-playbook/ [accessed 2021-12-21]
- HERA: A Health Platform for Refugees. URL: https://app.jogl.io/project/18 [accessed 2021-12-22]
- Vaccination Awareness Escape Game. URL: https://app.jogl.io/project/22 [accessed 2021-12-22]
- Masselot CM, Greshake Tzovaras B, Graham CLB, Jeyaram R, Finnegan G, Vitali I, et al. Co-Immune: A case study on open innovation for vaccination hesitancy and access [Data set]. Zenodo. 2021. URL: https://zenodo.org/record/4560273#.YVYlGZpBzb2 [accessed 2021-09-30]
- Qualitative Analysis of Tweets on Vaccination. URL: https://app.jogl.io/project/20 [accessed 2021-12-22]
- Commit to Get Vacc & to Promote Vaccination - HEROIC Santé. URL: https://app.jogl.io/project/115 [accessed 2021-12-22]
- Project APRICOT. URL: https://app.jogl.io/project/117 [accessed 2021-12-22]
- Harmonize Vaccination. URL: https://app.jogl.io/project/62 [accessed 2021-12-22]
- Pass It On: A Game About Vaccine Hesitancy. URL: https://app.jogl.io/project/70 [accessed 2021-12-22]
- Global Vaccination Risk Assessment. URL: https://app.jogl.io/project/73 [accessed 2021-12-22]
- Immuno. URL: https://app.jogl.io/project/9 [accessed 2021-12-22]
- Vaccine DataDump. URL: https://app.jogl.io/project/26 [accessed 2021-12-22]
- Measuring Vaccination Hesitancy From Social Media. URL: https://app.jogl.io/project/59 [accessed 2021-12-22]
- Mortality According to Access to Vaccines. URL: https://app.jogl.io/project/60 [accessed 2021-12-22]
- The Health System Matrices. URL: https://app.jogl.io/project/61 [accessed 2021-12-22]
- Meta Immune - Data Exploration of Existing DB. URL: https://app.jogl.io/project/63 [accessed 2021-12-22]
- Biloba. URL: https://app.jogl.io/project/67 [accessed 2021-12-22]
- Wakuchin Senshi. URL: https://app.jogl.io/project/68 [accessed 2021-12-22]
- Neutralizing Information About Vaccines. URL: https://app.jogl.io/project/69 [accessed 2021-12-22]
- Go Viral!. URL: https://app.jogl.io/project/71 [accessed 2021-12-22]
- Make Vaccines Affordable. URL: https://app.jogl.io/project/72 [accessed 2021-12-22]
- Identify Topics of Discussion in Vaccination Posts. URL: https://app.jogl.io/project/74 [accessed 2021-12-22]
- Detect Vaccine Administration in Social Media Patient Data. URL: https://app.jogl.io/project/78 [accessed 2021-12-22]
- Detect Vaccine Hesitancy in Social Media Patient Data. URL: https://app.jogl.io/project/79 [accessed 2021-12-22]
- Luo EM, Newman S, Amat M, Charpignon M, Duralde ER, Jain S, et al. MIT COVID-19 Datathon: Data without boundaries. BMJ Innov 2021 Jan;7(1):231-234 [FREE Full text] [CrossRef] [Medline]
- Kinder F, Harvey A. COVID-19: The medical students responding to the pandemic. BMJ 2020 Jun 15;369:m2160. [CrossRef] [Medline]
- Brereton B. EUvsVirus Hackathon project: A case study from a mentor's perspective. All Irel J Teach Learn Higher Educ 2020 Jun 30;12(2):1-8 [FREE Full text]
- Kim J, Kim H, Oh H, Ryu Y. A group recommendation system for online communities. Int J Inf Manage 2010 Jun;30(3):212-219. [CrossRef]
- Fagerholm F, Guinea A, Münch J, Borenstein J. The role of mentoring and project characteristics for onboarding in open source software projects. In: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. New York, NY: ACM Press; 2014 Presented at: 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement; September 18-19, 2014; Torino, Italy p. 1-10. [CrossRef]
- Schilling A, Laumer S, Weitzel T. Train and retain: The impact of mentoring on the retention of FLOSS developers. In: Proceedings of the 50th Annual Conference on Computers and People Research. New York, NY: ACM Press; 2012 Presented at: 50th Annual Conference on Computers and People Research; May 31-June 2, 2012; Milwaukee, WI p. 79-84. [CrossRef]
|APRICOT: Antigen Presentation Using Crispr for TB|
|CESI: Committee for Ethics, Science and Impact|
|CRI: Center for Research and Interdisciplinarity|
|HERA: Health Recording App|
|JOGL: Just One Giant Lab|
|SDG: Sustainable Development Goal|
|WHO: World Health Organization|
Edited by G Eysenbach, S Woods; submitted 18.07.21; peer-reviewed by S Rowbotham, M Das; comments to author 17.08.21; revised version received 30.09.21; accepted 03.10.21; published 21.01.22Copyright
©Camille Masselot, Bastian Greshake Tzovaras, Chris L B Graham, Gary Finnegan, Rathin Jeyaram, Isabelle Vitali, Thomas Landrain, Marc Santolini. Originally published in Journal of Participatory Medicine (https://jopm.jmir.org), 21.01.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in Journal of Participatory Medicine, is properly cited. The complete bibliographic information, a link to the original publication on https://jopm.jmir.org, as well as this copyright and license information must be included.