This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/, as well as this copyright and license information must be included.
Medical students show low levels of e–mental health literacy. Moreover, there is a high prevalence of common mental illnesses among medical students. Mobile health (mHealth) apps can be used to maintain and promote medical students’ well-being. To date, the potential of mHealth apps for promoting mental health among medical students is largely untapped because they seem to lack familiarity with mHealth. In addition, little is known about medical students’ preferences regarding mHealth apps for mental health promotion. There is a need for guidance on how to promote competence-based learning on mHealth apps in medical education.
The aim of this case study is to pilot an innovative concept for an educative workshop following a participatory co-design approach and to explore medical students’ preferences and ideas for mHealth apps through the design of a hypothetical prototype.
We conducted a face-to-face co-design workshop within an elective subject with 26 participants enrolled at a medical school in Germany on 5 consecutive days in early March 2020. The aim of the workshop was to apply the knowledge acquired from the lessons on e–mental health and mHealth app development. Activities during the workshop included group work, plenary discussions, storyboarding, developing personas (prototypical users), and designing prototypes of mHealth apps. The workshop was documented in written and digitalized form with the students’ permission.
The participants’ feedback suggests that the co-design workshop was well-received. The medical students presented a variety of ideas for the design of mHealth apps. Among the common themes that all groups highlighted in their prototypes were personalization, data security, and the importance of scientific evaluation.
Overall, this case study indicates the feasibility and acceptance of a participatory design workshop for medical students. The students made suggestions for improvements at future workshops (eg, use of free prototype software, shift to e-learning, and more time for group work). Our results can be (and have already been) used as a starting point for future co-design workshops to promote competence-based collaborative learning on digital health topics in medical education.
Medical students’ poor mental health remains a worldwide challenge. Despite the high prevalence of mental illness among medical students [
Previous research suggests that digital interventions can be an effective tool to promote university students’ mental health [
A possible explanation for the low uptake of mHealth apps might be that existing mHealth apps do not reflect medical students’ needs and preferences [
Given the relevance of the subject, educational concepts are needed to help implement mHealth in the medical curriculum.
The primary aim of this case study is to describe the piloting of a novel co-design workshop on mHealth and e–mental Health at a German medical school. We sought to explore the feasibility of co-design workshops as an educational concept and asked for participants’ evaluations and suggestions for improvements regarding future co-design workshops (iterative development). Furthermore, we were interested in medical students’ ideas and preferences for prototypes of mHealth apps and their application of the theoretical insights conveyed during the workshop.
The participants in this case study were preclinical and clinical medical students enrolled at the medical school of Heinrich Heine University Düsseldorf (HHU) in Germany. The inclusion criteria were age ≥18 years and registration for the elective course
We conducted the co-design workshop on 5 consecutive days on site from March 2 to 6, 2020, at the Faculty of Medicine at HHU, approximately 1 week before the first COVID-19 lockdown [
Generally, each day of the workshop was structured in 2 parts. The first part consisted of introductory lectures on eHealth and participatory design methods. Moreover, the students were shown how to identify existing mHealth apps that are safe to use and are also of high quality. During the second part, the participants were divided into smaller groups to work through relevant literature on e–mental health and to develop their own hypothetical prototype for a mental health intervention. For this, they used a range of methods grounded in participatory design, design thinking, and target population–centered approaches to intervention development (
Workshop contents.
Workshop day | Educational content | Activity |
1 | Introduction, components and types of guidance for mHealtha apps, quality criteria for mHealth apps, and legal framework | Introductory lecture, group work, presentations and plenary discussions on relevant literature, legal aspects, and identification of existing mHealth apps that are safe to use and are also of high quality (MARS-Gb [ |
2 | Acceptability and user orientation, co-design and participatory design methods, and strategies and model for designing mHealth apps | Expert lecture (building your own mHealth app and insights into a medical student’s back pain app start-up), focus groups part 1 (reported elsewhere [ |
3 | Gamification, development and adjustment of mHealth apps, and avatars | Expert lecture (assessment of avatar of a certified medical app for insomnia, |
4 | Acceptance-facilitating interventions, adherence-facilitation, and implementation | Expert lecture (web-based marketing), persona development, journey mapping, implementation mapping, prototyping, development of personas in groups, and mock-ups and prototypes |
5 | Presentations and workshop evaluation | Presentations of the mHealth app concepts, feedback questionnaires, and feedback round |
amHealth: mobile health.
bMARS-G: Mobile App Rating Scale, German version.
cIDEAS: Integrate, Design, Assess, and Share.
The first author (MD) and the last author (JAH) facilitated the workshop. JAH created the contents of the workshop with the support of MD. MD is a researcher and trained psychologist. JAH is a qualified psychologist with a background in medical psychology and an experienced researcher with focus on e–mental health and mHealth acceptance in different target groups, including medical students, as well as psychosocial stress research. Both facilitators conduct lectures for medical students and are involved in research on medical students’ well-being. They took turns conducting lectures; the other observed and took field notes.
Each group, consisting of 3-7 students, focused on a different psychological problem for their hypothetical prototype of an mHealth app. Participants could choose from among the following 5 predetermined group topics, selected on the basis of their relevance for promoting mental health among medical students [
Depression and anxiety (transdiagnostic; students chose to focus on depressive symptoms)
Stress management and subjective well-being
Test anxiety and procrastination
Insomnia (focus on health behavior and sleep quality)
Psychosomatic conditions (self-management of chronic conditions; students chose to focus on gastrointestinal problems)
The medical students also had the opportunity to adapt their topic or propose other health conditions. All groups were supervised and provided with feedback during the development phase of their mHealth app.
During the workshop, the facilitators took notes and documented the workshop with photographs. All written and designed material was collected with the permission of the participants. In addition, the participants filled out a so-called logbook with predefined tasks mirroring the contents of the workshop. The logbook was also used to document their thoughts, ideas, and progress regarding the development of their prototype. The final segment of the workshop was devoted to the group presentations of the hypothetical app concepts. The presentations were rated based on predefined evaluation criteria (
Quality of the content
Relevance for the target group and field of action
Overall concept and presentation: comprehensibility, rationale, and logic
Selection of content and components (based on evidence, empiricism, etc)
Practical transfer: strategies for dissemination and execution
Implementation
Manner of presentation
Visualization
At the end of the workshop, the participants completed a brief feedback questionnaire to evaluate the workshop. They were asked 3 questions regarding their perceived learning progress during the workshop on a scale from 1 (strongly disagree) to 6 (strongly agree), and they were given the opportunity to add suggestions for improvement in free text. In addition, feedback was collected during an oral feedback round and within a standardized anonymized evaluation form for lectures at medical schools. The latter is not reported in this study. The participants’ feedback was used to make alterations and improvements for future workshops. Statistical analysis of the paper-based questionnaire data was performed using the software SPSS (version 25.0; IBM Corp).
In all, 26 participants (women: 17/26, 65%; men: 9/26, 35%) aged 18-30 years (mean 23.35, SD 3.73 years) took part in the workshop. All participants were medical students from the third to the ninth semester (mean 4.31, SD 1.87) at HHU. Of the 26 students, 16 (62%) were in their third semester, 4 (15%) were in their fifth semester, 5 (19%) were in their seventh semester, and 1 (4%) was in their ninth semester (median third semester). All (26/26, 100%) participants attended on all 5 days of the workshop and completed the course with a group presentation of their app concepts and prototypes (ie, there were no dropouts). The participants gave permission to use their intellectual work and feedback for research and publication purposes.
The common themes described in
The students also discussed strategies to improve the uptake of novel e–mental health services such as advertisements using testimonials, including potential negative effects of testimonials. They expressed skepticism regarding such testimonials, especially when they exclusively involve positive ratings. Rather, they preferred balanced reviews (including positive and negative aspects) by trusted sources. Taken together, this points to the need to include user target groups in the design process of mHealth apps to increase their acceptance and use.
On the basis of scientific evidence
Certification
Transparent quality criteria
Free of cost or cost reimbursement
Personalization
Gamification (limited and not too playful)
Easy and intuitive handling
Daily use or daily commitment (eg, reminders)
In this section, the hypothetical prototypes developed by 40% (2/5) of the groups will be presented as exemplary concepts. The 2 prototypes in this paper were chosen based on their visual clarity and comprehensive concept. All creative theoretical work and design samples belong to the students and cannot be used without their permission.
The target group consists of not only medical students but also students in general who display depressive symptoms or are experiencing a mild depressive episode (ie, early intervention as the first step or additional support). Users are encouraged to seek professional support; the app informs them that it is not a substitute for treatment.
The app includes elements of gamification, including personalization, reminders to use the diary, and motivating messages. To increase adherence and use in the long run, users can adapt the app design to their needs and preferences. They can use emojis, upload a picture, or include Graphics Interchange Format files and stickers when making a diary entry. Moreover, the app is structured in a specific order. When students complete a level, they receive a
The students described the design as colorful and esthetic. Users can create a profile and chat and interact with other users. Furthermore, the app provides a variety of helpful videos and resources, for example, to deliver psychoeducation. Therapists and scientists will verify all content in the app. The students highlighted that the app is easy and intuitive to use.
As the students were worried about people who use the app having an acute mental health crisis, an
Before the app is launched in app stores, all students from Heinrich Heine University Düsseldorf would have the opportunity to test it. If they approve it, this can be extended to other universities. Finally, students could provide testimonials for the app in diverse app stores. The entire process would be monitored and assessed by scientists at Heinrich Heine University Düsseldorf.
Another major concern highlighted by the group presenting
The app should be used daily.
Logo of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Logo of the app
The students designed this mobile app for people with insomnia who want to improve their sleep quality.
Before the first use, users can answer questions (eg, about their age;
The app is not specially targeted at medical students but rather created for people working in shifts (eg, hospital staff) as they face additional challenges because of their irregular sleep schedule. The app is also suitable for students who often study and sleep in the same place (studio apartment) and face multiple distractions because of their extensive use of smartphones and other electronic devices.
In case of severe insomnia, users receive an alert to seek help from a professional.
The avatar
Wearables can be connected to the app to improve the quality of sleep tracking. If a user wakes up during sleep, the app recognizes this and immediately offers them help to go back to sleep. In addition, the app can restrict the use of other apps during the time users want to sleep (eg, social media apps). The students stated from personal experience that when they use these apps during the night, going back to sleep becomes harder.
Further personalization of the app is possible, such as the regulation of push notifications. The app has 3 main modes for night, morning, and day (
The group that designed
The app includes several elements of gamification. The mechanism behind points and rewards is positive reinforcement. If users sleep well, they gain points that they can use to unlock new characters (eg,
Users can also track their progress (
The app should be used on a daily basis to ensure a reliable sleep profile.
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
Mock-up of the app
The feedback from participants upon workshop completion showed that, among other things, it was well-received. The students showed great interest in the presented contents, including the acquisition of knowledge about quality-approved e–mental health solutions. In addition, the students emphasized the benefits of involving potential users in app development. However, the face-to-face workshop was seen to be in need of improvement. The students requested more self-learning components and web-based tools as well as more opportunities to test e–mental health solutions. Moreover, they suggested including more group work, individual work, and interactions in terms of mutual exchange of ideas and exercises of practical relevance. By expanding the digital elective components, the elective course could also be better integrated into everyday life and thus increase the learning effect (self-directed learning and e-learning).
The feedback questionnaire suggests that most of the students were not familiar with e–mental health before the workshop (
Students suggested the following improvements in the free-text questions of the feedback questionnaire: learning more about existing apps, testing specific apps, and learning more about app design and technical implementation (“What is needed to create a good app?”).
Workshop evaluation (scale from 1=strongly disagree to 6=strongly agree).
Question | Values, mean (SD) | Values, median (IQR) |
“Before the workshop, I already knew a lot about the topics covered in the workshop” | 2.38 (1.10) | 2 (1.25) |
“I learned a lot of new things in the workshop” | 4.58 (1.10) | 4.5 (2) |
“I have learned a lot of useful things for my profession” | 3.77 (1.24) | 4 (2) |
The aim of this study is to describe an innovative concept for an educative workshop following a participatory co-design approach. Moreover, we wanted to present samples of medical students’ prototypes and ideas for mHealth apps, highlighting their preferences and needs.
The co-design workshop was well-received by medical students. It created an environment that allowed participants to engage and be creative from both user and prospective health care provider perspective. The small groups ensured that all participants were able to engage in the design process, as suggested in previous research [
Moreover, the students provided insights into how mHealth apps can be designed to meet their needs: they agreed that it would be beneficial if the app considered challenges that are specific for shift workers and students in general. Throughout all groups, one of the main issues highlighted by participants was data security provided by their app. This finding fits into the international literature, where data protection concerns have been identified as a key barrier for the adoption of eHealth across various target groups [
Confidentiality was another related topic relevant for medical students, especially because many mental health apps do not fulfill this criterion [
In addition, the participants declared customizable elements, easy and flexible use, and daily commitment as essential during the presentation of their mHealth app prototypes. This is in line with the principles of persuasive design, which has also gained importance in the health informatics educational sector in recent years [
However, the students were critical of parts of the lectures on mHealth apps because they had difficulty following them on an abstract level without an mHealth app for practical demonstration. This was due to the limited availability of freely accessible quality-approved mHealth apps, which has been acknowledged as a barrier in previous mHealth educational research with medical students [
Interestingly, the group discussions and prototypes developed in this workshop suggest that most medical students do not see an urgent need for mHealth apps directly targeted at medical students [
Several quality-approved, effective digital mental health interventions for students exist [
Another goal of the workshop was knowledge acquisition and transfer in terms of competence-based collaborative learning (ie, application of acquired knowledge).
Overall, it is striking that only 31% (8/26) of the students agreed or strongly agreed that they had learned anything they consider valuable for their future practice. It is possible that the students did not deem eHealth relevant for their profession, which hints at the need for more education to familiarize students with eHealth. Generally, this finding can be seen in light of the period when the workshop was conducted, namely in early March 2020, 1 week before the first lockdown due to the COVID-19 pandemic in Germany. The pandemic was an unexpected driver for the transition of telemedicine into health care, especially the spread of videoconferencing consultations and also e–mental health services [
Furthermore, it is possible that some contents such as the demonstration of the UTAUT2 model [
The suggestions received at this pilot workshop have been transferred to a novel e-learning participatory design workshop with medical students. It was conducted for the first time in the 2021 summer semester (e–mental health literacy as an elective subject, with support of the Medical Faculty Quality Funds for innovative educational projects). A novelty of the new e-learning workshop on e–mental health and mHealth for medical students is that it offers an extensive e-learning module on the quality criteria of mHealth apps as well as new ways of implementing remote group work (under continuous guidance by the team of educators), and their integration into routine care, design thinking, and gamification. This helps to systematically guide small groups of medical students through the design of a prototype mHealth app, alongside engaging sessions and continuous tailored feedback. The platforms used are ILIAS (an open-source digital learning platform for asynchronous self-directed learning), Webex (Cisco Systems, Inc), and Microsoft Teams (videoconference-based
Thus, the subsequent workshop included new educational content of more practical relevance for medical students, such as the prescription of apps. Moreover, the switch to e-learning seems to have facilitated knowledge acquisition significantly: the second workshop was evaluated more positively compared with the pilot workshop, and knowledge acquisition was rated consistently as
Upon completion of the quality-improvement project, the participatory workshop will be implemented as a standard elective subject in the medical curriculum at HHU. The curriculum will likely be extended to other fields in eHealth as well, such as digital health for chronic conditions. Moreover, a collaboration with the university’s computer science department is planned that could offer the opportunity to translate medical students’ ideas into actual mHealth apps. This case study lays the foundation for these ambitions by exploring medical students’ perspective in detail, providing concept sketches, and initiating communication channels.
The exploratory nature of our case report entails several limitations that must be considered. First, our results concerning medical students’ ideas and preferences for mHealth apps are not conclusively generalizable to the entire medical student population. Rather, this case study offers subjective insights into participatory workshops for educational purposes from the educators’ perspective.
A second concern related to generalizability is that participants chose the workshop as an elective course. Thus, there might be a self-selection bias if especially students familiar with, or interested in, mHealth attended the workshop. Students with no interest in mHealth could have had other ideas or preferences compared with those of the workshop participants. However, the students indicated low familiarity with eHealth, and many chose the workshop as an elective course because it was held on 5 consecutive days during the semester holidays.
Third, all participants were regular smartphone users as well as digital natives and therefore widely familiar with smartphone apps. It became evident on different occasions (eg, in group discussions and feedback rounds) that they had already formed critical opinions on related topics (eg,
Furthermore, it is likely that the different lectures, exercises, and tasks throughout the workshop (eg, creating avatars and personas as well as learning about mHealth guidelines) influenced the students’ prototypes. However, they only included elements that they perceived as useful for their prototype (eg, no avatar in
Finally, the workshop was conducted in Germany where digital health is not yet a mandatory or widespread part of medical education [
Researchers might consider the following implications and recommendations:
Develop an empirical and theory-led guide for best practice through continuous evaluation of medical students’ preferences and needs using both qualitative and quantitative research methods.
Define and test outcomes of the learning success by combining subjective and objective measures based on digital health technology literacy frameworks [
Enable the cocreation of educational content using participatory research approaches (eg, person-based approach [
Lecturers might consider the following implications and recommendations:
Define a set of clear competencies and learning goals that should be obtained through the workshop [
Provide personalized and interactive digital learning platforms in line with recent trends [
Select and use novel educational tools and web-based platforms such as Psy-Q [
Learn how to create apps—easy and intuitive software tools to build initial apps exist (eg, iBuildApp [
Encourage the collaboration of physicians and informatics experts as lecturers, for example, as shown in the DigiWissMed project in Germany [
Offer trainings in digital health for medical educators.
Medical schools might consider the following implications and recommendations:
Note that in Germany, most eHealth-related topics are taught within elective subjects, and the number of such courses is very limited [
Note that not all medical students may be interested in eHealth or in creating their own apps in the same way. Thus, basic knowledge on eHealth could be implemented in the standard curriculum, whereas more advanced or in-depth courses could continue to be part of the elective curriculum [
Overall, the participatory workshop on e–mental health was well-received by medical students. Thus, it seems to be a feasible approach that can be used as a starting point for future educational activities with medical students. Moreover, the medical students had a clear vision for their ideal mHealth apps after being informed about key quality criteria and persuasive design features. As medical students are both potential users and future health care providers, the adoption of mHealth education into the medical curriculum should be considered.
Heinrich Heine University Düsseldorf
mobile health
Nationaler Kompetenzbasierter Lernzielkatalog Medizin
Unified Theory of Acceptance and Use of Technology, extended version
The authors thank Rebecca Kastl, Roni Memis, Onur Aydemir, Amina Doua, Sophia Kiemann, Benedikt Plaack, Medina Serifi, Doris von Janowsky, Rebecca Gulba, and Christian Boschenriedter for sharing their interesting prototypes and for giving permission to use them in this article. The authors also thank Elke Heidenreich for her assistance in transcribing and summarizing the contents of the logbooks. The authors also thank Ivano Sorrentino, Christian Weber, Alexander Rötger, and Nicola Voß for the expert lectures.
None declared.