This study used action research to conduct a thematic summary of issues that were raised by the lecturer and students in the class. Action research is a form of systematic inquiry that involves educators engaging in a cyclical process of problem-solving about their practices. It is often used to improve teaching by identifying and addressing specific issues or challenges within a specific educational setting [11]. In this approach, the teacher is both the researcher and the participant, and the ultimate goal is to improve the teacher’s own practice and their students’ learning experiences. The original R version for longitudinal data analysis was used to accomplish this task. Subsequently, a detailed illustration of the solutions to the problems created through teaching the original version of R was presented using Jupyter Notebook. The solutions were implemented with a new cohort of students, and the students’ average satisfaction scores were compared with those of the previous cohort to validate the solutions’ effectiveness. This analysis identified areas for potential improvement, which can be useful in enhancing the sustainability of this approach.

This study was based on the longitudinal data analysis class using the tidyverse package [12]. All students had background knowledge in using Basic R and the epiDisplay package [13]. The class instruction and learning materials were shared through a circulated email system. The Department of Epidemiology, Faculty of Medicine, Prince of Songkla University (PSU) routinely collected satisfaction information from students using a web-based questionnaire (shown in Multimedia Appendix 1). The questionnaire used a five-point Likert scale and was distributed to students after class. It assessed satisfaction across five dimensions: appropriate duration, media suitability, communication skills, discussion encouragement, and critical thinking promotion. These dimensions evaluate various aspects of course satisfaction: duration pertains to time allocation for topics; media suitability measures the effectiveness of instructional materials; communication skills rate the instructor’s clarity, organization, and engagement; discussion encouragement gauges the fostering of interaction and dialogue; and critical thinking promotion examines the support for in-depth analysis and problem-solving. Higher scores in each dimension signify a more satisfactory learning experience for students.

The questionnaire was created for internal use in an arbitrary manner due to the limited number of students per annum. Consequently, no reliability study was undertaken. Routine requests were made to the students to complete the questionnaire and include their reflections on a web-based sheet after class. All data reported by the students were anonymously recorded in a secured database. This mitigated the possibility of social desirability.

In accordance with the JupyterHub guidelines [14], we established a self-hosted Jupyter server on a dedicated machine (US $8700) procured from the Division of Digital Innovation and Data Analytics (DIDA), Faculty of Medicine, at PSU. The server is equipped with a 64-core CPU and 256 GB of RAM. For the default configuration, each student was allocated a server with 1 CPU core and 500 MB of RAM. This allocation sufficed for storing their notebook and any requisite data files for the course. However, it should be noted that individual access settings can be adjusted within the server’s capacity constraints.

To initiate the server, we created a virtual machine on the DIDA server and preinstalled all necessary packages. The cost of operating this instance amounted to approximately US $20 per month, as per the university’s established rates. The management of a JupyterHub server for users necessitated that the authentication be implemented via the PSU passport service, which is provided by the Computer Center of PSU, and that resources be allocated for each user. This ensured that every student had access to essential resources without overwhelming the server’s capacity.

With the participation of students and author PK as the teacher, the classroom action research was a collaborative learning method that changed specific actions. Participants in this study included PK and all graduate students in medical fields who were taking the longitudinal data analysis class run by PK. All students had already passed a basic epidemiology exam, so it could be inferred that they possessed a foundational understanding of epidemiological concepts and were familiar with relevant basic statistical techniques, including the R base and EpiDisplay packages. All students needed to independently analyze epidemiological data to finish their research and complete their PhD or MSc in epidemiology. The first class (class 1) was taught the original R version in October 2020, and the second class (class 2) was conducted using Jupyter Notebook in July 2022. Each class took 6 hours and comprised different students. After class finished, students from both class 1 and 2 were asked to answer the same web-based satisfaction questionnaire given by the educational assisting staff.

The intended learning outcome of both class 1 and 2 was for students to exhibit competence in using R programming for the analysis of longitudinal data. PK normally observed the action of students during each class. To facilitate individualized learning within the small class setting, students were required to independently interpret results or address parallel questions after completing exercise segments on a section-by-section basis. To further promote understanding, PK presented each student with a spontaneously devised distinct problem (improvised question) that used the same technique. For example:

This approach ensures that students do not merely replicate the code provided in instructions but rather gain a comprehensive grasp of the material.

PK noted the problems that occurred and retrieved the comments reported by the students in class 1 from the database. The notes and comments were thematically analyzed to create the problem list. The problems were reviewed and used to develop the R Jupyter for the instruction of longitudinal data analysis. The R Jupyter content was developed incrementally to solve the problems, and subsequently, a flipped class [15] assignment was included as a preclass assignment as group work. The flipped class assignment may introduce bias due to the confusing effects of using Jupyter R Notebooks. However, it is impossible to avoid since it was mandated by the university in 2022. This enabled the students to exchange ideas through the web-based platform and collaboratively prepare for the longitudinal data analysis class.

PK created a mitigation plan for class 2, which included the development of the Jupyter Notebook (see our GitHub [16]) and PDF instruction file (see Multimedia Appendix 2). These materials were distributed to students 2 weeks prior to the commencement of the class. The students were allowed to use the Jupyter server using their PSU passport account [17]. The problems detected during class 2 were noted by PK. The anonymized satisfaction scores and comments from students were sent to PK a week later. Additional details of the average age and sex distribution of the students were attached; however, those were not linked with the scores to protect personal data.

This study used a thematic analysis to examine the notes and comments made by PK, which were provided by students in class 2, and compare them with thematic issues in class 1. In addition, descriptive statistics were used to compare satisfaction scores between class 1 and class 2 by ignoring parametric assumptions due to the small sample size. Opportunities for improvement were identified based on the observations made in the notes and comments gathered by PK, which were not previously observed in class 1.

This study was approved by the Human Research Ethics Committee, PSU (REC 66-104-18-1), which authorized a waiver of consent.

Table 1 summarizes the characteristics of students in two different classes. Class 1 had 9 students with a mean age of 32.9 years, while class 2 had 8 students with a mean age of 30.9 years. Both classes had a similar number of male and female students. Before starting, class 2 was given a link to access a Jupyter server and a password for internet access, and students were allowed to use any device to connect to the classroom’s wireless internet. All students chose to use their laptop.

Table 2 presents a list of thematic issues that arose during class 1, along with their corresponding mitigation plans. It also outlines particular feedback provided by students in class 1 that had to be addressed before commencing class 2.

Table 3 presents feedback on the use of Jupyter R for instruction and improvement in class 2 problems, as well as comments from the students regarding their feelings about the changes in the class.

Figure 1 shows the satisfaction ratings of the two classes (class 1 and class 2) across different dimensions of satisfaction related to their learning experience. Overall, the findings suggest that class 2 (using Jupyter R) was more effective when compared to class 1, as clearly shown by the higher mean ratings and lower variability in the ratings for class 2. The explicit suitability of media in class 2 was found to be higher than that in class 1.

Table 4 presents the opportunities for improvement in class 2 and notes/comments from both the instructor and students. PK noted that there was uncertainty regarding the long-term effectiveness of the course, as there was no standard procedure in place to monitor whether students continued to engage with coding in R after class was completed. Furthermore, there is also a lack of knowledge regarding their proficiency in solving coding challenges independently. Despite PK’s emphasis that students are able to use Jupyter Notebook when they face compatibility issues related to traditional R (we also provided a Jupyter server for the alumni), it is unclear whether students would remember this or if they would use another program altogether to conduct their data analysis. Hence, a plan was developed to devise a system that could effectively monitor the adherence of students to the practice of coding in R, such as the R Skill Challenge Activity, through the Jupyter server. The student reflections highlight the need for a teaching assistant during class, as some practice sessions run slower when students require specific help. Even though the lecturer was able to cover all the necessary material with the students within the scheduled time by using Jupyter Notebook and teaching at a new pace, the students appeared to be unwilling to wait for assistance in resolving an error while the lecturer was assisting another student. Therefore, a few statisticians will be assigned as teaching assistants in upcoming classes.

This study focused on the challenges faced in teaching R programming in epidemiology classes and proposed the use of Jupyter Notebook as a potential solution. The study aimed to evaluate the effectiveness of Jupyter Notebook in a longitudinal data analysis class and collected reflections from students in a previous class regarding the problems they encountered in learning R programming. The findings of the study indicated that Jupyter Notebook could provide an interactive and collaborative environment that improves the effectiveness and efficiency of the learning process.

Reflections on the action research process revealed that compatibility issues and package installation crashes were the most common challenges faced when teaching R programming. These challenges were resolved by using Jupyter R Notebook, which also facilitated group work and collaborative learning. This study is innovative in its use of Jupyter Notebook as a pedagogical tool for the instruction of epidemiology and, to the best of the author’s knowledge, is the first study to do so. However, previous studies in other fields [9,19-22] have revealed that Jupyter Notebook is an effective tool for teaching data analysis.

The primary strength of this study was its collaboration with students, allowing their problems to be identified so that solutions could be found to address those issues. Moreover, the use of Jupyter Notebook as a tool to enhance learning is an innovative approach to teaching epidemiology. The use of this tool was a pragmatic remedy to the obstacles encountered when instructing students in R programming within epidemiology courses. Jupyter Notebook provided an effective and efficient learning environment, enabling students to explore data and document their analysis steps in a clear and reproducible way. Moreover, Jupyter Notebook facilitates collaboration between students and instructors, allowing instructors to create interactive tutorials, assignments, and quizzes.

Unfortunately, this study’s focus on a particular class and context constrains its generalizability. Additionally, the long-term efficacy of the Jupyter Notebook method in enhancing student learning outcomes remains unreported. Future research should assess the long-term effectiveness of the Jupyter Notebook strategy in augmenting student learning outcomes. Moreover, to adhere to ethical standards during student data collection, it is crucial to establish a research protocol that delineates the process for securing informed consent prior to further evaluation. The use of a flipped classroom assignment in class 2 may have influenced the overall feedback, complicating whether the observed outcomes could be exclusively attributed to the Jupyter Notebook approach.

Considering these constraints, we propose that subsequent research should examine the long-term effectiveness of the Jupyter Notebook approach in fostering student learning outcomes while accounting for confounding factors, such as flipped classroom assignments. This will facilitate a clearer understanding of the primary effect and aid in discerning the distinct contributions of the Jupyter R notebook method to student learning.

Jupyter Notebook can enhance the learning of epidemiological data analysis for graduate students by providing an interactive and collaborative environment that allows for more efficient and effective learning. The findings of this study demonstrate that Jupyter Notebook can help address the challenges of teaching R programming in epidemiology classes, which are caused by compatibility issues with different OSs and computers.