Mobile Technology in E-Learning for Undergraduate Medical Education on Emergent Otorhinolaryngology–Head and Neck Surgery Disorders: Pilot Randomized Controlled Trial

Background The use of mobile technology in e-learning (M-TEL) can add new levels of experience and significantly increase the attractiveness of e-learning in medical education. Whether an innovative interactive e-learning multimedia (IM) module or a conventional PowerPoint show (PPS) module using M-TEL to teach emergent otorhinolaryngology–head and neck surgery (ORL-HNS) disorders is feasible and efficient in undergraduate medical students is unknown. Objective The aim of this study was to compare the impact of a novel IM module with a conventional PPS module using M-TEL for emergent ORL-HNS disorders with regard to learning outcomes, satisfaction, and learning experience. Methods This pilot study was conducted at an academic teaching hospital and included 24 undergraduate medical students who were novices in ORL-HNS. The cognitive style was determined using the Group Embedded Figures Test. The participants were randomly allocated (1:1) to one of the two groups matched by age, sex, and cognitive style: the IM group and the PPS group. During the 100-min learning period, the participants were unblinded to use the IM or PPS courseware on a 7-inch tablet. Pretests and posttests using multiple-choice questions to evaluate knowledge and multimedia situational tests to evaluate competence were administered. Participants evaluated their satisfaction and learning experience by the AttrakDiff2 questionnaire, and provided feedback about the modules. Results Overall, the participants had significant gains in knowledge (median of percentage change 71, 95% CI 1-100, P<.001) and competence (median of percentage change 25, 95% CI 0-33, P=.007) after 100 min of learning. Although there was no significant difference in knowledge gain between the two groups (median of difference of percentage change 24, 95% CI −75 to 36; P=.55), competence gain was significantly lower in the IM group compared with the PPS group (median of difference of percentage change −41, 95% CI −67 to −20; P=.008). However, the IM group had significantly higher scores of satisfaction (difference 2, 95% CI 2-4; P=.01), pragmatic quality (difference 1.7, 95% CI 0.1-2.7; P=.03), and hedonic stimulation (difference 1.9, 95% CI 0.3-3.1; P=.01) compared with the PPS group. Qualitative feedback indicated that the various games in the IM module attracted the participants’ attention but that the nonlinearly arranged materials affected their learning. Conclusions Using M-TEL for undergraduate medical education on emergent ORL-HNS disorders, an IM module seems to be useful for gaining knowledge, but competency may need to occur elsewhere. While the small sample size reduces the statistical power of our results, its design seems to be appropriate to determine the effects of M-TEL using a larger group. Trial Registration ClinicalTrials.gov NCT02971735; https://clinicaltrials.gov/ct2/show/NCT02971735 (Archived by WebCite at http://www.webcitation.org/6waoOpCEV)

Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study for undergraduate medical students 1b) ABSTRACT: Structured summary of trial design, methods, results, and conclusions NPT extension: Description of experimental treatment, comparator, care providers, centers, and blinding status.

1b-i) Key features/functionalities/components of the intervention and comparator in the METHODS section of the ABSTRACT
Mention key features/functionalities/components of the intervention and comparator in the abstract. If possible, also mention theories and principles used for designing the site. Keep in mind the needs of systematic reviewers and indexers by including important synonyms. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The participants were randomly allocated (1:1) to one of two groups matched by age, sex, and cognitive style: the IM group, and the PPS group.

1b-ii) Level of human involvement in the METHODS section of the ABSTRACT
Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study During the 100minute learning period, the participants were unblinded to use the IM or PPS courseware on a 7inch tablet.
1b-iii) Open vs. closed, web-based (self-assessment) vs. face-to-face assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. o ine), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials). Note: In traditional o ine trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered. To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e. participants can self-enrol). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 1b-iii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Pretests and posttests using multiple choice questions to evaluate knowledge and multimedia situational tests to evaluate competence were administered. The global satisfaction scale and AttrakDiff2 learning experience questionnaire were also given to each participant, who also provided feedback about the modules.

1b-iv) RESULTS section in abstract must contain use data
Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study problems associated with otorhinolaryngology head and neck surgery (ORLHNS) comprise 20% to 50% of presenting complaints to a primary care provider. Therefore, educating medical students about ORLHNS is an extremely important part of their UME. However, there have been longstanding concerns regarding the low priority assigned to ORLHNS in the UME curriculum, and a substantial mismatch between this educational need and existing curricula has been reported to result in significant downstream effects on managing ORLHNS problems in family medical practice [2]. METHODS 3a) Description of trial design (such as parallel, factorial) including allocation ratio Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons Does your paper address CONSORT subitem 3b? * currently provided for undergraduate medical students. The successful application of elearning requires that it meets the needs of both the learners and program, and it should be aligned with the contexts in which it is used [9]. Furthermore, individual differences may also play an important role in the effectiveness of MTEL. For example, learners with a fieldindependent (FI) cognitive style have been reported to prefer e learning technologies and to have a better performance with hypermedia systems than fielddependent (FD) learners, because they use active approaches and make better transfer of concepts in new situations [10].
23; IBM, Armonk, NY, USA), computer generated lists of random numbers were created for the allocation of the students, who were stratified by center with a 1:1 allocation using a fixed block size of 6 (Rv. Uniform [0, 1]) in both parallel subgroups. We concealed the allocation sequence from those assigning participants to intervention groups until the moment of assignment and adhered to our computergenerated randomization protocol. The participants were randomly assigned (1:1) to the IM group and PPS group after they had provided consent and completed the GEFT and pretests.
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No, this study have no important changes to methods after trial commencement.

3b-i) Bug xes, Downtimes, Content Changes
Bug xes, Downtimes, Content Changes: ehealth systems are often dynamic systems. A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug xes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have in uenced study design such as staff changes, system failures/downtimes, etc. [2].

Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Major bug fixes were performed before the pilot study.

4a) Eligibility criteria for participants
Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The inclusion criteria were as follows: 1) age > 20 years; and 2) undergraduate medical students (defined as three or four years of medical school training [clerkship]). The exclusion criteria were: 1) previous ORL HNS training; and 2) declining to participate.

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clari ed.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study All of them had at least a basic level of computer literacy, and they were also introduced to the practical aspects of using tablets and applications.
4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. o ine), e.g., from an open access website or from a clinic, and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasianonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email con rmation, phone calls) were used to detect/prevent these.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Twentyfour volunteers were included in the prestudy at Linkou Chang Gung Memorial Hospital (an academic teaching hospital), Taoyuan, Taiwan from November 23, 2016 to January 14, 2017. There were six different facetoface assessments in this study.

4a-iii) Information giving during recruitment
Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.

Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Blinding to the purpose of the prestudy during recruitment was maintained to minimize preparation bias. Written informed consent was obtained from all participants.
https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1… 10/34 4b) Settings and locations where the data were collected Does your paper address CONSORT subitem 4b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Twentyfour volunteers were included in the prestudy at Linkou Chang Gung Memorial Hospital (an academic teaching hospital), Taoyuan, Taiwan from November 23, 2016 to January 14, 2017.

4b-i) Report if outcomes were (self-)assessed through online questionnaires
Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study There were no selfassessed outcomes through online questionnaire in this study.

4b-ii) Report how institutional a liations are displayed
Report how institutional a liations are displayed to potential participants [on ehealth media], as a liations with prestigious hospitals or universities may affect volunteer rates, use, and reactions with regards to an intervention.(Not a required item -describe only if this may bias results) 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.

5)
The interventions for each group with su cient details to allow replication, including how and when they were actually administered

5-i) Mention names, credential, a liations of the developers, sponsors, and owners
Mention names, credential, a liations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Con ict of interest" section or mentioned elsewhere in the manuscript).

Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.

Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial.
Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).
1 2 3 4 5 subitem not at all important essential emergency physicians and two ORLHNS department chiefs who designed the learning objectives and developed the instructional content according to the needs assessment. In the second round, 10 junior residents rated the relative importance of each item. We then developed a storyboard and courseware, implemented installation and distribution, managed the learner's activities, and evaluated their reactions, learning behavior, and results using an instructional system design model including five phases (analysis, design, development, implementation, and evaluation) [12].

Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Major bug fixes were performed before the pilot study.

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.

Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Two investigators from the study team reviewed the instructional content of each module using the Software Evaluation Checklist [15]. This checklist includes seven items (curriculum connections, age/grade appreciates, investment justification, layout, support materials, instructional content, graphics/multimedia) with two (yes, no) scales (a total of 28 questions). The overall items were confirmed to be significantly correlated (Spearman correlation test, r=0.91, P<.001).

5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screencapture video, and/or providing owcharts of the algorithms used
Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing owcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scienti c reporting.

Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study a parkour style) to obtain learning materials (seven text image slides per disorder) in the four domains of the 10 disorders ( Figure 5). The instructional materials were briefly explained using scrolling text. After they had read the material for two disorders or completed 10 disorders, they had to complete small gamebased quizzes that were designed to emphasize the key points and enhance their working memory [14].
Notably, contexts of the gamebased quizzes were different from those of the multiple choice questionnaires (MCQs) and multimedia situational tests (MSTs).

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No, the application has not been released to date.

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of speci c group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-vii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Before starting the MTEL, the functionality of the tablet was explained to the participants. The students were unblinded after randomization. The students in the PPS group used an app on a 7inch tablet to watch video lectures in 10 lineardesigned sessions and review the instructional materials in an ordinary office environment for 100 minutes. subitem not at all important essential Does your paper address subitem 5-viii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.

Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The students in the PPS group used an app on a 7inch tablet to watch video lectures in 10 lineardesigned sessions and review the instructional materials in an ordinary office environment for 100 minutes.

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).

Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study design, development, implementation, and evaluation) [12]. Subsequently, the content was translated into an elearning app including a novel gamified IM module and a conventional visualauditory textimage PPS module. We created an 80minute storyboard for each module, and both modules had the same design of user interface ( Figure 2). We also created a learning map to allow the learners to assess their progress in each session or their progress overall. Moreover, both modules contained simple slides for review purposes after completing the brief sessions ( Figure 3).
The students in the PPS group used an app on a 7inch tablet to watch video lectures in 10 lineardesigned sessions and review the instructional materials in an ordinary office environment for 100 minutes.

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study There were no prompts/reminders used.

5-xii) Describe any co-interventions (incl. training/support)
Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Before starting the MTEL, the functionality of the tablet was explained to the participants. 6a) Completely de ned pre-speci ed primary and secondary outcome measures, including how and when they were assessed Does your paper address CONSORT subitem 6a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The percentage change in MCQ score (i.e. "knowledge gain") after the M TEL was the primary outcome measure. The percentage changes in MST (i.e. "competence gain"), GSS, and AttrakDiff2 questionnaire scores were the secondary outcomes.

6a-i) Online questionnaires: describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed
If outcomes were obtained through online questionnaires, describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed [9].
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 6a-i?

Copy and paste relevant sections from manuscript text
There were no online questionnaire in this study.

6a-ii) Describe whether and how "use" (including intensity of use/dosage) was de ned/measured/monitored
Describe whether and how "use" (including intensity of use/dosage) was de ned/measured/monitored (logins, log le analysis, etc.). Use/adoption metrics are important process outcomes that should be reported in any ehealth trial. Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).

Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text Anonymous feedback We used anonymous feedback to assess "quality of learning". Each participant in this pilot study also provided anonymous feedback about the quality of the module used after the MTEL.
6b) Any changes to trial outcomes after the trial commenced, with reasons Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No, we did not change trial outcomes after the trial commenced.
7a) How sample size was determined NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed

7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size
Describe whether and how expected attrition was taken into account when calculating the sample size.

Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 7b) When applicable, explanation of any interim analyses and stopping guidelines Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Nothing.
8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 8b) Type of randomisation; details of any restriction (such as blocking and block size) Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study MCQ: mean 31, standard deviation 16, effect size 1.94; percentage change in MST: mean=45, standard deviation=52, effect size=0.87). In this pilot feasibility study, we needed to confirm the students could gain knowledge and competence significantly. We estimated the sample sizes by a priori calculation (onesample Wilcoxon signedrank test, two tailed, normal parent distribution, α=0.05, power=0.95) and found that we needed at least 7 subjects for knowledge gain and at least 21 subjects for competence gain. Due to a fixed block size of 6, we determined that the sample size of the pilot study was 24.
23; IBM, Armonk, NY, USA), computer generated lists of random numbers were created for the allocation of the students, who were stratified by center with a 1:1 allocation using a fixed block size of 6 (Rv. Uniform [0, 1]) in both parallel subgroups. We concealed the allocation sequence from those assigning participants to intervention groups until the moment of assignment and adhered to our computergenerated randomization protocol. The participants were randomly assigned (1:1) to the IM group and PPS group after they had provided consent and completed the GEFT and pretests.
Using the Random Number Generator in IBM SPSS software (version 23; IBM, Armonk, NY, USA), computergenerated lists of random numbers were created for the allocation of the students, who were stratified by center with a 1:1 allocation using a fixed block size of 6 (Rv. Uniform [0, 1]) in both parallel subgroups. 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study We concealed the allocation sequence from those assigning participants to intervention groups until the moment of assignment and adhered to our computergenerated randomization protocol.
10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Using the Random Number Generator in IBM SPSS software (version 23; IBM, Armonk, NY, USA), computergenerated lists of random numbers were created for the allocation of the students, who were stratified by center with a 1:1 allocation using a fixed block size of 6 (Rv. Uniform [0, 1]) in both parallel subgroups. Twentyfour volunteers were included in the prestudy at Linkou Chang Gung Memorial Hospital (an academic teaching hospital), Taoyuan, Taiwan from November 23, 2016 to January 14, 2017.
11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment

11a-i) Specify who was blinded, and who wasn't
Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The students were unblinded after randomization.

11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator"
Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".

Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Blinding to the purpose of the prestudy during recruitment was maintained to minimize preparation bias.

11b) If relevant, description of the similarity of interventions
(this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not relevant.
12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study

12a-i) Imputation techniques to deal with attrition / missing values
Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No missing value in this study.

12b) Methods for additional analyses, such as subgroup analyses and adjusted analyses
Does your paper address CONSORT subitem 12b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study

x26-ii) Outline informed consent procedures
Outline informed consent procedures e.g., if consent was obtained o ine or online (how? Checkbox, etc.?), and what information was provided (see 4a-ii). See [6] for some items to be included in informed consent documents.

Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No outline informed consent in this study.

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study We used anonymous feedback to assess "quality of learning". Each participant in this pilot study also provided anonymous feedback about the quality of the module used after the MTEL.

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study  Table 2 summarizes the variables of interest for the overall study cohort. There were no significant differences in age, sex, cognitive style, MCQ score, or MST score between the IM and PPS groups at baseline.
13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT ow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study  Table 2 summarizes the variables of interest for the overall study cohort. There were no significant differences in age, sex, cognitive style, MCQ score, or MST score between the IM and PPS groups at baseline. After randomization, all participants (100%) received the intended intervention. There was no protocol deviation in the prestudy.

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other gures or tables demonstrating usage/dose/engagement. Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the gure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study After randomization, all participants (100%) received the intended intervention. There was no protocol deviation in the prestudy.

14a) Dates de ning the periods of recruitment and followup
Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Twentyfour volunteers were included in the prestudy at Linkou Chang Gung Memorial Hospital (an academic teaching hospital), Taoyuan, Taiwan from November 23, 2016 to January 14, 2017.

14a-i) Indicate if critical "secular events" fell into the study period
Indicate if critical "secular events" fell into the study period, e.g., signi cant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No, there was no critical "secular events" fell into the study period.
14b) Why the trial ended or was stopped (early) Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The pilot study stopped after recruitment of a sufficient sample size.

15) A table showing baseline demographic and clinical characteristics for each group
NPT: When applicable, a description of care providers (case volume, quali cation, expertise, etc.) and centers (volume) in each group Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Table 2 summarizes the variables of interest for the overall study cohort. There were no significant differences in age, sex, cognitive style, MCQ score, or MST score between the IM and PPS groups at baseline.

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 15-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Table 2 summarizes the variables of interest for the overall study cohort. There were no significant differences in age, sex, cognitive style, MCQ score, or MST score between the IM and PPS groups at baseline.
16) For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups 16-i) Report multiple "denominators" and provide de nitions Report multiple "denominators" and provide de nitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at speci c pre-de ned time points of interest (in absolute and relative numbers per group). Always clearly de ne "use" of the intervention.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).

Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study After randomization, all participants (100%) received the intended intervention. There was no protocol deviation in the prestudy.
17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% con dence interval) Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Descriptive statistics were expressed as median and 95% confidence interval (CI).

17a-i) Presentation of process outcomes such as metrics of use and intensity of use
In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational de nitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is de ned (e.g., timeout after idle time) [1] (report under item 6a).

Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study

Yes.
18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-speci ed from exploratory Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information Overall, all participants showed a significant improvement in MCQ score (i.e. "knowledge gain"; median of percentage change, 71 [95% CI, 14-100]; P<.001; compared with the neutral value of 0) and a significant improvement in MST score (i.e. "competence gain"; median of percentage change, 25 [95% CI, 0-33]; P=.007; compared with the neutral value of 0) after 100 minutes of learning. The MTEL positively impacted the GSS (median of difference, 2.5; 95% CI, 1.0-4.0; P=.002; compared with the neutral value of 5), PQ (median of difference, 1.7; 95% CI 0 2 0 P 003 d ith th t l l f 0) HQ S not in the ms, or brie y explain why the item is not applicable/relevant for your study

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).

Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 19) All important harms or unintended effects in each group (for speci c guidance see CONSORT for harms) Does your paper address CONSORT subitem 19? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study There were no important harms nor unintended effects in each group.

19-i) Include privacy breaches, technical problems
Include privacy breaches, technical problems. This does not only include physical "harm" to participants, but also incidents such as perceived or real privacy breaches [1], technical problems, and other unexpected/unintended incidents. "Unintended effects" also includes unintended positive effects [2].

Does your paper address subitem 19-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study There were no privacy nor technical problems in each group.

19-ii) Include qualitative feedback from participants or observations from staff/researchers
Include qualitative feedback from participants or observations from staff/researchers, if available, on strengths and shortcomings of the application, especially if they point to unintended/unexpected effects or uses. This includes (if available) reasons for why people did or did not use the application as intended by the developers.

Does your paper address subitem 19-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study DISCUSSION 22) Interpretation consistent with results, balancing bene ts and harms, and considering other relevant evidence NPT: In addition, take into account the choice of the comparator, lack of or partial blinding, and unequal expertise of care providers or centers in each group

22-i) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use)
Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 22-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information The qualitative feedback from the PPS group emphasized that they found the PPS module 'easy to use and follow', 'clear layout', 'enhanced knowledge', 'suitable small sessions', and 'simulated lectures'. However, they also reported that the module was 'tedious', 'hypnogenetic', and 'difficult to play back'. The IM group reported that the IM module was 'fun learning (attractive)', contained 'enjoyable small gamebased quizzes', and was an 'amazing learning experience'. However, they also considered it 'difficult to use and follow', that it contained 'nonlinear instructional materials' and 'some tough games'.

Does your paper address subitem 22-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study However, results from this pilot study suggest that instructors may need to provide other learning methods to reinforce students' competency. While the small sample size reduces the statistical power of our results, especially with regards to cognitive style, its design seems to appropriate to determine the effects of MTEL using a larger group. 20) Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study regard to competence gained (small effect), the students preferred the IM module to the PPS module due to it being more efficient and enjoyable to use (smalltointermediate effect). However, qualitative feedback recommended that both modules needed to have better quality of design and function. Since the development of the IM module was more timeconsuming (3 months vs. 1 months) and more expensive (USD$12,500 vs. 2,500) than that of the PPS module, the IM module needs to be further improved with regard to competence gain in the future.
distribution of the cognitive styles will provide more accurate data. However, FD volunteers are not frequently encountered in our undergraduate medical students (less than 10%). Moreover, it is very difficult to perform probability sampling at a regular medical school. Second, we did not investigate social interaction, selfmotivation, and selfregulation (important elements of elearning [2224]) in detail. The effects of MTEL on these factors during the learning process will be closely monitored when the MTEL app is made available to the students. 21) Generalisability (external validity, applicability) of the trial ndings NPT: External validity of the trial ndings according to the intervention, comparators, patients, and care providers or centers involved in the trial

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 21-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The use of different learning strategies is one of the most important prerequisites of academic success among undergraduate medical students and can lead to a positive attitude towards learning [4]. MTEL using an IM module seems to be an effective, enjoyable and pragmatic way to instruct emergent ORLHNS disorders in undergraduate medical students.

21-ii) Discuss if there were elements in the RCT that would be different in a routine application setting
Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.

Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study OTHER INFORMATION 23) Registration number and name of trial registry impact on the outcome of competence gain. In the past, Bertini et al. [39] found that FI learners are more likely to be worse at "tests requiring learners to recall information in the form or structure that it was presented" than FD learners. In the study, MSTs have been designed for evaluating competence with regard to the clinical reasoning process. The nonlinear structure of presentation of the IM module might limit FI learners to recall information to answer a 5question MST. However, interactive gamebased learning seems to be a promising didactic tool to achieve higher longterm knowledge retention [40].

Does your paper address subitem X27-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The authors declare no competing financial interests.

About the CONSORT EHEALTH checklist
As a result of using this checklist, did you make changes in your manuscript? * yes, major changes yes, minor changes no What were the most important changes you made as a result of using this checklist?
Make the presentation of the study protocol and results more clearly.
How much time did you spend on going through the checklist INCLUDING making changes in your manuscript *

About 4 hours
As a result of using this checklist, do you think your manuscript has improved? * yes no Other:

Would you like to become involved in the CONSORT EHEALTH group?
This would involve for example becoming involved in participating in a workshop and writing an "Explanation and Elaboration" document yes no Other: