We undertook this pilot study to evaluate the feasibility and utility of a neck-mounted wearable camera (THINKLET, Fairy Devices Inc) for retrospective analysis in a simulated OSCE environment. Additionally, the study incorporated a questionnaire to assess the device’s usability and acceptability. The OSCE scenario focused on the placement of 12-lead electrocardiogram (ECG) electrodes, a skill frequently evaluated in OSCEs. We selected the ECG electrode placement because it requires close observation of hand positioning and electrode placement relative to anatomical landmarks, which can be difficult to capture adequately using fixed-position cameras.

A fifth-year medical student volunteered to serve as the examinee. Nine faculty members, all with experience in clinical education, participated as evaluators. Demographic data were available for 8 evaluators; 1 evaluator did not complete the demographic questionnaire. All 9 evaluators took part in the initial live assessment; however, 1 of 9 (11%) evaluators did not participate in the subsequent video-based reassessment. We summarized continuous variables using means and SDs and presented categorical variables as frequencies and percentages.

The research team developed the OSCE scenario and checklist in accordance with the publicly accessible guideline, “Learning and Evaluation Items for Skills and Attitudes Required for Students Participating in Clinical Clerkships,” published by the Common Achievement Tests Organization. The scenario encompassed a standard patient interaction and a procedural task related to the placement of ECG electrodes. The checklist, refined through expert consultation, comprised 26 binary items, with responses recorded as “1=performed” or “0=not performed.”

The neck-mounted wearable camera used in this study was the THINKLET system, operated through the on-premises version of the Mirai Connect platform (Mirait One Systems Corporation). This device comprises an integrated camera and microphone, facilitating real-time communication via long term evolution or Wi-Fi. For this study, we used video and audio recordings. We recorded offline on a general-purpose PC and subsequently stored the recordings on a dedicated server. We captured audio in pulse code modulation 24-bit 48 kHz format and provided it to evaluators as complete video files. We obtained video recordings from both the fixed camera and the neck-mounted camera (refer to Multimedia Appendix 2 for an example of wearable camera footage). Figure 1 illustrates the configuration of the neck-mounted device. The device was rented from Mirait One Systems Corporation as a commercial service. The company provided technical support for device operation and troubleshooting but had no role in the study design, data collection, data analysis, interpretation of the results, or manuscript preparation.

Following the evaluations, each participant completed a questionnaire designed to assess 2 primary aspects: the comfort and usability of the wearable device during use, and the perceived ease and effectiveness of evaluating student performance using footage from wearable devices rather than fixed-camera footage. The questionnaire comprised 8 closed-ended items rated on a 5-point Likert scale, along with 2 open-ended questions. The questionnaire items on device usability and video-based evaluation are provided in Tables S2 and S3 of Multimedia Appendix 1.

For each checklist item, we descriptively compared evaluability (“evaluable” vs “not evaluable”) between recordings from a fixed camera and those from a neck-mounted camera. Results are presented as counts and percentages of evaluable and nonevaluable observations for each camera type.

We summarized responses to the Likert-scale items by calculating frequencies and percentages. We used thematic coding to analyze free-text responses, facilitating the identification of recurring content categories. Furthermore, we manually applied a basic sentiment classification, categorized as positive, negative, or neutral, to each response based on its overall tone. We selected this manual approach to ensure contextual accuracy and minimize the risk of misclassification due to nuanced expressions or cultural subtleties that automated sentiment analysis tools frequently overlook.

We performed data analyses using Microsoft Excel and R (version 4.3.3; R Foundation for Statistical Computing).

The Juntendo University Ethical Review Board (approval number E24-0093) approved this study. The study followed the Strengthening the Reporting of Observational Studies in Epidemiology reporting guidelines. All participants, including the student examinee and faculty evaluators, provided written informed consent before participation, including consent for video recording and publication of selected materials. The student examinee received compensation for participation, whereas the faculty evaluators did not receive financial compensation. Because the study involved video and audio recordings that could contain identifiable information, the data were stored on a secure institutional server with access restricted to the research team. Identifiable information, such as faces and name badges, was masked in publication materials when appropriate. A selected video excerpt was submitted for publication.

A total of 9 evaluators participated. Demographic data were available for 8 of 9 (89%) evaluators, as 1 of 9 (11%) evaluators did not complete the demographic questionnaire. Among these 8 evaluators, 7 (88%) were men and 1 (13%) was a woman, with a mean age of 42.1 (SD 8.9) years and a mean of 14.9 years (SD 9.1) since graduation. All respondents had graduated from a medical school. Overall, 3 (38%) evaluators had prior experience as officially certified OSCE examiners, having completed the formal national accreditation process, while the remaining 5 (63%) had no such certification. In total, 5 (56%) evaluators had never previously participated in an OSCE evaluation, whereas 3 (33%) had done so ≥5 times.

Figure 2 presents visual examples of the perspectives captured by the neck-mounted and fixed cameras. In the fixed-camera recordings, some tasks requiring fine motor skills, such as electrode placements at C5 and C6, were more frequently judged as not evaluable than in the neck-mounted camera recordings (Table 1). Across all checklist observations, 206 were judged evaluable in the neck-mounted camera recordings, compared with 185 in the fixed-camera recordings (Table 2). Several electrode placement tasks, including C5 and C6 placement, were more frequently judged evaluable in the neck-mounted camera recordings than in the fixed-camera recordings (Table 1). Multimedia Appendix 2 provides an example of this type of footage.

In the context of video-based evaluation using wearable camera footage, 7 (88%) of 8 evaluators rewound and rewatched the videos during assessment. In total, 3 (38%) of 8 evaluators reported difficulty hearing audio, and 3 (38%) reported difficulty observing the desired area. Half (n=4, 50%) of the evaluators agreed, and 1 (13%) strongly agreed, that video quality was equivalent to that of on-site evaluation. Most (n=7, 88%) evaluators strongly agreed or agreed that video-based evaluation should be integrated into OSCE assessment. Tables S6 and S7 in Multimedia Appendix 1 provide detailed results, and Table 3 shows a summary of free-text feedback.

The evaluators’ positive comments commonly noted good usability and lack of discomfort, while their negative comments frequently mentioned heat sensation, device slippage, and difficulty handling the evaluation sheet. Counts represent the number of coded comments derived from the free-text responses.

This study investigated the feasibility of using an examiner-worn neck-mounted camera for OSCE assessment. Agreement between live and video-based scoring varied across evaluators, with Cohen κ values ranging from 0.258 to 0.913. In the descriptive comparison of camera perspectives, more checklist items were judged evaluable in neck-mounted camera recordings than in fixed-camera recordings, particularly for selected electrode placement tasks requiring fine motor observation. Evaluator feedback indicated both perceived usefulness and practical limitations. Although many evaluators supported incorporating video-based assessment into OSCE practice, others reported technical challenges, including audio issues, visibility of the target area, heat sensation, device slippage, and difficulty handling the evaluation sheet (Table 3). Together, these findings suggest that an examiner-worn neck-mounted camera may be useful as a supplementary tool for review and verification in selected OSCE tasks, rather than as a replacement for live assessment. Further refinement of device setup and recording conditions is needed before broader implementation.

This study directly compared live and video-based OSCE assessments using neck-mounted and fixed-camera recordings in a simulated educational scenario. Our methodology provided descriptive information regarding the evaluability of individual checklist items. We also gathered and reported evaluator background information, thereby enhancing the transparency of the study context. Furthermore, we incorporated both quantitative and qualitative feedback on device usability to support the interpretation of the feasibility findings.

The COVID-19 pandemic, which began in 2019, prompted a significant shift toward remote and online methods across various sectors, including medical education [20]. As telemedicine, online learning, and virtual assessments proliferated [21,22], the adoption of wearable technologies also gained traction [22].

Wearable devices have garnered increasing attention as instruments for clinical education and performance evaluation [23]. The expansion of remote and hybrid teaching during the COVID-19 pandemic further accelerated interest in their application for clinical skills training. These devices provide a first-person perspective, offering wearable camera recordings that facilitate detailed observation of fine motor skills and examinee behaviors [24,25]. In addition to passive recording, certain devices also support real-time, bidirectional communication, enabling remote instructors or supervisors to deliver immediate feedback during clinical tasks. This interactive capability further enhances their utility in remote education and simulation-based training environments. In online education, these recordings have been associated with increased student satisfaction and perceived educational effectiveness [23,26]. These characteristics may make wearable cameras a potentially relevant option for examiner-perspective recording in OSCE assessment.

Although wearable devices offer distinct advantages, they may not be suitable for all assessment domains; for example, fixed third-person cameras may be more appropriate for evaluating nonverbal communication [25]. Wearable and related observational technologies have been explored in health research, OSCE-related contexts, and other applied settings [27-31]. In this study, we focused specifically on the potential role of an examiner-worn neck-mounted camera in OSCE assessment.

Although various wearable devices, including smart glasses and action cameras, have been used globally to evaluate communication with standardized patients and develop instructional content [32,33], they are not without limitations. Reported challenges encompass device weight, image instability, limited field of view, and privacy concerns [34,35]. Smart glasses have been associated with visual distraction and diminished attentional focus because of their design, including a limited field of view and camera instability. These limitations may impede clear observation of procedural tasks and compromise the reliability of recorded footage, particularly in high-stakes assessment settings such as OSCEs [36-38]. These challenges indicate the importance of ergonomic and design considerations in wearable camera systems. These considerations are particularly relevant to the use of examiner-worn cameras in OSCE assessment, where stable first-person recording of procedural tasks is required. Research has identified smart glasses as less user-friendly regarding operability and comfort [36], which influenced our decision to implement a neck-mounted camera design. In this pilot study, evaluator feedback suggested that the neck-mounted camera was generally acceptable to some users and may offer practical advantages for reviewing selected procedural tasks.

Nonetheless, we observed some technical limitations. For example, visibility was occasionally obstructed when evaluators held the checklist close at hand, which sometimes blocked the camera’s view. In addition, there were challenges related to audio capture because the recording software did not allow remote control of the microphone’s on-off function due to privacy regulations. Consequently, evaluators could not easily confirm whether audio was being recorded, which occasionally made it difficult to ensure adequate audio quality for precise assessment.

Furthermore, we noted variability in interrater agreement (Cohen κ=0.258-0.913), potentially influenced by inconsistencies in video framing. In certain instances, key procedural steps were not clearly visible, and evaluators reported misaligned camera angles. These issues likely resulted from variations in how the device was worn, as factors such as shirt collars, ties, or fabric thickness affected the camera’s orientation and stability.

To address these challenges, future efforts should focus on standardizing attachment protocols, introducing supplementary supports to minimize clothing-related interference, enhancing microphone sensitivity, and providing user interfaces that indicate audio recording status. Conducting test recordings before actual assessments may also help ensure optimal video framing and sound quality. Moreover, broader implementation will require the establishment of standardized OSCE video-recording protocols, structured evaluator training programs, and secure data governance frameworks that comply with relevant ethical and legal standards.

This pilot study had some limitations. First, the small sample size and limited number of evaluators constrained the generalizability of the findings. Additionally, the examinee was a fifth-year medical student participating in a mock OSCE, which may not fully replicate actual examination conditions. Second, the evaluators were predominantly faculty members from a single institution, with only 1 evaluator from another university, thereby limiting the diversity of perspectives. Third, the study focused exclusively on a single OSCE scenario, ECG electrode placement, rendering it challenging to extrapolate the findings to other clinical skills or specialties. Additionally, this study did not evaluate scoring accuracy against an external reference standard; therefore, the findings should not be interpreted as evidence that neck-mounted recordings improve assessment accuracy. Furthermore, agreement analyses were based on reassessment using the neck-mounted recordings, whereas fixed-camera recordings were used only to judge whether checklist items were evaluable. Technical issues, including audio quality, visibility, and device positioning, also limited the consistency of video-based reassessment. Future research involving larger sample sizes, multi-institutional collaboration, and multiple OSCE stations is necessary to validate the broader applicability of neck-mounted cameras.

This pilot study investigated the feasibility of using an examiner-worn neck-mounted camera in preclinical OSCE assessments. A greater number of checklist items were judged evaluable in neck-mounted camera recordings than in fixed-camera recordings, particularly for selected tasks requiring fine motor observation. These findings suggest that an examiner-worn neck-mounted camera may be useful as a supplementary tool for review and verification in selected OSCE tasks rather than as a replacement for live assessment. However, technical and operational challenges remain, indicating the need for further refinement of device design and evaluation protocols in medical education before broader implementation.