In medical education, writing is not merely a record of clinical events but a vital learning activity that supports reflection and growth. Reflective writing (RW), including the reflective elements embedded in case report writing, fosters core competencies such as clinical reasoning and professional identity formation [1,2]. However, writing becomes educationally meaningful only when feedback effectively guides learners toward explicit goals [3]. Previous studies confirm that supportive, structured, and longitudinal feedback maximizes the impact of RW [2,3].

Delivering such feedback at scale faces persistent challenges in clinical settings. Time constraints and limited instructional skills often prevent adequate feedback [1-3]. Consequently, this feedback loop frequently breaks down, hindering the professional development of novice staff.

Large language models (LLMs) have demonstrated utility for feedback, clinical reasoning, and research assistance, including health care contexts [4-6]. A recent review suggests that artificial intelligence (AI) reduces workload and improves efficiency, particularly in medical writing and administrative support [7]. These developments have increased interest in applying LLMs to educational feedback for RW.

However, realizing this potential requires purpose-built systems that go beyond general-purpose chatbot interactions, along with careful consideration of data privacy. This tutorial addresses that gap.

This tutorial provides a practical guide for educators without programming experience to develop an AI-based feedback system for case report writing in clinical education. Specifically, it offers (1) a conceptual framework distinguishing custom AI systems from standard chatbot use, (2) a step-by-step implementation guide using 3 accessible tools (Dify [LangGenius, Inc]; Slack [Slack Technologies, LLC]; and Google Apps Script [GAS; Google LLC]), (3) ethical and data privacy guidelines for deploying AI in medical education settings, and (4) empirical evaluation data from a pilot implementation at a community care hospital.

The primary audience is clinical educators who supervise novice staff writing case reports, experience time pressure when providing feedback, and lack programming expertise. Although the examples and prompts presented here focus on rehabilitation settings as a working illustration, educators in other medical fields can adapt this framework by modifying the prompts to suit their specific clinical domains and educational objectives.

General-purpose chatbots (eg, a standard ChatGPT interface) pose substantial risks when used without customization in medical education. The medical domain requires specialized terminology and linguistic precision that general models may fail to process accurately [8]. Effective AI use in medical education depends on strategic prompt design; poorly constructed instructions can yield inaccurate or unsafe outputs [9]. Mastering such prompt design techniques requires technical expertise that most clinical educators do not possess [9], making accessibility a core design requirement. These considerations led us to develop a customized system rather than rely on off-the-shelf chatbot interactions.

Three requirements distinguish a custom educational system from standard chatbot use. First, standardization ensures fairness. In direct chatbot interaction, response quality depends entirely on the user’s prompting skills. Novice staff may receive suboptimal feedback simply due to limited prompting skills. A custom system uses preprogrammed instructions, known as meta-prompts (Textbox 1), to evaluate every draft against the same rubric, ensuring consistent feedback for all learners. Second, integration enables visibility and progress management. When novice staff use personal chatbot accounts, educators cannot monitor their interactions. Integrating the system into a shared workspace enables educators to transparently track progress and intervene when necessary. Third, centralized logging supports educational auditing. To improve the curriculum, educators need to identify common errors among novice staff. A custom system automatically records all interactions, generating valuable data for analyzing trends in learner performance.

This section is organized as follows. First, we describe the 3-component system architecture (Dify, Slack, and GAS) and the dataflow connecting them. Second, we present the feedback design principles and the specifications of 4 educational chatbots. Third, we provide a step-by-step implementation guide for reproducing the system. Fourth, we address ethical considerations and data privacy safeguards. Finally, we describe a pilot implementation conducted at a community care hospital to assess system usability and feasibility.

Figure 1 illustrates the operational workflow. From the novice staff’s perspective, the system provides a straightforward interaction: the novice staff tags the relevant bot in Slack (eg, @Discussion_Bot) and submits a draft text. The system processes the request and provides specific feedback within the same thread. The novice staff then revises the text and replies to the bot again. This iterative conversation continues until the draft meets the required standards. This workflow off-loads the time-consuming revision cycle to the AI, reserving human feedback for clinical validity and professional judgment.

The system architecture consists of 3 components selected to enable implementation without advanced programming knowledge (Figure 2). Programming is a well-documented barrier for health care professionals seeking to develop AI applications [10]. We therefore adopted an approach that does not require specialized programming expertise and uses the following 3 components: Dify, Slack, and GAS.

Dify [11] is an open-source LLM application development platform that serves as the system’s central processing engine. Hosted in the cloud, it receives user inputs, processes them through a predesigned workflow, and instructs the AI how to respond. Educators can build the AI’s reasoning process visually, without writing code. We selected Dify because it offered the most accessible visual interface among available open-source LLM workflow platforms, combined with built-in application programming interface (API) publishing and conversation logging. The free tier is suitable for initial testing but has limited message credits; a paid plan may be required for sustained use with multiple learners. Data sovereignty considerations are discussed in the Ethical Considerations and Data Privacy section.

Slack [12] serves as the interactive chat interface for novice staff. Its private channel structure enables educators to monitor all novice staff-AI interactions in real time and to communicate with novice staff within the same workspace. The free tier limits message history retention, which we addressed through automatic logging to Google Sheets via GAS.

GAS functions as the technical bridge linking Slack and Dify, securely relaying messages between the 2 platforms. We chose GAS for its native integration with Google Workspace, enabling automatic logging to Google Sheets and administrator error notifications without additional infrastructure. Although Dify provides a built-in Slack plug-in for direct integration, it does not support the logging and oversight functions considered essential for educational management. When a novice staff sends a message on Slack, the data travels through GAS to reach Dify, and the response returns along the same path. Every interaction is automatically logged, visible to educators, and monitored for errors.

A key design feature is the iterative pass or revise loop implemented within Dify’s Chatflow functionality (Figure 3). Unlike a simple chatbot that answers a question once, effective education requires iterative engagement [3,13]. A recent Association for Medical Education in Europe guide indicates that continuous, formative feedback aligned with explicit learning objectives develops competencies more effectively than one-time summative evaluation [14]. The system, therefore, implements a conditional workflow:

Because this system transmits case report content through multiple external platforms, safeguarding protected health information (PHI) is essential. This section describes the data protection measures organized into 3 categories: administrative, technical, and supervisory safeguards.

The ethics committee of Kyoto Min-iren Asukai Hospital approved this pilot evaluation (2024-0502). All participating staff members received verbal and written explanations of the study’s purpose and data handling procedures, and provided written informed consent. We explained deidentification requirements to all novice staff members individually and reinforced them through an instructional video prior to system use. For past case reports used in meta-prompt tuning, we implemented an opt-out procedure. Participants received no financial compensation.

The current Dify-based system evolved through iterative refinement of an earlier prototype. The initial version used a single-prompt chatbot approach. Early testing revealed 2 challenges: a ceiling effect in which most novice staff passed all checkpoints on their first submission, indicating that the meta-prompts were too lenient for meaningful learning, and declining instruction adherence as prompt length increased. These issues motivated the transition to the Dify-based Chatflow architecture, which evaluates checkpoints sequentially and allows educators to adjust passing criteria through its visual interface.

Some instructors also expressed concern that overreliance on AI feedback could diminish clinical reasoning skills. We therefore designed the system to limit AI feedback to structural and logical aspects and to prompt learners to reconsider their own reasoning rather than providing corrected example text.

To inform the design and iterative refinement of this tutorial, we conducted a single-center pilot implementation at Kyoto Min-iren Asukai Hospital. This pilot used a prototype version with direct Slack-GAS-AI integration prior to the introduction of Dify. Because the user interface (Slack) and the fundamental feedback logic (iterative revisions) remained identical, the usability data are directly applicable to the Dify-based system.

Five novice rehabilitation staff members (3 physiotherapists and 2 occupational therapists) who joined in April 2024 and 5 educational instructors participated. The pilot ran from April to June 2024. Prior to this pilot, the facility had not used Slack for official communication; thus, this pilot represented the participants’ first experience with the platform. During the pilot, participants tagged the specific bot (eg, “@Integration_Bot”) in Slack to initiate iterative feedback on structure and logic. Once the content was finalized, they used the “@Proofreading_Bot” for grammatical corrections before submitting the final draft to their human supervisors.

We collected data through 2 methods. First, postimplementation surveys via Google Forms assessed the system’s usability and acceptance. Novice staff completed the System Usability Scale [18] and rated the appropriateness of the AI feedback content. The System Usability Scale is a standardized 10-item questionnaire for measuring system usability. Scores above 68 indicate above-average usability, and scores above 85 indicate excellent usability. Instructors assessed changes in instructional efficiency and feedback appropriateness. Second, we analyzed use logs from the Slack-linked spreadsheet, including use frequency, revision counts per task, and the content of feedback.

This tutorial presents a practical framework for building an LLM-based feedback system for case report writing in clinical education. The principal contribution is demonstrating that educators can construct a customized AI feedback system by combining 3 accessible tools (Dify, Slack, and GAS) while maintaining educational quality and data privacy.

The system embodies 2 key design principles. First, comment-based iterative feedback encourages active revision rather than passive correction, preserving the cognitive effort essential for developing clinical reasoning skills [3,13,14]. Second, the human-in-the-loop principle limits AI to structural and logical evaluation, reserving clinical validity assessment for qualified educators and mitigating the risk of unwarranted clinical authority [19].

The pilot implementation experience supported the design decisions underlying this tutorial. In particular, the transition from the prototype to the Dify-based architecture was directly motivated by 2 challenges observed during early testing: a ceiling effect indicating that initial meta-prompts were insufficiently stringent and declining instruction adherence as prompt complexity increased. These observations illustrate a core advantage of the framework presented here: educators can iteratively adjust passing criteria and feedback logic through Dify’s visual interface, enabling continuous adaptation without programming expertise.

To our knowledge, few studies have provided practical, reproducible guidance for clinical educators to build customized LLM-based feedback systems. Systematic reviews of LLMs in medical education have predominantly identified studies evaluating LLM performance on examinations and clinical knowledge assessments, with very few reporting empirical implementation research [20,21]. Practical programming-free AI development guides in health care have focused on other domains, such as image classification [10], and exploratory studies in adjacent fields have examined similar accessible tools for providing formative feedback on student writing [22]. A step-by-step guide for building an LLM-based formative feedback system for clinical writing education has not been described. This tutorial addresses that gap.

The design principle of delegating structural and logical review to AI while reserving clinical judgment for educators reflects an emerging model in which AI supports, rather than replaces, expert judgment [19,20]. Prior studies on the use of AI in education have reported reductions in teachers’ workload and stress, along with improvements in instructional efficiency [23,24]. The pilot findings were consistent: instructors reported the ability to “focus on specialized feedback” rather than routine text review. This division of labor may offer a scalable approach to feedback delivery in resource-constrained educational settings.

Accessibility was a key design consideration. Prior research indicates that AI-based educational tools positively influence learning efficiency and cognitive motivation [24,25]. The pilot results supported this, with high usability ratings achieved even among staff with no prior experience using the chat platform.

This tutorial demonstrates that rubric-based LLM evaluation can be customized for domain-specific educational purposes without programming expertise. Although the potential of AI as a writing support tool is well documented [26,27] and LLMs can reliably evaluate clinical writing against structured rubrics [28], these capabilities have typically required programming expertise, limiting accessibility for clinical educators.

Overreliance on AI feedback poses recognized risks, including cognitive deskilling and diminished independent reasoning [29,30]. A recent scoping review of ethical challenges in AI-assisted medical education underscores the need for hybrid learning models that balance AI assistance with traditional teaching [19]. This tutorial addresses these concerns through the human-in-the-loop principle: the system limits AI feedback to structural and logical aspects, and all AI-generated comments include disclaimers directing novice staff to verify content with their supervisors.

This tutorial and its pilot evaluation have several limitations. First, the small sample size (5 novice staff and 5 instructors) limits causal inference. Second, as a single-center pilot, future research should involve multicenter collaborative studies. Third, the outcomes relied on subjective questionnaires. Future studies should analyze quantitative data such as the duration of instructor guidance and use independent reviewers to evaluate report quality objectively. Fourth, although we provided deidentification training through individual instruction and an instructional video, the pilot did not include an automated presubmission PHI detection mechanism. Recent studies have demonstrated that LLM-based systems can analyze conversation data while preserving privacy [31].

No incidents of PHI disclosure were identified during the pilot period, suggesting that the current approach, which aligns with standard security practices for cloud-based services, was sufficient within this controlled setting. However, future implementations should strengthen compliance through additional safeguards: supervisor-led presubmission review, prioritization of hypothetical cases during early adoption, and development of automated PHI detection systems. In contexts requiring stricter data governance, locally hosted LLMs may also be considered.

Two broader considerations apply to the adoption of this approach. The system relies on cloud-based services, which may not meet the data sovereignty requirements of all institutions. Additionally, because LLM capabilities and API policies evolve rapidly, educators should monitor LLM provider updates regarding data handling practices and model performance. Furthermore, while this tutorial provides detailed implementation instructions and reproducible resources, the step-by-step guide has not yet been independently tested by educators outside the development team.

Future directions include broader validation of the tutorial’s reproducibility across diverse clinical settings, development of automated PHI detection as a presubmission safeguard, and investigation of the system’s long-term impact on writing proficiency and clinical reasoning development.

This tutorial provides a reproducible guide for building a case report feedback system using Dify and Slack without programming expertise. The architecture enables educators to fine-tune AI logic through visual interfaces, separating AI reasoning from software code. The tutorial also addresses data privacy considerations for deploying AI systems that handle clinical educational content. By emphasizing human oversight at every stage and iterative refinement of feedback criteria, this framework supports educators in adapting the system to their institutional context and evolving educational needs.