Background Direct observation with feedback to learners should be a mainstay in resident education, yet it is infrequently done and its impact on consultation skills has rarely been assessed.

Approach This project presents the framework and implementation of a longitudinal low-frequency, high-intensity direct observation and coaching intervention, and elaborates on insights learned. Internal medicine interns at one residency training program were randomized to an ambulatory coaching intervention or usual precepting. Over one year, coached interns had three complete primary care visits directly observed by a faculty clinician-coach who provided feedback informed by a behavior checklist. Immediately after each of the coached patient encounters, interns completed a structured self-assessment and coaches led a 30-minute feedback session informed by intern self-reflection and checklist items. Interns with usual precepting had two mini-CEX observations over the course of the year without other formal direct observation in the ambulatory setting.

Evaluation As part of the post-intervention assessment, senior faculty members blinded to intervention and control group assignments evaluated videotaped encounters. Coached interns completed an average of 21/23 behaviors from the checklist, while interns from the control group completed 18 (p < 0.05). The median overall grade for coached interns was B+, compared to B−/C+ for controls (p < 0.05).

Reflection Coaching interns longitudinally using a behavior checklist is feasible and associated with improved consultation performance. Direct observation of complete clinical encounters followed by systematic coaching is educationally valuable, but time and resource intensive.