Introduction In postgraduate medical education, group decision-making has emerged as an essential tool to evaluate the clinical progress of residents. Clinical competency committees (CCCs) have been set up to ensure informed decision-making and provide feedback regarding performance of residents. Despite this important task, it remains unclear how CCCs actually function in practice and how their performance should be evaluated.

Methods In the prototyping phase of a design-based approach, a CCC meeting was developed, using three theoretical design principles: (1) data from multiple assessment tools and multiple perspectives, (2) a shared mental model and (3) structured discussions. The meetings were held in a university children’s hospital and evaluated using observations, interviews with CCC members and an open-ended questionnaire among residents.

Results The structured discussions during the meetings provided a broad outline of resident performance, including identification of problematic and excellent residents. A shared mental model about the assessment criteria had developed over time. Residents were not always satisfied with the feedback they received after the meeting. Feedback that had been provided to a resident after the first CCC meeting was not addressed in the second meeting.

Discussion The principles that were used to design the CCC meeting were feasible in practice. Structured discussions, based on data from multiple assessment tools and multiple perspectives, provided a broad outline of resident performance. Residency programs that wish to implement CCCs can build on our design principles and adjust the prototype to their particular context.