Table 3.

Main results from the interviews

Design principles	Cycle #1	Modification	Cycle #2	Quotes
1. Multiple assessment data and multiple perspectives	Not all members discussed resident performance with their colleagues before the meeting	The group leader asked the CCC members to consult colleagues before the meeting	More members consulted their colleagues and said they formed a broader picture about resident performance by doing so	M6: ‘I did not consult my colleagues before the meeting, but I am going to schedule time before the next meeting to do so, because I think it is valuable’
2. Shared mental model	Members learned from each other about different approaches of assessing from colleagues	No modification needed	Members learned from each other about different approaches of assessing from colleagues	M6: ‘All supervision levels must be the same at every ward. So that we assess residents in the same way’
3. Interaction during the meeting	Safe atmosphere	No modification needed	Safe atmosphere	M3: ‘There was a really good atmosphere, I had the feeling I could say everything’
	Some members were hesitant to give a negative opinion	The CCC discussed this hesitation and discussed the need to give negative opinions when necessary	The members were more comfortable in giving a negative opinion when necessary	M2: ‘I cannot do that, can I? To somebody who is such a nice person, to give, to (…) I have the feeling I cannot do that!’ [give a negative opinion]
Extra	Broad and rich picture about the performance of residents	No modification needed	Broad and rich picture about the performance of residents	M1: ‘I have a broader image of the resident, especially because I now know his extracurricular activities. We did not know that before the introduction of the CCC’
	They were concerned about the extent to which private matters of the resident should be discussed	It was decided to ask the permission of residents before the meeting to discuss private matters	Private matters were only discussed when residents gave their permission	M4: ‘This level of feedback and assessment cannot be reached with feedback on paper. Now we were able to ask questions about somebody’s opinion and discuss this’