The summer swelter is back, and I thought it was to blame for an ache in the joints of my right pointer finger. A moment’s reflection (and a rogue fluoro pedal-stomp?) reminded me that the boneon-bone MCP joint was a testament to my ten-millionth mouse click as the first cohort of Competency by Design (CBD) residents graduate after a hailstorm of assessment.
Some years prior, a klaxon screamed from the stained-glass and marble château of the Royal College, and education leaders met to enjoy braised beef and gammon luncheons and to cleave urology into 39 discrete parts — the Entrustable Professional Activities (EPAs) — and their composite CanMEDS-linked milestones. An unimpeachable idea! Competence as the goal of training is so obvious a principle that we held aloft that 61-page PDF like 1930s newsies and set about making competent urologists out of wide-eyed PGY1s. Which is what we thought we had been doing all along?!
Among the first acts of the newly minted resident is to be observed admitting a patient to the urology service (Transition to Practice EPA#2), and graded over five milestones on a three-point scale just in case they massively effed up the process and never learned to do it right in the subsequent 59 months. Dribs and drabs of assessments follow over the summer during the “TTP” stage until, like the moment one wonders where all the dang food is at a Brazilian churrasquiera, the resident and the attendings are suddenly inundated with meat (in this case, the hundreds of assessments required to become entrusted and therefore competent). There are no red cards to wave from your greasy hands to help you here.
A useful mental model is one I like to call “step zero,” the act of asking, “Is this a problem that actually needs solving?” Does that patient actually need the catheter the ER team has been struggling with? Does the other need the nephrostomy you’ve been tasked with facilitating? Did we have a competence issue with our graduating urologists that needed correction? Was there ever data to support this? I submit there was no such data or pressing need, and no corresponding analysis plan to elaborate the outcomes of training under CBD. The measures of adherence are easy (click the boxes to complete the EPAs, compile the data on the Elentra or One45 dashboard), but what is a measure of competence?
Most onerous is the burden of assessment. More discussion and more feedback are great, but with CBD we are trying to fit the work to the forms, a contortion that demands significant time of the residents to try and ensure their EPAs are met (it is a fact that about 10% of assessments are triggered by attendings, so trainees truly bear the burden). In their lives, running from task to task, pausing to remember the day or week’s work and to fit it to an incomplete EPA is draining. It also typically separates the event from the assessment so the latter almost always takes place asynchronously without discussion. This bleeds out the substance of the interaction in favor of prompted recall of granular points, a yellow flag for low validity.
Residents then very reasonably try to match the effort of submitting requests for assessment with the likelihood that said assessment will be successful. While the dream is to see an “up and to the right” shift in entrustment scores over an abundance of iterations, in reality a) effort in submitting assessments that won’t count towards passing EPAs feels wasted; and b) submitting an assessment expecting to be admonished feels unreasonable. Absent making assessments mandatory, psychology and time management principles suggest this issue is intractable.
The solution bleated forth again and again is “faculty development.” We’re seemingly forever one session, one meeting, one bon mot away from converting faculty to rigorous and dedicated assessment mavens, shaping our charges through deft deployment of, “I had to prompt them from time to time” clicks that prompt fulsome discussion and actionable advice. Here we are though, five years later, and we’re still hoping it’s just around the corner.
All of this has led to disquiet, dissatisfaction, and dissent. The state of the discipline is a bare-bones adherence to the minimum of entrustment-as-per-EPAs. Residents submit when they feel they will make progress, faculty breeze through their button-clicking, type “keep reading!” and snap their computers shut. The College has acknowledged “several unintended negative consequences” of the implementation of CBD and announced planned changes to provide increased flexibility and agency to programs.1
One could (and if we’ve chatted, you’ve heard it) go on endlessly with examples of the frustration of this implementation of a laudable idea. The Kafka-esque software imposed on us, with hidden menus and baffling navigation to achieve the simplest of tasks, the absence of fostering excellence in the paradigm, the fact that urology programs already cultivate close relationships between trainees and teachers. I was going to do this in fact — being crusty is easy and the vocabulary is fun — until It became clear that I could simply do better if I want.
A chance discussion with an outstanding educator at the Montreal CUA meeting pinned me to reconciling with the opportunities that arise as collateral to CBD. My definition of CBD is narrow, and I think correct. It is explicitly the series of EPAs and milestones that residents complete, and the administrative roles of the academic advisor and the competency committee. That said, by forcing frequent communication and creating named roles and standards, the door opens to using these moments for good.
There is an opportunity to define for our trainees how we intend to use EPA assessments as discussion points, not permanent blemishes. We can learn (!) the EPAs, or even just the phrase, “Is there an EPA for this that we should assess?” Real-time assessment takes all of three minutes from idea to submission. Three minutes spent in a capsized canoe is a lot less fun than three minutes celebrating an overtime goal, but it still passes as quickly so is hard to justify punting on the chance to have a helpful interaction. The 30 000-ft view of increased assessment density provides an opportunity to get face-to-face in coaching mode. If we choose to waste this (
) then we’ve indulged in cynicism and abrogated our #1 role as teachers.
Clear instructions about naming academic advisors (AAs), assembling competence committees (CCs), and empowering them with decision-making authority invests a greater pool of the faculty into the assessment process. Simply facing the need to compile assessment and opinion data and to discuss resident performance in some systematic way is a huge win. Residents crave feedback, and constructive feedback specifically, so pulling thoughts together is pure opportunity, which is not the usual mode of operation for training programs. Reducing repeated acts (whether the competencies themselves or the routines of AA and CC meetings) to checklists and nudging behaviors through association may feel inorganic, but it frees time and RAM to think on substantive issues rather than scrambling to rearticulate processes and agendas on the fly.
Identifying the struggling resident can be difficult. Postgrad offices are full of stories of trainees in trouble for whom only anecdote is available and so correction is handcuffed. Seeing clues in assessment activity is a chance to both identify a struggling trainee and to see the corrective maneuvers almost declare themselves on screen. Nobody wins when a good person in a bad way passes through the asteroid belt of training unidentified, then flails at the moment of examination or in practice.
I maintain that rigorous adherence to the current implementation of CBD has a tremendously poor signal-to-noise ratio. We are not one perfect rubric away from epiphany, and urology is not 39 tasks. Perhaps though, I’m pummelling a bit of a straw man. I’m not about to sit down with the Whos and carve the roast beast here, but maybe a sliver of positivity from myself and those of us (and that’s a lot of us!) who have the chance to participate in training seems sound.