There has been a large increase in the number of medical graduates in Australia, from 1544 in 2007 to a projected 2912 in 2012.1 Providing more training positions and rotations is one of the reasons the Prevocational General Practice Placement Program (PGPPP) exists. It includes 10–13 week rotations into general practice for interns (PGY1) and postgraduate year 2 and 3 doctors (PGY2/3).2 General practice registrar training numbers have also increased from 600 in 2007, to a projected 1200 in 2013 as a response to general practitioner shortages.3 It is necessary therefore to rapidly increase both the number of trainers and availability of training practices. This increase and the inexperience of the trainees (PGPPP and registrars) make the question of patient safety during the training process even more pertinent.
Due to the projected increase of medical graduates and
general practice registrars, a rapid increase in new trainers
and practices is required. The resulting mix of relatively
inexperienced trainers and trainees makes the examination of
the important question of patient safety even more pertinent.
To describe practical techniques that look beyond the
door of the closed consulting room to detect unconscious
incompetence in trainees.
Trainees can both be conscious of their incompetence and
ask for help, or unconscious of their incompetence. Many
articles have been written on teaching trainees who ask for
help, but it is the trainee who does not ask for help who may
be at most risk of serious problems, and therefore compromise
patient safety. Formative assessment and feedback should be
used to empower trainees as self-regulated learners. There are
seven principles of good feedback practice that help develop
self-regulation. This article provides practical teaching tips for
supervisors in general practice.
A patient safety event is an adverse event or a 'near miss'. Patient safety events may be 'sins of commission' or 'sins of omission'.4 One way of looking at adverse events is the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) taxonomy,5 which uses a standardised terminology and classification schema. There are five primary classifications in JCAHO (Table 1), this article will focus on impact or patient harm (Figure 1).
Given the solitary nature of general practice, the first requirement for the trainee practitioner is to be aware if harm has occurred or may occur. General practice differs from the hospital system where initial training for nurses and doctors has traditionally taken place. In hospitals, patients are usually seen by multiple health workers on multiple occasions within a treatment hierarchy in an open ward environment. This lends itself to ongoing scrutiny of health practitioner performance. In contrast, general practice tends to be a more isolated experience, consisting of a patient being seen by one health worker in a consulting room (behind a closed door) and often only once for any particular condition. Because presentations to general practice may be early in the disease process, patient adverse events are often subtle and difficult to ascertain.
In USA hospital practice, there is evidence of what is called the 'July effect'. This refers to mortality increases; rates of medical error increase and efficiency decreases in hospitals because of end of training or employment year change-over in staff.6 With general practice registrars changing practice every 6 months, and more junior trainees changing every 3 months, this effect may be seen up to four times every year in a training general practice.
Trainees who are conscious of their incompetence are likely to ask for help. However the 'unconsciously incompetent' are not capable of recognising when they need help. Literature abounds on teaching trainees who ask for help, but it is the trainee who does not ask for help who may be at most risk. Kruger and Dunning7 found that the lowest quartile of performers consistently rate their performance as better than actual measured performance.
Medical insight as defined by Hays et al8 is 'awareness of one's performance in the spectrum of medical practice'. It is essential to appropriate requests for help. The three related constructs: selfawareness; awareness of others; and reflection/judgement can be developed with a suitable educational framework which develops a capacity for self-regulated learning.
Table 1. JCAHO patient safety event taxonomy: primary classifications5
||The outcome or effects of medical error and systems failure, commonly referred to as harm to the patient
||The implied or visible processes that were faulty or failed
||The characteristics of the setting in which an incident occurred and the type of individuals involved
||The factors and agents that led to an incident
|Prevention and mitigation
||The measures taken or proposed to reduce incidence and effects of adverse occurrences
Figure 1. JCAHO patient safety event taxonomy: impact
In higher education, formative assessment and feedback is the accepted method of engaging students as self-regulated learners.9 This core assumption of active engagement in learning and learner responsibility for the management of learning is evident in the design of both the Australian College of Rural and Remote Medicine (ACRRM)10 and The Royal Australian College of General Practitioners (RACGP)11 curricula. However, there is still a strong tradition among doctors of relying mainly on transmitting information (cognitive external feedback) to trainees about strengths and weaknesses in the belief that the trainee will subsequently make improvements. Unfortunately this teaching style may lessen the development of self-regulation and there is a risk of the feedback being too complex or difficult to decipher. Giving information only may ignore the motivational beliefs of the trainee. It is also time intensive for the conscientious supervisor. This suggests that utilising good feedback principles to develop trainee self-regulation would be more effective and time efficient.
Nicol et al9 describe seven principles of good feedback practice that develop self-regulation (Table 2). Self-regulation generates internal feedback at a range of levels: cognitive, motivational and behavioural. Principle 3 emphasises the crucial role of external feedback in complementing internal feedback. It is essential for this external feedback to be timely, offer corrective advice instead of just information on strengths and weaknesses, direct trainees to higher goals and involve praise alongside constructive criticism.
Principle 5 emphasises the important role that belief patterns have in learners. Mangels et al12 differentiate students into two groups: those who believe that ability is fixed and so there is a limit to what they can achieve (entity view); and those who believe that ability is malleable and depends on the amount of effort (incremental view). The entity view learner has ego involvement in the task and the incremental view learner has task involvement. Ego involvement puts the learner at self-esteem risk and may lessen the likelihood of asking for help. Particular care must be taken to develop the capacity of these trainees to engage in self-regulated learning.
Experience shows that some characteristics of trainees should sound alarm bells:
- punctuality and time management problems (they always arrive late)
- communication problems with patients or staff (they do not listen or explain)
- avoidance of feedback opportunities (they never want to ask for help)
- does not learn from past experience (they keep making the same mistakes)
- defensive justifications when safety events occur (they always have an excuse)
- selective about information sources (the 'specialist/ hospital says this is the way to do it , so I will not do it the way you suggest')
- confirmation bias or selective hearing (they ignore suggestions on areas to work on and just hear what is going well).
Table 2. Seven principles of good feedback9
|Clarify what good performance is
|Facilitate self-assessment (reflection) in learning
|Deliver high quality feedback information
|Encourage teacher and peer dialogue rather than a 'transmission'
|Encourage positive motivation and self-esteem
|Provide opportunities to close the gap
|Use feedback to improve teaching
Practical teaching tips
Studies of junior doctors in hospital practice found that direct observation (82%), or recognition of critical incidents (59%) such as inappropriate prescribing or failure to organise investigations, accurately identified those doctors having difficulties.13,14 This supports the belief that direct observation and audit for pinpointing critical incidents is essential for both patient safety and appropriate training of practitioners within general practice. Because of the isolated nature of general practice the supervisor has to create a learning environment where not only is the trainee comfortable asking for help, but the supervisor has the opportunity to become aware of the trainee's unconscious incompetence. We call this 'looking beyond the door'. Practical teaching techniques that perform this function follow, with reference to Nicol's principles of good feedback.9
For teaching to be consistently delivered within a busy general practice, methods that are easy to use and deliver high quality feedback are necessary. We believe that regular teaching sessions facilitating self-assessment (principle 2), with appropriate feedback that enables self-regulation, are fundamental.
From long experience in medical education and GP supervision, we have found the following to be useful techniques to 'look beyond the door'.
Random chart review
As utilised in several overseas studies,4,15 a random chart review is a structured opportunity for self-assessment integrated with immediate tutor feedback (principle 2, 3). For PGY2/3 who are not required to have each patient signed off and registrars who work independently, assign one 30 minute teaching session per week. PGPPP interns need each patient to be 'signed off', creating the ideal time for chart review.16
Ensuring that feedback is provided in relation to pre-defined criteria (domains) increases quality (principle 2), such as:
- 'Open this record and assess it from the domains of general practice' (either the five RACGP or the seven ACRRM domains)
- 'What went well?'
- 'What would you have done differently?'
- Then, 'Here is what I think went well and here is what I think are some priorities for you to work on'. Emphasis on the specific performance NOT the person (principle 5)
- Limiting the amount of feedback increases the likelihood of use, so three well thought-out feedback comments are optimum.
Assign at least one regular weekly 30 minute session to sit in on the trainee. There are two advantages to this, unlike most teaching sessions the practice keeps running because one of you is consulting, and unlike video recording, the entirety of physical examination, history and communication skills are observed. We have found the following rules essential for efficacy:
- The trainee must tell the patient, 'I will be conducting the consultation today. Doctor [supervisor] is just sitting in and will not be participating unless I want an opinion'
- After the initial greeting exchange with the patient, the supervisor should not make eye contact with the patient for the next 90 seconds, rather, looking at the trainee
- After every consultation use the same questions, as for random chart review.
Daily trainee self-reflection
Daily self-reflection on consultations using a patient unmet needs/ doctor educational needs recording sheet creates a dynamic learning plan.17,18 This encourages a documentary approach rather than a 'still life' approach and develops a rotation long learning plan.
Audit of test ordering
A weekly random audit of pathology and radiology ordering is an essential rule. For example: 'Let us look at your inbox of pathology and radiology for today. What guideline did you use in ordering this particular test?'
Video recording and review
Discuss this consultation with respect to the domains of general practice. What went well? What would you have done differently? This can be useful every 3 months to allow the trainee to observe their communication style.
What we have found useful
As supervisors, we have found the following concepts to be useful:
- Supervisor familiarity with the curriculum and evidence based guidelines is necessary for the supervisor to be capable of clarifying what is good performance8 (principle 1)
- Aim for dialogue not a mere transmission of information when giving feedback (principle 4)
- We have found that 'pulsed teaching', ie. increasing the frequency of teaching at the beginning of a rotation and decreasing toward the end, helps mitigate the 'July effect'
- Reflection on the information coming from both trainee selffeedback and your feedback to the trainee is useful to shape future teaching (principle 7)
- Set topic tutorials can be a time trap and may not meet true imperatives. Tutorials, if used, should be in response to immediate educational needs
- Consider analysing the trainee's learning style if adult self-directed learning appears to be blocked.
This article has not attempted to examine the more difficult topic of patient safety with remote supervision. When the trainee is geographically remote from the supervisor 'looking beyond the door' is much more difficult.
We believe that patient safety depends upon detection of unconscious incompetence by regular use of practical techniques that look beyond the door of the closed consulting room.
Conflict of interest: none declared.
- Commonwealth of Australia. Medical Training Review Panel: fourteenth report. 2011. Available at www.health.gov.au/internet/main/publishing.nsf/Content/work-pubs-mtrp-14 [Accessed 2 December 2011].
- Australian General Practice Training. PGPPP. Available at www.agpt.com.au/PrevocationalTraining/PGPPPHome [Accessed 26 August 2011].
- Australian Government Department of Health and Ageing. Building a national health and hospitals network — training record numbers of GPs. 2010. Available at www.health.gov.au/internet/ministers/publishing.nsf/Content/mr-yr10-nr-nr047.htm [Accessed 2 December 2011].
- Hayward RA, Asch SM, Hogan MM, Hofer TP, Kerr EA. Sins of omission: getting too little medical care may be the greatest threat to patient safety. J Gen Intern Med 2005;20:8:686–91.
- Chang A, Schyve PM, Croteau RJ, O'Leary DS, Loeb JM. The JCAHO patient safety event taxonomy: a standardized terminology and classification schema for near misses and adverse events. Int J Qual Health Care 2005;17:95–105.
- Young JQ, Ranji SR, Wachter RM, Lee CM, Niehaus B, Auerbach AD. "July effect": impact of the academic year-end changeover on patient outcomes: a systematic review. Ann Intern Med 2011;155:309–15.
- Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc Psychol 1999;77:1121–34.
- Hays RB, Jolly BC, Caldon LJ, et al. Is insight important? Measuring capacity to change performance. Med Educ 2002;36:965–71.
- Nicol DJ, Macfarlane-Dick D. Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 2006;31:199–218.
- Australian College Rural and Remote Medicine. Curriculum. Available at www.acrrm.org.au/curriculum [Accessed 30 September 2011].
- he Royal Australian College of General Practitioners. Curriculum for general practice. Available at www.racgp.org.au/curriculum [Accessed 30 September 2011].
- Mangels JA, Butterfield B, Lamb J, Good C, Dweck CS. Why do beliefs about intelligence influence learning success? A social cognitive neuroscienceence model. Soc Cogn Affect Neurosci 2006;1:75–86.
- Yao DC, Wright SM. The challenge of problem residents. J Gen Intern Med 2001;16:486–92.
- Yao DC, Wright SM. National survey of internal medicine residency program directors regarding problem residents. JAMA 2000;284:1099–104.
- Gaal S, Verstappen W, Wolters R, Lankveld H, van Weel C, Wensing M. Prevalence and consequences of patient safety incidents in general practice in the Netherlands: a retrospective medical record review study. Implement Sci 2011;6:37.
- City Coast Country General Practice Training. PGPPP frequently asked questions. Available at www.ccctraining.org/public/Pgppp_faq_practices_supervisors.aspx [Accessed 2 December 2011].
- Doctor's educational needs (DENS) from patients unmet Needs (PUNS). Available at www.wentwest.com/treat/index_files/Page2194.htm [Accessed 30 September 2011].
- Eve R. PUNS and DENS: discovering learning needs in general practice. Milton Keynes: Radcliffe Publishing, 2003.