Performance Appraisal Inside the Medical Club Part 1

The previous topic concluded by arguing for the need to investigate the implementation of portfolio based performance appraisal within the medical profession.Revalidation was an unknown quantity when the research started, so the decision was made to focus upon exploring the introduction of appraisal within medicine through discussing with doctors their experience of annual appraisal as well as conducting trainee appraisal during clinical training placements. The following three research questions to guide the research:

1. What are the perceptions of ‘rank and file’ doctors involved in supervising and assessing medical students and junior doctors during workplace clinical placements concerning the recent introduction of portfolio based Annual Appraisal for them as part of their NHS contract?

2. What are the perceptions of ‘rank and file’ doctors involved in supervising and assessing medical students and junior doctors during workplace clinical placements concerning the introduction of portfolio based performance appraisal for trainees as part of recent broader reforms in the training of medical students and junior doctors?

3. Do ‘rank and file’ doctors involved in supervising medical students and junior doctors during workplace clinical placements think that the introduction of portfolio based performance appraisal has made them more personally accountable for their educational activities? Including for how they keep up to date and ‘fit to practice’ in their chosen medical specialty, as well as for how they themselves judge medical students and junior doctors as ‘fit to practice’? If not why not? If they do think they are more accountable, in what way do they think they are more accountable. What are their attitudes towards this self-reported increase in accountability?


The conduct of the research was determined by the British Sociological Association guidelines on ethical research practice. To protect their right to anonymity study participants have been given pseudonyms.The location the study took place in is referred to as ‘Blue Town’ and its medical school as ‘Blue School’. In order to answer the research questions a sample population had to be defined theoretically on the basis that to be included a doctor in Blue Town had to be subject to annual appraisal as part of their NHS employment contract, and therefore was either a consultant or general practitioner. Furthermore, they had to be involved in supervising and assessing medical students and junior doctors during their clinical training placements, and therefore had to have used a trainee’s portfolio to track and assess their progress. Not all doctors can become consultants and general practitioners. Not all consultants and general practitioners are involved in medical education. Those involved in the training of medical students are not necessarily involved in the training of junior doctors (and vice versa). 189 doctors in the Blue Town area are involved in supervising and assessing medical students and junior doctors during clinical placements. Each was contacted and asked if they would agree to participate in the research. 103 did.

The decision was made to use interviews to collect data. Primarily due to their utility as a research tool for exploring in detail with a social actor their points of view regarding a topic of mutual interest to both them and an interviewer. Good research practice dictates that data typically needs to be collected from between 30 and 50 interviewees before theoretical saturation is reached, with emergent themes fully explored and generalizable to comparable settings (Bryman 2006). 46 doctors from a range of specialities were eventually interviewed. Interviews ranged between 45 and 90 minutes in length. They were tape-recorded, transcribed in full, and securely stored. Although initially selected on the basis of their gender, age, ethnicity and medical speciality, interviewees were increasingly selected at random from the list of volunteers as key themes emerged over time and these variables were discounted as core explanatory factors concerning the ritualistic nature of the conduct of appraisal.

Data collection and analysis was divided into three overlapping stages typical to the conduct of qualitative research (Strauss and Corbin 1990). Stage one was the ‘open coding’ stage. It lasted for 15 interviews and focused upon asking open-ended questions, related to interviewee’s personal experience of appraisal. A key part of this first stage was the quantification of the rich qualitative data being generated. Counting reoccurring words and phrases is useful for categorizing informant’s experiences into themes, as well as exploring whether textual data can be safely generalized (Bryman 2006).

Stage two was the ‘axial coding’ stage and lasted for 16 interviews. Throughout this stage the research deliberately sought to disprove its developing understanding of the conduct of appraisal by framing interview questions in such a way that informants used their own personal experience to answer them in a either a positive, ‘supporting’, or negative, ‘disproving’, sense. To further ensure the validity, reliability and generalization of research data, emergent themes were thoroughly triangulated using informant characteristics.

Stage three, the ‘selective coding’ stage, lasted for 15 interviews. During this stage data is collected for confirmatory reasons. This ensured the two emergent key themes were fully saturated (i.e. no new or contradictory data was collected) and linked to an explanatory core theme, or ‘central storyline’, which as the topic will discuss later, was defined as ‘paperwork compliance’ (Strauss and Corbin 1990).

Setting the Scene: Recent Changes in Medical Training

‘Things have changed a lot around here recently’

Dr White (General Practitioner)

All forty-six interviewees recognised this fact. As the following comments from Dr Red (Surgeon) illustrate:

‘I would say I first noticed that the way we train students and juniors was beginning to change around the mid-90s. You see, the system really hadn ‘t changed since, well, I think you could reasonably argue that nothing much had changed since the 1940s (laughs)….I certainly remember starting to really notice changes after (Blue School) changed its course in 1996….For one thing students started to have better basic clinical and communication skills. They could really talk to patients, take a history and perform an initial (physical) examination, and in a far more advanced way than your average student could in my day…But then we never really got formally taught communication skills at medical school. We just got thrown in the deep-end when we were juniors…I also started to notice that I was being asked to do more and more formal teaching for students and juniors, that is on top of the more traditional ward round teaching. You could tell things were getting more structured and formal training-wise…There is more bureaucracy and a greater administrative workload involved in providing (clinical) placements for trainees these days….More and more paperwork keeps coming my way from the medical school and the (postgraduate) deanery…’.

Dr Red is directly and indirectly touching three important themes that emerged from discussion of recent reforms in medical training with research informants. First, interviewees recognised the need for changes in medical training and were supportive of recent changes to undergraduate and postgraduate training. Second, they felt that because of recent reforms clinical placements had become more structured than they had been previously. The introduction of portfolio based performance appraisal was bound up with this. Third, it was possible to identify from their accounts that although recent changes in medical training placed doctors under greater collegiate surveillance and control, their accountability to medical elites nevertheless possesses a ‘ritual quality’ (Pym 1973). Furthermore, it was possible to identify a mixture of ‘structural’ and ‘ideological’ factors which were directly affecting the implementation of portfolio based performance appraisal. This topic will now start to explore these three themes.

A Different Type of Medical Trainee

‘I think as a clinical teacher you have to accept that today’s trainee is different to yesterdays’.

Dr Orange (Physician)

All interviewees held that ‘new curriculum’ medical students and junior doctors possess a different set of skills than their predecessors. They were quick to note that today’s medical curricula are very different to the ones they themselves completed as trainees.

‘The old and new curricula are very different. You really are comparing a competency and skills based curriculum to, in a sense, a knowledge-based curriculum, which did not necessarily equip students with the skills and competence needed to be a good house officer.’

Dr Lilac (General Practitioner)

As Dr Lilac notes, medical training is becoming increasingly focused upon ensuring today’s young doctors possess key communication, study and practice skills relevant to the work tasks they will be required to complete when they begin their medical careers. Dr Yellow (Surgeon) concurs:

‘I think one of the strengths of the new system for medical students is that the students are much more able to go out and find things out for themselves, and do projects and research…You ask them to find out something and they are able and motivated to go and find it out…They also have very good communication skills. I think that overall the course does make a good job of preparing them for postgraduate training by focusing upon key practice skills’.

Given this increased focus upon preparing medical students for the job they will take up when they graduate, it should be no surprise to learn that interviewees’ held that today’s medical students are far better prepared for the role of junior doctor than their predecessors were:

‘Today’s trainees are far better prepared for their future jobs. I think a large reason why is because in final year they ‘shadow’ the house officer whose job they will take over when they graduate’.

Dr Green (General Practitioner)

The production of a ‘different type’ of junior doctor because of recent reforms in medical training is to be expected. When first published Tomorrows Doctors (GMC 1993 2002 2003) required medical schools change substantially the content of the undergraduate medical curricula and focus upon producing medical graduates who were prepared for their immediate junior house officer job (Bligh 1998). First, it required that medical schools remove the traditional preclinical-clinical training divide. Traditionally medical students had no contact with patients and learned the basic sciences in years one and two of the undergraduate course, before embarking upon ‘hands on’ clinical training during years three, four and five (GMC 1993 2002 2003). Tomorrows Doctors replaced this with a ‘core-plus-options’ model with earlier clinical contact with patients (in year one in most medical schools). As part of this, less emphasis was placed upon ensuring students memorised large amounts of basic science knowledge (Bligh 1995). Second, it required medical schools to establish ‘clinical skill training centres’ which focused upon developing core clinical and communication skills (Bradley and Bligh 1999). Third, it required medical schools to increase community-based training in the general practice setting (Parsell and Bligh 1995) Fourth, it required medical schools to develop student’s potential for independent learning and critical thinking (Bullimore 1998). This was seen as essential given the rapidly changing and expanding nature of professional knowledge, exponential growth of new medical technologies and a concurrent focus upon using ‘evidence based medicine’ to inform clinical practice (BMJ 1999).

In the context of junior doctor training, it was recognised from the 1990s onwards that junior doctors working hours and overnight ‘on-call’ duty in hospitals (with 100 hour working weeks being all too common) not only endangered patients, but also affected trainees’ well-being, educational achievements and career progression (McKee and Black 1993). The New Deal (Department of Health 1991) limited junior doctors to a maximum of 72 hours ‘on duty’ and 56 hours actual work in light of growing concern over the quality of medical practice, training and regulation (Stacey 1992). The Calman report of postgraduate training recommended more formal supervision for junior doctors as well as the introduction of protected teaching and learning time (Calman 1993). This started a process of formalisation within postgraduate training that eventually over the next decade would lead to the development of the Foundation Programme (BMA 2005). The two year foundation programme for junior doctors (introduced in August 2005) aims to provide trainees with a structured competency based, outcome focused, training programme, utilising nationally agreed clinical performance standards in specialty-specific ‘key skill’ domains (BMA 2005). These practice benchmarks are provided by the Royal Colleges. The implementation of the foundation programme locally, by the regional postgraduate deaneries responsible for junior doctor training, is supported by a regular quality assurance surveillance and inspection programme run jointly by PMETB and the GMC. This is to meet the contemporary requirement for transparent and accountable medical governance (Bruce 2007).

The Problem with Problem Based Learning

In summary, from the 1990s onwards it was accepted within the medical profession at large that how medical students and junior doctors were trained required modernisation (Bligh 1998). It should come as no surprise to learn that all the doctors interviewed recognised that recent reforms to undergraduate and junior doctor training were needed, as the following comments from Dr Red (Surgeon) and Dr White (General Practitioner) illustrate:

‘Oh yes, the changes were definitely needed, you have to keep up with the times, and I remember what it was like for me as a junior doctor…Under the old system you were very much thrown in at the deep end from day one…Often you were so tired towards the end of your shift that you could actually fall asleep standing up’.

Dr Red

‘I think it goes without saying that everybody recognised that it was about time the undergraduate course changed. I certainly had wanted things to change for a long time’.

Dr White

Although broadly supportive of recent reforms in medical training, concerns were expressed by all interviewees regarding the use of Problem Based Learning (PBL) by Blue School. PBL is a specific type of small group learning technique, first applied in the medical training context in the 1960s at McMaster medical school in Ontario. Post-Tomorrows Doctors it was adopted by many UK medical schools, such as Manchester, Glasgow, St Bartholomew’s, St Georges, Liverpool, Birmingham and Newcastle to name but a few (Maudsley 1999). The PBL method dates back to the American Pragmatist Philosopher, John Dewy, who advocated an educational method known as ‘multiple working hypothesis’ (Chamberlain 1965). Students are presented with real-life problem in the form of a written case scenario. In the context of medical education, the scenario is typically a ‘presenting patient’ who possesses certain clinical symptoms (for example, breathlessness, lack of appetite and night sweats). A group must solve the ‘problem’ and decide upon an appropriate course of patient treatment, through proactively identifying and accessing learning resources (Dolmans and Schmidt 1996). Although widely held to develop students’ self-directed and critical thinking skills, PBL has had a mixed reception within medical education (Norman 1988). It is seen by some to reduce the acquisition of basic science knowledge, due to its focus upon acquiring knowledge that is only of direct relevance to the immediate ‘problem case’ (Vernon and Blake 1993, Finucane 1998). The introduction of PBL by Blue School was held by research informants to have a direct affect upon ‘new curriculum’ medical students’ basic science knowledge base. This related to the depth of their understanding of anatomy, physiology and pharmacology. As Dr Orange (Physician) explains:

‘They do possess good clinical and communication skills but they lack a solid grounding in the key basic sciences. I am thinking of anatomy and physiology here. I know that there is a general philosophy nowadays of ‘you need to know what you need to know’, but I would argue that you have to build your general competence as a doctor upon a solid foundation of core basic science knowledge’.

It is not surprising to learn that their reduced presence within undergraduate curricula caused some comment amongst the doctors interviewed. Despite their concerns, interviewees did argue that the old system of undergraduate training tried to instil too much knowledge into medical students. They certainly did not want a return to ‘(the) old days of digesting bucket loads of facts, the vast majority of which you never really needed again once you left medical school’ Dr Lilac (General Practitioner). They recognised that a key goal of Tomorrows Doctors had been to reduce the ‘factual burden’ that medical education had long placed upon a medical student (GMC 1993 2002 2003). They also acknowledged that ‘new curriculum’ students had the necessary learning and critical thinking skills so they could ‘fill in’ gaps in their knowledge. They maintained that a typical final year medical student’s knowledge was an adequate level to work as a junior doctor. Nevertheless, they believed that Blue School should adopt more traditional didactic teaching methods in addition to using PBL. This reflected current argument within medical education circles that a ‘hybrid’ mixture of PBL and didactic teaching is the best way to train medical students, given the need to ensure they possess a core amount of basic science knowledge (Finucane, 1998). Interviewees argued that ‘(you) need to make sure that they have the core knowledge in anatomy that you will always need as a doctor, regardless of if you work in A&E or general practice, and the traditional lecture is a key way of getting that across’ Dr White (General Practitioner).

Further discussion revealed that informants argued this point because they were concerned with the possible negative impact a ‘PBL-heavy and lecture-light’ course could have upon a students’ future career progression. Trainees need to pass Royal College examinations once they finished junior doctor training, which as Dr Lilac (General Practitioner) pointed out are heavily "knowledge intensive". In summary, interviewees recognised that their own medical training had been too focused upon memorizing ‘facts and figures’, but they nevertheless felt it provided them with an essential core knowledge base that they could draw upon throughout their medical career. They argued that this was missing from the new ‘lecture-light’ PBL course at Blue School, as the following comments from Dr Black (Physician) and Dr Purple (Surgeon) serve to illustrate:

‘I know that we must not focus on knowledge of the basic sciences too much these days but I do think they should have more depth to their knowledge than they typically have at the moment…(They) can’t answer some quite key basic anatomical and physiological questions. OK, they can go and find out the answers, but that is not the point, the course teachers at (Blue School) need to make sure the fundamentals are in place really’.

Dr Black

‘They (Blue School) need to provide students with more formal teaching in key areas like anatomy, physiology andpharmacology….You have to have the building blocks in place so trainees have a fighting chance when they hit the (royal) college exams’.

Dr Purple

Exploration of doctors concerns over the use of PBL by Blue School revealed two issues highly relevant to analysis of the introduction of portfolio-based performance appraisal within the medical club. Although doctors were quite happy to discuss their concerns in interviews, their accounts showed a mixture of apathy and ambivalence when raising the topic with Blue School. It should be stressed that this was not because they felt they could not complain to Blue School out of fear of reprisal. Rather, two interrelated factors seemed to be at work here. First, they are not medical school or postgraduate deanery employees. Indeed, they felt a sense of distance between themselves and the medical school (a fact I will touch upon again later). Consequently, they may agree that changes to medical training were necessary to modernise it in the face of NHS reform, but as the following comments from Dr Yellow (Surgeon) highlight, they did not feel directly and personally responsible for something they saw as being a consequence of Blue Schools’ own choice to use PBL i.e. a reduction in students’ core basic science knowledge:

‘Yes, I did mention it in passing once at some meeting or other. I was told that the reduction in core knowledge was a consequence of the changes being made, and that was that really.It’s really down to them, not me, to do something about it. That is if they want things to be different of course, and I don’t get the impression that they do;.

Second, their accounts reinforced the fact that there is a natural tendency within the medical club for doctors to respect each others ‘turf’ and right to autonomous practice, regardless of if they are working in medical education or clinical practice (Stacey 1992 2000). As the following comments from Dr Green (General Practitioner) serve to illustrate:

‘Did I mention my concerns to (Blue School)? Look, when you have been doing this for as long as I have you realise that whatever happens at the medical school or the (postgraduate) deanery happens there, and whatever happens here in the hospital happens here, and never the twain shall meet. I mean, obviously we both want to turn people into good competent doctors. But do you really think the medical school or (postgraduate) deanery cares what we think or do past a certain level? No, we leave each other alone to get on with out jobs..(so) what matters to them most is making sure they are enough (clinical) placements for trainees…(and) that all the necessary paperwork is filled in and returned to them on time’.

Dr Green’s comments reinforce the importance of the need to pay close attention to ‘rank and file’ medical practitioners perceptions of the relationship between doctors responsible for providing trainees with clinical placements and assessing their clinical performance, and the medical elites responsible for the content, governance and quality assurance of medical training. In the context of the restratification thesis, it is important to identify how accountable they feel to medical elites for what they do educationally, given the contemporary move toward a ‘standards driven’ open, transparent and accountable form of governance within the medical club. Not least of all because this strategy is being used by medical elites to sustain professional autonomy, in the form of the principle of medical self-regulation, in the face of growing calls for lay, managerial and inter-professional involvement in medical regulation. The next section of this topic will discuss this point in relation to the implementation of portfolio based performance appraisal within medicine.

Next post:

Previous post: