PACES 2020 Short Life Working Group
The PACES 2020 short life working group (SLWG) has completed a root and branch review of the current PACES examination structure and content. The purpose of the review was to evaluate the current PACES model, identifying areas of good practice and opportunities for development. The introduction of nPACES in 2009 led to the implementation of a non-compensated skills-based marking scheme with new examiner marking methodology using seven key skills and the introduction of two new brief clinical consultation encounters in station five. PACES 2020 builds on these very successful developments to ensure that the examination remains relevant and fit for purpose in testing candidates’ clinical knowledge, skills, attitudes and behaviours as they move towards higher specialist training.
PACES 2020 Carousel
The review concluded that there are some elements of the examination that could be improved:
1. History Taking
Station 2 tests candidates on their ability to take a full and structured history. This skill remains essential to daily practice. However, it was felt that history taking is not a skill that is practiced in isolation. Candidates are not asked to examine the surrogates, but this is something that they would be expected to do in day-to-day practice. The group felt that the testing of history taking should remain in the examination, but could be better assessed as part of a more integrated clinical encounter
The brief clinical consultations in station 5 integrate communication and physical examination skills in a more realistic way. These integrated encounters contribute to the validity of the examination. The limited amount of time available for the interaction means that station 5 can be pressured for candidate and examiners. The group discussed ways in which the brief clinical consultations could be expanded to give candidates more time to demonstrate their skills. A successful proof of concept study tested new clinical consultation that lasted 20 minutes was undertaken; where candidates are given more time to complete an integrated encounter.
3. Communication and ethics
Station 4 tests candidates’ ability to communicate with patients and relatives, and demonstrate an understanding of ethical principles involved in the encounter. However, there was a sense that the 20 minute encounter is not being used as effectively as possible. Candidates often complete the clinical interaction before the end of the 15 minute allocated time, and the 5 minutes of examiner candidate interaction does not always add value to what the examiners have observed. The group concluded that there were opportunities to increase the number of communication encounters in the carousel while making the encounter shorter.
A second proof of concept study was undertaken to test ten minute encounters both with and without a minute of candidate/examiner interaction within the encounter. The results of the proof of concept and further discussion with the short life working group led to the decision to implement a ten minute communication encounter that is only observed by the examiners.
4. Reliability and validity
Recent analysis of the current PACES marking system highlighted aspects of the examination that could be altered to improve the reliability. PACES reliability rests on the high number of independent judgements made by each of the examiners. However, not all examiners contribute the same number of marks to the overall score. Examiners on stations two and four make fewer judgements in fewer skills than examiners in stations one, three and five. This reduces their impact on the overall reliability of each of the skills assessed in the examination and effectively weights the examiners’ judgements. The SLWG discussed ways to sequence the new proposed encounters to ensure that the spread of judgements by skill would be more even, and thus improve the overall reliability as examiners mark all candidates on as many skills as possible. This could be achieved by testing communication encounters alongside physical examination encounters, as shown in the new carousel diagram below.
Implementation of PACES 2020
The recommendations made by the PACES 2020 SLWG were accepted by the MRCP(UK) Academic Quality Management and Research Committee (AQMRC). The next stage of the process is to submit a request for approval to the GMC. In order to gather the relevant evidence for this submission, MRCP(UK) will be running two pilot examinations. These pilots will also be used to produce training material for examiners and candidates – further updates will follow.