Research publications

Introduction

Thousands of people around the world have an interest in our examinations and we work hard to keep everyone well informed.
 
MRCP(UK) is actively engaged in research and ongoing development work in order to prove the examinations’ relevance and importance to training and medical education. It is committed to academically validating the examinations. The research and analysis of candidate performance, production of statistics and standard-setting that MRCP(UK) carries out for the examinations is essential to maintain and improve the quality of the exams.

Academic research publications

Those with an interest in medical education might like to follow developments in the peer-reviewed medical literature. The papers listed below are a good starting point and are available to view.

Title The predictive validity of a situational judgment test, a clinical problem solving test and the core medical training selection methods for performance in specialty training
Description This paper examines the long-term validity of CMT and GP selection methods in predicting performance in the Membership of Royal College of Physicians (MRCP(UK)) examinations
Authors Fiona Patterson, Safiatu Lopes, Stephen Harding, Emma Vaux, Liz Berkin, and David Black
Date published 1 February 2017
Download Clinical medicine website

Title How valuable is physical examination of the cardiovascular system?
Description The review investigates the evidence for the clinical value of physical examination of the cardiovascular system.
Authors Andrew Elder, Alan Japp, Abraham Verghese
Date published 27 July 2016
Download BMJ Clinical Research

Title Exploring cultural and linguistic influences on clinical communication skills: a qualitative study of International Medical Graduates
Description This research looks at international medical graduates’ clinical communication skills and in particular their ability to seek, detect and acknowledge patients’ concerns in PACES examination, stations 2 and 4. Hofstede’s cultural dimensions framework is used to look at the impact of culture on examination performance.
Authors Anju Verma, Ann Griffin, Jane Dacre, Andrew Elder
Date published 10 June 2016
Download BMC Medical Education

Title Assessment at UK medical schools varies substantially in volume, type and intensity and correlates with postgraduate attainment
Description This study shows that there are substantial differences in the volume, format and intensity of undergraduate assessment between UK medical schools and that these differences appear to directly correlate with performance in postgraduate examinations.
Authors O P Devine, A C Harborne, I C McManus
Date published 11 September 2015
Download BMC Medical Education

Title

Cross-comparison of MRCGP & MRCP(UK) examinations' performance

Description This study performs a database linkage of 2,284 candidates taking both examinations highlighting the validity and differential performance by ethnicity.
Authors Richard Wakeford, MeiLing Denney, Katarzyna Ludka-Stempien, Jane Dacre, I C McManus
Date published 16 January 2015
Download BMC Medical Education

Title UK postgraduate medicine examinations: opportunities for international candidates
Description This article describes the examinations on offer internationally from the Federation of Royal Colleges of Physicians of the UK, explains their development, local benefits and explores their future.
Authors L McAlpine, E Selamaj, C Shannon, L Chis, J Dacre, A Elder
Date published 1 October 2014
Download Clinical Medicine October 2014 14:500-505

Title Implementing statistical equating for MRCP(UK) parts 1 and 2
Description Statistical equating was successfully introduced into the MRCP(UK) Part 1 and Part 2 written examinations in 2008 and 2010 respectively. This paper describes that equating has higher predictive validity than the previous Angoff/Hofstee standard setting, provides a robust standard-setting method, with a better theoretical foundation than judgemental techniques such as Angoff, and is more straightforward and requires far less examiner time to provide a more valid result.
Authors IC McManus, L Chis, R Fox, D Waller and P Tang
Date published 26 September 2014
Download BMC Medical Education

Title

Investigating possible ethnicity and sex bias in clinical examiners: an analysis of data from the MRCP(UK) PACES and nPACES examinations

Description This paper demonstrates that there is no evidence of ethnic or gender bias in examiner judgements in the MRCP(UK) Part 2 Clinical Examination (PACES). The method of analysis used in this study is based on comparison made between scores awarded by white and non-white examiners in the assessment of the same candidate, on the same station. This method is only effective in examinations where there are two independent examiners at all examination stations, as is required for PACES, which contributes to its status as a high-quality and reliable assessment.
Authors IC McManus, A Elder and J Dacre
Date published 30 July 2013
Download BMC Medical Education website

Title Performance at MRCP(UK): when should trainees sit examinations?
Description This paper analyses candidate performance in the three components of the MRCP(UK) examination to establish when they are most likely to pass.
Authors C Pinheiro-Torres, P Tang and J Dacre
Date published April 2013
Available Clinical Medicine, Vol 13, No 2: 166–169

Title Resitting a high-stakes postgraduate medical examination on multiple occasions: nonlinear multilevel modelling of performance in the MRCP(UK) examinations
Description The paper demonstrates that candidates continue to show evidence of true improvement in performance up to at least the 10th attempt at MRCP(UK) Part 1, although there are individual differences in the starting level, the rate of improvement and the maximum level that can be achieved. The authors point out that while this does not seem to support a fixed number of attempts at an examination, unlimited attempts are also difficult to justify because of the ever increasing role that luck must play with increasing numbers of resits.
Authors IC McManus and K Ludka
Date published 14 June 2012
Download BMC Medicine website

Title Changing PACES: developments to the examination in 2009
Description As a consequence of change in medical, educational and regulatory practice, MRCP(UK) successfully modified the international PACES examination in 2009. This brief paper explains the rationale for change and summarises the development and implementation process.
Authors A Elder, L McAlpine, N Bateman, J Dacre, P. Kopelman, IC McManus
Date published June 2011
Available Clinical Medicine, Vol 11, No 3: 231–234

Title Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis
Description A systematic review and meta-analysis of ethnicity and academic performance in UK-trained doctors and medical students, which includes and references MRCP(UK) data. The paper concludes that UK-trained doctors and medical students from minority ethnic groups tend to underperform academically compared with their white counterparts, but that it is not clear why this might be.
Authors K Woolf, HWW Potts, IC McManus
Date published 8 March 2011
Download BMJ website

Title What skills are tested in the new PACES examination?
Description The MRCP(UK) PACES examination assesses skills of relevance to the practice and delivery of high quality clinical care and is the leading international postgraduate summative assessment of this kind. In 2009, the examination was revised, and this article describes those revisions, focussing on the clinical skills assessed.
Authors A Elder, IC McManus, L McAlpine, J Dacre
Date published 3 March 2011
Download Annals Academy of Medicine Singapore 40:119–125

Title The standard error of measurement is a more appropriate measure of quality for postgraduate medical assessments than is reliability: an analysis of MRCP(UK) written examinations
Description The authors of this study tested the hypothesis that SEM is a better measure of the quality of an assessment, because it is unaffected by the ability range or number of candidates taking an examination.
Authors J Tighe, IC McManus, NG Dewhurst, L Chis, J Mucklow
Date published 2 June 2010
Download BMC Medical Education website

Title An empirical examination of the impact of group discussion and examinee performance information on judgments made in the Angoff standard-setting procedure
Description This study examines the impact of group discussion and candidate performance data on the judgments made by experts during the process of standard-setting. The paper provides a detailed analysis of the relationship between question difficulty and the degree of changes experts make in their judgments after group discussion of discrepancies.
Authors BE Clauser, P Harik, MJ Margolis, IC McManus, J Mollon, L Chis, S Williams
Date published 1 January 2009
Available Applied Measurement in Education, Vol 22, No 1: 1–21 (available with subscription). For more detail see Education Resources Information Center website.

Title Graduates of different UK medical schools show substantial differences in performance on MRCP(UK) Part 1, Part 2 and PACES examinations
Description The aim of this study is to assess the performance of UK graduates who have taken MRCP(UK) Part 1, Part 2 and PACES (Practical Assessment of Clinical Examination Skills) and explore the reasons for the differences between medical schools.
Authors IC McManus, AT Elder, A de Champlain, JE Dacre, J Mollon, L Chis
Date published 14 February 2008
Download BMC Medicine website

Title Performance in the MRCP(UK) Examination 2003–4: analysis of pass rates of UK graduates in relation to self-declared ethnicity and gender
Description Male students and students from ethnic minorities have been reported to underperform in undergraduate medical examinations. The aim of this study was to assess effects of ethnicity and gender for UK medical graduates on pass rates in the MRCP(UK) examination sat in the UK in 2003–4.
Authors N Dewhurst, IC McManus, J Mollon, J Dacre, AJ Vale
Date published 3 May 2007
Download BMC Medicine website

Title Assessment of examiner leniency and stringency ('hawk-dove effect') in the MRCP(UK) clinical examination (PACES) using multi-facet Rasch modelling
Description Clinical examinations require that judgements of candidates are made by experienced examiners. A potential vulnerability of any clinical examinations is that examiners differ in their relative leniency or stringency. This study looks at the paired judgements made by examiners in the MRCP(UK) Part 2 clinical examination (PACES) and calculates examiner stringency.
Authors IC McManus, J Mollon, M Thompson
Date published 18 August 2006
Download BMC Medical Education website

Title Changes in standard of candidates taking the MRCP(UK) Part 1 examination, 1985 to 2002: analysis of marker questions
Description This paper looks at marker questions (questions re-used unchanged in several examinations) in  the MRCP(UK) Part 1 examination between 1985 and 2002 to assess whether standards have changed. The authors conclude that there was a sudden, relatively steep decline in the performance of graduates from UK medical schools taking the MRCP(UK) Part 1 examination. The reasons for this are not clear and are not an artefact of changes in the mix of overseas and UK candidates, or changes in the time after qualifying of first or subsequent sittings of the examination, but they do have implications for medical education.
Authors IC McManus, J Mollon, OL Duke, AJ Vale
Date published 18 July 2005
Download BMC Medicine website

Title Reliability of the MRCP(UK) Part I Examination, 1984–2001
Description For an examination to be robust, it must be valid, reliable and practical. This review concludes that the reliability of the MRCP(UK) examination was maintained over the period 1984–2001.
Authors IC McManus, J Mooney-Somers, JE Dacre, AJ Vale
Date published 27 June 2003
Download Medical Education 2003;37:609–611