The reliability and validity of a matrix to assess the completed reflective personal development plans of general practitioners

被引:7
|
作者
Roberts, C
Cromarty, I
Crossley, J
Jolly, B
机构
[1] Univ Sheffield, Sheffield S10 2TN, S Yorkshire, England
[2] Postgrad GP Educ, Cambridge, England
[3] Chesterfield & N Derbyshire Royal Hosp Trust, Chesterfield, England
[4] Monash Univ, Fac Med Nursing & Hlth Sci, Ctr Med & Hlth Sci Educ, Clayton, Vic 3168, Australia
关键词
family practice; education; medical; graduate; standards; staff development; accreditation; documentation; Great Britain; clinical competence; psychometrics;
D O I
10.1111/j.1365-2929.2006.02413.x
中图分类号
G40 [教育学];
学科分类号
040101 ; 120403 ;
摘要
INTRODUCTION We wished to determine whether assessors could make reliable and valid judgements about the quality of completed reflective personal development plans (PDPs) for the purpose of accrediting UK general practitioners (GPs) for a postgraduate education allowance using a marking matrix, and secondly, to plan a feasible model of PDP assessment in the context of forthcoming GP appraisal/revalidation that would overcome the main sources of error identified from this study. METHODS Within generalisability theory, a variance components analysis on PDP scores estimated reliability and the effect on them of varying, for example, the number of assessors. We investigated the construct validity of the matrix through its internal consistency and detection of differences in the quality of PDPs. RESULTS For a single PDP and one assessor, 37.6% of the variance in scores was due to true differences in the quality of the PDP. Between 5 and 7 PDP assessors are needed to achieve summative reliability of greater than 0.8. While increasing the number of judges is important, reliability could also be improved by addressing assessor subjectivity. Construct validity was demonstrated, as the matrix distinguished between good, satisfactory and poor PDPs, and it had good internal consistency. CONCLUSION PDP assessment has reasonable summative characteristics for the purpose of assessing GPs' reflective continuing professional development. If doctors could include their PDPs within their revalidation folders as evidence of their reflections on pursuing better clinical performance, we have described a reliable, valid and feasible method of external assessment.
引用
收藏
页码:363 / 370
页数:8
相关论文
共 48 条
  • [41] A STRUCTURED PICTORIAL QUESTIONNAIRE TO ASSESS DSM-III-R-BASED DIAGNOSES IN CHILDREN (6-11 YEARS) - DEVELOPMENT, VALIDITY, AND RELIABILITY
    VALLA, JP
    BERGERON, L
    BERUBE, H
    GAUDET, N
    STGEORGES, M
    JOURNAL OF ABNORMAL CHILD PSYCHOLOGY, 1994, 22 (04) : 403 - 423
  • [42] Suicide Attempt Self-Injury Interview (SASII): Development, reliability, and validity of a scale to assess suicide attempts and intentional self-injury
    Linehan, Marsha M.
    Comtois, Katherine Anne
    Brown, Milton Z.
    Heard, Heidi L.
    Wagner, Amy
    PSYCHOLOGICAL ASSESSMENT, 2006, 18 (03) : 303 - 312
  • [43] The development of a psychometric scale to assess the influence of medical and non-medical factors upon general practitioners' decisions to refer patients with gastrointestinal (GI) symptoms
    Flynn, D
    van Schaik, P
    van Wersch, A
    Cann, P
    Douglass, A
    GUT, 2002, 50 : A97 - A97
  • [44] A NEW METHOD TO ASSESS ACTIVITY LIMITATIONS IN PATIENTS WITH HIP AND KNEE OSTEOARTHRITIS: THE ANIMATED ACTIVITY QUESTIONNAIRE (AAQ): PART 1. DEVELOPMENT, VALIDITY AND RELIABILITY
    Peter, W. F.
    Loos, M.
    de Vet, R. C.
    Boers, M.
    Harlaar, J.
    Roorda, L. D.
    Poolman, R. W.
    Scholtes, V. A.
    Boogaard, J.
    Buitelaar, H.
    Steultjens, M.
    Roos, E. M.
    Guillemin, F.
    Benedetti, M. G.
    Escobar Martinez, A.
    Terwee, C. B.
    ANNALS OF THE RHEUMATIC DISEASES, 2014, 73 : 463 - 464
  • [45] DEVELOPMENT, CONTENT VALIDITY AND INTER-RATER RELIABILITY TESTING OF THE CIMI-BRIGHT: AN INSTRUMENT TO ASSESS THE LEVEL OF CHRONIC ILLNESS MANAGEMENT IN TRANSPLANT CLINICAL PRACTICE
    Berben, Lut
    Russell, Cynthia
    Engberg, Sandra
    Dobbels, Fabienne
    Geest, SabinaDe
    TRANSPLANT INTERNATIONAL, 2013, 26 : 191 - 192
  • [46] The reliability and validity of the Japanese version of the Daily Record of Severity of Problems (J-DRSP) and Development of a Short-Form version (J-DRSP (SF)) to assess symptoms of premenstrual syndrome among Japanese women
    Ikeda, Yumie
    Egawa, Miho
    Okamoto, Kazuya
    Mandai, Masaki
    Takahashi, Yoshimitsu
    Nakayama, Takeo
    BIOPSYCHOSOCIAL MEDICINE, 2021, 15 (01)
  • [47] The reliability and validity of the Japanese version of the Daily Record of Severity of Problems (J-DRSP) and Development of a Short-Form version (J-DRSP (SF)) to assess symptoms of premenstrual syndrome among Japanese women
    Yumie Ikeda
    Miho Egawa
    Kazuya Okamoto
    Masaki Mandai
    Yoshimitsu Takahashi
    Takeo Nakayama
    BioPsychoSocial Medicine, 15
  • [48] Development, content validity and inter-rater reliability testing of the Chronic Illness Management Implementation - Building Research Initiative Group: Chronic Illness Management and Adherence in Transplantation: An instrument to assess the level of chronic illness management implemented in solid organ transplant programmes
    Berben, Lut
    Russell, Cynthia L.
    Engberg, Sandra
    Dobbels, Fabienne
    De Geest, Sabina
    INTERNATIONAL JOURNAL OF CARE COORDINATION, 2014, 17 (1-2) : 59 - 71