Try our beta test site
IMPORTANT: Listing of a study on this site does not reflect endorsement by the National Institutes of Health. Talk with a trusted healthcare professional before volunteering for a study. Read more...

Faculty Development to Improve Mini-CEX Ratings

This study has been completed.
Information provided by:
Mayo Clinic Identifier:
First received: April 23, 2008
Last updated: April 25, 2008
Last verified: April 2008
We will conduct a faculty development workshop with intent to improve interrater reliability and rater accuracy. Outcomes will include interrater reliability, accuracy, and workshop evaluations.

Condition Intervention
Faculty Development
Behavioral: Rater training workshop

Study Type: Interventional
Study Design: Allocation: Randomized
Intervention Model: Parallel Assignment
Masking: Open Label
Primary Purpose: Treatment
Official Title: Effect of Faculty Development on Mini-CEX Score Reliability and Accuracy: A Randomized, Controlled Trial

Further study details as provided by Mayo Clinic:

Primary Outcome Measures:
  • Interrater reliability (videotaped resident-patient encounters) [ Time Frame: April-June 2006 ]

Secondary Outcome Measures:
  • Rater accuracy (agreement with scripted levels of performance on videotaped resident-patient encounters): percent agreement, chance-corrected agreement, mean scores [ Time Frame: April-June 2006 ]
  • Interrater reliability (actual resident-patient encounters) [ Time Frame: March 2005 (historical) to June 2007 ]

Enrollment: 52
Study Start Date: March 2006
Study Completion Date: June 2007
Primary Completion Date: June 2006 (Final data collection date for primary outcome measure)
Arms Assigned Interventions
Experimental: Workshop
Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion
Behavioral: Rater training workshop
Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion
No Intervention: Delayed workshop
No specific intervention until after follow-up assessment, then receives same workshop.

Detailed Description:

Hypothesis: An intervention for training faculty preceptors on the use of the mini-CEX will improve inter-rater reliability and classification accuracy.

Methods. We will develop a faculty development workshop aimed at improving the inter-rater reliability and accuracy of mini-CEX scores. To evaluate the workshop we will conduct a randomized, controlled trial. Preceptors will be randomly assigned to immediate or delayed workshop. The primary outcome will be inter-rater reliability of mini-CEX scores as preceptors rate videotaped resident-patient encounters before and four weeks after the workshop. We will also evaluate the accuracy of performance classifications. We will also collect mini-CEX ratings of actual resident-patient interactions one year preceding and one year following the workshop, and calculate interrater reliability.

Statistical methods. Inter-rater reliability will be determined using the intraclass correlation coefficient (ICC) and corresponding confidence intervals. ICC will be used to determine chance-corrected agreement (accuracy). Discrimination (differences in mean scores among scripted levels of performance) will also be evaluated as a measure of accuracy.

Scientific basis. Clinical performance assessments constitute a key element of resident education. The effect of rater training on the inter-rater reliability of mini-CEX scores is unknown. This workshop will utilize methods (rater error and field of reference training) shown to be effective in improving rater accuracy in non-medical fields.

Inclusion criteria. All internal medicine residency continuity clinic preceptors are eligible.


Ages Eligible for Study:   Child, Adult, Senior
Sexes Eligible for Study:   All
Accepts Healthy Volunteers:   Yes

Inclusion Criteria:

  • Internal medicine residency continuity clinic preceptor at Mayo Clinic Rochester Minnesota

Exclusion Criteria:

  Contacts and Locations
Choosing to participate in a study is an important personal decision. Talk with your doctor and family members or friends about deciding to join a study. To learn more about this study, you or your doctor may contact the study research staff using the Contacts provided below. For general information, see Learn About Clinical Studies.

Please refer to this study by its identifier: NCT00667940

United States, Minnesota
Mayo Clinic
Rochester, Minnesota, United States, 55905
Sponsors and Collaborators
Mayo Clinic
Principal Investigator: David A Cook, MD, MHPE Mayo Clinic
  More Information

Additional Information:
Publications automatically indexed to this study by Identifier (NCT Number):
Responsible Party: David A. Cook, M.D., Mayo Clinic Identifier: NCT00667940     History of Changes
Other Study ID Numbers: 06-002730
Study First Received: April 23, 2008
Last Updated: April 25, 2008

Keywords provided by Mayo Clinic:
Medical education
faculty development
rater training
clinical competence
assessment processed this record on May 23, 2017