Working…
ClinicalTrials.gov
ClinicalTrials.gov Menu

A Study to Evaluate Strategies for Teaching Effective Use of Diagnostic Tests

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been evaluated by the U.S. Federal Government. Read our disclaimer for details.
 
ClinicalTrials.gov Identifier: NCT04130607
Recruitment Status : Completed
First Posted : October 17, 2019
Last Update Posted : October 17, 2019
Sponsor:
Collaborator:
McMaster University
Information provided by (Responsible Party):
John Brush, Sentara Norfolk General Hospital

Brief Summary:
A recent Institute of Medicine monograph brought attention to high rates of diagnostic error and called for better educational efforts to improve diagnostic accuracy.1 Educational methods, however, are rarely tested and some educational efforts may be ineffective and wasteful.2 In this study, we plan to examine whether explicit instruction on diagnostic methods will have an effect on diagnostic accuracy of 2nd-year medical students and internal medicine residents.

Condition or disease Intervention/treatment Phase
Instructional Methods Other: Conceptual teaching Not Applicable

Detailed Description:

Research has shown that expert diagnosticians use a two-step process to confirm a diagnosis: hypothesis generation to generate diagnostic possibilities, followed by hypothesis verification to confirm the most likely diagnostic possibility.3-5 The first step appears to be non-analytical, related to pattern recognition. The second step could be calculated using analytical reasoning, however, physicians rarely make an overt calculation of conditional probabilities. Instead, experienced clinicians typically use an implicit habit or heuristic called "anchoring and adjusting" to incorporate diagnostic testing information into their thinking.6,7 Cognitive psychologists have postulated that anchoring and adjusting provides a way that probability estimates can be updated based on additional new evidence. Most of the discussion in the literature focuses on how this heuristic can lead to biased thinking because of base-rate neglect or anchoring.6 Very little discussion is on how this heuristic could be improved to yield more accurate probability estimates and whether proper use of the heuristic could be taught.

The degree to which a diagnostic test should lead to an adjustment of a probability estimate depends on the operating characteristics of a test, that is, the sensitivity and specificity. Likelihood ratios, once understood, are easier to incorporate into one's thinking, and thus could be used to calibrate the anchoring and adjusting heuristic.7

In this randomized trial, we tested whether explicit conceptual instruction on Bayesian reasoning and likelihood ratios would improve Bayesian updating, compared with a second intervention where we provided multiple (27) examples of clinical problem solving. The third arm provided minimal teaching about diagnosis, but no explicit teaching or examples.

Layout table for study information
Study Type : Interventional  (Clinical Trial)
Actual Enrollment : 65 participants
Allocation: Randomized
Intervention Model: Parallel Assignment
Intervention Model Description: Randomized trial of teaching methods with 3 arms.
Masking: Single (Participant)
Masking Description: Participants are not aware of the other arms in the trial
Primary Purpose: Health Services Research
Official Title: A Study to Evaluate Strategies for Teaching Effective Use of Diagnostic Tests
Actual Study Start Date : May 15, 2018
Actual Primary Completion Date : January 1, 2019
Actual Study Completion Date : October 15, 2019

Arm Intervention/treatment
Experimental: Analytical
Students will receive brief instruction in probability, sensitivity, specificity, and likelihood ratios, with distributions and calculations. Pretest and posttest probabilities will be computed for two cases for each of the three conditions listed above.
Other: Conceptual teaching
The present study is designed to contrast two instructional methods - explicit instruction in likelihood ratios and pretest/posttest probabilities versus implicit instruction based on presentation of multiple cases. These will be compared to a "no intervention" control group.
Other Names:
  • Teaching through examples
  • No active teaching

Active Comparator: Experiential
Students will receive a brief instruction conceptually discussing sensitivity and specificity (e.g. "a sensitive test will be positive at even low levels of disease. However, this can lead to a number of false positive errors, when the test is positive even when there is no disease. As a result, it is most useful for ruling out a diagnosis"). They will then work through a total of 30 cases, 10 for each condition, in blocked sequence. For each brief written case they will be asked for a probability of diagnosis after the clinical information is presented. The test result will then be given and they will be asked for a post-test probability. Their estimate will be compared to the computed value based on published estimates of sensitivity and specificity and feedback provided.
Other: Conceptual teaching
The present study is designed to contrast two instructional methods - explicit instruction in likelihood ratios and pretest/posttest probabilities versus implicit instruction based on presentation of multiple cases. These will be compared to a "no intervention" control group.
Other Names:
  • Teaching through examples
  • No active teaching

Placebo Comparator: No Explicit Instruction or Examples
Students will receive 3 passages from a clinical text related to each of the 3 conditions in the study and asked to study them for 15 min each.
Other: Conceptual teaching
The present study is designed to contrast two instructional methods - explicit instruction in likelihood ratios and pretest/posttest probabilities versus implicit instruction based on presentation of multiple cases. These will be compared to a "no intervention" control group.
Other Names:
  • Teaching through examples
  • No active teaching




Primary Outcome Measures :
  1. Accuracy of participants probability revisions were compared to posttest probability revisions that were calculated using Bayes Rule. An effect size was calculated to measure how close students matched the calculated revision. [ Time Frame: Post-test was taken within 72 hours of instructional phase completion. ]
    To perform the effect size analysis, two transformations were performed. First, the difference between the subjective estimate and the Bayesian calculation of post-test probability was squared to remove negative differences and permit combining of the effects of positive and negative test results. Second, a correction based on the intrinsic error of a probability estimate was applied by dividing each squared difference by p(1-p). In this manner, we transformed each raw difference to a squared effect size (difference / error of difference). Finally, the square root was computed, to transform the data back to an effect size. The resulting effect size was then used for statistical analysis. For this primary analysis, a mixed model ANOVA was used.



Information from the National Library of Medicine

Choosing to participate in a study is an important personal decision. Talk with your doctor and family members or friends about deciding to join a study. To learn more about this study, you or your doctor may contact the study research staff using the contacts provided below. For general information, Learn About Clinical Studies.


Layout table for eligibility information
Ages Eligible for Study:   18 Years and older   (Adult, Older Adult)
Sexes Eligible for Study:   All
Accepts Healthy Volunteers:   Yes
Criteria

Inclusion Criteria:

  • Medical Student at McMaster University or Eastern Virginia Medical School
  • Completed 18 months of coursework

Information from the National Library of Medicine

To learn more about this study, you or your doctor may contact the study research staff using the contact information provided by the sponsor.

Please refer to this study by its ClinicalTrials.gov identifier (NCT number): NCT04130607


Locations
Layout table for location information
United States, Virginia
Sentara Norfolk General Hospital
Norfolk, Virginia, United States, 23507
Sponsors and Collaborators
Sentara Norfolk General Hospital
McMaster University
Publications automatically indexed to this study by ClinicalTrials.gov Identifier (NCT Number):
Layout table for additonal information
Responsible Party: John Brush, Cardiologist, Sentara Norfolk General Hospital
ClinicalTrials.gov Identifier: NCT04130607    
Other Study ID Numbers: 18-04-EX-0062
First Posted: October 17, 2019    Key Record Dates
Last Update Posted: October 17, 2019
Last Verified: October 2019
Individual Participant Data (IPD) Sharing Statement:
Plan to Share IPD: No

Layout table for additional information
Studies a U.S. FDA-regulated Drug Product: No
Studies a U.S. FDA-regulated Device Product: No
Keywords provided by John Brush, Sentara Norfolk General Hospital:
Teaching Strategies, Diagnostic reasoning