ClinicalTrials.gov
ClinicalTrials.gov Menu

Survey Study - Sensitivity Comparison Between MelaFind and Physician Group

The safety and scientific validity of this study is the responsibility of the study sponsor and investigators. Listing a study does not mean it has been evaluated by the U.S. Federal Government. Read our disclaimer for details.
ClinicalTrials.gov Identifier: NCT01011153
Recruitment Status : Completed
First Posted : November 11, 2009
Results First Posted : January 24, 2011
Last Update Posted : February 14, 2012
Sponsor:
Information provided by (Responsible Party):
MELA Sciences, Inc.

Study Type Observational
Study Design Observational Model: Cohort;   Time Perspective: Cross-Sectional
Condition Melanoma
Enrollment 241
Recruitment Details Physicians were selected from the membership list of the AAD and lists provided by SK&A Healthcare Information Solutions. A letter was then sent inviting them to participate. For each who replied positively,an access code was mailed with which to log on to the study. 241 subjects logged into system,183 signed consents & completed the Intake Survey.
Pre-assignment Details Participants were grouped according to responses to an Intake Survey, completed after Consent and before reading any study cases. A maximum of 130 cases were reviewed,each one consisting of 3 clinical images and a case history. MelaFind results were not provided as part of each case and were only used for sensitivity and specificity calculations.
Arm/Group Title General Dermatologists Pigmented Skin Lesion Experts Primary Care Physicians
Hide Arm/Group Description Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081 Pigmented Skin Lesion Expert were defined as board-certified dermatologists who spend at least 25% of their practice time evaluating pigmented skin lesions (PSLs) Primary Care Physicians(PCPs) were defined as physicians who deliver primary care service to adult patients (e.g., internists, general practitioners, family practitioners, and geriatricians).
Period Title: Overall Study
Started 54 75 54
Completed 46 64 45
Not Completed 8 11 9
Reason Not Completed
Less than 78 cases completed             8             11             9
Arm/Group Title General Dermatologists Pigmented Skin Lesion Experts Primary Care Physicians Total
Hide Arm/Group Description Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081 Pigmented Skin Lesion Expert were defined as board-certified dermatologists who spend at least 25% of their practice time evaluating pigmented skin lesions (PSLs) Primary Care Physicians(PCPs) were defined as physicians who deliver primary care service to adult patients (e.g., internists, general practitioners, family practitioners, and geriatricians). Total of all reporting groups
Overall Number of Baseline Participants 54 75 54 183
Hide Baseline Analysis Population Description
[Not Specified]
Age, Categorical  
Measure Type: Count of Participants
Unit of measure:  Participants
Number Analyzed 54 participants 75 participants 54 participants 183 participants
<=18 years
0
   0.0%
0
   0.0%
0
   0.0%
0
   0.0%
Between 18 and 65 years
54
 100.0%
75
 100.0%
54
 100.0%
183
 100.0%
>=65 years
0
   0.0%
0
   0.0%
0
   0.0%
0
   0.0%
Sex/Gender, Customized  
Measure Type: Number
Unit of measure:  Participants
Number Analyzed 54 participants 75 participants 54 participants 183 participants
Female 16 36 23 75
Male 32 35 26 93
Unknown 6 4 5 15
Region of Enrollment  
Measure Type: Number
Unit of measure:  Participants
United States Number Analyzed 54 participants 75 participants 54 participants 183 participants
54 75 54 183
1.Primary Outcome
Title Comparison of Biopsy/Referral Sensitivity of MelaFind and Dermatologists (Pigmented Skin Lesion Experts and General Dermatologists)
Hide Description Sensitivity is the proportion of positive cases (i.e., histologically confirmed melanoma) identified as positive. Specificity is the proportion of negative cases (i.e., histologically confirmed non-melanoma) identified as negative. Because the number of cases given to each dermatologist varied, both sensitivity and specificity were computed for each dermatologist. The primary outcome as stated was to compare the sensitivity and specificity of all dermatologists to that of MelaFind. These metrics, for both the dermatologists and MelaFind, were calculated based on the same 130 lesions.
Time Frame April 2010
Show Outcome Measure DataHide Outcome Measure Data
Hide Analysis Population Description
Participants were invited to enroll in this study via mail. Minimum number of participants was determined by statistician based on power analyses and number of cases completed. The time frame of the study was about 6 months and the comparison between Dermatologists and MelaFind is presented in the statistical analysis section below.
Arm/Group Title Dermatologists MelaFind
Hide Arm/Group Description:
Dermatologists were defined as board-certified dermatologists who were General Dermatologists and Pigmented Skin Lesion Experts, according to the Intake Survey. Dermatologists in this group did not participate in previous EOS Protocols 20061 and 20081. Each dermatologist was given up to 130 cases (each one consisting of 3 clinical images and a case history) consistinf of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma)
MelaFind imaged 130 cases which consisted of 65 positive cases (i.e., histolgoically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
Overall Number of Participants Analyzed 110 1
Mean (95% Confidence Interval)
Unit of Measure: Proportion of True Cases
Proportion of True Positive Cases (Sensitivity)
0.72
(0.66 to 0.78)
0.97
(0.9 to 0.99)
Proportion of True Negative Cases (Specificity)
0.51
(0.43 to 0.58)
0.09
(0.04 to 0.19)
Show Statistical Analysis 1 Hide Statistical Analysis 1
Statistical Analysis Overview Comparison Group Selection Dermatologists, MelaFind
Comments Using True Positives/All Positives, sensitivity values were calculated for MelaFind as well as the average of the 110 dermatologists.
Type of Statistical Test Non-Inferiority or Equivalence
Comments Sample sizes computed based on data from smaller studies provided 90% power for the comparison of sensitivity. Under the alternative hypothesis that the biopsy sensitivity of MelaFind would be 0.10 greater than the average biopsy/referral sensitivity of physicians (combined or separate), the probability that a 95% CI lies entirely above zero will be at least 90% when the standard error of the estimated difference is at most 0.0308.
Statistical Test of Hypothesis P-Value <0.0001
Comments [Not Specified]
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Mean Difference (Final Values)
Estimated Value 0.25
Confidence Interval (2-Sided) 95%
0.18 to 0.32
Parameter Dispersion
Type: Standard Deviation
Value: 0.03
Estimation Comments [Not Specified]
2.Secondary Outcome
Title Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians
Hide Description Sensitivity is the proportion of positive cases (i.e., histologically confirmed melanoma) identified as positive. Specificity is the proportion of negative cases (i.e., histologically confirmed non-melanoma) identified as negative. Because the number of cases given to each dermatologist varied, both sensitivity and specificity were computed for each dermatologist. The primary outcome as stated was to compare the sensitivity and specificity of each group of physicians to that of Melafind, which is presented in the statistical analysis.
Time Frame December 2009
Show Outcome Measure DataHide Outcome Measure Data
Hide Analysis Population Description
Each category was diminished after excluded subjects were taken into account. These included subjects who did not complete at least 78 cases, subjects who previously participated in other EOS studies, pediatricians, and a board eligible dermatologist.
Arm/Group Title General Dermatologists Pigmented Skin Lesion Experts Primary Care Physicians MelaFind
Hide Arm/Group Description:
General Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081. Each dermatologist was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma)
Physicians who spend at least 25% of their practice time examining pigmented skin lesions. Each PSL Expert was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
Physicians who are not Board Certified Dermatologists and Pediatricians. Each PCP was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
MelaFind imaged the 130 cases, the 65 positive cases (i.e., histologically confirmed melanoma) and the 65 negative cases (i.e., histologically confirmed non-melanoma). Sensitivity was calculated based on the correct identification of the 65 positive cases and specificity was calculated based on the correct identification of the 65 negative cases.
Overall Number of Participants Analyzed 46 64 45 1
Mean (95% Confidence Interval)
Unit of Measure: Proportion of True Cases
Proportion of True Positive Cases (Sensitivity)
0.73
(0.67 to 0.80)
0.71
(0.65 to 0.77)
0.71
(0.64 to 0.78)
0.97
(0.90 to 0.99)
Proportion of True Negative Cases (Specificity)
0.51
(0.43 to 0.60)
0.50
(0.42 to 0.58)
0.45
(0.37 to 0.53)
0.09
(0.04 to 0.19)
Show Statistical Analysis 1 Hide Statistical Analysis 1
Statistical Analysis Overview Comparison Group Selection Primary Care Physicians, MelaFind
Comments Sensitivity
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value <0.0001
Comments [Not Specified]
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Mean Difference (Final Values)
Estimated Value 0.26
Confidence Interval (2-Sided) 95%
0.19 to 0.34
Parameter Dispersion
Type: Standard Deviation
Value: 0.04
Estimation Comments [Not Specified]
Show Statistical Analysis 2 Hide Statistical Analysis 2
Statistical Analysis Overview Comparison Group Selection General Dermatologists, MelaFind
Comments Sensitivity
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value <0.0001
Comments [Not Specified]
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Mean Difference (Final Values)
Estimated Value 0.24
Confidence Interval (2-Sided) 95%
0.16 to 0.31
Parameter Dispersion
Type: Standard Deviation
Value: 0.04
Estimation Comments [Not Specified]
Show Statistical Analysis 3 Hide Statistical Analysis 3
Statistical Analysis Overview Comparison Group Selection Pigmented Skin Lesion Experts, MelaFind
Comments Sensitivity
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value <0.0001
Comments [Not Specified]
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Mean Difference (Final Values)
Estimated Value 0.26
Confidence Interval (2-Sided) 95%
0.19 to 0.33
Parameter Dispersion
Type: Standard Deviation
Value: 0.03
Estimation Comments [Not Specified]
Show Statistical Analysis 4 Hide Statistical Analysis 4
Statistical Analysis Overview Comparison Group Selection Primary Care Physicians, MelaFind
Comments Specificity
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value <0.0001
Comments [Not Specified]
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Mean Difference (Final Values)
Estimated Value -0.36
Confidence Interval (2-Sided) 95%
-0.46 to -0.25
Parameter Dispersion
Type: Standard Deviation
Value: 0.05
Estimation Comments [Not Specified]
Show Statistical Analysis 5 Hide Statistical Analysis 5
Statistical Analysis Overview Comparison Group Selection General Dermatologists, MelaFind
Comments Specificity
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value <0.0001
Comments [Not Specified]
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Mean Difference (Final Values)
Estimated Value -0.42
Confidence Interval (2-Sided) 95%
-0.53 to -0.31
Parameter Dispersion
Type: Standard Deviation
Value: 0.05
Estimation Comments [Not Specified]
Show Statistical Analysis 6 Hide Statistical Analysis 6
Statistical Analysis Overview Comparison Group Selection Pigmented Skin Lesion Experts, MelaFind
Comments Specificity
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value <0.0001
Comments [Not Specified]
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Mean Difference (Final Values)
Estimated Value -0.41
Confidence Interval (2-Sided) 95%
-0.51 to -0.30
Parameter Dispersion
Type: Standard Deviation
Value: 0.05
Estimation Comments [Not Specified]
3.Secondary Outcome
Title Determine the Interobserver Variability in Each of the Above Metrics Within Each of the Caregiver Groups.
Hide Description Each physician was given up to 130 cases and asked whether or not they would biopsy the lesion. Interobserver variability was measured via the kappa statistic indicating how well the physicians' answers to that question agreed within each group. Kappa statistics are reported in the statistical analysis. while numbers rep, they dont reflect the sgreement among the subjects
Time Frame December 2009
Show Outcome Measure DataHide Outcome Measure Data
Hide Analysis Population Description
The statistical analysis section contains the Kappa results within Each of the Caregiver Groups
Arm/Group Title Dermatologists Pigmented Skin Lesion Experts Primary Care Physicians All Partipants
Hide Arm/Group Description:
General Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081. Each dermatologist was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma)
Physicians who spend at least 25% of their practice time examining pigmented skin lesions. Each PSL Expert was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
Physicians who are not Board Certified Dermatologists and Pediatricians. Each PCP was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
All Participants are the General Dermatologists, Pigmented Skin Lesion Experts and Primary Care Physicians combined.
Overall Number of Participants Analyzed 46 64 45 155
Measure Type: Number
Unit of Measure: Number of Cases
5927 8263 5844 20034
Show Statistical Analysis 1 Hide Statistical Analysis 1
Statistical Analysis Overview Comparison Group Selection Dermatologists
Comments [Not Specified]
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value [Not Specified]
Comments No comparison is being made.
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Kappa
Estimated Value 0.313
Parameter Dispersion
Type: Standard Error of the mean
Value: 0.003
Estimation Comments There is no threshold level-this is a measurement of variability among responses provided by the participants.
Show Statistical Analysis 2 Hide Statistical Analysis 2
Statistical Analysis Overview Comparison Group Selection Pigmented Skin Lesion Experts
Comments [Not Specified]
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value [Not Specified]
Comments No comparison is being made
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Kappa
Estimated Value 0.276
Parameter Dispersion
Type: Standard Error of the mean
Value: 0.002
Estimation Comments There is no threshold level-this is a measurement of variability among responses provided by the participants.
Show Statistical Analysis 3 Hide Statistical Analysis 3
Statistical Analysis Overview Comparison Group Selection Primary Care Physicians
Comments [Not Specified]
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value [Not Specified]
Comments No comparison is being made
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Kappa
Estimated Value 0.200
Parameter Dispersion
Type: Standard Error of the mean
Value: 0.003
Estimation Comments There is no threshold level-this is a measurement of variability among responses provided by the participants.
Show Statistical Analysis 4 Hide Statistical Analysis 4
Statistical Analysis Overview Comparison Group Selection All Partipants
Comments [Not Specified]
Type of Statistical Test Superiority or Other
Comments [Not Specified]
Statistical Test of Hypothesis P-Value [Not Specified]
Comments N/A - No comparison is being made
Method ANOVA
Comments [Not Specified]
Method of Estimation Estimation Parameter Kappa
Estimated Value 0.256
Parameter Dispersion
Type: Standard Error of the mean
Value: 0.001
Estimation Comments There is no threshold level-this is a measurement of variability among responses provided by the participants.
4.Secondary Outcome
Title To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians.
Hide Description For each case reviewed, physicians were asked if they thought the lesion was a melanoma (diagnostic sensitivity/specificity) and whether or not they would biopsy or refer the lesion (biopsy/referral sensitivity/specificity. These measurements were compared using areas under the corresponding receiver operating characteristic curves. (see statistical analysis for results) ROC curves (reciver operating curves) are plotted on graphs with an x-axis of sensitivity and a y-axis of 1-specificity.
Time Frame June 2010
Show Outcome Measure DataHide Outcome Measure Data
Hide Analysis Population Description
[Not Specified]
Arm/Group Title General Dermatologists Pigmented Skin Lesion Experts Primary Care Physicians
Hide Arm/Group Description:
Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081
[Not Specified]
[Not Specified]
Overall Number of Participants Analyzed 46 64 45
Geometric Mean (Standard Deviation)
Unit of Measure: Area Under Curve for biopsy/referral
Area Under Curve for Biopsy/Referral 0.65  (0.03) 0.63  (0.03) 0.59  (0.02)
Area Under Curve for Diagnostic 0.68  (0.03) 0.66  (0.03) 0.61  (0.03)
Time Frame [Not Specified]
Adverse Event Reporting Description No safety analyses were planned for this survey study which used cases in which medical management had already been planned and carried out.
 
Arm/Group Title General Dermatologists Pigmented Skin Lesion Experts Primary Care Physicians
Hide Arm/Group Description Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081 Pigmented Skin Lesion Expert were defined as board-certified dermatologists who spend at least 25% of their practice time evaluating pigmented skin lesions (PSLs) Primary Care Physicians(PCPs) were defined as physicians who deliver primary care service to adult patients (e.g., internists, general practitioners, family practitioners, and geriatricians).
All-Cause Mortality
General Dermatologists Pigmented Skin Lesion Experts Primary Care Physicians
Affected / at Risk (%) Affected / at Risk (%) Affected / at Risk (%)
Total   --/--   --/--   --/-- 
Show Serious Adverse Events Hide Serious Adverse Events
General Dermatologists Pigmented Skin Lesion Experts Primary Care Physicians
Affected / at Risk (%) Affected / at Risk (%) Affected / at Risk (%)
Total   0/0   0/0   0/0 
Show Other (Not Including Serious) Adverse Events Hide Other (Not Including Serious) Adverse Events
Frequency Threshold for Reporting Other Adverse Events 0%
General Dermatologists Pigmented Skin Lesion Experts Primary Care Physicians
Affected / at Risk (%) Affected / at Risk (%) Affected / at Risk (%)
Total   0/0   0/0   0/0 
The only limitation to having complete data sets was the time a physician was willing to spend on completing the survey.
Certain Agreements
Principal Investigators are NOT employed by the organization sponsoring the study.
There IS an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.
Results Point of Contact
Name/Title: Joanna Adrian, Director of Clinical Operatons and Medical Affairs
Organization: MELA Sciences, Inc.
Phone: 914-591-3783 ext 732
Responsible Party: MELA Sciences, Inc.
ClinicalTrials.gov Identifier: NCT01011153     History of Changes
Other Study ID Numbers: 20063
First Submitted: November 10, 2009
First Posted: November 11, 2009
Results First Submitted: May 26, 2010
Results First Posted: January 24, 2011
Last Update Posted: February 14, 2012