Friday, 29 March, 2024
HomeClinical PracticeConcerns raised over accuracy of melanoma diagnoses

Concerns raised over accuracy of melanoma diagnoses

MelanomaA study of US pathologists shows that diagnoses can vary among pathologists, particularly for cases in the middle of the disease spectrum, suggesting the potential for both over-diagnosis and under-diagnosis.

Melanoma is a type of skin cancer that develops from skin cells called melanocytes. Diagnosis relies on visual assessment of skin samples (biopsies) under a microscope by a pathologist, but the reliability of the criteria used to diagnose these skin lesions have never been established with rigorous standards.

Previous studies have suggested high levels of diagnostic disagreement among pathologists when interpreting melanocytic lesions, but results are conflicting. So a team of researchers, led by Professor Joann Elmore at the University of Washington School of Medicine in Seattle, set out to measure the accuracy and reproducibility of pathologists' diagnoses of melanocytic skin lesions. The study was inspired by Elmore's experience as a patient undergoing a skin biopsy, which resulted in three different independent interpretations, ranging from benign to invasive melanoma. Ten years later she is healthy and doing research on the topic.

A total of 187 practicing pathologists from 10 US states were randomised to independently interpret the same set of skin biopsy cases on two separate occasions (phases 1 and 2), at least eight months apart. Each case had been independently reviewed by a panel of three experienced skin pathologists and a consensus reference diagnosis reached.

Participants' interpretations were assigned to one of five classes: I, mild atypia; II, moderate atypia; III, severe atypia or melanoma in situ; IV, early invasive melanoma; and V, invasive melanoma. Accuracy was measured by comparing the pathologists' interpretations with the panel's consensus reference diagnosis.

The highest levels of accuracy were found for class I mild lesions (92%) and class V high stage invasive melanoma (72%) – these cases are at the polar ends of the disease spectrum.

In contrast, interpretations for cases in the middle of the spectrum had noticeably lower accuracy, as less than half of the diagnoses were in concordance with the reference diagnosis; class II moderately atypical lesions (25%); class III severely atypical lesions and melanoma in situ (40%); and class IV early stage invasive melanoma (43%).
Pathologists' interpretations of the same case on two occasions also lacked reproducibility for cases in the middle of the spectrum.

At a population level, the researchers estimate that 83% of melanocytic skin biopsy diagnoses would have their diagnosis verified if reviewed by a consensus reference panel of experienced pathologists, with 8% of cases over-interpreted by the initial pathologist and 9% under-interpreted.

The authors outline some study limitations which could have introduced bias, but strengths included a large sample size and use of three reference standards to estimate accuracy. They also point out that, in clinical practice, pathologists may have the opportunity to review more slides, obtain second opinions from colleagues, or request additional tests before making a diagnosis.

These results show that diagnoses ranging from moderately atypical lesions to early stage invasive melanoma are neither accurate nor reproducible, say the authors.
Efforts to improve clinical practice should include use of a standardised classification format, acknowledging uncertainty of specific diagnoses in pathology reports, and development of more sophisticated diagnostic tools to support pathologists, they conclude.

Abstract
Objective: To quantify the accuracy and reproducibility of pathologists’ diagnoses of melanocytic skin lesions.
Design: Observer accuracy and reproducibility study.
Setting: 10 US states.
Participants Skin biopsy cases (n=240), grouped into sets of 36 or 48. Pathologists from 10 US states were randomized to independently interpret the same set on two occasions (phases 1 and 2), at least eight months apart.
Main outcome measures: Pathologists’ interpretations were condensed into five classes: I (eg, nevus or mild atypia); II (eg, moderate atypia); III (eg, severe atypia or melanoma in situ); IV (eg, pathologic stage T1a (pT1a) early invasive melanoma); and V (eg, ≥pT1b invasive melanoma). Reproducibility was assessed by intraobserver and interobserver concordance rates, and accuracy by concordance with three reference diagnoses.
Results: In phase 1, 187 pathologists completed 8976 independent case interpretations resulting in an average of 10 (SD 4) different diagnostic terms applied to each case. Among pathologists interpreting the same cases in both phases, when pathologists diagnosed a case as class I or class V during phase 1, they gave the same diagnosis in phase 2 for the majority of cases (class I 76.7%; class V 82.6%). However, the intraobserver reproducibility was lower for cases interpreted as class II (35.2%), class III (59.5%), and class IV (63.2%). Average interobserver concordance rates were lower, but with similar trends. Accuracy using a consensus diagnosis of experienced pathologists as reference varied by class: I, 92% (95% confidence interval 90% to 94%); II, 25% (22% to 28%); III, 40% (37% to 44%); IV, 43% (39% to 46%); and V, 72% (69% to 75%). It is estimated that at a population level, 82.8% (81.0% to 84.5%) of melanocytic skin biopsy diagnoses would have their diagnosis verified if reviewed by a consensus reference panel of experienced pathologists, with 8.0% (6.2% to 9.9%) of cases overinterpreted by the initial pathologist and 9.2% (8.8% to 9.6%) underinterpreted.
Conclusion: Diagnoses spanning moderately dysplastic nevi to early stage invasive melanoma were neither reproducible nor accurate in this large study of pathologists in the USA. Efforts to improve clinical practice should include using a standardized classification system, acknowledging uncertainty in pathology reports, and developing tools such as molecular markers to support pathologists’ visual assessments.

Authors
Joann G Elmore, Raymond L Barnhill, David E Elder, Gary M Longton, Margaret S Pepe, Lisa M Reisch, Patricia A Carney, Linda J Titus, Heidi D Nelson, Tracy Onega, Anna NA Tosteson, Martin A Weinstock, Stevan R Knezevich, Michael W Piepkorn

[link url="https://www.sciencedaily.com/releases/2017/06/170628200017.htm"]BMJ material[/link]
[link url="http://www.bmj.com/content/357/bmj.j2813"]BMJ abstract[/link]

MedicalBrief — our free weekly e-newsletter

We'd appreciate as much information as possible, however only an email address is required.