← ATOF interactiveMethods · v0.1

Methods & Citations

The Adaptive Threshold Optimisation Framework (ATOF) integrates three usually-siloed lenses - clinical net benefit, overdiagnosis harm, and regional workforce capacity - into a single answer to the implementation question for Australia's first national risk-stratified melanoma screening programme: at what risk score do you screen vs. not screen?

Author: Dr Yagiz Aksoy · The Daffodil Centre · University of Sydney & Cancer Council NSW · LinkedIn. Pairs with manuscript draft for BMJ submission and the Viertel Clinical Investigator Award.

1. Decision Curve Analysis

Net benefit is computed per the standard Vickers & Elkin (2006) framing:

NB(t) = TP/N − (FP/N) × (t / (1 − t))

where t is the threshold probability, TP/N the true-positive rate, FP/N the false-positive rate, and t / (1 − t) the exchange rate between false-positives and true-positives.

For each model we map the published area under the curve (AUC) to a sensitivity–specificity trade-off using a binormal approximation: assume normally-distributed risk scores in cases and controls with equal variance and a mean separation of d' = √2 × Φ⁻¹(AUC). At threshold t, the implied normal-deviate cut-point is solved against the prevalence-aware base rate, giving sensitivity and FPR (Φ(d' − z), Φ(−z)). Net benefit then follows.

⚠ Modelled from published AUCs. The final ATOF analysis reconstructs sensitivity–specificity directly from individual-level 45 and Up Study data via the Cust lab (Reyes-Marcelino et al., forthcoming).

2. Validated risk prediction models

Six external-validation AUCs from the 45 and Up Study plus the Roadmap Evidence Synthesis (§5.4). All except QSkin MP16 published; MP16 cited from Whiteman et al.'s internal recalibration in the Roadmap evidence synthesis.

ModelAUCCohortVariablesCitation
QSkin MP16
Highest discrimination of any Australian model. MP16 = 16-variable extended QSkin.
0.76QSkin Sun and Health Study (Queensland)16 variables · Age, Sex, Skin colour …Whiteman DC 2025
QSkin MP7
Parsimonious 7-variable QSkin model.
0.73QSkin Sun and Health Study (Queensland)7 variables · Age, Sex, Tanning ability …Olsen CM 2018
MIA/Vuong
Used as reference model in 45 and Up Study external validation.
0.70Australian Melanoma Family Study (development), 45 and Up (validation)7 variables · Age, Sex, Hair colour …Vuong K 2016
NCI MBRAT
US-derived; tested in 45 and Up.
0.70US case-control (Philadelphia, San Francisco)6 variables · Age, Sex, Sunburn history …Fears TR 2006
Cho
0.66Nurses' Health Study + Health Professionals (US)6 variables · Age, Sex, Family history of melanoma …Cho E 2005

3. Overdiagnosis adjustment

Net benefit can be deflated by an overdiagnosis rate so that true-positives detected via screening are weighted against the harm of identifying lesions that would never have caused clinical disease.

NB'(t) = NB(t) − (sensitivity × prevalence × OD-rate) × (1 / (1 − t))

We use named reference points so a reviewer can interrogate each pick:

  • 74% - Lindsay 2024 - In situ (2021). in situ · AU (range 71–76%) (Lindsay A 2024)
  • 69% - Lindsay 2024 - In situ (2017). in situ · AU (range 67–70%) (Lindsay A 2024)
  • 28% - Lindsay 2024 - Thin invasive. thin invasive · AU (range 22–34%) (Lindsay A 2024)
  • 54% - Glasziou 2020 - All melanoma (women). all melanoma · AU (Glasziou PP 2020)
  • 58% - Glasziou 2020 - All melanoma (men). all melanoma · AU (Glasziou PP 2020)
  • 29% - Bjørch 2024 - International low. all melanoma · INTL (Bjørch MF 2024)
  • 60% - Bjørch 2024 - International high. all melanoma · INTL (Bjørch MF 2024)
  • 50% - Adamson 2024 - US (men). all melanoma · US (Adamson AS 2024)
  • 65% - Adamson 2024 - US (women). all melanoma · US (Adamson AS 2024)

4. Workforce capacity model

Per-region capacity is computed from clinician density (dermatologists and SCCA-accredited skin-cancer GPs) multiplied by per-clinician annual consult capacity and the fraction of that capacity available for screening.

capacity = (D × 2800) + (G × 1200) × 0.3

where D = dermatologist FTE in region, G = skin-cancer GP FTE, 2800 = annual consults per derm FTE, 1200 = annual consults per skin-cancer-GP FTE, and 30% = screening-eligible fraction of total clinical time.

Annual demand:

demand = (cohort above threshold) / 2 (2-year cadence)

A region is "feasible" when capacity ≥ demand at the selected threshold.

ScenarioStatusMultiplierRationale
Dermatologists onlyempirical-Lindsay et al. 2026 Table 2 - adjusted clinical FTE × 2800 consults/yr × 30% screening fraction.
Skin cancer GPs onlyempirical-SCCA-trained GPs at adjusted FTE × 1200 consults/yr × 30% screening fraction.
Combined (Derm + GP)empirical-Sum of Derm + GP capacity - represents the practical Australian baseline.
Melanographer / TBPmodelled-extensionderm × 2Total Body Photography decouples image capture (technician, high throughput) from image reading (specialist, async, batched). Effective derm throughput doubled (×2.0) - defensible because the bottleneck shifts from physical consult time to image-reading time. Awaiting ACEMID empirical data.
Tiered hub-and-spokemodelled-extensionGP × 1.5, derm × 0.5Tiered model where GPs triage ~70% of patients locally and route ~30% to specialist confirmation. GP throughput ×1.5 (focused on dermoscopy), specialist load ×0.5 (only complex cases).

Annualised case detection in the sidebar: (sens × prevalence × eligible adult population) / 10. Eligible adults = 48% of total ERP (AIHW age-band weighting for adults 40+).

Time projection

The "Workforce projection" slider in the controls scales every region's workforce capacity by (1 + 0.03)years. Defensible anchors:

  • ACD pipeline: ~30 derm trainees graduating per year against a base of ~600 working derms → ~5%/yr gross, ~3%/yr net of retirement.
  • SCCA-accredited GP-with-skin pipeline is faster but smaller in absolute terms; weighted average pulls towards the derm number.
  • Roadmap Evidence Synthesis cites Lindsay 2026's central workforce scenario at 2.5–3.5%/yr depending on state. We use 3.0% as the central anchor (sensitivity: 2% gives 1.22× over a decade, 4% gives 1.48×).

5. Data provenance

  • Risk models. Published AUCs (QSkin MP16/MP7, MIA/Vuong, NCI MBRAT, Cho) - see Section 2 table.
  • Workforce density (QLD SA4s). Real - Lindsay et al. 2026 (Australasian Journal of Dermatology, Early View) supply–demand analysis.
  • Workforce density (NSW). Real - Blake et al. 2023 dermatologist density analysis (Australasian Journal of Dermatology 64(3)).
  • Workforce density (other states). Modelled - Modified Monash Model (MMM) inferred density. SA4 regions tagged 'mmm-modelled' carry a provenance pill in the live tool.
  • Incidence. AIHW Cancer Data in Australia 2024 + Australian Cancer Atlas 2.0 (Cancer Council Queensland, QUT).
  • Population (ERP). ABS Estimated Resident Population by SA4.
  • SEIFA / ARIA+. ABS Socio-Economic Indexes for Areas (SEIFA) quintiles and ARIA+ remoteness categories.

6. Methodological caveats

  • Decision curves are modelled. The displayed curves are derived from published AUCs using a binormal approximation. The final ATOF analysis reconstructs them from individual-level 45 and Up Study data.
  • Workforce density is partly modelled. Empirical SA4 numbers come from Lindsay 2026 (QLD) and Blake 2023 (NSW). Other regions use MMM-modelled estimates and are visually tagged.
  • Melanographer/TBP and tiered scenarios are modelled extensions. They assume specific throughput multipliers (×2.0, ×1.5/×0.5) which are defensible from first principles but await ACEMID empirical data.
  • Cost figures. MSAC 1699 ICER stated precisely as AUD $62,754/QALY. Annual overdiagnosis cost ≈ AUD $21.6M per Lindsay 2024.

7. Citations

22 references, alphabetised by first author.

  1. Adamson AS, Naik G, Jones BA, Bell KJL (2024). Overdiagnosis of melanoma in the United States. BMJ Evidence-Based Medicine, 29(3): 156–161. doi:10.1136/bmjebm-2023-112460 · PMID:38290826
  2. Aneja S, Aneja S, Bordeaux JS (2012). Association of increased dermatologist density with lower melanoma mortality. Archives of Dermatology, 148(2): 174–178. doi:10.1001/archdermatol.2011.345 · PMID:22351816
  3. Australasian College of Dermatologists (2023). Special Commission of Inquiry into Healthcare Funding - NSW submission. ACD. link
  4. Australian Institute of Health and Welfare (2024). Cancer data in Australia. AIHW Cancer Series. link
  5. Bjørch MF, Gram EG, Brodersen J (2024). Overdiagnosis of melanoma: a scoping review of screening settings. BMJ Evidence-Based Medicine, 29(1): 17–28. doi:10.1136/bmjebm-2023-112341 · PMID:37536952
  6. Blake C, et al. (2023). The association of dermatologist demographic density with melanoma survival in New South Wales, Australia. Australasian Journal of Dermatology, 64(3): 425–429. doi:10.1111/ajd.14113 · PMID:37353974
  7. Brentnall AR, Atakpa EC, Hill H, Santeramo R, Damiani C, Cuzick J, Montana G, Duffy SW (2023). An optimization framework to guide the choice of thresholds for risk-based cancer screening. npj Digital Medicine, 6:216. doi:10.1038/s41746-023-00967-9 · PMID:38017184
  8. Cancer Council Queensland, Queensland University of Technology, Australian Cancer Atlas (2024). Australian Cancer Atlas 2.0. Version 05-2024. link
  9. Cho E, Rosner BA, Feskanich D, Colditz GA (2005). Risk factors and individual probabilities of melanoma for whites. Journal of Clinical Oncology, 23(31): 8055–8060. doi:10.1200/JCO.2005.02.1773 · PMID:16258103
  10. Fears TR, Guerry D 4th, Pfeiffer RM, et al. (2006). Identifying individuals at high risk of melanoma: a practical predictor of absolute risk. Journal of Clinical Oncology, 24(22): 3590–3596. doi:10.1200/JCO.2005.04.1277
  11. Glasziou PP, Jones MA, Pathirana T, Barratt AL, Bell KJL (2020). Estimating the magnitude of cancer overdiagnosis in Australia. Medical Journal of Australia, 212(4): 163–168. doi:10.5694/mja2.50455 · PMID:31858632
  12. Lindsay A, Bell KJL, Olsen CM, Whiteman DC, Pathirana T, Collins IM (2024). Estimating the magnitude and healthcare costs of melanoma in situ and thin invasive melanoma overdiagnosis in Australia. British Journal of Dermatology, 191(6): 906–913. doi:10.1093/bjd/ljae296 · PMID:38953175
  13. Lindsay A, et al. (2026). Supply, demand and needs-based workforce modelling for skin cancer in Queensland. Australasian Journal of Dermatology, Early View. doi:10.1111/ajd.70063 · PMID:41674191
  14. McLoughlin K, Watts CG, Wade S, et al. (2025). A protocol for development of a microsimulation model platform to evaluate the potential benefits, harms, and cost-effectiveness of risk-tailored melanoma screening. PLoS One, 20(12): e0339177. doi:10.1371/journal.pone.0339177
  15. Medical Services Advisory Committee (2022). MSAC Application 1699: National Lung Cancer Screening Program - Public Summary Document. Australian Department of Health. link
  16. Olsen CM, Pandeya N, Thompson BS, et al. (2018). Risk stratification for melanoma: models derived and validated in a purpose-designed prospective cohort. Journal of the National Cancer Institute, 110(10): 1075–1083. doi:10.1093/jnci/djy023 · PMID:29546283
  17. Pashayan N, Morris S, Gilbert FJ, Pharoah PDP (2018). Cost-effectiveness and benefit-to-harm ratio of risk-stratified screening for breast cancer: a life-table model. JAMA Oncology, 4(11): 1504–1510. doi:10.1001/jamaoncol.2018.1901 · PMID:29978189
  18. Reyes-Marcelino G, et al. (2026). External validation of melanoma risk prediction models in the 45 and Up Study. (Forthcoming; preliminary data via Cust lab).
  19. Vickers AJ, Elkin EB (2006). Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making, 26(6): 565–574. doi:10.1177/0272989X06295361
  20. Vuong K, McGeechan K, Armstrong BK, et al. (2016). Risk prediction models for incident primary cutaneous melanoma: a systematic review and external validation in the 45 and Up Study. JAMA Dermatology, 152(8): 889–896. doi:10.1001/jamadermatol.2016.0939 · PMID:27050141
  21. Whiteman DC, et al. (QSkin MP16 update) (2025). QSkin Sun and Health Study - extended risk model (MP16) recalibration. (internal report cited in Roadmap Evidence Synthesis §5.4).
  22. Whiteman DC, Olsen CM, Thompson BS, et al. (2022). The effect of screening on melanoma incidence and biopsy rates. British Journal of Dermatology, 187(4): 515–522. doi:10.1111/bjd.21649