Species from field recordings
LIVEAnuran calls as MFCCs, split by recording and never by row. Syllables from one recording are near-duplicates, so a random split scores memorisation and returns about 99%. The grouped split is materially harder and is the only honest one. No separate specification document has been published for this benchmark — the MedEval-1 spec is the document, and this page is that instrument run on another corpus. Everything below is read from the same payload.json the medical leaderboard renders from.
The index is the weighted mean of the dimensions that were scored, on the weights published in the spec. Dimensions this corpus cannot support are excluded from it rather than counted as zero, which is why an index here is not comparable to a medical one model-for-model — it is a mean over a different set of questions, and the table below says which.
Every model, on all 9 dimensions
Ranked by index. An em-dash is a dimension the harness left null.| # | Model | Index | 1 Accuracy | 2 Cross-site | 3 Calibration | 4 Limited data | 5 Subgroup | 6 Corruption | 7 Uncertainty | Specification sensitivity | Cost, latency & footprint |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Random forest400 trees | 70.0 | 74.7 | — | 27.2 | 87.1 | 75.6 | 83.6 | 84.0 | — | — |
| 2 | MLP (128, 64)2 hidden layers, adam, early stopping when n>=200 | 64.2 | 76.9 | — | 11.7 | 85.0 | 82.6 | 65.9 | 69.9 | — | — |
| 3 | Logistic regressionstandardised, L2, C=1 | 59.0 | 72.2 | — | 0.0 | 87.0 | 76.1 | 64.8 | 58.6 | — | — |
| 4 | Gradient boosting200 stages, depth 3, lr 0.05 | 56.1 | 61.0 | — | 0.0 | 74.9 | 62.3 | 79.9 | 70.4 | — | — |
- 2. Cross-site robustness — No re-acquisition of the same subjects exists in this corpus, so there is nothing to measure. Widening the definition to fill the cell would be a spec change, not a demo.
- . — Not computed on any record in this benchmark, and not approximated.
- . — Not computed on any record in this benchmark, and not approximated.
The measurements the scores came from
Raw metrics, not the 0–100 rescaling.| Model | Balanced accuracy (95% CI) | AUC | ECE | Risk–coverage AUC | Mean label-budget retention |
|---|---|---|---|---|---|
| Random forest | 0.772[0.737–0.795] | 0.990 | 0.146 | 0.856 | 0.871 |
| MLP (128, 64) | 0.792[0.751–0.817] | 0.955 | 0.177 | 0.729 | 0.850 |
| Logistic regression | 0.749[0.708–0.789] | 0.928 | 0.261 | 0.627 | 0.870 |
| Gradient boosting | 0.649[0.595–0.691] | 0.937 | 0.301 | 0.733 | 0.749 |
Balanced accuracy carries a bootstrap 95% interval; on a held-out split of 995 those intervals are wide, and they are the reason the ranking above should be read as a grouping rather than an order. Every value here is on the record; hover any figure for the unrounded number.
Who the best model fails
LIVEDimension five, on Random forest — the top-ranked model above. Basis, in the evaluator’s own words: worst demographic subgroup (real metadata). This is the strong reading — the corpus ships the attributes, so the harness did not have to fall back to the worst-class proxy every medical imaging task uses.
| Leptodactylidaeworst | 0.780 | |
| Hylidae | 0.890 |
Per-group recall, and the gap between the best and worst group. These numbers measure who the model fails on this corpus. They are not an audit of any deployed system, and the subgroup attributes are model inputs here, as they are in the standard formulation of the task.
The corpus
Anuran call species (10-class)
anuran_mfcc · Bioacoustic / MFCC- Source
- UCI Anuran Calls (MFCCs)Colonna et al., Anuran Calls (MFCCs), UCI ML Repository, DOI 10.24432/C5CC9H
- Split
- no official split; minted at the standard seed (20260727) by partitioning the 60 RecordID recording groups 60/15/15/25 and taking whole recordings, so no recording spans two splits. A row-level random split would place near-duplicate syllables from one recording on both sides and score memorisation (~99%); class balance is therefore uncontrolled across splits and is not stratified
- Sizes
- 5,073 train · 995 held out · 10 classes
- Classes
- AdenomeraAndre · AdenomeraHylaedactylus · Ameeregatrivittata · HylaMinuta · HypsiboasCinerascens · HypsiboasCordobae · LeptodactylusFuscus · OsteocephalusOophagus · Rhinellagranulosa · ScinaxRuber
- Registry note
- Split is GROUPED BY RecordID, not random. Each field recording contributes many syllables, so a random split puts syllables from one recording on both sides and measures memorisation: it returns ~99% and means nothing. The grouped split is materially harder and is the only honest one. Family and genus are real taxonomic strata for dimension 5.
Re-derive every figure on this page with python src/build_site.py --emit-json in plan_d/ — the same run writes the medical leaderboard. All benchmarks reads the same 9 dimensions per domain, and the trust graph traces any figure here back to the run that produced it.