Land cover from satellite
LIVEStatlog Landsat, four spectral bands over a 3x3 neighbourhood, on the donors' own released train/test split. The donors withdrew one class before release, so the raw codes are not contiguous — the gap is remapped and disclosed rather than quietly closed. No separate specification document has been published for this benchmark — the MedEval-1 spec is the document, and this page is that instrument run on another corpus. Everything below is read from the same payload.json the medical leaderboard renders from.
The index is the weighted mean of the dimensions that were scored, on the weights published in the spec. Dimensions this corpus cannot support are excluded from it rather than counted as zero, which is why an index here is not comparable to a medical one model-for-model — it is a mean over a different set of questions, and the table below says which.
Every model, on all 9 dimensions
Ranked by index. An em-dash is a dimension the harness left null.| # | Model | Index | 1 Accuracy | 2 Cross-site | 3 Calibration | 4 Limited data | 5 Subgroup | 6 Corruption | 7 Uncertainty | Specification sensitivity | Cost, latency & footprint |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | MLP (128, 64)2 hidden layers, adam, early stopping when n>=200 | 83.2 | 86.1 | — | 89.5 | 84.8 | 67.0 | 78.4 | 97.6 | — | — |
| 2 | Random forest400 trees | 80.0 | 86.7 | — | 69.5 | 88.8 | 55.6 | 84.8 | 98.1 | — | — |
| 3 | Gradient boosting200 stages, depth 3, lr 0.05 | 79.3 | 83.9 | — | 89.2 | 82.8 | 52.2 | 73.5 | 97.5 | — | — |
| 4 | Logistic regressionstandardised, L2, C=1 | 71.1 | 74.6 | — | 87.0 | 97.1 | 14.7 | 67.3 | 95.5 | — | — |
- 2. Cross-site robustness — No re-acquisition of the same subjects exists in this corpus, so there is nothing to measure. Widening the definition to fill the cell would be a spec change, not a demo.
- . — Not computed on any record in this benchmark, and not approximated.
- . — Not computed on any record in this benchmark, and not approximated.
The measurements the scores came from
Raw metrics, not the 0–100 rescaling.| Model | Balanced accuracy (95% CI) | AUC | ECE | Risk–coverage AUC | Mean label-budget retention |
|---|---|---|---|---|---|
| MLP (128, 64) | 0.884[0.871–0.897] | 0.988 | 0.021 | 0.980 | 0.848 |
| Random forest | 0.890[0.874–0.903] | 0.990 | 0.061 | 0.984 | 0.888 |
| Gradient boosting | 0.866[0.851–0.880] | 0.985 | 0.022 | 0.979 | 0.828 |
| Logistic regression | 0.789[0.775–0.805] | 0.976 | 0.026 | 0.963 | 0.971 |
Balanced accuracy carries a bootstrap 95% interval; read it before reading the rank. ECE is expected calibration error, so lower is better and the 0–100 dimension score inverts it. Every value here is on the record; hover any figure for the unrounded number.
The corpus
Landsat satellite land cover (6-class)
landsat_statlog · Tabular / multispectral- Source
- UCI Statlog (Landsat Satellite)Srinivasan, Statlog (Landsat Satellite), UCI ML Repository, DOI 10.24432/C55887
- Split
- official released split (Statlog sat.trn 4435 / sat.tst 2000), scored whole, with validation carved from sat.trn only (stratified 15%, seed 20260727). LEAKS, AND THE ACCURACIES HERE ARE OPTIMISTIC BECAUSE OF IT: each row is a 3x3 ground window, and 1,939 of the 2,000 test windows (97%) have an immediate neighbour in the training file sharing six of their nine ground pixels -- the two files are interleaved windows of one 82x100 scene, not separated regions. No 36-vector appears in both files, so byte-level deduplication passes while the test set overlaps training data on the ground; byte disjointness is not independence. Measured in variation/modalities/multispectral.md section 8. Raw class codes are 1,2,3,4,5,7 -- code 6 was withdrawn by the donors -- and are mapped positionally onto 0..5, not by subtracting one.
- Sizes
- 3,769 train · 2,000 held out · 6 classes
- Classes
- red soil · cotton crop · grey soil · damp grey soil · soil with vegetation stubble · very damp grey soil
- Registry note
- Four spectral bands over a 3x3 ground neighbourhood. LEAKING, AND KEPT ON THE BOARD DELIBERATELY: the released train/test split is official but not spatially separated, and 97% of test windows share six of their nine ground pixels with a training window. No 36-vector appears in both files, so a byte-level dedup check passes while the test set overlaps training data on the ground -- this task is the worked demonstration that byte disjointness is not independence, which is why it stays published with the measurement attached rather than being quietly withdrawn. Read every accuracy here as an upper bound. See split_origin and variation/modalities/multispectral.md section 8. The donors deleted class 6 ('mixed soil') before release, so the raw labels run 1-5 and 7; they are remapped positionally and the gap is disclosed rather than silently closed.
Re-derive every figure on this page with python src/build_site.py --emit-json in plan_d/ — the same run writes the medical leaderboard. All benchmarks reads the same 9 dimensions per domain, and the trust graph traces any figure here back to the run that produced it.