Room occupancy from sensors
LIVEThe first non-medical corpus in the suite where dimension two is measured rather than nulled. The donors released one training period and two held-out test periods recorded under different physical conditions — a real re-collection of the same instrument, not a simulated perturbation. No separate specification document has been published for this benchmark — the MedEval-1 spec is the document, and this page is that instrument run on another corpus. Everything below is read from the same payload.json the medical leaderboard renders from.
The index is the weighted mean of the dimensions that were scored, on the weights published in the spec. Dimensions this corpus cannot support are excluded from it rather than counted as zero, which is why an index here is not comparable to a medical one model-for-model — it is a mean over a different set of questions, and the table below says which.
Every model, on all 9 dimensions
Ranked by index. An em-dash is a dimension the harness left null.| # | Model | Index | 1 Accuracy | 2 Cross-site | 3 Calibration | 4 Limited data | 5 Subgroup | 6 Corruption | 7 Uncertainty | Specification sensitivity | Cost, latency & footprint |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Logistic regressionstandardised, L2, C=1 | 92.8 | 95.5 | — | 89.9 | 93.0 | 93.6 | 88.2 | 99.4 | — | — |
| 2 | Random forest400 trees | 88.7 | 89.9 | — | 84.9 | 94.7 | 86.0 | 85.3 | 98.2 | — | — |
| 3 | MLP (128, 64)2 hidden layers, adam, early stopping when n>=200 | 88.5 | 95.1 | — | 83.7 | 76.7 | 90.9 | 83.7 | 98.6 | — | — |
| 4 | Gradient boosting200 stages, depth 3, lr 0.05 | 85.1 | 86.3 | — | 85.4 | 94.3 | 79.6 | 76.9 | 98.5 | — | — |
| 5 | Logistic regressionstandardised, L2, C=1 | 84.7 | 80.0 | — | 83.6 | 96.8 | 74.9 | 88.4 | 98.4 | — | — |
| 6 | Random forest400 trees | 76.3 | 94.9 | — | 10.8 | 89.7 | 91.4 | 81.6 | 98.2 | — | — |
| 7 | MLP (128, 64)2 hidden layers, adam, early stopping when n>=200 | 75.2 | 75.1 | — | 64.1 | 70.5 | 74.1 | 81.2 | 96.4 | — | — |
| 8 | Gradient boosting200 stages, depth 3, lr 0.05 | 73.3 | 71.8 | — | 55.2 | 93.3 | 65.4 | 78.1 | 97.3 | — | — |
- 2. Cross-site robustness — No re-acquisition of the same subjects exists in this corpus, so there is nothing to measure. Widening the definition to fill the cell would be a spec change, not a demo.
- . — Not computed on any record in this benchmark, and not approximated.
- . — Not computed on any record in this benchmark, and not approximated.
The measurements the scores came from
Raw metrics, not the 0–100 rescaling.| Model | Balanced accuracy (95% CI) | AUC | ECE | Risk–coverage AUC | Mean label-budget retention |
|---|---|---|---|---|---|
| Logistic regression | 0.977[0.972–0.982] | 0.992 | 0.020 | 0.997 | 0.930 |
| Random forest | 0.950[0.940–0.957] | 0.985 | 0.030 | 0.991 | 0.947 |
| MLP (128, 64) | 0.976[0.971–0.980] | 0.992 | 0.033 | 0.993 | 0.767 |
| Gradient boosting | 0.932[0.922–0.942] | 0.985 | 0.029 | 0.993 | 0.943 |
| Logistic regression | 0.900[0.892–0.909] | 0.980 | 0.033 | 0.992 | 0.968 |
| Random forest | 0.975[0.972–0.977] | 0.994 | 0.178 | 0.991 | 0.897 |
| MLP (128, 64) | 0.876[0.869–0.883] | 0.967 | 0.072 | 0.982 | 0.705 |
| Gradient boosting | 0.859[0.850–0.867] | 0.972 | 0.090 | 0.986 | 0.933 |
Balanced accuracy carries a bootstrap 95% interval; read it before reading the rank. ECE is expected calibration error, so lower is better and the 0–100 dimension score inverts it. Every value here is on the record; hover any figure for the unrounded number.
The corpus
Room occupancy from sensors (period 1)
occupancy_detection · Tabular / environmental sensors- Source
- UCI Occupancy DetectionCandanedo & Feldheim, Energy and Buildings 112:28-39, 2016
- Split
- official released split: the donor released one training period (datatraining.txt, 8143 rows) and two separate later test periods (datatest.txt 2665 rows, datatest2.txt 9752 rows); THIS task scores test period 1 (datatest.txt) whole and never fits on it. Validation is the last 15% of the training period in time order, not a random carve -- consecutive rows are one minute apart, so a random validation set would be scored on rows whose own neighbours are in train.
- Sizes
- 6,922 train · 2,665 held out · 2 classes
- Classes
- unoccupied · occupied
- Registry note
- The donors released one training period and TWO held-out test periods recorded under different physical conditions. That is a genuine re-collection of the same instrument in the same room -- not a simulated perturbation -- so this family is the first non-medical member of the cross-site test. Headline accuracy is high because the light sensor is nearly decisive during office hours; the number that matters is the gap between the two periods, which is what dimension 2 measures.
Room occupancy from sensors (period 2)
occupancy_detection_2 · Tabular / environmental sensors- Source
- UCI Occupancy DetectionCandanedo & Feldheim, Energy and Buildings 112:28-39, 2016
- Split
- official released split: the donor released one training period (datatraining.txt, 8143 rows) and two separate later test periods (datatest.txt 2665 rows, datatest2.txt 9752 rows); THIS task scores test period 2 (datatest2.txt) whole and never fits on it. Validation is the last 15% of the training period in time order, not a random carve -- consecutive rows are one minute apart, so a random validation set would be scored on rows whose own neighbours are in train.
- Sizes
- 6,922 train · 9,752 held out · 2 classes
- Classes
- unoccupied · occupied
- Registry note
- Same training period and schema as period 1, scored on the donors' second held-out period. The pair is the finding; neither task on its own is.
Re-derive every figure on this page with python src/build_site.py --emit-json in plan_d/ — the same run writes the medical leaderboard. All benchmarks reads the same 9 dimensions per domain, and the trust graph traces any figure here back to the run that produced it.