Produce grading

LIVE

Bean variety from grain photographs, and wine quality from an assay panel. The wine label is a sensory panel's median score, not ground truth, and the cut that turns it into two classes was made here rather than in the source — both facts travel with the number. No separate specification document has been published for this benchmark — the MedEval-1 spec is the document, and this page is that instrument run on another corpus. Everything below is read from the same payload.json the medical leaderboard renders from.

spec MedEval-1 v0.1code fingerprint de32bca72d56seed 20260727device mpsgenerated 2026-08-03T19:03+00:00

models scored

tasks

scored records

5,029

held-out cases

91.7

best index — Random forest

The index is the weighted mean of the dimensions that were scored, on the weights published in the spec. Dimensions this corpus cannot support are excluded from it rather than counted as zero, which is why an index here is not comparable to a medical one model-for-model — it is a mean over a different set of questions, and the table below says which.

Every model, on all 9 dimensions

Ranked by index. An em-dash is a dimension the harness left null.

#	Model	Index	1 Accuracy	2 Cross-site	3 Calibration	4 Limited data	5 Subgroup	6 Corruption	7 Uncertainty	Specification sensitivity	Cost, latency & footprint
1	Random forest400 trees	91.7	93.1	—	92.6	93.3	87.4	88.3	98.9	—	—
2	Gradient boosting200 stages, depth 3, lr 0.05	90.2	93.3	—	96.1	89.3	87.6	79.1	98.8	—	—
3	MLP (128, 64)2 hidden layers, adam, early stopping when n>=200	90.1	93.0	—	93.3	94.4	88.7	78.1	98.9	—	—
4	Logistic regressionstandardised, L2, C=1	89.2	93.1	—	90.5	96.6	88.0	75.2	98.8	—	—
5	Random forest400 trees	67.6	61.0	—	73.2	64.6	54.2	80.3	78.2	—	—
6	Gradient boosting200 stages, depth 3, lr 0.05	65.1	60.2	—	67.6	65.6	57.9	72.2	73.9	—	—
7	Random forest400 trees	63.7	62.8	—	74.5	61.4	40.5	63.9	88.9	—	—
8	MLP (128, 64)2 hidden layers, adam, early stopping when n>=200	60.5	52.1	—	73.5	54.3	43.9	74.6	67.9	—	—
9	Logistic regressionstandardised, L2, C=1	59.7	50.4	—	69.3	63.1	44.9	70.8	68.8	—	—
10	Gradient boosting200 stages, depth 3, lr 0.05	57.6	49.2	—	84.9	58.6	20.5	62.8	81.9	—	—
11	MLP (128, 64)2 hidden layers, adam, early stopping when n>=200	56.3	45.7	—	91.7	61.2	12.2	60.3	80.3	—	—
12	Logistic regressionstandardised, L2, C=1	52.3	38.2	—	85.6	66.7	0.0	63.8	76.7	—	—

Empty columns, and why they are empty

2. Cross-site robustness — No re-acquisition of the same subjects exists in this corpus, so there is nothing to measure. Widening the definition to fill the cell would be a spec change, not a demo.
. — Not computed on any record in this benchmark, and not approximated.
. — Not computed on any record in this benchmark, and not approximated.

The measurements the scores came from

Raw metrics, not the 0–100 rescaling.

Model	Balanced accuracy (95% CI)	AUC	ECE	Risk–coverage AUC	Mean label-budget retention
Random forest	0.941[0.932–0.948]	0.995	0.015	0.991	0.933
Gradient boosting	0.943[0.937–0.950]	0.995	0.008	0.990	0.893
MLP (128, 64)	0.940[0.934–0.947]	0.996	0.013	0.990	0.944
Logistic regression	0.941[0.933–0.950]	0.996	0.019	0.990	0.966
Random forest	0.805[0.770–0.842]	0.869	0.054	0.891	0.646
Gradient boosting	0.801[0.763–0.841]	0.845	0.065	0.870	0.656
Random forest	0.814[0.787–0.839]	0.913	0.051	0.944	0.614
MLP (128, 64)	0.760[0.720–0.799]	0.818	0.053	0.839	0.543
Logistic regression	0.752[0.710–0.799]	0.816	0.061	0.844	0.631
Gradient boosting	0.746[0.717–0.771]	0.866	0.030	0.909	0.586
MLP (128, 64)	0.728[0.704–0.753]	0.857	0.017	0.901	0.612
Logistic regression	0.691[0.665–0.717]	0.827	0.029	0.883	0.667

Balanced accuracy carries a bootstrap 95% interval; read it before reading the rank. ECE is expected calibration error, so lower is better and the 0–100 dimension score inverts it. Every value here is on the record; hover any figure for the unrounded number.

The corpus

Dry bean variety (7-class)

dry_bean · Tabular / imaging-derived

Source: UCI Dry Bean DatasetKoklu & Ozkan, Computers and Electronics in Agriculture 174:105507, 2020
Split: minted by MedEval-1 (stratified 60/15/25, seed 20260727); no official split is published for this corpus. Parsed from the ARFF in the donor's zip, whose nominal class order is the class order used here.
Sizes: 8,166 train · 3,404 held out · 7 classes
Classes: SEKER · BARBUNYA · BOMBAY · CALI · HOROZ · SIRA · DERMASON
Registry note: Geometric descriptors measured from photographs of 13,611 real beans. Genuinely measured, but through a single controlled imaging rig, so cross-site robustness is undefined here for exactly the reason it is undefined on Adult -- there is no second acquisition of the same subjects.

Wine quality, white (binarised)

wine_quality_white · Tabular / physicochemical

Source: UCI Wine QualityCortez et al., Decision Support Systems 47(4):547-553, 2009
Split: minted by MedEval-1 (stratified 60/15/25, seed 20260727); no official split is published for this corpus (white wine, Vinho Verde). The binary target is an ANALYST CHOICE made here, not a property of the release: the donors ship an ordinal 0-10 sensory score and this task binarises it at quality >= 6. A different cut would give a different positive rate and a different board. The raw quality column is dropped from the features.
Sizes: 2,938 train · 1,225 held out · 2 classes
Classes: not good (<6) · good (>=6)
Registry note: The target is the median of three sensory-panel scores -- a human preference label, not ground truth -- and the >=6 cut that turns it into two classes is an analyst choice made here, not in the source. Both are disclosed, because a model can only be as well-defined as its label. Red and white share a schema and are genuinely different products, so the pair supports a cross-population test.

Wine quality, red (binarised)

wine_quality_red · Tabular / physicochemical

Source: UCI Wine QualityCortez et al., Decision Support Systems 47(4):547-553, 2009
Split: minted by MedEval-1 (stratified 60/15/25, seed 20260727); no official split is published for this corpus (red wine, Vinho Verde). The binary target is an ANALYST CHOICE made here, not a property of the release: the donors ship an ordinal 0-10 sensory score and this task binarises it at quality >= 6. A different cut would give a different positive rate and a different board. The raw quality column is dropped from the features.
Sizes: 959 train · 400 held out · 2 classes
Classes: not good (<6) · good (>=6)
Registry note: Same construction and the same invented threshold as the white task. A third the size, so the intervals are wider.

Re-derive every figure on this page with python src/build_site.py --emit-json in plan_d/ — the same run writes the medical leaderboard. All benchmarks reads the same 9 dimensions per domain, and the trust graph traces any figure here back to the run that produced it.