Can Machine Learning Improve QMED?

Research question

Can a machine-learning challenger improve ungauged QMED accuracy over donor-adjusted SM2025 when both methods are judged on the same held-out stations and only training-fold donors are available? The competing hypotheses were an accuracy improvement from nonlinear combination of national inputs, or no improvement once geographic transfer and donor custody were handled fairly.

Data and cohort

The frozen cohort contains 918 stations whose QMED targets were derived from at least eight non-rejected AMAX years. The challenger used broad families describing catchment scale, hydroclimate, terrain, land cover and river-network context, together with the statistical-method estimate. Exact feature construction and selection are outside the public disclosure boundary.

Results are reported for the national cohort and for two deliberately visible small-catchment strata. The below 10 km² stratum has 26 stations and the 10–25 km² stratum has 55, so the subgroup evidence must be read with its sample size rather than as a stable guarantee for an individual catchment.

Method and validation question

Random K-fold validation asks how the fitted relationship interpolates when training stations may remain geographically close to evaluation stations. Spatial-cluster cross-validation asks the harder operational question: how well does the method transfer when a centroid-based geographic group is held out? The experiment used five unbuffered centroid/KMeans clusters. No distance buffer was imposed, so the result does not assert a protected separation distance.

Performance was evaluated in log space using R² and FSE, with median absolute error reported in flow space. The paired station result is the primary verdict; random-fold and donor-custody comparisons are sensitivity checks that explain why a more permissive design can produce a stronger-looking headline.

Information asymmetry and donor custody

Donor selection is part of the evaluation. An all-donor baseline can use nearby gauges that belong to the held-out population, whereas the train-only baseline limits donor information to what would be available within the fitted side of each fold. The primary comparison therefore uses train-only donors.

SM2025 was an input to the challenger. It is therefore a secondary challenger that tries to refine an established statistical signal, not a separate estimate built without that signal. URBEXT was zero throughout the frozen cohort, so the urban-adjustment chain was not exercised. Both facts narrow the claim before any score is read.

Results and the negative verdict

The challenger did not beat the benchmark. It was behind donor-adjusted SM2025 on the national R², median absolute error and FSE results, and the paired R² interval remained on the unfavourable side of zero. The result is a failed accuracy hypothesis, not an ambiguous tie.

The validation-design table shows why the random-fold result was not enough. Geographic grouping reduced the challenger score, while enforcing train-only donor custody reduced the apparent donor result. Fairness required both corrections at once. The small-catchment table also shows no recovery of the hypothesis: SM2025 retained lower median error in both reported bands and stronger R² in the larger of the two bands.

Small-catchment instability

Small catchments are not a footnote to the national score. The below-10 km² band combines a small sample with weak R² for both methods, so small changes in station composition can materially alter the statistic. The 10–25 km² band is larger but still modest and remains adverse to the challenger. These are cohort diagnostics, not local uncertainty intervals.

Operational implications

The accuracy-win claim failed, but the exercise retained three useful properties. National open-input access supports screening without presenting access as superior prediction. A frozen evidence revision, stated fold construction and train-only donor rule make the result repeatable. Exact tables, subgroup results and explicit information custody make it auditable by a hydrologist who cannot see the private implementation.

In practice, the challenger should not displace donor-adjusted SM2025 on the evidence reported here. Its surviving value is as an auditable research and review layer: it can expose sensitivity, support national comparison and test future hypotheses under the same fair protocol.

Limitations

• The geographic clusters were centroid-based and unbuffered, so the study did not test transfer across a guaranteed separation distance.
• SM2025 was already present in the challenger inputs, which limits the comparison to a secondary challenger rather than a standalone alternative.
• URBEXT carried no variation, so the experiment provides no evidence about performance when the urban-adjustment chain is active.
• The smallest area bands contain limited station counts and cannot support a catchment-specific performance guarantee.
• Cross-validation describes this frozen cohort and evidence cut-off; it does not remove rating, target or descriptor uncertainty.

What we do not conclude

• We do not conclude that the challenger improves QMED point accuracy over donor-adjusted SM2025.
• We do not conclude that machine learning has no role in flood hydrology; this verdict applies to the tested QMED challenger and frozen validation design.
• We do not conclude that the result validates urban QMED behaviour, establishes site-specific uncertainty or replaces analyst review.
• We do not claim approval, certification or replacement of official flood-estimation methods.

Disclosure boundary

We disclose the research question, broad cohort rules, feature families, validation logic, donor custody, information asymmetry, exact national and subgroup results, failed hypothesis, limitations and evidence revision. We withhold exact feature engineering and selection, learned coefficients, model weights, full training recipes, private data-construction mechanics, code-level workflow logic and operational infrastructure. The public record is sufficient to audit the claim without providing a deployable reconstruction.