Bridging the Interpretability Gap in Machine Learning for Water Quality

ABSTRACT: Machine learning (ML) has been widely used to predict nonlinear relationships among water quality variables and to address complex ecological problems. However, the ML models’ results are sometimes interpreted as real ecological findings. ML tools like explainable artificial intelligence explain model behavior but not the natural processes occurring in the water system. We define this limitation as an “interpretability gap” between the output of ML models and ecological causality. For example, in a random forest model of benthic chlorophyll-a, colloidal phosphorus was the most influential predictor, but algae cannot take up colloidal phosphorus directly. A practical rule is to check whether a variable can affect the response directly before calling it a driver. This can lead to incorrect conclusions, especially when input variables are highly collinear, as in nutrient cycling and across physical and biological processes. This interpretability gap is not only a technical issue in ML models but also a major scientific concern that may lead a new researcher down the wrong track. To address this limitation, we proposed a process-based hydraulic and biogeochemical model alongside ML models, because high accuracy must be supported by ecological understanding. Closing this gap is essential for making ML results more reliable for science and decision-making. We frame this challenge from a watershed perspective, since the watershed is widely recognized as the fundamental spatial unit for integrating the hydrological, biogeochemical, and land-use processes that jointly determine water quality outcomes.