Temporal Drift in Clinical Prediction Models: Implications for NHS Safety in the Digital Age

Abstract

Background: Clinical prediction models used for clinical decision support across the NHS can lose accuracy over time due to temporal drift resulting from population changes, evolving clinical practice, or shifts in data recording. Such drift leads to incorrect risk estimates, which may undermine patient safety. This issue aligns with priorities in the NHS Ten Year Plan and the Goldacre Review, both of which emphasise safe, transparent, and continuously monitored data-driven tools. Aims: To compare strategies for detecting and repairing temporal drift using a cardiovascular risk model aligned with QRISK-2, and to assess their practicality for wide-scale NHS deployment. Methods: Using Connected Bradford data, a logistic regression model predicting 10-year stroke or heart attack risk was evaluated monthly over seven years. Drift detection methods included monitoring performance metrics, analysing residuals through statistical process control and Kullback–Leibler divergence, and assessing input data changes using discrimination error. Model updates were triggered when predefined safety thresholds were exceeded. Results: Pronounced temporal drift was observed. Monitoring against performance thresholds produced the most accurate and stable model overall. Residual-based approaches showed similar trends. Regular three-year recalibration offered consistent performance while allowing predictable operational planning. Discrimination error detected drift without requiring long-term outcome data, making it suitable for prediction models targeting long-term patient outcomes. Conclusion: Effective monitoring and regulation around temporal drift is essential for safe clinical decision support within the NHS. Threshold-based strategies offer strong accuracy, while simpler or outcome-independent approaches provide operational advantages. These findings support the Goldacre Review’s call for robust monitoring frameworks and align with the NHS Ten Year Plan’s commitment to trustworthy, adaptive AI-driven healthcare. Findings will feed into an upcoming workshop with the MHRA, DHSC, and other policymakers about the regulatory framework to use moving forwards.