Detecting and Mitigating Data Leakage in Breast Cancer Prediction Using an Explainable Machine Learning Framework with XGBoost and SHAP
Machine learning models have shown considerable potential for clinical prediction; however, their reported performance can be substantially affected by data leakage, leading to unreliable estimates of predictive capability. This study proposes an explainable leakageaware framework for identifying and mitigating data leakage in breast cancer prediction models. The proposed framework integrates data preprocessing, feature auditing, Extreme Gradient Boosting (XGBoost)-based classification, and SHapley Additive exPlanations (SHAP) analysis to examine whether highly influential predictors represent clinically available information or outcome-related leakage. Experiments were conducted using two breast cancer datasets, including the Breast Cancer Wisconsin dataset from Kaggle and a clinical dataset from Mendeley Data. The initial model trained on the Mendeley dataset achieved an accuracy of 0.9710 and receiver operating characteristic-area under the curve (ROC-AUC) of 0.9968; however, SHAP analysis revealed that several dominant predictors were associated with post-outcome information, indicating potential target leakage. After removing leakage-prone variables, the model performance decreased to a more realistic level, achieving an accuracy of 0.8950 and ROC-AUC of 0.9416. Additional experiments using random forest (RF), support vector machine (SVM), and CatBoost confirmed that leakage effects were not limited to a specific algorithm. The results demonstrate that explainability-based auditing can support more reliable evaluation of clinical machine learning models by identifying misleading predictive patterns caused by inappropriate features.