Forecast-Driven Doctor Scheduling in Primary Healthcare: Ensemble Learning, a Reinforcement Learning Agent, and an Exact Baseline

Abstract

Doctor scheduling in primary healthcare facilities is challenging because patient visit volume fluctuates due to seasonality, holidays, weather, and unplanned staff absences, while rule-based scheduling used in many healthcare systems remains static and unresponsive. This study builds and rigorously validates a forecast-driven scheduling pipeline for a primary healthcare center (Puskesmas Talang Ratu, Palembang, Indonesia), using five years (2019–2024) of visit, attendance, weather, and holiday records engineered into 46 temporal, operational, and contextual features. Six ensemble regressors were compared using a chronological split; Random Forest generalised best (MAE = 7.08, RMSE = 10.44, R² = 0.82, next-day visits). The forecast was embedded in a Deep Q-Network (DQN) scheduling agent and benchmarked, under a leak-free walk-forward forecasting procedure with the forecast-to-decision day alignment corrected, against three baselines: the facility's existing rule-based practice, a constraint-aware forecast-to-staffing rule, and a myopic cost-optimal integer program that solves the DQN's own objective exactly each day. Over a 301-day held-out period, the exact optimizer achieved the best capacity match (MAE 8.11) and the lowest objective cost, ahead of the constraint-aware rule (MAE 8.42) and the DQN policy (MAE 9.90–10.26 across reward-scaling and seed variants), all of which greatly outperformed the existing rule-based practice (MAE 20.61). The DQN did not surpass the exact optimizer even on its own training objective, a result that held across four random seeds, extended training, and reward rescaling. We trace this to a structural property of the environment: because staffing decisions do not influence future states or rewards, the task reduces to a per-day (contextual-bandit-like) optimization rather than a genuinely sequential one, removing the advantage reinforcement learning derives from learning delayed returns. The contribution is therefore a leak-free, day-aligned forecasting-to-scheduling pipeline and an empirically grounded account of when reinforcement learning is, and is not, warranted for demand-responsive capacity allocation in primary healthcare.