Putting clinical records into neat time intervals can change the sequence an AI model appears to learn. A new reinforcement-learning study found that this preprocessing could produce optimistic retrospective evaluations even when the resulting models performed poorly in a simulated treatment environment. It did not measure harm to patients receiving an AI-directed treatment. Original study
The finding concerns a choice made before a model is evaluated: whether irregular observations and decisions retain their original timing.
A regular row can hide an irregular decision
Clinical events do not necessarily occur on an hourly schedule. A measurement may trigger an adjustment minutes later, followed by a longer period without another decision. Combining events inside a fixed interval can lose that ordering.
As an illustration, a measurement followed by an action may be recorded as one time-bin summary. The action can then appear to precede, rather than respond to, the information that triggered it. This is an explanatory example, not a patient record collected by Healthcare Edition.
The authors examined glucose measurements and insulin adjustments in MIMIC-IV intensive-care records to characterize such distortions. That observational analysis identifies changes to the representation of existing events; it is different from testing a new treatment policy on those patients. Real-record analysis and Figure 1
The treatment-policy experiment used virtual patients
The research then used a modified UVA/Padova diabetes simulator with 30 virtual patients. Eighteen were assigned to training, six to validation and six to testing, with adult, adolescent and child profiles represented in each group.
The researchers compared behavioral cloning, implicit Q-learning and conservative Q-learning across differently prepared versions of training data. Behavioral cloning supplied a supervised imitation baseline; the other methods sought better long-term policies.
| Data preparation | What changes | Why timing can matter |
|---|---|---|
| Unprocessed decisions | Original irregular decision intervals are retained | The model encounters the timing structure represented in the generated data |
| Interpolation | A sustained action is represented at more frequent regular steps | The model can learn to expect opportunities to act more often |
| Binning | Observations and actions are aggregated into broader intervals | Ordering and short-lived events can be obscured |
This table summarizes the study design, not a universal ranking of all preprocessing methods. Methods and simulated results
The retrospective test could share the distortion
Models trained on the unprocessed datasets generally performed better in the authors’ simulated evaluation. The paper also found that fitted Q-evaluation, a method for estimating policy performance from stored data, was overoptimistic for agents trained on binned data.
The important point is the combination: the evaluation could inherit a misleading representation of the environment used to train the model. A strong score on that representation was not necessarily a strong result when the agent interacted with the irregularly timed simulator. Calibration analysis, Figure 4
Reported reward scores are constructed model-performance measures. They are not percentages of patients helped, and a relative change in that score is not a patient-harm rate.
The limits are part of the result
The authors used model-free methods, simplified observation spaces and randomly varying decision intervals. A more complex real environment or another algorithm family could behave differently. They also distinguish irregular decision times from the separate challenge of irregular observations.
The paper provides simulated code and data, while the underlying MIMIC-IV records require credentialed access. This article reports the authors’ experiment; it does not claim to have rerun that experiment or accessed restricted patient data.
Our physician-learning analysis examines another route by which an AI system can influence decisions. Here the issue arises earlier: changing what happened when can change what a model learns, and can also change how convincing its evaluation looks.
