Responsible implementation AI-based radiological protocol recommendation in Denmark

There is pressure on hospitals to implement AI systems which promise to improve diagnoses and save time for the doctors. One use-case could be related to the automation of protocoling based on a physician referral. Currently, this requires a referral letter from a physician who has examined a patient and evaluates that there is a need for additional imaging studies. In this case, the physician would send an electronic referral letter to a radiologist who would allocate the appropriate scan protocol given the patient’s signs and symptoms. This process may be automated using AI.

The referral process currently takes radiologists at Rigshospitalet, who review the referrals from multiple hospitals, 1-2 hours a day. The task appears to be straightforward to automate, especially with the increasing availability of (large) language models. However, in practice there could be several issues with training and evaluating a language model with data collected from past referrals:

• Different level of experience of physicians and different practices across hospitals, for example abbreviations
• Very short reports that could be associated with multiple classes, for example “status” . More information on the patient is needed in this case
• Some protocols are much more common than others, the data is highly imbalanced
• The “ground truth” of what was recommended by the radiologist, might not accurately reflect what scan was actually needed
• Some patients might not have been referred for a scan when in fact they did need one, so the coverage of the data is not complete

Given these data characteristics, a model trained on existing data is likely to overfit on various shortcuts. For example a model might learn that a physician who uses their initials in the report, often recommends a specific protocol. Using the available data, the model would appear to produce correct protocol predictions, but what it learned is not actually related to the patient’s symptoms, leading to major problems over time.

There are multiple projects possible on this theme, for example:

• Identifying the requirements of the referral process (for example accuracy, time spent), what are acceptable and unacceptable errors of a potential AI system?
• Mapping out the variables involved, their interactions, and potential sources of bias, how could these affect different data distributions if translated to a machine learning task?
• Creating synthetic datasets which would reveal shortcut learning or other biases in trained models
• Few-shot transfer learning of existing models to Danish data
• Etc

The projects are suitable as MSc thesis projects and will be done in collaboration with Jack Xu (MD, PhD, Post-doc & Radiology Resident at Herlev-Gentofte Hospital). The projects will be done partially at the hospital due to the restrictions in data access.