A medical device that is software is still a medical device. An artificial intelligence-enabled medical device (AIMD) takes inputs, runs them through an algorithm, and returns an output, whether that output is an arrhythmia flag, a suspicious region on a mammogram, or a sepsis risk score. What is different is that it provides information based on learned behavior rather than a physical intervention, which changes how success is measured and what evidence is needed to demonstrate it.
The shift is clearest in the risk analysis. The core hazard of an AIMD is that it gives the wrong answer, or no answer at all. The core benefit is the reverse: it gives the right answer, in time to influence care. Because benefit and risk are both measured by accuracy, the way a model is built, trained, and tested is as much a part of the clinical evidence as it is an engineering exercise.
This article walks through how manufacturers can build compelling evidence for algorithm performance, data quality, bias, and clinical validation. Each of these areas benefits from early input by specialists who understand how reviewers evaluate AI-enabled devices. A useful anchor throughout is the International Medical Device Regulators Forum (IMDRF) Good Machine Learning Practice (GMLP) guiding principles, finalized in January 2025.1 GMLP is best applied as a way of working across the product life cycle rather than a pre-submission checklist, and manufacturers that build it into design controls from day one find clinical evidence far easier present and defend.
Building and training the model
Match the model to the data
The first step in building any model is choosing an architecture that suits the type and structure of the data. A convolutional neural network (CNN) is well suited to images such as mammograms or chest X-rays. A large language model (LLM) is a natural fit for the conversational speech and text managed by an ambient clinical scribe, but not for classifying an electrocardiogram (ECG) waveform. GMLP reflects this in its principle that model choice and design are tailored to the available data.
Start with a valid clinical association
Before training begins, the manufacturer should be able to answer a basic question: Do these inputs make sense for this clinical condition? The IMDRF software as a medical device (SaMD) clinical evaluation guidance (N41)2 and its EU counterpart, the Medical Device Coordination Group’s MDCG 2020-1,3 frame this as valid clinical association: the extent to which the software’s output, based on the inputs and algorithms selected, is associated with the targeted physiological state or clinical condition.
Consider an at-home device for sleep apnea diagnosis. Demonstrating valid clinical association means showing that the algorithm’s inputs, such as respiratory rate and electroencephalography (EEG), are established measures for evaluating sleep apnea. If the inputs cannot be tied to the condition clearly, then no amount of training data will produce a reliable model.
Build the right training dataset
A model is only as good as the data it learns from, and manufacturers should prioritize amount, quality, and diversity when curating the training dataset. There must be enough data for the model to learn the signal without overfitting, including enough examples of less common presentations of the condition. That data must also be acquired, cleaned, and labeled consistently. FDA’s January 2025 draft guidance on AI-enabled device software functions4 directs manufacturers to describe each of these steps, including the qualifications and training of the annotators.
A diverse dataset is an important way to reduce bias in the model. Bias, as defined in international AI standards, is a systematic difference in treatment of certain objects, people, or groups in comparison to others, and every AIMD carries some. To reduce the risk of bias, training data should represent the full intended patient population across age, sex, race and ethnicity, disease severity, care setting, geography, and acquisition equipment. If all training data come from one hospital, the model may be tuned to that hospital’s equipment, patient mix, and clinical practice rather than the disease. A skin-lesion classifier trained mostly on 20- to 30-year-olds, or an algorithm processing pulse oximetry signals, which are known to be affected by skin pigmentation, are at high risk of demographic bias. A mammography model trained on low-prevalence screening data may underperform on diagnostic follow-up scans of women with a suspected lesion, a form of disease-state bias.
The goal is to recognize bias, then reduce it or account for it. The newly published ISO/TS 24971-2:2026,5 which guides the application of ISO 14971 to machine learning, treats bias as a hazard like any other. Each potential source, whether development decisions, the data itself, or performance that still differs between groups, goes into hazard identification, with risk estimated and controlled. Controls might include more diverse data, a human in the loop, or a labeling change that narrows the intended population, though narrowing the population has health-equity implications, especially for demographic bias.
Establish an appropriate ground truth
For supervised learning, every training example carries a label derived from the reference standard or ground truth. GMLP calls for reference standards that are fit for purpose, and the first step is to establish the best available source. For a chest X-ray tool intended to diagnose COVID-19, that means a molecular test, not a radiologist’s impression of the same image. For a device that flags a patient’s risk of developing sepsis, a process where no single test settles the question, expert adjudicators should annotate the data. When clinicians do the annotation, getting the right experts is critical. Their specialty, training, and years of experience should match the clinical task, and annotation methods, including quality control, should be documented.
Validating and testing the model
To data scientists, “validation” often means tuning the model before it is locked. However, to regulators, it means evidence that the finished device meets its intended purpose.
Initial validation is analogous to bench testing. Once the model is locked, its technical performance is measured on an independent validation set, typically retrospective data. This is the technical or analytical performance pillar of IMDRF N41 and MDCG 2020-1. Endpoints typically include accuracy, as measured by analytical sensitivity and specificity.
Clinical validation represents testing in the appropriate clinical setting. Even when the test set contains real patient data, using retrospective datasets may not be considered comprehensive clinical performance testing. MDCG 2020-1 expects clinical performance to be shown for the intended use, target population, use conditions, use environment, and all intended user groups. Ideally this is done prospectively. GMLP reinforces this in two principles that emphasize assessing human-AI interaction in the intended use environment and showing performance under clinically relevant conditions. For a second-reader tool, the question is not only one of algorithm accuracy, but one of clinician-algorithm collaboration. Clinical endpoints resemble the analytical ones and include clinical sensitivity and specificity, positive and negative predictive values, and likelihood ratios. With a human in the loop, however, usability becomes paramount for the device to provide value in the clinical setting.
An accurate algorithm can still cause harm if it is used incorrectly or its output is misread. Human factors testing is especially important when retrospective datasets are used for validation, since a retrospective study shows the algorithm works but not that people using it in practice get the same result. Additional care is needed when the users are lay people, such as patients and caregivers, who may not know the device’s limitations or how to act on its output. Usability depends most on whether users know how to deploy the device and how to interpret its output. Common concerns like automation bias, where a clinician begins to blindly trust the output, are documented in both the usability file and the risk analysis.
Whether retrospective or prospective, the test dataset must be independent of the training data, not only in patients but in sites. GMLP asks manufacturers to consider all sources of dependence, including patients, sites and data acquisition, since images from the same patient or the same scanner are not truly independent. A random split of one pooled dataset is increasingly challenged in regulatory review, while test data from new sites make a much stronger case. Ground truth for the test set are generally established with more rigorous adjudication methods compared to the annotation used for training, and it should be done by a separate panel of experts not involved in training or tuning. Since even experts can disagree, an adjudication plan should be established in advance that documents these qualifications and the study-specific instructions. The plan should also specify how many readers annotate each case, whether they are blinded to each other, and how disagreements are resolved, such as by majority vote or a senior adjudicator. Regulators have scrutinized ground truth more closely in recent years and are requesting formal adjudication processes more often, so manufacturers should document both the plan and adherence to it.
Validation and test data may be retrospective and may come from open-source datasets. Public datasets can be useful, but manufacturers should confirm the quality, diversity, and appropriateness of the data before relying on it. It is beneficial to establish a line of communication with the data provider in case regulators have additional questions during review. Above all, the test set should represent the intended purpose, matching the intended population, disease prevalence and severity, care setting, and equipment, with enough cases to support subgroup analyses that can reveal bias introduced during training.
As with all medical devices, sample sizes for the test sets should be statistically justified. Confidence level and expected performance cover only part of that justification. Disease prevalence, intended use, and whether the device is used for screening or diagnosis also determine how many positive and negative cases the test set needs.
Validation does not end at market entry
For an AIMD, placement on the market is not the finish line, but rather the start of a new phase of validation. Real-world use can reveal rare failures, unanticipated user errors, and performance that slowly degrades. Regulators expect performance and risk to be monitored across the full life cycle of a deployed model.
Post-market validation should be continual, and two threats deserve particular attention. Data drift should be monitored because changes in the input population, hospital equipment, software versions, clinical practice, or disease patterns can quietly erode accuracy, even for a locked model. Additionally, bias may become more evident over time. Subgroup monitoring can reveal performance differences across a broader population than pre-market testing could cover.
FDA’s 2025 draft guidance recommends a performance monitoring plan, including a description of how updates, mitigations, and corrective actions will be deployed when performance changes. Plans to update the model, for example by retraining on new data to reduce bias, including the schedule and validation methods should be set out in a Predetermined Change Control Plan (PCCP), as described in FDA’s PCCP guidance.6 An authorized PCCP lets the manufacturer make the specified changes, validated according to the pre-agreed protocol, without a new submission. In the EU, the same activities belong in the post-market surveillance (PMS) plan and the post-market clinical or performance follow-up (PMCF/PMPF) plans, which are updated regularly in response to the clinical or performance evaluation process.
Planning for validation from the start
U.S., EU, and global regulatory frameworks ask a common question of every AIMD. Does the device give the right answer for the intended patients, in the hands of the intended users, and keep doing so after launch? That answer is determined long before a submission is drafted, and is supported by the clinical rationale for the inputs, the makeup of the training data, the independence of the test set, and the post-market monitoring plan.
Bringing clinical, data science, human factors, and regulatory expertise together early positions manufacturers to make those decisions deliberately. The result is evidence that holds up in regulatory review and gives clinicians a sound basis for trusting the device in practice.



