FDA Artificial Intelligence (AI) Guidance Highlights

Modified on March 5, 2026 to remove the “subscribe” option. This blog has been retired and replaced by the S.P.I.R.I.T. newsletter.

Good morning, good afternoon, and good evening IRBers, clinical research educators, and investigators from around the world!

I hope everyone had a great first of full week back to work! Now, I can officially say, back to the grind. I should be used to how busy the start of the semester is with outreach training efforts. I was also busy reviewing submissions. We typically see an increase in submissions around this time since it is the start of Spring semester.

Besides being busy with work, I was also working on the Show-and-Tell Series of blog posts. You can read about the series here:

Well, I wasn’t the only one busy this week. The FDA went on a guidance posting frenzy! Below is a list of guidance documents relevant to IRB and clinical research that were issued between Monday, January 6, 2025 – Friday, January 10, 2025:

• Study of Sex Differences in the Clinical Evaluation of Medical Products
• Considerations for Complying with 21 CFR 211.110
• Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products: Draft Guidance for Industry and Other Interested Parties
• Evaluation of Sex-Specific and Gender-Specific Data in Medical Device Clinical Studies: Draft Guidance for Industry and Food and Drug Administration Staff
• Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations: Draft Guidance for Industry and Food and Drug Administration Staff
• Developing Drugs for Optical Imaging
• Considerations for Including Tissue Biopsies in Clinical Trials
• Accelerated Approval and Considerations for Determining Whether a Confirmatory Trial is Underway

Though all these guidance documents should be reviewed, I plan to only deep dive into the following guidance documents:

• Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products: Draft Guidance for Industry and Other Interested Parties
• Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations: Draft Guidance for Industry and Food and Drug Administration Staff

You may be wondering why I’m solely focusing on these two guidance documents.

I’ll tell you why I’m into AI!

• AI can optimize productivity. I can’t tell you how many times this has saved me. For people like me who have trouble reading long documents, AI is great for summarizing key concepts. Of course, I will read all documents in their entirety. However, it’s nice to have a general idea of what I plan to read. This way, if there are any concepts or terms I’m unfamiliar with, I can simply look them up. Then, when I’m reading the document in full, I won’t have to waste time looking up terms and concepts.
• AI can help you be creative. I can’t wait to share a post related to this! Before I think about automating a task, I ask ChatGPT if it’s possible. Then, with my skills I’ve acquired over the years, I can attempt to act on my efficiency idea. I also love the DALL-E aspect. This is great for visual folks. I especially love to have DALL-E create flowcharts or diagrams. This helps me understand complex topics (such as the ethical codes in human subjects research).
• AI can be used in any facet. Whether you’re a writer, an artist, in IT, or even compliance, AI can be helpful anywhere! I will say that it’s important to make your audience aware when AI was used. I always love to give credit where credit is due. I feel that AI can even make the least creative person…a creator.

Though I sound pro-AI, I do see there are downsides.

• Privacy and confidentiality are major concerns. This likely goes without saying, but I will say it anyway. Compliance personnel such as myself likely know you shouldn’t place any personal information into ChatGPT. Well, others may not know this. What about ways to withdraw your data? Can you do that in ChatGPT? I know there is a way to export data that was entered into the tool via your ChatGPT settings. But can ChatGPT unlearn data that has been withdrawn? I’m not sure, but I hope to learn more about this.
• Machine learning bias is real. For those who may not be familiar with machine learning, this is a subset of AI. There isn’t actual programming (e.g., with Python). It learns from datasets and makes inferences based on patterns. This is why it is called “machine learning” because the tool learns over time. I feel this is GREAT for a highly specific function (such as a customer service chatbot). But what about when AI is being used for biomarker analysis or drug development? How can we ensure that we are applying the Belmont principle of Justice (subjects must be fairly selected)? How do we ensure we are fairly selecting datasets that are representative of the population of interest?
• Compliance professionals are in an arms race in how to regulate the rapid use of AI in research. Again, this likely goes without saying it. You can probably start calling me “Captain Obvious” now. Even as I learn about AI, it is hard to keep track of everything that’s going on. I follow federal agencies for guidance documents or if strategic plans are released discussing the ethical uses of AI. To me, the problem seems to be that everyone is developing their own guidance documents and best practices. It seems to me that something like this should have a best practice standard that agencies can adopt. I plan to learn about the EU AI Act in further detail as this seems like a great start. I recently completely GDPR trainings and felt this regulation really covered everything. We desperately need something like this in the United States.

In this post, I plan to highlight key takeaways from these lengthy AI guidance documents (90 pages total).

Then, I plan to analyze the documents even further as the request for comment is due Monday, April 7, 2025.

If you are interested in leaving a public comment with me, please email me at tmohseni@renovationinirbeducation.com.

Let’s make our voice count TOGETHER!

---

Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products

Per the FDA, this guidance provides recommendations to sponsors and other interested parties on the use of AI to produce informations or data intended to support regulatory decision-making regarding the safety, effectiveness, or quality of drugs. Though I am not an expert in drug development, I will say that I have intermediate understanding of AI. Let’s see what the FDA has to say.

The guidance provides a risk-based credibility assessment framework that may be used for establishing and evaluating the credibility of an AI model for a particular context of use (COU).

The COU defines the specific role and scope of the AI model to address a specific question. As a former auditor, I also appreciate that the FDA has defined the word “should”. The definition of “should” means that the FDA recommends actions within this guidance, but they aren’t required. I remember carefully reviewing policies for words like “should”, “shall”, or “must”. It’s important for institutions to define this as well. This way, when folks are reviewing their institution’s policy or guidance, they know what is required versus what is recommended.

A Risk-Based Credibility Assessment Framework

This is a 7-step process:

• Step 1: Define the question of interest that will be addressed by the AI model.
• Step 2: Define the COU for the AI model.
• Step 3: Assess the AI model risk.
• Step 4: Develop a plan to establish credibility of AI model output within the COU.
• Step 5: Execute the plan.
• Step 6: Document the results of the credibility assessment plan and discuss deviations from the plan.
• Step 7: Determine the adequacy of the AI model for the COU.

Okay, so we know the steps. What do we do for each of these steps?

Step 1 should describe the specific question, decision, or concern being addressed by the AI model. For step 2, the description of the COU should describe in detail what will be modeled and how model outputs will be used. It should also be notated on whether other information will be used in conjunction with the AI model’s output to answer the question of interest determined in step 1. Examples of other information include animal studies and/or clinical human research studies. In step 3, model risk is assessed by two factors: model influence and decision consequence. Model influence, like it sounds, compares data derived from the AI model to other evidence used to inform the question of interest in step 1. Decision consequence is the significance of an adverse outcome resulting from an incorrect decision concerning the question of interest in step 1. To appropriately assess these components of model risk, subject-matter expertise is strongly advised.

Step 4 describes what information should be in your credibility assessment plan. Below is a summarized list of information that should be considered:

• Describe the datasets used for training and tuning the AI model and which model development activities were performed using these datasets
• Describe how the development data have been or will be collected, processed, annotated, stored, controlled, and used for training and tuning the AI model
• Describe how the development data is fit for the COU
• Describe whether the development data are centralized
• Describe how the AI model was trained
• Specify if a pre-trained model was used
• Describe the use of ensemble methods
• Explain any calibration of the AI model
• Describe the quality assurance and control procedures of computer softwares and how version changes were tracked (as well as code verification)
• Describe the applicability of the test data to the COU to minimize data drift
• Describe the agreement between the model prediction and the observed data
• Provide rationale for the chosen model evaluation method
• Describe any model limitations and biases

For step 5, discussing the credibility assessment plan with the FDA prior to execution may be helpful. The last section of the document describes early engagement options s with the FDA. Step 6 should involved documenting the results and deviations from steps 1-4. Once this is complete, you can proceed to step 7. Step 7 is where you determine if the AI model is appropriate for the COU. Finally, the document concludes with life cycle maintenance of the credibility of the AI model output in certain COUs. This can be referred to as the management of changes to an AI model (whether incidentally or deliberately).

Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendation

Per the FDA:

This draft guidance, when finalized, will represent the current thinking of the FDA on this topic.

Though this document will represent FDA’s thoughts, I appreciate FDA’s flexibility in approaches towards these recommendations. So long as the applicable statutes and regulations are met and it has been discussed with the FDA, you can use an alternative approach. The guidance provides recommendations on the contents of marketing submissions for devices that include AI-enabled device software functions including documentation and information that will support FDA’s review. Similar to the previous guidance, the FDA defines the word “should” as “suggested” or “recommended”. Now…let’s get into this document!

The FDA promotes a total product life cycle (TPLC) approach to the oversight of medical devices. You can read more about TPLC here: Total Product Life Cycle for Medical Devices. They also discussed the recent efforts made such as the 10 tenets of Good Machine Learning Practice (GMLP). The document further defines terminology used by the FDA versus the general AI community. For example, using the term “validation” to represent “training” or “tuning” should be avoided in medical device marketing submissions. Instead, the word “development” should be used. The FDA Digital Health and Artificial Intelligence Glossary – Educational Resource provides a compilation of commonly used AI Terms and how the FDA defines them.

The next few sections within this guidance are what the FDA recommends including in marketing submissions. Each section provides a reason as to why it must be included, what must be included, and where to include it. Below is a general outline of what is recommended for submission:

General Outline for Marketing Submissions

• Device description
  • A statement that AI is used in the device
  • A description of device inputs and outputs
  • An explanation of how AI is used to achieve the device’s intended use
  • A description of the intended users, their characteristics, and the level and type of training they are expected to have and/or receive
  • A description of its intended use environment(s)
  • A description of the intended workflow for the use of the device
  • A description of installation and maintenance procedures
  • A description of any calibration and/or configuration procedures
  • If the device can be configured by a user, then the submission should include information about:
    • All configurable elements of the AI-enabled device
    • How these elements and their settings can be configured
    • The potential impact of the configurable elements on user decision-making
  • If a device contains multiple connected applications with separate interfaces, then the device description should address all these applications
• User Interface
  • A graphical representation of the device and its user interface
  • A written description of the device user interface
  • An overview of the operational sequence of the device and the user’s expected interactions with the user interface
  • Examples of the output format
  • A demonstration of the device
• Labeling
  • The following should be included at the age-appropriate reading level for the intended user:
    • Inclusion of AI
    • Model input
    • Model output
    • Automation
    • Model architecture
    • Model development data
    • Performance data
    • Device performance metrics
    • Performance monitroing
    • Limitations
    • Installation and use
    • Customization
    • Metrics and visualization
    • Patient and caregiver information
• Risk assessment
  • Risk management file
• Data management for both training and testing data
  • Data collection
  • Data processing and cleaning
  • Reference standard
  • Data annotation
  • Data storage
  • Management and independence of data
  • Representativeness
• Model description and development
• Performance validation
• Device performance monitoring
• Cybersecurity
  • Cybersecurity risk management report
  • How cybersecurity testing addresses the risks in the report
  • A security use case view(s) that covers the AI-enabled considerations for the Debi e
  • A description of controls
• Publication submission summary
  • A statement that AI is used in the device
  • An explanation of how AI is used as part of the device’s intended use
  • A description of the class of model and its limitations
  • A description of development and validation datasets
  • A description of the statistical confidence level of predictions
  • A description of how the model will be updated and maintained over time

---

I hope you found this content enlightening and useful! I strive to provide my readers “food for thought”. What did you think of my interpretation of the guidance documents? Please leave a comment below and let’s get this discussion started!