FDA seeks public input on competency-based premarket tests for medical generative AI

The Food and Drug Administration’s medical device center is considering a competency-based approach to premarket evaluation of generative AI-enabled devices, drawing loosely on how human clinicians are evaluated and credentialed but adapting the method for device regulation, according to a discussion draft published Tuesday. The agency proposes testing the final, user-facing generative AI product rather than foundational models or subcomponents.

Under the draft framework, the test would assess whether the product “demonstrates the clinical knowledge, analytic capabilities, safety behavior, communication, and generalizability to support reasonable assurance of safety and effectiveness of the device for its intended use.” The agency suggests using scoring rubrics and expert adjudication, with the rigor of the tests varying by the product’s risk profile. The risk profile could consider whether the product steers users toward actions or provides information, and the consequences of relying on faulty outputs.

The FDA acknowledges that testing alone may not suffice and proposes collecting real-world evidence to confirm that a generative AI product operates as intended. The agency is also soliciting public feedback on the draft through Oct. 19, emphasizing that the document is not a policy statement or a preview of future regulations.

FDA officials framed the effort as part of a broader push to regulate generative AI in health care, noting that the technology’s capacity to evolve and produce variable outputs has made oversight difficult. The agency has held advisory committee meetings in 2024 and 2025 and recently convened health AI companies to inform its approach. The discussion draft follows multiple requests for information about AI from the agency and signals a concrete step toward formal guidance.