The US Food and Drug Administration (FDA or the Agency) recently issued its draft guidance titled “Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions.” The draft guidance follows the passage of the Food and Drug Omnibus Reform Act of 2022 (FDORA), which explicitly authorized the Agency to approve or clear Predetermined Change Control Plans (PCCPs) for devices that require premarket approval (PMA) or premarket notification (510(k)).1 While the FDORA provision covers all devices subject to a PMA or 510(k), the draft guidance specifically applies to machine learning-enabled device software functions (ML-DSFs) that a manufacturer intends to modify over time.
PCCPs are intended to contain planned modifications that, if not otherwise included in a PCCP, would require a new marketing submission under applicable FDA regulations.2 If a PCCP is authorized, manufacturers of ML-DSFs are not required to submit a new marketing submission before implementing modifications that are delineated and made in accordance with the authorized PCCP.
Establishing and Modifying A PCCP
A PCCP must be reviewed by FDA and established as part of a marketing authorization prior to a manufacturer’s implementing any modifications under the PCCP. Accordingly, a PCCP must be included in and established through a marketing submission (i.e., a 510(k), PMA, or de novo request for classification).
Manufacturers who wish to establish a new PCCP for a previously authorized device must include in a marketing submission a proposed PCCP along with other appropriate marketing submission requirements for the device. Importantly, an authorized PCCP is applicable only to the version of the authorized device with which it was established. Accordingly, if the authorized device is significantly modified in a manner that requires a new marketing submission, for the original authorized PCCP to apply to the significantly modified device, it must be reviewed and established as part of the new marketing submission for the significantly modified device.
Further, FDA expects that a modified PCCP will need to be reviewed and established as part of a new marketing submission. As such, a manufacturer that would like to modify a PCCP for a previously authorized device with a PCCP should submit a new marketing submission that includes the proposed PCCP along with other appropriate marketing submission requirements for the device.
Content of a PCCP
A PCCP should contain the following components: a Description of Modifications, a Modification Protocol, and an Impact Assessment.
Description of Modifications
The Description of Modifications section in a PCCP identifies the “specific, planned modifications to the ML-DSF that the manufacturer intends to implement” and should include a detailed description of “changes to the device characteristics and performance resulting from implementation of the modifications.” FDA recommends that “a PCCP include only a limited number of modifications that are specific, and that can be verified and validated.”
The types of modifications that may be acceptable to include within a PCCP include:
- Modifications related to quantitative measures of ML-DSF performance specifications, such as improvements to analytical and clinical performance resulting from retraining the ML model based on new data within the intended use population from the same type and range of input signal
- Modifications related to device inputs to the ML-DSF, such as expansions to the algorithm to include new sources of the same signal type (e.g., different makes, models, or versions of a data acquisition system) or limited modifications related to new types of inputs
- Limited modifications related to the device’s use and performance (e.g., for use within a specific subpopulation that was not previously available)
Modifications included in a PCCP must maintain the device within the device’s intended use, and FDA further indicates in the draft guidance that “[a]t this time, FDA expects that modifications included in a PCCP should also maintain the device within the device’s indications for use” as FDA believes that “modifications to the indications for use would not allow the device to remain safe and effective.”
Modification Protocol
The Modification Protocol should describe the methods that will be followed when developing, validating, and implementing any modifications (as well as predefined acceptance criteria) to ensure that the device remains safe and effective. The Modification Protocol includes components that outline a manufacturer’s data management practices, retraining practices, performance evaluation tools, and update procedures (including communication and transparency to users and real-world monitoring plans) for each modification in the PCCP.
The PCCP should clearly delineate which parts of the Modification Protocol are applicable to each modification within the Description of Modifications, which for a PCCP with multiple modifications may be accomplished through a traceability table.
Impact Assessment
Last, the PCCP should include an Impact Assessment, or the documentation of the assessment of the benefits and risks of implementing a PCCP for an ML-DSF and the mitigation of such risks.
The Impact Assessment should:
- Compare the version of the device with each modification implemented to the version of the device without any modifications implemented
- Discuss the benefits and risks (including risks of social harm) of each individual modification
- Discuss how the activities proposed within the Modification Protocol continue to reasonably ensure the safety and effectiveness of the device
- Address how the implementation of one modification impacts the implementation of another
- Address the collective impact of implementing all modifications to not only the ML-DSF but the device as a whole
FDA recognizes that for some devices, it may be appropriate to include the content of the Impact Assessment within the Modification Protocol rather than as a separate section in the PCCP.
Additional Takeaways for Manufacturers of ML-DSFs
Significantly, as underscored by FDA during its April 13, 2023, webinar, the draft guidance is not intended to provide a complete description of what may be necessary to include in a marketing submission for an ML-DSF. Instead, the Agency states that the FDA review division determines “whether the scope of the modifications is appropriate for inclusion in a PCCP and what evidence and information are required to support proposed modifications in a marketing submission.”
FDA encourages manufacturers to engage with FDA early to discuss proposed PCCPs and to leverage the Q-Submission process to obtain feedback from the Agency.
Interested stakeholders may submit comments on the draft guidance to the docket (FDA-2022-D-2628) by July 3, 2023.
Contact Goodwin FDA team members for any questions or if you would like to consider submitting comments to FDA on the PCCP draft guidance.
[1]Consolidated Appropriations Act of 2023, Pub. L. No. 117-328, § 3308 (2022).
[2]21 C.F.R. § 807.81(a)(3) or 21 C.F.R. § 814.39(a).
