Health Care

On January 13, 2025, California Attorney General Rob Bonta (the California AG) issued two Legal Advisories regarding the utilization of artificial intelligence (AI). The first Legal Advisory provides guidance to consumers and entities developing, selling, and using AI describing their rights and obligations under California law. The second Legal Advisory, entitled Application of Existing California Law to Artificial Intelligence in Healthcare (the Health Care Legal Advisory), is expressly directed at health care providers, insurers, vendors, investors, and other health care entities who develop, sell, and use AI and other automated decision-making tools.

Governor Newsom signed over 1300 bills and vetoed almost 190 bills in 2024.  Consistent with prior years, in 2024, California enacted multiple laws significantly impacting the health care industry.  From artificial intelligence (AI) to providers to health plans, these new laws require proactive efforts to ensure on-going compliance.

We highlight below the most important of the heath care-focused bills passed during this year.  Unless otherwise noted, these new laws are effective January 1, 2025.

The Department of Health and Human Services (HHS) Office for Human Research Protections (OHRP) recently held its 2024 Exploratory Workshop titled “The Evolving Landscape of Human Research with AI – Putting Ethics to Practice” (the Workshop). Although the individual presentations and panel discussions throughout the Workshop covered a range of topics and raised a number of interesting questions and hypotheticals, the panelists did not draw any specific conclusions or reach any kind of consensus about next steps to address the critical issues. Even so, the panelists provided some crucial insights that companies and regulators must grapple with in the context of expanding use cases for AI in human research and creating rules governing such uses.

As the first state law to regulate the results of Artificial Intelligence System (AI System) use, Colorado’s SB24-205, “Concerning Consumer Protections in Interactions with Artificial Intelligence Systems” (the Act), has generated plenty of cross-industry interest, for good reason. In some ways similar to the risk-based approach taken by the European Union (EU) in the EU AI Act, the Act aims to regulate developers and deployers of AI Systems, which are defined by the Act as “any machine-based system that, for any explicit or implicit objective, infers from the inputs the system receives how to generate outputs, including content, decisions, predictions, or recommendations, that can influence physical or virtual environments.”

Preventing discrimination and bias in connection with the use of artificial intelligence (AI) in health care is among the principal current focuses of U.S. Department of Health and Human Services (HHS) and was among the health care directives in the recent Biden Administration Executive Order on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (Executive Order). Consistent with these priorities, on April 26, 2024, the HHS Office for Civil Rights (OCR) and the Centers for Medicare & Medicaid Services (CMS) released an unpublished version of a new final rule under Section 1557 of the Affordable Care Act (ACA) that aims to broadly address inequity across health care but also requires certain actions of entities covered under Section 1557 around their use of AI in clinical decision-making (Final Rule).

We have been writing about software as a medical device (SaMD) for years, tracking the Food and Drug Administration's (FDA) efforts to keep up with the fast-paced development of digital technology, such as launching the Digital Health Center of Excellence, implementing predetermined change control plans, and issuing various digital health guidances on device software functions, clinical decision support software, cybersecurity, and other topics. In anticipation of FDA’s Artificial Intelligence /Machine Learning (AI/ML) Medical Devices Workshop in October 2021, we posted a brief history of the agency’s regulatory oversight of software through the traditional medical device regulatory framework established in the 1970s, in which we highlighted the numerous challenges associated with such an approach. But now, with the rise of artificial intelligence and machine learning and the proliferation of AI/ML-enabled software throughout the health care industry, FDA is facing enormous challenges using an outdated, procrustean regulatory framework to maintain standards of safety and quality for such software devices. It is becoming increasingly clear that innovation in the AI/ML and digital health technology space is advancing rapidly, as FDA Commissioner Rob Califf has emphasized in many recent public appearances, and that the traditional device framework is quickly becoming unworkable for such technologies.

Keeping pace with the ever-evolving landscape of health care enforcement can prove challenging. In the latest EnforceMintz newsletter, Mintz’s Health Care Enforcement Defense practice group highlights emerging enforcement trends and developments in FCA investigations, and provides predictions for 2024, offering helpful guidance through the complexities of health care enforcement.

Health care companies using algorithms and AI applications face increased compliance risks. Previous technology-related enforcement suggests how relators and enforcement agencies might use AI to detect potential fraud and develop allegations based on how the technology is being used.

This Q&A with John Koss, Managing Director of Mintz’s E-Data Consulting Group, delves into the work of the group along with the benefits, challenges, and emerging issues related to using AI in e-discovery.

Since May 2023, federal legislators have introduced more than 50 bills focused on issues related to AI technologies, with patient privacy, the role of clinicians, and health equity emerging as areas of focus. In addition, the Biden administration released an Executive Order focused on promoting AI safety in October 2023.

As we reflect on the flurry of activity in the health care data privacy and security space in 2023 and look ahead to what will continue to be a busy 2024, we are seeing the early stages of federal agency movement to align the regulatory environment with modern health care delivery, cutting-edge technologies, and innovative data-sharing techniques. Some of this work has been done in the form of federal agency guidance in which health care organizations will be looking for additional updates and there are also a handful of pending U.S. Department of Health and Human Services (HHS) Office for Civil Rights (OCR) proposals that call for substantial changes to the HIPAA Privacy Rule.

The Department of Health and Human Services (HHS) was tasked with formalizing and coordinating efforts to regulate artificial intelligence (AI) in health care under the November 2023 Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (AI EO) and has already begun its regulation of AI within certain certified health IT.  HHS and Office of the National Coordinator for Health Information Technology (ONC) recently published the Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1) Final Rule.

On October 30, 2023, the Biden Administration released and signed an Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (Executive Order) that articulates White House priorities and policies related to the use and development of artificial intelligence (AI) across different sectors, including health care.

The capabilities and sophistication of digital health technologies are constantly expanding, especially with the integration of artificial intelligence and machine learning. Many companies, both existing and start-up, are developing software aimed at health care systems, hospitals, physicians, and lay-users for various levels of use, from assisting diagnoses in a clinical environment to tracking exercise and general health indicators in the home. In particular, a growing number of companies is focusing on the development of digital therapeutics - a class of software devices that deliver and monitor medical interventions for the purpose of treating, managing, or preventing certain diseases or conditions - for use in a patient’s home. With all of the new players jumping into the digital health technology game, we thought it might be useful to provide a brief primer on certain factors to consider when evaluating the potential regulatory requirements for such products.

The information age in which we live is reaching a new milestone with the development and ready access to conversational artificial intelligence based on advanced transformer algorithms, or AI chatbots, including their upcoming integration into multiple Internet search engines. This development creates exciting opportunities and potentially terrifying risks in the health care space. Inevitably, people will ask AI chatbot-enabled search engines for information on diseases, conditions, medicines, or medical devices and use the response in some way to make certain medical decisions. But what happens when the AI chatbot’s response is inaccurate or even provides advice that may lead to harm if the user follows it? Can AI chatbots be regulated by the U.S. Food and Drug Administration (FDA)? What are the liability implications if a user is harmed? We provide some initial thoughts on such legal issues in this post.

In the weeks leading up to FDA’s October 14, 2021 Transparency of AI/ML Enabled Medical Devices Workshop (Workshop) we took a brief look at the history of FDA’s regulation of medical device software and the agency’s more recent efforts in regulating digital health. In this post, we will provide an overview of the topics discussed at the Workshop and our impressions of the agency’s likely next steps.

In our last post, we took a brief look back through history at FDA’s approach to regulating medical device software and found that there is little distinction from the agency’s approach to hardware devices. Recently, however, FDA has announced several digital health initiatives aimed at improving the agency’s resources and policies governing software and data systems (including its own internal data systems) and changing the way the agency handles pre-market reviews of and compliance activities for software as a medical device (SaMD) and SaMD manufacturers. In this post, we will review FDA’s digital health improvement highlights from the past few years and take a quick look at the agenda for the transparency of AI/ML-enabled medical devices workshop scheduled for October 14, 2021.

In anticipation of FDA’s virtual public workshop on transparency of artificial intelligence/machine learning (AI/ML)-enabled medical devices scheduled for October 14, 2021, we will be posting a series detailing the history behind FDA’s regulation of software and then reporting our impressions of FDA’s presentations and statements from various attending stakeholders following meeting. In this part, we briefly summarize FDA’s traditional approach to regulating software and how software development quickly revealed the limitations of the original regulatory framework established in the 1976 Medical Device Amendments to the Federal Food, Drug, and Cosmetic Act (FD&C Act).

Artificial Intelligence is a growing part of our day-to-day life. And AI promises to improve our health care system. ML Strategies Vice President Christian Tomatsu Fjeld recently sat down with other experts for a panel discussion hosted by the San Francisco Business Times to discuss AI and some business and policy considerations across multiple industries. This viewpoint considers some of the impacts on health care specifically, and links out to the panel's discussion.

Regular readers of this blog know that we’re closely following the FDA’s proposed regulatory framework for software as a medical device (SaMD), known as precertification—Pre-Cert for short. Generally, Pre-Cert involves a premarket evaluation of a software developer’s culture of quality and organizational excellence and continual, real-time postmarket analyses to assure software meets the statutory standard of reasonable assurance of safety and effectiveness.