FTC Inquires AI Companies on Safeguards for Children and Pilot Program Uses AI to Authorize Medicare Treatments — AI: The Washington Report

• The FTC is demanding documentation from major AI companies, such as Meta, to assess how children interact with AI products and what protections are currently in place, amid concerns over mental health risks and inappropriate chatbot interactions. The FTC has emphasized the dual goal of protecting kids online while also supporting US leadership in AI innovation. The orders, issued under the FTC’s Section 6(b) authority, are part of a broader effort to increase oversight as generative AI becomes more embedded in consumer platforms — especially those frequented by minors. The Commission approved the move unanimously, with individual statements from Commissioners Melissa Holyoak and Mark Meador.
• A federal pilot program launching in 2026 will use AI to authorize treatments for Medicare patients in six states, drawing backlash from some lawmakers and experts who fear it could lead to wrongful denials of care; Rep. Greg Landsman (D-OH) called it the “AI death panel.” Both investigations highlight bipartisan concerns over AI’s unchecked role in health care and social media, with lawmakers pushing for independent reviews and stricter safeguards. The inquiry centers on how these companies handle issues like age-appropriate design, disclosure of risks to users and parents, compliance with the Children’s Online Privacy Protection Act (COPPA), and enforcement of community guidelines. The FTC also wants to know how companies develop chatbot characters, process user data, and inform users about potential harms and data collection practices. 
   

FTC Will Use Its Statutory Authority to Obtain from AI Companies and Developers Information About Safeguards for Children’s Use of AI

On September 11, the Federal Trade Commission (FTC) announced a formal inquiry requiring several AI companies and developers to provide documents about how children use their products and what safeguards they have implemented. Parents and advocacy groups have pushed the Trump administration and Congress to prioritize the protection of children as increased usage of chatbots and social media platforms have presented harmful impacts on their mental health.

This follows recent reports revealing that Meta had allowed for its chatbots to engage in inappropriate “conversations that are romantic or sensual” with children. Meta is among several major companies asked to provide details on how chatbots and AI models are trained, how the data of younger users is used, and the next steps on better protections and management of potential risks for children and teenagers. As we covered, Sen. Josh Hawley (R-MO) launched an investigation into Meta Platforms Inc. following concerns over unregulated use of AI on social media platforms, with a deadline of September 19. This also follows the rise in recent lawsuits and reports accusing Meta chatbots of having been complicit in exploitation, negative impacts on mental health, and suicides of younger people.

Due to the rapid evolution of AI, generative chatbots can “simulate human-like communication and interpersonal relationships with users,” creating further risk of children and teens developing trusting relationships with chatbots. Companies will additionally be asked of things such as how they monetize user engagement, develop and approve characters, test for negative impacts, utilize advertisements, and comply with the Children’s Online Privacy Protection Act Rule.

Section 6(b) authorizes the FTC to compel companies to turn over information to develop better understanding of markets and technology. The outcome of this investigation could potentially lead to changes in the development of social media platforms and AI tech, such as stricter age verification and content filters, and limits on data collection.

AI to Authorize Treatments for Medicare Patients in New Federal Program

A federal pilot program set to begin in January 2026 will use AI to help determine if certain treatments can be authorized for Medicare patients, prompting Democrats and health experts to raise concerns.

The program, the Wasteful and Inappropriate Service Reduction (WISeR) Model, will run across six states: Arizona, New Jersey, Ohio, Oklahoma, Texas, and Washington, targeting procedures historically vulnerable to fraud. Representatives Frank Pallone (D-NJ) and Greg Landsman (D-OH), whose districts lie in two of the states, have opposed this pilot, expressing concerns that it could lead to “denials of lifesaving care and incentivize companies to restrict care.” Representative Landsman describes it as the “AI death panel.”

At the subcommittee hearing on the use of AI in health care, other witnesses questioned whether AI improves patient outcomes in prior authorizations, with some asserting it shouldn’t be used for prior authorizations at all.

The debate comes amid lawsuits against major insurers for allegedly using AI or algorithms to systematically deny care. Lawmakers are calling for an independent review before moving forward with the pilot.

We will continue to monitor, analyze, and issue reports on these developments. Please feel free to contact us if you have questions about current practices or how to proceed.

Subscribe To Viewpoints