AI is here to say, but Dario Amodei’s admonition needs to be taken seriously.
The debate over artificial intelligence just took another major turn. Anthropic CEO Dario Amodei, one of the people at the center of the AI revolution and one of its most thoughtful leaders, has issued another major warning about where AI could be heading.
His message is not that we should abandon AI. Amodei has been outspoken about AI’s extraordinary potential to accelerate science, medicine, and economic growth. His concern is that capabilities may be advancing faster than our ability to understand and control the risks. In an essay, he said that, unless development slows and safeguards are put in place, rogue bots could take over the web in a few short months or bad actors could seize control of AI tools. His proposed reforms include having outside evalulators gain access to each froniter lab to ensure safety and ethics are paramount, creating collaborative industry standards on the pace of innovation, and ensuring global coordination of AI development.
He isn’t alone. Google DeepMind Chair Demis Hassabis has repeatedly discussed the need for serious international AI governance. Mustafa Suleyman, the DeepMind co-founder who now heads Microsoft AI, has written extensively about the challenge of “containing” increasingly powerful technology. And both Sam Altman, Open AI’s leader, and SpaceXAI’s Elon Musk quickly endorsed Amodei’s call. The Center for AI Safety similarly has raised concerns.
All these are not people standing outside the AI revolution throwing stones at it. While I do not trust all of them, the Amodei call and endorsements are consistent with past viewpoints and rising risks we have seen over the past many months. That is why I think their warnings deserve attention. AI has become a bullet train of advancement, with chilling tales of risks as uncontrolled bots work feverishly in a matrix to solve life’s greatest challenges. President Donald Trump quickly lashed out against the warnings, but I do think we have reached a breaking point on the issue. What Amodei and others are saying should not be dismissed lightly. The warnings are real. The way I think about it: if several reasoned AI thinkers are urging action based on what has become publicly known about AI’s downside, imagine what may be happening in the background!
Should we really slow AI down?
There is a legitimate argument for slowing development. Quite simply, we don’t fully understand what increasingly autonomous systems will be capable of doing. There are cybersecurity risks. There are concerns about AI helping bad actors develop biological weapons or other dangerous capabilities. While AI advances have helped Ukraine defend itself against Russian threats, AI has also helped Russia commit ever increasing war crimes. AI could also become an amazing equalizer for terrorist entities seeking to wreak havoc and destroy civilizations.
There are enormous questions about employment and what happens if AI substitutes for significant portions of human knowledge work, although so far we have not seen the displacement. Past innovations have led to job losses but also created other opportunities for people. But is AI so redefining that its impact is far broader than past discoveries?
And ultimately there is the bigger question: what happens if artificial intelligence reaches or exceeds human capabilities across a broad range of activities? Are the things of science fiction ridiculously fanciful or are we indeed on a road to a future where machines literally have supremacy over us and we lose control of human destiny? A few years ago – even months – I scoffed at the concept. I don’t know that I can now.
But there is the AI opportunity to transform.
I don’t dismiss any of this. But there is an equally powerful argument against slowing down – that involves what it can do for human destiny itself. AI’s potential could be seen throughout life but let me opine a bit on the area I know best. AI could help produce some of the greatest scientific and medical breakthroughs of our lifetime.
Drug discovery could be fundamentally changed. What has been years and sometime decades in the making for drug breakthroughs could soon become months. AI has the potential to accelerate drug discovery and genomics, fundamentally enhance diagnostics, conquer cancer and chronic disease, personalize medicine, and enrich countless other areas.
Consider what already is in play. The Food and Drug Administration’s (FDA) growing list of authorized AI-enabled medical devices includes applications across radiology, cardiovascular medicine, neurology and other specialties. The FDA says AI is being applied to early disease detection, diagnosis, prognosis, risk assessment, and the identification of new patterns in physiology and disease progression.
The geopolitical reality as well.
If America dramatically slows AI development and China doesn’t, we risk surrendering leadership to an autocratic regime in perhaps the most consequential technology of this century. China could have reasons to participate in international controls, including the Chinese government’s own interest in maintaining control over powerful technologies operating within its society. In short, the Chinese Communist Party fears being overthrown. But we certainly cannot assume every nation or bad actor will abide by the same rules.
So, I come down on the side of continuing to innovate, but with much stronger guardrails. More on this later in the blog.
AI could fundamentally change healthcare.
I am extremely bullish about AI in healthcare beyond drug discovery and scientific advancement. (In full disclosure, my former firm, now part of Medisolv, is actively developing AI solutions for health plans and providers.)
Healthcare has staggering amounts of data — claims, electronic medical records, pharmacy information, laboratory results, imaging, remote monitoring, wearables, patient-generated information and social determinants. We have spent billions creating all this data, but we have not been particularly good at using it. AI potentially changes that.
Imagine if we could combine years of medical history with current laboratory results, medications, vital signs, activity and nutrition information, behavioral health indicators, and claims. AI can potentially detect relationships across those data that no physician, health plan, or care manager could realistically identify manually.
As I alluded to above, that can translate into earlier identification of disease, risks, and intervention. A diabetic patient’s glucose control may be deteriorating. That same patient has heart disease and his or her weight and blood pressure may be moving in the wrong direction. Medication adherence may suddenly decline. A patient may repeatedly miss appointments. Behavioral-health indicators may begin changing.
Individually, those events might not appear to mean much. But bring them together from disparate data points, and they tell a concerning story. AI can quickly and efficiently put that holistic view together, with a plan to better that situation. AI can help us build dynamic representations of an individual’s health and continuously update risk as new information arrives. That could fundamentally change healthcare from one that is today very reactive – we act when sickness and disease is already evident – to one that increasingly predicts when someone is going to get sick and intervene to stop it.
AI could also radically change intervention and outreach as well as member experience.
I think this opportunity is underappreciated. Healthcare is enormously difficult to navigate. Members don’t understand their benefits. They don’t know which physician to see. They struggle with medical terminology. They don’t know why something requires prior authorization. They can’t decipher bills or explanation of benefits documents.
AI could become an extraordinarily sophisticated healthcare navigator. It can explain a diagnosis in plain English (and dozens of other languages), explain benefits and medications, identify appropriate providers, prepare patients for appointments and surgeries, and remind them about important screenings and prevention.
The potential is far-reaching. In my years at health plans, I learned quickly that we concentrated just on the most adverse to control their costs. My care management staff looked at the 3% to 5% (if we were lucky) that drove the costs. But any number of risk tools show the real costs to the system represented by those below that threshold. Medium adversity populations are already costly (sometimes collectively more than the most adverse) yet often are not intervened on due to staffing and budgetary constraints. We also know that ignoring prevention and wellness leads to disease burden down the road. For any number of reasons, we are now in an age where disease is striking younger generations, whether breast cancer, colorectal cancer, diabetes, or heart disease.
Health plans and providers, especially those at risk with plans, could only dream of creating a care plan for everyone that recognizes individual health risks. But AI-supported data analysis and agentic AI outreach now offer that opportunity. AI cannot only determine who needs a preventive screening, but also outreach to a member, explain the test, identify an appropriate provider, collect information, arrange the appointment, provide preparation instructions, and remind the member and follow up afterward. The potential to close every member’s care gap through personalized outreach becomes attainable at a fraction of the labor cost. Plans struggle every year with closing care gaps. Instead of repeatedly mailing or calling members, intelligent agents could personalize outreach based upon how individual members actually communicate, respond, and engage in care.
Millions of seniors get lost post-surgery due to lack of home supports and cognitive difficulties. Agentic AI offers the opportunity to continually track seniors after leaving the hospital or surgery center, ensure adequate home or other services, and identify risks of readmissions. It will be a boon for health plans, providers, and hospitals alike and help drive quality and reduce costs.
These AI-based approach use cases are endless — medication adherence, all post-discharge and transition follow-up, annual wellness visits, chronic-condition management, behavioral health engagement, and social-needs interventions. It is important to note that AI doesn’t necessarily replace a medical director, care manager, or provider. But it does allow each of them to track and engage with far more people and then direct more prescribed clinical attention to the people who need it the most.
We need AI because healthcare costs far too much to administer.
Healthcare administration is simply too expensive. Physicians spend enormous amounts of time documenting encounters, reviewing charts, obtaining authorizations, coding services, and responding to health plans. Health plans deploy enormous infrastructure to process claims, answer member questions, administer networks, and manage utilization. Government programs have their own massive administrative systems, whether administering the Exchanges or the fee-for-service claims program.
AI will automate much of this over time. Clinical notes can be summarized. Medical records can be searched. Coding can be assisted. Routine member questions can be answered. Claims can be processed more efficiently. Missing information can be identified automatically. It will not only save dollars but streamline and rationalize processes and engagement. At the same time, it will reduce human error and perhaps (more to come) be fairer and more unbiased.
If we are serious about healthcare affordability, administrative simplification has to be part of the solution. AI gives us an opportunity to attack administrative costs in ways that simply weren’t possible before.
The mixed truth about AI: a prior authorization use case.
But critics are right that left to its own devices, or the machinations of any number of special interests in healthcare who might manipulate AI algorithms, AI could be used for nefarious purposes as well. That is why caution and guardrails (see below) are key as AI unfolds rapidly in healthcare.
Take a prior authorization use case. Used correctly, AI could make prior authorization dramatically better. Imagine a physician orders an MRI. The AI then identifies the patient’s health plan and applicable coverage criteria. It searches the medical record, extracts the relevant diagnosis, previous treatment and clinical findings, identifies missing information and assembles the prior-authorization request. At the health plan, its prior auhorization system reviews it. A straightforward case could be approved almost immediately as is envisioned in new regulatory requirements. CMS estimates its broader interoperability policies could save approximately $15 billion over ten years, but to me that is not achievable without AI. That is exactly where technology should take us.
But AI can also be pointed in the opposite direction. An algorithm could be designed to aggressively identify cases for denial. It could apply coverage criteria without appreciating individual clinical circumstances. It could make predictions based upon historical utilization patterns that themselves contain biases. Some health plans have been accused of doing just this. Providers have been accused of aggressively coding prior authorizations and claims based on algorithms that predict whether plans will actually flag them.
In short, AI should help make the right coverage decision faster. It should not become a mechanism for denying care or claims or used to commit fraud or engage in over-utilization. The role of the clinician should not be undermined, even as that role will change over time.
Health equity and risk concerns.
Another criticism is the potential for bias in decision-making. For sure, that happens every day with humans in the current status quo. Done right, AI could make things fairer. Done wrong, bias continues or is even enhanced. Bias could be along ethnicity and race, other social determinants, gender, age, disability, and morbidity. Algorithms learn from data. Algorithms can both enhance these cohorts’ treatment or institutionalize discrimination against them in enrollment, clinical decision-making, and more. Left to its own devices and unsupervised, poor AI could make dramatically poor decisions. If you use AI, you know how wrong it can be. Often, AI lacks context and critical information, it is trained wrong, and it concludes too fast. It, at least today, lacks the human reasoning that is critical in healthcare.
AI does require oversight, especially in healthcare.
The Trump administration and some business interests (including until more recently the AI cabal itself) have taken a laissez-faire approach to AI regulation but seem to be understanding the risks of no regulation. The Trump administration’s AI policy continues to strongly emphasize American innovation and winning the international AI competition. As noted, Trump actually dismissed Amodei’s warning. However, the U.S.’ 2026 framework also recognizes areas requiring federal safeguards, while national-security initiatives now explicitly emphasize systems that are robust, controllable and accountable.
Some in the MAGA right and populist left seem to agree on either hyper regulation or turning down/off AI entirely. The Democratic and Republican middle seem to favor, as do I, the need for a balanced approach.
AI regulation should not be deemed a nefarious act in and of itself. The government regulates industry every day. It is a legitimate role of government. What would the world be like without occupational safety oversight, environmental protection, or transportation and railroad regulation (two modern age developments that transformed life). AI oversight should be no different. The judicious exercise of oversight by government is a necessity in free enterprise to protect the public, the economy, our nation, and the world at hand.
A short- and long-term, risk-based, guard-railed approach.
Government regulators will never understand cutting-edge AI as well as the people developing it. And traditional regulations move too slowly. But self-regulation isn’t sufficient either. AI companies have tremendous financial and competitive incentives to keep advancing. We therefore need both industry self-regulation and government oversight. My proposed framework would have several components:
- Short- and long-term risk assessments, constantly updated through swift and agile iterations rather than traditionally long regulatory cycles.
- Mandatory industry reporting of significant frontier capabilities, safety issues, and serious incidents to appropriate oversight bodies.
- Sector-specific assessments determining how AI should advance in healthcare, science, finance, defense, and other critical areas — including circumstances where deployment should proceed only with additional controls.
- Continuous balancing of risk and innovation, recognizing both the consequences of moving too quickly and the enormous costs of unnecessarily delaying beneficial innovation.
- International coordination, particularly around the relatively small category of AI capabilities that could create catastrophic consequences or outcomes, such as cybersecurity, biological weapons and other military use, as well as medical ethics and other societal risks.
AI, especially in healthcare, needs a risk-based approach. Both government and industries should be tasked with implementing the short- and long-term approach and incorporate risk-based guardrails in AI adoption. This would promote innovation and leverage the good of AI even as we assess and govern the potential negative effects.
The risk-based, guardrail approach I propose builds on Amodei’s suggestions and is most like the European Union’s AI Act, the most developed model in the world. The EU also uses a risk-based approach by categorizing AI systems into four levels: unacceptable risk (banned), high risk (strict requirements), limited risk (transparency obligations), and minimal risk (no obligations). I think adopting similar levels makes a great deal of sense.
Even if the EU standards are not exactly applied, regulatory intensity should increase with potential harm. Low-risk applications should have considerable freedom to innovate. Moderate-risk applications should have reasonable requirements around transparency, testing, and monitoring. High-risk applications — especially those affecting clinical decisions, coverage determinations, patient safety or access to healthcare — should require validation, auditability, human accountability and ongoing surveillance. Specific to healthcare, regulation has to consider privacy, security, bias, and misuse.
In the short term, the industry and government would collaboratively act to together to assess AI as we know it now and explicitly set guardrails on AI use – from relatively unfettered to flexibly governed. At the same time, long-term oversight would be applied to new emerging use cases that need to be assessed and eventually governed. As quickly as AI seemed to be advancing, long term could very well now be months not years.
A good short-term example: AI summarizing a patient’s medical record, collating, and submitting information is low risk and could be leveraged greatly to save dollars, better ensure accuracy, and promote interoperability. This may require few guardrails. But an autonomous AI system making treatment or coverage decisions is something else entirely. Here, regulation and guardrails would be far stricter and require stronger yet still flexible governance.
A long-term example: as AI advances, we are quickly arriving at the time where AI could begin rendering clinical decisions. That may be reasonable depending on the training of the model, its accuracy, and whether it is being overseen by an actual clinician on an ongoing basis. The risk-based strategy would continually assess these advancements and be ready to apply the all-important guardrails to this use case in the future. Consequential healthcare algorithms should be tested across populations before deployment and continuously monitored afterward. If outcomes diverge materially across groups, somebody needs to understand why.
The risk-based approach would also be very sensitive to AI evolution more broadly, especially model maturity, capability, and data availability. The risk-based approach needs to be sensitive to the fact that it will not be evaluating a relatively static product or environment. It must be sensitive to where AI is going, even if that to some degree is speculative right now. This is increasingly where federal healthcare policy is heading. FDA’s current approach differentiates among types of clinical decision-support software, and its newest work on generative-AI medical devices explicitly asks how risk assessment and post-market monitoring should operate.
The risk in my approach and conclusion.
Admittedly, there is one enormous weakness in my approach: innovation could get ahead of regulation. By the time we discover that a capability is dangerous, it may already have spread. Further, bad actors don’t have to comply with the regulatory scheme. They can operate inside democratic societies or beyond them.
Some technological developments likely cannot be put back into the bottle once they exist. That is Amodei’s strongest argument, and I take it seriously. But I still come down on the side of moving forward. The potential benefits of AI — particularly for science, medicine and healthcare — are simply too enormous not to pursue. Healthcare alone could be transformed. We could identify disease earlier. Understand complex relationships across enormous amounts of data. Personalize treatment. Engage patients continuously rather than episodically. Reduce administrative expense. Give physicians back time. Improve the member experience. And potentially improve both outcomes and affordability.
But innovation cannot mean AI at any cost. We need to use it aggressively when the risk is low and the benefits are clear. And we need to watch it closely where decisions affect people. We need to demand transparency and human accountability where AI affects access to healthcare. We need to regulate it most aggressively where getting the answer wrong can seriously harm someone.
In essence, I say proceed with AI given its thrilling potential, but build a regulatory framework that seeks to control for emerging risks and the anticipated misdeeds of bad actors. That isn’t stopping innovation. It is creating the guardrails that may ultimately allow us to innovate even faster, with greater confidence and consensus. And in healthcare, we desperately need both.
My proposal creates a balance. As I note, there are still risks. My bigger fear, though, is that a politicized Washington will not have the courage to think rationally about the issue and put in proper safeguards.
#healthcare #healthcarereform #ai #regulation
— Marc S. Ryan
