AI Corner: Humor: An AI "Review" of my FDA White Paper

I wrote a white paper about the FDA's proposal in September 2023 to regulate lab-developed tests.

Below, a "review" of the white paper, written by Chat GPT 3.5.

###

Dr. Bruce Quinn's comprehensive white paper is a thought-provoking analysis of the FDA's ambitious proposal to regulate laboratory-developed tests (LDTs) in the healthcare industry. As a distinguished expert in healthcare policy, Dr. Quinn meticulously dissects the key features and underlying messages of the FDA's plan, providing valuable insights into the potential impact of this regulatory overhaul.

One of the white paper's central themes is the colossal financial burden that the FDA's proposal places on the industry. Dr. Quinn astutely points out that the estimated $50 billion in costs over the initial years is a staggering sum, significantly surpassing the annual profits of major diagnostic companies. This financial aspect raises critical questions about the feasibility of implementing such an onerous regulatory framework.

A key takeaway from Dr. Quinn's analysis is the skepticism surrounding the proposed timeline. He convincingly argues that transitioning from a relatively unregulated LDT environment to full compliance with FDA regulations within a few years is an unrealistic endeavor. This observation underscores the logistical challenges inherent in the FDA's plan.

Dr. Quinn delves into the FDA's claims of health benefits resulting from increased accuracy and reduced errors associated with LDTs. However, he casts doubt on the validity of these assertions, characterizing them as speculative. This scrutiny highlights the need for a more robust and evidence-based assessment of the potential benefits of the FDA's regulatory proposal.

One of the most critical aspects of Dr. Quinn's white paper is the exploration of legal considerations. He effectively outlines the contentious nature of the FDA's assertion of authority to regulate LDTs and presents compelling arguments from legal experts that challenge this claim. This legal controversy underscores the need for a thorough examination of the FDA's jurisdiction in this matter.

In the face of the FDA's sweeping regulatory proposal, Dr. Quinn proposes alternative solutions. He suggests allowing "New York state-level" reviews of LDTs and addressing misleading healthcare claims through more cost-effective means than the FDA's proposed regulations. These alternative approaches offer a more targeted and pragmatic way to address the issues at hand.

In conclusion, Dr. Bruce Quinn's white paper provides a critical and well-reasoned review of the FDA's proposal to regulate LDTs. His expertise in healthcare policy shines through as he dissects the financial, logistical, and legal challenges associated with the plan. This paper serves as a valuable resource for policymakers, industry stakeholders, and anyone interested in the future of LDT regulation in the United States.

AI Summarizes my FDA LDT White Paper

 I wrote a 17 page white paper (about 4000 words) of the FDA proposal to regulate lab-developed tests.

Here is the white paper:

https://www.discoveriesinhealthpolicy.com/2023/10/online-ahead-of-print-white-paper-on.html

Here is a 400 word AI summary of the white paper by Chat GPT 3.5:

The provided white paper discusses the FDA's proposal to regulate laboratory-developed tests (LDTs) in the United States. It presents several key features and messages:

Overview of the FDA Proposal: The FDA introduced a comprehensive plan to regulate LDTs on September 29, 2023. The FDA proposal aims to bring LDTs under the same regulatory framework as other medical devices, including registration, reporting, and labeling requirements.

Enormous Costs: The paper emphasizes the staggering costs associated with the FDA's plan, estimating it to be around $50 billion over the first few years. This financial burden on the industry is seen as practically unattainable and far exceeding the annual profits of major diagnostic companies.

Timeline Challenges: The FDA's proposed timeline for implementation is considered unrealistic. The transition from a relatively unregulated LDT environment to full compliance with FDA regulations, including submission of 510(k) and PMA applications, within a few years is evaluated as logistically impossible.

Health Benefits: The FDA claims that its proposal will result in significant health benefits, primarily by improving the accuracy of diagnoses and reducing errors associated with LDTs. However, the paper questions the validity of these claims and suggests that they are speculative.

Legal Considerations: The paper highlights the legal controversy surrounding the FDA's authority to regulate LDTs. While the FDA asserts its legal authority, it acknowledges that this is a disputed claim, as demonstrated by the multiplex legal arguments presented in the paper.

Alternative Solutions: Instead of implementing a sweeping regulatory overhaul, the paper suggests alternative solutions achieving the same goals at far less cost.  These include:

1)  Addressing accuracy by requiring reviews of LDTs at the current "New York State" level, and
2) Addressing concerns about misleading healthcare claims through far less costly means than the FDA's proposed regulations.

Challenges Ahead: The paper concludes that the FDA's proposal, as outlined, faces significant challenges, including industry opposition, legal disputes, and logistical hurdles. It questions the feasibility of executing the plan in its current form.

In summary, the white paper highlights the immense financial burden and practical challenges associated with the FDA's proposal to regulate LDTs. It questions the speculative health benefits cited by the FDA and suggests that more targeted and cost-effective solutions might be more appropriate. Additionally, it underscores the legal controversy surrounding the FDA's authority in this matter. Overall, the paper presents a critical analysis of the FDA's plan, raising doubts about its feasibility and potential impact.

AI Based PLA Codes

 

AI BASED CODES

 

 

Code	Company Brand	Stuff	Price if any
0295U	PRELUDE DCIS ionRT	RT	$5435
0261U	HALIO DX	CRC Checkpoint	$5435
0220U	Precise Dx Br Ca Test	12 Histo Or IHC	$700 about
NEW
0376U	Artera AI Prostate	HE AI	$700 CW
X084U = 0414U	Lung OI, Imagene	WSI 8 genes, pos or neg each BM	$700 CW
X088U = 0418U	Precise Dx Br Biopsy	8 Histo and IHC	$700 CW

0084u

 

X084U: Oncology (lung), augmentative algorithmic analysis of digitized whole slide imaging for 8 genes (ALK, BRAF, EGFR, ERBB2, MET, NTRK1-3, RET, ROS1), and KRAS G12C and PD-L1, if performed, formalin-fixed paraffin-embedded (FFPE) tissue, reported as positive or negative for each biomarker  [**REVISED by CPT 6-12-2023]

Lung OI, Imagene (Oct 1 2023)

 

 

088U

X088U: Oncology (breast), augmentative algorithmic analysis of digitized whole slide imaging of 8 histologic and immunohistochemical features, reported as a recurrence score    [**REVISED by CPT 6-12-2023

JAMA IM // Op Ed and Research Note on AI in Clinical Decision Support

 

1.  Consolidated AI review.

2. Op Ed abridgement.

3. Research Note abridgement.

4.  Louis Black Essay.

250 WORD AI NOTE ABOUT BOTH PIECES

(249 words, 250 requested)

The intersection of artificial intelligence (AI) and healthcare has ushered in a new era of computerized clinical decision support (CDS) tools. In an op-ed, concerns are raised about the FDA's regulation of AI/ML-driven CDS devices in critical care. The op-ed argues that the current regulatory framework is inadequate, as evidenced by a case series examining the authorization of 10 such devices. Alarmingly, only two of these devices cited peer-reviewed publications, and just one outlined safety risks. Furthermore, none provided their software code for independent validation, evaluated clinical efficacy, or assessed potential biases. This lack of transparency and evidence raises serious questions about the reliability and safety of AI/ML CDS devices. The op-ed calls for greater transparency in algorithms, rigorous preapproval studies, and a focus on mitigating biases, urging urgent reforms in FDA guidance and enforcement.

In a related research note, the study delves deeper into the authorization of AI/ML CDS devices, focusing on critical care. The study identifies 10 such devices out of 521 in the FDA's database. 

It finds that most of these devices were authorized through the 510(k) pathway, which does not require clinical data submission. Clinical evidence varied, from absent to peer-reviewed assessments, and few devices mentioned safety evaluations or AI/ML predicates. Notably, a widely implemented model lacked FDA authorization. The research note underscores the need to update regulatory requirements to align with current AI/ML practices in healthcare.

Together, these pieces highlight the urgent necessity for regulatory reforms in the AI/ML-driven CDS device landscape. Transparency, rigorous evaluation, and bias mitigation should be prioritized to ensure the safety and efficacy of these technologies, particularly in high-stakes critical care scenarios. Failure to address these issues may jeopardize patient care and exacerbate healthcare disparities.

###

OP ED - WEISSMAN, PENN (AI SUMMARY)

https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2810620

The op-ed discusses the emerging era of computerized clinical decision support (CDS) tools, particularly those incorporating artificial intelligence and machine learning (AI/ML). It highlights a case series by Lee and colleagues, which examined the FDA approval process for 10 AI/ML CDS devices designed for critical care.

The key findings of concern are as follows:

• Lack of Transparency: Most companies did not disclose the algorithms, programming code, or data sets behind their AI/ML CDS devices, creating a "black box" problem where users cannot assess how the system works.
• 
  
• Insufficient Evidence: Only a few device authorizations cited peer-reviewed publications, and just one provided a detailed safety risk assessment. There is a dearth of high-quality evidence supporting the validity, safety, and efficacy of these devices.
• 
  
• 510(k) Pathway Eligibility: Some devices were approved through the 510(k) pathway despite differences from their predicates, which raises questions about the appropriateness of this regulatory pathway for AI/ML CDS devices.
• 
  
• Bias Evaluation: The FDA should scrutinize these devices for potential biases, especially those derived from data that lack diversity or perpetuate existing healthcare disparities.

The op-ed calls for urgent reforms in the regulation of AI/ML-driven CDS devices. It recommends greater transparency in algorithms, rigorous preapproval studies, postmarketing surveillance, reconsideration of the 510(k) pathway, and a focus on mitigating biases. Without these changes, it suggests that healthcare professionals and institutions should exercise caution when using these devices for critical decisions.

###

RESEARCH LETTER   - GROSS, YALE (*AI SUMMARY)

https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2810619

This JAMA research note discusses the regulatory landscape and evidence behind clinical decision support (CDS) devices that use machine learning (ML) or artificial intelligence (AI) in the critical care setting. Here's a summary of the key points:

• The use of predictive CDS devices with AI or ML has the potential to improve critical care outcomes, but there is a lack of a clear regulatory framework.
• 
  
• Recent guidance from the FDA indicates that most CDS tools for critical illness will be regulated due to the urgency of the decisions they inform. However, concerns exist regarding the effectiveness and safety of current regulatory frameworks designed before advanced statistical learning methods became widely available.
• 
  
• The study examined data from the FDA's AI and ML database as of December 15, 2022, to identify devices offering CDS in critical care and assess the evidence provided for their authorization.
• 
  
• Among 521 authorizations in the FDA AI/ML database, only 10 devices were relevant to critical care. Most of these devices were authorized through the 510(k) pathway, which does not require submission of clinical data, and few mentioned safety assessments or published data.
• 
  
• None of the devices mentioned an evaluation of performance bias, and only three devices had AI/ML predicates.
• 
  
• Notably, a high-profile sepsis-focused prediction model that could serve as a CDS device did not appear to have FDA authorization.
• 
  
• The study emphasizes the need to update regulatory requirements to align with the current use of AI/ML systems in various clinical practice settings, especially for high-risk decisions in critical care.
• 
  
• One limitation of the study is that it does not account for FDA approvals through recently ended precertification pilot pathways or for other devices in widespread use not present in the FDA database.

In summary, the research note highlights the challenges in regulating AI/ML-based CDS devices in critical care and calls for updating regulatory requirements to ensure their safety and effectiveness.

###
REVIEW THE ABOVE BY LOUIS BLACK

Write an essay in the satirical voice of Louis Black, harpooning the two articles (Op Ed and Reserach Note) and portraying the authors as Nervous Nellies.

ChatGPT

Ladies and gentlemen, gather 'round! It's time for a satirical rollercoaster ride through the land of Nervous Nellies and their newfound fear of artificial intelligence and machine learning in healthcare. You see, we've got an op-ed and a research note that read like the script of a "Chicken Little" reboot. Let's dive in!

First, we have this op-ed that paints a gloomy picture of AI/ML-driven clinical decision support (CDS) tools. According to these authors, we're entering a dystopian era where AI/ML might actually help doctors make better decisions! Shocking, I know. They wring their hands over a lack of "algorithmic transparency" and "evidentiary standards." Well, let me tell you, folks, transparency is overrated. Who needs to understand how these magical algorithms work when we've got doctors who can barely explain our lab results in plain English?

And don't get me started on the "bias evaluation" they're clamoring for. Apparently, AI/ML might pick up some bad habits from its human creators. Who cares if it inadvertently exacerbates biases? It's not like our healthcare system has ever been a shining example of fairness and equality.

Then we've got this research note, where they cherry-pick a few AI/ML devices and make it sound like they're about to unleash the robot apocalypse. They quiver in fear that some devices don't have enough "clinical evidence" or that they're not similar enough to their older, less advanced counterparts. But let's face it, if the older devices were so great, we wouldn't need these newfangled AI tools in the first place.

These Nervous Nellies want to slow down innovation and keep us stuck in the dark ages of medicine. Well, I say, let's embrace the future! AI and ML have the potential to revolutionize healthcare, making diagnoses faster, more accurate, and accessible to all. So, to the authors of these articles, I say, "Take a chill pill and let the robots help us out. The sky isn't falling, but your sense of humor sure is!"

Understanding the FDA LDT Financials

The FDA press release about the LDT proposal doesn't  mention finances at all, except a qualitative remark that "benefits would outweight the costs."

The proposed rule says very little about costs, except for some cryptic summary information at beginning and end.  The "annualized" 20 year benefits (the 20 year value divided by 20) will be $2B to $86B per year.  The annualized costs will be $2B to $19B per year (the 20 year costs divided by 20).

Find footnote 34 of the Rule and it takes you to a 127-page PDF at the FDA, the preliminary financial analysis.   Slog through to the end, to find tables 35 and 37.  Those are the "money slides."

Table 35 above shows that benefits begin at $14B per year as soon as registration is put in place, rapidly rising to about $26B per year out to year 20.  

It also shows that costs will be $10B and $27B in years 3 and 4, tapering to $3B per year out to year 20.

The costs are actual lab compliance costs.  The benefits, we learn, are mostly driven by value-years of about 10,000 patients living about 3 years longer for a value of about $2M ($2M x 10,000 = $20B).  All of those numbers are highly conjectural and reflect implied or intangible values.

In table 37, the 20-year costs are simple added up (with discounting) and then, divided by 20 to annualize them.   Thus, the "annualized" cost to industry is about $6B and the annualized benefits to society anywhere from $22B to $31B per year.   What's lost is that most of the industry costs are $40-50B in the first few years.  (This is also why the present-value costs are much less sensitive to the 3% or 7% discounter).

Chat GPT Summarizes My Several Blogs about FDA/LDT Regulations

On September 29, 2023, FDA released an 80-page proposed rulemaking and 127-page supplemental document.  It has over 200 pages, over 40 tables, some 150 references and 90 footnotes.

I wrote a main blog, and three sidebars (on legal aspects; on PR; and on the 127-page supplement, which many will never discover).   Today, I asked Chat GPT 3.5 to summarize my essays into one report.

###

CHAT GPT:

In the provided materials from Dr Quinn, there is an in-depth discussion of the FDA's proposed regulation for lab-developed tests (LDTs) and its associated economic analysis. The main focus is on the economic impact, potential benefits, and challenges of this regulation. Here is a detailed summary of the main blog and three sidebar blogs:

Main Blog: Link: 

The main blog discusses the FDA's proposed regulation for LDTs, emphasizing its economic aspects. It notes that the FDA's press release was strategically timed to control the narrative around the regulation. The main highlight is the significant economic impact of the proposed regulation. It suggests that there could be health benefits ranging from $2 billion to $60 billion per year, while the estimated annual costs are approximately $5 billion.

One key point of contention is that the costs are tangible (e.g., PMA costs, clinical trial costs), while the benefits are largely intangible, including extended patient lifespan and improved quality of life. The calculation involves "annualized costs," which are derived from a 20-year projection of costs, discounted to their net present value at either 3% or 7%. These annualized costs, when divided by 20, provide the "annualized benefit" or "annualized cost."

The blog stresses the importance of Table 35, which details the benefit and cost to the industry by year. It points out that the FDA anticipates industry costs of around $5 billion per year for 20 years, which are offset by "health benefits" estimated to be between $26 billion and $40 billion annually.  [BQ: The document makes obscure that the immediate cost to industry up front would be $30-40B(!).  While the financial "benefits" are mostly projected for future decades from intangible QALYs valued at millions per patient].  [See Suppl. tables 35, 37].

Sidebar One (Legal Justification): Link:

This sidebar focuses on the legal justification the FDA provides for its regulation of LDTs. It outlines several key arguments the FDA makes to assert its regulatory authority over LDTs, including the interpretation of the Federal Food, Drug, and Cosmetic Act (FD&C Act), functional equivalence with commercial in vitro diagnostic (IVD) devices, and the role of CLIA certification. It also highlights the FDA's stance on interstate commerce and commercial distribution.

The summary notes that some arguments, such as the interpretation of interstate commerce, may be perceived as weaker, while others, like the FDA's historical practice, are presented as strong foundations for the regulation.

Sidebar Two (PR Game): Link:

This sidebar discusses the FDA's strategic approach to public relations in unveiling the LDT regulation. It points out that the FDA released the rule on a Friday morning, accompanied by a press release emphasizing the urgency of the FDA's actions. The lack of a "Fact Sheet" or detailed summary made it challenging for journalists to fully understand the rule. The blog highlights the FDA's effective control of the narrative during the initial news cycle.

Sidebar Three (Secret Document - the FDA Financial Supplement): [Link]:

This sidebar unveils a "secret document," which is a 127-page preliminary economic analysis of the regulation. It discusses the document's discovery and emphasizes its importance for understanding the economic underpinnings of the regulation. The analysis suggests that the benefits of the regulation far outweigh the costs, with significant health benefits and a relatively lower financial burden on the industry.

The document explains how the FDA arrived at these figures using various assumptions, including error rates in healthcare and the potential impact of FDA approval on reducing errors. It also addresses issues such as consolidation in the lab industry, the exit of smaller labs, and the question of why few labs voluntarily seek FDA approval for LDTs.

In summary, these materials provide a comprehensive overview of the FDA's proposed regulation for LDTs, focusing on the economic impact, legal justifications, PR strategy, and a detailed economic analysis. The discussion highlights the potential benefits and challenges associated with the regulation and raises questions about its practical implementation and implications for the healthcare industry.