Saturday, June 3, 2023

Using AI to Help Understand Novitas LCD L39365 (Oncology Tests)

Master blog here:

http://www.discoveriesinhealthpolicy.com/2023/06/novitas-releases-anxiously-awaited.html


STRUCTURE OF LCD L39365

01

Definitions

4 pp, 1680 words

02

Coverage Rules & Authorities

9 pp, 1800 words

03

Authority Sources Analysis

17 pp, 7400 words

04

Specific Tests: Evidence & Analysis

40 pp, 19,000 words

 

 

Of the above, first 8 pp and 4500 words are “Evidence,” the rest,  “Analysis”

05

Bibliography – Final

17 pp, 7250 words, 230 citations

06

Bibliography – Proposed

3 pp, 520 words, 19 citations

 Part 03 is an extensive analysis of how authorities, such as NCCN, do or do not meet 8 criteria for valid authorities (e.g. "transparency of process.")  The criteria come from Institute of Medicine, 1990.   


Supplemental AI assistance (GPT4) here.

Please assess the definitions section.

This lengthy "definitions" section from the Medicare local coverage document (LCD L39365) provides comprehensive terminology to guide the understanding of oncology tests. It is designed to ensure clarity and consistency when interpreting the policy and its application in clinical practice. This section assists various stakeholders such as clinicians, medical coders, billing personnel, and patients in interpreting the coverage rules, ensuring that everyone understands the language used.

The author's goal in this section appears to be twofold:

Clarity: Each term is defined meticulously, providing a comprehensive understanding of each concept. For example, the document distinguishes between diagnostic, prognostic, predictive, and therapeutic biomarkers, explaining their unique roles.

Precision: The document is specific about how it uses certain terms. For instance, it avoids using "Comprehensive Genomic Profiling" (CGP) and "Genomic Testing" due to varying interpretations, while choosing to use a broader term like "Genetic Testing" to encompass a range of assay types and methodologies.

One unorthodox approach is the omission of certain commonly used terms. For instance, "Comprehensive Genomic Profiling" (CGP) and "Genomic Testing" are intentionally not used because the author believes they can be interpreted differently depending on the context, which may lead to confusion. Instead, the author uses a broad term, "Genetic Testing," to include any assays evaluating DNA and/or RNA, thereby reducing potential misinterpretations.

The author also specifies contexts where certain terms would not apply, such as the use of "substantiated suspicion of cancer." It is clearly mentioned that radiologic suspicion of cancer is not considered "substantiated" except where specified. This level of detail shows an attempt to prevent misinterpretation or over-application of certain terms.

Additionally, the author refers to federal regulations such as the Code of Federal Regulations (CFR) to define terms like "FDA-cleared or approved test system," "Kit," and "Reflex testing." This approach ensures alignment with federally recognized definitions and maintains legal and operational consistency.

In conclusion, the detailed definitions provided in this LCD contribute to a comprehensive understanding of the coverage and application of oncology tests. It gives a clear path for health professionals to determine what is covered, what conditions need to be met, and how to correctly use the terminology within the LCD. It further demonstrates a careful and deliberate approach to maintaining precision and avoiding potential ambiguities in the complex and evolving field of oncology.

Author includes RNA testing as "genetic" testing?  Is that usage a bit unusual?

The inclusion of RNA testing in the umbrella term "genetic testing" in this LCD reflects this shift towards a more holistic understanding of the complex interplay between DNA, RNA, and disease. It also ensures that coverage applies to a wide array of diagnostic tests, whether they involve DNA or RNA, thereby allowing for more comprehensive patient care.

However, your observation is correct in the sense that this might be seen as an unorthodox or unconventional approach, as not all frameworks or definitions in the field might include RNA testing under the moniker of "genetic" testing. The definitions in the LCD reflect an attempt to be as inclusive and comprehensive as possible in defining genetic testing for coverage purposes.

After definitions, there is a lengthy section explaining coverage rules and authorities, such as NCCN.  Can you discuss this section?

The section you mentioned from the LCD L39365 provides detailed rules and criteria for determining the coverage of genetic testing in the context of oncology. 

The rules, as laid out, emphasize the importance of evidence-based medicine and appropriate clinical judgment.

Here are the key takeaways:

Cancer Diagnosis or Suspicion: For a genetic test to be considered medically reasonable and necessary, the provider must have established a diagnosis of cancer or have significant evidence to create suspicion for cancer. This decision must be based on both a clinical evaluation and abnormal results from histologic, cytologic, and/or flow cytometric examination.

Clinical Impact: The genetic test must have a direct impact on the management of the patient’s specific condition. This means that the results of the genetic test should have the potential to change the course of treatment or patient management.

Evidence-Based Databases and Knowledge Bases: The LCD refers to three specific sources for determining whether the gene-disease association is actionable

  • NIH-sponsored Clinical Genome Resource (ClinGen), 
  • NCCN, and the 
  • Memorial Sloan Kettering Cancer Center-sponsored Oncology Knowledge Base (OncoKB).
  • Each of these sources has specific criteria that the gene-disease association must meet in order to be considered for coverage.

Next Generation Sequencing (NGS) tests: Additional requirements are specified for NGS tests. These include the FDA status of the NGS test, the patient and their cancer meeting specific criteria detailed in the NCD, and NCD requirements taking precedence over Local Coverage Determinations.

Limitations: Genetic tests that do not meet specific analytical validity, clinical validity, and/or clinical utility criteria will not be covered. This includes genetic testing in patients who do not have an established diagnosis of cancer or substantiated suspicion of cancer, genetic testing for the purposes of screening asymptomatic patients, repetitions of the same genetic test, and several specific tests listed in the LCD.

Provider Qualifications: The provider ordering the genetic test must be the treating clinician who is responsible for the patient’s cancer management, must understand how the test result will impact the patient’s condition, and must have communicated this information effectively to the patient.

The overall goal of these rules is to ensure that genetic testing is used appropriately and effectively in the management of patients with cancer, and that these tests provide value in terms of influencing clinical decision-making. This also ensures that patients are not exposed to unnecessary testing and that healthcare resources are used judiciously.


Please review the next section, which is a lengthy discussion of how these rules and evidence authorities (E.g. NCCN) meet or do not meet objective quality measures.

Quote:  Potential knowledge bases were evaluated against the Good Practice Guidelines found in the Clinical Practice Guidelines: Directions for a New Program written by the Institute of Medicine in 1990.  PDF here.

Based on the four-part analysis you have provided, here is a consolidated overview of how the National Comprehensive Cancer Network (NCCN), OncoKB, and ClinGen meet or do not meet the eight attributes outlined by the Good Practice Guidelines:

1 Transparent Process:

NCCN: Meets this attribute. They provide detailed guidelines and panel meeting records.

OncoKB: Meets this attribute but lacks transparency regarding its evidence sources.

ClinGen: Meets this attribute. They maintain a transparent and detailed process but could improve clarity around their internal decision-making processes.

2 Systematic Review of Evidence:

NCCN: Meets this attribute. Their guidelines are derived from rigorous evidence-based reviews.

OncoKB: Meets this attribute. They use a variety of data sources that are reviewed systematically.

ClinGen: Partially meets this attribute. There is a lack of information available regarding their systematic review process.

3 Rating the Quality and Applicability of Studies:

NCCN: Meets this attribute. They have a process for grading the quality of the literature.

OncoKB: Does not meet this attribute. There is a lack of clarity on how they assess the quality and applicability of studies.

ClinGen: Meets this attribute. They have a process for evaluating the quality and applicability of studies.

4 Clear Presentation of Recommendations:

NCCN: Meets this attribute. Their guidelines are clear and easily accessible.

OncoKB: Partially meets this attribute. They provide clear recommendations, but there's lack of detail about the evidence supporting these recommendations.

ClinGen: Partially meets this attribute. They provide clear recommendations, but the presentation of evidence could be improved.

5 Managing Conflicts of Interest:

NCCN: Meets this attribute. They have procedures in place to manage conflicts of interest.

OncoKB: Meets this attribute. They have clear policies and procedures for managing conflicts of interest.

ClinGen: Meets this attribute. They disclose conflicts of interest and have guidelines for managing them.

6 Multidisciplinary Process:

NCCN: Meets this attribute. Their guideline development process is multidisciplinary.

OncoKB: Meets this attribute, but could improve by including patient advocacy groups in their development process.

ClinGen: Meets this attribute. Their expert panels represent a variety of professions and backgrounds.

7 Scheduled Reviews:

NCCN: Meets this attribute. They regularly review and update their guidelines.

OncoKB: Meets this attribute. They have a rigorous review process and stay current with new information.

ClinGen: Partially meets this attribute. They have policies for reviewing gene-disease associations, but a similar policy for Clinical Actionability scores and recommendations is not evident.

8 Documentation:

NCCN: Partially meets this attribute. They provide detailed procedures but could be more transparent about the dialogue during guideline panel meetings.

OncoKB: Partially meets this attribute. They lack transparency in terms of the data and citations that support their scoring.

ClinGen: Partially meets this attribute. They provide overall scoring criteria and parameters, but protocols may vary across different workgroups.

In summary, all three knowledge bases generally meet or partially meet the attributes outlined in the Good Practice Guidelines. However, areas such as the presentation of evidence, scheduled reviews, and documentation could benefit from improvements in all three databases.

EVIDENCE REVIEW - 40 PAGES - HIGHLIGHTS

From a comprehensive analysis of all nine parts of the Test Evidence Review, it's clear that the author is assessing various genetic and genomic tests in a critical manner, focusing on their analytic validity, clinical validity, and clinical utility.

The assessment approach of the author is methodical and rigorous. The evidence review for each test is conducted in a manner that is systematic, with a focus on:

  • Methodology and results of the original studies that validate the tests
  • Investigating both the technical aspects of the tests (i.e., their analytic validity) and their clinical applicability and effectiveness (i.e., clinical validity and utility)
  • Examining how the tests are performed and validated, what they are measuring, how accurately they measure it, and what the clinical implications and outcomes of these measurements are.

The author seems critical but also balanced. Each test is evaluated against a set of stringent criteria for determining their validity and utility, which is a necessary part of any thorough and robust scientific review process. Some tests, like Colvera, have multiple studies examining their validity, which are analyzed in detail, while others, like PancreaSeq, have limited peer-reviewed evidence available, which is also pointed out.

From the broad perspective, it seems not all the tests "fail," but most of the tests reviewed don't meet all the necessary criteria to be considered clinically useful and reliable for the Medicare population. The author points out deficiencies and limitations in available data, study design, and follow-up procedures.

Overall, this review emphasizes the importance of stringent evidence-based evaluation in determining the clinical utility of novel diagnostic tests, and it highlights the gaps in current literature that need to be addressed before these tests can be confidently adopted in clinical practice, particularly for a Medicare population.

Please discuss the 230 citation bibliography. [AI switches to v3.5 here.]

Based on the provided bibliography, here are some inferences that can be made:

Author's Expertise: The author of the Medicare policy document regarding coverage of molecular cancer tests appears to have a strong background in oncology, genetics, and precision medicine. The extensive range of references in the bibliography suggests a deep understanding of the scientific literature in these fields.

Comprehensive Research: The author has conducted comprehensive research on various aspects of cancer testing and treatment. The bibliography includes references covering a wide range of topics, including genomics, molecular profiling, targeted therapies, diagnostic tests, and clinical utility of different tests.

Focus on Personalized Medicine: The bibliography highlights the author's focus on personalized medicine and the use of genomic information to guide cancer treatment decisions. There are references related to genomic profiling, genetic variants, gene expression profiling, and the integration of molecular data into clinical practice.

Integration of Clinical Guidelines: The author has incorporated references to clinical guidelines and resources such as OncoKB and the National Comprehensive Cancer Network (NCCN). This indicates an intention to align the Medicare policy with evidence-based recommendations from reputable organizations.

Emphasis on Validity and Clinical Utility: The inclusion of references related to gene-disease validity classification, interpretation and actionability of genetic variants, and effectiveness of laboratory-developed tests suggests that the author is concerned with the validity and clinical utility of molecular cancer tests. The author likely wants to ensure that the tests covered by Medicare have demonstrated accuracy, clinical relevance, and proven benefit for patient care.

Diverse Cancer Types: The bibliography covers a range of cancer types, including melanoma, urothelial carcinoma, pancreatic cysts, rhabdomyosarcoma, leukemia, and colorectal cancer. This indicates a broad approach to understanding the application of molecular tests across different malignancies.

Multidisciplinary Approach: The author has included references from various disciplines, including oncology, pathology, genomics, dermatology, and hematology. This suggests a multidisciplinary approach to evaluating molecular cancer tests and incorporating different perspectives into the Medicare policy.

Overall, this bibliography reflects a thorough and evidence-based approach taken by the author in developing the Medicare policy on molecular cancer test coverage. The author has aimed to provide a comprehensive overview of the scientific literature, guidelines, and clinical evidence to ensure appropriate and effective utilization of these tests in the management of cancer patients.This analysis provides important insights into the technology assessment in healthcare, emphasizing the need for rigorous validation studies, transparent methodology, and careful interpretation of results for tests to be considered effective and reliable.

Compare the Version 1 bibliography (19 citations) to the Version 2 bibliography (230 citations.)

Comparing the original short bibliography of Version 1 (19 citations) to the expanded bibliography of Version 2 (230 citations), several notable changes and major features can be observed:

  1. Increased Depth and Scope: The most apparent change is the significant expansion of the bibliography in Version 2. The initial 19 citations in Version 1 have grown to 230 citations, indicating a substantial increase in the depth and scope of research conducted to develop the Medicare policy. The expanded bibliography reflects a more comprehensive examination of the literature and a broader understanding of molecular cancer tests.
  2. Incorporation of Clinical Guidelines: In both versions, there is a focus on the utilization of clinical guidelines to inform the policy. Version 1 includes references to the National Comprehensive Cancer Network (NCCN) guidelines, while Version 2 expands upon this by including additional NCCN references and incorporating guidelines from other reputable organizations like the European Society of Cardiology and the American Joint Committee on Cancer.
  3. Emphasis on Gene-Disease Validity: Version 2 includes several references (such as ClinGen and the Clinical Genome Resource) that discuss the clinical validity of gene-disease associations and the evaluation of gene-disease evidence. This indicates an increased emphasis on assessing the validity and clinical relevance of genetic variants and their impact on patient care.
  4. Consideration of Biomarkers and Biomarker Testing: The inclusion of references on biomarkers and biomarker testing in both versions highlights the importance of these factors in molecular cancer testing. Version 2 expands upon this topic, incorporating references from the FDA-NIH Biomarker Working Group, emphasizing the significance of biomarkers in cancer diagnosis, prognosis, and treatment selection.
  5. Integration of Precision Oncology Knowledge Bases: Version 2 incorporates references to OncoKB and the precision oncology knowledge base, emphasizing the role of these resources in providing evidence-based information on actionable genetic variants and guiding treatment decisions.
  6. Enhanced Understanding of Clinical Actionability: Version 2 includes references that provide frameworks for assessing the clinical actionability of genetic disorders and evaluating the clinical validity of gene-disease associations. This indicates a deeper understanding of the importance of clinical actionability and evidence-based decision-making in the context of molecular cancer tests.
  7. Expansion to Multiple Cancer Types: While the initial bibliography covers a range of cancer-related topics, Version 2 further expands the coverage of different cancer types, including melanoma, urothelial carcinoma, pancreatic cysts, rhabdomyosarcoma, leukemia, and colorectal cancer. This reflects a broader consideration of molecular cancer tests in various malignancies.
  8. Emphasis on FDA Recognition and Database Validation: Version 2 includes references to FDA recognition of public genetic variant databases, highlighting the importance of validated and reliable databases for accurate interpretation of genetic variants. This indicates a focus on ensuring the quality and credibility of the data used in molecular cancer testing.

Overall, the evolution from Version 1 to Version 2 demonstrates a significant increase in the depth, breadth, and rigor of research conducted to develop the Medicare policy. The expanded bibliography in Version 2 reflects a more comprehensive understanding of molecular cancer tests, incorporating clinical guidelines, gene-disease validity, biomarkers, clinical actionability, and the utilization of reliable databases. This suggests an enhanced focus on evidence-based decision-making and a commitment to providing up-to-date and accurate information to guide the coverage of molecular cancer tests in Medicare.




No comments:

Post a Comment

Note: Only a member of this blog may post a comment.