Discussing CMS NCD Key Questions and Gaps

 Can you read the text in the image and write it out for me

ChatGPT

Sure, here's the text from the image:

vbnet

Copy code

B. Discussion of Evidence

1. Evidence Question(s)

Question 1: Is the evidence sufficient to determine that a blood-based biomarker screening test (at the time of this review Epi proColon® is the only FDA approved commercially available blood-based biomarker screening test) is a suitable colorectal cancer screening test for prevention or early detection in Medicare beneficiaries?

Question 2: Is the evidence sufficient to determine that colorectal cancer screening using a blood-based biomarker screening test is appropriate for Medicare beneficiaries?

2. External Technology Assessments..

###

User

I'm flummoxed.   CMS offered two key questions for evidence review of colorectal cancer screening tests, in a 2001 tech assessment.   One question asks if the tests are "suitable".  The other, if they are "appropriate."  Is this crazy?    How can a test be highly suitable for Medicare patients, but not appropriate.  Or highly appropriate, but not suitable.  It makes my eyes cross!   here are the full questions, am I crazy, or do they virtually overlap and in a vague way.  ###

Question 1:     Is the evidence sufficient to determine that a blood-based biomarker screening test (at the time of this review Epi proColon® is the only FDA approved commercially available blood-based biomarker screening test) is a suitable colorectal cancer screening test for prevention or early detection in Medicare beneficiaries?

Question 2:     Is the evidence sufficient to determine that colorectal cancer screening using a blood-based biomarker screening test is appropriate for Medicare beneficiaries?

ChatGPT

Your confusion is completely understandable; the language in the two questions is quite similar, and it can certainly seem like they're asking the same thing. However, there is a subtle distinction that might be implied by the choice of words "suitable" and "appropriate," which can be nuanced in the context of medical screening and policy.

"Suitable" often refers to the effectiveness and accuracy of the test itself—its sensitivity, specificity, and predictive value in the detection or prevention of disease in a general sense. It's about the fit of the tool for the task, considering the characteristics of the test in the context of a medical condition.

"Appropriate," on the other hand, might refer to the broader context of using the test in a specific population—Medicare beneficiaries, in this case. This includes considerations such as cost-effectiveness, the potential for harm versus benefit, the availability of resources to act on the results, and the likelihood that the use of the test will improve outcomes in this particular group. "Appropriate" carries with it a sense of suitability but adds a layer of ethical, economic, and practical considerations.

For example, a test could be highly suitable in that it is effective at detecting colorectal cancer, but not appropriate for Medicare beneficiaries if it's prohibitively expensive or if positive results would lead to interventions that are not safe or effective in an older population.

Alternatively, a test might be appropriate because it's cost-effective and has minimal side effects, but not suitable if it isn't sensitive enough to detect early-stage cancers, which is essential for a screening test where early detection is the goal.

It's important to look at both the clinical performance of a test and the implications of its use in a specific patient population. These two questions likely aim to address both the clinical efficacy and the broader implications of implementing such a test for Medicare beneficiaries. So no, you're not crazy—the overlap is there, but the emphasis is just slightly different in a way that's meaningful for policy and healthcare decision-making.

User

But it seems to fail their purported standard of writing for the public and at more or less the senior high school level.    Most people would be stumped by the closely parsed difference visible to experts.

ChatGPT

You’re absolutely right. Clarity and accessibility are crucial when it comes to public-facing documents, especially those related to healthcare. The subtleties between “suitable” and “appropriate” could easily be lost on a general audience, and even professionals might find the distinction to be needlessly opaque without further explanation.

When creating these documents, the aim should be to communicate effectively with the intended audience, which often includes patients, their families, and non-specialist healthcare providers. Using language that requires interpretation or a deep understanding of policy nuances can create barriers to understanding and can detract from the document’s usefulness.

The use of plain language is encouraged in public communication to ensure that the information is accessible to readers with varying levels of education and expertise. In the case you presented, it would be more effective if the document not only used clearer language but also provided context for why different aspects of screening tests are being evaluated, and how these evaluations impact the care and options available to Medicare beneficiaries.

Addressing this issue might involve restating these questions in simpler terms or providing a brief explanation of what is meant by "suitable" and "appropriate" in this context. It’s about striking a balance between precision and accessibility, ensuring that the important distinctions are communicated in a way that is understandable to all readers.

User

                  HEre is how CMS approached answers to #1 is it suitable and #2 is it appropriate.   Do these answers jive with your best guesses, above, at the difference in the two questions.   (They may or may not support your conjectures, be objective).   ##### CMS defines suitable in its answer to Q1  (is it SUITABLE):

No, the evidence (direct or indirect) does not demonstrate that screening in Medicare beneficiaries using the Epi proColon® test is SUITABLE for prevention or early detection of colorectal cancer.  The published evidence does not show that the available blood-based biomarker CRC screening test improves health outcomes for Medicare beneficiaries.  The indirect evidence (comparing a blood-based screening test to another test that has been shown to improve mortality such as screening fecal immunochemical test) does not show that blood-based screening test is equivalent in the detection of large adenomas or early stage cancer.  The test performance of the Epi proColon® test does not meet sensitivity and specificity levels established by prior evidence at which the benefits of using the screening test outweigh harms to Medicare patients.

And for Q2 (is it APPROPRIATE).

No, the published evidence does not demonstrate that the available blood-based biomarker screening test will prevent progression of advanced adenomas to cancer or improve health outcomes for patients found to have colorectal cancer.  To provide clarity to innovators and to reduce burden of a reconsideration of this NCD, we provide a direct pathway for coverage of appropriate blood-based biomarker CRC screening tests when the criteria are met.

Among the cited studies, the study populations had a mean age between 55 and 71 years, and for persons at least age 60 years old, the percent ranged from 35.8% to 76%.  Both genders were adequately represented in the evidence base.  Even though the study populations were relatively balanced, some were slightly in favor of women (Church, 2014; Ørntoft, 2015; Sun, 2019).  Epi pro Colon® instructions for use state that “The rate of false positive Epi proColon® results increases with age. Test results should be interpreted with caution in elderly patients. See Performance Characteristics in Section 13.” (Epigenomics AG, Instructions for Use [IFU] Epi proColon®. March 26, 2014.)   While some of the clinical research studies were conducted in the United States (4 studies), many studies were completed in Europe (4), China (8), and Russia (1).

ChatGPT

In the response to Question 1 regarding whether the Epi proColon® test is "suitable," CMS's answer focuses on the test’s clinical efficacy—its sensitivity and specificity. This is consistent with the idea that "suitable" refers to the technical performance of the test and its ability to detect the condition it’s being used to screen for. They state that the test does not meet the established thresholds at which the benefits outweigh the harms, which is a judgment based on clinical effectiveness.

In the response to Question 2 regarding whether the test is "appropriate," CMS seems to expand the scope of their assessment. They mention the impact on health outcomes and progression of disease, which involves considering the broader implications of using the test in the Medicare population. They talk about providing a "direct pathway for coverage" if certain criteria are met, which suggests they are considering factors beyond just the test’s performance—like how it would be implemented and utilized within the healthcare system.

This does support the earlier conjecture to an extent. The answer to Question 1 is focused on the clinical validity and utility of the test, while the answer to Question 2 involves additional considerations like the demographic suitability, the implications of false positives in an older population, and the relevance of the studies to the U.S. Medicare population.

In essence, the CMS answers differentiate between a test being clinically valid ("suitable") and its implications for care within the specific context of the Medicare population ("appropriate"), which includes practical, ethical, and policy considerations. This does align with the nuanced difference between the terms that we discussed, indicating that the use of the terms was intentional and meant to convey these different aspects of evaluation.

User

It's interesting that CMS answers include study of the international population used, FDA warning remarks about the eldelry, etc, and adenomas.    But, the final decision is that CMS will cover any LBX test that is 74% sensitive and 90% specific (period).    So all the diverse  ying and yang and pros and cons that are debated, are absent from the final decision which predicates coverage solely and immediately on thost two statistics 74% and 90%.

ChatGPT

It seems that in the end, CMS has decided to set specific quantitative benchmarks for test performance as the criteria for coverage. This decision to use sensitivity and specificity thresholds might simplify the coverage criteria and provide a clear, measurable standard for future tests to meet in order to be covered.

The consideration of factors such as the demographics of the study populations, FDA warnings, and the performance of the test in detecting adenomas are important parts of the evidence review. They provide context and justification for why certain benchmarks are set. However, when it comes to making a coverage decision, having clear, objective criteria like sensitivity and specificity percentages can make the policy more straightforward to apply and enforce.

The decision to focus on these two statistics for coverage does not negate the importance of the broader considerations discussed in the evidence review; rather, it distills the decision down to quantifiable metrics. This could be intended to create a transparent and standardized baseline for all manufacturers to meet, which can encourage the development of tests that are not only clinically valid but also practically applicable to the Medicare population.

Furthermore, this approach could also be a reflection of the need to balance detailed scientific review with the operational simplicity of the healthcare system. By having clear numerical targets, CMS provides a direct pathway for coverage that can be understood and met by innovators, potentially reducing the burden of lengthy coverage determinations while still upholding a standard that ensures a reasonable level of efficacy for the tests.

User

I think the analysis could be framed a little less charitably (but fairly and perhaps insightfully) that they have 40 pages of review of diverse factors like racial inclusion, age includion, whether international studies are in the data, and so on, yet on the 40th page at the end they just predicate all future decisions for new products on the 74/90 rule alone.    So I think it's fair to ask, what use were all the discussions.  It's like, by analogy, you require 20 kinds of info about presidential candidates, but rule 21, is that you vote based on eye color alone.   So what was the use of the other 20 factors.   

ChatGPT

Your analogy certainly drives the point home about the disparity between the complexity of the review and the simplicity of the final criteria. If a review process takes into account a myriad of factors but then the final decision rests on just one or two metrics, it’s natural to question the value and purpose of the comprehensive review. The situation you're describing with CMS could raise questions about the transparency and consistency of the decision-making process.

One possible explanation is that the thorough review process is intended to be comprehensive to ensure all relevant factors are considered before setting the benchmarks. The discussions could serve to justify why those particular benchmarks—74% sensitivity and 90% specificity—are chosen as the cut-off points for coverage. This approach might be aimed at ensuring due diligence has been performed and all angles considered before landing on the most crucial, decisive factors.

However, it could also be argued, as you suggest, that if only a select few criteria are ultimately used for the decision, it might indicate a disconnect between the evidence review process and the policy-making process. This can lead to frustration and confusion among stakeholders who expect that the breadth of factors discussed in the review will directly inform the final coverage decision.

Such a decision-making process might suggest the need for a more integrated approach where the variety of factors that are evaluated in the evidence review have a clearer impact on the final coverage decision. Transparency in how each aspect of the review influences the decision-making could help address concerns about the relevance of the extensive discussions that appear to precede a seemingly oversimplified conclusion.