AI Comparison of HHS ONC Proposal, Google & Amazon Comments (June 2023)

Main Blog: https://www.discoveriesinhealthpolicy.com/2023/12/hhs-scandalously-bad-x-2-costbenefit.html

AI backgrounder / deep dive

https://brucedocumentblog.blogspot.com/2023/12/chat-gpt-on-hhs-onc-ai-ehr-final-rule.html

###

Attached are comment letters from Google and Amazon on an HHS rule on EHR's and AI.   Please write a consolidated viewpoint of their comments.   Please summarize the consolidated comments in 12 major points.  Thank you!  You are writing for a policy expert.

ChatGPT

I have reviewed the comment letters from Google and Amazon on the HHS rule concerning Electronic Health Records (EHRs) and Artificial Intelligence (AI). Here's a consolidated viewpoint of their comments summarized into 12 major points:

Support for Enhanced Interoperability: Both companies emphasize the importance of improved interoperability in healthcare, facilitating better data access for patients and clinicians.

Empowering Patients and Clinicians: There is a shared belief that access to more comprehensive health data empowers patients and clinicians to make informed decisions.

Advocating for Transparency in AI/ML Tools: Both stress the need for transparency in the development and use of AI/ML tools, particularly in high-risk applications.

Risk-Based Approach to Regulation: They advocate for a risk-based approach to the regulation of AI/ML tools, focusing on high-risk use cases.

Data Privacy and Protection: The importance of data privacy and the protection of intellectual property and trade secrets in any transparency requirements is highlighted.

Quality Assurance Across Product Life Cycle: Emphasis on maintaining quality across the product life cycle, especially for high-risk AI/ML use cases.

Human-AI Collaboration: The necessity for human oversight in AI/ML systems is recognized, especially in high-risk scenarios, to ensure safety and accuracy.

Promoting Data Diversity: They call for diverse patient data in AI/ML development to address healthcare biases and ensure effective care for all patient populations.

Development of Reference Data Sets: The need for developing and using reference data sets to benchmark AI/ML technologies is emphasized.

Enabling Continuous AI/ML Learning: They highlight the importance of regulatory frameworks that allow for continuous improvement of AI/ML models.

Standardization and International Alignment: The companies advocate for the alignment with international standards for consistency and effectiveness in AI/ML use.

Incorporation of Third-Party Tools and Real-Time Data: Both emphasize the integration of third-party tools and real-time data capabilities in EHR systems to enhance healthcare delivery and decision-making.

These points reflect a consensus on the need for responsible, transparent, and innovative use of technology in healthcare, focusing on patient-centered care and data-driven decision-making.

User

Do they go so far as to suggest that HHS ONC should not finalize the proposed rule at this time?

ChatGPT

Both Amazon and Google, in their comments on the HHS ONC proposed rule regarding Electronic Health Records (EHRs) and Artificial Intelligence (AI), did not explicitly suggest that HHS ONC should not finalize the proposed rule at this time. However, they did express concerns and sought clarifications:

Amazon's Concerns: Amazon pointed out that the proposed rule covers a broad array of AI/ML-based tools, extending beyond its intended scope of predictive technologies. They encourage ONC to refine the definition to focus specifically on AI/ML tools that use data to train algorithms and predictive models for tasks associated with human intelligence, such as computer vision, natural language processing, and speech recognition​​.

Google's Perspective: Google highlighted the need for clarity regarding low-risk predictive DSI (Decision Support Intervention) use cases. They suggest that the rule should clearly state that low-risk use cases, such as non-clinical applications (e.g., EHR integrations for administrative notes and billing), are not covered by the proposed regulations. Google argues that providing such clarifications would help avoid consumer confusion and unnecessary compliance costs​​.

In summary, while neither company explicitly recommends against finalizing the rule, they both stress the need for more precise definitions and clarifications to ensure that the rule effectively targets high-risk use cases and avoids unnecessary regulatory burdens on low-risk applications.

User

What were the 2 or 3 greatest areas of dissimilarity between the independent views of Amazon and Google?

ChatGPT

The two or three greatest areas of dissimilarity between the independent views of Amazon and Google on the HHS ONC proposed rule regarding EHRs and AI are:

Focus on Data Diversity and Bias:

Amazon: Amazon places a significant emphasis on encouraging data diversity. They stress the need for sufficient data on diverse patient populations to reflect real-world use and address underrepresented attributes, thus providing safe and effective care. Amazon also highlights the importance of identifying and addressing existing biases in healthcare delivery that could influence AI/ML model performance. They support the development of reference data sets to benchmark technology performance and address healthcare biases​​.

Google: Google's focus, in contrast, is more on the regulatory scope and clarity, particularly for low-risk predictive DSI (Decision Support Intervention) use cases. They advocate for clarity in the proposed rules to ensure that low-risk use cases, such as EHR integrations for administrative notes and billing, are not covered by the proposed regulations. Google emphasizes avoiding consumer confusion and unnecessary compliance costs due to ambiguous application of the rules​​.

Approach to AI/ML Tools and Services:

Amazon: Amazon agrees with ONC on the importance of transparency in the development and use of AI/ML tools, particularly for higher-risk applications. They stress that AI/ML tools should be fair, appropriate, valid, effective, and safe. Amazon has developed Service Cards to provide customers with information on their AI/ML services, showcasing a commitment to transparency and responsible use of AI/ML in healthcare​​.

Google: Google, while also concerned with transparency and the appropriate use of AI/ML, does not specifically mention initiatives like Service Cards or similar tools for transparency. Their comments are more focused on the need for regulatory clarity and the exclusion of low-risk predictive DSI tools from certain regulatory burdens.

These dissimilarities highlight Amazon's focus on addressing bias and promoting diversity in data used for AI/ML, and their efforts towards transparency in AI/ML tools, compared to Google's emphasis on regulatory clarity and the distinction between high-risk and low-risk AI/ML applications.

Bonus:  Ten Main Points of UHG Letter (13pp)

The UnitedHealth Group (UHG) made several key points in their response to the Proposed Rule from the Office of the National Coordinator for Health Information Technology (ONC). Here's a concise summary:

Support for ONC's Engagement: UHG appreciates ONC's public engagement and transparency efforts during the comment process.

Definitions and Scope: They discuss the definition and scope of electronic health information (EHI) and the interoperability requirement.

Clarifications in Health IT Development: UHG seeks clarifications on the definitions related to self-developed health IT and the conditions under which health care providers are not considered health IT developers.

Exception for Trusted Exchange Framework: UHG supports the new exception for entities in the Trusted Exchange Framework under certain conditions.

FHIR Standards and USCDI Adoption: They express concerns about the transition complexity and timeline for adopting new FHIR standards and migrating from USCDI v1 to v3.

Predictive Decision Support Intervention (DSI): UHG suggests narrowing the definition of predictive DSI to clinical uses and advocates a risk-based approach to classifying DSIs.

Intervention Risk Management (IRM): There's a request for clarity on the proposed IRM structure and its relation to FDA regulations.

Disclosure Requirements: UHG seeks clarification on disclosure requirements for certified health IT developers, particularly concerning third-party predictive DSI developers.

Transparency in DSIs: UHG supports more transparency but has concerns about the practicality and risks associated with the requirement to disclose predictive DSI “source attributes.”

Additional Comments and Requests for Information: UHG provides additional comments on various aspects of the Proposed Rule, including the adoption of health IT standards, data segmentation, and lab data standards.

These points reflect UHG's focus on ensuring clear, practical, and risk-based approaches to health IT development and interoperability, with a keen interest in safeguarding against overly broad definitions and burdensome requirements.

User

What is UHG most upset about?  You can pick two topics and briefly explain why you chose them.

ChatGPT

UnitedHealth Group (UHG) expressed significant concerns in two main areas regarding the Proposed Rule by the Office of the National Coordinator for Health Information Technology (ONC):

Disclosure of Predictive Decision Support Intervention (DSI) Source Attributes:

UHG is particularly concerned about the requirement for health IT modules that enable or interface with predictive DSIs to allow end-users to review predictive DSI source attributes and summary information regarding intervention risk management practices. They find the list of required attributes for disclosure to be overly broad and potentially impractical to implement. UHG is worried that this requirement could lead to the sharing of sensitive or proprietary information with end users. They also express concerns that such detailed disclosures could result in user interfaces that are overly complex and designed primarily to meet regulatory requirements rather than user needs​​.

Documentation and Disclosure Requirements for Intervention Risk Management (IRM) Practices:

UHG has reservations about the nature of the information required to be disclosed concerning the IRM practices of certified health IT developers. They are concerned that depending on the level of detail expected by ONC Authorized Certification Bodies (ONC-ACBs), third-party DSI developers might be obliged to share proprietary technical and governance information. UHG urges ONC to provide clarity on the level of detail required in the documentation that demonstrates the employment of IRM practices and to consider the implications of obligating third-party developers to disclose potentially sensitive information​​.

These concerns indicate UHG's apprehension about the potential overreach of disclosure requirements, which could compromise proprietary information and lead to impractical implementation challenges.