The advent of generative AI (Gen AI) solutions, driven by the recent surge in AI capabilities, has sparked considerable interest among healthcare organizations and vendors alike. The healthcare Gen AI field is rapidly advancing, with a plethora of vendors all claiming that their AI features can tackle healthcare challenges.

However, this market saturation complicates the process of navigating potential vendors and determining desired features. There are limited insights about the features of enterprise large language model (LLM) products that can genuinely add value and alleviate existing problems. There is also a lack of guidance for assessing the risks associated with these features. As healthcare leaders scrutinize current market trends and investigate the Gen AI features offered by various vendors to address workflow challenges, they must assess the potential benefits, value, and quality of these features while also considering and mitigating any associated risks.

This buyers guide aims to serve as a valuable resource, offering market and vendor insights to equip healthcare leaders with the knowledge they need to make informed decisions about adopting enterprise LLM solutions.

Sharon Auma-Ebanyat
Research Director, Healthcare Industry
Info-Tech Research Group

Navigating potential vendors and desired features in the fast-evolving healthcare enterprise LLM landscape is challenging. The market is saturated with vendors, all claiming to have AI solutions that will address healthcare challenges.

You need information about features of enterprise LLM products that can drive value and minimize existing pain points.

You also need guidance to evaluate the risks associated with the Gen AI features offered by healthcare solutions.

Common obstacles include:

• Healthcare budget constraints and the lack of a strategic approach to accelerate the adoption
  of AI.
• Lack of understanding of the maturity of various AI features within the complex LLM ecosystem.
• Vendors who are unable to accommodate your way of working in the configuration, implementation, and operational support of their tools.

We review the current context of the healthcare enterprise LLM market to address healthcare challenges and to guide you through your decision-making process. This buyers guide contains:

• Market trends for enterprise LLM adoption.
• Use cases, challenges, and value opportunities to determine what to expect from enterprise LLM solutions.
• Evaluations of key vendors in the market and the features of their LLM solutions.

Market Size (2023-2032)

US$1.45 billion to $21.74 billion

CAGR (2023-2032)

35.14%

Largest Market

North America

Fastest-Growing Market

Asia Pacific

Medical transcription and EHR enhancement

LLMs can convert spoken medical observations into written health records, a process that is traditionally time-consuming and prone to errors. LLMs can analyze and extract valuable insights from vast amounts of electronic health record (EHR) data, which is a source of value for cost savings, efficiency, and patient and clinician experience.

Medical coding and billing

LLMs can help accelerate scientific discovery by formulating, refining, and answering research questions. This is a source of value for cost savings, revenue optimization, operational efficiency, and patient and clinician experience.

Clinical decision support

LLMs can help healthcare professionals make informed decisions by providing accurate diagnoses and treatment plans. This is a source of value for cost savings, revenue optimization, operational efficiency, and patient and clinician experience.

Medical research assistance

LLMs can help accelerate scientific discovery by formulating, refining, and answering research questions. This is a source of value for cost savings, operational efficiency, and clinician experience.

Personalized treatment plans and predictive outcomes

LLMs can help create personalized treatment plans for patients and can predict health outcomes based on patient history and other relevant data. This is a source of value for cost savings, operational efficiency, and patient and clinician experience.

Advanced drug discovery

LLMs can find potential drug treatments at a much quicker pace than traditional methods, with the ability to meticulously examine complex molecular formations, identify potential compounds with healing capabilities, and predict the effectiveness and safety characteristics of these prospects. This is a source of value for cost savings.

LLMs can generate plausible but incorrect content. This can be harmful, especially in medical advice or clinical decision-making. As LLMs advance, these hallucinations can become more convincing, requiring healthcare professionals to verify the generated information due to the models’ lack of transparency.

The data used for training LLMs can contain private personal information, posing a risk to patients’ privacy. LLMs deployed for biomedical applications can also present privacy risks, as they may have access to patients’ characteristics.

Language models are trained on extensive data, which can lead to the introduction of biases inherent in the data. If training data is biased or contains mistakes, a model might unintentionally propagate these biases or errors in its outputs. Carefully selecting training data and continuously detecting and mitigating bias are crucial steps.

The use of AI in medicine raises legal and ethical concerns, necessitating a robust legal framework. The acknowledgment of ChatGPT as an author in biomedical research has been identified as an ethical concern, raising questions about accountability, copyright, and the disclosure of LLM usage.

Comprehensive evaluation of LLMs is crucial but challenging. While some traditional NLP tasks have reliable automatic evaluation metrics, evaluating free-text LLM outputs is labor-intensive and not scalable. Therefore, it is important to arrive at a reporting consensus for evaluating biomedical LLMs and to design evaluation metrics that are both scalable and accurate.

15%

User Acceptance Testing (UAT) for Functional Modalities

How well does the vendor align with your top-priority modality requirements? What is the vendor’s functional breadth and depth across modalities? Are the modalities unimodal or multimodal? Does the modality proficiency satisfy end-user testing?

15%

Affordability

How affordable is the vendor? Consider a three-to-five-year total cost of ownership (TCO) that encompasses not just licensing and prompt costs, but also implementation, integration, training, and ongoing support costs. Do you need to bring in specialized labor to implement and maintain the solution (e.g. full-time equivalent/FTE developers or BI analysts)?

15%

Architectural Fit and Infrastructure

How well does the vendor align with your direction from an enterprise architecture and security perspective? How interoperable is the solution with existing applications in your technology stack? Does the solution meet your deployment model preferences? To what extent do you need to ensure that your organization has quality data for fine-tuning the solution?

15%

Scalability

How easy is it to expand the solution to support increased user, data, and/or customer volumes? Are there any capacity constraints in the solution? Do you need additional in-house resources to ensure optimal processing speeds during high-volume events? (For example, do you require additional GPUs?) How might this factor into affordability?

15%

Vendor Viability

How viable is the vendor? Is the vendor an established player with a proven track record, or a new and untested entrant into the market? What is the financial health of the vendor? How committed is the vendor to its current product roadmap? Does this align with your expectations?

10%

Vendor Vision

Does the vendor have a cogent and realistic product roadmap? Is the vendor making sensible investments that align with your organization’s internal direction?

5%

Emotional Footprint

How well does the vendor’s organizational culture and team dynamics align with yours?

10%

Third-Party Assessments and/or References

How well-received is the vendor by unbiased, third-party sources? For larger projects, how well does the vendor perform in reference checks (and how closely do those references mirror your own situation)?

Ambient AI Scribe Solutions

Ambient AI scribe solutions aim to document physician-patient conversations. They use AI to understand dialogue, extract information, and create accurate notes. The solutions also handle tasks such as retrieving EHRs, ordering tests, and scheduling appointments. This reduces the burden of documentation, improves workflow, and addresses challenges such as physician burnout.

AI-Powered Revenue Cycle Management Solutions

AI revenue cycle management solutions aim to streamline healthcare financial processes. They use AI to increase accuracy and efficiency in tasks such as claims processing and patient eligibility verification. RCM solutions also automate repetitive tasks, allowing staff to focus on higher-value tasks. Overall, they enhance revenue capture, reduce costs, and improve patient satisfaction.

Overview
DAX Copilot is a HITRUST CSF–certified solution built on Microsoft Azure and a long-standing healthcare dictation software solution in the healthcare industry.

AI Features

• Is integrated with LLM GPT-4, which transcribes conversations using natural language processing (the same technology used by speech-to-text-software).
• Was trained on real clinical data from 1 billion minutes of medical dictation and 10 million unique ambient encounters from data stored in Nuance software; has multiple models to optimize speed and accuracy and protects data from any misuse of these large language models.
• Can capture a multi-party conversation ambiently with an AI-powered voice-enabled solution (e.g. can captures clinician visit conversations at the point of care); uses advanced conversational and ambient Gen AI in a mobile application to create high-quality documentation. automatically by converting multi-party conversations after a patient encounter into structured clinical notes, integrated with existing clinician workflows.
• Integrates seamlessly with electronic health records, using Dragon Medical One for easy implementation.
• Offers customizable templates, which enable clinicians to customize their note templates to streamline workflows tailored to their needs.

Market Analysis

• Started with integration in EPIC EHR and recently integrated in MEDITECH solutions and 200+ EHRs.

Responsible AI

• Has enterprise-grade security, provided by Microsoft, to protect data within its environment – something specialized solutions cannot yet offer.

Licensing & Pricing

• Opaque (custom information available from vendor).

Quality Assessment Considerations

• Accurate
• Thorough
• Useful
• Free from hallucinations and bias
• Internally consistent
• Synthesized
• Succinct
• Comprehensive
• Organized

Overview
DeepScribe is an enterprise-grade AI medical scribe that can be fully customized by clinical specialty groups and individual clinicians.

AI Features

• Automates clinical note-taking by converting patient conversations into customizable clinical chart notes.
• Has seamless EHR integration of clinical notes, with no uploads or attachments required.
• Integrates coding with automatically generated ICD-10 codes for every patient encounter, with the capability to maximize coding accuracy by suggesting ICD-10 codes to include in billing and a risk-adjusted factor score that maximizes revenue.
• Has a notes customization studio with 50+ prebuilt specialty-specific customizable templates.
• Includes voice-activated macros that populate when certain words or phases are heard, similar to dot phrases in EHRs.
• Uses AWS SageMaker to train and host its LLM securely.
• Is health-AI trained on over 3 million clinical conversations in partnership with AWS HealthScribe.

Market Analysis

• Has formed a strategic alliance with Amazon Web Services (AWS) to securely and rapidly scale LLMs in the healthcare sector.
• Has specialized LLMs, which were trained on one of the most extensive labeled datasets in healthcare and act as a catalyst for scaling ambient AI in clinical environments.
• Offers level of customization that sets DeepScribe apart from its competitors.
• Outperforms GPT-4 by 32% on medical scribing.

Responsible AI

• Ensures enterprise-level security, including compliance with HIPAA, end-to-end encryption, anonymized patient data, multifactor authentication, and restricted user access.
• Has transparent design, allowing AI-generated notes to be traced back to original patient conversation within an encounter.
• Has audit dashboard for notes to monitor note accuracy, frequency, or edits and a trust score.

Licensing & Pricing

• Opaque (custom information available from vendor).

Quality Assessment Considerations

• Accurate
• Thorough
• Useful
• Free from hallucinations and bias
• Internally consistent
• Synthesized
• Succinct
• Comprehensive
• Organized

Overview
Heidi Health is an Australian-based AI medical scribe platform that assists clinicians by automating the creation of clinical notes during patient visits. It is designed to enhance the efficiency and accuracy of clinical documentation.

AI Features

• Automatically generates notes from voice prompts and builds a detailed patient clinical history.
• Has multiple AI note templates to select from.
• Has customized note structure, which allows users to teach their AI exactly how they like their notes structured, and even save templates for reuse; also has capabilities for making additions and addendums to notes and templates.
• Can generate letters and forms for patient handover.
• Has multilingual prompts, which can generate content in multiple languages.
• Has adaptive voice capabilities, where AI increasingly resembles user’s distinct style with every visit, addition, and modification.

Market Analysis

• Is now used by more than 100 general practitioners in about 30 clinics throughout Australia.
• Is based in Melbourne, with plans to expand around the world.

Responsible AI

• At present, safely keeps all information in Australia, complying rigorously with international norms, such as HIPAA, GDPR, PIPEDA, PHIPA, and PIPA.
• Has advanced processing and storage of LLM data in privately hosted servers that are ISO27001 and HIPAA compliant.
• Has specialized and secure data handling so that no recordings of patient information are stored.
• Ensures users transparency and responsibility, with full control over their data and with rights to access, correct, or raise concerns.

Licensing & Pricing

• Free version for basic note-generation features.
• Pro version at US$69/month with a 7-day trial.
• Clinic version for multi-clinician consults at US$99/month.

Contact vendor for updated and custom pricing.

Quality Assessment Considerations

• Accurate
• Thorough
• Useful
• Free from hallucinations and bias
• Internally consistent
• Synthesized
• Succinct
• Comprehensive
• Organized

Overview
Regard is an AI tool that helps clinicians analyze patient data and make faster, more informed decisions. It extracts insights from unstructured clinical data to provide relevant information quickly.

AI Features

• Creates possible diagnoses based on patient data using Gen AI model.
• Generates clinical notes based on voice input or patient data.
• Offers EHR integration with relevant information and a simple interface.
• Also offers rigorous documentation auditing against clinical guidelines.
• Helps clinicians quickly generate notes using autocomplete feature.

Market Analysis

• Stays competitive in the market by advancing LLM capabilities to create an interactive experience that can interact with Regard's AI clinical assistant to perform tasks such as requesting discharge summaries and diagnostic history.
• Has partnered with OpenAI to develop new chatbot functionalities.

Responsible AI

• Has a comprehensive auditing process for clinicians to review outputs in adherence with clinical guidelines with a human in the loop.
• Uses a wide range of data to mitigate any potential biases and improve accuracy, reliability, and diagnostic capabilities.

Licensing & Pricing

• Demo-based custom pricing model.
• Opaque (more specific pricing information available from vendor).

Quality Assessment Considerations

• Accurate
• Thorough
• Useful
• Free from hallucinations and bias
• Internally consistent
• Synthesized
• Succinct
• Comprehensive
• Organized