Welcome to HIMSS

This site uses technologies such as cookies to provide a better user experience by personalising content and ads, analysing web traffic and trends, and improving site operations. We may share information about your use of the site with third parties in accordance with our Privacy Policy. By continuing to use this site you agree that we can save cookies on your device, unless you have disabled them. You can change your cookie settings at any time by visiting our Cookie Policy, but parts of our site may not function correctly without them.

Skip to main content
Email

The latest news in Healthcare IT – straight to your inbox.

Home
  • Main Menu
  • Subscribe
  • Topics
    • Video
    • Analytics
    • Artificial Intelligence
    • Cloud Computing
    • EHR
    • Government & Policy
    • Interoperability
    • Patient Engagement
    • Population Health
    • Precision Medicine
    • Privacy & Security
    • Telehealth
    • Women In Health IT

Regions

  • ANZ
  • ASIA
  • EMEA
  • Global Edition
Global Edition
AI & ML Intelligence

Provider organizations need AI trainers to ensure quality outcomes

Even the best algorithm could still need months to verify its performance, a healthcare systems expert says. So IT leaders should ensure they budget for the training process whenever they evaluate any artificial intelligence technology.
By Bill Siwicki
February 08, 2024
11:12 AM

Matt Hollingsworth, cofounder and chief innovation officer at Carta Healthcare, with his mother, his inspiration for starting the company

Photo: Matt Hollingsworth

This year, healthcare data experts will also begin to take on a new role as artificial intelligence trainers, contends Matt Hollingsworth, cofounder and chief innovation officer at Carta Healthcare, a healthcare AI systems company.

"Although the adoption of AI in healthcare is nothing new, there will continue to be a growing need for AI technology in 2024 and beyond," he said. "With an overall lack of manpower in healthcare, as seen in nursing and staff shortage trends, AI looks like the best solution for retaining existing manpower at competitive compensation rates while increasing efficiency in workflow and improving clinician job satisfaction.

"According to the National Library of Medicine, the key to successful AI implementation is to do it in a clinically relevant way that clinical caregivers can get behind," he continued. "It's not only about the technology, it's about how technology and caregivers work together in a trusted way to believe in, train and commit their AI solutions to provide long-term value."

We interviewed Hollingsworth to better understand his beliefs on the need for AI trainers in healthcare.

Q. You say healthcare provider organizations today need artificial intelligence trainers. Why?

A. Fundamentally, it is because no class of AI produces high enough quality output for any given task to trust until it has been verified to perform that specific task well. To perform said verification, you need subject matter experts – and we call these people AI trainers.

Let's make this concrete. Imagine you want to build a system that will chat with patients to answer their clinical questions about diagnoses they've received (like a chatbot WebMD). In principle, generative AI could do that. Here is what happens when you ask ChatGPT to give you some details about a clinical diagnosis:

The first answer is entirely wrong and would mislead any patient who received it. The second answer is perfectly fine. Before you send your product out into the wild, answering patients' questions, you have a question that you absolutely must answer: How often is it right, and how often is it wrong?

Once you have that answer, you must decide whether it is good enough to help solve your problem or if it will cause more harm than good. How do you do that? In this case, you ask many questions and then have an "AI trainer" verify the output and score it based on accuracy. Then you take those findings and decide whether or not it's good enough. Unless you don't care whether your product works, there isn't any way around this – someone needs to check the veracity of the output.

Today, no single "generalized AI" algorithm can take any problem and perform it at a human level. So, for any class of AI, you choose – in the example above, it's a generative text AI model, ChatGPT – you must verify its performance against a specific, quantifiable problem before knowing if the algorithm will add value to solving your problem. We call the people who do that verification process "AI trainers."

Q. What does the role of an AI trainer look like? What exactly do they need to be doing?

A. The role of an AI trainer is multifaceted and involves critical evaluation of an AI algorithm's outputs using real-world data. This professional assesses whether the AI's performance aligns with expected outcomes and accuracy. The scope and methods of an AI trainer's work depend highly on the AI algorithm's specific application.

For example, in scenarios where the AI algorithm is tasked with responding to patient inquiries about medical diagnoses, the AI trainer must evaluate the responses for their relevance and correctness. This involves comparing the AI's answers with verified medical information to ensure accuracy.

The AI trainer's role becomes more intricate in more complex applications, such as when an AI algorithm is designed to estimate blood loss during surgery through image analysis. Here, they must measure the blood loss independently and then compare these measurements with the AI's estimates, ensuring the AI's precision in real-time medical situations.

Similarly, suppose the AI is involved in summarizing clinical documentation. In that case, the AI trainer must verify the AI-generated summaries are comprehensive and reflect the key points of the actual documents. This involves a detailed comparison between the AI's output and the original clinical records.

Lastly, in cases where the AI assists in detecting missed billing codes, the AI trainer's job is to confirm the codes suggested by the AI are relevant and applicable. They must cross-reference the AI's suggestions with the medical services provided, ensuring billing is accurate and comprehensive.

In summary, an AI trainer's role is crucial in validating and refining AI algorithms across various domains, ensuring the AI's output is technically correct, practically applicable and reliable in real-world scenarios.

Q. What titles or roles that exist today at hospitals and health systems need to take on the responsibilities of AI trainer, and why them?

A. In implementing AI in hospitals and health systems, the roles that would be best suited to take on the responsibilities of an AI trainer are those professionals who already possess deep subject matter expertise in the specific tasks the AI is designed to perform. However, it's important to note these professionals would need additional training in AI to bridge the gap between their domain expertise and the technical aspects of AI effectively.

Here's a breakdown of specific roles and why they are suitable.

Q&A Bot – The ideal AI trainer for an AI handling patient questions about diagnoses would be a doctor. Doctors have the necessary medical knowledge and experience to assess the accuracy and appropriateness of AI-generated responses. Their expertise in diagnosis and patient communication is crucial for ensuring the AI provides medically accurate and contextually relevant answers.

Blood Loss Estimation – An operating room nurse is well-placed to train the AI. OR nurses have firsthand experience in surgical settings and are skilled in assessing patient conditions during surgery, including estimating blood loss. Their practical knowledge is vital for AI training to analyze images and estimate blood loss accurately.

Clinical Summary – Doctors, physician assistants or nurse practitioners could effectively manage this task. These professionals are experienced in creating and interpreting detailed clinical documentation. Their expertise is essential to ensure AI-generated summaries of clinical documentation are accurate and include all critical medical information.

Billing Coding – A coding specialist is the most appropriate choice for training an AI in billing coding. Coding specialists comprehensively understand medical billing codes and their application in various healthcare scenarios. Their role in ensuring accurate and efficient billing aligns with the AI's purpose, making them ideal for training and overseeing the AI in this area.

In each of these cases, the selected professionals already have the domain knowledge and experience in the tasks that AI aims to automate or assist. The additional requirement for them to be effective AI trainers is a foundational understanding of AI principles and operations. This knowledge can be acquired through specialized training, enabling them to bridge their subject matter expertise with the technical nuances of AI algorithms and applications.

Q. When vendors with AI in their systems are involved, who should the AI trainers be working with and how should they be working as a conduit between vendors and users?

A. This highly specialized process has only been around at scale for a few years now, so it's unlikely hospitals have these people just sitting around yet. At least in the near future, the most common model will be having the vendors with AI in their systems provide the trainers necessary to implement their product via a services contract, most likely during the technology implementation.

These trainers will need to be subject matter experts both in the task at hand and the AI algorithms themselves, and having the latter experience will be infinitely easier if the trainer is an employee of the company that made the algorithm in the first place.

Generally, this will take the form of a hospital-employed AI trainer working with a larger team of vendor-side AI trainers during the implementation phase of any given project. The hospital-side trainer will be setting performance requirements, sanity-checking the output, and doing spot checks to ensure they trust the output of the process.

The vendor-side folks will be doing the grunt work of collecting enough example statistics to get certainty on the performance and working with the rest of the team to address any shortcomings that surfaced during the implementation process.

Q. What would you say are a couple big-picture issues AI trainers should be addressing with healthcare AI users?

A. The most important thing is to remind everyone we don't currently have – and likely will not have in our lifetimes – a generalized AI algorithm that can do any task we throw at it, so this training process is essential.

Suppose a given task/AI algorithm pair doesn't have the trainer's blessing in the form of a quantified measurement of accuracy for a given task in real life at the institution where it is being deployed. In that case, the users shouldn't use that AI tool. My earlier GPT example is an excellent example of why that is. GPT is great at answering GRE questions but not so great at answering clinical questions.

The blind application of any AI system to a task is a sure formula for disaster.

Budget for this training if you're buying an AI solution. Fundamentally, this is similar to a clinical trial for a medical device. We would never use an EKG machine whose manufacturer didn't verify its accuracy. Similarly, we should only use AI algorithms once we've verified they work.

The primary difference is that, unlike patient physiology, AI algorithms' data varies wildly from institution to institution based on their IT infrastructure and documentation practices. And these algorithms are not sentient. They can't just correct themselves magically because your EHR documents weight in kg when the place where the algorithm was trained documented weight in lbs.

That means any algorithms touching data in any EHR or similar system must be verified at each institution rather than globally. As a result, this is often a labor-intensive process, and people expecting this to happen overnight will be disappointed.

The best AI algorithm could still need months to verify its performance. As such, leaders should ensure they budget for the training process whenever they evaluate any AI solution. If they expect to "turn on the AI" and it will add value, they will be disappointed 100% of the time.

Follow Bill's HIT coverage on LinkedIn: Bill Siwicki
Email him: bsiwicki@himss.org
Healthcare IT News is a HIMSS Media publication.

Topics: 
Artificial Intelligence, Quality and Safety, Workforce

More regional news

Patient does a virtual care consult

Castlight Health intros virtual urgent care for members

By
Mike Miliard
April 18, 2025
HIMSSCast logo

HIMSSCast: Should every healthcare organization have an AI strategy?

By
Mike Miliard
April 18, 2025
Nurse checks tablet to communicate on shift

Zoom launches agentic AI-powered mobile comms for frontline staff

By
Andrea Fox
April 18, 2025
Want to get more stories like this one? Get daily news updates from Healthcare IT News.
Your subscription has been saved.
Something went wrong. Please try again.

Top Story

Nurse checks tablet to communicate on shift
Zoom launches agentic AI-powered mobile comms for frontline staff

Most Read

How UCHealth is reducing fall injuries with AI-enhanced risk modeling
2025: AI enhances personalized care; caregiver experience in the spotlight
Frontera launches with $32M in seed funding
Korea University Medical Center pursues brain, heart AI development and more briefs
Roundup: AI and cloud tackle cyber risk and improve workflows
HIMSSCast: Fundamentals of data governance - lessons from UNC Health, part 1

Research

White Papers

More Whitepapers

Telehealth
Create secure, connected omnichannel communications
Telehealth
Let us guide you to HIPAA compliance
Cloud Computing
How a cloud communications platform puts connection at the center of care

Webinars

More Webinars

Analytics
Standby Eligibility and Claims Solutions: Diversify Your Risk & Ensure Business Continuity
Interoperability
Nursing Leadership, Operational Innovation, and Emerging Technologies with AONL
Artificial Intelligence
Loving the AI Revolution: How Automation is Humanizing Healthcare and Improving Provider Well-Being

Video

Ilir Kullolli, Stanford Medicine Children's Health_Las Vegas skyline Photo by halbergman/E+/Getty Images
HIMSS-ACCE working together to advance digital health
Vik Bajaj, Foresite Labs_Medical research Photo by Edward Jenner/pexels.com
Healthcare research is being affected by federal budget cuts
Priyanka Jain, Evvy_Hand holding sample vial Photo courtesy of Evvy
How one women's health startup tests fertility outcomes
Keisuke Nakagawa, UC San Diego Health_Las Vegas skyline Photo by halbergman/E+/Getty Images
Can technology help bring the human touch back to medicine?

More Stories

Lee Kim, HIMSS_Las Vegas skyline Photo by halbergman/E+/Getty Images
Past year's data breaches often stemmed from remediable cybersecurity gaps
Cathy Menkiena, Health Catalyst_Las Vegas skyline Photo by halbergman/E+/Getty Images
Innovative – and useful – tech is key to empowering care teams
Sameer Sethi of Hackensack Meridian Health on AI
Hackensack Meridian Chief AI Officer on the intersection of business and technology
Doctor checking and tracking information on a computer
HHS updates regulatory guides for the safe use of EHRs
Sameer Sethi, Hackensack Meridian Health_Computer neural network concept Photo by dan/Moment/Getty Images
Chief AI Officer on becoming one and working with the C-suite
Businessperson signing piece of paper
White House releases guidance on federal AI use and procurement
Dr. Ateev Mehrotra of Brown University School of Public Health on telehealth policy
Brown University policy expert talks about the future of telehealth flexibilities
Micky Tripathi, former HHS acting chief AI officer
Former National Coordinator headed to Mayo Clinic, reports say
Home

More News

  • MobiHealthNews
  • Healthcare Finance News
  • Healthcare Payers News

Newsletter Signup

HIMSS25 European Health Conference & Exhibition
HIMSS25 European Health Conference & Exhibition
Get ready for knowledge-sharing, all the latest innovations, and in-depth demos with Europe's most influential healthcare community.
10 - 12 June, 2025 | Paris
Learn More
AI in Healthcare Forum
AI in Healthcare Forum
The HIMSS AI in Healthcare Forum cuts through the hype to showcase real-world examples illustrating the transformative potential, and realistic challenges of AI application across the care continuum.
10 - 11 July 2025 | New York
Learn More

Footer Menu

  • About
  • Advertise
  • Reprints
  • Contact
  • Privacy Policy

© 2025 Healthcare IT News is a publication of HIMSS Media

X

Topics

  • Video
  • Analytics
  • Artificial Intelligence
  • Cloud Computing
  • EHR
  • Government & Policy
  • Interoperability
  • Patient Engagement
  • Population Health
  • Precision Medicine
  • Privacy & Security
  • Telehealth
  • Women In Health IT

Career

  • Events
  • Jobs
  • Research Papers
  • Webinars

More

  • About
  • Advertise
  • Contact
  • Special Projects
  • Video

Regions

  • ANZ
  • ASIA
  • EMEA
  • Global Edition

The Daily Brief Newsletter

Get daily news updates from Healthcare IT News.

Search form

Top Stories

Nurse checks tablet to communicate on shift
Zoom launches agentic AI-powered mobile comms for frontline staff
HIMSSCast logo
HIMSSCast: Should every healthcare organization have an AI strategy?
Vik Bajaj, Foresite Labs_Medical research Photo by Edward Jenner/pexels.com
Healthcare research is being affected by federal budget cuts