Skip to main content
Email

The latest news in Healthcare IT – straight to your inbox.

Home
  • Main Menu
  • Subscribe
  • Topics
    • Video
    • Analytics
    • Artificial Intelligence
    • Cloud Computing
    • EHR
    • Government & Policy
    • Interoperability
    • Patient Engagement
    • Population Health
    • Precision Medicine
    • Privacy & Security
    • Telehealth
    • Women In Health IT

Regions

  • ANZ
  • ASIA
  • EMEA
  • Global Edition

Women In Health IT

Free Newsletter
Global Edition
Artificial Intelligence

Responsible AI governance is needed now, says UNC Health chief analytics officer

Rachini Ahmadi-Moosavi explains why planning and guardrails are so necessary as artificial intelligence becomes embedded in an array of clinical, financial and operational use cases – and explains how they should best be applied.
Bill Siwicki
July 05, 2023

Rachini Ahmadi-Moosavi, chief analytics officer at UNC Health

Photo: UNC Health

In the December 2022 AI & Analytics Study of 250 health system leaders, the Advisory Board depicted how artificial intelligence had shifted from transformational in nature (as reported in their 2018 study) to merely adding incremental value to healthcare.

The incremental value in healthcare came in the form of narrow AI, focused on solving individual problems – early detection of sepsis, falls risk, clinical documentation improvement, identifying best candidates for care, among others. Healthcare historically has lagged other industries in technology adoption, resulting in the 2018 AI hype fizzling down to incremental as opposed to transformational value, the Advisory Board concluded.

By January 2023, however, the buzz around AI became reinvigorated. OpenAI released ChatGPT to the world in November 2022. The ease of accessibility to and application of so-called large language models and natural language processing has led AI to the point of revolutionizing how everything, including healthcare, operates.

"However, as AI becomes more computationally complex and integrated into new and existing technology, the need for thoughtful and responsible AI governance becomes an imperative to ensure human-centered AI adoption that is ethical and trustworthy," said Rachini Ahmadi-Moosavi, chief analytics officer at UNC Health, a health system based in Chapel Hill, North Carolina.

Healthcare IT News sat down with Ahmadi-Moosavi to get her expert advice on what health IT leaders need to know – today – about responsible AI governance.

Q. How would you define responsible AI governance? Generally speaking, what is it and what problems or potential problems is it trying to solve?

A. UNC Health has developed homegrown AI capabilities since 2016. In the early stages of developing this capability, we self-governed AI within the solution teams and analytics organization.

However, as artificial intelligence pushes our organization to move faster and further into AI technology development and partnerships, the need to fully vet AI capabilities grows. Microsoft CEO Satya Nadella was recently quoted regarding the application of AI that “what we should do is maximize the benefits and mitigate the challenges.” Responsible AI governance aims to accomplish both goals.

By leveraging the expertise of leaders across our health system and outside partners, like Microsoft, UNC Health developed its own Responsible AI Framework. The framework defines four pillars of AI governance: fairness, transparency, accountability and trustworthiness.

Responsible AI governance ensures both built and bought AI technologies meet certain expectations: evaluations for bias, moral and ethical appropriateness, visibility beyond “the black box,” accountability for continued investment in models, model reliability and tuning, education about safe and effective adoption of the technology, and more.

In addition to a well-defined, human-centered framework, UNC Health is standing up an AI & Automation Advisory (AAA) workgroup to ensure that a multidisciplinary group of experts, clinicians and leaders across the health system will continue to vet these transformational capabilities.

Mitigating the risks and maximizing the benefits of AI adoption requires that scrutiny over the technology provides common metrics before the implementation and throughout the lifecycle of its deployment. Developing AI governance KPIs to track these key risks and benefits enables the organization and the AAA workgroup to continuously learn, monitor and improve the technology.

Q. More specifically, what kind of responsible AI governance is needed in the operational arena of healthcare?

A. AI can be leveraged in healthcare operations to improve patient call center experiences, automate appointment scheduling, drive efficiency in hospital throughput, and delivering effective patient communication.

Responsible AI governance holds the technology accountable to safe and enhanced healthcare operations by ensuring bias is not introduced in risk models and generative AI solutions. This further enables fairness and consistency in recommendations impacting patients and teammates and builds trust by the people who monitor or interact with the technology.

Q. What kind of responsible AI governance is needed in the clinical area of healthcare?

A. AI may be used in clinical care to summarize patient encounters in pre-visit preparations, alert clinicians to patient risk, draft nursing and provider documentation based on ambient listening devices and information from the electronic health record, spot anomalies in imaging scans, and connect risk of disease to testing, treatment and outcomes.

The risk is much greater in the clinical space due to the direct impact on patient health and wellbeing. Adoption of AI in this space must avoid previous pitfalls related to clinical bias (racial bias in transplant recommendations, gender bias in screening for liver disease, skin color bias in diagnosis, etc.); ensure that ethics are a cornerstone of care delivery; vet model inputs and reliability; and thoroughly test recommendations.

Clinicians will need to be engaged in the governance of clinical AI to earn their trust in the technology. Additionally, when aiming at improving health outcomes, reducing burnout and ameliorating staffing challenges, monitoring and evaluating the impact on these desired outcomes will be key for effective long-term adoption.

Q. And what kind of responsible AI governance is needed in the financial/administrative sphere of healthcare?

A. In the financial and administration realm, AI may help bend the cost curve, increase timely revenue attainment and improve efficiencies.

Many processes in revenue cycle management – streamlining prior authorizations, connecting self-pay to philanthropic aid services, fine-tuning registration, providing patient estimates, preventing and adjudicating denials, processing payment posting and write-offs – are touted to see great potential benefit from AI and automation. Similarly, supply chain, finance, human resources and other administrative areas likely can benefit from AI.

The accuracy of information surfaced and used in these processes is critical. Ensuring AI is well understood in terms of what it’s consuming and how recommendations are presented also is necessary in this space. With the sensitive data in this sphere, tracking anomalies and presenting findings through the right channels is critical.

Follow Bill's HIT coverage on LinkedIn: Bill Siwicki
Email him: bsiwicki@himss.org
Healthcare IT News is a HIMSS Media publication.

Topics: 
Artificial Intelligence, Quality and Safety, Women In Health IT

Women In Health ITResource Center

Handshake at business meeting
Taking a Seat on the Board
View More Resources

Stay Informed

Subscribe today to receive our FREE monthly e-newsletter

Have valuable and interesting information to share?

contribute today!

Featured Contributors

Susan Morse

Susan Morse is Executive Editor of Healthcare Finance and Women in Health IT contributor. You can follow her at @susanmorseHFN

Bill Siwicki

Bill Siwicki is Managing Editor of Healthcare IT News. Bill has 36 years of experience in journalism, with more than 25 years experience in healthcare IT.

Jessica Hagen

Jessica Hagen is an Editor at MobiHealthNews. She covers the growth of digital health, including the medical extended reality (XR) industry, and reports in-depth on the pharmaceutical and biotechnology sectors. As a hobby, Jessica co-hosts a podcast with Dr. Kim Bullock, a medical XR pioneer at Stanford University, about immersive technology use in psychiatric care. She received a bachelor's degree in English with a minor in Journalism from California State University, Northridge.

Want to get more stories like this one? Get daily news updates from Healthcare IT News.
Your subscription has been saved.
Something went wrong. Please try again.
Home

More News

  • MobiHealthNews
  • Healthcare Finance News
  • Healthcare Payers News

Newsletter Signup

HIMSS25 European Health Conference & Exhibition
HIMSS25 European Health Conference & Exhibition
Get ready for knowledge-sharing, all the latest innovations, and in-depth demos with Europe's most influential healthcare community.
10 - 12 June, 2025 | Paris
Learn More
AI in Healthcare Forum
AI in Healthcare Forum
The HIMSS AI in Healthcare Forum cuts through the hype to showcase real-world examples illustrating the transformative potential, and realistic challenges of AI application across the care continuum.
10 - 11 July 2025 | New York
Learn More

Footer Menu

  • About
  • Advertise
  • Reprints
  • Contact
  • Privacy Policy

© 2025 Healthcare IT News is a publication of HIMSS Media

X

Topics

  • Video
  • Analytics
  • Artificial Intelligence
  • Cloud Computing
  • EHR
  • Government & Policy
  • Interoperability
  • Patient Engagement
  • Population Health
  • Precision Medicine
  • Privacy & Security
  • Telehealth
  • Women In Health IT

Career

  • Events
  • Jobs
  • Research Papers
  • Webinars

More

  • About
  • Advertise
  • Contact
  • Special Projects
  • Video

Regions

  • ANZ
  • ASIA
  • EMEA
  • Global Edition

The Daily Brief Newsletter

Get daily news updates from Healthcare IT News.

Search form

Top Stories

Nurse checks tablet to communicate on shift
Zoom launches agentic AI-powered mobile comms for frontline staff
HIMSSCast logo
HIMSSCast: Should every healthcare organization have an AI strategy?
Vik Bajaj, Foresite Labs_Medical research Photo by Edward Jenner/pexels.com
Healthcare research is being affected by federal budget cuts