How Boards Should Engage in AI Decision-Making

00:00:00:03 - 00:00:19:02
Tom Haederle
Welcome to Advancing Health. In recent years, hospital trustee boards have had a new and extremely important responsibility added to their oversight portfolio: the integration of AI. Success depends on understanding the basics and knowing what questions to ask.

00:00:19:04 - 00:00:43:27
Rebecca Chickey
Hello, my name is Rebecca Chickey and I am the vice president of Behavioral Health and Trustee Services at the American Hospital Association. And it is my honor today to be joined by Ajay Gupta. He is bringing two hats to this conversation today. And so, Ajay, if you will, go ahead and tell the listeners what two hats you're bringing to the table.

00:00:44:00 - 00:01:06:28
Ajay Gupta
Rebecca is a pleasure to be here. Thank you for inviting me to be on the show today. My name is Ajay Gupta, as you mentioned, and I have the privilege of serving as this year as the board vice chair for Eternity Health Mid-Atlantic, which includes Trinity Health hospitals across Pennsylvania, Maryland and Delaware, and previously was on the board and board chair for Holy Cross Health, which are the hospitals for Trinity in Maryland.

00:01:07:00 - 00:01:30:04
Ajay Gupta
And so we went through a merger, and it's been it's been a great experience going from a still large regional system to an even larger multi-state system. I do, in my day job, run an artificial intelligence company that focuses on the health care space. Artificial intelligence, of course, applies to anyone. And what we do also applies to healthcare and to mission critical and to other organizations.

00:01:30:04 - 00:01:36:26
Ajay Gupta
But our focus is on healthcare, given that's our background, my own background and my partner's background as well.

00:01:37:01 - 00:02:04:01
Rebecca Chickey
So you nailed it. Those are the two hats I wanted you to share, because the focus of our podcast today, for the listeners to know, is really, how should boards be engaging with AI decision making in hospitals and health systems across the country? And this is happening at an incredibly rapid rate, I'm sure you are all aware. And more and more boards are saying, what questions do I need to be asking?

00:02:04:01 - 00:02:32:24
Rebecca Chickey
What do I need to know? What really is my role? What should my role be as hospitals and health systems continue on their AI journey? I want to level set though, because many of the listeners may not know all of the sort of common terms around AI. We use AI generically, but there are different types of AI. So could I ask you to put on your AI hat, tell the listeners what an LLM is, and go on and build beyond there the different types of AI.

00:02:32:26 - 00:03:13:26
Ajay Gupta
You are absolutely right that AI has become an industry term. When we use what we now call AI, we're often using LLMs and other types of artificial intelligence. LLMs are just large language models. These are the ones that have become popular: ChatGPT, Gemini, Claude, which is now very popular. These are large language models. Essentially what they do, or the core of what they do, is having been trained on the sum of human content creation, the internet and beyond, they predict the next word in a sequence of words, and that's really the core of what they do.

00:03:13:27 - 00:03:30:28
Ajay Gupta
They have grown and become far more sophisticated over the several past several years when people have been using them globally, but that's what they are, their models of understanding of the human language. We use them for everything. But that's what an LLM is.

00:03:31:01 - 00:03:35:16
Rebecca Chickey
So what's the next gen of that? I think the term may be agentic.

00:03:35:22 - 00:03:55:09
Ajay Gupta
Yeah, the next generation is always a tough question because we don't know if we're talking about three months in the future or three days in the future. Agentic AI is just an LLM - or just is a bad word, right? It minimizes everything. And when we're talking about artificial intelligence and what we've built in the modern computing world, these are very powerful tools.

00:03:55:12 - 00:04:22:04
Ajay Gupta
Agentic AI is an LLM that is connected to some other piece of software, could be an AI system as well, but it doesn't necessarily have to be that can do something, that can take an action, that can book a flight, right? That can make a hotel reservation, dinner reservation, that can order supplies, right. So it gives you the capacity to use artificial intelligence systems to ask questions, to have a chatbot experience.

00:04:22:04 - 00:04:46:18
Ajay Gupta
But then when you come to some decision point, it can execute that decision on your behalf. That's potentially very powerful, potentially dangerous. There's been a number of stories where unfettered agents have caused havoc on a network, but there have been a lot of positive use cases and increase in productivity at the personal level and company levels as well.

00:04:46:20 - 00:05:01:20
Rebecca Chickey
So given that as a foundation, we have LLMs, agentic, the latter one thinks more on its own, although my son would say they don't really think mom and we've had a conversation about that. Your thoughts on that?

00:05:01:26 - 00:05:28:02
Ajay Gupta
It's a deep question. What is the thinking, right? Are they thinking? They are certainly doing a math problem, right? They're determining what word comes next in a sequence of words at the greatest probability. That's math. That's hard. Right? It's a complex equation. It can be a complex equation. Does that relate to thinking? I don't know. So this is artificial intelligence, right?

00:05:28:03 - 00:05:52:08
Ajay Gupta
The concept of artificial comes from that these are originally mirrored after or patterned after how our human brain thinks. Do we really understand how our human brain thinks when we figure out what we want to say? You're asking me a question and I'm giving you an answer. How is my brain figuring out what the right sequence of words is that's responsive to your question?

00:05:52:10 - 00:06:14:25
Ajay Gupta
It may be figuring out probabilistic decision on a sequence of words. So we don't know. There are a lot of people, if you just go on to the, you know, the podcasting universe, you'll find a lot of interviews with people who are saying that LLMs today are reasoning. They are more advanced than the ones that came out initially, which were really just doing probabilistic math.

00:06:14:25 - 00:06:38:03
Ajay Gupta
And now there's some reasoning going on. I would say, though, from the perspective of hospital boards and health systems, we need to know what they can do for us. Whether they're reasoning, whether their level of intelligence is equivalent to or better than humans. Honestly doesn't matter. Let's figure out what they can do for us in terms of our ability to operate health systems and care for our patients.

00:06:38:06 - 00:07:08:02
Rebecca Chickey
Well, you've led me to my next question. I'm going to lay the foundations here. This is happening rapidly. AI is growing, changing, evolving, as are the applications of AI, not just in healthcare, but across the country, but particularly in hospitals and health systems. So what questions should board members be asking? What information should they be seeking to help shape the discussions that are being had at the board level around AI?

00:07:08:09 - 00:07:15:26
Rebecca Chickey
I've given you two questions there. I'll stop. So what information should they be seeking and what questions should they be asking?

00:07:15:28 - 00:07:37:16
Ajay Gupta
You know, I'd like to say that maybe board members shouldn't ask any different questions than they currently ask in all of the areas where they operate. Because at the end of the day, our role is still strategic advisory. And to help ensure quality, right. Make sure that the organization is sound, is financially sound, is doing the right things in the community and is caring for patients.

00:07:37:18 - 00:08:02:14
Ajay Gupta
AI is a very powerful tool along those lines across the entire spectrum of hospitals operations. AI systems can help both at the bedside as well as operational decisions in the administrative wing of the hospital, right. So to that extent, we should know how are we using AI? What's our vision for using artificial intelligence going forward? I'd suggest we should all ask who owns AI?

00:08:02:16 - 00:08:26:27
Ajay Gupta
I say that because as powerful as it is, as fast as it's moving, it's still fairly new. ChatGPT was released in November of 2022, not even four years ago. And that's really when the common consumer retail version of AI exploded. We were operating before November 2022. There wasn't anybody in October of 2022, in a hospital system in charge of AI necessarily.

00:08:26:27 - 00:08:49:22
Ajay Gupta
Maybe there were a few advanced systems, but this is so new that there may not be an owner designated for AI in a hospital. And that's, I think, the place to start. In order to have a strategy, typically you need to have one owner and they can have a committee to make decisions, but somebody who's going to be the champion. That person, that organization, that that structure probably needs to be defined.

00:08:49:26 - 00:08:56:15
Rebecca Chickey
Do you think there needs to be a subcommittee of the board related to AI?

00:08:56:18 - 00:09:20:04
Ajay Gupta
Sure. I think the way an individual board operates is so unique to the people who serve that board that I'm sure that answer is different for every everyone. I would say, though, the health system probably needs to have a committee or a task force designed to see how AI can help everywhere. I say that even though it's a tool and it is a tool, it's very powerful.

00:09:20:07 - 00:09:33:15
Ajay Gupta
It's not inconceivable to assume that we could take an entire administrative task and have 80 to 90% of the tasks within that job task given over to AI systems, whatever it might be.

00:09:33:18 - 00:09:58:08
Rebecca Chickey
I heard a podcast once that was talking about how there's a lot of equipment in a hospital, and there is information available about when, you know, on average, any one piece of equipment needs to be serviced or it may expire, needs to be replaced, and that AI can help monitor across the inventory of equipment at a hospital and help alert the individuals responsible for that.

00:09:58:08 - 00:10:09:09
Rebecca Chickey
This one needs to be serviced. You know, this one is coming up to its expiration date in a way that is really difficult for a human to monitor. It'd be a massive spreadsheet.

00:10:09:14 - 00:10:32:21
Ajay Gupta
It's a very interesting application, and I don't think that's all that novel, right? So building management companies have been using computerized systems, not necessarily AI, right. This is for some long time ago. But distributed sensors based systems to track the status of buildings for maybe decades. And now we're just saying that that concept can be applied to the equipment that's in the hospital.

00:10:32:21 - 00:10:58:15
Ajay Gupta
And that's true. Hospitals have a lot of heavy equipment. Hospital infrastructure is an expensive infrastructure. Can we use artificial intelligence systems, distributed sensors to track the utilization of those and the status of those systems? We probably should be doing those things. We should be enabling that level of intelligence to staff at throughout a hospital to be able to do their jobs of tracking and maintaining the infrastructure.

00:10:58:20 - 00:11:20:22
Rebecca Chickey
So I have two more questions as we bring the podcast to a close. One, what should the board be involved in related to AI oversight? You mentioned earlier patient safety and quality being one, and where they should not. Because as we are moving through this at a rapid pace, I anticipate that there could be some friction there.

00:11:20:25 - 00:11:22:18
Ajay Gupta
I think the reality is that the board

00:11:22:18 - 00:11:49:04
Ajay Gupta
is there for the strategic advice, right. So I think they should not necessarily get involved when it's coming to a decision between which two vendors to use, right. Do they want to use a system based on open AI or Claude? Maybe that's probably not the right decision, but strategic decisions are important. And the strategic decisions may be, for instance, are we going to use AI first on the clinical side or the administrative side?

00:11:49:07 - 00:12:13:27
Ajay Gupta
That could be a strategic decision. There's risk involved there, right. Obviously everything that we do on the patient side has greater risk and sensitivity. That's our core purpose. Those are areas where the board should be involved. And the board should be involved in not necessarily deciding that, although in some situations maybe that is what is called for, but they should be involved to ensure that the administration and the leadership have a plan to make these decisions.

00:12:13:28 - 00:12:36:04
Rebecca Chickey
Do you also see a role for the board in terms of risk mitigation as part of this whole process? Because I've read that as we add more AI, that also increases the potential for cyber security breaches. Does a board have a role in, you know, asking the question? So if we add this, what are the risk mitigation strategies that you have put in place?

00:12:36:04 - 00:12:45:03
Rebecca Chickey
Or how are we going to put up guardrails to make sure that this is the most effective? Is that a place you see for the board?

00:12:45:08 - 00:13:05:03
Ajay Gupta
Yeah, that's what I was getting back to saying that some of the questions that board will ask for AI are the same questions we ask everywhere. Managing risk or helping the institution manage risk. Ensuring the institution is managing risk is a core board function. So finding out what is the risk exposure to whatever decisions we make on artificial intelligence is important, right?

00:13:05:04 - 00:13:29:19
Ajay Gupta
So many companies in and outside of healthcare have used these tools. I'm sure you've seen them. There's many of them that transcribe a meeting into notes and then send those notes to everybody who participated. Well, that's a risk, right? That you're talking hospital business, and now you have some other agent that is privy to that business. So I'm not saying that one decision is right or wrong, use it or not use.

00:13:29:19 - 00:13:42:08
Ajay Gupta
And I'm certainly not making a selection between all of the tools that are out there. But there should be a process in place. There's a risk exposure there. The board should be on top of knowing that management has thought that through and made a clear decision.

00:13:42:12 - 00:13:50:07
Rebecca Chickey
Wonderful. So my last question to you is what's something that we haven't discussed here that you want the trustees to hear from you?

00:13:50:14 - 00:14:19:04
Ajay Gupta
I think that's incredibly important that we are engaged in artificial intelligence one way or another. And obviously engage in the way that makes the most sense for your institution. That's true. But this is moving fast. It is the next future of the computing world. It's already in the social world, right? People are talking about and using it in every aspect of their lives, and not just the young patients, although definitely young patients, but all patients are using it to care for themselves.

00:14:19:06 - 00:14:40:10
Ajay Gupta
So we have to engage. We cannot avoid the use. We can't just say, let the market figure out what we do, and then we'll be laggards in deploying AI. If we don't engage, we are simply going to have to adopt whatever solutions come about, whether they work for us and our patients or not. So we have to find a way to engage.

00:14:40:10 - 00:14:55:16
Ajay Gupta
And I know that that there's a risk, there's all the risks there. There's also potential cost there, right. So we have to figure out how we're going to manage and carry those costs. How are we going to share those costs to at least with the technology providers who can bring these solutions into our systems?

00:14:55:20 - 00:15:16:27
Rebecca Chickey
Well, thank you so much for sharing your time and your expertise with us here today. Thank you for the work that you're doing with both hats on to close the loop on that. Very grateful. I want to also point the listeners as we close out to AHA Trustee Services has a dedicated web page to AI and what trustees need to know.

00:15:16:27 - 00:15:26:08
Rebecca Chickey
So go to AHA.org/trustees and look for our AI landing page to learn more about this important topic. Thank you so much.

00:15:26:15 - 00:15:29:04
Ajay Gupta
Thank you. Thank you for having me.

00:15:29:07 - 00:15:37:28
Tom Haederle
Thanks for listening to Advancing Health. Please subscribe and rate us five stars on Apple Podcasts, Spotify, or wherever you get your podcasts.