Lessons from a Physician-CIO on AI Governance with Dr. Stacey Johnston
In In this episode of AI Explained, we are joined by Stacey Johnston, MD, Chief Information and Digital Execution Officer at Beacon Health System.
Dr. Johnston brings a rare dual perspective to healthcare AI. She trained as a physician and hospitalist before moving into health informatics, giving her firsthand insight into where technology helps clinicians and where it gets in the way. She discusses how Beacon built its AI governance council from scratch, the policies and vendor risk assessments that gate every new tool, and why requiring a defined ROI before approval has become a forcing function for disciplined adoption. She also shares how agentic AI is already delivering results in scheduling, autonomous benefits verification, and colon cancer screening, why ambient listening drove $10,000 in additional per-physician revenue over 12 months, how clinician trust is earned incrementally through seamless workflow fit and real time savings, and what a federated but centrally monitored AI model could look like as health systems scale.
Introduction & Guest Overview
[00:00:06] Krishna Gade: Welcome and thank you for joining us on today's AI explained. I'm Krishna Gade, the Founder and CEO of Fiddler AI. I'll be your host today. You can put your questions in the q and a box anytime in the fireside chat. Uh, today's session will also be recorded, uh, and send to all the attendees after the session.
[00:00:24] Krishna Gade: Um. We have a very special guest today on AI, and that is Dr. Stacy Johnston, uh, CIO and digital executive officer at, uh, Beacon Health System. And today we are exploring with her on AI in healthcare and, you know, ROI, governance, um, controls, observability, all of those fun things. Welcome Dr. Johnston.
[00:00:51] Dr. Stacey Johnston: All right. Thank you so much.
[00:00:54] Krishna Gade: Dr. Johnston joined Beacon in October, 2024. She most recently served as the vice President, chief Applications Officer for Baptist Health in Jacksonville, Florida. Prior to that, she was the Chief Medical Information Officer. Uh, at, you know, Beaufort Memorial Hospital in South Carolina. During her time in these roles, uh, she was al she'll always maintain an active medical staff privileges, worked as a hospitalist, and she plans to continue this as, as she feels it, as, gives her a unique insight into how technology and care in intersect.
[00:01:25] Krishna Gade: And prior to this, she received her bachelor's degree from Washington College and Doctor of Medicine at Emory. So great to, uh. You know, great background and it's, it's, it's great to see, you know, technologists coming from, you know, different backgrounds and, and, and, uh, very excited to host you here, Dr. Johnston. Um,
[00:01:45] Dr. Stacey Johnston: yeah, again, thanks for having me. And now my camera's working, so we're good now.
[00:01:49] Krishna Gade: Awesome. So, so I guess let's start with that, right? Like you started as a clinician. Uh,
From Clinician to Technologist
[00:01:54] Krishna Gade: what moment made you realize that healthcare needed systems level change and, you know, not just, uh, you know, bedside care?
[00:02:01] Dr. Stacey Johnston: So, honestly, a lot of this actually started straight out of residency.
[00:02:05] Dr. Stacey Johnston: Um, so, you know, back in the day, uh, we were still on paper health, uh, paper records and, um, and then, uh, you know, we were switching over to electronic health records in residency and have, of course, at that time, many attendings were not super excited about it. Of course they gave it to the residents to try to figure it out.
[00:02:24] Dr. Stacey Johnston: And even the residents were raising concerns and saying, you know, one thing I've heard a lot over my tenure in this role is, um, you know, we are the highest paid secretaries out there. And so this idea of putting in our orders, typing them in, um, you know, which just went against everything. A lot of people were trained, you know, to do, um, at that time.
[00:02:47] Dr. Stacey Johnston: And so, um. And so what I realized was either I could buck the system. I mean, this was coming whether we wanted it to or not at this point in time as a regulatory requirement. And so, you know, I could either buck it or embrace it. And so what I realized was if I embraced it, it, it made my life easier. You know, the more I embraced it, the more time I spent in developing.
[00:03:11] Dr. Stacey Johnston: My documentation templates or my order sets just kind of made my workflow easier and therefore made my colleagues' workflow easier. So at that time I was in family medicine and then became an attending as hospitalist. And interestingly enough, I went back to paper. And so then, uh, what led a second round of implementation of paper to an electronic health record by that time.
[00:03:34] Dr. Stacey Johnston: I was the only person in that health system at Beaufort, uh, that had been on, um, any sort of CPOE. So next thing I know, I became the Chief Medical Information Officer and, um, what it, you know, that it was that mindset of, okay, if we spend time in the beginning really evaluating our workflows, um, trying to make them better.
[00:03:56] Dr. Stacey Johnston: You know, by using the systems, how can it, you know, how can it save us time in the long run? And so versus if we don't put what I call care and feeding into the EHR, then it's just gonna, you know, all that pajama time, all of that, you know, that was happening in, in real life. And so I really, um, spent a lot of time on the hospitalist workflows and on their order sets and, um.
[00:04:21] Dr. Stacey Johnston: You know, uh, automation and using Dragon Medical one and the templates, and so really building up our templates. Really building up auto text. And so the more we could, and that, that was the version of automating back then. Mm. You know, it was templates and, um, and even the Dragon Medical one, and then we went to the DMO mobile app.
[00:04:41] Dr. Stacey Johnston: I mean, we're like, oh my gosh, this is amazing. And like AI, AI wasn't even, uh, and you know what thought? Yeah. I mean, we, no one even thought that that was even possible. Um, what, what I realized was. You know, taking the time to make it better for the physicians ultimately ended up making it better for patients.
[00:05:03] Dr. Stacey Johnston: You know, um, that decreasing time in the EHR obviously is, you know, that, that EHR can contribute to burnout and burnout obviously leads to a lesser quality care. And so we really, you know, initially that triple aim. Uh, that you might have heard about is, you know, good quality care, reduced cost and you know, patient, um, more about the patient.
[00:05:26] Dr. Stacey Johnston: And then adding in the quadruple aim was really talking about enhancing the caregiver experience. So I really took that to heart and, you know, making the caregiver experience better. And then now they're talking about the quintuple aim, which is equity in care. So, you know, I still think we have a lot of room to go.
[00:05:44] Dr. Stacey Johnston: Especially in those last two components, you know, enhancing the caregiver experience and then in the equity of care. But I really think that, you know, now. In, in this AI era we are going, we're just getting the glimpse of it, like just a little taste of it, you know? Um, we're doing a lot, you know, radiology's embracing AI pretty well.
[00:06:05] Dr. Stacey Johnston: Um, there are some cool note summarizer type of things, but, um, but we're not full AI in the clinical space yet. And I think once we get there, you know, of course through, you know, change management and appropriate data and trusting of the systems, um you know we're still, I think, you know, maybe a year or two out from that.
[00:06:26] Dr. Stacey Johnston: But once we get there, I think the sky's the limit. And I think the docs are really gonna see time spent in the EHR is decreasing. They're going home on time, you know? They're, they're starting to find joy in medicine, you know, really embracing ambient, but more than just documentation, you know, from the ambient listening, but really doing your full workflow with an ambient solution, I think is, is ultimately where we need to go.
[00:06:50] Krishna Gade: No, it's, it's great to see technology helping, uh, practitioners that way right now it comes to, no, no. So it's an interesting, you know, because you had this two, uh, kind of split level experience, in some ways they're connected. But what do you see, you know, like technology, pure play technologies fundamentally misunderstanding about clinical?
[00:07:09] Dr. Stacey Johnston: Yeah, I, I think ultimately, um, spearheading technology is. It's not a technology issue, it is a workflow issue. It is a change management issue. It's a training issue. And so if you do not understand and embrace the workflows, um, if you don't understand the needs for change management or if you don't really have a good training program, um, around these AI solutions, then you're just gonna put 'em in and they're gonna fail, you know?
[00:07:41] Dr. Stacey Johnston: And so there are gonna be a lot of times, um. Where we, you know, where we implement and it's not meeting our needs. Well, why isn't it meeting our needs? Well, chances are we put in something because, you know, maybe the operators thought it would help solve a problem. But ultimately, you know, one of the things as being in the informatics is, is you learn, you know, to ask a question, what problems are you trying to solve?
[00:08:03] Dr. Stacey Johnston: And maybe the problems of the physicians aren't necessarily aligning with your. Hospital operators. And so you need to make sure that we're in alignment from a system, what problems we truly are trying to solve. And that, you know, if you can get your, you know, your service line leaders, your physicians all to be in alignment and, you know, maybe it is about access for patients.
[00:08:27] Dr. Stacey Johnston: Um, and maybe it's about, you know, physician burnout, but whatever problem you're trying to solve, we need to make sure that you're aligned from a system and you have a singular system approach to that problem. Before trying to deploy the solutions.
[00:08:41] Krishna Gade: Interesting. And now we are in the biggest technology shift, uh, now, now era Right.
[00:08:46] Krishna Gade: You know, everyone is saying that AI will transform healthcare, especially in, and, and from your perspective, you know,
AI Use Cases That Are Actually Working
[00:08:53] Krishna Gade: what's actually working today? You know, is it still kind of PowerPoint decks or is it, are there any interesting applications that you've seen work? Uh, you know, and giving ROI.
[00:09:03] Dr. Stacey Johnston: Yeah, so, you know, I, I think most health systems are pretty, are advancing themselves in the backend pieces, you know, um, automatic coding, um, you know, we are pursuing fully autonomous coding, which I think would be, you know, a game changer for us.
[00:09:19] Dr. Stacey Johnston: Um, right now it's still mostly an augmented coding aspect. Um, we are moving forward. Towards, um, fully, um, autonomous verification of benefits, um, where an agentic AI solution is gonna do all of the, um, verification of benefits. We're also looking at, um, a agentic, um have call return. And so agents actually participating in, you know, voice call returns and, you know, interacting with the patients.
[00:09:46] Dr. Stacey Johnston: Uh, so there are some really cool solutions out there doing that. Um, other areas where we've actually, um, expanded into agent use, um, fully autonomous agent use is, um. When we converted, we actually just did a four hospital acquisition, uh, recently and we put them on our tech stack in seven months, which was pretty aggressive, but you know, you only get this system available to back load appointments.
[00:10:11] Dr. Stacey Johnston: We got it about two weeks of, uh, before we went live and two weeks to back load a hundred thousand appointments is not necessarily enough time. So we had to think differently. We could either hire 40 people to do this or we could use, um, some agentic AI. And so we were able, um, to basically do all of the ambulatory back load appointments with an agent.
[00:10:34] Dr. Stacey Johnston: It took us about three weeks to build the agent, and then two weeks, um, from that time that the system became available to back load all of the appointments. Um, so that was, that was game changing and that was fully agent, a AI and, um, doing all of our, all of our scheduling for our patients.
[00:10:53] Dr. Stacey Johnston: So I think that's pretty cool. We're looking at some. Um, digital FTEs and, um, you know, any, all these repeatable processes. We have a pretty good, robust, robust, um, robotic, um, process automation program. We have a good intake process and, and a governance around the RPA. But imagine laying, um, an age intake AI solution on top of your RPA.
[00:11:17] Dr. Stacey Johnston: Um, we're not there yet, but I think that's just, you know, that's right around the corner as well too. So those are kind of all of your backend processes. And then when you think about. Yeah, when you think about the clinical aspects, um. We actually have a white paper that has been written about, um, uh, agentic AI and colon colon cancer screening.
[00:11:38] Dr. Stacey Johnston: So really embracing the use of Cologuard and, uh, and as an initial screening instead of automatically going to a colonoscopy. Um, and a lot of this has to do with access a colonoscopies and Cologuard is now being viewed as, you know, a, a good first, um, solution. And so we, um, built an agent that automatically ordered Cologuard for the, this group of, um, patients that qualified.
[00:12:05] Dr. Stacey Johnston: So we ordered 7,000. Um. Cologuard screenings. Um, and, you know, ended up getting, uh, back, you know, I think it was about 40% returned and ended up, uh, doing 250 colon cancer screenings, um, additional with a colonoscopy. And we did actually have one positive colon cancer, um, returned. That was caught early. Um, so had a resection and full survival.
[00:12:31] Dr. Stacey Johnston: So I, I mean, that's just that is game changing, you know, clinical care where, uh, agentic AI solution can actually help, you know, help prevent, um, you know, further, um, further mortality. So I, I, I think that's pretty cool stuff.
[00:12:46] Krishna Gade: Wow, that's awesome. So automating appointments to clinical diagnosis. So, uh, what is like, maybe like, you know, maybe you can give us like audience also, like what's one use case that's often over hyped in, in, and, and also what's like, you think that is underappreciated, you know, both, you know, like in your experience that you've seen with AI.
[00:13:06] Dr. Stacey Johnston: Yes. Um, so one use case, so obviously radiology, a lot of the radiology use cases are fully embraced. Um, I don't know that, I don't know of any organizations that are fully autonomous with radiology yet. Um, we are using it as an augmentation to the workflow. So I think that's one that isn't overhyped, but is definitely used, you know, throughout multiple health systems.
[00:13:32] Dr. Stacey Johnston: Um, and one that I think is underappreciated, um, is the augmented responses to the in basket messaging. Um, and I think part of it is just understanding, you know, the large language model and, you know, having to say, well, I have to modify the message anyway. But yeah, modifying a message that's already written up is very different than typing a brand new message.
[00:13:57] Dr. Stacey Johnston: So I think that's one that we should continue to explore and push ourselves on. You know, there are studies out there that show that our, um, especially our primary care docs are getting 85% more in basket messages from the portal, these portal messages are, um, since COVID. And so, you know, that's additional burden, again, the cognitive burden for our physicians.
[00:14:22] Dr. Stacey Johnston: But yet also it's not being, it's not being reimbursed. It's not, you know, it's just additional time for our providers. The other thing is we need to look at the pools. For these messages, do they, does this particular message really need to go to that provider? Or can it go to an MA? Can it go to the front office staff?
[00:14:41] Dr. Stacey Johnston: So I think routing the messages better, and then when it truly is a provider message, having the AI help queue it up. Um, I, I think also some referral only, um, referral management is not as well utilized with AI. Um, we're, we're diving into that refill management. So refill medications, I really feel like as an opportunity with AI.
[00:15:05] Dr. Stacey Johnston: Again, cognitive burden. What is a repetitive task, you know? And so can the AI queue up the refills that are ready so the docs aren't having to click on those boxes all the time. So there are some solutions out there that are doing some pretty cool stuff with, um, refill meds.
[00:15:20] Krishna Gade: Awesome. So, uh, we touched upon a few things, right?
[00:15:23] Krishna Gade: You know, uh, uh, you know, you leveraging large language models to respond on behalf of doctors or even augment, you know, their answers. Now, one of the problems with LLMs is that there are a bunch of risks. You know, you know, they can hallucinate at times, you know, they could, you know, leak PII information and all of that, or PHI in the case of healthcare, uh.
[00:15:45] Krishna Gade: How do you, you know, when you joined, you know, AI,
Building AI Governance at Beacon
[00:15:47] Krishna Gade: how did you approach this problem of AI governance and, and where did you begin? You know,
[00:15:52] Dr. Stacey Johnston: uh, yeah, so great question. Um, so we at Beacon, um, upon my arrival didn't really have a good strong IS governance process. So it was, I was either the approvers or not approvers, and it was kind of just.
[00:16:08] Dr. Stacey Johnston: Tickets were submitted and people didn't hear back, and it was just not a great process. So that's the, actually, the very first thing that I did when I came here was stand up an IS governanceprocess. So we have an executive steering committee, we have eight different advisory levels, and then we have multiple work groups that report up to the advisory councils.
[00:16:27] Dr. Stacey Johnston: Um, one of the advisory councils that we did stand up as an AI council, um, and in that AI council we developed, um. Two AI policies. One's basically an AI policy, meaning that, you know, you don't go out and just sign up for AI. Um, you know, it has to come through the advisory council for approval. Um, and that, you know, if you're, if you, we have, uh, if you use AI, it has to be like a system approved AI.
[00:16:53] Dr. Stacey Johnston: So kind of more about like how to use AI and then the, um, and then we have a, a permitted and prohibitive use policy, meaning I can't put. PHI into ChatGPT or into Open Evidence. Um, something along those lines, but that we do have permitted clinical uses, which is more like augmenting the workflow. So there's ambient listening, you know, the clinical oracle's clinical AI agent, but you might use a bridge or, you know, um, DAX, uh, copilot.
[00:17:24] Dr. Stacey Johnston: So you know, some of those solutions that you know. Are augmenting your workflows. So you're still reviewing and you're signing off on that documentation. So in the radiology space, that's an augmentation to the workflow because it queues up the severity. So, you know, if an, um. That they get a, you know, hundreds of CTs a day.
[00:17:43] Dr. Stacey Johnston: It will do the read first and then it will queue up to the radiologist. These are ones with results that we think you need to review first. Um, and so it actually helps them become more efficient because they can then, you know, read that the most urgent reads first, and then go to the, you know, more of them.
[00:18:02] Dr. Stacey Johnston: Maintenance read. So, um, there's just a lot of cool things that we can be doing, but we need to make sure that we are governing it. So at first it comes through the AI Council for approval. There is a form that we created that the act, um, that the actual companies submit and fill out themselves. So it's about, you know, what is your data modeling look like?
[00:18:22] Dr. Stacey Johnston: Um, What, what are you using LLM? Which model are you using? Um. You know, what, what are you monitoring for drift and bias? Um, where are you storing the data? You know, so basically some vendor, uh, it's like a vendor risk assessment, but from an AI risk. Um, and then we are, um, also. Going, uh, we've done double down on our AI literacy program.
[00:18:48] Dr. Stacey Johnston: So now we have, um, we built out an AI literacy program and it's required training, um, for our managers on up, which is essentially to help us look for drift or bias or anything that is starting to, um, help us like manage these AI agents. You know, as you might manage an associate, you need to start managing these agents themselves too.
[00:19:11] Dr. Stacey Johnston: Um. And then it also helps us understand like what's a permitted use, what's not permitted so that we don't have our individual team members doing something with AI that they shouldn't be doing. And then one last component of governance is, uh, there are a few solutions out there that are doing AI monitoring.
[00:19:30] Dr. Stacey Johnston: So overlaying an another solution on top that is monitoring the AI for drift, for bias. And then they would escalate their concerns to the, you know, to the leadership, um, if, if something seems amiss. So, um, it's, you know, kind of like. You have the security, um, for monitoring for network, and it's, it's basically another security metric, but It's, it's monitoring for AI use.
[00:19:57] Krishna Gade: Yeah, makes a lot of sense. And, uh, very relevant to today. So, so I guess, you know, fundamentally, right, like it all comes down to trust and safety and governance. Um,
Trust in AI: The Hospital Board's Perspective
[00:20:08] Krishna Gade: what does trust in AI look from a hospital board's perspective? You know, like, because, uh, as you think about, uh, you know, like. Potentially AI recommendations conflicting with clinician judgment or, you know, AI maybe standardizing too much across, uh, and losing physician intuition and things like that.
[00:20:29] Krishna Gade: So, you know, how, how do you view of the, the world of like trust in AI from hospital perspective?
[00:20:37] Dr. Stacey Johnston: Yeah, so I, I think the administrators and operators, um, are. The leadership is most, most likely to embrace AI and trust it at this time because they've seen it successful in the backend processes. So, for instance, our revenue cycle team, they are on full on AI.
[00:20:57] Dr. Stacey Johnston: They're like, the more AI can layer on, bring it on. And so, um, so they've actually been great to work with. They are actually kind of leading our organization with the amount of AI they've embraced. Which then made it easier to tell our referral team, Hey, we're gonna embrace AI for your referral management.
[00:21:13] Dr. Stacey Johnston: And okay, now, now we've gotten that. So now let's del dive more into the clinical space. Um, the, the. AI that's more of an augmentation tool workflow. So as I mentioned, the in basket messaging, um, the clinical AI agent, we, we've really adopted the clinical AI agent, which is our ambient listening solution to queue up the documentation.
[00:21:38] Dr. Stacey Johnston: So about 70 to 80% of our providers and the ambulatory clinics are using the ambient listening solution for their documentation. And about 70 to 80% of their notes are generated, um, with the ambient solution. So, um, so it's being very widely used. Um, part of that was we train them, um, we spent a lot of time training them on how to use it to the, to help them become more efficient.
[00:22:05] Dr. Stacey Johnston: And they saw their documentation time go from seven and a half minutes. Per note to two and a half minutes per note. And so the fact that they saw the value in it and that they were starting to see that the note was actually producing a good quality note, so they trust that. Now, if I were to say, I'm gonna to total turn it on to be fully autonomous and you don't even get to read the note, I don't think we're there yet.
[00:22:28] Dr. Stacey Johnston: They're not trusting it enough. Um, and so you. You know, so you've gotta move the needle a little bit, have them trust that one component and then move the needle again. Have them trust that component. And so it, it's just a natural trust that builds on top of each other as as you start to expand and expand more.
[00:22:48] Krishna Gade: Very interesting and, uh, you know, like, let's, I think this is a great ROI thing that, you know, sort of mentioned, right? You know, healthcare industry runs on thin margins. You know, uh, you probably have to justify your AI investments. Where have you seen like the path
ROI & Fastest Path to Payback
[00:23:03] Krishna Gade: fastest payback, you know, cost reduction, clinician productivity, or new revenue?
[00:23:09] Dr. Stacey Johnston: Um, yeah, so I think that's still kind of for us on the backend pieces. Um, although I do have to say the clinical AI agent, you know, I remember when we first deployed it at another organization it was, gosh, I don't know if we can afford to. Do doctors pay for it? Do the, does the hospital pay for it? And there's no ROI, you know, maybe they might see one more patient per day.
[00:23:32] Dr. Stacey Johnston: Maybe they won't, you know? And so, um, but what we have actually found is, um, we've been live for about a year and a half and in, you know, over a 12 month period of time, um, we had an extra $10,000. Um. Per doc revenue and just better capturing of the documentation of the visit. Um, and so there was a definitive ROI by going live with the AI.
[00:23:56] Dr. Stacey Johnston: So then we're like, that's a no brainer. Let's just keep rolling that one out. So, um, so, um. And the other one.
[00:24:04] Krishna Gade: And what does this clinical AI agent do? Actually? Can you double?
[00:24:07] Dr. Stacey Johnston: It's like via bridge. It's within Oracle built it. Um, but you know, it's like the co you know, DAX 360, you know, the co-pilot, the
[00:24:15] Krishna Gade: right
[00:24:16] Dr. Stacey Johnston: A bridge that ambient, you know, it you is conversational AI that you have with your patient, the provider, and the patient that then queues up your documentation templates to review.
[00:24:27] Dr. Stacey Johnston: Um, but that one's built within Oracle by Oracle. Um. And, um, the other one that again, received great ROI was the, the colonoscopy. So why not fully, I mean, the Cologuard, why not fully automate that? Like why can't that be a fully autonomous decision that if we have, based off of your inclusion criteria, and we know your exclusion criteria.
[00:24:54] Dr. Stacey Johnston: Why isn't an a, why isn't an agent just automatically ordering the Cologuard screening? You know, I, I, I think that's a great use case. And then, you know, for all, for all of the decreased colonoscopies, un unnecessary colonoscopies and increase in appropriate colon cancer screening, then, um, you know, which is tied to a lot of the, you know, four and five star metrics from your.
[00:25:19] Dr. Stacey Johnston: You know, your Medicare Advantage plans. Um, so you know why not do something like that.
[00:25:26] Krishna Gade: Yeah, it makes sense. And as you bring these multiple tools, right, like you mentioned, you know, clinical agents developed by Oracle or average or other, other systems, are you seeing like the need for a
Centralized Control & Federated AI Models
[00:25:37] Krishna Gade: centralized control plane emerge where you are able to track all of these agents in one place and how know how they're performing?
[00:25:45] Dr. Stacey Johnston: Yeah, so that's, so, um, AI reports to me. I've, uh, formed a Department of AI and Transformational Technologies, and we are a centralized process at this point in time. However, um, you know, the recommendation in the studies are showing that. You're more likely to embrace AI if it's more of a federated model, um, you know, more of a decentralized model.
[00:26:07] Dr. Stacey Johnston: However, I still think the monitoring should be centralized.
[00:26:10] Krishna Gade: Right.
[00:26:10] Dr. Stacey Johnston: And so what does that, you know, federated, decentralized model really look like? Well, that means that, you know, you've got your department's owning, um, you know, just kind of day-to-day monitoring of the AI. Does this still make sense? Am I, you know, what is my adoption rate?
[00:26:26] Dr. Stacey Johnston: Are these metrics meeting, you know, my needs? Um, is this AI no longer needed? Do we turn it off? I mean, that's something we never ask ourselves in technology is do we turn off these systems? Um, and so I, I think the monitoring still should be a centralized process, but how well you adopt and deploy, um, the AI's agents or, you know.
[00:26:48] Dr. Stacey Johnston: If any RPA or anything like that, I think we can start to move towards more of this federated model. One of the things we're talking about developing, and there are some organizations that are doing this really well, which is this kind of this AI citizen program, which is you can start building out like low code, no code yourself.
[00:27:06] Dr. Stacey Johnston: Um, and so we have, um, you know, we have Microsoft Co-pilot and if you know, with a. You know, that higher level, um, um, license, you can actually start building out some of your agents and or do we use that? Do we use UiPath? You know, where you can maybe build out your own bots? And so how can we actually, um, through an approval process that maybe more centralized, then give people the tools to actually build their own bots.
[00:27:36] Dr. Stacey Johnston: Um, I think that's where we wanna go, but we're not quite there yet.
[00:27:40] Krishna Gade: Yeah, makes sense. Makes sense. And then like, as part of this, uh, control chain of centralized Observability, you know, like, uh, is there a need for a runtime AI governance, like, almost like that redact PII before inform? You know, I think this is an audience question I'm reading out that redact the PHI before any information enters LLM,
[00:28:00] Dr. Stacey Johnston: I, yeah.
[00:28:01] Dr. Stacey Johnston: I mean. Obviously I, I think you have to be real time because I, you know, without that, Um, we won't be responsive. So, um, It's It's critical for you, you know, you need to have kind of your standard processes, but there are gonna be times and needs where you need to be a little bit more agile. So how can you have a, you know, a governance that's, um, that has a standard process but yet is agile?
[00:28:31] Dr. Stacey Johnston: I think that's the sweet spot. And I don't know that a lot of organizations are doing that really well. We're not quite in a fully agile yet. Um, but I think having our de having a dedicated AI team has actually made us much quicker.
[00:28:43] Krishna Gade: Hmm. Interesting, interesting. And then, and another related question, I'm just taking a couple of audience questions here of, you know, there are so many that please keep coming them.
[00:28:52] Krishna Gade: And, and so there's one more question from John Sri's on uh, hi Dr. Johnston. How does Beacon ensure that is robust in their tool evaluation while not acting as a bottleneck for innovation? I think this is kind of that trade off between innovation and governance.
[00:29:09] Dr. Stacey Johnston: Yeah, no, that's, that's always hard. Um, I, I think the organization is definitely clamoring for more AI, but you know, I brought up in our large system meeting this morning that, hey, we can go as fast as you want.
[00:29:24] Dr. Stacey Johnston: I just need more people to do the work. You, you know, so, um, so is, you know, are these tools worth their investment? I think so. You know, if we work on these AI solutions, what are we not working on? So I think that's kind of the, you know, so we need, we need to understand the system priorities. Um, but also as we move into this more, um, federated model where if it's kind of this low code, no code, um, or, you know, a, a simple bot build program, then that I think helps us become more.
[00:29:56] Dr. Stacey Johnston: Innovative and that people could actually be building some of their own agents or building their own bots. Um. I, that's I think where most organizations are ultimately gonna have to go. And the other thing is that, um, do you buy or do you build? And we're doing a hybrid of both. So if it's a really complex workflow or if we need it turned around really quickly, we're gonna buy, we're gonna partner with someone and we're gonna say, Hey, we need this agent built in two weeks.
[00:30:22] Dr. Stacey Johnston: Um, can you, you know, we have it mapped out, you know, you can you build that for us versus if you, um. You know, if it's a, uh, a simple work workflow, um, you know, why don't we build that ourselves within our team? Um, so, you know, I think, I think it's a, you're gonna almost have to do all you, you're gonna have to be innovative and have a process.
[00:30:46] Dr. Stacey Johnston: You're gonna need to be agile. And also look at cost reduction. I mean, you, you have to, you have to kind of do all, and so, you know what I brought up this morning was I think, you know, yes, we might be. Spending more now, but in the long run, it's gonna save us more money by hope, you know, reducing FTEs, or honestly, where we are is we can't even fill those FTEs.
[00:31:08] Dr. Stacey Johnston: So we're paying contractors. And so rather, instead of filling it with contractors, can we fill that with an, an AI agent that can actually help reduce the workloads? You don't need as many FTEs. So for even it's not for us, it's not about reducing your FTEs, it's just about, you know. Filling the positions we have open or reducing our contractor spend.
[00:31:30] Krishna Gade: No, that's very, very interesting. And, and so, uh, I guess like for other, you know, I AI leaders like you working in healthcare, what's your advice? Like, what do they, how do they think about, uh, you know, bringing new AI tools to deployment? What is the one thing to put in place, you know, before they deploy anything in terms of AI, you know, in their, in their environment?
Deploying AI Responsibly: Governance & ROI
[00:31:51] Dr. Stacey Johnston: I do think having an AI governance council and advisory council makes sense. And so, um, we have physicians sitting on it, vice presidents, hospital presidents. Um, we have ethics, we have compliance, we have risk, we have security, um, I ask leaders um, so, you know, it's kind of the gamut of, you know, what's gonna Have HR.
[00:32:15] Dr. Stacey Johnston: Um, so, you know, we have the, kind of the gamut of who needs AI and who should be responsible for AI, and that just helps to ensure that we have some responsibility, um, for our solutions. Um, but it, you know, also helps us understand our training and needs, our change management needs. Um. So I think that's the first thing.
[00:32:34] Dr. Stacey Johnston: And then you need to understand, you know, the other thing we did very early on was at a minimum, understand what your permitted and prohibited uses are. Are you going to be, you know, what is your risk for fully autonomous in the clinical space? Are you there yet? Are you not? You know, you need to decide as an organization.
[00:32:51] Dr. Stacey Johnston: Um, and then I would, you know, so those are the first two things I think you need to do first before you really fully expand. And then I would. A hundred percent what I did of standing up a department that this is all I did. Because if you add AI on top of your team that's already busy with code upgrades, maintaining your systems, the break fix, you know, the, you know, the request from, you know, I need to change this order set, um, this documentation template, you know, we have now joint commissions coming.
[00:33:22] Dr. Stacey Johnston: So if, if. If you're a team that's doing that day-to-day maintenance and you know of your systems, and then they're also responsible for going out there and, and being a partner for implementing AI, I think it's gonna be very hard to move the needle. So I think having a team that they don't need to worry about that the mundane.
[00:33:42] Dr. Stacey Johnston: Yeah, their job is just to go and meet with the business partners, um, go, they're going to conferences. They are, you know, meeting with other organizations. They are out there just scanning the environment and finding out what's new. They're meeting with venture capitalists. Um, we're meeting with, um, you know, we're partners with Notre Dame, so we meet with Notre Dame frequently.
[00:34:03] Dr. Stacey Johnston: What are they working on? What are they seeing? Um. You know, try and we are setting up, um, some internships with Notre Dame so they can actually do some development in our, in our environment too. So before you get there though, you, you know, it's kind of that, um, you know, crawl, walk, run philosophy. Yeah. So you've gotta crawl first.
[00:34:24] Dr. Stacey Johnston: So choose a couple of good select use cases that you felt fairly confident in, um, before you start to, you know, um, walk and run. The other thing that we do, I forgot to mention with the AI, um. Policy is we only approve at the advisory council if it has a defined ROI. If there is no ROI, then you need to ask yourself why are we doing it?
[00:34:49] Dr. Stacey Johnston: But if it's soft, ROI, and it's really gonna give back, you know? Back to the clinical experience, then it goes to the steering committee, meaning that, you know, do, do the executives, so the C-suite do we buy in that we wanna give this to the clinical end users, even though we may not get any ROI from it.
[00:35:08] Dr. Stacey Johnston: And so, um, we haven't actually had anything that we've approved, um, without a defined ROI. So right now, you know. Finances are tight for all of us. So you, you all should be looking for ROI first with your AI before really going to the non ROI initiatives.
[00:35:26] Krishna Gade: Yeah, makes sense. So you got to measure your ROI continuously and evaluate how these things are working.
[00:35:31] Krishna Gade: I guess that's important. Uh,
[00:35:33] Dr. Stacey Johnston: yeah.
[00:35:34] Krishna Gade: So
[00:35:39] Krishna Gade: there is, um,
Winning Clinical Trust & Patient-Facing AI
[00:35:40] Krishna Gade: winning the confidence of clinical, clinical staff, right, when it comes to introduction of new tools, especially AI, right? What has been your journey, uh, in terms of, uh, doing that at Beacon? You know, how you successful or any lessons learned? And
[00:35:56] Dr. Stacey Johnston: I'm sorry, it broke up for just a second. Could you ask the question
[00:35:58] Krishna Gade: again?
[00:35:58] Krishna Gade: Yeah. So the question was basically how are you, uh, you know, one of the problems in, in healthcare is like, how do you win the confidence of AI clinical staff and who might be skeptical about AI and the tools and, and, you know, yeah. And what has been your experience doing that at Beacon? Yeah.
[00:36:14] Dr. Stacey Johnston: Yeah. So I think having, um, key physician partners, um, understanding their use cases.
[00:36:20] Dr. Stacey Johnston: So I actually spoke at a physician meeting last night and kind of talked about the future of healthcare and the future of healthcare technology. Obviously AI was probably a number one thing we talked about and. People were, you know, some were reticent, but some were like, Hey, I have really great ideas.
[00:36:36] Dr. Stacey Johnston: I'm like, submit 'em to me. You know, if it's a low hanging fruit, we could put that in, you know, from the time of request to an agent being built is, you know, probably about, you know, four to. Four to eight weeks, depending on the complexity. Right? And so why not, you know, if that, if there's gonna be ROI there, I don't see why we couldn't do that.
[00:36:58] Dr. Stacey Johnston: Um, if it's obviously more complicated workflow or it requires integration into our EHR, that's gonna be a longer, you know, ask. Um. But the, but you need to deliver on your promises, and then they need to see the value of what they, you know, what we've turned on. So, you know, one of the things I always say is don't implement AI for AI sake to say you're implementing AI, but it's gonna add, it's gonna be meaningless to the docs.
[00:37:23] Dr. Stacey Johnston: Or make sure you know where that data's pulling from. So again, if you have. Different dataset, definitions, and you're pulling from the wrong dataset. It's the data. You know, the docs aren't gonna trust the data and therefore they're not gonna use the systems. Um, and it needs to be easy to use, you know, just naturally intuitive.
[00:37:42] Dr. Stacey Johnston: It needs to be part of their workflow. Asking them to go out of their EHR to use another system that's gonna also not help your case. And so it needs to be seamless, you know, integrating into your system as much as possible. Um. And, and it just needs to provide some value to them. Either, you know, um, they can see more patients or they go home earlier.
[00:38:05] Dr. Stacey Johnston: They, um, less cognitive burden, you know, one less click, you know, throughout a day adds up, you know, a significant, um, time savings. So, you know, the, you're gonna gain the trust by doing solutions that are easy to use for them.
[00:38:21] Krishna Gade: Awesome. Yeah, and I think once you're able to trust your AI, then you can start thinking about customer facing AI as well.
[00:38:27] Krishna Gade: Right. That's actually an interesting, uh, you know, question from an audience, you know, how is there, how are the con, you know, how are you tracking the concerns? You know, how are you sort of overcoming the concerns about AI agents directly work, you know. Working with patient and, and sort of, uh, and potential patient reactions, you know, dealing with AI agents, you know, what has been, you know, I don't know if you are there yet, or like, how are you thinking about it?
[00:38:56] Dr. Stacey Johnston: We're, we're looking at it. I mean, we are investigating the AI agents making phone calls and so, you know, studies have shown, um, that that empathy from an AI agent call is actually higher than, you know, a person calling. And, um, you know, so, so that's probably where we're gonna start with our AI, true interaction with the patient.
[00:39:22] Dr. Stacey Johnston: Um. Now there's other things that AI could be doing and that I'd like to pursue, which is really AI detecting kind of emotional or mental crises. You know, when they're calling to make an appointment, there's AI solutions out there that can detect, you know, suicidal risks. So is that something that you, you know, wanna look at as well too?
[00:39:44] Dr. Stacey Johnston: Um, but I, I think, you know, I think. AI agents calling patients is pretty cutting edge. Hmm. Um, you know, the other thing that, you know, AI. Um, can be doing, there's a lot of solutions out there that, um, for in your inpatient stays, you know, AI and falls. A lot, a lot of people are doing AI with fall predictive modeling with camera use.
[00:40:06] Dr. Stacey Johnston: But what about if you could take that and then that, um, if the, um, you know, the patient is about to um, stand up, the agent itself calls into the room and says, Hey, please sit back down. Um, or do you need me to call your nurse for you? You know? So having that agent interact with the patient actually in the, the clinical setting in the care room, um, that's out there, I mean that's, we haven't done, we haven't done that yet, but that's definitely on our roadmap.
[00:40:33] Krishna Gade: Very interesting. Very interesting. Uh, I guess, uh, you know, coming back to sort of, uh. You know, copilots and autopilots in medicine, you know what, what, you know, apart, apart from like administrative assistance and clinical diagnosis, where do you see the world going? You know, for example, if you have to rebuild, you know, the entire healthcare system from scratch today, you know, with AI, what would you do differently?
[00:40:59] Dr. Stacey Johnston: Uh, yeah, so it's interesting. Um, so Oracle is rebuilding their or total EHR with this new AI semantic layer and we are going live with it in our ambulatory clinics this summer. So at least in pilot, you know, a pilot, a few clinics, um, so it'll be interesting to see if that significantly changes. But some of the AI things that we need to embrace within the EHR.
[00:41:27] Dr. Stacey Johnston: Um, again, the augmented responses to the patient messages, number one. Number two, um, summarizer. So kind of what happened between, from the time I saw you, you know, six months ago to now. You know, was there admission? What, you know, what um, what images were done? Did they follow up on any of their care gaps? What care gaps still need to be done?
[00:41:48] Dr. Stacey Johnston: You know, so basically kind of what happened to now and how can I help keep you healthy? Like layering the EHR on top of large language model, um, to be able to queue that up combined with the ambient solutioning. So then you can review the, you know, the note summarizer and say, okay, I see that you, you know, visited that Ad for chest pain. Did you follow up with a cardiologist? No, I didn't. Well, let me place this referral for you now. You know, let's get you referred. Do you have a particular cardiologist you want? Let's go ahead and order an echo and, Um, you know, follow up with me in a month that conversation. Should be actually placing the orders as you're speaking instead of you then getting the document.
[00:42:30] Dr. Stacey Johnston: Right now it's just queuing it up in the document. You read the document, you submit, it still saves you time. But wouldn't it be great if then it also actually placed those orders, or at a minimum, queued them up into your in basket so that you just had to sign off on them instead of actually having to manually type in the orders.
[00:42:45] Dr. Stacey Johnston: And because it was using a large language model, it could review to make sure you have the right CPT code and the right diagnosis code, and you know, um. And it also has access to your referral management system, so it knows which doctor to send it to and the right address. And then the patient gets a text, um, you know, 30 minutes later and say, Hey, we got your referral processed.
[00:43:07] Dr. Stacey Johnston: Um, you know, please, uh, you know, we'll text you again when we have an appointment made for you. So, um, or text in here. You know, use your texting solution to choose which time you want. So that's where we need to go. We're not there yet, but I definitely, um, think there are some. Organizations that are doing bits and pieces, like we're doing the texting and the online scheduling.
[00:43:29] Dr. Stacey Johnston: Um, but we are not fully, Um, the note summarizer. We won't go live with till this summer.
[00:43:35] Krishna Gade: Interesting. Interesting. And there was also a question on variables, uh, from the audience, uh, you know, from, um, Dr. Johnston, how are you envisioning utilizing lifestyle variable data from patients in their care?
[00:43:49] Krishna Gade: Variable devices like smart watches, rings, wristbands, continuously collect data, and Yeah. And then how can these data sets be utilized by leveraging AI tools to enhance patient care?
[00:44:00] Dr. Stacey Johnston: I mean, great question. So I mean, that's where I think the sky's the limit. So, you know, we've got all of this, this IOT world, you know, so we have all of this patient data.
[00:44:08] Dr. Stacey Johnston: So whether it's a continuous glucose monitor, blood pressure cuff, um, you know, we've got, you know, these telemetry devices, your smart watch, you know, so all of this wearable data, right? And so coming in, um. And so it can come into the EHR and ingest it pretty easily. That's not the hard part. What the hard part is from where I started at the very beginning is the workflow around that.
[00:44:32] Dr. Stacey Johnston: So if you have someone's whose blood pressure's reading at, uh, you know, 200. What are you gonna do? Like who's monitoring that alert and does that alert come to the primary care doc? Well, what if he's off that week? Does it, you know, does it go to a case management list? You know, 'cause we, you can embed it into a registry.
[00:44:49] Dr. Stacey Johnston: So is there someone that's always monitoring this registry? Are they, is it a nurse practitioner that's just monitor, like managing a hypertension clinic? So there are some organizations that are doing this really well in the wearable space where we are not there yet, just being totally honest. But, um.
[00:45:06] Dr. Stacey Johnston: You know, I, I do think the wearables are one components. Then I think the other that I talked about last night with the physicians is genomics. Um, genomics is, you know, um. I think the studies show that every three or four out of five, um, drugs now have some sort of genomic marker being tested as it's going through FDA approval and, you know, depression, meds, um, antipsychotics, um, blood thinners.
[00:45:29] Dr. Stacey Johnston: I mean, a lot of medications right now have some sort of genetic testing and that's additional cognitive burden to our docs. There is no way they are gonna remember everything that needs to be done for your patients at any given time. So embedding that genetic data to alert you and say, oh, you're about to order Pradaxa.
[00:45:47] Dr. Stacey Johnston: This patient isn't, uh, a candidate for Pradaxa because of their genetic profile, please consider one of these others. And then the solution actually queues up what the other recommendations are. So I think between both the wearables and the genetic data, you can now move from. Predictive analytics to truly prescriptive care, which is the right patient at the right time with the right care, um, you know, by, you know, really surrounding them with, you know, the data and analytics and the AI on top of that.
[00:46:17] Krishna Gade: Wow, that's amazing. I think that gets me into the future of healthcare delivery, right? Like, you know, fast forward five to 10 years, what does a hospital look like with deeply embedded AI? You know, according to you.
[00:46:29] Dr. Stacey Johnston: Yeah, so I think what will end up happening is, you know, there's a lot of consumer grade, um Things that are happening right now, like blood pressure cups and you know, and then, you know, all the wearables and even, um, telemedicine visits are, you know, I think gonna continue to expand. So I think in the primary care space, you are gonna actually see sticker patients in the. Less sick patients are gonna just kind of do their care at home.
[00:46:55] Dr. Stacey Johnston: You know, even prescriptions that required an actual visit in a prescription can now be bought over the counter. So there's a, you know, like, um, Omeprazole used to be a prescription only, now it's bought over the counter. So there's, and also with a large language model and ChatGPT people are gonna start to self care at home.
[00:47:14] Dr. Stacey Johnston: I think. And then, and then it's going to become sicker. Patients gonna be seeing that primary care office and they're gonna manage those at, at home as best as possible. And therefore your inpatient days are gonna be even sicker. And so, you know, I think you're. What we used to see in the inpatient space, some of these people that we would keep a little bit longer because they need some extra oxygen and we're gonna take care of those patients at home.
[00:47:40] Dr. Stacey Johnston: And that, that truly would then leave the beds for the, the sickest of the sick. So I think we're gonna really see the acuity in our hospitals going up as well too. Um, but on the flip side, I think we're gonna have more automation. We're gonna have, you know, these cameras in the rooms and the AI predicting the care.
[00:47:58] Dr. Stacey Johnston: Um, you know, um. AI right now without a, without an actual blood pressure cuff, is able to start detecting blood pressure, temperature, heart rate, um, just by cameras, you know? So I think there's just more we would be able to do, um, with the tech technical aspects and the technology surrounding, you know, the patient in the individual hospital room so that you know, that hospital room of the future.
[00:48:24] Dr. Stacey Johnston: Um, so that, uh, so you're monitoring these sicker patients, um. Kind of more around the clock instead of relying on people always going in and looking at them all the time.
[00:48:35] Krishna Gade: Yeah, it makes sense. And are there gonna be some new roles in healthcare because of these changes? Are there, like, are there's gonna be a job job shifting as well happening in healthcare because of this?
[00:48:44] Dr. Stacey Johnston: Yeah, I think we're gonna obviously continue to have to upskill our folks. Um, so, you know, everyone's gonna need to practice to their top of the license. But don't forget, we're also going, you know, already in a nursing shortage and a physician shortage. So how are we gonna face both, you know, the clinical caregiver shortage?
[00:49:01] Dr. Stacey Johnston: Well, we're gonna need to upskill our CNAs, we're gonna need to upskill, um, our, you know, transport team. So, you know, the more we can upskill the folks that we may have, um, that they can maybe go into some of these other roles. Um, I know some hospital systems are looking at, um. You know, partnering with community colleges that, you know, they come directly outta community college to go into some of these, um, you know, these roles, but I think upskill and definitely upskilling on your IT team.
[00:49:31] Dr. Stacey Johnston: So how to manage an AI model and, you know, building, are you, you know, a builder or a buy program and you know. Your smaller community hospitals may continue to be a BY program. I think us at Beacon managing a large language model is probably not gonna happen anytime soon. So that's why we partner, but you know, um, I, you know, upskilling your team, but upskilling the clinical people as well too.
[00:49:54] Krishna Gade: Awesome. Okay, so we are almost at the end of the webinar. Uh, let's actually wrap this up with like, uh. You know,
Lightning Round & Closing Thoughts
[00:50:01] Krishna Gade: lightning round. So like, you know, one or two word answers Dr. Johnston. So, uh, ask, quickly, ask these questions. You know, one AI myth in healthcare for you right now?
[00:50:11] Dr. Stacey Johnston: Uh, that I think it's gonna get rid of the caregivers anytime soon in the future.
[00:50:16] Dr. Stacey Johnston: Maybe five, 10 years from now. But right now we're not seeing that.
[00:50:19] Krishna Gade: Okay. Great. And one company doing it, right?
[00:50:23] Dr. Stacey Johnston: Uh, we've, I, I had to say right now I'm really impressed with the Oracle and their clinical AI agent and, um, you know, so we're doubling down on our Oracle experience and going all in with their new, um, agentic EHR.
[00:50:39] Krishna Gade: Okay. And one capability that you wish existed today?
[00:50:42] Dr. Stacey Johnston: Uh, I, I really think just being like totally keyboardless, like ambient everything. Just being able to have a conversation with the computer, a conversation with the patient. Just you know, your thought process and you know, queuing up your orders, everything being totally ambient.
[00:50:59] Dr. Stacey Johnston: I think it's coming, but we're not there yet.
[00:51:02] Krishna Gade: Got it. And this is probably interesting, AI as a co-pilot or autopilot in medicine.
[00:51:08] Dr. Stacey Johnston: Uh, I think it's both. I think it, for the most part should be a co-pilot in the clinical space, but I don't see any reason why it couldn't be autopilot in some of the backend processes.
[00:51:17] Krishna Gade: Okay. So finally, if you went back to being a physician today, how would you use AI differently?
[00:51:23] Dr. Stacey Johnston: Uh, definitely, you know, I would embrace it, you know, is, you know, seeing more and more patients, um mostly just to finish on time and go home on time. You know, I, the EHR is definitely hard work and it adds more clicks to the docs, and so if I could use AI to make my life easier, I, I would a hundred percent embrace it.
[00:51:46] Krishna Gade: Awesome. Well, thank you so much Dr. Johnston. Uh, thanks for your valuable time here with our audience and, uh, I learned a ton about AI in healthcare and your journey from being a physician to now a technologist. And, uh, some great insights and thanks.
[00:52:00] Dr. Stacey Johnston: Oh, thank you so much.
[00:52:02] Krishna Gade: Awesome. But that's it for this, uh, month AI Explained folks, uh, we'll come back to you with another great guest on, um, uh, in the month in, I think in the next few weeks then. See you all.
[00:52:15] Dr. Stacey Johnston: Great. Thank you.
