First US Trials Show Apple Vision Pro a “Revolutionary Tool” in the OR, Say UC San Diego Health Chief Clinical & Innovation Officer Dr. Christopher Longhurst & Associate Professor of Surgery Dr. Ryan Broderick

Dr. Longhurst: Thank you for having us.

Dr. Broderick: Thank you.

Anthony: Great. We’re going to talk about something fun today which is the Apple Vision Pro. You guys are doing the first US clinical trials for spatial computing in the OR with it, which is probably about as cutting edge as you’re going to get. To start off, Chris, give me a little bit about the organization and your role and then we’ll go from there.

Dr. Longhurst: I serve as the Chief Clinical and Innovation Officer for the health system at UC San Diego and I’m a Professor and Associate Dean in the School of Medicine. I also have the privilege of leading our Center for Health Innovation  which was funded with a generous gift from Joan and Irwin Jacobs. All three of those roles are really synergistic because our goal is to help solve challenges in healthcare delivery through innovative technologies and the like. We’re really delighted to partner with Dr. Broderick, Dr. Horgan and the Center for Future Surgery on this project.

Anthony: Excellent. Ryan, a little bit about your role.

Dr. Broderick: I’m Associate Professor of Surgery in Division of Minimally Invasive Surgery and I do my research out of the Center for the Future of Surgery which is located at the Medical School at UC San Diego. In that role, I have a background in biomedical engineering, so I like to think very innovatively, try to apply new technology to what we’re doing in the operating room, and it’s been great to have Chris as a collaborator on this project as we have been able to bring it to reality.

Anthony: Very good. Let’s get the 10,000-foot overview of what this project is all about. Again, I’m quite interested in governance and the process for something like this. Somebody had the idea, read about the technology and said “oh, let’s try this.”

This is even more cutting edge than reading about it being done somewhere else. From the idea of a clinician or a surgeon who thinks this would be interesting, to moving through the process of governance, to IT eventually getting involved, and to where we are today. Let’s start with the overview of what exactly you are doing, whoever wants to jump in there.

Dr. Broderick: I’m happy to take that one. When we do minimally invasive surgery, laparoscopic or robotic surgery or even endoscopic, the image from the small camera that’s inserted into the body whether for endoscopy, through the mouth or through the anus or if it’s inserted through the abdomen or chest through small incisions in laparoscopic or robotic surgery, that image is then put through an image processor and up onto a computer monitor, a screen, that allows the surgeon to see to do the operation itself.

As we get more and more equipment in the operating room, there’s more and more video feeds. We have lots of different feeds and therefore, lots of different monitors. The room tends to get a bit cluttered. But ultimately, as we got some of these Apple Vision headsets, a lot of the function really is using that as a monitor. You can make a digital screen that’s as big as you want it to be, that sits in digital space, but it can be pinned to real world situations, like in your room or in the operating room.

We have the idea of okay, we have all these stuff in the operating room, what if we can distill that down into this augmented reality headset where we create virtual monitors around the operating room, have the information that we need at our fingertips that we can resize, move and change the source to look at what we want to look at. That’s what we started looking at, and also how does that impact not just patient care, but the surgeon. Is it comfortable to wear? Is it feasible to do at all? How does the workflow of the operating room feel when we’re changing things like this in a way?

Anthony: Let me back up a little bit. I want to start before that to somebody reading about this particular device and thinking ‘oh this might be interesting in the surgical suite.’ Can you take me a little bit through that?

Dr. Broderick: Chris has actually worked with product kit devices or devices from other companies in the past as well.

Dr. Longhurst: Thanks, Ryan. The announcement of the Apple Vision Pro earlier this year was certainly head turning and there have been other AR/VR headsets. But one of the things that’s special about what’s been released is both the high-definition screens, I think it’s next generation in terms of the resolution compared to some of the other headsets, as well as the specificity of the head tracking and motion tracking. That combined with the gestures that can be made in a sterile field make this an obvious candidate for a lot of different use cases from a healthcare standpoint.

We previously had been announced as partners with our radiology vendor where we’re testing out these headsets for reading three-dimensional scans and Dr. Paul Murphy has been leading that work and we have some publications coming out soon. Then, in parallel, Dr. Horgan and Dr. Broderick and others in the Division of Minimally Invasive Surgery recognized the opportunity to use this to address some of those challenges that surgeons feel in the operating room suite.

When you hear about our first dozen or so cases, there’s some real wins here and real opportunities and it certainly turned heads even within our own institution given that we’re now generating interest from other proceduralists, from neurosurgery and ophthalmology to interventional pulmonology. I think that Dr. Broderick and his colleagues have certainly started a trend here.

Anthony: Chris, do you know if Apple positioned this as something that was designed specifically for use in healthcare or is this the case of a device they put out there and people are finding different uses for it in different industries?

Dr. Longhurst: First of all, I would say Apple certainly did not position this as a medical device. Like other Apple devices, they’re really looking at the software to be evaluated from a medical device standpoint. Dr. Broderick and his colleagues sort of homebrewed some software to make this work in our operating room environment as part of this research study and we also partnered with various vendors who we expect we’ll be going through 5K, 10K clearance processes for apps that make this work in the healthcare environment. More to come in that space but I think it is really critical to recognize it’s not the hardware itself that’s the medical device, it’s the software that we’re using on it.

Anthony: Let’s talk a little bit about the process. I’m very interested in this and these conversations to help other people. There are a million technologies, there are a million devices, things are coming out all the time, could be a surgeon, could be any physician who says “I think this device would help me in my work. I could imagine this being useful and I would like to try it.”

What is the process for that coming to fruition or eventually getting in front of IT whether it’s the CMIO or someone like yourself, Chris, and eventually the CIO. Eventually we have to talk about security, we have to talk about integration, right, and all roads are supposed to lead to the EMR. Whatever this device is capturing we think we want to get it in there. Eventually, the idea that comes from the user, probably a clinician, has to go through those governance processes, tell me about that.

Dr. Broderick: Yes, there certainly was that governance process. Once we have the idea we come up with a hypothesis or what we think we want to study by using this clinically. There were a couple of pathways that I went through in parallel.

One is we knew we didn’t want to trial this device in first cases on humans. We trialed this in the lab, both in dry lab and then in wet labs, to ensure that this would work and we did this over the course of multiple months where we would deal with the settings, set it up, make sure that we have the connections correct, make sure there’s no latency in what we’re seeing from imaging. Because we want this to perform exactly as it’s going to perform when we’re doing on people because what we care most about is the safety of our patients.

I went through the IACUC process and for multiple months, we trial that and in parallel worked with our Institutional Review Board (our IRB) that protects our patients and make sure that we go through the correct processes as far as talking with the FDA about using these devices on people, making sure we consent our patients in the appropriate manner and so that we can use them in real time and use them safely.

Anthony: Chris, anything you want to add to that sort of bubbling up and that going through the process and IT eventually coming in and checking things out?

Dr. Longhurst: Fundamentally, there’s two ways to bring new devices into the organization. One is through an innovation committee and the perioperative services has one for that. That’s often used for things that have demonstrated evidence elsewhere but are perhaps not yet FDA approved and necessary in one-off cases. The other is through, as Dr. Broderick described, the Institutional Review Board for Clinical Research.

In this case, we agreed that doing this from a research standpoint was the right way to do it because that allows us to rigorously evaluate what the outcomes are. As Ryan described, I think it’s important that before this is ever used in humans, there was a lot of education simulation in other models that allowed for the surgeons by the time they’re using it in a real case to have that facility and agility with the device to make sure it wasn’t going to slow down surgery or adversely affect it in any way.

Some of the outcomes that you’ll hear Dr. Broderick evaluating is length of case and satisfaction and also really importantly is surgeon comfort. It turns out that minimally invasive surgeons like Dr. Broderick and Dr. Horgan suffer some of the highest rates of ergonomic discomforts from twisted necks and back pain and other things because of the machinations they have to do while operating.

I’m excited about this device as a way of improving surgeon and proceduralist experience as well and so lots of opportunities here. But we don’t really know how these things work unless we actually study them, and that’s what’s important about this being first is that we don’t know others that are actually studying this and so the data that comes out will allow other health systems to make those decisions if they should bring this in as an innovation device.

Anthony: What would you say, Chris, is the role of the CIO in something like this? Josh Glandorf is your CIO. What do we want to educate them about – we’d like to put them in a position to at least say “oh, yeah, I heard of that,” right.

Dr. Longhurst: Absolutely.

Anthony: When they get approached by a surgeon two weeks from now, we want them to listen to this and say “oh, yeah, now I can entertain this idea intelligently.” What’s the role of the CIO in something like this?

Dr. Longhurst: Absolutely. Great question. First of all, Josh is terrific and he’s a member of my team and  he has a role in our Innovation Center as well. This was certainly not a surprise to Josh or the IT team. Secondly, whether this comes in through the clinical research direction or the hospital operations direction, it’s still going to need to be evaluated from a security privacy standpoint and certainly our teams work together to make that happen.

As we move from a research kind of evaluation to considering if this is showing meaningful outcomes, are we going to scale this, are we going to make an investment on an on-going basis, are we providing all or some of our surgeons with these devices, how are we going to have highly reliable processes for supporting them, plugging them in because it won’t be just Dr. Broderick and his team doing this type of support. The Information Services Department will be critical for all of that.

But the first stop is really just understanding is it worthwhile to have those discussions, are there meaningful outcomes that we can achieve. I think this is very similar to the discussions we’re all having about artificial intelligence in healthcare. Everybody wants an AI-based solution in a shiny object, but the first question is really, is it helping you solve problems? A big shout out to Ryan and his teammates because that’s the question that they’re fundamentally trying to answer right now.

Anthony: Where are you on that, Ryan? We know the discomfort, the multiple screens, the ergonomic issues that you have to deal with. You’ve used this Apple Vision Pro. What are your thoughts? Are you saying ‘oh my god, this is unbelievable’ or are you saying ‘I could see potential here but we’ve got some refining to do.’ I don’t know if there’s any conversations you can have with Apple, it’s sort of like a ‘hey, here’s some feedback.’ Maybe they don’t pick up the phone. You just got to use it as it is.

Dr. Broderick: I think this is a game changer. I think this is seeing is believing. In the first cases when you put this on, the surgeon who is operating is like “oh my gosh,” taken aback by the quality of visualization, the comfort of where that digital screen is put in physical space, and it is mind blowing. We really say seeing is believing.

We have multiple headsets that we can run at the same time now. After a few cases we had me and the fellow wearing it at the same time and they hear me talking about how great this is and how awesome and then when he goes to put the headset on, and we do a case with him, he’s again mind blown, just beside himself. He said, “you can’t even imagine how amazing this is until you get your head in it.”

I think it’s a game changer. We’ve done a dozen cases now. I have put together a short video clip (see below) so you can kind of see how we set up the operating room and how we do this and kind of what the surgeon in seeing in their headset with the multiple video streams coming through and how we’re doing cases, and we can link that to your website.

We’ve done a dozen cases. The patients have all done absolutely perfect, zero complications, zero concerns. Nobody has returned to the ER. Everybody has had a very good outcome in line with what we’d expect from our standard laparoscopic surgical practice. But what we’re studying is how does the surgeon really feel when they’re using this, what is our cognitive load, what is our physical load. We’re using the NASA TLX Score to determine how important this might be in application to our practice and we’re seeing very high ratings from the four surgeons that have used it already, and we expect pretty big things.

We’re going to continue to use it. We are currently slated for up to 65 patients. We’re planning to expand our IRB now that we’re going to start incorporating ophthalmology and our interventional pulmonologists and our endoscopists, et cetera. We’re going to expand this out to see what kind of potential we have in other proceduralists and also this scratching the surface of capabilities. Right now, it’s just a digital monitor. There may be some role for being able to mark on the screen, for better education for residents, for medical students, you mark the planes of where we’re going to do our dissection that you could see that on their digital screen. The sky is the limit in this case.

As far as talking with Apple, certainly they’re aware of what’s going on here and they put this out there for a lot of people to think about other sources and other things that they can do with it. All this feedback goes to people that are working in the medical field as well as Apple. Like Chris mentioned, I think there’s going to be some work on the back end from multiple different companies now looking to make app-based solutions to really streamline this to a broader audience.

Anthony: It’s wild. It’s like science fiction, it’s coming really fast. I wrote down a couple of things as you were talking. One is, are you getting to the point where you would prefer, almost to the sense that you don’t want to do surgery without it?

Dr. Broderick: I’d say it’s too early to say that but every case that I do, I’m asking to use it.

Anthony: It’s going to be a big difference, right.

Dr. Broderick: A big difference.

Anthony: We’re going to go quickly from trying it out to possibly saying, I prefer this. It could also happen on the patient side as they become more educated about it. You can go from someone saying ‘I will allow you to do this because you’ve explained it’ to ‘I want you to use this because you said you like it better. I want you happier when you do my surgery so use it please.’

Dr. Broderick: A hundred percent.

Dr. Longhurst: Anthony, I’m not a surgeon but I had the privilege of watching Dr. Broderick and his colleagues in one of these cases and I can tell you that it’s clear the difference between looking at a virtual 150-inch screen that’s right in your operating field and being able to see your hands in the operating field at the same time versus looking at the 12-inch screen that is 45 degrees to your hard left. It’s a far better experience for the surgeons as Dr. Broderick described and so as a patient, I would want my surgeons to have the best possible experience when they’re operating on me.

Anthony: Does it take a little bit to get used to it, Ryan? Or is it almost right away?

Dr. Broderick: No, you get it right away. The monitor, the visualization of the surgical field is the same as what we would have. Like I said, we’ve gone through it to make sure this is very safe. It’s the user interphase that you get used to a little bit, just how to grab and move the screen and space and how to increase the size and that sort of thing, just device settings really that take very little time to get used to, but the rest is really just an improvement on the monitors, the visualization that we have.

Anthony: We know that there’s competition between health systems and certainly you have surgeons that practice at multiple hospitals, right. This can become a competitive advantage. You could have a surgeon that says I prefer to do my surgeries over here because they allow me to use something like this, over here, they don’t. Can you see this becoming a competitive advantage, to be attractive to clinicians?

Dr. Broderick: Yeah, very much so. There’s potential for that. I think you can envision a world really where each surgeon is carrying around a small briefcase with their own Vision Pro or AR headset that suits them, that’s calibrated to their vision. They can then talk to your IT systems and make sure everything’s above board and safe and protected rather than having individual headsets in each room. I think this is going to end up becoming the new age doctor’s briefcase.

Dr. Longhurst: I would agree with Dr. Broderick. He alluded to some of the opportunities moving forward that I think it’s going to be beyond just education. I think with these images being digitally processed on the way to this augmented kind of reality headset, we’re going to see vendors changing their hardware so that surgeons can see in three dimensions rather than just two dimensions. I think that we can also introduce things from a safety standpoint to help provide those guardrails that you only see, for example, in robots today. There’s a lot of excitement and I do think solutions like this will be part of the future’s briefcase.

One note of caution I would say though is just like every doctor carries a stethoscope, but it can be easily interchanged. These headsets are not easily interchanged. The eye tracking has to be configured for each individual using it. If you wear glasses, it’s got to be pre setup with the right optics inside of it. That is going to raise the bar in terms of showing outcomes that matter because before the health systems are investing in headsets like this for every proceduralist, we’re really going to have to understand the difference that it makes because it adds to the cost of operating and providing care.

Anthony: Absolutely. Let me just ask you each one more question and I think we’ll have it pretty much covered. What would be your best advice to someone in your position at a comparably-sized health system across the country that wants to try this out.  What’s your best advice, Ryan, to again someone in your position at another health system?

Dr. Broderick: First, I congratulate all the forward-thinking folks out there. We try to be the tip of the spear at UCSD between JCHI and the Center for Future of Surgery. I really like all those forward-thinking tip-of-the-spear folks. Right now, it is still under study. There’s going to be stuff that’ll need to be FDA reviewed and approved. If you’re looking to do it at your own institution, go through those Institutional Review Boards, if you can do it in a dry lab or wet lab setting, definitely get that ball rolling and then once you’re feeling more comfortable, introduce it and make sure you’re doing it with full consent process and with all the protections in place in the operating room.

Anthony: Chris, what about you? What’s your advice to your peers out there if they’re really intrigued and want to move along with this?

Dr. Longhurst: Listen, we’re the first in the US trialing this device in the operating room environment. There was a cardiologist I think at Stanford. He used it in a cath lab environment and I think we’re going to see more of those coming. In the early days which we’re still in, they’ll either be done under IRB or through innovation committees and of course, we’re happy to partner. But not all health systems have the resources to be able to do that sort of thing.

My advice to those who are eager but not resourced to do it would be to just be patient because Dr. Broderick and Dr. Horgan and colleagues will publish the results and as those publications come out, that helps to move things from early innovation to standard practice, and we’ll see more of that occurring particularly as the vendors of the minimally invasive surgery devices get more involved as well. It’s a very exciting time in between AI-based solutions and augmented reality glasses, I think that Dr. Broderick is leading the evolution of his field.

Anthony: That’s wonderful, gentlemen. I appreciate your time so much today. I think people are going to be really intrigued and enjoy this. Thank you.

Dr. Broderick: Thank you.

Dr. Longhurst: Our privilege. Thank you, Anthony.