It may be a cliché, but for security leaders, knowledge is definitely power. And that knowledge must come from a number of directions. First off, CISOs and their teams must be ingesting the latest threat intelligence to know what the bad guys are up to, but that information can only be acted upon in a timely manner if they also have knowledge of their systems, according to WellSpan Health Director of Information Security Mike Shrader. Shrader, who essentially functions as the organization’s CISO, says the needed knowledge doesn’t stop there. To run a tight identity and access management program (often cited as an absolute key to any security program) cyber teams need to be kept in the know as roles change so permissions can flex up and down with them; the down part being key. In this interview with healthsystemCIO Founder & Editor-in-Chief Anthony Guerra, Shrader discusses these issues and many more.
LISTEN HERE USING THE PLAYER BELOW OR SUBSCRIBE THROUGH YOUR FAVORITE PODCASTING SERVICE.
.. we also have to have the knowledge of our systems. We have to know where is that (threat) applicable in our environment, what is truly exposed, what can be taken advantage of quickly …
As soon as HR finds out someone is terminated, we need to make sure that that is automatically fed into our system to deactivate those accounts as quickly as possible because otherwise accounts linger on. We’ve all heard stories like so and so left two years ago, the account is still active and they’re logging in or someone else is logging in with those accounts. Lifecycle management is absolutely huge to stay safe.
How do we make sure we can operate as a system if or when the (electronic) systems are down? We’re actually doing tabletops on these now – business impact analyses with third parties to help us really identify where those gaps are and make sure we address them.
Anthony: Welcome to healthsystemsCIO’s interview with Mike Shrader, Director of Information Security with WellSpan Health. I’m Anthony Guerra, Founder and Editor-in-Chief. Mike, thanks for joining me.
Mike: Thanks for having me, Anthony.
Anthony: Great. Looking forward to having a fun chat. Mike, let me start off by asking you a little bit about your organization and your role.
Mike: As you said, I’m Mike Shrader. I’m Director of Information Security for WellSpan Health. We are an integrated delivery network in south central Pennsylvania as well as the northern part of Maryland. We have 20,000 employees. Of that 20,000, 2,000 are employee providers. We have 220 locations across south central Pennsylvania and Maryland. We are an 8 hospital systems along with home care services as well as behavioral health.
Anthony: Very good, Mike. Do you report to a CISO or do you function as the CISO?
Mike: At this point, we do not have a CISO but I would be the highest functioning leader with security as my focus.
Anthony: Is that an open position or they’re going to fill it or they’re going to create it or they’re just going to stick with you as the director?
Mike: At this point, we haven’t decided if we’re going to have a CISO or if one is necessary. But I do report to our Chief Technology Officer who reports to our CIO. We all have a heavy focus and paranoia around security but at this point, no, we do not have a CISO.
Anthony: Excellent. Tell me, I’d like to start with an open-ended question, see what’s on people’s minds. What are some of the things you’re thinking about, looking at, trends you’re watching?
Mike: There are things that you have to think about. Things that are always in the news. Things that are constantly running through either my news feeds, my social media feeds, everything is related to some attack that is happening. Majority of those are ransomware. The topic, it’s probably top of mind for most healthcare organizations and organizations, in general.
We constantly have to review and evaluate the different techniques and compromise methods that are disclosed. Not everybody shares all the details but we pay close attention to all the details that are shared whether it’s publicly or through information sharing channels that we have. These are things that are always top of mind for me to make sure that we’re always staying ahead of that. Because patient safety is top of mind for not only everyone in our organization but for me, as a focus, how do I play into that and how does my team help make sure we can see and treat patients effectively.
Anthony: There’s actually a lot there, right? Let’s talk social feeds. It’s like drinking from a firehose. You feel like you could never stop. You hope the algorithms are helping make sure you see the most relevant stuff but you never know.
Mike: I’m hoping the algorithms are helping me get the most important thing, right. But after you watch the same folks, you really have to curate who you follow. Make sure you trust who you follow in those different social feeds. It’s really important and making sure that the information they share is credible. And then also where I usually find the most value is finding folks that are saying the same thing or are sharing the same articles, so then there’s a little more legitimacy behind those.
Again, you still have to read every article and take out of it what you can. I don’t know if there’s a secret sauce to that but that’s what I found, has been – I think has been effective for us or it feels has been effective for us, at least from my perspective. I’m the one that keeps track of all the social stuff for our groups between all the different social media platforms, whether it’s X or LinkedIn. And then seeing is the same information being shared in both areas because those seem to be the most credible when you correlate them together.
Anthony: As you said, security folks are looking at the info coming out, the vulnerability and possible patching, and they have to be fast to get those things addressed, because the bad guys get that info too, right?
Mike: It’s a dance for sure. The time we have to address vulnerabilities has significantly decreased over the last few years and it’s one of those things where we get a lot of information. There’s a lot of public information out there. The attackers are also leveraging that same information which makes them more likely to act quicker because now that everyone knows about the attacks, they’re out there, they have to move fast if they want to take advantage of them, right? We have to evaluate quickly too.
Us having not only the information that’s shared with us, so we have the knowledge on hand, we also have to have the knowledge of our systems. We have to know where is that applicable in our environment, what is truly exposed, what can be taken advantage of quickly, is it something that’s internet facing versus it’s behind our firewall so we have a little more time but we still need to move as quickly as possible. Yes, those timeframes are really shrinking but we have to be very transparent with our internal teams and partner with our internal teams. I think that’s something that we do really well in our organization here. It’s a partnership.
We understand the impacts of not addressing certain vulnerabilities and what could happen. But also the other side of this is it’s not just, you can’t just let one hole breach your systems. We have to have different defensive depth mechanisms of yes, we didn’t patch this one system this one day, somebody got in, there has to be other things that are there to either prevent or detect those attacks. Yes, we need to patch as quickly as possible but we also have to have those layers to help us protect ourselves and our systems.
Anthony: Really good points. It seems like that knowledge of what you have is so important. It’s not as easy as it sounds (I’m thinking of log4j), but it’s critical, right?
Mike: It’s critical. It’s one of the foundational parts of any good cybersecurity framework that you work with, whether it’s from NIST or whoever, understanding your assets is critical. For us, you get that information a lot of different ways; such as your vulnerability scanning. You scan all of your systems. You look for all the different vulnerabilities. That’s going to help you know what’s vulnerable, but then you have to have some asset tracking system so that way you can tie that all back together.
But you have to be able to understand what those assets are and so really working with the different teams to understand at least where you need to go to get the information. I don’t need to know what every single system does, all the servers that tie to each system, but we need to know who owns those systems and those system owners need to understand their infrastructure.
If we say, for instance, when you talked about log4j, the fire drill that it was, understanding and telling the teams, ‘hey, we found this vulnerability; it is very critical that you patch your system and here’s why’ but they need to understand, either they need to look into their systems, if they find that component themselves or knowing where to get that information from if they need to reach out to their vendors.
Anthony: What you’re talking about, is that a communication between you as security and the business owner? Or is it you as security and IT who is going to liaise with the business owner? I can’t imagine many business owners are going to have much technical knowledge about the applications they’re running and they’re going to say ‘I don’t know.’
Mike: You’re absolutely right. The way we’ve handled it in the past is we’ll gather the entire IT team – Because our IT teams will support the business. So they may not own the application themselves, per se, so they may not understand how the business uses it completely, but they understand the functioning of that and they’ll have to work with their business owner to relay any impacts of having to patch or take their system down for maintenance or whatever is necessary at our request, from a security perspective.
Anthony: You talked about knowing what you have. This is obviously a huge point in medical device security which comes up, getting an inventory in that area is very, very difficult and maintaining it, but it’s an important element of providing good security because you can’t secure or patch what you don’t know you have. That’s a tough one, right?
Mike: Yes. Medical device security is super important. We have to keep these devices secure so that they can efficiently and effectively treat our patients. It’s really important that we keep all those assets tracked. That is part of our standard work, of all of our biomedical technicians, so making sure that those things are tracked and into our inventory system. We also make sure that we have those devices up to date in the systems so that if there are recalls put in place, that we’re notified. We have some integration with some partners who send us FDA recalls since they’re now including cybersecurity.
So again, going back to Log4j. Log4j was found in these medical devices, and then it gives you guidance. Here are the compensating controls, here’s the mitigating factors or here’s the patch that you need to deploy, and then working through that. Because we can’t scan those devices, for the most part. We try not to use active scanning software on our medical devices, just due to some of the legacy technology.
Anthony: Right. There’s a fear that it can be damaged or disrupted by the scanning.
Mike: Right, absolutely.
Anthony: That’s tricky. Very, very interesting. Very interesting. You talked about defense in depth, right. Some CISOs I talked to said essentially that there’s the potential for a single click by an employee with mid-level permissions to take down the organization (through ransomware). That’s scary.
Mike: If a single click can take our entire organization down, then I think I may have failed in some areas. But yes, so from a defense in depth standpoint, yes, people are going to click on links. We’ve run our own phishing simulations that we send for awareness. We know folks are going to click on links. So we have to make sure that we have, again, defense in depth. Make sure we understand, we track those links that they click on within their email, using various technologies that we have so that we can react. Once we know that that link is bad or what that link does, we can take action on not only the user but maybe their device.
If they completely bypasses those technologies, having advanced tools on your endpoint to look for anomalous activity, having your SIEMs – your security incident and event management tools – having correlation and logging from all the different network systems, applications, any systems that you can get information from, getting that data into one place and having it looked at for anomalous activity is key.
But really, you mentioned this earlier – it sounds like they may have had some privilege on the systems. The key there is to make sure they were really limiting that privilege that the user has when they’re checking their email. Because all 20,000 of our users have access to email. We all use it in one way, shape or form and the user running that has to have these privileges and that’s something to push to make sure that when you need privilege, you need to escalate it using different accounts, really limiting that impact from a click; and there’s many other levels of defense in depth that we can go through but just to keep it at a high level and to answer your question, a click should not take all systems down now. There’s a lot of hoops they have to jump through after they get access to the system through clicking a link. But yes, defense in depth is key.
Anthony: You mentioned something I hadn’t heard this before – escalation and de-escalation of a user staying in the same role based on what they’re currently doing. Is that what you meant? Because I hadn’t heard that before. I’m in the same role, I haven’t change roles but based on what I’m doing, my permissions are going up and down. Is that what you said?
Mike: Not specifically for that user. When we talked about escalating your own privileges, there’s software that we can allow escalation of certain applications. If you have an application that says administrator, your (whole) account is not going to run that. We’re going to give the privilege to that application to run that. That way we’re not going to run the email client as an administrator.
But the other point of that escalation or elevation would be I have my own administrator account, I don’t log into my system with my administrator account, I just use it to connect to another server or connect to an application to do administration. Other things we are looking at is ways to even limit those accounts to then escalate as needed or elevate, request to elevate, you have to prove that I need this to do something that I don’t normally do.
I need to request that and have it approved by whoever, whether it’s the system owner or IT manager, to provide to whomever, so really making a lot of these administrative tasks and elevations are intentional and not just by default just because it’s what we’ve always done.
Anthony: Based on what you’re saying, what everyone is saying, identity and access management is a huge part of good security but doing it is really hard when you have a big health system, right?
Mike: Yes, identity and access management is very hard. That is something that has been new to me when I came to WellSpan nine nine years ago at this point. Dealing with an organization this large, we have 20,000 employees, we have an additional non-employee population, contractors, non-employed physicians, nursing students, folks that are not employed directly by WellSpan. You have to manage all of those identities as well.
Having a really good identity and access management platform to make sure that you have automation in place, HR processes are key as well, on-boarding, off-boarding, making sure that timely terminations occur. As soon as HR finds out someone is terminated, we need to make sure that that is automatically fed into our system to deactivate those accounts as quickly as possible because otherwise accounts linger on. We’ve all heard stories like so and so left two years ago, the account is still active and they’re logging in or someone else is logging in with those accounts. Lifecycle management is absolutely huge to stay safe.
You mentioned transfers. Transfers is a big challenge within healthcare because we constantly have folks moving to different areas of the hospital, they need different rights in different areas. Creating roles is a large effort to make sure one specific nurse is going to have a different set of permissions than the nurse in another area of the hospital. Making sure that they can do what they need to do and then make sure that those capabilities and permissions transfer as they need to as they move out.
Anthony: You don’t want to just keep adding, right. You just don’t want to keep adding rights. Maybe they did something where they need more rights and then at some point, they no longer need them, that’s got to roll off, right?
Mike: Correct, yes. Temporary, when you have so and so transfers from A to B. They’re going to be doing A’s job for the next two weeks as well as B’s job, so making sure you’re tracking all of those and make sure you strip those privileges after the transfer completes. Those are things that are always going to be a challenge and finding different ways to address those but it really comes down to making sure that you have certification processes, auditing and ways to follow up, just have regular check-ins with leadership who are responsible for the staff members to say hey, do you still need these privileges for these users, on a periodic basis.
Anthony: It must be tricky to ensure that IT and IT security gets notified of all those changes on the business side. Working with HR must be critical.
Mike: HR is just one of the many important relationships any security leader needs to have. We work very closely with HR, not just myself but the identity and access management team must work close with them. Mainly because, by policy, if an account is disabled because of a termination, we didn’t want to go too far on that but we’re not just going to take the word of the employee that says, ‘hey, I can’t get in.’ We take the information from HR.
We have to work with them to say, ‘okay, can we re-enable so and so’s account so that they can finish their work or continue based on what they need to do.’ But yes, constantly working with HR to better our processes. Some challenges we’d have to work through are things like leave of absence, things like, ‘hey I need to take off 4 weeks for whatever reason I may have, it’s approved through HR,’ how do we make sure those accounts are secure while I’m on leave or make sure that I am on leave, how do we better partner in those areas?
HR deals a lot with employee issues. We are tightly integrated if there’s audits that need to be performed. We don’t do audits unless HR directs us. We’re not the internet police. We’re not the network police. I’m not watching folks work, contrary to popular belief. But we have to work very closely with them because we support a lot of the work that they do and they rely on us to provide that information to backup whatever the truth may be in whatever situation they’re working through.
Anthony: Did you say WellSpan has a research arm?
Mike: We do, yes.
Anthony: I’ve heard from many that that certainly elevates your profile in terms of risk because that’s one of the things that bad actors are looking for. They want research information. With research, also, there can be requests for data sharing which is something that probably has to go by you, can we send this data over here. Research can sometimes involve travel.
There was one story told to me about an individual from the health system who did not check with IT and just went on an overseas trip, I think to China, and when that was found out about, their laptop was remotely disabled because they said no, no, no, lots of stuff. Any thoughts around all that?
Mike: Research became a big item during the pandemic for sure. Working with our different research teams to see how can we better support them. We protect everyone but are there additional things we could do for our research folks, see what different technologies, education is out there – can we provide additional education to them in different areas. This isn’t necessarily research specific for us, but when we talk about travel, we do restrict access to our systems from outside of the United States so our firewalls do not permit network connections outside of our business region.
We don’t do business outside of south central Pennsylvania but we do all our connections from within the United States. Whenever our folks travel, they have to work with us (ahead of time) or really they won’t be able to do work while they travel. There are different things that we have to work through and again, it goes back to our partnership with HR. If anyone reaches out and says, ‘hey, I can’t access the VPN from Canada.’ We understand your concern. We need you to work with HR to make sure that you’re approved to work while you’re travelling, ‘is it for business, is it for personal reasons,’ and then working with them to make sure that that travel is approved and then we find different ways to accommodate that.
Anthony: Very good. Let me ask you one more question, then I’ll get your final thoughts and let you go. I’ve been focusing a lot on business continuity planning and how CISOs can work with emergency management. What are your thoughts?
Mike: Yes, this is a huge topic for us and I think many other healthcare organizations, especially in light of all the attacks that are going on. Everybody has to answer those questions for their leadership and their boards. They want to know these things. You talked about emergency management. We work very closely with that team. We’re actually part of their assessments. When they say they do their risk assessment, floods, different scenarios, IT and cybersecurity is one of the risks that they rate in all the different locations that they service throughout our system.
You mentioned the business continuity piece of it. How do we make sure we can operate as a system if or when the systems are down? We’re actually doing tabletops on these now – business impact analyses with third parties to help us really identify where those gaps are and making sure that we address them. If we have plans in place, great, let’s test them out. Let’s make sure they actually work. We do a great job of keeping our systems up so a lot of folks have never really had to work off of paper. A lot of the newer clinicians that maybe coming into the system haven’t experienced any extended downtimes but hardware problems happen. Hopefully cybersecurity events never happen but it’s inevitable that systems will be down at some point, so we have to make sure that we’re prepared to do that.
Our role as cybersecurity experts in those situations is making sure that the different business partners understand what is available, what’s not, what’s safe to do and what’s not, so even if the system may look like it’s online, we may caution them not to use that just because it may not be available or it may be at risk. Each leader in all the different areas from a clinical standpoint and operationally, such as HR, needs to be prepared. How is HR going to onboard new staff if the systems are not online? Supply chain, how are we going to get the supplies we need, how are we going to coordinate deliveries, things like that.
From a clinical standpoint, we have to continue to treat patients. ‘Do you have the charts you need, do you have paper to be able to do that charting, do you have everything you need to continue to treat the patients in any extended downtime or incident coming in?’
Anthony: And again, you’re not making sure that they have paper. That’s emergency management.
Mike: Yes, right.
Anthony: You’re not going to Staples and buying ink and putting them in the closet, right.
Mike: No, we can help buy. I’m capable enough to carry some paper, but no, I’m definitely no clinician, for sure. (laughing)
Anthony: But those conversations must get interesting, when you’re describing a potential outage, how it could evolve, to emergency management. I would imagine they’re eating it up in the sense of you’re really educating them. Because even if you took me though it, I’d be like ‘wow, really?’ If you were to scenario it out, like, ‘here’s what’s going to happen, here’s what might happen, here’s scenario A,’ – is that how these conversations happen?
Mike: Oh yes, yes, absolutely. ‘Hey, what’s the worst case scenario,’ or, ‘here’s what actually could happen in our environment. The health records system is down, phone system down. You only have some analog phones that may exist in certain places. Email is definitely not available and what do you do with that situation?’ And then maybe in two days we’ll tell you something is back up and you may be able to use it. ‘Right now we are digitally dark, so we need to figure out how to do this.’ Our emergency management team is very – like I said, we are very involved with them because they’re the experts in that area. They know what happens in major emergencies. Without any IT systems, it’s definitely going to be an emergency.
Anthony: Mike, I can keep you all day but I’m just going to get your final thoughts and let you go. I really could. Any final piece of advice for someone in your position at a comparable sized health system based on your career, what you’ve achieved, things that you feel you bring to the table? Any piece of advice for your colleagues?
Mike: For me, I think the keys to success for any security leader are collaboration and empathy. Most cybersecurity folks that I have worked with have worked in all different areas of IT so they understand the impacts of security in all the different areas. Asking somebody to patch 100 systems is like, ‘yes, I have been there, I know it and I feel for you but we have to do those things.’ Understanding the impacts of what you’re doing and being empathetic but collaborative is key to success in a lot of different areas. We can’t be secure in a silo. We can’t fix and prevent everything on our own so we have to work together to be successful.
Anthony: Excellent. Excellent interview, Mike. A lot of great stuff in there, like the empathy. Appreciate your time today. Thank you so much.
Mike: Thank you very much, Anthony. I appreciate it.