We Bring the Cloud to Healthcare

Save Your Data (and Your Budget) from Cyberattacks

October 3, 2017


Save Your Data (and Your Budget) from Cyberattacks

MEDITECH Guest Blog Article by Jim Fitzgerald

Republished with permission.

It cannot be overstated how much of a disaster cyberattacks are on hospitals. Despite the severity of the crime, the moral, legal, and cultural norms that would stop a prospective vandal from crossing a physical fence and painting graffiti on your hospital don’t seem to apply on the internet.

I believe that enforcing a stronger deterrent for these crimes is just the beginning of accountability in the world of cybersecurity. We need to change the way we see cybercrime and begin to reconsider the punishment for those who attack us. Candidly, the only way to fully understand what a cyberattack can do to a health organization is to use the one metric everyone understands — money.

I have been living healthcare disaster recovery for nearly 30 years. The typical recovery time from a system’s problem can be five to seven days when a hospital has no plans or preparations, or less than four hours when the customer is hyper-prepared either in their facility or in a cloud service like CloudWave’s OpSus Recover.

Based on casual observations of incidents in the MEDITECH community for the last nine months, the down period from a cyberattack is an average of five to 20 days. Being in a dire situation where those who need access to vital data is truly a disaster. With that being said, there is one aspect that often gets overlooked: revenue.

The financial ripple

Downtime has a ripple effect throughout the entire hospital. When you start adding in A/R days, and the ability to sustain the cash flow to operate the business, it can be catastrophic to an organization.

The financial damage is enough to make your jaw hit the floor. An oft-cited healthcare industry study from 2014 suggests that a typical cybersecurity breach costs the organization $810,189 per incident (seriously, who calculates these numbers?) and that an unplanned system downtime is nearly half the price at $432,000 per incident.

For instance, Medicare and Medicaid require online documentation for billing for level 2 and above procedures. So if the hospital is down, you have to wait until you’re restored your system and then hire extra staff or even part-time consultants to manually transcribe paper records that are kept during downtime into the EHR so you can properly bill for those services. Best case, it causes a long delay in the task cycle. Worst case, you’re going to lose procedures that you should be able to bill for and that’s going to be a direct loss of revenue.

This isn’t examined too closely too often, but the last study funded by EMC commissioned by the Meritalk found that the average downtime cost per hour is $32,000. That figure includes human cost, lost revenue, inconvenience, and everything else.

So what can you do to keep your revenue streaming? It starts by defeating cybercriminals.

The three options to defeat cybercriminals

Most security experts agree that a sufficiently motivated and equipped hacker is hard to stop, which begs the question – “How do we protect ourselves?” or at least “How do we mitigate the impact of a potential attack?” The simplest and least expensive thing to do is to look at the existing backup and recovery strategy and make sure that you have an “air-gapped backup.” Having an air gap means that the backup is dropping to either an old-school physical tape library, which no one really does anymore or a virtual tape library or disc array of some kind that isn’t on any other network within the hospital and is only a resource to the backup servers.

A better option would be to make sure you have an air gap and also make a copy out to a cloud service. This gives you a geo-isolated copy that is definitely not on a hospital network that probably won’t be subject to any virus, malware, or phishing attacks on the local hospital network.

The best option isn’t a backup at all, since that’s only treating the disease after it happens. A preventative strategy might be to assess your security infrastructure and operations and start to fill gaps by adding needed security components and services to your annual IT operating budget on a permanent basis. Hardened security appliances like IDS/IPS inspect every packet of traffic coming in and out of your network. Periodic vulnerability scanning evaluates your internal and external security perimeters. Log analysis and examination reveal flaws that are often easily remedied by re-configuration or procedural means.

It’s worth repeating, nothing will stop all attacks. Some phishing attacks, for example, depend on social engineering or a pre-existing vulnerability to bring the virus or malware into the building. But widely known attacks, especially the type of attacks that were involved in recent crypto-like blocker-type attacks and WannaCry, have known “signatures” that could be caught in a well-maintained IDS/IPS or in a virtual pre-execution environment like that pioneered by Fireye and integrated into Microsoft Office 365.

A well-maintained IPS/IDS will probably stop 19 out of 20 Cyberattacks, if not more. A good, external web-based email service like Corporate Gmail or Exchange Online will vastly reduce the number of email-borne attack attempts.

The dollars and cents of it

From a CIO’s budget perspective, you have two strategic long-term choices to make. One is to either add a permanent recurring capital budget line each year that includes tech refreshes, software licensing, and maintenance to have an adequately comprehensive security infrastructure. Another approach is to move workloads onto a cloud service where the cloud service provider is mandated by contract, industry norms, and federal compliance standards to protect your data. Cloud providers who do not meet industry norms for security fail their SLAs and are penalized, so chances are, being compliant is a top priority for these companies.

For some hospitals, this is a significant, and not previously considered financial commitment, complicated by the current era of shrinking reimbursements. A few years ago, it may have been OK to defer cybersecurity investments in favor of the latest MRI machine. Now we need to give serious consideration to keeping the data from those expensive clinical modalities from being hijacked.

I have heard stories where a hospital had downtime from a disaster only to find themselves in yet another. What happens when you go to restore and the only good backup is three to four days old just because you haven’t been monitoring successes or failures during nightly, daily or in-progress backups? In other words, if it’s Wednesday and your most usable backup is from Sunday, you have lost all of the bills you’ve generated on Monday and Tuesday. They’re just gone. To reconstruct those manually will be a laborious, slow process consisting of combing through other data in the system to try and reconstruct what has happened. The simple step of monitoring your backups can really go a very long way.

From a patient care perspective, what matters is that when an employee of a hospital needs to access specific data, it is correct and it is readily available in a secure environment. When a cyberattack inevitably slips through all the safety nets, walls, and fences, a solid backup is your best basis for restoral. I’ve laid out a few options to strengthen that environment, but it doesn’t matter which option you choose if you don’t actively test your backup system. Monitoring the success of daily backups is a simple thing that everyone can do right now with whatever equipment they have.

If you choose to not be proactive with your data and computer security, there’s yet another hurdle you have to overcome just to get your environment certified safe so that data restoration can get underway. Security assets are very limited right now – there are only a few thousand information security professionals in the United States, and only a small percentage are qualified to do analysis and remediation work. So, getting your hands on those resources during a crisis is not a sure thing – although this is an area where a good cyber insurance policy can make a difference, as the Insurance provider often has these resources on retainer.

Make a plan and stick to it

During and after a cyberattack, it’s critical that all of the stakeholders involved work together as a team. Whether it’s the hospital IT staff living on coffee and encouragement or it’s the out of town malware consultant on her third-consecutive case, there is a camaraderie that happens when everyone only cares about one thing – getting a hospital’s data acquisition and management systems up and running securely and quickly.

The hospitals that take disaster recovery seriously usually have a plan in place, rehearse, and drill for disasters. And this could be for disasters other than IT-specific. They could have a physical on-campus disaster that IT survives such as a bomb threat against a wing of the hospital, or a physical disaster such as the devastating flooding in Texas and Florida.

Those hospitals have a cross-functional team that is made up of many different departments in the hospitals. Usually, the CIO is the leader in that effort, but it’s just as likely a non-IT person such as a clinician, nurse or administrator is in charge. Typical drills and disaster rehearsals based on a written disaster recovery plan happen once a year, sometimes twice. Those plans can be as detailed as “go to the closet with the red door and pull out the paper forms.” It depends on the hospital, but these groups would benefit greatly from planning a cyberattack rehearsal to their annual agenda.

The playbook for these recoveries is being written in real-time, in the field. Hospitals and their supporting cast like MEDITECH and CloudWave plan, design around, rehearse and openly work to prevent disasters. It’s time to bring that level of planning, diligence, and investment to the prevention and remediation of cybercrime, no matter the cost.

Jim Fitzgerald is EVP and Chief Strategy Officer at CloudWave. He values the role of community hospitals in healthcare and works to create solutions and services that allow them to flourish. Jim brings 25 years of experience with MEDITECH in both technical and executive roles. His contributions include co-architecting and leading “first of their kind” cloud-based technology solutions for healthcare. Jim is passionate about helping hospitals of all sizes and types implement sustainable technology strategies that enhance the performance and availability of MEDITECH and other enterprise software, whether in a hospital data center or in the cloud.

It cannot be overstated how much of a disaster cyberattacks are on hospitals. Despite the severity of the crime, the moral, legal, and cultural norms that would stop a prospective vandal from crossing a physical fence and painting graffiti on your hospital don’t seem to apply on the internet.

I believe that enforcing a stronger deterrent for these crimes is just the beginning of accountability in the world of cybersecurity. We need to change the way we see cybercrime and begin to reconsider the punishment for those who attack us. Candidly, the only way to fully understand what a cyberattack can do to a health organization is to use the one metric everyone understands — money.

I have been living healthcare disaster recovery for nearly 30 years. The typical recovery time from a system’s problem can be five to seven days when a hospital has no plans or preparations, or less than four hours when the customer is hyper-prepared either in their facility or in a cloud service like CloudWave’s OpSus Recover.

Based on casual observations of incidents in the MEDITECH community for the last nine months, the down period from a cyberattack is an average of five to 20 days. Being in a dire situation where those who need access to vital data is truly a disaster. With that being said, there is one aspect that often gets overlooked: revenue.

The financial ripple

Downtime has a ripple effect throughout the entire hospital. When you start adding in A/R days, and the ability to sustain the cash flow to operate the business, it can be catastrophic to an organization.

The financial damage is enough to make your jaw hit the floor. An oft-cited healthcare industry study from 2014 suggests that a typical cybersecurity breach costs the organization $810,189 per incident (seriously, who calculates these numbers?) and that an unplanned system downtime is nearly half the price at $432,000 per incident.

For instance, Medicare and Medicaid require online documentation for billing for level 2 and above procedures. So if the hospital is down, you have to wait until you’re restored your system and then hire extra staff or even part-time consultants to manually transcribe paper records that are kept during downtime into the EHR so you can properly bill for those services. Best case, it causes a long delay in the task cycle. Worst case, you’re going to lose procedures that you should be able to bill for and that’s going to be a direct loss of revenue.

This isn’t examined too closely too often, but the last study funded by EMC commissioned by the Meritalk found that the average downtime cost per hour is $32,000. That figure includes human cost, lost revenue, inconvenience, and everything else.

So what can you do to keep your revenue streaming? It starts by defeating cybercriminals.

The three options to defeat cybercriminals

Most security experts agree that a sufficiently motivated and equipped hacker is hard to stop, which begs the question – “How do we protect ourselves?” or at least “How do we mitigate the impact of a potential attack?” The simplest and least expensive thing to do is to look at the existing backup and recovery strategy and make sure that you have an “air-gapped backup.” Having an air gap means that the backup is dropping to either an old-school physical tape library, which no one really does anymore or a virtual tape library or disc array of some kind that isn’t on any other network within the hospital and is only a resource to the backup servers.

A better option would be to make sure you have an air gap and also make a copy out to a cloud service. This gives you a geo-isolated copy that is definitely not on a hospital network that probably won’t be subject to any virus, malware, or phishing attacks on the local hospital network.

The best option isn’t a backup at all, since that’s only treating the disease after it happens. A preventative strategy might be to assess your security infrastructure and operations and start to fill gaps by adding needed security components and services to your annual IT operating budget on a permanent basis. Hardened security appliances like IDS/IPS inspect every packet of traffic coming in and out of your network. Periodic vulnerability scanning evaluates your internal and external security perimeters. Log analysis and examination reveal flaws that are often easily remedied by re-configuration or procedural means.

It’s worth repeating, nothing will stop all attacks. Some phishing attacks, for example, depend on social engineering or a pre-existing vulnerability to bring the virus or malware into the building. But widely known attacks, especially the type of attacks that were involved in recent crypto-like blocker-type attacks and WannaCry, have known “signatures” that could be caught in a well-maintained IDS/IPS or in a virtual pre-execution environment like that pioneered by Fireye and integrated into Microsoft Office 365.

A well-maintained IPS/IDS will probably stop 19 out of 20 Cyberattacks, if not more. A good, external web-based email service like Corporate Gmail or Exchange Online will vastly reduce the number of email-borne attack attempts.

The dollars and cents of it

From a CIO’s budget perspective, you have two strategic long-term choices to make. One is to either add a permanent recurring capital budget line each year that includes tech refreshes, software licensing, and maintenance to have an adequately comprehensive security infrastructure. Another approach is to move workloads onto a cloud service where the cloud service provider is mandated by contract, industry norms, and federal compliance standards to protect your data. Cloud providers who do not meet industry norms for security fail their SLAs and are penalized, so chances are, being compliant is a top priority for these companies.

For some hospitals, this is a significant, and not previously considered financial commitment, complicated by the current era of shrinking reimbursements. A few years ago, it may have been OK to defer cybersecurity investments in favor of the latest MRI machine. Now we need to give serious consideration to keeping the data from those expensive clinical modalities from being hijacked.

I have heard stories where a hospital had downtime from a disaster only to find themselves in yet another. What happens when you go to restore and the only good backup is three to four days old just because you haven’t been monitoring successes or failures during nightly, daily, or in-progress backups? In other words, if it’s Wednesday and your most usable backup is from Sunday, you have lost all of the bills you’ve generated on Monday and Tuesday. They’re just gone. To reconstruct those manually will be a laborious, slow process consisting of combing through other data in the system to try and reconstruct what has happened. The simple step of monitoring your backups can really go a very long way.

From a patient care perspective, what matters is that when an employee of a hospital needs to access specific data, it is correct and it is readily available in a secure environment. When a cyberattack inevitably slips through all the safety nets, walls, and fences, a solid backup is your best basis for restoral. I’ve laid out a few options to strengthen that environment, but it doesn’t matter which option you choose if you don’t actively test your backup system. Monitoring the success of daily backups is a simple thing that everyone can do right now with whatever equipment they have.
If you choose to not be proactive with your data and computer security, there’s yet another hurdle you have to overcome just to get your environment certified safe so that data restoration can get underway. Security assets are very limited right now – there are only a few thousand information security professionals in the United States, and only a small percentage are qualified to do analysis and remediation work. So, getting your hands on those resources during a crisis is not a sure thing – although this is an area where a good cyber insurance policy can make a difference, as the Insurance provider often has these resources on retainer.

Make a plan and stick to it

During and after a cyberattack, it’s critical that all of the stakeholders involved work together as a team. Whether it’s the hospital IT staff living on coffee and encouragement or it’s the out of town malware consultant on her third-consecutive case, there is a camaraderie that happens when everyone only cares about one thing – getting a hospital’s data acquisition and management systems up and running securely and quickly.

The hospitals that take disaster recovery seriously usually have a plan in place, rehearse, and drill for disasters. And this could be for disasters other than IT-specific. They could have a physical on-campus disaster that IT survives such as a bomb threat against a wing of the hospital, or a physical disaster such as the devastating flooding in Texas and Florida.

Those hospitals have a cross-functional team that is made up of many different departments in the hospitals. Usually, the CIO is the leader in that effort, but it’s just as likely a non-IT person such as a clinician, nurse or administrator is in charge. Typical drills and disaster rehearsals based on a written disaster recovery plan happen once a year, sometimes twice. Those plans can be as detailed as “go to the closet with the red door and pull out the paper forms.” It depends on the hospital, but these groups would benefit greatly from planning a cyberattack rehearsal to their annual agenda.

The playbook for these recoveries is being written in real-time, in the field. Hospitals and their supporting cast like MEDITECH and CloudWave plan, design around, rehearse and openly work to prevent disasters. It’s time to bring that level of planning, diligence, and investment to the prevention and remediation of cybercrime, no matter the cost.