Erie County Medical Center: Anatomy of a ransomware attack
Buffalo, N.Y. hospital’s response to cybercriminals taking its data hostage provides valuable lessons for other healthcare providers.
It was a Sunday morning like any other in the emergency department of Erie County Medical Center, a 602-bed hospital in Buffalo, N.Y., and the Western New York area’s Level 1 trauma center.
However, around 2 a.m. on April 9, 2017—Palm Sunday—a member of ECMC’s clinical staff was the first to see an electronic ransom note on a workstation. “You must send us 1.7 BitCoin for each affected PC OR 24 BitCoins to receive ALL Private Keys for ALL affected PC’s,” read the note from the cybercriminals, demanding the equivalent of $44,000 in Bitcoin cryptocurrency in return for a key to unlock the hospital’s files.
No one “really expects something like this to happen to them and their hospital,” says Jennifer Pugh, MD, ECMC’s associate chief of emergency medicine, who was in the ER when the ransomware hit. She credits the quick response of the medical staff with enabling ECMC to manage the crisis.
As the day went on, the hospital found itself enmeshed in a major ransomware attack.
Ransomware, used by hackers to target all kinds of organizations worldwide, is a type of malicious software surreptitiously installed on a computer that encrypts files and then holds the data hostage in return for payment of a ransom. After a computer’s hard drive is encrypted, a ransom note typically appears on the user’s screen, demanding payment for a software “key,” similar to a password, which unencrypts the hard drive.
ECMC responded quickly to the attack, following a pre-arranged script. To prevent the rampant spread of the ransomware, the organization purposely shut down all its information systems, including an electronic health records system, email and website—among others.
The staff recognized the threat almost immediately and within minutes notified security executives—who limited the spread of the malware—called in experts to deal with the crisis and, soon after, employed a novel workaround for accessing patient data.
Ultimately, more than 6,000 of ECMC’s computers were infected by a common version of ransomware called SamSam. To recover, the hospital would need to meticulously clean the file-encrypting malware from the hard drive of each computer that was hit.
ECMC says patient records were never compromised during the incident. Even so, the incident took six weeks to resolve and cost millions of dollars to fix. Perpetrators of the attack have not been caught.
The hospital’s experience is a cautionary tale for other healthcare organizations, which are regularly targeted by ransomware attacks and may fall victim to similar incidents. The number of reported major ransomware events targeted against healthcare organizations increased from 19 reported in 2016 to 36 reported in 2017—an 89 percent increase in the frequency of ransomware attacks—according to cybersecurity vendor Cryptonite.
“What the ransomware attack at ECMC proved is that every organization has potential vulnerabilities,” says Peter Cutler, ECMC’s vice president of communications and external affairs. “What is important to emphasize and proved critical in the ECMC attack is quick detection of an attack and immediately taking appropriate steps to prevent widespread damage to an organization’s computer infrastructure.”
Despite its success in dealing with the attack, ECMC has been cautious in releasing information about the incident—which is not atypical for organizations hit with ransomware, according to Adam Cohen, special agent in charge of the FBI’s Buffalo Field Office. Cohen, who declined to either confirm or deny an FBI investigation of ECMC’s attack, notes that often victims of ransomware keep the details confidential because of concerns over privacy, business reputation or regulatory data breach reporting requirements.
While ECMC released some statements to the press from its executives during and after the crisis, the hospital declined multiple requests from Health Data Management to interview IT staff and executives about the incident. It did, however, grant interviews with Pugh, its associate chief of emergency medicine, and Reg Harnish, CEO of GreyCastle Security, the cybersecurity firm that managed ECMC’s response to the event.
Attack timeline
The Sunday morning of the attack, a member of ECMC’s clinical staff was the first to see the ransom note on a workstation. Alarmed by the message, the clinician, following ECMC’s protocol, immediately called the facility’s helpdesk, which in turn notified the medical center’s chief information security officer.
In response, ECMC executives made the decision by 3:30 a.m. to shut down all its IT systems—including a Meditech electronic health records (EHR) system, email and website—in an attempt to stop the ransomware from spreading throughout the organization.
With the EHR out of commission, Pugh says, the hospital executed an existing contingency plan and reverted to using paper-based charts and face-to-face communication. “We do practice this and prepare for it,” she adds. “It involves going to paper records and paper order forms.”
Shortly before 5 a.m., ECMC reached out for help to GreyCastle Security, a cybersecurity firm in Troy, N.Y., which operates a 24/7 emergency response hotline, to head up remediation efforts. Within 15 minutes of that call, Harnish says his company was involved in triage to contain the incident; he activated a six-member response team, who went from Troy to Buffalo to manage the crisis onsite.
“That Sunday morning, when we began triage, cybercriminals were still accessing (the ECMC) network,” Harnish says.
Despite that access, medical records weren’t compromised and patient care was not negatively impacted, hospital executives said in public statements. At no point during the incident did ECMC consider paying the $44,000 ransom demanded by hackers, Harnish adds. “Our advice (to ECMC) never changed, and it never changes with anyone, which is not to pay the ransom,” he says. “The reality is that even if you pay the ransom, there’s no guarantee that it’s actually going to work.”
Likewise, the FBI doesn’t support paying to resolve a ransomware attack, says Cohen. Rather, the agency urges prevention as a first step and regular data backups to recover in the event of an attack, with recovery data stored on media that’s not connected to the computers or network.
While ECMC had regularly backed up data in multiple ways before the ransomware attack, the hackers “looked for and deleted all of those backup files that were online,” thus complicating the recovery process, Harnish says. As a result, the hospital “had to resort to older backups that were offline and not connected to the network,” he adds.
Justin Armstrong, a security analyst for Meditech, contends that backing up data regularly and verifying the integrity of those backups is critical to getting EHR systems back after an attack.
“Whether to pay (the ransom) or not is a very individual thing,” ECMC President and CEO Thomas Quatroche told The Buffalo News. “If you have no backup, you have no choice.” By backing up its data, the hospital ensured that it did not have to give in to the ransom demand from hackers.
Restorations and workarounds
In the hours, days and weeks after the attack, ECMC made steady progress in restoring its computer systems through a multiphased approach.
With its EHR system down, ECMC turned to HEALTHeLINK, a regional health information exchange in Western New York. HEALTHeLINK provided critical access to some patient records for ECMC clinicians immediately after the attack. “That became a bit of a lifeline,” Harnish says of HEALTHeLINK, a collaborative effort started in 2006 by healthcare organizations in the Western New York area to share clinical information and make patient records available.
In working around the ransomware attack, HEALTHeLINK served as a source of data backup with the information safely stored in the cloud. ECMC was one of the first participants in the HIE and “has been very progressive” in its participation, according to HEALTHeLINK Executive Director Daniel Porreca, even building an interface to HEALTHeLINK into its Meditech EHR.
“We had invested a lot of time and money to upload all of ECMC’s prior records into HEALTHeLINK—literally, up to the moment our computer systems were intentionally shut down in the aftermath of the attack,” says Pugh. “We were able to go and look up prior patient records, surgical reports, CT scans, labs—everything that we would normally get out of our computer screens normally.”
While ECMC used paper records in the first hours of the system shutdown, HEALTHeLINK helped ECMC implement an EHR workaround that enabled hospital staff to use laptops with ad hoc Internet access to view patient records through a web-based portal that accessed the HIE’s database.
“Very quickly, we had one of our staff on a call to reset passwords to enable access, and by early Sunday afternoon, we had one of our staff in the hospital working directly with providers as they set up laptops to get Internet access,” says Porreca. “By Monday morning, we had seven people onsite working in the areas where the laptops were being deployed and getting access to (ECMC) data via HEALTHeLINK.
“Based on their involvement with us, ECMC was able to continue clinical operations almost immediately and to access their own data by using HEALTHeLINK,” he adds. “We were fortunate to be in a position where we could help.”
“Any hospital that has the ability to participate in a health information exchange such as HEALTHeLINK should do so—it was that important to us,” contends Pugh. “I don’t think our patients even noticed because we really tried to provide the same level of care, even without use of our EHR.”
Still, some processes, such as placing orders or detailing care plans, required the use of paper and pen, Pugh says. For some clinicians, there were benefits to going back to these old practices, such as spending more face time with patients and less time in front of a computer screen. At the same time, Pugh notes that for some of the hospital’s younger staff and medical students a paper-based process took some getting used to because they hadn’t practiced in an environment without computers before. “We spent a lot of time with our residents making sure they knew how to appropriately document patient charts without EHR prompts,” she adds.
Other processes were moved off-screen as well. For example, clinicians normally would look at X-rays or CT scans on a computer screen, but now temporarily, they had to view them directly on film or at the CT scanners, Pugh says.
To enable physicians to place medical orders, ECMC printed out paper versions of the forms that had to be signed with a pen instead of being initialized on screen. Electronic prescribing—which New York State mandated in 2016—was a bit challenging in the aftermath of the ransomware attack, Pugh notes, but physicians used paper prescription pads to place orders, and “all the local pharmacies were notified of our issues.”
According to Michael Vinson, manager of client support and a member of Meditech’s disaster recovery application team, ECMC has been an EHR customer since the late 1990s. In addition to the EHR system being down, he recounts that early in the aftermath of the ransomware attack, one of the big challenges in assisting ECMC remotely was that the hospital didn’t have an operating email system and could only communicate through “old school” phone and text messages.
Further recovery efforts
By April 21—12 days after the initial attack—the hospital website had been restored, temporary email was established, some financial systems began to come online, and more than 6,000 hard drives had been cleaned and returned to workstations. In addition, ECMC’s EHR system from Meditech was available to staff clinicians, but just in view-only mode.
During the week of April 24, the medical center installed a new hospital email system, continued the phased restoration of inpatient EHR-related functions and began to roll out restored desktop computers. And, by the week of May 1, ECMC started electronic transmission of radiological images as well as physician documentation, beginning with the emergency and psychiatric emergency departments, while continuing the rollout of restored desktop computers and restoration of inpatient EHR functions.
The quick recovery was enabled because of prior staff training, planning and quick response of ECMC staff to the breach, limiting the damage to its systems while ensuring the safety of patients. However, ECMC's recovery carried a huge financial cost. This past summer, the hospital initially reported a $10 million price tag for repairing the damage and restoring its information systems; more recently an ECMC spokesman said the final cost of rebuilding the hospital’s computer systems is “not yet finalized.”
But fortuitously, the hospital in late 2016 increased its cyber insurance coverage to $10 million from $2 million, increasing its financial protection against such cybersecurity events, ECMC’s CEO Quatroche told The Buffalo News.
Rising threats, old vulnerabilities
ECMC's experience is emblematic of the challenges of overcoming ransomware attacks, which are on the rise. Results from a survey conducted by HIMSS Analytics, released in December 2017, show that 78 percent of providers have experienced a ransomware or malware attack in the past 12 months.
“It’s something that affects the single, sole practice medical professional all the way up to major hospitals,” says FBI special agent Cohen, who believes the trend will continue to rise with “more incidents of ransomware, hacking and intrusions.”
The FBI has warned that in newer instances of ransomware, cybercriminals are increasingly capitalizing on unpatched software on end-user computers. For example, in May 2017, hundreds of thousands of computers worldwide were compromised by the WannaCry ransomware in at least 150 countries, including the National Health Service in the United Kingdom, where the cyberattack froze computers at hospitals and closed emergency rooms. WannaCry affected systems that did not have the latest security patches and were running older versions of the Windows operating system that are no longer supported by Microsoft.
WannaCry is not the only variant of ransomware being used to attack healthcare providers’ systems. According to GreyCastle’s Harnish, the SamSam ransomware that hit ECMC targets web server vulnerabilities to infiltrate computer networks, which is how he believes ECMC’s systems were hacked.
“It was a single technical vulnerability,” says Harnish. “It was a very common but very simple vulnerability—by simple, I mean one that is easily addressed and fixed.”
An alert from the FBI details that SamSam uses an automated script that crawls the Internet looking for server vulnerabilities involving JBoss (an open source application server program from Red Hat) and Remote Desktop Protocol (or RDP, a Microsoft remote management tool), exploiting either weak passwords or cracking default passwords with brute force attacks. Once it finds one, the script exploits the vulnerability—known as “patient zero”—and then gains access to the victim’s network.
For ECMC, the “patient-zero vulnerability was a default password on an Internet-facing asset,” Harnish believes. “It was what ended up being the initial vulnerability that was exploited and gave those criminals access.”
According to Harnish, the SamSam ransomware attack on ECMC did not involve a JBoss server, leaving the other possibility—an RDP vulnerability. “I can’t confirm or disconfirm that,” he adds.
Avi Rubin, director of the health and medical security lab at Johns Hopkins University, says a common technique hackers employ is scanning the Internet for computers that have insecure connections—called ports—and exploiting vulnerable applications such as RDP.
“Once the attackers gain a foothold in this manner, they can attack the passwords in the system by using sophisticated dictionaries and matching techniques to crack the passwords in the system,” notes Rubin.
Phillip Hallam-Baker, principal scientist and vice president at cybersecurity vendor Comodo, warns that if the password is a default password, the attacker already knows it.
“Quite often, software ships with an account ‘guest’ with password ‘password,’” remarks Hallam-Baker. “In the past, software often shipped with admin accounts with default passwords, but that happens much less now because it is flagged as an issue.”
In 2016, the FBI issued a warning about SamSam ransomware, detailing how cybercriminals were exploiting such vulnerabilities, particularly in the healthcare industry.
Nonetheless, Cutler, ECMC’s vice president of communications and external affairs, is dismissive of any fault or negligence on the part of the hospital. “Organizations across the country routinely receive information of cyberattack warnings from entities like the FBI,” Cutler says.
“When you have an environment (like ECMC) where there are 6,000-plus computers, the likelihood that the configuration on one of the computers was incorrect is pretty high,” contends Harnish.
Similarly, Meditech’s Armstrong says that, in a big complex computing environment like a hospital, “there’s always going to be something that has a vulnerability” that are going to put any devices that connect to the Internet at risk.
Resilience in facing ransomware
While there are other variants of ransomware, Harnish says SamSam is “rampant” in healthcare and will continue to pose a cybersecurity threat to hospitals. Earlier this month, health IT vendor Allscripts was hit by a ransomware attack, affecting its cloud-hosted EHR among other systems, with hospitals and physician group practices across the country reporting interruptions in service. The company acknowledged that it had been hit by a variant of SamSam.
Harnish would not reveal who was behind the ECMC attack or their country of origin. Based on his experience, FBI Special Agent Cohen says most of the ransomware attacks being launched on the U.S. are initiated in Eastern Europe.
“What’s happening is a form of terrorism like an attack on critical infrastructure,” ECMC’s Quatroche told The Buffalo News.
Harnish believes that most medical facilities are woefully unprepared for the kind of attack that hit ECMC, saying it’s not a question of if—but when—the next health system will fall victim to malware.
The FBI’s Cohen urges victims of ransomware to report incidents to the agency—regardless of the outcome—to help it gain a more comprehensive view of the current threat environment. “Our job is to help, however we can, and the more that we know about the types of attacks and the tactics used enables us to better understand the threat.”
To facilitate public-private collaboration between U.S. businesses and the FBI, InfraGard was established as a not-for-profit organization to expedite the timely exchange of information and promote mutual learning opportunities when it comes to cybersecurity. “That’s our way to not just take information but provide information back,” says Cohen.
For its part, the FBI suggests organizations focus on two main areas: prevention in terms of both awareness training for employees and robust technical prevention controls, as well as the creation of a solid business continuity plan in the event of a ransomware attack.
Healthcare organizations “need to build a response capability—this is about resilience in healthcare,” says Harnish.
Resilience is clearly a message that resonates with the industry. As results of the HIMSS Analytics survey released in December 2017 showed, 97 percent of providers have a high level of concern about cybersecurity and resilience—defined as an organization’s capacity to adapt and respond to adverse cyber events in ways that maintain the confidentiality, integrity and availability of data and services.
Calling cyber defense a “bit of a failed concept” for hospitals, Harnish recommends to facilities that they not give up on prevention but at the same time develop contingency planning and train their staffs in how their organization will deal with the loss of information systems as a result of such cybersecurity incidents.
John Glynn, chief information officer at Rochester Regional Health, another integrated healthcare delivery system serving Western New York and the Finger Lakes region, says the ECMC ransomware attack “really got the attention of our board—I’m getting sick just thinking about it.”
Glynn notes that in the aftermath of ECMC’s cybersecurity event, one of the benefits for Rochester Regional Health was it forced them to “do more system-wide downtime preparedness drills than maybe we had previously.” He acknowledges there are “multiple vectors of attack” that healthcare organizations have to be prepared for, which is difficult because of the wide range of cyber threats confronting healthcare organizations.
“The attackers are always going to find a way in—that’s why it’s really essential to be able to quickly detect and respond,” adds Meditech’s Armstrong. “These are complicated problems. When an attacker gets into a system, you want to make sure that the ransomware is gone and that they didn’t leave any backdoors in so they can come back later.”
Hospitals “need to be prepared for when prevention breaks down,” Harnish concludes. “Insulating patients from cyberattacks has to be their No. 1 priority because it’s potentially an issue of life and death. These types of intrusions and attacks will dramatically increase in frequency and sophistication. It’s pretty much inevitable.”
However, around 2 a.m. on April 9, 2017—Palm Sunday—a member of ECMC’s clinical staff was the first to see an electronic ransom note on a workstation. “You must send us 1.7 BitCoin for each affected PC OR 24 BitCoins to receive ALL Private Keys for ALL affected PC’s,” read the note from the cybercriminals, demanding the equivalent of $44,000 in Bitcoin cryptocurrency in return for a key to unlock the hospital’s files.
No one “really expects something like this to happen to them and their hospital,” says Jennifer Pugh, MD, ECMC’s associate chief of emergency medicine, who was in the ER when the ransomware hit. She credits the quick response of the medical staff with enabling ECMC to manage the crisis.
As the day went on, the hospital found itself enmeshed in a major ransomware attack.
Ransomware, used by hackers to target all kinds of organizations worldwide, is a type of malicious software surreptitiously installed on a computer that encrypts files and then holds the data hostage in return for payment of a ransom. After a computer’s hard drive is encrypted, a ransom note typically appears on the user’s screen, demanding payment for a software “key,” similar to a password, which unencrypts the hard drive.
ECMC responded quickly to the attack, following a pre-arranged script. To prevent the rampant spread of the ransomware, the organization purposely shut down all its information systems, including an electronic health records system, email and website—among others.
The staff recognized the threat almost immediately and within minutes notified security executives—who limited the spread of the malware—called in experts to deal with the crisis and, soon after, employed a novel workaround for accessing patient data.
Ultimately, more than 6,000 of ECMC’s computers were infected by a common version of ransomware called SamSam. To recover, the hospital would need to meticulously clean the file-encrypting malware from the hard drive of each computer that was hit.
ECMC says patient records were never compromised during the incident. Even so, the incident took six weeks to resolve and cost millions of dollars to fix. Perpetrators of the attack have not been caught.
The hospital’s experience is a cautionary tale for other healthcare organizations, which are regularly targeted by ransomware attacks and may fall victim to similar incidents. The number of reported major ransomware events targeted against healthcare organizations increased from 19 reported in 2016 to 36 reported in 2017—an 89 percent increase in the frequency of ransomware attacks—according to cybersecurity vendor Cryptonite.
“What the ransomware attack at ECMC proved is that every organization has potential vulnerabilities,” says Peter Cutler, ECMC’s vice president of communications and external affairs. “What is important to emphasize and proved critical in the ECMC attack is quick detection of an attack and immediately taking appropriate steps to prevent widespread damage to an organization’s computer infrastructure.”
Despite its success in dealing with the attack, ECMC has been cautious in releasing information about the incident—which is not atypical for organizations hit with ransomware, according to Adam Cohen, special agent in charge of the FBI’s Buffalo Field Office. Cohen, who declined to either confirm or deny an FBI investigation of ECMC’s attack, notes that often victims of ransomware keep the details confidential because of concerns over privacy, business reputation or regulatory data breach reporting requirements.
While ECMC released some statements to the press from its executives during and after the crisis, the hospital declined multiple requests from Health Data Management to interview IT staff and executives about the incident. It did, however, grant interviews with Pugh, its associate chief of emergency medicine, and Reg Harnish, CEO of GreyCastle Security, the cybersecurity firm that managed ECMC’s response to the event.
Attack timeline
The Sunday morning of the attack, a member of ECMC’s clinical staff was the first to see the ransom note on a workstation. Alarmed by the message, the clinician, following ECMC’s protocol, immediately called the facility’s helpdesk, which in turn notified the medical center’s chief information security officer.
In response, ECMC executives made the decision by 3:30 a.m. to shut down all its IT systems—including a Meditech electronic health records (EHR) system, email and website—in an attempt to stop the ransomware from spreading throughout the organization.
With the EHR out of commission, Pugh says, the hospital executed an existing contingency plan and reverted to using paper-based charts and face-to-face communication. “We do practice this and prepare for it,” she adds. “It involves going to paper records and paper order forms.”
Shortly before 5 a.m., ECMC reached out for help to GreyCastle Security, a cybersecurity firm in Troy, N.Y., which operates a 24/7 emergency response hotline, to head up remediation efforts. Within 15 minutes of that call, Harnish says his company was involved in triage to contain the incident; he activated a six-member response team, who went from Troy to Buffalo to manage the crisis onsite.
“That Sunday morning, when we began triage, cybercriminals were still accessing (the ECMC) network,” Harnish says.
Despite that access, medical records weren’t compromised and patient care was not negatively impacted, hospital executives said in public statements. At no point during the incident did ECMC consider paying the $44,000 ransom demanded by hackers, Harnish adds. “Our advice (to ECMC) never changed, and it never changes with anyone, which is not to pay the ransom,” he says. “The reality is that even if you pay the ransom, there’s no guarantee that it’s actually going to work.”
Likewise, the FBI doesn’t support paying to resolve a ransomware attack, says Cohen. Rather, the agency urges prevention as a first step and regular data backups to recover in the event of an attack, with recovery data stored on media that’s not connected to the computers or network.
While ECMC had regularly backed up data in multiple ways before the ransomware attack, the hackers “looked for and deleted all of those backup files that were online,” thus complicating the recovery process, Harnish says. As a result, the hospital “had to resort to older backups that were offline and not connected to the network,” he adds.
Justin Armstrong, a security analyst for Meditech, contends that backing up data regularly and verifying the integrity of those backups is critical to getting EHR systems back after an attack.
“Whether to pay (the ransom) or not is a very individual thing,” ECMC President and CEO Thomas Quatroche told The Buffalo News. “If you have no backup, you have no choice.” By backing up its data, the hospital ensured that it did not have to give in to the ransom demand from hackers.
Restorations and workarounds
In the hours, days and weeks after the attack, ECMC made steady progress in restoring its computer systems through a multiphased approach.
With its EHR system down, ECMC turned to HEALTHeLINK, a regional health information exchange in Western New York. HEALTHeLINK provided critical access to some patient records for ECMC clinicians immediately after the attack. “That became a bit of a lifeline,” Harnish says of HEALTHeLINK, a collaborative effort started in 2006 by healthcare organizations in the Western New York area to share clinical information and make patient records available.
In working around the ransomware attack, HEALTHeLINK served as a source of data backup with the information safely stored in the cloud. ECMC was one of the first participants in the HIE and “has been very progressive” in its participation, according to HEALTHeLINK Executive Director Daniel Porreca, even building an interface to HEALTHeLINK into its Meditech EHR.
“We had invested a lot of time and money to upload all of ECMC’s prior records into HEALTHeLINK—literally, up to the moment our computer systems were intentionally shut down in the aftermath of the attack,” says Pugh. “We were able to go and look up prior patient records, surgical reports, CT scans, labs—everything that we would normally get out of our computer screens normally.”
While ECMC used paper records in the first hours of the system shutdown, HEALTHeLINK helped ECMC implement an EHR workaround that enabled hospital staff to use laptops with ad hoc Internet access to view patient records through a web-based portal that accessed the HIE’s database.
“Very quickly, we had one of our staff on a call to reset passwords to enable access, and by early Sunday afternoon, we had one of our staff in the hospital working directly with providers as they set up laptops to get Internet access,” says Porreca. “By Monday morning, we had seven people onsite working in the areas where the laptops were being deployed and getting access to (ECMC) data via HEALTHeLINK.
“Based on their involvement with us, ECMC was able to continue clinical operations almost immediately and to access their own data by using HEALTHeLINK,” he adds. “We were fortunate to be in a position where we could help.”
“Any hospital that has the ability to participate in a health information exchange such as HEALTHeLINK should do so—it was that important to us,” contends Pugh. “I don’t think our patients even noticed because we really tried to provide the same level of care, even without use of our EHR.”
Still, some processes, such as placing orders or detailing care plans, required the use of paper and pen, Pugh says. For some clinicians, there were benefits to going back to these old practices, such as spending more face time with patients and less time in front of a computer screen. At the same time, Pugh notes that for some of the hospital’s younger staff and medical students a paper-based process took some getting used to because they hadn’t practiced in an environment without computers before. “We spent a lot of time with our residents making sure they knew how to appropriately document patient charts without EHR prompts,” she adds.
Other processes were moved off-screen as well. For example, clinicians normally would look at X-rays or CT scans on a computer screen, but now temporarily, they had to view them directly on film or at the CT scanners, Pugh says.
To enable physicians to place medical orders, ECMC printed out paper versions of the forms that had to be signed with a pen instead of being initialized on screen. Electronic prescribing—which New York State mandated in 2016—was a bit challenging in the aftermath of the ransomware attack, Pugh notes, but physicians used paper prescription pads to place orders, and “all the local pharmacies were notified of our issues.”
According to Michael Vinson, manager of client support and a member of Meditech’s disaster recovery application team, ECMC has been an EHR customer since the late 1990s. In addition to the EHR system being down, he recounts that early in the aftermath of the ransomware attack, one of the big challenges in assisting ECMC remotely was that the hospital didn’t have an operating email system and could only communicate through “old school” phone and text messages.
Further recovery efforts
By April 21—12 days after the initial attack—the hospital website had been restored, temporary email was established, some financial systems began to come online, and more than 6,000 hard drives had been cleaned and returned to workstations. In addition, ECMC’s EHR system from Meditech was available to staff clinicians, but just in view-only mode.
During the week of April 24, the medical center installed a new hospital email system, continued the phased restoration of inpatient EHR-related functions and began to roll out restored desktop computers. And, by the week of May 1, ECMC started electronic transmission of radiological images as well as physician documentation, beginning with the emergency and psychiatric emergency departments, while continuing the rollout of restored desktop computers and restoration of inpatient EHR functions.
The quick recovery was enabled because of prior staff training, planning and quick response of ECMC staff to the breach, limiting the damage to its systems while ensuring the safety of patients. However, ECMC's recovery carried a huge financial cost. This past summer, the hospital initially reported a $10 million price tag for repairing the damage and restoring its information systems; more recently an ECMC spokesman said the final cost of rebuilding the hospital’s computer systems is “not yet finalized.”
But fortuitously, the hospital in late 2016 increased its cyber insurance coverage to $10 million from $2 million, increasing its financial protection against such cybersecurity events, ECMC’s CEO Quatroche told The Buffalo News.
Rising threats, old vulnerabilities
ECMC's experience is emblematic of the challenges of overcoming ransomware attacks, which are on the rise. Results from a survey conducted by HIMSS Analytics, released in December 2017, show that 78 percent of providers have experienced a ransomware or malware attack in the past 12 months.
“It’s something that affects the single, sole practice medical professional all the way up to major hospitals,” says FBI special agent Cohen, who believes the trend will continue to rise with “more incidents of ransomware, hacking and intrusions.”
The FBI has warned that in newer instances of ransomware, cybercriminals are increasingly capitalizing on unpatched software on end-user computers. For example, in May 2017, hundreds of thousands of computers worldwide were compromised by the WannaCry ransomware in at least 150 countries, including the National Health Service in the United Kingdom, where the cyberattack froze computers at hospitals and closed emergency rooms. WannaCry affected systems that did not have the latest security patches and were running older versions of the Windows operating system that are no longer supported by Microsoft.
WannaCry is not the only variant of ransomware being used to attack healthcare providers’ systems. According to GreyCastle’s Harnish, the SamSam ransomware that hit ECMC targets web server vulnerabilities to infiltrate computer networks, which is how he believes ECMC’s systems were hacked.
“It was a single technical vulnerability,” says Harnish. “It was a very common but very simple vulnerability—by simple, I mean one that is easily addressed and fixed.”
An alert from the FBI details that SamSam uses an automated script that crawls the Internet looking for server vulnerabilities involving JBoss (an open source application server program from Red Hat) and Remote Desktop Protocol (or RDP, a Microsoft remote management tool), exploiting either weak passwords or cracking default passwords with brute force attacks. Once it finds one, the script exploits the vulnerability—known as “patient zero”—and then gains access to the victim’s network.
For ECMC, the “patient-zero vulnerability was a default password on an Internet-facing asset,” Harnish believes. “It was what ended up being the initial vulnerability that was exploited and gave those criminals access.”
According to Harnish, the SamSam ransomware attack on ECMC did not involve a JBoss server, leaving the other possibility—an RDP vulnerability. “I can’t confirm or disconfirm that,” he adds.
Avi Rubin, director of the health and medical security lab at Johns Hopkins University, says a common technique hackers employ is scanning the Internet for computers that have insecure connections—called ports—and exploiting vulnerable applications such as RDP.
“Once the attackers gain a foothold in this manner, they can attack the passwords in the system by using sophisticated dictionaries and matching techniques to crack the passwords in the system,” notes Rubin.
Phillip Hallam-Baker, principal scientist and vice president at cybersecurity vendor Comodo, warns that if the password is a default password, the attacker already knows it.
“Quite often, software ships with an account ‘guest’ with password ‘password,’” remarks Hallam-Baker. “In the past, software often shipped with admin accounts with default passwords, but that happens much less now because it is flagged as an issue.”
In 2016, the FBI issued a warning about SamSam ransomware, detailing how cybercriminals were exploiting such vulnerabilities, particularly in the healthcare industry.
Nonetheless, Cutler, ECMC’s vice president of communications and external affairs, is dismissive of any fault or negligence on the part of the hospital. “Organizations across the country routinely receive information of cyberattack warnings from entities like the FBI,” Cutler says.
“When you have an environment (like ECMC) where there are 6,000-plus computers, the likelihood that the configuration on one of the computers was incorrect is pretty high,” contends Harnish.
Similarly, Meditech’s Armstrong says that, in a big complex computing environment like a hospital, “there’s always going to be something that has a vulnerability” that are going to put any devices that connect to the Internet at risk.
Resilience in facing ransomware
While there are other variants of ransomware, Harnish says SamSam is “rampant” in healthcare and will continue to pose a cybersecurity threat to hospitals. Earlier this month, health IT vendor Allscripts was hit by a ransomware attack, affecting its cloud-hosted EHR among other systems, with hospitals and physician group practices across the country reporting interruptions in service. The company acknowledged that it had been hit by a variant of SamSam.
Harnish would not reveal who was behind the ECMC attack or their country of origin. Based on his experience, FBI Special Agent Cohen says most of the ransomware attacks being launched on the U.S. are initiated in Eastern Europe.
“What’s happening is a form of terrorism like an attack on critical infrastructure,” ECMC’s Quatroche told The Buffalo News.
Harnish believes that most medical facilities are woefully unprepared for the kind of attack that hit ECMC, saying it’s not a question of if—but when—the next health system will fall victim to malware.
The FBI’s Cohen urges victims of ransomware to report incidents to the agency—regardless of the outcome—to help it gain a more comprehensive view of the current threat environment. “Our job is to help, however we can, and the more that we know about the types of attacks and the tactics used enables us to better understand the threat.”
To facilitate public-private collaboration between U.S. businesses and the FBI, InfraGard was established as a not-for-profit organization to expedite the timely exchange of information and promote mutual learning opportunities when it comes to cybersecurity. “That’s our way to not just take information but provide information back,” says Cohen.
For its part, the FBI suggests organizations focus on two main areas: prevention in terms of both awareness training for employees and robust technical prevention controls, as well as the creation of a solid business continuity plan in the event of a ransomware attack.
Healthcare organizations “need to build a response capability—this is about resilience in healthcare,” says Harnish.
Resilience is clearly a message that resonates with the industry. As results of the HIMSS Analytics survey released in December 2017 showed, 97 percent of providers have a high level of concern about cybersecurity and resilience—defined as an organization’s capacity to adapt and respond to adverse cyber events in ways that maintain the confidentiality, integrity and availability of data and services.
Calling cyber defense a “bit of a failed concept” for hospitals, Harnish recommends to facilities that they not give up on prevention but at the same time develop contingency planning and train their staffs in how their organization will deal with the loss of information systems as a result of such cybersecurity incidents.
John Glynn, chief information officer at Rochester Regional Health, another integrated healthcare delivery system serving Western New York and the Finger Lakes region, says the ECMC ransomware attack “really got the attention of our board—I’m getting sick just thinking about it.”
Glynn notes that in the aftermath of ECMC’s cybersecurity event, one of the benefits for Rochester Regional Health was it forced them to “do more system-wide downtime preparedness drills than maybe we had previously.” He acknowledges there are “multiple vectors of attack” that healthcare organizations have to be prepared for, which is difficult because of the wide range of cyber threats confronting healthcare organizations.
“The attackers are always going to find a way in—that’s why it’s really essential to be able to quickly detect and respond,” adds Meditech’s Armstrong. “These are complicated problems. When an attacker gets into a system, you want to make sure that the ransomware is gone and that they didn’t leave any backdoors in so they can come back later.”
Hospitals “need to be prepared for when prevention breaks down,” Harnish concludes. “Insulating patients from cyberattacks has to be their No. 1 priority because it’s potentially an issue of life and death. These types of intrusions and attacks will dramatically increase in frequency and sophistication. It’s pretty much inevitable.”
More for you
Loading data for hdm_tax_topic #better-outcomes...