When you’re hacked, there is no such thing as reacting too quickly or too thoroughly.

We all want tools to give us advanced warning of a security hack. With nearly two weeks’ notice, could we lock down our defenses and purge an attacker? For one small town in Alabama, twelve days’ notice simply was not enough.

With the ever-increasing number of vulnerabilities, the increasing sophistication of attackers, and the chaos of dealing with COVID-19-related complications, most IT departments barely keep up with everyday tasks. Unless our organizations have planned ahead, special events such as a hack quickly overwhelm our capabilities.

The Florence Security Hack

On May 26, a cybersecurity firm tipped off security journalist Brian Krebs about potentially compromised computers within the municipal government of Florence, AL.

Krebs contacted the city and found himself transferred to three different people (Issue #1, below) before dead-ending in a voicemail for the police department. He left messages for both the department and the city’s emergency response team.

The next day, a system admin called to express gratitude for helping the city catch the issue so quickly. They found that a DHL-themed phishing attack compromised the city’s IT manager. The IT team then isolated the system (Issue #2, below) and changed protocols to quarantine the compromised system and attack.

Then, the city’s IT Teams worked to remediate the compromised system while simultaneously trying to convince the city council to approve funds (Issue#3, below) for a more thorough investigation. Unfortunately, on June 5, the attack shut down the city’s email system before approvals were gained to do so.

Instead, the mayor had to go on the press and tell the world about the attack. At the time, Mayor Holt said there was no indication of ransomware being involved. But, by Tuesday, June 9, he conceded that the city was being extorted by the DoppelPaymer ransomware gang.

The DoppelPaymer ransomware gang struck four different compromised networks within an hour, geographically, of Florence (including another municipality), and negotiated a 30 bitcoin (~$291,000) ransom of the city’s data.

Now, the city feels it must pay to avoid the publication of its citizens’ personal and financial data.

Lessons Learned

The town of Florence tried to take the right steps to limit their exposure once they learned about the compromised computer.

Unfortunately, three key issues prevented them from moving quickly and effectively enough to stave off the attack. To avoid becoming the next victim, let’s examine those three issues and the steps you can take to prepare your own organization for a similar attack.

Issue #1: No designated person for tips or cybersecurity issues.

When first contacted, the city gave the tipster the runaround.

The municipality wasted precious time that could have been used to fight the attack by making Krebs fight through its voicemail systems.

To avoid this issue, every organization should profile different types of disasters (IT, natural disasters, terrorist attacks, etc.), and also create a call list of important contacts. The contact list should be distributed widely and communicated well so that every member of the organization understands how to react and direct information.

No expertise is necessary.

Your receptionist does not need to understand the problem. Instead, you only need to know that, if there is a computer problem, you should:

Take a brief message that can be emailed to a pre-established inbox (i.e. ITProblem@myorg.org)
Text a preestablished group (ITProb) a short synopsis of the issue
Transfer the tipster to a designated voicemail number accessible by anyone on the team

These steps should also be posted in key departments so that, if the computers go down, the process will still be followed.

Issue #2 – If one is infected, assume many are infected. Time to go into lockdown.

The IT team for the town of Florence tried to isolate the infected machine, but they weren’t hacking experts.

The DoppelPaymer ransomware gang certainly deploys more expertise in hacking, thus maintaining a distinct advantage against a typical IT team. Ransomware gangs possess high-level hacking skills. They’ve been known to maintain control even as ransomware victims shut down systems and restore from backups.

Some even publish screenshots of internal IT team counterattack discussions after the fact.

Once one computer has been compromised, some cybersecurity response teams feel that the only guaranteed solution is to rebuild the network, as well as all computers and servers. At the very least, the IT team should shut down outside access for all potentially affected systems within that network.

While a complete shutdown will always be disruptive, the alternative is a forced shut down enforced by attackers… and, potentially, a large ransom. Organizations need to prepare for this possibility in advance, and create workarounds for vital services to run despite temporary IT interruptions.

For any approach other than a complete shutdown, your team or your vendor needs to be highly skilled. Specialist teams, such as Blue Bastion, can provide alternatives, but you’ll need to engage them quickly. There is no time to vet or budget – which brings us to the next issue.

Issue #3 – No emergency funding prepared in advance.

As part of disaster recovery, IT Teams and organization managers need to anticipate many different types of emergencies. They also must forecast potential funding needed to address them. These protocols should be funded well in advance, or at least pre-approved for minimum levels of emergency spending.

To prevent runaway spending, you may need to set predetermined limits, You don’t want to spend blindly, and find that the incident was a false alarm.

Florence took 12 days to properly respond to the event, which could have been anticipated. Clearly, that wasn’t a reasonable response.

Cyber Security Hacks Will Only Increase in Difficulty

IT security managers have their hands full with maintaining the status quo in an increasingly complex environment.

The COVID-19 quarantine has only stressed departments more, as they deal with remote workers, shifts to mobile applications, and cloud adoption.

Even basic maintenance can be challenging.

For the fourth month in a row, Microsoft released patches for over 100 security vulnerabilities including 19 security updates that could enable remote code execution.

While our teams work to approve and implement these patches, ransomware gangs and other attackers continue to expand their creativity to avoid detection and attack our systems:

Maze Ransomware now allows other gangs to use their data leak platform to release data
Thanos offers a build-your-own ransomware kit with antivirus evasion techniques
Ragnar Locker is only 49kB, but it operates within a 280 MB Windows XP virtual machine to avoid antivirus detection
Kingminer botnets attack vulnerable servers and then apply hotfixes to fix those same vulnerabilities so future attackers cannot interrupt their activities.

The Right Plan With the Best Support

Are there methods that can be used to counter attacks? Certainly.

We often try to remind organizations that multifactor authentication and microsegmentation can mitigate an attack that compromises credentials. However, there are many other useful techniques.

Municipalities, first responders, and healthcare organizations that offer critical services should isolate and segregate critical systems completely so that an attack on one network cannot spread to other networks. This segregation is accomplished by using microsegmentation technology, or even by creating a standard, simple, and unconnected network with different credentials.

Advanced firewalls, endpoint protection software, Secure DNS and other concepts can also play roles in providing layers of detection for your organization. Finding a solution that works best for you and your budget requires expertise and a broad background.

Ideal Integrations and Blue Bastion can leverage our expertise to provide your IT and security teams with expert advice on a wide array of products and services designed to help your team respond to attacks quickly and thoroughly.

How to Prepare For a Cyber Security Hack