Russian government hackers appear to be behind one of the most evasive espionage campaigns leveraging the software supply chain in recent memory.

While the full scope of the attack isn’t yet known, we learned over the weekend that as part of what appears to be a coordinated global intrusion campaign, hackers managed to breach the U.S. Treasury, Commerce, and Homeland departments, along with other U.S. government agencies, months ago.

To carry out the campaign, attackers leveraged software created by SolarWinds, a network monitoring company that’s primarily used by companies for system performance, asset discovery and management, and resilience.

In a classic supply chain scenario, attackers appear to have inserted malicious code next to trusted code, without SolarWinds knowledge, in order to gain a foothold – essentially a backdoor - into any organization that downloaded the code, dating back to at least March. The backdoor code - which exists in a legitimate library - has the file name SolarWinds.Orion.Core.BusinessLayer.dll, according to Microsoft, which posted a Microsoft Security Response Center blog on the attacks on Sunday.

The attack involved versions of SolarWinds’ Orion Platform software, 2019.4 HF 5 through 2020.2.1, released between March 2020 and June 2020. Orion allows administrators to carry out monitoring capabilities, analysis, and address issues across the IT stack.

It's not known how attackers got the backdoor code into SolarWinds; Microsoft suggested in its report that the attackers may have compromised internal build or distribution systems belonging to SolarWinds, something which allowed them to embed a backdoor. The conventional narrative, at least judging from SolarWinds' disclosure, is that attackers have been in systems for months, stealing data and spying on government workers, without officials being any the wiser.

A Reuters report cited a source that said once news of the hack surfaced, it was so severe that it prompted a National Security Council meeting at the White House on Saturday.

To aid the public, the Cybersecurity and Infrastructure Security Agency (CISA) – part of the Department of Homeland Security (DHS) - provided instructions on how to detect and analyze compromised systems on Sunday by issuing a rare emergency directive. In it, CISA urged all federal civilian agencies to review their networks for indicators of compromise and if they use SolarWinds Orion products, to disconnect or power them down immediately. By noon today, the directive asked organizations to report any existence of the following on its systems: 

a. [SolarWinds.Orion.Core.BusinessLayer.dll] with a file hash of [b91ce2fa41029f6955bff20079468448]

b. [C:\WINDOWS\SysWOW64\netsetupsvc.dll]

c. Other indicators related to this issue to be shared by CISA

Ironically, CISA is just one of several of SolarWinds' US government contracts. A slew of other federal entities, including US Cyber Command, the Department of Defense, the FBI, DHS, and Veterans Affairs also reportedly use the software. Of course, it’s unknown if any of those agencies are victims of this campaign however.

The Trump administration didn't divulge much about the attacks on Sunday. John Ullyot, a spokesman for the National Security Council said the US government was aware of the reports detailing the U.S. Treasury and Commerce hacks and that it was “taking all necessary steps to identify and remedy any possible issues related to this situation.” The Commerce Department said that one of its agencies had been affected but didn’t name it. That agency appears to be the National Telecommunications and Information Administration; Reuters' writeup on the attack suggested that staff emails at the NTIA may have been monitored for months.

It wasn't clear until Monday, per a Reuters report, that the Department of Homeland Security was also penetrated as part of the campaign. Details about that incident are not yet public.

Again, while the scope of the breach isn’t yet known, it could ultimately be massive.

Including the aforementioned agencies, SolarWinds has 300,000 customers, among them 425 of the US Fortune 500 companies, all branches of the US military, a handful of telecommunications giants and hundreds of colleges and universities.

In a Securities and Exchange Commission 8-K filing on Sunday, SolarWinds said it disclosed the incident to approximately 33,000 customers that use its Orion products over the weekend but that it believes the actual number of customers that may have had a vulnerable version of the Orion product is "fewer than 18,000."

As further details of the attack come into focus, it’s safe to assume there will be news of additional compromises to come.

Judging by the 8-K, it sounds as if an email compromise was to blame for the breach at SolarWinds. In the document, the company says it was made aware of an attack vector that was used to compromise the company's emails that may have provided access to other data in its office productivity tools.

SolarWinds admitted culpability on Sunday, acknowledging its systems "experienced a highly sophisticated, manual supply chain attack." It urged customers to update to Orion Platform version 2020.2.1 HF 1 and prepare to apply an additional hotfix release, 2020.2.1 HF 2, expected to be released Tuesday. In its advisory, SolarWinds said it was advised the attack was likely committed by an outside nation state but stopped short of saying it was Russia.

SolarWinds asks all customers to upgrade immediately to Orion Platform version 2020.2.1 HF 1 to address a security vulnerability. More information is available at https://t.co/scsUhZJCk8

— SolarWinds (@solarwinds) December 14, 2020

The Washington Post was the first to report that Russia's Foreign Intelligence Service, or SVR, initiated the attack and that FireEye, a cybersecurity firm that said last week it was hacked "by a nation with top-tier offensive capabilities,” was also targeted by the campaign.

According to FireEye, which published an analysis of the malware on Sunday, the malware hides in network traffic as Orion's native protocol, OIP, or Orion Improvement Program. It stores data within plugin configuration files, something that allows it to obscure its activity.

In a technical report on the malware, which FireEye dubbed SUNBURST, the company pointed to detection rules administrators can deploy on GitHub.

Microsoft, which posted its own blog post on Sunday – it refers to the malware as Solorigate - also shared steps defenders can take to protect their systems.

If the attack is formally attributed to Russia, it's poised to be the largest data theft by SVR aka Cozy Bear aka APT29 since 2014-2015, when the White House, State Department and Joint Chiefs of Staff were hacked.