All posts

The Week Everything Looked Like an Attack

Microsoft 365 failed, flights stopped, and an apparent FBI intrusion made the news. The records show where the outages and attacks diverge, and where investigators still owe answers.

Two coral routes bend apart and break at the same horizontal position.

What failed. What the records show.

On November 18, FBI Deputy Director Christopher Raia told Congress the FBI had seen no sign yet of an attack behind the week's outages. Two days later, CNN reported that the FBI was looking into a possible break-in at its National Crime Information Center. Police use that system to look up warrants and records. The Associated Press heard the same from its own source. Its scope was still unclear. Neither story tied it to the outages. What stopped working that week? Did someone get in?123

That same week, one attack left a trail at two firms. On November 16, someone slipped bad code into scripts on some sites that used Optimizely. Optimizely put the first change at 07:51 UTC. Salesforce found altered scripts on some of its sites and turned them off. It found no signs that its platform or paid services were hit. Both firms could point to the same code.45

When a system fails on its own

On November 12, parts of Microsoft 365 went down. Exchange Online took the brunt. Teams and other tools had trouble too. Microsoft's report points to a bug from July. A route failed. Teams traffic went through the same gateway as Exchange Online. Some of those servers were already down for planned work. The rest held up for hours. Then peak demand hit. As each server failed, its traffic went to the ones still up. That added load and made the next failure more likely.67

The next day, the FAA stopped flights from taking off nationwide. A flood of traffic had swamped older gear that carried radio and radar data. The network settled down after the contractor cut off two sites. The FAA thinks turning off a feature on the Juniper routers at those sites helped stem the flood. It turned off the same feature on more than 60 other routers to be safe. The FAA found no sign of an attack and kept looking for the root cause. Senators asked what the tests missed and why the backups could not take over.89

On November 16, bank customers had trouble with online services. Chase traced its delays to a bug in software running on its own hardware. When one bank stalls, a transfer can hang for people on both sides. Zelle's operator said its network stayed up. Three days later, Chase said hackers had hit some of its vendors. They may have seen customer names and email addresses. Visa and Mastercard each said a vendor had been hit too. Chase said it found no link between the breach and its earlier slowdown.1011

What a patch can explain

The NSA said on November 17 that planned work had disrupted some of its systems. It said its most critical work went on. On November 20, CNN said its classified network, NSANet, was back after more than three days. Jay Clayton, the director of national intelligence, blamed a PHP patch. Other officials told CNN they thought the patch caused the outage. They said it was not an attack. That answers how the network may have gone down. To see if anyone got in before the patch, I'd look at records from before the outage.121314

The NSA warned about the PHP flaws on October 21. Five days later, CISA added two of them to its list of bugs used in real attacks. A team can take down a service trying to fix the same flaw that someone else is using to break into another site. On November 18, NGINX shipped fixes for ten flaws of its own. The release shows what needs a patch. To say someone took over a server, we would need evidence from that server.151617

The public record points to separate failures. Microsoft showed how the load spread. Chase showed why people at two banks could see the same delay. To tie those failures to an attack, I'd look for what Optimizely and Salesforce found. What did both sides use? What changed, and when?

Say two sites go dark. Start with the logs at each site. Did both go down after the same change? Did one go down first and cut off a service the other one used? Look for the step that takes you from one failure to the next. The date on a press release tells you when you heard. The logs can tell you when it began.

The senators asked for an initial safety assessment and a joint technical briefing by November 23. I'd start with the traffic the FAA saw before it cut off the two sites. The agency can tell Congress when planes could take off again. Can it trace the first burst of traffic back to what set it off?18

Sources