AWS Outage—What Happened And What To Do Next

Amazon Web Services is starting to get back up and running after a massive global outage that affected multiple apps. The AWS outage affecting the US-East-1 region left users of Snapchat, Fortnite Ring, Roblox, Coinbase and Signal as well as Amazon services such as Prime Video and Alexa unable to access the services globally.

AWS users flocked to report the issue on Down Detector, with a spike at around 4 .a.m. EST.

AWS says it has identified the issue and it is now fixed. However, some business users are still reporting problems accessing certain services.

Here’s what you need to know.

What Happened?

Problems began at about 3 a.m. EST, according to users of Down Detector. An hour later there were around 5,000 reports of issues in the U.S. alone.

Soon afterwards, Amazon said it had identified a potential root cause for error rates for the DynamoDB APIs in the US-EAST-1 Region. “Based on our investigation, the issue appears to be related to DNS resolution of the DynamoDB API endpoint in US-EAST-1,” it wrote on its service health dashboard. “This issue also affects other AWS Services in the US-EAST-1 Region. Global services or features that rely on US-EAST-1 endpoints such as IAM updates and DynamoDB Global tables may also be experiencing issues.”

Several hours after the outage began, AWS says the DNS issue has “been fully mitigated”, and most AWS Service operations are succeeding normally now.

However, it added that some requests may be throttled “while we work toward full resolution.”

Additionally, some services are continuing to work through a backlog of events such as Cloudtrail and Lambda. “While most operations are recovered, requests to launch new EC2 instances (or services that launch EC2 instances such as ECS) in the US-EAST-1 Region are still experiencing increased error rates,” it wrote

AWS makes up about 30% of the global cloud infrastructure market, so an outage like this can hit hard across the world, says Jake Moore, global cybersecurity advisor, ESET:

“A large number of global apps and websites rely heavily on AWS for cloud hosting and data processing which means the disruption can rapidly become widespread and create a knock on effect to many services,” Moore says.

I contacted AWS for a comment, but was directed to the service health dashboard.

What To Do

If you are still affected by the outage, things might take a while to return to normal. AWS has also recommended some steps. “If you are still experiencing an issue resolving the DynamoDB service endpoints in US-EAST-1, we recommend flushing your DNS caches,” the firm wrote.

Going forward as the risk of outages surges, Charlotte Wilson, head of enterprise at Check Point Software advises businesses avoid keeping everything in one cloud. “Test your failovers, train your teams, and plan for downtime before it arrives.”

Firms should also look out for attackers taking advantage of the AWS outage. “When companies rush to restore access, systems and staff are stretched thin, and that’s when attackers strike,” says Wilson. Expect a spike in fake ‘refund’ or ‘discount’ offers, phishing emails, and scam links claiming to fix the problem.

The AWS outage was not the ideal start to the week. Here’s hoping everything is fully working soon and users and businesses can return to normal.

Forbes