Back to News
Advertisement

Ask HN: Any AWS billing issues known? Amazon forecast of 3 billion dollars

mmstolpm about 3 hours ago 52 comments
I receive an AWS Budgets alert that my budget is exceeding the alert threshold. Threshold is 5$. Forecasted amount is listed as $3,005,575,870.47. (Yepp, right, that’s 3 billion dollars.) I haven't even used AWS actively in the last year, but AWS console lists the amount as stated above. No feedback from AWS support yet, but the support AI chat bot says: "Die perfekt gleichmäßigen Tageskosten seit dem 1. Juli deuten stark auf einen Abrechnungs- oder Messfehler hin." ("The perfectly consistent daily costs since July 1 strongly suggest a billing or metering error.") Bot created a support ticket (or at least told me so). Anyone else seeing something like this? Already disabled all AWS IAM roles and deleted all other AWS resources I know of. AWS login not hacked as far as I can tell. But owing Amazon 3 billion dollars is a bit of a concern. Any ideas?
Advertisement

⚡ Community Insights

Discussion Sentiment

29% Positive

Analyzed from 921 words in the discussion.

Trending Topics

#billing#aws#dollars#billion#cost#data#estimated#pdt#don#owe

Discussion (52 Comments)Read Original on HackerNews

fronabout 2 hours ago
Woke up to a billing alarm email. Thought I had leaked my AWS keys accidentally and somehow run up 437 billion dollars of charges. Joke's on them though, I don't have 437 billion dollars

Anyways I didn't need coffee. That produced an adrenaline release unlike any I've experienced before. Thanks AWS

lijokabout 1 hour ago
If you owe AWS 437k bucks, that’s a big problem for you

If you owe AWS 437B bucks, that’s a big problem for AWS

binaryturtleabout 1 hour ago
I wonder how many people may have gotten an actual heart infarct because of that. There may be a person out there that may be dead as a result.

It's entirely irresponsible of Amazon to even display such values to the user.

jollyllama15 minutes ago
Above a bill of a certain amount, the alert should say "Sit down to view."
trial340 minutes ago
huge irl laugh at “i don’t have 437 billion dollars”
logicallee42 minutes ago
Worth a shot to give them a call and explain that. They can probably adjust it down to 100 billion.
Cthulhu_23 minutes ago
What is this, US health care negotiations?
astonexabout 1 hour ago
From their status page

>The second path involves rolling back a recent change to the billing computation subsystem.

Want to bet AI code was involved?

Cthulhu_22 minutes ago
I'm not a betting person but I am looking forward to the postmortem, whether or not AI was involved, and what their code to production verification stuff looks like now. This kind of thing should have been caught by automated tests.
CarRamrod32 minutes ago
>Spawning Sub-Agent: "Dr. Evil"
a01243 minutes ago
Is it still a bet if you have 99.999% chance to win?
verzali37 minutes ago
Can I bet you 437 billion dollars?
elondaitsabout 1 hour ago
Woke up to a 100 billion dollars in S3, which is above the USD 4 alarm threshold I had set (I pay $0.55 monthly) . Some AI decided to prune the most impressionable of us.
AngryKittenabout 1 hour ago
You folks are completely irresponsible with your finances. I only spent $2.4 million last night. You've got to learn to manage your money.
Cthulhu_21 minutes ago
Yeah, buy less cappuchinos and avocado toast so you can pay your AWS bills people!
hoppylukeabout 2 hours ago
My estimate was only $21M (vs ~$0.01 average bill). Wish I had checked status sooner and saved myself the panic!

My process went: verify email is not phishing (it was), login to console and check dashboard (same amount), attempt to understand cost (cost management kept contradicting itself), try to log support ticket and only on that part did I notice the status notification. At least I can breathe again now!

zengineerabout 1 hour ago
same process, same near heart attack - especially some days ago I had malicious activity on my platform and thought that now they actually found something.

Apparently you can trigger an Action (e.g. prevent uploads) when the billing alert triggers, but then my platform wouldn't work anymore, just because AWS had an issue. Also insane that Amazon still hasn't send an email to clarify.

jwikabout 3 hours ago
Yes, there is a known issue with cost estimation.

https://health.aws.amazon.com/health/status

mysterydipabout 2 hours ago
Bit of an understatement: “The displayed billing estimates do not reflect actual usage and charges.”
AngryKittenabout 1 hour ago
Whatever you do, AWS, don't post a related service health alert site wide on the console. Heck, don't even post one in the billing module. We wouldn't want to overdo the alerts, especially when we already have one being displayed to market the new FinOps Agent in Public Preview.
mstolpmabout 3 hours ago
Thank you. Have seen this after I posted.
coffee_is_nomabout 1 hour ago
Thanks, I panicked logged in and could not find the root cause of the bill.

Thanks AWS, no caffeine needed this morning!

throwatdem12311about 1 hour ago
Hey man AI makes mistakes sometimes that’s why you need to double check the output.
throwaway_5753about 2 hours ago
Scared me even though it was obviously a bug once I stopped to consider the magnitude ($bn). Very unfriendly that they don't allow for hard spend caps; closed my mostly dormant personal account as a result.
largbae26 minutes ago
If they did allow hard spend caps, it sounds like today would be a global outage.
Group_B14 minutes ago
Yeah nearly had a heart attack this morning. Thought keys were leaked for a sec.
sailfast33 minutes ago
Who is going to compensate us for the years taken off our lives when we received the alerts?
dpcxabout 1 hour ago
My estimate was over 2T. Talk about waking up quickly...
Cthulhu_20 minutes ago
For science I hope people are trying to figure out now how they could manage to rack up a 2T bill in a month.
Advertisement
_joelabout 1 hour ago
I've had a mysterious Neptune cluster appear on my billing. Never used it, no API key access (or IAM instance profile access, OIDC etc), nothing in my console shows I've ever had one in any region. Raised a case with support, they ignored it.
fuorileggeabout 3 hours ago
I have just received a similar alert for $ 5b

AWS on their support data is reporting this:

Inaccurate Estimated Billing Data

Jul 17 3:03 AM PDT We continue to work to resolve the issue affecting estimated cost and usage data displayed in the Billing and Cost Management Console. We have identified the root cause as an issue with unit pricing within the estimated billing computation subsystem and we are working on a mitigation. The displayed billing estimates do not reflect actual usage and charges. There are no customer actions required at this time. Once the issue has been mitigated, we expect full resolution to take multiple hours as we work through recomputing the estimated billing data. We will provide another update by 4:00 AM PDT or sooner if more information becomes available.

Jul 17 2:07 AM PDT Beginning on July 16 7:38 PM PDT, we began displaying incorrect estimated billing data in the Billing and Cost Management Console. Our engineering teams are engaged and investigating root cause. We will provide another update by 3:00 AM PDT or sooner if more information becomes available.

Jul 17 1:33 AM PDT We are investigating issues with Cost Explorer reflecting inaccurate estimated billing data.

alfiedotwtfabout 1 hour ago
Vibe Billing
lordleft44 minutes ago
Got a message that I owe 37 million on an account that I haven't used in probably...6 years?
themgtabout 1 hour ago
"If you owe AWS a hundred thousand dollars, that's your problem. If you owe AWS three billion dollars, that's Amazon's problem."
gioazziabout 1 hour ago
Heard of somebody who got 19 quadrillion dollars - I thought they meant Zimbabwean dollars
elashri39 minutes ago
one USD is about 362 Zimbabwean dollars. So it would still be about ~53 trillion dollars which is more than the nominal GDP of US and China combined.
Polizeiposaune7 minutes ago
For history here, see https://en.wikipedia.org/wiki/Hyperinflation_in_Zimbabwe

Zimbabwe redenominated its currency three times in 2006, 2008, and 2009, dropping a total of 25 trailing zeros in the process. the 4th Zimbabwean dollar in 2009 was worth 10^25 of the first zimbabwean dollar.

Waterluvian42 minutes ago
That would have been a great deal!
HarHarVeryFunnyabout 1 hour ago
They should have added "make no mistakes" to the prompt.
ohnoooooooooo43 minutes ago
Did it recover for you folks? I still see billions of dollars!
tedk-42about 1 hour ago
far out it's 10pm here and I was just about to sleep when my wife nugged me about a billing alert from AWS.

$151 billion the number for me.

dvhabout 1 hour ago
Prompt: bill our aws customers, make no mistakes.
vel0city35 minutes ago
Prompt: investigate new ways to fund additional AI datacenters.
zengineerabout 1 hour ago
or: Increase revenue!
Cthulhu_19 minutes ago
I mean just overcharging is one approach to achieving that goal I suppose.

But so is imprisoning or exterminating all humans for their own good, as most AI dystopias end up as.

Advertisement
ownagefoolabout 1 hour ago
Mines was $190,594,974,587,761.20 :)
gib444about 2 hours ago
Maybe they accidentally used the Argentine peso ;)
_joelabout 1 hour ago
Less hyper-scaler, more hyper-inflation
ohnoooooooooo43 minutes ago
did it recover for you? I still see billions
aweiland5 minutes ago
Mine has
th3o6a1dabout 2 hours ago
131 billion for me