GitHub Is Sinking

hherbertl about 3 hours ago 60 commentsRead Article on dbushell.com

DE version is available. Content is displayed in original English for accuracy.

⚡ Community Insights

Discussion Sentiment

43% Positive

Analyzed from 1841 words in the discussion.

Discussion (60 Comments)Read Original on HackerNews

johnfn•about 2 hours ago

Everyone wants to pin this on the Microsoft acquisition or incompetence but it seems pretty clear to me from the material GitHub has posted that AI has 10xed the amount of code being committed to GH, which has downstream effects everywhere - CI, Actions, code ingestion, everywhere. The author pins it on weird things like MS Copilot, which kind of feels like he’s listing off things he doesn’t like rather than casual favors. This is ignoring the 800 pound gorilla in the room.

einsteinx2•42 minutes ago

The graph in TFA shows the downtime pattern starting in January 2020. OpenAI released GPT-3.5 in November 2022 (basically December), and LLM/agentic coding didn’t really kick off in the way you’re describing until 2024, but really in 2025.

How can that explain the terrible uptime for the ~4 years post acquisition before all the AI stuff you’re talking about started?

johnfn•36 minutes ago

The subjective experience I and others report is that GitHub feels to have gotten significantly worse over the last few months. If you look at the month over month view of "Uptime history" in the cited link[1], it confirms this: it's been sub-90 (even sub-80 last month) essentially since the start of this year (i.e. when GitHub says that commit activity 10xed). Go back even a year and it's all in the high 9s.

I honestly can't explain the discrepancy between the graph in the article and the month over month stats on the same page, but the latter tracks both to my own subjective experience of GitHub and their own internal metrics.

[1]: https://mrshu.github.io/github-statuses/

silverwind•21 minutes ago

I think it's just a case of brain drain, followed by reckless AI adoption which both drove the quality down.

aliasxneo•about 2 hours ago

Yeah, I had the exact same response after reading the post. I mean, I'm all for jumping on the Microsoft hate train, but not if it misses the elephant in the room. Let's say the _perfect_ GitHub replacement spawns tomorrow? What's preventing the same infrastructure challenges of millions of lines of AI-generated code destroying it?

I think centralized code hosting is pretty much going to get killed by AI. Just like it's doing to social media.

einsteinx2•40 minutes ago

> I mean, I'm all for jumping on the Microsoft hate train, but not if it misses the elephant in the room.

That elephant didn’t even exist yet for the first few years of poor uptime shown in the graph in TFA… I don’t really disagree if we’re talking about the recent uptime issues, but how does that explain the years 2020-2023?

aliasxneo•25 minutes ago

It doesn't. It just means if they were having problems before, they've now been made significantly worse by AI (on the free tier). All I'm saying is that the problem is bigger than, "Microsoft sucks."

majormajor•about 2 hours ago

> I think centralized code hosting is pretty much going to get killed by AI. Just like it's doing to social media.

Private corporate codebases are a poor fit for GH because they don't benefit from public social graph effects. And the typical codebase isn't so large as to be technically challenging to deal with with OSS tools. I'd guess they make up a substantial share of revenue.

But once the reliability is called into question, self-hosted or smaller alternatives start to look good. Although there's some trickiness there if you want to be super cautious about making sure you can get to your code+infra in case of a vendor incident, especially if you're cloud based.

kyrra•about 2 hours ago

Saas code hosting seems to be the problem here. If companies self hosted, they could deal with the scaling problems themselves.

logicchains•about 1 hour ago

>What's preventing the same infrastructure challenges of millions of lines of AI-generated code destroying it?

There's something called "rate limits" that engineers not working for GitHub have probably heard of; it's this crazy idea that you should limit the load on your infra in order to avoid downtime. GitHub is not the first free service to ever have to deal with bots.

bdangubic•about 2 hours ago

of all the awful things AI is doing and will be doing to society, killing centralized code hosting and social media will be its shinniest moments, both deserve to die painful deaths

idiotsecant•about 2 hours ago

Yes, the terrible sin of ... Hosting code where people can find it

ako•about 2 hours ago

Why is centralized code hosting getting killed? I'm running an opensource project, >99% of the code is AI generated, could not do this without GitHub. Ai generated source code needs a place where AIs and people can collaborate. I'm expecting GitHub to hugely successful, but mostly for an AI audience.

chowells•about 2 hours ago

Because it's centralized. Your project pays the price for every unrelated project that's getting overloaded.

hirako2000•about 2 hours ago

GitHub hasn't changed in any positive way since the acquisition. A decade is a long time, it tells.

GitHub action, co pilot. Oh and that ugly AI search I'm unable to disable. Migration to azure.

Yes Microsoft managed to ruin the network effect. Outages? The straw that broke the camel's back.

madeofpalk•about 2 hours ago

3 months post Microsoft acquisition, GitHub expanded the free plan to include unlimited private repos.

The next year they removed the limitation on collaborators on private repos for free users.

In the last 4 years they’ve significantly improved their project management tools. I think a lot of teams can make do with GitHub Projects, they’re pretty decent.

Who knows if any of these are directly because of Microsoft or not. But there has naturally been material improvements to GitHub in the years after being bought by Microsoft.

politelemon•about 1 hour ago

> GitHub hasn't changed in any positive way since the acquisition.

It's more like any positive actions they have had are being outright dismissed or forgotten. They removed several restrictions that Github had over private accounts, as well as github actions. Aside from the downtimes, the Github of today is fantastic compared to pre-acquisition Github.

ako•about 2 hours ago

I'm loving it, running an opensource project mostly AI generated, i don't have to think about version control, building and testing my app, running AI code review, hosting my docs website, API and cli to enable Claude Code to interact with everything, etc.

It provides huge value for anyone running an opensource AI generated project.

Pay08•about 2 hours ago

How on earth is Actions a downside?

prerok•about 2 hours ago

I think they meant all the security holes that have been popping up and that there is no interest from Microsoft to fix them.

ExoticPearTree•18 minutes ago

I like to think that Microsoft is trying to run GitHub in Windows in their Azure cloud. And on the fact that every time GitHub is down I think of "someone updated the Windows Servers GH runs on and had to reboot everything".

While I'm 99% sure it is not true, it makes me sleep better at night. And giggle a little when it goes down.

QuercusMax•13 minutes ago

They definitely do something with Azure. Stuff related to GitHub action runs hosted on something.windows.net, which I believe is azure.

dwroberts•about 1 hour ago

Even if this is true: Microsoft own an entire cloud platform. They have enormous codebases of their own and they employ ~200k people. It’s just not an excuse, especially because they consciously made decisions such as e.g. private repositories being free

hackton•about 1 hour ago

If that's the case, we should also see the exact same pattern on Gitlab, Bitbucket, etc. Do we?

stusmall•about 1 hour ago

GitHub has been basically the default for free public git hosting for a long time. I was curious what bitbucket has and it looks like the free tier is so limited, I can't imagine a lot of people hosting vibe coded open source there.

fontain•about 1 hour ago

10x of nothing is nothing.

spiderfarmer•about 1 hour ago

What is easier to 10x? A tent or a flat?

veryfancy•about 1 hour ago

I’m with you here. Further: Even though I disagree with it, “GitHub down, Microsoft bad” is a defensible take, but we’ve seen it ad nauseam at this point.

delusional•about 1 hour ago

This would make sense if GitHub themselves cited increased traffic or load shedding as their root cause, but most of their incidents from the last month seems to cite misconfigured infrastructure or operational mistakes.

rvz•about 1 hour ago

Github had lots of outages even before AI was introduced.

senko•about 1 hour ago

The 800 pound gorilla in the room being a $3T company that also happens to be one of the largest cloud providers?

C'mon.

gverrilla•26 minutes ago

We want to thank you for your heroic service in our defense, sir. We really need people like you who know in what side they're at.

Microsoft investors

oarsinsync•about 2 hours ago

I went to look at a repo on Github today. Clicked on the "xxx commits" link to see the commit history, and got told I've hit a secondary rate limit and need to wait.

I'm the only person on this network that would even look at Github, and my connection has a dedicated IP, no CGN.

blinded•about 2 hours ago

The only real way to browse the site is to be logged in.

MYEUHD•about 1 hour ago

If you're on the desktop, refresh the page cache by using Ctrl + Shift + R

The page will load correctly

graemep•about 2 hours ago

Exactly the same here. I get that regularly.

noprocrasted•about 2 hours ago

Yeah this is just typical techbro gaslighting. There is no rate-limit and hasn't been for years (it's just default deny), but they refuse to change the wording to reflect.

dcrazy•about 2 hours ago

Would you care to cite your source that GitHub does not apply rate limits to unauthenticated requests?

noprocrasted•about 1 hour ago

The parent's experience which mirrors my own - on a clean residential IP that hasn't sent any traffic I hit that "rate-limit" on my first request to the commits list view.

So there is no rate-limit, it's a default deny for unauthenticated requests... which could be fine but at least update the error message to reflect that.

tbolt•about 2 hours ago

“GitLab - enterprise grade, meaning it’s bloated and confusing but it’ll impress your boss. This could be the choice if you need multiple meetings to make the choice.“

lol!

egwor•about 1 hour ago

We use gitlab at work, and I have to say that it is disappointing. The UI is so complicated to do the simplest things (e.g. to approve a MR you need to click a button that is actually a menu; the diffs are difficult to read; the 'To-do list' includes MRs that were already merged (how is that actionable?)) and it seems that they're struggling to turn around improvements quickly. The issue around the 'To-do list' including MRs that were already merged was raised years ago.

I also have to say that I'm surprised about the backlash against bitbucket. I find the UI incredibly simple and clear, as do all of the new joiners. With Script Runner you can do some pretty amazing things. It handles the huge repo's well too.

phyzix5761•about 1 hour ago

For $5 a month I can host a server and put a bunch of projects on there. Yeah, I don't have a million stars on my repos but it works for what I need and I can give access to whoever I want.

summa_tech•about 2 hours ago

It sort of feels like no major open source repository can be possibly left well enough alone. I remember how SourceForge went down the drain, it's a real pity to see same happen with GH.

Side note: I read the URL as "dBus hell". We've all been there m8

arikrahman•about 2 hours ago

No m80 it's a nushell based on decibel units dBu Shell

n_e•about 2 hours ago

I'm not sure what to make of the graph.

On the one hand the acquisition of GitHub may have caused the availability to be worse.

On the other hand, the 100.00% availability before the acquisition looks suspicious, wondering if it's not just the status page being better updated.

(I'm aware of the recent availability problems with GitHub, but on the graph the problems start in 2020 and don't seem to worsen significantly)

coolgoose•about 2 hours ago

So, what's the actual real alternative ? The one that also supports open source projects ? Ironically gitlab is costlier than github, and not without their faults, but that's "maybe" the only other alternative here, anything else ?

MrDrMcCoy•about 1 hour ago

Codeberg, Sourcehut, or self hosted Gitea.

kukkeliskuu•about 2 hours ago

I just installed a gitea. It seems decent.

TheCondor•about 1 hour ago

It absolutely is.

The only concerns are if it were exposed to the public internet and scale. For personal stuff? It's spectacular.

lawn•22 minutes ago

I've been running a self-hosted Forgejo. Extremely responsive and I've been really happy with it.

imagetic•about 1 hour ago

Anyone would buckle right now. Microsoft just sucks more at it.

mariocesar•about 2 hours ago

Agree with Gitlab as an enterprise alternative. Beautifully boring and safe to have complex teams and permissions. Also has a good enough Terraform support, and a nice workflow to host docker images

iamkrazy•about 2 hours ago

I installed forgejo on my home server and never looked back. The only problem I face is when hosting an app on DigitalOcean App platform, or vercel etc. They only connect to GitHub.

sinpif•2 minutes ago

Similar situation with Apple's Xcode Cloud.

lorecore•10 minutes ago

All of the reasons to avoid GitHub are also reasons to avoid the Digital Ocean App platform and Vercel. I use Digital Ocean, but just the VPSs. Don't let yourself get vendor locked in with these middle men, retain control and shoot for the most universal level of the stack you can.

cobbzilla•about 2 hours ago

I’m in a similar boat, I abandoned ship for Gitea years ago (prior to forgejo fork) and have no regrets.

For things that require GitHub I’ve been able to mirror repos there and get things working. Keeping code in sync is annoying though.

rbbydotdev•about 2 hours ago

I wasn't expecting to see the outages being nearly the same even before the 2023 ai inflection point

ChrisArchitect•about 1 hour ago

Ghostty is leaving GitHub

https://news.ycombinator.com/item?id=47939579

Before GitHub

https://news.ycombinator.com/item?id=47940921

Days without GitHub incidents

https://news.ycombinator.com/item?id=48012022

GitHub Actions is the weakest link

https://news.ycombinator.com/item?id=47933257

GitHub Copilot is moving to usage-based billing

https://news.ycombinator.com/item?id=47923357

QuiCasseRien•about 1 hour ago

onedev onedev ondev

I still don't see this tool when it's about a forge. It is a fantastic tool. Seriously guys, you should really consider it !

rvz•about 2 hours ago

I have lost count of how many times something went down on GitHub ever since documenting it on this comment chain [0] and also predicting 6 years ago [1], that going all in and centralizing everything on to GitHub was really not a good idea if you need stability or to push a critical fix and your GitHub actions doesn't work.

Now, are you going to finally self host or should we continue to expect another outage on GitHub?

This time, there is no CEO of GitHub to help us. It is Copilot, and Tay.ai that are still struggling to maintain GitHub.

[0] https://news.ycombinator.com/item?id=37395238

[1] https://news.ycombinator.com/item?id=22867803

LorenDB•about 1 hour ago

Why do I keep seeing people blaming Tay.ai? That was a one-off Twitter chatbot that was shut down a decade ago.