ES version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
49% Positive
Analyzed from 15107 words in the discussion.
Trending Topics
#canvas#instructure#students#don#more#https#university#software#important#com
Discussion (609 Comments)Read Original on HackerNews
We got our first email (from Academic Affairs) notifying us that it was down at 5:17pm EDT this afternoon, with little info; followup emails were sent at 6:24 and 6:57 with more info, but mostly about how we would be compensating for it and not about what actually was going on (other than, "nationwide shutdown" and "cybersecurity attacks", no further detail). I don't get a sense that they know much more than that, not that I would expect them to.
A perhaps telling detail: they're instructing us to have students email us directly with any work that had been submitted via Canvas. That suggests that they have no particular confidence that it will come back up soon.
I personally am only slightly affected; as a CS professor a lot of my students' work is done on department machines, and submitted that way, and I do the actual exams on paper. More importantly, I've never liked or trusted Canvas's gradebook, and so although I do upload grades to Canvas so students can see them, my primary gradebook is always a spreadsheet I maintain locally.
But I have a lot of colleagues for whom this is catastrophic at a level of "the whole building burnt down with all my exams and gradebooks in it"---even many of those that teach 100% in person have shifted much or all of their assessment into Canvas (using the Canvas "quiz" feature for everything up to and including final exams), and use the Canvas gradebook as their source-of-truth record. We've been encouraged to do so by our administration ("it makes submitting grades easier"). For faculty in that situation, they have few or zero artifacts that the students have produced, the students themselves don't have the artifacts to resubmit via email because they were done in Canvas in the first place, and they have no record of student grades or even attendance (because they managed that all inside Canvas). I guess they have access to the advisory midterm grades from March, if they submitted them (most do, some don't), but that might be it.
My gut feeling on this is that this is either resolved in hours (they have airgapped backups and can be working as soon as they can spin up new servers), or weeks (they don't). Very little in-between. And if that's true and we wake up tomorrow with this unresolved, I really have no idea what a lot of professors at my university and across the country are going to do to submit grades that are fair and reasonable. In the extreme case, they may have to revert to something we did in the pandemic semester (and before that, at my school, in the semester that two major academic buildings actually did burn to the ground a week before finals): let classes that normally count for a grade just submit grades as pass-fail. Because what else can you do?
(Well, one thing you can do is not put your eggs all in one basket, and not trust "the cloud" quite so much, but that ship's already sailed. I do wonder if in the longer term, anybody learns any lessons from this....)
UPDATE: As of 11:45pm EDT, my university's canvas instance is up and running! Here's hoping it stays (but I'll be downloading some stuff just in case...)
It’s so simple to send an e-mail to the student with relevant records on completion of a quiz or whatnot. They don’t do it, because they want to control the data. (And universities don’t insist on it for who knows what reason.)
It's better than nothing. (And good training for the real world.)
Also, most universities (and many schools now) issue academic e-mail addresses to students. In those cases, the email is definitive proof.
Ironically, this incident shows they don’t have control of anything.
What seems easy on hobby projects gets way more difficult at scale. Source: experience.
Either way, they were under no obligation to adopt this garbage technology regardless of whether it’s available, so this is 110% on them.
You are aware that you are posting on Hacker News, a forum for people who make their living selling software and the expertise to host it?
Edit: No idea why this was down voted so much. I'm not defending Canvas, just wondering what the alternative would be.
Everything we know has come from reddit threads / hackernews threads. There has been 0 official communication today indicating this was an attack, yet the login page was defaced by ShinyHunters.
That makes you one better than me. :( One thing's for sure--I'm never trusting it again.
I already had almost all my materials outside of Canvas and just used their API to upload it. So at least that's safe. But the grades... dang. Luckily we're only halfway through our quarter and it's not finals week.
Our instance is still down, but your update gives me hope.
Canvas does provide a lot of value (all courses, teachers', students', and parents' contact information, all learning plans, schedules, room numbers, all grades, a lot of tests and assignments themselves, all upcoming assignments and deadlines, a lot of other coursework is in there, as are the final grades) but it shows that with external SaaS you might be one attack away from not only losing all that convenience but also in a world of hurt 'cause you lost all the data and now have to figure out how to proceed without the data and the system.
US high schools are in the middle of the finals, and seniors are getting ready for college (the transcripts to be finalized and sent out in a few weeks) so that was a scary timing.
What good is having airgapped backups and spinning them up, if they are instantly vulnerable to the same attack again?
It does depend on what the attack is, but how do people approach that scenario?
And then wish for the death of saas and a day where you can deploy your own software you can control and modify as you need.
Does a future employer look at pass/fail vs the grade? do they care? Are there even jobs that matter enough to care out there for them?
This seems like, solving the problem but without actually seeing the broader goal or trajectory education is supposed to follow.
I don't know for a fact how pass/fail is treated by employers, but there are indeed some that look at your college GPA even 10+ years after you graduated. I suspect they don't care about the specifics of how your overall GPA was derived though, so pass/fail likely doesn't matter (unless you did really well and expected the grade to boost your GPA, and then pass/fail essentially does nothing to the GPA, thus kinda eliminating the GPA boost).
I got asked for my undergrad GPA (I graduated ~10 years ago) more than once over the last year by some finance/quant firms.
As for whether "do those jobs even matter enough," I guess it is more of a personal subjective take. I found the work that the people at those companies did (and the problems they solved) to be very interesting and challenging, I found the people working there to be extremely sharp, smart, and genuinely nice to interact with (which is an ideal work environment for me), and I found the total comp to be great. Honestly, I cannot think of much more to ask from an employer.
Canvas is mostly FOSS
https://github.com/instructure/canvas-lms
Does anyone have a list of affected schools?
I have an idea for the midterm (pun intended): Maybe don't jump feet first into the deep end of a single point of failure going forward.
... and assuming they have a documented, tested, and trusted restore process.
Some data was permanently lost, and then officers told reporters that multi-regional backup was not yet built because it was too hard at such a massive scale... of 858 TB.
There are probably many S3 buckets in existence that are bigger than that.
Not saying that they should've used S3, but it's definitely possible configure multi-regional backup (and a government can afford it).
As a parent of kids who are impacted by this, I’m not super concerned about the data being held for ransom, but I sure as fuck am concerned about how much it’s going to cost the district to move to another provider.
Does Canvas have cybersecurity insurance?
Schedule a single exam and that's your grade for that subject? That's how it should work anyway, credits for work during semester (or worse attendance) are not needed to evaluate if someone learned the material, give them an exam and done.
Which to me seems the best way, you still have to learn throughout the year. Especially to avoid cheating this works nice. And as an aside, most people I know that did a year abroad in the US got 1-2 grades higher, as it was quite easy to just farm extra credits.
At my school, tomorrow is the last day of exams. Most of the students have left campus. There's no time or mechanism to schedule an(other) exam.
Using attendance is a carrot to get students to show up, which leads to better learning outcomes overall - which should be the goal.
And from the hacker's message itself, it's clear they want money in exchange for not releasing private info, not for the data itself.
Do we live in a fear based culture? Why the panic? Even if everything was hosted on Instructure's infrastructure, it's all AWS. I'd be VERY surprised if there aren't multiple way to go back to a previous state.
Most of the work and delay is to make sure they figure out where the breach occurred.
Here in the Netherlands a data center's power source (not even the machines) burnt down, data center is offline and University of Utrecht, one of the biggest universities here, is closed. Access passes don't work, work from home environment doesn't work, student information system is down, system for grading doesn't work. No failover for any of them (or maybe it was in the same DC?)
https://nos.nl/artikel/2613485-storingen-in-hele-land-door-b...
Backups can be sabotaged (turned off or schedules manipulated) or compromised (say, by lateral movement).
> Even if everything was hosted on Instructure's infrastructure, it's all AWS.
AWS Backup isn't foolproof. Get your hands on administrator credentials as an attacker and suddenly the only thing between everything being gone for good and unrecoverable even for AWS is remembering to have put a permanent deletion protection on all resources in AWS Backup.
Incidentally I've always hated Canvas and probably every other LMS provider, but what is particularly amusing about this current outage is that it is occurring at exactly the time when universities are demanding that all professors put all of their materials on Canvas, without exception, due to ADA compliance regulations. It is explicitly forbidden for professors to, e.g., refer to pdfs posted on a personal website.
Other commentators here seem not to understand that many faculty also do not enjoy being forced to use Canvas.
The MS services have not improved teaching at all. What they do, is fragment communications, and add ever more places people have to look, in hopes of finding things.
But the administration loves them. "The bureaucracy is expanding, to meet the expanding needs of the bureaucracy."
Thankfully, I store my teaching materials on my personal non-uni webpage, and the student's marks in my office's computer (apart from the MS-based Uni system).
Whenever something happens with MS, chaos ensues throughout the whose Uni and the students end up paying the consequences.
And of course the other serious concern I have with Canvas is that they are likely using all the materials faculty upload to train their AI replacements. Many of my colleagues engage in dark humor about this but I haven't noticed much action.
Instructure (Canvas's developer) partnered with OpenAI last year [1], about a year after KKR and Dragoneer (PE firms) acquired it [2].
[1] https://www.forbes.com/sites/rayravaglia/2025/07/23/instruct...
[2] https://www.pehub.com/kkr-and-dragoneer-complete-4-8bn-take-...
I would guess these plugins are chosen so a majority of user won't want to live without them.
It also seems these plugins "link" to canvas-lms, so keeping the proprietary would be a GPL violation if anyone except Instructure holds part of the copyright to Canvas.
That calculus is about to shift.
I'm not sure where your stereotype even comes from, because Canvas is not trivial software. You can see for yourself as it's AGPL and I assume you looked at the code before criticizing it because any good engineer would do that.
It's been long enough that I can't claim to be in touch with the current generation of teaching faculty. But it might be an element of that, combined with the desire to provide accessibility for the handful of students who do in fact need the accommodation.
The administration has so far opened with one “Canvas said” and then an hour later one “Canvas is down indefinitely” email noting that they’re aware it’s serious.
(Canvas is a glorified wiki for teaching students, with quizzes and such, for those unaware.)
That's my biggest fear.
Note that little of this really helps the students that it is supposed to help, because as you wisely point out, raw HTML is almost by definition extremely accessible. I work in a field that uses Latex and the source code of Latex should also be considered more accessible than the compiled pdf. But for university administrators the only important thing is that the accessibility metric that appears (or used to appear, before today!) on Canvas shows 100% accessible.
Nobody has infinite energy, and disabled people don't have infinite social capital. It's a shame when energy from that shared pool gets spent on things that don't really impact meeting people's access needs.
And the other thing is that everyone's access needs are different. It can certainly be useful to try to set a baseline or propagate common guidance. But the most important thing, especially in a university setting, is for instructors to be flexible and responsive and for classes (and non-teaching workloads) to be structured in a way (e.g., small enough) that supports that.
I think metrics like "100% accessible" might even be dangerous. It makes it easy for able-bodied people who aren't in direct contact with disabled stakeholders to pat themselves on the back without actually knowing what's going on.
Bleh. Good luck doing right by your disabled students and disabled colleagues, and good luck resisting the bullshit.
I'm a prof. When I have a student with special needs in my class, the administration tells me ahead of time. I make the necessary allowances - and those differ from case to case, anyway: whether it's extra time in exams, or someone who is deaf, or someone who is blind, or whatever.
When it happens, I make the necessary allowances. When I don't, then...I don't.
The obsession that everything has to be 100% accessible, for every kind of disability, all of the time? That's just nuts, not to mention a complete waste of resources.
No this will not stop this and companies need to be held accountable for their lack of security investment. Every attack should be investigate if the company met an agreed industry standards best practices and staffing, etc. The penalties for not meeting the requirements should be punitive.
It should be illegal to host insecure services, especially when you're dealing with PII. Breaches keep happening and nobody gives a fuck, because the worst that'll happen is you might lose a handful of customers and buy some "credit monitoring".
Incidents like this should be followed by an audit and charges being laid. Send corp officers to jail for negligent security failures. If you can go to jail for accounting fraud, you should be able to go to jail for cybersecurity-promises-fraud.
They claim to be compliant with a number of security standards [1]. I would love to see a postmortem audit of how much of this they actually implemented.
[1] https://www.instructure.com/en-au/trust-center/compliance
Instead, there should be standard civil penalties for leaking various degrees of PII paid as restitution to the affected individual. Importantly, this must be applied REGARDLESS of "certification" or whether any security practices were "incorrect" or "insufficient". Even if there's a zero-day exploit and you did everything right, you pay. That's the cost of storing people's secrets.
This would make operating services whose whole "thing" is storing a bunch of information about individuals (like Canvas) much more expensive. Good! It's far to cheap to stockpile a ticking time bomb of private info and then walk away paying no damages just because you complied with some out-of-date list of rules or got the stamp of approval from a certification org that's incentivized to give out stamps of approval.
For most individuals impacted by these hacks, appropriate restitution would be $0. Anything more than that would go beyond making them whole.
I do agree with the audit and punishments for clear failure to adhere to established standards.
Just take a look at the recent Epic vs. Health Gorilla lawsuit to see how nonexistent the protection is around exchanging your medical records, one of the most sensitive types of PII.
Here’s an example. https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin...
These problems will continue as long as it is legal to operate in an unsafe way.
We've learned this in every other industry, but we can't seem to accept it in software. One of my hopes for AI is that it reduces the cost to behave responsibly to a level where this absurd resistance to acting responsibly erodes.
Every service that is online will be hacked eventually, it's only a matter of time.
Time is the most powerful force in the universe.
What? Why? Who died? This whole thing is perfectly dealt with through civil process.
There are no shortage of coins and no shortage of sketchy exchanges. The platforms do work with LEOs, when asked, but my understanding is that unless the perp was a serious nonce, chasing the transfers themselves is a fools errand.
Like is that your actual model? I’m curious
It's very easy to play with lives that aren't yours.
It's a familiar example of the perennial "[THING] could be solved overnight if [PERSON_OR_GROUP] would just start taking [THING] seriously" trope.
The best response to a cyberattack on critical systems is to take security seriously. Document the offense, avoid the same mistakes and invest in penetration testing. Of course, nobody is incentivized to do that until they're attacked, so the cycle perpetuates itself.
When appropriate. I.e. never.
If you're going to get the chair you might as well murder some witnesses or destroy some systems to hide the fact you got hacked. "Hack? What hack? Our servers all burned down in an arson attack".
An investigative body, the same kind that determines the who, the why, and the how when an airliner crashes or a bridge collapses. Obviously a lot of work needs to be done to get from point A to point B, and it won't happen overnight, but software development is currently a deeply unserious profession and at some point a genuine software engineering practice needs to be developed.
I am, perhaps naively, slightly hopeful that the LLM bullshit plaguing our industry will be the gust of wind needed for the house of cards to collapse and governments to realise that allowing the entire world to be vibe coded is not sustainable.
ShinyHackers, obviously.
Kids from the local uni having a lark, stalkers, vindictive ex employees, local gangs, criminals who understand their victims because they hail from the same community. These are your local hackers. Sift them from the nation states and international crime groups, then deal with the International as a matter of diplomacy. Because we do this so poorly locally, we have little ammunition to when it comes to diplomacy. "reduce attacks by your crime groups and we buy your natural gas, seel you wheat etc"
Want more motivation?- 75% of the local attacks by volume send funds back to terrorist or separatist organizations.
It is not an in-soluble problem. Sentences are a fraction of the answer, effective and receptive reporting processes are more important, then government backing for investigation and enforcement, then policy around home-team activities (ie don't do the bad things yourselves Mr Gov). Deterrence comes after all that.
There are already significant penalties for doing anything like this. The guy involved is in prison for a very long time. I don’t recall the exact number of years but I do remember it was so long that he wasn’t going to see his kids grow up.
I don’t think anyone who puts a little thought into a crime like this doesn’t understand that the penalties are already very huge. You don’t get a slap on the wrist for extorting a company (or person, for that matter)
Really, though, if you want someone to blame, Instructure is not a particularly compelling target. Let's review:
1. Iran is intentionally targeting infrastructure due to a war started by the current administration.
2. China is actively seeking corporate secrets to steal and commercialize for themselves, spurred by extreme protectionism and retaliatory tariffs.
3. North Korea is doing anything they can -- including just taking a remote job by proxy -- in order to extract any money.
4. And Russia is working with and aiding all of them, after everything else going on has forced the embargo to break.
5. All of this while completely alienating every single one of the United States' allies.
6. Meanwhile, the American DHS is currently shut down.
7. And this is after Trump cut funding and personnel for CISA severely enough they've had to end the contract with MS-ISAC, meaning all state and local entities can only remain in the organization if they foot the bill for it directly and CISA and other agencies responsible for cybersecurity are more thinly staffed than they have been in decades.
In short, the current administration systematically disassembled all the protections we have built over the last 100 years, and then placed infrastructure -- schools, in this case, but also power companies, water treatment facilities, communications companies, local governments, hospitals, food producers -- directly on the front lines of the modern geopolitical conflict.
That vast ocean that has kept us safe historically is a poor moat in the modern era.
You seem to think "if it's illegal it won't happen". Instead you need to think about unintended consequences and what would actually happen if this were law. People would hesitate to contact the police for help before they've decided, or not do it at all. And not report it.
One way to weaken any group that works on trust would be to make them less trustworthy. That way victims wouldn't be as confident paying the criminals and thereby making the effort by the criminals less attractive.
We're talking about vulnerabilities that have existed 10+ years but nobody noticed until AI.
I think in principle, its sound. Im also just baffled hearing anecdotes from friends that are in big corp world and hearing the type of incidents they have, and how they respond to it.. It makes me wonder if there is enough capable talent to go around for the "boring corp" crowd.
Hint: I don't think there is nearly enough talent to go round, but for these companies, its either that they think they have solid experts (and didn't), OR its not a real priority until you get hit.
a loved one, gun to the head: "please pay the ransom, i don't want to die!"
what's your play now? save loved one, and go to prison? or worse, bank blocks transfer, and they die?
go ahead and tax ransom payments (0 tax if human life at risk, 10x otherwise) if you have to, but making it illegal feels disconnected from the messiness of the real world. then, go after the attackers.
Apart from the 4% of the total worldwide annual turnover fine that theoretically could be levied under GDPR, but has never been imposed in full.
That makes as much sense as illegal to give your wallet to a mugger.
I.e. no sense.
2. The payout to the hackers should form part, but not all of the penalties. Pay those guys for their great service to humanity they earned it.
But it turns out that MIT used to have their own homegrown system, and recently switched to Canvas. Bet they're regretting that now.
The build vs. buy decision seems to have swung very hard toward buy in the last decade, and I think that's a shame. Yes, orgs need to focus on their core competency, and sometimes that means outsourcing things that aren't core competencies to third parties. But there are always downsides.
And it's pretty easy to customize which is nice.
Throw it in an auto-scale ECS cluster and you have something that goes from 100 students to 20k easy.
(I don't have experience in hosting either software so I can't really comment beyond that)
A lot can change in 10 years, sure. Maybe Moodle is better now (I doubt it). I'm all for self-hosting a LMS. But, can we at least self-host a good one?
it's MIT.
I believe the same applied to the professors themselves, although that was hardly enforced.
My highschool, for a while, had a website, which was eventually replaces by a large corporate CMS. Was the website as complicated or complex as the CMS? No, you would have needed to know HTML to publish to it. The CMS was no doubt "more user friendly", I suppose.
But … the original site had a soul. It was unique to the school. There was a student directory! All lost, because the CMS meant utter standardization between all the schools using it (their pages were all identical, except for each got like a different picture of the school as the banner at the top) and the CMS did not do directory anything.
Of course, the directory largely didn't matter in the end. (This was when you needed people's landlines! Quite laughable nowadays…) But it was still sad to see it lost, and several of us students worked on it, which provided us with some early real-world experience.
A large number of my college professors published their own sites, too, where they'd put their lecture notes, homework, etc. I loved those far more than I loved "Canvas" or whatever the ugly LMS we used was.
The one they had before Canvas was very very inadequate.
edit: also some of the more popular cs classes have custom websites and don’t really use canvas, but that isn’t the centralized IT department’s doing.
IT staff who are ambitious and talented don’t last long in education. The pay is very low compared to industry. Where I worked, you could retire with a comfortable pension after a number of service years, so the IT staff outsourced as much as possible so they needed to take zero risks to their nest egg. Blame all the problems on the consultants and do as little as possible.
It’s literally where dreams go to die.
MIT is known for the brilliant professors and students but at the end of the day, running a university is pretty standard stuff. They don’t need a genius rockstar to admin the courseware servers.
This would be like TurboTax "scheduling maintenance" on April 14th in the US.
There is a lot of people who likely are unaware the latest outage is because they were compromised again.
Them marking the incident as 'Under Maintenance' means the status page isn't reporting this as an outage and adding to downtime%.
It's not unreasonable that non-technical people would expect paid cloud services to be good custodians of the data entrusted to them.
These services also do everything they can to encourage you to work within the online platform rather then working offline and then uploading.
For example, there's no easy way to author a quiz, set up the answers offline and then later upload it.
Last month it was a presentation. She had to make a poster that would be displayed on the big electronic "whiteboard" running Windows of some sort. The page layout software was so terrible that she repeatedly deleted the entire thing on accident moving text around.
This month, it was a short paper she had to write in Word, but through Teams. Literally, the Word icon is in the Teams sidebar, and she also had all kinds of trouble with it freezing or misbehaving.
In both cases, I advised her to write all the content in Notes in macOS and when she had it all ready to go we'd paste it into the crappy software so she didn't have to worry about losing any more work.
Long story short, she's non-technical and she's learned a very valuable lesson about these systems and how much trust to place in them.
One thing to target coroporations but leave the students alone....
Heard you loud and clear sheesh
That doesn't excuse any of their other messaging though.
Also looks pretty bad their whole platform was compromised by the same hacker group again.
I also ran the entire DOMPurify sample XSS and managed to find one way to download custom content onto someone's computer.
I'm actually much more interested if there is any financial liability for Instructure here? It's interesting that it's the universities being ransomed, while the technical failure was Instructure's. We're used to uptime SLA's -- what about security breach SLA's?
My guess would be they get likelihood of getting paid when blackmailing 9,000 schools (at least a few would pay up) than blackmailing Canvas/Instructure.
I don't think any SLA/terms would change who gets to feel the pain.
Don't ransom all your eggs in one basket
I dont think a competent CS department requires their being a homegrown or on-prem system for use in the university. That could happen, but if resources could be better spent by purchasing rather than building, then that should be the correct choice.
Universities which do have large agriculture/farming related departments often operate their own small scale test/development/experimental farm.
Also yeah there is value in being able to blame another party, and also being down when everyone else is down.
Brought up a question I've had every time I read about these leaks... what kind of pipes do these shadowy groups have that they can grab all this data? I've spent days waiting just downloading a few 100 of GB from OneDrive. How do they grab all this data, are they just slowly gathering it for months via a compromised desktop somewhere, or if not, are the companies not monitoring for unexpected massive amounts of outbound traffic from their database or file servers?
do you mean equivalent ?.
Instructure, "the developer and publisher of Canvas," was founded in 2008 [1].
[1] https://en.wikipedia.org/wiki/Instructure
Edit: https://status.parchment.com/ says "While Canvas, Canvas Beta and Canvas test are currently unavailable, we are simultaneously monitoring all of our other product environments, including Parchment. We continue to see no reason to believe any Parchment resources have been impacted."
1: https://ibb.co/r29RjdnH
We received communication that Canvas is down for "Under Maintenance" although it seems ShineyHunters have compromised Canvas again with that message you posted.
We do not see that message anymore, although all instrucuture.com URLs are down. The list of schools in the ShinyHunters publication can be found here: https://web.archive.org/web/20260507042014/http://91.215.85....
Original now shows 404.
https://github.com/instructure/canvas-lms/wiki/Quick-Start
> It is recommended that you have at least 150GB of available hard drive space, 8GB of RAM, and a quad-core CPU to use this script.
As far as I can tell, this is not for running a production environment with assets. This is just the development environment.
I totally understand why a university wouldn’t want to bake their own learning portals but just feels like such a single point of risk to use third party solutions for something like this.
Back in my day… all we had was a school email via on-premise services. I guess we registered for classes in a web portal but that’s about it. The idea of online class was entirely foreign at the time. Ain’t nobody hacking a blue book.
There is a saying in the software security industry that (I'm paraphrasing from rusty memories) a system is secure if the cost of hacking it is higher than the value it protects.
Each system being completely distinct from another means that the cost of hacking the average student goes up by 9000 (from the article, Canvas is used by 9000 schools).
Still not saying that rolling out your own is the preferred solution, but the idea is not as ludicrous as it would seem, and should definitely be entertained and discussed, at least.
But also, the cost is much, much higher to the institutions, which is the salient point. You're going to spend years developing a system, deploying it, training staff and students, supporting it. I see mentions here of in-house systems being developed much more cheaply and I don't believe it. The economies of scale are at work.
I worked at a university for many years and I can't recall anyone I'd consider to be a competent software architect working for the IT department. Hell, we had students writing major webapps that kinda sorta worked well enough.
As a faculty member at a large university…I have a deep respect for the impossible job of university IT departments.
We originally rolled our on LMS decades ago. When we switched to canvas we kept the home brew running for five years past its expiration date because faculty refused to remove their files. Finally each one was manually moved by IT for the recalcitrant old faculty.
I used to work in academia and am now an LMS admin (in private industry). I've interviewed for LMS admin positions at educational institutions and each time I've ended up walking away. The questions I was asked at the last interview revealed what a ridiculously unplanned, spiraling mess their system was and that I would have no agency over it. No, thanks. And it was clear the reason for this was faculty recalcitrance and an inability to tell them no. Each one wanted a special plugin/special way of doing things, causing a giant mess of insecure bloat, and a fair amount of interview questions always amount to 'how do you wheedle faculty into doing things/placate their egos to keep things running?'
I'm not a rockstar candidate either: I'm a disabled, geographically-constrained, self-taught(ish) sort-of techie. The disability means I have substantial holes in my resume/work history, etc. I don't have a CS degree or any kind of formal IT education. If people at my level of knowledge are looking at these jobs and passing because they're not worth it, I can't imagine the actual pool of people who get hired is great.
LMS admins in particular are going to be harder to find/retain because we tend to have options we can jump to that would be less onerous than doing LMS admin for a dumpster fire. I could go straight IT or full Instructional Design, for example.
In private industry, I can tell people to kick rocks if they want to do something that the system doesn't support/is a really bad idea. And if I can't, I'm not held responsible for the consequences.
Well not with that attitude
https://moodle.org/
They used to, in the pre-cloud/SaaS era; and they were much simpler and better UX than the slop that they're renting today, because the actual users were not far from the developers.
The amount of corner cases and performance requirements during rush times (semester start) made it really infeasible for a university to roll their own.
* German universities have this funny system where 51% of such boards are controlled by the professors and the rest is made up of other employees/staff and students. They call it academic participation.
https://www.instructure.com/incident_update
This suggests a bad actor at any institution could do the same thing done here. No?
I believe FERPA's PII provisions apply to Canvas and contractors handing PII in general (at least as interpreted by the Department of Education). Now, will Canvas be held accountable by ED in this administration? Hah – DOGE probably ran that through the shredder as well.
doesn't seem that scheduled to me
That's just the quickest page/status update to throw up; it was a one-liner to push it live back when I was on the deploy rotation.
I'd hazard a guess they have more important things to worry about right now than exact status page messaging ;)
Funny how a lie is always quicker than the truth...
https://news.ycombinator.com/item?id=48025001
I'm a software dev who was affected by the outage. I was working on an app that connects to the Canvas SAML endpoints. One minute I was able to run my code, the next I couldn't. This was a little after 17:00 EST.
edit: here's the list of impacted universities (unsure if they all have their canvas instances offline, but i'd be surprised if not): http://91.215.85.103/pay_or_leak/instructure_affected_school...
Someone dumped the content into a google doc on reddit[1] if anyone's interested.
[1]: https://docs.google.com/document/d/1MTktVSwTUM5I_w7bKNGj94sT...
> Someone dumped the content into a google doc on reddit[1] if anyone's interested.
> [1]: https://docs.google.com/document/d/1MTktVSwTUM5I_w7bKNGj94sT...
Thanks for linking this. Ended up finding my kids school district on the list unfortunately.
Back when I worked for Instructure ~10 years ago, Canvas was effectively a single, giant, monolithic multitenant app with one instance backed by several thousand app servers and ~100 separate Postgres database clusters that any app server could talk to.
Schools were grouped onto pools of app severs and Postgres database clusters more or less according to locality and cluster availability. I want to say a handful of the largest schools got their own clusters, but I'm not certain, and at any rate their clusters could certainly all talk to each other.
It was actually kind of neat from a technical perspective: any Rails model across the entire Canvas world could have a "foreign key" pointing to any other Rails model anywhere else. Among other things, this allowed for users who could administer multiple Canvas organizations, even if those organizations resided on different Postgres clusters. https://github.com/instructure/switchman is their gem that made that all work. (I put "foreign key" in quotes because the whole thing was implemented in software, not with actual database FKs, for obvious reasons.)
---
Of course, the massive downside to that sort of thing is that if you manage to pop one Canvas app server, you have the keys to the kingdom. I wonder if they'll sharpen the edges between clusters in response to this...
---
(Disclaimer: I left Instructure back in 2017; much could have changed since then, and my memory could be faulty about the specifics. Caveat emptor.)
dig canvas.ucdavis.edu
dig canvas.duke.eduIs this accurate? Or is this still an ongoing issue?
Let's not side with the parasites.
What did Canvas PR do except do a poor job? Doing a poor job of PR is a whole, whole lot less worse than actively destroying people's lives for profit.
Our whole testing center is down. This is inconvenient, but mainly it's amusing. I swear strangers are talking to each other more. I'm noticing people just sitting in the sun and relaxing. Nature is healing.
(Of course, plenty of people have also just finished their exams, so it's hard to know the cause.)
Any idea what data Instructure-and-also-now-ShinyHunters even purport to have beyond names, profile photos, pronouns, homework assignments, school communications, phone numbers, and email addresses?
i.e. What makes this threat so different from what any old data brokers have already scraped?
What leverage besides aura farming do the ShinyHunters really have?
All I can think of that's really valuable is passwords. And private communications in Canvas DMs. But if you're being at all intimate over your school email, that's kinda on you.
Anyway surely Instructure only stores user public keys or something?
Alternate history question: If they just sold the data, never revealed the hack, and didn't make a scene, from a customer perspective, how different would this be from business as usual?
https://www.abc.net.au/news/2026-05-08/students-lose-access-...
I lost access when I left as it was tied to my work email. I downloaded a lot, but there was still some useful stuff on the boards.
I wonder what the havkers found out about me. Perhaps the class notes will be lifted to train AI, higher quality than a lot thats on the internet anyway.
Everything from middle school up to grad school.
It's a particularly interesting time to have this happen too -- many finals going on now.
We already bond over how awful the Canvas UX is (and she has a bunch of Chrome extensions to improve it.) Now we’ve got something else to gripe over together.
haha i went to go check and they haven't merged a PR since 2017
I'm friends with a professor who complained to me a couple times about how sometimes he will need to scroll through pages and pages of courses he taught in the past. He also mentioned that profs aren't able to delete their own course shells either.
If we increase the penalties for a company being hacked, you create even MORE incentive for hackers to try to break in, because if they succeed, they have a pretty big stick to threaten companies with when demanding a random payment - not only will the company have the negative effect of the data being leaked and the PR that accompanies it, they now know that if they don't pay and the attack becomes public knowledge, they face a big fine or other punishment.
A company is much more likely to pay a big ransom if they know they are just going to end up paying that much or more in fines if they refuse the ransom and report the hack instead.
If you take this route, and increase punishment for being hacked, you are making a pretty big bet that the main reason companies are hacked is because of poor security practices. I am not sure if that is true or not.
Consider surgery instead of software development. There are general best practices, but the difference between a good surgeon and a poor one is a small number of deaths. Malpractice insurance is high. Litigation is constant. And patients still die on the operating table. It's unclear what all the malpractice tort law actually gets you in the end.
In most of these cases, the companies involved did NOT follow standard security practices.
I am pretty sure that is what people mean when they say "held responsible", they mean "held responsible for failing to follow standard security practices", not for the actual act of getting hacked.
Is that really the analogy you want to use the bolster your argument? Licensing was forced on the medical profession because of rampant quackery causing a large number of deaths. Some of the horrors that went on before enforced medical licensing are well-nigh unbelievable, e.g. https://en.wikipedia.org/wiki/John_R._Brinkley
Just like 100 years from now, many of today's medical practices will also be seen as barbaric.
But even if they do everything right, is it really fair to let the companies just shrug their shoulders and say "it happens"? While their users are the ones who really get hurt.
I like this analogy, but deaths shouldn't be the leading indicator just an indicator. Family member had a surgery with well known procedures, say removing a gall bladder. Unfortunately, this surgeon skipped a step in lieu of setting a record for fastest procedure. Because steps were skipped, the gall bladder was not scooped into a net to avoid spilled gall stones which resulted stones spilling into the abdominal cavity requiring numerous follow up surgeries to remove the spilled stones as they made themselves known. So clearly not following accepted procedures should be a clear win in a malpractice case, yeah? Wrong. No doctor would testify against the surgeon and the case was dismissed. I feel like this is exactly how it would work in software security incidents as well.
That was the foundational premise of Dr. Atul Gawande's book The Checklist Manifesto, an expansion of his article The Checklist in The New Yorker [0]
[0] https://www.newyorker.com/magazine/2007/12/10/the-checklist
[1] https://digital-strategy.ec.europa.eu/en/policies/cyber-resi... [2] https://ec.europa.eu/commission/presscorner/detail/en/ip_22_...
I mean, maybe it changed in the last 10 years. But I was a TA grading CS majors for a while. Their C capstone or what have you.
Some were decent but naively coded. Most were pile of shit half hazardly put together so it output what is needed to get passing grade.
But I agree with you in spirit!
I'm under the impression files are getting released 12th May. I don't see any reporting on 800GB?
Of course if you can't complete your exams because of this, that's more of an issue!
I'm honestly surprised more people aren't talking about this.
The timing probably isn't a coincidence. Great time to stress out students and staff alike. Hopefully it doesn't affect them too much in the end, but I imagine it will.
What we don't have access to includes:
* Already graded work
* Ungraded work
* overall adn assignment grades
* lists of students and student emails from the course
* messages from students that are often sent through gradescope
Just...complete implosion.
this is really, really, REALLY bad. it's not great that names/emails/etc will potentially be leaked, but also private messages between students and instructors. and since many of the campus systems rely on canvas integration, things have pretty much ground to a halt a week before finals.
after they were breached on the 1st of this month, instructure had an announcement yesterday that "everything is great! we're good! hackers are gone! we've rotated our keys!".
no. nothing is great. we are not good.
This will keep happening, more and more, and never stop, until we create a software building code and legally require it for all online businesses.
Universities, Parents: ya'll actually have the political and economic power to get a software building code passed. This incident isn't the last.
...what does that DDB DNS issue have to do with anything?
Shame on your existence basically.
Is that a Pokemon reference?
They moved away from Teams because it objectively sucked, but I haven't heard of widespread compromises like this in Microsoft's systems so...
It's always been as stupid as requiring that your chalkboard, chalk, chairs, bluebooks, pens, paper, gradebook etc etc all come from the same company.
I, for one, am very much looking forward to my IT Gov council meeting tomorrow.
And GitHub doesn't provide a way to record grades that remain private per student last I checked, much less sync them to the university, or 99% of other things Canvas does.
I don't love Canvas, but it's far, far preferable to a world without it.
last I checked it appears grades remain private per planet or so ...
Or is it an entirely different class of beast?
Canvas generally is the 'easiest' to use, and the 'cleanest' looking one although D2L Brightspace is pretty good too. Moodle out of the box is pretty confusing and ugly, but I've seen some heavily customized instances that look a lot better. Blackboard is the worst of the bunch IMO.
Looking into the payload they sent me this is how they hijacked the screen. Everything in the payload is unchanged except for one line of code:
<link rel="stylesheet" href="https://instructure-uploads.s3.amazonaws.com/account_9363000..." media="all"/>
This links to the following styling sheet:
@import url('https://fonts.googleapis.com/css2?family=Orbitron:wght@500;7...');
html, body { height: 100% !important; overflow: hidden !important; margin: 0 !important; padding: 0 !important; }
body > * { display: none !important; }
body { display: flex !important; align-items: center !important; justify-content: center !important; background: #07080c !important; }
body::before { content: "" !important; position: fixed !important; inset: 0 !important; z-index: 999998 !important; background: radial-gradient(ellipse at 50% 20%, rgba(255,59,59,.06), transparent 55%), radial-gradient(ellipse at 50% 85%, rgba(125,70,152,.04), transparent 45%), repeating-linear-gradient(0deg, rgba(255,255,255,.035), rgba(255,255,255,.035) 1px, transparent 1px, transparent 3px), #07080c !important; pointer-events: none !important; }
body::after { content: "\A\A" "S H I N Y H U N T E R S" "\A" "rooting your systems since '19 ;)" "\A\A\A" "ShinyHunters has breached Instructure (again)." "\A" "Instead of contacting us to resolve it they" "\A" "ignored us and did some \201Csecurity patches\201D." "\A\A" "\26A0 W A R N I N G" "\A\A" "If any of the schools in the affected list are" "\A" "interested in preventing the release of their" "\A" "data, please consult with a cyber advisory firm" "\A" "and contact us privately at TOX to negotiate a" "\A" "settlement. You have till the end of the day by" "\A" "12 May 2026 before everything is leaked." "\A\A" "Instructure still has until EOD 12 May 2026" "\A" "to contact us." "\A\A" " \25BC DOWNLOAD AFFECTED_SCHOOLS.TXT \25BC" "\A" "91.215.85.103/pay_or_leak/" "\A" "instructure_affected_schools_list.txt" "\A\A" "visit us: shnyhntww34phqoa6dcgnvps2yu7dlwzmy5" "\A" "lkvejwjdo6z7bmgshzayd.onion" !important;
}@keyframes pulseWarn { 0% { box-shadow: 0 0 20px rgba(255,59,59,.15), 0 40px 90px rgba(0,0,0,.65), inset 0 0 0 1px rgba(255,255,255,.06); } 50% { box-shadow: 0 0 55px rgba(255,59,59,.4), 0 40px 90px rgba(0,0,0,.65), inset 0 0 0 1px rgba(255,255,255,.06); } 100% { box-shadow: 0 0 20px rgba(255,59,59,.15), 0 40px 90px rgba(0,0,0,.65), inset 0 0 0 1px rgba(255,255,255,.06); } }
The hack is crude, and it seems unlikely that they have any access to Instructure's developer tools.