DE version is available. Content is displayed in original English for accuracy.
We thought: what if lecture videos were written as code and compiled into video using computer graphics and TTS?
Then LLMs could write them, and lectures could be fixed with code edits instead of video production.
On correctness: LLMs may make mistakes. But these videos are code sitting in our repository, and we can maintain them. Every report and review becomes a fix in the source, and everyone who watches after that gets the corrected lecture. Every recorded lecture on the internet is stuck with its mistakes. Ours will continuously improve.
Each video also comes with an AI chat that understands everything said and shown in the lecture.
There is a longer write-up at the bottom of the page.
Happy to answer any questions.
Discussion (23 Comments)Read Original on HackerNews
I'm not sure listening to a robot reading me a transcript and chatting with it really gives me those same advantages. What's the upside to this versus just getting slides and/or transcript w/ sources and feeding it to my own LLM?
I've rarely, if ever, seen this. And if I had, I think most of the class would have just been annoyed, which you probably didn't notice.
Also, you’re not chatting with the transcript; you’re chatting with the video. The AI has much more than the transcript in its context. See this, for example: https://academa.ai/lectures/diffusion-models-learning-to-den...
And in the coming months, we’ll make the answers themselves real-time videos, which is kind of trivial for us at this point. We’ll ask the LLM to answer by writing code, and our software will render that code and show you the response directly as a video.
It suddenly starts talking about URLs, but it never really sets up the idea that we're using some kind of URL lookup service as an example. The way it introduces the memory limit feels similarly strange. At some point, it just says that we have 8GB of memory, but it does that halfway through the explanation, not as part of the setup of the problem we're actually solving.
The lecture description language isn’t public at the moment. It’s a core part of the technology we’re building the company around.
Also, if you’d like, you can share the video as a public link by clicking the share button!
I'm curious, what are the economics of producing this longer form content?
Actually, the economics are quite good.
We’re not using video generation models. We ask an LLM to write the lecture as code, then render it deterministically with computer graphics and TTS.
So the main costs are the LLM call, TTS, and some cloud GPU time for rendering. That ends up being much cheaper than generating long form video directly with video models.
Models change. Models are unpredictable. The assumption that we can constrain them between prompt bumpers has not been proven and seems unlikely to be provable given how these model works.
Beyond that, how do you know if the reports and reviews are even correct? Are you just gonna rely on the kindness of others to provide that value? Of all things that LLM’s have killed, I’m pretty sure the open sharing of knowledge and understanding are the first on the chopping block. I don’t care whether or not you’re pro or anti LLM, publishing knowledge for reputation is now a dead end for anyone wanting to make a living.
Wrapping others’ knowledge in a black box that may or may not accurately represent that knowledge is basically enshitification on steroids. The reason we hold folks like Lovelace, Sagan, or Feynman in high regard is because they weren’t wrong when they shared their understanding and knowledge.
But the best of luck to everyone involved! I’m sure everything will work out!
You wrote it well. This aspect irks me, too.
Without our system, there isn’t really an equivalent target language you can ask the LLM to write. Manim exists, but you won’t get videos like these by simply asking an LLM to generate Manim code.
The idea of using LLMs to write out a script and storyboard for the video is interesting, but I think it needs intermediary work to better instruct the speech and graphics on how to perform.
The future of humanity is dire if this is what's round the corner.
Thanks for making everything worse, you soulless bastards.
The one that asks questions at the beginning is Gemini Flash 3.7.