Human Bottlenecks

This was a fantastic article and beautifully captured something I had been circling but hadn't quite put into words:

> The notetaking people—and I say this with all the love in the world—are never, like, a researcher at the cutting edge of their field, building this vast cathedral of knowledge, note-by-note, so they can derive new insights. Never a historian who has to read tens of millions of words across thousands of sources to synthesize the life of some historical person. It’s never someone doing something hard. It’s always some blogger. Their “digital garden” is about how to keep a digital garden. It’s very solipsistic: there’s no output, no deliverables. The deliverable is you take a screenshot of your Obsidian graph and tweet about it to show off how much it looks like an incomprehensible ball of twine.

> Sometimes, tools don’t move the needle because there’s no needle to move.

It reminds me of something my old CS mentor, now elderly, had said about LLMs a few months ago: "it's a force multiplier, but there has to be some force to multiply."

This likewise is a basic fact I encounter over and over:

> Knowledge is another limiting factor. I find that even very educated people tend to underrate the importance of knowledge. A lot of people have this attitude that you can just Google everything just-in-time as it comes up. Like Babbage, I can’t rightly apprehend the confusion of ideas that would lead someone to think this.

As a product manager I have found people's understanding of what they want to be tenuous at best. They know they want it to be easier, less frustrating, and do the darn thing they want. But collecting the input of all the people with the power to make choices, boiling that all down, and actually figuring out what will work is a total bottle neck. Technology and programming have hardly ever actually been a blocker. Legacy systems, conflicting requirements, and ever shifting choices represent a much harder problem.

> Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something.

This was a good read but this felt like a personal attack :)

This is all very right, but I'd like to add this: As your capacity to deal with abstraction (which is a function of intelligence and executive function and, to some extent, knowledge) grows, you become more and more constrained by the extent to which you can manipulate symbols. AI-based solutions for that are potentially really powerful, and that's the mechanism through which, as TFA says, "intelligent, educated people with working reward circuitry stand to gain more from AI."

And I'd also add that AI strongly disaggregates the returns to different levels of the capability to deal with abstraction -- higher levels get more, lower levels get less -- rather than uniformly boosting returns across the board (unfortunately). Of course, this has been the trend of information technology since at least the '80s, but now the slice at the top is really small and the returns very high.

Heh, joke's on the author. I've built both a flashcard app and a notes app using AI, and I've learned more Polish in 6 months than other languages I've studied for years. Jury still out on the notes app.

But of course he's not really wrong. I've been a heavy user of Anki and a heavy reader of certain schools of academic literature on second language acquisition and knew exactly what I wanted and why and how it differed from existing tools.

The lesson I take is that you need a specific problem that you truly understand, whether it's your own problem or not.

This article is very well written and you can enjoy it regardless you agree or not.

It made me think, considering how much content is being written with AI, would an LLM ever write something like this?

For some reason they I can’t explain, the article smells human and I don’t use the word smell as a bad thing.

"Most people are not autodidacts because most people have no material reason to learn a specific topic (i.e. their job does not require it) and the problem with learning for the sake of learning is opportunity cost: there is no a priori reason to learn one thing over another, so better to do nothing and wait for something to appear which actually grabs your interest. Again, this is likely rational! Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something."

uhoh

i've been spotted

I've been trying to solve a lot of the issues brought up here, like personal automation, note taking, article summarizing/indexing, etc. What I've learned so far is that no level of AI-enabled automation will help me with my inherent ADD. All it means is that I have more started projects, but just as many finished ones. It has been a big enabler for things I'm already knowledgeable about, but the cognitive debt issue is real: A machine that thinks for me can't help me think better by itself.

Hm. Saying "AI cannot solve motivation" or "AI cannot solve judgment" may be too static. AI can change the conditions under which motivation or judgment appear. For many people, this is such the case.

Great read, I feel many of the topics discussed firsthand.

It gives me some relief to know there are others out there who struggle with some similar issues, but I was hoping the piece may offer some guidance, but sadly I do not feel it has.

As an aside, the term 'human bottleneck' resonated with me because I've been trying to describe situations that look like bottlenecks but are actually just "the process everything else is working towards".

I think using 'bottleneck' to describe a process that isn't amenable to automation frames the situation incorrectly in my head. 'positive bottleneck' isn't any help.

There are nuggets in this article that people enamored and, dare I say, crippled by their dependent use of AI and note-taking methods will never want to admit — or can’t — about themselves.

I completely agree with the author. As I've been saying repeatedly: AI is not magic. It's just another tool. An amazing search engine that simplifies a lot of mundane manual work to find what you're looking for. But of course, you have to have some idea of what you're looking for.

The missing part is usually the real workflow. If there is no output and no cost for being wrong, the AI tool just becomes another toy to maintain.

> Rather: if you don’t have the knowledge, you don’t understand the question, or why it matters, or how to judge the answers, and you won’t ever think to ask. You’re in a completely different continent from “writing the prompt”.

Reminds me of a time someone asked an influencer how to write better blog posts with LLMs. They responded "oh, it's easy", and then crafted a very specific and niche prompt about comparing and contrasting very specific things with technical details they clearly had deep knowledge in.

> Could you imagine if you found everything interesting? You’d spend years living in a basement curating a wiki of late Soviet military hardware or something.

Yes... imagine...

Tell me you've got little empathy for the autist mind without telling me...

Excellent read - thanks! Always felt there was a ‘self-licking ice cream cone’ at the heart of the present moment. If only AI had better context than the messy reality that is human knowledge…oh wait, we have AI to help with that problem

> Finally there’s the tools-for-thought/notetaking people. God save us. It’s always the same thing. Your folder with notes—pardon me, your “second brain”—plus an AI agent that writes, edits, synthesizes information, answers queries. You could build this in an afternoon, and it won’t move the needle in your life, for the same reason that building the second brain in the first place didn’t make a difference.

I laughed out loud when I read this paragraph. There are so many "consciousness" or "memory" or "learning" AI bullshit startups right now...all of them not understanding how positional encoding of LLMs work. It's getting so ridiculous that I don't even understand why they're in the (uncurated) news everywhere.

It's like everyone tries it out, and writes without any journalistic integrity because it's a sponsored article that costs 100$, and moves on. Spamming as a bought in service or something. It doesn't make sense to me, and I have no clue how broken the economics of this must be to get into the state of "AI news" we are in right now. Excuse my French, but something must be utterly broken.

Eventually AI will be good enough that whatever is inside a human brain isn't going to matter much. It's like how the most precise craftsmanship of swiss watches didn't matter because quartz watches were far more accurate. Once a new paradigm jumps the curve, the bottleneck of the old paradigm can just be circumnavigated entirely. It wouldn't make any sense to wait for technology to improve such that gear-based watched could get 10x better, the other tech is just better by nature.

AI is just better in a fundamental way vs human intelligence. It can be reproduced infinitely, has perfect logging, no mental illnesses/hangups over certain things, far faster, able to ingest more kinds of data, etc. The only limitation now is a lack of intuitive in-context learning during test time, but once that final bottleneck falls then humans will have nothing valuable comparatively.