ZH version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
73% Positive
Analyzed from 8141 words in the discussion.
Trending Topics
#more#better#brain#don#off#still#something#llm#loop#software
Discussion (139 Comments)Read Original on HackerNews
That _is_ valuable, and will remain so, but the temptation to turn it off is there because the loop is good and getting better at what it does. Once you're in the loop, with the rare high-value exception of catching total mistakes nad redirecting, you're mostly choosing between similar-yet-reasonable options. This isn't so much right-vs-wrong as relativistic optimization. Letting the loop do the work will get you a mediocre result quickly, and that's usually fine.
The most important time to use your brain is _before the prompt_. Once you engage with your LLM and agent, you start biasing yourself, and its reasonable suggestions constrain your visibility into other options, other worlds.
First, use your brain. Then write the prompt.
This isn't a new problem. I had it when working with outsource developers, too. In both cases, it seemed that just sitting back and describing what I wanted in plain English prevented me from properly wrapping my head around the problem. Which then meant I failed to develop the insight needed to find the really good improvements. Those usually consisted of identifying and removing the bits that were making things worse.[1] It's incredibly difficult for me to think of anything other than just adding more stuff on top of what's already there if I'm not getting my own hands dirty.
So I've been moving away from plan mode and letting the spin away on building large chunks, and back toward working in small increments and typing out the important code with my own hands. That seems to be the sweet spot. I'm still getting a productivity boost because there's more than enough boilerplate and trivial function definitions to farm out. (Easy prompt, too: "Implement `someFunctionStub`.") But I'm also not letting Plan Mode drag me back into the waterfall development pit of insanity.
1: https://en.wikipedia.org/wiki/No_Silver_Bullet
It's also unlikely that you're going to make those realizations, expressions, and decisions when your harness is asking you to choose between what _it_ identifies as being important decisions. If you're in the loop you're getting dumped on with lots of stuff that barely differentiates and you're biased away from catching things that matter.
If you're in the loop, use the loop. It's not a terrible construct. It will give you a mediocre outcome. And if mediocre is sufficient, you're done.
And here's the part that didn't make it into my prior comment -- if mediocre isn't sufficient, _stop the loop_. Play with what you've got. See how it behaves. Explore the edge cases yourself. Sketch how it works on paper, and how you want it to work, and think about how that should be constructed. Use your brain.
Then write a really long prompt (maybe referencing your files and notes and assets) to get back into the loop again.
I guess what I'm ultimately getting at is that the loop is useful _and_ a mind-killer, and using your mind is _also_ high value, so do both those things, but don't do them at the same time.
Your stance isn't new, if it's any comfort. I heard (and probably said) exactly the same thing when C compilers started getting good enough to eliminate the need for most assembly coding.
Does the prompter sit there with the llm all day answering the 1000s of questions necessary to make what he wanted, and the prompter then spent days testing it to make sure what he initially thought he wanted is actually what he wanted?
Then for every additional feature and integration the prompter will do this again?
Are we going to make a new job title for that? Should we call that job a programmer?
Tools are tools. Better tools let us do more faster, but they are still tools.
Even if I agree that you have to use your brain, when my boss is literally telling me to stop questioning whether AI automation is the right answer in team meetings/chats I'm just going to shut up and give them what they want.
Suppose I actually hire a professional to do it. I receive fewer of the benefits. However, he's much more likely to do a good job, and the legal system provides a variety of ways to hold him accountable for the work.
Suppose I hire a professional, but then insist that he use legions of guys from the park to do the actual work, since that's going to be some much faster and cheaper per quantum of work. I also tell him that if he's unwilling to do that, I'll find someone who will. I also tell him that, since he has so much more unoccupied time now that the original work is in the hands of all those other dudes, he gets to supervise still more work done by other swarms of guys from the park.
At this point it should be clear that I'm not interested in quality work, but in having someone poised to absorb blame. I want a patsy.
Now replace "guys at the park" with LLMs, and replace "free" with "nearly free, but with no warranty."
That means that they really don't care if, right there in the moment, just doing the job is faster and better. What they care about is that the actual work is being done by humans, and that's out of line with their grand vision of getting rid of humans.
Some would tell you that you need to find a different job with a better boss, but I'd say that this is quickly becoming something very hard to escape from. Investors want line to go up, and they all talk to each other about this "AI" thing that lets them cut staff and keep productivity, so naturally we're all getting it crammed down our throats.
It's have gotten to the point that I wanted to work on something manually, so I booked a room in the office. Just to not be at my desk, because I'm afraid my claude usage will be reflected in my performance review.
I spend a lot of time encouraging my manager and coworkers to ... write some actual english. To not just point an AI at a PR and call it a day. To actually collaborate before we jump into something - leaning on the knowledge we've built instead of being a cowboy and assuming you know it all (you don't, not yet).
In many ways it feels like we're back in some awful version of 90's based software dev. All these amazing tools have got us stuck deep in the shit swamp again.
Also: I would hate working with your team lead.
No? You solved the symptom not the issue?
I sometimes suggest timeboxing investigations for this reason. Spend 4h on this - if root cause is not found, just throw more hardware at it. Although I expect a writeup of what was investigated and ruled out, so that it can be used when someone returns to it.
This is rarely if ever done in a sensible way. Instead you find Managers, Engineers or PMs avoiding troublesome stakeholders, hallway convos or Slack DMs that are never properly written into public record in a searchable/readable way, last minute overrides from higher ups based on limited context of the problem space or their own unaligned career goals, poor recordkeeping, panic mode bandaid fixes when systems break, etc...
IMO a lot of the “meat proxy” and “loop obsession” behavior we’re seeing is simply people using the sense of short term productivity gains as self-soothing against anxiety from corporate dysfunction that they have no control over.
Then again, for many people their job security doesn't depend on what work they do, it depends on how well they are embedded in the organisation. Before AI you also already had plenty of people doing their job at a questionable level, yet never seem to get fired.
Sounds like a quiet few years and then effort once demanded. So I guess... being a meat proxy does work, after all?
> I started seeing people turn off their brain as they use LLMs
I mean, this was true for computers. "Computer says no"[0] was thing long, long before LLMs. I've had well educated, seemingly very intelligent people break when they couldn't log in steadfastly ignoring the [caps lock key is on] error message. This becomes even more "fun" and "interesting" when a person is being a meat proxy for some horrifically dangerous industrial process involving high pressures, excited chemicals, and enough rotational inertia to send god flying.
[0] https://en.wikipedia.org/wiki/Computer_says_no
(In my lifetime, I've even watched mass cellular communication move "replacing your own tire" from skilled to deskilled, since AAA is always one phone call away for most drivers).
If you want to shoehorn AI into this analogy at all then no one is even bothering going hiking at this point, let alone seeing anything worth capturing. It would just vomit up endless permutations of statistically average images that look like the kind of pictures people take of landscapes.
You can be creative and take the photo for your sake but at the same time accept that you can generate 1000s of them and many of the generated ones will be more liked by the general public than your creative one.
In the past few could afford to be photographers due to time and money. Many were not the most artistic or talented minds in the universe on the subject (you can’t find those until you give everyone a camera) but just people who enjoyed it and could make money off of it. Those who gained experience with the old tech were essentially AI models trained on statistically good probabilities of getting good shot. You had smaller selection and the model / brain quality determined what % of the best that photo was between 0 and 100% for any sunset. In modern days more people get a camera - more people with bad taste and more people with good taste. You also get more attempts at the shot - more photos taken, more tries. More tries means you are now preserving more of that moment from more angles and extending the time your taste people have to pick the best shot of all possible.
Your plethora of statistically average photos happen if you don’t have anyone filtering the ginormously increased output. If the problem is really valuable, someone will filter it. Possibly a model.
The real filter is value, not capability anymore. Do you have the empathy and taste to know what matters? The AI tokens/cost lets everyone make and become interested in making of someone hasn’t solved their problem. And then if search isn’t improved - everyone will just repeat the making of the same thing over and over. So if you don’t want repeated slop. fix search.
Or do you copy everything and everyone into making more of nothing.
I am so glad I am not you. What a miserable way to live your life.
Now if you know and analyze your photos in those terms you can get good faster. There is no shortcut to the value of quantity.
What we have never done here is define how much effort the "professional" has put into getting better. I've known a lot over the years that know the terms and have the "best" equipment, but they just take a measurement and put it into the tools without understanding the whole and so take really bad pictures. They consider composition as a checklist - which works okay (only okay) in a studio when it means put the chair here - but they can't handle a sunset where they can't control the scene.
Award winning photography means get those details perfect. If you have the details right a phone is often more than good enough to take the right shot - and if it can take more photos faster than is worth a lot more than the best equipment limited to one shot (or a few) and hope that you get hit the shutter at the right moment. If you can get the right moment the better equipment is better, but that is one of the least important parts of the shot.
It's not as if he was covering every combination possible of exposition, composition, depth of field, shutter speed etc. The tourist doesn't even know the existence of those settings.
Define wrong in this case.
I mean there are some obvious examples of a photo just being all black or all white. There are things so blurry that you can't identify the subject matter. But in between these extremes exist matters of preference and taste and everything goes subjective as hell. There are very few rights and wrongs here.
The credulous LLM users Dan Luu is complaining about don't take the equivalent of thousands of photos of the same sunset. They take a couple photos at most, say "LGTM", and move on to taking a photo of something else, even if their couple of photos of the sunset aren't great.
As someone who will often compare different things, I think doing that usually takes time. I think LLMs can help reduce the time (especially if it's programming, not necessarily so much in other areas), but I'm not seeing many people use LLMs to try a large number of independent approaches and compare them.
If that were the case then photo competitions would have been upended by now by tourists who accidentally took masterpieces with their phones.
I can take as many thousands of pictures as I want but, without taking the time to critically review them to see what works and what doesn't, I will rarely produce a prize-worthy picture. And then there's the risk that I'll discard it anyway because I don't have the eye to separate the prize-winning picture from the Instagram shot all my contacts are publishing.
I also think the analogy works if we're talking about average photographs, or even very good photographs, that are conventionally well-composed, balanced, etc, but lack the special spark that we associate with true art.
But if the photographer on the hill is a real artist, it's hard for me to imagine a current frontier LLM winning the contest. I suppose it may stumble upon a winning photo sooner than a roomful of monkeys would, but I believe the number of attempts would still be so vast that it would be more efficient for the human to make one single attempt at taking a photo, rather than dealing with prompts and sorting through the millions of AI-generated options.
EDIT: I also think it's very easy to underestimate the size of the nearly limitless set of possible "solutions" to the photograph, and also easy to overlook the constant feedback loop in the artist's mind as he considers each option and looks through the viewfinder, while the LLM is shooting blind the entire time.
Making things easy is good as long as it is not too easy, then it's Spotify muzak.
Of course "manual" programming can always be a hobby that provides emotional gratification, but that's not what dominates the economy.
Except it sometimes shows you a photo of something completely different than what you're looking at for or alters the landscape to "improve" it. Maybe that's not a good analogy.
That's the point of the analogy, not that people are passing it off "as if it's art."
It worked for a time, and manufactures employed experienced, legendary even, watchmakers and specialized "watch setters" to create and tune to the best possible extent a few experimental watch movements. They would be sold as collection pieces, the firm would gain boasting rights in its advertising, and the experience informed the design and tuning of more mainline movements.
By the end of the history of (the revival* of) this concours, the winner was always Tissot, a second-rate company. What hides behind Tissot is ETA, a huge watch movement manufacturer for all of the Swatch group (dozens of brands, from Omega to the worst no-name stuff you can find).
What they were doing to win was just to pick the best watch movements out of the millions of basic, bland, uninteresting "workhorse" ones they were producing each year. Just by chance, there was always one combination of the hundreds of components, each with their manufacturing tolerances and variations, that would fit all together to make an exceptionally accurate movement.
In the end, yes, Tissot won repeatedly. But the concours is dead. And Tissot is still a second-rate brand making second rate watches. Nobody is interested by their watch movements, no collector, no historian, no customer.
Incidentally, analog film photography is having a huge revival, and Instax is still Fujifilm's best seller. Maybe there is something missing from "I did an award-winning photo but I have have no clue how, lol".
Is there anybody here who wants to have their portrait taken by the "tourist"? Is there anyone here who would prompt an AI to generate their portrait, apart from Ghibli-style remixes (which are funny five minutes, and stale a month later)?
https://www.instax.com/printer/
Also the AI powered instax Up! app...
Have even more fun with the instax UP!™ app. With the introduction of AI, you can now scan an instax™ photo regardless of the background or angle, or even in your hand.
But a better analogy is how we hired a guy to take photos of us in front of the Eiffel tower on Christmas day. He came with a DSLR, directed the different poses, then sent us the photos later that day. Whole thing took less than 20 minutes.
It's the fact that the floor has been lowered so substantially.
Of course every modernized, automated craft ends up having its hobbyist revival. Whether it's analog photography, hand tool woodworking, knitting, assembly programming, or whatever. And that's lovely, but it doesn't mean much for economic trends.
If you are using these systems you still need to know what you want and how to specify it. You can use agents to automate much of the work but it can’t tell you that your ideas are bad, that there’s a better way to accomplish your goals; it can’t tell you what you want. And if you can’t understand what it is outputting you’re failing in your primary task as a developer: understanding the problem.
I agree with the title. I just don’t think these systems will ever be capable of replacing humans. As much as product managers and executives would love it and not have to deal with us; they’ll just be shooting themselves in the foot if they try to replace professional developers with AI harnesses and fire all the people with the institutional knowledge and skill.
There are still some things in an airplane that an autopilot cannot do. There is nothing on a train in normal operation (as opposed to maintenance) that a computer cannot do - as evidenced by multiple trains around the world running without humans operators since the 1990s!
There are no other jobs on the island that pay decently. Cutting edge engineering work will make you half of what it will in either the DC or Boston areas.
The conductors are highly paid in part because it's the local custom, and part because a third of the economy would collapse if that train stops running.
It would be better for society to pay these people to do nothing while we run more trains automatically.
All US passenger rail runs off of massive state subsidies. The absurd pay is the taxpayers money.
Meanwhile Boeing did enough stock buybacks from 2014 to 2019 to give every single boeing employee $200k... and when the 737 max crashes happened, the stock crashed back to pre buyback levels. THAT is truly sending money to the void to be lit on fire, money that could of been used to change the lives of thousands of employees and factory workers.
Or if we replace that well-compensated train conductor with automation I sure hope it's high quality stuff and not lowest-bidder slop.
It's going to make passengers' lives better by making their train rides more affordable, allowing them to improve their lives by leveraging those funds.
Or, it's going to make shareholders' lives better by increasing their profits.
Either way, some people will benefit from it. The question is -- what is the real, practical tradeoff, and is the tradeoff worth it to the participants in the system.
What, you don't want to save a few bucks on a train ticket? Come on, you'll love it!
New concept: the AI patsy
But wouldn’t it be way more economical to have some sort of AI-insurance service? i.e. protection against AI fucking things up?
I have limited tolerance to take demeaning workload that I know "the boss" could do himself. I think my pride and satisfaction is in my intellectual contribution, not my ability to relay messages nor do repetitive slop work.
But the real revulsion is being a scapegoat. I will not rubber stamp and take responsibility for things not under my control. I have limited capacity to review and sign off work I've delegated to someone else. It is usually easier for me to do it myself and know it inside-out than to delegate and then review enough to have the requisite confidence in the result.
When the new plan is to increase throughput via delegation to these generators, I can't possibly review to my standards. So, I refuse to accept any of it as my work.
It also freaks me out a bit to think of the collective mess that is developing across society, with so many people apparently ready and willing to shovel and rubber-stamp. I think the "move fast and break everything" cult is going too far.
But, primarily, I just need to protect my own sanity and live to my own standards. And hope that I can back away far enough to avoid some of the splash damage when entropy catches up...
Calling someone a "meat proxy" for being the human in the loop is wild.
Your underlying supposition is right, though. If you are truly thinking and in the loop, you are by definition not a meat proxy. I think those are the people who are successful with AI coding tools, but it takes discipline and it's still hard to find the joy and reward when you're not the one solving the problems.
[1] https://swiftrocks.com/using-ai-without-losing-critical-thin...
Even with small things, you need to be at the wheel. The other day I had to stop copilot from literally deleting the file I was working on. Why did it choose to go down that route? I don't know, but I nixed and reported the chat once it did that.
LLMs are dangerous and if for some reason you are stuck with using them, do NOT ever turn your brain off.
Ideally, you should not be using LLMs at all.
The problem is, that digital work, the new frontier of employment for a lot of epeople, got and will continue to get A LOT cheaper, faster and more accessable.
The PO who always complains about someone? They will just vibe stuff. It might even be shit but it will work good enough until LLMs/AI/AGI/ASI is good enough or better anyway than an avg developer.
Man i have seen so so much shitty code even in software companies.
You know when your HR software takes seconds to load? Yeah an LLM can do that today too
What's difficult, is I now spend most of my time on these high-throughput tasks, where my brain isn't exactly off, but focused on quantity over quality. It's often difficult to identify when a task requires "real" brain-power, and when it does, it adds a lot of friction to dig into the code and orient myself. So it's hard to know when it's the right time to do that.
The human's purpose is to keep an eye on the LLM, occasionally intervene/interfere with its work, and most importantly, incur social/legal liability for anything that the LLM does wrong.
Kind of like the backup drivers used by self-driving cars.
That sounds miserable.
Welcome to our future that was already here years ago.
https://en.wikipedia.org/wiki/Bullshit_Jobs
Fulfilling work disappeared a long time ago, the book bringing this up was from 2018 before chatbots were really a thing. We are just experiencing the acceleration to all jobs being bullshit jobs.
Developers like to "turn their brain off" during "super hard debuging" sessions. Mental flow state is a well documented as pleasurable experience. It is like a drug!
Auditing and reviewing autogenerated commits, is hard gruelling mental labour, compared to that. You do not even get a credit as an author! You can not get into groove to get your fix!
For example, the average driving commuter is not "zoning out" because of some bad experience with roads or cars, but because the route is extremely familiar.
Even at work, I can spend a week on developing something, implement it, and once it's done will have almost no memory of it a month later because it was just one more feature in a long series of features that are all pretty much the same.
My solution is saying “gnite” and coming back in a new terminal window with a clear restatement of my purpose. Or “what are you working on and why” can help.
Terminal windows are not great for keeping context; waiting for better design. It is possible.
But when the human suggests changing the course, the LLMs will often agree the new direction is better and start heading there...
Another problem is that when something breaks then I am very limited in my ability to reason through the breakage. It's necessary to go back to the LLM and ask it why it broke. At which point, I'm able to follow the reasoning — but not in any durable manner where I'll be able to remember it. To me, it's sort of like the difference between memorizing a passage and having a ready reference at hand at all time. We had this same problem with information before — now it's come for our reasoning.
Company is pleased! Cost savings!
Alas, the joy doesn't last long as pretty soon after the company realizes that after an LLM replaces almost all their employees, the much easier task of replacing the entire company begins, because the company itself has just become a much thinner meat proxy for getting an LLM to do whatever it is the customer relies on the company for.
Customer is pleased! Cost savings!
Alas, the joy doesn't last long as whatever savings they have from replacing the company with their own LLM loop is offset by whatever it is they do to earn income being similarly impacted.
All of the moats degrade together though they may fully give out at slightly different speeds.
If this is true, turning off the brain... is completely rational. Why burn out the brain for something happening either way? It's not like being fully engaged is going to result in anything that much better for the employee. In a world where code is this disposable, memorizing a codebase that won't recognizably exist in 3 years is also stupid.
In particular, tech is very open to people cheating and if billions of dollars could be had by somehow communicating the chess engine's work to a human at chess, the world of business would cheat at it.
Tech has a number of large startups that basically started with the assumption of "what if we assumed the law had no teeth?"
Clients will cut the middle man and run the LLM themselves.
Why do you think management is so gung-ho about LLMs? Employees are expensive, and laying them off is therefore extremely appealing.
The entire, explicitly stated goal of the AI movement is to produce an ASI, which is better across every metric than a human. Even if you refuse to turn off your brain, the economics of paying you to keep the lights on upstairs aren't going to work out.
This was never going to work out well for the human.
I'm not sure what else to say. You listened to these people speak, what the heck did you expect?
If you can do your job while watching TV your brain is turned off.
I was dreaming to be promoted to a cushy management role where I could talk bs for a living.
I managed to attain 'hands-on team lead' and later an ill-fated 'bootstrapped cryptocurrency CTO' role... Nobody wanted to pay me to talk as a career. Now they do!
So I'm very happy about that.
I feel contempt for all the people who had it easy and 'chose' to be engineers voluntarily.
he was writing in reaction to the McCarthy era which, arguably, is very similar to kind of bombastic nativism we're seeing now from our political elites. his argument essentially being that there are cultural and institutionalized forces that are driving people to, well, turn their brains off - specifically to no longer be skeptical of what they consume, to not want to refine and improve their toolbox of analysis, to not want to understand how things work for the sake of knowing
and that these acts were deliberate - a populace is so much easier to control when it's complacent and unknowing, who believes, unskeptically, that Oceania has always been at war with Eastasia, unquestioning about who was writing the PR missives that feed the news cycle (see also: https://www.commondreams.org/opinion/what-is-copaganda), how verbatim they are, and what the real intent of the authors were
like, I get that you don't want to put in effort for a job you don't like. but that effort also makes you better at your work. it makes it easier on you, it lets you create less incidents, it empowers you to be more agent in your own life, to, who knows, have the guts to start a consulting gig because your skill-sets actually are better than the average
there's so many benefits from just being more diligent, longterm, proactive benefits. but that's not everybody's extrinsic motivator and it's so much easier to just be Skinner-boxed by social media companies into doomscrolling, I suppose
Here's the thing I'm finding - LLMs allow me to do more coding when my brain isn't firing on all six cylinders. It means I get more code written because I don't have to be "in the zone" to exhaustively hunt down every piece of API and dependency and have my flow continuously broken up by missing information. The code written in that state is lower quality, but I can patch up five-cylinder-brain code faster than I can synthesize code from a blank page outside of my flow state.
Of course, this becomes a lot more difficult when you're unemployed.
Then there's the software that I write for a living, and almost my entire day is now spent making hard decisions. /grilling gives me difficult technical design decisions endlessly and I am forever doing product management on features from agents. My brain is back in CTO mode, which isn't what I'd planned for myself after I'd left that particular cursed path, but fine. I am doing the opposite of turning my brain off, because I no longer can just chill out and mindlessly write code.
Both these states can exist simultaneously. But my brain work has shifted into something different, while getting no less intense.
”Nowhere is this clearer than in entry-level software engineering—though the picture depends on whom you ask. Brynjolfsson discovered that employment among developers aged 22 to 25 has fallen nearly 20% from its late-2022 peak. The online job site Indeed also paints a stark picture: software development job postings have fallen 53% from the same starting point.”
https://insights.som.yale.edu/insights/the-real-job-destruct...
They aren’t firing the meat proxies to that degree yet, but they absolutely have stopped hiring the meat proxies. It’s going to be a very dirty year on Wallstreet when you’ll see massive sneaky layoffs of the existing meat proxies.
You can turn your brain off with these models. It is what it is.
We've seen layoffs as a result of this, and nobody's hiring a cohort of juniors when they're trying to afford the employees they already have.
On top of that, companies are firing the money cannon at AI initiatives and experiments (some of which will be successful and others which won't), leaving less money for humans.
Anyway, it's just a single occurrence of a hiring drop in a single economic cycle. Unfortunately, it just happens to perfectly line up with the rise of GenAI. We may never be able to tease apart cause and effect in this period, and we certainly won't be able to do so until we have at least a handful more years of data to see how junior hiring does or does not rebound.
[1]https://newsletter.pragmaticengineer.com/p/zirp
”Unfortunately, it just happens to perfectly line up with the rise of GenAI.”
Right. That’s just something I think we should keep our pulse on and keep looking at this data every six months (seriously, this needs to be tracked by devs because this data may start forming into that proverbial “writing on the wall”).
No one should be planning on being a software engineer in about 3-5 years, all need to plan for alternative income sources or early retirement. I think, at least.
Interesting. I recall Geoffrey Hinton saying something similar about Radiologists.
“People should stop training radiologists now” "it is just completely obvious that within 5 years deep learning will do better than radiologists”
https://youtu.be/2HMPRXstSvQ?si=prlQ_uGkCrWAZwgU&t=22