RU version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
69% Positive
Analyzed from 4343 words in the discussion.
Trending Topics
#code#llm#more#don#tests#llms#something#things#using#thing
Discussion (86 Comments)Read Original on HackerNews
I am happy about all the little side-projects, and ideas it help my realize, and I enjoy exploring this new world, but I've noticed LLMs feed my unhealthy "don't want to take a break and waste time being idle" mindset, and I need to correct it.
W.r.t. article's main complain - I think the similar thing happened due to factory manufacturing automation. What used to be a varied skillful craft in a shop became standing in a single place of an assembly line doing the exact same thing whole day. LLM took away the more creative and variable part of the work, and left the repetitive QA rubber-stamping. Probably some of the mitigations used back then could be rediscovered today.
This 1000 times. The number of times I now have to use my brain to do something engineer-y is now something like 5 times per week, usually eyeballing some architectural decision from (as you mentioned above) someone else's LLM.
I really loved writing code, and I loved that I could do it for a living. Obviously, nothing is stopping me from continuing to write code and solve problems with my own brain, but that's absolutely not the efficient way to work anymore.
The most high-leverage activity I've done in the last 6 months is a build-out of internal AI orchestration platforms and data access layers so every employee in the company can built with AI against our own data (particularly non-engineers). It has unlocked so much for everyone. Yet, my only real claim is a) coming up with the idea and general architecture and b) ensuring smooth roll-outs and hosting "office hours" for months to help new employees onboard to these new AI tools I built.
High-level design and architecture decisions, quality assurance, and balancing decision-making through subjective areas like UI/UX. That's mostly all I do. Now that I'm leading engineering, I just think even higher-level while the engineers use orchestrated AI do built vast swaths of code. Then I frantically bounce around approving sensitive PRs and architecture decisions -- most of which were written by Claude, described by Claude, and usually don't need a lot of correction.
Spoken like someone who is not at an org/team that has undergone layoffs and reduced hiring in the last 3 years.
You might be in the minority there - especially when it comes to those who are facing burnout.
I confess that the above variant on the quotation is how I originally read it. And that's just about how I feel now with trying to sort through vibe-coded slop projects that are put forth by (well-meaning, probably good intentioned, not evil) people who represent them as if they're the handcrafted result of one dedicated developer.
"I did a Chat output, please fix and review it " is the kind of thing that empowers the people who used to have a minimal productivity, and now lets them to wreck things on an industrial scale.
It's not. There is no one person that has universally good taste. Also, we're not in your head, no matter how much better of a coder or whatever. We're not in your head and it's all terribly painful to navigate.
AI is not a productivity multiplier. There are diminishing results.
The ones that notice the highest increases of productivity are usually the ones that were unproductive at best and dangerously incompetent at worst.
Lots of companies (nearly all, I’d wager) of any size were leaving bare-minimum a 2x software development speed increase on the table before LLMs, having nothing whatsoever to do with how fast anyone was typing or thinking up code, and everything to do with how they organized and supported development work, and with your basic ordinary corporate dysfunction.
My company, I’d say it was more like 4x or 5x they could have achieved before LLMs, by fixing processes and reducing how often management steps on their own dicks.
All the people I’m seeing with crazy-high LLM productivity at my company? They’ve been given enormous autonomy to basically go do WTF ever they want, and people are jumping to get them anything they need (and most of what they’re doing is prototyping, for that matter). So right off the bat, if they’re competent, they should see a notable multiplier on productivity even if they weren’t using LLMs. Not that those aren’t helping, too, but if you don’t change processes they’re not all that effective, because the problem wasn’t speed of code-writing (and if you can change processes, you already could have sped up development a lot before LLMs…)
I got into programming because the problems of programming were interesting to me. But if the problems go from "figure out why this calculator is off by one in France" to "Get this LLM to stop spamming cutsey emojis", then maybe it's time for a career change.
It'd be rather beautiful if all jobs were purely passion driven, but that is simply not the case. Nor could it be. And yeah, there are programmers with jobs that are mostly driven by passion, but most would pack it up and go home immediately if there was a sudden "we have stopped paying you" announcement.
I'm not looking forward to using computers or technology over the next decade. There is a non-zero chance myself or a loved one is killed because of vibe coding.
My latest is, I'm really into fizzy/soda water and wanted my own continuous carbonator. My entire build from water source to tap with an ESP32 controlled pump, pressure, water level, cooling fans.
There were so many areas I made mistakes in my shopping cart and it found it - like Home Brewer likes 8mm lines but water filter systems like 9.5mm. Really optimized the versions from a simple on/off pump w/ float switch to effectively a full on PLC system. So many iterations gained by chatting with "someone more experienced". Once I get the parts I can build and have the software side running in less than an hour.
It doesn't make money, but man I really enjoy it.
Sorry, some of us have a joy for programming where the how is just as important, if not more so, than the what and the why. No matter how much people proclaim that the how doesn't matter to them, it isn't going to suddenly make it true for others.
We’ve had 3 production incidents this week that slipped past CI because there’s a whole team that is just shoving out PRs without understanding what’s going out.
The experience is much closer to working with an external API that you don't have control over and which simply doesn't do what the documentation says. Those have always been the most frustrating parts of programming, but at least previously you could reverse engineer the actual implementation to work around bugs. You can't even do that now because the "boundary" randomly change every day.
It's impossible to undo some of these linguistic wobbles. Even if you could filter out 100% of LLM input, the humans themselves are learning to say "land" at a higher frequency now.
Prior to the last 12mos AI companies were hell bent on squeezing out the best results from mediocre models.
But... now that the top models have progressed, those same AI companies have switched their efforts into reducing the computation (cost of a producing a result) as much as possible without being too obvious.
What was an exponential slope in the quality of results over the last 36 months has now nearly flat lined.
This is what you want. You want comprehensive tests at every level, far more than is reasonable for a human to build or maintain, from unit, functional, to full end to end and beyond. Adversarial testing (both TDD-style "write tests to demonstrate this bug", and posthoc "prove this patch wrong with a new test") is the best way to keep AI on track and make those diffs you have to read clean and easy.
An even better way is to use a more strongly typed language and really lock it down, but you can use testing in any language. I feel like my background in TDD and "TATFT" has been secret sauce when working with AI
Yes tests are conceptually isolated and that helps, but I've personally seen unit tests get generated that are semantically incorrect - that is, they test the structure of the code (e.g. they can check function output types and values), but they can't know _why_ the unit tests need to be there, so the really really helpful tests never get generated. Not to mention the obvious issues with generated tests only testing is x = x, or needless redundant tests for the same thing, or them essentially testing basic features of the language.
I actually have a public (AGPL) example here: https://github.com/pgdogdev/pgdog/tree/main/integration/sql - pgdog is particularly testable since it is trying for complete transparency, so you have a perfect oracle in hand via base postgresql, but it demonstrates the concept at least.
I've been burned by this in my honeymoon period with unit testing (pretty much the reason it ended). These days, I prefer broader scope of testing, especially user-facing part. The users may be other developers or end users. I only do unit testing for tricky algorithms or math formulae.
https://github.com/dprkh/eventfs
It has good test coverage, mostly unit tests but also a number of end-to-end tests. I also made the LLM build a benchmark, which you can find at the bottom of the readme. It is obviously slow, but I thought that it is good enough to work. When I tried to write a 1 GiB file, I found that it broke down, and after writing half the file, the speed went to under one megabyte per second. Implementation is 10k+ LoC, and I have no idea what is going on there.
At least with agent-run tests I care about loop speed a lot, but I care about complete coverage more, so having the odd heavy weight full stack integration test is fine, I think.
It is different though. Basically a lot of what I do has changed over the last 2 years. I totally get that a lot of people won't want to adapt though.
Or people don't want to be reverse centaur keeping the clankers happily running. Instead of helping to solve users/consumers problem.
I save myself by skimming things like tests, templates, some UI. Anything cosmetic. But I have to read the majority of code that ends up on my back end systems.
In my personal experience, the ones most enthusiastic about LLM magic are those that can't code, but can now walk away with something functional if not quite the best code. Now that they can produce workable code, it will make everyone better. Yet, they have no idea how maintainable the slop is or if it's slop at all.
When you see a perfectly clear function or object that just isn't your style, you have to accept it and move on. Where there are concrete concerns, or it's unreadable, demand excellence, but treat it like a coworker, not an IDE.
Review code line by line is like watch movies frame by frame, and is impossible, very difficult, terribly boring, or abandoned sooner or later.
Just because we work with computers doesn't mean we don't take, er, social-damage. Or perhaps parasocial damage, in this case.
As a long-time engineering manager, PM and, eventually, product owner my response is, "Congrats! You've just been promoted to management." :-)
As a new manager, your first challenge will be successfully delivering commercial results using only a team of 'differently abled' new grad interns. Don't complain, new managers don't get to pick their first team! To be honest, these guys are more like alien brains raised in a vat with no direct senses. They've only ever experienced a data feed of the internet and, oh yeah, they get near-total amnesia a few times a day (but maybe you can teach them to write notes for themselves). They also have ADHD and are somewhere on the spectrum. But don't worry because what they lack in common sense, experience and intuition is offset by having a sort-of photographic memory and a willingness to grind on a problem 24/7. You should be fine. Good luck, we're all counting you...
The productivity drive and the sheer feature set you can generate in record time makes it easy to forget proper sdlc hygiene.
Anyone else working on something like this or know of any projects attempting it?
Directionally if what you're doing is straightforward it's an amazing experience to be able to slap in an epic planning document and wake up the next day to it being "done", with a big asterisk that done-ness is directly proportional to how good of a spec and how good of a model you were using.
That being said, these days if you use Fable, slap in an epic planning document, and ask it to run a workflow (be sure to specify that subagents should use, say, Sonnet, or wave goodbye to your wallet), it's almost as good as gastown/gascity but far more predictable.
I do not understand these complaints. Yes, those are the defaults and they're annoying, although the general public seems to like them. But you are not stuck with these. You can just tell the LLM how it should interact with you. If you're using any sort of harness beyond the chat window in a web browser, you can codify these instructions in a rules.md file or similar and have it automatically included in any new chat. It's not any harder than changing the default wallpaper or color scheme on your desktop operating system.
In reverse order, you can just tell the LLM to never use emojis. I don't like emphatic staccato fragments either, so I tell it to eschew the language of marketing and hype and stick to a factual and plain language, or to employ an academic tone. I explicitly instruct mine to ask clarifying questions whenever context is ambiguous and to push back on false assumptions or common misconceptions (by me). Hallucinationsa re the biggest problem of those you mention; it's not easy to totally eliminate them (for the same reason it's not easy to instruct people to not fall for scams or disinformation), but you can considerably reduce them by setting standards for citations.
I have ideas about reducing hallucinations over work material (ie a codebase) but am omitting them here as they are not fully thought out or tested.
My mind still can't function well without having knowledge about everything.
Getting sent IM responses that are copy pasted LLM nonsense. Getting a massive PR to review that was generated overnight and the author didn't read it first.
Of course if you're supposed to achieve so much output that it's not possible to do anything but vibe it, fair enough.
What helped was a sleep and work system, oriented around being offline that was inspired by nature and from my earlier days in working in tech while car camping across the national parks.
Basically: the sun wins in terms of how all energy on the earth is structured, and expressed. All manners of cycles of organisms and living systems are in relation to its rise and fall, and even its particular color spectrum phases (whether thats night oriented or day). I call this our real circadian rhythm; it's used to being signaled by the light of the sun and maybe fire for millions of years and it isn't until recent centuries when we started tricking our biology with LEDs and lights. So the solution is simple. Orient yourself around the light of the sun and make sure it's the first and last major light source you see; blue limiting is the most important part BEFORE sunrise and AFTER CUT OFF ALL BLUE LIGHT. On my Mac I use a red light filter (using it now, it's 11:07pm ET and the sun went down about 2.5 hours ago). It's really hard to stay alert and chatting with an LLM when the only light sources are red and you keep them dim at that. Our ancestors would rest when the sun's at its peak (~1:05 pm today) and that's a good time to divide my own day productively as well. With intentional breaks diving the middle of the day with sunlight anchoring it, my nervous system is more relaxed, and by the evening time, it's also ready to transition out of anything blue-light assisted and most intellectual work and problem solving falls into this bucket. It's really hard to explain but it really works so simply. To enjoy the process a little more I made this fun sun clock, check it out at https://sunsignal.app
https://github.com/JuliusBrussee/caveman
It's for getting it to output shorter answers, but also could help with your burnout.