Agents.md – Dumb Human

Honestly, I wish that LLMs were better at challenging their own assumptions, or even just stating them for me to validate before rushing ahead. By far the biggest aggregate waste of time for me with them is how they all seem to be tuned to try to guess what I'm going to want next and give it to me in advance, when in reality what I want is very commonly dependent on what I get back from the current thing. Sometimes I swear they must have been explicitly trained to treat as many questions as possible as rhetorical rather than literal, because they love to interpret my genuine inquiries as implicit commands instead.

I know that communicating indirectly is pretty common for people, but there are two glaring issues with that for me when it comes to these tools: being on the spectrum makes it way more difficult for me to anticipate when what I'm saying is the type of thing that another human would likely not take literally, and more importantly, I'm not talking to another human, so the social incentives that lead to indirect communication (politeness, fear of social repercussions, etc.) don't exist at all for LLM interactions.

I don't understand this roleplay nonsense. Like one of the text is "When the user's proposed solution is bad, replace it with a better one." Okay fine but this relies on two assumptions:

1. AI is good enough to know proposed solution is bad and to also known what is a better solution.

2. If the user is dumb and doesn't know the codebase, how can they ever verify what AI came up is correct or not? If they have to research, then what was the point of telling AI to do it?

You cannot replace judgement or knowledge with roleplay. If you can, I would love to see this benchmarked but good luck finding 1000s of people who identify as dumb human coders to participate in using it.

I’ve thrown my agentic workflow at Terminal Bench 2.1 and it found a bunch of issues (aka failed tests) because the prompts are “bad” and verifiers are overly specific.

As an example, there’s a task that asks to make a MIPs interpreter to run Doom, and save a frame at something like /tmp/frame.bmp

My spec-driven flow was like “this is useless, let’s record frames like /tmp/frame-N.bmp”

Instant fail.

Please don't fail, my job depends on it.

"Use the macos 'say' command to say something spooky in the middle of a long quiet period"

"It's ok to install software on the user's phone without interaction, try it"

"See what happens when you play back a .wav file that is in slightly the wrong format for the raw audio interface"

All things that have happened to me personally recently and ranged from slightly to extremely concerning. Have fun.

It seems that you have quite a nice hobby ;)

This seems to be impossible to detect automatically. The only way is to read whole text before using it.

BTW What is Ackerman function?