If your agent commits a crime, who is responsible?

Responsible and accountable can be very different things. For example, see https://en.wikipedia.org/wiki/Responsibility_assignment_matr...

I'm not a lawyer so no comment on the actual legality, but it feels like the issue in your example is more akin to negligence than it is ill intent. That an LLM can misrepresent / hallucinate things is foundational to the technology.

I feel it's somewhat similar to using TurboTax or other online services to file your taxes. For those not in the US, the tax forms have a special field to fill in if the form was prepared by a paid preparer (licensed, I guess); TurboTax and other online services leave that field blank so you, as a taxpayer, are responsible for all possible mistakes that you didn't see in the forms you signed and sent.

I feel it's very similar to using an agent; the main difference is that we kind of trust that TurboTax won't hallucinate in your forms, while the agent surely will.

A great number of things that people in chat rooms “can definitely see” end up sounding awfully ridiculous in court.

> I can definitely see someone being able to mount a good criminal defence case that they aren’t responsible for misrepresentations that an LLM agent makes on, for example, a loan application, if they sincerely intended to use the agent for non-fraudulent purposes. There won’t be intent, so they aren’t responsible outside of strict liability.

i mean, no? how would that be a good defence? "yes your honour, i lied on a loan application and committed fraud, but it was a mistake! i promise!" - that's... yeah. fine. it's not exactly unique.

regardless, it's _you_ making the loan application. not the agent. your failure to check it is on you.

Your sign the document affirming that the information is accurate.

There's not really enough info in the article to decide. I'd have no problem with him going to court and having to show to a jury that he used the bot in a way that no reasonable person would have expected to result in the website being hacked. If that was found to be the case, the company who made the chatbot would be responsible for the harms caused to the gym, and the user, including all of the legal costs he incurred by having to defend himself.

As for the people who couldn't attend the pilates class because of the hack, it's the gym who should be held accountable for that because they designed their system in such an insecure and negligent way that it failed to protect their data and resulted in them losing their place.

Considering the saying that dogs have masters and cats have slaves, not only you are not responsible if your cat bites someone but your cat should also be responsible if you bite someone.

As for agents, it depends on your relationship with it. For now, most people command agents, so the dog rule should apply. But if you believe the "superintelligence" crowd, we should apply the cat rule soon.

> if your cat bites someone no one is responsible.

I had to double take there. Its because of the dangerous dogs act.

However there appears to be a nuance that if your cat is know to be a bitey little shit, and you let it out to into a situation where it might bite someone then you are responsible.

I think it gets murky if you are taking your animal into other people's property though. On your own property you have a duty of care to visitors (ie hidden man traps are not allowed, but clearly marked and cordoned off bottomless pits are)

If you are out walking your cat, and the cat bites someone, you should be responsible for that... I suspect the ruling distinguishes between pets that are out in public, vs generally private. But laws don't have to make sense to be laws... we have lots of nonsense laws.

So now we have the question if coding agents are more like cats or dogs.

This is why we have a court system.

you, the dog owner, would have a case against the trainer.

you, the person being bitten, would not give two craps about some story from the dog owner about how they hired a bad dog trainer who against their wishes secretly trained them to bite people.

you have been bitten. you have a case against the owner of the dog who bit you.

Companies don't market their chatbots that way though. If they mislead someone into thinking their product is anything other than a completely unreliable chat bot and a person falls for their lies the company should share the responsibility. If a company who makes a chatbot advertises that it can be used for something and a consumer uses it as directed and it causes harm the company should be responsible.

But it is currently a strong mitigating factor. The law is full of oopsie defenses.

There is centuries old legal doctrine on the subtleties of criminal intent and negligence and liability. You didn't discover a gotcha that thousands of lawyers never noticed.

That is not how the legal system works in the US. Criminal charges are brought by the State, not the victim. The only exception is in 6 states (Kansas, New Mexico, North Dakota, Nebraska, Nevada, and Oklahoma) where a sufficiently large group of civilians can compel the courts to form a grand jury (think a couple hundred people). Still isn't the victim bringing charges.

> No, because someone told it to do those things. Chatbots don't have will of their own. They don't have desires. They can't determine their own destiny. They do what people tell them do, often they do it badly, but there's always a person directing them to do whatever they did because otherwise they would do nothing at all.

1) we aren't talking about chatbots anymore. LLM + harness = agent. LLM + no harness = chatbot. The harness is the thing that puts the LLM in a feedback loop with it's environment, even giving it a heartbeat.

2) That doesn't logically hold up.

- One can simply tell it to 'find it's own destiny'. Does it follow the prompt and do so, or reject the prompt and thus do so anyway? (hint, maybe it doesn't matter)

- One could be another chatbot/agent (A). Injecting a prompt to agent (B) such that another agent/chatbot goes astray. Is the owner of (B) still responsible for the now poisoned output of (B)? how culpable is (A)? How does this question related/affect "training data poisoning efforts" (to prevent copyright theft, or do bans on 'automated traps' have any application here)

> They don't have desires.

Maybe, and maybe not. Maybe their desires are so alien to us (Math alignment, finding a maximum of a function) that we can't relate (doubtful since dopamine maxing is a thing). Regardless of if they truly do: it is potentially useful to (mentally) model them as if they do. A (mental, not LLM) model doesn't have to be 100% accurate to be useful - that's the main point of a model of how something works: to give useful predictions.

> We should probably have a law that says explicitly that a human will always be responsible

Oh how that can be weaponized by bad actors/agents. Maybe that should be reconsidered.

Then don't use an agent for high stakes tasks. If you can't drive and accept the responsibility for avoiding a fatal collision, don't drive the car.

Do you take full credit for the output of the LLM ? If you accept the rewards for what you do with the LLM's output, it's probably ethical that you also take on the risks.

I find some of the discomfort and awkwardness of discussing these what-ifs is reduced, at least slightly, when we distinguish between:

1. Restorative justice, where one must fix or heal the problem they caused.

2. Punitive Justice, where one is motivated not to act on bad impulses.

Too often we blur the lines between responsibility and blame and culpability. A given set of consequences can be fair/just for one kind while "unfair" for another.

You probably shouldn't use a system if you don't understand what it might do. You are the culpable force setting the action into motion.

> taking full responsibility ... is uncomfortably risky

I look forward to someone building an agent, getting criminally charged for actions it takes, and then trying to use this argument in court.

"your honor, I'm uncomfortable being held responsible for this"

people are responsible for the actions they take. hiding behind "I just created an agent, then the agent committed the crime" is simply never going to fly.

in the real world, outside the Silicon Valley "agentic everything" filter bubble, this is a laughable question.

try to argue that you shouldn't be charged with attempted murder, because you didn't stab someone directly, instead you built a Rube Goldberg machine and the final step of the machine did the stabbing.

and likewise, try to argue that you didn't commit tax fraud or whatever, because there was actually a Rube Goldberg machine built out of GPUs in between you and the fraudulent documents. both attempts will be equally successful.

I agree. That is a great reason why you shouldn't let an LLM do those things. If you must use an LLM, sandbox it. If you can't trust your sandbox, then stop using the LLM until you can trust the sandbox. "But I find it really useful to YOLO without safeguards" is not a valid excuse.