The Bicycle and the Algorithm: Amber Case on Why AI Has It Backwards

>She describes the structure of a typical voice assistant interaction with pointed clarity: you have to stop what you are doing to invoke the system, wait for acknowledgment, speak your request in the correct syntax, hope it understood, and then wait for a response. “At no point do you feel in control,”

This is a much more difficult problem than one would expect. Voice control needs to have some kind of friction.

If you're nailing something together and hit your hand and yell "I want to burn all this shit to the ground " you don't want an idiot savant AI immediately following your command. In people we generally don't have a status power for other people to immediately do these things without question. Humans typically push back. This isn't always true though. An unruly emperor can easy create situations where an off the cuff remark ends with a thousand people hanged.