FR version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
55% Positive
Analyzed from 10288 words in the discussion.
Trending Topics
#don#more#those#state#technology#power#used#ethics#police#https
Discussion (258 Comments)Read Original on HackerNews
[1] https://gdcvault.com/play/1012227/Development-Telemetry-in-V...
[2] https://www.army.mil/article/85934/solution_to_weapon_develo...
The same code was used in another project of this same company. The project was a missile. It felt weird.
Eg. Flock cameras don’t help the military but do help the police state.
I would expect any invading forces would need to be surveilled by planes or satellites, or perhaps drones, since they would immediately destroy something like flock cameras.
https://www.brennancenter.org/our-work/analysis-opinion/cour...
It's the last step that's the problem, not the people who built all the things they stand on. Otherwise we'd be suggesting that no one build a database engine or a digital camera or a computer ever again in case they're used to build evil tech.
This isn't what the article is talking about at all. It never says don't build military technology. It says don't build things like a database of who in the country is and isn't gay and what religion everyone practices, for fear a future bigoted administration might use it to target those people. If your government is turning howitzers on you, that's less a police state and more a civil war.
I don't know what you do against that, but it's a different situation. There is precedent in the US with the Army being used to round up and forcibly displace native populations, but that really seems to be the kind of thing that requires a political solution, not prevention of any possibility. If we didn't have an Army at all, then secession would have succeeded, there would still be chattel slavery, and one of the hostile European colonial powers likely would have simply conquered the west itself and killed all the natives anyway.
But maybe easier than it sounds.
All technology can be weaponized. It can also be de-weaponized.
One of my favorite late-night thought experiments, is "what would it take to turn an AK47 or an AR15 into a water pump?" .. the possibilities are endless. "How could a tank be turned into a bicycle factory?", another great one worth the effort to think about.
The only difference between peaceful technology and a weapon is the intention of the user. The very real spiritual intention.
De-weaponization starts with the human being. Those who think weapons are necessary must always take lesser priority than those who invent technology to make the lives of their fellow human beings better. There are so many examples of the tool-makers winning harder and at greater scale than the bomb-manufacturer. We wouldn't be here now if the balance was the other way - our world is however threatened by having this scale weighed heavily, by those who think murder is the solution to anything.
It's a spirit thing. Not spiritu-al, a spirit thing. What is the spirit of the designer? If they want to help humanity, it will be difficult technology to weaponize (although not impossible). It will be easy to de-weaponize, also.
We need to focus more on the spirit of help, and less on hurt.
Imagine if those F35 mounting pylons were used to drop field-portable water-purification systems directly where they are needed most.
Imagine if they could be used to drop a copy of archive.org, anywhere in the world, in any language.
Imagine if those ships were being used to bring power to the villages that need it most. Instead of glide bombs, we have glide libraries. Instead of cluster munitions, we instead drop cluster medical kits!
Oh, the glory to be had by those who would do this - the allies in the poor and weak of the world to be found! Imagine this happening, changing the intention from hate and fear, to respect and love.
All it takes is the spirit, a bit of imagination, and any and all technology can be de-weaponized.
It is the warrior narcissist who stands in the way of this. Disarm them too - smash their mirror: Drop books, not bombs!
Except we can (and do!) accomplish that with much simpler airplanes. The staggering costs and complexities of an F35 (the thrust:weight, maneuverability, and stealth) don't make sense for any "peaceful" application.
And then a group that prioritizes weapons over human-life-improving-tools kills the group that prioritizes life and takes your resources.
This is like the larger group version of suicide by cop.
We've been playing this game for as long as human beings have existed.
>And then a group that prioritizes weapons over human-life-improving-tools kills the group that prioritizes life and takes your resources
Only for as long as those who make peace, don't pay attention to the prioritization of weapons - and that group of warrior narcissists whose identity is more important than human life must never be allowed to be bigger than the group who make peace - as is happening today at a pace far too great - because otherwise, we all die.
>We've been playing this game for as long as human beings have existed.
Yes, humans have been making peace at far greater scale than those limited few who make war. You just don't see them because a) you have your own warrior narcissist mirror obscuring your view, and b) those people are not narcissists, and are living human lives instead of primping their warrior narcissist identities...
Good luck - this flies in the face of the entirety of human history.
> "what would it take to turn an AK47 or an AR15 into a water pump?"
As someone who has manufactured these from billet: why would you do this? (hence "word salad") Why does one thing at all relate to the other? Why not just manufacture a pump from the same raw resources? Why not both?
> Those who think weapons are necessary must always take lesser priority than those who invent technology to make the lives of their fellow human beings better.
Did the Allies in WWII not "make the lives of their fellow human beings better" using violence? Why didn't they just ask mean old Hitler: "Hey, be nice! Have a good spirit!"? The world has finite resources, inherently making anything often denies those resources (your time, actual materials, etc.) to other causes. "Better" is subjective.
> There are so many examples of the tool-makers winning harder and at greater scale than the bomb-manufacturer.
These organizations are often one in the same.
> We wouldn't be here now if the balance was the other way
The balance is already the other way regardless of your citation-free assertion. You're using the internet to proliferate this tripe - famously a DARPA invention.
> Imagine if those F35 mounting pylons were used to drop field-portable water-purification systems directly where they are needed most.
Why is a high-performance stealth aircraft required for this? Why does one thing have anything to do with the other? What peaceful purpose does stealth capability serve?
> Imagine if those ships were being used to bring power to the villages that need it most.
No need to imagine, this already happens: https://en.wikipedia.org/wiki/Powership
> Disarm them too.
With what? Words? Precisely how do you non-violently disarm someone who is willing and demonstrated to use violence to prevent you from disarming them? Again, the world's resources are finite, and control of these resources inherently requires violence.
Your ability to spread these "ideas" are rooted in violence: taxation, freedom generally. Freedom isn't free, and the price is paid in blood.
I get that folks don't like war. I hate war. I've seen much more evidence that deterrence produces peace more readily than disarmament. On the flip side, more destructive weapons may be used more rarely, but they kill more people when they are used.
I'm not sure what the optimal approach is, but it almost certainly requires nations to arm themselves in case they are attacked. "Provide for the common defense" was not specified idly.
It might actually be wise [1] to create a new, slightly less permissioned license for open source wherein the open is defined as anything that isn’t in a kill or surveillance chain or, more broadly, for military use…
[1] for the community
[0]: https://opensource.org/osd
[1]: https://www.json.org/license.html
[2]: https://www.gnu.org/licenses/license-list.en.html#JSON
If you actually wanted to respect such a license,you would have to endlessly argue what is dual use technology for example, the same as people argue if advertisements on project website or tangentially related donations make the thing "commercial"?
While at the same time, any "bad guys" will just ignore both.
If better artillery is being built on open source tech then everyone has better artillery. Not just the people "protecting you" but also the people trying to kill those people
+
>risking their lives to protect your freedom
=
nonsense.
The only thing that protects my freedom is active peace-making at levels greater than active war-making.
We are all under a great deal of threat precisely because borderline religious warrior narcissism has taken precedence over emphatic, unstoppable peace-making.
The USA has been bombing people into oblivion "protecting peace", for decades now. Is the world safer now?
No, it very definitely is not.
>Why do you want the people risking their lives ..
I want people risking their lives to make great peace. That is not happening at the scale that it should be.
Instead, the lazy, less noble warrior narcissist, has the world by the nose. The only difference between an utter criminal and a war fighter is a very, very thin line of paper which makes murder in one case illegal and in another case, legal. This paper is tattered and torn and covered in the muck of the victims, held high by the victors to justify their crime.
However, we see today the fallacy of this condition.
If your country is going to shoot something, I want to be damn sure it has done everything ethically, technologically, and administratively possible to have first made peace, at all costs.
That is simply not evident in any of the current world conflicts.
Instead of engaging in foreign cultures, we in the West march in lock-step behind those who have deemed foreign cultures inferior and worthy of nothing but destruction and despair.
I want to see the warrior narcissist who profit from this condition, disarmed and made irrelevant. The only way that happens is if human life takes precedence over human identity.
No humans identity is worth more than another humans life.
Risk your life for true peace this way: Before you murder, abandon yourself!
Hi. Veteran here. I protected his freedoms to do just that. You asking this question in that way insults veterans, whether you intended it or not. People using this line of thinking usually only care about veterans when it can win them arguments, and do not care any other time.
> I don't understand this thinking at all
Then, instead of asking questions with inflammatory language, you should humble yourself and ask respectfully so you can understand.
Don't build technology that interacts with people in ways they did't consent to. Don't build skinner boxes. Let the user decide who gets whatever data they generate while using the thing. Let the user control how it will work, insofar as it doesn't contradict the other heuristics.
These are political principles, but they're pretty basic. You don't have to keep up to date with the news, just check in with your morals once in a while.
And, to be clear, I’m not saying “politics” as in “did you hear what Trump said?” I’m saying politics as in “what is your city council doing, how are your neighbors doing, and what are the consequences of your work and economic choices and who’s paying them.” Most of the news is a circus; that stuff is not.
The nit pick arguments are just a defence to avoid honesty and introspection IMO.
I like some platforms do give us choice over our data but rarely do they not have our data be a component of the product. OpenAI, and anthropic for example use our data for "analytic purposes" which is broad enough that we compromise our agency over our data.
So what do you do when the majority of the systems you use on a daily basis to function in a normal life, require you (or atleast the majority of the populace) to compromise on morals? Do you know its expensive rise-wise, cost-wise, socially etc to check in with you morals in that kind of environment. I'm just curious your thoughts on this or if I'm misreading it. edit: And I'm not all doom gloom, I have hope and desire and trust in a future where morals aren't compromised in this way - I just see it as coming with a higher up-front cost and being aware that the most common environments around you systematically actively oppose you.
You replace those systems, one by one, with systems that don't. You figure out who is doing the work that makes that possible, and you figure out how to ensure they're supported while they do that work. You educate your friends and family that doing so as well is an investment in their future safety. Consider contributing to systems that help, consider damaging systems that harm. Fix a graphene OS bug. Shame a Palantir employee. Donate to Signal. Put a bag over a flock camera. Break somebody's pervert glasses (that is, supposing they're using them without consent of the people around them). There are a million ways to resist, some easy, some hard, some risky, some safe. Find something that jives with your tolerances and your skills and take action.
Of course you can avoid politics. Just be a good person. Vote for who you think is the best candidate at the time. And move on with your life.
Ironically the people most involved in politics don't have their own house in order. They're often mentally ill, over educated, underemployed and without meaningful ties in their life (e.g. family, real life community, etc).
Watch how police states behave. What tools do they use? Typically, it's roots of trust. They take down sites by tampering with their domain name. They MITM web traffic by compelling certificate authorities to do unstrustworthy things. They spy on cell phones by mimicking external authorities that the phones are wired to trust explicitly and without input from the user. They insert devices into ISP infrastructure which users tend to assume is neutral.
So if we want to prevent this: Don't tolerate roots of trust that the user cannot configure. Use a threat model that includes nation states. Don't design with the assumption that trusted third parties will remain trustworthy. Think about how to protect your users privacy. Prefer peer-to-peer where possible, because your server may at one point become compromised and harm the users. Think about how they can protect themselves if that happens. Give them control over whether to install updates. Publish your API so users can make unauthorized clients if they don't trust yours. Prefer repeatable builds, and check hashes to spot malware. Review contributions and be wary of binary blobs that may have hidden use cases.
Mostly this is just sound engineering. It means doing your job well, which is not always the same as doing the job the shareholders asked for. They invested so they could have leverage, so ask yourself ahead of time if those are the same kinds of levers that police states want to pull. Shoot them down in the design phase if possible, sabotage them after the fact if necessary.
"Being aware"
"advocate for"
What are you talking about? Should you hound co-workers that don't work for you? Leave pamphlets on cars with bad bumper stickers? Subscribe to the New Yorker?
If you're interested in politics you should read it, but you should not let it affect your life in any meaningful way.
That is definitionally not avoiding politics.
I think you misunderstand what politics is, at a fundamental level.
All of the above. Get involved with political organisations, NGOs, volunteering groups, go to protests and network with people there. So that when the government starts doing something that approaches a police state, you, your group, and many other groups of people will be out there ready to make their will known. Block roads, burn tires, occupy squares, remove cameras, etc. Is is easy? Not at all. Can you do it alone? No. Is it worth doing to avoid living in a police state? Yes.
Take a look at Russia if you want an example of what widespread political apathy leads to.
If someone blocks traffic because of [topic], I don't think "gee maybe we are overly-dependent on oil", but the reverse. Most people are the same.
If I see someone defiling a public space, I am more accepting to more authoritative means to restore order.
If someone cuts ties with me because of their political beliefs, I am further convinced they are part of a cult. Because that's essentially what cults advocate for.
You can believe what you want, but you should not be deluded to think you're changing anyone's mind by behaving this way.
Stallman was right again
https://www.azquotes.com/quote/755239
That said, they are both compatible, you can refuse to do something, and make efforts so that nobody does it.
But I think the the latter is the actual core security model of our society, and refusals aren't really effective to the point that I wouldn't worry much about it and just focus on compartimentalizing and executing on my duties. Isn't it theoretically possible that upper management divides duties so that engineers cannot accurately judge the ethical correctness of the task and the overall mission? As an individual you don't have the right tools to make such an evaluation. You can be pushed to investigate further, but the moment you do so, you cease your role as an engineer and put your citizen hat on.
I personally wouldn't object on most cases, my threshold would be very high, a database of people's sexuality is not even close to even being concerning to me, that's like a Wikipedia infobox property. But even if it were a feature that did concern me, like a citizen-ID to real time coordinates system, I wanna be at the helm of it, either way (if I get the privilege). If something is wrong, I'd be in a privileged position to do something about it internally, judicially or through congress, there's also whistleblowing if it comes to that, but that's not in me.
Correct, and the lessons in the CIA's very own Simple Sabotage Field Manual are still as helpful as when they were written.
You can't avoid building technologies that could be used by a police state, because literally everything can be used for warfare or aggression. The first person who sharpened a stick or a rock created a weapon. Everything is a weapon. Everything is dual-use.
If you want to prevent a police state or any other undesirable political outcome, you have to confront the politics directly. "Change the tools and you change the rules," an old saying from 1990s cyber-culture, is an idea that I think has failed. Instead what happens is that the rules determine what's done with the tools. Those 90s cyber-hippie and cyber-libertarian types vastly underestimated the power of the political to the point of making a category error.
Cryptocurrency is a great case in point. It started as an attempt to create a technology that would act as an escape hatch from corrupt casino capitalism and state-backed financialization. Instead it became an insane absurdist parody of those things. The same types of corrupt individuals who profited off "too big to fail" and the rest of it simply took cryptocurrency and used it to implement the same type of system but on a block chain. The rules (and rulers) changed the tools, not the other way around.
Cryptocurrency is clearly a counterexample to your main point. The fact that some people use cryptocurrency tech to implement centrally managed private currencies is largely irrelevant. To the extent that cryptocurrencies are used to implement private currencies, it largely just hampers them technolgicially if it has any effect at all. These projects just like to use the term cryptocurrency to imply some level of legitimate decentralization or security that doesn't really exist.
It's like saying that because you can use a screwdriver as a weapon, it's inherently a destructive technology (or some mixture of destructive and constructive). On closer inspection it's clear that the screwdriver doesn't sit on the pareto front for weapons, but it does sit on the pareto front for manufacturing tools. So a screwdriver is clearly a technology whose existence favors construction over destruction. Similarly I would argue that technology like bittorrent or bitcoin favor decentralization over centralization.
As another example, surveilance technology favors centralization. None of what we are seeing now with flock would be possible without cheap mass produced digital cameras. The balance of technology favors cheap cameras (and counter-surveilance is often impossible or expensive) and it favors centralization because you need a lot of data to create inferences based on this spurrious data.
If we could have predicted these effects, perhaps we could have delayed the development of cheap cameras, but it just goes to show how complicated this game is. We could just as easily have said that more cameras would lead to unprecedented public transparency (e.g. bodycams on cops, dash cams). Unintended consequences.
The problem is not just the tech, its building systems where humans are essentially slaves to the system, with no recourse or power over whats happening.
The solution is the same solution as any financial risk: diversification. I have many accounts at many banks. If any one of them goes down, I’m fine. If multiple go down, I’m still fine in the interim.
It ideally makes people better at identifying when something bad is happening and gives them tools to handle it. If they're amoral in the first place though a course isn't going to change that.
All the great people in history who would be regarded as ethical never took an ethics course.
Its hard to know whether an ethics course makes people behave ethically, but its clearly not a necessary requirement to ethical behaviour
Although lot of terrible people in history were likely at least outwardly religious, so not sure that proves anything one way or the other.
I don't know exactly how that looks, and yes there are many issues with it, but we're living in the age of cowboy coding.
I also think there's a long, long way between here and there, especially in the US. Requiring ethics in CS programs seems more immediately attainable to me, but I agree that it isn't a standalone fix.
Ethical stance is not something you can teach in one course. If you don't have this disposition by the time you're in college, I think you're likely not learning it.
(And, to be fair, it's a discussion board on the Internet, so maybe my expectations are just out of whack.)
"My ethics teacher has gotten his sons girlfriend pregnant. I think about this often"
Reply: "To be fair, you have to be really good a ethics to get away with that"
How are you supposed to 'learn' ethics?
Never understood the point of those. My university had over 4 for the cyber students and over half the grads went on to work for the MIC contributing to blowing up brown children in the middle east anyway.
Clearly, our problem must have been that they didn't have 5 mandatory time-waster ethics courses. That surely would have solved the matter. Add in a few extra "racism is bad" and "don't be a meanie" courses to round out the collection, just for good measure.
What program? Or if you'd rather stay anonymous, can you share the curriculum somehow? I'm very curious about what a CS program of study looks like with that many ethics courses.
In the US, many undergrad programs award a minor with 15-18 hours of study in the subject (or 5-6 standard courses). Four ethics courses would basically get you to a philosophy minor.
Edit: To make this comment have a little substance, what I'm getting at is, universities are already way too biased and one-sided on humanities issues to be teaching ethics. Students notice it too. I've gotten As on some insanely bad papers just cause I knew what they wanted to hear.
It was entirely performative. You can teach a curriculum of ethics, but that won't make a person empathetic. These are courses for credit hours, not firmware updates.
And a person with empathy can always find or be given reasons to not apply his empathy to a person or a group of people.
The problem is all the techbros who take those ethics courses and laugh at them as easy and pointless and don't care about them.
There's a lot of very unserious people right now.
What about an ad system that can predict beliefs and home addresses? Ooops.
Google, Microsoft, Apple, every telecoms provider, every modern car maker..
For example, I just voted in the Minnesota primaries. Ours are open primaries, so you don't need to register with any party and the ballot has every party on it. Though, it spoils your ballot to vote in multiple parties' primaries. So you gotta choose, but you do that privately.
[0] https://ballotpedia.org/Primary_election_types_by_state
I don’t belong to one, but it is sort of annoying to not be able to vote in the “primary elections” (where the parties pick their candidates). My state has a strong partisan lean so this is the only election that actually matters here. Oh well.
I'm given a promotion for adding invasive features based on "insights".
I'm incentivized to create something patent-able to restrict competition.
I'm rewarded for more detailed metrics to reveal more.
I'm hired by how many people were affected by my previous work regardless of ethics.
There is no reward for protection, privacy, and restricting access to data.
> “For my friends, everything. For my enemies, the law”
I prefer speed cameras, red-light cameras, and so on because automated enforcement removes discretionary power from enforcement agents. I believe that enforcement agents are the arms of the civic body tasked with ensuring the commons are not ruined. Consequently, controlling the exertion of discretion is entirely an engineering approach to controlling quality of enforcement. Others believe that enforcement agents are our rulers or agents thereof that we must submit to - and therefore to whom we must give great discretion so that they can choose whether we are worthy or not.
That is incompatible with all being equal before the law.
There can still be corruption with automated enforcement. If the punishment is fees and the fees go towards the budget of those in charge of the automation parameters, then you'll see manipulation of parameters to increase fees paid. Shorter yellow lights, shorter distance between a speed limit sign and the speed camera, requiring more effort to contest the automation when it's wrong than to just pay the fee, etc.
Flaws exist but we can constraint-engineer to minimize adversarial optimization of parameters. Improvements allow us to asymptotically approach our goal optimum. Human discretion, on the other hand, is fluid and dodges all attempts at enforcement.
We require many human agents solely because transforming unstructured vision space into structured judgment was impossible in the past. Currently, in a sufficiently constrained environment (roadways) we are able to solve sufficiently constrained problems (speeding, running a red light).
One hopes that we are able to solve this problem in more spaces as time goes by.
I agree with you on our obscene debt and our obsession with "conflicts" around the world, but I'd argue we're seeing the mass reduction of state employees and it's just making everything worse (eg massive deregulation and the destruction of checks and balances).
So, Either do it yourself (China) or get run by them (India).
I'm not dismissing the sentiment here. It's morally sound. You should act in accordance with your principles and it useful to be reminded how those principles intersect the world. I have similar guardrails (probably not the same as the author's). I just wouldn't want people to overestimate what can be accomplished with this approach.
I call this https://wiki.roshangeorge.dev/w/Story-Logic_Bias
But that seems normal and historical. Stories are how we encode memorable concepts for transmission to each other and probably why the Bible is full of parables and so on.
Someone pointed out to me that said Harry Potter fanfiction author has written about this specific bias as well https://www.lesswrong.com/posts/rHBdcHGLJ7KvLJQPk/the-logica...
edit: posted is a podcast by Behind the Bastards about the Zizians, a death cult that branched out of rationalist community and which was responsible for six deaths, but it covers the rationalist community and the Harry Potter fanfiction as well.
[0]https://www.youtube.com/watch?v=9mJAerUL-7w
If I had told you 30 years ago that we are going to build a surveillance network and make the people we are tracking pay for it all you would have laughed me out of the room. But that is what we have built. The modern Smart Phone.
Now we use our iPhones to complain about cameras on public roads as if they are watching us.
Do you seriously think that governments need Flock cameras when they have Apple, Microsoft and Google?
What's the recourse here?
see war on drugs and more
https://en.wikipedia.org/wiki/IBM_and_the_Holocaust
I fully agree with your base perspective, that the main focus should be on preventing authoritarians from coming to power in the first place.
I also agree with your final point, that the government doesn't need tech to lean on a person.
We disagree in the middle, on how tech that makes evil orders of magnitude easier might be a contributing factor and might enable the government to lean on many more people than it would otherwise.
We also never seem to accept this answer for other forms of tech. Should we ban encryption because it makes it easier for terrorists to hide? Should your phone run ever photo you take through a CSAM database to catch preditors? Should we ban 3d printers because someone might print gun parts? For some reason we understand that people motivated enough to commit crimes will find a way to do it. This is much truer when it comes to the actions of the state than it is for individuals. I'm never going to learn to smith parts for a gun, but I could figure out 3d printing pretty easily. If the state wants me dead or in jail, I don't think the Flock cameras are that meaningful.
* https://dave.autonoma.ca/blog/2019/06/06/web-of-knowledge/
* https://i.ibb.co/WWWYznHX/flock-future.png
* https://www.cfr.org/education/learn/reading/what-fascism
Don't create gradients you don't have dams for.
No, but they were extremely happy to hop over to France and realize that we kept lists that indirectly listed jews. Actions have consequences that we may not be able to predict.
Sure, we could all live without IDs and use motio belli for defence but this arrangement will be soon replaced by someone more organized because there is a huge return to that organization despite its risks.
This is not about technologies (after all, an authoritarian government will use many existing technologies, from the car to the computer), but about powers granted to a government, that could be used differently by the next party/majority in power.
The answer to that is to not grant government too much power in the first place.
Of course, this is often a contentious point for leftists, since they do want to grant government a lot of power in the first place; they're just concerned that the exact same powers will then be used by a different political side in a way that bothers a different category of people.
the solution is not to have the power exist at all, anywhere.
This is nonsensical. The power always exists, it's just your fellow human beings doing things, to you, around you, with you, without you.
You cannot do anything to not have that power. It doesn't matter if your neighbors are lynching you because the evil dictatorial police state said to, or because the rough consensus of a mob of lunatics want your food.
You as a human lack power. Humans working together have far more power. Sometimes, you can use the combined power of some humans working together to empower other humans. That's what laws are. That's what a State can do.
That's why corporations can have power even though they are not part of government. Being able to direct 1000 humans to do something is what power is.
You cannot eliminate that power. It's a fundamental part of reality. It's the entire reason animals evolved to be social at all, to take advantage of that power.
“Become ungovernable.”
>It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.
https://blog.codinghorror.com/to-serve-man-with-software/
But still, it's not possible to prevent abuse altogether and sometimes we even applaud breaking the rules (the movie Hunt for Red October for example kicks off with the Soviet captain jumping the guardrails).
Also, the most "benign" public infrastructure is usually the most impactful, e.g., water systems and electrical systems. Don't like the people in that one town who keep protesting and voting against you? Turn off their power, water, and also make it so that their trash doesn't get picked up for a year. See how they like it.
“If war is not impossible, then every advance in scientific technique in an advance in mass murder”
As true today as then
You can replace war with “police state” and it’s the same message.
https://m.youtube.com/watch?v=GzTlTVaYDtk
Or we could stop all tech development. But it is not going to happen.
Everyone in that pod agreed the Gov was broken. Nobody could put their finger on it or articulate in what AI might deem a "load bearing" - HHAHAH capacity. The other day i saw a comment on here by bob1029 [1] and he put it better than I ever could. It's broken because nobody can execute across all the moving parts; its a coordination issue in many ways - the cost of collaboration has become so large that it has become the thing to overcome instead of whatever first principal is the whole point of the shit e.g.: Healthcare, Food, Housing etc... The bottleneck was never intelligence it's coordination. The cost of getting many parties to act together without the whole thing blowing up.
The Anthropic multi-agent article hit it too...turf war, the 2.4M-request queue, the price collusion, the merge silos. Every one is the same thing: free-form runtime "negotiation" blowing up. And every success is an agent ad-hoc reinventing an institution mid-fight inventing a "verifiable criterion," writing an apology commit and pulling in a human, running a tournament as a commitment device. They keep hand-rolling the institution, badly, under fire.
So stop making them reinvent it. The generalization of the "constrained action space" idea isn't just bounding one agent's moves it's what I call an intent: intents don't just constrain each agent's actions, they codify the coordination itself.
And I make the unit of coordination a first-class object: the Deal a structured negotiation instead of a free-form one. The Deal is the container those agents keep reinventing, handed to them up front: scope and terms, the parties and their roles, the ordered steps, a verification gate as the neutral arbiter, and an appeal / human-escalation path as recourse, all over an append-only record. Agents negotiate inside a bounded contract instead of improvising one mid-fight.
How it works from what you say to what actually happens (the "Deal")
1. You speak the intent. You can use a form but I found people like the verbal shit much more.
2. Break the problem into an ontology. The ask gets decomposed into the real work it implies mapped to actual job roles (O*NET). Problem → Solution.
3. Vet the solution with the gov council agents. Every domain has one public AI expert in that agency's real systems Medicare, Medicaid, SSA, OPM (pda:codify.<domain>). Policies get deliberated and voted, elected officials in the loop, before anything runs. Stakeholders.
4. We staff it. It matches the specific expertise needed the exact specialty, not a generic "doctor." Down to the right role for your case.
5. We set up the financing. The money that follows the person eligibility, benefits, the payment rail. Financing.
6. An agent manages the process with expertise humans in the loop for anything high-risk, adjusting as outcomes come back, like an adaptive trial, until the outcome is met or it fails honestly. Expertise → Outcome.
The atomic unit of value here is a "Deal". Problem broken down, solution vetted, staffed, financed, executed under contract. intent → Deal → outcome.
Overview
- Intent: https://wiki.erpo.law/agent/wa-erpo
- Program: https://erpo.law/program/71069-petition-for-an-erpo-as-a-fam...
- Protocol: https://erpo.law/protocol/71079
I'm building it in the open because I think thats the only way this can be done
https://medicare.dev
https://epa.dev
etc...
Learning how to take criticism better so the more criticism the better.
[1] - https://news.ycombinator.com/item?id=49316271
Seriously, if the coordination problem you've identified isn't solved by the instantaneous global communications tech that we've had for between thirty and a hundred years (depending on how you count), it's a people problem that's not going to be solved by more tech. Notice how everything except maybe steps six and three in your plan require people to agree on the nature of the problem, whether it is a problem that even should be solved, and how many resources to commit to solving it. Getting people to agree to solve problems that need to be solved -even in the cases where solving those problems makes them personally worse off- is the blocker. Your use of shiny new software tools does nothing to remove that blocker.
-----------
This is one of those things that make for great post title and gets you all riled up at first but when you think about it, it's really shallow.
Most civic databases can be abused by any sufficiently authoritarian government. A homeless shelter's records may contain exactly the sort of information that could be used to get "undesirables" off the street. But those same records are useful for coordinating beds, outreach, medical care, etc. Surely the conclusion isn't that the shelter shouldn't keep records.
It seems much more useful to focus on limiting what gets collected, who gets access to it, and how long it can be retained. In other words, boring things like data-minimisation and retention laws. Somewhat amusingly, that gets you pretty close to the sort of principles behind GDPR, which isn't exactly universally loved around here.
I don't really disagree with the blog post's moral intuition. What bothers me is the emphasis on individual moral choices by the people building these systems. I've seen plenty of my peers happily work for companies that have spent the last twenty years building exactly this kind of technology, there will always be someone willing to build it. That's not an argument for abandoning personal ethics, but it is an argument against treating personal ethics as any form of serious societal safeguard. "good engineers should refuse to build them" means nothing.
That a system can be avoided offends the investor who want their products to be ubiquitous.
The system that is not perfectly enforced offends the engineer who dislikes broken things.
Yet both are required if we as a society are to grow. I think the thing that it might be good to learn is when to say "I've done/had enough."
Nope.
Now that the red pedophiles are in charge instead of the blue ones, he changes his tune.
many countries like Singapore, etc. prove this easily
Authoritarianism can even be desired by Capitalists if they know who to bribe to remove regulations and get approvals
and no surprise, that's exactly where the United States is right now, teetering on the edge of that hole, about to fall in
"AI" is going to be weaponized, already is, and any resistance against its abuse is simply overriden with threats to destroy profits or even company survival at all
Economic freedom is a necessary but not sufficient condition for political freedom.
If you're a nerd developing open source (not an insult; I'm in that category), our job is to develop effective technology.
If you want to do politics, go into law and/or politics (or into business, so you can fund/control politics).
>"But software which OpenBSD uses and redistributes must be free to all (be they people or companies), for any purpose they wish to use it, including modification, use, peeing on, or even integration into baby mulching machines or atomic bombs to be dropped on Australia. " -- Theo De Raadt
It's embarassing for an adult to think like this. There will always be someone willing to build that thing (and that was true even before llms)
… with sufficient time and money. As opposed to the minimal time and money required if the baddies are just pulling an existing solution off the shelf.
We may all play some part in societies that have some degree of monstrous outcomes, but I feel like staying on the "tragic civilian casualty" side versus the "legitimate target" side of a freedom fighter's bombing campaign is probably the way to go.
Joe's gonna die anyway. I might as well kill him before someone else does.
It's embarassing for an adult to think like this.