Back to News
Advertisement
ggregpr07 about 4 hours ago 20 commentsRead Article on github.com

ES version is available. Content is displayed in original English for accuracy.

Hey HN,

We got tired of browser frameworks restricting the LLM, so we removed the framework and gave the LLM maximum freedom to do whatever it's trained on. We gave the harness the ability to self correct and add new tools if the LLM wants (is pre-trained on) that.

Our Browser Use library is tens of thousands of lines of deterministic heuristics wrapping Chrome (CDP websocket). Element extractors, click helpers, target managemenet (SUPER painful), watchdogs (crash handling, file downloads, alerts), cross origin iframes (if you want to click on an element you have to switch the target first, very anoying), etc.

Watchdogs specifically are extremely painful but required. If Chrome triggers for example a native file popup the agent is just completely stuck. So the two solutions are to: 1. code those heuristics and edge cases away 1 by 1 and prevent them 2. give LLM a tool to handle the edge case

As you can imagine - there are crazy amounts of heuristics like this so you eventually end up with A LOT of tools if you try to go for #2. So you have to make compromises and just code those heuristics away.

BUT if the LLM just "knows" CDP well enough to switch the targets when it encounters a cross origin iframe, dismiss the alert when it appears, write its own click helpers, or upload function, you suddenly don't have to worry about any of those edge cases.

Turns out LLMs know CDP pretty well these days. So we bitter pilled the harness. The concepts that should survive are: - something that holds and keeps CDP websocket alive (deamon) - extremely basic tools (helpers.py) - skill.md that explains how to use it

The new paradigm? SKILL.md + a few python helpers that need to have the ability to change on the fly.

One cool example: We forgot to implement upload_file function. Then mid-task the agent wants to upload a file so it grepped helpers.py, saw nothing, wrote the function itself using raw DOM.setFileInputFiles (which we only noticed that later in a git diff). This was a relly magical moment of how powerful LLMs have become.

Compared to other approaches (Playwright MCP, browser use CLI, agent-browser, chrome devtools MCP): all of them wrap Chrome in a set of predefined functions for the LLM. The worst failure mode is silent. The LLM's click() returns fine so the LLM thinks it clicked, but on this particular site nothing actually happened. It moves on with a broken model of the world. Browser Harness gives the LLM maximum freedom and perfect context for HOW the tools actually work.

Here are a few crazy examples of what browser harness can do: - plays stockfish https://x.com/shawn_pana/status/2046457374467379347 - sets a world record in tetris https://x.com/shawn_pana/status/2047120626994012442 - figures out how to draw a heart with js https://x.com/mamagnus00/status/2046486159992480198?s=20

You can super easily install it by telling claude code: `Set up https://github.com/browser-use/browser-harness for me.`

Repo: https://github.com/browser-use/browser-harness

What would you call this new paradigm? A dialect?

Advertisement

⚡ Community Insights

Discussion Sentiment

100% Positive

Analyzed from 505 words in the discussion.

Trending Topics

#browser#agent#https#com#harness#paradigm#tools#remote#self#using

Discussion (20 Comments)Read Original on HackerNews

mattaustinabout 1 hour ago
I submitted a remote code execution to the browser-use about 40 days ago. GHSA-r2x7-6hq9-qp7v I am a bit stunned by the lack of response. Any safety concerns in this project?
dataviz10006 minutes ago
Yes, why the F*&( is there a remote-debugging warning!

The consent checkbox at chrome://inspect/#remote-debugging only gates Chrome's primary/default user-data-dir -- your everyday browser. The reason to use this is so that all your history, cookies, and everything else is not available to the agent. Give the agent an empty sandbox profile to start with. If it needs a persistent login, ... hmmmmm roll a browser in a browser, BRB.

Browser Use knows this! [0]

[0] https://github.com/browser-use/browser-use/issues/1520

LarsenCC20 minutes ago
Hey! Where did you submit this exactly? Can you provide a link? Will ask others on the team also, but I am not sure what you are referring to.
esafak2 minutes ago
1. Can you elaborate on the self healing aspect of it?

2. Can you publish a tabular comparison on your README?

3. What information gets sent to your API server?

syl5xabout 2 hours ago
That's pretty good, I've achieved pretty much the same thing using the vercel's agent-browser, but I've tried playwright and it worked easily as good. Its good for scraping, automating stuff in the browser.
reaganhsu29 minutes ago
agent-browser uses playwright so it struggles with things like cross-origin-iframes - on the other hand, browser harness uses raw cdp, which is unrestrictive. It's discussed in this blog post! https://browser-use.com/posts/bitter-lesson-agent-harnesses
debarshriabout 1 hour ago
I think the usecase here is to go beyond scraping. I think you can use it as a tool for agent harnesses and make it part of a larger workflow.
SomewhatLikely15 minutes ago
If I'm understanding correctly, this might be the first example of just-in-time agentic coding that I've come across.
embedding-shapeabout 1 hour ago
> The new paradigm? SKILL.md + a few python helpers that need to have the ability to change on the fly. [...] What would you call this new paradigm? A dialect?

It's called "agentic coding" for all I know, and isn't a new paradigm, the whole purpose with agentic coding is that it uses tools to do their thing, then those tools could be structured as the good old JSON schema tools next to the implemented runtime, or as MCP, or HTTP API or whatever, the "paradigm" is the same: Have a harness, have a LLM, let the harness define tools that the LLM can use those.

codethief12 minutes ago
IIUC the point is that the agent has the ability to modify itself? So one possible term could be "self-evolving" or "self-modifying agent".
embedding-shape10 minutes ago
Any agent that accepts "work in this working directory" (which AFAIK, all of them do) have had this ability, even the initial GPT2/3 experiments around matching LLMs with primitive tool-calling.
ameliusabout 1 hour ago
I was wondering when someone would finally build this.

Anyway, of course this will be superseded by a harness that provides freedom to complete any task within the OS.

reaganhsu34 minutes ago
it will be crazy when someone builds this
mvelbaumabout 1 hour ago
Sawyer Hood's dev-browser[0] allows the browser to write playwright JS code directly. Do you have cases where his approach fails and yours works?

[0] https://github.com/SawyerHood/dev-browser

LarsenCC15 minutes ago
PW is usually easier to detect. Using raw CDP is in our opinion much better for this.
jstanley8 minutes ago
We published a survey of stealth browsers just yesterday https://botforensics.com/blog/stealth-browser-survey-april-2...

There's still plenty that Browser-Use could improve in terms of stealthiness.

We didn't detect it using CDP (good!) but can still detect that it is Browser-Use.

Amekedlabout 2 hours ago
I’d call it “open washing”, but it looks cool. Good luck with it
LarsenCC17 minutes ago
Curious why? You can just take this and run locally or deploy anywhere you'd like with any provider agent provider.
doublerabbitabout 1 hour ago
And that's how I woke up with an LLM roleplaying with itself while looking at porn.
LarsenCC16 minutes ago
Lmaooo.