Back to News
Advertisement
bbrian_kuan about 7 hours ago 8 commentsRead Article on github.com

DE version is available. Content is displayed in original English for accuracy.

Hi HN, I'm Brian, I spent the last few years at Vanta (YC W18), helping startups and enterprises become compliant and I recently started exploring what that might look like in a post-agentic world.

The problem Halo solves is: when a company buys an AI agent from a vendor and gives it access to their data, they have no way to check what the agent did with that data. Vendors may have built observability dashboards and audit logs, but those are editable and partisan. SOC 2 and ISO 27001 audit a company's controls, but controls are less predictive when the software is agentic. TLDR: give an agent the same prompt 50 times, and you get 50 slightly different actions/answers - so the only thing worth auditing in a post-agentic world is what happened at runtime.

Halo is an open-source project that produces agent runtime evidence. It's a small recorder that records every action an agent takes (eg. tool calls, model calls, data access, etc), and becomes a record in an append-only log. It's hash-chained, so anyone can re-verify.

Run the following command to see a fictional example:

     uvx --from halo-record halo demo --serve
Then, delete a line from one of the .jsonl files and reload, and the report will catch that it's been tampered with.

To wire up your own agent, run this line of Python:

     agent = trace(run_my_agent, profile="my-agent", log="audit.jsonl")
Then use this to generate a real report and give it to your customers:

     halo report audit.jsonl -o report.html
Disclaimer: this proves integrity, not completeness (as a self-held chain proves nothing was edited but does NOT prove that nothing was omitted). Catching this requires a witness outside the vendor and is what I'm working on next.

Halo is Apache-2.0, contains zero runtime dependencies, and is about 4,300 lines of Python with 125 tests (if you prefer TypeScript, here's that repo: https://github.com/bkuan001/halo-record-ts).

Give it a try, and please let me know if you have any feedback!

Advertisement

⚡ Community Insights

Discussion Sentiment

100% Positive

Analyzed from 248 words in the discussion.

Trending Topics

#agent#framework#love#vendor#working#glad#products#stack#built#audit

Discussion (8 Comments)Read Original on HackerNews

all2about 1 hour ago
I'm currently working on an agent framework that has auditability as one of its core promises. I'm glad to see others are working in this domain!

I've seen other products/apps in this space farther up the stack at the API boundary.

What frameworks does your package work with? How does it handle intercept?

brian_kuan35 minutes ago
There's no interception - it runs in-process, so it works with any Python code (with or without a framework). There's a TS version too if your stack is in JS.

Would love to hear more about your agent framework!

all27 minutes ago
I don't want to hijack your thread. My email is in my profile, I'd be glad to shoot you an in depth description of what I have so far.
ambicapter43 minutes ago
> may have built observability dashboards and audit logs, but those are editable and partisan

Why would these be editable?

brian_kuan22 minutes ago
Because they live in infrastructure the vendor itself controls - there's some conflict of interest there. Things like retention, rotation, deletion, what gets included/excluded, etc are all decisions the vendor makes. Same reason why compliance requires audits!

The hash chain doesn't make the log unwritable, it makes any edit detectable.

brian_kuanabout 2 hours ago
PS: please try to break it - if you find that the report does not catch a deleted line, changed number, or modified record, I'd love to know!

And to start a discussion: if you sell or buy AI agent products, what do security reviews ask about them?

nf-xabout 1 hour ago
Looks very slop, but it’s a good idea. The main difficulty is that no big name is hosting a witness.
brian_kuan26 minutes ago
Fair point - I am a marketer by training and built this with some help from Claude! But appreciate the love.

If you find something/have feedback, please let me know and I'll gladly fix it.

Separately - 100% agree on the witness - only someone/thing outside the vendor can prove nothing was omitted. Who do you think fills that gap - is it an audit firm (my world), a standards body, or something else?