Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
82% Positive
Analyzed from 1281 words in the discussion.
Trending Topics
#durable#feature#execution#architecture#command#state#replay#model#executions#workflow
Discussion Sentiment
Analyzed from 1281 words in the discussion.
Trending Topics
Discussion (22 Comments)Read Original on HackerNews
So we switched to replay. The biggest benefit of replay is that it lets you implement Durable Execution in any language as a library without a complex runtime. It also supports code changes while workflows are in flight (Temporal calls this patching). Making snapshots of arbitrary code state backward-compatible with code changes isn't practical.
I personally think that, in the long term, Durable Execution will use a runtime that supports both snapshotting and determinism. That way, snapshots can be taken infrequently, and replay can bring workflow code to the latest state. Similarly to a database recovering from a WAL. WASM is the most promising technology to achieve this.
My company started around working on this problem because it's the basis for how you train programming models/reliably deploy LLMs to do specific tasks. It allowed me to build a much better mental model for LLMs because I saw how weirdly fickle/inconsistent/picky they could actually be outside of a "chat" where it feels like they have a coherent persona or consistent knowledge/capability.
Initially I thought of it as a search over prompts for capability at completing specific tasks, but now I think the speed/reliability and operations (eg can I switch models without degrading perforamnce?) benefits are even bigger benefits for most users.
A little "secret" since labs are making it harder to even use their models in this way and it's important that it be more widely understood: distribution-aware replay/re-sampling is a key technique in post-training LLMs. But it's also something that allows you to automatically identify the best model for some subset of your tasks, which can save you a lot of money.
If you first approach a tool because of buzzwords, don't be surprised that you think of the buzzwords instead of the tool.
Durable executions greatly simplify how workflows can be implemented and audited, and they literally allow eternal workflow executions that are not tied to the lifetime of an instance assigned to execute them. Durable executions do not require fancy infrastructure, only a terribly simple database.
If you understand continuations, you understand durable executions. Otherwise, you'll be stuck with the excuse that they are buzzwords.
In a distributed or concurrent system, for full granularity, that can require specialized timing or virtualization techniques up to ensuring fully atomic snapshots and deterministic execution environments (and whether or not that properly models the SUT in real environments, or introduces bias/breaks the reproducibility in a way you care about)
Otherwise if you're only running against fixed checkpoints you have something closer to traces that maybe you could re-run or test against, in some cases, if you put in the work to set it up. In distributed systems that can be a lot of work so it's a bit vague if left unspecified. Because it's not enough to merely replay something if things can drift or don't accurately model the real system
You really need to read up on durable executions before commenting. Your comment reads as if you are completely oblivious to them. Their whole point is benefiting from an execution model where your workflow is comprised of idempotent pure functions whose inputs and outputs are tracked by the durable task persistence.
FWIW, I think we'll see a rise of AI-ready interpreters. In some sense, I like that it challenges traditional microservice architectures as an aside.
I am curious about what prompted you to build a durable execution framework.
We didn't know we were building one until later as that is where our application development trajectory naturally lead us.
So, hello frenemies, I suppose :)
We are primarily focused on our Feature Architecture. Durable execution is only for command features, requires tight coupling with our architecture, and is not a standalone dedicated tool like Temporal.
Open to further discussion on both of our work's trajectories rather than digress here.
Building for extreme scale constrained the possible architectural space. For a high level overview of the application architecture, refer to the seperate comment here in this thread.
It started initially by realizing that our app, Slyp, requires extremely distributed invoice and coupon processing as we started rolling out. So the architecture must handle such scale without issues.
This lead to persisting every incoming request (for commands) from the frontend and dismissing them only to be informed later of continuing with the status. This avoids processing requests when the system is overloaded with high traffic. So naturally the minimum is a log style ordered persistence. Initially kafka. Later our own Monolog built in rust which is substantially faster for our workloads and zero jvm and low memory and can run on servers and mobile alike.
Naturally the right point to consume this was into the command ingestion point in the Feature Architecture as noted in the other comment. This expanded the capability to all feature invocations including inter and intra service feature invocations.
This then progressively evolved into a much capable engine at which point we realized, this is a durable execution engine for command features but as a minor part of the whole Feature Architecture itself.
Our architecture has had somewhat of a different primitive for years that yields the same outcome.
Our application architecture is named The Feature Architecture.
A unit of work is feature. A lot of common implementation can be derived from (or compressed into) the name of a feature. They are invoked by Features.invoke(feature_name,...) and seamlessly calls same service, service to service or frontend to backend service or even backend to frontend (with client ID and userid) seamlessly.
A command feature by default does durable execution by being a consumer as well as a feature as the machinery ensures all incoming command feature messages are injected into monolog (in house built akin to kafka). So command feature errors are retried by the machinery by default.
All Dip operations are idempotent and hence can be retried maximally.
Our arcc (The architecture compiler) enforces at compile time that 1) there are zero CQRS violations of query to command invocations 2) zero violations of query to Dip.insert/update/remove (extended CQRS for persistence) 3) all Dip mutate operations are idempotent (arcc --strict), 4) zero alien Dip collection access (a feature leaf and its handlers owns exactly one collection) 5) and a whole myriad of around 10 different architecture rules that usually depend on developer discipline and conventions.
One can set a command operation to be not a durable execution by specifying bypass: true for that feature in the registry but those are the outliers.
Checkpointing and replay are not mutually exclusive in our architecture as they emerge when a command feature is either a default or one with bypass: true.
So durable executions are inherently native and first class for all commands in our Feature Architecture.