Back to News
Advertisement
nnonadhocproblem about 2 hours ago 7 commentsRead Article on github.com

RU version is available. Content is displayed in original English for accuracy.

I've spent the last few months working on this codec.

It has the following characteristics:

  - SOTA decompression throughput in its ratio class
  - Decent ratios (comparable to LZ4 at high effort levels)
  - Slow compression
Most of the gains can be attributed to reducing branches and making decompression very friendly to out-of-order cores, by using a smart format.

Results on the tarred Silesia corpus on Intel x86-64 follow:

  codec       decode      ratio    encode
  misa77 -0   5219 MB/s   42.64%   54.5 MB/s
  misa77 -1   4274 MB/s   39.65%   51.2 MB/s
  lz4         2505 MB/s   47.59%   371 MB/s
  lz4hc -12   2531 MB/s   36.45%   7.31 MB/s
Advertisement

⚡ Community Insights

Discussion Sentiment

100% Positive

Analyzed from 193 words in the discussion.

Trending Topics

#interesting#compression#developer#integrate#misa#code#project#low#cpu#encode

Discussion (7 Comments)Read Original on HackerNews

scottchiefbaker6 minutes ago
Almost double the decompression speed, *and* a higher compression ratio than LZ4? That's very promising.

I don't see much in your README that talks about how a developer would integrate misa into their code. I might suggest some basic code samples to help a developer integrate misa decode into their project.

Congrats, looks like a cool project.

Sesse__12 minutes ago
Interesting, but if you are not robust to corrupted/malicious data, it is really in a different class of algorithm and it is hard to compare speeds directly.

From memory, 2505 MB/sec also sounds on the low side for LZ4 on a modern CPU?

nonadhocproblem8 minutes ago
Someone asked this on encode.su too (see my reply here: https://encode.su/threads/4514-misa77-ridiculously-fast-deco...).

In short, my decompressor is very simple, and a naive safe version of the decompressor is only about 5% slower than the current unsafe one (and I will add this safe version in v0.3.0).

As for the raw throughput numbers being low here, it's because Intel Turbo (frequency boost) was disabled for stability, and the CPU was running at a fixed frequency of 2.1 GHz (I've confirmed that the relative performance scales similarly even with Turbo enabled).

logdahl13 minutes ago
Nice results, I will keep a watch on this! Would be interesting to see benchmarks vs Oodle compression, I think the most similar one is Selkie?
bootlegbillyabout 1 hour ago
this is super interesting! im excited to give this a look this afternoon, since I specifically have wanted faster throughout for decompressing maps in a game engine.
throwaway7474713 minutes ago
What's the Weissman score?