Zero-copy in Go: sendfile, splice, and the cost of io.Copy

sly010•8 minutes ago

Beware, there are versions of go where sendfile is broken and only sends the first 4k of a file on macos.

sanxiyn•about 4 hours ago

A good reminder. It is surprising first time you encounter it.

Same for Rust. As https://doc.rust-lang.org/stable/std/io/fn.copy.html says, std::io::copy can use copy_file_range(2), sendfile(2), or splice(2).

drivebyhooting•about 1 hour ago

How is the byte counting reader supposed to work in user space without putting the buffer in user space? The article claims there is a way but I want to see what is meant by counting bytes in that case.

flakes•about 1 hour ago

sendfile(2) and io.ReaderFrom both return the number of bytes transmitted. The issue is that users are unaware of (or forget about) the optional interface upgrades and fail to define all the methods required for interface upgrades on their wrapper structs. You can definitely make a counting reader with a minimal performance loss, but the proper solution is less obvious than it ideally should be.

drivebyhooting•15 minutes ago

Isn’t this a symptom of structural/duck typing where interfaces are not declared? In Java for all its faults this wouldn’t happen because you’d be forced to implement all the interfaces.

mike_hock•about 4 hours ago

Zero-Copy in Go: Why magic is an antipattern, and: performance is observable behavior.

sanxiyn•about 3 hours ago

What would you prefer?

I do think it is criminal this is not documented (https://pkg.go.dev/io#Copy), but I think io.Copy is fine as an API.

arccy•about 2 hours ago

it is documented by saying it calls ReadFrom or WriteTo

inigyou•about 2 hours ago

This is almost like the expression problem. Copy is a new operation, and you introduced a new type, thus creating a new grid cell nobody from either side could have reasonably known about - except for the fact Copy is in the standard library so you could have known about it but not done anything.

throwrioawfo•about 3 hours ago

Ugh, AI slop writing.

drivebyhooting•about 3 hours ago

Definitely written by codex.

joaohaas•about 3 hours ago

Interesting premise for a post, but I had to stop midway due to the AI slop writing adding meaningless information.

Zero-copy in Go: sendfile, splice, and the cost of io.Copy

⚡ Community Insights

Discussion (12 Comments)Read Original on HackerNews