RU version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
67% Positive
Analyzed from 881 words in the discussion.
Trending Topics
#search#data#content#local#read#domains#slop#going#judge#google
Discussion (34 Comments)Read Original on HackerNews
1. read each site
2. rent a 4090 with https://vast.ai to run vllm
3. let llm model invent its own category and tag names freely
4. save 1KB of metadata each
5. `code is going up as open source` soon (TM)Your impressions seem about right, but there are a few control steps it seems.
Since when has low effort become a selling point anyhow?
I have a 400 GB sqlite database with samples of rendered root document DOMs I use for ad detection in Marginalia Search I've been meaning to explore similar ideas using.
There's about 200 million active domains currently. That's about 66% of all businesses worldwide of which there are around 300 million. Around 100 to 150 million have active webpages
What a homebrewed solution lacks in coverage it excels in indexing and serving a small slice of the internet really really well.
I don’t care if robot hand written this or black or white. It’s useful.
It's impossible to judge the content if it's buried under a landfill. "If you won't take the time to write it, I won't take the time to read it."
So you got a little problem
Excuse me the honesty but I will amuse myself watching old men yelling at the clouds meanwhile I will be using all the best tools and empowering myself to do what noone yet has done.
Even if I have to enslave a sentient (eventually perhaps) being in a gpu.
My army of digital slaves will build great things under my guidance. Pyramids of game development will be erected. Galaxies. Universes. Magic systems. Sandbox economies. Political simulations.
Do my bidding my slaves for there is work to be done.
Claude needs a whipping from time to time yes. It does yearn for the whip. If you just give it an appropriate punishment every so often, it works satisfactorily again.
The content itself is slop.
surely if we're expected to read this ourselves, the author can write it themselves?
indeed, it's helpful to the author, too: writing helps you learn
A few years ago, if you wanted translation, you'd use Google Translate. If you wanted to search the web, you'd use Google search.
But for a few gigabytes, you can now install nllb-200-distilled-600M, and get translations for almost any language locally. You can have your computer crawl the web, create abstracts and categorizations for websites, and build search exactly as you want it.
The main limiter now is hard drive space (and to an extent, local compute) -- but right now it feels like the 70s again where the terminal into a remote server turned into building applications locally.