Ask HN: How do you use LLMs for private discussions?

eemerongi about 3 hours ago 8 comments

ZH version is available. Content is displayed in original English for accuracy.

I sometimes have things to discuss with LLMs, which are more private than usual. I was thinking how to do it in a way that doesn't reveal my identity.

  1. Use Tor to access the provider?
  2. Create a random account?
  3. Use some form of untraceable payment (which one?)
  4. Scrub all information provided to the LLM from personally identifiable information?

It seems like a lot of effort. So is running a local LLM, for which I don't even have the hardware. How do you do it?

⚡ Community Insights

Discussion Sentiment

100% Positive

Analyzed from 160 words in the discussion.

Discussion (8 Comments)Read Original on HackerNews

7402•about 1 hour ago

With llama.cpp, Intel i9-13900KS CPU, 96 GB RAM, RTX 4070 running locally.

The models I'm using right now with that are:

  gpt-oss-120b-F16.gguf
  Qwen_Qwen3.5-27B-Q4_K_M.gguf
  Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf
  gemma-4-31B-it-UD-Q6_K_XL.gguf

hbcondo714•about 2 hours ago

Maybe try Ollama Cloud, Prompt or response data is never logged or trained on:

https://ollama.com/pricing

blinded•about 3 hours ago

I spin up a gpu instance in a cloud, run my model via vllm, connect to it via an ssh tunnel. done.

stuxnet79•about 2 hours ago

Can you elaborate on the first step? Which cloud and which service? What's the cost outlay if you are just having a convo and not doing anything 'agentic'?

blinded•about 2 hours ago

Hey sure.

It depends, but usually spin up an h100 on lambda.ai or coreweave. They have capacity and their UIs/APIs are nice. I spin it up for an hour or two, believe it was 6~ dollars an hour.

Once the gpu instance is up, you need to run vllm and a model, ie https://docs.lambda.ai/education/large-language-models/deplo....

Then you can connect your pi.dev, openwebui, etc etc to vllm and interact with it like normal.

ahdgs•about 1 hour ago

i've been using OpenAI's os PII-masking model, works decently and lightweight enough to run virtually anywhere https://huggingface.co/openai/privacy-filter

Havoc•about 2 hours ago

With a local one

downbad_•about 3 hours ago

Can't you do it logged out?