DE version is available. Content is displayed in original English for accuracy.
I am excited to share with you TERMy, a terminal assistant built on top of the NPC-Forge framework. Unlike everything else being built today, TERMy does not use embeddings, machine-learning or LLMs. It runs on the CPU (even on a Raspberry Pi Zero) both in the terminal or client-side in a browser tab and responds in milliseconds. It is a cynical but very knowledgeable Linux terminal assistant that translates your natural language into shell commands without relying on a single artificial neuron.
I had a chance to focus for 2 months on my personal projects since early July, during the strange times of AI price hikes and the end of subsidized tokenmaxing. I was curious to see if I could develop from scratch a terminal assistant capable of handling simple natural language requests. I have a bad memory and got used to ask to copilot "activate the virtual environment" or similar trivial operations spending a non negligible sum every month. I started thinking, maybe I can do something to make my workflow more efficient? Do I really need trillions of parameters to accomplish those tasks?
How it Works
When you type a prompt, it goes through a lightweight NLU pipeline written in ~1000 lines of Python that implement the following steps:
1. Strip expletives, interjections, encouraging, discouraging and thanking words (remove noise)
2. Sentiment analysis
3. Exact Match (very fast)
4. Template Match (slower)
5. Probabilistic Match (even slower)
Step 5 relies on:
1. IDF (Inverse Document Frequency) to identify rare words.
2. BOW (Bag Of Words) to accommodate word inversions.
3. IDF weighted Levenshtein to safely handle typos.
Permission gating is hardcoded into the dataset and enforced for all potentially destructive commands, so it's inherently safer than letting an unpredictable LLM run wild on your machine.
- TERMy in operation: https://www.youtube.com/watch?v=qeIp0xePLBg
- Variance and typo tolerance: https://www.youtube.com/watch?v=tQvGDk6fkk0
- Copilot integration: https://www.youtube.com/watch?v=Wzzouhq2a8A
- Advanced features: https://www.youtube.com/watch?v=qeIp0xePLBg
- Source Code: https://github.com/gioblu/NPC-Forge
Discussion (23 Comments)Read Original on HackerNews
paper: https://arxiv.org/abs/1802.08979
WOW! With that dataset the capabilities of TERMy could be vastly extended!
Thank you.
What do you think about it?
TERMy (or is it the NPC-forge) seems to be worth a try.
It isn't. At least not by design, even though in practice it often can be. If you do greedy decoding (or use a preset seed) and deterministically compute everything (e.g. only use integer math) then it will be 100% always deterministic.
This tool has a finite amount of outputs for an infinite amount of inputs. Which is different from an llm based tool.
Have you considered/tried using a model that's, well, more appropriate size-wise for an use case like this? These are relatively big. Something like FunctionGemma [1] finetuned for a given set of tasks would be a lot more speedy.
[1] https://blog.google/innovation-and-ai/technology/developers-...
I really look forward to a hypothetical LFM3-230M, because LFM2.5-230M is so close to being usable, while FunctionGemma is miles away from being usable.
But, yes, still tangential to TERMy.
I hope the community will help me to enhance it :) it is just a proof of concept for now
I really like it, this flavor of specialization gives the user a win on privacy and speed. Seems like the right idea for such a tool.
$ termy create file test.txt and write Hello
TERMy | template match | Confidence: 100.00%
Thinking: Ok, I am asked to create the file test.txt.
echo 'Hello' > 'test.txt' && termy_set_context 'active_file' 'test.txt'
Description: Writes Hello in file test.txt.
Response: Affirmative
Now that I think about it, I should let TERMy use tldr...