µllm

I am not an avid LLM user, but sometimes I use it to ask some specific question where I expected the answer to be more specific than the one I would get from Google. I have always used the desktop app, but I found various drawbacks and as a developer I wanted to do something simple myself.

There are multiple reasons for it, here are the main ones.

• Use different models. Since the beginning of 2026 open weight models are becoming increasingly more powerful so I wanted to have the opportunity to try them.
• Command Line Interface. I live in the terminal most of the time, so I wanted to something that I can use directly from the CLI instead of having a desktop or browser application.
• Privacy-first. It's not always easy to make sure your data is not going to be used by companies who do the inference. I want to make sure that everything I say to the LLMs have a strict zero data retention policy.
• No subscription. I'm not a fan of subscriptions overall, and since I'm not a heavy user I would be paying much less if I pay per token instead of always paying a subscription. It depends of course which model I use and for how long.
• Multi-platform. I use multiple operating systems and multiple architectures, so this project needs to work on all of them.

With this premise I started looking around and found out that the best service for this is OpenRouter, and I only needed to build a simple CLI around it.

Here are the features that I built.

• Support for multiple models. The CLI is not supporting all the OpenRouter models because there are simply too many. I made a selection of both open weight and proprietary models that is good for all my use cases.
• Support for sessions. Sometimes I need to quit a session but I want to save it for either reviewing or continuing it later. That's why you can save and load the sessions.
• Basic support for artifacts. If the model is generating an artifact (e.g. a snippet of code) you should be able to open it in your editor and change or save it. There is a basic support for this.
• Support for attachments. In the prompt you can add a file using the prefix '@'. This is useful to add more context to the model so it can personalize the response for you.

There are various commands supported in the CLI. Here is the list.

• /models shows the list of available models /model <model> selects a specific model (default: openrouter/auto) /stats [on|off] turns on/off the statistics (default: on) /save-session <name> saves a session /load-session <name> load a session /artifacts show the list of artifacts /artifact <number> opens a specific artifact /compose opens an editor where you can write a more complex prompt

There is no configuration file for µllm. There are (I think) sane defaults, but if you want to change them my suggestion is to create an alias in your shell configuration passing your preferred flags.

The only environment variable requires is ULLM_OPENROUTER_API_KEY containing the API key of OpenRouter to use.

Download the binary or the latest binary or the source code here.