Shared resident runtime
1
Model process pooled for editors, scripts, and the desktop chat — queued instead of duplicated.
Editor → llama3.2
Notes → embed
CLI → same pool
Local-first · nothing leaves your machine
One endpoint for every local model.Zero duplication.
AnyLM runs quietly in the background, pools every LLM already installed on your computer, and gives all your apps a single OpenAI-compatible endpoint to talk to.
Works withOpenAI SDKOllamaCursorContinueCustom scripts
Insights
One resident runtime. Every app benefits.
Illustrative view of how AnyLM sits between your tools and the models already on disk — no second copy of the weights, no cloud round-trip for inference.
Model pool activity
Liquidity of local weights
Cloud round-trips for inference
0
Tokens stay on-device. Connectors are opt-in and explicit.
Projects + RAG
Docs → chunks
Attach references; retrieve grounded context automatically.
Governance peek
Policy-ready for teams
Organisation rules over which models can be used, with usage recorded per member.
Allowed
llama3.2
Usage
local
Live pool
Requests float in. One gauge holds the pool.
Apps speak OpenAI; AnyLM routes them through a single resident runtime so memory stays sane and answers stay local.
Sent
Cursor → llama3.2
chat.completions
Routed
Notes → nomic-embed
embeddings
Queued
Script → same pool
no reload
Step 01
Single endpoint
Local-onlyOpenAI-compatibleSigned buildsPolicy-ready
Compare
How AnyLM sits next to the usual desktop stack
Ollama and LM Studio remain excellent runtimes and chat GUIs. AnyLM's wedge is a shared endpoint, pooled weights, projects, and governance — often alongside Ollama, not instead of it.
| Capability | AnyLM | Ollama | LM Studio | Jan | GPT4All |
|---|---|---|---|---|---|
| Desktop app | Yes | Partial | Yes | Yes | Yes |
| OpenAI-compatible API | Yes | Yes | Yes | Partial | Partial |
| Model pooling / shared resident runtime | Yes | Partial | Partial | — | — |
| Projects + local RAG | Yes | — | Partial | Partial | Partial |
| Org governance / usage | Yes | — | — | — | — |
| Background always-on router | Yes | Yes | Partial | Partial | — |
| Works with models already installed | Yes | Yes | Yes | Yes | Yes |
| Multi-agent orchestration | Yes | — | — | — | — |
| Project-first coding / file writes | Yes | — | Partial | Partial | — |
| Load protection (RAM soft-stop) | Yes | — | — | — | — |
Based on typical public product positioning; features change. Partial means available with limits, toggles, or a different workflow.
Enhance local models
Your weights, with hands and memory
AnyLM does not replace Ollama — it makes local models more useful: tools, skills, retrieval, and a single endpoint your whole machine can share.
Tool calling on local models
Ollama function calling turns a chat model into an agent that can use the tool registry — files, web, shell, docs — without leaving your machine.
Skills = instructions + tools
Enable a skill and the model gets a focused system prompt plus a curated tool bundle. Built-ins cover research and calendars; custom skills are yours.
Projects with real RAG
Attach reference docs. They are chunked, embedded, and retrieved so local answers stay grounded in your material.
One shared router
Point editors and scripts at :3227. One resident runtime serves everyone — no second copy of the same weights in RAM.
Connectors that act
Optional Google Calendar and Outlook skills let the model schedule and mail. Risky writes always ask before they run.
Multi-agent on complex turns
Heuristics detect hard requests. An orchestrator plans, routes independent work in parallel, and synthesizes one reply — with a collapsed agent trail you can expand.
Project-first coding
Scaffold with official CLIs when they exist, write app code with file tools, look up current docs online when possible, and keep the chat bubble to a short summary.
Guided Ollama setup
If Ollama is missing or stopped, AnyLM offers Install or Start — plus a boot splash so you never flash login before the dashboard.
Catalog
Skills, tools, and platform features
Everything the desktop app can offer a local model — from filesystem tools to connector skills and the background router itself.
Built-in
Web research
Search the web and fetch page contents for live URLs or current facts.
Project-first coding
On coding turns, scaffold and write files in a working folder, research docs when online, and reply with a short summary — not a full source dump.
Research specialist
Coming soonPhase 2+ multi-agent research role that plans deeper web + doc passes on complex questions.
Connector
Google Calendar
Read and create events on the user's primary Google Calendar.
Outlook
Read calendar and mail, and send mail via Microsoft 365.
Slack
Coming soonRead channels and post messages in workspaces the user connects (confirm on send).
User-defined
Custom skills
Bundle your own instructions with any tool picks; enable per chat.
Shared team skills
Coming soonPublish a skill once and enable it across an organisation's members.
Skill marketplace
Coming soonBrowse and install community skills without hand-writing instruction bundles.
Platform
Everything the router does
A background service with a desktop app on top. Install it once and every other tool on the machine gets a well-behaved model server for free.
One endpoint for every app
A local OpenAI-compatible server at a fixed port. Point your editor, your notes app or your own scripts at it and they all share the same runtime.
Model pooling, not model copies
AnyLM discovers what is already installed, keeps one copy resident, and queues requests instead of loading a second set of weights into memory.
Projects with real context
Attach reference documents to a project. They are chunked, embedded and retrieved automatically so answers stay grounded in your own material.
Streaming chat, locally
Token-by-token responses from local models, with conversation history stored on your machine rather than someone else's.
Policy and governance built in
Organisation-level rules over which models can be used and by whom, with usage recorded per member.
Updates that arrive on their own
Signed builds published straight from CI. The app checks for new versions and updates itself in place.
Multi-agent when it matters
Simple chats stay single-agent and fast. Complex turns plan, route, and run parallel Phase 1 roles, then return one synthesized answer with an agent trail.
Project-first coding
Coding requests create or update files in a real folder — CLI scaffolds when available — and finish with a file/command summary instead of pasting the whole program into chat.
Cloud API backends
Coming soonOptional Claude, OpenAI, and similar API keys as selectable backends beside local models — same chat flow, cloud when you choose it.
Drop-in
Works with anything already speaking OpenAI
Change the base URL and you are done. Same request shape, same streaming format, same client libraries. The router decides which local model answers and makes sure it is only loaded once.
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:3227/v1",
api_key="anylm-local",
)
client.chat.completions.create(
model="llama3.2",
messages=[{"role": "user", "content": "ship it"}],
stream=True,
)