Plans Sign In Register
DMJBot - Personal AI Assistant

Local Models over the Bridge

Wiki

Local Models over the Bridge

DMJBot normally talks to a model provider over the internet — OpenAI, Anthropic, Google, DeepSeek. Device Models is the alternative: the model runs on your machine, and the DMJBot that uses it can live anywhere.

That gives you a combination that was previously impossible: a bot that is online 24/7 on a server, answering from a model that never leaves your desk.

When this is for you

  • You already run Ollama on a laptop, desktop or home server.
  • You want a permanently available bot (server, cloud, mobile app) rather than one that only works while your laptop is open.
  • You do not want the text of certain conversations sent to a model vendor.
  • You have a capable machine and would rather use it than pay per token.

Keep in mind that the answers are only as good — and as fast — as the model your machine can run, and that the machine has to be awake and online for the bot to use it. Most people keep a cloud model as the primary one and switch to the local one when it matters, or set one as the other's fallback.

How it works

  1. The bridge you already installed to connect the machine as a device opens a second connection to your bot.
  2. Over that connection it tells the bot which models are installed on the machine.
  3. When you chat, the bot sends the request down that connection; the bridge passes it to the Ollama running locally and sends the answer back.

Nothing new needs to be opened on your firewall or router: the bridge connects out to the bot, exactly like the device connection does. Your Ollama is never exposed to the internet. (If you point the bridge at an Ollama on a different machine, that one has to be reachable over your local network — see the "Ollama on a different machine" section below.)

Setting it up

1. Install and run Ollama on your machine

Follow ollama.com, then pull a model:

ollama pull qwen3:8b
ollama list

Pick a model that supports tools — DMJBot uses tool calling for most of its work, and a model without it can only chat.

If you also want the bot to look at screenshots and photos, pull a vision model (llava, llama3.2-vision, qwen2.5vl, gemma3 …). The bridge asks Ollama which of your models can read images and tells the bot, so pictures are only ever sent to one that can.

2. Connect the machine as a device

If you have not already: install the bridge on that machine and connect it in Settings → Devices. See the Bridge page.

3. Share the Ollama

On the machine:

dmjbot-bridge enable-ollama

That checks the standard address http://127.0.0.1:11434 and lists the models it finds. If your Ollama listens somewhere else — a different port, or another machine altogether — pass the address, and a login if it sits behind a protected proxy:

dmjbot-bridge enable-ollama http://127.0.0.1:11500
dmjbot-bridge enable-ollama http://192.168.0.5:11434
dmjbot-bridge enable-ollama 192.168.0.5:11434       # http:// is assumed
dmjbot-bridge enable-ollama https://ollama.example.com --username me --password secret

Restart the bridge so the link comes up:

dmjbot-bridge service stop && dmjbot-bridge service start

Check it any time with dmjbot-bridge status — it prints the shared address, whether that address is answering right now, and the model list.

If the machine cannot reach the address yet but you want to save the setting anyway, add --skip-verify. The setting is also kept in the bridge's config.json, in an ollama block with enabled, host, username and password, so you can edit it by hand and restart instead of re-running the command.

4. Enable the provider in the bot

Open Settings → Models, find Device Models, and press Enable. There is nothing else to fill in — no key, no address. The card then shows how many devices are sharing an Ollama and how many models they offer.

5. Choose the model

The model selectors on the same page now include a group per device, named after the device. Pick a model, press Save Models, and the next message is answered by your machine.

You can also see the shared Ollama and its model list on the device's card in Settings → Devices, where the vision-capable ones are marked "reads images".

Ollama on a different machine

The bridge and Ollama do not have to be on the same computer. A common setup is a small always-on machine running the bridge and a workstation or GPU box running the models; only the bridge machine has to be connected to the bot as a device, and it relays to whatever address you gave enable-ollama.

Two things to know before you point the bridge at another host.

Ollama must listen beyond its own loopback. By default it only accepts connections from the machine it runs on, so another computer gets "connection refused" no matter which address you pass. On the Ollama machine, start it with:

OLLAMA_HOST=0.0.0.0:11434 ollama serve

For a background service, set the same variable where the service is defined — on Linux systemctl edit ollama.service with Environment="OLLAMA_HOST=0.0.0.0:11434", on macOS launchctl setenv OLLAMA_HOST "0.0.0.0:11434" before restarting the app. Then check it from the bridge machine before running enable-ollama:

curl http://192.168.0.5:11434/api/tags

The privacy picture changes slightly. With Ollama on the same machine as the bridge, the prompt never leaves that machine. With Ollama on another host, the prompt travels from the bridge machine to that host across your network — as plain HTTP unless you put the Ollama behind an HTTPS proxy and use the https:// address. That is usually fine on a home or office network, and it is still a world away from sending the text to a model vendor, but be deliberate about it: keep that port on the local network and never forward it from your router. The link to your bot is unaffected — still one outbound connection from the bridge.

Images

Uploading a picture works the same as with a hosted provider: attach it to a message and ask about it. The bot fetches the image and sends it to a vision-capable model — so with a vision model on your machine selected, the screenshot is analysed locally and never leaves it.

If the model you picked cannot read images, the bot says so instead of quietly answering without looking: nothing is dropped silently.

Using several machines

Every connected device can share its own Ollama. Each appears as its own group in the model list, so you can, for example, keep a small fast model on a laptop and a large one on a workstation and switch between them — or set one as the fallback for the other.

During first setup

You can pick Device Models as the provider in the setup wizard, but you cannot pick a model there: devices can only be connected once setup is finished. The wizard lets you continue without one, and the bot then shows a "No model selected" notice and keeps the chat box disabled until you connect a device and choose a model (or configure another provider).

If something is not working

The model list is empty. Check on the machine that Ollama is running and has models: ollama list. Then dmjbot-bridge status — it shows whether the bridge is running and whether it can reach Ollama.

The device shows "Not shared". enable-ollama was not run on that machine, or the bridge was not restarted after it.

"Connection refused" for an Ollama on another host. Almost always Ollama listening on loopback only. Start it with OLLAMA_HOST=0.0.0.0:11434 on that machine and confirm with curl http://<host>:11434/api/tags from the bridge machine. If the curl works but enable-ollama does not, a firewall is allowing your shell but not the bridge's user, or the address you saved differs — dmjbot-bridge status prints the one in use.

"No configured model supports image input". The model you selected is text-only. Pull a vision model (ollama pull llava), restart the bridge, and pick it — or keep a hosted vision model in one of the fallback slots.

Answers stop halfway or the model ignores tools. Small models often struggle with tool calling and long contexts. Try a larger model, or one documented to support tools.

Everything is slow. The generation happens on your machine; a model too large for its memory will crawl. Try a smaller variant.

See also