Notes on using AI in Debian without installing anything or using Docker

Background and what did not work

I wished just to experiment.  I am using an nVidia 2080Ti on an X58 motherboard with an Intel i7-920 CPU.  It is an old and reliable setup.

I do not wish to install apps that alter my operating system.  This includes Docker. Additionally, Docker requires further complex setup to avoid running it as root.

FlatHub FlatPak apps are acceptable. I want to install and run AI as a separate, non-privileged user. 

Fooocus required conflicting versions of Python dependencies. Jan.ai produced a blank screen whether running from FlatPak or AppInage. LMStudio from FlatHub kept crashing. 

What did work

Manual download of Ollama

To set up Ollama on your Debian system without using Docker, follow these steps:

1. Download Ollama Software:
• cd Downloads
• mkdir ollama
• cd ollama
•  wget https://ollama.com/download/ollama-linux-amd64.tar.zst
•  tar xf ollama-linux-amd64.tar.zst 
2. Run Ollama service:
• cd bin/
• ./ollama serve
3. Check if Ollama is running by visiting:
• http://localhost:11434 
4. Download a model (run in a new terminal in the ollama/bin directory)
•  ./ollama pull deepseek-r1:7b
5. Run model and use command prompt
• ./ollama run deepseek-r1:7b
6. Or just to explore the TUI
• ./ollama 

This method is straightforward and effective for running Ollama on your system without Docker, making it a simple choice compared to other tools that encountered issues.

Experimental 

ThunderAI

ThunderAI extension downloadable from https://services.addons.thunderbird.net/EN-us/thunderbird/addon/thunderai/ 

Set an environment variable when running ollama server:

• Environment="OLLAMA_ORIGINS=moz-extension://*" ./ollama serve