Background and what did not work
I wish to just experiment. I am using an nVidia 2080Ti on an X58 motherboard with an Intel i7-920 CPU. It is an old and reliable setup.
I do not wish to install apps that alter my operating system. This includes Docker. Additionally, Docker requires further complex setup to avoid running it as root.
FlatHub FlatPak apps are acceptable. I want to install and run AI as a separate, non-privileged user.
I am running the programmes below as a separate user to keep them sandboxed
What did not work
- Fooocus required conflicting versions of Python dependencies.
- Jan.ai produced a blank screen, whether running from a FlatPak or AppInage.
- StabilityMatrix kept crashing.
- Open Web UI in python - needs an older version of Python (3.11)
- Open WebUI desktop - the download link for the AppImage is broken
What does work
Manual download of Ollama
To set up Ollama on your Debian system without using Docker, follow these steps:
- Download Ollama Software:
- cd Downloads
- mkdir ollama
- cd ollama
- wget https://ollama.com/download/ollama-linux-amd64.tar.zst
- tar xf ollama-linux-amd64.tar.zst
- Run Ollama service:
- cd bin/
- ./ollama serve
- Check if Ollama is running by visiting:
- http://localhost:11434
- Download a model (run in a new terminal in the ollama/bin directory)
- ./ollama pull deepseek-r1:7b
- Run model and use command prompt
- ./ollama run deepseek-r1:7b
- Or just to explore the TUI
- ./ollama
This method is straightforward and effective for running Ollama on your system without Docker, making it a simple choice compared to other tools that encountered issues.
ThunderAI
ThunderAI extension downloadable from https://services.addons.thunderbird.net/EN-us/thunderbird/addon/thunderai/
Set an environment variable when running ollama server:
- OLLAMA_ORIGINS=moz-extension://* ./ollama serve
Gemma 2 may be a better model for this purpose
- ./ollama pull gemma2:9b
Easy Diffusion
Easy Diffusion has worked. I used its start.sh script.
Experimental
Page assist: https://github.com/n4ze3m/page-assist?ref=itsfoss.com - no, I want to upload documents.AnythingLLM looks good: https://anythingllm.com/
