Background and what did not work
I wish to just experiment. I am using an nVidia 2080Ti on an X58 motherboard with an Intel i7-920 CPU. It is an old and reliable setup.
I do not wish to install apps that alter my operating system. This includes Docker. Additionally, Docker requires further complex setup to avoid running it as root. Downloaded .debs are a security risk and usually block debian distribution upgrades.
FlatHub FlatPak apps are acceptable. I want to install and run AI as a separate, non-privileged user.
I am running the programmes below as a separate user to keep them sandboxed
What did not work
- Fooocus required conflicting versions of Python dependencies.
- Jan.ai produced a blank screen, whether running from a FlatPak or AppInage.
- StabilityMatrix kept crashing.
- Open Web UI in python - needs an older version of Python (3.11)
- Open WebUI desktop - the download link for the AppImage is broken
What does work
Manual download of Ollama
To set up Ollama on your Debian system without using Docker, follow these steps:
- Download Ollama Software:
- cd Downloads
- mkdir ollama
- cd ollama
- wget https://ollama.com/download/ollama-linux-amd64.tar.zst
- tar xf ollama-linux-amd64.tar.zst
- Run Ollama service:
- cd bin/
- ./ollama serve
- Check if Ollama is running by visiting:
- http://localhost:11434
- Download a model (run in a new terminal in the ollama/bin directory)
- ./ollama pull deepseek-r1:7b
- ./ollama pull llama3.2-vision:11b
- Run model and use command prompt
- ./ollama run deepseek-r1:7b
- Or just to explore the TUI
- ./ollama
- According to Google Gemini, I can ./ollama run llama3.2-vision 'Then, simply drag and drop your image file into the terminal window and type your question.' This needs testing.
This method is straightforward and effective for running Ollama on your system without Docker, making it a simple choice compared to other tools that encountered issues.
ThunderAI
ThunderAI extension downloadable from https://services.addons.thunderbird.net/EN-us/thunderbird/addon/thunderai/
Set an environment variable when running ollama server:
- OLLAMA_ORIGINS=moz-extension://* ./ollama serve
Gemma 2 may be a better model for this purpose
- ./ollama pull gemma2:9b
Easy Diffusion
Easy Diffusion has worked. I used its start.sh script.
Alpaca
Alpaca installed from flatpak.Recommended vision-capable model:
- /ollama pull llama3.2-vision:11b
Experimental
- AnythingLLM looks good: https://anythingllm.com/
- Github incognide looks fascinating, but is a .deb
- Page assist: https://github.com/n4ze3m/page-assist?ref=itsfoss.com - no, I want to upload documents.
