Repository files navigation

A simple Flask-based web GUI that enables local AI (LLMs) inference usingollama for model serving. This project is currently inAlpha phase and open to any contributions. Created by@qusaismael.

If you'd like to support this project, consider donating via PayPal:

• Features
• System Requirements & Recommendations
• Installation
• Usage
• Security Notice
• Troubleshooting
• Project Status
• Roadmap
• Contributing
• License
• Acknowledgments
• References

• Multiple Model Support: Easily switch between different local LLM models (e.g.,deepseek-r1,qwen2.5,codellama, etc.)
• Streaming Responses: See tokens appear in real time using server-sent events (SSE)
• Markdown and Code Block Rendering: Code blocks with syntax highlighting and copy-to-clipboard
• Raw Output Toggle: Debug with raw text output visibility
• Cross-Platform: Works on Windows, Linux, and macOS
• Keyboard Shortcuts:
  • Shift+Enter: New line
  • Enter: Send message

• Python 3.7+
  Required for Flask compatibility

• pip/venv
  For dependency management and environment isolation

• ollama
  Installation required
  Verify installation:

  ollama --version

• Hardware:

  • Minimum: 8GB RAM (for smaller models)
  • Recommended: 16GB+ RAM + NVIDIA GPU (for larger models)
  • Disk Space: 10GB+ for model storage

1. Clone Repository

   git clone https://github.com/qusaismael/localllm.gitcd localllm

2. Setup Virtual Environment

   # Linux/macOSpython3 -m venv venvsource venv/bin/activate# Windowspython -m venv venvvenv\Scripts\activate

3. Install Dependencies

   pip install -r requirements.txt

4. Configure Ollama

   • Ensure Ollama is running:

     ollama serve

   • Download models first:

     ollama pull deepseek-r1:14b

1. Start Server

   python app.py

   Access athttp://localhost:5000

2. First-Time Setup

   • Select model from available options
   • If models aren't listed, ensure they're downloaded via Ollama

3. Basic Operations

   • Type prompt & press Enter to send
   • Toggle raw output for debugging
   • Copy code blocks with one click

⚠️Important Security Considerations:

• Default binding:0.0.0.0 (accessible on your network)
• Not recommended for public internet exposure
• No authentication layer implemented
• Use firewall rules to restrict access if needed

Common Issues:

1. "Model not found" error

   ollama pull<model-name>

2. Port conflictModifyPORT variable inapp.py

3. Slow responses

   • Try smaller models first
   • Check system resource usage
   • Ensure GPU acceleration is enabled if available

4. Windows path issuesUpdateOLLAMA_PATH inapp.py to your installation path

Alpha Release
Current version: 0.1.0

Known Limitations:

• No conversation history
• Basic error handling
• Limited model configuration

• Conversation history support
• Model download UI
• Docker support
• System resource monitoring

Welcome! Please follow these steps:

1. Fork repository
2. Create feature branch
3. Submit PR with description

Development Setup:

pip install -r requirements-dev.txtpre-commit install

Guidelines:

• Follow PEP8 style
• Add tests for new features
• Update documentation accordingly

MIT License - SeeLICENSE for details

• Built withFlask
• LLM backend byOllama
• Inspired by the Open Source AI community

• Ollama Documentation
• Flask Documentation
• CUDA Installation Guide

Created by@qusaismael
Open Source • Contributions Welcome!