Repository files navigation

LXC AutoScale ML is a resource management daemon specifically designed for Proxmox environments. It automatically monitor resources and adjusts CPU and memory allocations with no downtime based on real-time usage metrics and predefined thresholds with the help of a machine learning pipeline.

• ✅ Works withProxmox 8.2.4

LXC AutoScale ML in action :)

2024-08-20 13:07:56,393 [INFO] Data loaded successfully from /var/log/lxc_metrics.json.2024-08-20 13:07:56,399 [INFO] Data preprocessed successfully.2024-08-20 13:07:56,416 [INFO] Feature engineering, spike detection, and trend detection completed.2024-08-20 13:07:56,417 [INFO] Features used for training: ['cpu_memory_ratio', 'cpu_per_process', 'cpu_trend', 'cpu_usage_percent', 'filesystem_free_gb', 'filesystem_total_gb', 'filesystem_usage_gb', 'io_reads', 'io_writes', 'max_cpu', 'max_memory', 'memory_per_process', 'memory_trend', 'memory_usage_mb', 'min_cpu', 'min_memory', 'network_rx_bytes', 'network_tx_bytes', 'process_count', 'rolling_mean_cpu', 'rolling_mean_memory', 'rolling_std_cpu', 'rolling_std_memory', 'swap_total_mb', 'swap_usage_mb', 'time_diff']2024-08-20 13:07:56,549 [INFO] IsolationForest model training completed.2024-08-20 13:07:56,549 [INFO] Processing containers for scaling decisions...2024-08-20 13:07:56,600 [INFO] Applying scaling actions for container 104: CPU - Scale Up, RAM - Scale Up | Confidence: 87.41%2024-08-20 13:07:57,257 [INFO] Successfully scaled CPU for LXC ID 104 to 4 CPU units.2024-08-20 13:07:57,916 [INFO] Successfully scaled RAM for LXC ID 104 to 8192 RAM units.2024-08-20 13:07:57,916 [INFO] Sleeping for 60 seconds before the next run.

• Overview
• Key Features
• System Requirements
• Installation
• Components Overview
  • API Component
  • Monitor Component
  • Model Component
• Usage and Control
• Monitoring and Alerts
• Documentation
• Uninstallation
• Contributing
• License

LXC AutoScale ML is a tool designed to manage LXC containers on Proxmox hosts using machine learning for automatic scaling. It dynamically adjusts container resources to maintain optimal performance and efficient resource utilization.

• Proxmox Integration: Seamless integration with Proxmox hosts via API and CLI.
• ML-Driven Autoscaling: Utilizes machine learning models to predict and respond to resource demands.
• Modular Architecture: Components (API, Monitor, Model) designed to handle specific autoscaling tasks.
• Customizable Policies: Define custom scaling rules and thresholds.
• Real-Time Monitoring and Logging: Provides detailed logs for auditing and optimization.

• Proxmox Host: Version 6.x or higher
• Operating System: Linux (Debian-based preferred)
• Python: Version 3.x
• Dependencies:

  git, python3-flask, python3-requests, python3-sklearn, python3-pandas, python3-numpy, python3-aiofiles

To installLXC AutoScale ML, execute the following command:

curl -sSL https://raw.githubusercontent.com/fabriziosalmi/proxmox-lxc-autoscale-ml/main/install.sh| bash

• Root or Sudo Privileges: Ensure the command is executed with appropriate privileges.
• Internet Access: Required on the Proxmox server for downloading files and packages.

The installation script will:

1. Verify System Requirements: Ensures all necessary packages and dependencies are present.
2. Download & Set Up: Retrieves the required files and configures services.
3. Enable Services: Starts the API, monitoring, and ML model services.

[Unit]Description=FUSE filesystem for LXCConditionVirtualization=!containerBefore=lxc.serviceDocumentation=man:lxcfs(1)[Service]OOMScoreAdjust=-1000ExecStartPre=/bin/mkdir -p /var/lib/lxcfs# ExecStart=/usr/bin/lxcfs /var/lib/lxcfsExecStart=/usr/bin/lxcfs /var/lib/lxcfs -lKillMode=processRestart=on-failureExecStopPost=-/bin/fusermount -u /var/lib/lxcfsDelegate=yesExecReload=/bin/kill -USR1 $MAINPID[Install]WantedBy=multi-user.target

TheAPI provides RESTful endpoints for managing autoscaling services.

• Scaling Operations: Trigger container scaling manually.
• Configuration Management: Dynamically update scaling configurations.
• Monitoring and Health Checks: Access real-time metrics and system status.
• Audit Logging: Logs all API interactions for security purposes.

TheMonitor service continuously tracks the performance and resource usage of LXC containers.

• Real-Time Metrics Collection: Collects CPU, memory, disk, and network usage statistics.
• Anomaly Detection: Detects unusual patterns in resource usage.
• Threshold Alerts: Triggers alerts or scaling actions when predefined thresholds are exceeded.
• Data Aggregation: Aggregates metrics for analysis and reporting.

TheModel uses machine learning algorithms to analyze metrics and predict scaling needs.

• Anomaly Detection: UsesIsolationForest to detect unusual usage patterns.
• Predictive Scaling: Forecasts when scaling actions are necessary.
• Adaptive Learning: Continuously refines predictions based on new data.
• Customizable Models: Supports various ML algorithms and configurations.

Manage the autoscaling services with the following commands:

• Check Status:

  systemctl status lxc_autoscale_api.servicesystemctl status lxc_monitor.servicesystemctl status lxc_autoscale_ml.service

• Start/Stop Services:

  systemctl start lxc_autoscale_api.servicesystemctl stop lxc_monitor.servicesystemctl restart lxc_autoscale_ml.service

• Metrics Dashboard: Integrate with tools like Grafana or Prometheus for visualization.
• Alerting: Configure alerts for critical events, such as spikes in CPU or memory usage.

For detailed documentation, including advanced usage, configuration options, and troubleshooting, please refer to theExtended Documentation.

To uninstallLXC AutoScale ML, execute the following command:

curl -sSL https://raw.githubusercontent.com/fabriziosalmi/proxmox-lxc-autoscale-ml/main/uninstall.sh| bash

We welcome contributions! Please follow these steps:

1. Fork the repository.
2. Create a new branch (git checkout -b feature/your-feature).
3. Commit your changes (git commit -m 'Add your feature').
4. Push to the branch (git push origin feature/your-feature).
5. Open aPull Request.

If You like my projects, you may also like these ones:

• caddy-waf Caddy WAF (Regex Rules, IP and DNS filtering, Rate Limiting, GeoIP, Tor, Anomaly Detection)
• patterns Automated OWASP CRS and Bad Bot Detection for Nginx, Apache, Traefik and HaProxy
• blacklists Hourly updated domains blacklist 🚫
• proxmox-vm-autoscale Automatically scale virtual machines resources on Proxmox hosts
• UglyFeed Retrieve, aggregate, filter, evaluate, rewrite and serve RSS feeds using Large Language Models for fun, research and learning purposes
• proxmox-lxc-autoscale Automatically scale LXC containers resources on Proxmox hosts
• DevGPT Code togheter, right now! GPT powered code assistant to build project in minutes
• websites-monitor Websites monitoring via GitHub Actions (expiration, security, performances, privacy, SEO)
• caddy-mib Track and ban client IPs generating repetitive errors on Caddy
• zonecontrol Cloudflare Zones Settings Automation using GitHub Actions
• lws linux (containers) web services
• cf-box cf-box is a set of Python tools to play with API and multiple Cloudflare accounts.
• limits Automated rate limits implementation for web servers
• dnscontrol-actions Automate DNS updates and rollbacks across multiple providers using DNSControl and GitHub Actions
• csv-anonymizer CSV fuzzer/anonymizer
• iamnotacoder AI code generation and improvement

This project is licensed under the MIT License. See theLICENSE file for more details.