Repository files navigation

Python-Devops, gathers Python stack for DevOps, these are usually my basic templates use for my implementations, so, feel free to use it and evolve it! Everything is Docker!

• Basic Python
• Basic Backend
• Scaling-up Backend
• Big data piping
• Apache big data
• Unit test
• Stress test
• Miscellaneous

1. AutoPEP8
2. Graph function dependencies

1. Flask
2. Flask with MongoDB
3. REST API Flask
4. Flask Redis PubSub
5. Flask Mysql with REST API
6. Flask Postgres with REST API
7. Flask Elasticsearch
8. Flask Logstash with Gunicorn
9. MLFlow with Nginx reversed proxy

1. Flask SocketIO with Redis
2. Multiple Flask with Nginx Loadbalancer
3. Multiple Flask SocketIO with Nginx Loadbalancer
4. RabbitMQ and multiple Celery with Flask
5. Flask + Gunicorn + HAproxy

1. Streaming Tweepy to Elasticsearch
2. Scheduled crawler using Luigi Spotify to Elasticsearch
3. Airflow to Elasticsearch

1. Flask with Hadoop
2. Flask with Kafka
3. Flask with Hadoop Hive
4. PySpark with Jupyter and Hadoop
5. Flink with Jupyter (not working)
6. Apache Storm with Redis

1. Pytest

1. Locust

1. Elasticsearch + Kibana
2. Elasticsearch + Cerebro
3. Jupyter notebook
4. Jupyterhub
5. Jupyterhub + Github Auth
6. Hadoop cluster + Luigi + Jupyter Notebook
7. Kafka cluster
8. Apache Storm

Every folders contain .yml for docker-compose. You need to install Docker-Compose first.

To run,

compose/build

To safely close

compose/down

To remove all images

docker rmi$(docker images -q)

To removenone images

docker rmi$(docker images -f"dangling=true" -q)

To remove all containers

docker rm$(docker ps -aq)