Repository files navigation

Visualize arbitrary realtime data streams with just a few lines of code!

Visualizing data flows are important in a project where maintaining data streams are of priority.The abiliy to visualize data in realtime can contribute with huge advantages while debugging code or demonstrating. In this implementation we are visualizing data in charts using Chart.JS, a simple powerful library for creating charts in Javascript. The server can now also visualize image streams and heatmaps using Plotly, a modern analytics app for enterprises. This implementation utilizes Flask and is developed mainly in python3 and Javascript.

This implementation is started by running themain.py file. TheStarter will then set a timer for triggering the webbrowser after one second and then start theFlask Handler. TheFlask Handler will start the website containing aFlask-SocketIO server. TheFlask-Server will receive json objects and update theGUI listview. When theFlask-Server is started theScheduler will be triggered. TheScheduler will start the middle-man servers consisting of aTCP Socket Server, aTCP Socket Server for large files and aHTTP Server which has the purpose of receiving messages and proxy them to the flask server. In thedemo we also start some data streams with the scheduler. You basicly have two alternatives on sending data to this implementation. Either create a data stream in thescheduler or create a seperatetcp socket client orhttp client and send data to the tcp socket server while running. See examples.

See program structure image below:

1. Install program by firstly installing all the required packages in python3:Note: If you want to use docker seehere.

pip install -r requirements.txt

2. Start the implementation by running:

python3 main.py

3. If the website doesn't open automatically, open a webbrowser of your choice and go to:https://127.0.0.1:5000/

4. Edit theScheduler file and Comment thedemo() line, to make sure the demo data streams won't start.

5. Create your stream by looking at theheading below.

Example data json:

data = {  "id":1337,   "value": [1,1],   "type":"line",  "active_points": 20,  "label":"Label",  "legend": ["one", "two"],  "name":"Example",  "borderColor":["#3e95cd", "#e8c3b9"],  "backgroundColor":["#3e95cd","#e8c3b9"],  "api_crypt":"password-1"  }

Ports and addresses can be changed in theconfig.ini file.

As mentioned in thesolution heading above there are two ways of sending data streams to this implementation, creating a socket client or creating data stream in the implementation.

I created two simple example of tcp socket clients inexamples/. One that creates a data stream from live stock share prices and one that create a data stream from random numbers.

Take a closer look at thescheduler.py file, where more functions can be added to server start. In the current implementation there are two stream examples using scheduler in thedemo() function.

The first one is a seperate tcp socket client started from the scheduler. Check out the scheduler andsocket_client.py. The second one is a stream using theDataStream class. Check outdata_streams/samples.py.

To create a video feed send a json with rtsp address as value and a unique id. Example:

video_data= {            'id': 12,            'value': rtsp://192.168.0.25:554/live.sdp,            'type': 'video_stream',            'name': 'Video Stream HTTP Example',            # Crypt password from config.ini            'api_crypt':CRYPT         }

1. Build the docker image for this project by running:

docker build . --tag="JSChart-flask:1.0"

2. Run the image in background by running:

docker run -d -p 80:80 JSChart-flask:1.0

2. Or Run image in interactive mode by running:

docker run -it -p 80:80 JSChart-flask:1.0

3. Open your webbrowser of choice and go to:

http://127.0.0.1/orhttp://localhost/

Note: all ports and addresses can be changed inconfig.ini

This is the output on the console during execution.

This is how the implementation looks like during execution of thedemo().

Examples of how each chart look and how the data should be represented in json see:

• https://www.chartjs.org/samples/latest/
• https://tobiasahlin.com/blog/chartjs-charts-to-get-you-started

Flask is a micro web framework, enabling websites to be hosted in python.

Chart.JS is an opensource project with the main purpose to provide awesome charts for html5 and javascript.

Plotly is a collection of open source Graphing Libraries for visualizing data in a vast amount of formats.

See

List of known issues:

• Ignore theWebSocket transport not available. Install eventlet or gevent and gevent-websocket for improved performance. warning as eventlet doesn't support Flask-socketio! Make sure eventlet is not installed in your python environment!

The main inspiration and solutions comes from the following sources:

• https://gitlab.com/patkennedy79/flask_chartjs_example
• https://github.com/roniemartinez/real-time-charts-with-flask

A deprecated demo can be intresting to see how the application has developed from earlier versions!

These are functions that I will add

• geo tagging free api map
• 3d graphs
• 3d algoritm
• Bubble matrix
• Tree