Repository files navigation

This system produces a set of comprehensible, step-by-step instructions from video input. The videos can be of two categories: Academic Videos and Exercise Videos. To enable extracting procedures from these videos, Text Recognition, Speech Recognition and Action Recognition are used.

• python >= 3.5
• Opencv >= 3.4.1
• scikit-learn==0.22.2
• tensorflow & keras
• numpy & scipy
• pathlib
• pytesseract
• SpeechRecognition

• Download the graph models fromhere, and place it under the Pose folder.
• python main.py e videoname.mp4, willperform action and speech recognition and write the actions and speech recognized from the video toexercise_video_output.txt.
• python main.py e webcam, willperform action recognition in realtime.
• python main.py a videoname.mp4, willperform text and speech recognition and write the text and speech recognized from the video toacademic_video_output.txt.

• prepare data(actions) by runningmain.py, remember touncomment the code of data collecting, the origin data will be saved as a.txt.
• transforming the.txt to.csv, you can use EXCEL to do this.
• do the training with thetrain.py inAction/training/, remember tochange the action_enum and output-layer of model.

• Check exercise_video_output.txt for exercise videos and academic_video_output.txt for academic videos.

Thanks to the following works:

• Online-Realtime-Action-Recognition-based-on-OpenPose