Repository files navigation

SinceGoogle Summer of Code 2018, FFMpeg supportsthesrfilter for applying super-resolution methods based on convolutional neuralnetworks. However, compiling FFMpeg with proper libraries and preparing modelsfor super-resolution requires expert knowledge. This repository provides aDockerfile that makes super-resolution in FFMpeg a breeze!

• Docker: For running containerized FFMpeg with super-resolution support.
• NVIDIA Container Toolkit: For GPU acceleration.

Simply clone the repo and build the container with the following commands:

git clone https://github.com/MIR-MU/ffmpeg-tensorflow.gitdocker build --compress --no-cache --force-rm --squash -t miratmu/ffmpeg-tensorflow ffmpeg-tensorflow/

If you wish to use different versions of Libtensorflow, FFMpeg, CUDA, or Ubuntuyou can also build a customized container. Keep in mind that your version ofLibtensorflow (here1.15.0) should match your version of CUDA (here ≥ 10.0),seethe compatibility table. Your version of CUDAshould match your NVIDIA driver, seeNVIDIA CUDA Toolkit Release Notes, Table2:

docker build --compress --no-cache --force-rm --squash --build-arg VERSION_LIBTENSORFLOW=1.15.0 --build-arg VERSION_CUDA=10.0-cudnn7 --build-arg VERSION_UBUNTU=18.04 --build-arg VERSION_FFMPEG=4.3.1 -t miratmu/ffmpeg-tensorflow ffmpeg-tensorflow/

You should now seemiratmu/ffmpeg-tensorflow among your Docker images:

docker images

You can now remove the downloaded during the installation:

rm -rf ffmpeg-tensorflow/

If you skip this section, a pre-build Docker image will be downloaded fromDocker Hub.

Download anexample video and run themiratmu/ffmpeg-tensorflowdocker image to upscale it using one of the super-resolution models (hereESPCN):

wget https://media.xiph.org/video/derf/y4m/flower_cif.y4malias ffmpeg-tensorflow='docker run --rm --gpus all -u $(id -u):$(id -g) -v "$PWD":/data -w /data -it miratmu/ffmpeg-tensorflow'ffmpeg-tensorflow -i flower_cif.y4m -filter_complex'[0:v] format=pix_fmts=yuv420p, extractplanes=y+u+v [y][u][v]; [y] sr=dnn_backend=tensorflow:scale_factor=2:model=/models/espcn.pb [y_scaled]; [u] scale=iw*2:ih*2 [u_scaled]; [v] scale=iw*2:ih*2 [v_scaled]; [y_scaled][u_scaled][v_scaled] mergeplanes=0x001020:yuv420p [merged]' -map [merged] -sws_flags lanczos -c:v libx264 -crf 17 -c:a copy -y flower_cif_2x.mp4

Theflower_cif_2x.mp4 file with the upscaled example video should be produced.Compare upscaling using Lanczos filtering (left) with upscaling using the ESPCNsuper-resolution model (right):

Besides ESPCN, the docker image includes pre-trained SRCNN, VESPCN, and VSRNETmodels in the/models directory. Thearchitectures andexperimental results for the super-resolution results aredescribed in theHighVoltageRocknRoll/sr GitHub repository.