Repository files navigation

A Node-Based Image, Video and Data Processing Tool

• Overview
• Installation
  • Using the Installer
  • Building from Source
• Quick Start
  • User
  • Developer
• References

Graph-ICS is a tool for visualizing and creating image, video, and data streams. A stream always consists of an input and a series of filter nodes. Graph-ICS provides out-of-the-box usage of many filtering algorithms from the libraries ITK,OpenCV, andQt. Up to 10 different streams can be processed and displayed simultaneously. A core concept of Graph-ICS is to give its users the freedom to easily extend the available nodes for their desired use-case.

• Go to theReleases section and download the installer for your platform.
• You can find an installer forWindows,MacOS, andLinux.
• Extract the content and run the executable.

The linux binary for Graph-ICS was built and tested on Ubuntu 18.04 LTS. According tolinuxdeployqt the binary should also work on newer systems and different distributions.

I tried running the binary onEndeavourOS (Arch-based), but didnot get it to work.

The distributed mac binary was built on macOSX High Sierra 10.13.6 and cannot run on older systems.

If the Graph-ICS binary doesn't run on your system you can alwaysbuild from source!

If you encounter a problem anywhere in the process look upTroubleshooting! Maybe the problem has already been solved for you :)

Worth Mentioning:

The guide walks thorugh the steps on how to build Graph-ICS for theDebug configuration. You just need to replaceDebug withRelease in all following commands to produce theRelease configuration.

• Install Git
• Install Build Tools
• Install CMake
• Install Qt Library and QtCreator
• Build OpenCV
• Build ITK
• Build Graph-ICS
• Develop via QtCreator
• Plugins, Examples and Documentation (optional)
• Troubleshooting

• Download and install Git from theofficial website or via your package manager.

Windows:

• Download and install Visual Studio Community 2019 fromhere.
• Install Windows 10 SDK for C++ using the Visual Studio Installer.
• Make sure the compiler tools are in yourPATH.
• e.g.C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64

Linux:

• Installgcc,g++ andmake packages.

e.g. onUbuntu use:

$ sudo apt install gcc g++ make

MacOS:

• Download and install XCode using the AppStore.

• Go to the officialCMake download website.
• Select the binary distributions of the latest relase for your platform.
• Make sure you have a CMake version greater than3.14 installed.

Note: On Linux you can also install CMake via your package manager.

Make surecmake runs from the commandline and check your version:

$ cmake --version

• Download theQt Online Installer for open-source.
• We needQt 5.15.2.
• Make sure to install the binaries for the compiler you use (e.g. on Windows using Visual Studio Community 2019:MSVC2019, on Linux/MacOS:Desktop gcc).
• Make sure to install all Qt Modules to have no dependency issues.

Windows:

• Additionally installQt Creator CDB Debugger Support andDebugging Tools for Windows.

• Open a commandline window.
• cd to a location where you want to store the library (e.g.C:/Libraries/) and execute the following commands.
• Pay close attention to the commands and execute them in the specified order!

Do not choose a path wich is too long! CMake paths are restricted to 50 characters by default.

$ mkdir opencv&&cd opencv$ git clone https://github.com/opencv/opencv.git src&&cd src$ git checkout tags/4.5.5 -b 4.5.5&&cd ..$ mkdir build-4.5.5&&cd build-4.5.5

In the following you will see the-j8 option. This is to speed up the build by a large factor. Replace8 with the number of threads your CPU supports.

Inside the/opencv/build-4.5.5 directory execute:

Windows:

$ cmake ../src -DWITH_FFMPEG=ON$ cmake --build. -j8 --config Debug

Linux:

Before configuring OpenCV you need to make sure to have all needed packages installed.

To install the dependencies on Ubuntu run:

$ sudo apt install pkg-config ffmpeg libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libavresample-dev mesa-common-dev libglu1-mesa-dev

$ mkdir Debug&&cd Debug$ cmake ../../src -DCMAKE_BUILD_TYPE=Debug -DWITH_FFMPEG=ON

Make sure that OpenCV findsFFMPEG and the output in theVideo I/O section looks something like this:

Video I/O:--     DC1394:                      NO--     FFMPEG:                      YES--       avcodec:                   YES (57.107.100)--       avformat:                  YES (57.83.100)--       avutil:                    YES (55.78.100)--       swscale:                   YES (4.8.100)--       avresample:                YES (3.7.0)

cmake --build. -j8

MacOS:

$ mkdir Debug&&cd Debug$ cmake ../../src -DCMAKE_BUILD_TYPE=Debug$ cmake --build. -j8

• Only configure and start the build after OpenCV has finished.
• I know the build process takes long but you can grab a hot drink ☕ and enjoy some fresh air while waiting :)
• The next steps will seem familiar to you.
• Open a commandline window.
• cd to a location where you want to store the library and execute the following commands.
• Pay close attention to the commands and execute them in the specified order!

Do not choose a path wich is too long! CMake paths are restricted to 50 characters by default.

$ mkdir itk&&cd itk$ git clone https://github.com/InsightSoftwareConsortium/ITK.git src&&cd src$ git checkout tags/v5.2.1 -b v5.2.1&&cd ..$ mkdir build-v5.2.1&&cd build-v5.2.1

Make sure to adjust the path for theOpenCV_DIR option depending on your OpenCV build directory.

ITK has a known issue inv5.2.1 regarding theVideoBridgeOpenCV module and the testing of it. ITK will only compile successfully without testing so we need to specify the-DBUILD_TESTING=OFF option to build without errors.

Inside the/itk/build-v5.2.1 directory execute:

Windows:

$ cmake ../src -DOpenCV_DIR=<your path to>/opencv/build-4.5.5 -DModule_ITKVideoBridgeOpenCV=ON -DBUILD_TESTING=OFF$ cmake --build. -j8 --config Debug

Linux and MacOS:

$ mkdir Debug&&cd Debug$ cmake ../../src -DCMAKE_BUILD_TYPE=Debug -DOpenCV_DIR=<your path to>/opencv/build-4.5.5/Debug -DModule_ITKVideoBridgeOpenCV=ON -DBUILD_TESTING=OFF$ cmake --build. -j8

• Open a commandline window andcd to the directory where you want to store Graph-ICS.

$ git clone https://github.com/Graph-ICS/Graph-ICS.git&&cd Graph-ICS$ mkdir build&&cd build

Inside the/Graph-ICS/build directory execute:

$ mkdir Debug&&cd Debug

Make sure to adjust the path for theQt5_DIR andITK_DIR options depending on your Qt and ITK-build directories.

Windows:

We need to specify theCMAKE_BUILD_TYPE to automatically place shared libraries into the directory with the Graph-ICS executable. We also keepDebug andRelease build in seperate directories in contrast to building OpenCV and ITK (on Windows) to import the build easier into QtCreator.

$ cmake ../.. -DCMAKE_BUILD_TYPE=Debug -DQt5_DIR=<your path to>/Qt/5.15.2/<compiler>/lib/cmake/Qt5 -DITK_DIR=<your path to>/itk/build-v5.2.1$ cmake --build. -j8 --config Debug

Linux and MacOS:

$ cmake ../.. -DCMAKE_BUILD_TYPE=Debug -DQt5_DIR=<your path to>/Qt/5.15.2/<compiler>/lib/cmake/Qt5 -DITK_DIR=<your path to>/itk/build-v5.2.1/Debug$ cmake --build. -j8

• Start QtCreator and selectOpen Project.
• Find your Graph-ICS source code and select theCMakeLists.txt.
• Deselect all kits.
• SelectImport Build From, enter the location of yourDebug build and clickImport.
• ClickConfigure Project.
• Now you should be able to develop Graph-ICS using the QtCreator.

• Plugins
• Examples
• Documentation

Many errors initkOpenCV...BridgeTest.cxx files?

• I had this issue on Linux and MacOS.
• Building ITK without testing worked for me.
• Append-DBUILD_TESTING=OFF to the cmake configuration and rebuild.

CMake can't find Qt?

• Try deleting all Qt related CMake cache entries (or your whole build directory) and settingQt5_DIR cache variable to:your path to/Qt/5.15.2/your compiler/lib/cmake/Qt5

QtCreator shows errors in C++ code after importing the build but compiles without a problem?

• Try disabling the QtCreator pluginClangCodeModel (worked for me).
• I found the solution in thisstackoverflow post.

• UI Tour
• Adding Nodes
• Node Structure
• Tasks and Views
  • Edit your Video
  • Save your Image or Video
• Save your Configurations
• Customize your Favoritesbar
• Node Overview

1: Searchpanel

• Here you can find all available nodes and query them using the searchbar at the top.
• Drag & drop, doubleclick or use the context menu to create a node (more inAdding Nodes).

2: Tasks

• This area is used to control your defined tasks.
• Read more about Tasks inTasks and Views.

3: Messages

• Warnings and messages are shown in themessages-tab.

4: Canvas

• All created nodes will be placed inside the canvas.
• Here you can connect and arrange your nodes to your liking.
• You can use standard editing methods like copy, paste, remove using shortcuts or themenubar'sedit menu

5: Favoritesbar

• All your favorite nodes in one place, for quick-access.

6: Viewarea

• The output of a stream will be shown in one of the views.
• Connect the output of a node with the view where you want to display your stream.
• You can add and remove views using the buttons above (+) and below (-).
• More about views inTasks and Views.

There are several ways to create nodes in Graph-ICS. Nodes can be created in the same way using thesearchpanel or thefavoritesbar. You can create nodes via:

• Doubleclick
• Context menu
• Drag & drop

• Drag & drop a file

Input nodes (Image andVideo) canalso be created by dragging and dropping a file (e.g.jpg ormp4) from the file explorer (or desktop) into thecanvas.

A node can be either aninput or afilter. Available inputs areImage,Video orCamera.Image andVideo represent a file on your system.Camera represents a camera device on your system.

A node can have any number of input and output ports (not practical, but possible). Ports can be connected with each other to represent a data flow from one node to the other. Ports always indicate the transfered data with an icon (image or data).

Nodes have attributes to change the parameters of for example a filtering algorithm. You can change the values and check the processed result right after.

Tasks andViews take a special role in the processing of streams. A stream can only be processed if the output is connected to a view. That is when tasks come into play. A task gets created as soon as the output of a node is connected with a view. The type of the task depends on the input node of the stream. For example aVideo node as input will always lead to the creation of a video task. The same can be applied toCamera andImage nodes.

Each view-task relation can be identified by a color.

After defining your tasks you can control and start processing the stream. You can control your tasks using the global or task specific options. Additionally you can select a view and use theRun menu in themenubar or the context menu in theview to control your task.

Views can be added and removed using the buttons above (+) and below (-) theviewarea. The views to date arehighly inflexible andimpractical. This topic will be adressed in a future release!

Videos or camera recordings can be edited (cut, copy, paste) by clicking theedit button (pencil icon) at the right side of a video or camera task.

You can select a range to cut out using the range slider below and select a position to paste frames using the slider above the regular frame slider. Use the buttons at the top to execute or undo/redo your editing.

The context menu in aview can be used to save your image or video to a file. Also you will be asked to save your video if you recorded a camera stream or edited your video so you don't accidentally loose your progress.

A node setup with all attribute values and connections can be saved to a config file (.gconfig) to persist your configuration work. You can use standard keyboard shortcuts or thefile menu in themenubar to interact with a configuration.

To customize yourfavoritesbar you can use the context menu in thesearchpanel or a node. Also you canpress and hold an item in thesearchpanel and drag & drop into thefavoritesbar. Use the context menu orpress and hold an item in thefavoritesbar to move it. Remove nodes from the favorites using the context menu.

Input:

Filter:

Every contribution is appreciated! Make sure to read thecontributing guide first.

This guide assumes you already have builtGraph-ICS from source.

• Adding a New Filter Node
  • Creating the Class Files using QtCreator
  • Implementing the Filter Class
  • Registering the Filter
  • Creating a Custom Front-End (optional)

Graph-ICS aims to keep an open and easy interface to add new filter nodes. The following guide shows the steps that need and also could be done to add a new filter node. The example shows the creation of theCvSobelOperator node with a custom front-end.

QtCreator quickly generates a nice starting point for filter development and saves you much time writing the class definition.

• Right-click onapp/src/model/opencv and selectAdd New... .
• SelectC++,C++ Class and click onchoose.
• Enter a class name, hereCvSobelOperator.
• AddNode as the custom base class and selectAdd Q_OBJECT.
• Clicknext, thenfinish.
• Right-click on the project root and selectRun CMake (or from the menuBuild/Run CMake)

Header File:

#ifndef CVSOBELOPERATOR_H#defineCVSOBELOPERATOR_H#include"node.h"// Add to the Graph-ICS NamespacenamespaceG {// Derive from NodeclassCvSobelOperator :publicNode{// Important Q_OBJECT Macro    Q_OBJECTpublic:explicitCvSobelOperator();// Pure virtual function// Filter functionality needs to go in hereboolretrieveResult()override;// Virtual function// Define special behavior if an attribute value changedvoidonAttributeValueChanged(Attribute* attribute)override;private:// Declare your in- and out-ports// You can have as many as you want and need    Port m_inPort;    Port m_outPort;// Declare the attributes// CvSobelOperator has two attributes to specify the edge detection direction    Attribute* m_xDerivative;    Attribute* m_yDerivative;};}// namespace G#endif// CVSOBELOPERATOR_H

Source File:

#include"cvsobeloperator.h"#include<cv.h>#include<opencv2/imgproc/imgproc.hpp>// Wrap the definition into the namspace to avoid writing G::Foo everywherenamespaceG{CvSobelOperator::CvSobelOperator()// !!!Pass the classname to the Node constructor!!!    : Node("CvSobelOperator")// Initialize the ports with a supported datatype (GIMAGE or GDATA)    , m_inPort(Port::TYPE::GIMAGE)    , m_outPort(Port::TYPE::GIMAGE)// The Attribute class is really generiy and can be customized to your liking// attributeFactory is a prototype factory for predefined Attribute instances// For simplicity we just use the predefined IntTextField-Attribute    , m_xDerivative(attributeFactory.makeIntTextField(1,0,2,1,"x-Derivative"))    , m_yDerivative(attributeFactory.makeIntTextField(0,0,2,1,"y-Derivative")){// Register your in-and out-ports to the port systemregisterPorts({&m_inPort}, {&m_outPort});// Register your attributes to the attribute systemregisterAttribute("xDerivative", m_xDerivative);registerAttribute("yDerivative", m_yDerivative);}boolCvSobelOperator::retrieveResult(){try    {// We know that the in-port has an image when entering this function        cv::Mat& cvImage = m_inPort.getGImage()->getCvMatImage();// We need to make sure the image has only one channelif (cvImage.channels() >1)        {cv::cvtColor(cvImage, cvImage, CV_BGR2GRAY);        }// Retrieve the values from the attributes to use them in the filteringint xDerivative = m_xDerivative->getValue().toInt();int yDerivative = m_yDerivative->getValue().toInt();// Do the filteringcv::Sobel(cvImage, cvImage, CV_8U, xDerivative, yDerivative);// Forward the result to the out-port        m_outPort.getGImage()->setImage(cvImage);    }catch (int e)    {qDebug() <<"CvSobelOperator. Exception Nr." << e <<'\n';returnfalse;    }returntrue;}voidCvSobelOperator::onAttributeValueChanged(Attribute* attribute){// Clear the cached data in the output stream// Because an attribute value changed and the result needs to be calculated againclearStreamCache();// The specification of the cv::Sobel function does not allow both parameters to be 0// This is why this function needs to make sure that the values are never both 0.if (attribute->getValue().toInt() ==0)    {        Attribute* other;if (attribute == m_xDerivative)        {            other = m_yDerivative;        }else        {            other = m_xDerivative;        }if (other->getValue().toInt() ==0)        {            other->setValue(1);        }    }}}// namespace G}

• OpenNodes.cmake located inGraph-ICS/app and add your newly created node to theOPENCV_NODES variable

set(OPENCV_NODES    CvDilation    CvErosion    CvFourierTransformPSD    CvHistogramCalculation    CvHistogramEqualization    CvMedian    CvTransformation# Our newly added Filter    CvSobelOperator)

Now you just need to run CMake and build Graph-ICS.

You also can customize the node front-end if you don't like the auto generated look or want additional features. As an example we add a button to the node to swap the attribute values.

• Add a new .qml file toapp/qml/node/custom
• The file needs to have the exact same name as the filter class (here: CvSobelOperator).

importQtQuick2.15importQtQuick.Layouts1.15// Import our ThemeimportTheme1.0import"../"import"../attribute"import"../../components"// Import the c++ modelimportModel.CvSobelOperator1.0// GNode provides a Component interface that can be used for customization// GNode provides all functionality and default ComponentsGNode {    id: sobel// Instantiate the c++ model and assign it to the model property    model: CvSobelOperatorModel {}// Override the background// Do not specitfy size or anchors of the Rectangle, as it will fit to the content    background: Rectangle {        color:Theme.primary        radius:8border.width:3border.color: {if (isSelected) {returnTheme.secondary            }if (isHovered) {returnTheme.secondaryLight            }returnTheme.primaryDark        }    }// Override the content// Everything a sobel contains is inside of the content Component// e.g. Title and Attributes// Specify a size for this Component if no implicit size is available (e.g. Item)// You don't need to use Layouts (but they are convenient for sizing)    content: ColumnLayout {        spacing:Theme.smallSpaceing// Display the node name (id)Text {Layout.fillWidth:true            text: id            font:Theme.font.body2            color:Theme.onprimary            topPadding:Theme.largeSpaceing            leftPadding:Theme.largeSpaceing            rightPadding:Theme.largeSpaceing            bottomPadding:Theme.smallSpaceing            verticalAlignment:Text.AlignVCenter            horizontalAlignment:Text.AlignHCenter        }        RowLayout {Layout.leftMargin:Theme.largeSpaceingLayout.rightMargin:Theme.smallSpaceingLayout.fillHeight:true            Item {Layout.preferredHeight:xDerivative.height+yDerivative.height+Theme.smallSpaceingLayout.minimumWidth:Math.max(xDerivative.width,yDerivative.width)// Specify the Attributes// Custom Attributes that derive from GAttribute can be used too                TextFieldAttribute {                    id: xDerivative// make sure to use the exact same name as in registerAttribute(...) (c++)                    name:"xDerivative"// get the attribute model from the node model by the name                    node: sobel                    model:node.model.getAttribute(name)// Don't forget to add the Attributes to the attributes array// Otherwise saving and loading the node will not work properly// It is necessary to use the onCompleted function to push the// attribute into the array, because content is a Component// and the content object is only created during GNode's onCompleted functionComponent.onCompleted:attributes.push(xDerivative)                }                TextFieldAttribute {                    id: yDerivativeanchors.top:xDerivative.bottomanchors.topMargin:Theme.smallSpaceing                    name:"yDerivative"                    node: sobel                    model:node.model.getAttribute(name)Component.onCompleted:attributes.push(yDerivative)                }            }// Simple demonstration// swap the attribute values            GIconButton {Layout.preferredHeight:Theme.baseHeightLayout.preferredWidth:Theme.baseHeight// Make sure that the attribute values can not be altered if the node is locked// Otherwise unexpected behavior may occur if the values are changed// while a stream is processing                enabled:!isLockedicon.source:Theme.icon.swapVerticon.color: {if (hovered|| pressed) {returnTheme.primaryDark                    }returnTheme.onprimary                }                transparent:true                onClicked: {let value=xDerivative.valuexDerivative.setValue(yDerivative.value)yDerivative.setValue(value)                }            }        }    }// If you just want to create your individual Attributes and don't want to override the content// you can just override the predefined attribute Components e.g.:// textFieldAttributeComponent: MyCustomTextFieldAttribute {}}

Special thanks ❤️ to all open-source projects making life easier for developers all over the world and especially:

• linuxdeployqt
• macos-virtualbox
• ScreenToGif
• Greenshot
• Big Buck Bunny (Example Video)
• OpenCV
• ITK
• Qt