Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up

OpenISS -- a unified multimodal motion data delivery framework.

License

NotificationsYou must be signed in to change notification settings

OpenISS/OpenISS

Repository files navigation

CII Best Practices

OpenISS is a motion capture data aggregation and delivery framework for VFX thathas library instances abstracting various middleware and cameras for many application types.

Applications include AI+Art, performing arts, interactive film, facial animation,NUI applications using OpenGL, Processing, TensorFlow, Keras, and MARF.

The OpenISS framework provides a uniform abstraction layer over libfreenect,libfreenect2, librealsense, OpenNI2, NiTE2, NuiTrack, Azure SDK, PCL, and ROS and hasREST and SOAP web services. There are different front-end modules aswell based on OpenGL, Processing, and web browsers (JS).

The public GitHub updates are done periodically from private forks wherea lot of integration / contributions happen.

History

OpenISS is inspired by the development of ISSv1, ISSv2, ISSv3 and MultiCamTk++, but is being builtfrom scratch using C/C++ and various wrappers and uses similar or same open-source libraries, middleware and toolkits forsensors and creative coding. Various wrappers are being developed. See background on the inspirational ISS below.OpenISS API is likewise for the first time made it possble to access Kinect 1 and 2as well as OpenCV as REST and SOAP services for creative near-realtime online broadcasting.OpenISS is poised to be the core replacement for ISSv2's pipeline. It also serves asan educational tool for graduate and undergraduate students in computer vision, computationarts, pattern recognition, AI, machine learning, and game development. It is designedto be portable.

  • OpenISS component development began in 2016 in C
  • Expanded to C++ and Java since and ongoing

Contributors

Current

  • Kyle Ross, Unity plugin and related rigging
  • Konstantinos (Kosta) Psimoulis: Web Services lead (REST and SOAP), SimpleOpenNIRS/liblrealsense2 support
  • Jonathan Llewellyn: actvity/action recognition, VFX pipeline
  • Dominique Cartier, point clouds integration and object/surface reconstruction

Recent

  • Jashanjot Singh: Gesture and Hand Tracking Framework, NiTE2/NuiTrack, and ROS support, SWIG support for Java/Processing
  • Haotao (Eric) Lai: Person Re-identifcation, Green Screening, device abstracion (Kinects 1 and 2, RealSense D435), TensorFlow/Keras/Python abstractions; Kinect 2 web service
  • Yiran (Bernie) Shen: Facial Recognition and data (landmarks and expressions) provider framework abstraction over OpenFace, OpenCV, including TensortFlow for VFX
  • Yuhao Mao: Art and AI, Sound Visualization, Style Transfer
  • Chao Wang: Style Transfer, Magenta support, motion-based audio support, finger tracking
  • Zihao Song: Singnal Processing for Audio and Image Processing effects and MARF

Web Services

  • Seehere for more documentation.

Early C API and Linux Builds

  • See detailed listhere.

  • Updates for Linux and C in EL6 (CentOS 6.x), CSI230-101 Fall 2017 course students teams:

    • Calum Phillips, Rosser Martinez, Matthew Roy
    • Alex Rader, Cory Smith, Nicholas Robbins
  • Original build automation contributors for Linux in EL6 (CentOS 6.x), CSI230-101 Fall 2016 course students teams:

    • Brian Baron, Colin Brady, Robert Gentile
    • Gabriel Pereyra, Justin Mulkin, Duncan Carrol, Lucas Spiker

Build Instructions

Docker

  • We are working on our Docker imageshere

Ubuntu

  • Please usecmake
  • More documentation to follow

MacOS

  • Please usecmake
  • More documentation to follow

EL7

EL7 (RHEL, CentOS, Scientific Linux) are as of September 2018 default andpreferred build, so the development effort focuses around this platform,but it is known to run on macOS and Ubuntu.

EL6

EL6 is now considered legacy.

Notice, EL6 requires a newer kernel forproper USB3 and NVIDIA support. If you preferto install dependencies manually, you can followthe scripts referenced here (build.sh andel6.sh) and repeat theirrelevant steps one by one.

Install:yum install gitgit clone https://github.com/OpenISS/OpenISS.gitgit submodule update --init --recursiveNavigate to OpenISS/srcRun the command:make depsThis will install all of the 3rd partydependenciesCompile:cd OpenISS/srcmakeClean:Navigate to OpenISS/srcRun the command:make cleanRemoves the installed dependencies

TODO

Build Support

  • OS X (mostly there)
  • Windows (partially there for some components)

Wrappers

  • Java (in progress)
  • C++ (mostly there)
  • Processing (in progress)
  • Max (in progress)
  • PureData (in progress)
  • Python (started)
  • ...

Background on the ISSv1, ISSv2, ISSv3

OpenISS’s core goals are to enable achieving something akinto the original ISS below in an open educational setting. Then,to build custom applications based on it, using OpenISS APIas a core.

Academic Publications

Videos

References and Acknowledgements


[8]ページ先頭

©2009-2025 Movatter.jp