Repository files navigation

imagenode enables Raspberry Pi computers to capture images with thePiCamera, perform image transformations and send them to a centralimagehub forfurther processing. It can also send other sensor data such as temperature dataand GPIO data. The processing power of the Raspberry Pi is used to detectevents (like the water meter flowing or a coyote crossing the back yard), andthen send a limited number of images of the event. It also works on other typesof (non Raspberry Pi) computers with USB cams or webcams.

Here are a couple of screenshots showing images sent by a Raspberry Pi PiCameraand displayed on a Mac. In the top screenshot, a ballpoint pen hanging from astring is still. In the bottom screenshot, the ballpoint pen is swinging backand forth. The largest image in each screenshot is the full frame sent by thePiCamera. The smaller windows are showing theimagenode motion detectorparameter tuning displays including the detected motion state of "still" and"moving":

imagenode is the image capture and sending portion of a computer visionpipeline that is typically run on multiple computers. For example, a RaspberryPi computer runsimagenode to capture images with a PiCamera and performsome simple image processing. The images are transferred byimageZMQ (seereference) to a hub computer runningimagehub (often a Mac) for furtherimage processing. The real benefit ofimagenode is that it can use thethe processing power of the Raspberry Pi to:

• Continuously capture images (around 10 frames a second is typical)
• Analyze the images to detect events (e.g., water meter started flowing)
• When a detected event occurs:
  • Send an event message about the event to theimagehub
  • Send a select few "detected state change" images to theimagehub

So, instead of 36,000 images an hour being sent from our water meter cam to ourimagehub, only about 20 images are sent each time the water starts flowingor stops flowing. Instead of many thousands of images an hour showing a mostlyunmoving farm area, our critter cams spot coyotes, raccoons and rabbits and onlysend event messages and images when something is actually seen moving about.

imagenode provides image capture, event detection and transmission servicesas part of a distributed computer vision system that includes multiplecomputers with cameras, sensors, database hubs and communication links.SeeUsing imagenode in distributed computer vision projectsfor a more detailed explanation of the overall project design. See theYin Yang Ranch projectfor more details about the architecture of theimagenode <-->imageZMQ <-->imagehub system.

• Continuously captures images using PiCameras or USB webcams.
• Performs image transformation and motion, light or color detection.
• Sends detected events and relevant images to an image hub usingimageZMQ.
• Can capture and send other sensor data gathered using the GPIO pins.
• Can control lighting (e.g., white LED or Infrared LED area lights).
• Sends event messages (e.g., water is flowing) as well as images.

imagenode has been tested with:

• Python 3.6 and newer
• OpenCV 3.3 and 4.0 and newer
• Raspberry Pi OS Buster, Raspbian Stretch and Raspbian Jessie
  • NOT yet tested with Raspberry Pi OS Bullseye. Waiting for a productionreplacement for the Python PiCamera module.
• PyZMQ 16.0 and newer
• RPi.GPIO 0.6 and newer (imported only if using GPIO pins)
• picamera 1.13 (imported only if using PiCamera)
• imageZMQ 1.1.1 and newer
• imutils 0.4.3 and newer (used get to images from PiCamera)
• psutil 5.7.2 and newer
• PyYAML 5.3 and newer
• w1thermsensor 1.3 (if using DS18S20 temperature sensor)
  • NOT yet compatible with w1thermsensor version 2 which uses a new API
• adafruit-circuitpython-dht 3.4.2 and newer (if using DHT11 or DHT22 sensor)

imagenode captures images and usesimageZMQ to transfer the images.It is best to install and testimageZMQ before installingimagenode.The instructions for installing and testingimageZMQ are in theimageZMQ GitHub repository.

imagenode is still in early development, so it is not yet in PyPI. Get it bycloning the GitHub repository:

git clone https://github.com/jeffbass/imagenode.git

Once you have clonedimagenode to a directory on your local machine,you can run the tests using the instructions below. The instructions assume youhave clonedimagehub to the user home directory.

imagenode requires aLOT of settings: settings for the camera, settingsfor the GPIO pins, settings for each detector and each ROI, etc. The settings arekept in a YAML file and are changed to "tune" the image capture, ROIs, motiondetection and computer vision parameters. An example YAML file is included inthe "yaml" directory. An explanation of the yaml file and how to adjust the settingsis inimagenode Settings and YAML files.

imagenode should be tested in stages, with each stage testing a little morefunctionality. The tests are numbered in the order in which they should be runto determine ifimagenode is running correctly on your systems.

Testimagenode in the same virtualenv in which you testedimagenZMQ. FortheimageZMQ testing and for theimagenode testing, my virtualenv iscalled py3cv3.

imagenode requiresimageZMQ be installed and working. Before running anytests withimagenode, be sure you have successfully installedimageZMQand run all of its tests. TheimageZMQ tests must run successfully on everycomputer you will be usingimagenode on. You can use pip to installimageZMQ.

imagenode is not far enough along in developmentto be pip installable. So it should both be git-cloned to any computer thatit will be running on. I have done all testing at the user homedirectory of every computer. Here is a simplified directory layout:

~ # user home directory+--- imagenode.yaml  # copied from one of the imagenode yaml files & edited|+--- imagenode    # the git-cloned directory for imagenode     +--- sub directories include docs, imagenode, tests, yaml

This directory arrangement, including docs, imagenode code, tests, etc. is acommon development directory arrangement on GitHub. Using git clone from youruser home directory (either on a Mac, a RPi or other Linux computer) willput theimagenode directories in the right place for testing. Each testdescribed below requires you to copy the appropriatetestN.yaml file toimagenode.yaml in the user home directory as shown in the above directorydiagram. Thereceive_test.py program acts as the image hub test receiver foreach imagenode test. It must be started and running before runningimagenode.py.

The first test runs both the sending programimagenode and the receivingprogramreceive_test.py (acting as a test hub) ona Mac (or linux computer) with a webcam. It tests that theimagenode softwareis installed correctly and that theimagenode.yaml file has been copied andedited in a way that works. It uses the webcam on the Mac for testing. It uses a"lighted" versus "dark" detector applied to a specified ROI.

The second test runsimagenode on a Raspberry Pi, usingreceive_test.py(acting as a test hub) on a Mac (or Linux computer). It tests that theimagenode software is installed correctly on the RPi and thattheimagenode.yaml file has been copied and edited in a way that works.It tests that theimageZMQ communication is working between the Raspberry Piand the Mac. It also tests the Picamera. It uses a "lighted" versus "dark"detector applied to a specified ROI.

The third test runsimagenode on a Raspberry Pi, usingreceive_test.py(acting as a test hub) on a Mac (or Linux computer). It is very similar to Test2, except that it uses a "moving" versus "still" motion detector applied to aspecified ROI.

The fourth test runsimagenode on a Raspberry Pi, usingreceive_test.py(acting as a test hub) on a Mac (or Linux computer). It allows testing of thetemperature sensor capabilities ofimagenode. It requires setting up aDS18B20 temperature sensor and connecting it appropriately to RPi GPIO pin 4.

The details of running the 4 tests arehere.

Running the test programs requires that you leave a terminal window open, whichis helpful for testing, but not for production runs. I use systemctl / systemdto startimagenode in production. I have provided an exampleimagenode.service unit configuration file that shows how I startimagenode for the production programs observing my small farm. I have foundthe systemctl / systemd system to be best way to start / stop / restart andcheck the running status ofimagenode over several years of testing. Forthose who prefer using a shell script to startimagenode, I have included anexampleimagenode.sh. It is important to runimagenode in the rightvirtualenv in production, regardless of your choice of program startup tools.

In production, you would want to set the test options used to print settingstoFalse; they are only helpful during testing. All errors andimagenodeevent messages are saved in the fileimagehub.log which defaults to thesame directory as imagenode.py. You might want the log to be in a differentdirectory for production; the log file location can be set by changing it in thelogging function at the bottom of the imagenode.py program file.

• More details on running the tests.
• How imagenode works.
• How imagenode is used in a larger project.
• Version History and Changelog.
• Research and Development Roadmap.
• The imageZMQ classes that allow transfer of images.
• The imagehub software that saves events and images.
• The larger farm automation / computer vision project.This project shows the overall system architecture. It also containslinks to myPyCon 2020 talk video and slides explaining the project.

imagenode is in early development and testing. I welcome open issues andpull requests, but because the programs are still rapidly evolving, it is bestto open an issue for some discussion before submitting pull requests. We canexchange ideas about your potential pull request and how to best test your code.

Thanks for all contributions big and small. Some significant ones:

• ZeroMQ is a great messaging library with great documentationatZeroMQ.org.
• PyZMQ serialization examples provided a starting point forimageZMQ.See thePyZMQ documentation.
• OpenCV and its Python bindings provide great scaffolding for computervision projects large or small:OpenCV.org.
• The motion detection function detect_motion() borrowed a lot of helpful codefrom a motion detectortutorial postby Adrian Rosebrock of PyImageSearch.com.