Repository files navigation

This is a demonstration for low latency streaming of the Pi's camera module toany reasonably modern web browser, utilizing Dominic Szablewski's excellentJSMPEG project. Other dependencies arethe Pythonws4py library, mypicameralibrary (specifically version 1.7 or above),andFFmpeg.

Firstly make sure you've got a functioning Pi camera module (test it withraspistill to be certain). Then make sure you've got the following packagesinstalled:

$ sudo apt-get install ffmpeg git python3-picamera python3-ws4py

Next, clone this repository:

$ git clone https://github.com/waveform80/pistreaming.git

Run the Python server script which should print out a load of stuffto the console as it starts up:

$ cd pistreaming$ python3 server.pyInitializing websockets server on port 8084Initializing HTTP server on port 8082Initializing cameraInitializing broadcast threadSpawning background conversion processStarting websockets threadStarting HTTP server threadStarting broadcast thread

Now fire up your favourite web-browser and visit the addresshttp://pi-address:8082/ - it should fairly quickly start displaying the feedfrom the camera. You should be able to visit the URL from multiple browserssimultaneously (although obviously you'll saturate the Pi's bandwidth sooner orlater).

If you find the video stutters or the latency is particularly bad (more than asecond), please check you have a decent network connection between the Pi andthe clients. I've found ethernet works perfectly (even with things likepowerline boxes in between) but a poor wifi connection doesn't provide enoughbandwidth, and dropped packets are not handled terribly well.

To shut down the server press Ctrl+C - you may find it'll take a whileto shut down unless you close the client web browsers (Chrome in particulartends to keep connections open which will prevent the server from shutting downuntil the socket closes).

The server script is fairly simple but may look a bit daunting to Pythonnewbies. There are several major components which are detailed in the followingsections.

This is implemented in theStreamingHttpServer andStreamingHttpHandlerclasses, and is quite simple:

• In response to an HTTP GET request for "/" it will redirect the client to"/index.html".
• In response to an HTTP GET request for "/index.html" it will serve up thecontents of index.html, replacing @ADDRESS@ with the Pi's IP address andthe websocket port.
• In response to an HTTP GET request for "/jsmpg.js" it will serve up thecontents of jsmpg.js verbatim.
• In response to an HTTP GET request for anything else, it will return 404.
• In response to an HTTP HEAD request for any of the above, it will simplydo the same as for GET but will omit the content.
• In response to any other HTTP method it will return an error.

This is implemented in theStreamingWebSocket class and is ridiculouslysimple. In response to a new connection it will immediately send a headerconsisting of the four characters "jsmp" and the width and height of the videostream encoded as 16-bit unsigned integers in big-endian format. This header isexpected by the jsmpg implementation. Other than that, the websocket serverdoesn't do much. The actual broadcasting of video data is handled by thebroadcast thread object below.

TheBroadcastOutput class is an implementation of apicamera customoutput.On initialization it starts a background FFmpeg process (avconv) which isconfigured to expect raw video data in YUV420 format, and will encode it asMPEG1. As unencoded video data is fed to the output via thewrite method, theclass feeds the data to the background FFmpeg process.

TheBroadcastThread class implements a background thread which continuallyreads encoded MPEG1 data from the background FFmpeg process started by theBroadcastOutput class and broadcasts it to all connected websockets. In theevent that no websockets are currently connected thebroadcast method simplydiscards the data. In the event that no more data is available from the FFmpegprocess, the thread checks that the FFmpeg process hasn't finished (withpoll) and terminates if it has.

Finally, themain method may look long and complicated but it's mostlyboiler-plate code which constructs all the necessary objects, wraps several ofthem in background threads (the HTTP server gets one, the main websocketsserver gets another, etc.), configures the camera and starts it recording totheBroadcastOutput object. After that it simply sits around callingwait_recording until someone presses Ctrl+C, at which point it shutseverything down in an orderly fashion and exits.

Since authoring thepicamera library, afrequent (almost constant!) request has been "how can I stream video to a webpage with little/no latency?" I finally had cause to look into this whileimplementing a security camera system using the Pi.

My initial findings were that streaming video over a network is pretty easy:open a network socket, shove video over it, done! Low latency isn't much of anissue either; you just need a player that's happy to use a small buffer (e.g.mplayer). Better still there's plenty of applications which will happily decodeand play the H.264 encoded video streams which the Pi's camera produces ...unfortunately none of them are web browsers.

When it comes to streaming video to web browsers, the situation at the time ofwriting is pretty dire. There's a fair minority of browsers that don't supportH.264 at all. Even those that do have rather variable support for streamingincluding weird not-really-standards like Apple's HLS (which usually involveslots of latency). Then there's the issue that the Pi's camera outputs rawH.264, and what most browsers want is a nice MPEG transport stream (TS). FFmpegseemed like the answer to that, but the version that ships with Raspbiandoesn't seem to like outputting valid PTS (Presentation Time Stamps) with thePi's output. Perhaps later versions work better, but I was looking for asolution that wouldn't involve users having to jump through hoops to create acustom FFmpeg build (mostly because I could just imagine the amount of extrasupport questions I'd get from going that route)!

So, what about other formats? Transcoding to almost anything else (WebM, Ogg,etc.) is basically out of the question because the Pi's CPU just isn't fastenough, not to mention none of those really solve the "universal client"problem as there's plenty of browsers that don't support these formats either.MJPEG looked an intruiging (if thoroughly backward) possibility but I found itrather astonishing that we'd have to resort to something as primitive as that.Surely in this day and age we could at least manage a proper video format?!

Then, out of the blue, and by sheer coincidence a group in Canada got incontact to ask whether the Pi could produce raw (i.e. unencoded) video output.This wasn't something I'd ever been asked before but it turned out to befairly simple, so I added it to the list of tickets for 1.7 and finished thecode for it about a week later. I confess I pretty much skimmed the rest oftheir e-mail the first time I read it, but with the implementation done I wentback and read it properly. They wanted to know whether they could use thepicamera library with Dominic Szablewski'sJavascript-based MPEG1decoder.

This was an interesting idea! Javascript implementations are near universal inbrowsers these days, and Dominic's decoder was fast enough that it would runhappily even on relatively small platforms (for example it runs on iPhones andreasonably modern Androids like a Nexus). Furthermore, the Pi is just aboutfast enough to handle MPEG1 transcoding with FFmpeg (at least at lowresolutions).

Okay, it's not a modern codec like the excellent H.264. It's not using "proper"HTML5 video tags. All round, it's still basically a hack, and yes it's prettyappalling that we have to resort to hacks like this just to come up with auniversally accessible video streaming solution. But hey ... it works, and it'snot (quite) as primitive as MJPEG so I'm happy to declare victory. I spent anevening bashing together a Python version of the server side. It turned out abit too complex to include as a recipe in the docs, hence why it's here, but Ithink it provides a reasonable basis for others to work from and extend.

Enjoy!

Dave.