When I started with Home Assistant a couple years ago, one of the main use cases was making a nice dashboard which included my cameras. Unfortunately this simple desire has been far from easy to achieve. In my quest for something reliable and real time, I think I have now tried every option that is open to me and realized that none of them is really perfect, but I’ve learned enough that it’s at least worth sharing so that other people have some better idea of their options and which one might fit their use case first. So, let’s start.
To begin, I am running 4x 1080p hikvision cameras configured with a 1080p h.264 main stream and a ~700x500 h.264 substream and they are working perfectly using the same streams with my synology and frigate object detection. The stream has never dropped from those systems that I can remember. Not the case with hass unfortunately.
I have tested the MJPEG stream from the substream too. Also during my testing I switched everything over from TCP to UDP to reduce lag a couple seconds and tried to enable multicast but I’m not sure if it’s working. TI’m going to list the camera platforms I’ve tried in order of worst to best for my use case (which involves a few tablets around the house with both persistent small previews and temporary popups for when humans are detected) with ratings for various properties. You can see my config here
MJPEG (with and without streaming component enabled):
MJPEG cameras are extremely close to realtime, even with streaming component turned on. You can choose from a live view which gives a decent framerate (but which my tablets can’t handle) or a The downside seems to be that only one thing can be streaming the camera URL at a time. This even applies to hass endpoints, So if one hass session is viewing the camera, the other hass sessions cannot. That alone makes it unusable for my use case as I have multiple tablets that need the stream simultaneously. Also I noticed frequent cases where the streams would smear and have other visual artifacts.
Also, synology forces you to choose H.264 or MJPEG for all the streams, so if I switch the substream on my camera to MJPEG (main stream doesn’t support it), it means that I have to choose to only use the main stream or substream in Synology.
MJPEG Camera Rankings:
Lag: 10/10 (this is the least laggy camera platform in hass, 1 sec or less)
Framerate 8/10 (decent, but not the best)
Kindle Fire/ Fully Kiosk Tablet friendliness: 10/10 (can get high framerate 1080p previews for the one tablet that grabs the stream)
Reliability: 7/10 (if you only have one device ever accessing the stream, it’s rock solid, add one more device however, and the reliability is a perfect 0. Unfortunately sometimes the cameras smear, maybe due to CPU load on the cameras).
Usability: 2/10 (only usable if I have only 1 hass dashboard open with the camera streams, with more the streams fail)
Server CPU: 10/10 (no noticeable effect on hass server cpu)
Camera CPU: 3/10 (the cameras don’t appear to like serving MJPEG streams and bog down)
FFMPEG Camera Platform without Stream Component Enabled:
I had high hopes for FFMPEG to give me the reliability of MJPEG type streams that it transcodes from the h.264 streams that don’t kill my cameras. Unfortunately it brought the CPU on my 2011 mac mini Ubuntu server to it’s knees. I tested it on an 2019 i7 mac mini to see if I could get the kind of real time feed and reliability I wanted if I threw enough processor at it, but unfortunately even when given plenty of overhead to play with, it didn’t produce reliable feeds. It also created a lot of smearing in the camera feeds.
FFMPEG Camera Component Rankings:
Lag: 7/10 (2-3 seconds, not bad.)
Framerate 7/10 (Probably 1-2 FPS IIRC)
Kindle Fire/ Fully Kiosk Tablet friendliness: 10/10 (I can’t remember it ever failing to display. Most reliable one out of all of them.)
Reliabiliy 4/10 (A lot of dropouts and failures to load in lovelace, even on a powerful CPU, also smeared the stream)
Usability: 4/10 (The CPU requirements and lack of reliability mean it’s not real practical)
Server CPU: 2/10 (the CPU killer)
Camera CPU: 10/10 (cameras aren’t even serving the stream)
FFMPEG Camera Platform with Stream Component Enabled:
Turning on the stream component with FFMPEG increases realiability and framerate a bit at the expense of 10 additional seconds of lag.
FFMPEG Camera Component Rankings:
Lag: 2/10 (13 seconds, not very useful)
Framerate 8/10 (Something near full 4 fps)
Kindle Fire/ Fully Kiosk Tablet friendliness: 4/10 (Can’t use the live stream so stuck with the 1 frame every 10 seconds)
Reliabiliy 4/10 (A lot of dropouts and failures to load in lovelace, even on a powerful CPU)
Usability: 3/10 (The lag, CPU requirements and lack of reliability mean it’s not real practical)
Server CPU: 2/10 (the CPU killer)
Camera CPU: 10/10 (cameras aren’t even serving the stream)
Generic Camera Platform Without Stream Component:
This works okay, pretty low lag and you can choose between a 1 frame every 10 second preview or a “live” preview which gets maybe 1 frame every 1-2 seconds. The problem is that the streams tend to drop off the UI or not load, so unfortunately reliability isn’t very good.
Generic Camera without Stream Component Rankings:
Lag: 8/10 (2-3 seconds of lag)
Framerate 5/10 ( 1 frame every 1-2 seconds for “live”. does the job, but not good)
Kindle Fire/ Fully Kiosk Tablet friendliness: 3/10 (don’t display reliably on really any device)
Reliability:: 3/10. Streams would often blank out or not load
Usability: 3/10: Reliability kills it
Server CPU: 10/10 (no noticeable effect on hass server cpu)
Camera CPU: 10/10 (cameras don’t mind serving h.264 streams)
Generic Camera Platform Without Stream Component:
I was using my cameras this was for a long time simply due to the reliability of the streams and framerates. Only very occasionally they wouldn’t appear on my tablets, but often on my desktop I would have to click the … and do a refresh before they would update. But they had about 13 seconds of lag which in the end I realized was just useless for anything other than instant replay when the kids were fighting (could be a nice feature to add to other realtime camera streams :-)). This camera platform needs some parameters to tweak the HLS stream to reduce latency, then it will be pretty good.
Generic Camera with Stream Component Rankings:
Lag: 2/10 (13 seconds of lag)
Framerate 5/10 ( very good, my cameras only do 4 fps but I appeared to get all of that)
Kindle Fire/ Fully Kiosk Tablet friendliness: 5/10 (fully kiosk on kindle wouldn’t allow the live stream for more than 1 camera, it would crash, so I had to run without live stream which worked quite reliably but was very slow)
Reliabiliy 8/10 (Live stream not super reliable, but non live is quite reliable)
Usability: 8/10 (work well if you don’t mind the lag)
Server CPU: 10/10 (no noticeable effect on hass server cpu)
Camera CPU: 10/10 (cameras don’t mind serving h.264 streams)
Synology Camera Platform:
This is a funny one, it only grabs the substream of the camera from synology and gives you 1 frame every 5 seconds or so. This limited it’s usefulness because my substream is 4x3 aspect ratio which doesn’t look good on my dashboard with my portrait orentation camers which I stack next to each other. But what I did use it for a while for was to do the preview popups which despite having low framerate was more realtime and in a good aspect ratio for popups on my landscape orientation tablets.
Synology Camera Component Rankings:
Lag: 7/10 (when it finally refreshes the frame, it only has 2-3 seconds of lag)
Framerate 4/10 ( 1 frame every 5 seconds, not so good)
Kindle Fire/ Fully Kiosk Tablet friendliness: 10/10 (I can’t remember it ever failing to display. Most reliable one out of all of them.)
Reliabiliy 10/10 (The only reliable camera platform in hass that I’ve found)
Usability: 8/10 (work well if you don’t mind the substream and the frame rate)
Server CPU: 10/10 (no noticeable effect on hass server cpu)
Camera CPU: 10/10 (cameras aren’t even serving the stream)
ONVIF Camera Platform with stream component:
This was one of the last ones I tested because I didn’t see any reason why it should be any different than the generic camera, but it was. First off, it seems to be a bit more reliable. Secondly it gives you the sensors from the camera like motion detection. One of these sensors is very useful and not available from the other camera/sensor platform, which is the CPU level sensor. Once I had this I started to realize how much of a factor my camera CPU was in the performance of the whole camera display. Very useful.
ONVIF Camera Component with Stream Component Enabled Rankings:
Lag: 2/10 (13 seconds of lag)
Framerate 5/10 ( very good, my cameras only do 4 fps but I appeared to get all of that)
Kindle Fire/ Fully Kiosk Tablet friendliness: 5/10 (fully kiosk on kindle wouldn’t allow the live stream for more than 1 camera, it would crash, so I had to run without live stream which worked quite reliably but was very slow)
Reliabiliy 8/10 (Live stream not super reliable, but non live is quite reliable)
Usability: 8/10 (work well if you don’t mind the lag)
Server CPU: 10/10 (no noticeable effect on hass server cpu)
Camera CPU: 10/10 (cameras don’t mind serving h.264 streams)
ONVIF Camera Platform without stream component:
This is where the reliability of the platform really started to shine. I was able to get these 1080p streams to come up pretty reliably (though not perfectly) in lovelace using the live view of ~1fps even on tablets. This level of real time with pretty good reliability was not something I had found before.
I am still running with these ONVIF cameras with the addition of a proxy camera (next rating) for preview, but I use the full 4 fps substream to get a near real time (1 second lag) popup with ~2-3 FPS on the tablets when people are detected outside. It’s not 100% reliable but maybe 95%, the drop in reliability from synology component is made up for by having ~10 times higher framerate.
ONVIF Camera Component without Stream Component Enabled Rankings:
Lag: 9/10 (~1 second of lag, very good)
Framerate 8/10 (getting 2-3 out of the 4 fps I think)
Kindle Fire/ Fully Kiosk Tablet friendliness: 6/10 (I don’t recall if I had them working with live view on the tablet, but I don’t think they would at 1080p)
Reliabiliy 8/10 (Not perfect, but better than most)
Usability: 8/10 (Lack of lag and good framerate make them a good choice)
Server CPU: 10/10 (no noticeable effect on hass server cpu)
Camera CPU: 10/10 (cameras don’t mind serving h.264 streams)
Proxy Camera Platform without stream component:
I was pretty excited when I found the proxy camera component because the only stream I had with the 16x9 aspect ratio I needed was 1080p and I knew that that was causing problems on my frontend.
Proxy camera allowed me to divide the resolution by 6 in each direction which was still fine for the DPI of the displays I was using, so now I’m running 320x180 preview streams that are downsized by the proxy component. This means that they load up very quickly in the frontend and I can run them in live view on the tablets very nicely.
Unfortunately they do seem to load up the camera’s CPU a bit, but I seem to be able to just barely squeak by with this enabled and not totally kill the camera’s CPU.
ONVIF Camera Component without Stream Component Enabled Rankings:
Lag: 9/10 (~1 second of lag, very good)
Framerate 8/10 (getting 2-3 out of the 4 fps I think)
Kindle Fire/ Fully Kiosk Tablet friendliness: 9/10 (fully kiosk doesn’t choke on these and can be run at 1 frame every second or 2 which is fine for a small preview)
Reliabiliy 8/10 (Not perfect, but better than most. Sometimes drops or stops updating for a while)
Usability: 8/10 (Lack of lag and good framerate make them a good choice)
Server CPU: 7/10 (pretty minimal, at least how I use them… Something like extra 8%)
Camera CPU: 4/10 (added something like 30% additional load to the CPU, bringing them right to the limits of usability)
Conclusion:
So currently I am running the ONVIF cameras with the proxy component. I use the substream when I have a popup on my tablets to give a decent framerate and low latency stream which is about the most those tablets can handle. Proxy is giving the previews I have in the lovelace cards on the tablet and my main hass dashboard on iOS or browser. Not 100% reliable but maybe 95% for both the lovelace and popups… I’d love to see 100% someday, but for now this is the best configuration I’ve found.
Special mention: MQTT camera. I use this one for my snapshots of humans detected on our property. Never had any problems with it, but I’m not really putting it through it’s paces. I only need 1 frame every 5 seconds or something.
2 posts - 1 participant
Read full topic
