This is a great project: the code is super neat and the write-up very clear. I like the sousveillance aspect of it!
For a project I needed a low-latency RTSP stream as well. When reading a video stream with OpenCV, the default video buffer is quite big, which, when filling up, makes the video lag behind a second or two. It then becomes impossible to perform any interaction on it.
I wasn't familiar with the setting you use to overcome this: setting cv2.CAP_PROP_BUFFERSIZE to 1 on the VideoCapture.
I am not sure, but you might get even lower latency by turning to OpenCV's GStreamer support. For me the trick was:
When testing, I also found out that the codec and image settings of the camera matter. With a h264 stream, the images came in batches of a number of frames, whereas MJPEG provided a more constant image stream with lower latency. Lastly, disabling 3D noise reduction also removed some delay.
If you want a hardware upgrade it's actually reasonably inexpensive to build something direct drive (which is how the real sub-30kg ones work). This gives you many advantages over the plastic gear type including a much faster response time and more accurate positioning.
Look for "gimbal motors", basically a large skinny pancake brushless motor. Combine those with a storm32, simplebgc or odrive and a magnetic encoder. A 3D printer will help.
You can also have a look at low cost suppliers of cheap UAV stuff if you want something fully integrated for you. A basic Gremsy, Viewpro or Siyi isn't that much more than your Amazon thing. Various software bugs but they can be worked around. The DJI units can sometimes be had used and some of the protocols have been RE'd already.
Super interesting project and impressive engineering. It wasn't quite clear to me how he solved the problem with
> Their motors are designed for slow, dampened pans across a stage, not for tracking a jet moving at 300 knots. The mechanical and electronics latency is significant; if you simply tell the camera to “follow that plane,” by the time the motors react, the target has often moved out of the frame.
Is he able to move the motors faster than they are designed to be moved? Is this the __Control (PID + Feed-Forward Loop)__ fix?
That's really interesting. One of my unfinished projects it's a system that take input from ADSB data and from a microphone and get a picture of a flying aircraft (I am living under the approaching path of a regional airport)
Hey HN, thrilled my silly little project was posted here. I had been noodling on this idea for quite some time and ran into a bunch of roadblocks early on that prevented me from getting reliable control over the PTZ camera. Once those were overcome, it came together fairly quickly.
I'm looking forward to doing more experiments including integrating my own payloads (thermal and optical) with an off the shelf motion control system. I also have an automotive radar unit coming which may provide some interesting options for cueing without using ADS-B in some situations (with relatively close targets).
The "We" is short for Westinghouse, or at least it was: Westinghouse Steered Stabilized Camera Mount, thus WESSCAM. Then they dropped an "S".
Granted pronouncing the name is ambiguous, Wes-cam or We-scam. But they're known well enough in the industry at this point that it's not a problem for them.
It's the we-scam part that's kinda funny. But of course they don't target consumers at all so that connotation doesn't matter I guess. Professionals just go by the product specs and not brand feeling.
This is a fun project that combines a cheap PTZ camera + OpenCV, Kalman filtering, PID control, and digital stabilization to not just snap photos of aircraft flying past, but do rock solid, pixel-level tracking as long as they're in sight.
And it can be combined with a source of ADS-B data so it knows what it's looking at, displaying the info on the OSD.
For a project I needed a low-latency RTSP stream as well. When reading a video stream with OpenCV, the default video buffer is quite big, which, when filling up, makes the video lag behind a second or two. It then becomes impossible to perform any interaction on it.
I wasn't familiar with the setting you use to overcome this: setting cv2.CAP_PROP_BUFFERSIZE to 1 on the VideoCapture. I am not sure, but you might get even lower latency by turning to OpenCV's GStreamer support. For me the trick was:
When testing, I also found out that the codec and image settings of the camera matter. With a h264 stream, the images came in batches of a number of frames, whereas MJPEG provided a more constant image stream with lower latency. Lastly, disabling 3D noise reduction also removed some delay.If you want a hardware upgrade it's actually reasonably inexpensive to build something direct drive (which is how the real sub-30kg ones work). This gives you many advantages over the plastic gear type including a much faster response time and more accurate positioning.
Look for "gimbal motors", basically a large skinny pancake brushless motor. Combine those with a storm32, simplebgc or odrive and a magnetic encoder. A 3D printer will help.
You can also have a look at low cost suppliers of cheap UAV stuff if you want something fully integrated for you. A basic Gremsy, Viewpro or Siyi isn't that much more than your Amazon thing. Various software bugs but they can be worked around. The DJI units can sometimes be had used and some of the protocols have been RE'd already.
> Their motors are designed for slow, dampened pans across a stage, not for tracking a jet moving at 300 knots. The mechanical and electronics latency is significant; if you simply tell the camera to “follow that plane,” by the time the motors react, the target has often moved out of the frame.
Is he able to move the motors faster than they are designed to be moved? Is this the __Control (PID + Feed-Forward Loop)__ fix?
I'm looking forward to doing more experiments including integrating my own payloads (thermal and optical) with an off the shelf motion control system. I also have an automotive radar unit coming which may provide some interesting options for cueing without using ADS-B in some situations (with relatively close targets).
Granted pronouncing the name is ambiguous, Wes-cam or We-scam. But they're known well enough in the industry at this point that it's not a problem for them.
And it can be combined with a source of ADS-B data so it knows what it's looking at, displaying the info on the OSD.