使用 python 和 OpenCV 与 VideoCapture 深入了解多个网络摄像头捕获

Question

I'm experimenting with trying to capture an image from multiple webcams simultaneously (or near-simultaneously). I've been playing with OpenCV and with VideoCapture and programming in python. But have some confusion and hoped someone could help explain things...

To Start, I tried VideoCapture (Markus Gritsch's work found here: http://videocapture.sourceforge.net/ ). This is a very easy to use add-in for python. If I just wanted to capture a simple image. It works just fine... for the most part.

My setup is 2 cheapie webcams in a USB hub on a single USB port and my laptop's built-in webcam.

I have read all about using the same model webcam on the same USB bus and how it may not work, etc. but decided to give it a go. (I also have some photobooth program that I must have installed eons ago called "Cyberlink YouCam" For some reason, this program "looks" like a camera to both OpenCV and VideoCapture.

VideoCapture connects to and captures from each camera like this:

Cam0 = Device(devnum=0)
Cam0.saveSnapshot("filename0.jpg")
del Cam0
Cam1 = Device(devnum=1)
Cam1.saveSnapshot("filename1.jpg")
del Cam1

With VideoCapture and the setup I described, I can independently capture from all the cameras on my system (4 total, including the YouCam... 0 - 3)

The problem is that this program does not seem to be able to connect to more than one camera at the same time... if I don't close the previous camera instance, it simply freezes and I have to disconnect and reconnect the first webcam from USB to regain access to it.

This won't work:

cam0 = Device(devnum=0)
cam1 = Device(devnum=1)
cam0.saveSnapshot("filename0.jpg")
cam1.saveSnapshot("filename1.jpg")
del cam0
del cam1

Cam0 will open, but that is the end of it. Frozen.

Another thing with VideoCapture is that on the cheapo webcams, there is a major delay (almost a second) until the picture comes alive... in order to do a successful capture, I had to do something like this:

Cam1 = Device(devnum=1)
Cam1.saveSnapshot("filename1.jpg") #gets the camera going and saves black image
time.sleep(.75) #delay
Cam1.saveSnapshot("filename1.jpg") #captures the image second time around

Effectively saving the image twice...

I wouldn't have minded if the images from each camera were a few milliseconds apart taken in squence

#open connection to cam, take image
#close connection to cam

#connect to next cam, take next image
#close connection to cam

#etc.

But the delay was way too much... I thought the delay was the cheap webcams, but I saw a different result with Open CV so it must be VideoCapture's fault.

For one thing, I COULD open more than one camera at the same time with OpenCV... but only of different types. (One of the cheapo cams and my built-in cam... and / or the Cyberlink program) OpenCV seemed to see the 2 cheapo cams on the same USB hub as one (it only turned on one of the cams.)

capture0 = CaptureFromCAM(0) #this was Cyberlink Program
capture1 = CaptureFromCAM(1) #this was cheapo cam
capture2 = CaptureFromCAM(2) #this was built-in cam
#CaptureFromCAM(3) resulted in error... did not find a 4th "camera"

frame0 = QueryFrame(capure0)
frame1 = QueryFrame(capure1)
frame2 = QueryFrame(capure2)
cv.SaveImage("filename0.jpg",frame0)
cv.SaveImage("filename1.jpg",frame1)
cv.SaveImage("filename2.jpg",frame2)

Whereas VideoCapture could see the 2 similar cameras independently, OpenCV could not.

Anyone know why that would be? How are the two packages interacting with the computer differently that one can determine different cams on the same USB and the other cannot?

Secondly, OpenCV opened my cheap cams instantaneously... no .75 second delays there.

Again, I am curious how the two packages (videoCapture vs OpenCV) communicating with the cameras differently from each other?

What I am ultimately interested in doing is being able to capture from 2 or 3 cameras at the same time (or close to it, if there were a few milliseconds delay, that is fine). Looks like OpenCV would be the package of choice, however, I am trying to understand better how the software is interacting with the cameras.

Thank you all for the insight!

J

Answer 1

I can tell you some information about OpenCV , but not about VideoCapture. I do, however, use VideoCapture for getting nice camera names.

I have used OpenCV in python to successfully talk to two cameras at the same time. In some cases, the two cameras were different, other times they were the same but scientific level so perhaps the drivers were better. I can't say that having two identical cameras isn't causing you a problem.

However, I can say that you are using the old and possibly unmaintained camera API. Use the new cv2 namespace/API also called VideoCapture. See OpenCV VideoCapture class python bindings beginning with cv2 . If you have a new version of OpenCV in your python distribution, it is accessed via import cv2 . The old bindingss are in cv2.cv (also note that as of OpenCV 2.4.6, many of the important constants for VideoCapture::get(...) and ::set(...) are only found in cv2.cv , this is supposed to be fixed in the next version)

Answer 2

More insight...

I can connect to two Logitech WebCams, and capture from them simultaneously on my MacBook Pro (Late 2011) using OpenCV 2.4.7, but not on older MacBook Pro (Late 2009)!

Specifically:

MacBook Pro 2011 -- 
WORKS: 2 x SAME MODEL plugged into SEPARATE USB Ports
FAILS: 2 x SAME MDOEL plugged into SAME USB Port (via a hub)
WORKS: 1 x SAME MODEL plugged into HUB and 1 x SAME MODEL plugged into laptop directly

MacBook Pro 2009 --
WORKS: 2 x DIFFERENT MODELS plugged into SEPARATE USB Ports
FAILS: 2 x SAME MANUFACTURER (!) plugged into SEPARATE USB Ports
FAILS: 2 x SAME MODEL

I think the only real news here is that a newer computer with the latest OpenCV works ... at least for me!

In ALL CASES, I can successfully enumerate ALL cameras as long as I only have one USB camera open at a time.

Answer 3

it's been a while since I worked on this project thanks to work and life getting in the way, as usual...

I recently built an all new computer running linux and I am at it again - trying to capture input from multiple cameras. This time I am using 4 decent Microsoft Webcams rather than super cheap cameras...

I am, again, able to communicate with all the cameras (I even tried putting them all on the same USB controller via hub and was able to talk to each one independently) - but not simultaneously.

This doesn't matter to me so much, as I don't need simultaneous connection to them, however when they are first powered up, the internal shutters have to open and adjust and the auto-focus has to adjust...

I am trying the camera library of Pygame and the time my script takes to open the camera, grab the frame and save it is not enough time for the cameras to ready-themselves and thus my captures images are either dark (or half dark) and out of focus.

I was using OpenCV (which I am going back to trying now) was able to communicate with the cameras at the same time (the lights on all 4 cams were on at the same time) which I am HOPING means that I can "Turn them on" in advance and then capture the frames as I need them. Right now I am able to see live video from each camera (not at the same time, but I can test them individually by changing the device number) but unfortunately, since I put this project down for a while, I forgot some of the code I need to do more with it without running into errors so I have to do some work on it.

In the mean time, i just found this online: http://codeplasma.com/2012/12/03/getting-webcam-images-with-python-and-opencv-2-for-real-this-time/

I haven't tried this code yet, but I like the idea that this person added "throw away frames" to allow the camera time to adjust itself. My aim is to have the camera on continuously, and then simply take the photo (capture the frame) when I press a key or something.

J

A FEW MORE UPDATES:

Using OpenCV, I am able to connect to as many as 5 webcams simultaneously (4 Microsoft and one Logitech) when each one is connected on a different USB controller (the motherboard I'm using seems to have 8 and I have 6 USB ports, currently). But when I connect more than one cam on a single USB controller (ie 2 ports that are on the same controller or using a hub) OpenCV only seems to control one of the two cameras on each controller.

However, using PyGame's camera module, I am able to individually connect to as many as 5 cameras on a single USB controller (connected with a powered hub) however I can not connect to more than one at the same time (this doesn't work even when they are on different USB controllers). Again, I don't need a live feed from all the cams at the same time, but the cameras require time to set themselves up so without being able to "turn the cameras on" in advance, I can't "snap a picture" worthwhile.

and it continues...

Answer 4

I also tinkered with this a bit. I can confirm that you can only do one camera at a time. I've tried threading and a myriad of other things, but all failed.

Here's the code I came up with. Perhaps it will help someone. I have a bunch of "test code" in there. You're welcome to keep going where I left off: https://gist.github.com/dovy/4fe2924ea31869def1e8

The code is fully working, you just have to get cv2 installed. ;)

Answer 5

I think this is a USB bandwidth issue. Webcams without built in compression use an enormous amount of bandwidth. Multiply the width * height of the resolution of the camera by the color depth (for example, 8 bits for 256 colors) by the frame rate to get the number of bits per second (without any USB framing overhead) and you'll see that a single USB2 high-speed (480 megabit/sec) channel can't carry more than 1 HD stream (1280 * 1024 * 8 * 30 fps ~= 314.6 megabits/sec). When using a hub, that bandwidth is shared among all the ports. When the driver goes to open the camera, it will read the bandwidth requirement and refuse to continue if the remaining bandwidth on the USB channel is insufficient. That's why the first camera you open works, but the subsequent ones fail. Your choices are to use a lower-resolution camera, move the hi-res camera to a different root hub, or use an H264 or MJPEG compressed camera.