iPhone 7 Plus - AVFoundation dual camera

Question

I'm actively researching this at the moment, but now that the iPhone 7 Plus has a dual camera system, will AVFoundation allow you to handle video frames from each specific camera simultaneously?

I am thinking/hoping that I'll be able to handle output from two AVCaptureDevice instances at the same time given a certain position.

Answer 1

You can only add one camera at the time to an AVCaptureSession. For example you can switch between the front and the back camera, but not use both at the same time. It is the same with the two back cameras on the 7 Plus, you have to choose either. However, there is a small difference since you can also call a "duo camera" that merges the images from both cameras when you zoom. But that is only available for still photo and you will only get one image/capture buffer. For video you have to pick either camera.

To pick the camera you can use the new AVCaptureDeviceDiscoverySession. To use the duo camera:

@property (nonatomic) AVCaptureDevice *backCamera;
@property (nonatomic) AVCaptureDeviceInput *backCameraInput;


if([AVCaptureDeviceDiscoverySession class]){
    NSArray *allTypes = @[AVCaptureDeviceTypeBuiltInDuoCamera, AVCaptureDeviceTypeBuiltInWideAngleCamera, AVCaptureDeviceTypeBuiltInTelephotoCamera ];
    AVCaptureDeviceDiscoverySession *discoverySession = [AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:allTypes mediaType:AVMediaTypeVideo position:AVCaptureDevicePositionBack];

    for(AVCaptureDevice *device in discoverySession.devices) {
        if(device.deviceType== AVCaptureDeviceTypeBuiltInDuoCamera){
            self.backCamera = device;
            self.backCameraInput = [AVCaptureDeviceInput deviceInputWithDevice:self.backCamera error:&error];
        }
    }
}

if(!self.backCamera){
    self.backCamera = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
    self.backCameraInput = [AVCaptureDeviceInput deviceInputWithDevice:self.backCamera error:&error];
}

To use wide and tele camera individually

@property (nonatomic) AVCaptureDevice *backCamera;
@property (nonatomic) AVCaptureDeviceInput *backCameraInput;
@property (nonatomic) AVCaptureDevice *teleCamera;
@property (nonatomic) AVCaptureDeviceInput *teleCameraInput;


if([AVCaptureDeviceDiscoverySession class]){
    NSArray *allTypes = @[AVCaptureDeviceTypeBuiltInDuoCamera, AVCaptureDeviceTypeBuiltInWideAngleCamera, AVCaptureDeviceTypeBuiltInTelephotoCamera ];
    AVCaptureDeviceDiscoverySession *discoverySession = [AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:allTypes mediaType:AVMediaTypeVideo position:AVCaptureDevicePositionBack];

    for(AVCaptureDevice *device in discoverySession.devices) {
        if(device.deviceType==AVCaptureDeviceTypeBuiltInWideAngleCamera){
            self.backCamera = device;
            self.backCameraInput = [AVCaptureDeviceInput deviceInputWithDevice:self.backCamera error:&error];
        }

        if(device.deviceType==AVCaptureDeviceTypeBuiltInTelephotoCamera){
            self.teleCamera = device;
            self.teleCameraInput = [AVCaptureDeviceInput deviceInputWithDevice:self.teleCamera error:&error];
        }
    }
}

if(!self.backCamera){
    self.backCamera = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
    self.backCameraInput = [AVCaptureDeviceInput deviceInputWithDevice:self.backCamera error:&error];
}

If you don't do this, or keep your old code you will only use the wide camera even if you zoom.

EDIT: In iOS 11 there is new AVCapturePhotoSettings called dualCameraDualPhotoDeliveryEnabled. It allows you to take two still images simultaneously, however, no streaming/video.

Answer 2

In the updated AVFoundation documentation ( AVCaptureDeviceType ) there're new device types: builtInWideAngleCamera and builtInTelephotoCamera . Hence, it should be possible to create multiple capture sessions and get the feedback from both of them at the same time.