getUserMedia 和 NotReadableError 与 iOS Safari 13

Question

Edit : camera access was automatically allowed when I hosted my site with https on Github. However streaming is still not working properly, I'll make another question about that.

I have followed a tutorial on getUserMedia API. It works fine on my desktop Chrome and Safari. I have now created a tunnel from localhost 8080 with ngrok ( https://....ngrok.io) and I'm able to see my app on my phone with Safari 13.

However, it does not react when I press the "play" button that opens webcam on desktop version. When I inspect the Safari console from myiPhone, it gives me this error

Unhandled Promise Rejection: NotReadableError: The I/O read operation failed.

from index.js:76, which is const stream = await navigator.mediaDevices.getUserMedia(constraints);

I wonder if anyone could help with this? I'll attach my index.js code below. I wonder if I need to add something that asks for user's permission to use the camera?

Also: I know that getUserMedia API might get tricky to use. My original plan included an app that can access users rear camera and classify content with Tensorflow.js in my browser (browser because I would want to have a server connection). If anyone has alternative tips for doing something similar I'm really happy to hear!

const controls = document.querySelector('.controls');
const cameraOptions = document.querySelector('.video-options>select');
const video = document.querySelector('video');
const canvas = document.querySelector('canvas');
const screenshotImage = document.querySelector('img');
const buttons = [...controls.querySelectorAll('button')];
let streamStarted = false; // was false

const [play, pause, screenshot] = buttons;

const constraints = {
  video: {
    width: {
      min: 224,
      ideal: 224,
      max: 224,
    },
    height: {
      min: 224,
      ideal: 224,
      max: 224
    },
  }
};

cameraOptions.onchange = () => {
  const updatedConstraints = {
    ...constraints,
    deviceId: {
      exact: cameraOptions.value
    }
  };

  startStream(updatedConstraints);
};

play.onclick = () => {
  if (streamStarted) {
    video.play();
    play.classList.add('d-none');
    pause.classList.remove('d-none');
    return;
  }
  if ('mediaDevices' in navigator && navigator.mediaDevices.getUserMedia) {
    const updatedConstraints = {
      ...constraints,
      deviceId: {
        exact: cameraOptions.value
      }
    };
    startStream(updatedConstraints);
    console.log("Camera available")
  }
};

const pauseStream = () => {
  video.pause();
  play.classList.remove('d-none');
  pause.classList.add('d-none');
};

const doScreenshot = () => {
  canvas.width = video.videoWidth;
  canvas.height = video.videoHeight;
  canvas.getContext('2d').drawImage(video, 0, 0);
  screenshotImage.src = canvas.toDataURL('image/webp');
  screenshotImage.classList.remove('d-none');
};

pause.onclick = pauseStream;
screenshot.onclick = doScreenshot;

const startStream = async (constraints) => {
  const stream = await navigator.mediaDevices.getUserMedia(constraints);
  handleStream(stream);
};


const handleStream = (stream) => {
  video.srcObject = stream;
  play.classList.add('d-none');
  pause.classList.remove('d-none');
  screenshot.classList.remove('d-none');

};


const getCameraSelection = async () => {
  const devices = await navigator.mediaDevices.enumerateDevices();
  const videoDevices = devices.filter(device => device.kind === 'videoinput');
  const options = videoDevices.map(videoDevice => {
    return `<option value="${videoDevice.deviceId}">${videoDevice.label}</option>`;
  });
  cameraOptions.innerHTML = options.join('');
};

getCameraSelection();

Answer 1

Several possible issues.

1. Safari doesn't support image/webp as a MIME type. See here . Use screenshotImage.src = canvas.toDataURL('image/jpeg') or screenshotImage.src = canvas.toDataURL('image/png') if you need lossless coding.
2. Try relaxing your constraints, giving only ideal values.
3. getUserMedia() in Safari is a bit like the proverbial talking donkey. It's not astounding it works badly, it's astounding it works at all. You definitely need a catch handler on getUserMedia() something like this.

let devices
try {
  devices = await navigator.mediaDevices.enumerateDevices();
} catch (error) {
  console.error(error)
}

because gUM is likely to throw errors. And, of course to see the console on mobile safari you have to use desktop Safari to debug it. Lots of tutorials explain this. Here's just one. https://appletoolbox.com/use-web-inspector-debug-mobile-safari/