如何使用 Detectron2 在视频上获得训练有素的 model 结果？

Question

I am new on using Detectron2. I want to load the video from local drive. And then, do detection using my trained model using Detectron2's VideoVisualizer.

I tried to find a tutorial about this. But it does not exist. Could you please what do I do?

Thank you

import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()

# import some common libraries
import numpy as np
import tqdm
import cv2

# import some common detectron2 utilities
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.video_visualizer import VideoVisualizer
from detectron2.utils.visualizer import ColorMode, Visualizer
from detectron2.data import MetadataCatalog
import time

video = cv2.VideoCapture('gdrive/My Drive/video.mp4')
width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))


cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.OUTPUT_DIR = 'gdrive/My Drive/mask_rcnn/'
cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.7  # set threshold for this model 
predictor = DefaultPredictor(cfg)

v = VideoVisualizer(MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), ColorMode.IMAGE)

Answer 1

First, check the following tutorial (you can skip training parts if you don't want to train on your own data). https://colab.research.google.com/drive/16jcaJoc6bCFAQ96jDe2HwtXj7BMD_-m5#scrollTo=Vk4gID50K03a

Then, look at the following code to inference on video. https://github.com/facebookresearch/detectron2/blob/master/demo/demo.py