将 2d openpose 关键点转换为 2d 图像中的 3d

Question

我正在使用https://github.com/CMU-Perceptual-Computing-Lab/openpose从图像/视频上的 openpose 获取 2d 关键点

提取的人/人 ROI 被传递给 openpoe 的关键点

for dirname, _, filenames in os.walk('/content/extracted_roi'):
  if (len(dirname.split('/'))) == 5:
    out_json_folder = os.path.join('/content/output_json', dirname.split('/')[-2], dirname.split('/')[-1])
    create_folder(out_json_folder)
    !cd openpose && ./build/examples/openpose/openpose.bin --image_dir {dirname} --write_json {out_json_folder} --display 0 --render_pose 0

import json
import numpy as np
for dirname, _, filenames in os.walk('/content/output_json'):
  print(dirname)
  json_files  = filenames
  if len(json_files) > 0:
    kps = []
    for file in json_files:
        f = open(os.path.join(dirname, file), 'r')
        data = json.load(f)
        if len(data["people"]) == 1:
          frame_kps = []
          pose_keypoints = data["people"][0]["pose_keypoints_2d"]

          #loop through 18 pose keypoints (total = 54, 18x3 (x, y and accuracy))
          j = 0
          for i in range(36):
            frame_kps.append(pose_keypoints[j])
            j += 1
            if ((j+1) % 3 == 0):
              j += 1
          kps.append(frame_kps)

    kps = np.array(kps)
    
    output_file_folder = os.path.join('/content/output_list', dirname.split('/')[-2], dirname.split('/')[-1].split('.')[0])
    create_folder(output_file_folder)
    output_file = os.path.join(output_file_folder, 'all_kp.txt')
    with open(output_file, "w") as text_file:
        for i in range(len(kps)):
          for j in range(36):
            text_file.write('{}'.format(kps[i][j]))
            if j < 35:
              text_file.write(',')
          text_file.write('\n')
    text_file.close()

如何为通过上述二维图像的人获取 3d 关键点？

Answer 1

Openpose 具有 3D 重建模块。 你可以在这里查看

https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/advanced/3d_reconstruction_module.md