How to skip lines when reading a yaml file in python?

Question

I'm familiar with similar questions, but they don't seem to address what should be a simple problem. I am using Python 2.7x and trying to read a YAML file that looks similar to this:

%YAML:1.0
radarData: !!opencv-matrix
rows: 5
cols: 2
dt: u
data: [0, 0, 0, 0, 0, 10, 5, 3, 1, 22]

For now I only need the 'data:' document. I have tried a vanilla approach and then tried to force skip the first 4 lines (second code snippet that is commented out). Both approaches gave errors.

import yaml
stream = file('test_0x.yml', 'r') 
yaml.load(stream)
# alternative code snippet
# with open('test_0x.yml') as f:
#  stream = f.readlines()[4:]
# yaml.load(stream)

Any suggestions about how skip the first few lines would be very appreciated.

Answer 1

Actually, you only need to skip the first 2 lines.

import yaml

skip_lines = 2
with open('test_0x.yml') as infile:
    for i in range(skip_lines):
        _ = infile.readline()
    data = yaml.load(infile)

>>> data
{'dt': 'u', 'rows': 5, 'data': [0, 0, 0, 0, 0, 10, 5, 3, 1, 22], 'cols': 2}
>>> data['data']
[0, 0, 0, 0, 0, 10, 5, 3, 1, 22]

Skipping the first 5 lines also works.

Answer 2

I completely missed the point here, but I leave my original answer at the bottom as a humbling reminder.

mhawke's answer is short and sweet, and is probably preferable. A more complicated solution: strip that malformed directive, correct your custom tag, and add a constructor for it. This has the advantage of correcting that tag wherever it appears in a file, not just in the first couple of lines.

My implementation here does have some disadvantages - it slurps whole files, and it hasn't been tested on complex data, where the effect of replacing the tag with a proper one might have different results than intended.

import yaml

def strip_malformed_directive(yaml_file):
    """
    Strip a malformed YAML directive from the top of a file.

    Returns the slurped (!) file.
    """
    lines = list(yaml_file)
    first_line = lines[0]
    if first_line.startswith('%') and ":" in first_line:
       return "\n".join(lines[1:])
    else:
       return "\n".join(lines)


def convert_opencvmatrix_tag(yaml_events):
    """
    Convert an erroneous custom tag, !!opencv-matrix, to the correct 
    !opencv-matrix, in a stream of YAML events.
    """
    for event in yaml_events:
        if hasattr(event, "tag") and event.tag == u"tag:yaml.org,2002:opencv-matrix":
            event.tag = u"!opencv-matrix"
        yield event


yaml.add_constructor("!opencv-matrix", lambda loader, node: None)
with open("test_0x.yml") as yaml_file:
    directive_processed = strip_malformed_directive(yaml_file)
    yaml_events = yaml.parse(directive_processed)
    matrix_tag_converted = convert_opencvmatrix_tag(yaml_events)
    fixed_document = yaml.emit(matrix_tag_converted)

    data = yaml.load(fixed_document)
    print data

Original Answer

That yaml.load function you're using returns a dictionary, which can be accessed like so:

import yaml

with open("test_0x.yml") as yaml_file:
    test_data = yaml.load(yaml_file)

print test_data["data"]

Does that help?

Answer 3

I have camera matrix generated by aruco_calibration_fromimages.exe , here is yml file:

%YAML:1.0
---
image_width: 4000
image_height: 3000
camera_matrix: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 3.1943912478853654e+03, 0., 1.9850941722590378e+03, 0.,
       3.2021356095317910e+03, 1.5509955246019449e+03, 0., 0., 1. ]
distortion_coefficients: !!opencv-matrix
   rows: 1
   cols: 5
   dt: d
   data: [ 1.3952810090687282e-01, -3.8313647492178071e-01,
       5.0555840762660396e-03, 2.3753464602670597e-03,
       3.3952514744179502e-01 ]

Load this yml with this code:

import cv2
fs = cv2.FileStorage("./calib_asus_chess/cam_calib_asus.yml", cv2.FILE_STORAGE_READ)
fn = fs.getNode("camera_matrix")
print(fn.mat())

And get this result:

[[  3.19439125e+03   0.00000000e+00   1.98509417e+03]
 [  0.00000000e+00   3.20213561e+03   1.55099552e+03]
 [  0.00000000e+00   0.00000000e+00   1.00000000e+00]]