python3 - edit value in YAML file in place

Question

I am trying to use PyYAML and Python3 to edit values within a YAML file (it's a Hiera data structure, but that's a little beside the point).

It's easy enough to achieve with yaml.safe_load and yaml.dump . The problem I have is that I need to preserve the comments, whitespace, ordering, and other formatting. That is to say, when I edit the value for a given key, that should be the only change within the file. (There is no requirement to add or remove keys themselves.)

In my case, these are second level keys. I can do it with a regex or some form of state machine - but it's pretty nasty. Is anyone aware of a library that does this neatly already?

Here's a dummied up example of the YAML concerned:

---
# Some form of comment block.
# Some form of comment block.
# Some form of comment block.

# This is my config block.
config::me:
  key0: 123
  key1: 456

# This is another config block:
applications:
  frontend:
    version: '2.4.2'
    enabled: true
  backend:
    version: '4.3.9'
    enabled: false

# More comments etcetera.

Essentially what I need to be able to do is target applications.frontent.version and update the value from 2.4.2 to 2.4.3 without touching anything else in the file.

Answer 1

You can use ruamel.yaml for this which is a derivative of PyYAML ¹. It was especially created to do this kind of round-tripping, preserving comments, and several other things from the original file, that most parsers discard of during round-tripping (it is also YAML 1.2 compatible, whereas PyYAML supports 1.1)

import sys
import ruamel.yaml
from ruamel.yaml.util import load_yaml_guess_indent
from ruamel.yaml.scalarstring import SingleQuotedScalarString

ys = """\
---
# Some form of comment block.
# Some form of comment block.
# Some form of comment block.

# This is my config block.
config::me:
  key0: 123
  key1: 456

# This is another config block:
applications:
  frontend:
    version: '2.4.2'
    enabled: true
  backend:
    version: '4.3.9'
    enabled: false

# More comments etcetera.
"""

data, indent, block_seq_indent = load_yaml_guess_indent(ys, preserve_quotes=True)
data['applications']['frontend']['version'] = SingleQuotedScalarString('2.4.3')
ruamel.yaml.round_trip_dump(data, sys.stdout, explicit_start=True)

gives you:

---
# Some form of comment block.
# Some form of comment block.
# Some form of comment block.

# This is my config block.
config::me:
  key0: 123
  key1: 456

# This is another config block:
applications:
  frontend:
    version: '2.4.3'
    enabled: true
  backend:
    version: '4.3.9'
    enabled: false

# More comments etcetera.

The preserve_quotes option and use of SingleQuotedScalarString() are only necessary because you have quotes around the scalars 2.4.3 and 4.3.9 that are superfluous and would normally be dropped.

---

¹ _{Disclaimer: I am the author of ruamel.yaml .}