如何在使用 ruamel 更新 yaml 文件時保持格式？

Question

我有一個多文檔 YAML 文件。 我只對修改第三個文檔感興趣（稍后將使用其他代碼和條件進行此修改）。 經過一些研究，我選擇了 ruamel，因為據報道它可以保留順序和格式。

我的 YAML 看起來像這樣（不包括整個內容，因為它超過 3000 行）：

---
"SOURCE": "mmmmm"
"VERSION": "5.4.2"
"DATE_WRITTEN": "Tue Oct 25 06:09:34 2022"
"CONFIG_CHECKSUM": "0XCD44F064"
"CONFIG_SIZE": "231212"
...
---
"moduleVersion": ["5.4.2   (AUG 2022)", "20:FIO w/2070-2A"]
"moduleModel": ["mmmmm", "mmmmm Linux Actuated Controller Unit"]
"maxPhases": 16
"maxVehicleDetectors": 72
"maxPedestrianDetectors": 8
"etcsAscPhsBanksMax": 4
"maxOverlaps": 16
"maxRings": 4
"etcsAscPriorityBanksMax": 4
"etcsAscMaxPriorityQueues": 6
"maxPatterns": 253
"etcsAscSFMapsMaskSize": 16
"etcsAscPFMapsMaskSize": 16
"etcsMaxSpcFuncMaps": 47
"etcsMaxPhsFuncMaps": 192
"maxTimebaseAscActions": 255
"maxTimebaseScheduleEntries": 255
"maxDayPlanEvents": 15
"maxDayPlans": 255
"maxDaylightSavingEntries": 2
"rs232Number": 3
"maxSequences": 16
"etcsAscMaxSerialPorts": 2
"maxChannels": 32
"ipAdEntAddr": [[192, 168, 1, 100], [192, 168, 0, 77]]
"etcsAscMaxSpatDestinations": 16
"etcsUnitBankMax": 4
"etcsMaxOutputLoadswitches": 32
"etcsPeerFunctionMax": 64
"etcsAscMaxPriorities": 12
"maxSplits": 253
"maxPreempts": 12
...
---
"phaseWalk": [0, 7, 0, 7, 0, 7, 0, 7, 0, 0, 0, 0, 0, 0, 0, 7]
"phasePedestrianClear": [0, 28, 0, 32, 0, 28, 0, 32, 0, 0, 0, 0, 0, 0, 0, 32]
"phaseMinimumGreen": [5, 7, 5, 7, 5, 7, 5, 7, 0, 0, 0, 0, 0, 0, 0, 7]
"phasePassage": [20, 10, 20, 25, 20, 10, 20, 25, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseMaximum1": [5, 25, 5, 15, 5, 25, 5, 15, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseMaximum2": [20, 0, 20, 55, 20, 0, 20, 65, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseYellowChange": [44, 44, 40, 40, 44, 44, 40, 40, 0, 0, 0, 0, 0, 0, 0, 30]
"phaseRedClear": [20, 20, 26, 26, 20, 20, 26, 26, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseAddedInitial": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseMaximumInitial": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseTimeBeforeReduction": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseCarsBeforeReduction": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseTimeToReduce": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseMinimumGap": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseDynamicMaxLimit": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseDynamicMaxStep": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseStartup": [2, 3, 2, 2, 2, 3, 2, 2, 1, 1, 1, 1, 1, 1, 1, 2]
"phaseOptions": [33, 165, 33, 1059, 33, 165, 33, 1059, 0, 0, 0, 0, 0, 0, 0, 1]
"phaseConcurrency": [[5, 6], [5, 6], [7, 8], [7, 8], [1, 2], [1, 2], [3, 4], [3, 4], [], [], [], [], [], [], [], []]
"etcsAscPhaseFlashWalk": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"etcsAscPhaseExtPedClear": [0, 3, 0, 3, 0, 3, 0, 3, 0, 0, 0, 0, 0, 0, 0, 3]

這是我的代碼：

#importing the yaml
directory = input("Please enter the directory path: ")
yml_file = glob.glob('*.yaml')
import ruamel.yaml

try:
    # Create a new YAML object
    yaml = ruamel.yaml.YAML()
    # Open the YAML file in read-write mode
    with open(yml_file[0], 'r+') as file:
        # Load the YAML documents using the ruamel.yaml.safe_load() method
        data = list(yaml.load_all(file))
        # Get the third document
        doc3 = data[2]
        # Make changes to the third document
        doc3["phaseWalk"][0] == 5
        # Seek to the beginning of the file
        file.seek(0)
        # Overwrite the file with the updated documents
        yaml.dump_all(data, file)
        file.write("...\n")
        # Close the file
        file.truncate()

except FileNotFoundError:
    print("The file 'file.yaml' was not found.")

except PermissionError:
    print("You do not have permission to write to the file 'file.yaml'.")

except Exception as e:
    print(f"An unexpected error occurred: {e}")

它產生以下 YAML：

SOURCE: mmmmm
VERSION: 5.4.2
DATE_WRITTEN: Tue Oct 25 06:09:34 2022
CONFIG_CHECKSUM: 0XCD44F064
CONFIG_SIZE: '231212'
---
moduleVersion: [5.4.2   (AUG 2022), 20:FIO w/2070-2A]
moduleModel: [mmmmm, mmmmm Linux Actuated Controller Unit]
maxPhases: 16
maxVehicleDetectors: 72
maxPedestrianDetectors: 8
etcsAscPhsBanksMax: 4
maxOverlaps: 16
maxRings: 4
etcsAscPriorityBanksMax: 4
etcsAscMaxPriorityQueues: 6
maxPatterns: 253
etcsAscSFMapsMaskSize: 16
etcsAscPFMapsMaskSize: 16
etcsMaxSpcFuncMaps: 47
etcsMaxPhsFuncMaps: 192
maxTimebaseAscActions: 255
maxTimebaseScheduleEntries: 255
maxDayPlanEvents: 15
maxDayPlans: 255
maxDaylightSavingEntries: 2
rs232Number: 3
maxSequences: 16
etcsAscMaxSerialPorts: 2
maxChannels: 32
ipAdEntAddr: [[192, 168, 1, 100], [192, 168, 0, 77]]
etcsAscMaxSpatDestinations: 16
etcsUnitBankMax: 4
etcsMaxOutputLoadswitches: 32
etcsPeerFunctionMax: 64
etcsAscMaxPriorities: 12
maxSplits: 253
maxPreempts: 12
---
phaseWalk: [0, 7, 0, 7, 0, 7, 0, 7, 0, 0, 0, 0, 0, 0, 0, 7]
phasePedestrianClear: [0, 28, 0, 32, 0, 28, 0, 32, 0, 0, 0, 0, 0, 0, 0, 32]
phaseMinimumGreen: [5, 7, 5, 7, 5, 7, 5, 7, 0, 0, 0, 0, 0, 0, 0, 7]
phasePassage: [20, 10, 20, 25, 20, 10, 20, 25, 0, 0, 0, 0, 0, 0, 0, 0]
phaseMaximum1: [5, 25, 5, 15, 5, 25, 5, 15, 0, 0, 0, 0, 0, 0, 0, 0]
phaseMaximum2: [20, 0, 20, 55, 20, 0, 20, 65, 0, 0, 0, 0, 0, 0, 0, 0]
phaseYellowChange: [44, 44, 40, 40, 44, 44, 40, 40, 0, 0, 0, 0, 0, 0, 0, 30]
phaseRedClear: [20, 20, 26, 26, 20, 20, 26, 26, 0, 0, 0, 0, 0, 0, 0, 0]
phaseAddedInitial: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
phaseMaximumInitial: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
phaseTimeBeforeReduction: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
phaseCarsBeforeReduction: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
phaseTimeToReduce: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
phaseMinimumGap: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
phaseDynamicMaxLimit: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
phaseDynamicMaxStep: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
phaseStartup: [2, 3, 2, 2, 2, 3, 2, 2, 1, 1, 1, 1, 1, 1, 1, 2]
phaseOptions: [33, 165, 33, 1059, 33, 165, 33, 1059, 0, 0, 0, 0, 0, 0, 0, 1]
phaseConcurrency: [[5, 6], [5, 6], [7, 8], [7, 8], [1, 2], [1, 2], [3, 4], [3, 4],
  [], [], [], [], [], [], [], []]
etcsAscPhaseFlashWalk: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
etcsAscPhaseExtPedClear: [0, 3, 0, 3, 0, 3, 0, 3, 0, 0, 0, 0, 0, 0, 0, 3]

我遇到的問題是：

1. 第一個文檔的前 3 個破折號在哪里？
2. 為什么更改了數據類型？ 我的大部分數據類型都定義為字符串。 他們不再那樣了。
3. 對第三個文件的修改沒有生效？ 我做錯了什么嗎？

我正在運行 ruamel v 0.17.21

Answer 1

長話短說；博士； 跳到水平線后的文本

YAML文件的各個部分根據情況是可選的，同樣多文檔文件中的文件分離也有可選部分。

從歷史上看， ruamel.yaml開始（僅）處理保留由 PyYAML 刪除的行尾注釋，方法是將 PyYAMLs 分開但在很大程度上重疊的源 Python 2 和 3（因此只需在一個地方進行更改) 然后添加保留注釋的代碼。 （源組合更改和 YAML 1.2 支持的更改首先作為 PR 請求提供給 PyYAML，但被忽略了，迫使我分叉）

其他的東西，比如縮進，被“規范化”了，即到處都一樣。 縮進仍然是標准化的，盡管您現在對映射和序列有單獨的縮進控制。

規范化通常會消除多余的元素：序列中元素之間的額外空間。 大多數這些規范化，包括刪除注釋，都在原始 PyYAML 代碼中。 鑒於解析 YAML 是一個多步驟過程（掃描、標記化、組合結構、生成 Python 個對象），如果在掃描過程中丟棄了某些內容，您可能會想象添加更改丟棄信息的復雜性。 此外，盡管 PyYAML 在內部與各種類的實例一起工作，將一個額外的參數添加到load或dump function，例如選擇性地保留標量周圍的引號，但需要在多個文件中的每個位置進行更改。 這就是為什么 ruamel.yaml 切換到使用YAML()實例，您可以在該實例上設置屬性（並且底層代碼可以根據需要進行查詢）。

除了主要根據ruamel.yaml的主要開發人員的懶惰來添加此類代碼之外，YAML 往返的某些方面也存在問題，是使用 PyYAML 的原始規范化，使規范化成為可選的還是始終保留。 除了易於實施之外，答案可能取決於個人偏好，並且兩種方式的決定並不總是一致的。

后來添加到 ruamel.yaml 的東西是保留整數/浮點數格式； 文字標量（最初）和引用/折疊標量； 根級映射冒號后的空格。 其中一些保留始終提供，一些取決於YAML()實例上的設置屬性。

---

鑒於這種情況，簡短的回答是，除非您設置.preserve_quotes （否則它們被規范化），否則標量周圍多余的引號將被刪除，並且文檔結束標記（ ... ）在不必要時不會被保留（即當有像%YAML 1.2這樣的指令時），除非你設置.explicit_end 。 所以你必須明確地告訴你的YAML()實例你想要什么。

（您仍然在CONFIG_SIZE的值周圍加上引號，否則該值將被解釋為數字。）

在我知道更改正確之前，我通常不會覆蓋輸入文件（當它們部分是您想要的並且您必須在下一次測試運行之前恢復輸入時，這很痛苦）

doc3["phaseWalk"][0] == 5行的計算結果為False並且沒有進一步的副作用，因此當然沒有修改鍵phaseWalk值的第一個元素，也沒有任何更新。

如果你運行：

import sys
import ruamel.yaml
import pathlib

path = next(Path('.').glob('*.yaml'))  # first matching path
    
yaml = ruamel.yaml.YAML()
# yaml.indent(mapping=4, sequence=4, offset=2)  # this is the default, doesn't affect your code
yaml.preserve_quotes = True  # added in ruamel.yaml
yaml.explicit_start = True   # control also available in PyYAML
yaml.explicit_end = True     # control also available in PyYAML

data = list(yaml.load_all(path))
doc3 = data[2]
doc3['phaseWalk'][0] = 5    # single '=' for assignment
yaml.dump_all(data, sys.stdout)   # yaml.dump_all(data, path) -> overwrite original file

這使：

---
"SOURCE": "mmmmm"
"VERSION": "5.4.2"
"DATE_WRITTEN": "Tue Oct 25 06:09:34 2022"
"CONFIG_CHECKSUM": "0XCD44F064"
"CONFIG_SIZE": "231212"
...
---
"moduleVersion": ["5.4.2   (AUG 2022)", "20:FIO w/2070-2A"]
"moduleModel": ["mmmmm", "mmmmm Linux Actuated Controller Unit"]
"maxPhases": 16
"maxVehicleDetectors": 72
"maxPedestrianDetectors": 8
"etcsAscPhsBanksMax": 4
"maxOverlaps": 16
"maxRings": 4
"etcsAscPriorityBanksMax": 4
"etcsAscMaxPriorityQueues": 6
"maxPatterns": 253
"etcsAscSFMapsMaskSize": 16
"etcsAscPFMapsMaskSize": 16
"etcsMaxSpcFuncMaps": 47
"etcsMaxPhsFuncMaps": 192
"maxTimebaseAscActions": 255
"maxTimebaseScheduleEntries": 255
"maxDayPlanEvents": 15
"maxDayPlans": 255
"maxDaylightSavingEntries": 2
"rs232Number": 3
"maxSequences": 16
"etcsAscMaxSerialPorts": 2
"maxChannels": 32
"ipAdEntAddr": [[192, 168, 1, 100], [192, 168, 0, 77]]
"etcsAscMaxSpatDestinations": 16
"etcsUnitBankMax": 4
"etcsMaxOutputLoadswitches": 32
"etcsPeerFunctionMax": 64
"etcsAscMaxPriorities": 12
"maxSplits": 253
"maxPreempts": 12
...
---
"phaseWalk": [5, 7, 0, 7, 0, 7, 0, 7, 0, 0, 0, 0, 0, 0, 0, 7]
"phasePedestrianClear": [0, 28, 0, 32, 0, 28, 0, 32, 0, 0, 0, 0, 0, 0, 0, 32]
"phaseMinimumGreen": [5, 7, 5, 7, 5, 7, 5, 7, 0, 0, 0, 0, 0, 0, 0, 7]
"phasePassage": [20, 10, 20, 25, 20, 10, 20, 25, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseMaximum1": [5, 25, 5, 15, 5, 25, 5, 15, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseMaximum2": [20, 0, 20, 55, 20, 0, 20, 65, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseYellowChange": [44, 44, 40, 40, 44, 44, 40, 40, 0, 0, 0, 0, 0, 0, 0, 30]
"phaseRedClear": [20, 20, 26, 26, 20, 20, 26, 26, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseAddedInitial": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseMaximumInitial": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseTimeBeforeReduction": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseCarsBeforeReduction": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseTimeToReduce": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseMinimumGap": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseDynamicMaxLimit": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseDynamicMaxStep": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"phaseStartup": [2, 3, 2, 2, 2, 3, 2, 2, 1, 1, 1, 1, 1, 1, 1, 2]
"phaseOptions": [33, 165, 33, 1059, 33, 165, 33, 1059, 0, 0, 0, 0, 0, 0, 0, 1]
"phaseConcurrency": [[5, 6], [5, 6], [7, 8], [7, 8], [1, 2], [1, 2], [3, 4], [3, 4],
  [], [], [], [], [], [], [], []]
"etcsAscPhaseFlashWalk": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
"etcsAscPhaseExtPedClear": [0, 3, 0, 3, 0, 3, 0, 3, 0, 0, 0, 0, 0, 0, 0, 3]
...

請注意，在正常情況下，您的 YAML output 和輸入都加載到相同的內部數據結構（ruamel.yaml 實現的往返解析器是一個例外）。 因此，出於實際目的，除非您必須處理不符合 YAML 規范的解析器，否則您不應該關心丟失的引號。

如果您因為比較困難而不想要這樣的更改，或者您不希望存儲庫中有這樣的額外更改，您應該考慮硬着頭皮（byte？）子彈，這正是您運行代碼時必須做的源代碼上的格式化程序（例如oitnb ）。