你如何以編程方式讀取 Tensorboard 文件？

Question

如何在不啟動 GUI tensorboard --logdir=...的情況下編寫 python 腳本來讀取 Tensorboard 日志文件，提取損失和准確度等數值數據？

Answer 1

您可以使用TensorBoard的Python類或腳本來提取數據：

如何從TensorBoard導出數據？

如果您想將數據導出到其他地方可視化（例如iPython Notebook），那也是可能的。 您可以直接依賴TensorBoard用於加載數據的基礎類： python/summary/event_accumulator.py （用於從單次運行加載數據）或python/summary/event_multiplexer.py （用於從多次運行加載數據，並保留它）組織）。 這些類加載事件文件組，丟棄TensorFlow崩潰“孤立”的數據，並按標記組織數據。

作為另一種選擇，有一個腳本（ tensorboard/scripts/serialize_tensorboard.py ）將像TensorBoard一樣加載logdir，但是將所有數據作為json寫入磁盤而不是啟動服務器。 此腳本設置為“偽TensorBoard后端”進行測試，因此它的邊緣有點粗糙。

使用EventAccumulator ：

# In [1]: from tensorflow.python.summary import event_accumulator  # deprecated
In [1]: from tensorboard.backend.event_processing import event_accumulator

In [2]: ea = event_accumulator.EventAccumulator('events.out.tfevents.x.ip-x-x-x-x',
   ...:  size_guidance={ # see below regarding this argument
   ...:      event_accumulator.COMPRESSED_HISTOGRAMS: 500,
   ...:      event_accumulator.IMAGES: 4,
   ...:      event_accumulator.AUDIO: 4,
   ...:      event_accumulator.SCALARS: 0,
   ...:      event_accumulator.HISTOGRAMS: 1,
   ...:  })

In [3]: ea.Reload() # loads events from file
Out[3]: <tensorflow.python.summary.event_accumulator.EventAccumulator at 0x7fdbe5ff59e8>

In [4]: ea.Tags()
Out[4]: 
{'audio': [],
 'compressedHistograms': [],
 'graph': True,
 'histograms': [],
 'images': [],
 'run_metadata': [],
 'scalars': ['Loss', 'Epsilon', 'Learning_rate']}

In [5]: ea.Scalars('Loss')
Out[5]: 
[ScalarEvent(wall_time=1481232633.080754, step=1, value=1.6365480422973633),
 ScalarEvent(wall_time=1481232633.2001867, step=2, value=1.2162202596664429),
 ScalarEvent(wall_time=1481232633.3877788, step=3, value=1.4660096168518066),
 ScalarEvent(wall_time=1481232633.5749283, step=4, value=1.2405034303665161),
 ScalarEvent(wall_time=1481232633.7419815, step=5, value=0.897326648235321),
 ...]

size_guidance ：

size_guidance: Information on how much data the EventAccumulator should
  store in memory. The DEFAULT_SIZE_GUIDANCE tries not to store too much
  so as to avoid OOMing the client. The size_guidance should be a map
  from a `tagType` string to an integer representing the number of
  items to keep per tag for items of that `tagType`. If the size is 0,
  all events are stored.

Answer 2

要完成user1501961的答案，您可以使用pandas pd.DataFrame(ea.Scalars('Loss)).to_csv('Loss.csv')輕松地將標量列表導出到csv文件pd.DataFrame(ea.Scalars('Loss)).to_csv('Loss.csv')

Answer 3

對於任何有興趣的人，我已將user1501961的答案改編為 function 用於將張量板標量解析為 pandas 數據幀的字典：

from tensorboard.backend.event_processing import event_accumulator
import pandas as pd


def parse_tensorboard(path, scalars):
    """returns a dictionary of pandas dataframes for each requested scalar"""
    ea = event_accumulator.EventAccumulator(
        path,
        size_guidance={event_accumulator.SCALARS: 0},
    )
    _absorb_print = ea.Reload()
    # make sure the scalars are in the event accumulator tags
    assert all(
        s in ea.Tags()["scalars"] for s in scalars
    ), "some scalars were not found in the event accumulator"
    return {k: pd.DataFrame(ea.Scalars(k)) for k in scalars}

Answer 4

嘗試這個：

    python << EOF | bat --color=always -pltsv
from contextlib import suppress
import sys
from time import gmtime, strftime

import pandas as pd
from tensorboard.backend.event_processing.event_file_loader import (
    EventFileLoader,
)

df = pd.DataFrame(columns=["Step", "Value"])
df.index.name = "YYYY-mm-dd HH:MM"

for event in EventFileLoader("$1").Load():
    with suppress(IndexError):
        df.loc[strftime("%F %H:%M", gmtime(event.wall_time))] = [  # type: ignore
            event.step,  # type: ignore
            event.summary.value[0].tensor.float_val[0],  # type: ignore
        ]
df.index = pd.to_datetime(df.index)  # type: ignore
df.Step = df.Step.astype(int)
df.to_csv(sys.stdout, "\t")
EOF

蝙蝠是最佳的。