如何在 python 的大查询中存储 tf-agents 的轨迹对象并将其作为轨迹对象检索回来

Question

I wanted to save the trajectories from the tf-agents into a big query table and wanted to retrieve them back as needed into python again.

In the python dataframe, the trajectories are saved as trajectory object. But, I am not sure how to save these trajectories object to big query and retrieve them back into python. Saving them as strings is not working nor is saving the individuals components (Action, reward etc. as strings) in big query.

Here is a sample trajectory object that I Wanted to save and retrieve back as a trajectory into pandas

Trajectory(
{'action': <tf.Tensor: shape=(1,), dtype=int32, numpy=array([2], dtype=int32)>,
 'discount': <tf.Tensor: shape=(1,), dtype=float32, numpy=array([0.], dtype=float32)>,
 'next_step_type': <tf.Tensor: shape=(1,), dtype=int32, numpy=array([2], dtype=int32)>,
 'observation': <tf.Tensor: shape=(1, 3), dtype=int32, numpy=array([[0, 1, 1]], dtype=int32)>,
 'policy_info': PolicyInfo(log_probability=(), predicted_rewards_mean=(), multiobjective_scalarized_predicted_rewards_mean=(), predicted_rewards_optimistic=(), predicted_rewards_sampled=(), bandit_policy_type=()),
 'reward': <tf.Tensor: shape=(1,), dtype=float32, numpy=array([-1000.], dtype=float32)>,
 'step_type': <tf.Tensor: shape=(1,), dtype=int32, numpy=array([2], dtype=int32)>})

Answer 1

Stored each trajectory as pickle data, using pickle.dumps(), to the big query column. The big query data type used is 'bytes' for the trajectory object.

Again retrieved the pickle back using pickle.dumps()