Getting error in Jupyter Notebook on Linux but not on Windows

Question

I'm trying to run the following notebook ( https://github.com/carlosnatalino/optical-rl-gym/blob/master/examples/stable_baselines/DeepRMSA.ipynb ) on Linux but I keep getting the following error in block 8:

KeyError                                  Traceback (most recent call last)
<ipython-input-8-4b84d49bf60d> in <module>()
----> 1 agent.learn(total_timesteps=100000, callback=callback)

~/.conda/envs/project4/lib/python3.7/site-packages/stable_baselines/ppo2/ppo2.py in learn(self, total_timesteps, callback, log_interval, tb_log_name, reset_num_timesteps)
    334                 callback.on_rollout_start()
    335                 # true_reward is the reward without discount
--> 336                 rollout = self.runner.run(callback)
    337                 # Unpack
    338                 obs, returns, masks, actions, values, neglogpacs, states, ep_infos, true_reward = rollout

~/.conda/envs/project4/lib/python3.7/site-packages/stable_baselines/common/runners.py in run(self, callback)
     46         self.callback = callback
     47         self.continue_training = True
---> 48         return self._run()
     49 
     50     @abstractmethod

~/.conda/envs/project4/lib/python3.7/site-packages/stable_baselines/ppo2/ppo2.py in _run(self)
    480             if isinstance(self.env.action_space, gym.spaces.Box):
    481                 clipped_actions = np.clip(actions, self.env.action_space.low, self.env.action_space.high)
--> 482             self.obs[:], rewards, self.dones, infos = self.env.step(clipped_actions)
    483 
    484             self.model.num_timesteps += self.n_envs

~/.conda/envs/project4/lib/python3.7/site-packages/stable_baselines/common/vec_env/base_vec_env.py in step(self, actions)
    148         """
    149         self.step_async(actions)
--> 150         return self.step_wait()
    151 
    152     def get_images(self, *args, **kwargs) -> Sequence[np.ndarray]:

~/.conda/envs/project4/lib/python3.7/site-packages/stable_baselines/common/vec_env/dummy_vec_env.py in step_wait(self)
     40         for env_idx in range(self.num_envs):
     41             obs, self.buf_rews[env_idx], self.buf_dones[env_idx], self.buf_infos[env_idx] =\
---> 42                 self.envs[env_idx].step(self.actions[env_idx])
     43             if self.buf_dones[env_idx]:
     44                 # save final observation where user can get it, then reset

~/.conda/envs/project4/lib/python3.7/site-packages/stable_baselines/bench/monitor.py in step(self, action)
     97             ep_info = {"r": round(ep_rew, 6), "l": eplen, "t": round(time.time() - self.t_start, 6)}
     98             for key in self.info_keywords:
---> 99                 ep_info[key] = info[key]
    100             self.episode_rewards.append(ep_rew)
    101             self.episode_lengths.append(eplen)

KeyError: 'service_blocking_rate_since_reset'

The notebook runs fine in windows(with different directory ofc) and I made sure to install the same versions of python and the same packages in both environments (made an environment.yml and installed it on linux). What is causing this error when using linux?

Answer 1

Are you sure you get this error in the DeepRMSA.pynb notebook? I get this error only in the https://github.com/carlosnatalino/optical-rl-gym/blob/main/examples/stable_baselines/QoSConstrainedRA.ipynb notebook.

There seems to be an inaccuracy at the code you point at. QoSConstrainedRA instantiates the Environment with this subclass: https://github.com/carlosnatalino/optical-rl-gym/blob/main/optical_rl_gym/envs/qos_constrained_ra.py This subclass explicitly returns info for each step with the following dictionary:

info = {
    'service_blocking_rate': (self.services_processed - self.services_accepted) / self.services_processed,
    'episode_service_blocking_rate': (self.episode_services_processed - self.episode_services_accepted) / self.episode_services_processed,
}

So, when in cell 7 the environment variable is defined with the following info_keywords :

env = Monitor(env, log_dir + 'training', info_keywords=('service_blocking_rate_since_reset',))

it cannot but crash in the following line of stable-baselines https://github.com/hill-a/stable-baselines/blob/b3f414f4f2900403107357a2206f80868af16da3/stable_baselines/bench/monitor.py#L99

For a quick fix, you can substitute the above line of cell 7 with the following:

env = Monitor(env, log_dir + 'training', info_keywords=('service_blocking_rate','episode_service_blocking_rate'))

Why would this run in Windows, I can really not tell. For me it would make sense if it didn't. Are you sure you test the same notebook in both platforms?