如何在kedro管道中访问环境名称

Question

Is there any way to access the kedro pipeline environment name? Actually below is my problem.

I am loading the config paths as below

conf_paths = ["conf/base", "conf/local"]  
conf_loader = ConfigLoader(conf_paths)
parameters = conf_loader.get("parameters*", "parameters*/**")
catalog = conf_loader.get("catalog*")

But I have few environments like "conf/server" , "conf/test" etc, So if I have env name available I can add it to conf_paths as "conf/<env_name>" so that kedro will read the files from the respective env folder. But now if the env path is not added to conf_paths, the files are not being read by kedro even if i specify the env name while I run kedro like kedro run --env=server I searched for all the docs but was not able to find any solution.

EDIT: Elaborating more on the problem. I am using the above-given parameters and catalog dicts in the nodes. I only have keys that are common for all runs in conf/base/parameters.yml and the environment specific keys in conf/server/parameters.yml but when i do kedro run --env=server I am getting keyerror which means the keys in conf/server/parameters.yml is not available in the parameters dict. If I add conf/server to config_paths kedro is running well without keyerror.

Answer 1

You don't need to define config paths, config loader etc unless you are trying to override something.

If you are using kedro 0.17.x, the hooks.py will look something like this.

Kedro will pass, base, local and the env you specified during runtime in conf_paths into ConfigLoader .

class ProjectHooks:
    @hook_impl
    def register_config_loader(
        self, conf_paths: Iterable[str], env: str, extra_params: Dict[str, Any]
    ) -> ConfigLoader:
        return ConfigLoader(conf_paths)

    @hook_impl
    def register_catalog(
        self,
        catalog: Optional[Dict[str, Dict[str, Any]]],
        credentials: Dict[str, Dict[str, Any]],
        load_versions: Dict[str, str],
        save_version: str,
        journal: Journal,
    ) -> DataCatalog:
        return DataCatalog.from_config(
            catalog, credentials, load_versions, save_version, journal
        )

In question, I can see you have defined conf_paths and conf_loader and the env path is not present. So kedro will ignore the env passed during runtime.