您如何使用过滤器收集 CKAN 数据集

Question

I'm using CKAN 2.9 version with the Remote harvesting extension.

I don't want to harvest all datasets, I want to harvest select datasets of name or of the ID within packages on a website.

Can I use a filter eg with CQL filter in the config or another way possibility (another ext?)? Do you have an idea of how I can harvest the datasets with a filter and a simple example if I have to use the config?

Answer 1

By default the CKAN harvester extension allows you to filter by organization or group datasets to be included or excluded from harvesting. For eg you can harvest the dataset only for a org_1 by adding the "organizations_filter_include": ['org_1'] key. For more info check the harvester docs in Github . If that doesn't work for you, you can use the IHarvester interface and add your own logic, which could be more complicated than using the one above provided by the harvester.