Model SageMaker中的管理：Model Package组的使用

Question

We have a huge number of models which we plan to train/test/deploy/use. We have several levels of products - starting from Country -> Region -> Warehouse -> Department -> Class A -> Class B -> Product ID etc., Some of the models will be for overall country level, some Country+Region level, and of course, similarly it will go down to the Class B and Product level. Which means if there are total 10,000 products - we will have 10,000 models in the product level. In this kind of scenario, where we can have 10s of thousands of models, we were wondering how to manage these models. Naming convention as well as grouping them together for easy look up etc. are part of the model management also. Our initial thought was to use Sagemaker Model Package Group to group these models - so we can easily organize and find them as needed. However, I just learned that one SageMaker Model Package Group is a flat structure - which means one Model Package Group cannot contain another Model Package Group - so this hierarchical organization of models could not be replicated using Model Package Group.

Question is: How do we achieve this kind model organization in production? This is not an unusual situation or anything, I believe there are many retailers who are dealing with model management scenarios like this. How do they do it? If model package is not used, what else can be used to organize them and how?

Any suggestions/idea will be highly appreciated.

Thanks

Answer 1

Using SageMaker Model registry, you can catalog models by creating model package groups that contain * different versions of a model . You can create a model group that tracks all of the models that you train to solve a particular problem .

So the basic idea is that you have a business problem and you create 1 group for it and store all the models along with their versions.

Going by that definition, having a model package group under another model package group is an invalid scenario. Thats not how model group is defined and not how model group is intended to be used.

Also, this problem is not only restricted to models and you can see it everywhere is s/w engineering. Whether is it storing jar files or python packages etc.

And i would suggest to treat this problem as a software engineering problem and rely on namespace convention. Rather than thinking in a hierarchical way: Country -> Region -> Warehouse -> Department -> Class A -> Class B -> Product ID

Can we think of it in country.region.warehouse.department.classA.classB.product_id ?

Similar to how you also might be managing the docker images you can create such namespaces and go as deep as you want. You are still respecting the hierarchy but also not maintaining the hierarchy per say.

Let me know what you think of this approach.