| @@ -5,7 +5,7 @@ Market | |||
| The learnware market receives high-performance machine learning models from developers, incorporates them into the system, and provides services to users by identifying and reusing learnware to help users solve current tasks. Developers voluntarily submit various learnwares to the learnware market, and the market conducts quality checks and further organization of these learnwares. When users submit task requirements, the learnware market automatically selects whether to recommend a single learnware or a combination of multiple learnwares. | |||
| The learnware market will receive various kinds of learnwares, and learnwares from different feature/label spaces form numerous islands of specifications. All these islands together constitute the "specification world" in the learnware market. The market should discover and establish connections between different islands, and then merge them into a unified specification world. This further organization of learnwares can make the learnware search among all learnwares, not just learnwares which has the same feature space and label space as the user's task requirements. | |||
| The learnware market will receive various kinds of learnwares, and learnwares from different feature/label spaces form numerous islands of specifications. All these islands together constitute the "specification world" in the learnware market. The market should discover and establish connections between different islands, and then merge them into a unified specification world. This further organization of learnwares can make the learnware search among all learnwares, not just learnwares which has the same feature space and label space with the user's task requirements. | |||
| Framework | |||
| ====================================== | |||
| @@ -40,17 +40,17 @@ One important case is that models have different feature spaces. In order to ena | |||
| Current Checkers | |||
| ====================================== | |||
| The checkers check a learnware object in different aspects, including environment configuration (``CondaChecker``), semantic specifications (``EasySemanticChecker``), and statistical specifications (``EasyStatChecker``). The ``__call__`` method of each checker is designed to be invoked as a function to conduct the respective checks on the learnware and return the outcomes. Currently, we have three checkers, which are described below. | |||
| The checkers check a learnware object in different aspects, including environment configuration (``CondaChecker``), semantic specifications (``EasySemanticChecker``), and statistical specifications (``EasyStatChecker``). The ``__call__`` method of each checker is designed to be invoked as a function to conduct the respective checks on the learnware and return the outcomes. It defines three types of learnwares: INVALID_LEARNWARE denotes the learnware does not pass the check, NONUSABLE_LEARNWARE denotes the learnware pass the check but cannot make prediction due to some env dependency, USABLE_LEARWARE denotes the leanrware pass the check and can make prediction. Currently, we have three checkers, which are described below. | |||
| CondaChecker Class | |||
| ------------------ | |||
| This checker checks a the environment of the learnware object. It creates a ``LearnwaresContainer`` instance to handle the Learnware and uses ``inner_checker`` to check the Learnware. If an exception occurs, it logs the error and returns ``BaseChecker.INVALID_LEARNWARE`` status and error message. | |||
| This checker checks a the environment of the learnware object. It creates a ``LearnwaresContainer`` instance to handle the Learnware and uses ``inner_checker`` to check the Learnware. If an exception occurs, it logs the error and returns ``BaseChecker.NONUSABLE_LEARNWARE`` status and error message. | |||
| EasySemanticChecker Class | |||
| ------------------------- | |||
| This checker checks the semantic specification of a learnware object. It checks if the given semantic specification conforms to predefined standards. It verifies each key in ``semantic_spec`` dictionary against expected types and values. If the check fails, it logs the error and returns ``INVALID_LEARNWARE`` status and error message. | |||
| This checker checks the semantic specification of a learnware object. It checks if the given semantic specification conforms to predefined standards. It verifies each key in predefined dictionary. If the check fails, it logs the error and returns ``NONUSABLE_LEARNWARE`` status and error message. | |||
| EasyStatChecker Class | |||
| --------------------- | |||
| This checker checks the statistical specification and functionality of a learnware object. It performs multiple checks to validate the learnware. It checks for model instantiation, verifies input shape and statistical specifications, and test output shape and dimensions using random generated data. In case of any exceptions, it logs the error and returns ``INVALID_LEARNWARE`` status and error message. | |||
| This checker checks the statistical specification and functionality of a learnware object. It performs multiple checks to validate the learnware. It checks for model instantiation, verifies input shape and statistical specifications, and test output shape using random generated data. In case of any exceptions, it logs the error and returns ``NONUSABLE_LEARNWARE`` status and error message. | |||