| @@ -1,4 +1,5 @@ | |||
| .. _quick: | |||
| ============================================================ | |||
| Quick Start | |||
| ============================================================ | |||
| @@ -7,13 +8,14 @@ Quick Start | |||
| Introduction | |||
| ==================== | |||
| This ``Quick Start`` guide tries to demonstrate that it's very easy to build a complete ``Learnware Market`` workflow and use ``Learnware Market`` to deal with users' tasks. | |||
| This ``Quick Start`` guide aims to illustrate the straightforward process of establishing a full ``Learnware Market`` workflow | |||
| and utilizing ``Learnware Market`` to handle user tasks. | |||
| Installation | |||
| ==================== | |||
| Learnware is currently hosted on `PyPI <https://pypi.org/>`__. You can easily intsall ``learnware`` according to the following steps: | |||
| Learnware is currently hosted on `PyPI <https://pypi.org/>`__. You can easily intsall ``learnware`` by following these steps: | |||
| - For Windows and Linux users: | |||
| @@ -32,51 +34,70 @@ Learnware is currently hosted on `PyPI <https://pypi.org/>`__. You can easily in | |||
| Prepare Learnware | |||
| ==================== | |||
| The Learnware Market consists of a wide range of learnwares. A valid learnware is a zipfile which | |||
| is composed of the following four parts. | |||
| The Learnware Market encompasses a board variety of learnwares. A valid learnware is a zipfile that | |||
| includes the following four components: | |||
| - ``__init__.py`` | |||
| A python file offering interfaces for your model's fitting, predicting and fine-tuning. | |||
| A Python file that provides interfaces for fitting, predicting, and fine-tuning your model. | |||
| - ``rkme.json`` | |||
| A json file containing the statistical specification of your data. | |||
| A JSON file that contains the statistical specification of your data. | |||
| - ``learnware.yaml`` | |||
| A config file describing your model class name, type of statistical specification(e.g. Reduced Kernel Mean Embedding, ``RKMEStatSpecification``), and | |||
| A configuration file that details your model's class name, the type of statistical specification(e.g. ``RKMEStatSpecification`` for Reduced Kernel Mean Embedding), and | |||
| the file name of your statistical specification file. | |||
| - ``environment.yaml`` | |||
| - ``environment.yaml`` or ``requirements.txt`` | |||
| - ``environment.yaml`` for conda: | |||
| A Conda environment configuration file for running the model. If the model environment is incompatible, this file can be used for manual configuration. | |||
| Here's how you can generate this file: | |||
| - Create env config for conda: | |||
| - For Windows users: | |||
| .. code-block:: | |||
| conda env export | findstr /v "^prefix: " > environment.yaml | |||
| - For macOS and Linux users | |||
| A Conda environment configuration file for running the model (if the model environment is incompatible, you can rely on this for manual configuration). | |||
| You can generate this file according to the following steps: | |||
| .. code-block:: | |||
| - Create env config for conda: | |||
| conda env export | grep -v "^prefix: " > environment.yaml | |||
| - Recover env from config: | |||
| .. code-block:: | |||
| .. code-block:: | |||
| conda env export | grep -v "^prefix: " > environment.yaml | |||
| - Recover env from config: | |||
| conda env create -f environment.yaml | |||
| - ``requirements.txt`` for pip: | |||
| A plain text documents that lists all packages necessary for executing the model. These dependencies can be effortlessly installed using pip with the command: | |||
| .. code-block:: | |||
| .. code-block:: | |||
| pip install -r requirements.txt. | |||
| conda env create -f environment.yaml | |||
| We've also detailed the format of the learnware zipfile in :ref:`Learnware Preparation<workflow/submit:Prepare Learnware>`. | |||
| We also demonstrate the detail format of learnware zipfile in [DOC link], and also please refer to [Code link] for concrete learnware zipfile example. | |||
| Learnware Market Workflow | |||
| ============================ | |||
| Users can start an ``Learnware Market`` workflow according to the following steps: | |||
| Users can start a ``Learnware Market`` workflow according to the following steps: | |||
| Initialize a Learware Market | |||
| ------------------------------- | |||
| The ``EasyMarket`` class implements the most basic set of functions in a ``Learnware Market``. | |||
| You can use the following code snippet to initialize a basic ``Learnware Market`` named "demo": | |||
| The ``EasyMarket`` class provides the core functions of a ``Learnware Market``. | |||
| You can initialize a basic ``Learnware Market`` named "demo" using the code snippet below: | |||
| .. code-block:: python | |||
| @@ -86,15 +107,16 @@ You can use the following code snippet to initialize a basic ``Learnware Market` | |||
| learnware.init() | |||
| easy_market = EasyMarket(market_id="demo", rebuild=True) | |||
| Upload Leanwares | |||
| Upload Leanware | |||
| ------------------------------- | |||
| Before uploading your learnware into the ``Learnware Market``, | |||
| create a semantic specification ``semantic_spec`` by selecting or filling in values for the predefined semantic tags | |||
| Before uploading your learnware to the ``Learnware Market``, | |||
| you'll need to create a semantic specification, ``semantic_spec``. This involves selecting or inputting values for predefined semantic tags | |||
| to describe the features of your task and model. | |||
| For example, the following code snippet demonstrates the semantic specification | |||
| of a Scikit-Learn type model, which is designed for business scenario and performs classification on tabular data: | |||
| For instance, the dictionary snippet below illustrates the semantic specification for a Scikit-Learn type model. | |||
| This model is tailored for business scenarios and performs classification tasks on tabular data: | |||
| .. code-block:: python | |||
| @@ -107,8 +129,8 @@ of a Scikit-Learn type model, which is designed for business scenario and perfor | |||
| "Name": {"Values": "demo_learnware", "Type": "String"}, | |||
| } | |||
| Once the semantic specification is defined, | |||
| you can easily upload your learnware with a single line of code: | |||
| After defining the semantic specification, | |||
| you can upload your learnware using a single line of code: | |||
| .. code-block:: python | |||
| @@ -116,41 +138,41 @@ you can easily upload your learnware with a single line of code: | |||
| Here, ``zip_path`` is the directory of your learnware zipfile. | |||
| Semantic Specification Search | |||
| ------------------------------- | |||
| To search for learnwares that fit your task purpose, | |||
| you should also provide a semantic specification ``user_semantic`` that describes the characteristics of your task. | |||
| The ``Learnware Market`` will perform a first-stage search based on ``user_semantic``, | |||
| identifying potentially helpful leranwares whose models solve tasks similar to your requirements. | |||
| To find learnwares that align with your task's purpose, you'll need to provide a semantic specification, ``user_semantic``, that outlines your task's characteristics. | |||
| The ``Learnware Market`` will then perform an initial search using ``user_semantic``, identifying potentially useful learnwares with models that solve tasks similar to your requirements. | |||
| .. code-block:: python | |||
| # construct user_info which includes semantic specification for searching learnware | |||
| # construct user_info which includes a semantic specification | |||
| user_info = BaseUserInfo(id="user", semantic_spec=semantic_spec) | |||
| # search_learnware performs semantic specification search if user_info doesn't include a statistical specification | |||
| # search_learnware: performs semantic specification search when user_info doesn't include a statistical specification | |||
| _, single_learnware_list, _ = easy_market.search_learnware(user_info) | |||
| # single_learnware_list is the learnware list by semantic specification searching | |||
| # single_learnware_list: the learnware list returned by semantic specification search | |||
| print(single_learnware_list) | |||
| Statistical Specification Search | |||
| --------------------------------- | |||
| If you choose to porvide your own statistical specification file ``stat.json``, | |||
| the ``Learnware Market`` can perform a more accurate leanware selection from | |||
| the learnwares returned by the previous step. This second-stage search is based on statistical information | |||
| and returns one or more learnwares that are most likely to be helpful for your task. | |||
| If you decide in favor of porviding your own statistical specification file, ``stat.json``, | |||
| the ``Learnware Market`` can further refine the selection of learnwares from the previous step. | |||
| This second-stage search leverages statistical information to identify one or more learnwares that are most likely to be beneficial for your task. | |||
| For example, the following code is designed to work with Reduced Set Kernel Embedding as a statistical specification: | |||
| For example, the code below executes learnware search when using Reduced Set Kernel Embedding as the statistical specification: | |||
| .. code-block:: python | |||
| import learnware.specification as specification | |||
| user_spec = specification.rkme.RKMEStatSpecification() | |||
| # unzip_path: directory for unzipped learnware zipfile | |||
| user_spec.load(os.path.join(unzip_path, "rkme.json")) | |||
| user_info = BaseUserInfo( | |||
| semantic_spec=user_semantic, stat_info={"RKMEStatSpecification": user_spec} | |||
| @@ -158,26 +180,25 @@ For example, the following code is designed to work with Reduced Set Kernel Embe | |||
| (sorted_score_list, single_learnware_list, | |||
| mixture_score, mixture_learnware_list) = easy_market.search_learnware(user_info) | |||
| # sorted_score_list is the learnware scores based on MMD distances, sorted in descending order | |||
| # sorted_score_list: learnware scores(based on MMD distances), sorted in descending order | |||
| print(sorted_score_list) | |||
| # single_learnware_list is the learnwares sorted in descending order based on their scores | |||
| # single_learnware_list: learnwares, sorted by scores in descending order | |||
| print(single_learnware_list) | |||
| # mixture_learnware_list is the learnwares whose mixture is helpful for your task | |||
| # mixture_learnware_list: collection of learnwares whose combined use is beneficial | |||
| print(mixture_learnware_list) | |||
| # mixture_score is the score of the mixture of learnwares | |||
| # mixture_score: score assigned to the combined set of learnwares in `mixture_learnware_list` | |||
| print(mixture_score) | |||
| Reuse Learnwares | |||
| ------------------------------- | |||
| Based on the returned list of learnwares ``mixture_learnware_list`` in the previous step, | |||
| you can easily reuse them to make predictions your own data, instead of training a model from scratch. | |||
| We provide two baseline methods for reusing a given list of learnwares, namely ``JobSelectorReuser`` and ``AveragingReuser``. | |||
| Simply replace ``test_x`` in the code snippet below with your own testing data and start reusing learnwares! | |||
| With the list of learnwares, ``mixture_learnware_list``, returned from the previous step, you can readily apply them to make predictions on your own data, bypassing the need to train a model from scratch. | |||
| We offer two baseline methods for reusing a given list of learnwares: ``JobSelectorReuser`` and ``AveragingReuser``. | |||
| Just substitute ``test_x`` in the code snippet below with your own testing data, and you're all set to reuse learnwares! | |||
| .. code-block:: python | |||
| @@ -189,7 +210,10 @@ Simply replace ``test_x`` in the code snippet below with your own testing data a | |||
| reuse_ensemble = AveragingReuser(learnware_list=mixture_learnware_list) | |||
| ensemble_predict_y = reuse_ensemble.predict(user_data=test_x) | |||
| Auto Workflow Example | |||
| ============================ | |||
| ``Learnware Market`` also provides an auto workflow example, which includes preparing learnwares, upload and delete learnware from markets, search learnware with semantic specifications and statistical specifications. The users can run ``examples/workflow_by_code.py`` to try the basic workflow of ``Learnware Market``. | |||
| The ``Learnware Market`` also offers an automated workflow example. | |||
| This includes preparing learnwares, uploading and deleting learnwares from the market, and searching for learnwares using both semantic and statistical specifications. | |||
| To experience the basic workflow of the Learnware Market, users can run [workflow code link]. | |||