Browse Source

[MNT ]Update quick.rst

tags/v0.3.2
HarryLiu-34 GitHub 2 years ago
parent
commit
1d2cb45c95
No known key found for this signature in database GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 73 additions and 49 deletions
  1. +73
    -49
      docs/start/quick.rst

+ 73
- 49
docs/start/quick.rst View File

@@ -1,4 +1,5 @@
.. _quick:

============================================================
Quick Start
============================================================
@@ -7,13 +8,14 @@ Quick Start
Introduction
====================

This ``Quick Start`` guide tries to demonstrate that it's very easy to build a complete ``Learnware Market`` workflow and use ``Learnware Market`` to deal with users' tasks.
This ``Quick Start`` guide aims to illustrate the straightforward process of establishing a full ``Learnware Market`` workflow
and utilizing ``Learnware Market`` to handle user tasks.


Installation
====================

Learnware is currently hosted on `PyPI <https://pypi.org/>`__. You can easily intsall ``learnware`` according to the following steps:
Learnware is currently hosted on `PyPI <https://pypi.org/>`__. You can easily intsall ``learnware`` by following these steps:

- For Windows and Linux users:

@@ -32,51 +34,70 @@ Learnware is currently hosted on `PyPI <https://pypi.org/>`__. You can easily in
Prepare Learnware
====================

The Learnware Market consists of a wide range of learnwares. A valid learnware is a zipfile which
is composed of the following four parts.
The Learnware Market encompasses a board variety of learnwares. A valid learnware is a zipfile that
includes the following four components:

- ``__init__.py``

A python file offering interfaces for your model's fitting, predicting and fine-tuning.
A Python file that provides interfaces for fitting, predicting, and fine-tuning your model.

- ``rkme.json``

A json file containing the statistical specification of your data.
A JSON file that contains the statistical specification of your data.

- ``learnware.yaml``
A config file describing your model class name, type of statistical specification(e.g. Reduced Kernel Mean Embedding, ``RKMEStatSpecification``), and
A configuration file that details your model's class name, the type of statistical specification(e.g. ``RKMEStatSpecification`` for Reduced Kernel Mean Embedding), and
the file name of your statistical specification file.

- ``environment.yaml``
- ``environment.yaml`` or ``requirements.txt``

- ``environment.yaml`` for conda:

A Conda environment configuration file for running the model. If the model environment is incompatible, this file can be used for manual configuration.
Here's how you can generate this file:

- Create env config for conda:

- For Windows users:
.. code-block::

conda env export | findstr /v "^prefix: " > environment.yaml
- For macOS and Linux users

A Conda environment configuration file for running the model (if the model environment is incompatible, you can rely on this for manual configuration).
You can generate this file according to the following steps:
.. code-block::

- Create env config for conda:
conda env export | grep -v "^prefix: " > environment.yaml
- Recover env from config:

.. code-block::
.. code-block::

conda env export | grep -v "^prefix: " > environment.yaml
- Recover env from config:
conda env create -f environment.yaml
- ``requirements.txt`` for pip:

A plain text documents that lists all packages necessary for executing the model. These dependencies can be effortlessly installed using pip with the command:

.. code-block::
.. code-block::
pip install -r requirements.txt.

conda env create -f environment.yaml
We've also detailed the format of the learnware zipfile in :ref:`Learnware Preparation<workflow/submit:Prepare Learnware>`.

We also demonstrate the detail format of learnware zipfile in [DOC link], and also please refer to [Code link] for concrete learnware zipfile example.

Learnware Market Workflow
============================

Users can start an ``Learnware Market`` workflow according to the following steps:
Users can start a ``Learnware Market`` workflow according to the following steps:

Initialize a Learware Market
-------------------------------

The ``EasyMarket`` class implements the most basic set of functions in a ``Learnware Market``.
You can use the following code snippet to initialize a basic ``Learnware Market`` named "demo":
The ``EasyMarket`` class provides the core functions of a ``Learnware Market``.
You can initialize a basic ``Learnware Market`` named "demo" using the code snippet below:

.. code-block:: python
@@ -86,15 +107,16 @@ You can use the following code snippet to initialize a basic ``Learnware Market`
learnware.init()
easy_market = EasyMarket(market_id="demo", rebuild=True)

Upload Leanwares

Upload Leanware
-------------------------------

Before uploading your learnware into the ``Learnware Market``,
create a semantic specification ``semantic_spec`` by selecting or filling in values for the predefined semantic tags
Before uploading your learnware to the ``Learnware Market``,
you'll need to create a semantic specification, ``semantic_spec``. This involves selecting or inputting values for predefined semantic tags
to describe the features of your task and model.

For example, the following code snippet demonstrates the semantic specification
of a Scikit-Learn type model, which is designed for business scenario and performs classification on tabular data:
For instance, the dictionary snippet below illustrates the semantic specification for a Scikit-Learn type model.
This model is tailored for business scenarios and performs classification tasks on tabular data:

.. code-block:: python

@@ -107,8 +129,8 @@ of a Scikit-Learn type model, which is designed for business scenario and perfor
"Name": {"Values": "demo_learnware", "Type": "String"},
}

Once the semantic specification is defined,
you can easily upload your learnware with a single line of code:
After defining the semantic specification,
you can upload your learnware using a single line of code:
.. code-block:: python
@@ -116,41 +138,41 @@ you can easily upload your learnware with a single line of code:

Here, ``zip_path`` is the directory of your learnware zipfile.


Semantic Specification Search
-------------------------------

To search for learnwares that fit your task purpose,
you should also provide a semantic specification ``user_semantic`` that describes the characteristics of your task.
The ``Learnware Market`` will perform a first-stage search based on ``user_semantic``,
identifying potentially helpful leranwares whose models solve tasks similar to your requirements.
To find learnwares that align with your task's purpose, you'll need to provide a semantic specification, ``user_semantic``, that outlines your task's characteristics.
The ``Learnware Market`` will then perform an initial search using ``user_semantic``, identifying potentially useful learnwares with models that solve tasks similar to your requirements.

.. code-block:: python

# construct user_info which includes semantic specification for searching learnware
# construct user_info which includes a semantic specification
user_info = BaseUserInfo(id="user", semantic_spec=semantic_spec)

# search_learnware performs semantic specification search if user_info doesn't include a statistical specification
# search_learnware: performs semantic specification search when user_info doesn't include a statistical specification
_, single_learnware_list, _ = easy_market.search_learnware(user_info)

# single_learnware_list is the learnware list by semantic specification searching
# single_learnware_list: the learnware list returned by semantic specification search
print(single_learnware_list)

Statistical Specification Search
---------------------------------

If you choose to porvide your own statistical specification file ``stat.json``,
the ``Learnware Market`` can perform a more accurate leanware selection from
the learnwares returned by the previous step. This second-stage search is based on statistical information
and returns one or more learnwares that are most likely to be helpful for your task.
If you decide in favor of porviding your own statistical specification file, ``stat.json``,
the ``Learnware Market`` can further refine the selection of learnwares from the previous step.
This second-stage search leverages statistical information to identify one or more learnwares that are most likely to be beneficial for your task.

For example, the following code is designed to work with Reduced Set Kernel Embedding as a statistical specification:
For example, the code below executes learnware search when using Reduced Set Kernel Embedding as the statistical specification:

.. code-block:: python

import learnware.specification as specification

user_spec = specification.rkme.RKMEStatSpecification()

# unzip_path: directory for unzipped learnware zipfile
user_spec.load(os.path.join(unzip_path, "rkme.json"))
user_info = BaseUserInfo(
semantic_spec=user_semantic, stat_info={"RKMEStatSpecification": user_spec}
@@ -158,26 +180,25 @@ For example, the following code is designed to work with Reduced Set Kernel Embe
(sorted_score_list, single_learnware_list,
mixture_score, mixture_learnware_list) = easy_market.search_learnware(user_info)

# sorted_score_list is the learnware scores based on MMD distances, sorted in descending order
# sorted_score_list: learnware scores(based on MMD distances), sorted in descending order
print(sorted_score_list)

# single_learnware_list is the learnwares sorted in descending order based on their scores
# single_learnware_list: learnwares, sorted by scores in descending order
print(single_learnware_list)

# mixture_learnware_list is the learnwares whose mixture is helpful for your task
# mixture_learnware_list: collection of learnwares whose combined use is beneficial
print(mixture_learnware_list)

# mixture_score is the score of the mixture of learnwares
# mixture_score: score assigned to the combined set of learnwares in `mixture_learnware_list`
print(mixture_score)


Reuse Learnwares
-------------------------------

Based on the returned list of learnwares ``mixture_learnware_list`` in the previous step,
you can easily reuse them to make predictions your own data, instead of training a model from scratch.
We provide two baseline methods for reusing a given list of learnwares, namely ``JobSelectorReuser`` and ``AveragingReuser``.
Simply replace ``test_x`` in the code snippet below with your own testing data and start reusing learnwares!
With the list of learnwares, ``mixture_learnware_list``, returned from the previous step, you can readily apply them to make predictions on your own data, bypassing the need to train a model from scratch.
We offer two baseline methods for reusing a given list of learnwares: ``JobSelectorReuser`` and ``AveragingReuser``.
Just substitute ``test_x`` in the code snippet below with your own testing data, and you're all set to reuse learnwares!

.. code-block:: python

@@ -189,7 +210,10 @@ Simply replace ``test_x`` in the code snippet below with your own testing data a
reuse_ensemble = AveragingReuser(learnware_list=mixture_learnware_list)
ensemble_predict_y = reuse_ensemble.predict(user_data=test_x)


Auto Workflow Example
============================

``Learnware Market`` also provides an auto workflow example, which includes preparing learnwares, upload and delete learnware from markets, search learnware with semantic specifications and statistical specifications. The users can run ``examples/workflow_by_code.py`` to try the basic workflow of ``Learnware Market``.
The ``Learnware Market`` also offers an automated workflow example.
This includes preparing learnwares, uploading and deleting learnwares from the market, and searching for learnwares using both semantic and statistical specifications.
To experience the basic workflow of the Learnware Market, users can run [workflow code link].

Loading…
Cancel
Save