Browse Source

[DOC] Modify docs

tags/v0.3.2
bxdd 3 years ago
parent
commit
a6464f0dd4
6 changed files with 173 additions and 172 deletions
  1. +3
    -1
      docs/index.rst
  2. +0
    -170
      docs/introduction/quick.rst
  3. +3
    -0
      docs/start/install.rst
  4. +0
    -0
      docs/start/intro.rst
  5. +166
    -0
      docs/start/quick.rst
  6. +1
    -1
      setup.py

+ 3
- 1
docs/index.rst View File

@@ -23,7 +23,9 @@ Document Structure
:maxdepth: 3
:caption: GETTING STARTED:

Introduction <introduction/intro.rst>
Introduction <start/intro.rst>
Quick Start <start/quick.rst>
Installation Guide <start/install.rst>

.. toctree::
:maxdepth: 3


+ 0
- 170
docs/introduction/quick.rst View File

@@ -1,170 +0,0 @@
.. _quick:
============================================================
Quick Start
============================================================


Introduction
====================

This ``Quick Start`` guide tries to demonstrate

- It's very easy to build a complete Learnware Market workflow and use ``learnware`` to deal with users' tasks.


Installation
====================

Learnware is currently hosted on `PyPI <https://pypi.org/>`__. You can easily intsall ``learnware`` according to the following steps:

- For Windows and Linux users:

.. code-block::

pip install learnware

- For macOS users:

.. code-block::

conda install -c pytorch fais
pip install learnware


Prepare Learnware
====================

The Learnware Market consists of a wide range of learnwares. A valid learnware is a zip file which
is composed of the following four parts. Please refer to
:ref:`script` for examples of these components.

- ``__init__.py``

A python file offering interfaces for your model's fitting, predicting and fine-tuning.

- ``rkme.json``

A json file containing the statistical specification of your data.

- ``learnware.yaml``
A config file describing your model class name, type of statistical specification(e.g. Reduced Kernel Mean Embedding, ``RKMEStatSpecification``), and
the file name of your statistical specification file.

- ``environment.yaml``

A Conda environment configuration file for running the model (if the model environment is incompatible, you can rely on this for manual configuration).
You can generate this file according to the following steps:

- Create env config for conda:

.. code-block::

conda env export | grep -v "^prefix: " > environment.yml
- Recover env from config:

.. code-block::

conda env create -f environment.yml


Learnware Market Workflow
============================

Users can start an Learnware Market workflow according to the following steps:

1. Initialize a Learware Market:

.. code-block:: python
import learnware
from learnware.market import EasyMarket

learnware.init()
easy_market = EasyMarket(market_id="demo", rebuild=True)

2. Upload leanware:

Here, ``zip_path`` is the directory of your learnware zip file.

.. code-block:: python

semantic_spec = {
"Data": {"Values": ["Tabular"], "Type": "Class"},
"Task": {"Values": ["Classification"], "Type": "Class"},
"Library": {"Values": ["Scikit-learn"], "Type": "Class"},
"Scenario": {"Values": ["Business"], "Type": "Tag"},
"Description": {"Values": "", "Type": "String"},
"Name": {"Values": "learnware_1", "Type": "String"},
}
semantic_spec["Name"]["Values"] = "learnware_user"
semantic_spec["Description"]["Values"] = "test_learnware_user"
easy_market.add_learnware(zip_path, semantic_spec)

3. Semantic specification search:

The Learnware Market will perform first-step searching based on the semantic specification
``semantic_spec`` you provided.
This searching process will indentify potentially helpful leranwares whose models
solve tasks similar to your requirements.

.. code-block:: python

user_semantic = {
"Data": {"Values": ["Tabular"], "Type": "Class"},
"Task": {
"Values": ["Classification"],
"Type": "Class",
},
"Library": {"Values": ["Scikit-learn"], "Type": "Tag"},
"Scenario": {"Values": ["Business"], "Type": "Class"},
"Description": {"Values": "", "Type": "String"},
"Name": {"Values": "", "Type": "String"},
}
user_info = BaseUserInfo(id="user", semantic_spec=user_semantic)
_, single_learnware_list, _ = easy_market.search_learnware(user_info)

4. Statistical specification search:

If you choose to porvide your own statistical specification file ``rkme.json``,
the Learnware Market can perform a more accurate leanware selection from
the learnwares returned by the previous step. This second-step searching is carried out
at the level of data distribution information and returns
one or more learnwares that are most likely to be helpful for your task.

Here, ``unzip_path`` is the directory where you unzip your learnware file.

.. code-block:: python

import learnware.specification as specification

user_spec = specification.rkme.RKMEStatSpecification()
user_spec.load(os.path.join(unzip_path, "rkme.json"))
user_info = BaseUserInfo(
id="user", semantic_spec=user_semantic, stat_info={"RKMEStatSpecification": user_spec}
)
(sorted_score_list, single_learnware_list,
mixture_score, mixture_learnware_list) = easy_market.search_learnware(user_info)

5. Reuse learnwares:

Based on the returned list of learnwares ``mixture_learnware_list`` in the previous step,
you can easily reuse them to make predictions your own data, instead of training a model from scratch.
We provide two baseline methods for reusing a given list of learnwares, namely ``JobSelectorReuser`` and ``AveragingReuser``.

.. code-block:: python

reuse_job_selector = JobSelectorReuser(learnware_list=mixture_learnware_list)
job_selector_predict_y = reuse_job_selector.predict(user_data=test_x)

reuse_ensemble = AveragingReuser(learnware_list=mixture_learnware_list, mode='vote')
ensemble_predict_y = reuse_ensemble.predict(user_data=test_x)

.. _script:

Example: Learnware Files
-------

Below is an example learnware that includes an SVM model and uses Reduced Kernel Mean Embedding as its statistical reduction method.
We have listed the files that it needs to include.

+ 3
- 0
docs/start/install.rst View File

@@ -0,0 +1,3 @@
===================
Installation Guide
===================

docs/introduction/intro.rst → docs/start/intro.rst View File


+ 166
- 0
docs/start/quick.rst View File

@@ -0,0 +1,166 @@
.. _quick:
============================================================
Quick Start
============================================================


Introduction
====================

This ``Quick Start`` guide tries to demonstrate

- It's very easy to build a complete Learnware Market workflow and use ``learnware`` to deal with users' tasks.


Installation
====================

Learnware is currently hosted on `PyPI <https://pypi.org/>`__. You can easily intsall ``learnware`` according to the following steps:

- For Windows and Linux users:

.. code-block::

pip install learnware

- For macOS users:

.. code-block::

conda install -c pytorch fais
pip install learnware


Prepare Learnware
====================

The Learnware Market consists of a wide range of learnwares. A valid learnware is a zip file which
is composed of the following four parts.

- ``__init__.py``

A python file offering interfaces for your model's fitting, predicting and fine-tuning.

- ``rkme.json``

A json file containing the statistical specification of your data.

- ``learnware.yaml``
A config file describing your model class name, type of statistical specification(e.g. Reduced Kernel Mean Embedding, ``RKMEStatSpecification``), and
the file name of your statistical specification file.

- ``environment.yaml``

A Conda environment configuration file for running the model (if the model environment is incompatible, you can rely on this for manual configuration).
You can generate this file according to the following steps:

- Create env config for conda:

.. code-block::

conda env export | grep -v "^prefix: " > environment.yml
- Recover env from config:

.. code-block::

conda env create -f environment.yml


Learnware Market Workflow
============================

Users can start an Learnware Market workflow according to the following steps:

Initialize a Learware Market
-------------------------------

.. code-block:: python
import learnware
from learnware.market import EasyMarket

learnware.init()
easy_market = EasyMarket(market_id="demo", rebuild=True)

Upload Leanwares
-------------------------------

Here, ``zip_path`` is the directory of your learnware zip file.

.. code-block:: python

semantic_spec = {
"Data": {"Values": ["Tabular"], "Type": "Class"},
"Task": {"Values": ["Classification"], "Type": "Class"},
"Library": {"Values": ["Scikit-learn"], "Type": "Class"},
"Scenario": {"Values": ["Business"], "Type": "Tag"},
"Description": {"Values": "", "Type": "String"},
"Name": {"Values": "learnware_1", "Type": "String"},
}
semantic_spec["Name"]["Values"] = "learnware_user"
semantic_spec["Description"]["Values"] = "test_learnware_user"
easy_market.add_learnware(zip_path, semantic_spec)

Semantic Specification Search
-------------------------------

The Learnware Market will perform first-step searching based on the semantic specification
``semantic_spec`` you provided.
This searching process will indentify potentially helpful leranwares whose models
solve tasks similar to your requirements.

.. code-block:: python

user_semantic = {
"Data": {"Values": ["Tabular"], "Type": "Class"},
"Task": {
"Values": ["Classification"],
"Type": "Class",
},
"Library": {"Values": ["Scikit-learn"], "Type": "Tag"},
"Scenario": {"Values": ["Business"], "Type": "Class"},
"Description": {"Values": "", "Type": "String"},
"Name": {"Values": "", "Type": "String"},
}
user_info = BaseUserInfo(id="user", semantic_spec=user_semantic)
_, single_learnware_list, _ = easy_market.search_learnware(user_info)

Statistical Specification Search
---------------------------------

If you choose to porvide your own statistical specification file ``rkme.json``,
the Learnware Market can perform a more accurate leanware selection from
the learnwares returned by the previous step. This second-step searching is carried out
at the level of data distribution information and returns
one or more learnwares that are most likely to be helpful for your task.

Here, ``unzip_path`` is the directory where you unzip your learnware file.

.. code-block:: python

import learnware.specification as specification

user_spec = specification.rkme.RKMEStatSpecification()
user_spec.load(os.path.join(unzip_path, "rkme.json"))
user_info = BaseUserInfo(
id="user", semantic_spec=user_semantic, stat_info={"RKMEStatSpecification": user_spec}
)
(sorted_score_list, single_learnware_list,
mixture_score, mixture_learnware_list) = easy_market.search_learnware(user_info)

Reuse Learnwares
-------------------------------

Based on the returned list of learnwares ``mixture_learnware_list`` in the previous step,
you can easily reuse them to make predictions your own data, instead of training a model from scratch.
We provide two baseline methods for reusing a given list of learnwares, namely ``JobSelectorReuser`` and ``AveragingReuser``.

.. code-block:: python

reuse_job_selector = JobSelectorReuser(learnware_list=mixture_learnware_list)
job_selector_predict_y = reuse_job_selector.predict(user_data=test_x)

reuse_ensemble = AveragingReuser(learnware_list=mixture_learnware_list, mode='vote')
ensemble_predict_y = reuse_ensemble.predict(user_data=test_x)

+ 1
- 1
setup.py View File

@@ -51,7 +51,7 @@ def get_platform():
# What packages are required for this module to be executed?
# `estimator` may depend on other packages. In order to reduce dependencies, it is not written here.
REQUIRED = [
"numpy>=1.20.0",
"numpy>=1.19.2",
"pandas>=0.25.1",
"scipy>=1.0.0",
"matplotlib>=3.1.3",


Loading…
Cancel
Save