learnware/docs/start/quick.rst

.. _quick:
============================================================
Quick Start
============================================================


Introduction
==================== 

This ``Quick Start`` guide tries to demonstrate

- It's very easy to build a complete Learnware Market workflow and use ``learnware`` to deal with users' tasks.


Installation
====================

Learnware is currently hosted on `PyPI <https://pypi.org/>`__. You can easily intsall ``learnware`` according to the following steps:

- For Windows and Linux users:

    .. code-block::

        pip install learnware

- For macOS users:

    .. code-block::

        conda install -c pytorch fais
        pip install learnware


Prepare Learnware
====================

The Learnware Market consists of a wide range of learnwares. A valid learnware is a zip file which 
is composed of the following four parts.

- ``__init__.py``

    A python file offering interfaces for your model's fitting, predicting and fine-tuning.

- ``rkme.json``

    A json file containing the statistical specification of your data. 

- ``learnware.yaml``
    
    A config file describing your model class name, type of statistical specification(e.g. Reduced Kernel Mean Embedding, ``RKMEStatSpecification``), and 
    the file name of your statistical specification file.

- ``environment.yaml``

    A Conda environment configuration file for running the model (if the model environment is incompatible, you can rely on this for manual configuration). 
    You can generate this file according to the following steps:

    - Create env config for conda:

        .. code-block::

            conda env export | grep -v "^prefix: " > environment.yml
        
    - Recover env from config:

        .. code-block::

            conda env create -f environment.yml


Learnware Market Workflow
============================

Users can start an Learnware Market workflow according to the following steps:

Initialize a Learware Market
-------------------------------

The ``EasyMarket`` class implements the most basic set of functions in a Learnware Market. 
You can use the following code snippet to initialize a basic Learnware Market named "demo":

.. code-block:: python
    
    import learnware
    from learnware.market import EasyMarket

    learnware.init()
    easy_market = EasyMarket(market_id="demo", rebuild=True)

Upload Leanwares
-------------------------------

Before uploading your learnware into the Learnware Market,
create a semantic specification ``semantic_spec`` by selecting or filling in values for the predefined semantic tags 
to describe the features of your task and model.

For example, the following code snippet demonstrates the semantic specification 
of a Scikit-Learn type model, which is designed for business scenario and performs classification on tabular data:

.. code-block:: python

    semantic_spec = {
        "Data": {"Values": ["Tabular"], "Type": "Class"},
        "Task": {"Values": ["Classification"], "Type": "Class"},
        "Library": {"Values": ["Scikit-learn"], "Type": "Class"},
        "Scenario": {"Values": ["Business"], "Type": "Tag"},
        "Description": {"Values": "", "Type": "String"},
        "Name": {"Values": "user learnware", "Type": "String"},
    }

Once the semantic specification is defined, 
you can easily upload your learnware with a single line of code:
    
.. code-block:: python
    
    easy_market.add_learnware(zip_path, semantic_spec) 

Here, ``zip_path`` is the directory of your learnware zip file.

Semantic Specification Search
-------------------------------

To search for learnwares that fit your task purpose, 
you should also provide a semantic specification ``user_semantic`` that describes the characteristics of your task.
The Learnware Market will perform a first-stage search based on ``user_semantic``,
identifying potentially helpful leranwares whose models solve tasks similar to your requirements. 

.. code-block:: python

    user_semantic = semantic_spec
    user_semantic["Name"]["Values"] = ""
    user_info = BaseUserInfo(id="user", semantic_spec=user_semantic)
    _, single_learnware_list, _ = easy_market.search_learnware(user_info) 
    # search_learnware performs semantic specification search if user_info doesn't include a statistical specification

Statistical Specification Search
---------------------------------

If you choose to porvide your own statistical specification file ``stat.json``, 
the Learnware Market can perform a more accurate leanware selection from 
the learnwares returned by the previous step. This second-stage search is based on statistical information 
and returns one or more learnwares that are most likely to be helpful for your task. 

For example, the following code is designed to work with Reduced Set Kernel Embedding as a statistical specification:

.. code-block:: python

    import learnware.specification as specification

    user_spec = specification.rkme.RKMEStatSpecification()
    user_spec.load(os.path.join(unzip_path, "rkme.json"))
    user_info = BaseUserInfo(
        id="user", semantic_spec=user_semantic, stat_info={"RKMEStatSpecification": user_spec}
    )
    (sorted_score_list, single_learnware_list,
        mixture_score, mixture_learnware_list) = easy_market.search_learnware(user_info)
    print(sorted_score_list) # learnware scores based on MMD distances, sorted in descending order
    print(single_learnware_list) # learnwares sorted in descending order based on their scores
    print(mixture_learnware_list) # learnwares whose mixture is helpful for your task
    print(mixture_score) # the score of the mixture of learnwares


Reuse Learnwares
-------------------------------

Based on the returned list of learnwares ``mixture_learnware_list`` in the previous step, 
you can easily reuse them to make predictions your own data, instead of training a model from scratch. 
We provide two baseline methods for reusing a given list of learnwares, namely ``JobSelectorReuser`` and ``AveragingReuser``.
Simply replace ``test_x`` in the code snippet below with your own testing data and start reusing learnwares!

.. code-block:: python

    reuse_job_selector = JobSelectorReuser(learnware_list=mixture_learnware_list)
    job_selector_predict_y = reuse_job_selector.predict(user_data=test_x)

    reuse_ensemble = AveragingReuser(learnware_list=mixture_learnware_list)
    ensemble_predict_y = reuse_ensemble.predict(user_data=test_x)