diff --git a/docs/index.rst b/docs/index.rst index e35cc7f..dad4708 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -23,7 +23,9 @@ Document Structure :maxdepth: 3 :caption: GETTING STARTED: - Introduction + Introduction + Quick Start + Installation Guide .. toctree:: :maxdepth: 3 diff --git a/docs/introduction/quick.rst b/docs/introduction/quick.rst deleted file mode 100644 index 9322077..0000000 --- a/docs/introduction/quick.rst +++ /dev/null @@ -1,170 +0,0 @@ -.. _quick: -============================================================ -Quick Start -============================================================ - - -Introduction -==================== - -This ``Quick Start`` guide tries to demonstrate - -- It's very easy to build a complete Learnware Market workflow and use ``learnware`` to deal with users' tasks. - - -Installation -==================== - -Learnware is currently hosted on `PyPI `__. You can easily intsall ``learnware`` according to the following steps: - -- For Windows and Linux users: - - .. code-block:: - - pip install learnware - -- For macOS users: - - .. code-block:: - - conda install -c pytorch fais - pip install learnware - - -Prepare Learnware -==================== - -The Learnware Market consists of a wide range of learnwares. A valid learnware is a zip file which -is composed of the following four parts. Please refer to -:ref:`script` for examples of these components. - -- ``__init__.py`` - - A python file offering interfaces for your model's fitting, predicting and fine-tuning. - -- ``rkme.json`` - - A json file containing the statistical specification of your data. - -- ``learnware.yaml`` - - A config file describing your model class name, type of statistical specification(e.g. Reduced Kernel Mean Embedding, ``RKMEStatSpecification``), and - the file name of your statistical specification file. - -- ``environment.yaml`` - - A Conda environment configuration file for running the model (if the model environment is incompatible, you can rely on this for manual configuration). - You can generate this file according to the following steps: - - - Create env config for conda: - - .. code-block:: - - conda env export | grep -v "^prefix: " > environment.yml - - - Recover env from config: - - .. code-block:: - - conda env create -f environment.yml - - -Learnware Market Workflow -============================ - -Users can start an Learnware Market workflow according to the following steps: - -1. Initialize a Learware Market: - - .. code-block:: python - - import learnware - from learnware.market import EasyMarket - - learnware.init() - easy_market = EasyMarket(market_id="demo", rebuild=True) - -2. Upload leanware: - - Here, ``zip_path`` is the directory of your learnware zip file. - - .. code-block:: python - - semantic_spec = { - "Data": {"Values": ["Tabular"], "Type": "Class"}, - "Task": {"Values": ["Classification"], "Type": "Class"}, - "Library": {"Values": ["Scikit-learn"], "Type": "Class"}, - "Scenario": {"Values": ["Business"], "Type": "Tag"}, - "Description": {"Values": "", "Type": "String"}, - "Name": {"Values": "learnware_1", "Type": "String"}, - } - semantic_spec["Name"]["Values"] = "learnware_user" - semantic_spec["Description"]["Values"] = "test_learnware_user" - easy_market.add_learnware(zip_path, semantic_spec) - -3. Semantic specification search: - - The Learnware Market will perform first-step searching based on the semantic specification - ``semantic_spec`` you provided. - This searching process will indentify potentially helpful leranwares whose models - solve tasks similar to your requirements. - - .. code-block:: python - - user_semantic = { - "Data": {"Values": ["Tabular"], "Type": "Class"}, - "Task": { - "Values": ["Classification"], - "Type": "Class", - }, - "Library": {"Values": ["Scikit-learn"], "Type": "Tag"}, - "Scenario": {"Values": ["Business"], "Type": "Class"}, - "Description": {"Values": "", "Type": "String"}, - "Name": {"Values": "", "Type": "String"}, - } - user_info = BaseUserInfo(id="user", semantic_spec=user_semantic) - _, single_learnware_list, _ = easy_market.search_learnware(user_info) - -4. Statistical specification search: - - If you choose to porvide your own statistical specification file ``rkme.json``, - the Learnware Market can perform a more accurate leanware selection from - the learnwares returned by the previous step. This second-step searching is carried out - at the level of data distribution information and returns - one or more learnwares that are most likely to be helpful for your task. - - Here, ``unzip_path`` is the directory where you unzip your learnware file. - - .. code-block:: python - - import learnware.specification as specification - - user_spec = specification.rkme.RKMEStatSpecification() - user_spec.load(os.path.join(unzip_path, "rkme.json")) - user_info = BaseUserInfo( - id="user", semantic_spec=user_semantic, stat_info={"RKMEStatSpecification": user_spec} - ) - (sorted_score_list, single_learnware_list, - mixture_score, mixture_learnware_list) = easy_market.search_learnware(user_info) - -5. Reuse learnwares: - - Based on the returned list of learnwares ``mixture_learnware_list`` in the previous step, - you can easily reuse them to make predictions your own data, instead of training a model from scratch. - We provide two baseline methods for reusing a given list of learnwares, namely ``JobSelectorReuser`` and ``AveragingReuser``. - - .. code-block:: python - - reuse_job_selector = JobSelectorReuser(learnware_list=mixture_learnware_list) - job_selector_predict_y = reuse_job_selector.predict(user_data=test_x) - - reuse_ensemble = AveragingReuser(learnware_list=mixture_learnware_list, mode='vote') - ensemble_predict_y = reuse_ensemble.predict(user_data=test_x) - -.. _script: - -Example: Learnware Files -------- - -Below is an example learnware that includes an SVM model and uses Reduced Kernel Mean Embedding as its statistical reduction method. -We have listed the files that it needs to include. \ No newline at end of file diff --git a/docs/start/install.rst b/docs/start/install.rst new file mode 100644 index 0000000..0c61488 --- /dev/null +++ b/docs/start/install.rst @@ -0,0 +1,3 @@ +=================== +Installation Guide +=================== diff --git a/docs/introduction/intro.rst b/docs/start/intro.rst similarity index 100% rename from docs/introduction/intro.rst rename to docs/start/intro.rst diff --git a/docs/start/quick.rst b/docs/start/quick.rst new file mode 100644 index 0000000..e5863c7 --- /dev/null +++ b/docs/start/quick.rst @@ -0,0 +1,166 @@ +.. _quick: +============================================================ +Quick Start +============================================================ + + +Introduction +==================== + +This ``Quick Start`` guide tries to demonstrate + +- It's very easy to build a complete Learnware Market workflow and use ``learnware`` to deal with users' tasks. + + +Installation +==================== + +Learnware is currently hosted on `PyPI `__. You can easily intsall ``learnware`` according to the following steps: + +- For Windows and Linux users: + + .. code-block:: + + pip install learnware + +- For macOS users: + + .. code-block:: + + conda install -c pytorch fais + pip install learnware + + +Prepare Learnware +==================== + +The Learnware Market consists of a wide range of learnwares. A valid learnware is a zip file which +is composed of the following four parts. + +- ``__init__.py`` + + A python file offering interfaces for your model's fitting, predicting and fine-tuning. + +- ``rkme.json`` + + A json file containing the statistical specification of your data. + +- ``learnware.yaml`` + + A config file describing your model class name, type of statistical specification(e.g. Reduced Kernel Mean Embedding, ``RKMEStatSpecification``), and + the file name of your statistical specification file. + +- ``environment.yaml`` + + A Conda environment configuration file for running the model (if the model environment is incompatible, you can rely on this for manual configuration). + You can generate this file according to the following steps: + + - Create env config for conda: + + .. code-block:: + + conda env export | grep -v "^prefix: " > environment.yml + + - Recover env from config: + + .. code-block:: + + conda env create -f environment.yml + + +Learnware Market Workflow +============================ + +Users can start an Learnware Market workflow according to the following steps: + +Initialize a Learware Market +------------------------------- + +.. code-block:: python + + import learnware + from learnware.market import EasyMarket + + learnware.init() + easy_market = EasyMarket(market_id="demo", rebuild=True) + +Upload Leanwares +------------------------------- + +Here, ``zip_path`` is the directory of your learnware zip file. + +.. code-block:: python + + semantic_spec = { + "Data": {"Values": ["Tabular"], "Type": "Class"}, + "Task": {"Values": ["Classification"], "Type": "Class"}, + "Library": {"Values": ["Scikit-learn"], "Type": "Class"}, + "Scenario": {"Values": ["Business"], "Type": "Tag"}, + "Description": {"Values": "", "Type": "String"}, + "Name": {"Values": "learnware_1", "Type": "String"}, + } + semantic_spec["Name"]["Values"] = "learnware_user" + semantic_spec["Description"]["Values"] = "test_learnware_user" + easy_market.add_learnware(zip_path, semantic_spec) + +Semantic Specification Search +------------------------------- + +The Learnware Market will perform first-step searching based on the semantic specification +``semantic_spec`` you provided. +This searching process will indentify potentially helpful leranwares whose models +solve tasks similar to your requirements. + +.. code-block:: python + + user_semantic = { + "Data": {"Values": ["Tabular"], "Type": "Class"}, + "Task": { + "Values": ["Classification"], + "Type": "Class", + }, + "Library": {"Values": ["Scikit-learn"], "Type": "Tag"}, + "Scenario": {"Values": ["Business"], "Type": "Class"}, + "Description": {"Values": "", "Type": "String"}, + "Name": {"Values": "", "Type": "String"}, + } + user_info = BaseUserInfo(id="user", semantic_spec=user_semantic) + _, single_learnware_list, _ = easy_market.search_learnware(user_info) + +Statistical Specification Search +--------------------------------- + +If you choose to porvide your own statistical specification file ``rkme.json``, +the Learnware Market can perform a more accurate leanware selection from +the learnwares returned by the previous step. This second-step searching is carried out +at the level of data distribution information and returns +one or more learnwares that are most likely to be helpful for your task. + +Here, ``unzip_path`` is the directory where you unzip your learnware file. + +.. code-block:: python + + import learnware.specification as specification + + user_spec = specification.rkme.RKMEStatSpecification() + user_spec.load(os.path.join(unzip_path, "rkme.json")) + user_info = BaseUserInfo( + id="user", semantic_spec=user_semantic, stat_info={"RKMEStatSpecification": user_spec} + ) + (sorted_score_list, single_learnware_list, + mixture_score, mixture_learnware_list) = easy_market.search_learnware(user_info) + +Reuse Learnwares +------------------------------- + +Based on the returned list of learnwares ``mixture_learnware_list`` in the previous step, +you can easily reuse them to make predictions your own data, instead of training a model from scratch. +We provide two baseline methods for reusing a given list of learnwares, namely ``JobSelectorReuser`` and ``AveragingReuser``. + +.. code-block:: python + + reuse_job_selector = JobSelectorReuser(learnware_list=mixture_learnware_list) + job_selector_predict_y = reuse_job_selector.predict(user_data=test_x) + + reuse_ensemble = AveragingReuser(learnware_list=mixture_learnware_list, mode='vote') + ensemble_predict_y = reuse_ensemble.predict(user_data=test_x) diff --git a/setup.py b/setup.py index 106e033..b14d169 100644 --- a/setup.py +++ b/setup.py @@ -51,7 +51,7 @@ def get_platform(): # What packages are required for this module to be executed? # `estimator` may depend on other packages. In order to reduce dependencies, it is not written here. REQUIRED = [ - "numpy>=1.20.0", + "numpy>=1.19.2", "pandas>=0.25.1", "scipy>=1.0.0", "matplotlib>=3.1.3",