Browse Source

[ENH] polish submit doc

tags/v0.3.2
2652610018@qq.com 2 years ago
parent
commit
d9a4b7231a
1 changed files with 42 additions and 42 deletions
  1. +42
    -42
      docs/workflow/submit.rst

+ 42
- 42
docs/workflow/submit.rst View File

@@ -2,23 +2,23 @@
Learnware Preparation and Submission
==========================================

In this section, a detailed guide on how to submit your own learnware to the Learnware Market will be provided.
Specifically, we will first elaborate on the components that a valid learnware file should include, then explain
how learnwares are uploaded and removed within ``Learnware Market``.
In this section, we provide a comprehensive guide on submitting your custom learnware to the Learnware Market.
We will first discuss the necessary components of a valid learnware, followed by a detailed explanation on how to upload and remove learnwares within ``Learnware Market``.


Prepare Learnware
====================

A valid learnware is a zipfile which consists of four essential parts. Here we demonstrate the detail format of a learnware zipfile.
A valid learnware is encapsulated in a zipfile, comprising four essential components.
Below, we illustrate the detailed structure of a learnware zipfile.

``__init__.py``
---------------

In ``Learnware Market``, each uploader is required to provide a set of unified interfaces for their model,
which enables convenient usage by future users.
``__init__.py`` is the python file offering interfaces for your model's fitting, predicting and fine-tuning. For example,
the code snippet below trains and saves a SVM model for a sample dataset on sklearn digits classification:
Within ``Learnware Market``, every uploader must provide a unified set of interfaces for their model,
facilitating easy utilization for future users.
The ``__init__.py`` file serves as the Python interface for your model's fitting, prediction, and fine-tuning processes.
For example, the code snippet below is used to train and save a SVM model for a sample dataset on sklearn digits classification:

.. code-block:: python

@@ -36,7 +36,7 @@ the code snippet below trains and saves a SVM model for a sample dataset on skle
joblib.dump(clf, "svm.pkl") # model is stored as file "svm.pkl"


Then the ``__init__.py`` for this SVM model should be as follows:
Then the corresponding ``__init__.py`` for this SVM model should be structured as follows:

.. code-block:: python
@@ -61,38 +61,38 @@ Then the ``__init__.py`` for this SVM model should be as follows:
def finetune(self, X: np.ndarray, y: np.ndarray):
pass
As a kind reminder, don't forget to fill in ``input_shape`` and ``output_shape`` corresponding to the model
(in our sklearn digits classification example, (64,) and (10,) respectively).
Please remember to specify the ``input_shape`` and ``output_shape`` corresponding to your model.
In our sklearn digits classification example, these would be (64,) and (10,) respectively.


``stat.json``
-------------

In order to better match users with learnwares suitable for their tasks,
we need the information of your training dataset. Specifically, you need to provide a statistical specification
stored as a json file, e.g., ``stat.json``, which contains statistical information of the dataset.
This json file is all we required regarding your training data, and there is no need for you to upload your own data.
To accurately and effectively match users with appropriate learnwares for their tasks, we require information about your training dataset.
Specifically, you are required to provide a statistical specification
stored as a json file, such as ``stat.json``, which contains the statistical information of the dataset.
This json file meets all our requirements regarding your training data, so you don't need to upload the actual data.

There are multiple approaches to generate statistical specification.
If Reduced Kernel Mean Embedding (RKME) is chosen to be as statistical specification,
the following code snippet provides guidance on how to build and store the RKME of a dataset:
There are various methods to generate a statistical specification.
If you choose to use Reduced Kernel Mean Embedding (RKME) as your statistical specification,
the following code snippet offers guidance on how to construct and store the RKME of a dataset:

.. code-block:: python
import learnware.specification as specification
# generate rkme specification for digits dataset
spec = specification.utils.generate_rkme_spec(X=data_X, gamma=0.1, cuda_idx=0)
spec.save(os.path.join("stat.json"))
spec = specification.utils.generate_rkme_spec(X=data_X)
spec.save("stat.json")


``learnware.yaml``
------------------

We also require you to prepare a configuration file in YAML format,
which describes your model class name, type of statistical specification(e.g. Reduced Kernel Mean Embedding, ``RKMEStatSpecification``), and
the file name of your statistical specification file. The following ``learnware.yaml`` demonstrates
how your learnware configuration file should be structured based on our previous example:
Additionally, you are asked to prepare a configuration file in YAML format.
The file should detail your model's class name, the type of statistical specification(e.g. Reduced Kernel Mean Embedding, ``RKMEStatSpecification``), and
the file name of your statistical specification file. The following ``learnware.yaml`` provides an example of
how your learnware configuration file should be structured, based on our previous discussion:

.. code-block:: yaml

@@ -109,14 +109,14 @@ how your learnware configuration file should be structured based on our previous
``environment.yaml`` or ``requirements.txt``
--------------------------------------------

In order to allow others to run your learnware, you need to specify the dependencies of your model. You
can either provide an ``environment.yaml`` file or an ``requirements.txt`` file.
In order to allow others to execute your learnware, it's necessary to specify your model's dependencies.
You can do this by providing either an ``environment.yaml`` file or a ``requirements.txt`` file.


- ``environment.yaml`` for conda:

if you provide ``environment.yaml``, a new conda environment will be created according to this file
when users install your learnware. The yaml file can be generated by the following command:
If you provide an ``environment.yaml``, a new conda environment will be created based on this file
when users install your learnware. You can generate this yaml file using the following command:

.. code-block::

@@ -125,26 +125,26 @@ can either provide an ``environment.yaml`` file or an ``requirements.txt`` file.

- ``requirements.txt`` for pip:

if you provide ``requirements.txt``, dependent packages will be installed through -r option of pip.
More description of ``requirements.txt`` can be found in
If you provide a ``requirements.txt``, the dependent packages will be installed using the `-r` option of pip.
You can find more information about ``requirements.txt`` in
`pip documentation <https://pip.pypa.io/en/stable/user_guide/#requirements-files>`_.
It is recommended to use ``environment.yaml`` which can reduce conflics between different packages.
We recommend using ``environment.yaml`` as it can help minimize conflicts between different packages.

.. note::
Wether you use ``environment.yaml`` or ``requirements.txt``, you should make the dependencies as minimal as possible.
You may need to open the file and delete unnecessary packages manually.
Whether you choose to use ``environment.yaml`` or ``requirements.txt``,
it's important to keep your dependencies as minimal as possible.
This may involve manually opening the file and removing any unnecessary packages.


Upload Learnware
==================

Once you have prepared the four required files mentioned above,
you can package them as your own learnware zipfile. Combined with the generated semantic specification that
briefly describes the features of your task and model (
Please refer to :ref:`semantic specification<components/spec:Semantic Specification>` for more details),
you can easily upload your learnware to the ``Learnware Market`` with a single line of code:
After preparing the four required files mentioned above,
you can bundle them into your own learnware zipfile. Along with the generated semantic specification that
succinctly describes the features of your task and model (for more details, please refer to :ref:`semantic specification<components/spec:Semantic Specification>`),
you can effortlessly upload your learnware to the ``Learnware Market`` using a single line of code:

.. code-block:: python

@@ -159,18 +159,18 @@ you can easily upload your learnware to the ``Learnware Market`` with a single l
# single line uploading
easy_market.add_learnware(zip_path, semantic_spec)

Here, ``zip_path`` is the directory of your learnware zipfile.
Here, ``zip_path`` refers to the directory of your learnware zipfile.


Remove Learnware
==================

As ``Learnware Market`` administrators, it is necessary to remove learnwares with suspicious uploading motives.
With required permissions and approvals, you can use the following code to remove a learnware
As administrators of the ``Learnware Market``, it's crucial to remove learnwares that exhibit suspicious uploading motives.
Once you have the necessary permissions and approvals, you can use the following code to remove a learnware
from the ``Learnware Market``:

.. code-block:: python

easy_market.delete_learnware(learnware_id)

Here, ``learnware_id`` is the market ID of the learnware to be removed.
Here, ``learnware_id`` refers to the market ID of the learnware to be removed.

Loading…
Cancel
Save