From eade801d88503384a9ab1faccc3791c2ddd296fc Mon Sep 17 00:00:00 2001 From: Frozenmad Date: Sun, 27 Jun 2021 10:36:13 +0000 Subject: [PATCH] update documents for new version --- README.md | 24 ++++++-- docs/docfile/tutorial/t_nas.rst | 77 +++++++++++++++++++++++++- docs/docfile/tutorial/t_quickstart.rst | 2 +- docs/docfile/tutorial/t_solver.rst | 24 +++----- docs/index.rst | 14 +++-- 5 files changed, 110 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index fd04576..205fb43 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ The workflow below shows the overall framework of AutoGL. AutoGL uses `datasets` to maintain dataset for graph-based machine learning, which is based on Dataset in PyTorch Geometric with some support added to corporate with the auto solver framework. -Different graph-based machine learning tasks are solved by different `AutoGL solvers`, which make use of four main modules to automatically solve given tasks, namely `auto feature engineer`, `auto model`, `hyperparameter optimization`, and `auto ensemble`. +Different graph-based machine learning tasks are solved by different `AutoGL solvers`, which make use of five main modules to automatically solve given tasks, namely `auto feature engineer`, `neural architecture search`, `auto model`, `hyperparameter optimization`, and `auto ensemble`. Currently, the following algorithms are supported in AutoGL: @@ -33,6 +33,7 @@ Currently, the following algorithms are supported in AutoGL: Feature Engineer Model + NAS HPO Ensemble @@ -40,13 +41,26 @@ Currently, the following algorithms are supported in AutoGL: Generators
graphlet
eigen
more ...

Selectors
SeFilterConstant
gbdt

Subgraph
netlsd
NxAverageClustering
more ... Node Classification
GCN
GAT
GraphSAGE

Graph Classification
GIN
TopKPool + + Algorithms
+ Random
+ RL
+ more ...

+ Spaces
+ SinglePath
+ GraphNas
+ more ...

+ Estimators
+ Oneshot
+ Scratch
+ Grid
Random
Anneal
Bayes
CAMES
MOCAMES
Quasi random
TPE
AutoNE Voting
Stacking -This toolkit also serves as a platform for users to implement and test their own autoML or graph-based machine learning models. +This toolkit also serves as a framework for users to implement and test their own autoML or graph-based machine learning models. ## Installation @@ -56,11 +70,11 @@ Please make sure you meet the following requirements before installing AutoGL. 1. Python >= 3.6.0 -2. PyTorch (>=1.5.1) +2. PyTorch (>=1.6.0) see for installation. -3. PyTorch Geometric +3. PyTorch Geometric (>=1.7.0) see for installation. @@ -71,7 +85,7 @@ Please make sure you meet the following requirements before installing AutoGL. Run the following command to install this package through `pip`. ``` -pip install auto-graph-learning +pip install autogl ``` #### Install from source diff --git a/docs/docfile/tutorial/t_nas.rst b/docs/docfile/tutorial/t_nas.rst index 7d60674..10ed15d 100644 --- a/docs/docfile/tutorial/t_nas.rst +++ b/docs/docfile/tutorial/t_nas.rst @@ -8,11 +8,59 @@ To be more flexible, we modulize NAS process with three part: algorithm, space a Different models in different parts can be composed in some certain constrains. If you want to design your own NAS process, you can change any of those parts according to your demand. +Usage +----- + +You can directly enable architecture search for node classification tasks by passing the algorithms, spaces and estimators to +solver. Following shows an example: + +.. code-block:: python + + # Use graphnas to solve cora + from autogl.datasets import build_dataset_from_name + from autogl.solver import AutoNodeClassifier + + solver = AutoNodeClassifier( + feature = 'PYGNormalizeFeatures', + graph_models = (), + hpo = 'tpe', + ensemble = None, + nas_algorithms='rl', + nas_spaces='graphnasmacro', + nas_estimators='scratch' + ) + + cora = build_dataset_from_name('cora') + solver.fit(cora) + +The code above will first find the best architecture in space ``graphnasmacro`` using ``rl`` search algorithm. +Then the searched architecture will be further optimized through hyperparameter-optimization ``tpe``. + +.. note:: The ``graph_models`` argument is not conflict with nas module. You can set ``graph_models`` to + other hand-crafted models beside the ones found by nas. Once the architectures are derived from nas module, + they act in the same way as hand-crafted models directly passed through graph_models. + Search Space ------------ The space definition is base on mutable fashion used in NNI, which is defined as a model inheriting BaseSpace There are mainly two ways to define your search space, one can be performed with one-shot fashion while the other cannot. +Currently, we support following search space: + ++------------------------+-----------------------------------------------------------------+ +| Space | Description | ++========================+=================================================================+ +| ``singlepath`` [4]_ | Architectures with several sequential layers with each layer | +| | choosing only one path | ++------------------------+-----------------------------------------------------------------+ +| ``graphnas`` [1]_ | The graph nas micro search space designed for fully supervised | +| | node classification models | ++------------------------+-----------------------------------------------------------------+ +| ``graphnasmacro`` [1]_ | The graph nas macro search space designed for semi-superwised | +| | node classification models | ++------------------------+-----------------------------------------------------------------+ + +You can also define your own nas search space. If you need one-shot fashion, you should use the function ``setLayerChoice`` and ``setInputChoice`` to construct the super network. Here is an example. @@ -111,8 +159,17 @@ But you can only use sample-based search strategy. Performance Estimator --------------------- -The performance estimator estimates the performance of an architecture. -Here is an example of estimating an architecture without training (used in one-shot space). +The performance estimator estimates the performance of an architecture. Currently we support following estimators: + ++-------------------------+-------------------------------------------------------+ +| Estimator | Description | ++=========================+=======================================================+ +| ``oneshot`` | Directly evaluating the given models without training | ++-------------------------+-------------------------------------------------------+ +| ``scratch`` | Train the models from scratch and then evaluate them | ++-------------------------+-------------------------------------------------------+ + +You can also write your own estimator. Here is an example of estimating an architecture without training (used in one-shot space). .. code-block:: python @@ -135,7 +192,20 @@ Here is an example of estimating an architecture without training (used in one-s Search Strategy --------------- -The space strategy defines how to find an architecture. +The space strategy defines how to find an architecture. We currently support following search strategies: + ++-------------------------+-------------------------------------------------------+ +| Strategy | Description | ++=========================+=======================================================+ +| ``random`` | Random search by uniform sampling | ++-------------------------+-------------------------------------------------------+ +| ``rl`` [1]_ | Use rl as architecture generator agent | ++-------------------------+-------------------------------------------------------+ +| ``enas`` [2]_ | efficient neural architecture search | ++-------------------------+-------------------------------------------------------+ +| ``darts`` [3]_ | differentiable neural architecture search | ++-------------------------+-------------------------------------------------------+ + Sample-based strategy without weight sharing is simpler than strategies with weight sharing. We show how to define your strategy here with DFS as an example. @@ -227,3 +297,4 @@ Different search strategies should be combined with different search spaces and .. [1] Gao, Yang, et al. "Graph neural architecture search." IJCAI. Vol. 20. 2020. .. [2] Pham, Hieu, et al. "Efficient neural architecture search via parameters sharing." International Conference on Machine Learning. PMLR, 2018. .. [3] Liu, Hanxiao, Karen Simonyan, and Yiming Yang. "DARTS: Differentiable Architecture Search." International Conference on Learning Representations. 2018. +.. [4] Guo, Zichao, et al. “Single Path One-Shot Neural Architecture Search with Uniform Sampling.” European Conference on Computer Vision, 2019, pp. 544–560. diff --git a/docs/docfile/tutorial/t_quickstart.rst b/docs/docfile/tutorial/t_quickstart.rst index cd27ad0..78a6765 100644 --- a/docs/docfile/tutorial/t_quickstart.rst +++ b/docs/docfile/tutorial/t_quickstart.rst @@ -12,7 +12,7 @@ Based on the concept of autoML, auto graph learning aims at **automatically** so The diagram below describes the workflow of AutoGL framework. -To reach the aim of autoML, our proposed auto graph learning framework is organized as follows. We have ``dataset`` to maintain the graph datasets given by users. A ``solver`` object needs to be built for specifying the target tasks. Inside ``solver``, there are four submodules to help complete the auto graph tasks, namely ``auto feature engineer``, ``auto model``, ``hyperparameter optimization`` and ``auto ensemble``, which will automatically preprocess/enhance your data, choose and optimize deep models and ensemble them in the best way for you. +To reach the aim of autoML, our proposed auto graph learning framework is organized as follows. We have ``dataset`` to maintain the graph datasets given by users. A ``solver`` object needs to be built for specifying the target tasks. Inside ``solver``, there are five submodules to help complete the auto graph tasks, namely ``auto feature engineer``, ``auto model``, ``neural architecture search``, ``hyperparameter optimization`` and ``auto ensemble``, which will automatically preprocess/enhance your data, choose and optimize deep models and ensemble them in the best way for you. Let's say you want to conduct an auto graph learning on dataset ``Cora``. First, you can easily get the ``Cora`` dataset using the ``dataset`` module: diff --git a/docs/docfile/tutorial/t_solver.rst b/docs/docfile/tutorial/t_solver.rst index 59c6ccf..028e36c 100644 --- a/docs/docfile/tutorial/t_solver.rst +++ b/docs/docfile/tutorial/t_solver.rst @@ -3,10 +3,11 @@ AutoGL Solver ============= -Our AutoGL project use ``solver`` to handle the auto-solvation of tasks. Currently, we support the following tasks: +AutoGL project use ``solver`` to handle the auto-solvation of tasks. Currently, we support the following tasks: * ``AutoNodeClassifier`` for semi-supervised node classification * ``AutoGraphClassifier`` for supervised graph classification +* ``AutoLinkPredictor`` for link prediction Initialization -------------- @@ -16,7 +17,7 @@ A solver can either be initialized from its ``__init__()`` or from a config dict Initialize from ``__init__()`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If you want to build a solver by ``__init__()``, you need to pass the four key modules to it, namely as ``auto feature engineer``, ``auto model list``, ``hyperparameter optimizer`` and ``auto ensemble``. You can either pass the keywords of corresponding modules or the initialized instances: +If you want to build a solver by ``__init__()``, you need to pass the key modules to it. You can either pass the keywords of corresponding modules or the initialized instances: .. code-block:: python @@ -43,13 +44,13 @@ If you want to build a solver by ``__init__()``, you need to pass the four key m Where, the argument ``device`` means where to perform the training and searching, by setting to ``auto``, the ``cuda`` is used when it is available. -If you want to disable one module (except graphModuleList), you can set it to ``None``: +If you want to disable one module, you can set it to ``None``: .. code-block:: python solver = AutoNodeClassifier(feature_module=None, hpo_module=None, ensemble_module=None) -You can also pass some important arguments of modules directly to solver, which will automatically set them for you: +You can also pass some important arguments of modules directly to solver, which will automatically be set for you: .. code-block:: python @@ -89,9 +90,9 @@ You can use ``fit()`` or ``fit_predict()`` to perform optimization, which shares # load your dataset here dataset = some_dataset() - solver.fit(dataset, inplace=True, time_limit=3600) + solver.fit(dataset, inplace=True) -The inplace argument is used for saving memory if set to ``True``. It will modify your dataset in an inplace manner during feature engineering. You can also set ``time_limit`` to limit the time cost of the whole auto process. +The inplace argument is used for saving memory if set to ``True``. It will modify your dataset in an inplace manner during feature engineering. You can also specify the ``train_split`` and ``val_split`` arguments to let solver auto-split the given dataset. If these arguments are given, the split dataset will be used instead of the default split specified by the dataset provided. All the models will be trained on ``train dataset``. Their hyperparameters will be optimized based on the performance of ``valid dataset``, as well as the final ensemble method. For example: @@ -107,16 +108,7 @@ You can also specify the ``train_split`` and ``val_split`` arguments to let solv For the node classification problem, we also support balanced sampling of train and valid: force the number of sampled nodes in different classes to be the same. The balanced mode can be turned on by setting ``balanced=True`` in ``fit()``, which is by default set to ``True``. -For the graph classification problem, we also provide a way to conduct cross-validation. You can enable cross-validation by specifying ``cross_validation=True``. ``cv_fold`` is also provided to determine the number of folds. Then, the ``train dataset`` will be further split into ``cv_fold`` folds for each model to be trained and optimized hyperparameters on. The auto ensemble will base on the model performance of ``valid dataset``. - -.. note:: - - If you want to use cross validation, please make sure the ``dataset`` receives train/val/test split before cross validated. By default, the graph dataset derived directly from ``build_dataset_from_name`` is not splitted yet. To split the dataset, you can: - - * use ``autogl.datasets.utils.graph_random_split`` to pre-split ``dataset`` outside of ``solver``. - * pass ``train_split`` and ``val_split`` directly to ``solver``, which will pre-split the ``dataset`` for you. - -.. note:: Solver will maintain the models with the best hyper-parameter of each model architecture you pass to solver (the ``graphModelList`` argument when initialized). The maintained models will then be ensembled by ensemble module. When cross-validation is used, solver will maintain ``cv_fold`` models of each model architecture for each fold. +.. note:: Solver will maintain the models with the best hyper-parameter of each model architecture you pass to solver (the ``graph_models`` argument when initialized). The maintained models will then be ensembled by ensemble module. After ``fit()``, solver maintains the performances of every single model and the ensemble model in one leaderboard instance. You can output the performances on valid dataset by: diff --git a/docs/index.rst b/docs/index.rst index e2c0a60..e203f08 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -6,7 +6,7 @@ AutoGL *Actively under development by @THUMNLab* -AutoGL is developed for researchers and developers to quickly conduct autoML on the graph datasets & tasks. See our documentation for detailed information! +AutoGL is developed for researchers and developers to quickly conduct autoML on the graph datasets & tasks. The workflow below shows the overall framework of AutoGL. @@ -15,7 +15,7 @@ The workflow below shows the overall framework of AutoGL. AutoGL uses ``AutoGL Dataset`` to maintain datasets for graph-based machine learning, which is based on the dataset in PyTorch Geometric with some support added to corporate with the auto solver framework. -Different graph-based machine learning tasks are solved by different ``AutoGL Solvers`` , which make use of four main modules to automatically solve given tasks, namely ``Auto Feature Engineer``, ``Auto Model``, ``HyperParameter Optimization``, and ``Auto Ensemble``. +Different graph-based machine learning tasks are solved by different ``AutoGL Solvers`` , which make use of four main modules to automatically solve given tasks, namely ``Auto Feature Engineer``, ``Auto Model``, ``Neural Architecture Search``, ``HyperParameter Optimization``, and ``Auto Ensemble``. Installation ------------ @@ -27,11 +27,11 @@ Please make sure you meet the following requirements before installing AutoGL. 1. Python >= 3.6.0 -2. PyTorch (>=1.5.1) +2. PyTorch (>=1.6.0) see `PyTorch `_ for installation. -3. PyTorch Geometric +3. PyTorch Geometric (>=1.7.0) see `PyTorch Geometric `_ for installation. @@ -45,7 +45,7 @@ Run the following command to install this package through pip. .. code-block:: shell - pip install auto-graph-learning + pip install autogl Install from source ^^^^^^^^^^^^^^^^^^^ @@ -71,10 +71,12 @@ If you are a developer of the AutoGL project, please use the following command t Modules ------- -In AutoGL, the tasks are solved by corresponding learners, which in general do the following things: +In AutoGL, the tasks are solved by corresponding solvers, which in general do the following things: 1. Preprocess and feature engineer the given datasets. This is done by the module named **auto feature engineer**, which can automatically add/delete useful/useless attributes in the given datasets. Some topological features may also be extracted & combined to form stronger features for current tasks. +2. Find the best suitable model architectures through neural architecture search. This is done by modules named **nas**. AutoGL provides several search spaces, algorithms and estimators for finding the best architectures. + 2. Automatically train and tune popular models specified by users. This is done by modules named **auto model** and **hyperparameter optimization**. In the auto model, several commonly used graph deep models are provided, together with their hyperparameter spaces. These kinds of models can be tuned using **hyperparameter optimization** module to find the best hyperparameter for the current task. 3. Find the best way to ensemble models found and trained in the last step. This is done by the module named **auto ensemble**. The suitable models available are ensembled here to form a more powerful learner.