From 66c34aeadab4de17a170257ccfe717cbb1c9837c Mon Sep 17 00:00:00 2001 From: Gao Enhao Date: Thu, 21 Dec 2023 20:24:52 +0800 Subject: [PATCH] [DOC] remove BaseDataElement --- abl/data/structures/list_data.py | 44 +++++++++++++++++++++++++++----- docs/Intro/Basics.rst | 3 +-- docs/Intro/Datasets.rst | 6 ++--- 3 files changed, 41 insertions(+), 12 deletions(-) diff --git a/abl/data/structures/list_data.py b/abl/data/structures/list_data.py index 25bb0fc..1b04aa8 100644 --- a/abl/data/structures/list_data.py +++ b/abl/data/structures/list_data.py @@ -18,13 +18,43 @@ IndexType = Union[str, slice, int, list, LongTypeTensor, BoolTypeTensor, np.ndar # https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/data_structures/instance_data.py # noqa class ListData(BaseDataElement): """ - Data structure for example-level data. - - Subclass of :class:`BaseDataElement`. All value in `data_fields` - should have the same length. This design refer to - https://github.com/facebookresearch/detectron2/blob/master/detectron2/structures/instances.py - - ListData supports `index` and `slice` for data field. The type of value in data field can be either `None` or `list` of base data structures such as `torch.Tensor`, `numpy.ndarray`, `list`, `str` and `tuple`. + Abstract Data Interface used throughout the ABL-Package. + + `ListData` is the underlying data structure used in the ABL-Package, + designed to manage diverse forms of data dynamically generated throughout the + Abductive Learning (ABL) framework. This includes handling raw data, predicted + pseudo-labels, abduced pseudo-labels, pseudo-label indices, etc. + + As a fundamental data structure in ABL, `ListData` is essential for the smooth + transfer and manipulation of data across various components of the ABL framework, + such as prediction, abductive reasoning, and training phases. It provides a + unified data format across these stages, ensuring compatibility and flexibility + in handling diverse data forms in the ABL framework. + + The attributes in ``ListData`` are divided into two parts, + the ``metainfo`` and the ``data`` respectively. + + - ``metainfo``: Usually used to store basic information about data examples, + such as symbol number, image size, etc. The attributes can be accessed or + modified by dict-like or object-like operations, such as ``.`` (for data + access and modification), ``in``, ``del``, ``pop(str)``, ``get(str)``, + ``metainfo_keys()``, ``metainfo_values()``, ``metainfo_items()``, + ``set_metainfo()`` (for set or change key-value pairs in metainfo). + + - ``data``: raw data, labels, predictions, and abduced results are stored. + The attributes can be accessed or modified by dict-like or object-like operations, + such as ``.``, ``in``, ``del``, ``pop(str)``, ``get(str)``, ``keys()``, + ``values()``, ``items()``. Users can also apply tensor-like + methods to all :obj:`torch.Tensor` in the ``data_fields``, such as ``.cuda()``, + ``.cpu()``, ``.numpy()``, ``.to()``, ``to_tensor()``, ``.detach()``. + + ListData supports `index` and `slice` for data field. The type of value in + data field can be either `None` or `list` of base data structures such as + `torch.Tensor`, `numpy.ndarray`, `list`, `str` and `tuple`. + + This design is inspired by and extends the functionalities of the `BaseDataElement` + class implemented in MMEngine. + https://github.com/open-mmlab/mmengine/blob/main/mmengine/structures/base_data_element.py # noqa E501 Examples: >>> from abl.data.structures import ListData diff --git a/docs/Intro/Basics.rst b/docs/Intro/Basics.rst index baebbd9..4066405 100644 --- a/docs/Intro/Basics.rst +++ b/docs/Intro/Basics.rst @@ -22,8 +22,7 @@ AI: data, models, and knowledge. .. image:: ../img/ABL-Package.png **Data** part manages the storage, operation, and evaluation of data. -It first features class ``ListData`` (derived from base class -``BaseDataElement``), which defines the data structures used in +It first features class ``ListData``, which defines the data structures used in Abductive Learning, and comprises common data operations like insertion, deletion, retrieval, slicing, etc. Additionally, a series of Evaluation Metrics, including class ``SymbolAccuracy`` and ``ReasoningMetric`` (both diff --git a/docs/Intro/Datasets.rst b/docs/Intro/Datasets.rst index 9c11403..0bfebb6 100644 --- a/docs/Intro/Datasets.rst +++ b/docs/Intro/Datasets.rst @@ -53,11 +53,11 @@ As an illustration, in the MNIST Addition example, the data used for training ar Data Structure -------------- -Besides the user-provided dataset, various forms of data are utilized and dynamicly generate throughout the training and testing process of Abductive Learning framework. Examples include raw data, predicted pseudo-label, abduced pseudo-label, pseudo-label indices, and so on. To manage this diversity and ensure a stable, versatile interface, ABL-Package employs `abstract data interfaces <../API/abl.data.html>`_ to encapsulate different forms of data that will be used in the total learning process. +Besides the user-provided dataset, various forms of data are utilized and dynamicly generated throughout the training and testing process of Abductive Learning framework. Examples include raw data, predicted pseudo-label, abduced pseudo-label, pseudo-label indices, and so on. To manage this diversity and ensure a stable, versatile interface, ABL-Package employs `abstract data interfaces <../API/abl.data.html#data-structure>`_ to encapsulate different forms of data that will be used in the total learning process. -``BaseDataElement`` is the base class for all abstract data interfaces. Inherited from ``BaseDataElement``, ``ListData`` is the most commonly used abstract data interface in ABL-Package. As the fundamental data structure, ``ListData`` implements commonly used data manipulation methods and is responsible for transferring data between various components of ABL, ensuring that stages such as prediction, training, and abductive reasoning can utilize ``ListData`` as a unified input format. +``ListData`` is the underlying abstract data interface utilized in ABL-Package. As the fundamental data structure, ``ListData`` implements commonly used data manipulation methods and is responsible for transferring data between various components of ABL, ensuring that stages such as prediction, abductive reasoning, and training can utilize ``ListData`` as a unified input format. -Before proceeding to other stages, user-provided datasets are firstly converted into ``ListData``. For flexibility, ABL-Package also allows user to directly supply data in ``ListData`` format, which similarly requires the inclusion of three attributes: ``X``, ``gt_pseudo_label``, and ``Y``. The following code shows the basic usage of ``ListData``. More information can be found in the `API documentation <../API/abl.data.html>`_. +Before proceeding to other stages, user-provided datasets are firstly converted into ``ListData``. For flexibility, ABL-Package also allows user to directly supply data in ``ListData`` format, which similarly requires the inclusion of three attributes: ``X``, ``gt_pseudo_label``, and ``Y``. The following code shows the basic usage of ``ListData``. More information can be found in the `API documentation <../API/abl.data.html#data-structure>`_. .. code-block:: python