Merge branch 'Dev' of https://github.com/AbductiveLearning/ABL-Package into Dev

docs/Intro/Basics.rst

@@ -28,13 +28,13 @@ such as ``SymbolAccuracy`` and ``ReasoningMetric`` (both specialized metrics
 inherited from the ``BaseMetric`` class), for evaluating model quality from a 
 data perspective.
 
-**Learning** part focuses on the construction, deployment, and
+:blue-bold:`Learning` part focuses on the construction, deployment, and
 training of machine learning models. The ``ABLModel`` class is the 
 central class that encapsulates the machine learning model. This class is
 compatible with various frameworks, including those based on Scikit-learn
 or PyTorch neural networks constructed by the ``BasicNN`` class.
 
-**Reasoning** part concentrates on constructing domain knowledge and 
+:green-bold:`Reasoning` part concentrates on constructing domain knowledge and 
 performing reasoning. The ``KBBase`` class allows users to define a 
 domain knowledge base. For diverse types of knowledge, we also offer
 implementations like ``GroundKB`` and ``PrologKB`` (both inherited 
@@ -45,7 +45,7 @@ responsible for minimizing the inconsistency between the knowledge base
 and data.
 
 The integration of these three parts are achieved through the
-**Bridge** part, which features the ``SimpleBridge`` class (derived 
+:yellow-bold:`Bridge` part, which features the ``SimpleBridge`` class (derived 
 from the ``BaseBridge`` class). The Bridge part synthesizes data, 
 learning, and reasoning, facilitating the training and testing 
 of the entire ABL framework.
@@ -54,33 +54,33 @@ Use ABL-Package Step by Step
 ----------------------------
 
 In a typical ABL process, as illustrated below, 
-data inputs are first predicted by a machine learning model, and the outcomes are a pseudo-label 
-example (which consists of multiple pseudo-labels). 
-These labels then pass through a knowledge base :math:`\mathcal{KB}`
-to obtain the reasoning result by deductive reasoning. During training, 
+data inputs are first predicted by the learning model ``ABLModel.predict``, and the outcomes are pseudo-labels.
+These labels then pass through deductive reasoning of the domain knowledge base ``KBBase.logic_forward``
+to obtain the reasoning result. During training, 
 alongside the aforementioned forward flow (i.e., prediction --> deduction reasoning), 
 there also exists a reverse flow, which starts from the reasoning result and 
-involves abductive reasoning to generate possible pseudo-label examples. 
-Subsequently, these examples are processed to minimize inconsistencies with machine learning, 
-which in turn revise the outcomes of the machine learning model, and then 
-fed back into the machine learning model for further training. 
-To implement this process, the following five steps are necessary:
+involves abductive reasoning ``KBBase.abduce_candidates`` to generate possible revised pseudo-labels. 
+Subsequently, these pseudo-labels are processed to minimize inconsistencies with the learning part, 
+which in turn revise the outcomes of the learning model, and then 
+fed back for further training ``ABLModel.train``.  
 
 .. image:: ../img/usage.png
 
-1. Prepare datasets
+To implement this process, the following five steps are necessary:
+
+1. Prepare **datasets**
 
     Prepare the data's input, ground truth for pseudo-labels (optional), and ground truth for reasoning results.
 
-2. Build the learning part
+2.  :blue:`Build the` :blue-bold:`learning` :blue:`part`
 
     Build a machine learning base model that can predict inputs to pseudo-labels. 
     Then, use ``ABLModel`` to encapsulate the base model.
 
-3. Build the reasoning part
+3. :green:`Build the` :green-bold:`reasoning` :green:`part`
 
     Define a knowledge base by building a subclass of ``KBBase``, specifying how to 
-    map pseudo-label examples to reasoning results.
+    process pseudo-label examples to reasoning results.
     Also, create a ``Reasoner`` for minimizing inconsistencies 
     between the knowledge base and data.
 
@@ -89,7 +89,7 @@ To implement this process, the following five steps are necessary:
     Define the metrics by building a subclass of ``BaseMetric``. The metrics will 
     specify how to measure performance during the training and testing of the ABL framework.
 
-5. Bridge learning and reasoning
+5. :yellow-bold:`Bridge` :yellow:`learning and reasoning`
 
     Use ``SimpleBridge`` to bridge the learning and reasoning part
     for integrated training and testing. 

docs/Intro/Bridge.rst

@@ -14,7 +14,6 @@ In this section, we will look at how to bridge learning and reasoning parts to t
 
 .. code:: python
 
-    # Import necessary modules
     from abl.bridge import BaseBridge, SimpleBridge
 
 ``BaseBridge`` is an abstract class with the following initialization parameters:

docs/Intro/Datasets.rst

@@ -10,34 +10,35 @@
 Dataset & Data Structure
 ========================
 
-In this section, we will look at the datasets and data structures in ABL-Package.
+In this section, we will look at the dataset and data structure in ABL-Package.
 
 .. code:: python
 
-    # Import necessary libraries and modules
     import torch
     from abl.data.structures import ListData
 
 Dataset
 -------
 
-ABL-Package assumes user data to be either structured as a tuple or a ``ListData`` which is the underlying data structure utilized in the whole package and will be introduced in the next section. Regardless of the chosen format, the data should encompass the following three essential components:
+ Training data should be in the form of  or a ListData .
+
+ABL-Package requires user data to be either structured as a tuple ``(X, gt_pseudo_label, Y)`` or a ``ListData`` (the underlying data structure utilized in ABL-Package and will be introduced in the next section) object with ``X``, ``gt_pseudo_label`` and ``Y`` attributes . Regardless of the chosen format, the data should encompass three essential components:
 
 - ``X``: List[List[Any]]
     
-    A list of sublists representing the input data. We refer to each sublist in ``X`` as an example and each example may contain several instances.
+    A list of sublists representing the input data. We refer to each sublist in ``X`` as an **example** and each example may contain several **instances**.
 
 - ``gt_pseudo_label``: List[List[Any]], optional
     
-    A list of sublists with each sublist representing a ground truth pseudo-label example. Each example consists of ground truth pseudo-labels for each **instance** within a example of ``X``. 
+    A list of sublists with each sublist representing a ground-truth pseudo-label example. Each example consists of ground-truth pseudo-labels for each **instance** within a example of ``X``. 
     
     .. note::
 
-        ``gt_pseudo_label`` is only used to evaluate the performance of the learning part but not to train the model. If the pseudo-label of the instances in the datasets are unlabeled, ``gt_pseudo_label`` can be ``None``.
+        ``gt_pseudo_label`` is only used to evaluate the performance of the learning part but not to train the model. If the pseudo-label of the instances in the datasets are unlabeled, ``gt_pseudo_label`` should be ``None``.
 
 - ``Y``: List[Any]
     
-    A list representing the ground truth reasoning result for each **example** in ``X``.
+    A list representing the ground-truth reasoning result for each **example** in ``X``.
 
 
 .. warning::
@@ -58,12 +59,12 @@ As an illustration, in the MNIST Addition task, the data are organized as follow
    :alt: alternate text
    :scale: 55%
 
-where each sublist in ``X``, e.g., |data_example|, is a data example and each image in the sublist, e.g., |instance|, is a data instance.
+where each sublist in ``X``, e.g., |data_example|, is a data example and each image in the sublist, e.g., |instance|, is an instance.
 
 Data Structure
 --------------
 
-Besides the user-provided data, various forms of data are utilized and dynamicly generated throughout the training and testing process of Abductive Learning framework. Examples include raw data, predicted pseudo-label, abduced pseudo-label, pseudo-label indices, etc. To manage this diversity and ensure a stable, versatile interface, ABL-Package employs `abstract data interfaces <../API/abl.data.html#structure>`_ to encapsulate different forms of data that will be used in the total learning process.
+Besides the user-provided dataset, various forms of data are utilized and dynamicly generated throughout the training and testing process of ABL framework. Examples include raw data, predicted pseudo-label, abduced pseudo-label, pseudo-label indices, etc. To manage this diversity and ensure a stable, versatile interface, ABL-Package employs `abstract data interfaces <../API/abl.data.html#structure>`_ to encapsulate different forms of data that will be used in the total learning process.
 
 ``ListData`` is the underlying abstract data interface utilized in ABL-Package. As the fundamental data structure, ``ListData`` implements commonly used data manipulation methods and is responsible for transferring data between various components of ABL, ensuring that stages such as prediction, abductive reasoning, and training can utilize ``ListData`` as a unified input format. Before proceeding to other stages, user-provided datasets will be firstly converted into ``ListData``.
 
@@ -71,20 +72,20 @@ Instead of providing a tuple consisting of ``X``, ``gt_pseudo_label``, and ``Y``
 
 .. code-block:: python
 
-    # prepare data
+    # Prepare data
     X = [list(torch.randn(3, 28, 28)), list(torch.randn(3, 28, 28))]
     gt_pseudo_label = [[1, 2, 3], [4, 5, 6]]
     Y = [1, 2]
 
-    # convert data into ListData
+    # Convert data into ListData
     data = ListData(X=X, Y=Y, gt_pseudo_label=gt_pseudo_label)
 
-    # get data
+    # Get data
     X = data.X
     Y = data.Y
     gt_pseudo_label = data.gt_pseudo_label
 
-    # set data
+    # Set data
     data.X = X
     data.Y = Y
     data.gt_pseudo_label = gt_pseudo_label

docs/Intro/Evaluation.rst

@@ -14,7 +14,6 @@ In this section, we will look at how to build evaluation metrics.
 
 .. code:: python
 
-    # Import necessary modules
     from abl.data.evaluation import BaseMetric, SymbolAccuracy, ReasoningMetric
 
 ABL-Package seperates the evaluation process from model training and testing as an independent class, ``BaseMetric``. The training and testing processes are implemented in the ``BaseBridge`` class, so metrics are used by this class and its sub-classes. After building a ``bridge`` with a list of ``BaseMetric`` instances, these metrics will be used by the ``bridge.valid`` method to evaluate the model performance during training and testing. 

docs/Intro/Learning.rst

@@ -19,7 +19,6 @@ In ABL-Package, building the learning part involves two steps:
 
 .. code:: python
 
-    # Import necessary libraries and modules
     import sklearn
     import torchvision
     from abl.learning import BasicNN, ABLModel

docs/Intro/Quick-Start.rst

@@ -14,8 +14,8 @@ We use the MNIST Addition task as a quick start example. In this task, pairs of
 Working with Data
 -----------------
 
-ABL-Package assumes data to be in the form of ``(X, gt_pseudo_label, Y)``  where ``X`` is the input of the machine learning model, 
-``gt_pseudo_label`` is the ground truth label of each element in ``X`` and ``Y`` is the ground truth reasoning result of each instance in ``X``. Note that ``gt_pseudo_label`` is only used to evaluate the performance of the machine learning model but not to train it. If elements in ``X`` are unlabeled, ``gt_pseudo_label`` can be ``None``.
+ABL-Package requires data in the format of ``(X, gt_pseudo_label, Y)``  where ``X`` is a list of input examples containing instances, 
+``gt_pseudo_label`` is the ground-truth label of each example in ``X`` and ``Y`` is the ground-truth reasoning result of each example in ``X``. Note that ``gt_pseudo_label`` is only used to evaluate the machine learning model's performance but not to train it. If examples in ``X`` are unlabeled, ``gt_pseudo_label`` should be ``None``.
 
 In the MNIST Addition task, the data loading looks like
 
@@ -23,8 +23,8 @@ In the MNIST Addition task, the data loading looks like
 
    from examples.mnist_add.datasets.get_mnist_add import get_mnist_add
    
-   # train_data and test_data both consists of multiple (X, gt_pseudo_label, Y) tuples.
-   # If get_pseudo_label is False, gt_pseudo_label in each tuple will be None.
+   # train_data and test_data are tuples in the format (X, gt_pseudo_label, Y)
+   # If get_pseudo_label is set to False, the gt_pseudo_label in each tuple will be None.
    train_data = get_mnist_add(train=True, get_pseudo_label=True)
    test_data = get_mnist_add(train=False, get_pseudo_label=True)
 
@@ -33,9 +33,8 @@ Read more about `preparing datasets <Datasets.html>`_.
 Building the Learning Part
 --------------------------
 
-Learnig part is constructed by first defining a machine learning base model and then wrap it into an instance of ``ABLModel`` class. 
-The flexibility of ABL package allows the base model to be any machine learning model conforming to the scikit-learn style, which requires implementing the ``fit`` and ``predict`` methods, or a PyTorch-based neural network, provided it has defined the architecture and implemented the ``forward`` method.
-In the MNIST Addition example, we build a simple LeNet5 network as the base model.
+Learnig part is constructed by first defining a base model for machine learning. The ABL-Package offers considerable flexibility, supporting any base model that conforms to the scikit-learn style (which requires the implementation of ``fit`` and ``predict`` methods), or a PyTorch-based neural network (which has defined the architecture and implemented ``forward`` method).
+In this example, we build a simple LeNet5 network as the base model.
 
 .. code:: python
 
@@ -44,7 +43,7 @@ In the MNIST Addition example, we build a simple LeNet5 network as the base mode
    # The number of pseudo-labels is 10
    cls = LeNet5(num_classes=10)
 
-To facilitate uniform processing, ABL-Package provides the ``BasicNN`` class to convert PyTorch-based neural networks into a format similar to scikit-learn models. To construct a ``BasicNN`` instance, we need also define a loss function, an optimizer, and a device aside from the previous network.
+To facilitate uniform processing, ABL-Package provides the ``BasicNN`` class to convert a PyTorch-based neural network into a format compatible with scikit-learn models. To construct a ``BasicNN`` instance, aside from the network itself, we also need to define a loss function, an optimizer, and the computing device.
 
 .. code:: python
 
@@ -54,10 +53,9 @@ To facilitate uniform processing, ABL-Package provides the ``BasicNN`` class to
    loss_fn = torch.nn.CrossEntropyLoss()
    optimizer = torch.optim.RMSprop(cls.parameters(), lr=0.001, alpha=0.9)
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-   scheduler = torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr=0.001, pct_start=0.1, total_steps=100)
-   base_model = BasicNN(cls, loss_fn, optimizer, scheduler=scheduler, device=device)
+   base_model = BasicNN(model=cls, loss_fn=loss_fn, optimizer=optimizer, device=device)
 
-However, Base model built above are trained to make predictions on instance-level data (e.g., a single image), which is not suitable enough for our task. Therefore, we then wrap the ``base_model`` into an instance of ``ABLModel``. This class serves as a unified wrapper for base models, facilitating the learning part to train, test, and predict on example-level data, (e.g., images that comprise the equation).
+The base model built above are trained to make predictions on instance-level data (e.g., a single image), while ABL deals with example-level data. To bridge this gap, we wrap the ``base_model`` into an instance of ``ABLModel``. This class serves as a unified wrapper for base models, facilitating the learning part to train, test, and predict on example-level data, (e.g., images that comprise an equation).
 
 .. code:: python
 
@@ -70,11 +68,7 @@ Read more about `building the learning part <Learning.html>`_.
 Building the Reasoning Part
 ---------------------------
 
-To build the reasoning part, we first define a knowledge base by
-creating a subclass of ``KBBase``, which specifies how to map a pseudo 
-label example to its reasoning result. In the subclass, we initialize the 
-``pseudo_label_list`` parameter and override the ``logic_forward`` 
-function specifying how to perform (deductive) reasoning.
+To build the reasoning part, we first define a knowledge base by creating a subclass of ``KBBase``. In the subclass, we initialize the ``pseudo_label_list`` parameter and override the ``logic_forward`` method, which specifies how to perform (deductive) reasoning that processes pseudo-labels of an example to the corresponding reasoning result. Specifically for the MNIST Addition task, this ``logic_forward`` method is tailored to execute the sum operation.
 
 .. code:: python
 
@@ -87,15 +81,11 @@ function specifying how to perform (deductive) reasoning.
       def logic_forward(self, nums):
          return sum(nums)
 
-   kb = AddKB(pseudo_label_list=list(range(10)))
+   kb = AddKB()
 
-Then, we create a reasoner by instantiating the class
-``Reasoner`` and passing the knowledge base as an parameter.
-Due to the indeterminism of abductive reasoning, there could 
-be multiple candidates compatible to the knowledge base. 
-When this happens, reasoner can minimize inconsistencies between 
-the knowledge base and pseudo-labels predicted by the learning part, 
-and then return only one candidate that has the highest consistency.
+Next, we create a reasoner by instantiating the class ``Reasoner``, passing the knowledge base as a parameter.
+Due to the indeterminism of abductive reasoning, there could be multiple candidate pseudo-labels compatible to the knowledge base. 
+In such scenarios, the reasoner can minimize inconsistency and return the pseudo-label with the highest consistency.
 
 .. code:: python
 
@@ -121,7 +111,7 @@ Read more about `building evaluation metrics <Evaluation.html>`_
 Bridging Learning and Reasoning
 ---------------------------------------
 
-Now, we use ``SimpleBridge`` to combine learning and reasoning in a unified model.
+Now, we use ``SimpleBridge`` to combine learning and reasoning in a unified ABL framework.
 
 .. code:: python
 

docs/Intro/Reasoning.rst

@@ -15,14 +15,13 @@ leverage domain knowledge and perform deductive or abductive reasoning.
 In ABL-Package, building the reasoning part involves two steps:
 
 1. Build a knowledge base by creating a subclass of ``KBBase``, which
-   specifies how to map pseudo-label examples to reasoning results.
+   specifies how to process pseudo-label of an example to the reasoning result.
 2. Create a reasoner by instantiating the class ``Reasoner``
    to minimize inconsistencies between the knowledge base and pseudo
    labels predicted by the learning part.
 
 .. code:: python
 
-   # Import necessary modules
    from abl.reasoning import KBBase, GroundKB, PrologKB, Reasoner
 
 Building a knowledge base

docs/References.rst

@@ -1,7 +1,7 @@
 References
 ==========
 
-Zhi-Hua Zhou. Abductive learning: `Towards bridging machine learning and logical reasoning. <https://link.springer.com/epdf/10.1007/s11432-018-9801-4?author_access_token=jgJe1Ox3Mk-K7ORSnX7jtfe4RwlQNchNByi7wbcMAY7_PxTx-xNLP7Lp0mIZ04ORp3VG4wioIBHSCIAO3B_TBJkj87YzapmdnYVSQvgBIO3aEpQWppxZG25KolINetygc2W_Cj2gtoBdiG_J1hU3pA==>`_. **Science China Information Sciences**, 2019, 62: 076101.
+Zhi-Hua Zhou. `Abductive learning: Towards bridging machine learning and logical reasoning. <https://link.springer.com/epdf/10.1007/s11432-018-9801-4?author_access_token=jgJe1Ox3Mk-K7ORSnX7jtfe4RwlQNchNByi7wbcMAY7_PxTx-xNLP7Lp0mIZ04ORp3VG4wioIBHSCIAO3B_TBJkj87YzapmdnYVSQvgBIO3aEpQWppxZG25KolINetygc2W_Cj2gtoBdiG_J1hU3pA==>`_. **Science China Information Sciences**, 2019, 62: 076101.
 
 Zhi-Hua Zhou and Yu-Xuan Huang. `Abductive learning <https://www.lamda.nju.edu.cn/publication/chap_ABL.pdf>`_. In P. Hitzler and M. K. Sarker eds., **Neuro-Symbolic Artificial Intelligence: The State of the Art**, IOP Press, Amsterdam, 2022, p.353-379
 

docs/_static/custom.css

@@ -1,3 +1,24 @@
 div.code-out > div.highlight > pre {
     background-color: #d3effd !important; 
 }
+.green-bold {
+    color: green; 
+    font-weight: bold;
+}
+.blue-bold {
+    color: blue;
+    font-weight: bold;
+}
+.yellow-bold {
+    color: rgb(255, 192, 0);
+    font-weight: bold;
+}
+.green {
+    color: green;
+}
+.blue {
+    color: blue;
+}
+.yellow {
+    color: rgb(255, 192, 0);
+}

docs/conf.py

@@ -1,6 +1,20 @@
 import os
 import re
 import sys
+from docutils import nodes
+from docutils.parsers.rst import roles
+
+def colored_text_role(role, rawtext, text, lineno, inliner, options={}, content=[]):
+    node = nodes.inline(rawtext, text, classes=[role])
+    return [node], []
+
+roles.register_local_role('green-bold', colored_text_role)
+roles.register_local_role('blue-bold', colored_text_role)
+roles.register_local_role('yellow-bold', colored_text_role)
+roles.register_local_role('green', colored_text_role)
+roles.register_local_role('blue', colored_text_role)
+roles.register_local_role('yellow', colored_text_role)
+
 
 
 if "READTHEDOCS" not in os.environ: