Browse Source

[MNT] change SymbolMetric to SymbolAccuracy

pull/1/head
Gao Enhao 2 years ago
parent
commit
fb832c0fa0
15 changed files with 39 additions and 46 deletions
  1. +2
    -2
      abl/data/evaluation/__init__.py
  2. +1
    -1
      abl/data/evaluation/symbol_accuracy.py
  3. +3
    -3
      docs/Examples/HED.rst
  4. +3
    -3
      docs/Examples/HWF.rst
  5. +3
    -3
      docs/Examples/MNISTAdd.rst
  6. +1
    -1
      docs/Intro/Basics.rst
  7. +3
    -3
      docs/Intro/Evaluation.rst
  8. +3
    -3
      docs/Intro/Quick-Start.rst
  9. +3
    -3
      examples/hed/hed.ipynb
  10. +3
    -3
      examples/hwf/hwf.ipynb
  11. +2
    -2
      examples/hwf/main.py
  12. +2
    -2
      examples/mnist_add/main.py
  13. +5
    -12
      examples/mnist_add/mnist_add.ipynb
  14. +2
    -2
      examples/zoo/main.py
  15. +3
    -3
      examples/zoo/zoo.ipynb

+ 2
- 2
abl/data/evaluation/__init__.py View File

@@ -1,5 +1,5 @@
from .base_metric import BaseMetric
from .reasoning_metric import ReasoningMetric
from .symbol_metric import SymbolMetric
from .symbol_accuracy import SymbolAccuracy

__all__ = ["BaseMetric", "ReasoningMetric", "SymbolMetric"]
__all__ = ["BaseMetric", "ReasoningMetric", "SymbolAccuracy"]

abl/data/evaluation/symbol_metric.py → abl/data/evaluation/symbol_accuracy.py View File

@@ -6,7 +6,7 @@ from ..structures import ListData
from .base_metric import BaseMetric


class SymbolMetric(BaseMetric):
class SymbolAccuracy(BaseMetric):
"""
A metrics class for evaluating symbol-level accuracy.


+ 3
- 3
docs/Examples/HED.rst View File

@@ -32,7 +32,7 @@ model.
from examples.models.nn import SymbolNet
from abl.learning import ABLModel, BasicNN
from examples.hed.reasoning import HedKB, HedReasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log
from examples.hed.bridge import HedBridge

@@ -259,7 +259,7 @@ Building Evaluation Metrics

Next, we set up evaluation metrics. These metrics will be used to
evaluate the model performance during training and testing.
Specifically, we use ``SymbolMetric`` and ``ReasoningMetric``, which are
Specifically, we use ``SymbolAccuracy`` and ``ReasoningMetric``, which are
used to evaluate the accuracy of the machine learning model’s
predictions and the accuracy of the final reasoning results,
respectively.
@@ -267,7 +267,7 @@ respectively.
.. code:: ipython3

# Set up metrics
metric_list = [SymbolMetric(prefix="hed"), ReasoningMetric(kb=kb, prefix="hed")]
metric_list = [SymbolAccuracy(prefix="hed"), ReasoningMetric(kb=kb, prefix="hed")]

Bridge Learning and Reasoning
-----------------------------


+ 3
- 3
docs/Examples/HWF.rst View File

@@ -30,7 +30,7 @@ machine learning model.
from examples.models.nn import SymbolNet
from abl.learning import ABLModel, BasicNN
from abl.reasoning import KBBase, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log
from abl.bridge import SimpleBridge

@@ -353,14 +353,14 @@ Building Evaluation Metrics

Next, we set up evaluation metrics. These metrics will be used to
evaluate the model performance during training and testing.
Specifically, we use ``SymbolMetric`` and ``ReasoningMetric``, which are
Specifically, we use ``SymbolAccuracy`` and ``ReasoningMetric``, which are
used to evaluate the accuracy of the machine learning model’s
predictions and the accuracy of the final reasoning results,
respectively.

.. code:: ipython3

metric_list = [SymbolMetric(prefix="hwf"), ReasoningMetric(kb=kb, prefix="hwf")]
metric_list = [SymbolAccuracy(prefix="hwf"), ReasoningMetric(kb=kb, prefix="hwf")]

Bridge Learning and Reasoning
-----------------------------


+ 3
- 3
docs/Examples/MNISTAdd.rst View File

@@ -28,7 +28,7 @@ machine learning model.
from examples.models.nn import LeNet5
from abl.learning import ABLModel, BasicNN
from abl.reasoning import KBBase, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log
from abl.bridge import SimpleBridge

@@ -300,14 +300,14 @@ Building Evaluation Metrics

Next, we set up evaluation metrics. These metrics will be used to
evaluate the model performance during training and testing.
Specifically, we use ``SymbolMetric`` and ``ReasoningMetric``, which are
Specifically, we use ``SymbolAccuracy`` and ``ReasoningMetric``, which are
used to evaluate the accuracy of the machine learning model’s
predictions and the accuracy of the final reasoning results,
respectively.

.. code:: ipython3

metric_list = [SymbolMetric(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]
metric_list = [SymbolAccuracy(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]

Bridge Learning and Reasoning
-----------------------------


+ 1
- 1
docs/Intro/Basics.rst View File

@@ -26,7 +26,7 @@ It first features class ``ListData`` (derived from base class
``BaseDataElement``), which defines the data structures used in
Abductive Learning, and comprises common data operations like insertion,
deletion, retrieval, slicing, etc. Additionally, a series of Evaluation
Metrics, including class ``SymbolMetric`` and ``ReasoningMetric`` (both
Metrics, including class ``SymbolAccuracy`` and ``ReasoningMetric`` (both
specialized metrics derived from base class ``BaseMetric``), outline
methods for evaluating model quality from a data perspective.



+ 3
- 3
docs/Intro/Evaluation.rst View File

@@ -15,7 +15,7 @@ In this section, we will look at how to build evaluation metrics.
.. code:: python

# Import necessary modules
from abl.data.evaluation import BaseMetric, SymbolMetric, ReasoningMetric
from abl.data.evaluation import BaseMetric, SymbolAccuracy, ReasoningMetric

ABL-Package seperates the evaluation process from model training and testing as an independent class, ``BaseMetric``. The training and testing processes are implemented in the ``BaseBridge`` class, so metrics are used by this class and its sub-classes. After building a ``bridge`` with a list of ``BaseMetric`` instances, these metrics will be used by the ``bridge.valid`` method to evaluate the model performance during training and testing.

@@ -25,11 +25,11 @@ To customize our own metrics, we need to inherit from ``BaseMetric`` and impleme
- The ``compute_metrics`` method uses all the information saved in ``self.results`` to calculate and return a dict that holds the evaluation results.

Besides, we can assign a ``str`` to the ``prefix`` argument of the ``__init__`` function. This string is automatically prefixed to the output metric names. For example, if we set ``prefix="mnist_add"``, the output metric name will be ``character_accuracy``.
We provide two basic metrics, namely ``SymbolMetric`` and ``ReasoningMetric``, which are used to evaluate the accuracy of the machine learning model's predictions and the accuracy of the final reasoning results, respectively. Using ``SymbolMetric`` as an example, the following code shows how to implement a custom metrics.
We provide two basic metrics, namely ``SymbolAccuracy`` and ``ReasoningMetric``, which are used to evaluate the accuracy of the machine learning model's predictions and the accuracy of the final reasoning results, respectively. Using ``SymbolAccuracy`` as an example, the following code shows how to implement a custom metrics.

.. code:: python

class SymbolMetric(BaseMetric):
class SymbolAccuracy(BaseMetric):
def __init__(self, prefix: Optional[str] = None) -> None:
# prefix is used to distinguish different metrics
super().__init__(prefix)


+ 3
- 3
docs/Intro/Quick-Start.rst View File

@@ -107,13 +107,13 @@ Read more about `building the reasoning part <Reasoning.html>`_.
Building Evaluation Metrics
---------------------------

ABL-Package provides two basic metrics, namely ``SymbolMetric`` and ``ReasoningMetric``, which are used to evaluate the accuracy of the machine learning model's predictions and the accuracy of the ``logic_forward`` results, respectively.
ABL-Package provides two basic metrics, namely ``SymbolAccuracy`` and ``ReasoningMetric``, which are used to evaluate the accuracy of the machine learning model's predictions and the accuracy of the ``logic_forward`` results, respectively.

.. code:: python

from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy

metric_list = [SymbolMetric(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]
metric_list = [SymbolAccuracy(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]

Read more about `building evaluation metrics <Evaluation.html>`_



+ 3
- 3
examples/hed/hed.ipynb View File

@@ -26,7 +26,7 @@
"from examples.models.nn import SymbolNet\n",
"from abl.learning import ABLModel, BasicNN\n",
"from examples.hed.reasoning import HedKB, HedReasoner\n",
"from abl.data.evaluation import ReasoningMetric, SymbolMetric\n",
"from abl.data.evaluation import ReasoningMetric, SymbolAccuracy\n",
"from abl.utils import ABLLogger, print_log\n",
"from examples.hed.bridge import HedBridge"
]
@@ -335,7 +335,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolMetric` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolAccuracy` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
]
},
{
@@ -345,7 +345,7 @@
"outputs": [],
"source": [
"# Set up metrics\n",
"metric_list = [SymbolMetric(prefix=\"hed\"), ReasoningMetric(kb=kb, prefix=\"hed\")]"
"metric_list = [SymbolAccuracy(prefix=\"hed\"), ReasoningMetric(kb=kb, prefix=\"hed\")]"
]
},
{


+ 3
- 3
examples/hwf/hwf.ipynb View File

@@ -27,7 +27,7 @@
"from examples.models.nn import SymbolNet\n",
"from abl.learning import ABLModel, BasicNN\n",
"from abl.reasoning import KBBase, Reasoner\n",
"from abl.data.evaluation import ReasoningMetric, SymbolMetric\n",
"from abl.data.evaluation import ReasoningMetric, SymbolAccuracy\n",
"from abl.utils import ABLLogger, print_log\n",
"from abl.bridge import SimpleBridge"
]
@@ -434,7 +434,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolMetric` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolAccuracy` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
]
},
{
@@ -443,7 +443,7 @@
"metadata": {},
"outputs": [],
"source": [
"metric_list = [SymbolMetric(prefix=\"hwf\"), ReasoningMetric(kb=kb, prefix=\"hwf\")]"
"metric_list = [SymbolAccuracy(prefix=\"hwf\"), ReasoningMetric(kb=kb, prefix=\"hwf\")]"
]
},
{


+ 2
- 2
examples/hwf/main.py View File

@@ -10,7 +10,7 @@ from examples.hwf.datasets import get_dataset
from examples.models.nn import SymbolNet
from abl.learning import ABLModel, BasicNN
from abl.reasoning import KBBase, GroundKB, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log
from abl.bridge import SimpleBridge

@@ -152,7 +152,7 @@ def main():
)

### Building Evaluation Metrics
metric_list = [SymbolMetric(prefix="hwf"), ReasoningMetric(kb=kb, prefix="hwf")]
metric_list = [SymbolAccuracy(prefix="hwf"), ReasoningMetric(kb=kb, prefix="hwf")]

### Bridge Learning and Reasoning
bridge = SimpleBridge(model, reasoner, metric_list)


+ 2
- 2
examples/mnist_add/main.py View File

@@ -9,7 +9,7 @@ from examples.mnist_add.datasets import get_dataset
from examples.models.nn import LeNet5
from abl.learning import ABLModel, BasicNN
from abl.reasoning import KBBase, GroundKB, PrologKB, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log
from abl.bridge import SimpleBridge

@@ -117,7 +117,7 @@ def main():
)

### Building Evaluation Metrics
metric_list = [SymbolMetric(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]
metric_list = [SymbolAccuracy(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]

### Bridge Learning and Reasoning
bridge = SimpleBridge(model, reasoner, metric_list)


+ 5
- 12
examples/mnist_add/mnist_add.ipynb View File

@@ -26,7 +26,7 @@
"from examples.models.nn import LeNet5\n",
"from abl.learning import ABLModel, BasicNN\n",
"from abl.reasoning import KBBase, Reasoner\n",
"from abl.data.evaluation import ReasoningMetric, SymbolMetric\n",
"from abl.data.evaluation import ReasoningMetric, SymbolAccuracy\n",
"from abl.utils import ABLLogger, print_log\n",
"from abl.bridge import SimpleBridge"
]
@@ -122,7 +122,7 @@
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAD1CAYAAADNj/Z6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAKUklEQVR4nO3df6zVdR3H8XPuDy4iCHcRUnKRQIFkOFHzV+Z0maVhM5dZyzY1JWCUbmk/tkwrXWXRQB0xayVUarOcthhmFJpL5GflYgTiryyV3wgIXrjnnv6x/kh4fy+de++58H48/n2d+z3ff+7hyXe7n1OuVqvVEgCQVkO9bwAAqC8xAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAguaauvvADDZf15H0AXfC7zgfqfQsHzWcH1F/RZ4cnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBIrqneNwBAbg2DBoV7+Z1H99KdHNiLlw4L9/ahneHetCv+v/foezeGe2Xt+nCvlScDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLOGehGjUMGh/tzd48M99XvnRfu7dWOcD971afDvVyuxvtDbwv3hko412zob9aFe2Xzlp69AaAunr9hYrj//do5vXQn9XP/J1rD/Sfjju3R9/dkAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEjOOQPdqLJjV7hfevxfw72zFJ8D0FxuDPelp9wb7g2lcvz+J8fv39OmTD033Ddc/PZwr2za1I13A3RV07Ft4d7x4kvh3m9HfP3jfj4t3C86b0V8gW6w8A+nhnv/zfHn67CV7eHe/Hj870OpFJ8zUytPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJOfQoe7UWQnnXzx2Vrh//eN/7s67OeTc3fZYuH/oxCnh3vR7hw5BPTTP3xvu65aeGe6jv/hkTe+/tqaf7prRpSU9ev36HvnmyQAApCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnHMGetG4b8R/DTtx54yarj/0tA3hfuaw58O9s1oO9wdXnhLuyy+cFe6DG/qHO9A3bfh8fEbK0uNmh/v4Z6d15+3QAzwZAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJJzzkAvqmzbFu7Hfq1nvy/76cJXxN+o/e4JO8L9tQ/GPz+4ID2/s2VCuPdbsibcO+PLAwfQ2Noa7rd8bn64b620h/vYufFO/XkyAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACTnnAG6bM+sN8J9ZNMRNV1/0ZffF+4tu5fXdH1g/9bMHBPulxy5ONzHzrsx3N+1rGfPUKF2ngwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyTlngP+qnHdyuP903B0FV4jPGViwe3C4H7lmY7h3FLw7sH8v33hWuD9+/u0FVxgYrsfNXBfulYKrU3+eDABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJOWcgkYb+/cP9nnnxOQJDG+NzBHZX94b7HdMuD/fm51eGO7B/uy47PdxXXD873FvK8TkCReauejjcZ286J9yX3faecB/w4NKDvicOjicDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLOGUhkz6+Hh3vROQJFzllxdbgPX+QcAfh/NI04Jtx/9r2ZBVdoCdevbpwY7le1LomvXo7ffeY7VoX7rG9uDfdH/zgm3Cubt8Q3QCFPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDknDNwGNk446xwXzHhrnDvLLj+svb4j4lHTI3/Vrij4PrAAXTEvz1T1n8y3F9e1BbuI771ZLgvL50d7o1HDwv3zzzxVLhf3/pCuP/gugvDfdRN8TkIFPNkAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEjOOQOHkPKkCeG+4ivxOQKN5YL2q8YnDVz74xnh3vZq/LfKcCja+PD4cB82cFfhNRo/9nq4V7ZtC/eOVzeEe8P74/cfUXopfkGNqrv3hPuWysCCK+wI136vxWecUDtPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDknDPQl5xxYjjf98DccO8stcTXLzhH4PhfTg/3cd//S8H7w+Hn6EE7w/2R8QsKr3HC9Ph3q+22vn1GR7m5X7ivvTU+A2XK4CfC/Ve7jgr3th+tDvdKuNIVngwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyTlnoBc1DBgQ7ttvjr/zfGBDwTkCBS5/7oJwH3vDqnDv3Le3pveHQ9G61SPiF4wvvsa3r7wn3L/UdGW4D3kmPsVj4D/eCPddI/uHe0dLOdxPnx5/NjxyTHwGyu7O+LPj1juuCPdh2/v2OQyHA08GACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOScM9CLXrnmpHBffuKdNV3/6b3xt3q/flF7uFedIwBvMX7O1nBfffGewmt85MiC/bNzDuaW3mJzJT6jZGhjwQ3UaFn7vnC/dvb14T78LucI1JsnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByzhnoRo0njA33R2/8bsEV4u8c31eNzxGYfvN14T5k55KC9wf+V2XNM+E+efGMwmusv+CH4d5Yru3/ZT19jsD9O1vDff4l54f78DXOEejrPBkAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAknPOwEFoGDAg3J+9OT4noLUh3ouc+tTV4d423zkC0NvGXrWy8DWTHvpUuD992n3ddTv7dcUL54b7qgUnhPuoBzaEe2VdfBYDfZ8nAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAknPo0EF45ZqTwv1vZ99Z0/X/9EZzuI+aVnDwR03vDvSUEVe+HO7nT4oPFHvhw/3CvW1R/Nvf77cr4p+vPhnuPlsOf54MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMk5Z+A/TptY+JKFN9xe8IojwvWfHXvC/ZYZU8O9ZdPygvcH+qLK9tfCvXHxqnAfs7g77wbeypMBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDnnDLxp31Hx94WXSqXS0Mb4HIEiN/1rcri3LHSOAAC9z5MBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDnnDHSjL7xyRrhv/WjxWQYA0Ns8GQCA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSc87Am5oXrSx8zeRjTil4xb6CfWOX7wcAeosnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLlarVarfdNAAD148kAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJPdvx8SA9zSwVzUAAAAASUVORK5CYII=",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAD1CAYAAADNj/Z6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAKUklEQVR4nO3df6zVdR3H8XPuDy4iCHcRUnKRQIFkOFHzV+Z0maVhM5dZyzY1JWCUbmk/tkwrXWXRQB0xayVUarOcthhmFJpL5GflYgTiryyV3wgIXrjnnv6x/kh4fy+de++58H48/n2d+z3ff+7hyXe7n1OuVqvVEgCQVkO9bwAAqC8xAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAguaauvvADDZf15H0AXfC7zgfqfQsHzWcH1F/RZ4cnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBIrqneNwBAbg2DBoV7+Z1H99KdHNiLlw4L9/ahneHetCv+v/foezeGe2Xt+nCvlScDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLOGehGjUMGh/tzd48M99XvnRfu7dWOcD971afDvVyuxvtDbwv3hko412zob9aFe2Xzlp69AaAunr9hYrj//do5vXQn9XP/J1rD/Sfjju3R9/dkAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEjOOQPdqLJjV7hfevxfw72zFJ8D0FxuDPelp9wb7g2lcvz+J8fv39OmTD033Ddc/PZwr2za1I13A3RV07Ft4d7x4kvh3m9HfP3jfj4t3C86b0V8gW6w8A+nhnv/zfHn67CV7eHe/Hj870OpFJ8zUytPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJOfQoe7UWQnnXzx2Vrh//eN/7s67OeTc3fZYuH/oxCnh3vR7hw5BPTTP3xvu65aeGe6jv/hkTe+/tqaf7prRpSU9ev36HvnmyQAApCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnHMGetG4b8R/DTtx54yarj/0tA3hfuaw58O9s1oO9wdXnhLuyy+cFe6DG/qHO9A3bfh8fEbK0uNmh/v4Z6d15+3QAzwZAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJJzzkAvqmzbFu7Hfq1nvy/76cJXxN+o/e4JO8L9tQ/GPz+4ID2/s2VCuPdbsibcO+PLAwfQ2Noa7rd8bn64b620h/vYufFO/XkyAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACTnnAG6bM+sN8J9ZNMRNV1/0ZffF+4tu5fXdH1g/9bMHBPulxy5ONzHzrsx3N+1rGfPUKF2ngwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyTlngP+qnHdyuP903B0FV4jPGViwe3C4H7lmY7h3FLw7sH8v33hWuD9+/u0FVxgYrsfNXBfulYKrU3+eDABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJOWcgkYb+/cP9nnnxOQJDG+NzBHZX94b7HdMuD/fm51eGO7B/uy47PdxXXD873FvK8TkCReauejjcZ286J9yX3faecB/w4NKDvicOjicDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLOGUhkz6+Hh3vROQJFzllxdbgPX+QcAfh/NI04Jtx/9r2ZBVdoCdevbpwY7le1LomvXo7ffeY7VoX7rG9uDfdH/zgm3Cubt8Q3QCFPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDknDNwGNk446xwXzHhrnDvLLj+svb4j4lHTI3/Vrij4PrAAXTEvz1T1n8y3F9e1BbuI771ZLgvL50d7o1HDwv3zzzxVLhf3/pCuP/gugvDfdRN8TkIFPNkAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEjOOQOHkPKkCeG+4ivxOQKN5YL2q8YnDVz74xnh3vZq/LfKcCja+PD4cB82cFfhNRo/9nq4V7ZtC/eOVzeEe8P74/cfUXopfkGNqrv3hPuWysCCK+wI136vxWecUDtPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDknDPQl5xxYjjf98DccO8stcTXLzhH4PhfTg/3cd//S8H7w+Hn6EE7w/2R8QsKr3HC9Ph3q+22vn1GR7m5X7ivvTU+A2XK4CfC/Ve7jgr3th+tDvdKuNIVngwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyTlnoBc1DBgQ7ttvjr/zfGBDwTkCBS5/7oJwH3vDqnDv3Le3pveHQ9G61SPiF4wvvsa3r7wn3L/UdGW4D3kmPsVj4D/eCPddI/uHe0dLOdxPnx5/NjxyTHwGyu7O+LPj1juuCPdh2/v2OQyHA08GACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOScM9CLXrnmpHBffuKdNV3/6b3xt3q/flF7uFedIwBvMX7O1nBfffGewmt85MiC/bNzDuaW3mJzJT6jZGhjwQ3UaFn7vnC/dvb14T78LucI1JsnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByzhnoRo0njA33R2/8bsEV4u8c31eNzxGYfvN14T5k55KC9wf+V2XNM+E+efGMwmusv+CH4d5Yru3/ZT19jsD9O1vDff4l54f78DXOEejrPBkAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAknPOwEFoGDAg3J+9OT4noLUh3ouc+tTV4d423zkC0NvGXrWy8DWTHvpUuD992n3ddTv7dcUL54b7qgUnhPuoBzaEe2VdfBYDfZ8nAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAknPo0EF45ZqTwv1vZ99Z0/X/9EZzuI+aVnDwR03vDvSUEVe+HO7nT4oPFHvhw/3CvW1R/Nvf77cr4p+vPhnuPlsOf54MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMk5Z+A/TptY+JKFN9xe8IojwvWfHXvC/ZYZU8O9ZdPygvcH+qLK9tfCvXHxqnAfs7g77wbeypMBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDnnDLxp31Hx94WXSqXS0Mb4HIEiN/1rcri3LHSOAAC9z5MBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDnnDHSjL7xyRrhv/WjxWQYA0Ns8GQCA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSc87Am5oXrSx8zeRjTil4xb6CfWOX7wcAeosnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLlarVarfdNAAD148kAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJPdvx8SA9zSwVzUAAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 640x480 with 2 Axes>"
]
@@ -380,7 +380,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolMetric` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolAccuracy` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
]
},
{
@@ -389,7 +389,7 @@
"metadata": {},
"outputs": [],
"source": [
"metric_list = [SymbolMetric(prefix=\"mnist_add\"), ReasoningMetric(kb=kb, prefix=\"mnist_add\")]"
"metric_list = [SymbolAccuracy(prefix=\"mnist_add\"), ReasoningMetric(kb=kb, prefix=\"mnist_add\")]"
]
},
{
@@ -420,7 +420,7 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@@ -432,13 +432,6 @@
"bridge.train(train_data, loops=5, segment_size=1/3, save_interval=1, save_dir=weights_dir)\n",
"bridge.test(test_data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {


+ 2
- 2
examples/zoo/main.py View File

@@ -8,7 +8,7 @@ import openml

from abl.learning import ABLModel
from abl.reasoning import KBBase, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.bridge import SimpleBridge
from abl.utils.utils import confidence_dist
from abl.utils import ABLLogger, print_log
@@ -182,7 +182,7 @@ train_data = transform_tab_data(X_unlabel, y_unlabel)

# %%
# Set up metrics
metric_list = [SymbolMetric(prefix="zoo"), ReasoningMetric(kb=kb, prefix="zoo")]
metric_list = [SymbolAccuracy(prefix="zoo"), ReasoningMetric(kb=kb, prefix="zoo")]

# %% [markdown]
# ### Bridge Machine Learning and Logic Reasoning


+ 3
- 3
examples/zoo/zoo.ipynb View File

@@ -25,7 +25,7 @@
"from abl.learning import ABLModel\n",
"from examples.zoo.kb import ZooKB\n",
"from abl.reasoning import Reasoner\n",
"from abl.data.evaluation import ReasoningMetric, SymbolMetric\n",
"from abl.data.evaluation import ReasoningMetric, SymbolAccuracy\n",
"from abl.utils import ABLLogger, print_log, confidence_dist\n",
"from abl.bridge import SimpleBridge"
]
@@ -276,7 +276,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolMetric` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolAccuracy` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
]
},
{
@@ -285,7 +285,7 @@
"metadata": {},
"outputs": [],
"source": [
"metric_list = [SymbolMetric(prefix=\"zoo\"), ReasoningMetric(kb=kb, prefix=\"zoo\")]"
"metric_list = [SymbolAccuracy(prefix=\"zoo\"), ReasoningMetric(kb=kb, prefix=\"zoo\")]"
]
},
{


Loading…
Cancel
Save