AbductiveLearning
/
ABLkit
Not watched
1
0
Fork 0
Code
Releases 0 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
Browse Source

[MNT] change SymbolMetric to SymbolAccuracy

pull/1/head
Gao Enhao 2 years ago
parent
941bcb867d
commit
fb832c0fa0
15 changed files with 39 additions and 46 deletions
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +2
    -2
      abl/data/evaluation/__init__.py
  2. +1
    -1
      abl/data/evaluation/symbol_accuracy.py
  3. +3
    -3
      docs/Examples/HED.rst
  4. +3
    -3
      docs/Examples/HWF.rst
  5. +3
    -3
      docs/Examples/MNISTAdd.rst
  6. +1
    -1
      docs/Intro/Basics.rst
  7. +3
    -3
      docs/Intro/Evaluation.rst
  8. +3
    -3
      docs/Intro/Quick-Start.rst
  9. +3
    -3
      examples/hed/hed.ipynb
  10. +3
    -3
      examples/hwf/hwf.ipynb
  11. +2
    -2
      examples/hwf/main.py
  12. +2
    -2
      examples/mnist_add/main.py
  13. +5
    -12
      examples/mnist_add/mnist_add.ipynb
  14. +2
    -2
      examples/zoo/main.py
  15. +3
    -3
      examples/zoo/zoo.ipynb

+ 2
- 2
abl/data/evaluation/__init__.py View File

@@ -1,5 +1,5 @@
from .base_metric import BaseMetric from .base_metric import BaseMetric
from .reasoning_metric import ReasoningMetric from .reasoning_metric import ReasoningMetric
from .symbol_metric import SymbolMetric
from .symbol_accuracy import SymbolAccuracy


__all__ = ["BaseMetric", "ReasoningMetric", "SymbolMetric"]
__all__ = ["BaseMetric", "ReasoningMetric", "SymbolAccuracy"]

abl/data/evaluation/symbol_metric.py → abl/data/evaluation/symbol_accuracy.py View File

@@ -6,7 +6,7 @@ from ..structures import ListData
from .base_metric import BaseMetric from .base_metric import BaseMetric




class SymbolMetric(BaseMetric):
class SymbolAccuracy(BaseMetric):
""" """
A metrics class for evaluating symbol-level accuracy. A metrics class for evaluating symbol-level accuracy.



+ 3
- 3
docs/Examples/HED.rst View File

@@ -32,7 +32,7 @@ model.
from examples.models.nn import SymbolNet from examples.models.nn import SymbolNet
from abl.learning import ABLModel, BasicNN from abl.learning import ABLModel, BasicNN
from examples.hed.reasoning import HedKB, HedReasoner from examples.hed.reasoning import HedKB, HedReasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log from abl.utils import ABLLogger, print_log
from examples.hed.bridge import HedBridge from examples.hed.bridge import HedBridge


@@ -259,7 +259,7 @@ Building Evaluation Metrics


Next, we set up evaluation metrics. These metrics will be used to Next, we set up evaluation metrics. These metrics will be used to
evaluate the model performance during training and testing. evaluate the model performance during training and testing.
Specifically, we use ``SymbolMetric`` and ``ReasoningMetric``, which are
Specifically, we use ``SymbolAccuracy`` and ``ReasoningMetric``, which are
used to evaluate the accuracy of the machine learning model’s used to evaluate the accuracy of the machine learning model’s
predictions and the accuracy of the final reasoning results, predictions and the accuracy of the final reasoning results,
respectively. respectively.
@@ -267,7 +267,7 @@ respectively.
.. code:: ipython3 .. code:: ipython3


# Set up metrics # Set up metrics
metric_list = [SymbolMetric(prefix="hed"), ReasoningMetric(kb=kb, prefix="hed")]
metric_list = [SymbolAccuracy(prefix="hed"), ReasoningMetric(kb=kb, prefix="hed")]


Bridge Learning and Reasoning Bridge Learning and Reasoning
----------------------------- -----------------------------


+ 3
- 3
docs/Examples/HWF.rst View File

@@ -30,7 +30,7 @@ machine learning model.
from examples.models.nn import SymbolNet from examples.models.nn import SymbolNet
from abl.learning import ABLModel, BasicNN from abl.learning import ABLModel, BasicNN
from abl.reasoning import KBBase, Reasoner from abl.reasoning import KBBase, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log from abl.utils import ABLLogger, print_log
from abl.bridge import SimpleBridge from abl.bridge import SimpleBridge


@@ -353,14 +353,14 @@ Building Evaluation Metrics


Next, we set up evaluation metrics. These metrics will be used to Next, we set up evaluation metrics. These metrics will be used to
evaluate the model performance during training and testing. evaluate the model performance during training and testing.
Specifically, we use ``SymbolMetric`` and ``ReasoningMetric``, which are
Specifically, we use ``SymbolAccuracy`` and ``ReasoningMetric``, which are
used to evaluate the accuracy of the machine learning model’s used to evaluate the accuracy of the machine learning model’s
predictions and the accuracy of the final reasoning results, predictions and the accuracy of the final reasoning results,
respectively. respectively.


.. code:: ipython3 .. code:: ipython3


metric_list = [SymbolMetric(prefix="hwf"), ReasoningMetric(kb=kb, prefix="hwf")]
metric_list = [SymbolAccuracy(prefix="hwf"), ReasoningMetric(kb=kb, prefix="hwf")]


Bridge Learning and Reasoning Bridge Learning and Reasoning
----------------------------- -----------------------------


+ 3
- 3
docs/Examples/MNISTAdd.rst View File

@@ -28,7 +28,7 @@ machine learning model.
from examples.models.nn import LeNet5 from examples.models.nn import LeNet5
from abl.learning import ABLModel, BasicNN from abl.learning import ABLModel, BasicNN
from abl.reasoning import KBBase, Reasoner from abl.reasoning import KBBase, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log from abl.utils import ABLLogger, print_log
from abl.bridge import SimpleBridge from abl.bridge import SimpleBridge


@@ -300,14 +300,14 @@ Building Evaluation Metrics


Next, we set up evaluation metrics. These metrics will be used to Next, we set up evaluation metrics. These metrics will be used to
evaluate the model performance during training and testing. evaluate the model performance during training and testing.
Specifically, we use ``SymbolMetric`` and ``ReasoningMetric``, which are
Specifically, we use ``SymbolAccuracy`` and ``ReasoningMetric``, which are
used to evaluate the accuracy of the machine learning model’s used to evaluate the accuracy of the machine learning model’s
predictions and the accuracy of the final reasoning results, predictions and the accuracy of the final reasoning results,
respectively. respectively.


.. code:: ipython3 .. code:: ipython3


metric_list = [SymbolMetric(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]
metric_list = [SymbolAccuracy(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]


Bridge Learning and Reasoning Bridge Learning and Reasoning
----------------------------- -----------------------------


+ 1
- 1
docs/Intro/Basics.rst View File

@@ -26,7 +26,7 @@ It first features class ``ListData`` (derived from base class
``BaseDataElement``), which defines the data structures used in ``BaseDataElement``), which defines the data structures used in
Abductive Learning, and comprises common data operations like insertion, Abductive Learning, and comprises common data operations like insertion,
deletion, retrieval, slicing, etc. Additionally, a series of Evaluation deletion, retrieval, slicing, etc. Additionally, a series of Evaluation
Metrics, including class ``SymbolMetric`` and ``ReasoningMetric`` (both
Metrics, including class ``SymbolAccuracy`` and ``ReasoningMetric`` (both
specialized metrics derived from base class ``BaseMetric``), outline specialized metrics derived from base class ``BaseMetric``), outline
methods for evaluating model quality from a data perspective. methods for evaluating model quality from a data perspective.




+ 3
- 3
docs/Intro/Evaluation.rst View File

@@ -15,7 +15,7 @@ In this section, we will look at how to build evaluation metrics.
.. code:: python .. code:: python


# Import necessary modules # Import necessary modules
from abl.data.evaluation import BaseMetric, SymbolMetric, ReasoningMetric
from abl.data.evaluation import BaseMetric, SymbolAccuracy, ReasoningMetric


ABL-Package seperates the evaluation process from model training and testing as an independent class, ``BaseMetric``. The training and testing processes are implemented in the ``BaseBridge`` class, so metrics are used by this class and its sub-classes. After building a ``bridge`` with a list of ``BaseMetric`` instances, these metrics will be used by the ``bridge.valid`` method to evaluate the model performance during training and testing. ABL-Package seperates the evaluation process from model training and testing as an independent class, ``BaseMetric``. The training and testing processes are implemented in the ``BaseBridge`` class, so metrics are used by this class and its sub-classes. After building a ``bridge`` with a list of ``BaseMetric`` instances, these metrics will be used by the ``bridge.valid`` method to evaluate the model performance during training and testing.


@@ -25,11 +25,11 @@ To customize our own metrics, we need to inherit from ``BaseMetric`` and impleme
- The ``compute_metrics`` method uses all the information saved in ``self.results`` to calculate and return a dict that holds the evaluation results. - The ``compute_metrics`` method uses all the information saved in ``self.results`` to calculate and return a dict that holds the evaluation results.


Besides, we can assign a ``str`` to the ``prefix`` argument of the ``__init__`` function. This string is automatically prefixed to the output metric names. For example, if we set ``prefix="mnist_add"``, the output metric name will be ``character_accuracy``. Besides, we can assign a ``str`` to the ``prefix`` argument of the ``__init__`` function. This string is automatically prefixed to the output metric names. For example, if we set ``prefix="mnist_add"``, the output metric name will be ``character_accuracy``.
We provide two basic metrics, namely ``SymbolMetric`` and ``ReasoningMetric``, which are used to evaluate the accuracy of the machine learning model's predictions and the accuracy of the final reasoning results, respectively. Using ``SymbolMetric`` as an example, the following code shows how to implement a custom metrics.
We provide two basic metrics, namely ``SymbolAccuracy`` and ``ReasoningMetric``, which are used to evaluate the accuracy of the machine learning model's predictions and the accuracy of the final reasoning results, respectively. Using ``SymbolAccuracy`` as an example, the following code shows how to implement a custom metrics.


.. code:: python .. code:: python


class SymbolMetric(BaseMetric):
class SymbolAccuracy(BaseMetric):
def __init__(self, prefix: Optional[str] = None) -> None: def __init__(self, prefix: Optional[str] = None) -> None:
# prefix is used to distinguish different metrics # prefix is used to distinguish different metrics
super().__init__(prefix) super().__init__(prefix)


+ 3
- 3
docs/Intro/Quick-Start.rst View File

@@ -107,13 +107,13 @@ Read more about `building the reasoning part <Reasoning.html>`_.
Building Evaluation Metrics Building Evaluation Metrics
--------------------------- ---------------------------


ABL-Package provides two basic metrics, namely ``SymbolMetric`` and ``ReasoningMetric``, which are used to evaluate the accuracy of the machine learning model's predictions and the accuracy of the ``logic_forward`` results, respectively.
ABL-Package provides two basic metrics, namely ``SymbolAccuracy`` and ``ReasoningMetric``, which are used to evaluate the accuracy of the machine learning model's predictions and the accuracy of the ``logic_forward`` results, respectively.


.. code:: python .. code:: python


from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy


metric_list = [SymbolMetric(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]
metric_list = [SymbolAccuracy(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]


Read more about `building evaluation metrics <Evaluation.html>`_ Read more about `building evaluation metrics <Evaluation.html>`_




+ 3
- 3
examples/hed/hed.ipynb View File

@@ -26,7 +26,7 @@
"from examples.models.nn import SymbolNet\n", "from examples.models.nn import SymbolNet\n",
"from abl.learning import ABLModel, BasicNN\n", "from abl.learning import ABLModel, BasicNN\n",
"from examples.hed.reasoning import HedKB, HedReasoner\n", "from examples.hed.reasoning import HedKB, HedReasoner\n",
"from abl.data.evaluation import ReasoningMetric, SymbolMetric\n",
"from abl.data.evaluation import ReasoningMetric, SymbolAccuracy\n",
"from abl.utils import ABLLogger, print_log\n", "from abl.utils import ABLLogger, print_log\n",
"from examples.hed.bridge import HedBridge" "from examples.hed.bridge import HedBridge"
] ]
@@ -335,7 +335,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolMetric` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolAccuracy` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
] ]
}, },
{ {
@@ -345,7 +345,7 @@
"outputs": [], "outputs": [],
"source": [ "source": [
"# Set up metrics\n", "# Set up metrics\n",
"metric_list = [SymbolMetric(prefix=\"hed\"), ReasoningMetric(kb=kb, prefix=\"hed\")]"
"metric_list = [SymbolAccuracy(prefix=\"hed\"), ReasoningMetric(kb=kb, prefix=\"hed\")]"
] ]
}, },
{ {


+ 3
- 3
examples/hwf/hwf.ipynb View File

@@ -27,7 +27,7 @@
"from examples.models.nn import SymbolNet\n", "from examples.models.nn import SymbolNet\n",
"from abl.learning import ABLModel, BasicNN\n", "from abl.learning import ABLModel, BasicNN\n",
"from abl.reasoning import KBBase, Reasoner\n", "from abl.reasoning import KBBase, Reasoner\n",
"from abl.data.evaluation import ReasoningMetric, SymbolMetric\n",
"from abl.data.evaluation import ReasoningMetric, SymbolAccuracy\n",
"from abl.utils import ABLLogger, print_log\n", "from abl.utils import ABLLogger, print_log\n",
"from abl.bridge import SimpleBridge" "from abl.bridge import SimpleBridge"
] ]
@@ -434,7 +434,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolMetric` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolAccuracy` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
] ]
}, },
{ {
@@ -443,7 +443,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"metric_list = [SymbolMetric(prefix=\"hwf\"), ReasoningMetric(kb=kb, prefix=\"hwf\")]"
"metric_list = [SymbolAccuracy(prefix=\"hwf\"), ReasoningMetric(kb=kb, prefix=\"hwf\")]"
] ]
}, },
{ {


+ 2
- 2
examples/hwf/main.py View File

@@ -10,7 +10,7 @@ from examples.hwf.datasets import get_dataset
from examples.models.nn import SymbolNet from examples.models.nn import SymbolNet
from abl.learning import ABLModel, BasicNN from abl.learning import ABLModel, BasicNN
from abl.reasoning import KBBase, GroundKB, Reasoner from abl.reasoning import KBBase, GroundKB, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log from abl.utils import ABLLogger, print_log
from abl.bridge import SimpleBridge from abl.bridge import SimpleBridge


@@ -152,7 +152,7 @@ def main():
) )


### Building Evaluation Metrics ### Building Evaluation Metrics
metric_list = [SymbolMetric(prefix="hwf"), ReasoningMetric(kb=kb, prefix="hwf")]
metric_list = [SymbolAccuracy(prefix="hwf"), ReasoningMetric(kb=kb, prefix="hwf")]


### Bridge Learning and Reasoning ### Bridge Learning and Reasoning
bridge = SimpleBridge(model, reasoner, metric_list) bridge = SimpleBridge(model, reasoner, metric_list)


+ 2
- 2
examples/mnist_add/main.py View File

@@ -9,7 +9,7 @@ from examples.mnist_add.datasets import get_dataset
from examples.models.nn import LeNet5 from examples.models.nn import LeNet5
from abl.learning import ABLModel, BasicNN from abl.learning import ABLModel, BasicNN
from abl.reasoning import KBBase, GroundKB, PrologKB, Reasoner from abl.reasoning import KBBase, GroundKB, PrologKB, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.utils import ABLLogger, print_log from abl.utils import ABLLogger, print_log
from abl.bridge import SimpleBridge from abl.bridge import SimpleBridge


@@ -117,7 +117,7 @@ def main():
) )


### Building Evaluation Metrics ### Building Evaluation Metrics
metric_list = [SymbolMetric(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]
metric_list = [SymbolAccuracy(prefix="mnist_add"), ReasoningMetric(kb=kb, prefix="mnist_add")]


### Bridge Learning and Reasoning ### Bridge Learning and Reasoning
bridge = SimpleBridge(model, reasoner, metric_list) bridge = SimpleBridge(model, reasoner, metric_list)


+ 5
- 12
examples/mnist_add/mnist_add.ipynb View File

@@ -26,7 +26,7 @@
"from examples.models.nn import LeNet5\n", "from examples.models.nn import LeNet5\n",
"from abl.learning import ABLModel, BasicNN\n", "from abl.learning import ABLModel, BasicNN\n",
"from abl.reasoning import KBBase, Reasoner\n", "from abl.reasoning import KBBase, Reasoner\n",
"from abl.data.evaluation import ReasoningMetric, SymbolMetric\n",
"from abl.data.evaluation import ReasoningMetric, SymbolAccuracy\n",
"from abl.utils import ABLLogger, print_log\n", "from abl.utils import ABLLogger, print_log\n",
"from abl.bridge import SimpleBridge" "from abl.bridge import SimpleBridge"
] ]
@@ -122,7 +122,7 @@
}, },
{ {
"data": { "data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAD1CAYAAADNj/Z6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAKUklEQVR4nO3df6zVdR3H8XPuDy4iCHcRUnKRQIFkOFHzV+Z0maVhM5dZyzY1JWCUbmk/tkwrXWXRQB0xayVUarOcthhmFJpL5GflYgTiryyV3wgIXrjnnv6x/kh4fy+de++58H48/n2d+z3ff+7hyXe7n1OuVqvVEgCQVkO9bwAAqC8xAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAguaauvvADDZf15H0AXfC7zgfqfQsHzWcH1F/RZ4cnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBIrqneNwBAbg2DBoV7+Z1H99KdHNiLlw4L9/ahneHetCv+v/foezeGe2Xt+nCvlScDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLOGehGjUMGh/tzd48M99XvnRfu7dWOcD971afDvVyuxvtDbwv3hko412zob9aFe2Xzlp69AaAunr9hYrj//do5vXQn9XP/J1rD/Sfjju3R9/dkAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEjOOQPdqLJjV7hfevxfw72zFJ8D0FxuDPelp9wb7g2lcvz+J8fv39OmTD033Ddc/PZwr2za1I13A3RV07Ft4d7x4kvh3m9HfP3jfj4t3C86b0V8gW6w8A+nhnv/zfHn67CV7eHe/Hj870OpFJ8zUytPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJOfQoe7UWQnnXzx2Vrh//eN/7s67OeTc3fZYuH/oxCnh3vR7hw5BPTTP3xvu65aeGe6jv/hkTe+/tqaf7prRpSU9ev36HvnmyQAApCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnHMGetG4b8R/DTtx54yarj/0tA3hfuaw58O9s1oO9wdXnhLuyy+cFe6DG/qHO9A3bfh8fEbK0uNmh/v4Z6d15+3QAzwZAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJJzzkAvqmzbFu7Hfq1nvy/76cJXxN+o/e4JO8L9tQ/GPz+4ID2/s2VCuPdbsibcO+PLAwfQ2Noa7rd8bn64b620h/vYufFO/XkyAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACTnnAG6bM+sN8J9ZNMRNV1/0ZffF+4tu5fXdH1g/9bMHBPulxy5ONzHzrsx3N+1rGfPUKF2ngwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyTlngP+qnHdyuP903B0FV4jPGViwe3C4H7lmY7h3FLw7sH8v33hWuD9+/u0FVxgYrsfNXBfulYKrU3+eDABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJOWcgkYb+/cP9nnnxOQJDG+NzBHZX94b7HdMuD/fm51eGO7B/uy47PdxXXD873FvK8TkCReauejjcZ286J9yX3faecB/w4NKDvicOjicDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLOGUhkz6+Hh3vROQJFzllxdbgPX+QcAfh/NI04Jtx/9r2ZBVdoCdevbpwY7le1LomvXo7ffeY7VoX7rG9uDfdH/zgm3Cubt8Q3QCFPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDknDNwGNk446xwXzHhrnDvLLj+svb4j4lHTI3/Vrij4PrAAXTEvz1T1n8y3F9e1BbuI771ZLgvL50d7o1HDwv3zzzxVLhf3/pCuP/gugvDfdRN8TkIFPNkAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEjOOQOHkPKkCeG+4ivxOQKN5YL2q8YnDVz74xnh3vZq/LfKcCja+PD4cB82cFfhNRo/9nq4V7ZtC/eOVzeEe8P74/cfUXopfkGNqrv3hPuWysCCK+wI136vxWecUDtPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDknDPQl5xxYjjf98DccO8stcTXLzhH4PhfTg/3cd//S8H7w+Hn6EE7w/2R8QsKr3HC9Ph3q+22vn1GR7m5X7ivvTU+A2XK4CfC/Ve7jgr3th+tDvdKuNIVngwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyTlnoBc1DBgQ7ttvjr/zfGBDwTkCBS5/7oJwH3vDqnDv3Le3pveHQ9G61SPiF4wvvsa3r7wn3L/UdGW4D3kmPsVj4D/eCPddI/uHe0dLOdxPnx5/NjxyTHwGyu7O+LPj1juuCPdh2/v2OQyHA08GACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOScM9CLXrnmpHBffuKdNV3/6b3xt3q/flF7uFedIwBvMX7O1nBfffGewmt85MiC/bNzDuaW3mJzJT6jZGhjwQ3UaFn7vnC/dvb14T78LucI1JsnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByzhnoRo0njA33R2/8bsEV4u8c31eNzxGYfvN14T5k55KC9wf+V2XNM+E+efGMwmusv+CH4d5Yru3/ZT19jsD9O1vDff4l54f78DXOEejrPBkAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAknPOwEFoGDAg3J+9OT4noLUh3ouc+tTV4d423zkC0NvGXrWy8DWTHvpUuD992n3ddTv7dcUL54b7qgUnhPuoBzaEe2VdfBYDfZ8nAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAknPo0EF45ZqTwv1vZ99Z0/X/9EZzuI+aVnDwR03vDvSUEVe+HO7nT4oPFHvhw/3CvW1R/Nvf77cr4p+vPhnuPlsOf54MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMk5Z+A/TptY+JKFN9xe8IojwvWfHXvC/ZYZU8O9ZdPygvcH+qLK9tfCvXHxqnAfs7g77wbeypMBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDnnDLxp31Hx94WXSqXS0Mb4HIEiN/1rcri3LHSOAAC9z5MBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDnnDHSjL7xyRrhv/WjxWQYA0Ns8GQCA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSc87Am5oXrSx8zeRjTil4xb6CfWOX7wcAeosnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLlarVarfdNAAD148kAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJPdvx8SA9zSwVzUAAAAASUVORK5CYII=",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAD1CAYAAADNj/Z6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAKUklEQVR4nO3df6zVdR3H8XPuDy4iCHcRUnKRQIFkOFHzV+Z0maVhM5dZyzY1JWCUbmk/tkwrXWXRQB0xayVUarOcthhmFJpL5GflYgTiryyV3wgIXrjnnv6x/kh4fy+de++58H48/n2d+z3ff+7hyXe7n1OuVqvVEgCQVkO9bwAAqC8xAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAguaauvvADDZf15H0AXfC7zgfqfQsHzWcH1F/RZ4cnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBIrqneNwBAbg2DBoV7+Z1H99KdHNiLlw4L9/ahneHetCv+v/foezeGe2Xt+nCvlScDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLOGehGjUMGh/tzd48M99XvnRfu7dWOcD971afDvVyuxvtDbwv3hko412zob9aFe2Xzlp69AaAunr9hYrj//do5vXQn9XP/J1rD/Sfjju3R9/dkAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEjOOQPdqLJjV7hfevxfw72zFJ8D0FxuDPelp9wb7g2lcvz+J8fv39OmTD033Ddc/PZwr2za1I13A3RV07Ft4d7x4kvh3m9HfP3jfj4t3C86b0V8gW6w8A+nhnv/zfHn67CV7eHe/Hj870OpFJ8zUytPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJOfQoe7UWQnnXzx2Vrh//eN/7s67OeTc3fZYuH/oxCnh3vR7hw5BPTTP3xvu65aeGe6jv/hkTe+/tqaf7prRpSU9ev36HvnmyQAApCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnHMGetG4b8R/DTtx54yarj/0tA3hfuaw58O9s1oO9wdXnhLuyy+cFe6DG/qHO9A3bfh8fEbK0uNmh/v4Z6d15+3QAzwZAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJJzzkAvqmzbFu7Hfq1nvy/76cJXxN+o/e4JO8L9tQ/GPz+4ID2/s2VCuPdbsibcO+PLAwfQ2Noa7rd8bn64b620h/vYufFO/XkyAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACTnnAG6bM+sN8J9ZNMRNV1/0ZffF+4tu5fXdH1g/9bMHBPulxy5ONzHzrsx3N+1rGfPUKF2ngwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyTlngP+qnHdyuP903B0FV4jPGViwe3C4H7lmY7h3FLw7sH8v33hWuD9+/u0FVxgYrsfNXBfulYKrU3+eDABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJOWcgkYb+/cP9nnnxOQJDG+NzBHZX94b7HdMuD/fm51eGO7B/uy47PdxXXD873FvK8TkCReauejjcZ286J9yX3faecB/w4NKDvicOjicDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLOGUhkz6+Hh3vROQJFzllxdbgPX+QcAfh/NI04Jtx/9r2ZBVdoCdevbpwY7le1LomvXo7ffeY7VoX7rG9uDfdH/zgm3Cubt8Q3QCFPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDknDNwGNk446xwXzHhrnDvLLj+svb4j4lHTI3/Vrij4PrAAXTEvz1T1n8y3F9e1BbuI771ZLgvL50d7o1HDwv3zzzxVLhf3/pCuP/gugvDfdRN8TkIFPNkAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEjOOQOHkPKkCeG+4ivxOQKN5YL2q8YnDVz74xnh3vZq/LfKcCja+PD4cB82cFfhNRo/9nq4V7ZtC/eOVzeEe8P74/cfUXopfkGNqrv3hPuWysCCK+wI136vxWecUDtPBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDknDPQl5xxYjjf98DccO8stcTXLzhH4PhfTg/3cd//S8H7w+Hn6EE7w/2R8QsKr3HC9Ph3q+22vn1GR7m5X7ivvTU+A2XK4CfC/Ve7jgr3th+tDvdKuNIVngwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyTlnoBc1DBgQ7ttvjr/zfGBDwTkCBS5/7oJwH3vDqnDv3Le3pveHQ9G61SPiF4wvvsa3r7wn3L/UdGW4D3kmPsVj4D/eCPddI/uHe0dLOdxPnx5/NjxyTHwGyu7O+LPj1juuCPdh2/v2OQyHA08GACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOScM9CLXrnmpHBffuKdNV3/6b3xt3q/flF7uFedIwBvMX7O1nBfffGewmt85MiC/bNzDuaW3mJzJT6jZGhjwQ3UaFn7vnC/dvb14T78LucI1JsnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByzhnoRo0njA33R2/8bsEV4u8c31eNzxGYfvN14T5k55KC9wf+V2XNM+E+efGMwmusv+CH4d5Yru3/ZT19jsD9O1vDff4l54f78DXOEejrPBkAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAknPOwEFoGDAg3J+9OT4noLUh3ouc+tTV4d423zkC0NvGXrWy8DWTHvpUuD992n3ddTv7dcUL54b7qgUnhPuoBzaEe2VdfBYDfZ8nAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAknPo0EF45ZqTwv1vZ99Z0/X/9EZzuI+aVnDwR03vDvSUEVe+HO7nT4oPFHvhw/3CvW1R/Nvf77cr4p+vPhnuPlsOf54MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMk5Z+A/TptY+JKFN9xe8IojwvWfHXvC/ZYZU8O9ZdPygvcH+qLK9tfCvXHxqnAfs7g77wbeypMBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDnnDLxp31Hx94WXSqXS0Mb4HIEiN/1rcri3LHSOAAC9z5MBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDnnDHSjL7xyRrhv/WjxWQYA0Ns8GQCA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSc87Am5oXrSx8zeRjTil4xb6CfWOX7wcAeosnAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHLlarVarfdNAAD148kAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJPdvx8SA9zSwVzUAAAAASUVORK5CYII=",
"text/plain": [ "text/plain": [
"<Figure size 640x480 with 2 Axes>" "<Figure size 640x480 with 2 Axes>"
] ]
@@ -380,7 +380,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolMetric` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolAccuracy` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
] ]
}, },
{ {
@@ -389,7 +389,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"metric_list = [SymbolMetric(prefix=\"mnist_add\"), ReasoningMetric(kb=kb, prefix=\"mnist_add\")]"
"metric_list = [SymbolAccuracy(prefix=\"mnist_add\"), ReasoningMetric(kb=kb, prefix=\"mnist_add\")]"
] ]
}, },
{ {
@@ -420,7 +420,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 14,
"execution_count": null,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
@@ -432,13 +432,6 @@
"bridge.train(train_data, loops=5, segment_size=1/3, save_interval=1, save_dir=weights_dir)\n", "bridge.train(train_data, loops=5, segment_size=1/3, save_interval=1, save_dir=weights_dir)\n",
"bridge.test(test_data)" "bridge.test(test_data)"
] ]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
} }
], ],
"metadata": { "metadata": {


+ 2
- 2
examples/zoo/main.py View File

@@ -8,7 +8,7 @@ import openml


from abl.learning import ABLModel from abl.learning import ABLModel
from abl.reasoning import KBBase, Reasoner from abl.reasoning import KBBase, Reasoner
from abl.data.evaluation import ReasoningMetric, SymbolMetric
from abl.data.evaluation import ReasoningMetric, SymbolAccuracy
from abl.bridge import SimpleBridge from abl.bridge import SimpleBridge
from abl.utils.utils import confidence_dist from abl.utils.utils import confidence_dist
from abl.utils import ABLLogger, print_log from abl.utils import ABLLogger, print_log
@@ -182,7 +182,7 @@ train_data = transform_tab_data(X_unlabel, y_unlabel)


# %% # %%
# Set up metrics # Set up metrics
metric_list = [SymbolMetric(prefix="zoo"), ReasoningMetric(kb=kb, prefix="zoo")]
metric_list = [SymbolAccuracy(prefix="zoo"), ReasoningMetric(kb=kb, prefix="zoo")]


# %% [markdown] # %% [markdown]
# ### Bridge Machine Learning and Logic Reasoning # ### Bridge Machine Learning and Logic Reasoning


+ 3
- 3
examples/zoo/zoo.ipynb View File

@@ -25,7 +25,7 @@
"from abl.learning import ABLModel\n", "from abl.learning import ABLModel\n",
"from examples.zoo.kb import ZooKB\n", "from examples.zoo.kb import ZooKB\n",
"from abl.reasoning import Reasoner\n", "from abl.reasoning import Reasoner\n",
"from abl.data.evaluation import ReasoningMetric, SymbolMetric\n",
"from abl.data.evaluation import ReasoningMetric, SymbolAccuracy\n",
"from abl.utils import ABLLogger, print_log, confidence_dist\n", "from abl.utils import ABLLogger, print_log, confidence_dist\n",
"from abl.bridge import SimpleBridge" "from abl.bridge import SimpleBridge"
] ]
@@ -276,7 +276,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolMetric` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
"Next, we set up evaluation metrics. These metrics will be used to evaluate the model performance during training and testing. Specifically, we use `SymbolAccuracy` and `ReasoningMetric`, which are used to evaluate the accuracy of the machine learning model’s predictions and the accuracy of the final reasoning results, respectively."
] ]
}, },
{ {
@@ -285,7 +285,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"metric_list = [SymbolMetric(prefix=\"zoo\"), ReasoningMetric(kb=kb, prefix=\"zoo\")]"
"metric_list = [SymbolAccuracy(prefix=\"zoo\"), ReasoningMetric(kb=kb, prefix=\"zoo\")]"
] ]
}, },
{ {


Write Preview
Loading…
Cancel
Save