diff --git a/examples/Demo Notebook/TODSBlockchainNotebook.ipynb b/examples/Demo Notebook/TODSBlockchainNotebook.ipynb
new file mode 100644
index 0000000..dadfb4a
--- /dev/null
+++ b/examples/Demo Notebook/TODSBlockchainNotebook.ipynb
@@ -0,0 +1,2247 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Anomaly Detection in Blockchain System with TODS"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Introduction Summary"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### What is TODS?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "TODS, developed by DATA Lab @ Texas A&M University, is a full-stack automated machine learning system for outlier detection on multivariate time-series data. TODS provides exhaustive modules for building machine learning-based outlier detection systems, including: data processing, time series processing, feature analysis (extraction), detection algorithms, and reinforcement module."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### What is the Blockchain System?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\"A blockchain is, in the simplest of terms, a time-stamped series of immutable records of data that is managed by a cluster of computers not owned by any single entity. Each of these blocks of data (i.e. block) is secured and bound to each other using cryptographic principles (i.e. chain).\" \n",
+ "\n",
+ "More information about blockchain can be found at https://blockgeeks.com/guides/what-is-blockchain-technology/"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### How does TODS detect anomalies from Blockchain System?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "TODS system can perform three common outlier detection scenarios on time-series data: point-wise detection (time points as outliers), pattern-wise detection (subsequences as outliers), and system-wise detection (sets of time series as outliers).\n",
+ "\n",
+ "In this notebook, we use Google BigQuery Bitcoin Blockchain Dataset, which contains information about dates, transactions, blocks and prices of Bitcoin. According to Google, this dataset updates every 10 minutes in the following link:\n",
+ "https://bigquery.cloud.google.com/dataset/bigquery-public-data:bitcoin_blockchain\n",
+ "\n",
+ "With TODS, we first collect and process the data to put in a dataframe. Next, we use two primitives from TODS to fit and predict on all dimensions of the data, to get the prediction labels and scores. Last, we use the prediction information from TODS primitives to plot and visualize the outliers within the data.\n",
+ "\n",
+ "The searcher of TODS will take the dataset and automatically build a pipeline to find the best possible model with its scores. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "zV8tJwO6-U9h"
+ },
+ "source": [
+ "## Packages and Imports"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import warnings\n",
+ "warnings.filterwarnings(\"ignore\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "id": "mnF3OkUJ-U9i",
+ "outputId": "af2dc0fd-9b87-4a0a-d95c-285f7101f654"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Requirement already satisfied: google-cloud-bigquery in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (2.20.0)\n",
+ "Requirement already satisfied: google-resumable-media<2.0dev,>=0.6.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (1.3.0)\n",
+ "Requirement already satisfied: protobuf>=3.12.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (3.17.3)\n",
+ "Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (2.23.0)\n",
+ "Requirement already satisfied: proto-plus>=1.10.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (1.18.1)\n",
+ "Requirement already satisfied: google-cloud-core<2.0dev,>=1.4.1 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (1.6.0)\n",
+ "Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.29.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (1.31.0)\n",
+ "Requirement already satisfied: packaging>=14.3 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (21.0)\n",
+ "Requirement already satisfied: six in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.15.0)\n",
+ "Requirement already satisfied: google-crc32c<2.0dev,>=1.0; python_version >= \"3.5\" in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.1.2)\n",
+ "Requirement already satisfied: chardet<4,>=3.0.2 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery) (3.0.4)\n",
+ "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery) (1.25.11)\n",
+ "Requirement already satisfied: certifi>=2017.4.17 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery) (2021.5.30)\n",
+ "Requirement already satisfied: idna<3,>=2.5 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery) (2.10)\n",
+ "Requirement already satisfied: google-auth<2.0dev,>=1.24.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (1.33.1)\n",
+ "Requirement already satisfied: setuptools>=40.3.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.29.0->google-cloud-bigquery) (57.4.0)\n",
+ "Requirement already satisfied: pytz in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.29.0->google-cloud-bigquery) (2021.1)\n",
+ "Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.29.0->google-cloud-bigquery) (1.53.0)\n",
+ "Requirement already satisfied: grpcio<2.0dev,>=1.29.0; extra == \"grpc\" in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.29.0->google-cloud-bigquery) (1.36.1)\n",
+ "Requirement already satisfied: pyparsing>=2.0.2 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from packaging>=14.3->google-cloud-bigquery) (2.4.7)\n",
+ "Requirement already satisfied: cffi>=1.0.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-crc32c<2.0dev,>=1.0; python_version >= \"3.5\"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.14.0)\n",
+ "Requirement already satisfied: rsa<5,>=3.1.4; python_version >= \"3.6\" in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.24.0->google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (4.7.2)\n",
+ "Requirement already satisfied: pyasn1-modules>=0.2.1 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.24.0->google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (0.2.8)\n",
+ "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.24.0->google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (4.2.2)\n",
+ "Requirement already satisfied: pycparser in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0; python_version >= \"3.5\"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (2.20)\n",
+ "Requirement already satisfied: pyasn1>=0.1.3 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from rsa<5,>=3.1.4; python_version >= \"3.6\"->google-auth<2.0dev,>=1.24.0->google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (0.4.8)\n",
+ "Note: you may need to restart the kernel to use updated packages.\n"
+ ]
+ }
+ ],
+ "source": [
+ "pip install google-cloud-bigquery\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {
+ "id": "hI2JIaHC-U9i",
+ "outputId": "4424ee78-f84f-4e1c-9cdf-c0086ce3dca5",
+ "scrolled": true
+ },
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "d3m.primitives.tods.detection_algorithm.LSTMODetector: Primitive is not providing a description through its docstring.\n"
+ ]
+ }
+ ],
+ "source": [
+ "from google.cloud import bigquery\n",
+ "from scipy.stats.mstats import zscore\n",
+ "from sklearn.metrics import precision_recall_curve\n",
+ "from sklearn.metrics import accuracy_score\n",
+ "from sklearn.metrics import confusion_matrix\n",
+ "from sklearn.metrics import classification_report\n",
+ "from tods.sk_interface.detection_algorithm.Telemanom_skinterface import TelemanomSKI\n",
+ "from tods.sk_interface.detection_algorithm.DeepLog_skinterface import DeepLogSKI\n",
+ "from sklearn.preprocessing import MinMaxScaler, QuantileTransformer\n",
+ "from sklearn.cluster import KMeans\n",
+ "from sklearn.manifold import TSNE\n",
+ "from mpl_toolkits.mplot3d import Axes3D\n",
+ "import matplotlib.pyplot as plt\n",
+ "plt.style.use('fivethirtyeight')\n",
+ "import matplotlib as mpl\n",
+ "from pathlib import Path\n",
+ "from sklearn.decomposition import PCA\n",
+ "from sklearn.preprocessing import StandardScaler\n",
+ "import seaborn as sns\n",
+ "import datetime as dt\n",
+ "from datetime import datetime,tzinfo\n",
+ "import scipy, json, csv, time, pytz\n",
+ "from pytz import timezone\n",
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "seed = 135\n",
+ "%config InlineBackend.figure_format = 'retina'\n",
+ "%matplotlib inline\n",
+ "import os\n",
+ "# os.listdir('./')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "oFJ3d5MW-U9j"
+ },
+ "source": [
+ "## Processing Data"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "id": "1Laxlepi-U9j"
+ },
+ "outputs": [],
+ "source": [
+ "#Connecting to Google datastore (use path to ur private key)\n",
+ "os.environ['GOOGLE_APPLICATION_CREDENTIALS']=\"blockchain-316820-a21123305db7.json\"\n",
+ "client = bigquery.Client()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {
+ "id": "ZkCH3LR5-U9k"
+ },
+ "outputs": [],
+ "source": [
+ "# The query to get date, number of transactions from Google BigQuery bitcoin blockchain dataset \n",
+ "# Select records from the last three years and group them with respect to date\n",
+ "query_1 = \"\"\"\n",
+ "SELECT \n",
+ " DATE(TIMESTAMP_MILLIS(timestamp)) AS Date,\n",
+ " COUNT(transactions) AS Transactions\n",
+ "FROM `bigquery-public-data.bitcoin_blockchain.blocks`\n",
+ "GROUP BY date\n",
+ "HAVING date >= '2009-01-09' AND date <= '2020-06-12'\n",
+ "ORDER BY date\n",
+ "\"\"\"\n",
+ "query_job_1 = client.query(query_1)\n",
+ "# Waits for the query to finish\n",
+ "iterator_1 = query_job_1.result(timeout=30)\n",
+ "rows_1 = list(iterator_1)\n",
+ "df_1 = pd.DataFrame(data=[list(x.values()) for x in rows_1], columns=list(rows_1[0].keys()))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {
+ "id": "NXpxzJ5R-U9k"
+ },
+ "outputs": [],
+ "source": [
+ "# The query to get sum of all satoshis spent each day and number of blocks\n",
+ "query_2 = \"\"\"\n",
+ "SELECT\n",
+ " o.Date,\n",
+ " COUNT(o.block) AS Blocks,\n",
+ " SUM(o.output_price) AS Output_Satoshis\n",
+ "FROM (\n",
+ " SELECT\n",
+ " DATE(TIMESTAMP_MILLIS(timestamp)) AS Date,\n",
+ " output.output_satoshis AS output_price,\n",
+ " block_id AS block\n",
+ " FROM\n",
+ " `bigquery-public-data.bitcoin_blockchain.transactions`,\n",
+ " UNNEST(outputs) AS output ) AS o\n",
+ "GROUP BY\n",
+ " o.date\n",
+ "HAVING o.date >= '2009-01-09' AND o.date <= '2020-06-12'\n",
+ "ORDER BY o.date, blocks\n",
+ "\"\"\"\n",
+ "query_job_2 = client.query(query_2)\n",
+ "# Waits for the query to finish\n",
+ "iterator_2 = query_job_2.result(timeout=30)\n",
+ "rows_2 = list(iterator_2)\n",
+ "df_2 = pd.DataFrame(data=[list(x.values()) for x in rows_2], columns=list(rows_2[0].keys()))\n",
+ "\n",
+ "df_2[\"Output_Satoshis\"]= df_2[\"Output_Satoshis\"].apply(lambda x: float(x/100000000))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {
+ "id": "TQzZev9C-U9k",
+ "outputId": "c643c8e9-b222-4ce9-f306-ae3a8812988d"
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " Date | \n",
+ " Transactions | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 2009-01-09 | \n",
+ " 14 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 2009-01-10 | \n",
+ " 61 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 2009-01-11 | \n",
+ " 93 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 2009-01-12 | \n",
+ " 94 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 2009-01-13 | \n",
+ " 123 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " Date | \n",
+ " Blocks | \n",
+ " Output_Satoshis | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 2009-01-09 | \n",
+ " 14 | \n",
+ " 700.0 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 2009-01-10 | \n",
+ " 61 | \n",
+ " 3050.0 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 2009-01-11 | \n",
+ " 93 | \n",
+ " 4650.0 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 2009-01-12 | \n",
+ " 106 | \n",
+ " 4879.0 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 2009-01-13 | \n",
+ " 123 | \n",
+ " 6150.0 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " Date | \n",
+ " Transactions | \n",
+ " Blocks | \n",
+ " Output_Satoshis | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 2009-01-09 | \n",
+ " 14 | \n",
+ " 14 | \n",
+ " 700.0 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 2009-01-10 | \n",
+ " 61 | \n",
+ " 61 | \n",
+ " 3050.0 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 2009-01-11 | \n",
+ " 93 | \n",
+ " 93 | \n",
+ " 4650.0 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 2009-01-12 | \n",
+ " 94 | \n",
+ " 106 | \n",
+ " 4879.0 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 2009-01-13 | \n",
+ " 123 | \n",
+ " 123 | \n",
+ " 6150.0 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " Transactions | \n",
+ " Blocks | \n",
+ " Output_Satoshis | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | count | \n",
+ " 3532.000000 | \n",
+ " 3.532000e+03 | \n",
+ " 3.532000e+03 | \n",
+ "
\n",
+ " \n",
+ " | mean | \n",
+ " 153.053228 | \n",
+ " 2.623631e+05 | \n",
+ " 1.358730e+06 | \n",
+ "
\n",
+ " \n",
+ " | std | \n",
+ " 37.978411 | \n",
+ " 2.799358e+05 | \n",
+ " 2.193636e+06 | \n",
+ "
\n",
+ " \n",
+ " | min | \n",
+ " 2.000000 | \n",
+ " 4.000000e+00 | \n",
+ " 2.000000e+02 | \n",
+ "
\n",
+ " \n",
+ " | 25% | \n",
+ " 138.000000 | \n",
+ " 1.161975e+04 | \n",
+ " 3.047457e+05 | \n",
+ "
\n",
+ " \n",
+ " | 50% | \n",
+ " 153.000000 | \n",
+ " 1.586145e+05 | \n",
+ " 9.436964e+05 | \n",
+ "
\n",
+ " \n",
+ " | 75% | \n",
+ " 169.000000 | \n",
+ " 5.145980e+05 | \n",
+ " 1.835461e+06 | \n",
+ "
\n",
+ " \n",
+ " | max | \n",
+ " 692.000000 | \n",
+ " 2.035035e+06 | \n",
+ " 6.735430e+07 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "