{ "cells": [ { "cell_type": "markdown", "id": "6c6ded2d-2213-4594-8f6f-91fff22d7fba", "metadata": { "tags": [] }, "source": [ "# Bland-Altman analysis to compare segmentation algorithms\n", "Assume we used a segmentation algorithm for many years and we are now considering to replace it by a newer, faster version. We need to make sure that we can compare results between these two. As segmentation algorithms typically do not label objects in the same order and even the number of objects might differ, we cannot easily compare objects pair-wise. It is recommended to summarize segmented objects per image and then compare results produced on folders of images.\n", "\n", "In this notebook we will compare statistics derived from segmentation results produced by two algorithms on a folder of images. " ] }, { "cell_type": "code", "execution_count": 1, "id": "dbe79181-9f68-48ef-9e15-5813075adac4", "metadata": { "tags": [] }, "outputs": [], "source": [ "folder = '../../data/BBBC007_batch/' " ] }, { "cell_type": "code", "execution_count": 2, "id": "dcfd3ec8-5aa2-4658-b11b-258807f9c70f", "metadata": { "tags": [] }, "outputs": [], "source": [ "from skimage.io import imread\n", "from skimage.measure import regionprops\n", "from utils import bland_altman_plot\n", "import napari_segment_blobs_and_things_with_membranes as nsbatwm\n", "import pyclesperanto_prototype as cle\n", "import os\n", "import numpy as np\n", "import pandas as pd\n", "import stackview" ] }, { "cell_type": "markdown", "id": "c30b1464-6e2c-465d-ba90-76856da27fcc", "metadata": {}, "source": [ "# Segmentation algorithms to compare\n", "Here we write the two segmentation algorithms as Python functions and test them on an image." ] }, { "cell_type": "code", "execution_count": 3, "id": "7b147983-ea3c-4906-80ee-4394a28ec11f", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
shape(340, 340)
dtypeuint16
size225.8 kB
min1
max255
\n", "\n", "
" ], "text/plain": [ "StackViewNDArray([[3, 3, 3, ..., 2, 3, 3],\n", " [5, 4, 4, ..., 3, 3, 2],\n", " [6, 5, 4, ..., 2, 3, 2],\n", " ...,\n", " [2, 1, 1, ..., 1, 1, 1],\n", " [1, 2, 2, ..., 2, 1, 1],\n", " [2, 2, 1, ..., 1, 1, 1]], dtype=uint16)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "test_image = imread(folder + \"17P1_POS0013_D_1UL.tif\")\n", "stackview.insight(test_image)" ] }, { "cell_type": "code", "execution_count": 4, "id": "f284cfe9-fce1-4b0b-978d-177358b1ef8a", "metadata": { "tags": [] }, "outputs": [], "source": [ "def segmentation_1(image):\n", " return nsbatwm.voronoi_otsu_labeling(image)" ] }, { "cell_type": "code", "execution_count": 5, "id": "638815da-a1e4-4546-ace4-83de276e9ae8", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "\n", "nsbatwm made image
\n", "\n", "\n", "\n", "\n", "\n", "
shape(340, 340)
dtypeint32
size451.6 kB
min0
max46
\n", "\n", "
" ], "text/plain": [ "StackViewNDArray([[0, 0, 0, ..., 0, 0, 0],\n", " [0, 0, 0, ..., 0, 0, 0],\n", " [0, 0, 0, ..., 0, 0, 0],\n", " ...,\n", " [0, 0, 0, ..., 0, 0, 0],\n", " [0, 0, 0, ..., 0, 0, 0],\n", " [0, 0, 0, ..., 0, 0, 0]])" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "segmentation_1(test_image)" ] }, { "cell_type": "code", "execution_count": 6, "id": "6650ec55-46ab-442f-a7ca-3a4294f93f47", "metadata": { "tags": [] }, "outputs": [], "source": [ "def segmentation_2(image):\n", " return nsbatwm.gauss_otsu_labeling(image)" ] }, { "cell_type": "code", "execution_count": 7, "id": "dd2a44fa-8888-482b-aabd-4decdbc51e9d", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", "\n", "nsbatwm made image
\n", "\n", "\n", "\n", "\n", "\n", "
shape(340, 340)
dtypeint32
size451.6 kB
min0
max41
\n", "\n", "
" ], "text/plain": [ "StackViewNDArray([[0, 0, 0, ..., 0, 0, 0],\n", " [0, 0, 0, ..., 0, 0, 0],\n", " [0, 0, 0, ..., 0, 0, 0],\n", " ...,\n", " [0, 0, 0, ..., 0, 0, 0],\n", " [0, 0, 0, ..., 0, 0, 0],\n", " [0, 0, 0, ..., 0, 0, 0]])" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "test_labels = segmentation_2(test_image)\n", "test_labels" ] }, { "cell_type": "markdown", "id": "93ce89da-fabb-4951-b69a-6a517bae935e", "metadata": {}, "source": [ "## Quantiative measurements\n", "Later, we want to compare measurements. Thus, we write a Python function that determines these measurements. In this example, we will compute the mean area of segmented nuclei." ] }, { "cell_type": "code", "execution_count": 8, "id": "266c5a5d-e498-44e5-80bf-531a50a6afe5", "metadata": { "tags": [] }, "outputs": [], "source": [ "def mean_metric(image, label_image, metric):\n", " \n", " properties = regionprops(label_image, image)\n", " \n", " values = [p[metric] for p in properties]\n", " \n", " return np.mean(values)" ] }, { "cell_type": "code", "execution_count": 9, "id": "53a01e25-a416-41e1-b511-5c33ca40988c", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "235.70731707317074" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mean_metric(test_image, test_labels, \"area\")" ] }, { "cell_type": "markdown", "id": "63dfe5c9-5145-44fe-848d-d62a0954e77d", "metadata": {}, "source": [ "## Collecting measurements from folders\n", "We now apply these two algorithms and the measurements in a folder of images." ] }, { "cell_type": "code", "execution_count": 10, "id": "81e92138-56d9-4827-b0c2-5347200ba387", "metadata": { "tags": [] }, "outputs": [], "source": [ "def compare_measurements_from_algorithms(algorithm_1, algorithm_2, folder, metric):\n", " measurements = {\n", " metric + '_1':[],\n", " metric + '_2':[]\n", " }\n", "\n", " # Iterate over all files in the folder\n", " for filename in os.listdir(folder):\n", " file_path = os.path.join(folder, filename)\n", "\n", " # Check if the current item is a file\n", " if os.path.isfile(file_path) and filename.endswith(\".tif\"):\n", " # load image\n", " image = imread(file_path)\n", "\n", " # segment it using both algorithms\n", " labels_1 = algorithm_1(image)\n", " labels_2 = algorithm_2(image)\n", "\n", " # determine mean area and store it\n", " measurements[metric + '_1'].append(mean_metric(image, labels_1, metric))\n", " measurements[metric + '_2'].append(mean_metric(image, labels_2, metric))\n", " \n", " return measurements" ] }, { "cell_type": "code", "execution_count": 11, "id": "d212a937-39fe-4fdb-af7a-cf7069da8214", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
area_1area_2
0210.086957235.707317
1206.866667244.973684
2203.023256268.615385
3185.103448214.720000
4184.147059362.956522
5267.057692730.894737
\n", "
" ], "text/plain": [ " area_1 area_2\n", "0 210.086957 235.707317\n", "1 206.866667 244.973684\n", "2 203.023256 268.615385\n", "3 185.103448 214.720000\n", "4 184.147059 362.956522\n", "5 267.057692 730.894737" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "measurements = compare_measurements_from_algorithms(segmentation_1, \n", " segmentation_2, \n", " folder, \n", " 'area')\n", "\n", "pd.DataFrame(measurements)" ] }, { "cell_type": "markdown", "id": "ba5fe7c9-2e12-416f-a994-52d07079dc2d", "metadata": {}, "source": [ "## Bland-Altman plots\n", "We now use the Bland-Altman plot to visualize differences." ] }, { "cell_type": "code", "execution_count": 12, "id": "2bacd00c-6ff4-42cc-99a1-f09bb263f663", "metadata": { "tags": [] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkcAAAGzCAYAAAAlqLNlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA+tUlEQVR4nO3de3QU9f3/8dfmtgkhWQghl8UAqQJtiGIhxQZtI0i4iAheUSglivwERMGA2tAqagvxArZIFW/fAl6RI1KvKIgaRAEhgBKhCBhIIIlBwE24JSGZ3x+cjLMmC1nMZROej3PmHGbmvbufGSezLz/zmVmbYRiGAAAAIEnya+oGAAAA+BLCEQAAgAXhCAAAwIJwBAAAYEE4AgAAsCAcAQAAWBCOAAAALAhHAAAAFoQjAAAAC8IRAACARUBTN6CuMjMz9eabb+p///ufQkJC1KdPHz366KPq1q2bWWMYhh566CE999xzOnz4sC655BI99dRT6t69u1lTVlamadOm6bXXXtPx48d1xRVX6Omnn9Z5551X57ZUVVWpoKBAYWFhstls9bqdAACgYRiGodLSUjmdTvn5naZ/yGgmBg4caCxYsMDIyckxtmzZYgwZMsTo2LGjceTIEbPmkUceMcLCwoylS5caW7duNUaMGGHExsYaJSUlZs348eONDh06GCtXrjQ2bdpk9O3b1+jRo4dx8uTJOrclPz/fkMTExMTExMTUDKf8/PzTfs/bDKN5/vDsgQMHFBUVpaysLP3xj3+UYRhyOp2aMmWK7rvvPkmneomio6P16KOP6vbbb5fL5VL79u310ksvacSIEZKkgoICxcXF6f3339fAgQPr9Nkul0tt2rRRfn6+wsPDG2wbAQBA/SkpKVFcXJx+/PFHORwOj3XN5rLaz7lcLklSRESEJCk3N1dFRUUaMGCAWWO325WSkqIvvvhCt99+u7Kzs1VRUeFW43Q6lZiYqC+++MJjOCorK1NZWZk5X1paKkkKDw8nHAEA0MycaUhMsxyQbRiG0tPTddlllykxMVGSVFRUJEmKjo52q42OjjbXFRUVKSgoSG3btvVYU5vMzEw5HA5ziouLq8/NAQAAPqRZhqNJkybp66+/1muvvVZj3c/ToGEYZ0yIZ6rJyMiQy+Uyp/z8/LNrOAAA8HnNLhzdeeedevvtt/XJJ5+43WEWExMjSTV6gIqLi83epJiYGJWXl+vw4cMea2pjt9vNS2hcSgMAoGVrNuHIMAxNmjRJb775pj7++GPFx8e7rY+Pj1dMTIxWrlxpLisvL1dWVpb69OkjSerVq5cCAwPdagoLC5WTk2PWAACAc1uzGZB9xx136NVXX9Vbb72lsLAws4fI4XAoJCRENptNU6ZM0axZs9SlSxd16dJFs2bNUqtWrTRy5EizduzYsZo6daratWuniIgITZs2TRdeeKH69+/flJsHAAB8RLMJR/Pnz5ckXX755W7LFyxYoLS0NEnSvffeq+PHj2vixInmQyBXrFihsLAws/6f//ynAgICdOONN5oPgVy4cKH8/f0ba1MAAIAPa7bPOWpKJSUlcjgccrlcjD8CAKCZqOv3d7MZcwQAANAYCEcAAAAWhCMAAAALwhEAAIAF4QgAAMCi2dzK74vKy8tVXl5eY7mfn58CAgLc6jyx2WwKDAw8q9qKigp5utmwoWolKSgo6KxqT548qaqqqnqpDQwMNH/ypaFqKysrVVlZWS+1AQEB8vPz85naqqoqnTx50mOtv7+/+XgLX6g1DEMVFRX1Umv9+2yoWun0f8ucI2qv5RzBOaIxzhF1QTj6BebMmaPg4OAay7t06WI+eFKSZs+e7fE/WqdOncznNEnS3LlzdezYsVprnU6nxo0bZ84/9dRTcrlctda2b99eEydONOeff/55HThwoNZah8OhKVOmmPMLFy5UQUFBrbWtWrXSPffcY86/8sor2rt3b621gYGBmj59ujm/ZMkS7dy5s9ZaSZoxY4b572XLlmnbtm0eazMyMswT5bvvvquvvvrKY+20adMUGhoqSfrwww+1ceNGj7WTJ09WmzZtJEmrVq3S2rVrPdZOmDBBUVFRkqTPPvtMWVlZHmtvu+02dejQQZK0bt06ffTRRx5rx4wZo86dO0uSsrOztXz5co+1N998s7p27SpJ2rp1q9566y2Ptddff726d+8uSdq+fbveeOMNj7XDhg3TxRdfLEnatWtXrb9jWG3w4MHq3bu3JCkvL0+LFi3yWNu/f39deumlkk49nf6FF17wWJuSkmI+1+zAgQPms85qk5ycrAEDBkiSXC6X5s6d67E2KSlJQ4YMkSQdO3ZMs2fP9ljbo0cPDR8+XNKpL/nMzEyPtQkJCbrhhhvM+dPVco44hXPETzhHnNIY54i64LIaAACABQ+BPAvVD5E6cOBArQ+Rosu89lq6zOky57Ka97WcI86ulnPEL6v1hb/7hjhH1PUhkISjs8ATsgEAaH54QjYAAMBZIBwBAABYEI4AAAAsCEcAAAAWhCMAAAALwhEAAIAF4QgAAMCCcAQAAGBBOAIAALAgHAEAAFgQjgAAACwIRwAAABaEIwAAAAvCEQAAgAXhCAAAwIJwBAAAYEE4AgAAsCAcAQAAWBCOAAAALAhHAAAAFoQjAAAAC8IRAACABeEIAADAgnAEAABgQTgCAACwIBwBAABYEI4AAAAsCEcAAAAWhCMAAAALwhEAAIAF4QgAAMCCcAQAAGDRrMLR6tWrNXToUDmdTtlsNv33v/91W28Yhh588EE5nU6FhITo8ssv1zfffONWU1ZWpjvvvFORkZEKDQ3V1VdfrX379jXiVgAAAF/WrMLR0aNH1aNHD/373/+udf1jjz2mJ554Qv/+97+1YcMGxcTEKDU1VaWlpWbNlClTtGzZMi1evFhr1qzRkSNHdNVVV6mysrKxNgMAAPgwm2EYRlM34mzYbDYtW7ZMw4cPl3Sq18jpdGrKlCm67777JJ3qJYqOjtajjz6q22+/XS6XS+3bt9dLL72kESNGSJIKCgoUFxen999/XwMHDqzTZ5eUlMjhcMjlcik8PLxBtg8AANSvun5/N6ueo9PJzc1VUVGRBgwYYC6z2+1KSUnRF198IUnKzs5WRUWFW43T6VRiYqJZU5uysjKVlJS4TQAAoGVqMeGoqKhIkhQdHe22PDo62lxXVFSkoKAgtW3b1mNNbTIzM+VwOMwpLi6unlsPAAB8RYsJR9VsNpvbvGEYNZb93JlqMjIy5HK5zCk/P79e2goAAHxPiwlHMTExklSjB6i4uNjsTYqJiVF5ebkOHz7ssaY2drtd4eHhbhMAAGiZWkw4io+PV0xMjFauXGkuKy8vV1ZWlvr06SNJ6tWrlwIDA91qCgsLlZOTY9YAAIBzW0BTN8AbR44c0a5du8z53NxcbdmyRREREerYsaOmTJmiWbNmqUuXLurSpYtmzZqlVq1aaeTIkZIkh8OhsWPHaurUqWrXrp0iIiI0bdo0XXjhherfv39TbRYAAPAhzSocbdy4UX379jXn09PTJUljxozRwoULde+99+r48eOaOHGiDh8+rEsuuUQrVqxQWFiY+Zp//vOfCggI0I033qjjx4/riiuu0MKFC+Xv79/o2wMAAHxPs33OUVPiOUcAADQ/59xzjgAAAOoD4QgAAMCCcAQAAGBBOAIAALAgHAEAAFgQjgAAACwIRwAAABaEIwAAAAvCEQAAgAXhCAAAwIJwBAAAYEE4AgAAsCAcAQAAWBCOAAAALAhHAAAAFoQjAAAAC8IRAACABeEIAADAgnAEAABgQTgCAACwIBwBAABYEI4AAAAsCEcAAAAWhCMAAAALwhEAAIAF4QgAAMCCcAQAAGBBOAIAALAgHAEAAFgQjgAAACwIRwAAABaEIwAAAAvCEQAAgAXhCAAAwIJwBAAAYEE4AgAAsCAcAQAAWBCOAAAALAhHAAAAFoQjAAAAC8IRAACAxTkbjp5++mnFx8crODhYvXr10meffdbUTQIAAD7gnAxHr7/+uqZMmaK//vWv2rx5s/7whz9o8ODBysvLa+qmAQCAJmYzDMNo6kY0tksuuUQ9e/bU/PnzzWW/+c1vNHz4cGVmZp7x9SUlJXI4HHK5XAoPD2/IpgIAgHpS1+/vc67nqLy8XNnZ2RowYIDb8gEDBuiLL76o9TVlZWUqKSlxmwAAQMt0zoWjH374QZWVlYqOjnZbHh0draKiolpfk5mZKYfDYU5xcXGN0VQAANAEzrlwVM1ms7nNG4ZRY1m1jIwMuVwuc8rPz2+MJgIAgCYQ0NQNaGyRkZHy9/ev0UtUXFxcozepmt1ul91ub4zmAQCAJnbO9RwFBQWpV69eWrlypdvylStXqk+fPk3UKgAA4CvOuZ4jSUpPT9fo0aOVlJSk5ORkPffcc8rLy9P48eObumkAAKCJnZPhaMSIETp48KAefvhhFRYWKjExUe+//746derU1E0DAABN7Jx8ztEvxXOOAABofnjOEQAAwFkgHAEAAFick2OOcHqVVYa+zD2k4tITigoLVu/4CPn71f4MKAAAWhrCEdx8kFOoh97ZpkLXCXNZrCNYM4YmaFBibBO2DACAxsFlNR9RWWVo7e6DemvLfq3dfVCVVY0/Tv6DnEJNeHmTWzCSpCLXCU14eZM+yCls9DYBANDY6DnyAb7QW1NZZeihd7aptkhmSLJJeuidbUpNiOESGwCgRaPnqIn5Sm/Nl7mHarTBypBU6DqhL3MPNUp7AABoKoSjJnSm3hrpVG9NY1xiKy71HIzOpg4AgOaKcNSEfKm3JiosuE51P5SWNcl4KAAAGgvhqAn5Um9N7/gIxTqCdabRRH9/b7sue/RjBmcDAFoswlETqmtvTV3rfgl/P5tmDE2QpDMGJO5eAwC0ZISjJnSm3hqbTt211js+olHaMygxVvP/1FMxjtOHscYeDwUAQGMiHDWh0/XWVM/PGJrQqLfOD0qM1Zr7+un+Ib85bR13rwEAWirCURPz1FsT4wjW/D/1bJKnUvv72RQZZq9TLXevAQBaGh4C6QMGJcYqNSHGp37PzJfGQwEA0JgIRz7C38+m5PPbNXUzTNXjoYpcJ2p9DpNNp3q3Gms8FAAAjYXLaqiVL46HAgCgMRCO4JEvjocCAKChcVkNp+WL46EAAGhIZxWOjh49qqysLOXl5am8vNxt3V133VUvDYPv8LXxUAAANCSvw9HmzZt15ZVX6tixYzp69KgiIiL0ww8/qFWrVoqKiiIcAQCAZs3rMUd33323hg4dqkOHDikkJETr1q3T3r171atXL82ePbsh2ggAANBovA5HW7Zs0dSpU+Xv7y9/f3+VlZUpLi5Ojz32mKZPn94QbQQAAGg0XoejwMBA2WynBuNGR0crLy9PkuRwOMx/A9Uqqwyt3X1Qb23Zr7W7D/JbbAAAn+f1mKPf/va32rhxo7p27aq+ffvqgQce0A8//KCXXnpJF154YUO0Ec3UBzmFeuidbSp0/fQTI7GOYM0YmsBjAAAAPsvrnqNZs2YpNvbUF9vf//53tWvXThMmTFBxcbGee+65em8gmqcPcgo14eVNbsFIkopcJzTh5U36IKewiVoGAMDp2QzD4DqHl0pKSuRwOORyuRQeHt7UzfE5lVWGLnv04xrBqFr1T4+sua8fz0sCADSaun5/n9UTsk+ePKmPPvpIzz77rEpLSyVJBQUFOnLkyNm1Fi3Kl7mHPAYjSTIkFbpO6MvcQ43XKAAA6sjrMUd79+7VoEGDlJeXp7KyMqWmpiosLEyPPfaYTpw4oWeeeaYh2olmpLjUczA6mzoAABqT1z1HkydPVlJSkg4fPqyQkBBz+TXXXKNVq1bVa+PQPEWFBZ+5yIs6AAAak9c9R2vWrNHnn3+uoKAgt+WdOnXS/v37661haL56x0co1hGsItcJ1TagrXrMUe/4iMZuGgAAZ+R1z1FVVZUqKytrLN+3b5/CwsLqpVFo3vz9bJoxNEHSqSBkVT0/Y2gCg7EBAD7J63CUmpqqf/3rX+a8zWbTkSNHNGPGDF155ZX12TY0Y4MSYzX/Tz0V43C/dBbjCNb8P/XkOUcAAJ/l9a38+/fvV79+/eTv76+dO3cqKSlJO3fuVGRkpFavXq2oqKiGaqvP4Fb+uqusMvRl7iEVl55QVNipS2n0GAEAmkJdv7/P6jlHx48f1+LFi5Wdna2qqir17NlTo0aNchug3ZIRjgAAaH4aJBxVVFSoW7duevfdd5WQkFAvDW2OCEcAADQ/DfIQyMDAQJWVlZk/PAsAANDSeD0g+84779Sjjz6qkydPNkR7AAAAmpTXzzlav369Vq1apRUrVujCCy9UaGio2/o333yz3hoHAADQ2LwOR23atNF1113XEG0BAABocl6HowULFjREOwAAAHyC12OOAAAAWrKzCkdvvPGGbrzxRv3+979Xz5493aaGMnPmTPXp00etWrVSmzZtaq3Jy8vT0KFDFRoaqsjISN11110qLy93q9m6datSUlIUEhKiDh066OGHH9ZZPOoJAAC0UF6HoyeffFK33HKLoqKitHnzZvXu3Vvt2rXTd999p8GDBzdEGyVJ5eXluuGGGzRhwoRa11dWVmrIkCE6evSo1qxZo8WLF2vp0qWaOnWqWVNSUqLU1FQ5nU5t2LBB8+bN0+zZs/XEE080WLsBAEAzY3ipW7duxquvvmoYhmG0bt3a2L17t2EYhnH//fcbd9xxh7dv57UFCxYYDoejxvL333/f8PPzM/bv328ue+211wy73W64XC7DMAzj6aefNhwOh3HixAmzJjMz03A6nUZVVVWd2+ByuQxJ5vsCAADfV9fvb697jvLy8tSnTx9JUkhIiEpLSyVJo0eP1muvvVafuc0ra9euVWJiopxOp7ls4MCBKisrU3Z2tlmTkpIiu93uVlNQUKA9e/Z4fO+ysjKVlJS4TQAAoGXyOhzFxMTo4MGDkqROnTpp3bp1kqTc3NwmHbtTVFSk6Ohot2Vt27ZVUFCQioqKPNZUz1fX1CYzM1MOh8Oc4uLi6rn1AADAV3gdjvr166d33nlHkjR27FjdfffdSk1N1YgRI3TNNdd49V4PPvigbDbbaaeNGzfW+f1q+1kTwzDclv+8pjrQne4nUTIyMuRyucwpPz+/zm0CAADNi9fPOXruuedUVVUlSRo/frwiIiK0Zs0aDR06VOPHj/fqvSZNmqSbbrrptDWdO3eu03vFxMRo/fr1bssOHz6siooKs3coJiamRg9RcXGxJNXoUbKy2+1ul+IAAEDL5XU48vPzk5/fTx1ON954o2688caz+vDIyEhFRkae1Wt/Ljk5WTNnzlRhYaFiY2MlSStWrJDdblevXr3MmunTp6u8vFxBQUFmjdPprHMIAwAALVuzeQhkXl6etmzZory8PFVWVmrLli3asmWLjhw5IkkaMGCAEhISNHr0aG3evFmrVq3StGnTNG7cOIWHh0uSRo4cKbvdrrS0NOXk5GjZsmWaNWuW0tPTT3tZDQAAnDtsRlOOovZCWlqaFi1aVGP5J598ossvv1zSqQA1ceJEffzxxwoJCdHIkSM1e/Zst0tiW7du1R133KEvv/xSbdu21fjx4/XAAw94FY5KSkrkcDjkcrnM4AUAAHxbXb+/m0048iWEIwAAmp+6fn83m8tqAAAAjeGswtHJkyf10Ucf6dlnnzUfAllQUGCO/wEAAGiuvL5bbe/evRo0aJDy8vJUVlam1NRUhYWF6bHHHtOJEyf0zDPPNEQ7AQAAGoXXPUeTJ09WUlKSDh8+rJCQEHP5Nddco1WrVtVr4wAAABqb1z1Ha9as0eeff24+J6hap06dtH///nprGAAAQFPwuueoqqpKlZWVNZbv27dPYWFh9dIoAACApuJ1OEpNTdW//vUvc95ms+nIkSOaMWOGrrzyyvpsGwAAQKPz+jlHBQUF6tu3r/z9/bVz504lJSVp586dioyM1OrVqxUVFdVQbfUZPOcIAIDmp67f316POXI6ndqyZYsWL16s7OxsVVVVaezYsRo1apTbAG0AAIDmiCdkn4Xq5HngwIFak6efn58CAn7KneXl5R7fy2azKTAw8KxqKyoq5Ok/X0PVSnIbjO9N7cmTJ1VVVVUvtYGBgeZPvjRUbWVlZa3j686mNiAgwPzBZl+oraqq0smTJz3W+vv7y9/f32dqDcNQRUVFvdRa/z4bqlY6/d8y54jaazlHcI5o6HNEg/UcZWZmKjo6Wrfeeqvb8v/85z86cOCA7rvvPm/fstmaM2eOgoODayzv0qWLRo4cac7Pnj3b43+0Tp06KS0tzZyfO3eujh07Vmut0+nUuHHjzPmnnnpKLper1tr27dtr4sSJ5vzzzz+vAwcO1FrrcDg0ZcoUc37hwoUqKCiotbZVq1a65557zPlXXnlFe/furbU2MDBQ06dPN+eXLFminTt31lorSTNmzDD/vWzZMm3bts1jbUZGhnmifPfdd/XVV195rJ02bZpCQ0MlSR9++KE2btzosXby5Mlq06aNJGnVqlVau3atx9oJEyaYl5E/++wzZWVleay97bbb1KFDB0nSunXr9NFHH3msHTNmjDp37ixJys7O1vLlyz3W3nzzzerataukU78b+NZbb3msvf7669W9e3dJ0vbt2/XGG294rB02bJguvvhiSdKuXbv02muveawdPHiwevfuLenU7xvW9huI1fr3769LL71UklRYWKgXXnjBY21KSor5u4kHDhzQ/PnzPdYmJydrwIABkiSXy6W5c+d6rE1KStKQIUMkSceOHdPs2bM91vbo0UPDhw+XdOpLPjMz02NtQkKCbrjhBnP+dLWcI07hHPETzhGnNMY5oi68HpD97LPP6te//nWN5d27d+cBkAAAoNnz+rJacHCwtm/frvj4eLfl3333nRISEnTixIl6baAv4rIaXebe1tJl/stquazGOcLbWs4Rv6zWF/7um9Vltbi4OH3++ec1wtHnn38up9Pp7ds1a0FBQTUehumpzpv3rCvryao51Fq/DJpDrfUPr6XV+vn51flY84Vam83WrGqlhvu75xzhO7W+8LfMOeIUb/8+z8TrcHTbbbdpypQpqqioUL9+/SSduu567733aurUqfXWMAAAgKbgdTi69957dejQIU2cONHs3g0ODtZ9992njIyMem8gAABAYzrrW/mPHDmi7du3KyQkRF26dJHdbq/vtvksHgIJAEDz02Bjjqq1bt1av/vd78725QAAAD7J63B09OhRPfLII1q1apWKi4trjOr/7rvv6q1xAAAAje2sBmRnZWVp9OjRio2NNW9TBAAAaAm8DkfLly/Xe++9Zz7BEgAAoCXx+gnZbdu2VUREREO0BQAAoMl5HY7+/ve/64EHHvD42z4AAADNmdeX1ebMmaPdu3crOjpanTt3rvH0002bNtVb4wAAABqb1+Go+heqAQAAWqKzfgjkuYyHQAIA0PzU9fvb6zFHkvTjjz/qhRdeUEZGhg4dOiTp1OW0/fv3n11rAQAAfITXl9W+/vpr9e/fXw6HQ3v27NG4ceMUERGhZcuWae/evXrxxRcbop0AAACNwuueo/T0dKWlpWnnzp0KDg42lw8ePFirV6+u18YBAAA0Nq/D0YYNG3T77bfXWN6hQwcVFRXVS6MAAACaiteX1YKDg1VSUlJj+Y4dO9S+fft6aRTQnFRWGfoy95CKS08oKixYveMj5O/Hz+oAQHPldTgaNmyYHn74YS1ZskSSZLPZlJeXp7/85S+67rrr6r2BgC/7IKdQD72zTYWuE+ayWEewZgxN0KDE2CZsGQDgbHl9WW327Nk6cOCAoqKidPz4caWkpOiCCy5QWFiYZs6c2RBtBHzSBzmFmvDyJrdgJElFrhOa8PImfZBT2EQtAwD8El73HIWHh2vNmjX6+OOPtWnTJlVVValnz57q379/Q7QP8EmVVYYeemebantImCHJJumhd7YpNSGGS2wA0Mx4FY5Onjyp4OBgbdmyRf369VO/fv0aql2AT/sy91CNHiMrQ1Kh64S+zD2k5PPbNV7DAAC/mFeX1QICAtSpUydVVlY2VHuAZqG41HMwOps6AIDv8HrM0d/+9je3J2MD56KosOAzF3lRBwDwHV6POXryySe1a9cuOZ1OderUSaGhoW7rN23aVG+NA3xV7/gIxTqCVeQ6Ueu4I5ukGMep2/oBAM2L1+Fo+PDhDdAMoHnx97NpxtAETXh5k2ySW0CqHn49Y2gCg7EBoBmyGYZR2//44jTq+qu+aPl4zhEANB91/f72uudIkn788Ue98cYb2r17t+655x5FRERo06ZNio6OVocOHc660UBzMygxVqkJMTwhGwBaEK/D0ddff63+/fvL4XBoz549GjdunCIiIrRs2TLt3btXL774YkO0E/BZ/n42btcHgBbE67vV0tPTlZaWpp07dyo4+Kc7cQYPHqzVq1fXa+Oq7dmzR2PHjlV8fLxCQkJ0/vnna8aMGSovL3ery8vL09ChQxUaGqrIyEjdddddNWq2bt2qlJQUhYSEqEOHDnr44YfFlUUAAFDN656jDRs26Nlnn62xvEOHDioqKqqXRv3c//73P1VVVenZZ5/VBRdcoJycHI0bN05Hjx7V7NmzJUmVlZUaMmSI2rdvrzVr1ujgwYMaM2aMDMPQvHnzJJ261piamqq+fftqw4YN+vbbb5WWlqbQ0FBNnTq1QdoOAACaF6/DUXBwsEpKSmos37Fjh9q3b18vjfq5QYMGadCgQeb8r371K+3YsUPz5883w9GKFSu0bds25efny+l0SpLmzJmjtLQ0zZw5U+Hh4XrllVd04sQJLVy4UHa7XYmJifr222/1xBNPKD09XTYb40QAADjXeX1ZbdiwYXr44YdVUVEhSbLZbMrLy9Nf/vIXXXfddfXeQE9cLpciIn56hszatWuVmJhoBiNJGjhwoMrKypSdnW3WpKSkyG63u9UUFBRoz549Hj+rrKxMJSUlbhMAAGiZvA5Hs2fP1oEDBxQVFaXjx48rJSVFF1xwgcLCwjRz5syGaGMNu3fv1rx58zR+/HhzWVFRkaKjo93q2rZtq6CgIPNyX2011fOnuySYmZkph8NhTnFxcfW1KQAAwMd4HY7Cw8O1Zs0aLV26VI888ogmTZqk999/X1lZWTWeln0mDz74oGw222mnjRs3ur2moKBAgwYN0g033KDbbrvNbV1tl8UMw3Bb/vOa6sHYp7uklpGRIZfLZU75+flebScAAGg+6jTmKCIiQt9++60iIyN16623au7cuerXr5/69ev3iz580qRJuummm05b07lzZ/PfBQUF6tu3r5KTk/Xcc8+51cXExGj9+vVuyw4fPqyKigqzdygmJqZGD1FxcbEk1ehRsrLb7W6X4gAAQMtVp56j8vJyc5zNokWLdOJE/fzSeGRkpH7961+fdqp+XMD+/ft1+eWXq2fPnlqwYIH8/NybnpycrJycHBUWFprLVqxYIbvdrl69epk1q1evdru9f8WKFXI6nW4hDAAAnLvq9PMhqamp+v7779WrVy8tWrRII0aMUEhISK21//nPf+q9kQUFBUpJSVHHjh314osvyt/f31wXExMj6dSt/BdffLGio6P1+OOP69ChQ0pLS9Pw4cPNW/ldLpe6deumfv36afr06dq5c6fS0tL0wAMPeHUrPz8fAgBA81OvPx/y8ssv65///Kd2794t6VTIqK/eo7pYsWKFdu3apV27dum8885zW1ed7fz9/fXee+9p4sSJuvTSSxUSEqKRI0eat/pLksPh0MqVK3XHHXcoKSlJbdu2VXp6utLT0xttWwAAgG/z+odn4+PjtXHjRrVrd+7+XAI9RwAAND91/f6u05ijiIgI/fDDD5Kkvn37KigoqH5aCQAA4GOadEA2AACAr6nTmKPk5GQNHz5cvXr1kmEYuuuuuxp1QDYAAEBj8XpAts1ma/QB2QAAAI2FAdlngQHZAAA0P/V6K79Vbm7uL2oYAACAL6tTOHryySf1//7f/1NwcLCefPLJ09bedddd9dIwAACAplCny2rWS2nx8fGe38xm03fffVevDfRFXFYDAKD5qdfLatZLaVxWAwAALVmdnnMEAABwrqhTz5E3vz32xBNPnHVjAAAAmlqdwtHmzZvd5rOzs1VZWalu3bpJkr799lv5+/urV69e9d9CAACARlSncPTJJ5+Y/37iiScUFhamRYsWqW3btpKkw4cP65ZbbtEf/vCHhmklAABAI/H6IZAdOnTQihUr1L17d7flOTk5GjBggAoKCuq1gb6Iu9UAAGh+6vr97fWA7JKSEn3//fc1lhcXF6u0tNTbtwMAAPApXoeja665RrfccoveeOMN7du3T/v27dMbb7yhsWPH6tprr22INgIAADQar38+5JlnntG0adP0pz/9SRUVFafeJCBAY8eO1eOPP17vDQQAAGhMXo85qnb06FHt3r1bhmHoggsuUGhoaH23zWcx5ggAgOanwX54tlpoaKguuuiis305AACAT+IJ2QAAABaEIwAAAAvCEQAAgAXhCAAAwIJwBAAAYEE4AgAAsCAcAQAAWBCOAAAALAhHAAAAFoQjAAAAC8IRAACABeEIAADAgnAEAABgQTgCAACwIBwBAABYEI4AAAAsCEcAAAAWhCMAAAALwhEAAIAF4QgAAMCCcAQAAGBBOAIAALAgHAEAAFg0m3B09dVXq2PHjgoODlZsbKxGjx6tgoICt5q8vDwNHTpUoaGhioyM1F133aXy8nK3mq1btyolJUUhISHq0KGDHn74YRmG0ZibAgAAfFizCUd9+/bVkiVLtGPHDi1dulS7d+/W9ddfb66vrKzUkCFDdPToUa1Zs0aLFy/W0qVLNXXqVLOmpKREqampcjqd2rBhg+bNm6fZs2friSeeaIpNAgAAPshmNNNuk7ffflvDhw9XWVmZAgMDtXz5cl111VXKz8+X0+mUJC1evFhpaWkqLi5WeHi45s+fr4yMDH3//fey2+2SpEceeUTz5s3Tvn37ZLPZ6vTZJSUlcjgccrlcCg8Pb7BtBAAA9aeu39/NpufI6tChQ3rllVfUp08fBQYGSpLWrl2rxMREMxhJ0sCBA1VWVqbs7GyzJiUlxQxG1TUFBQXas2ePx88rKytTSUmJ2wQAAFqmZhWO7rvvPoWGhqpdu3bKy8vTW2+9Za4rKipSdHS0W33btm0VFBSkoqIijzXV89U1tcnMzJTD4TCnuLi4+tokAADgY5o0HD344IOy2WynnTZu3GjW33PPPdq8ebNWrFghf39//fnPf3YbTF3bZTHDMNyW/7ym+vWnu6SWkZEhl8tlTvn5+We9zQAAwLcFNOWHT5o0STfddNNpazp37mz+OzIyUpGRkeratat+85vfKC4uTuvWrVNycrJiYmK0fv16t9cePnxYFRUVZu9QTExMjR6i4uJiSarRo2Rlt9vdLsUBAICWq0nDUXXYORvVPT5lZWWSpOTkZM2cOVOFhYWKjY2VJK1YsUJ2u129evUya6ZPn67y8nIFBQWZNU6n0y2EAQCAc1ezGHP05Zdf6t///re2bNmivXv36pNPPtHIkSN1/vnnKzk5WZI0YMAAJSQkaPTo0dq8ebNWrVqladOmady4ceaI9JEjR8putystLU05OTlatmyZZs2apfT09DrfqQYAAFq2ZhGOQkJC9Oabb+qKK65Qt27ddOuttyoxMVFZWVnm5S5/f3+99957Cg4O1qWXXqobb7xRw4cP1+zZs833cTgcWrlypfbt26ekpCRNnDhR6enpSk9Pb6pNAwAAPqbZPueoKfGcIwAAmp8W/ZwjAACAhkI4AgAAsCAcAQAAWBCOAAAALAhHAAAAFoQjAAAAC8IRAACABeEIAADAgnAEAABgQTgCAACwIBwBAABYEI4AAAAsCEcAAAAWhCMAAAALwhEAAIAF4QgAAMCCcAQAAGBBOAIAALAgHAEAAFgQjgAAACwIRwAAABaEIwAAAAvCEQAAgAXhCAAAwIJwBAAAYEE4AgAAsCAcAQAAWBCOAAAALAhHAAAAFoQjAAAAC8IRAACABeEIAADAgnAEAABgQTgCAACwIBwBAABYEI4AAAAsCEcAAAAWhCMAAAALwhEAAIAF4QgAAMCCcAQAAGDR7MJRWVmZLr74YtlsNm3ZssVtXV5enoYOHarQ0FBFRkbqrrvuUnl5uVvN1q1blZKSopCQEHXo0EEPP/ywDMNoxC0AAAC+LKCpG+Cte++9V06nU1999ZXb8srKSg0ZMkTt27fXmjVrdPDgQY0ZM0aGYWjevHmSpJKSEqWmpqpv377asGGDvv32W6WlpSk0NFRTp05tis0BAAA+plmFo+XLl2vFihVaunSpli9f7rZuxYoV2rZtm/Lz8+V0OiVJc+bMUVpammbOnKnw8HC98sorOnHihBYuXCi73a7ExER9++23euKJJ5Seni6bzdYUmwUAAHxIs7ms9v3332vcuHF66aWX1KpVqxrr165dq8TERDMYSdLAgQNVVlam7OxssyYlJUV2u92tpqCgQHv27PH42WVlZSopKXGbAABAy9QswpFhGEpLS9P48eOVlJRUa01RUZGio6PdlrVt21ZBQUEqKiryWFM9X11Tm8zMTDkcDnOKi4v7JZsDAAB8WJOGowcffFA2m+2008aNGzVv3jyVlJQoIyPjtO9X22UxwzDclv+8pnow9ukuqWVkZMjlcplTfn6+N5sJAACakSYdczRp0iTddNNNp63p3Lmz/vGPf2jdunVul8MkKSkpSaNGjdKiRYsUExOj9evXu60/fPiwKioqzN6hmJiYGj1ExcXFklSjR8nKbrfX+GwAANAyNWk4ioyMVGRk5BnrnnzySf3jH/8w5wsKCjRw4EC9/vrruuSSSyRJycnJmjlzpgoLCxUbGyvp1CBtu92uXr16mTXTp09XeXm5goKCzBqn06nOnTvX89YBAIDmqFmMOerYsaMSExPNqWvXrpKk888/X+edd54kacCAAUpISNDo0aO1efNmrVq1StOmTdO4ceMUHh4uSRo5cqTsdrvS0tKUk5OjZcuWadasWdypBgAATM0iHNWFv7+/3nvvPQUHB+vSSy/VjTfeqOHDh2v27NlmjcPh0MqVK7Vv3z4lJSVp4sSJSk9PV3p6ehO2HAAA+BKbweOhvVZSUiKHwyGXy2X2SgEAAN9W1+/vFtNzBAAAUB8IRwAAABaEIwAAAAvCEQAAgAXhCAAAwIJwBAAAYEE4AgAAsCAcAQAAWBCOAAAALAhHAAAAFoQjAAAAC8IRAACABeEIAADAIqCpG9CclZeXq7y8vMZyPz8/BQQEuNV5YrPZFBgYeFa1FRUVMgyjUWslKSgo6KxqT548qaqqqnqpDQwMlM1ma9DayspKVVZW1kttQECA/Pz8fKa2qqpKJ0+e9Fjr7+8vf39/n6k1DEMVFRX1Umv9+2yoWun0f8ucI2qv5RzBOaIxzhF1QTj6BebMmaPg4OAay7t06aKRI0ea87Nnz/b4H61Tp05KS0sz5+fOnatjx47VWut0OjVu3Dhz/qmnnpLL5aq1tn379po4caI5//zzz+vAgQO11jocDk2ZMsWcX7hwoQoKCmqtbdWqle655x5z/pVXXtHevXtrrQ0MDNT06dPN+SVLlmjnzp211krSjBkzzH8vW7ZM27Zt81ibkZFhnijfffddffXVVx5rp02bptDQUEnShx9+qI0bN3qsnTx5stq0aSNJWrVqldauXeuxdsKECYqKipIkffbZZ8rKyvJYe9ttt6lDhw6SpHXr1umjjz7yWDtmzBh17txZkpSdna3ly5d7rL355pvVtWtXSdLWrVv11ltveay9/vrr1b17d0nS9u3b9cYbb3isHTZsmC6++GJJ0q5du/Taa695rB08eLB69+4tScrLy9OiRYs81vbv31+XXnqpJKmwsFAvvPCCx9qUlBRdfvnlkqQDBw5o/vz5HmuTk5M1YMAASZLL5dLcuXM91iYlJWnIkCGSpGPHjmn27Nkea3v06KHhw4dLOvUln5mZ6bE2ISFBN9xwgzl/ulrOEadwjvgJ54hTGuMcURdcVgMAALCwGafr70StSkpK5HA4dODAAYWHh9dYT5d57bV0mdNlzmU172s5R5xdLeeIX1brC3/3DXGOqP7+drlctX5/VyMcnYW67lwAAOA76vr9zWU1AAAAC8IRAACABeEIAADAgnAEAABgwXOOAACAT6isMvRl7iEVl55QVFiwesdHyN/P1ujtIBwBAIAm90FOoR56Z5sKXSfMZbGOYM0YmqBBibGN2hYuqwEAgCb1QU6hJry8yS0YSVKR64QmvLxJH+QUNmp7CEcAAKDJVFYZeuidbartoYvVyx56Z5sqqxrvsYyEIwAA0GS+zD1Uo8fIypBU6DqhL3MPNVqbCEcAAKDJFJd6DkZnU1cfCEcAAKDJRIUF12tdfSAcAQCAJtM7PkKxjmB5umHfplN3rfWOj2i0NhGOAABAk/H3s2nG0ARJqhGQqudnDE1o1OcdEY4AAECTGpQYq/l/6qkYh/ulsxhHsOb/qWejP+eIh0ACAIAmNygxVqkJMTwhGwAAoJq/n03J57dr6mZwWQ0AAMCKcAQAAGBBOAIAALAgHAEAAFgQjgAAACwIRwAAABaEIwAAAAvCEQAAgAXhCAAAwIInZJ8FwzAkSSUlJU3cEgAAUFfV39vV3+OeEI7OQmlpqSQpLi6uiVsCAAC8VVpaKofD4XG9zThTfEINVVVVKigoUFhYmGy2+vtBvJKSEsXFxSk/P1/h4eH19r4tFfur7thXdce+8g77q+7YV95piP1lGIZKS0vldDrl5+d5ZBE9R2fBz89P5513XoO9f3h4OH84XmB/1R37qu7YV95hf9Ud+8o79b2/TtdjVI0B2QAAABaEIwAAAAvCkQ+x2+2aMWOG7HZ7UzelWWB/1R37qu7YV95hf9Ud+8o7Tbm/GJANAABgQc8RAACABeEIAADAgnAEAABgQTgCAACwIBw1sMzMTP3ud79TWFiYoqKiNHz4cO3YscOtxjAMPfjgg3I6nQoJCdHll1+ub775xq2mrKxMd955pyIjIxUaGqqrr75a+/bta8xNaXB12VdpaWmy2Wxu0+9//3u3mnNhX0nS/PnzddFFF5kPSEtOTtby5cvN9RxXPznTvuK48iwzM1M2m01Tpkwxl3FseVbb/uL4OuXBBx+ssR9iYmLM9T51XBloUAMHDjQWLFhg5OTkGFu2bDGGDBlidOzY0Thy5IhZ88gjjxhhYWHG0qVLja1btxojRowwYmNjjZKSErNm/PjxRocOHYyVK1camzZtMvr27Wv06NHDOHnyZFNsVoOoy74aM2aMMWjQIKOwsNCcDh486PY+58K+MgzDePvtt4333nvP2LFjh7Fjxw5j+vTpRmBgoJGTk2MYBseV1Zn2FcdV7b788kujc+fOxkUXXWRMnjzZXM6xVTtP+4vj65QZM2YY3bt3d9sPxcXF5npfOq4IR42suLjYkGRkZWUZhmEYVVVVRkxMjPHII4+YNSdOnDAcDofxzDPPGIZhGD/++KMRGBhoLF682KzZv3+/4efnZ3zwwQeNuwGN6Of7yjBOnWSGDRvm8TXn6r6q1rZtW+OFF17guKqD6n1lGBxXtSktLTW6dOlirFy50khJSTG/7Dm2audpfxkGx1e1GTNmGD169Kh1na8dV1xWa2Qul0uSFBERIUnKzc1VUVGRBgwYYNbY7XalpKToiy++kCRlZ2eroqLCrcbpdCoxMdGsaYl+vq+qffrpp4qKilLXrl01btw4FRcXm+vO1X1VWVmpxYsX6+jRo0pOTua4Oo2f76tqHFfu7rjjDg0ZMkT9+/d3W86xVTtP+6sax9cpO3fulNPpVHx8vG666SZ99913knzvuOKHZxuRYRhKT0/XZZddpsTERElSUVGRJCk6OtqtNjo6Wnv37jVrgoKC1LZt2xo11a9vaWrbV5I0ePBg3XDDDerUqZNyc3N1//33q1+/fsrOzpbdbj/n9tXWrVuVnJysEydOqHXr1lq2bJkSEhLMEwXH1U887SuJ4+rnFi9erE2bNmnDhg011nHOqul0+0vi+Kp2ySWX6MUXX1TXrl31/fff6x//+If69Omjb775xueOK8JRI5o0aZK+/vprrVmzpsY6m83mNm8YRo1lP1eXmubK074aMWKE+e/ExEQlJSWpU6dOeu+993Tttdd6fL+Wuq+6deumLVu26Mcff9TSpUs1ZswYZWVlmes5rn7iaV8lJCRwXFnk5+dr8uTJWrFihYKDgz3WcWydUpf9xfF1yuDBg81/X3jhhUpOTtb555+vRYsWmQPUfeW44rJaI7nzzjv19ttv65NPPtF5551nLq8eqf/z1FtcXGwm6JiYGJWXl+vw4cMea1oST/uqNrGxserUqZN27twp6dzbV0FBQbrggguUlJSkzMxM9ejRQ3PnzuW4qoWnfVWbc/m4ys7OVnFxsXr16qWAgAAFBAQoKytLTz75pAICAszt5dg65Uz7q7KyssZrzuXjyyo0NFQXXnihdu7c6XPnLMJRAzMMQ5MmTdKbb76pjz/+WPHx8W7r4+PjFRMTo5UrV5rLysvLlZWVpT59+kiSevXqpcDAQLeawsJC5eTkmDUtwZn2VW0OHjyo/Px8xcbGSjp39pUnhmGorKyM46oOqvdVbc7l4+qKK67Q1q1btWXLFnNKSkrSqFGjtGXLFv3qV7/i2LI40/7y9/ev8Zpz+fiyKisr0/bt2xUbG+t756x6Hd6NGiZMmGA4HA7j008/dbt98dixY2bNI488YjgcDuPNN980tm7datx888213r543nnnGR999JGxadMmo1+/fi3uNs8z7avS0lJj6tSpxhdffGHk5uYan3zyiZGcnGx06NDhnNtXhmEYGRkZxurVq43c3Fzj66+/NqZPn274+fkZK1asMAyD48rqdPuK4+rMfn73FcfW6Vn3F8fXT6ZOnWp8+umnxnfffWesW7fOuOqqq4ywsDBjz549hmH41nFFOGpgkmqdFixYYNZUVVUZM2bMMGJiYgy73W788Y9/NLZu3er2PsePHzcmTZpkREREGCEhIcZVV11l5OXlNfLWNKwz7atjx44ZAwYMMNq3b28EBgYaHTt2NMaMGVNjP5wL+8owDOPWW281OnXqZAQFBRnt27c3rrjiCjMYGQbHldXp9hXH1Zn9PBxxbJ2edX9xfP2k+rlFgYGBhtPpNK699lrjm2++Mdf70nFlMwzDqN++KAAAgOaLMUcAAAAWhCMAAAALwhEAAIAF4QgAAMCCcAQAAGBBOAIAALAgHAEAAFgQjgAAACwIRwAAABaEIwAAAAvCEYAWpbKyUlVVVU3dDK+Ul5c3dRMAWBCOADSYDz74QJdddpnatGmjdu3a6aqrrtLu3bvN9cnJyfrLX/7i9poDBw4oMDBQn3zyiaRTweHee+9Vhw4dFBoaqksuuUSffvqpWb9w4UK1adNG7777rhISEmS327V3715t2LBBqampioyMlMPhUEpKijZt2uT2Wf/73/902WWXKTg4WAkJCfroo49ks9n03//+16zZv3+/RowYobZt26pdu3YaNmyY9uzZ43GbKysrNXbsWMXHxyskJETdunXT3Llz3WrS0tI0fPhwZWZmyul0qmvXrnX6rLpsE4BfjnAEoMEcPXpU6enp2rBhg1atWiU/Pz9dc801Zs/OqFGj9Nprr8n6+9evv/66oqOjlZKSIkm65ZZb9Pnnn2vx4sX6+uuvdcMNN2jQoEHauXOn+Zpjx44pMzNTL7zwgr755htFRUWptLRUY8aM0WeffaZ169apS5cuuvLKK1VaWipJqqqq0vDhw9WqVSutX79ezz33nP7617+6tf/YsWPq27evWrdurdWrV2vNmjVq3bq1Bg0a5LG3p6qqSuedd56WLFmibdu26YEHHtD06dO1ZMkSt7pVq1Zp+/btWrlypd599906fdaZtglAPTEAoJEUFxcbkoytW7ea8wEBAcbq1avNmuTkZOOee+4xDMMwdu3aZdhsNmP//v1u73PFFVcYGRkZhmEYxoIFCwxJxpYtW0772SdPnjTCwsKMd955xzAMw1i+fLkREBBgFBYWmjUrV640JBnLli0zDMMw/u///s/o1q2bUVVVZdaUlZUZISEhxocffljn7Z44caJx3XXXmfNjxowxoqOjjbKyMnPZ2XzWz7cJQP2g5whAg9m9e7dGjhypX/3qVwoPD1d8fLwkKS8vT5LUvn17paam6pVXXpEk5ebmau3atRo1apQkadOmTTIMQ127dlXr1q3NKSsry+3yXFBQkC666CK3zy4uLtb48ePVtWtXORwOORwOHTlyxPzsHTt2KC4uTjExMeZrevfu7fYe2dnZ2rVrl8LCwszPjoiI0IkTJ9w+/+eeeeYZJSUlqX379mrdurWef/5583OrXXjhhQoKCvLqs860TQDqR0BTNwBAyzV06FDFxcXp+eefl9PpVFVVlRITE90uSY0aNUqTJ0/WvHnz9Oqrr6p79+7q0aOHpFOXqPz9/ZWdnS1/f3+3927durX575CQENlsNrf1aWlpOnDggP71r3+pU6dOstvtSk5ONj/bMIwar/m5qqoq9erVywxvVu3bt6/1NUuWLNHdd9+tOXPmKDk5WWFhYXr88ce1fv16t7rQ0FCvP+tM2wSgfhCOADSIgwcPavv27Xr22Wf1hz/8QZK0Zs2aGnXDhw/X7bffrg8++ECvvvqqRo8eba777W9/q8rKShUXF5vvUVefffaZnn76aV155ZWSpPz8fP3www/m+l//+tfKy8vT999/r+joaEmnBjxb9ezZU6+//rqioqIUHh5e58/t06ePJk6caC47XS+TN591pm0CUD+4rAagQVTfcfXcc89p165d+vjjj5Wenl6jLjQ0VMOGDdP999+v7du3a+TIkea6rl27atSoUfrzn/+sN998U7m5udqwYYMeffRRvf/++6f9/AsuuEAvvfSStm/frvXr12vUqFEKCQkx16empur888/XmDFj9PXXX+vzzz83B2RX9yiNGjVKkZGRGjZsmD777DPl5uYqKytLkydP1r59+zx+7saNG/Xhhx/q22+/1f33318jdNWmLp91pm0CUD8IRwAahJ+fnxYvXqzs7GwlJibq7rvv1uOPP15r7ahRo/TVV1/pD3/4gzp27Oi2bsGCBfrzn/+sqVOnqlu3brr66qu1fv16xcXFnfbz//Of/+jw4cP67W9/q9GjR+uuu+5SVFSUud7f31///e9/deTIEf3ud7/Tbbfdpr/97W+SpODgYElSq1attHr1anXs2FHXXnutfvOb3+jWW2/V8ePHPfbujB8/Xtdee61GjBihSy65RAcPHnTrRfKkLp91pm0CUD9shmG5hxYAzmGff/65LrvsMu3atUvnn39+UzcHQBMhHAE4Zy1btkytW7dWly5dtGvXLk2ePFlt27atdWwUgHMHA7IBnLNKS0t17733Kj8/X5GRkerfv7/mzJnT1M0C0MToOQIAALBgQDYAAIAF4QgAAMCCcAQAAGBBOAIAALAgHAEAAFgQjgAAACwIRwAAABaEIwAAAIv/D70OrkBtT3kOAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "bland_altman_plot(measurements['area_1'], measurements['area_2'], 'area')" ] }, { "cell_type": "markdown", "id": "b967ab80-4680-40fe-8580-48e85ee9b452", "metadata": {}, "source": [ "In the case shown above, the average difference of the area measurement is about -100, which means that the first algorithm produces on average smaller area measurements than the second.\n", "\n", "For demonstration purposes we will now compare the same algorithm in a CPU and a GPU variant." ] }, { "cell_type": "code", "execution_count": 13, "id": "367da703-3adb-412c-b3e7-2e7d30edf353", "metadata": { "tags": [] }, "outputs": [], "source": [ "def segmentation_1_gpu(image):\n", " return cle.voronoi_otsu_labeling(image)" ] }, { "cell_type": "code", "execution_count": 14, "id": "af44eb00-73b9-4227-922f-c3559bb47c2f", "metadata": { "tags": [] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAGwCAYAAABRgJRuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAxPElEQVR4nO3de3QUdZ7+8adzDyFpCCEkgRBYETSgqCAuisNlgQEVAZVlBBEccAXkrjjCbzTq7GxULgOoIMpw2VFBlgOozAgCYriMoBCuAyMEIwESJkGcTiCSW9fvDw49NElDd+hOpyvv1zk5h6r+dPcn+dpVj1XfrrIYhmEIAAAgwAX5uwEAAABvINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTCPF3AzXJbrcrNzdX0dHRslgs/m4HAAC4wTAMFRUVKSkpSUFBro/H1KlQk5ubq+TkZH+3AQAAquHkyZNq1qyZy8frVKiJjo6WdOmPEhMT4+duAACAOwoLC5WcnOzYj7tSp0LN5VNOMTExhBoAAALM9aaOMFEYAACYAqEGAACYAqEGAACYAqEGAACYAqEGAACYAqEGAACYAqEGAACYAqEGAACYAqEGAACYAqEGAACYAqEGAACYAqEGAACYQp26oeVlpaWlKi0trbQ+KChIISEhTnWuWCwWhYaGVqu2rKxMhmHUaK0khYWFVau2vLxcdrvdK7WhoaGOG5L5qraiokIVFRVeqQ0JCVFQUFCtqbXb7SovL3dZGxwcrODg4FpTaxiGysrKvFJ75efTV7XStT/LbCOqrmUbwTaiJrYR7qiToWbWrFmKiIiotP7mm2/WkCFDHMszZ850+cdOSUnRiBEjHMtz585VcXFxlbVJSUl6+umnHcvvvPOObDZblbWNGzfW2LFjHcvvv/++CgoKqqy1Wq2aNGmSY3np0qXKzc2tsrZevXqaOnWqY/nDDz/UiRMnqqwNDQ3V9OnTHcsrV67UsWPHqqyVpLS0NMe/16xZo8OHD7usnTZtmmMDt27dOu3fv99l7fPPP6+oqChJ0oYNG7R7926XtRMnTlSDBg0kSZs3b9bXX3/tsnbMmDGKj4+XJG3btk0ZGRkua0eNGqWmTZtKknbu3KlNmza5rB0+fLhatGghSdqzZ48+//xzl7WPP/64WrduLUk6ePCgPvnkE5e1jz32mNq2bStJOnLkiFatWuWytn///rrjjjskSVlZWVq+fLnL2r59+6pTp06SpJycHC1btsxlbc+ePXXfffdJkvLy8rRo0SKXtV27dlW3bt0kSQUFBVqwYIHL2s6dO6t3796SJJvNprlz57qs7dixox588EFJUnFxsWbOnOmytn379howYICkSzvn9PR0l7WpqakaNGiQY/latWwjLmEb8S9sIy6piW2EOzj9BAAATMFiXOv4oskUFhbKarWqoKBAMTExlR7n0HLVtRxa5tAyp588r2UbUb1athE3VlsbPve+2EZc3n/bbLYq99+X1clQc70/CgAAqD3c3X9z+gkAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJgCoQYAAJhCwISa9PR03X333YqOjlZ8fLwGDBig7777zt9tAQCAWiJgQk1GRoaeffZZ7dy5Uxs3blR5ebl69+6tCxcu+Ls11CIVdkNfH/9Rn+w7ra+P/6gKu+HvlgAANcRiGEZAbvULCgoUHx+vjIwM/eIXv3DrOYWFhbJarbLZbIqJifFxh6hp6w/l6dXPDivPdtGxLtEaobR+qerTLtGPnQEAboS7+++AOVJzNZvNJkmKjY11WVNSUqLCwkKnH5jT+kN5GvNBplOgkaQztosa80Gm1h/K81NnAICaEpChxjAMTZkyRV26dFG7du1c1qWnp8tqtTp+kpOTa7BL1JQKu6FXPzusqg45Xl736meHORUFACYXkKFm3LhxOnDggJYvX37NumnTpslmszl+Tp48WUMdoiZ9k32u0hGaKxmS8mwX9U32uZprCgBQ40L83YCnxo8fr08//VRbt25Vs2bNrlkbHh6u8PDwGuoM/pJf5DrQVKcOABCYAibUGIah8ePHa82aNfrqq6/UsmVLf7eEWiI+OsKrdQCAwBQwoebZZ5/VRx99pE8++UTR0dE6c+aMJMlqtSoyMtLP3cGfOrWMVaI1QmdsF6ucV2ORlGCNUKeWrieVAwACX8DMqVmwYIFsNpu6deumxMREx8/HH3/s79bgZ8FBFqX1S5V0KcBc6fJyWr9UBQdd/SgAwEwC5khNgF5OBzWkT7tELXjirkrXqUngOjUAUGcETKgBrqdPu0T1Sk3QN9nnlF90UfHRl045cYQGAOoGQg1MJTjIos43NfJ3GwAAPwiYOTUAAADXQqgBAACmQKgBAACmQKgBAACmQKgBAACmQKgBAACmQKgBAACmwHVqblCF3eBibwAA1AKEmhuw/lBepcvyJ3JZfgAA/ILTT9W0/lCexnyQ6RRoJOmM7aLGfJCp9Yfy/NQZAAB1E6GmGirshl797LCqusXm5XWvfnZYFXZuwgkAQE0h1FTDN9nnKh2huZIhKc92Ud9kn6u5pgAAqOMINdWQX+Q60FSnDgAA3DhCTTXER0d4tQ4AANw4Qk01dGoZq0RrhFx9cduiS9+C6tQytibbAgCgTiPUVENwkEVp/VIlqVKwubyc1i+V69UAAFCDCDXV1KddohY8cZcSrM6nmBKsEVrwxF1cpwYAgBrGxfduQJ92ieqVmsAVhQEAqAUINTcoOMiizjc18ncbAADUeZx+AgAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAAphBQoWbr1q3q16+fkpKSZLFYtHbtWn+3BAAAaomACjUXLlxQ+/bt9fbbb/u7FQAAUMuE+LsBT/Tt21d9+/b1dxsAAKAWCqhQ46mSkhKVlJQ4lgsLC/3YDQAA8KWAOv3kqfT0dFmtVsdPcnKyv1sCAAA+YupQM23aNNlsNsfPyZMn/d0SAADwEVOffgoPD1d4eLi/2wAAADXA1EdqAABA3RFQR2rOnz+vrKwsx3J2drb27dun2NhYNW/e3I+dAQAAfwuoULN79251797dsTxlyhRJ0vDhw7V06VI/dQUAAGqDgAo13bp1k2EY/m4DAADUQsypAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAApkCoAQAAplCt69RcuHBBGRkZysnJUWlpqdNjEyZM8EpjAAAAnvA41Ozdu1cPPPCAiouLdeHCBcXGxurs2bOqV6+e4uPjCTUAAMAvPD79NHnyZPXr10/nzp1TZGSkdu7cqRMnTqhDhw6aOXOmL3oEAAC4Lo9Dzb59+/Tcc88pODhYwcHBKikpUXJyst58801Nnz7dFz0CAABcl8ehJjQ0VBaLRZLUpEkT5eTkSJKsVqvj3wAAADXN4zk1d955p3bv3q3WrVure/fuevnll3X27Fn96U9/0m233eaLHgEAAK7L4yM1//M//6PExERJ0u9+9zs1atRIY8aMUX5+vt577z2vNwgAAOAOi2EYhr+bqCmFhYWyWq2y2WyKiYnxdzsAAMAN7u6/q3XxvfLycm3atEkLFy5UUVGRJCk3N1fnz5+vXrcAAAA3yOM5NSdOnFCfPn2Uk5OjkpIS9erVS9HR0XrzzTd18eJFvfvuu77oEwAA4Jo8PlIzceJEdezYUT/99JMiIyMd6wcOHKjNmzd7tTkAAAB3eXykZvv27dqxY4fCwsKc1qekpOj06dNeawwAAMATHh+psdvtqqioqLT+1KlTio6O9kpTAAAAnvI41PTq1Utz5sxxLFssFp0/f15paWl64IEHvNkbAACA2zz+Svfp06fVo0cPBQcH69ixY+rYsaOOHTumuLg4bd26VfHx8b7q9YbxlW4AAAKPu/tvj+fUNG3aVPv27dOKFSu0Z88e2e12jRw5UkOHDnWaOAwAAFCTPDpSU1ZWpjZt2mjdunVKTU31ZV8+wZEaAAACj08uvhcaGqqSkhLHDS0BAABqC48nCo8fP15vvPGGysvLfdEPAABAtXg8p2bXrl3avHmzvvjiC912222Kiopyenz16tVeaw4AAMBdHoeaBg0a6NFHH/VFLwAAANXmcahZsmSJL/oAAAC4IdW6SzcAAEBt4/GRGklatWqVVq5cqZycHJWWljo9lpmZ6ZXGAAAAPOHxkZp58+bpqaeeUnx8vPbu3atOnTqpUaNG+v7779W3b19f9AgAAHBdHoea+fPn67333tPbb7+tsLAwvfDCC9q4caMmTJggm83mix4BAACuy+NQk5OTo3vvvVeSFBkZqaKiIknSsGHDtHz5cu92BwAA4CaPQ01CQoJ+/PFHSVJKSop27twpScrOzpaH98YEAADwGo9DTY8ePfTZZ59JkkaOHKnJkyerV69eGjx4sAYOHOj1BgEAANzh0Q0tJclut8tutysk5NIXp1auXKnt27erVatWGj16tMLCwnzSqDdwQ0sAAAKPu/tvj0NNICPUAAAQeHxyl24AAIDailADAABMgVADAABMgVADAABMoVqhpry8XJs2bdLChQsdF9/Lzc3V+fPnvdocAACAuzy+oeWJEyfUp08f5eTkqKSkRL169VJ0dLTefPNNXbx4Ue+++64v+gQAALgmj4/UTJw4UR07dtRPP/2kyMhIx/qBAwdq8+bNXm0OAADAXR4fqdm+fbt27NhR6SJ7KSkpOn36tNcaAwAA8ITHR2rsdrsqKioqrT916pSio6O90hQAAICnPA41vXr10pw5cxzLFotF58+fV1pamh544AFv9gYAAOA2j2+TkJubq+7duys4OFjHjh1Tx44ddezYMcXFxWnr1q2Kj4/3Va83jNskAAAQeNzdf3s8pyYpKUn79u3TihUrtGfPHtntdo0cOVJDhw51mjgMAABQk7ihJQAAqNV8dkPL9PR0LV68uNL6xYsX64033vD05QAAALzC41CzcOFC3XLLLZXWt23blgvvAQAAv/E41Jw5c0aJiYmV1jdu3Fh5eXleaQoAAMBTHoea5ORk7dixo9L6HTt2KCkpyStNAQAAeMrjbz+NGjVKkyZNUllZmXr06CFJ2rx5s1544QU999xzXm/wavPnz9eMGTOUl5entm3bas6cObr//vs9eo3S0lKVlpZWWh8UFKSQkBCnOlcsFotCQ0OrVVtWViZX87N9VSvJ6SrQntSWl5fLbrd7pTY0NFQWi8WntRUVFVVeILI6tSEhIQoKCqo1tXa7XeXl5S5rg4ODFRwcXGtqDcNQWVmZV2qv/Hz6qla69meZbUTVtWwj2EbUxDbCHR6HmhdeeEHnzp3T2LFjHR/SiIgI/eY3v9G0adM8fTmPfPzxx5o0aZLmz5+v++67TwsXLlTfvn11+PBhNW/e3O3XmTVrliIiIiqtv/nmmzVkyBDH8syZM13+sVNSUjRixAjH8ty5c1VcXFxlbVJSkp5++mnH8jvvvCObzVZlbePGjTV27FjH8vvvv6+CgoIqa61WqyZNmuRYXrp0qXJzc6usrVevnqZOnepY/vDDD3XixIkqa0NDQzV9+nTH8sqVK3Xs2LEqayUpLS3N8e81a9bo8OHDLmunTZvm2MCtW7dO+/fvd1n7/PPPKyoqSpK0YcMG7d6922XtxIkT1aBBA0mXQvbXX3/tsnbMmDGO6ylt27ZNGRkZLmtHjRqlpk2bSpJ27typTZs2uawdPny4WrRoIUnas2ePPv/8c5e1jz/+uFq3bi1JOnjwoD755BOXtY899pjatm0rSTpy5IhWrVrlsrZ///664447JElZWVlavny5y9q+ffuqU6dOkqScnBwtW7bMZW3Pnj113333SZLy8vK0aNEil7Vdu3ZVt27dJEkFBQVasGCBy9rOnTurd+/ekiSbzaa5c+e6rO3YsaMefPBBSVJxcbFmzpzpsrZ9+/YaMGCApEs75/T0dJe1qampGjRokGP5WrVsIy5hG/EvbCMuqYlthDs8Pv1ksVj0xhtvqKCgQDt37tT+/ft17tw5vfzyy56+lMdmz56tkSNHatSoUbr11ls1Z84cJScnX/MPAgAA6oaAuU5NaWmp6tWrp//7v//TwIEDHesnTpyoffv2VZmkS0pKVFJS4lguLCxUcnKyCgoKqvyeO4eWq67l0DKHljn95Hkt24jq1bKNuLHa2vC598U2wmdXFL5w4YJef/11bd68Wfn5+ZX+I/r+++89fUm3nD17VhUVFWrSpInT+iZNmujMmTNVPic9PV2vvvpqpfVhYWGV7jJeFXdqqlN75UYmEGqv3IgHQq0n52ADrTYoKMjt/9ZqQ63FYgmoWsl3n3u2EbWntjZ8ltlGXOLp5/N6qjVROCMjQ8OGDVNiYqIjzdaUq9/PMAyXPUybNk1TpkxxLF8+UgMAAMzH41Dz+eef689//rNjQlBNiYuLU3BwcKWjMvn5+ZWO3lwWHh6u8PDwmmgPAAD4mccThRs2bKjY2Fhf9HJNYWFh6tChgzZu3Oi0fuPGjbr33ntrvB8AAFC7eBxqfve73+nll192+dVEX5oyZYoWLVqkxYsX68iRI5o8ebJycnI0evToGu8FAADULh6ffpo1a5aOHz+uJk2aqEWLFpUmk2VmZnqtuasNHjxYP/74o1577TXl5eWpXbt2+stf/qKUlBSfvScAAAgMHoeayxe08pexY8c6XXgKAABACqDr1HiDu99zBwAAtYe7+2+P59RI0j//+U8tWrRI06ZN07lz5yRdOu10+vTp6nULAABwgzw+/XTgwAH17NlTVqtVP/zwg55++mnFxsZqzZo1OnHihP73f//XF30CAABck8dHaqZMmaIRI0bo2LFjTjeF7Nu3r7Zu3erV5gAAANzlcaj59ttv9cwzz1Ra37RpU5e3KwAAAPA1j0NNRESECgsLK63/7rvv1LhxY680BQAA4CmPQ03//v312muvOe6qabFYlJOToxdffFGPPvqo1xsEAABwh8ehZubMmSooKFB8fLx+/vlnde3aVa1atVJ0dLR+//vf+6JHAACA6/L4208xMTHavn27vvzyS2VmZsput+uuu+5Sz549fdEfAACAWzwKNeXl5YqIiNC+ffvUo0cP9ejRw1d9AQAAeMSj008hISFKSUlRRUWFr/oBAACoFo/n1Pz2t791upIwAABAbeDxnJp58+YpKytLSUlJSklJUVRUlNPjvrxLNwAAgCsBd5duAACAqnCXbgAAUKtxl24AAFCncJduAABgCtylGwAAmAJ36QYAAKbAXboBAIApcJduAABgCtylGwAAmAJ36QYAAKbg1sX3YmNjdfToUcXFxenXv/615s6dq+jo6Jroz6u4+B4AAIHHqxffKy0tdUwOXrZsmS5evOidLgEAALzErdNPnTt31oABA9ShQwcZhqEJEyYoMjKyytrFixd7tUEAAAB3uBVqPvjgA/3hD3/Q8ePHJUk2m42jNQAAoFbx+IaWLVu21O7du9WoUSNf9eQzzKkBACDweHVOTWxsrM6ePStJ6t69u8LCwrzTJQAAgJcwURgAAJgCE4UBAIApeDxR2GKxMFEYAADUOkwUBgAAtZq7+2+Pb5OQnZ19Q40BAAD4gluhZt68efqv//ovRUREaN68edesnTBhglcaAwJBhd3QN9nnlF90UfHREerUMlbBQRZ/twUAdZJbp5+uPOXUsmVL1y9msej777/3aoPeVBtOP7ETNI/1h/L06meHlWf71/yyRGuE0vqlqk+7RD92BgDm4u7+2+M5NYHM36GGnaB5rD+UpzEfZOrqD8/leLrgibsYUwDwEq9efA837vJO8MpAI0lnbBc15oNMrT+U56fO4KkKu6FXPztcKdBIcqx79bPDqrDXmf9fAIBawa05NVOmTHH7BWfPnl3tZszqejtBiy7tBHulJnAqKgB8k32uUji9kiEpz3ZR32SfU+ebAu9bggAQqNwKNXv37nVa3rNnjyoqKtSmTRtJ0tGjRxUcHKwOHTp4v0MTYCdoLvlF7l2jyd06AIB3uBVqtmzZ4vj37NmzFR0drWXLlqlhw4aSpJ9++klPPfWU7r//ft90GeDYCZpLfHSEV+skJpADgDd4fJ2aWbNm6YsvvnAEGklq2LCh/vu//1u9e/fWc88959UGzcAXO0H4T6eWsUq0RuiM7WKVpxQtkhKsl4KJO5hADgDe4fFE4cLCQv3jH/+otD4/P19FRUVeacpsLu8EXf1/t0WXdmLu7gThX8FBFqX1S5WkSmN6eTmtX6pbR1qYQA4A3uNxqBk4cKCeeuoprVq1SqdOndKpU6e0atUqjRw5Uo888ogvegx43twJonbo0y5RC564SwlW56NrCdYIt7/OzbeoAMC7PL5OTXFxsZ5//nktXrxYZWVlkqSQkBCNHDlSM2bMUFRUlE8a9QauUwNvu5G5MF8f/1GPv7/zunXLn/53JpADqNN8du+nevXqaf78+ZoxY4aOHz8uwzDUqlWrWh1maos+7RLVKzWBCaEmEhxkqXbgYAI5AHiXx6HmsqioKN1+++3e7KVOuJGdIMyFCeQA4F1cURjwEyaQA4B3EWoAP2ECOQB4F6EG8CNvfIsKAHBJtefUAPAOJpADgHcQaoBagAnkAHDjOP0EAABMgVADAABMgVADAABMgVADAABMIWBCze9//3vde++9qlevnho0aODvdgAAQC0TMKGmtLRUgwYN0pgxY/zdCgAAqIUC5ivdr776qiRp6dKl/m0EAADUSgETaqqjpKREJSUljuXCwkI/dgMAAHwpYE4/VUd6erqsVqvjJzk52d8tAQAAH/FrqHnllVdksViu+bN79+5qv/60adNks9kcPydPnvRi9wAAoDbx6+mncePG6Ve/+tU1a1q0aFHt1w8PD1d4eHi1nw8AAAKHX0NNXFyc4uLi/NkCAAAwiYCZKJyTk6Nz584pJydHFRUV2rdvnySpVatWql+/vn+bAwAAfhcwoebll1/WsmXLHMt33nmnJGnLli3q1q2bn7oCAAC1hcUwDMPfTdSUwsJCWa1W2Ww2xcTE+LsdAADgBnf336b+SjcAAKg7CDUAAMAUCDUAAMAUCDUAAMAUCDUAAMAUCDUAAMAUCDUAAMAUAubiewAAoHaqsBv6Jvuc8osuKj46Qp1axio4yFLjfRBqAABAta0/lKdXPzusPNtFx7pEa4TS+qWqT7vEGu2F008AAKBa1h/K05gPMp0CjSSdsV3UmA8ytf5QXo32Q6gBAAAeq7AbevWzw6rqXkuX17362WFV2GvubkyEGgAA4LFvss9VOkJzJUNSnu2ivsk+V2M9EWoAAIDH8otcB5rq1HkDoQYAAHgsPjrCq3XeQKgBAAAe69QyVonWCLn64rZFl74F1allbI31RKgBAAAeCw6yKK1fqiRVCjaXl9P6pdbo9WoINQAAoFr6tEvUgifuUoLV+RRTgjVCC564q8avU8PF9wAAQLX1aZeoXqkJXFEYAAAEvuAgizrf1MjfbXD6CQAAmAOhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmEJAhJoffvhBI0eOVMuWLRUZGambbrpJaWlpKi0t9XdrAACglgjxdwPu+Pvf/y673a6FCxeqVatWOnTokJ5++mlduHBBM2fO9Hd7AACgFrAYhmH4u4nqmDFjhhYsWKDvv//e7ecUFhbKarXKZrMpJibGh90BAABvcXf/HRBHaqpis9kUGxt7zZqSkhKVlJQ4lgsLC33dFgAA8JOAmFNztePHj+utt97S6NGjr1mXnp4uq9Xq+ElOTq6hDgEAQE3za6h55ZVXZLFYrvmze/dup+fk5uaqT58+GjRokEaNGnXN1582bZpsNpvj5+TJk778dQAAgB/5dU7N2bNndfbs2WvWtGjRQhEREZIuBZru3bvrnnvu0dKlSxUU5FkmY04NAACBJyDm1MTFxSkuLs6t2tOnT6t79+7q0KGDlixZ4nGgAQAA5hYQE4Vzc3PVrVs3NW/eXDNnzlRBQYHjsYSEBD92BgAAaouACDVffPGFsrKylJWVpWbNmjk9FqDfSAcAAF4WEOdwRowYIcMwqvwBAACQAiTUAAAAXA+hBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmAKhBgAAmEKIvxvwh9LSUpWWllZaHxQUpJCQEKc6VywWi0JDQ6tVW1ZWJsMwarRWksLCwqpVW15eLrvd7pXa0NBQWSwWn9ZWVFSooqLCK7UhISEKCgqqNbV2u13l5eUua4ODgxUcHFxrag3DUFlZmVdqr/x8+qpWuvZnmW1E1bVsI9hG1MQ2wh11MtTMmjVLERERldbffPPNGjJkiGN55syZLv/YKSkpGjFihGN57ty5Ki4urrI2KSlJTz/9tGP5nXfekc1mq7K2cePGGjt2rGP5/fffV0FBQZW1VqtVkyZNciwvXbpUubm5VdbWq1dPU6dOdSx/+OGHOnHiRJW1oaGhmj59umN55cqVOnbsWJW1kpSWlub495o1a3T48GGXtdOmTXNs4NatW6f9+/e7rH3++ecVFRUlSdqwYYN2797tsnbixIlq0KCBJGnz5s36+uuvXdaOGTNG8fHxkqRt27YpIyPDZe2oUaPUtGlTSdLOnTu1adMml7XDhw9XixYtJEl79uzR559/7rL28ccfV+vWrSVJBw8e1CeffOKy9rHHHlPbtm0lSUeOHNGqVatc1vbv31933HGHJCkrK0vLly93Wdu3b1916tRJkpSTk6Nly5a5rO3Zs6fuu+8+SVJeXp4WLVrksrZr167q1q2bJKmgoEALFixwWdu5c2f17t1bkmSz2TR37lyXtR07dtSDDz4oSSouLtbMmTNd1rZv314DBgyQdGnnnJ6e7rI2NTVVgwYNcixfq5ZtxCVsI/6FbcQlNbGNcAennwAAgClYjGsdXzSZwsJCWa1WFRQUKCYmptLjHFquupZDyxxa5vST57VsI6pXyzbixmprw+feF9uIy/tvm81W5f77sjoZaq73RwEAALWHu/tvTj8BAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTCPF3AzXp8g3JCwsL/dwJAABw1+X99uX9uCt1KtQUFRVJkpKTk/3cCQAA8FRRUZGsVqvLxy3G9WKPidjtduXm5io6OloWi0XSpfSXnJyskydPKiYmxs8d4loYq8DAOAUOxipw1PWxMgxDRUVFSkpKUlCQ65kzdepITVBQkJo1a1blYzExMXXyP5RAxFgFBsYpcDBWgaMuj9W1jtBcxkRhAABgCoQaAABgCnU+1ISHhystLU3h4eH+bgXXwVgFBsYpcDBWgYOxck+dmigMAADMq84fqQEAAOZAqAEAAKZAqAEAAKZAqAEAAKZgylCzdetW9evXT0lJSbJYLFq7dq3T4+fPn9e4cePUrFkzRUZG6tZbb9WCBQucakpKSjR+/HjFxcUpKipKDz/8sE6dOlWDv4X5paen6+6771Z0dLTi4+M1YMAAfffdd041hmHolVdeUVJSkiIjI9WtWzf97W9/c6phrHzvemNVVlam3/zmN7rtttsUFRWlpKQkPfnkk8rNzXV6HcbK99z5XF3pmWeekcVi0Zw5c5zWM1a+5+5YHTlyRA8//LCsVquio6P17//+78rJyXE8zlj9iylDzYULF9S+fXu9/fbbVT4+efJkrV+/Xh988IGOHDmiyZMna/z48frkk08cNZMmTdKaNWu0YsUKbd++XefPn9dDDz2kioqKmvo1TC8jI0PPPvusdu7cqY0bN6q8vFy9e/fWhQsXHDVvvvmmZs+erbffflvffvutEhIS1KtXL8d9vCTGqiZcb6yKi4uVmZmpl156SZmZmVq9erWOHj2qhx9+2Ol1GCvfc+dzddnatWu1a9cuJSUlVXqMsfI9d8bq+PHj6tKli2655RZ99dVX2r9/v1566SVFREQ4ahirKxgmJ8lYs2aN07q2bdsar732mtO6u+66y/jtb39rGIZh/POf/zRCQ0ONFStWOB4/ffq0ERQUZKxfv97nPddV+fn5hiQjIyPDMAzDsNvtRkJCgvH66687ai5evGhYrVbj3XffNQyDsfKXq8eqKt98840hyThx4oRhGIyVv7gaq1OnThlNmzY1Dh06ZKSkpBh/+MMfHI8xVv5R1VgNHjzYeOKJJ1w+h7FyZsojNdfTpUsXffrppzp9+rQMw9CWLVt09OhR/fKXv5Qk7dmzR2VlZerdu7fjOUlJSWrXrp3++te/+qtt07PZbJKk2NhYSVJ2drbOnDnjNA7h4eHq2rWrYxwYK/+4eqxc1VgsFjVo0EASY+UvVY2V3W7XsGHDNHXqVLVt27bScxgr/7h6rOx2u/785z+rdevW+uUvf6n4+Hjdc889TlMqGCtndTLUzJs3T6mpqWrWrJnCwsLUp08fzZ8/X126dJEknTlzRmFhYWrYsKHT85o0aaIzZ874o2XTMwxDU6ZMUZcuXdSuXTtJcvytmzRp4lR75TgwVjWvqrG62sWLF/Xiiy9qyJAhjpvvMVY1z9VYvfHGGwoJCdGECROqfB5jVfOqGqv8/HydP39er7/+uvr06aMvvvhCAwcO1COPPKKMjAxJjNXV6tRdui+bN2+edu7cqU8//VQpKSnaunWrxo4dq8TERPXs2dPl8wzDkMViqcFO645x48bpwIED2r59e6XHrv6buzMOjJXvXGuspEuThn/1q1/Jbrdr/vz51309xsp3qhqrPXv2aO7cucrMzPT4785Y+U5VY2W32yVJ/fv31+TJkyVJd9xxh/7617/q3XffVdeuXV2+Xl0dqzp3pObnn3/W9OnTNXv2bPXr10+33367xo0bp8GDB2vmzJmSpISEBJWWluqnn35yem5+fn6lowa4cePHj9enn36qLVu2qFmzZo71CQkJklTp/zauHAfGqma5GqvLysrK9J//+Z/Kzs7Wxo0bHUdpJMaqprkaq23btik/P1/NmzdXSEiIQkJCdOLECT333HNq0aKFJMaqprkaq7i4OIWEhCg1NdWp/tZbb3V8+4mxclbnQk1ZWZnKysoUFOT8qwcHBztScYcOHRQaGqqNGzc6Hs/Ly9OhQ4d077331mi/ZmYYhsaNG6fVq1fryy+/VMuWLZ0eb9mypRISEpzGobS0VBkZGY5xYKxqxvXGSvpXoDl27Jg2bdqkRo0aOT3OWNWM643VsGHDdODAAe3bt8/xk5SUpKlTp2rDhg2SGKuacr2xCgsL0913313pa95Hjx5VSkqKJMaqEj9MTva5oqIiY+/evcbevXsNScbs2bONvXv3Or6F0bVrV6Nt27bGli1bjO+//95YsmSJERERYcyfP9/xGqNHjzaaNWtmbNq0ycjMzDR69OhhtG/f3igvL/fXr2U6Y8aMMaxWq/HVV18ZeXl5jp/i4mJHzeuvv25YrVZj9erVxsGDB43HH3/cSExMNAoLCx01jJXvXW+sysrKjIcfftho1qyZsW/fPqeakpISx+swVr7nzufqald/+8kwGKua4M5YrV692ggNDTXee+8949ixY8Zbb71lBAcHG9u2bXPUMFb/YspQs2XLFkNSpZ/hw4cbhmEYeXl5xogRI4ykpCQjIiLCaNOmjTFr1izDbrc7XuPnn382xo0bZ8TGxhqRkZHGQw89ZOTk5PjpNzKnqsZIkrFkyRJHjd1uN9LS0oyEhAQjPDzc+MUvfmEcPHjQ6XUYK9+73lhlZ2e7rNmyZYvjdRgr33Pnc3W1qkINY+V77o7VH//4R6NVq1ZGRESE0b59e2Pt2rVOjzNW/2IxDMPw7bEgAAAA36tzc2oAAIA5EWoAAIApEGoAAIApEGoAAIApEGoAAIApEGoAAIApEGoAAIApEGoAAIApEGoAAIApEGoAAIApEGoA1AoVFRWy2+3+bsMjpaWl/m4BwBUINQAqWb9+vbp06aIGDRqoUaNGeuihh3T8+HHH4507d9aLL77o9JyCggKFhoZqy5Ytki7t8F944QU1bdpUUVFRuueee/TVV1856pcuXaoGDRpo3bp1Sk1NVXh4uE6cOKFvv/1WvXr1UlxcnKxWq7p27arMzEyn9/r73/+uLl26KCIiQqmpqdq0aZMsFovWrl3rqDl9+rQGDx6shg0bqlGjRurfv79++OEHl79zRUWFRo4cqZYtWyoyMlJt2rTR3LlznWpGjBihAQMGKD09XUlJSWrdurVb7+XO7wTgxhFqAFRy4cIFTZkyRd9++602b96soKAgDRw40HEkZejQoVq+fLmuvB/uxx9/rCZNmqhr166SpKeeeko7duzQihUrdODAAQ0aNEh9+vTRsWPHHM8pLi5Wenq6Fi1apL/97W+Kj49XUVGRhg8frm3btmnnzp26+eab9cADD6ioqEiSZLfbNWDAANWrV0+7du3Se++9p//3//6fU//FxcXq3r276tevr61bt2r79u2qX7+++vTp4/Loit1uV7NmzbRy5UodPnxYL7/8sqZPn66VK1c61W3evFlHjhzRxo0btW7dOrfe63q/EwAv8fNdwgEEgPz8fEOScfDgQcdySEiIsXXrVkdN586djalTpxqGYRhZWVmGxWIxTp8+7fQ6//Ef/2FMmzbNMAzDWLJkiSHJ2Ldv3zXfu7y83IiOjjY+++wzwzAM4/PPPzdCQkKMvLw8R83GjRsNScaaNWsMwzCMP/7xj0abNm0Mu93uqCkpKTEiIyONDRs2uP17jx071nj00Ucdy8OHDzeaNGlilJSUONZV572u/p0AeAdHagBUcvz4cQ0ZMkT/9m//ppiYGLVs2VKSlJOTI0lq3LixevXqpQ8//FCSlJ2dra+//lpDhw6VJGVmZsowDLVu3Vr169d3/GRkZDidxgoLC9Ptt9/u9N75+fkaPXq0WrduLavVKqvVqvPnzzve+7vvvlNycrISEhIcz+nUqZPTa+zZs0dZWVmKjo52vHdsbKwuXrzo9P5Xe/fdd9WxY0c1btxY9evX1/vvv+9438tuu+02hYWFefRe1/udAHhHiL8bAFD79OvXT8nJyXr//feVlJQku92udu3aOZ26GTp0qCZOnKi33npLH330kdq2bav27dtLunQqJzg4WHv27FFwcLDTa9evX9/x78jISFksFqfHR4wYoYKCAs2ZM0cpKSkKDw9X586dHe9tGEal51zNbrerQ4cOjtB1pcaNG1f5nJUrV2ry5MmaNWuWOnfurOjoaM2YMUO7du1yqouKivL4va73OwHwDkINACc//vijjhw5ooULF+r++++XJG3fvr1S3YABA/TMM89o/fr1+uijjzRs2DDHY3feeacqKiqUn5/veA13bdu2TfPnz9cDDzwgSTp58qTOnj3rePyWW25RTk6O/vGPf6hJkyaSLk3EvdJdd92ljz/+WPHx8YqJiXH7fe+9916NHTvWse5aR3U8ea/r/U4AvIPTTwCcXP4Gz3vvvaesrCx9+eWXmjJlSqW6qKgo9e/fXy+99JKOHDmiIUOGOB5r3bq1hg4dqieffFKrV69Wdna2vv32W73xxhv6y1/+cs33b9Wqlf70pz/pyJEj2rVrl4YOHarIyEjH47169dJNN92k4cOH68CBA9qxY4djovDlIzhDhw5VXFyc+vfvr23btik7O1sZGRmaOHGiTp065fJ9d+/erQ0bNujo0aN66aWXKoWlqrjzXtf7nQB4B6EGgJOgoCCtWLFCe/bsUbt27TR58mTNmDGjytqhQ4dq//79uv/++9W8eXOnx5YsWaInn3xSzz33nNq0aaOHH35Yu3btUnJy8jXff/Hixfrpp5905513atiwYZowYYLi4+MdjwcHB2vt2rU6f/687r77bo0aNUq//e1vJUkRERGSpHr16mnr1q1q3ry5HnnkEd1666369a9/rZ9//tnl0ZTRo0frkUce0eDBg3XPPffoxx9/dDpq44o773W93wmAd1gM44rvZAJAANqxY4e6dOmirKws3XTTTf5uB4CfEGoABJw1a9aofv36uvnmm5WVlaWJEyeqYcOGVc79AVB3MFEYQMApKirSCy+8oJMnTyouLk49e/bUrFmz/N0WAD/jSA0AADAFJgoDAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABTINQAAABT+P9OaKKmBGjkHAAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "measurements_cpu_vs_gpu = compare_measurements_from_algorithms(segmentation_1, \n", " segmentation_1_gpu, \n", " folder, \n", " 'area')\n", "bland_altman_plot(measurements_cpu_vs_gpu['area_1'], measurements_cpu_vs_gpu['area_2'], 'area')" ] }, { "cell_type": "markdown", "id": "5440def7-a4f3-47c7-8cfa-2359763bb185", "metadata": {}, "source": [ "In this case, we see the average difference is about 0. Furthermore, the confidence interval is much smaller than compared before." ] }, { "cell_type": "markdown", "id": "fc636fa5-537c-4b24-94c8-d00a8ded1c4c", "metadata": {}, "source": [ "## Exercise\n", "Also compare the second segmentation algorithm with its GPU-variant." ] }, { "cell_type": "code", "execution_count": null, "id": "86642f3e-e22b-46e8-bfe5-24da94fc49e2", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "8278d15a-eb1d-4fa7-a396-3716a22a7aa5", "metadata": {}, "source": [ "## Exercise\n", "\n", "Compare mean intensity measurements of two algorithms were the area seems quite different. Can you pridict how the Bland-Altman plot looks like?" ] }, { "cell_type": "code", "execution_count": null, "id": "053ece57-f468-4ff8-ad7b-dc2648d97e72", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 5 }