Image segmentation quality measurements

Image segmentation quality measurements#

For determining how good a segmentation algorithm is, and to compare different algorithms, we need a metric. A common metric is the Jaccard Index, which is a measure of overlap between a reference segmentation and the segmentation an algorithm produced for example. If we work with label images, a fair method is to determine the overlap of every annotated object with the most overlapping object in the automatic segmentation. If we average this value over all annotated objects, we receive the Sparse Jaccard Index as defined in The Segmentation Game.

import os
import napari
import napari_segment_blobs_and_things_with_membranes as nsbatwm
from the_segmentation_game import metrics
from skimage.io import imread

To demonstate this, we pick a random example image from the BBBC007 dataset (Jones et al., Proc. ICCV Workshop on Computer Vision for Biomedical Image Applications, 2005), available from the Broad Bioimage Benchmark Collection [Ljosa et al., Nature Methods, 2012].

image_folder = "../../data/BBBC007_batch/"
sparse_annotation_folder = "../../data/BBBC007_sparse_instance_annotation/"

test_image_filename = "17P1_POS0013_D_1UL.tif"

For visualizing the image and the corresponding manual annotation, we use napari.

viewer = napari.Viewer()

The example image#

image = imread(folder + test_image_filename)

viewer.add_image(image)

napari.utils.nbscreenshot(viewer)

The manual annotation#

sparse_labels = imread(sparse_annotation_folder + test_image_filename)

viewer.add_labels(sparse_labels)

napari.utils.nbscreenshot(viewer)

# hide last labels layer
viewer.layers[-1].visible = False

The automatic segmentation#

For demonstration purposes, we use background-subtraction and Voronoi-Otsu-Labeling to segment the nuclei in this image automatically.

def my_segmentation_algorithm(input_image):

    # background subtraction
    background_subtracted = nsbatwm.white_tophat(input_image, radius = 10)
    
    # instance segmenation / labeling
    labels_result = nsbatwm.voronoi_otsu_labeling(background_subtracted, spot_sigma=5, outline_sigma=1)

    return labels_result

labels = my_segmentation_algorithm(image)

viewer.add_labels(labels)

napari.utils.nbscreenshot(viewer)

Quality estimation: Sparse Jaccard Index#

From the two label images loaded and produced above we can compute the sparse Jaccard Index.

metrics.jaccard_index_sparse(sparse_labels, labels)

0.8357392602053431

Exercise#

Use the following for-loop and code snippets from above to compute the segmentation quality of all images in the folder. Provide the average quality over all images.

for image_filename in os.listdir(image_folder):
    print(image_folder + image_filename)

../../data/BBBC007_batch/17P1_POS0013_D_1UL.tif
../../data/BBBC007_batch/20P1_POS0005_D_1UL.tif
../../data/BBBC007_batch/20P1_POS0007_D_1UL.tif
../../data/BBBC007_batch/20P1_POS0010_D_1UL.tif
../../data/BBBC007_batch/A9 p7d.tif
../../data/BBBC007_batch/AS_09125_040701150004_A02f00d0.tif
Contents