Pixel classification on OpenCL-compatible GPUs

Alternatively to object or “instance” segmentation, we can also do semantic segmentation by classifying pixels.

Let’s start with loading an example image and some ground truth:

from skimage.io import imread
import matplotlib.pyplot as plt
import numpy as np
import apoc
import pyclesperanto_prototype as cle

image = imread('../../data/blobs.tif')
cle.imshow(image)

manual_annotations = imread('../../data/blobs_annotations.tif')

from skimage.io import imshow
cle.imshow(manual_annotations, labels=True)

Training

We now train a PixelClassifier, which is under the hood a scikit-learn RandomForestClassifier. After training, the classifier will be converted to clij-compatible OpenCL code and save to disk under a given filename.

# define features: original image, a blurred version and an edge image
features = "original gaussian_blur=2 sobel_of_gaussian_blur=2"

# this is where the model will be saved
cl_filename = 'my_model.cl'

apoc.erase_classifier(cl_filename)
clf = apoc.PixelClassifier(opencl_filename=cl_filename)
clf.train(features, manual_annotations, image)

Prediction

The classifier can then be used to classify all pixels in the given image. Starting point is again, the feature stack. Thus, the user must make sure that the same features are used for training and for prediction.

result = clf.predict(image=image)
cle.imshow(result, labels=True)