Generating code for processing images

In case chatGPT’s knowledge base contains solutions for tasks we ask for, such as image analysis tasks, it can create quite sophisticated image analysis scripts.

We define a helper function for making prompts first.

import openai
import matplotlib.pyplot as plt

def prompt(message:str, model="gpt-3.5-turbo"):
    """A prompt helper function that sends a message to openAI
    and returns only the text response.
    """
    response = openai.ChatCompletion.create(
        model=model,
        messages=[{"role": "user", "content": message}]
    )
    return response['choices'][0]['message']['content']

A simple task could be described like in the following. We explictly specify that this should execute from Jupyter to prevent windows popping up.

simple_question = """
Write Python code only and no additional explanatory text.

Write a python program, that 
* loads the file `../../data/blobs.tif`,
* labels objects in this image, and
* visualize results.

Assume this program would be executed in a Jupyter notebook.
It is not necessary to save the results. Show the results in Jupyter.
"""

The generated code looks like this.

code = prompt(simple_question)
print(code)

```python
from skimage.io import imread
from skimage.filters import threshold_otsu
from skimage.measure import label
from skimage.color import label2rgb
import matplotlib.pyplot as plt

image = imread('../../data/blobs.tif')
thresh = threshold_otsu(image)
binary = image > thresh
labels = label(binary)
image_label_overlay = label2rgb(labels, image=image)

fig, ax = plt.subplots(figsize=(10, 6))
ax.imshow(image_label_overlay)

plt.show()
```

This code needs a little cleaning, before we can execute it.

cleaner_code = code.replace("```python", "").replace("```", "")

We now excute the code. Note in more advanced scenarios there is a risk when executing generated code. We could delete files unintentionally for example.

exec(cleaner_code)

Exercises

• Rerun the code above. Is the output the same? If not, why?

• Modify the code above so that the output of the script is the number of blobs in the image.