Generating code for processing images#

In case chatGPT’s knowledge base contains solutions for tasks we ask for, such as image analysis tasks, it can create quite sophisticated image analysis scripts.

We define a helper function for making prompts first.

import openai
import matplotlib.pyplot as plt

def prompt(message:str, model="gpt-3.5-turbo"):
    """A prompt helper function that sends a message to openAI
    and returns only the text response.
    """
    response = openai.ChatCompletion.create(
        model=model,
        messages=[{"role": "user", "content": message}]
    )
    return response['choices'][0]['message']['content']

A simple task could be described like in the following. We explictly specify that this should execute from Jupyter to prevent windows popping up.

simple_question = """
Write Python code only and no additional explanatory text.

Write a python program, that 
* loads the file `../../data/blobs.tif`,
* labels objects in this image, and
* visualize results.

Assume this program would be executed in a Jupyter notebook.
It is not necessary to save the results. Show the results in Jupyter.
"""

The generated code looks like this.

code = prompt(simple_question)
print(code)
```python
from skimage.io import imread
from skimage.filters import threshold_otsu
from skimage.measure import label
from skimage.color import label2rgb
import matplotlib.pyplot as plt

image = imread('../../data/blobs.tif')
thresh = threshold_otsu(image)
binary = image > thresh
labels = label(binary)
image_label_overlay = label2rgb(labels, image=image)

fig, ax = plt.subplots(figsize=(10, 6))
ax.imshow(image_label_overlay)

plt.show()
```

This code needs a little cleaning, before we can execute it.

cleaner_code = code.replace("```python", "").replace("```", "")

We now excute the code. Note in more advanced scenarios there is a risk when executing generated code. We could delete files unintentionally for example.

exec(cleaner_code)
../_images/cac8a0cdb4e38be53e02abda9f642a1844701a2e1ebda5d837328b852fe18057.png

Exercises#

  • Rerun the code above. Is the output the same? If not, why?

  • Modify the code above so that the output of the script is the number of blobs in the image.