Skip to content
Fondazione ITS INCOM

Examples · Web Development

Alt text for your images

The images of a web page go to a vision model, and an alt text comes back for each one. Your code checks the length, the words a screen reader does not need, and every number of a chart against the data behind it; only what passes goes into the page.

Beginner · 30 min

The idea

Every <img> needs an alt text. A screen reader reads it to a blind visitor, and the browser shows it when the image does not load. Writing it for every image is boring, so it is often skipped, or it says image1.png.

A model that reads images can write a first draft. A draft can sound right and still be wrong, and nobody listens to the alt texts of their own site. So:

  1. a vision model writes the alt text of each image, with the words of the page around it;
  2. your code checks it: at most 125 characters (a common rule of thumb), no “image of”, and for the chart, every number must be one of the real ones;
  3. only the alt texts that pass go into the page.

What you need

A key and the openai library, as in Your first API call in Python. To draw the test images, Pillow 10.1 or newer: pip install pillow.

Two models read images: gemma-4-26b and qwen3.6-35b. We tried both.

The page and its images

The page of a bike repair shop in Legnano (the shop is invented). Save it as page.html. Its three images have no alt text:

<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>Six months of Ciclofficina Brambilla</title></head>
<body>
  <header><a href="/"><img src="logo.png"></a></header>
  <h1>Six months in Legnano</h1>
  <p>We opened our bike repair shop in January. Spring brought the bikes out of the cellars:</p>
  <img src="chart.png">
  <p>From today you can book a repair online. Choose a day, write your name, and you are done:</p>
  <img src="booking.png">
</body>
</html>

Save this as make_images.py, and run it with python3 make_images.py. It draws the same three images we used:

from PIL import Image, ImageDraw, ImageFont


def font(size):
    return ImageFont.load_default(size=size)  # a font that comes with Pillow 10.1 or newer


# 1. The logo: a bicycle and the name of the shop
img = Image.new("RGB", (480, 160), "white")
d = ImageDraw.Draw(img)
d.ellipse((20, 60, 90, 130), outline="#c0392b", width=6)    # back wheel
d.ellipse((110, 60, 180, 130), outline="#c0392b", width=6)  # front wheel
d.line([(55, 95), (95, 95), (130, 55), (75, 55), (55, 95)], fill="#2c3e50", width=6)  # frame
d.line([(95, 95), (75, 45)], fill="#2c3e50", width=6)       # seat tube
d.line([(130, 55), (145, 95)], fill="#2c3e50", width=6)     # fork
d.text((200, 45), "Ciclofficina", font=font(36), fill="#2c3e50")
d.text((200, 90), "BRAMBILLA", font=font(36), fill="#c0392b")
img.save("logo.png")

# 2. The chart: bikes repaired each month, with a scale and no numbers on the bars
LABELS = False  # True writes the number on top of each bar
repairs = {"Jan": 42, "Feb": 38, "Mar": 65, "Apr": 91, "May": 118, "Jun": 104}
img = Image.new("RGB", (640, 400), "white")
d = ImageDraw.Draw(img)
d.text((20, 15), "Bikes repaired in 2026", font=font(28), fill="black")
for value in (0, 50, 100):
    y = 350 - value * 2
    d.line([(60, y), (620, y)], fill="#cccccc" if value else "black", width=1 if value else 2)
    d.text((15, y - 12), str(value), font=font(20), fill="black")
for i, (month, count) in enumerate(repairs.items()):
    x = 80 + i * 90
    d.rectangle((x, 350 - count * 2, x + 60, 350), fill="#2e86c1")
    d.text((x + 10, 360), month, font=font(22), fill="black")
    if LABELS:
        d.text((x + 12, 350 - count * 2 - 28), str(count), font=font(22), fill="black")
img.save("chart.png")

# 3. A screenshot of the booking page, with its text
img = Image.new("RGB", (640, 360), "#f4f6f7")
d = ImageDraw.Draw(img)
d.rectangle((0, 0, 640, 40), fill="#dfe3e6")                 # the browser bar
d.text((20, 10), "localhost:8000/book", font=font(18), fill="#555555")
d.text((40, 60), "Book a repair", font=font(32), fill="black")
for y, label, value in [(120, "Name", "Giulia Ferrario"), (190, "Day", "14/10/2026")]:
    d.text((40, y), label, font=font(18), fill="#333333")
    d.rectangle((40, y + 25, 400, y + 60), outline="#999999", fill="white")
    d.text((50, y + 33), value, font=font(20), fill="black")
d.rounded_rectangle((40, 290, 220, 335), radius=8, fill="#27ae60")
d.text((70, 300), "Book now", font=font(22), fill="white")
img.save("booking.png")
print("logo.png, chart.png and booking.png are ready")

Open chart.png: it has a scale (0, 50, 100) and no numbers on the bars, like many charts on the web.

The program

Save it as alt_text.py, in the same folder:

import base64
import json
import os
import re
import sys

from openai import OpenAI

client = OpenAI(
    base_url="https://api.ai.itsincom.org/v1",
    api_key=os.environ["API_KEY"],
)
MODEL = sys.argv[1] if len(sys.argv) > 1 else "gemma-4-26b"

# The numbers behind the chart: your code has them, the model only sees the picture.
DATA = {"chart.png": {"42", "38", "65", "91", "118", "104", "2026"}}

page = open("page.html").read()
page_text = " ".join(re.sub(r"<[^>]+>", " ", page).split())  # the words of the page, for context

schema = {
    "type": "object",
    "properties": {"alt": {"type": "string"}},
    "required": ["alt"],
}


def write_alt(src):
    with open(src, "rb") as f:
        data_url = "data:image/png;base64," + base64.b64encode(f.read()).decode()
    answer = client.chat.completions.create(
        model=MODEL,
        temperature=0,
        messages=[
            {"role": "system", "content": "You write the alt text of images for a web page. At most 125 "
                                          "characters. Say what the image tells a reader of this page; do "
                                          "not start with «image of» or «picture of». For a logo, the name. "
                                          "For text in the image, the text that matters. For a chart, the "
                                          "main message with the numbers that support it."},
            {"role": "user", "content": [
                {"type": "text", "text": f"The page says: {page_text}\n\nWrite the alt text for {src}."},
                {"type": "image_url", "image_url": {"url": data_url}},
            ]},
        ],
        response_format={"type": "json_schema", "json_schema": {"name": "alt_text", "schema": schema}},
    )
    if answer.choices[0].finish_reason != "stop":
        raise SystemExit(f"The answer for {src} was cut off.")
    return json.loads(answer.choices[0].message.content)["alt"].strip()


def problems(src, alt):
    found = []
    if len(alt) > 125:
        found.append(f"{len(alt)} characters, more than 125")
    for words in ["image of", "picture of", "photo of"]:
        if words in alt.lower():
            found.append(f"says «{words}»: a screen reader already says «image»")
    for number in re.findall(r"\d+(?:\.\d+)?", alt):
        if src in DATA and number not in DATA[src]:
            found.append(f"{number} is not in the data of the chart")
    return found


for src in re.findall(r'<img src="([^"]+)">', page):
    alt = write_alt(src)
    found = problems(src, alt)
    print(f"{src} ({len(alt)} characters): {alt}")
    for problem in found:
        print(f"    NO: {problem}")
    if not found:
        # Only an alt text that passed the checks goes into the page.
        page = page.replace(f'<img src="{src}">', f'<img src="{src}" alt="{alt.replace(chr(34), "&quot;")}">')

with open("page-with-alt.html", "w") as f:
    f.write(page)
print("page-with-alt.html is ready")

Run it with python3 alt_text.py for Gemma, or python3 alt_text.py qwen3.6-35b. It takes a few seconds.

What you should see

Gemma:

logo.png (22 characters): Ciclofficina Brambilla
chart.png (95 characters): Bar chart showing bikes repaired in 2026: Jan 43, Feb 39, Mar 66, Apr 91, May 118, and Jun 104.
    NO: 43 is not in the data of the chart
    NO: 39 is not in the data of the chart
    NO: 66 is not in the data of the chart
booking.png (87 characters): Online booking form for a bike repair with fields for Name, Day, and a Book now button.
page-with-alt.html is ready

The logo and the form pass, and go into page-with-alt.html. Now look at the chart. The sentence reads well, and three of its six numbers are wrong: the bars are 42, 38 and 65, Gemma read 43, 39 and 66 from the scale. A person checking the page would hardly notice. The check does, because the program has the real data.

Qwen, on the same page:

logo.png (27 characters): Ciclofficina Brambilla logo
chart.png (115 characters): Bar chart showing bikes repaired monthly from January to June 2026, with May having the highest number at over 100.
    NO: 100 is not in the data of the chart
booking.png (201 characters): A screenshot of a web form titled 'Book a repair'. The form has fields for 'Name' and 'Day', with 'Giulia Ferrario' and '14/10/2026' filled in, respectively. A green 'Book now' button is at the bottom.
    NO: 201 characters, more than 125
page-with-alt.html is ready

Here only the logo passes.

What the checks caught

Now put the numbers on the bars: in make_images.py write LABELS = True, and run both programs again. In our runs, six per model, every number both models wrote was a real one. A chart that shows its numbers is better for every visitor, and for the model too.

Why a failed alt text stays empty

An image with no alt is an error that the usual checkers find: the W3C validator, Lighthouse, axe. An image with a wrong alt passes all of them. So the program writes only what passed, and leaves the rest for you to write.

Good to know

Exercise

All examples