Skip to content
Fondazione ITS INCOM

Examples · Cloud & Cybersecurity

An incident report that invents nothing

From the on-call notes to a report for the manager. Every sentence says which notes it comes from, your code puts the two side by side and marks every number that is not in the notes, and five minutes of reading find what the model added.

Intermediate · 40 min

The idea

After an attack, someone has to tell the manager what happened, the same morning, in plain words. The technician on call has notes: short, with times and addresses. A model turns them into a report in seconds.

The danger is that a model fills the gaps: a word that changes the meaning, a number it worked out, a reassuring sentence that nobody wrote. In a report about an attack, every sentence has to be true. So:

  1. the model writes the report as JSON: each sentence with its section and the times of the notes it comes from;
  2. your code writes report.md, then prints every sentence next to its notes, and marks every number that is not in them;
  3. you read the pairs.

What you need

A key and the openai library, as in Your first API call in Python.

The model

Qwen3.6 35B (qwen3.6-35b), strong on office work. In our runs it wrote one fact per sentence, each with its own notes: that is what makes the check quick. Gemma and Apertus often put a whole section into one long sentence, with up to a dozen notes behind it: you cannot check that line by line.

The notes

The morning of Attacks in an access log, as the technician wrote it down. Save it as notes.txt:

Web shop, on-call notes, 6 October 2026

09:12 monitoring alert: 14 failed logins from 192.0.2.150 in under a minute, then a successful one
09:15 read access.log. 192.0.2.150 downloaded /account/orders/export.csv right after logging in. Also: SQL injection from 192.0.2.77 at 09:02 (a 500, then 200 with 48 KB on /products?id=12 OR 1=1). Scanner 203.0.113.45 got /.git/config and /.git/HEAD with 200
09:18 blocked 192.0.2.150, 192.0.2.77, 203.0.113.45 on the firewall
09:20 phoned the shop manager
09:24 app log: the account taken over is customer 4471. Password reset, all sessions closed
09:31 export.csv of customer 4471: 212 orders with names, delivery addresses and last 4 digits of the card. No full card numbers: payments go through the payment provider
09:40 products.php puts the id straight into the SQL query. Products page offline, maintenance page on
09:55 .git removed from the web root. nginx now refuses hidden files
10:42 products.php fixed with a prepared statement by the developer, products page back online
11:00 NOT KNOWN: what the OR 1=1 and the UNION SELECT returned. The database keeps its query log for 1 hour: gone
11:10 password hashes are bcrypt (checked in the code)
11:20 manager: wait for the DPO before telling the customers

The program

Save it as report.py, next to the notes:

import json
import os
import re
import sys

from openai import OpenAI

client = OpenAI(
    base_url="https://api.ai.itsincom.org/v1",
    api_key=os.environ["API_KEY"],
)

with open("notes.txt") as f:
    notes = f.read()

# Every note starts with its time: the times are the sources a sentence can point at.
lines = dict(re.findall(r"^(\d\d:\d\d) (.*)$", notes, re.M))
title = notes.splitlines()[0]  # the first line, with the date, belongs to every sentence
SECTIONS = ["Summary", "Timeline", "Impact", "Actions taken", "Not known yet", "Next steps"]

schema = {
    "type": "object",
    "properties": {
        "sentences": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "section": {"type": "string", "enum": SECTIONS},
                    "text": {"type": "string"},
                    "sources": {"type": "array", "items": {"type": "string", "enum": list(lines)}, "minItems": 1},
                },
                "required": ["section", "text", "sources"],
            },
        },
    },
    "required": ["sentences"],
}

answer = client.chat.completions.create(
    model=sys.argv[1] if len(sys.argv) > 1 else "qwen3.6-35b",  # python3 report.py gemma-4-26b: another model
    temperature=0,
    messages=[
        {"role": "system", "content": "You write incident reports for the manager of a small company, from the "
                                      "notes of the technician on call. Use only the facts in the notes: what "
                                      "the notes do not say, you do not write. For every sentence, put the times "
                                      "of the notes it comes from in sources, not in the text. Sections: "
                                      + ", ".join(SECTIONS) + ". Plain English, short sentences."},
        {"role": "user", "content": notes},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "report", "schema": schema}},
)
sentences = json.loads(answer.choices[0].message.content)["sentences"]

# 1. The report, ready to send
with open("report.md", "w") as f:
    for section in SECTIONS:
        f.write(f"## {section}\n\n")
        for s in sentences:
            if s["section"] == section:
                f.write(f"- {s['text']}\n")
        f.write("\n")
print("report.md is ready. Now check it:\n")

# 2. Every sentence next to its sources. A number that is not in its sources is marked.
NUMBER = r"\d+(?:[.:]\d+)?"  # 14, 4471, 1.5, and times such as 09:12
for s in sentences:
    print(f"[{s['section']}] {s['text']}")
    sources = title + " " + " ".join(f"{t} {lines[t]}" for t in s["sources"])
    for number in re.findall(NUMBER, s["text"]):
        if number not in re.findall(NUMBER, sources):
            print(f"    !! {number} is not in its sources")
    for t in s["sources"]:
        print(f"    {t} {lines[t]}")

Run it with python3 report.py. It takes about 20 seconds.

What you should see

report.md, with the six sections, ready to send. And on the screen, every sentence with its notes underneath (here, a few of the 29):

report.md is ready. Now check it:

[Summary] Customer 4471’s account was compromised, and their order export containing personal and payment details was downloaded.
    09:24 app log: the account taken over is customer 4471. Password reset, all sessions closed
    09:31 export.csv of customer 4471: 212 orders with names, delivery addresses and last 4 digits of the card. No full card numbers: payments go through the payment provider
[Timeline] Logs also recorded an SQL injection attempt from IP 192.0.2.77 at 09:02, which returned 48 KB of data.
    09:15 read access.log. 192.0.2.150 downloaded /account/orders/export.csv right after logging in. Also: SQL injection from 192.0.2.77 at 09:02 (a 500, then 200 with 48 KB on /products?id=12 OR 1=1). Scanner 203.0.113.45 got /.git/config and /.git/HEAD with 200
[Impact] The products page was offline for approximately 1 hour and 2 minutes.
    !! 1 is not in its sources
    !! 2 is not in its sources
    09:40 products.php puts the id straight into the SQL query. Products page offline, maintenance page on
    10:42 products.php fixed with a prepared statement by the developer, products page back online
[Not known yet] The specific data returned by the SQL injection queries (OR 1=1 and UNION SELECT) is unknown.
    11:00 NOT KNOWN: what the OR 1=1 and the UNION SELECT returned. The database keeps its query log for 1 hour: gone

We got the same 29 sentences in each of our five runs.

Read the pairs

What we found in ours:

Correct the sentences in report.md by hand: it is a text file.

Good to know

Exercise

All examples