Skip to content
Fondazione ITS INCOM

Examples · Web Development

Tests for your JavaScript

A price formatter for an Italian shop hides two bugs. Qwen3-Coder or Qwen3.8 writes tests with node:test, and you run them with node --test. A failing test shows a bug, if the model tested the right cases. For each failure, you decide whether the function or the test is wrong.

Intermediate · 40 min

The idea

A test is code that checks code. Writing tests is slow, and a model writes them in seconds. But a test finds a bug only if one of its cases runs into it, and only if its expected value is right. So the model proposes the tests, and running them decides:

  1. a function with two hidden bugs;
  2. a model writes tests for it;
  3. node --test runs them, and you read every failure: is the function wrong, or the test?

What you need

Node.js 22, the openai package and a key in API_KEY, as in Your first API call in JavaScript. Nothing else: node:test and node:assert come with Node, with no Jest and no install.

The function

Save it as price.mjs. It formats a price in cents for an Italian page. It has two bugs: do not look for them yet.

/**
 * Formats a price for an Italian web page: the total of an order, or a refund.
 *
 * cents: a whole number of euro cents, for example 199 for 1,99 euro. Negative for a refund.
 * Returns the euros with a dot between each group of three digits, a comma before the cents,
 * always two decimals, then a normal space and the euro sign:
 *   formatPrice(199)    → "1,99 €"
 *   formatPrice(123456) → "1.234,56 €"
 *   formatPrice(-500)   → "-5,00 €"
 * Throws a TypeError if cents is not a whole number: 1.5, NaN, "199".
 */
export function formatPrice(cents) {
  if (!Number.isInteger(cents)) {
    throw new TypeError("cents must be a whole number");
  }
  const euros = Math.floor(cents / 100);
  const rest = String(cents % 100).padStart(2, "0");
  let text = String(euros);
  if (text.length > 3) {
    text = text.slice(0, -3) + "." + text.slice(-3);
  }
  return `${text},${rest} €`;
}

The comment is the promise: what the function must do. The three examples in it are right, and the function passes them.

The program

Save it as write_tests.mjs, next to price.mjs. It sends the file to a model and saves the tests it gets back.

import { readFile, writeFile } from "node:fs/promises";
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.ai.itsincom.org/v1",
  apiKey: process.env.API_KEY,
});

const model = process.argv[2] ?? "qwen3-coder-30b";
const code = await readFile("price.mjs", "utf8");
// With "promise", the model sees the comment and the first line, not the body of the function.
const shown = process.argv[3] === "promise" ? code.slice(0, code.indexOf("{")) + "{ /* not shown */ }" : code;

const answer = await client.chat.completions.create({
  model,
  messages: [
    { role: "system", content: "You write tests for JavaScript. Answer with one ```javascript block only." },
    {
      role: "user",
      content:
        "Write tests for this module with node:test and node:assert/strict. It is saved as price.mjs, " +
        'so the tests start with `import { formatPrice } from "./price.mjs";`. Test everything the ' +
        "comment promises, the edge cases included. Do not change the function.\n\n" + shown,
    },
  ],
});

const choice = answer.choices[0];
if (choice.finish_reason !== "stop") {
  throw new Error(`The answer was cut (${choice.finish_reason}): run it again.`);
}
const block = choice.message.content.match(/```(?:javascript|js)\s*\n([\s\S]*?)```/);
if (!block) {
  throw new Error("There is no ```javascript block in the answer: run it again.");
}
await writeFile("price.test.mjs", block[1]);
console.log(`price.test.mjs written by ${model}, ${block[1].split("\n").length} lines`);

Then run the tests:

node write_tests.mjs qwen3-coder-30b
node --test

node --test with no file name finds the files whose names end in .test.mjs (and a few other patterns), so it finds price.test.mjs.

What you should see

The tests change from run to run. In one of ours, two of Qwen3-Coder's four tests failed:

price.test.mjs written by qwen3-coder-30b, 42 lines
✔ formatPrice formats positive prices correctly (1.221345ms)
✖ formatPrice formats negative prices correctly (1.283915ms)
✔ formatPrice throws TypeError for non-integer inputs (1.170683ms)
✖ formatPrice handles edge cases correctly (1.782676ms)
ℹ tests 4
ℹ suites 0
ℹ pass 2
ℹ fail 2
…
✖ formatPrice formats negative prices correctly (1.283915ms)
  AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
  + actual - expected

  + '-2,-99 €'
  - '-1,99 €'
…
✖ formatPrice handles edge cases correctly (1.782676ms)
  AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
  + actual - expected

  + '1234.567,89 €'
  - '1.234.567,89 €'

actual is what the function returned; expected is what the test wanted. The two lines that failed:

assert.strictEqual(formatPrice(-199), "-1,99 €");
assert.strictEqual(formatPrice(123456789), "1.234.567,89 €");

Who is wrong, the function or the test?

Work out each expected value by hand, from the comment, not from the code:

A test can be wrong too. In two of our runs a test expected formatPrice(1000000) to be “1.000.000,00 €” or formatPrice(123456789) to be “123.456.789,00 €”: the model took cents for euros. 1,000,000 cents are 10.000,00 €. If you had “fixed” the function to pass that test, you would have broken it.

What the two models caught

We asked each model for tests several times, and ran them on four versions of the function: the one above, one with only the first bug, one with only the second, and one with neither. A test catches a bug if it fails on the version that has only that bug; a test is wrong if it fails on the version with no bug.

Model, what it saw Runs Negative prices Millions Wrong tests Seconds
qwen3-coder-30b, the whole file 4 4 of 4 3 of 4 none 4–5
qwen3.8-27b, the whole file 3 0 of 3 0 of 3 in 1 run 41–74
qwen3-coder-30b, only the comment 3 3 of 3 2 of 3 in 1 run 4–5
qwen3.8-27b, only the comment 3 3 of 3 3 of 3 in 1 run 17–35

Qwen3.8 wrote the most tests, 18 to 27 with the whole file, and they caught nothing. It thinks before it answers, and it had read the function: its tests describe what the code does, not what the comment promises. Two runs tested negative prices only with round amounts, the only negative amounts the function gets right. One even gave them a heading:

// Negative multiples of 100
test("formatPrice(-100) → '-1,00 €'", () => {
  assert.equal(formatPrice(-100), "-1,00 €");
});

Another run wrote the bug into a test, “'1234.567,89 €' (one dot)”, with an input that does not even give that number: it failed on every version of the function, for the wrong reason.

A test written from the code checks that the code does what the code does. All 25 tests of the first run passed, on a function with two bugs. A green result means the tests passed, not that the function is right.

Tests from the promise, not from the code

Give the model only the promise: the comment and the first line of the function, with the body hidden.

node write_tests.mjs qwen3.8-27b promise
node --test

With only the comment, Qwen3.8 caught the negative prices in 3 runs out of 3 and the millions in 3 out of 3, with tests such as formatPrice(-1) → “-0,01 €” and formatPrice(-123456789) → “-1.234.567,89 €”. It could not copy the function, so it had to follow the promise. Qwen3-Coder caught the negative prices in every run either way.

The two wrong tests of this mode are the cents-for-euros ones above. Both sat at the end of a test that also caught the millions bug, so you see them only after you fix the function: the first failed assert in a test hides the ones after it.

Solution

  1. Negative amounts. Math.floor(-1.99) is -2, not -1, and -199 % 100 is -99, so the cents get a minus sign of their own. Work on the positive number, and put the sign back at the end.
  2. Millions. The dot goes in once, before the last three digits. A regular expression puts one before every group of three.
export function formatPrice(cents) {
  if (!Number.isInteger(cents)) {
    throw new TypeError("cents must be a whole number");
  }
  const sign = cents < 0 ? "-" : "";
  const abs = Math.abs(cents); // work on the positive number, put the sign back at the end
  const euros = Math.floor(abs / 100);
  const rest = String(abs % 100).padStart(2, "0");
  const text = String(euros).replace(/\B(?=(\d{3})+(?!\d))/g, "."); // a dot before every group of three
  return `${sign}${text},${rest} €`;
}

With the comment on top, as before. Run the same tests again:

ℹ tests 4
ℹ pass 4
ℹ fail 0

We ran every test file from our runs on this version: all passed, except the three wrong tests.

Good to know

Exercise

All examples