Five synthetic users read the new home page, and four of five would buy
The synthetic-user study↗ was run again, against the home page v1.25.1↗ rebuilt from a business partner's HTML. Same five invented readers, same six-step loop, same window sizes, sixty screenshots in total. Both readings are now in one vault, published read-only, and every persona in it carries two runs.
The finding the first study made did not recur. At v1.20.1 the home page never said what the product was called: Agent Behaviour Policy first appeared at character 5,387 of 9,270, and two of five readers took "Buy one, from £5" for an insurance policy costing five pounds. Measured again on the same window:
| v1.20.1 | v1.25.1 | |
|---|---|---|
| where Agent Behaviour Policy first appears | character 5,387 of 9,270 | character 67 of 6,081 |
| bare policy / policies in the page body | 6 above the fold alone | 0 |
| behaviour policy, qualified | — | 12 |
| insurance-register words above the fold | 10 | 0 |
| screens to the four levels, on a phone | — | 8.9 |
The reader who works in insurance understood on the first screen what took her three pages before. The reader on a phone, who stalled last time without ever learning what was for sale, reached a purchase decision. Four of five would now spend money, against two of five — which is not a sale, and two of them said the entry level is cheap enough not to be a decision.
Seven findings replaced it, and five were fixed the same day.
- The bare word, where it survived. Eighteen vault pages said Buy this policy on their primary
button and for-corporate↗ said See a real policy — the exact phrase two
readers took for insurance last time. Both now say behaviour policy. The first is fixed in
build-abp-pages.mjs, which writes it. - One unhedged sentence about somebody else's industry. "What earns cover back is a written record…" states as settled a thing only an underwriter decides. The insurance page↗ already says it conditionally, and that is now the sentence the home page carries.
- Two dated 2026 facts with no source. The insurance page gives the exclusion's announcement date and says why the figures beside it are qualitative; the home page stated both facts bare. It now points at the sourced version and says neither is our observation.
- The line the broker asked for. We are not an insurer and place no cover, beside the two admissions the page already volunteers.
Two were not fixed and are recorded instead. The vocabulary is split across pages — the home page says reach and gap where the behaviour-policy page↗ still says the grant and the delta in its own prose, and three readers hit it within two minutes. And every delivery window on levels 3 and 4 starts from the buyer's reply, so the step with no time on it is ours; the reader who would have paid £1,500 the same day could not.
A tooling bug found while doing this, and fixed. release.mjs replaced every >vX.Y.Z</a> on
every page, not just the version chip in the header — so a sentence on the study page saying a bug
was fixed in v1.20.2 was carried forward release by release until it claimed v1.25.1. It now
matches the chip only, and the sentence says v1.20.2 again. A page that names a release in its own
text is a dated record.
Two corrections to the runner itself, both of which would have quietly falsified the study: the
page grows while its webfonts settle, so a scroll issued too early lands short; and the site sets
scroll-behavior: smooth, so a screenshot 280ms after scrollTo catches the page mid-flight — one
step aimed at 4,340px landed at 3,308px and showed a different section. Every step now records the
scroll position it actually reached.