Whether the terms and the score a tool gives you today are the ones it gives you tomorrow.

The second run is a new analysis in each product, not yesterday’s report reopened. Reopening tests whether the page still renders; it cannot see whether the tool would answer the same question the same way.

Two layers, kept apart. For the term list we lowercase both runs’ words and compare the sets, and we also count the plainer figure: of the words run one recommended, how many were gone by run two. For the score we take the URLs both runs scored and measure the absolute move on each one. A tool’s term list can churn while its scores sit still, which is what happened, so a single stability figure would hide the result.

Both tools with a term list were recomputed from their raw captures under one implementation rather than copied from their own reports, and the recomputed values match what each run published. Frase has no raw capture to recompute, so its row carries its report’s own figures and says so. The threshold below which we call a list changed, 0.7, is set in our method, not by any vendor.

Nothing new was fetched. The captures were already in the repo from the runs themselves, so this cost no quota and no API call. The script is scripts/tests/content-opt/cross/mine_findings.py --finding3, and the report it writes is data/reports/tests/cross-findings-2026-08-25.json.

Two of the three tools publish a list of terms to cover. Neither returned the same list a day later, and neither missed by a consistent amount.

Same two runs, different question: on the URLs both runs scored, how far did the number travel? Much less than the term lists did. The three scores are three different 0–100 definitions, so the comparable thing is each tool’s own gap between its two runs, never the scores themselves.

The obvious objection: these tools read the current top ten, the top ten changes overnight, so of course the output changes. So we pulled out the keywords where the competitor set barely moved — a set overlap of 0.9 or better, several of them identical — and looked only at those.

Read that overlap for what it measures. It compares the two runs’ competitor URLs after normalisation and nothing else — not the order they sat in, not the text on the pages behind them. It rules out one explanation, a different set of reference pages, and leaves the rest standing.

The cleanest row is Surfer SEO on cost of living comparison. The two runs saw exactly the same competitor URLs, and the term list still swapped 20 of its 80 words. The URLs it was pointed at did not change; the output did. The most likely explanation left is what the tool did with those pages between the two runs.

Most likely, not proven. A matching URL set does not guarantee matching input: NeuronWriter’s run records 3 URLs whose second crawl read a body more than three times the length of the first, so the same address can hand a tool a different page. Only NeuronWriter’s run carries that check, so on the other rows we cannot rule the same thing out. What these rows do settle is that Google reshuffling the top ten is not the explanation.

The control sits in the same table, on the same tool — the rows where no more than 5% of the first run’s list dropped out. On ai presentation maker, Surfer SEO swapped 1 word of 79 (0.975), and its score moved by at most 2. On ai content detector, Surfer SEO returned the list word for word (1.000), and its score did not move at all. Same method, same day, same competitor URLs: when a run holds still, this measurement says so. The churn on the rows above is not an artefact of the ruler.

gpu comparison was the worst keyword in both tools: NeuronWriter 0.272, Surfer SEO 0.524. Two products built independently landing on the same keyword points at the SERP behind it rather than at either analyser. Where the ranking pages are near-identical, ordinary crawl noise reshuffles which ones a tool leans on, and the recommended words follow. The steadiest keyword is not shared: NeuronWriter held best on cost of living comparison at 0.802, Surfer SEO held best on ai content detector at 1.000. Whatever a keyword contributes, it does not put the two tools in the same place.

Surfer SEO has the higher mean Jaccard (0.746 against 0.637), yet it has 3 keywords under the threshold to NeuronWriter’s 2. One is split, a couple of keywords near-identical and the rest adrift; the other is middling everywhere. Pick whichever summary you like and you can name a different winner.

The gaps are not equal either. Frase waited 25.0–25.1 hours, NeuronWriter waited 25.9–36.0 hours, Surfer SEO waited 25.6–25.8 hours. Waiting longer gives the SERP more room to move, so part of any difference is the clock. Five keywords, no significance test, unequal intervals: this sample supports a statement about each tool and nothing about the order they belong in.

A term list is not a fact about a keyword. It is what one tool recommended on one day, and the same tool will hand you a different one tomorrow. Whenever you quote one — in a brief, in a report to a client, in an article about which tool covers more — carry the timestamp with it, the way you would with a rank check. Our own term-accuracy numbers on how accurate these term suggestions are follow that rule for the same reason: they score the list a tool gave us that day, and the noise measured here is baked into any comparison between two tools tested on different days.

The scores travel better. Within one tool, a move of a point or two between runs is background, so treat a small gain after an edit as nothing until it clears the tool’s own noise. That does not make the score useful for ranking, which is a separate question with its own answer.

We would rather be corrected than quoted. Any of the following would change what this page says.