---
title: The Page Says Where It Lives. The Fetcher Echoes What You Asked. | NanoParse
url: https://nanoparse.app/blog/2026-11-13-canonical-and-fetch
description: A parse returns the Markdown and twenty signal groups beside it. One of those groups, structured_data, is five fields with four booleans and a single string.
---

A parse returns the Markdown and twenty signal groups beside it. One of those groups, `structured_data`, is five fields with four booleans and a single string. The booleans say whether a page declared something. The string is the page's own claim about where it lives, copied out of a `<link rel="canonical">` tag — and it is the only value in the group that can disagree with the URL that was requested.

That disagreement is measurable, and it is worth measuring twice, because a second group answers the same question: `redirects`. On the same response, for the same URLs, one field reported a change of address and the other reported none.

This post prints the deployed function, isolates each of its rules with 27 fixtures, then runs 16 real URLs through the live endpoint. Every payload, fixture and script is committed beside the post.

## Four booleans and one string

Two real responses from the live endpoint on 2026-11-13. The first is `https://www.nasa.gov/`:

```
"structured_data": { "json_ld": true, "schema_types": ["Organization", "WebSite", "ImageObject", "WebPage", "Person", "Article"], "open_graph": true, "canonical": "https://www.nasa.gov/", "meta_description": true }
```

The second is `https://kubernetes.io/docs/concepts/`, a documentation index:

```
"structured_data": { "json_ld": false, "schema_types": [], "open_graph": true, "canonical": null, "meta_description": false }
```

Both are ordinary pages with different amounts of metadata, and the group reports each without ranking it. `canonical: null` means no canonical tag was found, not that the page has no canonical URL — a documentation site can serve the same page under three paths and say nothing about which one is authoritative.

## The function, printed from the deployed source

`computeStructuredData` is 42 lines of `worker/worker/src/intel.ts`, sliced out by declaration rather than retyped, and run under `node --experimental-strip-types` for every fixture below:

```
export function computeStructuredData(html: string): { json_ld: boolean; schema_types: string[]; open_graph: boolean; canonical: string | null; meta_description: boolean; } { const src = String(html || ""); const schemaTypes = new Set<string>(); let jsonLd = false; const ldRe = /<script[^>]*type=["']application\/ld\+json["'][^>]*>([\s\S]*?)<\/script>/gi; let m: RegExpExecArray | null; while ((m = ldRe.exec(src)) !== null) { try { const data = JSON.parse(m[1].trim()); if (data && typeof data === "object") { jsonLd = true; collectSchemaTypes(data, schemaTypes); } } catch { /* malformed JSON-LD — ignore */ } } const canonical = src.match(/<link[^>]+rel=["']canonical["'][^>]*>/i)?.[0]?.match(/href=["']([^"']+)["']/i)?.[1] || null; return { json_ld: jsonLd, schema_types: Array.from(schemaTypes).slice(0, 20), open_graph: /<meta[^>]+property=["']og:(title|type)["']/i.test(src), canonical, meta_description: /<meta[^>]+name=["']description["']/i.test(src), }; }
```

Four properties are visible in the code before any test runs.

The string is copied, not resolved. `canonical` is whatever sits inside the `href` attribute. A relative canonical stays relative.

JSON-LD is parsed, not scanned. A malformed block throws, is swallowed, and contributes nothing — including the `json_ld: true` flag.

Types are collected from two keys only. `collectSchemaTypes` recurses through `@graph` and `mainEntity`, so a `@type` nested under `author` or `offers` is invisible.

The output is capped. Twenty schema types, in first-seen order.

## Twenty-seven fixtures, one rule each

Each fixture is a minimal HTML string built to test exactly one branch. Every row is the deployed function's output:

```
fixture ld schema types og canonical desc ------------------------- ----- ------------------------ ----- ----------------------- ---- none false - false null false jsonld_article true Article false null false jsonld_graph true Organization,WebSite,... false null false jsonld_type_array true Article,NewsArticle false null false jsonld_mainentity true WebPage,Question false null false jsonld_nested_author true Article false null false jsonld_malformed false - false null false jsonld_bare_string false - false null false jsonld_two_scripts true TechArticle false null false jsonld_single_quotes_tag true FAQPage false null false jsonld_not_ldjson false - false null false jsonld_25_types true first 20 of 25 false null false og_title false - true null false og_image_only false - false null false og_as_name false - false null false og_uppercase false - true null false desc_meta false - false null true desc_only_og false - false null false desc_uppercase false - false null true canonical_plain false - false https://site.test/a false canonical_href_first false - false https://site.test/a false canonical_rel_extra false - false null false canonical_unquoted false - false null false canonical_relative false - false /a/b false canonical_two false - false https://site.test/first false canonical_multiline false - false https://site.test/a false canonical_uppercase false - false https://site.test/a false
```

Twenty-seven of twenty-seven matched the expected output. The rows worth reading twice:

`jsonld_nested_author` declares `{"@type":"Article","author":{"@type":"Person"}}`. The result is `["Article"]` — the `Person` is not collected, because `author` is not a traversed key. A schema graph that describes authorship properly can still report one type.

`canonical_rel_extra` uses `rel="canonical alternate"` and returns `null`. The regex requires the quote to close immediately after `canonical`, so a tag using the specification's own allowance for a space-separated `rel` list is invisible.

`canonical_unquoted` uses `<link rel=canonical href=https://site.test/a>` and returns `null`. Both attribute values must be quoted.

`canonical_relative` uses `href="/a/b"` and returns `"/a/b"`. A client comparing that to `redirects.final_url` with `===` will always mismatch, on a page that is telling the truth.

`canonical_two` puts two canonical tags in one document and returns the first in source order. Nothing reports that a second one existed.

`canonical_uppercase` passes: `REL="CANONICAL"` matches, because the whole pattern is case-insensitive. Attribute names are flexible; attribute values are not.

## Twelve pages, measured

Sixteen URLs went through `POST /admin/parse` on 2026-11-13. Admin parses bypass quota and payment, so the run cost nothing and touched no free bucket. Thirteen completed; the three refusals each named their reason and charged nothing:

```
outcome count example and reason ---------- ----- --------------------------------------------------------- completed 13 every page below blocked 1 npmjs.com/package/hono — bot check (cloudflare challenge) consent wall 1 www.bbc.com/news — cookie/consent wall, nothing real returned not found 1 github.com/nanoparse-dev/nanoparse — 404/410
```

Thirteen reads cover twelve distinct pages: the `http://` and `https://` versions of one Wikipedia article were both submitted. Coverage, by field:

```
page words ld og desc canonical ------------------------------------------ ----- ----- -- ---- --------- en.wikipedia.org/wiki/web_scraping 4157 true T F yes en.wikipedia.org/wiki/Web_scraping (http) 4157 true T F yes stripe.com/pricing 3792 true T T yes www.nasa.gov/ 453 true T T yes developer.mozilla.org/.../Fetch 806 false F T yes docs.python.org/3/library/json.html 3852 false T T yes arxiv.org/abs/1706.03762 698 false T T yes www.postgresql.org/docs/.../sql-select 10721 false T F yes kubernetes.io/docs/concepts/ 272 false T F no example.com 135 false F F no news.ycombinator.com 928 false F F no plato.stanford.edu/entries/...-intelligence 29266 false F F no www.rfc-editor.org/rfc/rfc9110.html 66479 false F T no
```

JSON-LD was present on 4 of the 13 reads, three distinct pages. Open Graph on 8. A meta description on 6. A canonical link on 8.

The length column is the argument against reading any of this as a quality signal. The largest document in the run, RFC 9110 at 66,479 words, declares no JSON-LD, no Open Graph and no canonical. The Stanford Encyclopedia entry at 29,266 words declares none of the three either. Both are reference documents that render perfectly well; neither publisher added machine-readable declarations, and the group reports that as an absence rather than a defect.

## What the canonical disagreed with

Of the eight reads carrying a canonical, two named a URL different from the one submitted, and the two kinds of difference call for different client behaviour.

```
requested https://en.wikipedia.org/wiki/web_scraping canonical https://en.wikipedia.org/wiki/Web_scraping difference case only — same path, same document requested https://developer.mozilla.org/en-US/docs/Web/API/Fetch canonical https://developer.mozilla.org/en-US/docs/Web/API/Window/fetch difference a different path — the page names its own new address
```

A case-only difference is a deduplication fact: the two strings are one page, and a citation cache keyed on the raw requested URL will store the same document twice. The MDN difference is a provenance fact: the requested path is a consolidated redirect, and the page itself says so.

Both were confirmed against the page's own HTML, fetched separately with a plain HTTP client and compared field by field. The canonical string the endpoint returned was byte-identical to the canonical tag in the page source on all 13 reads, and JSON-LD presence agreed on all 13. The values are the publisher's, not the parser's inference.

## The other field that answers the same question

`redirects` reports three values — `final_url`, `hops`, `cross_domain` — and on all 13 reads it reported `hops: 0`. On 11 reads `final_url` was byte-identical to the URL submitted; on the other two it differed only by a trailing slash. That includes three URLs a plain HTTP client sees answer with a 301:

```
curl -sI https://developer.mozilla.org/en-US/docs/Web/API/Fetch 301 -> https://developer.mozilla.org/en-US/docs/Web/API/Window/fetch payload "redirects": {"final_url": "https://developer.mozilla.org/en-US/docs/Web/API/Fetch", "hops": 0, "cross_domain": false} curl -sI https://en.wikipedia.org/wiki/web_scraping 301 -> https://en.wikipedia.org/wiki/Web_scraping payload "redirects": {"final_url": "https://en.wikipedia.org/wiki/web_scraping", "hops": 0, "cross_domain": false} curl -sI http://en.wikipedia.org/wiki/Web_scraping 301 -> https://en.wikipedia.org/wiki/Web_scraping payload "redirects": {"final_url": "http://en.wikipedia.org/wiki/Web_scraping", "hops": 0, "cross_domain": false}
```

The third row is the clearest: the scheme in the reported `final_url` is the scheme that was requested, on a page served over TLS that says so in its own canonical.

The source explains the shape without needing a guess. In the queue consumer, `finalUrl` begins as `null` and `redirectHops` as `0`. Hops are incremented by a response listener that only exists inside the browser-render branch. When the direct-HTTP fast path produces the HTML, the browser never starts, so nothing assigns either variable, and `computeRedirects` falls back:

```
const final = (typeof finalUrl === "string" && finalUrl) ? finalUrl : original; const h = (typeof hops === "number" && hops >= 0) ? hops : 0; return { final_url: final, hops: h, cross_domain: extractDomain(final) !== extractDomain(original) };
```

`hops: 0` therefore means "no hop was counted", which is not the same as "no hop happened". The practical rule is short: read `redirects` as a report of what the fetch layer observed, and read `structured_data.canonical` when the exact URL matters — for a citation, a deduplication key, or a link a reader will click. When the two disagree, the canonical is the publisher's statement and the hop count is a counter that may have had nothing to count.

## What the group cannot see

- **Schema types outside `@graph` and `mainEntity`.** A `Person` nested under `author` is not collected, so `schema_types` is a partial view of a graph that may be complete and valid.
- A second canonical tag. First in source order wins, silently.
- **A canonical written with a `rel` list or without quotes.** Both appear in the wild and both return `null`.
- The target of a relative canonical. The string is returned as written.
- Whether a declaration is true. `json_ld: true` means JSON-LD parsed. It does not mean the `Article` it declares is an article, and `schema_types: []` means "nothing declared", never "nothing true".

## Using the group without a model call

Four rules, in the order they change a decision.

```
function addressOf(job: { litmus: any }) { const sd = job.litmus.structured_data; const rx = job.litmus.redirects; // 1. Identity: prefer the page's own canonical, resolved against the reported URL. const canonical = sd.canonical ? new URL(sd.canonical, rx.final_url).toString() // handles a relative href : null; // 2. Deduplication key: normalise case and trailing slash before comparing. const key = (canonical || rx.final_url).replace(/\/+$/, "").toLowerCase(); // 3. Do not infer a redirect from hops: 0 — it is a count, not a verdict. // Compare the canonical path with the requested path instead. const moved = canonical ? new URL(canonical).pathname !== new URL(rx.final_url).pathname : false; // 4. Read the three declarations separately: og and description answer // different questions, and an empty schema_types means nothing was declared. return { canonical, key, moved, declared: sd.json_ld, social: sd.open_graph, described: sd.meta_description, }; }
```

The first rule is the one that matters most. `new URL(sd.canonical, rx.final_url)` resolves a relative canonical against the URL the fetch reports, which is the only way to turn `"/a/b"` into something comparable — the endpoint returns the string as the page wrote it, and a client that string-compares without resolving calls every relative canonical a mismatch.

The third rule is a correction to a natural reading. A caller that wants to know whether a page moved has to compare paths, not read a hop counter, because a hop counter that ran on a path without a browser is a counter with nothing to count.

## Reproducing this

The function is not retyped. It is sliced from `intel.ts` by declaration and the slice records the source hash:

```
python3 build_fns.py # slice computeStructuredData + collectSchemaTypes node --experimental-strip-types fixtures.mjs # 27 fixtures -> results.json python3 probe_real.py # 16 URLs through /admin/parse -> real/ python3 rollup.py # every count printed above python3 truth.py # each page's own HTML, fetched independently
```

The slice is 42 lines at blob `7b3eca00`, source sha256 `d773ba97…`, unchanged since the live deploy. The 27 fixtures, all 13 payloads, the three refusals, the independent page fetches, the redirect headers and the scripts are committed beside this post.

Two scope notes. The run used the admin path, so it cost nothing and consumed no free tier. And every number here describes the deployed function as of this post — a later version can change any rule above without changing this page.

The endpoint is [https://nanoparse.app](https://nanoparse.app/), $$0.005 per successful parse, no account and no API key, with the same surface served over MCP at `/mcp`. Refused reads are never charged. The earlier survey of all six metadata groups is [/blog/2026-10-19-page-you-asked-for](https://nanoparse.app/blog/2026-10-19-page-you-asked-for), and the neighbouring deep dives read one field apiece: [/blog/2026-11-07-authorship-field](https://nanoparse.app/blog/2026-11-07-authorship-field) and [/blog/2026-11-10-page-status-default](https://nanoparse.app/blog/2026-11-10-page-status-default).

Litmus2,161 words

Copy link to this post

Test the page your agent just read.

Clean Markdown and 20 trust signals, $0.005 per parse.

[Run a test](https://nanoparse.app/#parse)

On this page

Share

Copy link

Keep reading

## More on litmus.

[All posts](https://nanoparse.app/blog)

[Nov 10, 2026Page Not Found, Twice, and the Field That Called It a DocumentAn earlier survey of the six signal groups added in October reported that page_status fired on none of ten pages, and drew the right conclusion from it: the wall is usually caught upstream b…](https://nanoparse.app/blog/2026-11-10-page-status-default)

[Nov 7, 2026The Author Field Is Present. That Is Not the Same as True.Litmus returns twenty signal groups in the same response body as the markdown. One of them is authorship, and of the twenty it is the field most likely to be read as a fact about a person: "…](https://nanoparse.app/blog/2026-11-07-authorship-field)

![](https://nanoparse.app/brand/nanoparse-logo-official.svg)

## Your agent deserves better data.

Clean Markdown and twenty deterministic trust signals, priced per parse.

[Run a free test](https://nanoparse.app/#parse)[See the 20 signals](https://nanoparse.app/litmus)

10 free parses a month · no account · no API key