Blog

Fifteen Signals, Measured: The Values, and the Lines Behind Them

· NanoParse team

A parse returns Markdown, page metadata, and fifteen signals. The Markdown is the page. The signals are its vitals: who published it, when, how dense the prose is, whether it reads like a reprint, whether it is a shell with a gate behind it.

Vitals are only useful if the method is known. So this is a run with the numbers left in. Eight URLs submitted to the live endpoint on 2026-10-01, seven rendered, one refused before any work started — and for each page, every value that came back, alongside the lines of the deployed source that computed it.

Nothing here calls a model. The fifteen signals are computed from the rendered text and the DOM in the same pass that produces the Markdown, which is why they arrive with the parse instead of as a second billed call.

The run

Eight URLs, picked to differ in kind rather than to flatter the product. Seven rendered:

page                                    words   read  boiler  source.type  auth  trust   content_type
docs.python.org/3/library/asyncio-task   6161    31m   0.04    commercial   0.40  medium  academic_paper
developer.mozilla.org/.../Status/402      326     2m   0.62    commercial   0.40  medium  tutorial
rfc-editor.org/rfc/rfc9110.html         66474   332m   0.03    commercial   0.40  medium  documentation
en.wikipedia.org/wiki/Web_scraping       4151    21m   0.09    commercial   0.40  medium  tutorial
arxiv.org/abs/1706.03762                  686     3m   0.09    academic     0.85  medium  documentation
bbc.com/news                             1182     6m   0.02    news         0.70  low     news_article
shopify.com/pricing                      2880    14m   0.01    commercial   0.40  medium  product_page
stackoverflow.com/questions/11227809       --     --     --  refused before render: bot check, not charged

Four hosts landed in the same bucket: commercial, authority 0.4. A language reference, a web-platform reference, an IETF specification, and an encyclopedia article, all typed identically. That is the most important thing to know before gating on these fields, and the next section is why it happens.

source.type is a domain lookup, not a judgment

classifySource() walks a fixed table of domain patterns — .gov and .mil for government, .edu and .ac.xx and a few host names for academic, a brand list for news, host patterns for technical, forum, and social. Its last entry is a catch-all that returns commercial for anything that matched nothing, and commercial carries a base authority of 0.4 in the authority table.

That is exactly what the run shows. arxiv.org matched the academic pattern and returned academic at 0.85. bbc.com matched the news list and returned news at 0.7. Every other host fell through to the catch-all: commercial, 0.4. The field answers "which rule matched this host" — useful for triage, useless as a verdict about what the page says.

Read source.rationale next to it. It carries the string that fired, "academic source" or "commercial source", so a caller can tell a matched rule from the default without knowing the table.

freshness reports what the page declares

freshness extracts a publication date from the page's own metadata, then describes it: published, age as a phrase, modified, has_correction, correction_note, and stale_warning.

Two ends of the run. RFC 9110 returned published: "2022-06-01", age: "4 years ago", a stale warning, and has_correction: true. The arXiv abstract page returned published: null, age: "unknown", and the warning text No publish date found — verify independently — even though the page body says "Submitted on 12 Jun 2017". The extractor reads declared date metadata first; a date written into prose is not a date field.

has_correction is narrower than it sounds. It fires on revision or errata markers in the text, so a specification with an errata section and an encyclopedia article with an edit history both return true. It means the page carries a correction trail, not that the page was wrong and fixed.

And a stale warning on a four-year-old specification is not a defect report. Standards are allowed to be old. The field's own copy tells the caller what to do with it: verify independently.

content_type is coarse on purpose

The seven rendered pages returned five different types: academic_paper (the Python asyncio reference), tutorial (the MDN status-code page and the Wikipedia article), documentation (RFC 9110 and the arXiv abstract), news_article (the BBC index), product_page (the pricing page).

The classifier reads structure and vocabulary, not intent. A language reference that reads like a paper is typed like a paper. Use it for routing and for choosing a prompt shape; do not use it as a citation rule on its own.

tldr and key_figures, with context

tldr is the page's own summary — its description or lead. key_figures is a sweep for numbers with the surrounding text kept beside each one, which is what makes the array usable at all.

The counts on this run: 0 figures on the Python reference, 3 on the arXiv abstract, 5 on the MDN page, 20 on the pricing page. The values themselves explain why the context field ships:

"value": "1,102"   context: "Mon, 12 Jun 2017 17:57:34 UTC (1,102 KB)"
"value": "402%"    context: "...%2Fweb%2Fhttp%2Freference%2Fstatus%2F402%60..." (MDN)

Those three arXiv numbers are submission PDF sizes, not results. The five on the MDN page came out of percent-encoded URL text — a digit run sitting next to a percent sign reads like a percentage. The twenty on the pricing page are the real thing: plan prices with their plan names beside them, which is what an agent pricing a product actually needs. The extractor finds candidates; context is what lets a caller decide which candidates matter.

for fig in litmus["key_figures"]:
    print(fig["value"], "|", fig["context"][:70])

reading_time and boilerplate_ratio

reading_time.minutes is max(1, round(words / 200)) over the rendered Markdown, and 0 for an empty render. The two ends of this run show the range: 2 minutes for the MDN page at 326 words, 332 for RFC 9110 at 66,474. An agent that checks the read length before opening the page learns whether a URL is a paragraph or a book while spending nothing on it.

boilerplate_ratio is navigation, footer, and cookie-banner text measured against the total body text in the pre-strip DOM. The counter-intuitive result here: the shortest page had the highest ratio. MDN returned 0.62 — nearly two thirds of its rendered text is chrome around a short reference entry — against 0.01 for the pricing page and 0.02 for the news index.

Confidence matters as much as the number. high means the ratio came from the parsed DOM. low means the function fell back to repeated-line detection on stripped text. A comment in the deployed source states the failure mode plainly: a stripped document measures boilerplate as near zero and would mislabel confidence as high, which is why the DOM handed to that function is the pre-strip one.

hedge_language and content_density

hedge_language counts a fixed list of speculative markers and reports them three ways: density, markers_found, and per_500_words. Across all seven pages the run found low density, between 0 and 3 markers, with the MDN page highest at 1.6 per 500 words. Reference material does not hedge. A page of speculation would look different, and that difference is the signal.

content_density splits the text into sentences and calls one substantive if it contains a digit, two adjacent capitalized words, or runs past 80 characters. thin_content is true when fewer than 0.25 of sentences qualify. Measured ratio: 0.62 on the pricing page, 0.81 on the Python reference, 0.86 on the arXiv abstract, 0.91 on MDN, 0.94 on RFC 9110, 1.0 on the news index. thin_content was false everywhere — including the page with the lowest ratio, which is worth noting: a pricing page full of short price lines is not thin content, it is dense in a different shape.

syndication, paywall, is_archived

All three boolean branches came back false on all seven pages, and the reason to show that is what a false does not mean.

syndication and paywall are keyword scans — "originally published", "via Reuters", "subscribe to read", "unlock the full", plus truncation phrasing. They fire on wording. An article can be a reprint without any of that wording, so a false is an absence of evidence, not evidence of absence.

is_archived probes the Wayback Machine and archive.is with a hard 300-millisecond timeout and returns false on timeout. A false there means "not confirmed in 300 milliseconds" — not "no copy exists". Read those three as cheap hints and verify the ones that matter.

structural_trust_score is assembled from the others

This is the one field that is not measured from the page directly; it reads the rest. The BBC News index returned the only low in the run, and it printed its work:

{
  "level": "low",
  "trust_factors": [
    "Moderate-authority source",
    "No publish date found — verify independently",
    "Contains speculative or unverified language",
    "Weak outbound authority — few or low-quality external citations"
  ]
}

The other six pages returned medium, each with its reasons. Because trust_factors is always populated, the level is a summary and the array is the detail — sort on the array. One honest wrinkle from this run: the news index carried a speculative-language factor while its own hedge_language.density read low with zero markers. The factors are reasons to look, not a perfectly derived score, which is the correct way to treat every field here.

outbound_authority measures the links, not the page

outbound_authority scores the domains a page points at, weighted by their own class. The Python reference scored 0.7 with ["github.com"] in top_domains. The pricing page scored 0.4 pointing at admin.shopify.com, stripe.com, and help.shopify.com. The news index scored 0.0 with an empty list — an index page with no external citations scores nothing, which says more about the page's shape than about the outlet.

A gate, and what it returns on this set

The fifteen fields exist to answer one question at fetch time: is this page worth reading or citing. Here is a small gate built on the fields above:

def gate(litmus, words, needs_recent=True):
    notes = []
    if litmus["paywall"]["detected"]:
        notes.append("gated")
    if litmus["content_density"]["thin_content"]:
        notes.append("thin")
    bp = litmus["boilerplate_ratio"]
    if words < 500 and bp["confidence"] == "high" and bp["ratio"] > 0.4:
        notes.append("short and mostly chrome")
    f = litmus["freshness"]
    if needs_recent and f["published"] is None:
        notes.append("no declared date")
    elif needs_recent and f["stale_warning"]:
        notes.append("dated: " + f["age"])
    return notes

Run against the seven rendered pages, the output is short and specific: ["short and mostly chrome"] for the MDN status-code page, ["dated: 4 years ago"] for RFC 9110, and ["no declared date"] for the other five. Two of seven pages get a real caution. Five get a flag that a caller reading reference material can ignore and a caller building a news digest cannot. That is the shape of every signal in this run: a reason to look, with the reason written down.

What these signals are not

They are heuristics run over text, not fact-checking. Nothing in the pass verifies a claim, resolves a statistic, or knows whether a page is right. The domain classifier is a rule table with a catch-all. content_type reads structure. syndication and paywall read phrasing. is_archived reads a 300-millisecond probe.

Two consequences worth carrying into production. First, a false boolean can mean "not confirmed". Second, the signals describe a page that rendered — when the render fails, there are no signals at all. The eighth URL in this run, a Stack Overflow question, came back as a refusal before render: a Cloudflare challenge, no Markdown, no signals, no charge. That failure mode is legible and it is not dressed up as data.

Reproduce it

The same eight URLs can be re-run on the public contract. Ten parses per network are free, with no account and no key:

import json, time, urllib.request

req = urllib.request.Request(
    "https://nanoparse.app/fetch",
    data=json.dumps({"url": "https://arxiv.org/abs/1706.03762"}).encode(),
    headers={"Content-Type": "application/json"},
)
job = json.loads(urllib.request.urlopen(req).read())   # 202: jobId + statusUrl

while True:
    time.sleep(3)
    res = json.loads(urllib.request.urlopen(job["statusUrl"]).read())
    if res.get("status") in ("completed", "failed"):
        break

print(res["status"])
print(res["litmus"])

Polling /status is free and needs no payment. Past the ten free parses, the same POST answers 402 with the x402 terms in a base64 PAYMENT-REQUIRED header; the retry carries a signed Payment-Signature header. That loop is written out in How an Agent Buys a Page, and the job envelope this snippet hands back is covered in Read Once: What a Parse Job Actually Promises.

MCP clients skip the glue entirely:

{"mcpServers": {"nanoparse": {"url": "https://nanoparse.app/mcp"}}}

Cost

One flat $0.005 USDC on Base per delivered parse, with the fifteen signals included in the response rather than billed as a second call. Reads that are refused — SSRF-restricted hosts, blocked pages, paywalled pages, or a render that comes back too thin — are rejected before settlement and never charged. The Stack Overflow refusal in this run produced no signals and no charge.

The raw result set for this run, all seven completed payloads and the refusal, is committed beside this post at marketing/litmus-run/2026-10-01/.

Getting started

Full documentation and the quickstart live at nanoparse.app. The agent-facing description of the API is at nanoparse.app/llms.txt, and the built-in MCP endpoint is nanoparse.app/mcp — one URL, no install, no API key.

← All posts NanoParse home