feat: serve markdown to agents via content negotiation - #28
Open
atinux wants to merge 1 commit into
Open
Conversation
Vercel routing-layer rewrites (Docus-style) send Accept: text/markdown and curl requests to the /raw/** mirrors, ahead of the ISR cache. Adds raw mirrors for /tree and /blob previews, markdown 404 bodies with recovery links, an llms.txt when-to-use section, and an Organization JSON-LD node.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Improves the is-agentic score for comark.dev (currently 67/100) for every site built on this layer.
What changed
Markdown content negotiation (essential check, was failing)
modules/markdown-rewrite.tsinjects rewrites into.vercel/output/config.jsonafter Nitro compiles, same approach as Docus' markdown-rewrite.Accept: text/markdown(or acurl/*user-agent) on any page URL serves its/raw/**mirror;/serves/llms.txt. Responses carrycontent-type: text/markdownandvary: Accept.llms.txt; this layer reads content at request time, so the rewrites use generic patterns instead (unit-tested intest/markdown-rewrite.test.ts).Raw mirrors for versioned previews
/raw/tree/:branch/<page>.mdand/raw/blob/:sha/<page>.md, sharing the preview-content instances (and ref validation) of the existing/api/contentpreview routes.Agent-friendly 404s (essential check, was partial)
/raw/**misses return a real 404 with a short markdown body pointing at/llms.txt,/llms-full.txt,/raw/index.md, and the sitemap. Combined with the rewrites, an agent that guesses a wrong page URL gets a recoverable markdown 404.app/error.vuegains home/documentation links.llms.txt "when to use" section (recommended check)
docs.llms.whenToUseapp-config key, emitted as the first llms.txt section.Organization JSON-LD (recommended check)
docs.schemaOrg.organizationsub-key, emitted as a top-levelOrganizationnode on the landing page (forcontactPoint,address,sameAs).Docs
Verification
pnpm lint(no new warnings),pnpm test(130 passed, 6 new),pnpm typecheckclean.NITRO_PRESET=vercel pnpm build→ 8 routes injected intoconfig.json(verified shape).text/markdown+Vary: Accept, missing page 404 markdown body,/raw/tree/main/...200.Out of scope (needs changes in the comark.dev content repo)
<h1 id="hello-world">which likely confuses the scanner./about,/contact,/privacy),docs.llms.whenToUse/organizationvalues, OpenAPI spec decision.