Skip to content

FindLocation returns Location.None for any value the file writes with a JSON escape: C:\temp, say "hi", and System.Text.Json-written non-ASCII names like Ω·m² can't be located #68

Open

Description

@matt-edmondson

What goes wrong

MetadataFile.IndexOfToken (SourceGeneratorToolkit/MetadataFile.cs, around lines 139-143) searches the raw file text: first for "<value>", then for <value>. Both FindLocation(needle) and FindLocation(anchor, needle) go through it.

A generator normally passes a value it got back from Deserialize<T>, which is unescaped. The file holds the escaped JSON form, so for any such value neither search can match.

Reproduction

The metadata file:

{
  "things": [
    { "name": "Alpha", "pattern": "C:\\temp" },
    { "name": "Caf\u00e9" },
    { "name": "say \"hi\"" }
  ]
}

I ran this in a scratch test against the current main:

Call Result
FindLocation(@"C:\temp") Location.None
FindLocation("Café") Location.None
FindLocation("say \"hi\"") Location.None
FindLocation("Alpha", @"C:\temp") Location.None
FindLocation("Alpha") (control) (2,15)-(2,20), correct

Why it matters

  • JSON requires \ and " to be escaped. So no value containing a backslash or a quote can ever be located: Windows paths, regex patterns, format strings.
  • System.Text.Json's default encoder escapes all non-ASCII and + < > & '. Serializing "Ω·m²" gives "\u03A9\u00B7m\u00B2". Units and physics metadata, which are the main consumers, hit this directly.
  • When the anchor is the escaped value, the scoped overload falls back to the first unscoped match of the needle. The diagnostic then lands on the wrong entry, which is exactly what the scoped overload exists to prevent.
  • The README says FindLocation "maps a name from the metadata back to a position in the file".

Suggested fix

Match against decoded JSON string tokens rather than raw text:

  1. Scan Text for string literals.
  2. Unescape each one, compare it with the needle, and return the literal's span. The span should cover the literal's content between the quotes, as now.
  3. Keep the raw search as the fallback for non-string tokens such as numbers.

This is distinct from #66 (needle key written before the anchor key), #33 and #63 (substring matches), and #28.

Acceptance criteria

  • FindLocation locates values whose file form contains \\, \", or \uXXXX escapes, for both upper- and lowercase hex, in both the unscoped and the scoped overloads.
  • The returned span covers the escaped literal in the file.
  • Tests cover a backslash path, an embedded quote, and a \u-escaped non-ASCII name.

Activity

  1. matt-edmondson commented on Nov 6, 2026

    @matt-edmondson
    ContributorAuthor

    Triage

    Next steps: Scan the string literals, unescape each one, compare it with the needle, and return the content span of the escaped literal. Keep the raw search as a fallback for non-string tokens. Add tests for a backslash path, an embedded quote, and \u escapes in both hex cases, across both overloads.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions