Skip to content

connector: cut off long messages so they fit in a Matrix event - #308

Merged
highesttt merged 2 commits into
mainfrom
indent/line-cut-off-long-messages
Nov 4, 2026
Merged

highesttt merged 2 commits into
mainfrom
indent/line-cut-off-long-messages

Conversation

@indent

@indent indent Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Production logs from the last 24 h show 60 Failed to send message part to Matrix errors with M_BAD_JSON: event content can't be over 64 KiB, across 5 accounts and 30 distinct LINE messages. The encrypted send requests were ~75 KB, so the plaintext content was around 55 KB. These messages are never bridged. Since they're never saved, the recent-message backfill re-queues them on every restart and they fail again.

Cause: convertLineMessage has no size limit. A long LINE text keeps the full text in body, and when it has mentions or inline sticons, the same text is copied again into formatted_body, together with m.mentions.

Change: after conversion, each part whose JSON-encoded content is over 40,000 bytes goes through a cutoff modeled on mautrix-whatsapp (whatsapp#988/#995). Ghost mentions are dropped. If a formatted body is present, the plain-text fallback is shortened to 500 bytes. If it's still too large, the formatted body (then the body) is cut at a rune and tag boundary, with a "[long message cut off]" marker. The cut point is found by binary search on the encoded size, because quotes, <, > and & expand when escaped. A warn line logs the LINE content type and the encoded size before and after the cut.

View in Indent
Tag @indent to continue the conversation here.

Inbound LINE messages whose body, formatted body and mentions add up to
more than the homeserver's 64 KiB event limit once encrypted were
rejected with M_BAD_JSON and never bridged. Apply the same cutoff as
mautrix-whatsapp to every converted part: drop ghost mentions, shorten
the plain-text fallback when a formatted body is present, and cut the
remaining text at 40k bytes.
@indent

indent Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor Author

Issues

No issues found.

CI Checks

All CI checks passed on 4e3be78.

Review agents

Select any unchecked box below to run or rerun that agent.

Passed (1)
  • Indent Review Agent · No bugs found in the long-message cutoff.
Full results

Indent Review Agent

  • Summary: No bugs found in the long-message cutoff.
  • Last ran on commit: 4e3be785
  • Latest result
    {
      "summary": "No bugs found in the long-message cutoff.",
      "findings": []
    }

@indent indent Bot added the to be tested label Nov 4, 2026
@iFixRobots
iFixRobots requested a review from highesttt November 4, 2026 11:08

@highesttt highesttt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Size check uses raw bytes, but the limit is on the JSON-encoded event. Quote-heavy HTML can pass the check and still be too large. Measure the encoded size instead

The homeserver limit applies to the encoded event, where quotes, angle
brackets and ampersands expand to escape sequences, so raw string
lengths could pass the check while the event stayed too large. Measure
json.Marshal output, and binary-search the cut point so escape-heavy
text isn't cut far shorter than needed.
@highesttt
highesttt enabled auto-merge (squash) November 4, 2026 15:33
@highesttt
highesttt self-requested a review November 4, 2026 15:33
@highesttt
highesttt disabled auto-merge November 4, 2026 15:33
@highesttt
highesttt merged commit a9cefaf into main Nov 4, 2026
9 checks passed
@highesttt
highesttt deleted the indent/line-cut-off-long-messages branch November 4, 2026 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants