Skip to content

fix(pgsql): opt into Npgsql dynamic JSON so ProfileData reads work again — PRODUCTION OUTAGE - #131

Merged
MusaMisto merged 1 commit into
mainfrom
fix/npgsql8-dynamic-json
Sep 1, 2026
Merged

fix(pgsql): opt into Npgsql dynamic JSON so ProfileData reads work again — PRODUCTION OUTAGE#131
MusaMisto merged 1 commit into
mainfrom
fix/npgsql8-dynamic-json

Conversation

@MusaMisto

Copy link
Copy Markdown
Member

🔴 Production outage — fixes it

InvalidCastException: Reading as 'IEnumerable<ProfileDataItem>' is not supported
for fields having DataTypeName 'jsonb'
 ---> NotSupportedException: Type 'IEnumerable`1' required dynamic JSON serialization,
      which requires an explicit opt-in; call 'EnableDynamicJson'

This is a regression from #129, which moved Npgsql 5.0.10 → 8.0.11 to fix the EF Core TypeLoadException. That fixed startup and introduced this: Npgsql 8 removed the implicit dynamic JSON serializer. Mapping an arbitrary POCO collection to a json/jsonb column now needs EnableDynamicJson() on the data source.

Three properties depend on it via StoreAsJson()Tenant.ProfileData, TenantMembership.ProfileData, Account.ProfileData — and mtm.account.profile_data is jsonb in production.

The blast radius is wider than login. UseApiKeyAsRequestContext materialises Account on every apikey-authenticated request, so any such request 500s — not just /accounts/login.

Fix

Startup.cs builds a single NpgsqlDataSource with EnableDynamicJson() and passes it to UseNpgsql. Built once, outside the AddDbContext lambda: NpgsqlDataSource owns the connection pool, so constructing one per DbContext instance would leak pools.

18 lines, one file, PgSql path only — the MySql and MsSql branches are untouched.

Verification — A/B against the real production database

Identical environment, identical request, same data:

GET /api/accounts POST /api/accounts/login exceptions
ghcr.io/simplify9/mtm:8.0.3 (live now) 500 500 8
this build 200 — returns real account data 400 — handler reached, bad password rejected 0

The 400 is the correct result: it proves the request got past the jsonb read and into the handler, which then rejected a deliberately wrong password.

No writes to production. The 2021 migrations are already applied, so startup migration was a no-op — confirmed, no Applying migration / DDL in the logs.

Not the connection string

Worth recording, since that was the initial hypothesis: the secret parses correctly (Database=funride-prod, correct host), the pod is Ready with 0 restarts, and /mtm/health returns 200. The app connects and queries fine — only the jsonb materialisation fails.

After merge

CI publishes 8.0.4. Pin funride-mtm's cicd-gateway.yml to it and redeploy Prod. I'll verify over the wire.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VHkpqv6dB6wjALyFe9ocMU

Production login returns HTTP 500:

  InvalidCastException: Reading as 'IEnumerable<ProfileDataItem>' is not
  supported for fields having DataTypeName 'jsonb'
  ---> NotSupportedException: Type 'IEnumerable`1' required dynamic JSON
  serialization, which requires an explicit opt-in; call 'EnableDynamicJson'

Regression from #129, which moved Npgsql 5.0.10 -> 8.0.11 to fix the EF Core
TypeLoadException. Npgsql 8 removed the implicit dynamic JSON serializer:
mapping an arbitrary POCO collection to a json/jsonb column now requires
EnableDynamicJson() on the data source. Three properties rely on it via
StoreAsJson() — Tenant.ProfileData, TenantMembership.ProfileData and
Account.ProfileData — and account.profile_data is jsonb in production.

The blast radius is wider than login: UseApiKeyAsRequestContext materialises
Account on every apikey-authenticated request, so ANY such request 500s.

Startup.cs now builds a single NpgsqlDataSource with EnableDynamicJson() and
hands it to UseNpgsql. It is built once outside the AddDbContext lambda —
NpgsqlDataSource owns the connection pool, so constructing one per DbContext
instance would leak pools.

Verified A/B against the REAL production database, same request, same data:

  ghcr.io/simplify9/mtm:8.0.3   GET /api/accounts   -> 500  (8 exceptions)
                                POST .../login      -> 500
  this build                    GET /api/accounts   -> 200  (0 exceptions,
                                                      real account data)
                                POST .../login      -> 400  (handler reached,
                                                      bad password rejected)

No schema changes were applied during the test — the 2021 migrations are
already present, so startup migration is a no-op.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHkpqv6dB6wjALyFe9ocMU
@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: simplify9/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: a20844f8-0000-4c15-92f7-d8effccd2be7

📥 Commits

Reviewing files that changed from the base of the PR and between ecb883e and 98835db.

📒 Files selected for processing (1)
  • SW.Mtm.Web/Startup.cs

📝 Walkthrough

What changed

  • Upgraded Npgsql compatibility by creating one NpgsqlDataSource.
  • Enabled dynamic JSON serialization with EnableDynamicJson().
  • Passed the data source to PostgreSQL UseNpgsql.
  • Left MySQL and SQL Server configuration unchanged.
  • This restores jsonb reads for Tenant.ProfileData, TenantMembership.ProfileData, and Account.ProfileData.

Risk

risk:low

The change affects PostgreSQL startup and data access configuration. It does not change schemas, writes, or application authorization logic.

Security-sensitive areas

API-key-authenticated requests and login previously failed when reading profile data. The change restores request processing but does not alter authentication or authorization behavior.

Test coverage impact

Production verification confirmed:

  • Account requests changed from HTTP 500 to HTTP 200.
  • Login reached its handler and returned the expected HTTP 400 for an invalid password.

No automated test coverage changes were reported.

Deployment and operational concerns

  • No migration or schema change is required.
  • No production writes were performed during verification.
  • Monitor PostgreSQL startup and jsonb reads after deployment.
  • Rollback requires reverting the Npgsql data-source configuration and restoring the previous package/configuration state.

Walkthrough

The PostgreSQL startup path now builds one shared NpgsqlDataSource, enables dynamic JSON handling, and passes the data source to Entity Framework through UseNpgsql. Other database paths remain unchanged.

Changes

PostgreSQL data source

Layer / File(s) Summary
Configure shared PostgreSQL data source
SW.Mtm.Web/Startup.cs
The PostgreSQL path imports Npgsql, enables dynamic JSON handling, builds one shared NpgsqlDataSource, and uses it for Entity Framework configuration instead of passing the connection string directly.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: security, database, risk:high


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@MusaMisto
MusaMisto merged commit d35c027 into main Sep 1, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant