Skip to content

fix: bound PACK inflation and push body size in parsePush (sfind-79af3f3c) - #47

Open
devin-ai-integration[bot] wants to merge 4 commits into
mainfrom
devin/sfind-79af3f3c-pack-inflate-limits
Open

devin-ai-integration[bot] wants to merge 4 commits into
mainfrom
devin/sfind-79af3f3c-pack-inflate-limits

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Description

Corrige o finding sfind-79af3f3c (low/high): o parser de PACK em parsePush inflava objetos sem limite de saída e com uma escrita no zlib (e uma Promise) por byte de entrada. Como parsePush roda antes de qualquer autenticação/checagem de repositório, um push anônimo podia travar ou derrubar o proxy por CPU/memória.

O que mudou

  • decompressGitObjects (src/proxy/processors/push-action/parsePush.ts)
    • Cada objeto é inflado numa única chamada zlib.inflate assíncrona (promisificada, info: true + maxOutputLength), executada no threadpool do libuv — não bloqueia o event loop. O fim de cada objeto é localizado via engine.bytesWritten (sem mais escrita byte a byte).
    • Limites: MAX_INFLATED_OBJECT_SIZE = 100 MiB por objeto e MAX_INFLATED_PACK_SIZE = 256 MiB no total do PACK. Objetos vazios ainda são aceitos quando o orçamento chega exatamente a zero.
    • No máximo MAX_CONCURRENT_PACK_INFLATIONS = 2 PACKs são inflados ao mesmo tempo (fila FIFO; o slot é liberado em finally), limitando a memória agregada de inflação entre pushes concorrentes.
    • Para após o número de entradas declarado no header do PACK (getContents(buffer, numEntries)).
  • getContents passa a logar só metadados (contentLength) em vez de serializar o conteúdo completo dos objetos.
  • Limite de body: extractRawBody (src/proxy/routes/index.ts) usava getRawBody(..., { limit: '1gb' }); agora usa MAX_PACK_BODY_SIZE ('100000kb'), o mesmo valor já usado pelo Express em src/proxy/index.ts (ambos referenciam a constante em src/proxy/processors/constants.ts).

Mudança de comportamento (sem mudança de contrato de API)

Pushes com body acima de ~100 MB, objeto inflado acima de 100 MiB ou PACK inflado acima de 256 MiB passam a ser rejeitados com erro de inflação. Nenhuma API pública/assinatura exportada mudou.

Fora de escopo

Limitar quantos bodies de push (~100 MB cada) podem estar bufferizados simultaneamente é decidido na camada HTTP/rota, antes do parser — exigiria um limite de concorrência/rate limit em extractRawBody ou no reverse proxy à frente do GitProxy (ver thread de review).

Medições locais (harness chamando getContents, Node 22)

Payload main Esta branch
1 MB comprimido, dados aleatórios 15,3 s 16 ms
4 MB comprimido, dados aleatórios 69,9 s 59 ms
~300 KB de zeros inflando para 300 MiB 7,8 s, +957 MiB RSS, depois Invalid string length rejeitado em ~31 ms (sync) / ~420 ms (async, event loop seguiu respondendo)

Related Issue

Code scan finding sfind-79af3f3c070b42aa9b72f931bea858cd (scan scan-70082f73b42b43fda8ae0a7ecbe7fedf).

Checklist

General

Documentation

  • Documentation has been added/updated for any new features

Configuration

  • If configuration schema (config.schema.json) was modified: (não modificado)

Tests

  • Tests have been added/updated for new functionality
  • Unit tests pass (npm test) — veja nota abaixo
  • Linting and formatting pass (npm run lint and npm run format:check) — job Linting verde no CI
  • Type checks pass (npm run check-types) — passa no CI

Como testar

npx vitest run test/testParsePush.test.ts test/parsePushLimits.test.ts test/extractRawBody.test.ts
  • test/parsePushLimits.test.ts (novo): limites por objeto/PACK (com constantes mockadas), objeto vazio no limite exato, yield ao event loop durante a inflação e no máximo MAX_CONCURRENT_PACK_INFLATIONS inflações simultâneas (falha se o limite for removido).
  • test/testParsePush.test.ts: parada no número de entradas declarado e rejeição rápida (<5 s) de um payload real acima de MAX_INFLATED_OBJECT_SIZE.
  • test/extractRawBody.test.ts: getRawBody chamado com MAX_PACK_BODY_SIZE.

Localmente: 63/63 nesses 3 arquivos. No CI, Linting, e2e (vitest), e2e (cypress) e CodeQL passam; nos jobs build-*, 60 arquivos de teste passam. As falhas restantes não vêm desta mudança e também aparecem em outros PRs abertos (ex.: #46):

  • test/ConfigLoader.test.ts > loadRemoteConfig busca proxy.config.json do upstream finos/git-proxy e falha com Invalid value for key "sink".
  • Matriz MongoDB 8.0: MongoDB did not initialize within 20 seconds antes da instalação.

Devin-Org: engineering

Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/a53cfdf248f24a4ab811ed0da7f0bbdc
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/a53cfdf248f24a4ab811ed0da7f0bbdc?variant=devin


Devin Review

Status Commit
⚪ Not started —

Run Devin Review

Enable auto-review to review your pull requests automatically.

Devin Review (Staging)


Devin Review

…79af3f3c)

Inflate each PACK object with a single inflateSync call capped by MAX_INFLATED_OBJECT_SIZE / MAX_INFLATED_PACK_SIZE instead of one zlib write per input byte, stop after the declared entry count, log object metadata instead of full contents, and cap the buffered git pack request body at MAX_PACK_BODY_SIZE instead of 1gb.
@devin-ai-integration

Copy link
Copy Markdown
Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Devin Review

activeInflations++;
return;
}
await new Promise<void>((resolve) => inflationQueue.push(resolve));

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟥 Unbounded push queue exhausts proxy memory

When two pushes occupy inflation slots, inflationQueue retains every additional push's buffered PACK without a limit. Anonymous requests reach this queue before permission checks, so enough queued pushes can exhaust proxy memory.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm leaving this one without a code change, for these reasons:

  • The queue adds no memory of its own. It stores only a resolve closure per waiting push. The compressed PACK it waits on is req.bodyRaw, which extractRawBody has already fully buffered (capped at MAX_PACK_BODY_SIZE) before the chain or parsePush runs. Each waiting push therefore holds exactly what it would hold on main, or with no queue at all. The difference is that without the queue, all of those pushes would also be inflating at once, each up to MAX_INFLATED_PACK_SIZE.
  • The real gap is concurrent body buffering. How many ~100 MB bodies can be buffered at once is decided by the HTTP/route layer, before this parser is reached. Capping the queue length here would reject some requests sooner, but would not stop an attacker from holding N connections that each buffer a body.
  • The fix belongs elsewhere. That needs a concurrency or rate limit on isPackPost requests in extractRawBody, or a body/connection limit in the reverse proxy in front of GitProxy. That is a separate change with its own trade-offs, such as rejecting legitimate pushes under load, so I've flagged it for the maintainers rather than adding it to this parser fix.

I'm leaving the thread open so a maintainer can decide whether they want a route-level limit, either in this PR or as a follow-up.

Written by Devin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants