Repository navigation
fix: validate pack POSTs before buffering and cap body size (bug) #48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
570b076
fa27846
90cb45f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,8 +19,12 @@ | |
| import { PassThrough } from 'stream'; | ||
| import getRawBody from 'raw-body'; | ||
| import { executeChain } from '../chain'; | ||
| import { exec as parseAction } from '../processors/pre-processor/parseAction'; | ||
| import { exec as checkRepoInAuthorisedList } from '../processors/push-action/checkRepoInAuthorisedList'; | ||
| import { exec as audit } from '../processors/post-processor/audit'; | ||
| import { processUrlPath, validGitRequest } from './helper'; | ||
| import { getAllProxiedHosts } from '../../db'; | ||
| import { getMaxPackSizeBytes } from '../../config'; | ||
| import { ProxyOptions } from 'express-http-proxy'; | ||
| import { handleErrorAndLog } from '../../utils/errors'; | ||
|
|
||
|
|
@@ -162,41 +166,80 @@ | |
| req.method === 'POST' && | ||
| /^(?:\/[^/]+)*\/[^/]+\.git\/(?:git-upload-pack|git-receive-pack)$/.test(req.url); | ||
|
|
||
| const rejectPackPost = (req: Request, res: Response, message: string): void => { | ||
| logAction(req.url, req.headers.host, req.headers['user-agent'], ActionType.ERROR, message); | ||
| sendErrorResponse(req, res, message); | ||
| }; | ||
|
|
||
| const rejectTooLarge = (req: Request, res: Response, limit: number): void => { | ||
| const message = `Request body exceeds the maximum pack size of ${limit} bytes`; | ||
| logAction(req.url, req.headers.host, req.headers['user-agent'], ActionType.ERROR, message); | ||
| res.set('connection', 'close'); | ||
| res.status(413).end(message); | ||
| }; | ||
|
|
||
| const extractRawBody = async (req: Request, res: Response, next: NextFunction) => { | ||
| if (!isPackPost(req)) { | ||
| return next(); | ||
| } | ||
|
|
||
| const proxyStream = new PassThrough({ | ||
| highWaterMark: 4 * 1024 * 1024, | ||
| }); | ||
| const pluginStream = new PassThrough({ | ||
| highWaterMark: 4 * 1024 * 1024, | ||
| }); | ||
| // run the cheap checks before reading any of the body into memory | ||
| const urlComponents = processUrlPath(req.url); | ||
| if ( | ||
| !urlComponents || | ||
| urlComponents.gitPath === undefined || | ||
| !validGitRequest(urlComponents.gitPath, req.headers) | ||
| ) { | ||
| return rejectPackPost(req, res, 'Invalid request received'); | ||
| } | ||
|
|
||
| req.pipe(proxyStream); | ||
| req.pipe(pluginStream); | ||
| try { | ||
| const action = await checkRepoInAuthorisedList(req, await parseAction(req)); | ||
| if (!action.continue()) { | ||
| // keep the audit record the action chain would have written, without reading the body | ||
| await audit(req, action); | ||
|
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟨 Unauthenticated denials exhaust audit storage An unauthenticated client can send bodyless Was this helpful? React with 👍 or 👎 to provide feedback.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The behavior exists, but this PR doesn't introduce it. On The two possible changes pull in opposite directions:
Both need a maintainer decision, so I'm leaving behavior unchanged here and flagging it for whoever owns the policy. |
||
| return rejectPackPost( | ||
| req, | ||
| res, | ||
| action.errorMessage ?? 'Rejecting repo not in the authorised whitelist', | ||
| ); | ||
| } | ||
| } catch (error: unknown) { | ||
| const message = handleErrorAndLog(error, 'Error checking repository authorisation'); | ||
| return rejectPackPost(req, res, message); | ||
| } | ||
|
|
||
| const limit = getMaxPackSizeBytes(); | ||
| const contentLength = Number(req.headers['content-length']); | ||
| if (Number.isFinite(contentLength) && contentLength > limit) { | ||
| return rejectTooLarge(req, res, limit); | ||
| } | ||
|
|
||
| try { | ||
| const buf = await getRawBody(pluginStream, { limit: '1gb' }); | ||
| // read straight from the request so backpressure only depends on this reader; the | ||
| // buffered body is replayed to the upstream proxy once the plugins have inspected it | ||
| const buf = await getRawBody(req, { limit }); | ||
| req.bodyRaw = buf; | ||
| req.pipe = (dest, opts) => proxyStream.pipe(dest, opts); | ||
| req.pipe = (dest, opts) => { | ||
| const replay = new PassThrough(); | ||
| replay.end(buf); | ||
| return replay.pipe(dest, opts); | ||
| }; | ||
| next(); | ||
| } catch (error: unknown) { | ||
| if (error instanceof Error) { | ||
| console.error(error.message); | ||
| proxyStream.destroy(error); | ||
| } else { | ||
| console.error(String(error)); | ||
| proxyStream.destroy(new Error(String(error))); | ||
| const err = error instanceof Error ? error : new Error(String(error)); | ||
| console.error(err.message); | ||
| if ((err as { type?: string }).type === 'entity.too.large') { | ||
| return rejectTooLarge(req, res, limit); | ||
| } | ||
| res.set('connection', 'close'); | ||
| res.status(500).end('Proxy error'); | ||
| } | ||
| }; | ||
|
|
||
| const getRouter = async () => { | ||
| const router = Router(); | ||
| router.use(extractRawBody); | ||
Check failureCode scanning / CodeQL Missing rate limiting High
This route handler performs
authorization Error loading related location Loading |
||
|
|
||
| const originsToProxy = await getAllProxiedHosts(); | ||
| const proxyKeys: string[] = []; | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 Pack-size setting missing from schema reference
Adding
maxPackSizeBytesleaves the generated website reference unchanged, despite the repository's required schema documentation update. Readers cannot discover the configurable limit there.Learn more
The website's schema reference is generated from the JSON schema. Contributing guidance requires regenerating it whenever a configuration property changes. The new property has no entry in the existing reference, leaving the public setting and its default undocumented there.
Example: An operator consulting the website schema reference after encountering HTTP 413 cannot find
maxPackSizeBytesor its default value.Recommended fix: Run
npm run gen-schema-docand commit the resulting update to the website reference with the schema change.Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed this should be regenerated.
npm run gen-schema-doccouldn't run in my environment: npm and pip registries are unreachable, sojson-schema-for-humansisn't installed. Editingreference.mdxby hand would mean renumbering every section afterrateLimit, which the generator should do instead. I'm leaving this thread open so the docs get regenerated before merge. It's flagged as an unchecked item in the PR checklist.