Skip to content

feat: export all project contacts as CSV - #526

Closed
Pratham2511 wants to merge 1 commit into
useplunk:nextfrom
Pratham2511:codex/export-contacts-csv-461
Closed

Pratham2511 wants to merge 1 commit into
useplunk:nextfrom
Pratham2511:codex/export-contacts-csv-461

Conversation

@Pratham2511

Copy link
Copy Markdown

Description

Adds Export CSV to Contacts for the entire authenticated project, including contacts beyond the visible page and custom fields found on later pages. The dialog explains that search, filters and selected rows do not limit the export.

A dedicated BullMQ worker reads indexed batches of 100, snapshots rows to a private temporary file and uploads CSV in bounded multipart buffers. Columns are deterministic; quoting, Unicode, null/nested values and spreadsheet formula protection are covered. Authenticated downloads enforce project membership and expire after 24 hours. Queued cancellation/expiry supports immediate restart; active work retains its lock until cleanup.

Self-hosted setup and CSV semantics are documented in CONTRIBUTING.md. The worker requires temporary disk space and permissions to manage a separate private <S3_BUCKET>-contact-exports bucket. It requests one-day expiry and incomplete-upload abort rules; operators must verify backend support. Concurrent edits are not a point-in-time backup. XLSX and a column selector are outside this PR.

Type of Change

  • feat: New feature (MINOR version bump)
  • fix: Bug fix (PATCH version bump)
  • feat!: Breaking change - new feature (MAJOR version bump)
  • fix!: Breaking change - bug fix (MAJOR version bump)
  • docs: Documentation update (no version bump)
  • chore: Maintenance/dependencies (no version bump)
  • refactor: Code refactoring (no version bump)
  • test: Adding tests (no version bump)
  • perf: Performance improvement (PATCH version bump)

Testing

Independent verification on the unchanged patch, using the existing CI profile:

  • Base e0b4faf: yarn test:run — 1,387/1,387 pass.
  • Candidate: yarn test:run — 1,422/1,422 pass.
  • Focused export regressions — 38/38 pass (35 supplied tests plus three temporary reviewer tests); no skips in these three runs.
  • Prior independent lint, root API typecheck, production dashboard build, formatting and real browser/API CSV download pass. The downloaded 305-row, 13-column CSV matches the authenticated API response; tenant/authentication, formula protection, cancellation/restart and phone layout checks pass.

The earlier six local failures were reproduced in a mismatched setup. PostgreSQL used Asia/Kolkata and CI=true was absent; the cutoff timezone effect was independently reproduced. Corrected runs use Node 20.20.2, PostgreSQL 16.15, Redis 7.4.11 and the CI MinIO release, UTC, the existing CI performance profile and unchanged four-worker/30-second limits. Earlier near-floor benchmark and resource-timeout failures remain recorded; no checks, dependencies or unrelated source were changed. Native local services differ from the hosted container environment; remote candidate CI is not yet verified.

Existing unmasked build limitations remain disclosed and were not rerun: direct yarn workspace api build cannot find tsc, and wiki/combined build lacks openapi.local.json. Passing root typecheck/dashboard build does not establish an all-app build pass; CI masks its separate final combined build with || true.

Actual day-long deletion, crash-abandoned upload/temporary-file recovery, million-row heap behavior and Microsoft Excel remain unverified. The earlier Silo backend omitted the requested abort-incomplete-upload rule. Handled upload failures abort correctly; this does not establish crash cleanup.

Checklist

  • PR title follows conventional commits format
  • Code builds successfully (unmasked direct API/wiki limitations above)
  • Tests pass locally
  • Documentation updated (if needed)

Related Issues

Closes #461

Copilot AI balanced review requested due to automatic review settings October 20, 2026 03:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@driaug

driaug commented Oct 20, 2026

Copy link
Copy Markdown
Member

Hey @Pratham2511, thanks for taking this on and for the work you put into it. Contact export is a feature we want, and #461 has been open for a while.

We've decided to close this PR and build the feature ourselves. I want to explain why so it doesn't come across as a brush-off.

The changes it needs are to the design, not small fixes. The main ones:

  • Storage. We'd want exports to go through the existing S3Service, with a private key prefix and short-lived signed download URLs. This PR adds its own S3 client, which creates a bucket and changes its lifecycle rules at runtime. Our production credentials deliberately can't do that, and on AWS a bucket name derived from S3_BUCKET can collide with someone else's. You had no way of knowing about those constraints from the outside. That's on us, not you.
  • Processing. We'd want the export done in one streaming pass. The local-file snapshot and the single-concurrency worker don't hold up well on hosted Plunk, where one large project would block exports for everyone else.
  • CSV encoding. The formula escaping puts an apostrophe in front of values starting with +, which breaks phone numbers when the file is re-imported.
  • UX. Closing the dialog cancels the export. We'd want it to keep running in the background.

Fixing all of that would replace most of the PR, and going back and forth on that much is slower for both of us than us writing it with the infrastructure context we already have.

We're keeping what you got right: the overall flow (start, check status, download), the cursor-based walk over the (projectId, email) index, and the CSV quoting. Those will shape the version we ship. We'll link it back to #461 when it lands.

One tip for future contributions: smaller PRs with a short description are much easier for us to review and merge. For anything that touches infrastructure (storage, queues, new workers), opening an issue or draft PR with the planned approach first is a good way to catch constraints like these before you put the time in.

Thanks again. We'd be glad to see more from you.

@driaug driaug closed this Oct 20, 2026
@Pratham2511

Copy link
Copy Markdown
Author

Hey, no worries at all, I understand. Thanks for taking the time to explain the decisions and what would need to change. The tip about keeping PRs smaller and discussing storage or worker changes up front was really helpful, and I’ll keep that in mind for future contributions. I’m glad the cursor-based approach and parts of the flow will still be useful. I learned a lot from working on this and from your feedback, and I’d be happy to contribute again. Thanks for the opportunity!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Contacts Export Feature

3 participants