Skip to content

feat: add bulkCreate() to create large amounts of objects - #1172

Draft
nikophil wants to merge 1 commit into
zenstruck:2.xfrom
nikophil:feat/bulk-create
Draft

nikophil wants to merge 1 commit into
zenstruck:2.xfrom
nikophil:feat/bulk-create

Conversation

@nikophil

@nikophil nikophil commented Sep 22, 2026 •

Copy link
Copy Markdown
Member

Closes #883.

Creating a large amount of objects is impossible today: createMany() materializes every factory and every object upfront, so memory grows linearly with the count — 20k objects already peak at 256MB.

bulkCreate() trades the return value for constant memory: factories are yielded one at a time, and each batch is flushed then detached from the object manager.

MessageFactory::new()->many(1_000_000)->bulkCreate(batchSize: 500);

Measured on 1M entities: flat memory from start to finish (+5MB overall, 76MB peak, 478s). The same run with createMany() needs several GB and never completes.

It deliberately reimplements nothing. Hooks, events, relations and identifiers behave exactly as with create(), which is why the diff stays this small and why it works for both ORM and ODM.

Draft — not verified yet: ODM, generated proxies, and a run without dama/doctrine-test-bundle (ie. with a real commit per batch). The method name is also open for discussion.

⚠️ this is an experiment, and might not be merged

@seb-jean could you try the PR?

createMany() materializes every factory and every object upfront, so memory
grows linearly with the count and creating a large amount of them is impossible.

bulkCreate() trades the return value for constant memory: factories are yielded
one at a time, and each batch is flushed then detached from the object manager.
Comment thread src/FactoryCollection.php
throw new \LogicException('bulkCreate() can only be used with a factory which persists its objects.');
}

Configuration::instance()->persistence()->flushInBatches(

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very reluctant to couple FactoryCollection and PersistenceManager, this is one of my main problems with this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Insert a lot of data - batch processing

1 participant