Skip to content

Parallelize issue hashes computation - #3639

Open
marco-c wants to merge 1 commit into
mozilla:masterfrom
marco-c:parallel_hashes
Open

marco-c wants to merge 1 commit into
mozilla:masterfrom
marco-c:parallel_hashes

Conversation

@marco-c

@marco-c marco-c commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Fixes #3638

@marco-c
marco-c force-pushed the parallel_hashes branch 2 times, most recently from 8abef66 to 92f7369 Compare September 14, 2026 17:09
@marco-c
marco-c marked this pull request as ready for review September 14, 2026 17:17
@marco-c
marco-c requested a review from La0 September 14, 2026 17:17

@La0 La0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not much to say on the refactoring, I'll test on a revision and report on perf.

Comment thread bot/code_review_bot/__init__.py
self.patch = patch

@property
def local_repository(self):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: could be named local_repository_path to explicitly state this does not provide a Repository instance

@La0

La0 commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

On a clean repo (full checkout) on that branch:

real	8m11,888s
user	4m44,746s
sys	2m2,043s

Second run (no checkout):

real	0m30,358s
user	0m6,765s
sys	0m4,917s

On current master 7bbadf2 , warm cache (no checkout):

real	0m27,762s
user	0m7,051s
sys	0m4,654s

That test revision is pretty inconclusive as it had very few hashes to calc (almost all issues silent). I'll retry with a revision with more publishable issues

@marco-c

marco-c commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator Author

The case I saw was in https://firefoxci.taskcluster-artifacts.net/ErAfZ8x-Tymdm4O5QCsU8A/0/public/logs/live_backing.log, you could try with that revision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

We are spending a lot of time computing issue hashes

2 participants