Attempt to reducing CI strain through compile time

[borisdevos]

Description

I first considered splitting symmetric and non-symmetric code, but then many files would be super tiny. The next best thing I thought of was the finite-infinite split. This is in no way definitive, just wanted to get the ball rolling on considering this, especially before merging #509 which is ridiculously expensive.

Summary:

• Split the test files according to finite, infinite, window, mixed, gpu or misc testing. The workers will then work in phases, so whatever they compile within a phase is useful when they're done with a file and move on to the next file within the same phase.
• Introduced fast tests, skipping certain symmetries and scalar types. I deliberately didn't skip at the algorithmic level, but this can be considered.

None of the content or hyperparameters of tests have been changed, it's really just copy-paste, introducing some lists and truncating them depending on fast_tests, or skipping certain test sets based on this.

The downside to this approach is that all checks will take longer to finish, but at least they'll finish instead of time out. And who knows, maybe the time saved on compiling might be enough to be comparable with full parallelisation.

Checklist

• Tests pass locally (julia --project=test test/runtests.jl, or the relevant subset)
• Documentation updated, if this PR changes public API (docstrings, docs/src/)
• Runic formatter is run
• Changelog entry added under [Unreleased] in docs/src/changelog.md, if this PR is user-facing (new feature, behavior change, bug fix, deprecation, or removal)

[lkdvos]

I'd be somewhat surprised if this helps too much, it might be worth it to measure where the actual compilation is happening but I'd expect most of it to be in the tensor kernels, which are largely shared between finite and infinite implementations. A different thing could be to just split up the tests into more groups, i.e. just use more runners, for example splitting off time evolution algorithms from groundstate/leading boundary algorithms etc

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[borisdevos]

I'd be somewhat surprised if this helps too much, it might be worth it to measure where the actual compilation is happening but I'd expect most of it to be in the tensor kernels, which are largely shared between finite and infinite implementations. A different thing could be to just split up the tests into more groups, i.e. just use more runners, for example splitting off time evolution algorithms from groundstate/leading boundary algorithms etc

Yeah, testing both fast and normal mode, it seems that there's no global speedup this way. I'm looking into --trace-compile and SnoopCompile.jl to see if I can more literally catch what's taking up compile times. I'll also check how beneficial QuantumKitHub/TensorKit.jl#487 is here afterwards.

[lkdvos]

Good point, I almost forgot about that PR. I'll try and take that back up, since I do think that should help out quite severely.

[borisdevos]

I haven't tested yet with TensorKit precompilation, but at least within MPSKit the compilation bottlenecks are (unsurprisingly) eigsolve/linsolves specialised to some operator types, followed by the actual construction of some of these operators like JordanMPOTensor and JordanMPO_AC(2)_Hamiltonian. Other constructors like (in)finite hamiltonians or states also contribute, but comparatively less. I'll look now at how the tests perform with TensorKit precompilation.