Skip to content

Provide report options to separate benchmarks by groups #115

Description

@sobolevn

Hi, I am using codspeed in my project: https://github.com/wemake-services/django-modern-rest / https://codspeed.io/wemake-services/django-modern-rest

And I really like the experience. Thanks a lot for building it!

I have several different benchmarks for different parts of the app: https://github.com/wemake-services/django-modern-rest/tree/master/benchmarks/tests

Each benchmark is in its own test module.
Currently the report looks like so:

Image

It is already hard to compare different benchmarks. But, we plan to add even more benches.

My proposal is to add an option to group the final report based on modules.

So it would be like so: --codspeed-group=module --codspeed-sort=slowest:

                                 Benchmark Results                                 
                                 test_router.py                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃                      Benchmark ┃ Time (best) ┃ Rel. StdDev ┃ Run time ┃   Iters ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│           test_negotiation_raw │        19ns │        2.5% │    2.95s │ 102,737 │
│        test_router_path_native │      13.1ms │        0.6% │    2.98s │     112 │
│           test_router_path_dmr │      10.9ms │        1.9% │    2.97s │     134 │
└────────────────────────────────┴─────────────┴─────────────┴──────────┴─────────┘

                                 test_negotiation.py                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃                      Benchmark ┃ Time (best) ┃ Rel. StdDev ┃ Run time ┃   Iters ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ test_negotiation_django_native │       275ns │        2.0% │    2.93s │  27,051 │
│      test_negotiation_compiled │         8ns │        2.6% │    2.87s │ 153,786 │
└────────────────────────────────┴─────────────┴─────────────┴──────────┴─────────┘

Now - everything is clear:

  • Everything is groupped by module from --codspeed-group=module
  • Everything is sorted by the Time (best) from --codspeed-sort=slowest

What do you think? :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions