Hi, I am in a pickle. I upgraded from 0.x to 1.2.3 and I saw that the "diff" column is no longer added. I read about the data frame and the expressions, but it doesn't seem to do what I need.
I am using this (borrowed from @tillkruss from https://github.com/cachewerk/relay/blob/main/phpbench.json):
"expressions":
{
"diff": "format(\"%.2fx (%.2f%%)\", mode(suite[subject_groups = [\"routing\"]][\"result_time_avg\"]) / mode(result_time_avg), percent_diff(mode(result_time_avg), mode(suite[subject_groups = [\"routing\"]][\"result_time_avg\"])))",
"routing:mode": "mode(suite[subject_groups = [\"routing\"]][\"result_time_avg\"])"
}
The problem is that the "diff" is calculated from the mode of the whole "routing" group, which is different from the individual modes of each subject.
kt@kt-ThinkPad-T460:~/Projects/benchmark-php-routing$ php vendor/bin/phpbench run benchmark/Symfony/CompiledUrlMatcher_Benchmark.php --report=short
PHPBench (1.2.3) running benchmarks...
with configuration file: /home/kt/Projects/benchmark-php-routing/phpbench.json
with PHP version 7.4.27, xdebug ✔, opcache ❌
\Benchmark_Routing\Benchmark\Symfony\CompiledUrlMatcher_Benchmark
benchSetup..............................I4 - Mo820.453μs (±0.33%)
benchLast...............................I4 - Mo834.985μs (±1.61%)
benchLongest............................I4 - Mo842.870μs (±4.68%)
benchAll................................I4 - Mo837.447μs (±0.71%)
Subjects: 4, Assertions: 0, Failures: 0, Errors: 0
+------------------------------+--------------+-----------+-----------+-----------+----------------+-----------------+
| benchmark | subject | set | mean | mode | diff | routing:mode |
+------------------------------+--------------+-----------+-----------+-----------+----------------+-----------------+
| CompiledUrlMatcher_Benchmark | benchSetup | bitbucket | 819.937μs | 820.453μs | 1.02x (1.57%) | 833.35714285715 |
| CompiledUrlMatcher_Benchmark | benchLast | bitbucket | 841.426μs | 834.985μs | 1.00x (-0.19%) | 833.35714285715 |
| CompiledUrlMatcher_Benchmark | benchLongest | bitbucket | 870.174μs | 842.870μs | 0.99x (-1.13%) | 833.35714285715 |
| CompiledUrlMatcher_Benchmark | benchAll | bitbucket | 840.391μs | 837.447μs | 1.00x (-0.49%) | 833.35714285715 |
+------------------------------+--------------+-----------+-----------+-----------+----------------+-----------------+
See, that 833.357 value is the mode of the group. What I need is the min value from the "mode" column, e.g. the best mode from all the subjects in the group. In this case that will be 820.453 from benchSetup subject. I need to compare the best mode against all of the other modes, and in this way the diff for the best subject will be 1.00x.
Hi, I am in a pickle. I upgraded from 0.x to 1.2.3 and I saw that the "diff" column is no longer added. I read about the data frame and the expressions, but it doesn't seem to do what I need.
I am using this (borrowed from @tillkruss from https://github.com/cachewerk/relay/blob/main/phpbench.json):
The problem is that the "diff" is calculated from the mode of the whole "routing" group, which is different from the individual modes of each subject.
See, that 833.357 value is the mode of the group. What I need is the min value from the "mode" column, e.g. the best mode from all the subjects in the group. In this case that will be 820.453 from
benchSetupsubject. I need to compare the best mode against all of the other modes, and in this way the diff for the best subject will be 1.00x.