Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
store: Optimized common cases for time selecting tiny amount series. Avoid looking up symbols. #3531
Conversation
…Avoid looking up symbols. Key part: Majority of query latency comes from symbols.Lookup. Also cleaned up the code. ## Benchmarks Baseline: ``` /tmp/___BenchmarkTelemeterRealData_Series_in_github_com_thanos_io_thanos_pkg_store -test.v -test.bench ^\QBenchmarkTelemeterRealData_Series\E$ -test.run ^$ -test.benchtime=1m -test.benchmem goos: linux goarch: amd64 pkg: github.com/thanos-io/thanos/pkg/store BenchmarkTelemeterRealData_Series Built index header; Starting BenchmarkTelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 43 1866550478 ns/op 395648807 B/op 4348808 allocs/op BenchmarkTelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 50 1417560470 ns/op 160595177 B/op 3017177 allocs/op BenchmarkTelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 392 168130862 ns/op 49921099 B/op 493076 allocs/op BenchmarkTelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 14 5295236741 ns/op 1083815895 B/op 11798226 allocs/op BenchmarkTelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 18 4149971387 ns/op 436568360 B/op 8577264 allocs/op PASS Process finished with exit code 0 ``` New ``` benchstat -delta-test=none _dev/bench/store_symb_a.txt _dev/bench/store_symb_b.txt name old time/op new time/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 1.87s ± 0% 1.66s ± 0% -10.85% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 1.42s ± 0% 0.25s ± 0% -82.17% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 168ms ± 0% 157ms ± 0% -6.72% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 5.30s ± 0% 4.53s ± 0% -14.53% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 4.15s ± 0% 0.68s ± 0% -83.67% name old alloc/op new alloc/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 396MB ± 0% 396MB ± 0% -0.00% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 161MB ± 0% 95MB ± 0% -40.95% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 49.9MB ± 0% 49.9MB ± 0% +0.00% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 1.08GB ± 0% 1.08GB ± 0% -0.00% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 437MB ± 0% 228MB ± 0% -47.77% name old allocs/op new allocs/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 4.35M ± 0% 4.35M ± 0% -0.00% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 3.02M ± 0% 0.05M ± 0% -98.36% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 493k ± 0% 493k ± 0% +0.00% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 11.8M ± 0% 11.8M ± 0% -0.00% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 8.58M ± 0% 0.02M ± 0% -99.78% ``` Cmp: ``` benchstat -delta-test=none _dev/bench/store_symb_a.txt _dev/bench/store_symb_b.txt name old time/op new time/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 1.87s ± 0% 1.66s ± 0% -10.85% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 1.42s ± 0% 0.25s ± 0% -82.17% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 168ms ± 0% 157ms ± 0% -6.72% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 5.30s ± 0% 4.53s ± 0% -14.53% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 4.15s ± 0% 0.68s ± 0% -83.67% name old alloc/op new alloc/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 396MB ± 0% 396MB ± 0% -0.00% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 161MB ± 0% 95MB ± 0% -40.95% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 49.9MB ± 0% 49.9MB ± 0% +0.00% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 1.08GB ± 0% 1.08GB ± 0% -0.00% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 437MB ± 0% 228MB ± 0% -47.77% name old allocs/op new allocs/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 4.35M ± 0% 4.35M ± 0% -0.00% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 3.02M ± 0% 0.05M ± 0% -98.36% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 493k ± 0% 493k ± 0% +0.00% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 11.8M ± 0% 11.8M ± 0% -0.00% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 8.58M ± 0% 0.02M ± 0% -99.78% ``` Code: https://gist.github.com/bwplotka/cbcbbcd1802181b7785da11dcc0f5cfd Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Key rationales: The majority of query latency comes from symbols.Lookup.
Also cleaned up the code.
Benchmarks
Baseline:
``
New
Cmp:
Code: https://gist.github.com/bwplotka/cbcbbcd1802181b7785da11dcc0f5cfd
Signed-off-by: Bartlomiej Plotka bwplotka@gmail.com