Collection Benchmarks by dougqh · Pull Request #10674 · DataDog/dd-trace-java

dougqh · 2026-02-24T20:46:56Z

What Does This Do

Benchmarks used to demonstrate most performant approach to common tracer programming problems
Each benchmark includes a header comment that includes benchmark results and the conclusions drawn from those results.

Motivation

The benchmarks are intended to serve as a reference for developers contributing to dd-java-agent.
The benchmarks are intended to be a living example that will be updated as new solutions are found / created.

Additional Notes

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

… situations

github-actions · 2026-02-24T20:47:10Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

dougqh · 2026-02-24T20:49:56Z

internal-api/src/jmh/java/datadog/trace/util/CaseInsensitiveMapBenchmark.java

+@Fork(2)
+@Warmup(iterations = 2)
+@Measurement(iterations = 3)
+@Threads(8)


I've chosen to run the benchmarks with 8 threads because I found that the total system impact was often hidden when only using a single thread.

With 8 threads, solutions that perform more allocation tend to fall behind because of stop-the-world pauses caused by garbage collection.

The thinking is that in a real world system, the tracer is often running inside many application threads, so designing for many threads typically makes sense.

dougqh · 2026-02-24T20:50:37Z

internal-api/src/jmh/java/datadog/trace/util/HashingBenchmark.java

- * common situation where the JIT's escape analysis is unable to elide the var-arg array allocation.
+ *
+ *
+ * <ul>


Tweaked this benchmark to match comment to adhere to the same format as the others

pr-commenter · 2026-02-24T21:29:26Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	dougqh/collection-benchmarks
git_commit_date	1772200579	1772203028
git_commit_sha	`c6896b7`	`cf3b9cb`
release_version	1.60.0-SNAPSHOT~c6896b7cf7	1.60.0-SNAPSHOT~cf3b9cb4d8

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1772204670	1772204670
ci_job_id	1462741116	1462741116
ci_pipeline_id	99430715	99430715
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-l43617uq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-l43617uq 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 11 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~cf3b9cb4d8, baseline=1.60.0-SNAPSHOT~c6896b7cf7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.068 s) : 0, 1068123
Total [baseline] (10.91 s) : 0, 10909902
Agent [candidate] (1.071 s) : 0, 1071392
Total [candidate] (11.061 s) : 0, 11061177
section appsec
Agent [baseline] (1.24 s) : 0, 1240157
Total [baseline] (11.045 s) : 0, 11045433
Agent [candidate] (1.242 s) : 0, 1241902
Total [candidate] (11.021 s) : 0, 11021403
section iast
Agent [baseline] (1.232 s) : 0, 1232246
Total [baseline] (11.229 s) : 0, 11229473
Agent [candidate] (1.232 s) : 0, 1232047
Total [candidate] (11.15 s) : 0, 11150092
section profiling
Agent [baseline] (1.193 s) : 0, 1192979
Total [baseline] (10.927 s) : 0, 10926815
Agent [candidate] (1.193 s) : 0, 1192836
Total [candidate] (10.973 s) : 0, 10972935

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.068 s	-
Agent	appsec	1.24 s	172.034 ms (16.1%)
Agent	iast	1.232 s	164.122 ms (15.4%)
Agent	profiling	1.193 s	124.856 ms (11.7%)
Total	tracing	10.91 s	-
Total	appsec	11.045 s	135.53 ms (1.2%)
Total	iast	11.229 s	319.571 ms (2.9%)
Total	profiling	10.927 s	16.913 ms (0.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.071 s	-
Agent	appsec	1.242 s	170.51 ms (15.9%)
Agent	iast	1.232 s	160.655 ms (15.0%)
Agent	profiling	1.193 s	121.444 ms (11.3%)
Total	tracing	11.061 s	-
Total	appsec	11.021 s	-39.774 ms (-0.4%)
Total	iast	11.15 s	88.915 ms (0.8%)
Total	profiling	10.973 s	-88.242 ms (-0.8%)

gantt
    title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~cf3b9cb4d8, baseline=1.60.0-SNAPSHOT~c6896b7cf7

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.201 ms) : 0, 1201
crashtracking [candidate] (1.208 ms) : 0, 1208
BytebuddyAgent [baseline] (630.448 ms) : 0, 630448
BytebuddyAgent [candidate] (633.299 ms) : 0, 633299
AgentMeter [baseline] (29.077 ms) : 0, 29077
AgentMeter [candidate] (29.412 ms) : 0, 29412
GlobalTracer [baseline] (257.725 ms) : 0, 257725
GlobalTracer [candidate] (259.876 ms) : 0, 259876
AppSec [baseline] (33.004 ms) : 0, 33004
AppSec [candidate] (33.212 ms) : 0, 33212
Debugger [baseline] (63.705 ms) : 0, 63705
Debugger [candidate] (63.224 ms) : 0, 63224
Remote Config [baseline] (621.319 µs) : 0, 621
Remote Config [candidate] (608.432 µs) : 0, 608
Telemetry [baseline] (10.82 ms) : 0, 10820
Telemetry [candidate] (9.089 ms) : 0, 9089
Flare Poller [baseline] (5.341 ms) : 0, 5341
Flare Poller [candidate] (5.223 ms) : 0, 5223
section appsec
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (658.898 ms) : 0, 658898
BytebuddyAgent [candidate] (659.383 ms) : 0, 659383
AgentMeter [baseline] (12.005 ms) : 0, 12005
AgentMeter [candidate] (12.044 ms) : 0, 12044
GlobalTracer [baseline] (258.837 ms) : 0, 258837
GlobalTracer [candidate] (259.016 ms) : 0, 259016
AppSec [baseline] (167.724 ms) : 0, 167724
AppSec [candidate] (168.318 ms) : 0, 168318
Debugger [baseline] (66.618 ms) : 0, 66618
Debugger [candidate] (66.14 ms) : 0, 66140
Remote Config [baseline] (666.039 µs) : 0, 666
Remote Config [candidate] (674.401 µs) : 0, 674
Telemetry [baseline] (9.268 ms) : 0, 9268
Telemetry [candidate] (9.425 ms) : 0, 9425
Flare Poller [baseline] (3.601 ms) : 0, 3601
Flare Poller [candidate] (4.333 ms) : 0, 4333
IAST [baseline] (25.358 ms) : 0, 25358
IAST [candidate] (25.3 ms) : 0, 25300
section iast
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.187 ms) : 0, 1187
BytebuddyAgent [baseline] (795.596 ms) : 0, 795596
BytebuddyAgent [candidate] (795.18 ms) : 0, 795180
AgentMeter [baseline] (11.358 ms) : 0, 11358
AgentMeter [candidate] (11.326 ms) : 0, 11326
GlobalTracer [baseline] (247.693 ms) : 0, 247693
GlobalTracer [candidate] (247.612 ms) : 0, 247612
AppSec [baseline] (34.014 ms) : 0, 34014
AppSec [candidate] (33.239 ms) : 0, 33239
Debugger [baseline] (66.738 ms) : 0, 66738
Debugger [candidate] (67.7 ms) : 0, 67700
Remote Config [baseline] (558.368 µs) : 0, 558
Remote Config [candidate] (529.168 µs) : 0, 529
Telemetry [baseline] (8.665 ms) : 0, 8665
Telemetry [candidate] (8.626 ms) : 0, 8626
Flare Poller [baseline] (3.448 ms) : 0, 3448
Flare Poller [candidate] (3.512 ms) : 0, 3512
IAST [baseline] (26.949 ms) : 0, 26949
IAST [candidate] (27.039 ms) : 0, 27039
section profiling
ProfilingAgent [baseline] (99.43 ms) : 0, 99430
ProfilingAgent [candidate] (98.458 ms) : 0, 98458
crashtracking [baseline] (1.16 ms) : 0, 1160
crashtracking [candidate] (1.169 ms) : 0, 1169
BytebuddyAgent [baseline] (683.355 ms) : 0, 683355
BytebuddyAgent [candidate] (684.297 ms) : 0, 684297
AgentMeter [baseline] (8.597 ms) : 0, 8597
AgentMeter [candidate] (8.573 ms) : 0, 8573
GlobalTracer [baseline] (216.446 ms) : 0, 216446
GlobalTracer [candidate] (216.233 ms) : 0, 216233
AppSec [baseline] (32.646 ms) : 0, 32646
AppSec [candidate] (32.526 ms) : 0, 32526
Debugger [baseline] (67.163 ms) : 0, 67163
Debugger [candidate] (67.149 ms) : 0, 67149
Remote Config [baseline] (616.971 µs) : 0, 617
Remote Config [candidate] (615.74 µs) : 0, 616
Telemetry [baseline] (8.914 ms) : 0, 8914
Telemetry [candidate] (8.844 ms) : 0, 8844
Flare Poller [baseline] (3.815 ms) : 0, 3815
Flare Poller [candidate] (3.759 ms) : 0, 3759
Profiling [baseline] (99.996 ms) : 0, 99996
Profiling [candidate] (99.033 ms) : 0, 99033

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~cf3b9cb4d8, baseline=1.60.0-SNAPSHOT~c6896b7cf7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.072 s) : 0, 1072373
Total [baseline] (8.801 s) : 0, 8800908
Agent [candidate] (1.068 s) : 0, 1068094
Total [candidate] (8.764 s) : 0, 8764232
section iast
Agent [baseline] (1.242 s) : 0, 1242346
Total [baseline] (9.421 s) : 0, 9420912
Agent [candidate] (1.234 s) : 0, 1233875
Total [candidate] (9.367 s) : 0, 9367400

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.072 s	-
Agent	iast	1.242 s	169.973 ms (15.9%)
Total	tracing	8.801 s	-
Total	iast	9.421 s	620.004 ms (7.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.068 s	-
Agent	iast	1.234 s	165.781 ms (15.5%)
Total	tracing	8.764 s	-
Total	iast	9.367 s	603.168 ms (6.9%)

gantt
    title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~cf3b9cb4d8, baseline=1.60.0-SNAPSHOT~c6896b7cf7

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.197 ms) : 0, 1197
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (631.725 ms) : 0, 631725
BytebuddyAgent [candidate] (629.838 ms) : 0, 629838
AgentMeter [baseline] (29.223 ms) : 0, 29223
AgentMeter [candidate] (29.122 ms) : 0, 29122
GlobalTracer [baseline] (258.862 ms) : 0, 258862
GlobalTracer [candidate] (258.075 ms) : 0, 258075
AppSec [baseline] (33.249 ms) : 0, 33249
AppSec [candidate] (33.168 ms) : 0, 33168
Debugger [baseline] (65.045 ms) : 0, 65045
Debugger [candidate] (64.427 ms) : 0, 64427
Remote Config [baseline] (626.36 µs) : 0, 626
Remote Config [candidate] (610.316 µs) : 0, 610
Telemetry [baseline] (11.593 ms) : 0, 11593
Telemetry [candidate] (10.115 ms) : 0, 10115
Flare Poller [baseline] (4.589 ms) : 0, 4589
Flare Poller [candidate] (5.383 ms) : 0, 5383
section iast
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.212 ms) : 0, 1212
BytebuddyAgent [baseline] (804.023 ms) : 0, 804023
BytebuddyAgent [candidate] (797.479 ms) : 0, 797479
AgentMeter [baseline] (11.393 ms) : 0, 11393
AgentMeter [candidate] (11.324 ms) : 0, 11324
GlobalTracer [baseline] (249.908 ms) : 0, 249908
GlobalTracer [candidate] (248.477 ms) : 0, 248477
IAST [baseline] (27.288 ms) : 0, 27288
IAST [candidate] (27.127 ms) : 0, 27127
AppSec [baseline] (32.439 ms) : 0, 32439
AppSec [candidate] (33.176 ms) : 0, 33176
Debugger [baseline] (67.601 ms) : 0, 67601
Debugger [candidate] (66.563 ms) : 0, 66563
Remote Config [baseline] (540.534 µs) : 0, 541
Remote Config [candidate] (529.883 µs) : 0, 530
Telemetry [baseline] (8.563 ms) : 0, 8563
Telemetry [candidate] (8.566 ms) : 0, 8566
Flare Poller [baseline] (3.447 ms) : 0, 3447
Flare Poller [candidate] (3.434 ms) : 0, 3434

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	dougqh/collection-benchmarks
git_commit_date	1772200579	1772203028
git_commit_sha	`c6896b7`	`cf3b9cb`
release_version	1.60.0-SNAPSHOT~c6896b7cf7	1.60.0-SNAPSHOT~cf3b9cb4d8

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1772205248	1772205248
ci_job_id	1462741119	1462741119
ci_pipeline_id	99430715	99430715
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-p6swfw6w 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-p6swfw6w 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 1 performance regressions! Performance is the same for 15 metrics, 18 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:code_origins:high_load	better [-1.548ms; -0.796ms] or [-8.498%; -4.370%]	better [-2.377ms; -1.247ms] or [-8.044%; -4.219%]	unstable [-10.808op/s; +42.370op/s] or [-4.295%; +16.839%]	17.043ms	27.733ms	267.406op/s	18.215ms	29.545ms	251.625op/s
scenario:load:petclinic:appsec:high_load	worse [+552.794µs; +1360.674µs] or [+3.026%; +7.447%]	same [-214.837µs; +1542.774µs] or [-0.717%; +5.146%]	unstable [-35.012op/s; +15.512op/s] or [-14.007%; +6.206%]	19.228ms	30.644ms	240.219op/s	18.271ms	29.980ms	249.969op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~cf3b9cb4d8, baseline=1.60.0-SNAPSHOT~c6896b7cf7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.163 ms) : 1151, 1175
.   : milestone, 1163,
iast (3.064 ms) : 3020, 3107
.   : milestone, 3064,
iast_FULL (5.861 ms) : 5802, 5921
.   : milestone, 5861,
iast_GLOBAL (3.587 ms) : 3528, 3646
.   : milestone, 3587,
profiling (2.147 ms) : 2127, 2167
.   : milestone, 2147,
tracing (1.861 ms) : 1843, 1880
.   : milestone, 1861,
section candidate
no_agent (1.166 ms) : 1155, 1178
.   : milestone, 1166,
iast (3.115 ms) : 3077, 3153
.   : milestone, 3115,
iast_FULL (5.806 ms) : 5748, 5864
.   : milestone, 5806,
iast_GLOBAL (3.589 ms) : 3530, 3647
.   : milestone, 3589,
profiling (2.064 ms) : 2045, 2082
.   : milestone, 2064,
tracing (1.78 ms) : 1766, 1795
.   : milestone, 1780,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.163 ms [1.151 ms, 1.175 ms]	-
iast	3.064 ms [3.02 ms, 3.107 ms]	1.901 ms (163.4%)
iast_FULL	5.861 ms [5.802 ms, 5.921 ms]	4.698 ms (404.0%)
iast_GLOBAL	3.587 ms [3.528 ms, 3.646 ms]	2.424 ms (208.4%)
profiling	2.147 ms [2.127 ms, 2.167 ms]	984.113 µs (84.6%)
tracing	1.861 ms [1.843 ms, 1.88 ms]	698.36 µs (60.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.166 ms [1.155 ms, 1.178 ms]	-
iast	3.115 ms [3.077 ms, 3.153 ms]	1.949 ms (167.0%)
iast_FULL	5.806 ms [5.748 ms, 5.864 ms]	4.64 ms (397.8%)
iast_GLOBAL	3.589 ms [3.53 ms, 3.647 ms]	2.422 ms (207.7%)
profiling	2.064 ms [2.045 ms, 2.082 ms]	897.117 µs (76.9%)
tracing	1.78 ms [1.766 ms, 1.795 ms]	614.038 µs (52.6%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~cf3b9cb4d8, baseline=1.60.0-SNAPSHOT~c6896b7cf7
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.924 ms) : 18727, 19122
.   : milestone, 18924,
appsec (18.67 ms) : 18478, 18862
.   : milestone, 18670,
code_origins (18.543 ms) : 18356, 18730
.   : milestone, 18543,
iast (17.962 ms) : 17785, 18139
.   : milestone, 17962,
profiling (19.065 ms) : 18875, 19255
.   : milestone, 19065,
tracing (17.493 ms) : 17323, 17663
.   : milestone, 17493,
section candidate
no_agent (19.384 ms) : 19180, 19587
.   : milestone, 19384,
appsec (19.435 ms) : 19239, 19631
.   : milestone, 19435,
code_origins (17.447 ms) : 17276, 17617
.   : milestone, 17447,
iast (18.134 ms) : 17953, 18314
.   : milestone, 18134,
profiling (18.583 ms) : 18394, 18772
.   : milestone, 18583,
tracing (17.683 ms) : 17507, 17859
.   : milestone, 17683,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.924 ms [18.727 ms, 19.122 ms]	-
appsec	18.67 ms [18.478 ms, 18.862 ms]	-254.472 µs (-1.3%)
code_origins	18.543 ms [18.356 ms, 18.73 ms]	-381.493 µs (-2.0%)
iast	17.962 ms [17.785 ms, 18.139 ms]	-962.642 µs (-5.1%)
profiling	19.065 ms [18.875 ms, 19.255 ms]	140.897 µs (0.7%)
tracing	17.493 ms [17.323 ms, 17.663 ms]	-1.431 ms (-7.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.384 ms [19.18 ms, 19.587 ms]	-
appsec	19.435 ms [19.239 ms, 19.631 ms]	51.282 µs (0.3%)
code_origins	17.447 ms [17.276 ms, 17.617 ms]	-1.937 ms (-10.0%)
iast	18.134 ms [17.953 ms, 18.314 ms]	-1.25 ms (-6.4%)
profiling	18.583 ms [18.394 ms, 18.772 ms]	-800.631 µs (-4.1%)
tracing	17.683 ms [17.507 ms, 17.859 ms]	-1.701 ms (-8.8%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	dougqh/collection-benchmarks
git_commit_date	1772200579	1772203028
git_commit_sha	`c6896b7`	`cf3b9cb`
release_version	1.60.0-SNAPSHOT~c6896b7cf7	1.60.0-SNAPSHOT~cf3b9cb4d8

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1772204965	1772204965
ci_job_id	1462741122	1462741122
ci_pipeline_id	99430715	99430715
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-7wk0uvxs 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-7wk0uvxs 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~cf3b9cb4d8, baseline=1.60.0-SNAPSHOT~c6896b7cf7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (2.498 ms) : 2444, 2552
.   : milestone, 2498,
iast (2.248 ms) : 2179, 2317
.   : milestone, 2248,
iast_GLOBAL (2.292 ms) : 2223, 2361
.   : milestone, 2292,
profiling (2.09 ms) : 2035, 2144
.   : milestone, 2090,
tracing (2.057 ms) : 2005, 2110
.   : milestone, 2057,
section candidate
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (3.691 ms) : 3475, 3906
.   : milestone, 3691,
iast (2.237 ms) : 2168, 2305
.   : milestone, 2237,
iast_GLOBAL (2.288 ms) : 2219, 2357
.   : milestone, 2288,
profiling (2.1 ms) : 2044, 2156
.   : milestone, 2100,
tracing (2.056 ms) : 2002, 2109
.   : milestone, 2056,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.473 ms [1.461 ms, 1.484 ms]	-
appsec	2.498 ms [2.444 ms, 2.552 ms]	1.025 ms (69.6%)
iast	2.248 ms [2.179 ms, 2.317 ms]	775.261 µs (52.6%)
iast_GLOBAL	2.292 ms [2.223 ms, 2.361 ms]	819.133 µs (55.6%)
profiling	2.09 ms [2.035 ms, 2.144 ms]	616.944 µs (41.9%)
tracing	2.057 ms [2.005 ms, 2.11 ms]	584.82 µs (39.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.46 ms, 1.483 ms]	-
appsec	3.691 ms [3.475 ms, 3.906 ms]	2.219 ms (150.8%)
iast	2.237 ms [2.168 ms, 2.305 ms]	765.388 µs (52.0%)
iast_GLOBAL	2.288 ms [2.219 ms, 2.357 ms]	816.734 µs (55.5%)
profiling	2.1 ms [2.044 ms, 2.156 ms]	628.795 µs (42.7%)
tracing	2.056 ms [2.002 ms, 2.109 ms]	584.178 µs (39.7%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~cf3b9cb4d8, baseline=1.60.0-SNAPSHOT~c6896b7cf7
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.004 s) : 15004000, 15004000
.   : milestone, 15004000,
appsec (14.958 s) : 14958000, 14958000
.   : milestone, 14958000,
iast (18.313 s) : 18313000, 18313000
.   : milestone, 18313000,
iast_GLOBAL (17.878 s) : 17878000, 17878000
.   : milestone, 17878000,
profiling (15.408 s) : 15408000, 15408000
.   : milestone, 15408000,
tracing (14.754 s) : 14754000, 14754000
.   : milestone, 14754000,
section candidate
no_agent (15.109 s) : 15109000, 15109000
.   : milestone, 15109000,
appsec (15.113 s) : 15113000, 15113000
.   : milestone, 15113000,
iast (18.042 s) : 18042000, 18042000
.   : milestone, 18042000,
iast_GLOBAL (17.508 s) : 17508000, 17508000
.   : milestone, 17508000,
profiling (14.982 s) : 14982000, 14982000
.   : milestone, 14982000,
tracing (14.814 s) : 14814000, 14814000
.   : milestone, 14814000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.004 s [15.004 s, 15.004 s]	-
appsec	14.958 s [14.958 s, 14.958 s]	-46.0 ms (-0.3%)
iast	18.313 s [18.313 s, 18.313 s]	3.309 s (22.1%)
iast_GLOBAL	17.878 s [17.878 s, 17.878 s]	2.874 s (19.2%)
profiling	15.408 s [15.408 s, 15.408 s]	404.0 ms (2.7%)
tracing	14.754 s [14.754 s, 14.754 s]	-250.0 ms (-1.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.109 s [15.109 s, 15.109 s]	-
appsec	15.113 s [15.113 s, 15.113 s]	4.0 ms (0.0%)
iast	18.042 s [18.042 s, 18.042 s]	2.933 s (19.4%)
iast_GLOBAL	17.508 s [17.508 s, 17.508 s]	2.399 s (15.9%)
profiling	14.982 s [14.982 s, 14.982 s]	-127.0 ms (-0.8%)
tracing	14.814 s [14.814 s, 14.814 s]	-295.0 ms (-2.0%)

dougqh added 2 commits February 24, 2026 15:43

Adding benchmarks to illustrate recommended approach to common tracer…

73b19dd

… situations

spotless

bba3921

dougqh requested a review from a team as a code owner February 24, 2026 20:46

dougqh requested a review from amarziali February 24, 2026 20:46

dougqh commented Feb 24, 2026

View reviewed changes

Merge branch 'master' into dougqh/collection-benchmarks

53901eb

dougqh added comp: core Tracer core tag: performance Performance related changes tag: diagnostics Diagnostics related changes type: enhancement Enhancements and improvements labels Feb 26, 2026

Merge branch 'master' into dougqh/collection-benchmarks

cf3b9cb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Collection Benchmarks#10674

Collection Benchmarks#10674
dougqh wants to merge 4 commits intomasterfrom
dougqh/collection-benchmarks

dougqh commented Feb 24, 2026

Uh oh!

github-actions bot commented Feb 24, 2026 •

edited

Loading

Uh oh!

dougqh Feb 24, 2026

Uh oh!

dougqh Feb 24, 2026

Uh oh!

pr-commenter bot commented Feb 24, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

dougqh commented Feb 24, 2026

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions bot commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dougqh Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

dougqh Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

pr-commenter bot commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions bot commented Feb 24, 2026 •

edited

Loading

pr-commenter bot commented Feb 24, 2026 •

edited

Loading