Skip to content

Implement Config Inversion with Default Strictness of Warning #9539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 21 commits into
base: mhlidd/test
Choose a base branch
from
Open

Implement Config Inversion with Default Strictness of Warning #9539

wants to merge 21 commits into from

Conversation

mhlidd
Copy link
Contributor

@mhlidd mhlidd commented Sep 15, 2025 •
edited by bric3
Loading

What Does This Do

This PR implements basic Config Inversion, aiming to document all DD/OTEL environment variables used in the repo in a supported-configurations.json file.

Components:

  • ConfigInversionStrictStyle.java
    • Enum for how strict to run Config Inversion with.
    • Options are Strict which does not allow any usage of an unknown DD/OTEL environment variable, Warning which allows the usage but sends telemetry about unknown environment variables, and Test which allows the usage of unknown environment variables without sending telemetry.
    • Default value for Strictness is Warning
  • ConfigHelper.java
    • Layer of abstraction between Environment Variable Source and Environment Component that directly compares data loaded from supported-configurations.json to ensure the environment variable queried for is "known"
    • Determines what behavior to take given the ConfigInversionStrictStyle that is set.

Motivation

This PR supports the general Config Inversion theme that has already been implemented in dd-trace-js and currently being implemented in dd-trace-go and dd-trace-rb. Here is the RFC that documents what this project entails.

Additional Notes

This is a follow-up to

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@mhlidd mhlidd force-pushed the mhlidd/config_inversion_base branch 2 times, most recently from 1a576ea to 512b2c6 Compare September 15, 2025 21:34
@pr-commenter
Copy link

pr-commenter bot commented Sep 15, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/config_inversion_base
git_commit_date 1759773830 1759777876
git_commit_sha 3745413 63e5d5a
release_version 1.55.0-SNAPSHOT~37454135e4 1.54.0-SNAPSHOT~63e5d5a068
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1759779749 1759779749
ci_job_id 1165618143 1165618143
ci_pipeline_id 78519612 78519612
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-31kdx8t7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-31kdx8t7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 6 performance improvements and 2 performance regressions! Performance is the same for 47 metrics, 4 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:AppSec better
[-1.739ms; -0.766ms] or [-4.890%; -2.153%]		 34.304ms 35.556ms
scenario:startup:insecure-bank:tracing:AppSec better
[-2.465ms; -1.194ms] or [-7.471%; -3.617%]		 31.167ms 32.997ms
scenario:startup:petclinic:appsec:GlobalTracer worse
[+4.686ms; +6.768ms] or [+2.001%; +2.890%]		 239.893ms 234.167ms
scenario:startup:petclinic:profiling:ProfilingAgent better
[-7.908ms; -5.786ms] or [-7.310%; -5.348%]		 101.328ms 108.175ms
scenario:startup:petclinic:profiling:GlobalTracer worse
[+6.739ms; +9.517ms] or [+3.110%; +4.392%]		 224.794ms 216.666ms
scenario:startup:petclinic:profiling:AppSec better
[-1.954ms; -1.361ms] or [-5.918%; -4.121%]		 31.359ms 33.017ms
scenario:startup:petclinic:profiling:Profiling better
[-8.344ms; -6.676ms] or [-7.623%; -6.100%]		 101.944ms 109.453ms
scenario:startup:petclinic:tracing:AppSec better
[-2.638ms; -1.183ms] or [-8.018%; -3.594%]		 30.996ms 32.907ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.54.0-SNAPSHOT~63e5d5a068, baseline=1.55.0-SNAPSHOT~37454135e4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.025 s) : 0, 1024585
Total [baseline] (10.76 s) : 0, 10759911
Agent [candidate] (1.008 s) : 0, 1007614
Total [candidate] (10.704 s) : 0, 10703905
section appsec
Agent [baseline] (1.193 s) : 0, 1193394
Total [baseline] (11.033 s) : 0, 11032662
Agent [candidate] (1.187 s) : 0, 1186537
Total [candidate] (11.041 s) : 0, 11040552
section iast
Agent [baseline] (1.154 s) : 0, 1154060
Total [baseline] (10.922 s) : 0, 10922123
Agent [candidate] (1.146 s) : 0, 1145890
Total [candidate] (10.961 s) : 0, 10961143
section profiling
Agent [baseline] (1.16 s) : 0, 1159753
Total [baseline] (11.079 s) : 0, 11079321
Agent [candidate] (1.154 s) : 0, 1154347
Total [candidate] (11.07 s) : 0, 11070441
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.025 s -
Agent appsec 1.193 s 168.808 ms (16.5%)
Agent iast 1.154 s 129.474 ms (12.6%)
Agent profiling 1.16 s 135.168 ms (13.2%)
Total tracing 10.76 s -
Total appsec 11.033 s 272.751 ms (2.5%)
Total iast 10.922 s 162.211 ms (1.5%)
Total profiling 11.079 s 319.409 ms (3.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.008 s -
Agent appsec 1.187 s 178.923 ms (17.8%)
Agent iast 1.146 s 138.276 ms (13.7%)
Agent profiling 1.154 s 146.733 ms (14.6%)
Total tracing 10.704 s -
Total appsec 11.041 s 336.647 ms (3.1%)
Total iast 10.961 s 257.239 ms (2.4%)
Total profiling 11.07 s 366.537 ms (3.4%) 
gantt
    title petclinic - break down per module: candidate=1.54.0-SNAPSHOT~63e5d5a068, baseline=1.55.0-SNAPSHOT~37454135e4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.478 ms) : 0, 1478
crashtracking [candidate] (1.472 ms) : 0, 1472
BytebuddyAgent [baseline] (698.958 ms) : 0, 698958
BytebuddyAgent [candidate] (689.276 ms) : 0, 689276
GlobalTracer [baseline] (242.831 ms) : 0, 242831
GlobalTracer [candidate] (248.69 ms) : 0, 248690
AppSec [baseline] (32.907 ms) : 0, 32907
AppSec [candidate] (30.996 ms) : 0, 30996
Debugger [baseline] (6.393 ms) : 0, 6393
Debugger [candidate] (6.392 ms) : 0, 6392
Remote Config [baseline] (690.676 µs) : 0, 691
Remote Config [candidate] (676.701 µs) : 0, 677
Telemetry [baseline] (9.24 ms) : 0, 9240
Telemetry [candidate] (8.975 ms) : 0, 8975
Flare Poller [baseline] (10.813 ms) : 0, 10813
section appsec
crashtracking [baseline] (1.464 ms) : 0, 1464
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (717.482 ms) : 0, 717482
BytebuddyAgent [candidate] (712.043 ms) : 0, 712043
GlobalTracer [baseline] (234.167 ms) : 0, 234167
GlobalTracer [candidate] (239.893 ms) : 0, 239893
AppSec [baseline] (175.502 ms) : 0, 175502
AppSec [candidate] (172.428 ms) : 0, 172428
Debugger [baseline] (6.072 ms) : 0, 6072
Debugger [candidate] (5.949 ms) : 0, 5949
Remote Config [baseline] (630.129 µs) : 0, 630
Remote Config [candidate] (650.341 µs) : 0, 650
Telemetry [baseline] (8.392 ms) : 0, 8392
Telemetry [candidate] (8.451 ms) : 0, 8451
Flare Poller [baseline] (3.907 ms) : 0, 3907
IAST [baseline] (24.742 ms) : 0, 24742
IAST [candidate] (24.591 ms) : 0, 24591
section iast
crashtracking [baseline] (1.456 ms) : 0, 1456
crashtracking [candidate] (1.457 ms) : 0, 1457
BytebuddyAgent [baseline] (817.421 ms) : 0, 817421
BytebuddyAgent [candidate] (810.933 ms) : 0, 810933
GlobalTracer [baseline] (232.479 ms) : 0, 232479
GlobalTracer [candidate] (237.684 ms) : 0, 237684
AppSec [baseline] (35.472 ms) : 0, 35472
AppSec [candidate] (32.94 ms) : 0, 32940
Debugger [baseline] (6.079 ms) : 0, 6079
Debugger [candidate] (6.11 ms) : 0, 6110
Remote Config [baseline] (610.47 µs) : 0, 610
Remote Config [candidate] (592.156 µs) : 0, 592
Telemetry [baseline] (8.597 ms) : 0, 8597
Telemetry [candidate] (8.308 ms) : 0, 8308
Flare Poller [baseline] (4.162 ms) : 0, 4162
IAST [baseline] (26.443 ms) : 0, 26443
IAST [candidate] (26.666 ms) : 0, 26666
section profiling
ProfilingAgent [baseline] (108.175 ms) : 0, 108175
ProfilingAgent [candidate] (101.328 ms) : 0, 101328
crashtracking [baseline] (1.425 ms) : 0, 1425
crashtracking [candidate] (1.452 ms) : 0, 1452
BytebuddyAgent [baseline] (719.366 ms) : 0, 719366
BytebuddyAgent [candidate] (719.468 ms) : 0, 719468
GlobalTracer [baseline] (216.666 ms) : 0, 216666
GlobalTracer [candidate] (224.794 ms) : 0, 224794
AppSec [baseline] (33.017 ms) : 0, 33017
AppSec [candidate] (31.359 ms) : 0, 31359
Debugger [baseline] (7.337 ms) : 0, 7337
Debugger [candidate] (7.321 ms) : 0, 7321
Remote Config [baseline] (692.24 µs) : 0, 692
Remote Config [candidate] (702.413 µs) : 0, 702
Telemetry [baseline] (15.822 ms) : 0, 15822
Telemetry [candidate] (15.547 ms) : 0, 15547
Profiling [baseline] (109.453 ms) : 0, 109453
Profiling [candidate] (101.944 ms) : 0, 101944
Flare Poller [baseline] (4.216 ms) : 0, 4216
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.54.0-SNAPSHOT~63e5d5a068, baseline=1.55.0-SNAPSHOT~37454135e4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.02 s) : 0, 1020499
Total [baseline] (8.675 s) : 0, 8674934
Agent [candidate] (1.013 s) : 0, 1012637
Total [candidate] (8.664 s) : 0, 8663759
section iast
Agent [baseline] (1.151 s) : 0, 1150895
Total [baseline] (9.242 s) : 0, 9241797
Agent [candidate] (1.147 s) : 0, 1147128
Total [candidate] (9.27 s) : 0, 9269787
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.02 s -
Agent iast 1.151 s 130.396 ms (12.8%)
Total tracing 8.675 s -
Total iast 9.242 s 566.862 ms (6.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.013 s -
Agent iast 1.147 s 134.491 ms (13.3%)
Total tracing 8.664 s -
Total iast 9.27 s 606.028 ms (7.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.54.0-SNAPSHOT~63e5d5a068, baseline=1.55.0-SNAPSHOT~37454135e4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.483 ms) : 0, 1483
crashtracking [candidate] (1.485 ms) : 0, 1485
BytebuddyAgent [baseline] (695.869 ms) : 0, 695869
BytebuddyAgent [candidate] (693.451 ms) : 0, 693451
GlobalTracer [baseline] (242.296 ms) : 0, 242296
GlobalTracer [candidate] (249.237 ms) : 0, 249237
AppSec [baseline] (32.997 ms) : 0, 32997
AppSec [candidate] (31.167 ms) : 0, 31167
Debugger [baseline] (6.453 ms) : 0, 6453
Debugger [candidate] (6.415 ms) : 0, 6415
Remote Config [baseline] (700.9 µs) : 0, 701
Remote Config [candidate] (686.522 µs) : 0, 687
Telemetry [baseline] (9.268 ms) : 0, 9268
Telemetry [candidate] (9.001 ms) : 0, 9001
Flare Poller [baseline] (10.289 ms) : 0, 10289
section iast
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.466 ms) : 0, 1466
BytebuddyAgent [baseline] (814.459 ms) : 0, 814459
BytebuddyAgent [candidate] (811.744 ms) : 0, 811744
GlobalTracer [baseline] (231.832 ms) : 0, 231832
GlobalTracer [candidate] (237.179 ms) : 0, 237179
IAST [baseline] (26.788 ms) : 0, 26788
IAST [candidate] (26.111 ms) : 0, 26111
AppSec [baseline] (35.556 ms) : 0, 35556
AppSec [candidate] (34.304 ms) : 0, 34304
Debugger [baseline] (6.123 ms) : 0, 6123
Debugger [candidate] (6.105 ms) : 0, 6105
Remote Config [baseline] (616.769 µs) : 0, 617
Remote Config [candidate] (592.387 µs) : 0, 592
Telemetry [baseline] (8.689 ms) : 0, 8689
Telemetry [candidate] (8.414 ms) : 0, 8414
Flare Poller [baseline] (4.161 ms) : 0, 4161
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/config_inversion_base
git_commit_date 1759773830 1759777876
git_commit_sha 3745413 63e5d5a
release_version 1.55.0-SNAPSHOT~37454135e4 1.54.0-SNAPSHOT~63e5d5a068
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1759779430 1759779430
ci_job_id 1165618144 1165618144
ci_pipeline_id 78519612 78519612
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-slk5tida 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-slk5tida 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 2 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:profiling:high_load worse
[+308.107µs; +635.985µs] or [+3.495%; +7.214%]		 unstable
[-91.982op/s; +38.919op/s] or [-17.477%; +7.395%]		 9.287ms 499.781op/s 8.815ms 526.312op/s
scenario:load:insecure-bank:iast_FULL:high_load better
[-1.484ms; -0.861ms] or [-9.844%; -5.710%]		 unstable
[-10.345op/s; +61.908op/s] or [-3.345%; +20.017%]		 13.901ms 335.062op/s 15.074ms 309.281op/s
scenario:load:petclinic:profiling:high_load worse
[+2.073ms; +3.129ms] or [+4.324%; +6.526%]		 unstable
[-10.829op/s; +3.174op/s] or [-11.096%; +3.252%]		 50.552ms 93.772op/s 47.951ms 97.600op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~63e5d5a068, baseline=1.55.0-SNAPSHOT~37454135e4
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.409 ms) : 36115, 36702
.   : milestone, 36409,
appsec (49.206 ms) : 48768, 49644
.   : milestone, 49206,
code_origins (43.904 ms) : 43524, 44284
.   : milestone, 43904,
iast (44.895 ms) : 44516, 45274
.   : milestone, 44895,
profiling (47.951 ms) : 47451, 48451
.   : milestone, 47951,
tracing (44.145 ms) : 43772, 44517
.   : milestone, 44145,
section candidate
no_agent (36.541 ms) : 36244, 36839
.   : milestone, 36541,
appsec (47.795 ms) : 47372, 48218
.   : milestone, 47795,
code_origins (42.959 ms) : 42584, 43333
.   : milestone, 42959,
iast (45.972 ms) : 45567, 46376
.   : milestone, 45972,
profiling (50.552 ms) : 50071, 51033
.   : milestone, 50552,
tracing (45.001 ms) : 44612, 45390
.   : milestone, 45001,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.409 ms [36.115 ms, 36.702 ms] -
appsec 49.206 ms [48.768 ms, 49.644 ms] 12.797 ms (35.1%)
code_origins 43.904 ms [43.524 ms, 44.284 ms] 7.495 ms (20.6%)
iast 44.895 ms [44.516 ms, 45.274 ms] 8.486 ms (23.3%)
profiling 47.951 ms [47.451 ms, 48.451 ms] 11.542 ms (31.7%)
tracing 44.145 ms [43.772 ms, 44.517 ms] 7.736 ms (21.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.541 ms [36.244 ms, 36.839 ms] -
appsec 47.795 ms [47.372 ms, 48.218 ms] 11.254 ms (30.8%)
code_origins 42.959 ms [42.584 ms, 43.333 ms] 6.417 ms (17.6%)
iast 45.972 ms [45.567 ms, 46.376 ms] 9.43 ms (25.8%)
profiling 50.552 ms [50.071 ms, 51.033 ms] 14.011 ms (38.3%)
tracing 45.001 ms [44.612 ms, 45.39 ms] 8.459 ms (23.2%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~63e5d5a068, baseline=1.55.0-SNAPSHOT~37454135e4
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.195 ms) : 4148, 4242
.   : milestone, 4195,
iast (10.005 ms) : 9836, 10173
.   : milestone, 10005,
iast_FULL (15.074 ms) : 14774, 15374
.   : milestone, 15074,
iast_GLOBAL (10.68 ms) : 10492, 10869
.   : milestone, 10680,
profiling (8.815 ms) : 8680, 8951
.   : milestone, 8815,
tracing (7.537 ms) : 7431, 7643
.   : milestone, 7537,
section candidate
no_agent (4.317 ms) : 4259, 4375
.   : milestone, 4317,
iast (9.838 ms) : 9674, 10003
.   : milestone, 9838,
iast_FULL (13.901 ms) : 13623, 14180
.   : milestone, 13901,
iast_GLOBAL (10.697 ms) : 10510, 10885
.   : milestone, 10697,
profiling (9.287 ms) : 9120, 9455
.   : milestone, 9287,
tracing (7.556 ms) : 7440, 7672
.   : milestone, 7556,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.195 ms [4.148 ms, 4.242 ms] -
iast 10.005 ms [9.836 ms, 10.173 ms] 5.81 ms (138.5%)
iast_FULL 15.074 ms [14.774 ms, 15.374 ms] 10.879 ms (259.3%)
iast_GLOBAL 10.68 ms [10.492 ms, 10.869 ms] 6.485 ms (154.6%)
profiling 8.815 ms [8.68 ms, 8.951 ms] 4.621 ms (110.1%)
tracing 7.537 ms [7.431 ms, 7.643 ms] 3.342 ms (79.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.317 ms [4.259 ms, 4.375 ms] -
iast 9.838 ms [9.674 ms, 10.003 ms] 5.521 ms (127.9%)
iast_FULL 13.901 ms [13.623 ms, 14.18 ms] 9.584 ms (222.0%)
iast_GLOBAL 10.697 ms [10.51 ms, 10.885 ms] 6.38 ms (147.8%)
profiling 9.287 ms [9.12 ms, 9.455 ms] 4.97 ms (115.1%)
tracing 7.556 ms [7.44 ms, 7.672 ms] 3.239 ms (75.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mhlidd/config_inversion_base
git_commit_date 1759773830 1759777876
git_commit_sha 3745413 63e5d5a
release_version 1.55.0-SNAPSHOT~37454135e4 1.54.0-SNAPSHOT~63e5d5a068
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1759779968 1759779968
ci_job_id 1165618145 1165618145
ci_pipeline_id 78519612 78519612
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-etpylwjw 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-etpylwjw 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~63e5d5a068, baseline=1.55.0-SNAPSHOT~37454135e4
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.98 s) : 14980000, 14980000
.   : milestone, 14980000,
appsec (15.035 s) : 15035000, 15035000
.   : milestone, 15035000,
iast (18.248 s) : 18248000, 18248000
.   : milestone, 18248000,
iast_GLOBAL (18.085 s) : 18085000, 18085000
.   : milestone, 18085000,
profiling (15.363 s) : 15363000, 15363000
.   : milestone, 15363000,
tracing (15.092 s) : 15092000, 15092000
.   : milestone, 15092000,
section candidate
no_agent (15.487 s) : 15487000, 15487000
.   : milestone, 15487000,
appsec (15.326 s) : 15326000, 15326000
.   : milestone, 15326000,
iast (18.479 s) : 18479000, 18479000
.   : milestone, 18479000,
iast_GLOBAL (18.257 s) : 18257000, 18257000
.   : milestone, 18257000,
profiling (15.433 s) : 15433000, 15433000
.   : milestone, 15433000,
tracing (15.4 s) : 15400000, 15400000
.   : milestone, 15400000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.98 s [14.98 s, 14.98 s] -
appsec 15.035 s [15.035 s, 15.035 s] 55.0 ms (0.4%)
iast 18.248 s [18.248 s, 18.248 s] 3.268 s (21.8%)
iast_GLOBAL 18.085 s [18.085 s, 18.085 s] 3.105 s (20.7%)
profiling 15.363 s [15.363 s, 15.363 s] 383.0 ms (2.6%)
tracing 15.092 s [15.092 s, 15.092 s] 112.0 ms (0.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.487 s [15.487 s, 15.487 s] -
appsec 15.326 s [15.326 s, 15.326 s] -161.0 ms (-1.0%)
iast 18.479 s [18.479 s, 18.479 s] 2.992 s (19.3%)
iast_GLOBAL 18.257 s [18.257 s, 18.257 s] 2.77 s (17.9%)
profiling 15.433 s [15.433 s, 15.433 s] -54.0 ms (-0.3%)
tracing 15.4 s [15.4 s, 15.4 s] -87.0 ms (-0.6%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~63e5d5a068, baseline=1.55.0-SNAPSHOT~37454135e4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (3.671 ms) : 3457, 3884
.   : milestone, 3671,
iast (2.214 ms) : 2150, 2278
.   : milestone, 2214,
iast_GLOBAL (2.246 ms) : 2182, 2310
.   : milestone, 2246,
profiling (2.069 ms) : 2017, 2122
.   : milestone, 2069,
tracing (2.026 ms) : 1976, 2076
.   : milestone, 2026,
section candidate
no_agent (1.478 ms) : 1467, 1490
.   : milestone, 1478,
appsec (3.704 ms) : 3485, 3922
.   : milestone, 3704,
iast (2.202 ms) : 2139, 2265
.   : milestone, 2202,
iast_GLOBAL (2.251 ms) : 2187, 2314
.   : milestone, 2251,
profiling (2.048 ms) : 1996, 2099
.   : milestone, 2048,
tracing (2.022 ms) : 1973, 2071
.   : milestone, 2022,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 3.671 ms [3.457 ms, 3.884 ms] 2.201 ms (149.8%)
iast 2.214 ms [2.15 ms, 2.278 ms] 744.651 µs (50.7%)
iast_GLOBAL 2.246 ms [2.182 ms, 2.31 ms] 776.835 µs (52.9%)
profiling 2.069 ms [2.017 ms, 2.122 ms] 600.047 µs (40.8%)
tracing 2.026 ms [1.976 ms, 2.076 ms] 556.367 µs (37.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.478 ms [1.467 ms, 1.49 ms] -
appsec 3.704 ms [3.485 ms, 3.922 ms] 2.225 ms (150.5%)
iast 2.202 ms [2.139 ms, 2.265 ms] 723.459 µs (48.9%)
iast_GLOBAL 2.251 ms [2.187 ms, 2.314 ms] 772.485 µs (52.3%)
profiling 2.048 ms [1.996 ms, 2.099 ms] 569.223 µs (38.5%)
tracing 2.022 ms [1.973 ms, 2.071 ms] 543.521 µs (36.8%)

@datadog-official Datadog Official
Copy link

datadog-official bot commented Sep 16, 2025 •
edited
Loading

🎯 Code Coverage
• Patch Coverage: 84.21%
• Total Coverage: 57.83% (+0.43%)

View detailed report

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 63e5d5a | Docs | Was this helpful? Give us feedback!

@mhlidd mhlidd changed the title Mhlidd/config inversion base Implement Config Inversion with Default Strictness of Warning Sep 17, 2025
@mhlidd mhlidd marked this pull request as ready for review September 18, 2025 15:13
@mhlidd mhlidd requested review from a team as code owners September 18, 2025 15:13
@mhlidd mhlidd requested review from AlexeyKuznetsov-DD and mcculls and removed request for a team September 18, 2025 15:13
@github-actions GitHub Actions
Copy link
Contributor

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@mhlidd mhlidd added type: enhancement Enhancements and improvements type: documentation comp: api Tracer public API labels Sep 18, 2025
@mhlidd mhlidd mentioned this pull request Sep 18, 2025
Merged
@mhlidd mhlidd changed the base branch from master to mhlidd/config_inversion_plugins September 18, 2025 21:25
@mhlidd mhlidd force-pushed the mhlidd/config_inversion_base branch from aa18112 to 606dfd0 Compare September 18, 2025 21:32
@mhlidd mhlidd changed the base branch from mhlidd/config_inversion_plugins to mhlidd/migrate_config-utils_tests September 18, 2025 22:06
@mhlidd mhlidd force-pushed the mhlidd/config_inversion_base branch from 606dfd0 to 7cea99b Compare September 18, 2025 22:09
PerfectSlayer
PerfectSlayer reviewed Sep 19, 2025
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏 praise: ‏Thanks for splitting the PR related to your config changes, that really helps to review 👍

🎯 suggestion: ‏I would recommend trying to decrease the overall (cognitive) complexity by refactoring using new few methods to de-duplicate code and give more meaning.

For example this should have its own function with a meaningful name:

if (key.startsWith("DD_")
          || key.startsWith("OTEL_")
          || configSource.getAliasMapping().containsKey(key))

@bric3 bric3 mentioned this pull request Sep 23, 2025
Merged
@mhlidd mhlidd force-pushed the mhlidd/migrate_config-utils_tests branch from 57483a2 to 1574d9b Compare September 23, 2025 21:02
@mhlidd mhlidd force-pushed the mhlidd/config_inversion_base branch from a20edf0 to bb4f47c Compare September 23, 2025 21:09
@mhlidd mhlidd force-pushed the mhlidd/config_inversion_base branch from 12e5514 to 5fe0a88 Compare September 25, 2025 00:25
@mhlidd mhlidd requested review from a team as code owners September 25, 2025 00:25
@mhlidd mhlidd requested review from ojung and daniel-mohedano and removed request for a team September 25, 2025 00:25
@mhlidd mhlidd changed the base branch from master to alexeyk/refactored-env-vars-to-be-testable September 25, 2025 00:26
@mhlidd mhlidd requested review from a team as code owners September 25, 2025 00:26
@mhlidd mhlidd changed the base branch from alexeyk/refactored-env-vars-to-be-testable to mhlidd/test September 25, 2025 00:26
@mhlidd mhlidd changed the base branch from mhlidd/test to alexeyk/refactored-env-vars-to-be-testable September 25, 2025 00:28
@mhlidd mhlidd changed the base branch from alexeyk/refactored-env-vars-to-be-testable to mhlidd/test September 25, 2025 00:29
@pr-commenter
Copy link

pr-commenter bot commented Sep 25, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1759778501 1759778846
end_time 2025-10-06T19:23:01 2025-10-06T19:28:46
git_branch master mhlidd/config_inversion_base
git_commit_sha 3745413 63e5d5a
start_time 2025-10-06T19:21:42 2025-10-06T19:27:27
See matching parameters
Baseline Candidate
ci_job_id 1165618150 1165618150
ci_pipeline_id 78519612 78519612
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1759777876 1759777876

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 6 unstable metrics.

See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-20.027µs; +25.922µs] or [-7.250%; +9.384%]		 unstable
[-34.056µs; +36.165µs] or [-10.756%; +11.422%]		 unstable
[-41.189µs; +49.123µs] or [-12.501%; +14.910%]		 unstable
[-171.354µs; +52.168µs] or [-17.452%; +5.313%]		 same
scenario:basic same same same unstable
[-152.873µs; +51.729µs] or [-17.978%; +6.083%]		 unstable
[-175.146op/s; +175.146op/s] or [-6.656%; +6.656%]
scenario:loop same same same same same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (316.631 µs) : 286, 347
.   : milestone, 317,
basic (281.639 µs) : 275, 288
.   : milestone, 282,
loop (8.961 ms) : 8958, 8965
.   : milestone, 8961,
section candidate
noprobe (317.685 µs) : 281, 354
.   : milestone, 318,
basic (277.696 µs) : 271, 284
.   : milestone, 278,
loop (8.959 ms) : 8954, 8963
.   : milestone, 8959,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 316.631 µs [286.387 µs, 346.875 µs]
basic 281.639 µs [275.083 µs, 288.196 µs]
loop 8.961 ms [8.958 ms, 8.965 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 317.685 µs [281.339 µs, 354.031 µs]
basic 277.696 µs [271.027 µs, 284.366 µs]
loop 8.959 ms [8.954 ms, 8.963 ms]

PerfectSlayer
PerfectSlayer approved these changes Sep 25, 2025
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good.

Left a comment about unnecessary allocation.

AlexeyKuznetsov-DD
AlexeyKuznetsov-DD reviewed Oct 6, 2025
View reviewed changes
Copy link
Contributor

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some notes and questions.

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigHelper.java Outdated
configs.put(key, value);
// If this environment variable is the alias of another, and we haven't processed the
// original environment variable yet, handle it here.
} else if (null != primaryEnv && !configs.containsKey(primaryEnv)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we OK with Yoda style? Here and other similar places.
Maybe natural way is better or more human: primaryEnv != null, WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember in a previous PR, I was told that there was some benefit to doing Yoda style, but that may be for a different context. I can't find the old comment but I can reverse the ordering here for readability.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is not strict request, can you search code base to get some stats?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a quick search on GitHub, instances of != null and == null vs null != and null == seem to be very comparable 🤷‍♂️

Copy link
Member

@jpbempel jpbempel Oct 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was told that there was some benefit to doing Yoda style

in C++, not in Java
see https://jpbempel.github.io/2015/09/21/yoda-conditions.html for more rationale

utils/config-utils/src/main/java/datadog/trace/config/inversion/ConfigHelper.java Outdated
Comment on lines 41 to 42
// Cache for configs, init value is null
private Map<String, String> configs;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just thinking out loud: do we need null state? Maybe empty map will be better? No null checks in usages? WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use empty map instead. Here is my thought process:

The only situation where initializing configs as null would be better is when no env vars are set at all, since our cache would have the same effect as if we had not yet processed env vars. This is a very improbable situation since we expect customers to set env vars, so I'm happy to change to initializing to an empty map.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I learned that Collections.emptyMap() is a EmptyMap type, which is different than an empty HashMap. There is no reason to not use it here :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dougqh dougqh dougqh left review comments

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@jpbempel jpbempel jpbempel left review comments

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD AlexeyKuznetsov-DD left review comments

@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-java

@ojung ojung Awaiting requested review from ojung ojung was automatically assigned from DataDog/debugger-java

@daniel-mohedano daniel-mohedano Awaiting requested review from daniel-mohedano daniel-mohedano was automatically assigned from DataDog/ci-app-libraries-java

@robertpi robertpi Awaiting requested review from robertpi robertpi was automatically assigned from DataDog/asm-java

@jandro996 jandro996 Awaiting requested review from jandro996 jandro996 was automatically assigned from DataDog/asm-java

Assignees
No one assigned
Labels
comp: api Tracer public API type: documentation type: enhancement Enhancements and improvements
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@mhlidd @dougqh @PerfectSlayer @jpbempel @AlexeyKuznetsov-DD