1.31.1

Components

Continuous Integration Visibility

• 🐛 Fix source path calculation for classes in default package (#6781 - @nikita-tkachenko-datadog)
• Update Spock ITR instrumentation to skip spec setup methods if all features in a spec are skipped (#6780 - @nikita-tkachenko-datadog)

1.31.0

Potentially Breaking Changes

Warning

Setting propagation style to None should extract headers and baggage (#6718 - @amarziali)

Components

Application Security Management (IAST)

• Add support for precise tainting of byte array objects (#6741 - @manuel-alvarez-alvarez)
• Add Lookup api support for reflection injection (#6708 - @jandro996)
• ⚡ Add bounds to all range related operations (#6698 - @manuel-alvarez-alvarez)
• ⚡ Optimizations to check if string arrays can be tainted (#6690 - @manuel-alvarez-alvarez)
• Attach values to sources coming from non char sequences (#6675 - @manuel-alvarez-alvarez)
• ⚡ Improve application module performance and security bounds (#6647 - @manuel-alvarez-alvarez)
• Add Reflection Injection support (#6622 - @jandro996)
• Partial IAST opt-out when AppSec is enabled (#6621 - @manuel-alvarez-alvarez)

Application Security Management (WAF)

• Added API Security option DD_API_SECURITY_ENABLED (#6735 - @ValentinZakharov)

Continuous Integration Visibility

• Update JUnit 4 method name extraction logic to properly handle Kotlin test names (#6751 - @nikita-tkachenko-datadog)
• Implement telemetry for early flakiness detection (#6746 - @nikita-tkachenko-datadog)
• Implement early flakiness detection (#6727 - @nikita-tkachenko-datadog)
• Make signal client timeout configurable (#6724 - @nikita-tkachenko-datadog)
• Add ITR correlator to test events (#6674 - @nikita-tkachenko-datadog)
• Implement telemetry for CI Visibility events (#6669 - @nikita-tkachenko-datadog)
• Implement telemetry for CI Visibility intakes (#6668 - @nikita-tkachenko-datadog)
• Implement telemetry for CI Visibility APIs (#6667 - @nikita-tkachenko-datadog)

Data Streams Monitoring

• Add DSM support to SNS (#6721 - @vandonr)

Dynamic Instrumentation

• Add "applicationkey" as redacted keyword (#6769 - @jpbempel)

JMX fetch

• Bump integrations-core submodule to 7.51.0 (#6750 - @mcculls)

Profiling

• Collapse implementations of timer and checkpointer interfaces into profiling context (#6738 - @richardstartin)
• Add failure to load dd profiler as periodic JFR event (#6726 - @richardstartin)
• Refactor profiling module initialisation and composition (#6719 - @richardstartin)
• Avoid permission issues for the crash uploader script (#6697 - @jbachorik)

Configuration at Runtime

• 🐛 Fix NPE on traceState() in DDSpanLink.java (#6758 - @cecile75)
• Custom Tags via Remote Config for Tracing (#6649 - @nayeem-kamal)
• Implement extra_services for remote config (#6130 - @jandro996)

Telemetry

• Implement telemetry for early flakiness detection (#6746 - @nikita-tkachenko-datadog)
• ✨ Improve dependency resolver heuristics for MANIFEST.MF (#6744 - @smola)
• 🐛 Limit DependencyResolverQueue size with a config flag (#6729 - @nayeem-kamal)
• 🐛 Fix dependency resolution for nested JARs (with fix for Spring Boot) (#6720 - @smola)
• Standardize common configuration telemetry (#6694 - @ygree)

Trace context propagation

• Support context propagation over WebSphere Axis2 transports (#6745 - @mcculls)
• ⚠️ Setting propagation style to None should extract headers and baggage (#6718 - @amarziali)

Tracer core

• 🐛 Ensure span links trace ids and attributes can't be null (#6766 - @PerfectSlayer)
• ⚠️ Setting propagation style to None should extract headers and baggage (#6718 - @amarziali)
• ⚡ Ensure superclasses and interfaces are loaded before classes that extend/implement them (#6677 - @mcculls)
• ⚡ Configure byte-buddy to use simpler method-graph (#6579 - @mcculls)

Instrumentations

Apache Spark instrumentation

• Spark Executor Tasks Instrumentation (#6515 - @paul-laffon-dd)

AWS SDK instrumentation

• Add aws.object.key tag for aws-sdk-2 (#6709 - @paul-laffon-dd)

Couchbase instrumentation

• Add option to not include Couchbase internal spans (#6763 - @brentm5 - thanks for the contribution!)

JAX-WS instrumentation

• Support context propagation over WebSphere Axis2 transports (#6745 - @mcculls)

JDBC instrumentation

• 🐛 Don't inject comment for callable JDBC statements (#6743 - @sethsamuel)

OpenTelemetry instrumentation

• 🧪 Do not generate contextStore() from OTel instrumentation if no context store is used (#6739 - @PerfectSlayer)
• 🧪 Add OpenTelemetry integrations support (#6526 - @PerfectSlayer)

Spring instrumentation

• 🧪 Spring boot: autodetect service name from environment (#6663 - @amarziali)

Other changes

• Update JNR dependencies to pick up fix for GLIBC minimum version (#6749 - @mcculls)

1.30.1

Components

Application Security Management (WAF)

• 🐛 Update WAF bindings to 9.0.2 (fixes segmentation fault) (#6710 - @smola)

Telemetry

• 🐛 Revert "Fix dependency resolution to nested JARs (#6603)" (#6713 - @smola)

1.30.0

Known Issues

• This release contains two critical bugs that may break applications in the following cases:
  • Some Spring Boot applications.
  • Applications with ASM enabled (DD_APPSEC_ENABLED=true).
• In both cases, it is strongly recommended to upgrade to v1.30.1 or later.

Components

Application Security Management (IAST)

• 🐛 Fix dynamic invoke for string concat in IBM J9 (#6634 - @manuel-alvarez-alvarez)
• ⚡ Move away from soft references in sources (#6631 - @manuel-alvarez-alvarez)
• Add application vulnerabilities to telemetry (#6624 - @manuel-alvarez-alvarez)
• Detect insecure auth protocol (#6614 - @jandro996)
• Fix insecure cookies false positives when unsetting cookies (#6586 - @jandro996)

Application Security Management (WAF)

• Upgrade to libddwaf 1.16.0 (libsqreen 9.0.1) (#6658 - @ValentinZakharov)
• 🧪 Exclude WAF timeout from telemetry logs (#6618 - @smola)
• Add _dd.appsec.waf.timeouts tag (#6597 - @smola)

Continuous Integration Visibility

• 🐛 Fix null test framework name in JUnit 5 instrumentation (#6684 - @nikita-tkachenko-datadog)
• Do not ignore caught exceptions in CI Visibility code (#6671 - @nikita-tkachenko-datadog)
• Implement telemetry for code coverage operations (#6666 - @nikita-tkachenko-datadog)
• Implement telemetry for shell git commands (#6665 - @nikita-tkachenko-datadog)
• Implement telemetry for CI Visibility (#6664 - @nikita-tkachenko-datadog)

Database Monitoring

• Fix context store access with null key on DBM MySQL connection (#6616 - @mcculls)

Dynamic Instrumentation

• ✨ Add class filtering and exception probe conditions (#6641 - @jpbempel)
• ✨ Add instrumentation of exception stacktraces (#6609 - @jpbempel)

Profiling

• Upgrade to ddprof 0.99.0 (#6679 - @richardstartin)

Remote Configuration

• 🐛 Fix remote config activation configuration name (#6642 - @PerfectSlayer)
• Add target service+env check prior configuration activation (#6636 - @ygree)

Telemetry

• 🐛 Fix intake telemetry client URL for EU site (#6683 - @nikita-tkachenko-datadog)
• Update telemetry intake client to use CI Visibility agentless URL, if configured (#6653 - @nikita-tkachenko-datadog)
• Do not switch to agent telemetry client on detecting agent telemetry endpoint availability when intake client is preferred (#6644 - @nikita-tkachenko-datadog)
• Add support for periodic submission of telemetry distribution series (#6643 - @nikita-tkachenko-datadog)
• Use intake telemetry client by default when running in agentless mode (#6607 - @nikita-tkachenko-datadog)

Tracer core

• Implement a mechanism for adding ad-hoc listeners to OKHTTP requests (#6660 - @nikita-tkachenko-datadog)
• Override tracing preferred service name and pin to dynamic config (#6648 - @amarziali)
• 🐛 Fixes to trace agent service rates feedback - e.g. MAX TPS support (#6628 - @dougqh)
• Switching to case insensitive matching (#6613 - @dougqh)
• ✨ Add scheduledWithFixedDelay method (#6587 - @jpbempel)

Serverless

• Use connection pool for http communications with extension. (#6417 - @purple4reina)

Instrumentations

Apache Spark instrumentation

• Flush traces when spark application is finished (#6670 - @paul-laffon-dd)

JDBC instrumentation

• Fix context store access with null key on DBM MySQL connection (#6616 - @mcculls)

All other instrumentations

• Align apache http client 5 instrumentation names (#6596 - @amarziali)

1.30.0-RC1

Warning

This is a RELEASE CANDIDATE and is NOT intended for use in production.

Known Issues

• This release contains two critical bugs that may break applications in the following cases:
  • Some Spring Boot applications.
  • Applications with ASM enabled (DD_APPSEC_ENABLED=true).
• In both cases, it is strongly recommended to upgrade to v1.30.1 or later.

Components

Application Security Management (IAST)

• 🐛 Fix dynamic invoke for string concat in IBM J9 (#6634 - @manuel-alvarez-alvarez)
• Add application vulnerabilities to telemetry (#6624 - @manuel-alvarez-alvarez)
• Detect insecure auth protocol (#6614 - @jandro996)
• Fix insecure cookies false positives when unsetting cookies (#6586 - @jandro996)

Application Security Management (WAF)

• 🧪 Exclude WAF timeout from telemetry logs (#6618 - @smola)
• Add _dd.appsec.waf.timeouts tag (#6597 - @smola)

Database Monitoring

• Fix context store access with null key on DBM MySQL connection (#6616 - @mcculls)

Dynamic Instrumentation

• ✨ Add instrumentation of exception stacktraces (#6609 - @jpbempel)

Telemetry

• Add support for periodic submission of telemetry distribution series (#6643 - @nikita-tkachenko-datadog)
• Use intake telemetry client by default when running in agentless mode (#6607 - @nikita-tkachenko-datadog)

Tracer core

• ✨ Add scheduledWithFixedDelay method (#6587 - @jpbempel)
• 🧹 Introduce concept of an InstrumenterGroup (#6575 - @mcculls)
• 🐛 Fixes to trace agent service rates feedback - e.g. MAX TPS support (#6628 - @dougqh)
• 🐛 Switching to case insensitive matching (#6613 - @dougqh)

Instrumentations

JDBC instrumentation

• Fix context store access with null key on DBM MySQL connection (#6616 - @mcculls)

All other instrumentations

• Align apache http client 5 instrumentation names (#6596 - @amarziali)

1.29.0

Known Issues

• This release contains a critical bug that may break some Spring Boot applications. It is strongly recommended to upgrade to v1.30.1 or later.

Components

Application Security Management (IAST)

• 🐛 Fix issue with cookie parser and empty cookie values (#6588 - @manuel-alvarez-alvarez)
• 🐛 Fix org.json instrumentation (bad JSONObject advice) (#6578 - @smola)
• Improve header injection redaction (#6577 - @jandro996)
• ⚡ Cache IAST metric tag values to be used in metrics and spans (#6561 - @manuel-alvarez-alvarez)
• 🧹 Get access to tag values from the top of TraceSegments (#6560 - @manuel-alvarez-alvarez)
• ⚡ Improve the performance of the IAST cookie parser (#6558 - @manuel-alvarez-alvarez)
• Fix header injection exclusions (#6550 - @jandro996)
• ⚡ Do not compute DBInfo on every statement call (#6549 - @manuel-alvarez-alvarez)
• ⚡ Reduce the memory footprint of IAST sources (#6501 - @manuel-alvarez-alvarez)
• Kafka support as IAST source (#6465 - @manuel-alvarez-alvarez)
• Add hardcoded secrets detection (#6461 - @jandro996)
• Improve IAST coverage by not relying on an active context for tainting purposes (#6094 - @manuel-alvarez-alvarez)

Application Security Management (WAF)

• 🐛 Trigger WAF request metrics only at end of request only (#6595 - @smola)
• ✨ Configure API Security configuration via remote-config (#6340 - @ValentinZakharov)

Continuous Integration Visibility

• Report retried TestNG tests as failed rather than skipped (#6592 - @nikita-tkachenko-datadog)
• Implement flaky test retries for Karate framework (#6582 - @nikita-tkachenko-datadog)
• Add support for GZIP for CI Visibility intakes and EVP endpoints in agentless/evp-proxy-V4 modes (#6569 - @nikita-tkachenko-datadog)
• Adjust log level or error messages written by repo indexer (#6556 - @nikita-tkachenko-datadog)

Dynamic Instrumentation

• ✨ Add exception debugging for span error (#6568 - @jpbempel)

GraalVM native-image

• Fix profiling on Quarkus native-image (#6562 - @jbachorik)

Profiling

• Include aas related tags in profile tag-map (#6612 - @jbachorik)
• Fix profiling on Quarkus native-image (#6562 - @jbachorik)
• Upgrade to ddprof 0.96.0 (#6553 - @richardstartin)

Remote Configuration

• Handle remote config 204 responses gracefully (#6576 - @smola)

Telemetry

• 🧪 Do not send agent API errors to telemetry logs (#6605 - @smola)
• Fix dependency resolution for nested JARs (#6603 - @smola)
• Add Datadog-Entity-Id header (#6559 - @ygree)
• 🧪 Improve content of redacted stacktraces in telemetry logs (#6557 - @smola)

Testing

• Force test failure on unhandled instrumented exception (#6546 - @amarziali)
• Add latestDepTest for Jvm 21 on CI (#6542 - @amarziali)

Tracer core

• Implement ScopedContext and Baggage API (#6608 - @mcculls)
• ✨ Adding support for tag matching of non-String types (#6591 - @dougqh)
• 🔍 Capture JVM arguments in tracer-flare (#6581 - @mcculls)
• 🐛 Avoid NumberFormatException when logging 128bit trace ID (#6570 - @amarziali)
• 🐛 Fix for missing ToJson annotation SamplingRules deserialization exception (#6548 - @ygree)
• 🧹 Refactor how we apply custom transformers (#6541 - @mcculls)
• 🔍 Add requested and completed times to flare_info.txt (#6531 - @mcculls)

Instrumentations

Apache Spark instrumentation

• 🧪 Initial flush interval for long running spans (#6540 - @paul-laffon-dd)
• 🧪 Reduce long running default flush interval to 120 seconds (#6536 - @paul-laffon-dd)
• Use spark.app.name as default service if not set by the user (#6460 - @paul-laffon-dd)
• Capture relationships between spark SQL stages (#6459 - @paul-laffon-dd)

AWS SDK instrumentation

• 🐛 Avoid NumberFormatException when logging 128bit trace ID (#6570 - @amarziali)
• Add aws.object.key tag for aws-sdk-1 (#6555 - @paul-laffon-dd)

Jetty instrumentation

• ✨ Support jetty client 10 and 11 (#6590 - @amarziali)

Kafka instrumentation

• 🐛 KafkaConsumer: avoid null key context access (#6545 - @amarziali)

All other instrumentations

• Mute tracing for subprocesses executed in by the tracer (#6584 - @smola)
• Add aerospike 8 latestDepTests (#6543 - @amarziali)
• ✨ Allow axis server span to set local root resource name (#6537 - @amarziali)
• ✨ Add component and fix cmd.exit_code tag in subprocess instrumentation (#6502 - @smola)
• Attach command execution spans to current span if it exists (#6481 - @smola)

1.28.0

Components

Application Security Management (IAST)

• ⚡🧹 Refactor http headers to reduce memory allocations (#6516 - @manuel-alvarez-alvarez)
• Add IAST exclusion for OpenTelemetry SDK package (#6511 - @smola)
• ⚡ Replace string.format calls with regular concatenation (#6510 - @manuel-alvarez-alvarez)
• Add call sites in spring ldap template (#6509 - @manuel-alvarez-alvarez)
• Add IAST exclusion for org.openjdk.jol.vm (#6490 - @smola)
• Detect servlet configuration vulnerabilities with IAST (#6300 - @jandro996)

Continuous Integration Visibility

• Enable flaky tests retries based on remote settings (#6454 - @nikita-tkachenko-datadog)
• Implement flaky tests retries for Scalatest (#6447 - @nikita-tkachenko-datadog)
• Implement flaky test retries for JUnit 4 (#6435 - @nikita-tkachenko-datadog)
• Synchronously flush test session finish event (#6434 - @nikita-tkachenko-datadog)
• Implement flaky test retries for Test NG (#6347 - @nikita-tkachenko-datadog)

Dynamic Instrumentation

• ✨ Add TriageReport for Dynamic Instrumentation (#6519 - @jpbempel)
• Support special case of empty catch block (#6507 - @jpbempel)
• Use java.io.tmpdir for classfile dump directory (#6486 - @jpbempel)
• ✨ Add @exception as local var in snapshots (#6480 - @jpbempel)
• Rename isUndefined expression to isDefined (#6470 - @jpbempel)
• Enable diagnostics on debugger track (#6462 - @jpbempel)
• 🐛 Fix issue for parsing classes from debugger.symbol (#6455 - @jpbempel)
• Enforce ordering of probe instrumentation (#6444 - @jpbempel)
• Add finally block for @duration metric (#6436 - @jpbempel)
• Fix support of string primitives for index in EL (#6429 - @jpbempel)

Metrics

• 🔍 Provide summary of JMXFetch for tracer-flare (#6439 - @mcculls)
• 🔍 Tidy up tracer health metrics (#6438 - @mcculls)
• Keep historical tracer health/span counts (#6426 - @mcculls)
• 🔍 Add tracer health and span count summaries to tracer-flare (#6416 - @mcculls)

OpenTracing

• 🧪 Introduce blackhole span (#6326 - @amarziali)

Profiling

• Update the enablement logic for allocation and liveheap (#6492 - @jbachorik)
• add operation label to endpoint events (#6463 - @richardstartin)
• ✨ Proactively cleanup dangling JFR repositories on startup (#6441 - @jbachorik)
• Update ddprof to 0.94.0 (#6433 - @jbachorik)
• Stop collecting jdk.ClassLoadingStatistics (#6538 - @richardstartin)

Telemetry

• Improve telemetry log deduplication (#6525 - @smola)
• Decouple telemetry logs from minimum log level (#6493 - @smola)
• 🐛 Fix Telemetry App-Started event retry that prevents the application from exiting. (#6451 - @ygree)

Trace context propagation

• Be consistent when checking if tasks are excluded from wrapping/injection (#6446 - @mcculls)

Tracer core

• 🔍 Rename version.txt to tracer_version.txt in tracer-flare (#6532 - @mcculls)
• 🔍 Update Java tracer flare name to be dd-java-flare-<runtimeId>-<currentTimeMillis>.zip (#6506 - @mcculls)
• 🔍 Add thread-dump to tracer-flare when triage or debug mode is enabled (#6485 - @mcculls)
• 🔍 Support delayed trigger of local triage report (#6479 - @mcculls)
• 🔍 Add Instrumenter metrics to tracer-flare (#6472 - @mcculls)
• 🧪 Introduce blackhole span (#6326 - @amarziali)

Instrumentations

Apache Spark instrumentation

• Add spark SQL plan the spark.stage spans (#6305 - @paul-laffon-dd)

JAX-RS instrumentation

• 🐛 support PATCH and other custom jaxrs annotations (#6471 - @amarziali)

JMS instrumentation

• Add auto-instrumentation for Message Driven Beans (#6216 - @am312)

Kafka instrumentation

• Added messaging.kafka.bootstrap.servers to consume spans (#6327 - @kr-igor)

OpenTelemetry instrumentation

• ✨ Add support for tracestate extraction from OpenTelemetry API (#6362 - @PerfectSlayer)

Trace annotations instrumentation

• 🧪 Introduce blackhole span (#6326 - @amarziali)

All other instrumentations

• ✨ Extract user principal on jakarta servlets 5+ (#6527 - @amarziali)
• ✨ Add missing error tags to process spans (#6489 - @smola)
• 🐛 log4j: preserve mdc when copied and injection is disabled (#6487 - @amarziali)
• Don't create a process span for agent internal threads (#6478 - @bantonsson)
• 🐛 zio: fix context propagation (#6442 - @amarziali)

1.28.0-RC3

Components

Application Security Management (IAST)

• ⚡🧹 Refactor http headers to reduce memory allocations (#6516 - @manuel-alvarez-alvarez)
• Add IAST exclusion for OpenTelemetry SDK package (#6511 - @smola)
• ⚡ Replace string.format calls with regular concatenation (#6510 - @manuel-alvarez-alvarez)
• Add call sites in spring ldap template (#6509 - @manuel-alvarez-alvarez)
• Add IAST exclusion for org.openjdk.jol.vm (#6490 - @smola)
• Fix flaky test due to usage of Thread.sleep (#6457 - @manuel-alvarez-alvarez)
• Detect servlet configuration vulnerabilities with IAST (#6300 - @jandro996)

Continuous Integration Visibility

• Enable flaky tests retries based on remote settings (#6454 - @nikita-tkachenko-datadog)
• Implement flaky tests retries for Scalatest (#6447 - @nikita-tkachenko-datadog)
• Implement flaky test retries for JUnit 4 (#6435 - @nikita-tkachenko-datadog)
• Synchronously flush test session finish event (#6434 - @nikita-tkachenko-datadog)
• Implement flaky test retries for Test NG (#6347 - @nikita-tkachenko-datadog)

Dynamic Instrumentation

• Use java.io.tmpdir for classfile dump directory (#6486 - @jpbempel)
• Rename isUndefined expression to isDefined (#6470 - @jpbempel)
• Enable diagnostics on debugger track (#6462 - @jpbempel)
• 🐛 Fix issue for parsing classes from debugger.symbol (#6455 - @jpbempel)
• Enforce ordering of probe instrumentation (#6444 - @jpbempel)
• Add finally block for @duration metric (#6436 - @jpbempel)
• Fix support of string primitives for index in EL (#6429 - @jpbempel)

Metrics

• 🔍 Provide summary of JMXFetch for tracer-flare (#6439 - @mcculls)
• 🔍 Tidy up tracer health metrics (#6438 - @mcculls)
• Keep historical tracer health/span counts (#6426 - @mcculls)
• 🔍 Add tracer health and span count summaries to tracer-flare (#6416 - @mcculls)

OpenTracing

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Profiling

• Update the enablement logic for allocation and liveheap (#6492 - @jbachorik)
• add operation label to endpoint events (#6463 - @richardstartin)
• ✨ Proactively cleanup dangling JFR repositories on startup (#6441 - @jbachorik)
• Update ddprof to 0.94.0 (#6433 - @jbachorik)

Telemetry

• Decouple telemetry logs from minimum log level (#6493 - @smola)
• 🐛 Fix Telemetry App-Started event retry that prevents the application from exiting. (#6451 - @ygree)

Trace context propagation

• Be consistent when checking if tasks are excluded from wrapping/injection (#6446 - @mcculls)

Tracer core

• 🔍 Update Java tracer flare name to be dd-java-flare-<runtimeId>-<currentTimeMillis>.zip (#6506 - @mcculls)
• 🔍 Add thread-dump to tracer-flare when triage or debug mode is enabled (#6485 - @mcculls)
• 🔍 Support delayed trigger of local triage report (#6479 - @mcculls)
• 🔍 Add Instrumenter metrics to tracer-flare (#6472 - @mcculls)
• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Tracer public API

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Instrumentations

Apache Spark instrumentation

• Add spark SQL plan the spark.stage spans (#6305 - @paul-laffon-dd)

AWS SDK instrumentation

• Log object key name to spans for AWS s3 (#6311 - @hokitam)

JAX-RS instrumentation

• 🐛 support PATCH and other custom jaxrs annotations (#6471 - @amarziali)

JMS instrumentation

• Add auto-instrumentation for Message Driven Beans (#6216 - @am312)

Kafka instrumentation

• Added messaging.kafka.bootstrap.servers to consume spans (#6327 - @kr-igor)

OpenTelemetry instrumentation

• ✨ Add support for tracestate extraction from OpenTelemetry API (#6362 - @PerfectSlayer)

Trace annotations instrumentation

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

All other instrumentations

• ✨ Add missing error tags to process spans (#6489 - @smola)
• 🐛 log4j: preserve mdc when copied and injection is disabled (#6487 - @amarziali)
• Don't create a process span for agent internal threads (#6478 - @bantonsson)
• 🐛 zio: fix context propagation (#6442 - @amarziali)

Other changes

• Revert "Log object key name to spans for AWS s3" (#6529 - @bm1549)

1.28.0-RC2

Components

Application Security Management (IAST)

• ⚡🧹 Refactor http headers to reduce memory allocations (#6516 - @manuel-alvarez-alvarez)
• Add IAST exclusion for OpenTelemetry SDK package (#6511 - @smola)
• ⚡ Replace string.format calls with regular concatenation (#6510 - @manuel-alvarez-alvarez)
• Add call sites in spring ldap template (#6509 - @manuel-alvarez-alvarez)
• Add IAST exclusion for org.openjdk.jol.vm (#6490 - @smola)
• Fix flaky test due to usage of Thread.sleep (#6457 - @manuel-alvarez-alvarez)
• Detect servlet configuration vulnerabilities with IAST (#6300 - @jandro996)

Continuous Integration Visibility

• Enable flaky tests retries based on remote settings (#6454 - @nikita-tkachenko-datadog)
• Implement flaky tests retries for Scalatest (#6447 - @nikita-tkachenko-datadog)
• Implement flaky test retries for JUnit 4 (#6435 - @nikita-tkachenko-datadog)
• Synchronously flush test session finish event (#6434 - @nikita-tkachenko-datadog)
• Implement flaky test retries for Test NG (#6347 - @nikita-tkachenko-datadog)

Dynamic Instrumentation

• Use java.io.tmpdir for classfile dump directory (#6486 - @jpbempel)
• Rename isUndefined expression to isDefined (#6470 - @jpbempel)
• Enable diagnostics on debugger track (#6462 - @jpbempel)
• 🐛 Fix issue for parsing classes from debugger.symbol (#6455 - @jpbempel)
• Enforce ordering of probe instrumentation (#6444 - @jpbempel)
• Add finally block for @duration metric (#6436 - @jpbempel)
• Fix support of string primitives for index in EL (#6429 - @jpbempel)

Metrics

• 🔍 Provide summary of JMXFetch for tracer-flare (#6439 - @mcculls)
• 🔍 Tidy up tracer health metrics (#6438 - @mcculls)
• Keep historical tracer health/span counts (#6426 - @mcculls)
• 🔍 Add tracer health and span count summaries to tracer-flare (#6416 - @mcculls)

OpenTracing

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Profiling

• Update the enablement logic for allocation and liveheap (#6492 - @jbachorik)
• add operation label to endpoint events (#6463 - @richardstartin)
• ✨ Proactively cleanup dangling JFR repositories on startup (#6441 - @jbachorik)
• Update ddprof to 0.94.0 (#6433 - @jbachorik)

Telemetry

• Decouple telemetry logs from minimum log level (#6493 - @smola)
• 🐛 Fix Telemetry App-Started event retry that prevents the application from exiting. (#6451 - @ygree)

Trace context propagation

• Be consistent when checking if tasks are excluded from wrapping/injection (#6446 - @mcculls)

Tracer core

• 🔍 Update Java tracer flare name to be dd-java-flare-<runtimeId>-<currentTimeMillis>.zip (#6506 - @mcculls)
• 🔍 Add thread-dump to tracer-flare when triage or debug mode is enabled (#6485 - @mcculls)
• 🔍 Support delayed trigger of local triage report (#6479 - @mcculls)
• 🔍 Add Instrumenter metrics to tracer-flare (#6472 - @mcculls)
• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Tracer public API

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

Instrumentations

Apache Spark instrumentation

• Add spark SQL plan the spark.stage spans (#6305 - @paul-laffon-dd)

AWS SDK instrumentation

• Log object key name to spans for AWS s3 (#6311 - @hokitam)

JAX-RS instrumentation

• 🐛 support PATCH and other custom jaxrs annotations (#6471 - @amarziali)

JMS instrumentation

• Add auto-instrumentation for Message Driven Beans (#6216 - @am312)

Kafka instrumentation

• Added messaging.kafka.bootstrap.servers to consume spans (#6327 - @kr-igor)

OpenTelemetry instrumentation

• ✨ Add support for tracestate extraction from OpenTelemetry API (#6362 - @PerfectSlayer)

Trace annotations instrumentation

• 🧪 R&D week: blackhole span (#6326 - @amarziali)

All other instrumentations

• ✨ Add missing error tags to process spans (#6489 - @smola)
• 🐛 log4j: preserve mdc when copied and injection is disabled (#6487 - @amarziali)
• Don't create a process span for agent internal threads (#6478 - @bantonsson)
• 🐛 zio: fix context propagation (#6442 - @amarziali)

1.27.0

Components

Application Security Management (IAST)

• Add time and bucket size boundaries to the tainted map (#6389 - @manuel-alvarez-alvarez)
• Add Header injection vulnerability detection (#6299 - @DDJavierSantos)
• Add support for names in jackson propagation (#6132 - @manuel-alvarez-alvarez)
• Add support for SSRF in Apache HttpClient V4 (#6112 - @jandro996)

Continuous Integration Visibility

• ⚡ Optimise per-test code coverage for multi-threaded tests (#6412 - @nikita-tkachenko-datadog)
• Avoid submitting multiple identical coverage reports when test span has children that are top-level spans (#6411 - @nikita-tkachenko-datadog)

Data Streams Monitoring

• ✨ Track Kafka Avro schemas using Data Streams (#6370 - @piochelepiotr)
• Add Data Streams Monitoring support to SQS v1 (#6259 - @vandonr)

Database Monitoring

• 🐛 Fix context store access with null key on DBM connection (#6400 - @amarziali)

Dynamic Instrumentation

• ✨ Add instanceof operator in Expression Language (#6425 - @jpbempel)
• Remove retransformation on failed resolution (#6423 - @jpbempel)
• Introduce special toString for well known classes (#6419 - @jpbempel)
• ✨ Add specific support for Optional (#6406 - @jpbempel)
• ✨ Add emitting status for probes (#6365 - @jpbempel)

Metrics

• Introduce more local counters for tracer health metrics (#6424 - @mcculls)
• Update JMXFetch to 0.49.0 (#6397 - @mcculls)

Profiling

• Add a number of 'new' JFR events to be enabled by default (#6430 - @jbachorik)
• Capture the SELinux status as a profiler setting event (#6414 - @jbachorik)
• Auto-generate native-image argument to enable JFR when profiling enabled (#6369 - @jbachorik)

Telemetry

• Include dynamic_instrumentation product into the app_started telemetry event (#6422 - @ygree)
• Add install_signature to app-started (#6418 - @ygree)

Tracer core

• Add new javaassist proxy separator to ignores (#6413 - @amarziali)
• 🔍 Report instrumenter state in tracer-flare (#6387 - @mcculls)

Instrumentations

JDBC instrumentation

• 🐛 Fix context store access with null key on DBM connection (#6400 - @amarziali)

OpenTelemetry instrumentation

• ✨ Use OpenTelemetry SDK context key instead of Datadog proprietary one for local root span (#6399 - @PerfectSlayer)

All other instrumentations

• Add instrumentation to allow sending kafka payload size on produce (#6228 - @vandonr)

Other changes

• Bump integrations-core submodule to 7.50.0 (#6396 - @mcculls)
• Bump byte-buddy to 1.14.9 (#6395 - @mcculls)