test(ffe): add agentless serverless exposure contract by leoromanovsky · Pull Request #7477 · DataDog/system-tests

leoromanovsky · 2026-08-07T21:17:17Z

Motivation

Our business goal is to make the Datadog OpenFeature client simple to integrate.

Agentless delivery is the default. Customers can send exposures directly or add serverless-init later. All three paths must produce identical exposure behavior.

The existing system test covers only Remote Configuration with Agent egress. It does not cover either agentless exposure path.

flowchart LR
  Evaluation["Feature flag evaluation"] --> Cache["Exposure cache"]
  Cache --> Agent["Agent EVP proxy"]
  Agent --> Intake["Exposure intake"]

  Cache -. "Missing contract" .-> Sidecar["serverless-init EVP proxy"]
  Sidecar -.-> Intake

  Cache -. "Missing contract" .-> Direct["Direct HTTPS EVP"]
  Direct -.-> Intake

The solid path has system-test coverage before this PR. The dashed paths do not.

Changes and Decisions

Add one scenario for agentless sidecar egress.
Add one scenario for agentless direct egress.
Define one exposure setup and one test method in ExposureEgressContract.
Use three small adapters to select the scenario and capture interface.
Use one test definition and identical semantic expectations across all three paths.
Start serverless-init:1.9.13 only for the sidecar scenario.
Run the direct scenario without a local receiver.
Assert that the unused path receives no duplicate exposure.
Gate unsupported scenario adapters in each SDK manifest.

flowchart TB
  RC["Remote Configuration"] --> SDK["Evaluation and exposure cache"]
  UFC["Agentless UFC"] --> SDK

  SDK --> Agent["Agent EVP proxy"]
  SDK --> Sidecar["serverless-init 1.9.13"]
  SDK --> Direct["Direct HTTPS EVP"]

  Agent --> AgentCapture["Agent capture"]
  Sidecar --> SidecarCapture["Sidecar capture"]
  Direct --> DirectCapture["Direct capture"]

  AgentCapture --> Contract["One shared side-effects contract"]
  SidecarCapture --> Contract
  DirectCapture --> Contract

Test matrix

Configuration delivery	Local receiver	Exposure egress	Scenario	Test file	Capture interface
Remote Configuration	Datadog Agent	Agent EVP proxy	`FEATURE_FLAGGING_AND_EXPERIMENTATION`	`test_exposure_egress.py`	`interfaces.agent`
Agentless UFC	`serverless-init:1.9.13`	Sidecar EVP proxy	`FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_SERVERLESS`	`test_exposure_egress.py`	`interfaces.datadog_sidecar`
Agentless UFC	None	Direct HTTPS EVP	`FEATURE_FLAGGING_AND_EXPERIMENTATION_AGENTLESS_DIRECT`	`test_exposure_egress.py`	`interfaces.datadog_direct`

Each adapter inherits the same contract. Five identical evaluations must produce one exposure with identical semantic fields.

Note to reviewers

Failures in test dd-gitlab/K8S_LIB_INJECTION_PROFILING_ENABLED: [dd-lib-java-init-test-app, ${PRIVATE_DOCKER_REGISTRY}/system-tests/dd-lib-java-init-test-app are on main branch and known, not related to this diff.

github-actions · 2026-08-07T21:17:53Z

CODEOWNERS have been resolved as:

tests/ffe/test_exposure_egress.py                                       @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
tests/ffe/utils/exposures.py                                            @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
.github/workflows/run-end-to-end.yml                                    @DataDog/system-tests-core
manifests/cpp_httpd.yml                                                 @DataDog/dd-trace-cpp
manifests/cpp_nginx.yml                                                 @DataDog/dd-trace-cpp
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
manifests/golang.yml                                                    @DataDog/dd-trace-go-guild
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
manifests/nodejs.yml                                                    @DataDog/dd-trace-js
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
manifests/python.yml                                                    @DataDog/apm-python @DataDog/asm-python
manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
manifests/rust.yml                                                      @DataDog/apm-rust
mirror_images.lock.yaml                                                 @DataDog/system-tests-core
mirror_images.yaml                                                      @DataDog/system-tests-core
tests/ffe/README.md                                                     @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
tests/test_the_test/test_compute_libraries_and_scenarios.py             @DataDog/system-tests-core
tests/test_the_test/test_group_rules.py                                 @DataDog/system-tests-core
tests/test_the_test/test_mock_ffe_agentless_backend.py                  @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core
utils/_context/_scenarios/endtoend.py                                   @DataDog/system-tests-core
utils/_context/containers.py                                            @DataDog/system-tests-core
utils/interfaces/__init__.py                                            @DataDog/system-tests-core
utils/proxy/core.py                                                     @DataDog/system-tests-core
utils/proxy/ports.py                                                    @DataDog/system-tests-core
utils/scripts/libraries_and_scenarios_rules.yml                         @DataDog/system-tests-core
tests/ffe/test_exposures_datadog_agent.py                               @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core

datadog-official · 2026-08-07T21:23:01Z

✨ Unblock PR with BitsAI

⚠️ Warnings

🚦 7 Pipeline jobs failed

Testing the test | System Tests (php, dev) / End-to-end #2 / laravel11x 2

See error
Upload attempts for artifact 'logs_endtoend_php_laravel11x_dev_2_7937f32f968da314' failed due to connectivity issues.
🧪 1 Test failed
tests.stats.test_stats.Test_Peer_Tags.test_peer_tags[laravel11x] from system_tests_suite
AssertionError: Client spans should have peer tags, found: []
assert 0 &gt; 0
 &#43;  where 0 = len([])

self = &lt;tests.stats.test_stats.Test_Peer_Tags object at 0x7fe77ca76ba0&gt;

    def test_peer_tags(self):
        &#34;&#34;&#34;Test that client spans include peer tags while server spans don&#39;t&#34;&#34;&#34;
        client_stats_found = False
        server_stats_found = False
...
DataDog/system-tests | K8S_LIB_INJECTION_PROFILING_ENABLED: [dd-lib-java-init-test-app, ${PRIVATE_DOCKER_REGISTRY}/system-tests/dd-lib-java-init-test-app, 235494822917.dkr.ecr.us-east-1.amazonaws.com/ssi/cluster-agent:7.81.1, 235494822917.dkr.ecr.us-east-1.amazonaws.com/ssi/apm-inject:latest, 3.231.5, 235494822917.dkr.ecr.us-east-1.amazonaws.com/ssi/dd-lib-java-init:latest]

DataDog/system-tests | K8S_LIB_INJECTION_PROFILING_ENABLED: [dd-lib-java-init-test-app, ${PRIVATE_DOCKER_REGISTRY}/system-tests/dd-lib-java-init-test-app, 235494822917.dkr.ecr.us-east-1.amazonaws.com/ssi/cluster-agent:7.81.1, 235494822917.dkr.ecr.us-east-1.amazonaws.com/ssi/apm-inject:latest, 3.231.5, 235494822917.dkr.ecr.us-east-1.amazonaws.com/ssi/dd-lib-java-init:latest_snapshot]
View all 7 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: e230b5d | Docs | Datadog PR Page | Give us feedback!}

Copilot

Pull request overview

Adds system-test coverage for Feature Flags & Experimentation (FFE) exposure delivery across three egress topologies (Agent EVP proxy, agentless direct HTTPS, and agentless serverless-init sidecar), aiming to enforce an identical exposure contract regardless of deployment path.

Changes:

Introduces two new agentless end-to-end scenarios (direct + serverless-init sidecar) and wires them into CI selection/rules.
Extends the proxy/interface layer with two new capture routes (datadog_direct, datadog_sidecar) and a serverless-init:1.9.13 helper container.
Adds a shared exposure egress contract test (tests/ffe/test_exposure_egress.py) and a route-neutral helper module for exposure assertions.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
utils/scripts/libraries_and_scenarios_rules.yml	Maps new scenarios/tests into the dynamic library/scenario selection rules.
utils/proxy/ports.py	Adds proxy ports for sidecar vs direct Datadog traffic capture.
utils/proxy/core.py	Routes new ports through mocked-backend handling and assigns new interface names.
utils/interfaces/init.py	Exposes new proxy-based interfaces (`datadog_sidecar`, `datadog_direct`).
utils/_context/containers.py	Adds `ServerlessInitContainer` definition for sidecar scenario.
utils/_context/_scenarios/endtoend.py	Extends agentless FFE scenario to support sidecar/direct exposure egress + capture interfaces.
utils/_context/_scenarios/init.py	Registers two new scenario names for direct + serverless agentless exposure egress.
tests/test_the_test/test_mock_ffe_agentless_backend.py	Adds unit-style checks for new agentless exposure scenario topology/env wiring.
tests/test_the_test/test_group_rules.py	Ensures new scenarios are excluded from tracer-release grouping and are in `ffe` group.
tests/test_the_test/test_compute_libraries_and_scenarios.py	Updates scenario computation expectations for new files/scenarios.
tests/ffe/utils/exposures.py	Introduces shared exposure matching + contract assertions used across egress paths.
tests/ffe/test_exposures_datadog_agent.py	Refactors to reuse shared exposure helpers and removes redundant same-subject caching test (replaced by new contract).
tests/ffe/test_exposure_egress.py	New shared contract test covering exposure semantics + “no duplicate egress” expectations.
tests/ffe/README.md	Updates FFE test index to reflect new/renamed exposure tests.
mirror_images.yaml	Adds `datadog/serverless-init:1.9.13` to mirrored images list.
mirror_images.lock.yaml	Locks digest/target for the new mirrored `serverless-init` image.
manifests/rust.yml	Updates skips/versions for new exposure tests (agentless tests marked missing_feature).
manifests/ruby.yml	Updates skips/versions for new exposure tests (agentless tests marked missing_feature).
manifests/python.yml	Updates skips/versions for new exposure tests (agentless tests marked missing_feature).
manifests/php.yml	Updates skips/versions for new exposure tests (agentless tests marked missing_feature).
manifests/nodejs.yml	Updates skips/versions for new exposure tests (agentless tests marked missing_feature with issue link).
manifests/java.yml	Updates skips/versions for new exposure tests (agentless tests marked missing_feature).
manifests/golang.yml	Updates skips/versions for new exposure tests (agentless tests marked missing_feature).
manifests/dotnet.yml	Updates skips/versions for new exposure tests (agentless tests marked missing_feature).
manifests/cpp_nginx.yml	Updates skips/versions for new exposure tests (missing_feature).
manifests/cpp_httpd.yml	Updates skips/versions for new exposure tests (missing_feature).
.github/workflows/run-end-to-end.yml	Adds CI steps to run the two new agentless exposure scenarios.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

test(ffe): add agentless serverless exposure contract

4c36f02

leoromanovsky commented Aug 7, 2026

View reviewed changes

test(ffe): separate serverless contract from SDK enablement

f387372

leoromanovsky mentioned this pull request Aug 8, 2026

test(ffe): enable Node.js agentless serverless exposures [nodejs@ffl-2697-nodejs-serverless-evp-routing] #7478

Draft

fix(ffe): use integer agentless poll interval

972b475

leoromanovsky added a commit that referenced this pull request Aug 8, 2026

Merge PR #7477 into Node.js exposure enablement

ecac126

leoromanovsky commented Aug 8, 2026

View reviewed changes

Comment thread utils/proxy/core.py Outdated

Comment thread utils/proxy/ports.py Outdated

Comment thread utils/scripts/libraries_and_scenarios_rules.yml

leoromanovsky added 7 commits August 8, 2026 11:33

test(ffe): use Datadog exposure route names

ab7e4e5

test(ffe): cover direct agentless exposure egress

009373c

test(ffe): account for capture key scrubbing

6a43bea

test: register direct exposure scenario group exception

4c2aca8

ci: run direct agentless exposure scenario

be461a4

test(ffe): identify Agent exposure tests

690a85b

test: share exposure egress contract

fd59899

leoromanovsky requested a lite review from Copilot August 9, 2026 16:20

Copilot started reviewing on behalf of leoromanovsky August 9, 2026 16:21 View session

Copilot AI reviewed Aug 9, 2026

View reviewed changes

Comment thread tests/ffe/test_exposure_egress.py Outdated

test: accept scrubbed sidecar API key

e230b5d

leoromanovsky marked this pull request as ready for review August 9, 2026 17:12

leoromanovsky requested review from a team as code owners August 9, 2026 17:12

leoromanovsky requested review from ZStriker19, christophe-papazian, claponcet, dromanol, mhlidd, sameerank, tabgok and typotter and removed request for a team August 9, 2026 17:12

leoromanovsky mentioned this pull request Aug 9, 2026

feat(openfeature): route agentless exposure events DataDog/dd-trace-js#9527

Draft

Navigation Menu

Uh oh!

test(ffe): add agentless serverless exposure contract - #7477

test(ffe): add agentless serverless exposure contract#7477
leoromanovsky wants to merge 11 commits into
mainfrom
agent/nodejs-agentless-serverless-exposures

leoromanovsky commented Aug 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 7, 2026 •

edited

Loading

Uh oh!

datadog-official Bot commented Aug 7, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

leoromanovsky commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes and Decisions

Test matrix

Note to reviewers

Uh oh!

github-actions Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datadog-official Bot commented Aug 7, 2026 • edited by datadog-datadog-prod-us1-2 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

leoromanovsky commented Aug 7, 2026 •

edited

Loading

github-actions Bot commented Aug 7, 2026 •

edited

Loading

datadog-official Bot commented Aug 7, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading