Skip to content

RUM-19112: Adopt FactoryKit Foundation - #4013

Merged
mariusc83 merged 2 commits into
developfrom
mariusc83/RUM-19112/factorykit-foundation
Oct 22, 2026
Merged

mariusc83 merged 2 commits into
developfrom
mariusc83/RUM-19112/factorykit-foundation

Conversation

@mariusc83

@mariusc83 mariusc83 commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

What does this PR do?

Adds Foundation-only repository guidance, a documentation map, and a manually invoked documentation gate to dd-sdk-android. Contributors and agents get one route for documentation review and deterministic validation before submitting changes.

Ticket: https://datadoghq.atlassian.net/browse/RUM-19112

The ten-file change preserves existing agent guidance and adds:

  • docs/README.md: routes to existing guides, documentation ownership, review expectations, and validation commands.
  • ci/scripts/foundation_check.py: the repository-owned entry selecting AGENTS.md, the documentation map, built-in document checks, and an advisory reminder.
  • factoryspec.yaml: selects Foundation and references the report and reminder descriptors.
  • factory/checks/foundation-report.yaml and factory/nudges/baseline-documentation-review.yaml: declare implementations, schemas, and bindings to the Python entry.
  • Four JSON schemas: define the input/output contracts for the report and reminder.

The Python entry remains executable authority. The YAML declares and validates its selected bindings; it does not install the validator or activate automatic execution.

Motivation

Catch documentation regressions and make validation repeatable for contributors and coding agents. Keep mechanical validation distinct from the review of whether documentation accurately explains the implementation.

Additional Notes

Runtime requirement and installation boundary

Anyone running these commands needs a Python environment with FactoryKit and its dependencies installed. The entry imports foundation_entry from factorykit.document_checks; this PR does not vendor that implementation. Cloning dd-sdk-android alone is therefore insufficient to execute the gate. The YAML and JSON files configure the validator and validate data shapes; they are not standalone executables.

For the local validation reported here, FactoryKit was installed from a separate local FactoryKit source checkout with that checkout's bash scripts/setup.sh. That script selects Python 3.10 or newer, creates an isolated environment by default at ~/.cache/factorykit/runtime, and installs FactoryKit plus its dependencies. Installation can download dependencies. The environment lives outside the Android repository and contains the Python interpreter, dependencies, and command launcher. The source remains in the FactoryKit checkout.

This is an editable installation: subsequent changes to that source checkout can change validator behavior. It demonstrates local execution but does not establish a reproducible team-wide runtime. This PR does not select a pinned distribution, dependency lock, or team installation mechanism. Those need a separate reviewed follow-up; contributors should not infer a generic package-index installation command from this PR.

When run locally, the contributor or agent needs that environment. If a later CI integration provisions it on the runner, contributors can receive PR validation results without installing it locally. Local execution would still require an installation.

What runs, and when

The Foundation gate evaluates tracked supported documentation across the selected Git snapshot, including files outside the documentation map. It checks required entry/map files, entry-to-map navigation, and supported local links and anchors. It compares findings with a baseline and returns allow or block, plus existing/new/resolved findings, coverage limits, and evaluation errors.

The reminder emits an advisory about documentation impact, navigation, instruction scope, and prose review for changed paths. It does not perform semantic review, edit files, or block the gate. An author or reviewer must separately assess factual accuracy, reader usefulness, and whether changed behavior needs documentation, and record that assessment in the PR.

Run the gate before committing or submitting changes, and repeat affected validation after review edits. Run Factoryspec validation and binding validation when the configuration, schemas, or entry selections change. This PR instructs agents through AGENTS.md; it does not guarantee compliance through automation.

From the Android repository, with the local runtime used for this validation:

# Validate staged configuration and its agreement with the entry.
~/.cache/factorykit/runtime/bin/factorykit --root . --format json \
  factoryspec validate --path factoryspec.yaml --snapshot index
~/.cache/factorykit/runtime/bin/factorykit --root . --format json \
  factoryspec bindings --path factoryspec.yaml \
  --entry-point ci/scripts/foundation_check.py --snapshot index

# Gate staged changes against the current commit.
~/.cache/factorykit/runtime/bin/python ci/scripts/foundation_check.py \
  --snapshot index --against HEAD --format json

# Gate the committed onboarding branch against the fetched base branch.
~/.cache/factorykit/runtime/bin/python ci/scripts/foundation_check.py \
  --snapshot HEAD --against origin/develop --format json

The executable paths above describe the local environment used here, not an installation delivered by this PR. Factoryspec reads require tracked files in the selected snapshot; an untracked or partially staged draft is not validation evidence for the intended complete configuration.

These validation operations read local Git content. They do not need model calls, embeddings, network retrieval, or a SQLite index. They do not compile the SDK or run Kotlin tests. External links and unsupported Markdown syntax remain outside coverage; a passing gate does not prove documentation accuracy.

Validation evidence

  • Staged Factoryspec validation: passed.
  • Staged entry bindings: both matched.
  • Staged Foundation gate: allow, no new findings or required evaluation errors.
  • Committed onboarding branch gate against fetched origin/develop: allow, no new findings or required evaluation errors.
  • Whitespace, Python/JSON syntax, new documentation routes, and staged/worktree consistency: passed during preparation.
  • Two pre-existing findings remain on this branch: the historical API-surface link in CHANGELOG.md and the OpenFeature README anchor hidden by an unclosed code fence. They are addressed separately in RUM-19113: Repair historical API link and OpenFeature code fence #4014.
  • External-link/Markdown coverage limitations remain visible in the report; its coverage-message list reached its output cap.

No SDK behavior, CI configuration, Git hooks, or host settings change. Creating this PR does not itself trigger FactoryKit validation. No automatic documentation-review command is configured.

Integration with current SDK agent skills

Rebased onto develop at fda9e7176a. The guidance now links the five canonical SDK skill sources under ai/skills/: tests, review, commits, PR preparation, and PR stacks. Existing .agents/skills/ and .claude/skills/ aliases remain unchanged.

The documentation map explains when each skill applies and how to run the Foundation gate alongside sdk-pr-prepare, which does not invoke it automatically. It distinguishes local preparation from documentation-only validation and uses the actual parent branch as the base for stacked PRs.

Canonical links avoid the current Foundation limitation when resolving tracked directory symlinks. Staged validation passes with all five skill-link findings resolved and no new required errors; the pre-existing historical changelog link remains assigned to cleanup PR #4014. factoryspec.yaml keeps skills: []: repository skill discovery does not require registration as FactoryKit invocation contracts.

Next steps and follow-up PRs

The selected onboarding scope is Foundation only. Foundation can stand alone; optional peers are not prerequisites for merging this PR.

  1. Merge this onboarding PR after review. This prepares repository-owned guidance and checks. It does not activate a host control or CI gate.
  2. Merge the separate stacked cleanup PR RUM-19113: Repair historical API link and OpenFeature code fence #4014 / RUM-19113. It restores the historical release 1.2.0 API link and closes the OpenFeature dependency example's code fence. Its gate passed against the onboarding parent with both findings resolved and no remaining findings. After the parent merges, retarget/reconcile the child against develop and validate the resulting diff before merging.
  3. Proposed follow-up: reproducible runtime setup. Select an approved built, pinned FactoryKit distribution and installation mechanism, document supported Python/environment requirements and upgrade/recovery procedures, and verify a clean installation. The repository maintainers own this decision. No installation mechanism or version is selected by this PR.
  4. Proposed follow-up: automatic Foundation validation. If maintainers want enforcement, add a separate reviewed CI integration that provisions the pinned runtime, evaluates the intended PR snapshot/base, and exposes reports. Local hook or host integration is another optional decision. CI/host activation and any required branch-protection configuration need separate approval. Until then, execution remains manual.
  5. Optional later onboarding sessions. Fresh-scan the repository, select scope, review an exact no-write plan, and apply only approved changes. Knowledge would add deterministic local metadata/retrieval; Literate would add selected reader-focused designs/guides and narratives; Agent-team would add coordinated work and independent verification. These peers can be selected later in any order after Foundation. None is selected or activated here. The current Agent-team onboarding route is Linear-only; the Jira tickets for these PRs do not establish Agent-team adoption or Jira support.

Items 1–2 are the current two-PR stack. Items 3–5 are proposals, not approved or scheduled implementation. Runtime installation, external configuration, and activation remain separate from repository preparation.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

@mariusc83
mariusc83 requested review from a team as code owners October 21, 2026 07:41
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T07:33:30.867276Z 2b13514 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-datadog-us1-prod datadog-datadog-us1-prod Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Static validation found consistent Foundation bindings, valid schemas, and resolving documentation references. The manual gate’s runtime behavior remains unverified because FactoryKit is not installed.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 6b123d6

@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog retried 2 tests - 2 passed on retry View in Datadog

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 71.81% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 2b13514 | Docs | View more details | Give us feedback!

@0xnm 0xnm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Be aware of #3997, we probably need to revisit this PR after.

@mariusc83

Copy link
Copy Markdown
Member Author

Be aware of #3997, we probably need to revisit this PR after.

@0xnm oh thanks for the heads-up, yes there will be several improvements after, this is the introduction to factoryKit.

@mariusc83
mariusc83 marked this pull request as ready for review October 21, 2026 08:38
0xnm
0xnm previously approved these changes Oct 21, 2026
@buraizu buraizu self-assigned this Oct 21, 2026

buraizu commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Created DOCS-16079 for the editorial review.

@buraizu buraizu removed their assignment Oct 21, 2026
Link canonical skill sources and document Foundation validation alongside SDK preparation.

Ref: RUM-19112
@mariusc83
mariusc83 force-pushed the mariusc83/RUM-19112/factorykit-foundation branch from 6b123d6 to 2b13514 Compare October 22, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants