Skip to content

egress: add credential provider which reads from k8s secret - #1335

Open
yufan-su wants to merge 2 commits into
agent-substrate:mainfrom
yufan-su:cred-provider-plugin
Open

egress: add credential provider which reads from k8s secret#1335
yufan-su wants to merge 2 commits into
agent-substrate:mainfrom
yufan-su:cred-provider-plugin

Conversation

@yufan-su

@yufan-su yufan-su commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

First of a two-PR stack for atenet egress credential injection change. This PR adds the credential provider; the follow-up adds the egress injector that calls it.

What this adds

A credprovider gRPC service implementing the CredentialProvider.RequestSecret plugin API (pkg/proto/credproviderpb), backed by Kubernetes Secrets. It resolves substrate-secret://kubernetes.io/<provider>/<namespace>/<secret>[/<key>] URIs to a Secret value — brokering a read it's authorized to perform rather than persisting the secret itself.

Authorization

  • Default-deny atespace→namespace mapping (internal/proto/nsauthzpb); an atespace can only resolve secrets from its permitted namespaces.
  • The atespace is derived from the attested actor SPIFFE identity (internal/actorspiffe, shared with the injector in the next PR).
  • mTLS with optional client-CA verification for callers.

Included

  • cmd/credprovider (service + kubeprovider backend)
  • pkg/proto/credproviderpb (plugin API), internal/proto/nsauthzpb (authz mapping), internal/actorspiffe (SPIFFE parser)
  • Deploy manifest + namespace-policy / sample-secret fixtures

@EItanya

Eitan Yarmush (EItanya) commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

I think this PR is honestly a bit noisy. There's both the credential provider, and the implementation of yet another extproc server which uses it. Can we instead review the credential provider in isolation, especially given that there are still open questions in the doc, and then we can tackle the thing which calls it?

Doc since it's not linked anywhere: https://docs.google.com/document/d/1-zQKvu-oYJlva_kwL3xbAOaFoJJaqO0qEqmqRlt1r0c/edit?resourcekey=0-TDrWohfRuYolcXYO-QPVTQ&tab=t.0#heading=h.rx0okwgtp5yu

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed offline but for posterity, Eitan Yarmush (@EItanya) 's idea to split the pr sounds good.

Comment thread manifests/egress-credential-injection/README.md Outdated
@yufan-su
yufan-su force-pushed the cred-provider-plugin branch 2 times, most recently from 02a01c3 to d2731aa Compare September 1, 2026 03:59
@yufan-su yufan-su changed the title egress: add credential injector and credential provider egress: add credential provider which reads from k8s secret Sep 1, 2026
@yufan-su

yufan-su commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

Split into two PRs #1335 and #1360 and this one only contains the credential provider change.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks yufan-su !

Comment thread cmd/credprovider/internal/kubeprovider/kubeprovider.go Outdated
Comment thread internal/proto/nsauthzpb/nsauthz.proto Outdated
Comment thread manifests/egress-credential-injection/namespace-policy.yaml Outdated
namespace: ate-system
---
apiVersion: apps/v1
kind: Deployment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creating YET another deployment for this feels like it spreads our deployment footprint out quite a bit. What do you think about running this as a sidecar for the egress policy, or from the control-plane somehow? I'm worried about:

  1. Latency
  2. deployment footprint

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to see if we consider this more of an example vs something that a lot of users will use.

Key things that would make some of the decisions:

  1. Would this be enabled by default (with opt-out)
  2. How we document the positioning of this (it's for demos vs it's for prod)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's a fair point, what do we consider it? How would a user actually deploy secrets and providers to their env?

Comment thread internal/proto/nsauthzpb/nsauthz.proto Outdated
// The attested actor identity on whose behalf the secret is fetched. Today
// this is the actor's SPIFFE URI as verified by the egress gateway; a
// verifiable Actor JWT is the intended future form.
string actor_identity = 1;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this secure? Wouldn't we need the actual actor identity document?

Comment on lines +27 to +32
# atespace "team-a" may resolve secrets in namespace "ns1" (matches the
# sample policy's substrate-secret://kubernetes.io/team-secrets/ns1/example-api).
policy {
atespace: "team-a"
allowed_namespaces: "ns1"
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has this API been discussed at all? This seems like an important piece we need to come to agreement on which was not in the doc Bowei Du (@bowei) Lior Lieberman (@LiorLieberman)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's somewhat separate than the doc as this is specific to cred provider implementation.

I was thinking for this example, we do something very simple like a static YAML map of atespace(s) to k8s namespaces.

Nit: probably shouldn't use textproto, this is not a very popular format (outside of Google)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so this who PR is really meant to be an example of what one of these plugins could be? I think overall it's not clear to me how various plugins will work, what their lifecycle will be, how they will be accessed by the PEP, etc

Comment thread internal/proto/nsauthzpb/nsauthz.proto Outdated

// TrustDomain is the SPIFFE host an actor identity URI carries, matching the URI
// minted in cmd/ateapi/internal/actoridentity.
const TrustDomain = "substrate-actor.local"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This relates to my comment about the actual actor credential, I think this is a code smell from that

Comment thread cmd/credprovider/main.go Outdated
metricsAddr = pflag.String("metrics-address", ":9090", "Prometheus/health HTTP listen address")
serverBundle = pflag.String("server-cred-bundle", "", "credential bundle (PEM key+chain) presented for serving TLS; empty serves plaintext (dev only)")
clientCAFile = pflag.String("client-ca-file", "", "CA bundle that caller (injector) client certificates must chain to; empty accepts any client when TLS is on")
defaultKey = pflag.String("default-secret-key", "", "Secret data key used when a credential URI omits one; empty requires a single-key Secret")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a good idea

@yufan-su
yufan-su force-pushed the cred-provider-plugin branch from 775787a to 6e30f5a Compare September 3, 2026 20:38
@yufan-su
yufan-su force-pushed the cred-provider-plugin branch from 6e30f5a to 150c6b2 Compare September 3, 2026 23:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants