diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 3385ffd..a049cf1 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -38,7 +38,7 @@ CI runs the same verification on pull requests and `main`; see `.github/workflow | `SOURCEGRAPH_SEED_DEMO` | Set `true` to register the two public example repositories | | `SOURCEGRAPH_GITHUB_TOKEN` | Optional GitHub token, used until a global GitHub token is saved; the earlier name `SOURCEGRAPH_GIT_TOKEN` is still read when this is unset | | `SOURCEGRAPH_CUSTOM_GIT_TOKEN` | Optional token for the custom Git source, used until a global one is saved | -| `SOURCEGRAPH_CONFIG_FILE` | Optional JSON file that overrides code sources at startup; see [code sources](docs/interfaces.md#code-sources) | +| `SOURCEGRAPH_CONFIG_FILE` | Optional JSON file that overrides code sources at startup; see [code sources](docs/code-sources.md) | | `SOURCEGRAPH_CREDENTIAL_KEY` | Optional base64 32-byte key for stored code credentials; otherwise `code-credentials.key` is generated in the data root. Required, and identical, on all instances sharing a database | | `SOURCEGRAPH_API_TOKEN` | Optional service automation bearer credential, at least 32 characters | | `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET` | Google web OAuth client | diff --git a/README.md b/README.md index 73d13b3..2c2f950 100644 --- a/README.md +++ b/README.md @@ -73,7 +73,7 @@ docker run --rm -p 8080:8080 -v sourcegraph-data:/data \ context-sourcegraph ``` -Open . Public GitHub repositories need no token; a custom Git host can be added in system settings or a startup file ([code sources](docs/interfaces.md#code-sources)). Without OAuth credentials the site is an anonymous, read-only instance; add Google or GitHub OAuth to enable administration — the first verified user becomes the administrator ([authentication](docs/authentication.md)). +Open . Public GitHub repositories need no token; a custom Git host can be added in system settings or a startup file ([code sources](docs/code-sources.md)). Without OAuth credentials the site is an anonymous, read-only instance; add Google or GitHub OAuth to enable administration — the first verified user becomes the administrator ([authentication](docs/authentication.md)). All environment variables are listed in [DEVELOPMENT.md](DEVELOPMENT.md#configuration). diff --git a/docs/README.md b/docs/README.md index 3103ccb..d91d7d7 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,6 +1,7 @@ # Documentation - [Authentication and demo access](authentication.md) +- [Code sources and tokens](code-sources.md) - [Deployment, storage and backups](deployment.md) - [HTTP, MCP and CLI](interfaces.md) - [Repository plugins](ref/repository-plugins.md) diff --git a/docs/code-sources.md b/docs/code-sources.md new file mode 100644 index 0000000..5adf269 --- /dev/null +++ b/docs/code-sources.md @@ -0,0 +1,78 @@ +# Code sources and tokens + +A code source is the Git host a repository is registered against; the service fetches the repository over that host's HTTPS address. This page covers the available sources, how to configure them, and which token each fetch uses. + +## Sources + +Every instance has exactly two code sources, listed GitHub first: + +| id | Default host | Token variable | Anonymous | Editable on the website | +| --- | --- | --- | --- | --- | +| `github` | `https://github.com` | `SOURCEGRAPH_GITHUB_TOKEN` | yes | no | +| `git` | none | `SOURCEGRAPH_CUSTOM_GIT_TOKEN` | no | yes | + +- GitHub has a fixed host. Public repositories need no token; a read-only token lifts anonymous rate limits and reaches private repositories. +- The custom Git source connects one self-hosted Git server, such as Gitea or GitLab. It has no host until one is set, and repositories cannot be registered against it before then. + +Each source has these fields: + +| Field | Meaning | +| --- | --- | +| `name` | Name shown when registering a repository | +| `host` | HTTPS origin of the Git server, without a path, user info, query or fragment | +| `username` | Username Git sends with the token over HTTPS; most hosts accept `oauth2`, the default | +| `token_env` | Environment variable holding the deployment token; set only in release defaults or the startup file | + +## Settings pages + +- **System settings → Code sources** (administrators) shows one card per source. The custom Git card holds its name, host and HTTPS username; every card holds the global token and an optional expiry, used to warn 7 days ahead. One save button submits whatever changed on the card. +- **User settings → Code source tokens** holds your personal token for each source. + +Saved tokens are never shown again; leave the field blank to keep the current one. Changes apply to the next Git operation without a restart. + +## Which token a fetch uses + +Each fetch tries, in order: + +1. the repository creator's personal token for that source; +2. the global token for that source; +3. the environment token for that source, only while no global token is saved; +4. anonymous access, for GitHub only. + +Sources never share tokens: a custom Git repository never reads a GitHub token. Tokens are sent only to the host of the current remote, with the source's HTTPS username. Authentication failures move on to the next token; timeouts, cancellations and disk errors do not. + +`github` still reads the earlier variable `SOURCEGRAPH_GIT_TOKEN` while `SOURCEGRAPH_GITHUB_TOKEN` is unset (`legacy_token_env` in the release defaults); a startup file that sets `token_env` stops that fallback. Clearing a saved global token does not bring back the environment token. + +## Configuration layers + +Settings apply in this order, later non-empty fields winning: + +1. Release defaults in `config/deployment.json`, which fix the set and order of sources. +2. The custom Git card on the website, stored in site settings and applied immediately. +3. The startup file named by `SOURCEGRAPH_CONFIG_FILE`, read once at startup: + +```json +{"code_sources": {"git": {"name": "Gitea", "host": "https://git.example.com", "username": "oauth2", "token_env": "SOURCEGRAPH_CUSTOM_GIT_TOKEN"}}} +``` + +- The file can only override existing ids; an unknown id, an invalid host or an invalid `token_env` stops startup. Never put a token in the file. +- Fields set by the file are read-only on the website (`409 CODE_SOURCE_LOCKED`). + +## Registered repositories + +- Registration pins the source host to the repository as `code_host`; later configuration changes never move existing repositories. Startup backfills a missing `code_host` and only logs when it differs from the current configuration. +- While any repository record uses the custom source, changing its host returns `409 CODE_SOURCE_IN_USE`; name and username can still change. +- Registering against a source without a host returns `409 CODE_SOURCE_NOT_CONFIGURED`. + +## HTTP + +| Route | Method | Purpose | +| --- | --- | --- | +| `/sourcegraph/v1/code-sources` | GET | Sources in display order: `id/name/host/username/anonymous/editable/locked`; public, without tokens or variable names | +| `/sourcegraph/api/admin/settings/code_source` | PATCH | Administrators edit the custom Git source with `{revision, values:{name, host, username}}` | +| `/sourcegraph/auth/code-credentials` | GET / PUT | Your personal tokens | +| `/sourcegraph/api/admin/settings/code-credentials` | GET / PUT | Global tokens, administrators only | + +Token routes need a browser session, and writes are CSRF-protected; API tokens cannot manage them. GET returns `configured`, `expires_at`, `revision`, `state` and `source` per source, never the token. PUT updates one source with `{kind, revision, token?, expires_at, clear?}`: an omitted `token` keeps the current one, while an omitted or null `expires_at` clears the expiry; a stale `revision` returns `REVISION_CONFLICT`. + +Tokens are stored encrypted with AES-256-GCM; see [deployment](deployment.md) for the encryption key and backups. diff --git a/docs/interfaces.md b/docs/interfaces.md index 67af621..2701c27 100644 --- a/docs/interfaces.md +++ b/docs/interfaces.md @@ -52,28 +52,7 @@ Repository registration, maintenance, statistics and site settings live under `/ ### Code sources -Every instance has exactly two code sources, listed GitHub first: - -| id | Default host | Token variable | Anonymous | Editable on the website | -| --- | --- | --- | --- | --- | -| `github` | `https://github.com` | `SOURCEGRAPH_GITHUB_TOKEN` | yes | no | -| `git` | none | `SOURCEGRAPH_CUSTOM_GIT_TOKEN` | no | yes | - -`github` still reads the earlier variable `SOURCEGRAPH_GIT_TOKEN` while `SOURCEGRAPH_GITHUB_TOKEN` is unset (`legacy_token_env` in the release defaults); a startup file that sets `token_env` stops that fallback. Environment tokens apply only until a global token is saved. - -On the website, **System settings → Code sources** shows one card per source: the custom Git card holds its name, host and HTTPS username, and every card holds the global token and its optional expiry. **User settings → Code source tokens** holds personal tokens, which are tried first. - -Settings apply in this order, later non-empty fields winning: release defaults in `config/deployment.json`, the website's custom Git settings (`PATCH /sourcegraph/api/admin/settings/code_source` with `{revision, values:{name, host, username}}`), then the startup file named by `SOURCEGRAPH_CONFIG_FILE`: - -```json -{"code_sources": {"git": {"name": "Gitea", "host": "https://git.example.com", "username": "oauth2", "token_env": "SOURCEGRAPH_CUSTOM_GIT_TOKEN"}}} -``` - -- The file can only override existing ids. Hosts must be HTTPS origins without a path. `token_env` names an environment variable; never put a token in the file. -- Fields set by the file are read-only on the website (`409 CODE_SOURCE_LOCKED`). -- Registration pins the source host to the repository as `code_host`; later configuration changes never move existing repositories. Startup backfills a missing `code_host` and only logs when it differs from the current configuration. While any repository record uses the custom source, changing its host returns `409 CODE_SOURCE_IN_USE`. -- Registering against a source without a host returns `409 CODE_SOURCE_NOT_CONFIGURED`. -- Tokens are tried in this order: the creator's personal token, the global token, the environment token, then anonymous access for GitHub only. +Code sources, their settings and token order are described in [code sources and tokens](code-sources.md). ## CLI diff --git a/internal/codecredentials/manager.go b/internal/codecredentials/manager.go index 91c5042..5b2d6dc 100644 --- a/internal/codecredentials/manager.go +++ b/internal/codecredentials/manager.go @@ -226,42 +226,6 @@ func (m *Manager) Save(ctx context.Context, owner, kind string, in Input) (Statu return status(r, token, time.Now()), nil } -// Move re-encrypts a retired alias token under a visible source that has no token, -// then clears the alias, so the plaintext never leaves the service. -func (m *Manager) Move(ctx context.Context, owner, from, to string, revision int64) (Status, error) { - if !codesources.IsAlias(from) { - return Status{}, contract.Fail("INVALID_CREDENTIAL_MOVE", "Only a retired code source's token can be moved.", 400) - } - if _, ok := codesources.Lookup(to); !ok { - return Status{}, contract.Fail("INVALID_CREDENTIAL_KIND", "Unknown code source.", 400) - } - src, token, err := m.load(ctx, owner, from) - if err != nil { - return Status{}, err - } - if src.Revision != revision { - return Status{}, conflict - } - if token == "" { - return Status{}, contract.Fail("INVALID_CREDENTIAL_MOVE", "The retired code source has no token to move.", 400) - } - dst, existing, err := m.load(ctx, owner, to) - if err != nil { - return Status{}, err - } - if existing != "" { - return Status{}, contract.Fail("CREDENTIAL_TARGET_CONFIGURED", "The target code source already has a token; clear it first to replace it.", 409) - } - moved, err := m.Save(ctx, owner, to, Input{Revision: dst.Revision, Token: &token, Expires: src.Expires}) - if err != nil { - return Status{}, err - } - if _, err = m.Save(ctx, owner, from, Input{Revision: src.Revision, Clear: true}); err != nil { - return Status{}, err - } - return moved, nil -} - // Credential providers are isolated; repository owners take precedence. func (m *Manager) Candidates(ctx context.Context, owner string, provider ...string) ([]Candidate, error) { kind := defaultProvider diff --git a/internal/codecredentials/manager_test.go b/internal/codecredentials/manager_test.go index 53fec1b..708228b 100644 --- a/internal/codecredentials/manager_test.go +++ b/internal/codecredentials/manager_test.go @@ -68,17 +68,3 @@ func TestEnvironmentTokenFollowsSourceVariable(t *testing.T) { t.Fatal("custom git must not reuse the GitHub variables") } } - -func TestOnlyRetiredSourcesMove(t *testing.T) { - ctx := context.Background() - m := testManager(t) - token := "custom_git_token" - saved, err := m.Save(ctx, Global, "git", Input{Token: &token}) - if err != nil { - t.Fatal(err) - } - var ce *contract.Error - if _, err = m.Move(ctx, Global, "git", "github", saved.Revision); !errors.As(err, &ce) || ce.Code != "INVALID_CREDENTIAL_MOVE" { - t.Fatalf("visible source moved: %v", err) - } -} diff --git a/internal/httpserver/code_credentials.go b/internal/httpserver/code_credentials.go index b53737d..3443596 100644 --- a/internal/httpserver/code_credentials.go +++ b/internal/httpserver/code_credentials.go @@ -56,7 +56,6 @@ func codeCredentials(w http.ResponseWriter, r *http.Request, m *codecredentials. Token *string `json:"token,omitempty"` Expires *time.Time `json:"expires_at"` Clear bool `json:"clear"` - MoveTo string `json:"move_to,omitempty"` } raw, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 8192)) decoder := json.NewDecoder(bytes.NewReader(raw)) @@ -65,16 +64,7 @@ func codeCredentials(w http.ResponseWriter, r *http.Request, m *codecredentials. writeError(w, "", 400, "INVALID_CREDENTIAL", "Invalid credential settings.") return } - var out codecredentials.Status - if input.MoveTo != "" { - if input.Token != nil || input.Clear || input.Expires != nil { - writeError(w, "", 400, "INVALID_CREDENTIAL_MOVE", "A move carries only kind, revision and move_to.") - return - } - out, err = m.Move(ctx, owner, input.Kind, input.MoveTo, input.Revision) - } else { - out, err = m.Save(ctx, owner, input.Kind, codecredentials.Input{Revision: input.Revision, Token: input.Token, Expires: input.Expires, Clear: input.Clear}) - } + out, err := m.Save(ctx, owner, input.Kind, codecredentials.Input{Revision: input.Revision, Token: input.Token, Expires: input.Expires, Clear: input.Clear}) if err != nil { credentialError(w, err) return diff --git a/web/assets/code-credentials.js b/web/assets/code-credentials.js index e575c31..67750e7 100644 --- a/web/assets/code-credentials.js +++ b/web/assets/code-credentials.js @@ -50,12 +50,10 @@ +ACTIONS+''; } function retiredCard(kind){ - const v=state[kind],targets=sources.filter(s=>!state[s.id]?.configured); - const options=targets.map(s=>'').join(''); - return '

'+esc(kind)+t(' 令牌（已退役类型）',' token (retired type)')+'

'+t('这个类型已不能登记新仓库，原来登记的仓库已归入对应的代码源。令牌可以原样转存到一个还没有令牌的代码源，或直接清除。','This type can no longer register repositories; existing ones now belong to its replacement source. Move the token unchanged to a source without a token, or clear it.')+'

' + const v=state[kind]; + return '

'+esc(kind)+t(' 令牌（已退役类型）',' token (retired type)')+'

'+t('这个类型已不能登记新仓库，原来登记的仓库已归入对应的代码源，这个令牌不再用于拉取，可以清除。','This type can no longer register repositories and existing ones now belong to its replacement source. The token is no longer used for fetching and can be cleared.')+'

' +'

'+esc(tokenStatus(v))+'

' - +(targets.length?'':'

'+t('所有代码源都已有令牌；如需改用这个令牌，先清除目标代码源的令牌。','Every source already has a token; clear the target source first to use this one.')+'

') - +ACTIONS+''+(targets.length?'':'')+'
'; + +ACTIONS+''; } function render(){ if(!state)return; @@ -74,10 +72,7 @@ } const clear=form.querySelector('[data-clear]'); if(clear)clear.onclick=()=>{if(confirm(t('确认清除这个令牌？之后拉取将改用其他可用令牌或匿名访问。','Clear this token? Later fetches use other available tokens or anonymous access.')))run(()=>API.request(endpoint,{method:'PUT',body:{kind,revision:v.revision,clear:true,expires_at:null}}));}; - if(form.dataset.retired){ - form.onsubmit=e=>{e.preventDefault();run(()=>API.request(endpoint,{method:'PUT',body:{kind,revision:v.revision,move_to:form.elements.target.value}}));}; - return; - } + if(form.dataset.retired)return; const expires=form.elements.expires; expires.value=localExpiry(v.expires_at); expires.addEventListener('click',()=>{try{expires.showPicker?.();}catch(_){/* Native keyboard editing remains available. */}});