diff --git a/AGENTS.md b/AGENTS.md index 434db9e8..128a8380 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,7 +25,7 @@ - `packages/utils` - `@proxy/utils`, the Result helpers shared by every package: `Do`, `requirePresent` and `parseSchema` (`src/parse.ts`), and `httpRequest`, the fetch wrapper that returns a `FetchError` union instead of throwing (`src/fetch.ts`). Also `ApiError`/`ApiErr` and `wrapDb` (`src/api-error.ts`), the error every server-side function returns, and the session cookie's name and parsing (`src/session-cookie.ts`) - `packages/integrations` - `@proxy/integrations`, the catalog the api and the web app share: each integration's actions, collections and fields (`src/catalog.ts`), the agents a login can be made for (`src/agent-providers.ts`), the email providers' servers (`src/email-providers.ts`), and access (`src/access.ts`): per action of a connection, an agent's own setting, else the connection's default, an action counting only with the one it `requires`, and tools (`src/tools.ts`): what an agent can do with a connection, generated from the catalog, one per list, open, write and command of each collection (`emails_list`, `emails_archive`), each with the action it needs and a JSON Schema of its params. A custom MCP server (`mcp` in the catalog, `custom`) has no actions of its own: `integrationFor(connection)` (`src/mcp-tools.ts`) builds its integration from the tools the connection keeps (`McpTool`, as the server lists them), one action per tool named by the tool, its risk from the tool's annotations (low when it only reads, medium when what it changes can be undone, high when it can't, and `unknown`, shown without a label, when it says nothing), and `integrationTools` passes them on as `call` tools with the server's own input schema (`ServerTool`, beside the catalog's `CollectionTool`). The web app adds the icons (`src/lib/integrations.ts`, whose `integrationOf(connection)` is what every page uses) - `packages/db` - `@proxy/db`: Prisma 6 on Postgres. `src/index.ts` is the client; each module (`auth.ts`, `user.ts`, `organization.ts`, `connection.ts`, `info.ts`, `agent.ts`, `audit.ts`) exports functions returning `Result` through `wrapDb`. A connection's `tools` column holds a custom MCP server's tools as JSON, and `name` what the person called it; both null for every other integration. `relationMode = "prisma"`: no foreign keys in the database, so Prisma runs the cascades and every relation column has its own index. Locally the database is Prisma's embedded Postgres (PGlite, `scripts/dev-db.ts`), which needs `connection_limit=1&pgbouncer=true` in `DATABASE_URL` -- `packages/api` - `@proxy/api`: Express on Bun. Routes are `handleXRoute` functions mounted in `src/server/create-api-router.ts`; JSON routes return `Result` through `withAuthResult`, which puts the signed-in user on `req.user`. Every environment variable is in `src/utils/env-schema.ts` with its docs, which the docs' Environment variables page is built from; add new ones there. Personal Agent Proxy is one origin everywhere: when `packages/web/dist` exists, as in the Docker image, the api serves the web app's build too (`src/server/serve-web.ts`), and every address the api builds starts from `APP_URL`. Browser-called writes take `requireBrowserOrigin` (CSRF); routes that read a cookie have no CORS. `src/connections/` lists, deletes and sets defaults on an organization's connections, connects a mailbox with an app password after signing in to its IMAP and SMTP servers (`email/`), and connects Granola, Notion and Linear by signing in to their MCP servers (`mcp/`, shared by all three: OAuth with PKCE against the server's authorization server, a client registered for each sign-in so nothing is set up beforehand, the handshake in an encrypted cookie, `mcpConnectRoutes` making both routes on the sign-in flow every OAuth integration shares, `connect-flow.ts`; tokens are refreshed as they run out, one refresh at a time per connection on each server, read again first so ones another call or server saved meanwhile are used rather than refreshed twice, and saved only over the ones they replace, whatever else the credential holds kept beside them, by `connections/token-session.ts`, which Google's connection will share; each tool call is one stateless `tools/call`, `mcp-client.ts`, which also lists a server's tools and, for a server that may keep a session (`handshake`), initializes one around each call as the spec asks). Every request to an MCP server, its metadata or its authorization server goes through `utils/checked-request.ts`, which checks the address as a database's host is checked (`utils/host-check.ts`: nothing private unless `ALLOW_PRIVATE_HOSTS`, and `https` unless the address is private) before each request, since a custom server and its endpoints are wherever it said, and never follows a redirect. A custom MCP server is connected by its address (`custom/`): `POST /api/connections/mcp` with `{name, url, token?, returnTo?}` (the name is the person's, kept through the sign-in in its cookie, since a server says nothing about itself that would do) asks the server for its tools, connects a server that answers at once, with no sign-in or the token typed in as a bearer token (as written when it has its scheme in front, `Basic …`), and for one that turns a request without a token down begins its sign-in there, under the browser's origin check, answering `{kind: 'signIn', url}` with the sign-in page the browser goes to: `mcp-discovery.ts` finds the authorization server from the server's protected resource metadata (the `WWW-Authenticate` header's `resource_metadata`, else the well-known paths), its endpoints from that server's metadata (else `/authorize`, `/token` and `/register` under it) and the scopes the server's own metadata names, and the sign-in, begun with `signInFlow` (`connect-flow.ts`, which `mcpConnectRoutes` builds on for the servers known in advance), keeps the endpoints in the handshake cookie and in the credential beside the tokens. The connection's account is the address without its scheme (`mcp.example.com/mcp`), connecting the same address again replaces its credential and tools, a new connection's defaults are the tools that say they only read, `POST /api/connections/:id/tools` asks the server for its tools again (replacing the tools drops the defaults and agents' settings for tools the server no longer lists, `updateConnectionTools`), and a tool is run as it is (`custom-mcp.ts`): its params go to the server as given, a tool that fails answers the agent with the server's own text as a `validation_error`, and the run is logged under the collection `tools` by the tool's name, with its params as JSON, cut to 200 characters. Granola's notes and transcripts are on by default, Notion's Read pages and Linear's Read issues alone; a Notion or Linear sign-in is for one workspace, and a Notion one lasts at most 180 days, however often it is refreshed. `src/agents/` makes, revokes and deletes agent logins (the password is sent once, only its argon2id hash is kept) and sets an agent's own settings per action of a connection where they differ from the default. `src/records/` reads and writes a collection's records through the integration's `Connector` (`connectors.ts`): Information's live in the database (`info-connector.ts`), a mailbox's are fetched over IMAP on every request (`email-connector.ts`), which merges the inbox and the folders the server marks as Drafts and Sent into one list (ids start with the folder: `inbox-…`, `drafts-…`, `sent-…`) and runs the commands: flags, moves to the folders the server marks as Archive (Gmail's All Mail) or Trash, and sending over SMTP, a draft or a new email, filing a copy in Sent where the server doesn't (Gmail does). Received emails are never edited or deleted for good; only drafts are. Granola's notes are fetched from its MCP server on every request (`granola-connector.ts`; the reading of the XML-like text Granola answers in is in `connections/granola/granola-mcp.ts`): its Notes and Transcripts collections are the same meetings by Granola's meeting id, and since Granola's tools have no pages, a page is 30 days of meetings; one that finds none skips back to the newest meeting before it, so the list ends where the meetings do. Granola's dates and attendees are shown as Granola writes them. Notion's workspace is fetched from its MCP server the same way (`notion-connector.ts`, the reading of its answers in `connections/notion/notion-mcp.ts`), as the person who signed in, as a tree in one nested collection: Notion has no list of every page, so the top is their Recents and a search (Notion's own, AI search where the workspace has it) finds the rest, one page of 50; inside a page are the pages and databases in its content, inside a database its saved views (`view-…` ids), or the rows of its only one, and inside a view its rows, paged with Notion's cursor (view queries have no plan quota; Notion's row and SQL queries do). A page reads as Notion-flavored Markdown, a row's properties beside it as "Name: value" lines (`notion-properties.ts`), and only a page's, database's or view's id is ever sent to Notion's fetch; a new page is a private draft at the top, or goes inside the page, database or view it is made in, a row's title in the database's title property; an edit sends the title, the properties written differently and the content, each only where it changed, and pages are never deleted. Notion's MCP server has no tool that deletes or trashes a page. Linear's issues are fetched from its MCP server the same way (`linear-connector.ts`, the reading of its answers in `connections/linear/linear-mcp.ts`), as the person who signed in, by their identifier (`ENG-123`): a list is the issues updated last first, 50 to a page with Linear's cursor, narrowed by the type of status (`?statusType=Started`) and searched in titles and descriptions, and an issue opens with its comments, oldest first. Teams, statuses, people, projects and labels are written by name, as Linear shows them and finds them, its refusals passed on to the agent; an edit sends only the fields written differently, an empty one clearing it. Linear's MCP server has no tool that archives or deletes an issue. Google is connected by signing in to Google with the app's own Google client, the one Google sign-in uses (`GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, a second redirect URI `/api/connections/google/callback`), asking for every scope at once, Gmail, Drive, Docs, Sheets and Slides, with offline access (`connections/google/`: `google-oauth.ts` the sign-in and refresh, with the client's secret, keeping the refresh token Google doesn't send again; `google-client.ts` the calls with a bearer token and Google's errors, a disabled API passing Google's own message on, which names the API and where to enable it); the credential holds the tokens, the scopes Google granted, since the consent screen lets a person untick some, and the address; the connection keeps Google's id for the account (`Connection.providerAccountId`, its `sub`), so signing in again renews the same connection, and its address, even after the address changed (`findAccountConnection`, by address for one without an id and for the MCP integrations). Its Drive is one nested collection, `files` (`records/google-files.ts`, behind `google-connector.ts`, which dispatches on the collection), fetched live from Drive's and Sheets' APIs (`drive-api.ts`, `sheets-api.ts`): at the top four places, My Drive, Shared with me, Shared drives (`drive-…` ids) and Recent, inside them and inside folders their files, folders first, 50 to a page with Drive's page token, and a search is Drive's full-text one, anywhere or inside the folder opened; the trail is walked up a file's parents to My Drive, its shared drive, or, where the account can't see further up, Shared with me. A document opens as Markdown and a presentation as text, as Drive exports them, up to its 10 MB export cap, beyond which opening it says so; a spreadsheet opens into its tabs (`sheet--`) and a tab into its rows (`row---`), 50 to a page, the page token the next row's number, each row's values "Header: value" lines under the tab's first row, a column without a header going by its letter (`Column C`); a page past the tab's last row is empty and a row past it not found without asking Sheets, which refuses a range outside the grid. Drive isn't written to yet; writes, and Gmail, come later. PostgreSQL and MySQL (MariaDB too) are two integrations on one code path: a database is connected with its host, port, database, user, password and encryption (`connections/sql/`: `route.ts` makes both routes per dialect, `sql-client.ts` opens one connection per request with `Bun.sql`, to the address `utils/host-check.ts` resolves the host to, refusing private addresses unless `ALLOW_PRIVATE_HOSTS` is set, and `sql-statement.ts` checks an agent's query is one statement that reads), and `records/sql-connector.ts` reads and writes it through `records/sql-dialect.ts`, where the two differ: the tables are one nested collection, a table holding its rows, a row read and written as "column: value" lines and found again by its primary key (ids `public.users:42`, `pairs:3,4`; a table without a key is never written to), and a query, a second collection that is only ever run, goes through a read-only transaction with a time limit, a cursor on Postgres and `sql_select_limit` on MySQL, 200 rows at most, answered as JSON lines. MySQL's read-only transaction doesn't stop DDL, so the statement check is what keeps a query to reading; the docs tell people to make a read-only database user as well. Lists take `?search=` (only where the collection has a `searchHint`; Granola has none), `?filter=` (the collection's `filterField`, such as `?filter=Draft`), `?parent=` (only in a `nested` collection: the list inside a record, answered with its `trail`, the records above it; a record made there goes in it) and `?page=` (`list-query.ts`) and answer a page, newest first (a mailbox's 50 emails, Granola's 30 days), with the next page's token; a mailbox's token holds where each folder goes on, so paging through the merged list shows every email once, and its search uses Gmail's own syntax on Gmail (`X-GM-EXT-1`) and IMAP text search elsewhere. Agents sign in at `/agent-auth/login` with a cookie of their own (`src/agent-auth/`); `src/agent-side/` serves them: the connections an agent can use, the tools it has with each, and `runAgentTool` (`tools.ts`), the one place a tool runs, which checks it against the agent's access as it is now, validates its params against the tool's schema (an edit changes only the fields given), calls the connector, or a custom server's tool as it is, answering `{text}`, and logs it, denied ones too, to the activity log that `src/activity/` lists. Over HTTP that is `/api/agent/me`, `GET /api/agent/connections/:id/tools` and `POST /api/agent/connections/:id/tools/:tool` with `{params}`; `src/mcp/` serves the same over MCP at `/mcp` (Streamable HTTP without sessions, JSON answers, no stream): four tools whatever the login may do, `list_connections` (the entry point), `list_tools`, `read` (only tools that read, marked read-only) and `run`, and server instructions naming the agent's connections. MCP clients sign in with OAuth 2.1 (`src/oauth/`), Personal Agent Proxy being its own authorization server: metadata under `/.well-known/`, Dynamic Client Registration at `/oauth/register` (open to anyone, each registration first deleting the clients that registered over a day ago and never finished a sign-in), `/oauth/authorize` sealing the request into the web app's `/connect-agent` page, where the person chooses the agent login the client works as or makes one, and `/oauth/token` with PKCE (S256 only) and refresh tokens replaced on every use; codes and tokens are kept as their SHA-256, and revoking an agent ends its clients' tokens. These endpoints take any origin (CORS), since none reads a cookie; credentials and Information records are stored encrypted with `ENCRYPTION_KEY` (`src/utils/secret-crypto.ts`). Has its own `.prettierrc.json` with a 200-column print width +- `packages/api` - `@proxy/api`: Express on Bun. Routes are `handleXRoute` functions mounted in `src/server/create-api-router.ts`; JSON routes return `Result` through `withAuthResult`, which puts the signed-in user on `req.user`. Every environment variable is in `src/utils/env-schema.ts` with its docs, which the docs' Environment variables page is built from; add new ones there. Personal Agent Proxy is one origin everywhere: when `packages/web/dist` exists, as in the Docker image, the api serves the web app's build too (`src/server/serve-web.ts`), and every address the api builds starts from `APP_URL`. Browser-called writes take `requireBrowserOrigin` (CSRF); routes that read a cookie have no CORS. `src/connections/` lists, deletes and sets defaults on an organization's connections, connects a mailbox with an app password after signing in to its IMAP and SMTP servers (`email/`), and connects Granola, Notion and Linear by signing in to their MCP servers (`mcp/`, shared by all three: OAuth with PKCE against the server's authorization server, a client registered for each sign-in so nothing is set up beforehand, the handshake in an encrypted cookie, `mcpConnectRoutes` making both routes on the sign-in flow every OAuth integration shares, `connect-flow.ts`; tokens are refreshed as they run out, one refresh at a time per connection on each server, read again first so ones another call or server saved meanwhile are used rather than refreshed twice, and saved only over the ones they replace, whatever else the credential holds kept beside them, by `connections/token-session.ts`, which Google's connection will share; each tool call is one stateless `tools/call`, `mcp-client.ts`, which also lists a server's tools and, for a server that may keep a session (`handshake`), initializes one around each call as the spec asks). Every request to an MCP server, its metadata or its authorization server goes through `utils/checked-request.ts`, which checks the address as a database's host is checked (`utils/host-check.ts`: nothing private unless `ALLOW_PRIVATE_HOSTS`, and `https` unless the address is private) before each request, since a custom server and its endpoints are wherever it said, and never follows a redirect. A custom MCP server is connected by its address (`custom/`): `POST /api/connections/mcp` with `{name, url, token?, returnTo?}` (the name is the person's, kept through the sign-in in its cookie, since a server says nothing about itself that would do) asks the server for its tools, connects a server that answers at once, with no sign-in or the token typed in as a bearer token (as written when it has its scheme in front, `Basic …`), and for one that turns a request without a token down begins its sign-in there, under the browser's origin check, answering `{kind: 'signIn', url}` with the sign-in page the browser goes to: `mcp-discovery.ts` finds the authorization server from the server's protected resource metadata (the `WWW-Authenticate` header's `resource_metadata`, else the well-known paths), its endpoints from that server's metadata (else `/authorize`, `/token` and `/register` under it) and the scopes the server's own metadata names, and the sign-in, begun with `signInFlow` (`connect-flow.ts`, which `mcpConnectRoutes` builds on for the servers known in advance), keeps the endpoints in the handshake cookie and in the credential beside the tokens. The connection's account is the address without its scheme (`mcp.example.com/mcp`), connecting the same address again replaces its credential and tools, a new connection's defaults are the tools that say they only read, `POST /api/connections/:id/tools` asks the server for its tools again (replacing the tools drops the defaults and agents' settings for tools the server no longer lists, `updateConnectionTools`), and a tool is run as it is (`custom-mcp.ts`): its params go to the server as given, a tool that fails answers the agent with the server's own text as a `validation_error`, and the run is logged under the collection `tools` by the tool's name, with its params as JSON, cut to 200 characters. Granola's notes and transcripts are on by default, Notion's Read pages and Linear's Read issues alone; a Notion or Linear sign-in is for one workspace, and a Notion one lasts at most 180 days, however often it is refreshed. `src/agents/` makes, revokes and deletes agent logins (the password is sent once, only its argon2id hash is kept) and sets an agent's own settings per action of a connection where they differ from the default. `src/records/` reads and writes a collection's records through the integration's `Connector` (`connectors.ts`): Information's live in the database (`info-connector.ts`), a mailbox's are fetched over IMAP on every request (`email-connector.ts`), which merges the inbox and the folders the server marks as Drafts and Sent into one list (ids start with the folder: `inbox-…`, `drafts-…`, `sent-…`) and runs the commands: flags, moves to the folders the server marks as Archive (Gmail's All Mail) or Trash, and sending over SMTP, a draft or a new email, filing a copy in Sent where the server doesn't (Gmail does). Received emails are never edited or deleted for good; only drafts are. Granola's notes are fetched from its MCP server on every request (`granola-connector.ts`; the reading of the XML-like text Granola answers in is in `connections/granola/granola-mcp.ts`): its Notes and Transcripts collections are the same meetings by Granola's meeting id, and since Granola's tools have no pages, a page is 30 days of meetings; one that finds none skips back to the newest meeting before it, so the list ends where the meetings do. Granola's dates and attendees are shown as Granola writes them. Notion's workspace is fetched from its MCP server the same way (`notion-connector.ts`, the reading of its answers in `connections/notion/notion-mcp.ts`), as the person who signed in, as a tree in one nested collection: Notion has no list of every page, so the top is their Recents and a search (Notion's own, AI search where the workspace has it) finds the rest, one page of 50; inside a page are the pages and databases in its content, inside a database its saved views (`view-…` ids), or the rows of its only one, and inside a view its rows, paged with Notion's cursor (view queries have no plan quota; Notion's row and SQL queries do). A page reads as Notion-flavored Markdown, a row's properties beside it as "Name: value" lines (`notion-properties.ts`), and only a page's, database's or view's id is ever sent to Notion's fetch; a new page is a private draft at the top, or goes inside the page, database or view it is made in, a row's title in the database's title property; an edit sends the title, the properties written differently and the content, each only where it changed, and pages are never deleted. Notion's MCP server has no tool that deletes or trashes a page. Linear's issues are fetched from its MCP server the same way (`linear-connector.ts`, the reading of its answers in `connections/linear/linear-mcp.ts`), as the person who signed in, by their identifier (`ENG-123`): a list is the issues updated last first, 50 to a page with Linear's cursor, narrowed by the type of status (`?statusType=Started`) and searched in titles and descriptions, and an issue opens with its comments, oldest first. Teams, statuses, people, projects and labels are written by name, as Linear shows them and finds them, its refusals passed on to the agent; an edit sends only the fields written differently, an empty one clearing it. Linear's MCP server has no tool that archives or deletes an issue. Google is connected by signing in to Google with the app's own Google client, the one Google sign-in uses (`GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, a second redirect URI `/api/connections/google/callback`), asking for every scope at once, Gmail, Drive, Docs, Sheets and Slides, with offline access (`connections/google/`: `google-oauth.ts` the sign-in and refresh, with the client's secret, keeping the refresh token Google doesn't send again; `google-client.ts` the calls with a bearer token and Google's errors, a disabled API passing Google's own message on, which names the API and where to enable it); the credential holds the tokens, the scopes Google granted, since the consent screen lets a person untick some, and the address; the connection keeps Google's id for the account (`Connection.providerAccountId`, its `sub`), so signing in again renews the same connection, and its address, even after the address changed (`findAccountConnection`, by address for one without an id and for the MCP integrations). Each action needs one of the scopes allowing it (`google/scopes.ts`), so an action whose scope wasn't granted is unavailable: `connections/unavailable.ts`, the one place the rest of the api asks, reads it from the credential on every request (`unavailable` on a connection; one that can't be read leaves every action unavailable rather than failing the list), no default or agent setting can turn it on, `runAgentTool` refuses it saying to sign in to Google again, and the connection page shows its checkbox disabled with a "Sign in again" link (`?account=` opening Google on the account with `login_hint`). An integration's `disclosure`, Google's Limited Use line, is shown above a connection's permissions. Disconnecting a Google connection revokes its grant at Google first (`connections/disconnect.ts`), unless another connection in any organization is for the same account, since a grant covers all of the client's tokens for it, and deletes it even when Google can't be reached. Its Drive is one nested collection, `files` (`records/google-files.ts`, behind `google-connector.ts`, which dispatches on the collection), fetched live from Drive's and Sheets' APIs (`drive-api.ts`, `sheets-api.ts`): at the top four places, My Drive, Shared with me, Shared drives (`drive-…` ids) and Recent, inside them and inside folders their files, folders first, 50 to a page with Drive's page token, and a search is Drive's full-text one, anywhere or inside the folder opened; the trail is walked up a file's parents to My Drive, its shared drive, or, where the account can't see further up, Shared with me. A document opens as Markdown and a presentation as text, as Drive exports them, up to its 10 MB export cap, beyond which opening it says so; a spreadsheet opens into its tabs (`sheet--`) and a tab into its rows (`row---`), 50 to a page, the page token the next row's number, each row's values "Header: value" lines under the tab's first row, a column without a header going by its letter (`Column C`); a page past the tab's last row is empty and a row past it not found without asking Sheets, which refuses a range outside the grid. Drive isn't written to yet; writes, and Gmail, come later. PostgreSQL and MySQL (MariaDB too) are two integrations on one code path: a database is connected with its host, port, database, user, password and encryption (`connections/sql/`: `route.ts` makes both routes per dialect, `sql-client.ts` opens one connection per request with `Bun.sql`, to the address `utils/host-check.ts` resolves the host to, refusing private addresses unless `ALLOW_PRIVATE_HOSTS` is set, and `sql-statement.ts` checks an agent's query is one statement that reads), and `records/sql-connector.ts` reads and writes it through `records/sql-dialect.ts`, where the two differ: the tables are one nested collection, a table holding its rows, a row read and written as "column: value" lines and found again by its primary key (ids `public.users:42`, `pairs:3,4`; a table without a key is never written to), and a query, a second collection that is only ever run, goes through a read-only transaction with a time limit, a cursor on Postgres and `sql_select_limit` on MySQL, 200 rows at most, answered as JSON lines. MySQL's read-only transaction doesn't stop DDL, so the statement check is what keeps a query to reading; the docs tell people to make a read-only database user as well. Lists take `?search=` (only where the collection has a `searchHint`; Granola has none), `?filter=` (the collection's `filterField`, such as `?filter=Draft`), `?parent=` (only in a `nested` collection: the list inside a record, answered with its `trail`, the records above it; a record made there goes in it) and `?page=` (`list-query.ts`) and answer a page, newest first (a mailbox's 50 emails, Granola's 30 days), with the next page's token; a mailbox's token holds where each folder goes on, so paging through the merged list shows every email once, and its search uses Gmail's own syntax on Gmail (`X-GM-EXT-1`) and IMAP text search elsewhere. Agents sign in at `/agent-auth/login` with a cookie of their own (`src/agent-auth/`); `src/agent-side/` serves them: the connections an agent can use, the tools it has with each, and `runAgentTool` (`tools.ts`), the one place a tool runs, which checks it against the agent's access as it is now, validates its params against the tool's schema (an edit changes only the fields given), calls the connector, or a custom server's tool as it is, answering `{text}`, and logs it, denied ones too, to the activity log that `src/activity/` lists. Over HTTP that is `/api/agent/me`, `GET /api/agent/connections/:id/tools` and `POST /api/agent/connections/:id/tools/:tool` with `{params}`; `src/mcp/` serves the same over MCP at `/mcp` (Streamable HTTP without sessions, JSON answers, no stream): four tools whatever the login may do, `list_connections` (the entry point), `list_tools`, `read` (only tools that read, marked read-only) and `run`, and server instructions naming the agent's connections. MCP clients sign in with OAuth 2.1 (`src/oauth/`), Personal Agent Proxy being its own authorization server: metadata under `/.well-known/`, Dynamic Client Registration at `/oauth/register` (open to anyone, each registration first deleting the clients that registered over a day ago and never finished a sign-in), `/oauth/authorize` sealing the request into the web app's `/connect-agent` page, where the person chooses the agent login the client works as or makes one, and `/oauth/token` with PKCE (S256 only) and refresh tokens replaced on every use; codes and tokens are kept as their SHA-256, and revoking an agent ends its clients' tokens. These endpoints take any origin (CORS), since none reads a cookie; credentials and Information records are stored encrypted with `ENCRYPTION_KEY` (`src/utils/secret-crypto.ts`). Has its own `.prettierrc.json` with a 200-column print width - `skills/` - Agent skills, symlinked into `.claude/skills` and `.codex/skills` by `bun run setup` ## The app @@ -33,7 +33,7 @@ Personal Agent Proxy holds the integrations and personal information an AI agent needs, so every agent company does not have to rebuild them and you do not have to trust each one's security. A human connects services and hands out agent logins; an agent signs in with one and works through what it was given. Everything is served by `packages/api`; the web app has no data of its own. - **Model**: an integration from the catalog (`@proxy/integrations`: Email over IMAP and SMTP, Google's Drive, Docs, Sheets and Slides over Google's APIs, one sign-in for a Google account, Granola's meeting notes, Notion's pages and Linear's issues over their MCP servers, PostgreSQL and MySQL over their own protocols) has the actions an agent can be allowed with a connection of it (Read, Archive, Write drafts, Send) and collections, each a list of fields with the actions reading and writing it need, and the commands it offers beyond editing (Mark as read, Archive, Send), each needing one action. A command runs on a `record`, where its `where` picks out which (Archive only in the inbox, Send only on a draft), or on values typed in (`new`: sending a new email). A mailbox is one collection, its folder a field the agent side filters by. A `nested` collection is a tree, as Notion's pages are: a record can hold others of the collection (`hasChildren`), listed inside it with the list tool's `parent` and made inside it with the create tool's, and access is the same anywhere in it. A collection without a `read` action is never listed or opened, only run, as a database's queries are, and a `new` command marked `readOnly` counts as a read for MCP clients. Integrations join the catalog once they work end to end. Any other service with an MCP server is connected as a custom MCP server: the server's tools are the actions, each allowed on its own, and there are no collections; an agent runs a tool with the params the server describes and gets back what it answered. A connection is a connected integration and its account, owned by the organization; an integration can be connected more than once, such as two mailboxes. Information is the built-in `info` connection (addresses, payment cards, notes), made with the organization, so access and the agent side treat it like any other. Nothing is copied from a provider: every request goes to it live. Fields marked `system` are set by the provider, never typed in. -- **Access** is one list of actions per connection, and every action the catalog lists can be set: what Personal Agent Proxy can do with the provider is the limit. Two layers: the connection's default (a `ConnectionDefault` row per action that is on) and an agent's own setting, on or off, where it differs (`AgentGrant`). An agent gets its own setting, else the default as it is now; an action that `requires` another counts only with it (Archive needs Read; Send needs nothing, so an agent can send without reading the mailbox). A collection's `writes` name the action creating, editing and deleting need, and `editable` which records they apply to (drafts). An agent login is for one agent from `AGENT_PROVIDERS`; each organization can have one login per company, named by it, and any number of custom MCP agents (`custom`, for any MCP client not in the list), each named by the person and told apart by their names (`Agent` is unique by organization, provider and name). It has a generated username and a password that is shown once. Every request on the agent side is an audit entry, allowed or denied, with what a list searched for. +- **Access** is one list of actions per connection, and every action the catalog lists can be set: what Personal Agent Proxy can do with the provider is the limit. Two layers: the connection's default (a `ConnectionDefault` row per action that is on) and an agent's own setting, on or off, where it differs (`AgentGrant`). An agent gets its own setting, else the default as it is now; an action that `requires` another counts only with it (Archive needs Read; Send needs nothing, so an agent can send without reading the mailbox), and an action a sign-in didn't cover, as Google's consent screen allows, is unavailable whatever is ticked. A collection's `writes` name the action creating, editing and deleting need, and `editable` which records they apply to (drafts). An agent login is for one agent from `AGENT_PROVIDERS`; each organization can have one login per company, named by it, and any number of custom MCP agents (`custom`, for any MCP client not in the list), each named by the person and told apart by their names (`Agent` is unique by organization, provider and name). It has a generated username and a password that is shown once. Every request on the agent side is an audit entry, allowed or denied, with what a list searched for. - **Accounts**: a human signs in with Google or a magic link (`src/auth/` in the api); the sign-in page says that continuing accepts the terms and the privacy policy, on the website, and a sign-up records it (`User.termsAcceptedAt`); accounts from before the notice have none until they are asked. A session is a `Session` row and an httpOnly cookie; the web app's `/api/me` loader on the human side's route redirects to `/login?callbackUrl=…` without one. Every user gets one `Organization` at sign-up, created with the user. `ALLOWED_SIGNUP_EMAILS` limits who can sign up; the login page asks `/auth/methods` whether to offer Google. In development the login page has a Create dev user button; in development, or without `RESEND_KEY`, the api prints magic links instead of emailing them. Cookies are secure when `APP_URL` is HTTPS. A new organization first goes through the welcome flow (`/welcome`, outside the human side's layout; the human side redirects there until `Organization.onboardedAt` is set, which `POST /api/me/onboarded` does when the flow is finished or skipped): step one is `AgentSetup` (`src/components/agent-setup.tsx`, which New agent and setting an agent up again use too): it picks the agent from `AGENT_PROVIDERS`, the choice in the address (`?agent=claude`) so the browser's back button returns to it, and does what its `connects` says, an `mcp` agent (Claude, Claude Code, Claude Desktop, Hermes Agent, ChatGPT, Poke) shown the server address and that agent's steps (Claude Code and Hermes Agent use terminal commands) while the page polls for the login its sign-in makes, and over http, as in development, only a `local` agent (Claude Code, Claude Desktop, Hermes Agent) can be picked, the rest greyed out with a pointer to self-hosting on a server with a domain, a `login` agent given a login at once, as a message to paste to it with the sign-in page, username and password, or, for a `vault` agent (Instinct), which won't take a password in chat, as three steps: a message asking it for a link to its vault, the login to fill in on the page it sends, and a message telling it to sign in; step two (`/welcome/connections`) is the catalog, a plus per integration with how many accounts are connected: a mailbox goes to `/welcome/connections/email`, the form on a page with Continue once signed in, and a sign-in started with `?returnTo=welcome` comes back to `/welcome/connections/:id`, the new connection's permissions (its defaults, saved as they are ticked) with Confirm back to the catalog; a database goes to `/welcome/connections/postgres` or `/welcome/connections/mysql` the same way as a mailbox. Organizations from before the flow were marked done by its migration. - **Human side** behind `/login`: `/connections` (a right click on a row renames the connection, `PUT /api/connections/:id/name`, an empty name being none of its own, which a custom MCP server can't have, or deletes it, each behind a dialog), `/connections/new` (the catalog, a plus per integration with how many accounts are connected: `/connections/new/email` connects a mailbox with an app password, with a Test connection that signs in and saves nothing, `/connections/new/postgres` and `/connections/new/mysql` a database the same way, `/connections/new/mcp` a custom MCP server under a name of the person's, by its address, with an access token where the server takes one instead of a sign-in (`ConnectMcpForm`, which sends the browser to the server's sign-in when the api says it asks for one), and Google, Granola, Notion and Linear are connected by signing in to them; `ConnectEmailForm`, `ConnectDatabaseForm`, `ConnectMcpForm` and `IntegrationCatalog` serve the welcome flow too, at `/welcome/connections/`), `/connections/:id` (titled by the name the person gave it where they did, as the list and the agent side are, the defaults under Permissions, with how many agents differ per action, a custom MCP server's tools being its actions, with a button that asks the server for them again, and a "Connection added successfully" when `?added` says it was just connected), `/info`, `/agents`, `/agents/new` (the welcome flow's agent step, among the companies without a login yet and Custom MCP agent, on to the agent's page), `/agents/:id/setup` (the same for one agent, to hand it a new password or add the connector again; the only place a password shows, once), `/agents/:id` (what the agent can reach and did, the MCP clients working as it when there are any; access laid out like an editor's settings: one checkbox per action of each connection, the only control for it, each connection folded by default to a row saying Default or Changed; a setting of the agent's own is marked with a bar, a button that makes it the connection's default for every agent, and one that resets it, and a filter and Changed only narrow the list; `ConnectionAccess` serves both pages), `/activity` (filters in the URL, so the agent and connection pages link to their part of the log; a custom server's tool reads "Ran Search docs with {…}"). Each page has a loader in `src/loaders.ts` and revalidates after a change. Layout follows the spanish repo: the shadcn sidebar as a rail, an `h-12` header, one `max-w-3xl` column, one-line `RowList` rows. Type has one size, `text-sm`, in rows, summaries and hints alike, except column headers (`RowHeader`), which are `text-xs`: the thing itself in the normal colour, what is said about it (column headers, times, summaries, descriptions) in `text-muted-foreground`, and `font-medium` only for page and section titles. - **MCP**: an agent that speaks MCP (Poke, Claude, Claude Code, Claude Desktop, Hermes Agent, ChatGPT, or any other as a Custom MCP agent) adds `/mcp` as a custom connector and signs in; the person picks which agent login it works as on `/connect-agent`, or makes one there: a client whose name points at no company's agent is offered a new custom agent named after it, which the person can rename. Setting up a custom agent waits for one that wasn't there before to sign in, since there can be several. The agent page shows the server's address and the MCP clients working as the login, each with a Disconnect; the activity log marks requests that came in over MCP (`AuditEntry.via`). diff --git a/packages/api/src/agent-side/tools.test.ts b/packages/api/src/agent-side/tools.test.ts index 387ba07c..873058be 100644 --- a/packages/api/src/agent-side/tools.test.ts +++ b/packages/api/src/agent-side/tools.test.ts @@ -1,7 +1,7 @@ import {beforeEach, describe, expect, mock, test} from 'bun:test'; -import {Err, Ok} from 'ts-results-es'; +import {Err, Ok, type Result} from 'ts-results-es'; -import {ApiErr} from '@proxy/utils'; +import {ApiErr, type ApiError} from '@proxy/utils'; import {findIntegration, integrationFor, type Integration} from '@proxy/integrations'; const getAgent = mock(); @@ -12,10 +12,15 @@ const logAgentRequest = mock(); mock.module('@proxy/db/audit', () => ({logAgentRequest})); -const listConnections = mock(); -const updateConnectionCredential = mock(); +const listConnectionsWithCredential = mock(); -mock.module('@proxy/db/connection', () => ({listConnections, updateConnectionCredential})); +mock.module('@proxy/db/connection', () => ({listConnectionsWithCredential, updateConnectionCredential: mock(), getConnectionCredential: mock(async () => Ok(null))})); + +// Readable stand-ins for encryption, so a test can write the credential it means. +mock.module('../utils/secret-crypto', () => ({ + encryptSecret: (plaintext: string): Result => Ok(`enc:${plaintext}`), + decryptSecret: (stored: string): Result => Ok(stored.replace(/^enc:/, '')), +})); const loadConnection = mock(); @@ -49,6 +54,16 @@ const notionConnection = { credential: 'v1.x.y.z', }; +// Read files is on by default, but the person unticked Drive on Google's consent screen. +const googleConnection = { + id: 'google-1', + integrationId: 'google', + account: 'alex@example.com', + createdAt: new Date(), + defaults: [{actionId: 'readFiles'}], + credential: `enc:${JSON.stringify({accessToken: 'access-1', refreshToken: 'refresh-1', expiresAt: null, scopes: ['openid', 'https://www.googleapis.com/auth/userinfo.email'], email: 'alex@example.com'})}`, +}; + // Reads cards of its own, and doesn't read the mailbox. const agent = { id: 'agent-1', @@ -75,7 +90,7 @@ const send = mock(); const sendNew = mock(); const connector = {list: mock(), get: mock(), create: mock(), update: mock(), remove: mock(), commands: {archive, send}, newCommands: {sendNew}}; -function useConnection(connection: typeof infoConnection | typeof emailConnection | typeof notionConnection) { +function useConnection(connection: typeof infoConnection | typeof emailConnection | typeof notionConnection | typeof googleConnection) { loadConnection.mockResolvedValue(Ok({connection, integration: integration(connection.integrationId), connector})); } @@ -83,7 +98,7 @@ beforeEach(() => { mock.clearAllMocks(); getAgent.mockResolvedValue(Ok(agent)); logAgentRequest.mockResolvedValue(Ok(undefined)); - listConnections.mockResolvedValue(Ok([infoConnection, emailConnection])); + listConnectionsWithCredential.mockResolvedValue(Ok([infoConnection, emailConnection, googleConnection])); connector.list.mockResolvedValue(Ok({records: [card], nextPage: null})); connector.get.mockResolvedValue(Ok(card)); useConnection(infoConnection); @@ -101,6 +116,47 @@ describe('listAgentConnections', () => { }); }); +describe("actions the connection's sign-in didn't cover", () => { + test('are not among what the agent can do, whatever is ticked', async () => { + getAgent.mockResolvedValue(Ok({...agent, grants: [{connectionId: 'google-1', actionId: 'readFiles', allowed: true}]})); + + const {listAgentConnections} = await import('./tools'); + const result = await listAgentConnections(signedIn); + + expect(result.unwrap().connections.map((connection) => connection.id)).toEqual(['info-1', 'mail-1']); + }); + + test('offer no tools', async () => { + useConnection(googleConnection); + + const {listAgentTools} = await import('./tools'); + const result = await listAgentTools(signedIn, 'google-1'); + + expect(result.unwrap()).toEqual([]); + }); + + test('running one is refused, saying how to allow it, and logged as denied', async () => { + useConnection(googleConnection); + + const {runAgentTool} = await import('./tools'); + const result = await runAgentTool(caller, 'google-1', 'files_list', {search: 'budget'}); + + expect(result.unwrapErr()).toMatchObject({kind: 'validation_error', message: "Google didn't allow Read files at sign-in. Sign in to Google again to allow it."}); + expect(connector.list).not.toHaveBeenCalled(); + expect(logAgentRequest.mock.calls[0]?.[0]).toMatchObject({connectionId: 'google-1', collectionId: 'files', action: 'list', query: 'budget', outcome: 'denied'}); + }); + + test("one the agent's own setting turns off anyway is plainly forbidden", async () => { + getAgent.mockResolvedValue(Ok({...agent, grants: [{connectionId: 'google-1', actionId: 'readFiles', allowed: false}]})); + useConnection(googleConnection); + + const {runAgentTool} = await import('./tools'); + const result = await runAgentTool(caller, 'google-1', 'files_list', {}); + + expect(result.unwrapErr().kind).toBe('forbidden'); + }); +}); + describe('listAgentTools', () => { test("the tools of the connection's actions the agent has", async () => { useConnection(emailConnection); @@ -351,7 +407,7 @@ describe('custom MCP servers', () => { beforeEach(() => { loadConnection.mockResolvedValue(Ok({connection: customConnection, integration: integrationFor({integrationId: 'mcp', tools: [search, publish]}), connector: null})); - listConnections.mockResolvedValue(Ok([customConnection])); + listConnectionsWithCredential.mockResolvedValue(Ok([customConnection])); callCustomTool.mockResolvedValue(Ok('3 results')); }); diff --git a/packages/api/src/agent-side/tools.ts b/packages/api/src/agent-side/tools.ts index 384d848e..e1bafc66 100644 --- a/packages/api/src/agent-side/tools.ts +++ b/packages/api/src/agent-side/tools.ts @@ -3,7 +3,7 @@ import {Err, Ok, type Result} from 'ts-results-es'; import {ApiErr, type ApiError, Do, requirePresent} from '@proxy/utils'; import {getAgent, type AgentRow, type SignedInAgent} from '@proxy/db/agent'; import {logAgentRequest} from '@proxy/db/audit'; -import {listConnections, type ConnectionRow} from '@proxy/db/connection'; +import {listConnectionsWithCredential, type ConnectionRow} from '@proxy/db/connection'; import { allowedTools, applies, @@ -22,6 +22,7 @@ import { } from '@proxy/integrations'; import {callCustomTool} from '../connections/custom/custom-mcp'; +import {unavailableActions, unavailableError} from '../connections/unavailable'; import type {Connector, DataRecord, RecordPage, RecordTarget, RecordValues} from '../records/connector'; import {parseRecordValues} from '../records/record-values'; import {loadConnection} from '../records/target'; @@ -53,12 +54,14 @@ export type ToolResult = RecordPage | {record: DataRecord | null} | {text: strin // How much of a custom tool's params the activity log keeps. const MAX_LOGGED_PARAMS = 200; -export function actionsOf(agent: AgentRow, connection: ConnectionRow, integration: Integration): string[] { +/** What the agent can do with the connection, none of the `unavailable` actions its sign-in didn't cover among them. */ +export function actionsOf(agent: AgentRow, connection: ConnectionRow, integration: Integration, unavailable: string[]): string[] { const own: OwnSettings = Object.fromEntries(agent.grants.filter((grant) => grant.connectionId === connection.id).map((grant) => [grant.actionId, grant.allowed])); return effectiveActions( integration, connection.defaults.map((stored) => stored.actionId), own, + unavailable, ); } @@ -73,13 +76,13 @@ export function requireAgent(signedIn: SignedInAgent): Promise> { return Do(async ($) => { const agent = $(await requireAgent(signedIn)); - const connections = $(await listConnections(signedIn.orgId)) + const connections = $(await listConnectionsWithCredential(signedIn.orgId)) .flatMap((connection) => { const integration = integrationOf(connection); if (!integration) { return []; } - const actions = actionsOf(agent, connection, integration); + const actions = actionsOf(agent, connection, integration, unavailableActions(connection)); return [ { id: connection.id, @@ -101,7 +104,7 @@ export function listAgentTools(signedIn: SignedInAgent, connectionId: string): P return Do(async ($) => { const agent = $(await requireAgent(signedIn)); const {connection, integration} = $(await loadConnection(signedIn.orgId, connectionId)); - return allowedTools(integration, actionsOf(agent, connection, integration)); + return allowedTools(integration, actionsOf(agent, connection, integration, unavailableActions(connection))); }); } @@ -132,7 +135,8 @@ function log(run: ToolRun, outcome: 'allowed' | 'denied', title: string | null, /** * Runs one of the connection's tools with `params`, as its input schema describes them. A * connection outside the organization, or a tool its integration doesn't have, is not found and - * not logged; a tool whose action the agent doesn't have is refused and logged as denied. + * not logged; a tool whose action the agent doesn't have is refused and logged as denied, saying + * how to allow it where the connection's sign-in didn't cover the action. * `readOnly` limits the run to tools that only read, or only to those that change something, as * the MCP server's `read` and `run` do; any other is refused, unlogged. */ @@ -153,14 +157,25 @@ export function runAgentTool(caller: AgentCaller, connectionId: string, toolName // A list's search is logged, a refused one's too, so its params are read first. const listParams = tool.kind === 'list' ? $(parseToolParams(tool, params)) : null; - if (!actionsOf(agent, connection, integration).includes(tool.action)) { + const unavailable = unavailableActions(connection); + if (!actionsOf(agent, connection, integration, unavailable).includes(tool.action)) { $(await log(run, 'denied', null, searchOf(listParams))); - return $(Err(ApiErr.forbidden())); + return $(Err(refusal(integration, agent, connection, unavailable, tool.action))); } return $(await execute(run, listParams ?? $(parseToolParams(tool, params)))); }); } +// Why an action is refused: the sign-in, naming the action it didn't cover, where signing in again +// would allow it as the agent's settings stand; else plainly forbidden. +function refusal(integration: Integration, agent: AgentRow, connection: ConnectionRow, unavailable: string[], actionId: string): ApiError { + if (!actionsOf(agent, connection, integration, []).includes(actionId)) { + return ApiErr.forbidden(); + } + const required = integration.actions.find((action) => action.id === actionId)?.requires; + return unavailableError(integration, unavailable.includes(actionId) || required === undefined ? actionId : required); +} + function searchOf(params: Record | null): string | null { return params?.search?.trim() || null; } @@ -206,7 +221,7 @@ function runServerTool( outcome, via: caller.via, }); - if (!actionsOf(agent, connection, integration).includes(tool.action)) { + if (!actionsOf(agent, connection, integration, unavailableActions(connection)).includes(tool.action)) { $(await logRun('denied')); return $(Err(ApiErr.forbidden())); } diff --git a/packages/api/src/agents/route.test.ts b/packages/api/src/agents/route.test.ts index e036dde2..3b65315c 100644 --- a/packages/api/src/agents/route.test.ts +++ b/packages/api/src/agents/route.test.ts @@ -1,7 +1,7 @@ import {beforeEach, describe, expect, mock, test} from 'bun:test'; -import {Err, Ok} from 'ts-results-es'; +import {Err, Ok, type Result} from 'ts-results-es'; -import {ApiErr} from '@proxy/utils'; +import {ApiErr, type ApiError} from '@proxy/utils'; const listAgents = mock(); const getAgent = mock(); @@ -14,9 +14,14 @@ const isProviderConflict = mock(() => false); mock.module('@proxy/db/agent', () => ({listAgents, getAgent, createAgent, updateAgent, deleteAgent, setAgentGrants, isUsernameConflict, isProviderConflict})); -const getConnection = mock(); +const getConnectionWithCredential = mock(); -mock.module('@proxy/db/connection', () => ({getConnection})); +mock.module('@proxy/db/connection', () => ({getConnectionWithCredential})); + +// Readable stand-ins for encryption, so a test can write the credential it means. +mock.module('../utils/secret-crypto', () => ({ + decryptSecret: (stored: string): Result => Ok(stored.replace(/^enc:/, '')), +})); const requireUserOrgId = mock(); @@ -44,7 +49,9 @@ beforeEach(() => { createAgent.mockResolvedValue(Ok(agentRow)); updateAgent.mockResolvedValue(Ok(true)); setAgentGrants.mockResolvedValue(Ok(undefined)); - getConnection.mockResolvedValue(Ok({id: 'conn-1', integrationId: 'email', account: 'alex@example.com', createdAt: new Date(), tools: null, name: null, defaults: []})); + getConnectionWithCredential.mockResolvedValue( + Ok({id: 'conn-1', integrationId: 'email', account: 'alex@example.com', createdAt: new Date(), tools: null, name: null, defaults: [], credential: 'enc:{}'}), + ); isUsernameConflict.mockReturnValue(false); isProviderConflict.mockReturnValue(false); }); @@ -179,8 +186,24 @@ describe('handleSetAgentGrantsRoute', () => { expect(setAgentGrants).not.toHaveBeenCalled(); }); + test("refuses turning on an action the connection's sign-in didn't cover, but not turning it off", async () => { + const credential = {accessToken: 'access-1', refreshToken: 'refresh-1', expiresAt: null, scopes: ['openid'], email: 'alex@example.com'}; + getConnectionWithCredential.mockResolvedValue( + Ok({id: 'conn-1', integrationId: 'google', account: 'alex@example.com', createdAt: new Date(), defaults: [], credential: `enc:${JSON.stringify(credential)}`}), + ); + + const {handleSetAgentGrantsRoute} = await import('./route'); + const on = await handleSetAgentGrantsRoute(makeRequest(params, {actions: {readFiles: true}})); + expect(on.unwrapErr()).toMatchObject({kind: 'validation_error', message: "Google didn't allow Read files at sign-in. Sign in to Google again to allow it."}); + expect(setAgentGrants).not.toHaveBeenCalled(); + + const off = await handleSetAgentGrantsRoute(makeRequest(params, {actions: {readFiles: false}})); + expect(off.isOk()).toBe(true); + expect(setAgentGrants).toHaveBeenCalledWith({agentId: 'agent-1', connectionId: 'conn-1', actions: {readFiles: false}}); + }); + test('refuses a connection outside the organization', async () => { - getConnection.mockResolvedValue(Ok(null)); + getConnectionWithCredential.mockResolvedValue(Ok(null)); const {handleSetAgentGrantsRoute} = await import('./route'); const result = await handleSetAgentGrantsRoute(makeRequest(params, {actions: {read: true}})); diff --git a/packages/api/src/agents/route.ts b/packages/api/src/agents/route.ts index c837ae6d..ad7ebf75 100644 --- a/packages/api/src/agents/route.ts +++ b/packages/api/src/agents/route.ts @@ -3,9 +3,10 @@ import {z} from 'zod'; import {ApiErr, type ApiError, Do, parseSchema, requirePresent} from '@proxy/utils'; import {deleteAgent, getAgent, listAgents, setAgentGrants, updateAgent} from '@proxy/db/agent'; -import {getConnection} from '@proxy/db/connection'; +import {getConnectionWithCredential} from '@proxy/db/connection'; import {deleteOAuthGrant, listAgentOAuthGrants} from '@proxy/db/oauth'; +import {refuseUnavailable, unavailableActions} from '../connections/unavailable'; import type {AuthenticatedRequest} from '../server/middleware/require-auth'; import {parseActionChanges, requireIntegration} from '../utils/connection-actions'; import {requireUserOrgId} from '../utils/user-org'; @@ -92,7 +93,7 @@ export function handleDeleteAgentRoute(request: AuthenticatedRequest): Promise> { return Do(async ($) => { @@ -100,9 +101,10 @@ export function handleSetAgentGrantsRoute(request: AuthenticatedRequest): Promis const agentId = agentIdOf(request); const connectionId = request.params.connectionId ?? ''; $(await requireAgent(orgId, agentId)); - const connection = $(requirePresent($(await getConnection(orgId, connectionId)), ApiErr.notFound('connection', connectionId))); + const connection = $(requirePresent($(await getConnectionWithCredential(orgId, connectionId)), ApiErr.notFound('connection', connectionId))); const integration = $(requireIntegration(connection)); const actions = $(parseActionChanges(integration, z.boolean().nullable(), request.body)); + $(refuseUnavailable(integration, unavailableActions(connection), actions)); $(await setAgentGrants({agentId, connectionId, actions})); return $(await requireAgent(orgId, agentId)); diff --git a/packages/api/src/connections/connect-flow.ts b/packages/api/src/connections/connect-flow.ts index 5357d5d7..136b24c8 100644 --- a/packages/api/src/connections/connect-flow.ts +++ b/packages/api/src/connections/connect-flow.ts @@ -40,8 +40,12 @@ type ReturnTo = z.infer; type Route = (request: AuthenticatedRequest, res: Response) => Promise; -/** What the provider's sign-in is sent: where it comes back to, the state, and the PKCE challenge. */ -export type SignInRequest = {redirectUri: string; state: string; challenge: string}; +/** + * What the provider's sign-in is sent: where it comes back to, the state, the PKCE challenge, and + * the account a sign-in again is for (`?account=`), as a connection's page passes it, for a provider + * that can open its sign-in on it. + */ +export type SignInRequest = {redirectUri: string; state: string; challenge: string; account: string | null}; /** The provider's sign-in page, and what to keep in the cookie for the callback. */ type SignInStart = (request: SignInRequest) => Promise>; @@ -146,12 +150,12 @@ export function signInFlow(args: SignInArgs) { } /** Keeps the handshake, and what `start` keeps beside it, in a cookie on `res`, and answers the provider's sign-in page, for the browser to go to. */ - function beginSignIn(res: Response, begin: {userId: string; returnTo: ReturnTo | undefined; start: SignInStart}): Promise> { + function beginSignIn(res: Response, begin: {userId: string; returnTo: ReturnTo | undefined; account: string | null; start: SignInStart}): Promise> { return Do(async ($) => { const appUrl = $(requirePresent(env.APP_URL, ApiErr.internalError(new Error('APP_URL is not configured')))); const state = randomBytes(16).toString('hex'); const {verifier, challenge} = makePkce(); - const {url, kept} = $(await begin.start({redirectUri: redirectUri(appUrl), state, challenge})); + const {url, kept} = $(await begin.start({redirectUri: redirectUri(appUrl), state, challenge, account: begin.account})); const payload: StatePayload = {state, verifier, userId: begin.userId, ...(begin.returnTo ? {returnTo: begin.returnTo} : {}), kept}; const cookie = $(encryptSecret(JSON.stringify(payload))); res.cookie(stateCookieName, cookie, {...stateCookieOptions(), maxAge: stateCookieMaxAgeMs}); @@ -225,7 +229,7 @@ export function signInRoutes(args: SignInArgs & {start: SignInStart< } const returnTo = returnToSchema.safeParse(queryParam(request, 'returnTo')).data; - const begun = await flow.beginSignIn(res, {userId: request.user.userId, returnTo, start: args.start}); + const begun = await flow.beginSignIn(res, {userId: request.user.userId, returnTo, account: queryParam(request, 'account'), start: args.start}); if (begun.isErr()) { flow.redirectToConnectError(res, appUrl, returnTo, begun.error); return; diff --git a/packages/api/src/connections/connection-response.ts b/packages/api/src/connections/connection-response.ts index bb499e62..647cacae 100644 --- a/packages/api/src/connections/connection-response.ts +++ b/packages/api/src/connections/connection-response.ts @@ -16,9 +16,12 @@ export type ConnectionResponse = { // What the person called the connection, where they named it; null for every other integration, // which is called by its integration. name: string | null; + // The actions the connection's sign-in didn't cover, in the catalog's order: no agent gets them + // until the person signs in again and allows them. + unavailable: string[]; }; -export function toConnectionResponse(row: ConnectionRow): ConnectionResponse { +export function toConnectionResponse(row: ConnectionRow, unavailable: string[]): ConnectionResponse { const integration = integrationOf(row); const actions = integration?.actions ?? []; return { @@ -28,5 +31,6 @@ export function toConnectionResponse(row: ConnectionRow): ConnectionResponse { defaults: actions.map((action) => action.id).filter((id) => row.defaults.some((stored) => stored.actionId === id)), tools: integration?.mcpTools ?? null, name: row.name, + unavailable, }; } diff --git a/packages/api/src/connections/custom/route.test.ts b/packages/api/src/connections/custom/route.test.ts index ac0bb11e..19f76b7a 100644 --- a/packages/api/src/connections/custom/route.test.ts +++ b/packages/api/src/connections/custom/route.test.ts @@ -163,7 +163,10 @@ describe('handleConnectCustomMcpRoute', () => { tools: JSON.stringify([search, publish]), name: 'Docs', }); - expect(res.sent).toEqual({kind: 'connected', connection: {id: 'mcp-1', integrationId: 'mcp', account: 'mcp.example.com/mcp', defaults: ['search_docs'], tools: [search, publish], name: 'Docs'}}); + expect(res.sent).toEqual({ + kind: 'connected', + connection: {id: 'mcp-1', integrationId: 'mcp', account: 'mcp.example.com/mcp', defaults: ['search_docs'], tools: [search, publish], name: 'Docs', unavailable: []}, + }); }); test('the connection needs a name, as nothing a server says would do', async () => { diff --git a/packages/api/src/connections/custom/route.ts b/packages/api/src/connections/custom/route.ts index 2393a0fe..d0057d80 100644 --- a/packages/api/src/connections/custom/route.ts +++ b/packages/api/src/connections/custom/route.ts @@ -74,12 +74,12 @@ function connectCustomMcp(request: AuthenticatedRequest, res: Response): Promise if (probe.kind === 'signIn') { const server = $(await discoverMcpServer(url)); const start = (signIn: SignInRequest) => startMcpSignIn(server, signIn, {server: mcpEndpoints(server), name: body.name}); - return {kind: 'signIn' as const, url: $(await beginSignIn(res, {userId: request.user.userId, returnTo: body.returnTo, start}))}; + return {kind: 'signIn' as const, url: $(await beginSignIn(res, {userId: request.user.userId, returnTo: body.returnTo, account: null, start}))}; } const credential: CustomCredential = token === null ? {auth: 'none', url: url.toString()} : {auth: 'token', url: url.toString(), token}; const orgId = $(await requireUserOrgId(request)); const {row} = $(await saveConnection(orgId, 'mcp', {account: mcpAccount(url), credential, defaults: readOnlyToolNames(probe.tools), tools: probe.tools, name: body.name})); - return {kind: 'connected' as const, connection: toConnectionResponse(row)}; + return {kind: 'connected' as const, connection: toConnectionResponse(row, [])}; }); } @@ -100,6 +100,6 @@ export function handleRefreshCustomMcpToolsRoute(request: AuthenticatedRequest): const tools = $(await listCustomTools(row)); const stored = JSON.stringify(tools); $(await updateConnectionTools({connectionId, tools: stored, actions: tools.map((tool) => tool.name)})); - return toConnectionResponse({...row, tools: stored}); + return toConnectionResponse({...row, tools: stored}, []); }); } diff --git a/packages/api/src/connections/disconnect.ts b/packages/api/src/connections/disconnect.ts new file mode 100644 index 00000000..292b6e35 --- /dev/null +++ b/packages/api/src/connections/disconnect.ts @@ -0,0 +1,35 @@ +import {type ApiError, Do} from '@proxy/utils'; +import {countSameAccountConnections} from '@proxy/db/connection'; + +import {log, serializeError} from '../observability/log'; +import {readGoogleCredential, revokeGoogleTokens} from './google/google-oauth'; + +/** + * What is undone at the provider before a connection is deleted, where the provider has a way: + * Google's grant is revoked, so its tokens stop working there too. Only Google has one so far. A + * grant is the account's for the app's Google client as a whole, every refresh token it has, so + * it is kept while another connection, in this organization or any other, is for the same + * account, by Google's id for it or its address. A failure is logged and the connection is + * deleted anyway; the person can still revoke it at myaccount.google.com/permissions. + */ + +type DisconnectedConnection = {id: string; integrationId: string; credential: string | null}; + +export async function beforeDisconnect(connection: DisconnectedConnection): Promise { + if (connection.integrationId !== 'google') { + return; + } + + const revoked = await Do(async ($) => { + const sharing = $(await countSameAccountConnections(connection.id)); + if (sharing > 1) { + log.info('Kept the Google grant another connection uses', {connectionId: connection.id}); + return; + } + $(await revokeGoogleTokens($(readGoogleCredential(connection.credential)))); + }); + if (revoked.isErr()) { + const error = revoked.error; + log.warn('Revoking a Google connection at Google failed', {connectionId: connection.id, kind: error.kind, ...('cause' in error ? serializeError(error.cause) : {})}); + } +} diff --git a/packages/api/src/connections/email/route.ts b/packages/api/src/connections/email/route.ts index 7b9d1b5e..98219699 100644 --- a/packages/api/src/connections/email/route.ts +++ b/packages/api/src/connections/email/route.ts @@ -89,6 +89,7 @@ export function handleConnectEmailRoute(request: AuthenticatedRequest): Promise< $(await checkMailbox(credential)); const row = $(await createConnection({orgId, integrationId: 'email', account: email, credential: encrypted})); - return toConnectionResponse(row); + // A password signs in to all of it: every action is available. + return toConnectionResponse(row, []); }); } diff --git a/packages/api/src/connections/google/google-oauth.ts b/packages/api/src/connections/google/google-oauth.ts index 62a80046..31c7b485 100644 --- a/packages/api/src/connections/google/google-oauth.ts +++ b/packages/api/src/connections/google/google-oauth.ts @@ -4,6 +4,7 @@ import {z} from 'zod'; import {ApiErr, type ApiError, Do, type FetchError, httpRequest, parseSchema, requirePresent} from '@proxy/utils'; import type {GoogleClient} from '../../utils/google-oauth-client'; +import {decryptSecret} from '../../utils/secret-crypto'; /** * Signing in to Google for a connection: OAuth 2.0 with PKCE against Google's own endpoints, with @@ -15,6 +16,7 @@ import type {GoogleClient} from '../../utils/google-oauth-client'; const AUTHORIZE_URL = 'https://accounts.google.com/o/oauth2/v2/auth'; const TOKEN_URL = 'https://oauth2.googleapis.com/token'; +const REVOKE_URL = 'https://oauth2.googleapis.com/revoke'; const TIMEOUT_MS = 15_000; @@ -44,6 +46,17 @@ export const googleCredentialSchema = z.object({ export type GoogleCredential = z.infer; +/** A Google connection's stored credential, decrypted; one that won't decrypt or read is an internal error. */ +export function readGoogleCredential(stored: string | null): Result { + const unreadable = (cause: unknown): ApiError => ApiErr.internalError(cause); + return Do(($) => { + const present = $(requirePresent(stored, unreadable(new Error('Google connection without a credential')))); + const json = $(decryptSecret(present)); + const parsed = $(Result.wrap((): unknown => JSON.parse(json)).mapErr(unreadable)); + return $(parseSchema(googleCredentialSchema, parsed).mapErr(unreadable)); + }); +} + const tokensSchema = z.object({ access_token: z.string().min(1), refresh_token: z.string().optional(), @@ -62,8 +75,14 @@ const idTokenPayloadSchema = z.object({sub: z.string().min(1), email: z.string() /** Who a sign-in was for: Google's id for the account, and its address as it is now. */ export type GoogleAccount = {id: string; email: string}; -/** Google's sign-in page for a connection, coming back to `redirectUri` with `state`. */ -export function googleAuthorizeUrl(client: GoogleClient, args: {redirectUri: string; state: string; challenge: string}): string { +// The account a sign-in again is for, as Google takes it in `login_hint`. +const loginHintSchema = z.email(); + +/** + * Google's sign-in page for a connection, coming back to `redirectUri` with `state`. Signing in + * again from a connection names its `account`, so Google opens on it. + */ +export function googleAuthorizeUrl(client: GoogleClient, args: {redirectUri: string; state: string; challenge: string; account: string | null}): string { const url = new URL(AUTHORIZE_URL); url.searchParams.set('response_type', 'code'); url.searchParams.set('client_id', client.clientId); @@ -76,6 +95,10 @@ export function googleAuthorizeUrl(client: GoogleClient, args: {redirectUri: str url.searchParams.set('access_type', 'offline'); url.searchParams.set('prompt', 'consent'); url.searchParams.set('include_granted_scopes', 'true'); + const hint = loginHintSchema.safeParse(args.account); + if (hint.success) { + url.searchParams.set('login_hint', hint.data); + } return url.toString(); } @@ -168,3 +191,20 @@ export async function refreshGoogleTokens(client: GoogleClient, credential: Goog export function isExpiring(credential: GoogleCredential): boolean { return credential.expiresAt !== null && Date.parse(credential.expiresAt) - EXPIRY_MARGIN_MS < Date.now(); } + +/** + * Ends the connection's grant at Google, as disconnecting it does: revoking the refresh token + * revokes the access tokens made from it too. + */ +export async function revokeGoogleTokens(credential: GoogleCredential): Promise> { + const revoked = await httpRequest( + REVOKE_URL, + { + method: 'POST', + headers: {'content-type': 'application/x-www-form-urlencoded'}, + body: new URLSearchParams({token: credential.refreshToken ?? credential.accessToken}).toString(), + }, + {parse: 'text', timeoutMs: TIMEOUT_MS}, + ); + return revoked.map(() => undefined).mapErr(tokenError); +} diff --git a/packages/api/src/connections/google/route.test.ts b/packages/api/src/connections/google/route.test.ts index 1c2762e4..db43b7cf 100644 --- a/packages/api/src/connections/google/route.test.ts +++ b/packages/api/src/connections/google/route.test.ts @@ -143,6 +143,20 @@ describe('handleStartGoogleRoute', () => { expect(google.requests).toEqual([]); }); + test('signing in again from a connection opens Google on its account', async () => { + const {authorizeUrl} = await start({account: 'alex@example.com'}); + + expect(authorizeUrl?.searchParams.get('login_hint')).toBe('alex@example.com'); + }); + + test('an account that is not an address is not passed on', async () => { + const {authorizeUrl} = await start({account: 'not an address'}); + const {authorizeUrl: none} = await start(); + + expect(authorizeUrl?.searchParams.has('login_hint')).toBe(false); + expect(none?.searchParams.has('login_hint')).toBe(false); + }); + test('without a Google client set up, goes back to the catalog with the error', async () => { env.GOOGLE_CLIENT_ID = undefined; const {res, cookie} = await start(); diff --git a/packages/api/src/connections/google/scopes.ts b/packages/api/src/connections/google/scopes.ts new file mode 100644 index 00000000..63d5870e --- /dev/null +++ b/packages/api/src/connections/google/scopes.ts @@ -0,0 +1,23 @@ +import {findIntegration} from '@proxy/integrations'; + +import type {GoogleCredential} from './google-oauth'; + +/** + * Which of Google's scopes allow each action. Google's consent screen lets a person untick some of + * what a sign-in asks for, and a connection made before an action joined the catalog lacks its + * scope, so an action is only available where the sign-in granted one of the scopes allowing it. + */ + +const scope = (name: string): string => `https://www.googleapis.com/auth/${name}`; + +// Any one of an action's scopes is enough. Drive's export and the Docs and Sheets APIs all take +// `drive`, so it alone covers reading files. +const ACTION_SCOPES: Partial> = { + readFiles: [scope('drive')], +}; + +/** The actions a Google sign-in didn't cover, in the catalog's order; one the map leaves out is never covered. */ +export function unavailableGoogleActions(credential: GoogleCredential): string[] { + const actions = findIntegration('google')?.actions ?? []; + return actions.filter((action) => !(ACTION_SCOPES[action.id] ?? []).some((allowing) => credential.scopes.includes(allowing))).map((action) => action.id); +} diff --git a/packages/api/src/connections/mcp/mcp-connect.ts b/packages/api/src/connections/mcp/mcp-connect.ts index f853269a..09f8e5d2 100644 --- a/packages/api/src/connections/mcp/mcp-connect.ts +++ b/packages/api/src/connections/mcp/mcp-connect.ts @@ -18,10 +18,10 @@ import {exchangeMcpCode, mcpAuthorizeUrl, registerMcpClient, type McpServer, typ * Registers with the server for this sign-in, and its sign-in page, keeping the client for the * callback beside `kept`. */ -export function startMcpSignIn(server: McpServer, request: SignInRequest, kept: K): Promise> { +export function startMcpSignIn(server: McpServer, {redirectUri, state, challenge}: SignInRequest, kept: K): Promise> { return Do(async ($) => { - const clientId = $(await registerMcpClient(server, request.redirectUri)); - return {url: mcpAuthorizeUrl(server, {clientId, ...request}), kept: {...kept, clientId}}; + const clientId = $(await registerMcpClient(server, redirectUri)); + return {url: mcpAuthorizeUrl(server, {clientId, redirectUri, state, challenge}), kept: {...kept, clientId}}; }); } diff --git a/packages/api/src/connections/route.test.ts b/packages/api/src/connections/route.test.ts index 91a23bc2..2bb170e3 100644 --- a/packages/api/src/connections/route.test.ts +++ b/packages/api/src/connections/route.test.ts @@ -1,26 +1,50 @@ -import {beforeEach, describe, expect, mock, test} from 'bun:test'; -import {Err, Ok} from 'ts-results-es'; +import {beforeEach, describe, expect, mock, spyOn, test} from 'bun:test'; +import {Err, Ok, type Result} from 'ts-results-es'; -import {ApiErr} from '@proxy/utils'; +import {ApiErr, type ApiError} from '@proxy/utils'; -const listConnections = mock(); -const getConnection = mock(); +const listConnectionsWithCredential = mock(); +const getConnectionWithCredential = mock(); const deleteConnection = mock(); const setConnectionDefaults = mock(); const updateConnectionName = mock(); +const countSameAccountConnections = mock(); mock.module('@proxy/db/connection', () => ({ - listConnections, - getConnection, + listConnectionsWithCredential, + getConnectionWithCredential, deleteConnection, setConnectionDefaults, updateConnectionName, + countSameAccountConnections, })); const getUserOrgId = mock(); mock.module('@proxy/db/organization', () => ({getUserOrgId})); +const warn = mock(); + +mock.module('../observability/log', () => ({ + log: {debug: mock(), info: mock(), warn, error: mock()}, + serializeError: mock(() => ({})), +})); + +// Readable stand-ins for encryption, so a test can write the credential it means. +mock.module('../utils/secret-crypto', () => ({ + encryptSecret: (plaintext: string): Result => Ok(`enc:${plaintext}`), + decryptSecret: (stored: string): Result => Ok(stored.replace(/^enc:/, '')), +})); + +// Google's revoke endpoint: each request, and its answer. +const google = {requests: [] as Array<{url: string; body: URLSearchParams}>, status: 200}; + +spyOn(globalThis, 'fetch').mockImplementation((async (input: string | URL | Request, init?: RequestInit) => { + const url = typeof input === 'string' ? input : input instanceof URL ? input.href : input.url; + google.requests.push({url, body: new URLSearchParams(typeof init?.body === 'string' ? init.body : '')}); + return new Response(google.status === 200 ? '{}' : '{"error":"invalid_token"}', {status: google.status}); +}) as typeof fetch); + const emailRow = { id: 'conn-1', integrationId: 'email', @@ -29,8 +53,17 @@ const emailRow = { tools: null, name: null, defaults: [{actionId: 'read'}], + credential: 'enc:{}', }; +function googleRow(scopes: string[]) { + const credential = {accessToken: 'access-1', refreshToken: 'refresh-1', expiresAt: null, scopes, email: 'alex@example.com'}; + return {...emailRow, id: 'conn-2', integrationId: 'google', defaults: [], credential: `enc:${JSON.stringify(credential)}`}; +} + +// The person unticked Drive on Google's consent screen. +const withoutDrive = googleRow(['openid', 'https://www.googleapis.com/auth/userinfo.email', 'https://www.googleapis.com/auth/gmail.modify']); + function makeRequest(params: Record = {}, body: unknown = undefined) { return { user: {userId: 'user-1', email: 'alex@example.com'}, @@ -41,9 +74,12 @@ function makeRequest(params: Record = {}, body: unknown = undefi beforeEach(() => { mock.clearAllMocks(); + google.requests = []; + google.status = 200; + countSameAccountConnections.mockResolvedValue(Ok(1)); getUserOrgId.mockResolvedValue(Ok('org-1')); - listConnections.mockResolvedValue(Ok([emailRow])); - getConnection.mockResolvedValue(Ok(emailRow)); + listConnectionsWithCredential.mockResolvedValue(Ok([emailRow])); + getConnectionWithCredential.mockResolvedValue(Ok(emailRow)); deleteConnection.mockResolvedValue(Ok(true)); setConnectionDefaults.mockResolvedValue(Ok(undefined)); }); @@ -53,10 +89,31 @@ describe('handleListConnectionsRoute', () => { const {handleListConnectionsRoute} = await import('./route'); const result = await handleListConnectionsRoute(makeRequest()); - expect(listConnections).toHaveBeenCalledWith('org-1'); + expect(listConnectionsWithCredential).toHaveBeenCalledWith('org-1'); const [connection] = result.unwrap().connections; expect(connection).toMatchObject({id: 'conn-1', integrationId: 'email', account: 'alex@example.com'}); expect(connection?.defaults).toEqual(['read']); + expect(connection?.unavailable).toEqual([]); + }); + + test("a Google credential that can't be read leaves the rest listed, itself with no action available", async () => { + listConnectionsWithCredential.mockResolvedValue(Ok([emailRow, {...googleRow([]), credential: 'v1.other-key'}])); + const {handleListConnectionsRoute} = await import('./route'); + const {connections} = (await handleListConnectionsRoute(makeRequest())).unwrap(); + + expect(connections.map((connection) => [connection.id, connection.unavailable])).toEqual([ + ['conn-1', []], + ['conn-2', ['readFiles']], + ]); + }); + + test("says which actions a Google connection's sign-in didn't cover, never the credential", async () => { + listConnectionsWithCredential.mockResolvedValue(Ok([emailRow, withoutDrive])); + + const {handleListConnectionsRoute} = await import('./route'); + const [, connection] = (await handleListConnectionsRoute(makeRequest())).unwrap().connections; + + expect(connection).toEqual({id: 'conn-2', integrationId: 'google', account: 'alex@example.com', defaults: [], tools: null, name: null, unavailable: ['readFiles']}); }); test('a user without an organization is signed out', async () => { @@ -74,12 +131,12 @@ describe('handleGetConnectionRoute', () => { const {handleGetConnectionRoute} = await import('./route'); const result = await handleGetConnectionRoute(makeRequest({connectionId: 'conn-1'})); - expect(getConnection).toHaveBeenCalledWith('org-1', 'conn-1'); + expect(getConnectionWithCredential).toHaveBeenCalledWith('org-1', 'conn-1'); expect(result.unwrap().id).toBe('conn-1'); }); test('an unknown connection is not found', async () => { - getConnection.mockResolvedValue(Ok(null)); + getConnectionWithCredential.mockResolvedValue(Ok(null)); const {handleGetConnectionRoute} = await import('./route'); const result = await handleGetConnectionRoute(makeRequest({connectionId: 'other'})); @@ -103,11 +160,11 @@ describe('handleRenameConnectionRoute', () => { test('a custom MCP server needs a name, and Information keeps its own', async () => { updateConnectionName.mockResolvedValue(Ok(undefined)); - getConnection.mockResolvedValue(Ok({...emailRow, integrationId: 'mcp', tools: '[]', name: 'Docs'})); + getConnectionWithCredential.mockResolvedValue(Ok({...emailRow, integrationId: 'mcp', tools: '[]', name: 'Docs'})); const {handleRenameConnectionRoute} = await import('./route'); expect((await handleRenameConnectionRoute(makeRequest({connectionId: 'conn-1'}, {name: ' '}))).unwrapErr().kind).toBe('validation_error'); - getConnection.mockResolvedValue(Ok({...emailRow, integrationId: 'info'})); + getConnectionWithCredential.mockResolvedValue(Ok({...emailRow, integrationId: 'info'})); expect((await handleRenameConnectionRoute(makeRequest({connectionId: 'conn-1'}, {name: 'Mine'}))).unwrapErr().kind).toBe('forbidden'); expect(updateConnectionName).not.toHaveBeenCalled(); }); @@ -122,8 +179,51 @@ describe('handleDeleteConnectionRoute', () => { expect(result.isOk()).toBe(true); }); + test('revokes a Google connection at Google first, its refresh token ending the whole grant', async () => { + getConnectionWithCredential.mockResolvedValue(Ok(withoutDrive)); + + const {handleDeleteConnectionRoute} = await import('./route'); + const result = await handleDeleteConnectionRoute(makeRequest({connectionId: 'conn-2'})); + + expect(result.isOk()).toBe(true); + expect(google.requests.map(({url, body}) => [url, body.get('token')])).toEqual([['https://oauth2.googleapis.com/revoke', 'refresh-1']]); + expect(deleteConnection).toHaveBeenCalledWith('org-1', 'conn-2'); + }); + + test('keeps the grant at Google while another connection, in any organization, is for the same account', async () => { + getConnectionWithCredential.mockResolvedValue(Ok(withoutDrive)); + countSameAccountConnections.mockResolvedValue(Ok(2)); + + const {handleDeleteConnectionRoute} = await import('./route'); + const result = await handleDeleteConnectionRoute(makeRequest({connectionId: 'conn-2'})); + + expect(result.isOk()).toBe(true); + expect(countSameAccountConnections).toHaveBeenCalledWith('conn-2'); + expect(google.requests).toEqual([]); + expect(deleteConnection).toHaveBeenCalledWith('org-1', 'conn-2'); + }); + + test('deletes a Google connection even when Google turns the revoke down', async () => { + getConnectionWithCredential.mockResolvedValue(Ok(withoutDrive)); + google.status = 400; + + const {handleDeleteConnectionRoute} = await import('./route'); + const result = await handleDeleteConnectionRoute(makeRequest({connectionId: 'conn-2'})); + + expect(result.isOk()).toBe(true); + expect(warn).toHaveBeenCalledTimes(1); + expect(deleteConnection).toHaveBeenCalledWith('org-1', 'conn-2'); + }); + + test('asks nothing of the provider for any other connection', async () => { + const {handleDeleteConnectionRoute} = await import('./route'); + await handleDeleteConnectionRoute(makeRequest({connectionId: 'conn-1'})); + + expect(google.requests).toEqual([]); + }); + test('Information is built in and stays', async () => { - getConnection.mockResolvedValue(Ok({...emailRow, integrationId: 'info'})); + getConnectionWithCredential.mockResolvedValue(Ok({...emailRow, integrationId: 'info'})); const {handleDeleteConnectionRoute} = await import('./route'); const result = await handleDeleteConnectionRoute(makeRequest({connectionId: 'conn-1'})); @@ -133,7 +233,7 @@ describe('handleDeleteConnectionRoute', () => { }); test('an unknown connection is not found', async () => { - getConnection.mockResolvedValue(Ok(null)); + getConnectionWithCredential.mockResolvedValue(Ok(null)); const {handleDeleteConnectionRoute} = await import('./route'); const result = await handleDeleteConnectionRoute(makeRequest({connectionId: 'other'})); @@ -159,6 +259,26 @@ describe('handleSetConnectionDefaultsRoute', () => { expect(setConnectionDefaults).not.toHaveBeenCalled(); }); + test("refuses turning on an action the sign-in didn't cover, saying how to allow it", async () => { + getConnectionWithCredential.mockResolvedValue(Ok(withoutDrive)); + + const {handleSetConnectionDefaultsRoute} = await import('./route'); + const result = await handleSetConnectionDefaultsRoute(makeRequest({connectionId: 'conn-2'}, {actions: {readFiles: true}})); + + expect(result.unwrapErr()).toMatchObject({kind: 'validation_error', message: "Google didn't allow Read files at sign-in. Sign in to Google again to allow it."}); + expect(setConnectionDefaults).not.toHaveBeenCalled(); + }); + + test("turns off an action the sign-in didn't cover", async () => { + getConnectionWithCredential.mockResolvedValue(Ok(withoutDrive)); + + const {handleSetConnectionDefaultsRoute} = await import('./route'); + const result = await handleSetConnectionDefaultsRoute(makeRequest({connectionId: 'conn-2'}, {actions: {readFiles: false}})); + + expect(result.unwrap().unavailable).toEqual(['readFiles']); + expect(setConnectionDefaults).toHaveBeenCalledWith({connectionId: 'conn-2', actions: {readFiles: false}}); + }); + test('refuses a setting that is not on or off', async () => { const {handleSetConnectionDefaultsRoute} = await import('./route'); const result = await handleSetConnectionDefaultsRoute(makeRequest({connectionId: 'conn-1'}, {actions: {read: null}})); diff --git a/packages/api/src/connections/route.ts b/packages/api/src/connections/route.ts index 3c63f093..2cd5302d 100644 --- a/packages/api/src/connections/route.ts +++ b/packages/api/src/connections/route.ts @@ -2,40 +2,58 @@ import {Err, type Result} from 'ts-results-es'; import {z} from 'zod'; import {ApiErr, type ApiError, Do, parseSchema, requirePresent} from '@proxy/utils'; -import {deleteConnection, getConnection, listConnections, setConnectionDefaults, updateConnectionName} from '@proxy/db/connection'; +import {deleteConnection, getConnectionWithCredential, listConnectionsWithCredential, setConnectionDefaults, updateConnectionName, type ConnectionRow} from '@proxy/db/connection'; import type {AuthenticatedRequest} from '../server/middleware/require-auth'; import {integrationOf, parseActionChanges, requireIntegration} from '../utils/connection-actions'; import {requireUserOrgId} from '../utils/user-org'; import {toConnectionResponse, type ConnectionResponse} from './connection-response'; +import {beforeDisconnect} from './disconnect'; +import {refuseUnavailable, unavailableActions} from './unavailable'; + +type StoredConnectionRow = ConnectionRow & {credential: string | null}; + +// The connection as the web app gets it, with the actions its sign-in didn't cover, read from its +// credential on every request. +function respond(row: StoredConnectionRow): ConnectionResponse { + return toConnectionResponse(row, unavailableActions(row)); +} + +function requireConnection(orgId: string, connectionId: string): Promise> { + return Do(async ($) => { + return $(requirePresent($(await getConnectionWithCredential(orgId, connectionId)), ApiErr.notFound('connection', connectionId))); + }); +} export function handleListConnectionsRoute(request: AuthenticatedRequest): Promise> { return Do(async ($) => { const orgId = $(await requireUserOrgId(request)); - const rows = $(await listConnections(orgId)); - return {connections: rows.map(toConnectionResponse)}; + const rows = $(await listConnectionsWithCredential(orgId)); + return {connections: rows.map(respond)}; }); } export function handleGetConnectionRoute(request: AuthenticatedRequest): Promise> { return Do(async ($) => { const orgId = $(await requireUserOrgId(request)); - const connectionId = request.params.connectionId ?? ''; - const row = $(await getConnection(orgId, connectionId)); - return toConnectionResponse($(requirePresent(row, ApiErr.notFound('connection', connectionId)))); + return respond($(await requireConnection(orgId, request.params.connectionId ?? ''))); }); } -/** Agents lose the connection at once. Information is built in and can't be disconnected. */ +/** + * Agents lose the connection at once, and the provider's grant is revoked first where it has a way, + * as Google does. Information is built in and can't be disconnected. + */ export function handleDeleteConnectionRoute(request: AuthenticatedRequest): Promise> { return Do(async ($) => { const orgId = $(await requireUserOrgId(request)); const connectionId = request.params.connectionId ?? ''; - const row = $(requirePresent($(await getConnection(orgId, connectionId)), ApiErr.notFound('connection', connectionId))); + const row = $(await requireConnection(orgId, connectionId)); if (integrationOf(row)?.builtIn) { return $(Err(ApiErr.forbidden())); } + await beforeDisconnect(row); const deleted = $(await deleteConnection(orgId, connectionId)); if (!deleted) { return $(Err(ApiErr.notFound('connection', connectionId))); @@ -56,7 +74,7 @@ export function handleRenameConnectionRoute(request: AuthenticatedRequest): Prom return Do(async ($) => { const orgId = $(await requireUserOrgId(request)); const connectionId = request.params.connectionId ?? ''; - const row = $(requirePresent($(await getConnection(orgId, connectionId)), ApiErr.notFound('connection', connectionId))); + const row = $(await requireConnection(orgId, connectionId)); const integration = $(requireIntegration(row)); if (integration.builtIn) { return $(Err(ApiErr.forbidden())); @@ -67,24 +85,25 @@ export function handleRenameConnectionRoute(request: AuthenticatedRequest): Prom } $(await updateConnectionName({connectionId, name: name === '' ? null : name})); - return toConnectionResponse({...row, name: name === '' ? null : name}); + return respond({...row, name: name === '' ? null : name}); }); } /** * Turns actions on or off for every agent without a setting of its own, such as - * `{actions: {read: true, send: false}}`. An action the integration doesn't have is refused. + * `{actions: {read: true, send: false}}`. An action the integration doesn't have is refused, and so + * is turning on one the connection's sign-in didn't cover. */ export function handleSetConnectionDefaultsRoute(request: AuthenticatedRequest): Promise> { return Do(async ($) => { const orgId = $(await requireUserOrgId(request)); const connectionId = request.params.connectionId ?? ''; - const row = $(requirePresent($(await getConnection(orgId, connectionId)), ApiErr.notFound('connection', connectionId))); + const row = $(await requireConnection(orgId, connectionId)); const integration = $(requireIntegration(row)); const actions = $(parseActionChanges(integration, z.boolean(), request.body)); + $(refuseUnavailable(integration, unavailableActions(row), actions)); $(await setConnectionDefaults({connectionId, actions})); - const updated = $(await getConnection(orgId, connectionId)); - return toConnectionResponse($(requirePresent(updated, ApiErr.notFound('connection', connectionId)))); + return respond($(await requireConnection(orgId, connectionId))); }); } diff --git a/packages/api/src/connections/sql/route.ts b/packages/api/src/connections/sql/route.ts index 7f517f72..b29e91c5 100644 --- a/packages/api/src/connections/sql/route.ts +++ b/packages/api/src/connections/sql/route.ts @@ -68,7 +68,8 @@ export function sqlConnectRoutes(dialect: SqlDialect): { $(await checkDatabase(dialect, credential)); const row = $(await createConnection({orgId, integrationId: dialect, account, credential: encrypted, defaults: DEFAULTS})); - return toConnectionResponse(row); + // A password signs in to all of it: every action is available. + return toConnectionResponse(row, []); }), }; } diff --git a/packages/api/src/connections/unavailable.test.ts b/packages/api/src/connections/unavailable.test.ts new file mode 100644 index 00000000..d6b9320f --- /dev/null +++ b/packages/api/src/connections/unavailable.test.ts @@ -0,0 +1,83 @@ +import {describe, expect, mock, test} from 'bun:test'; +import {Err, Ok, type Result} from 'ts-results-es'; + +import {ApiErr, type ApiError} from '@proxy/utils'; + +const warn = mock(); + +mock.module('../observability/log', () => ({ + log: {debug: mock(), info: mock(), warn, error: mock()}, + serializeError: mock(() => ({})), +})); + +// Readable stand-ins for encryption; a credential stored under another key won't decrypt. +mock.module('../utils/secret-crypto', () => ({ + decryptSecret: (stored: string): Result => (stored.startsWith('enc:') ? Ok(stored.slice('enc:'.length)) : Err(ApiErr.internalError(new Error('Unreadable secret')))), +})); + +const ALL_SCOPES = [ + 'openid', + 'https://www.googleapis.com/auth/userinfo.email', + 'https://www.googleapis.com/auth/gmail.modify', + 'https://www.googleapis.com/auth/gmail.send', + 'https://www.googleapis.com/auth/drive', + 'https://www.googleapis.com/auth/documents', + 'https://www.googleapis.com/auth/spreadsheets', + 'https://www.googleapis.com/auth/presentations', +]; + +function google(scopes: string[]) { + const credential = {accessToken: 'access-1', refreshToken: 'refresh-1', expiresAt: null, scopes, email: 'alex@example.com'}; + return {id: 'google-1', integrationId: 'google', credential: `enc:${JSON.stringify(credential)}`}; +} + +describe('unavailableActions', () => { + test('a Google sign-in that granted every scope covers every action', async () => { + const {unavailableActions} = await import('./unavailable'); + + expect(unavailableActions(google(ALL_SCOPES))).toEqual([]); + }); + + test('a Google sign-in without Drive leaves Read files out', async () => { + const {unavailableActions} = await import('./unavailable'); + + expect(unavailableActions(google(ALL_SCOPES.filter((scope) => !scope.endsWith('/drive'))))).toEqual(['readFiles']); + }); + + test('every other integration covers all its actions, whatever its credential', async () => { + const {unavailableActions} = await import('./unavailable'); + + expect(unavailableActions({id: 'email-1', integrationId: 'email', credential: 'v1.unreadable'})).toEqual([]); + expect(unavailableActions({id: 'info-1', integrationId: 'info', credential: null})).toEqual([]); + }); + + test("a Google credential that won't decrypt or read covers no action, and is logged", async () => { + const {findIntegration} = await import('@proxy/integrations'); + const {unavailableActions} = await import('./unavailable'); + const every = findIntegration('google')?.actions.map((action) => action.id); + + for (const credential of ['v1.other-key', 'enc:{"accessToken":1}', 'enc:not json', null]) { + expect(unavailableActions({id: 'google-1', integrationId: 'google', credential})).toEqual(every ?? []); + } + expect(warn).toHaveBeenCalledWith('A Google credential could not be read', {connectionId: 'google-1'}); + }); +}); + +describe('refuseUnavailable', () => { + test('refuses turning an unavailable action on, not off or back to the default', async () => { + const {findIntegration} = await import('@proxy/integrations'); + const {refuseUnavailable} = await import('./unavailable'); + const integration = findIntegration('google'); + if (!integration) { + throw new Error('No Google integration'); + } + + expect(refuseUnavailable(integration, ['readFiles'], {readFiles: true}).unwrapErr()).toMatchObject({ + kind: 'validation_error', + message: "Google didn't allow Read files at sign-in. Sign in to Google again to allow it.", + }); + expect(refuseUnavailable(integration, ['readFiles'], {readFiles: false}).isOk()).toBe(true); + expect(refuseUnavailable(integration, ['readFiles'], {readFiles: null}).isOk()).toBe(true); + expect(refuseUnavailable(integration, [], {readFiles: true}).isOk()).toBe(true); + }); +}); diff --git a/packages/api/src/connections/unavailable.ts b/packages/api/src/connections/unavailable.ts new file mode 100644 index 00000000..063de3cb --- /dev/null +++ b/packages/api/src/connections/unavailable.ts @@ -0,0 +1,48 @@ +import {Err, Ok, type Result} from 'ts-results-es'; + +import {ApiErr, type ApiError} from '@proxy/utils'; +import {findIntegration, type Integration} from '@proxy/integrations'; + +import {log, serializeError} from '../observability/log'; +import {readGoogleCredential} from './google/google-oauth'; +import {unavailableGoogleActions} from './google/scopes'; + +/** + * The actions a connection's sign-in didn't cover, which no default or agent setting turns on: + * Google's consent screen lets a person untick some of what is asked for, so its connections keep + * what Google granted. Every other integration's sign-in covers all its actions. + */ + +type StoredConnection = {id: string; integrationId: string; credential: string | null}; + +/** + * The connection's actions its sign-in didn't cover, in the catalog's order. A credential that + * can't be read, as after a change of `ENCRYPTION_KEY`, covers none, so the rest of the + * organization's connections are listed and used as before and this one can still be deleted. + */ +export function unavailableActions(connection: StoredConnection): string[] { + if (connection.integrationId !== 'google') { + return []; + } + const credential = readGoogleCredential(connection.credential); + if (credential.isErr()) { + log.warn('A Google credential could not be read', {connectionId: connection.id, ...('cause' in credential.error ? serializeError(credential.error.cause) : {})}); + return findIntegration(connection.integrationId)?.actions.map((action) => action.id) ?? []; + } + return unavailableGoogleActions(credential.value); +} + +/** The refusal of an action the sign-in didn't cover, saying how to allow it. */ +export function unavailableError(integration: Integration, actionId: string): ApiError { + const label = integration.actions.find((action) => action.id === actionId)?.label ?? actionId; + return ApiErr.validationError(`${integration.name} didn't allow ${label} at sign-in. Sign in to ${integration.name} again to allow it.`); +} + +/** Refuses turning on an action the sign-in didn't cover; turning one off, or back to the default, is fine. */ +export function refuseUnavailable(integration: Integration, unavailable: string[], actions: Record): Result { + const turnedOn = Object.keys(actions).find((id) => actions[id] === true && unavailable.includes(id)); + if (turnedOn !== undefined) { + return Err(unavailableError(integration, turnedOn)); + } + return Ok(undefined); +} diff --git a/packages/db/src/connection.ts b/packages/db/src/connection.ts index 3b954a36..70a8b91a 100644 --- a/packages/db/src/connection.ts +++ b/packages/db/src/connection.ts @@ -17,7 +17,8 @@ export type ConnectionRow = { defaults: Array<{actionId: string}>; }; -// Never the credential: only `getConnectionWithCredential` and `getConnectionCredential` read it. +// Never the credential: only `getConnectionWithCredential`, `listConnectionsWithCredential` and +// `getConnectionCredential` read it. const connectionSelect = { id: true, integrationId: true, @@ -38,6 +39,22 @@ export async function listConnections(orgId: string): Promise, ApiError>> { + return wrapDb(() => + db.connection.findMany({ + where: {orgId}, + select: {...connectionSelect, credential: true}, + orderBy: {createdAt: 'asc'}, + }), + ); +} + /** `Ok(null)` means no such connection in the organization. */ export async function getConnection( orgId: string, @@ -191,6 +208,28 @@ export async function updateConnectionName(data: { }); } +/** + * How many connections, in every organization, are for the same account as this one, itself + * included: those with the provider's id for it, or with its address, as one connected before ids + * were kept is known by. Zero once the connection is gone. + */ +export async function countSameAccountConnections( + connectionId: string, +): Promise> { + return wrapDb(async () => { + const connection = await db.connection.findUnique({ + where: {id: connectionId}, + select: {integrationId: true, account: true, providerAccountId: true}, + }); + if (!connection) { + return 0; + } + const {integrationId, account, providerAccountId} = connection; + const same = providerAccountId === null ? [{account}] : [{providerAccountId}, {account}]; + return db.connection.count({where: {integrationId, OR: same}}); + }); +} + /** `Ok(false)` means no such connection in the organization. Its defaults go with it. */ export async function deleteConnection( orgId: string, diff --git a/packages/integrations/src/access.test.ts b/packages/integrations/src/access.test.ts index 2c8242d5..8729efc8 100644 --- a/packages/integrations/src/access.test.ts +++ b/packages/integrations/src/access.test.ts @@ -62,6 +62,12 @@ describe('effectiveActions', () => { test('an action the catalog no longer has is dropped', () => { expect(effectiveActions(email, ['read', 'gone'])).toEqual(['read']); }); + + test("an action the sign-in didn't cover is off whatever is ticked, and so are those requiring it", () => { + expect(effectiveActions(email, ['read', 'archive'], {}, ['read'])).toEqual([]); + expect(effectiveActions(email, ['send'], {send: true}, ['send'])).toEqual([]); + expect(effectiveActions(email, ['read', 'send'], {}, ['send'])).toEqual(['read']); + }); }); describe('requiredAction', () => { diff --git a/packages/integrations/src/access.ts b/packages/integrations/src/access.ts index c4119e48..75238be3 100644 --- a/packages/integrations/src/access.ts +++ b/packages/integrations/src/access.ts @@ -9,15 +9,18 @@ export type OwnSettings = Partial>; /** * What an agent can do with a connection: its own setting for each action, else the default as it - * is now, leaving out an action whose required one is off, as Archive is without Read. + * is now, leaving out an action whose required one is off, as Archive is without Read, and the + * `unavailable` ones the connection's sign-in didn't cover, whatever is ticked. */ export function effectiveActions( integration: Integration, defaults: string[], own: OwnSettings = {}, + unavailable: string[] = [], ): string[] { const chosen = integration.actions.filter( - (action) => own[action.id] ?? defaults.includes(action.id), + (action) => + !unavailable.includes(action.id) && (own[action.id] ?? defaults.includes(action.id)), ); return chosen .filter((action) => !action.requires || chosen.some((other) => other.id === action.requires)) diff --git a/packages/integrations/src/catalog.ts b/packages/integrations/src/catalog.ts index 0d7541a3..e8da63ab 100644 --- a/packages/integrations/src/catalog.ts +++ b/packages/integrations/src/catalog.ts @@ -519,6 +519,9 @@ const google: Integration = { id: 'google', name: 'Google', description: 'Gmail, Drive, Docs, Sheets and Slides for one Google account', + // Google's Limited Use policy asks for what happens with its data to be said where access is given. + disclosure: + 'What is ticked here is what your agents can read and do with this Google account through Personal Agent Proxy, each time you ask them to. Personal Agent Proxy keeps only the sign-in and the activity log; nothing from Google is used to train a model.', actions: [ { id: 'readFiles', diff --git a/packages/integrations/src/types.ts b/packages/integrations/src/types.ts index 0e0dbd64..3f7ae269 100644 --- a/packages/integrations/src/types.ts +++ b/packages/integrations/src/types.ts @@ -89,6 +89,9 @@ export type Integration = { // Everything an agent can be allowed with a connection of it, in the order it is shown. actions: Action[]; collections: Collection[]; + // Shown above a connection's permissions: what ticking them hands the person's agents, where a + // provider's policy asks for it to be said. + disclosure?: string; // Information lives in Personal Agent Proxy itself: it is always there and never connected or // disconnected. builtIn?: boolean; diff --git a/packages/web/src/client/schemas.ts b/packages/web/src/client/schemas.ts index ab6842ff..9747cf0e 100644 --- a/packages/web/src/client/schemas.ts +++ b/packages/web/src/client/schemas.ts @@ -13,6 +13,8 @@ export const connectionSchema = z.object({ tools: z.array(mcpToolSchema).nullable(), // What the person called the connection, where they named it; null for every other integration. name: z.string().nullable(), + // The actions the connection's sign-in didn't cover: no agent gets them until it is signed in again. + unavailable: z.array(z.string()), }); export const agentSchema = z.object({ diff --git a/packages/web/src/components/connection-access.test.tsx b/packages/web/src/components/connection-access.test.tsx index 239ce99d..a43e5a9d 100644 --- a/packages/web/src/components/connection-access.test.tsx +++ b/packages/web/src/components/connection-access.test.tsx @@ -1,9 +1,9 @@ /// -import {afterEach, describe, expect, test} from 'bun:test'; +import {afterEach, describe, expect, mock, test} from 'bun:test'; import {act, cleanup, fireEvent, render, screen} from '@testing-library/react'; import {ConnectionAccess} from '@/components/connection-access'; -import type {Action} from '@proxy/integrations'; +import {findIntegration, type Action} from '@proxy/integrations'; afterEach(() => { cleanup(); @@ -220,3 +220,54 @@ describe('ConnectionAccess', () => { expect(changes).toEqual([{read: null}, {read: null}]); }); }); + +const actions = findIntegration('email')?.actions ?? []; + +describe('ConnectionAccess with what the sign-in covered', () => { + test("an action the sign-in didn't cover is off and disabled, saying how to allow it, whatever is ticked", () => { + render( + , + ); + + const read = screen.getByRole('checkbox', {name: /^Read/}); + expect(read.getAttribute('aria-checked')).toBe('false'); + expect(read.hasAttribute('data-disabled')).toBe(true); + expect(screen.getByText(/Not allowed at sign-in\. Sign in again to allow it\./)).toBeTruthy(); + + const archive = screen.getByRole('checkbox', {name: /^Archive/}); + expect(archive.hasAttribute('data-disabled')).toBe(false); + }); + + test("an agent's own setting for an action the sign-in didn't cover can be reset, not made the default", () => { + const onMakeDefault = mock(); + render( + , + ); + + expect( + screen.getAllByRole('button', {name: 'Make this the default for every agent'}), + ).toHaveLength(1); + expect(screen.getAllByRole('button', {name: 'Reset to default'})).toHaveLength(2); + }); + + test('every action can be set when the sign-in covered them all', () => { + render(); + + const read = screen.getByRole('checkbox', {name: /^Read/}); + expect(read.getAttribute('aria-checked')).toBe('true'); + expect(read.hasAttribute('data-disabled')).toBe(false); + expect(screen.queryByText(/Not allowed at sign-in/)).toBeNull(); + }); +}); diff --git a/packages/web/src/components/connection-access.tsx b/packages/web/src/components/connection-access.tsx index 2e25441a..3a7ca971 100644 --- a/packages/web/src/components/connection-access.tsx +++ b/packages/web/src/components/connection-access.tsx @@ -19,12 +19,16 @@ type Props = { onChange: (actions: Record) => Promise; // On an agent's page: makes the agent's own setting for an action the connection's default. onMakeDefault?: (actionId: string, allowed: boolean) => Promise; + // The actions the connection's sign-in didn't cover: off and disabled until it is signed in again. + unavailable?: string[]; }; // A connection's access, the way an editor shows its settings: one checkbox per action, the only // control for it. On an agent's page every action starts on the connection's default; one the // agent has its own setting for is marked with a bar, a button that makes it the default for -// every agent, and one that resets it to the default. +// every agent, and one that resets it to the default. An action the connection's sign-in didn't +// cover is off whatever is set, its checkbox disabled, and can't be made the default from here, +// since it shows off whatever the agent's own setting is. export function ConnectionAccess({ actions, defaults, @@ -32,6 +36,7 @@ export function ConnectionAccess({ differing, onChange, onMakeDefault, + unavailable = [], }: Props) { // A connection's actions show on more than one page section, so the ids need their own prefix. const idPrefix = useId(); @@ -43,8 +48,9 @@ export function ConnectionAccess({ defaultsRef.current = defaults; ownRef.current = own; const isDefault = (id: string) => defaults.includes(id); + const isUnavailable = (id: string) => unavailable.includes(id); const storedIsOn = (id: string) => (own ? (own[id] ?? isDefault(id)) : isDefault(id)); - const isOn = (id: string) => pending[id] ?? storedIsOn(id); + const isOn = (id: string) => !isUnavailable(id) && (pending[id] ?? storedIsOn(id)); const isOwn = (id: string) => own?.[id] !== undefined; // On an agent's page, an action that matches the default follows it rather than being stored. const setting = (id: string, next: boolean): boolean | null => { @@ -98,6 +104,7 @@ export function ConnectionAccess({ id={inputId} className="mt-0.5" checked={checked} + disabled={isUnavailable(action.id)} onCheckedChange={(next) => change(action.id, next)} /> {others > 0 && `Changed for ${others} ${others === 1 ? 'agent' : 'agents'}`} - {isOwn(action.id) && onMakeDefault && ( + {isOwn(action.id) && onMakeDefault && !isUnavailable(action.id) && ( @@ -145,3 +154,17 @@ export function ConnectionAccess({ ); } + +// Under a connection's permissions when its sign-in didn't cover some actions. Signing in again +// gives the same connection new tokens, its defaults and every agent's settings kept. +export function SignInAgain({href, className}: {href: string; className?: string}) { + return ( +

+ Some actions weren't allowed when this account was signed in.{' '} + + Sign in again + {' '} + to allow them. +

+ ); +} diff --git a/packages/web/src/components/integration-catalog.tsx b/packages/web/src/components/integration-catalog.tsx index c3df3333..f90b3842 100644 --- a/packages/web/src/components/integration-catalog.tsx +++ b/packages/web/src/components/integration-catalog.tsx @@ -4,16 +4,12 @@ import {cn} from '@proxy/ui/lib/utils'; import {PlusIcon} from 'lucide-react'; import {IntegrationLogo} from '@/components/brand-logo'; import {RowList} from '@/components/row-list'; -import {INTEGRATIONS} from '@/lib/integrations'; +import {INTEGRATIONS, signInPath} from '@/lib/integrations'; import type {Connection} from '@/lib/types'; -// The integrations connected by signing in to them. Their sign-in starts at -// `/api/connections//start`: the api sends the browser on to the service, and back to the new -// connection, or to the catalog with `?error=` when it didn't finish. Started with -// `?returnTo=welcome`, it comes back to the welcome flow's page for the connection, or its catalog. -// The rest are connected on a form of their own: a mailbox with an app password, a database with -// a user's name and password, a custom MCP server by its address. -const SIGN_IN: IntegrationId[] = ['google', 'granola', 'notion', 'linear']; +// The integrations not connected by signing in to them are connected on a form of their own: a +// mailbox with an app password, a database with a user's name and password, a custom MCP server by +// its address. // The whole row is the link or button; its plus is styled as a ghost icon button that lights up // with the row. @@ -37,7 +33,6 @@ export function IntegrationCatalog({ onConnectForm: (integrationId: IntegrationId) => void; }) { const available = INTEGRATIONS.filter((integration) => !integration.builtIn); - const startQuery = returnTo ? `?returnTo=${returnTo}` : ''; return ( @@ -45,6 +40,7 @@ export function IntegrationCatalog({ const count = connections.filter( (connection) => connection.integrationId === integration.id, ).length; + const signIn = signInPath(integration.id, {returnTo}); const content = ( <> @@ -64,8 +60,8 @@ export function IntegrationCatalog({ ); return (
  • - {SIGN_IN.includes(integration.id) ? ( - + {signIn ? ( + {content} ) : ( diff --git a/packages/web/src/lib/access.ts b/packages/web/src/lib/access.ts index 568643dd..6dc71e58 100644 --- a/packages/web/src/lib/access.ts +++ b/packages/web/src/lib/access.ts @@ -45,7 +45,8 @@ export function ownSettings(agent: AgentLogin, connectionId: string): OwnSetting ); } -// What the agent can do with the connection: its own settings, else the defaults. +// What the agent can do with the connection: its own settings, else the defaults, never what the +// connection's sign-in didn't cover. export function actionsFor(agent: AgentLogin, connection: Connection): string[] { const integration = integrationOf(connection); if (!integration) { @@ -55,6 +56,7 @@ export function actionsFor(agent: AgentLogin, connection: Connection): string[] integration, defaultActions(connection), ownSettings(agent, connection.id), + connection.unavailable, ); } diff --git a/packages/web/src/lib/integrations.ts b/packages/web/src/lib/integrations.ts index 1c7eb96b..0aaa267b 100644 --- a/packages/web/src/lib/integrations.ts +++ b/packages/web/src/lib/integrations.ts @@ -49,6 +49,29 @@ export function integrationOf(connection: { return {...integration, logo: LOGOS[integration.id]}; } +// The integrations connected by signing in to them. Their sign-in starts at +// `/api/connections//start`: the api sends the browser on to the service, and back to the new +// connection, or to the catalog with `?error=` when it didn't finish. Started with +// `?returnTo=welcome`, it comes back to the welcome flow's page for the connection, or its catalog; +// with `?account=`, as signing in again from a connection does, the service opens on that account +// where it can. +const SIGN_IN: IntegrationId[] = ['google', 'granola', 'notion', 'linear']; + +/** Where signing in to an integration starts, or null for one connected on a form of its own. */ +export function signInPath( + integrationId: string, + query: {returnTo?: 'welcome'; account?: string} = {}, +): string | null { + if (!SIGN_IN.some((id) => id === integrationId)) { + return null; + } + const params = new URLSearchParams( + Object.entries(query).flatMap(([key, value]) => (value === undefined ? [] : [[key, value]])), + ); + const search = params.size > 0 ? `?${params.toString()}` : ''; + return `/api/connections/${integrationId}/start${search}`; +} + /** What the catalog says when a sign-in came back with `?error=`. */ export function signInFailedMessage(integrationId: string | null): string | null { const failed = findIntegration(integrationId ?? ''); diff --git a/packages/web/src/pages/agent-page.tsx b/packages/web/src/pages/agent-page.tsx index 005216ee..be33d920 100644 --- a/packages/web/src/pages/agent-page.tsx +++ b/packages/web/src/pages/agent-page.tsx @@ -338,6 +338,7 @@ function AccessGrid({agent, connections, onChange, onMakeDefault}: GridProps) { onChange(connection.id, changes)} onMakeDefault={(actionId, allowed) => diff --git a/packages/web/src/pages/connection-page.tsx b/packages/web/src/pages/connection-page.tsx index 5e1f30fb..e689c877 100644 --- a/packages/web/src/pages/connection-page.tsx +++ b/packages/web/src/pages/connection-page.tsx @@ -10,7 +10,7 @@ import { import {AppShell, PageTitle} from '@/components/app-shell'; import {AuditList} from '@/components/audit-list'; import {BackButton} from '@/components/back-button'; -import {ConnectionAccess} from '@/components/connection-access'; +import {ConnectionAccess, SignInAgain} from '@/components/connection-access'; import {ConfirmDialog} from '@/components/confirm-dialog'; import {IconButton} from '@/components/icon-button'; import {AgentLogo, IntegrationLogo} from '@/components/brand-logo'; @@ -26,6 +26,7 @@ import { ownSettings, describeActions, } from '@/lib/access'; +import {signInPath} from '@/lib/integrations'; import {describeFetchError} from '@/lib/loader-utils'; import type {AgentLogin} from '@/lib/types'; import type {connectionLoader} from '@/loaders'; @@ -48,6 +49,10 @@ export function ConnectionPage() { } const withAccess = agentsWithAccess(agents, connection); + const signInAgain = + connection.unavailable.length > 0 + ? signInPath(connection.integrationId, {account: connection.account}) + : null; return ( + {integration.disclosure && ( +

    {integration.disclosure}

    + )} {integration.custom && integration.actions.length === 0 && (

    The server lists no tools.

    )} { const result = await setConnectionDefaults(connection.id, defaultsOnOrOff(actions)); @@ -125,6 +134,7 @@ export function ConnectionPage() { await revalidator.revalidate(); }} /> + {signInAgain && } {error &&

    {error}

    }
    diff --git a/packages/web/src/pages/welcome-page.tsx b/packages/web/src/pages/welcome-page.tsx index db2644b7..c13d2f11 100644 --- a/packages/web/src/pages/welcome-page.tsx +++ b/packages/web/src/pages/welcome-page.tsx @@ -1,4 +1,4 @@ -import {AGENT_PROVIDERS, type Action} from '@proxy/integrations'; +import {AGENT_PROVIDERS} from '@proxy/integrations'; import {useState, type ReactNode} from 'react'; import {Link, useLoaderData, useNavigate, useRevalidator, useSearchParams} from 'react-router'; import {setConnectionDefaults} from '@/client/connections-client'; @@ -8,13 +8,19 @@ import type {DatabaseDialect} from '@/client/connections-client'; import {ConnectDatabaseForm} from '@/components/connect-database-form'; import {ConnectEmailForm} from '@/components/connect-email-form'; import {ConnectMcpForm} from '@/components/connect-mcp-form'; -import {ConnectionAccess} from '@/components/connection-access'; +import {ConnectionAccess, SignInAgain} from '@/components/connection-access'; import {IntegrationCatalog} from '@/components/integration-catalog'; import {StepHeading} from '@/components/step-heading'; import {SuccessLine} from '@/components/success-line'; import {Button} from '@proxy/ui/components/button'; import {defaultActions, defaultsOnOrOff, integrationOf} from '@/lib/access'; -import {describeDatabase, EMAIL_FORM, signInFailedMessage} from '@/lib/integrations'; +import { + describeDatabase, + EMAIL_FORM, + signInFailedMessage, + signInPath, + type Integration, +} from '@/lib/integrations'; import {describeFetchError} from '@/lib/loader-utils'; import type {AgentLogin, Connection} from '@/lib/types'; import type {welcomeConnectionLoader, welcomeLoader} from '@/loaders'; @@ -115,7 +121,7 @@ export function WelcomeConnectionPage() { ); @@ -264,16 +270,20 @@ function McpStep({onContinue}: {onContinue: (connection: Connection) => void}) { function ConnectionStep({ agentName, connection, - actions, + integration, }: { agentName: string; connection: Connection; - actions: Action[]; + integration: Integration; }) { const navigate = useNavigate(); const {revalidate} = useRevalidator(); const [error, setError] = useState(null); const defaults = defaultActions(connection); + const signInAgain = + connection.unavailable.length > 0 + ? signInPath(connection.integrationId, {returnTo: 'welcome', account: connection.account}) + : null; return ( <> @@ -284,10 +294,14 @@ function ConnectionStep({ any time on the connection's page. Connection added successfully + {integration.disclosure && ( +

    {integration.disclosure}

    + )}
    { const result = await setConnectionDefaults(connection.id, defaultsOnOrOff(changes)); setError(result.isErr() ? describeFetchError(result.error) : null); @@ -295,7 +309,8 @@ function ConnectionStep({ }} />
    - {defaults.length === 0 && ( + {signInAgain && } + {defaults.every((id) => connection.unavailable.includes(id)) && (

    Nothing is ticked, so {agentName} can't use this account yet.