From cadcc9c7d1fb5bfd9c33622a7d4e53d504cdb5ca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rodrigo=20M=C3=A9ndez?= Date: Thu, 10 Sep 2026 12:12:57 -0600 Subject: [PATCH 1/3] docs: add ADR for authorization schema source tracking Adds 0025 ADR describing how authorization schema definitions and their source are tracked. Part 1 of the schema loading pipeline stack. --- ...5-authorization-schema-source-tracking.rst | 146 ++++++++++++++++++ 1 file changed, 146 insertions(+) create mode 100644 docs/decisions/0025-authorization-schema-source-tracking.rst diff --git a/docs/decisions/0025-authorization-schema-source-tracking.rst b/docs/decisions/0025-authorization-schema-source-tracking.rst new file mode 100644 index 00000000..4233694e --- /dev/null +++ b/docs/decisions/0025-authorization-schema-source-tracking.rst @@ -0,0 +1,146 @@ +0025: Track the Source of Compiled Authorization Definitions +############################################################ + +Status +****** + +**Draft** + +Context +******* + +`ADR 0019`_ discovers static schema resources from multiple applications and `ADR 0018`_ +compiles them into policy rows during deployment. Several requirements need to know *where* +each compiled definition came from: + +* Multiple applications may contribute to the same role. When a module adds a permission to a + built-in role (for example ``courses.export_grades`` on ``course_admin``), the system must be + able to tell the core-provided grants apart from the module-provided grant even though both + live in the same role. +* Operators and developers benefit from seeing which application contributed a role or + permission, for debugging and auditing. +* A future capability to remove an application should be able to drop only the definitions that + application provided, while shared definitions remain. + +Today none of this is stored. Compiled definitions exist only as Casbin ``p`` rows, which encode +``role, action, scope, effect`` and carry no display metadata and no origin. + +Two representations are possible for the origin: an extra field on the Casbin policy row, or a +value held only in the in-memory registry. Both were rejected. A policy-row field cannot hold +more than one contributing source, risks interfering with the enforcement matcher, and cannot +attribute display metadata or categories, which are not policy rows. An in-memory value does not +survive restarts, cannot be shared across processes, and cannot support idempotent re-deploys or +future removal. + +Decision +******** + +1. Store compiled definitions and their sources in dedicated tables +==================================================================== + +The schema loader persists compiled definitions in first-class tables owned by ``openedx-authz``. +Casbin ``p`` rows remain the enforcement representation and are rendered from these tables in the +same transaction; the definition tables are the authoritative record that the API reads and that +deployment diffs. + +The definition tables are: + +* a permission-category table (stable id and display fields); +* a permission-definition table (namespace and name forming the complete permission id, plus + display fields, category, and supported scopes); +* a role-definition table (stable role id, display fields, supported scopes, and the ``hidden`` + flag from `ADR 0023`_); and +* a role-permission table holding one row per ``(role, permission, scope)`` relationship. + +The role-permission relationship is the atomic unit of attribution, because it corresponds one +to one with a rendered ``p`` row and is where role extensions take effect. + +2. Attribute sources through an explicit many-to-many link +=========================================================== + +A source table records each distinct contribution. Its identity is ``(distribution, module)`` — +the installed distribution and the Python module that owns the resource. The file path is stored +as a non-identifying attribute only, so moving a definition between files within the same module +does not change its source and produces no add/remove churn. A content digest may be recorded for +diagnostics, but change detection relies on diffing compiled definitions rather than on the +digest, so the digest is advisory. + +Every definition and every role-permission relationship links to one or more sources through +explicit link tables. Each link records whether the contribution was a base definition or an +extension, and the contributing file's priority so the winning metadata source is derivable. The +common case is a single link; the many-to-many exists to represent shared ownership and to make +removal precise. + +All sources are treated equally. ``openedx-authz`` is a schema provider like any other +distribution, so there is no core-versus-module flag; callers that care about a particular origin +compare the distribution name directly. + +3. Extending a built-in role keeps both origins distinct +======================================================== + +Because attribution lives at the role-permission grain, a core grant and a module-added grant on +the same role remain individually attributed. For ``course_admin``: + +* the role definition links to the distribution that defines it, as a base contribution; +* each core permission links to that same distribution as a base contribution; and +* ``courses.export_grades`` links to the contributing module as an extension. + +The two coexist in one role, yet each relationship row carries its own origin. If two applications +add the same permission to the same role, the single relationship row gains two source links. + +4. Attribution is queryable and may be exposed by the API +========================================================= + +Given any role or permission, its origin can be queried from the definition and link tables — for +a role as a whole, for a single permission, or for a specific role-permission grant. The +authorization definition API (`ADR 0021`_) may expose these sources so a client such as the +Administrative Console can show which application contributed a role or permission. Exposing the +sources is an additive, optional API change and is not required by this decision. + +5. Metadata changes update definitions in place +================================================ + +When a source file changes a display name, description, icon, or similar field, the next +deployment recompiles and updates the existing definition row, keyed by its stable identifier. No +new definition row is created and relationships and assignments are unaffected. + +6. Adopt pre-existing policy rows; leave unmanaged rows untouched +================================================================= + +On the first deployment after this feature ships, existing Casbin ``p`` rows are adopted rather +than duplicated: for each rendered ``(role, permission, scope)`` that already exists as a policy +row without a definition record, the loader creates the definition and relationship rows and links +them to the contributing source. A pre-existing policy row that no schema declares is left in place +and enforceable, but is not attributed and does not appear in the definition tables. Pruning such +unmanaged rows is out of scope. + +Consequences +************ + +* Compiled definitions, including display metadata that previously had no home, are persisted and + readable through the API. +* The origin of any role, permission, or individual role-permission grant is queryable, and core + and module contributions to the same role remain distinguishable. +* Moving a definition between files in the same module does not change its recorded source. +* Removing an application becomes tractable: relationships and definitions whose only source is the + removed application can be pruned, while shared ones remain. Removal itself remains out of scope + and follows the assignment-safety rules of `ADR 0018`_. +* The existing ``ExtendedCasbinRule`` model is not reused; it continues to describe role + assignments (``g`` rows), while the new tables own definitions and provenance. +* Compilation must track provenance at the role-permission grain, and the apply step writes the + definition, relationship, and source tables in the same transaction as the policy rows. + +References +********** + +* `ADR 0017`_ +* `ADR 0018`_ +* `ADR 0019`_ +* `ADR 0021`_ +* `ADR 0023`_ + +.. _ADR 0017: 0017-static-authorization-schema.rst +.. _ADR 0018: 0018-authorization-schema-lifecycle.rst +.. _ADR 0019: 0019-authorization-schema-discovery.rst +.. _ADR 0021: 0021-authorization-definition-api.rst +.. _ADR 0023: 0023-extend-static-roles.rst From 408aca53098b633092766a4a1e27eef0b4dd4711 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rodrigo=20M=C3=A9ndez?= Date: Tue, 15 Sep 2026 16:09:05 -0600 Subject: [PATCH 2/3] squash!: Improvements based on comments --- ...5-authorization-schema-source-tracking.rst | 50 ++++++++++++------- 1 file changed, 33 insertions(+), 17 deletions(-) diff --git a/docs/decisions/0025-authorization-schema-source-tracking.rst b/docs/decisions/0025-authorization-schema-source-tracking.rst index 4233694e..df729e35 100644 --- a/docs/decisions/0025-authorization-schema-source-tracking.rst +++ b/docs/decisions/0025-authorization-schema-source-tracking.rst @@ -1,5 +1,5 @@ -0025: Track the Source of Compiled Authorization Definitions -############################################################ +0025: Track the Source of Compiled Static Authorization Definitions +################################################################### Status ****** @@ -9,13 +9,21 @@ Status Context ******* +.. note:: + + This ADR concerns only **static** authorization definitions — roles and permissions declared in + YAML schema files and compiled into policy rows during deployment. It does not cover **dynamic** + definitions created at runtime through the API or written directly to the database. Source + tracking for dynamically created definitions, if needed, is out of scope and left to a future + decision. + `ADR 0019`_ discovers static schema resources from multiple applications and `ADR 0018`_ compiles them into policy rows during deployment. Several requirements need to know *where* each compiled definition came from: * Multiple applications may contribute to the same role. When a module adds a permission to a - built-in role (for example ``courses.export_grades`` on ``course_admin``), the system must be - able to tell the core-provided grants apart from the module-provided grant even though both + pre-existing role (for example ``courses.export_grades`` on ``course_admin``), the system must be + able to tell the grants apart between the modules that defined them, even though both live in the same role. * Operators and developers benefit from seeing which application contributed a role or permission, for debugging and auditing. @@ -38,7 +46,8 @@ Decision 1. Store compiled definitions and their sources in dedicated tables ==================================================================== -The schema loader persists compiled definitions in first-class tables owned by ``openedx-authz``. +The schema loader persists definitions compiled from static YAML schemas in first-class tables +owned by ``openedx-authz``. Casbin ``p`` rows remain the enforcement representation and are rendered from these tables in the same transaction; the definition tables are the authoritative record that the API reads and that deployment diffs. @@ -71,18 +80,20 @@ extension, and the contributing file's priority so the winning metadata source i common case is a single link; the many-to-many exists to represent shared ownership and to make removal precise. -All sources are treated equally. ``openedx-authz`` is a schema provider like any other -distribution, so there is no core-versus-module flag; callers that care about a particular origin -compare the distribution name directly. +All sources are treated equally. ``openedx-authz`` is a schema provider like any other module, so +there is no core-versus-module flag; every definition originates from some module, including those +that ship with ``openedx-authz`` itself. Callers that care about a particular origin compare the +distribution name directly. -3. Extending a built-in role keeps both origins distinct -======================================================== +3. Extending a role keeps both origins distinct +=============================================== -Because attribution lives at the role-permission grain, a core grant and a module-added grant on -the same role remain individually attributed. For ``course_admin``: +Because attribution lives at the role-permission grain, the grant that first defined a role and a +later grant added by another module remain individually attributed. For ``course_admin``, assuming +``openedx-authz`` defines it and its base permissions: -* the role definition links to the distribution that defines it, as a base contribution; -* each core permission links to that same distribution as a base contribution; and +* the role definition links to the module that defines it, as a base contribution; +* each base permission links to that same module as a base contribution; and * ``courses.export_grades`` links to the contributing module as an extension. The two coexist in one role, yet each relationship row carries its own origin. If two applications @@ -97,6 +108,11 @@ authorization definition API (`ADR 0021`_) may expose these sources so a client Administrative Console can show which application contributed a role or permission. Exposing the sources is an additive, optional API change and is not required by this decision. +YAML schema files will always be the source of truth, display names and descriptions in the database +will always follow what the schema defines. This means that any translation strings derived from the +YAML files will match the text that the API will query, so the API can use those for applying +translations. The details on this mechanism is out of scope for this ADR. + 5. Metadata changes update definitions in place ================================================ @@ -107,7 +123,7 @@ new definition row is created and relationships and assignments are unaffected. 6. Adopt pre-existing policy rows; leave unmanaged rows untouched ================================================================= -On the first deployment after this feature ships, existing Casbin ``p`` rows are adopted rather +On deployment, when policies are loaded, existing Casbin ``p`` rows are adopted rather than duplicated: for each rendered ``(role, permission, scope)`` that already exists as a policy row without a definition record, the loader creates the definition and relationship rows and links them to the contributing source. A pre-existing policy row that no schema declares is left in place @@ -119,8 +135,8 @@ Consequences * Compiled definitions, including display metadata that previously had no home, are persisted and readable through the API. -* The origin of any role, permission, or individual role-permission grant is queryable, and core - and module contributions to the same role remain distinguishable. +* The origin of any role, permission, or individual role-permission grant is queryable, and the + contributions of different modules to the same role remain distinguishable. * Moving a definition between files in the same module does not change its recorded source. * Removing an application becomes tractable: relationships and definitions whose only source is the removed application can be pruned, while shared ones remain. Removal itself remains out of scope From c06af6e37adb071c640b8564e19021d5eebea587 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rodrigo=20M=C3=A9ndez?= Date: Tue, 15 Sep 2026 17:03:44 -0600 Subject: [PATCH 3/3] squash!: Improve table definitions --- ...5-authorization-schema-source-tracking.rst | 109 ++++++++++++++---- 1 file changed, 89 insertions(+), 20 deletions(-) diff --git a/docs/decisions/0025-authorization-schema-source-tracking.rst b/docs/decisions/0025-authorization-schema-source-tracking.rst index df729e35..3e9c025c 100644 --- a/docs/decisions/0025-authorization-schema-source-tracking.rst +++ b/docs/decisions/0025-authorization-schema-source-tracking.rst @@ -40,6 +40,14 @@ attribute display metadata or categories, which are not policy rows. An in-memor survive restarts, cannot be shared across processes, and cannot support idempotent re-deploys or future removal. +Reusing the existing ``ExtendedCasbinRule`` model was also considered and rejected. It was an +early candidate, but it is geared toward role assignments (``g`` rows) and is one-to-one with its +corresponding ``CasbinRule``. Schema source tracking has the opposite shape: a single role or +permission can have more than one contributing source, which a one-to-one model cannot represent. +Overloading a model built for assignments to also carry definition provenance would blur two +distinct concerns; keeping them separate leaves ``ExtendedCasbinRule`` focused on assignments and +gives definitions dedicated tables. + Decision ******** @@ -47,19 +55,20 @@ Decision ==================================================================== The schema loader persists definitions compiled from static YAML schemas in first-class tables -owned by ``openedx-authz``. -Casbin ``p`` rows remain the enforcement representation and are rendered from these tables in the -same transaction; the definition tables are the authoritative record that the API reads and that -deployment diffs. +owned by ``openedx-authz``. Casbin ``p`` rows remain the enforcement representation and are rendered +from these tables in the same transaction; the definition tables are the authoritative record that +the API reads and that deployment diffs. The definition tables are: -* a permission-category table (stable id and display fields); -* a permission-definition table (namespace and name forming the complete permission id, plus - display fields, category, and supported scopes); -* a role-definition table (stable role id, display fields, supported scopes, and the ``hidden`` - flag from `ADR 0023`_); and -* a role-permission table holding one row per ``(role, permission, scope)`` relationship. +* a **permission-category** table — a stable category id and display fields (display name, + description, icon). Categories group permissions for display and grant no access on their own; +* a **permission-definition** table — the complete permission id (a namespace and a name), display + fields, its category, and the supported scopes; +* a **role-definition** table — a stable role id, display fields, supported scopes, and the + ``hidden`` flag from `ADR 0023`_; and +* a **role-permission** table — one row per ``(role, permission, scope)`` grant, relating a role + definition and a permission definition. The role-permission relationship is the atomic unit of attribution, because it corresponds one to one with a rendered ``p`` row and is where role extensions take effect. @@ -67,24 +76,82 @@ to one with a rendered ``p`` row and is where role extensions take effect. 2. Attribute sources through an explicit many-to-many link =========================================================== -A source table records each distinct contribution. Its identity is ``(distribution, module)`` — -the installed distribution and the Python module that owns the resource. The file path is stored +A **source** table records each distinct contribution. Its identity is ``(distribution, module)`` — +the installed distribution and the code module that owns the resource. The resource path is stored as a non-identifying attribute only, so moving a definition between files within the same module does not change its source and produces no add/remove churn. A content digest may be recorded for -diagnostics, but change detection relies on diffing compiled definitions rather than on the -digest, so the digest is advisory. +diagnostics, but change detection relies on diffing compiled definitions rather than on the digest, +so the digest is advisory. Every definition and every role-permission relationship links to one or more sources through -explicit link tables. Each link records whether the contribution was a base definition or an -extension, and the contributing file's priority so the winning metadata source is derivable. The -common case is a single link; the many-to-many exists to represent shared ownership and to make -removal precise. +explicit link tables — one per definition kind (category, permission, role, and role-permission). +Each link records whether the contribution was a base definition or an extension, and the +contributing file's priority so the winning metadata source is derivable. The common case is a +single link; the many-to-many exists to represent shared ownership and to make removal precise. All sources are treated equally. ``openedx-authz`` is a schema provider like any other module, so there is no core-versus-module flag; every definition originates from some module, including those that ship with ``openedx-authz`` itself. Callers that care about a particular origin compare the distribution name directly. +Data model +---------- + +The relationships between the entities are shown below. Four definition entities hold the compiled +schema; a single source entity records each distinct contribution; and one link entity per +definition kind attributes definitions to sources, each carrying the origin (base or extension) and +priority of the contribution. + +.. code-block:: text + + Permission Category ─┐ + │ │ (base/extension, priority) + │ groups └──────────────┐ + ▼ │ + Permission Definition ──────────────┤ + │ │ + │ related by a │ ┌──────────────┐ + │ Role-Permission grant ├─────►│ Source │ + ▼ │ │ (distribution,│ + Role Definition ────────────────────┤ │ module) │ + │ │ └──────────────┘ + │ has grants │ + ▼ │ + Role-Permission grant ──────────────┘ + (role, permission, scope) + → one rendered Casbin ``p`` row + + Each definition kind links to one or more Sources through its own link entity; + every link carries the contribution's origin (base or extension) and priority. + +Each entity's purpose: + +.. list-table:: + :header-rows: 1 + :widths: 28 72 + + * - Entity + - Purpose + * - Permission category + - Display/grouping category for permissions. Grants no access; carries display metadata only. + * - Permission definition + - A compiled permission, identified by its namespace and name, with display metadata, its + category, and supported scopes. + * - Role definition + - A compiled role, identified by a stable role id, with display metadata, supported scopes, and + the ``hidden`` flag from `ADR 0023`_. + * - Role-permission grant + - One ``(role, permission, scope)`` grant. Corresponds one-to-one with a rendered Casbin ``p`` + row and is the atomic unit of attribution. + * - Source + - A distinct contribution, identified by ``(distribution, module)``. Resource path and content + digest are advisory, non-identifying attributes. + * - Definition-source links + - One link entity per definition kind (category, permission, role, and role-permission), + joining each definition to its contributing sources. Each link records the origin (base or + extension) and priority, so the winning metadata source is derivable and shared ownership is + representable. + 3. Extending a role keeps both origins distinct =============================================== @@ -141,8 +208,10 @@ Consequences * Removing an application becomes tractable: relationships and definitions whose only source is the removed application can be pruned, while shared ones remain. Removal itself remains out of scope and follows the assignment-safety rules of `ADR 0018`_. -* The existing ``ExtendedCasbinRule`` model is not reused; it continues to describe role - assignments (``g`` rows), while the new tables own definitions and provenance. +* The existing ``ExtendedCasbinRule`` model is not reused, for the reasons given in the Context: it + is one-to-one with its ``CasbinRule`` and geared toward assignments, whereas a definition may + have multiple sources. It continues to describe role assignments (``g`` rows), while the new + tables own definitions and provenance, keeping the two concerns separated. * Compilation must track provenance at the role-permission grain, and the apply step writes the definition, relationship, and source tables in the same transaction as the policy rows.