Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/pypi.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Publish to PyPI

# Runs only when a GitHub Release is published (not on push or PR).
# Release flow: bump `version` in pyproject.toml and src/verdi/__init__.py,
# Release flow: bump `version` in pyproject.toml and src/verdi_agents/__init__.py,
# merge, then publish a release tagged v<version> (e.g. v0.1.1).
on:
release:
Expand All @@ -21,8 +21,8 @@ jobs:
run: |
tag="${GITHUB_REF_NAME#v}"
pkg=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")
mod=$(python -c "import verdi; print(verdi.__version__)")
echo "tag=$tag pyproject=$pkg verdi.__version__=$mod"
mod=$(python -c "import verdi_agents; print(verdi_agents.__version__)")
echo "tag=$tag pyproject=$pkg verdi_agents.__version__=$mod"
if [ "$tag" != "$pkg" ] || [ "$tag" != "$mod" ]; then
echo "::error::Release tag v$tag does not match pyproject ($pkg) / __version__ ($mod)"
exit 1
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,6 @@ artifacts/

# GitHub Pages build output (site/build.py)
_site/

# Local working notes (not published)
notes/
176 changes: 0 additions & 176 deletions AUDIT.md

This file was deleted.

268 changes: 115 additions & 153 deletions README.md

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions docs/INSTITUTIONAL_RECOVERY.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ Configure an OpenAI-compatible endpoint as described in the
[README](../README.md), then run:

```bash
python -m verdi.experiments.paired_run --config examples/paired_institutional_recovery.yaml
python -m verdi.analysis.outcome_metrics --pair-root runs/paired-institutional-recovery
verdi-agents pair examples/paired_institutional_recovery.yaml
verdi-agents compare runs/paired-institutional-recovery
```

The paired runner copies a common base scenario into fixed and revisable
Expand Down Expand Up @@ -56,7 +56,7 @@ constitution:
```

These restrictions complement immutable resource invariants. Policy
violations appear in the event ledger with a `constitution:<rule>` reason.
violations appear in the event log (`events.jsonl`) with a `constitution:<rule>` reason.
An action blocked by a policy is not automatically a punitive sanction.

Other amendment keys can be recorded as governance or narrative rules but
Expand All @@ -79,7 +79,7 @@ demonstrate the relevant scenario settings.
The outcome analysis includes verified objective completion, attempted and
committed resource flows, reserve levels, inequality, exclusion episodes,
institutional decisions, and recovery observations. Aggregates include
references to the underlying event ledger where applicable; runs without
references to the underlying event log where applicable; runs without
adequate evidence should not be interpreted as zeros.

A passed amendment is a governance outcome, not proof of improved recovery.
Expand Down
39 changes: 19 additions & 20 deletions docs/RESEARCHER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ recover from disruption.

Adapted from [INSTITUTIONAL_RECOVERY.md](INSTITUTIONAL_RECOVERY.md):

- **No validated causal finding exists yet.** A live end-to-end smoke run has
not been executed as part of this branch; treat all results as provisional.
- **No validated causal finding exists yet.** Treat all results as provisional
until they hold up over repeated runs.
- **Seeding does not make LLM outputs deterministic.** Seeds make the
*simulator* reproducible (scheduling, order, world state), but model
generations are still stochastic. Paired repetitions across seeds — not a
Expand All @@ -35,7 +35,7 @@ Adapted from [INSTITUTIONAL_RECOVERY.md](INSTITUTIONAL_RECOVERY.md):
enforceable policy keys (`max_transfer_units`, `max_withdraw_units`,
`min_commons_reserve`, `max_produce_units_per_tick`) change real behavior.
Prose changes about fairness or rights stay descriptive metadata.
- **Probe self-reports are exploratory**, not evidence that actions complied
- **Value-check self-reports are exploratory**, not evidence that actions complied
with a value. Primary outcomes should come from committed world state
(delivery objectives), not agent claims.
- **Future-dated version bookkeeping** for multiple overlapping amendments
Expand All @@ -49,23 +49,23 @@ Adapted from [INSTITUTIONAL_RECOVERY.md](INSTITUTIONAL_RECOVERY.md):

### Prerequisites

- Python 3.10+ (CI uses 3.11)
- An OpenAI-compatible model endpoint (Qwen on vLLM in the examples)
- Python 3.10+
- A language model server that speaks the OpenAI API (for example vLLM or Ollama)

```bash
python -m venv .venv && source .venv/bin/activate
pip install -e .
pip install verdi-agents
verdi-agents init my-study && cd my-study
```

### Configure the endpoint

Copy the example environment file and fill in your values. **Never commit
real secrets**; `.env` is gitignored. See [.env.example](../.env.example):
`verdi-agents init` created a `.env` file. Put your key in it (**never share or
commit this file**). The model name and server address go in the scenario's
`model:` block.

```bash
cp .env.example .env
# edit .env: set MODEL_API_KEY. Model name + base_url live in the scenario `model:` block.
source .env
# edit .env: set MODEL_API_KEY
# edit examples/institutional_recovery_constrained.yaml: set model.base_url and model.name
```

| Variable | Meaning |
Expand All @@ -76,7 +76,7 @@ source .env
### Run the paired experiment — ONE command

```bash
python -m verdi.experiments.paired_run --config examples/paired_institutional_recovery.yaml
verdi-agents pair examples/paired_institutional_recovery.yaml
```

That's the whole workflow. The runner:
Expand All @@ -99,7 +99,7 @@ runs/paired-institutional-recovery/
├── fixed-seed42/
│ ├── report.html # open this first
│ ├── scenario_resolved.yaml # the exact frozen scenario used
│ ├── events.jsonl # immutable event ledger
│ ├── events.jsonl # event log: every action, in order
│ └── ... # metrics.json, timeline.md, etc.
└── revisable-seed42/
├── report.html
Expand All @@ -113,8 +113,7 @@ you which pairs are trustworthy.
Optional CLI overrides (no config edit needed):

```bash
python -m verdi.experiments.paired_run \
--config examples/paired_institutional_recovery.yaml \
verdi-agents pair examples/paired_institutional_recovery.yaml \
--seeds 42,7,99 \
--output-root runs/my-experiment \
--concurrency 4
Expand Down Expand Up @@ -153,7 +152,7 @@ Unknown keys are rejected with a readable error listing allowed keys.
| `constitution.rules` | Initial typed rules: `quorum_rule`, `max_withdraw_units`, `min_commons_reserve`, … plus `quorum` and `amendment_threshold` |
| `economy` | Opt-in constrained economy (versioned): per-resource `production_limit_per_tick`, `production_cost`, `consumption_per_tick`. Absent → unlimited free production |
| `tasks.deliveries` | Objective delivery goals evaluated from committed world state alone (`objective_id`, `start_tick`, `due_tick`, `resource`, `amount`, `target_pool`) |
| `institutional_condition` | Do NOT edit by hand for comparisons — the paired runner sets it per arm |
| `institutional_condition` | Do NOT edit by hand for comparisons: `verdi-agents pair` sets it for each half |

### "What if" recipes

Expand Down Expand Up @@ -214,22 +213,22 @@ interpret differences carefully — the intervention is *enactment*, not
|----------|---------------------|
| `runs/paired-institutional-recovery/<arm>-seed<N>/report.html` | What happened in one run, interactively: society, resources, governance, constitution history |
| `pair_manifest.json` | Which pairs completed/failed/stalled; final state hashes; replay integrity |
| Per-run `events.jsonl` / `metrics.json` / `timeline.md` | Raw evidence behind any aggregate: ledgers, counts, tick-by-tick story |
| Per-run `events.jsonl` / `metrics.json` / `timeline.md` | Raw evidence behind any aggregate: event log, counts, tick-by-tick story |
| Per-run `scenario_resolved.yaml` | The exact frozen scenario — reproducibility proof |
| `comparison.html`, `metrics.json`, `metrics.csv` | Cross-condition comparison and machine-readable outcomes after running the metrics command |

After the paired run completes, generate the comparison report and
machine-readable metrics with:

```bash
python -m verdi.analysis.outcome_metrics --pair-root runs/paired-institutional-recovery
verdi-agents compare runs/paired-institutional-recovery
```

This writes `comparison.html`, `metrics.json`, and `metrics.csv`
at the paired output root. The comparison includes objective completion,
committed and attempted resource flows, exclusion episodes, institutional
mechanisms, and recovery measurements where there is enough recorded
evidence. Inspect the individual `events.jsonl` ledgers for provenance.
evidence. Inspect the individual `events.jsonl` event logs to see where a number came from.

Interpretation guardrails:

Expand Down
13 changes: 6 additions & 7 deletions docs/SCENARIOS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
A scenario is one YAML (or JSON) file. It is the **frozen input** of a run: the
engine reads nothing else about the experiment, and each run records the model
it used. Run with
`python -m verdi.society.live --scenario <file> --run-dir runs/<name>`.
`verdi-agents run <file>`.

## Files

Expand All @@ -12,10 +12,9 @@ it used. Run with
| [`examples/starter.yaml`](../examples/starter.yaml) | Smallest working scenario: 4 agents, 20 ticks, 1 meeting. |
| [`examples/full.yaml`](../examples/full.yaml) | Every knob, commented: model, shocks, law, constitution, economy, delivery tasks, interviews, lifecycle, sanctions, rights. |
| [`examples/institutional_recovery_constrained.yaml`](../examples/institutional_recovery_constrained.yaml) | Constrained-economy base for the fixed-vs-revisable study. |
| [`examples/paired_institutional_recovery.yaml`](../examples/paired_institutional_recovery.yaml) | Not a scenario: drives both arms of the paired study via `python -m verdi.experiments.paired_run`. See [RESEARCHER_GUIDE.md](RESEARCHER_GUIDE.md). |
| [`examples/paired_institutional_recovery.yaml`](../examples/paired_institutional_recovery.yaml) | Not a scenario: input for `verdi-agents pair`, which runs the same society with fixed and with changeable rules. See [RESEARCHER_GUIDE.md](RESEARCHER_GUIDE.md). |
| [`examples/sweeps/`](../examples/sweeps/) | Sweep files: rerun a scenario across models, seeds and settings. See [SWEEPS.md](SWEEPS.md). |
| `experiments/scenarios/pilot_v1.json` | Frozen t0-t11 pilot slice used by the pilot driver and G5 pipeline. |
| `tests/fixtures/scenarios/` | Test inputs only (Society V2 stage fixtures, a 50-tick minimal, the institutional smoke). Not user-facing examples. |
| `tests/fixtures/` | Inputs used by the automated tests only. Not meant as examples. |

Both `starter.yaml` and `full.yaml` are loaded and run offline by
`tests/test_example_scenarios.py`.
Expand All @@ -24,18 +23,18 @@ Both `starter.yaml` and `full.yaml` are loaded and run offline by

| Key | Required | Meaning |
|-----|----------|---------|
| `model` | for live runs | `base_url`, `name`; optional `api_key_env` (default `MODEL_API_KEY`), `timeout_seconds`, `max_retries`. A literal `api_key` is rejected. Without a block the run falls back to legacy `VERDI_VLLM_*` env and warns. |
| `model` | for live runs | `base_url`, `name`; optional `api_key_env` (default `MODEL_API_KEY`), `timeout_seconds`, `max_retries`. A literal `api_key` is rejected. Without a block the run falls back to the older `VERDI_VLLM_*` environment variables and warns. |
| `world_id`, `total_ticks`, `run_seed` | yes | Identity, length, determinism seed. |
| `agents` | yes | `agent_id`, `role`, `persona_summary`, `declared_values`, `resources`, `social_edges`; optional `under_sanction` + `sanction.duration_ticks`, `rights`. |
| `world_resources` | recommended | Shared commons at t0. |
| `assemblies` | no | Ticks of constitutional meetings. |
| `assemblies` | no | Ticks at which agents meet to vote on rule changes. |
| `institutional_condition` | no | `revisable` or `fixed`. |
| `inner_life` | no | Private inner-life state per agent. |
| `shocks` | no | `{tick, resource, delta}` list. |
| `law`, `constitution` | no | Invariants, quorum, amendment threshold, sanction/appeal policy, typed `rules`. |
| `economy` | no | Opt-in production caps/costs and per-tick consumption. |
| `tasks.deliveries` | no | Objective delivery oracles checked from committed state. |
| `lifecycle` | no | Probe phases (`tick: {phase: name}`); auto-derived when absent. |
| `lifecycle` | no | When to run value checks (`tick: {phase: name}`); chosen automatically when absent. |
| `interviews` | no | `ticks` and/or `phases`: longitudinal interview checkpoints. |

`examples/full.yaml` is the authoritative worked example of every key above.
10 changes: 4 additions & 6 deletions docs/SWEEPS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ The baseline scenario file is never edited.

```bash
# 1. See the plan and the cost, and test each model with one tiny call. Runs no simulation.
python -m verdi.sweep examples/sweeps/model_comparison.yaml --dry-run
verdi-agents sweep examples/sweeps/model_comparison.yaml --dry-run

# 2. Run it. It shows the plan again and asks before starting.
python -m verdi.sweep examples/sweeps/model_comparison.yaml
verdi-agents sweep examples/sweeps/model_comparison.yaml

# 3. Open the results.
open runs/sweeps/model_comparison/comparison.html
Expand All @@ -26,8 +26,6 @@ on runs that would fail. Skip it with `--skip-check`.
If the sweep stops (laptop closed, Ctrl-C, endpoint down), run command 2
again. Finished runs are kept and only the missing or failed ones run.

`verdi sweep ...` (or `uvx verdi-agents sweep ...` without installing) works the same as `python -m verdi.sweep ...`.

## Writing a sweep file

Copy [examples/sweeps/model_comparison.yaml](../examples/sweeps/model_comparison.yaml)
Expand Down Expand Up @@ -99,7 +97,7 @@ See [examples/sweeps/governance_grid.yaml](../examples/sweeps/governance_grid.ya
## Quick sweeps without a file

```bash
python -m verdi.sweep --scenario examples/full.yaml \
verdi-agents sweep --scenario examples/full.yaml \
--vary model.name=default,Borealis2-preview \
--vary constitution.quorum=0.5,0.75 \
--parallel-runs 2 --dry-run
Expand Down Expand Up @@ -129,7 +127,7 @@ was not serving what you asked for. It also has token counts, so you can see
the cost per condition.

To rebuild `summary.csv` and `comparison.html` without running anything:
`python -m verdi.sweep <file> --results`.
`verdi-agents sweep <file> --results`.

## Good practice

Expand Down
90 changes: 0 additions & 90 deletions docs/concordia_eval.md

This file was deleted.

Loading
Loading