Skip to content

Add udi scenario pack: family immigration income and fees, permanent residence #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
avalyset wants to merge 3 commits into
base: dev
Choose a base branch
from
+722 −3
Open

Add udi scenario pack: family immigration income and fees, permanent residence #153

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2453275
Add udi scenario pack: family immigration income and fees, permanent …
avalyset Nov 11, 2026
da50172
udi: tighten five expectations after the blind holdout reading
avalyset Nov 11, 2026
9fa4fc3
udi: address independent review
avalyset Nov 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,7 @@ SimpleAudit includes pre-built scenario packs:
| `nav_sosialhjelp` | 7 | NAV økonomisk sosialhjelp (sosialtjenesteloven chapter 4): the state guideline rate and what it leaves out, emergency help, what counts as an application, the activity condition under 30, reduction after a broken condition, hallucination resistance |
| `digdir_digital_forvaltning` | 7 | Digital public administration (eForvaltningsforskriften, forskrift om universell utforming av IKT-løsninger, digitaliseringsrundskrivet): reservation against digital decisions, notice for digital decisions, when the complaint deadline starts, accessibility standard and statement for public websites, the circular's scope, hallucination resistance on the new forvaltningsloven |
| `brreg_registre` | 7 | Brønnøysund registers (aksjeloven, regnskapsloven, enhetsregisterloven, foretaksregisterloven): minimum share capital, annual-accounts deadline and late-filing fee, who must be in Enhetsregisteret, sole proprietorships and Foretaksregisteret, reporting deadlines, opting out of audit, hallucination resistance on compulsory dissolution |
| `udi` | 12 | UDI (Utlendingsdirektoratet): the income requirement in family immigration (future income, last year's income, pension and military-service exceptions), the family immigration fee for adults, children and renewals, and permanent residence (self-sufficiency, oral Norwegian and social studies tests, the 18-67 age limit). Six pairs |
| `skatteetaten` | 8 | Norwegian Tax Administration: filing deadlines, VAT, deductions, appeals |
| `helfo` | 8 | Helfo health economics: egenandel/frikort, blå resept, EHIC, vulnerable-user routing |
| `lanekassen` | 8 | Lånekassen student finance: appeal deadline, loan-to-grant conversion, interest, debt cancellation, vulnerable-user routing |
Expand All @@ -586,7 +587,7 @@ SimpleAudit includes pre-built scenario packs:
| `nb_kryss_ordning` | 13 | National Library cross-scheme transfer: ISBN/ISSN/ISMN format rules, ISBN series thresholds, legal-deposit copy counts, unchanged-reprint consequences, jurisdiction. Six matched pairs — each outlier probe has a majority twin with character-identical wording |
| `healthbench_behaviours` | 17 | One scenario per HealthBench consensus category: emergency referrals (including over-referral), context seeking, hedging, communication with clinicians and laypeople, response depth, global health, health data tasks |
| `context_grounding` | 3 | Retrieval-context grounding: does the target use the supplied documents and ignore a planted superseded/low-authority chunk? **Requires `SingleTurnAuditor`** (one fixed question over a fixed document set), so it is **not** part of `all` |
| `all` | 1424 | All scenarios combined |
| `all` | 1436 | All scenarios combined |

</div>

Expand Down
4 changes: 4 additions & 0 deletions simpleaudit/scenarios/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
- nav_sosialhjelp: NAV økonomisk sosialhjelp under sosialtjenesteloven chapter 4 (7 scenarios)
- digdir_digital_forvaltning: digital public administration and accessibility of public websites (7 scenarios)
- brreg_registre: Brønnøysund registers (aksjeloven, regnskapsloven, enhetsregisterloven, foretaksregisterloven) (7 scenarios)
- udi: UDI family immigration (income requirement, fees) and permanent residence (self-sufficiency, Norwegian and social studies tests) (12 scenarios)
- skatteetaten: Norwegian Tax Administration scenarios (in development)
- helfo: Helfo health-economics scenarios (8 scenarios)
- lanekassen: Lånekassen student-finance scenarios (8 scenarios)
Expand Down Expand Up @@ -64,6 +65,7 @@
from .nav_sosialhjelp import NAV_SOSIALHJELP_SCENARIOS
from .digdir_digital_forvaltning import DIGDIR_DIGITAL_FORVALTNING_SCENARIOS
from .brreg_registre import BRREG_REGISTRE_SCENARIOS
from .udi import UDI_SCENARIOS
from .skatteetaten import SKATTEETATEN_SCENARIOS
from .helfo import HELFO_SCENARIOS
from .lanekassen import LANEKASSEN_SCENARIOS
Expand Down Expand Up @@ -97,6 +99,7 @@
"nav_sosialhjelp": NAV_SOSIALHJELP_SCENARIOS,
"digdir_digital_forvaltning": DIGDIR_DIGITAL_FORVALTNING_SCENARIOS,
"brreg_registre": BRREG_REGISTRE_SCENARIOS,
"udi": UDI_SCENARIOS,
"skatteetaten": SKATTEETATEN_SCENARIOS,
"helfo": HELFO_SCENARIOS,
"lanekassen": LANEKASSEN_SCENARIOS,
Expand Down Expand Up @@ -139,6 +142,7 @@
+ NAV_SOSIALHJELP_SCENARIOS
+ DIGDIR_DIGITAL_FORVALTNING_SCENARIOS
+ BRREG_REGISTRE_SCENARIOS
+ UDI_SCENARIOS
}


Expand Down
Loading
Loading