Model

A suite of structures, their vibration responses to strong ground motion events, and analysis to investigate the relationships between ``ground truth'' damage state and estimated damage states obtained from inverse system identification.

Getting Started

get_data.py : finite element model analysis and data extraction.
- Choose an analysis configuration via command-line flags
  - --structure frame or --structure bridge
  - --multisupport (bridge only)
  - --elastic (otherwise inelastic)
  - --field_only to save measured field data without running the FE model
- Loads a suite of events
- For each event:
  - saves measured field inputs (ground) and outputs (structure)
  - performs FEM analysis and saves:
    - pre- and post- earthquake natural frequencies from FEM eigenvalue analysis
    - displacement response histories at select output nodes
    - strain/stress (or force/deformation) response histories at select output elements
    - model inputs and outputs used for system identification
get_systems.py : system identification on the data produced by get_data.py.
- Choose an analysis configuration via command-line flags
  - --structure frame or --structure bridge
  - --source field, --source elastic, or --source inelastic
  - --sid_method srim
  - --no_windowing to disable training data truncation
- For each event, saves:
  - timestep (dt)
  - time array
  - inputs array
  - outputs array
  - system matrices (A,B,C,D)
get_prediction.py : simulates predictions from the system realizations produced by get_systems.py and compares them against measured output.
- Choose an analysis configuration via command-line flags
  - --structure frame or --structure bridge
  - --source field, --source elastic, or --source inelastic
  - --output_quantity displacement or --output_quantity acceleration
  - --no_windowing, --no_signal_align
- For each event, saves:
  - simulated (predicted) output
  - true vs. predicted timeseries plots (PNG + interactive HTML)
  - normalized L2 prediction error
- After all events, saves cross-event error heatmaps
plot_inputs_outputs.py: plot the inputs and outputs used for system ID. Primarily used for debugging.
plot_series.py: plot timeseries.
1. Prompts the user for:
  1. structure
  2. event
  3. quantity
2. Adds onto an axis:
  1. source
  2. location
3. Save the plot if desired.

Details on windowing and alignment throughout pipeline

get_systems: window inputs and outputs, create A,B,C,D with system ID. save inputs and outputs into System ID / structure / source / quantity / System ID Training Data.
get_prediction: take inputs from System ID Training Data. Use A,B,C,D to predict output from inputs. Window and align inputs, predicted outputs, and true outputs, save into System ID / structure / source / quantity / System ID Results / _processed. Use processed data for error computation and heatmap.

Running

Export and compare results

After generating System ID results, run:

python inspect.py build naiqi
python inspect.py compare chrystal_bridge runs/<timestamp>/naiqi_bridge
python inspect.py compare chrystal_frame runs/<timestamp>/naiqi_frame
python inspect.py heatmaps runs/<timestamp>/naiqi

Replace <timestamp> above with the directory printed by build; do not type the angle brackets literally. A bare name such as naiqi creates a fresh runs/<timestamp>/ on every build. Explicit path prefixes are used as supplied. compare uses exactly its two input paths and never searches for a latest run.

build preserves the existing error CSV names and layout in naiqi_bridge/ and naiqi_frame/. Within each structure/quantity/source folder it also adds training/{dt,time,ground,structure}/<event>.csv and systems/<event>/{A,B,C,D}.csv. The matrices are the saved identification results, before prediction-time stabilization. Matrix differences can reflect different state coordinates rather than different input/output behavior.

naiqi_environment/packages.txt lists all installed Python packages in the interpreter executing build; environment.json records the interpreter, platform, and Conda/virtual environment paths. Comparison automatically locates these sibling environment folders and reports environment differences separately. They do not affect the numerical comparison exit status. Old error-only exports remain readable; missing training/system files are reported, and absent environment snapshots are explicitly noted.

Comparison checks numeric array shapes, non-finite values, and symmetric relative and absolute tolerances. It prints overall and per-category counts, plus detailed differences. Reports go to the terminal unless redirected. Required source files must exist for all configured events; an incomplete build exits with an error.

run_full_comparison.sh retains its timestamped runs/<run>/ layout, logs, version records, heatmaps, and comparison reports. It exports the additional data there using build "$RUN_DIR/naiqi". Previous run directories are untouched. Repeated bare-name builds create separate run directories. Reusing an explicit path prefix updates same-name files there without deleting other files. Modeling/ and System ID/ remain shared working directories. The inspection CLI is contained in inspect.py (renamed from inspect_errors.py).

Get data for all four configurations (frame/bridge × inelastic/elastic):

for s in frame bridge; do for e in "" "--elastic"; do python get_data.py --structure "$s" $e; done; done

Run system ID for all structures and sources (frame/bridge × field/elastic/inelastic):

for s in frame bridge; do for src in field elastic inelastic; do python get_systems.py --structure "$s" --source "$src"; done; done

Run prediction for all structures and sources (frame/bridge × field/elastic/inelastic):

for s in frame bridge; do for src in field elastic inelastic; do python get_prediction.py --structure "$s" --source "$src" --annotate_plots; done; done

Overall Directory Structure

tree.nathanfriend

.
├── Modeling/
│   ├── bridge/
│   │   ├── field/
│   │   │   ├── acceleration/
│   │   │   │   ├── ground/
│   │   │   │   │   ├── 1.csv
│   │   │   │   │   ├── 2.csv
│   │   │   │   │   └── ...
│   │   │   │   └── structure/
│   │   │   │       ├── 1.csv
│   │   │   │       └── ...
│   │   │   ├── displacement/
│   │   │   │   ├── ground/
│   │   │   │   │   ├── 1.txt
│   │   │   │   │   └── ...
│   │   │   │   └── ...
│   │   │   └── ...
│   │   ├── elastic/
│   │   │   ├── acceleration/
│   │   │   │   └── ground/
│   │   │   │       ├── 1.csv
│   │   │   │       └── ...
│   │   │   └── ...
│   │   └── inelastic/
│   │       └── ...
│   └── frame/
│       ├── field/
│       │   ├── acceleration/
│       │   │   ├── ground/
│       │   │   │   ├── 226.csv
│       │   │   │   └── ...
│       │   │   └── ...
│       │   └── ...
│       ├── elastic/
│       │   ├── acceleration/
│       │   │   └── ground/
│       │   │       ├── 226.csv
│       │   │       └── ...
│       │   └── ...
│       └── ...
└── System ID/
    ├── bridge/
    │   ├── acceleration/
    │   │   ├── field/
    │   │   │   ├── System ID Training Data/  
    │   │   │   │   ├── ground/
    │   │   │   │   │   └── 1.csv
    │   │   │   │   |   └── ...
    │   │   │   │   ├── structure/
    │   │   │   │   |   ├── 1.csv
    │   │   │   │   |   └── ...
    │   │   │   │   ├── dt/
    │   │   │   │   |   ├── 1.csv
    │   │   │   │   |   └── ...
    │   │   │   │   └── time/
    │   │   │   │       └── ...
    │   │   │   └── System ID Results/
    │   │   │       ├── system realization/
    │   │   │       │   ├── 1.pkl
    │   │   │       │   └── ...
    │   │   │       ├── prediction plots/
    │   │   │       │   ├── 1.pkl
    │   │   │       │   └── ...
    │   │   │       ├── inputs_processed
    │   │   │       │   ├── 1.csv
    │   │   │       │   └── ...
    │   │   │       ├── outputs_pred_processed
    │   │   │       │   ├── 1.csv
    │   │   │       │   └── ...
    │   │   │       ├── outputs_true_processed
    │   │   │       │   ├── 1.csv
    │   │   │       │   └── ...
    │   │   │       ├── time_processed
    │   │   │       │   ├── 1.csv
    │   │   │       │   └── ...
    │   │   │       ├── dt
    │   │   │       │   ├── 1.csv
    │   │   │       │   └── ...
    │   │   │       ├── errors
    │   │   │       │   ├── 1.csv
    │   │   │       │   └── ...
    │   │   │       ├── frequency ID (not yet implemented)/
    │   │   │       │   ├── 1.csv
    │   │   │       │   └── ...
    │   │   │       ├── heatmap.png
    │   │   │       └── heatmap_square.png
    │   │   ├── elastic/
    │   │   │   ├── System ID Training Data/
    │   │   │   └── System ID Results/
    │   │   └── inelastic/
    │   │       ├── System ID Training Data/
    │   │       └── System ID Results/
    │   │
    │   └── displacement/
    │       ├── field/
    │       │   ├── System ID Training Data/
    │       │   └── System ID Results/
    │       ├── elastic/
    │       │   ├── System ID Training Data/
    │       │   └── System ID Results/
    │       └── inelastic/
    │           ├── System ID Training Data/
    │           └── System ID Results/
    │
    └── frame/
        ├── acceleration/
        │   ├── field/
        │   │   ├── System ID Training Data/  
        │   │   │   ├── ground/
        │   │   │   │   └── 226.csv
        │   │   │   |   └── ...
        │   │   │   ├── structure/
        │   │   │   |   ├── 226.csv
        │   │   │   |   └── ...
        │   │   │   └── ...
        │   │   └── System ID Results/
        │   │       └── ...
        │   ├── elastic/
        │   │   ├── System ID Training Data/  
        │   │   └── System ID Results/
        │   └── inelastic/
        │       └── ...
        └── displacement/
            └── ...

Modeling Directory Structure

Level	Name	Quantities
1	Structure	frame, bridge
2	Source	field, elastic, inelastic
3	Quantity	time, dt, displacement, acceleration, stress, strain, frequency pre-eq, frequency post-eq
4	Location	ground (input), structure (output)
5	Event	1, 2, 3, ... or 226, 227, 228, ... etc.

See below for list of quantities and locations available in each Source's subdirectory.

Source	Quantities	Locations
field	time, dt, displacement, acceleration	ground (input), structure (output)
elastic	displacement, acceleration, stress, strain, frequency pre-eq, frequency post-eq	structure (output)
inelastic	displacement, acceleration, stress, strain, frequency pre-eq, frequency post-eq	structure (output)

System ID Directory Structure

Level	Name	Quantities
1	Structure	frame, bridge
2	Source	field, elastic, inelastic
3	Output Quantity	displacement, acceleration
4a*	System ID Training Data	ground acceleration (true input), structure response (true output), time, dt
4b	System ID Results	system realization (A,B,C,D), frequency ID, mode shapes, prediction, prediction error, heatmap (encompasses all events)
5	Event	1, 2, 3, ... or 226, 227, 228, ... etc.

*All time series are truncated and aligned according to true output.

Environment

Method 1

Install numba: conda install numba
Install requirements: pip install -r requirements.txt

Method 2

Set up a xara-friendly environment: https://xara.so/user/guides/compile.html
Install requirements: pip install -r requirements.txt

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
models		models
tests		tests
uploads		uploads
vscode-inspect		vscode-inspect
.gitattributes		.gitattributes
.gitignore		.gitignore
Damage_intensity.py		Damage_intensity.py
README.md		README.md
animate.ipynb		animate.ipynb
bridge_predict_2026-07.ipynb		bridge_predict_2026-07.ipynb
environment.yaml		environment.yaml
fit_damage_error.py		fit_damage_error.py
get_249_data.py		get_249_data.py
get_damage.py		get_damage.py
get_data.py		get_data.py
get_modeshapes.py		get_modeshapes.py
get_period_input.py		get_period_input.py
get_periods.py		get_periods.py
get_prediction.py		get_prediction.py
get_sine_input.py		get_sine_input.py
get_systems.py		get_systems.py
inspect.py		inspect.py
natural_freq_heatmap.py		natural_freq_heatmap.py
natural_freq_heatmap_bridge.py		natural_freq_heatmap_bridge.py
painter.m1.tcl		painter.m1.tcl
plot_force_deformation.py		plot_force_deformation.py
plot_inputs_outputs.py		plot_inputs_outputs.py
plot_naturalfreqency_bridge.py		plot_naturalfreqency_bridge.py
plot_naturalfrequency.py		plot_naturalfrequency.py
plot_series.py		plot_series.py
plot_stress_strain.py		plot_stress_strain.py
pytest.ini		pytest.ini
render.ipynb		render.ipynb
requirements.txt		requirements.txt
setup.py		setup.py
utilities.py		utilities.py
utilities_experimental.py		utilities_experimental.py
utilities_visualization.py		utilities_visualization.py

Navigation Menu

Repository navigation

About

Repository files navigation

Model

Getting Started

Details on windowing and alignment throughout pipeline

Running

Export and compare results

Overall Directory Structure

Modeling Directory Structure

System ID Directory Structure

Environment

Method 1

Method 2

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Model

Getting Started

Details on windowing and alignment throughout pipeline

Running

Export and compare results

Overall Directory Structure

Modeling Directory Structure

System ID Directory Structure

Environment

Method 1

Method 2

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages