RampNet: A Two-Stage Pipeline for Bootstrapping Curb Ramp Detection in Streetscape Images from Open Government Metadata

John S. O'Meara, Jared Hwang, Zeyu Wang, Michael Saugstad, Jon E. Froehlich

University of Washington

💻 Code / 📄 Paper / 🛠️ Demo / 🗃️ Dataset / 🏨 ICCV'25 Workshop

RampNet is a two-stage pipeline that addresses the scarcity of curb ramp detection datasets by using government location data to automatically generate over 210,000 annotated Google Street View panoramas. This new dataset is then used to train a state-of-the-art curb ramp detection model that significantly outperforms previous efforts. In this repo, we provide code for training and testing our system.

Note

Looking for a fixed version of this repository? There are two tagged releases: v1.0-iccv2025 is the repository frozen at paper state — use it to reproduce the ICCV'25 paper exactly as written. v1.1-corrected-eval is the same code and the same model weights with the corrected evaluation protocol (see the Erratum below) — use it to score RampNet's model under standard one-to-one matching. The main branch keeps evolving (post-paper analyses, benchmarks, and groundwork for what comes next), while the published model and dataset on Hugging Face are versioned independently and match the paper.

Citation

If you use our code, dataset, or build on ideas in our paper, please cite us as:

@inproceedings{omeara2025rampnet,
  author    = {John S. O'Meara and Jared Hwang and Zeyu Wang and Michael Saugstad and Jon E. Froehlich},
  title     = {{RampNet: A Two-Stage Pipeline for Bootstrapping Curb Ramp Detection in Streetscape Images from Open Government Metadata}},
  booktitle = {{ICCV'25 Workshop on Vision Foundation Models and Generative AI for Accessibility: Challenges and Opportunities (ICCV 2025 Workshop)}},
  year      = {2025},
  doi       = {https://doi.org/10.48550/arXiv.2508.09415},
  url       = {https://cv4a11y.github.io/ICCV2025/index.html},
  note      = {DOI: forthcoming}
}

Erratum: Evaluation Metrics (July 2026)

After publication, we found that the evaluation protocol described in §3.3 of the paper — and implemented in the evaluators at tag v1.0-iccv2025 — differs from standard detection evaluation in two ways, both of which bias precision and recall upward:

1. One detection could count as multiple true positives. A single predicted point within the matching radius of two ground-truth curb ramps (a common configuration — dual-ramp corners, pedestrian islands) was credited with both. 41% of gold-set ground-truth points sit within 2× the matching radius of a neighbor.
2. Redundant detections were ignored rather than counted as false positives. A second detection of an already-matched ramp appeared in neither the TP nor the FP column; standard (VOC/COCO-style) protocols count it as a false positive.

Additionally, the Stage 1 and Stage 2 evaluators implemented different matching rules, so the paper's direct Stage 1 vs. Stage 2 comparison mixed protocols.

We re-evaluated the released model on the same 1,000-panorama gold set with greedy one-to-one matching (each detection claims at most one ground-truth point; duplicates are false positives), now implemented in rampnet/metrics.py and used by stage_two/evaluate.py:

Metric (gold set, TTA, conf ≥ 0.55)	As published	Corrected
Model precision	0.938	0.949
Model recall	0.935	0.873
Model AP	0.9236¹	0.9205²

¹ Uninterpolated AP under the paper's matching protocol. ² Interpolated AP under one-to-one matching — the conventions differ, so compare with care.

Swapping only the matching rule on identical model outputs moves precision by −1.0 points and recall by −4.4 points; the remaining difference is reproduction drift (environment, JPEG re-encoding). The paper's Stage 1 dataset-agreement precision (94.0%) is also slightly optimistic because redundant points were ignored rather than counted as false positives; its recall is unaffected. That number has not yet been re-measured, and when it is, two changes will move it, not one: stage_one/dataset_evaluation/evaluate.py now counts redundant points as false positives and matches through the same rampnet/metrics.py core as Stage 2 — nearest unclaimed ground truth rather than first-in-list-order — so the two evaluators no longer implement different rules (see issue #18). The comparison with prior work is unaffected (both systems were scored under the same protocol, and the gap is far larger than the correction).

Full analysis — including the exact published code, executable traces, and visual examples of double-counted detections — is in docs/eval_protocol_verification.html (open in a browser) and issue #9. Corrected result curves and metrics are committed in stage_two/evaluation_results_new/; the as-published results remain unchanged in stage_two/evaluation_results/.

Dataset note: the 360° seam (August 2026)

A panorama wraps: the left and right edges of an equirectangular image are the same place. Stage 1's label generator extracts curb ramp locations from a 4096×2048 equirectangular heatmap with peak_local_max(min_distance=40), and that suppression does not carry across the wrap. A ramp sitting on the seam can therefore produce a peak on each edge and be labelled twice.

Measured across the whole published dataset: 8,361 seam-crossing label pairs among 849,904 labels (0.98%), affecting 7,987 of 214,385 panoramas (3.7%) — 17.2× more than a uniform-azimuth null predicts. Most pairs are one ramp labelled twice, but not all: a 14-pair human adjudication puts roughly one pair in seven as two genuinely adjacent ramps, so they cannot simply all be merged. Independently of the azimuth null, panoramas containing such a pair carry on average +0.591 more labels than they have source government ramp records, where panoramas without one carry −0.408 — a gap of +1.000 label per pair (z = +14.0).

We are not correcting the published 1.0 dataset. It is the artifact the paper's numbers were computed on; replacing it would mean a downloader gets different data than the paper used, which we think is worse for reproducibility than a documented defect.

The seam also costs the detector some response, though less than it costs the labels. Rolling a panorama so the seam falls elsewhere moves the model's activation at the ramp by more than 0.05 for 12 of 25 benchmark ramps within ~4° of the seam — raising it for 9, lowering it for 3 — against 0 of 77 control ramps in the same panoramas (19 of the 25 are gold-set). The effect varies with how much of a ramp falls on each side of the split; in this sample no ramp's response sat below the detection threshold with the seam in place (a separate detection-level check found 24 of 25 seam-band ramps detected), but a shift of this size would flip a ramp whose response is already marginal. The evaluation path itself is unaffected.

Separately, the 1,000-panorama gold set carries 10 human-adjudicated duplicate marks of its own, which slightly understate recall for every model scored on it; corrected figures will follow when that set is re-scored.

Full analysis — including two findings we initially reported and then retracted — is in docs/seam.md and issue #132.

Curb Ramp Detection Example

For a step-by-step walkthrough, see our Google Colab notebook, which includes a visualization in addition to the code below.

For basic usage of our detection model, you do not need to be working within the RampNet project directory or use any custom libraries. However, we strongly recommend using a GPU. See code example below:

import torch
from transformers import AutoModel
from PIL import Image
import numpy as np
from torchvision import transforms
from skimage.feature import peak_local_max

IMAGE_PATH = "example.jpg"
DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model = AutoModel.from_pretrained("projectsidewalk/rampnet-model", trust_remote_code=True).to(DEVICE).eval()

preprocess = transforms.Compose([
    transforms.Resize((2048, 4096), interpolation=transforms.InterpolationMode.BILINEAR),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])

img = Image.open(IMAGE_PATH).convert("RGB")
img_tensor = preprocess(img).unsqueeze(0).to(DEVICE)

with torch.no_grad():
    heatmap = model(img_tensor).squeeze().cpu().numpy()

peaks = peak_local_max(np.clip(heatmap, 0, 1), min_distance=10, threshold_abs=0.5)
scale_w = img.width / heatmap.shape[1]
scale_h = img.height / heatmap.shape[0]
coordinates = [(int(c * scale_w), int(r * scale_h)) for r, c in peaks]

# Coordinates of detected curb ramps
print(coordinates)

Predicted Heatmap	Extracted Points

Google Colab Notebook

Choosing a Detection Threshold

Different parts of this repo use different peak-extraction thresholds (threshold_abs passed to peak_local_max), which has caused confusion. Here is the provenance:

• stage_two/evaluate.py uses PEAK_THRESHOLD_ABS = 0.0 by design. It collects all heatmap peaks and sweeps the confidence axis to generate the full precision–recall and precision/recall-vs-confidence curves. It is not an operating point.
• The 0.5 in the example above and the 0.4 in stage_two/demo.py are illustrative visualization choices, not tuned values.
• A principled default operating point is 0.55, which achieves precision 0.949 / recall 0.873 on the 1,000-panorama manually labeled gold set under corrected one-to-one matching (see the Erratum; the originally published figures at this operating point were P 0.938 / R 0.935). You can read this (or any other operating point) directly from the committed curve data in stage_two/evaluation_results_new/pr_rc_vs_c_data_manual_r0.022_pt0.0.csv.

Important caveat — test-time augmentation. All committed evaluation curves were computed with horizontal-flip TTA: the panorama is evaluated twice (original and mirrored) and the two heatmaps are combined with an elementwise max (see stage_two/evaluate.py). If you deploy the model with single-pass inference (as in the quick-start example above), expect performance somewhat below these curves, and derive your own threshold curve without TTA before choosing an operating point.

Picking a per-city / per-deployment operating point. Curb ramp appearance and imagery vary by city, so a threshold tuned on our gold set (NYC, Portland, Bend) may not transfer. The recipe: manually label ~100 panoramas from your target area in the manual_labels/ format, point stage_two/evaluate.py at them, and read the threshold that meets your precision or recall requirement from the resulting pr_rc_vs_c_data_*.csv.

---

We now describe how to generate the dataset (Stage 1) and train the model (Stage 2). We also describe how to evaluate both of these stages.

Environment Setup

Create the conda environment (Linux with an NVIDIA GPU; CUDA 12.6 builds are selected automatically — for CPU-only or macOS, remove the cuda-version line from environment.yml):

conda env create -f environment.yml
conda activate sidewalkcv2
pip install -e .

The pip install -e . step installs the small shared rampnet package (model definition, checkpoint loading, evaluation metrics) that the stage 1 and stage 2 scripts import.

Alternatives:

• requirements.txt — the same dependency set for pip/venv/Colab users.
• environment.lock.yml — the exact full conda export used for the paper results (linux-64 only), kept for provenance. Note that despite the paper-era README saying "CUDA 11.8", the lock actually pins CUDA 12.6 pytorch builds.

Running the Tests

A pytest suite in tests/ covers the shared rampnet package — the model definition and its checkpoint compatibility, the evaluation metrics and prediction/ground-truth matching — along with the benchmark bundles and the Hugging Face export tooling. It is CPU-only, needs no network, and reads only fixtures committed to this repo, so it takes about 30 seconds:

pytest -q

If you created the conda environment above, you already have everything it needs. For a minimal install that skips the training, geo, and plotting stack:

pip install "torch>=2.6,<3" "torchvision>=0.21" --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements-dev.txt
pip install -e .

Install torch and torchvision together from the CPU index as shown. On Linux, PyPI's torchvision wheel is built against a CUDA torch, and mixing it with a CPU-only torch fails at import with operator torchvision::nms does not exist.

GitHub Actions runs the same suite on Python 3.10 and 3.12 for every pull request (.github/workflows/tests.yml). Because CI installs CPU-only pip wheels rather than the conda environment, a green run verifies the code — not that environment.yml still solves or that the CUDA builds are intact.

The suite does not train, evaluate a checkpoint, or touch the Google Street View endpoints; those runs are far too slow and network-dependent to gate a commit on, and are documented in the stage sections below.

Dataset Summary

Name	Description	# of Panoramas	# of Labels
Open Government Datasets	The initial source of curb ramp locations (<lat, long> coordinates) from 3 US cities (NYC, Portland, Bend) with "Good" location precision. Used as input for Stage 1.	N/A (Geo-data)	276,615¹
Project Sidewalk Crop Pre-training Set	A subset of Project Sidewalk data used to initially pre-train the crop-level model in Stage 1, which identifies curb ramps within a small, directional image crop. Published as rampnet-crop-model-dataset-round1 — prefer it over re-running stage_one/crop_model/ps_model/data/download_data.py, which reads the live, still-growing Project Sidewalk servers and builds a different set.	20,698	27,704
Manual Crop Model Training Set	A small, fully and manually labeled dataset used for a second round of training on the crop-level model to improve its precision and recall.	312	1,212
RampNet Stage 1 Dataset (Final Output)	The main, large-scale dataset generated by the Stage 1 auto-translation pipeline, containing curb ramp pixel coordinates on GSV panoramas. This is the primary dataset contribution.	214,376	849,895
Manual Ground Truth Set (1k Panos)	A set of 1,000 panoramas randomly sampled and then fully and manually labeled. This serves as the "gold standard" for evaluating both Stage 1 and Stage 2 performance. Images are included in the Stage 1 Dataset on Hugging Face, but the labels themselves are in manual_labels.	1,000	3,919

¹This number is the sum of curb ramp locations from the three cities with "Good" location precision listed in Table 1: New York City (217,680), Portland (45,324), and Bend (13,611). The inventory files now committed under stage_one/dataset_generation/location_data/ hold 276,071 records, 544 fewer — the gap is between Table 1 and the files, not anywhere in the pipeline, and docs/data_provenance.md §3.3 reconciles what is known about it. Quote 276,071 for anything derived from the committed inputs.

Provenance note: see docs/data_provenance.md for the registry of which cities' data entered training (evaluations in those cities are optimistically biased), the undocumented Google endpoints the regeneration pipeline depends on, and why the HuggingFace dataset — not a re-run of split_dataset.py — is the split of record for the paper.

Published Artifacts

Everything RampNet publishes lives under the projectsidewalk organisation on Hugging Face, indexed by the RampNet collection. This table is the map from "what an experiment needs" to "where it is", so that no input is discoverable only by asking us.

artifact	where	notes
Stage 2 model weights	rampnet-model	the curb ramp detector
Stage 1 generated dataset	rampnet-dataset	214k annotated panoramas, 463 GB; the split of record for the paper
Round-1 PS crops	rampnet-crop-model-dataset-round1	27,704 crops, 13.37 GB Parquet; the filename-encoded labels made a real column
Round-2 manual crops	rampnet-crop-model-dataset-round2	1,212 crops for the second crop-model round; renamed from rampnet-crop-model-dataset 2026-08-05, the old id redirects
Government curb ramp inventories	this repo, stage_one/dataset_generation/location_data/	the exact NYC/Portland/Bend files the paper ran on, sha256-pinned
Street centrelines	this repo, stage_one/dataset_generation/street_data/*.min.geojson.gz	18.7 MB derivative of the 801 MB downloads, proven equivalent
City boundaries	this repo, stage_one/dataset_generation/cityboundaries.geojson
Gold-set labels	this repo, manual_labels/	1,000 panoramas, YOLO-format points
Benchmark detections + verdicts	this repo, benchmark/	per-split records, human verdicts, and the rubrics they were made under
Crop-model checkpoints (Stage 1)	rampnet-crop-model	rounds 1 and 2; Stage 1 does not run without round 2
Stage 1 manifests, raw street data	rampnet-stage1-inputs	finaldataset.jsonl is the exact manifest the paper consumed
Benchmark panoramas and ground truth ‡	rampnet-benchmark	11.41 GB Parquet; configs records (the ground truth — score a model without cloning anything), native, 4096x2048, galleries (#21)

‡ Post-publication, not part of the paper. Every other row above is a paper-era artifact. The 9-city benchmark was built from 2026-07 onward to evaluate the published model out of domain; the paper's own evaluation was the 1,000-panorama gold set in manual_labels/. See benchmark/README.md.

What is not yet published is stated as plainly as what is. docs/replication.md is the ledger — for every input, whether someone outside the lab can obtain it, and where they are blocked if they cannot. docs/data_provenance.md records where each input came from, its hash, and the caveats that limit re-running Stage 1.

Dataset Setup

Before reproducing our results, certain datasets will need to be downloaded.

• City curb ramp location data. We use NYC, Bend, and Portland.
  • In stage_one/dataset_generation/location_data, there should be three files
    • bend.geojson
    • nyc.csv (using csv here as NYC uses different file format than geojson)
    • portland.geojson
  • These three files are now committed to this repo — no download needed. They are the exact files the paper ran on, hash-pinned in docs/data_provenance.md. Prefer them over the hyperlinks above: the portals serve current data and it drifts, so a fresh download reproduces a different experiment.
• City Street Data. We use this when generating null panos (picking a random street until we find one with no curb ramp nearby)
  • Also committed — no download needed, as stage_one/dataset_generation/street_data/<city> - Streets.min.geojson.gz. The raw downloads total 801 MB, so what is committed is an 18.7 MB derivative holding only the geometry and name field the pipeline reads; scripts/build_street_derivative.py verify proves it yields an identical street network. generate_negative_panos.py uses a full download instead if you have one.
  • The originals, should you want them:
    • Bend - Streets.geojson
    • New York - Streets.geojson
    • Portland - Streets.geojson
• We also need cityboundaries.geojson file in stage_one/dataset_generation for negative pano generation. It included in this repo - no download needed.
• The tiny set of manually labeled crops can be downloaded here. The test, train, and val folders belong in stage_one/crop_model/ps_and_manual_model/dataset_1
• Manual annotations for evaluation of both stages (included in this repo, no download needed). Note that while we include the manual annotations in this repo, the images themselves are not included because they are assumed to be included in the dataset that will be generated.

If you only wish to setup for Stage 2, then you can download our Stage 1-generated dataset here or using the download_dataset.py script in the project directory.

Stage 1: Dataset Generation

We detail how to reproduce our Stage 1 results. Please ensure you have downloaded all the necessary files before proceeding with this step.

Getting the Crop Model Ready

The crop model is the model that takes in a crop that faces a curb ramp and localized where the curb ramp is within that crop. It is crucial to our auto-translation technique and must be trained before we can proceed with dataset generation.

In stage_one/crop_model/ps_model, we will initiate our first round of training. In stage_one/crop_model/ps_and_manual_model, we will follow up with a final round that trains on manual data.

In stage_one/crop_model/ps_model/data, run download_data.py. This will take a very long time. You should have a resulting directory called dataset_1. Run ./splititup.sh dataset_1 to split the dataset into its test, train, and val splits.

In stage_one/crop_model/ps_model/model, run train.py. This will take a very long time. In the code, the number of epochs is set to 100 for comprehensiveness, but we suggest training for no more than 25 epochs. You should have a resulting file called best_model.pth.

Now, we will transition into the second round of training on manual data. Copy best_model.pth from the aforementioned process into stage_one/crop_model/ps_and_manual_model. Instead of best_model.pth, rename it to ps_model.pth in this folder.

In stage_one/crop_model/ps_and_manual_model, run train.py. This will take a very long time. In the code, the number of epochs is set to 100 for comprehensiveness, but we suggest training for no more than 25 epochs. You should have a resulting file called best_model.pth. This is what we will use to auto-translate government location data to pixel coordinates in street view panoramas.

(Optional step) If you want to evaluate the crop model, use evaluate.py.

Preparing for Auto-Translation

In this section, we will exclusively work in the stage_one/dataset_generation folder.

First, run combine_location_data.py. You will get a resulting all_locations.csv file.

Next, run generate_dataset_meta.py. This will probably take a long time. You will get a resulting dataset.jsonl file. IMPORTANT: This dataset.jsonl file does not yet contain null panos. The next step describes how we infuse our dataset with null panos.

Next, run generate_negative_panos.py. This will probably take a long time. You will get a resulting negativepanos.jsonl file. It is up to your discretion on how many of these null panos you want to include in your dataset. We did 20% in our paper. Create a finaldataset.jsonl file that contains lines from both the aforementioned dataset.jsonl file and the negativepanos.jsonl file. If you want 20% of the panos to be null panos, then 20% of the lines in finaldataset.jsonl should be from the negativepanos.jsonl file.

Auto-Translating Government Location Data to Pixel Coordinates

We now must download the GSV panoramas and convert government location data to pixel coordinates. Run the download_dataset.py file. This will take a very long time as there is hundreds of thousands of panos that need to be downloaded. In the end, we should have a dataset folder that is created at the root of the RampNet project folder.

Splitting the Generated Dataset

As discussed in our paper, this splitting step must be performed carefully to avoid data leakage. Specifically, we must ensure that no panoramas used for manual evaluation are included in the training or validation splits. While this does not affect dataset evaluation (since it is conducted independently of the splits), it could compromise model evaluation in Stage 2.

Users also must take care to avoid including the same curb ramps in different panoramas/viewpoints. We have build a custom script called split_dataset.py that takes care of this. After running it, you will have a folder called dataset_split next to the original dataset folder. Delete the old dataset folder and rename dataset_split to dataset.

IMPORTANT: If you plan to use this generated dataset for training the next stage and intend to rely on the same manual labels we created, do not use the split_dataset.py script as-is. Because it performs a random split, there is a risk of data leakage. Specifically, panoramas selected for manual evaluation could end up in the training or validation sets. In such cases, you must use a modified version of the script that explicitly excludes these manually evaluated panoramas from the training and validation splits. There is a variable in split_dataset.py called CONSIDER_MANUAL that should be set to True if you are planning on doing this.

Evaluating the Results

Run evaluate.py in stage_one/dataset_evaluation:

Precision (TP / (TP + FP)): 0.9403
Recall    (TP / Total GT):  0.9245

These are the as-published Stage 1 agreement numbers. Note that the precision figure is slightly optimistic — redundant generated points near an already-matched ground-truth ramp were ignored rather than counted as false positives (see the Erratum); the recall figure is unaffected.

A re-run will not reproduce these numbers. The current script differs from the published one in two ways: redundant points now count as false positives, and matching now goes through the shared rampnet/metrics.py core (nearest unclaimed ground truth, where the published script took the first ground-truth ramp in list order). Both changes are expected to be small, and they can pull in opposite directions — the first lowers precision, the second can raise it by assigning points to ramps more sensibly. Note the second change moves recall as well: which ground-truth ramps end up claimed can differ, so the true-positive count itself shifts. Attribute any corrected figures to both changes, not to redundancy alone. Re-measuring needs the generated Stage 1 dataset, which is not in git; see issue #18.

Stage 2: Curb Ramp Detection

We detail how to reproduce our Stage 2 results.

You can either start where you left off in Stage 1, with the dataset fully generated, or you can skip that process and download our full dataset here or using the download_dataset.py script in the project directory.

Training

Run train.py in stage_two (see python train.py --help for options). This will take a very long time (> 24 hours). We trained on 16x NVIDIA L40s GPUs on a slurm cluster. We train for only one epoch (--epochs) but you may increase this if you desire. The model is saved at best_model.pth.

To fine-tune from existing weights (e.g. the released RampNet model, or a previous run's checkpoint) instead of training from ImageNet initialization:

python train.py --preset finetune --init-weights path/to/checkpoint.pth

The finetune preset lowers the learning rate to 3e-6 (override with --lr). Note that an existing latest_checkpoint.pth resume file always takes precedence over --init-weights — delete it if you intend to start a fresh fine-tuning run.

Evaluating the Results

There are two benchmarks you can evaluate against: (1) the test split of the generated dataset or (2) the manually annotated panoramas. We place more emphasis on the latter due to it being less prone to errors and it being directly from a human source instead of machine-derived. Select with --dataset manual (default) or --dataset test:

python evaluate.py --checkpoint checkpoints/your_checkpoint.pth --dataset manual

After running evaluate.py (which will take some time), you should have results printed in the console and in the evaluation_results directory: the precision vs recall and precision & recall vs model confidence curves (PNG + CSV), plus a machine-readable metrics_*.json. Note that the repo has the evaluation_results included from our past runs, so if evaluation_results is present, it doesn't necessarily mean evaluation was successful - it might just be the folder that was included in the github repo.

The curve above is from the corrected re-evaluation (evaluation_results_new/, one-to-one matching — what current evaluate.py reproduces). The as-published curves remain in evaluation_results/ for reference; new runs will not match them because the matching rule changed (see the Erratum).

Cached heatmaps are keyed by checkpoint hash and TTA setting, so switching checkpoints is safe without clearing anything; pass --fresh to force recomputation (e.g. after code changes to inference).

Acknowledgments

This work is supported by the NSF and is part of the OSCUR initiative.