Skip to content
saiketaPublic

About

This is the code repository for the paper "BenHAR: Benchmarking Self-Supervised Learning for Generalizable Sensor-based Activity Recognition".

Topics

Resources

Stars

7 stars

Watchers

0 watching

Forks

Latest commit

 

History

9 Commits

Folders and files

Repository files navigation

BenHAR

This is the code repository for the paper BenHAR: Benchmarking Self-Supervised Learning for Generalizable Sensor-based Activity Recognition, a large-scale benchmark for evaluating the generalization capability of self-supervised sensor-based human activity recognition (HAR) models.

BenHAR framework overview

🔍 Key Features

  • Large-scale multi-dataset integration: Curates and standardizes a large-scale (~258K samples, ~430 hours) sensor-based HAR benchmark that is multi-source (14 widely used datasets), diverse (300+ subjects, 11 countries, 13 body locations, and 2 device types), and multimodal (accelerometer and gyroscope).

  • Cross-dataset generalization benchmark:

    • Evaluates four representative self-supervised learning paradigms, including 1 recognition method, 2 reconstruction methods, 3 contrastive methods, and 2 hybrid methods.
    • Benchmarks 12 architecture combinations consisting of 3 encoders (CNN, ResNet, Transformer) and 4 classifiers (CNN, GRU, MLP, Transformer).
  • Analysis of data scale and domain shift factors:

    • Investigates the impact of unlabeled data scale, labeled data scale, and irrelevant activity data scale on generalization performance.
    • Studies the impact of cross-subject, cross-device, and cross-location domain shifts on generalization performance.

⚙️ Installation

Create the environment using requirements.txt:

pip install -r requirements.txt

📦 Preparing data

Download

Dataset Country # Subject # Activity Device # Location Download
UCI Italy 30 6 Custom 1 Link
HHAR Denmark 9 6 Custom 2 Link
Shoaib The Netherlands 10 7 Custom 4 Link
Motion England 24 6 Custom 1 Link
DSADS Turkey 8 18 Research 3 Link
USC-HAD United States 14 12 Research 1 Link
KU-HAR Bangladesh 90 18 Custom 1 Link
PAMAP2 Germany 9 18 Research 3 Link
TNDA-HAR China 50 8 Research 5 Link
Mhealth Spain 10 12 Research 2 Link
WISDM United States 51 18 Custom 2 Link
RealWorld Germany 15 8 Custom 7 Link
HARSense Meghalaya 12 6 Custom - Link
UT-Complex The Netherlands 10 13 Custom 2 Link

All datasets are publicly available. Please download and extract the datasets into the datasets/ directory, and name each dataset folder according to the dataset names listed in the table.

Standardization

Run the following command to standardize all datasets:

bash scripts/preprocess.sh

The preprocessing scripts for each dataset can be found in the preprocess/ folder. All datasets are standardized into non-overlapping 6-second windows with a sampling rate of 20 Hz. More details are provided in the paper.

Dataset statistics.

🌍 Cross-dataset Benchmark

Methods

Method Paradigm Venue Link
BioBankSSL Recognition npj Digital Medicine 2024 Link
LIMU-BERT Reconstruction SenSys 2021 Link
CRT Reconstruction TNNLS 2023 Link
TS-TCC Contrastive IJCAL 2021 Link
TS2Vec Contrastive AAAI 2022 Link
FOCAL Contrastive NeurIPS 2023 Link
SimMTM Hybrid NeurIPS 2023 Link
CrossHAR Hybrid IMWUT 2024 Link

Overview of SSL paradigms.

Evaluation

The 14 datasets are partitioned into five dataset-level folds. Models are trained on four folds and evaluated on the held-out fold.

bash scripts/run_dataset_5fold.sh

Example: BioBankSSL with transformer encoder and CNN classifier using accelerometer + gyroscope:

MODE=BioBankSSL \
ENCODER_BACKBONE=transformer \
CLASSIFIER_PREFIX=cnn \
INPUT_CHANNELS=6 \
bash scripts/run_dataset_5fold.sh

📊 Analysis

Impact of unlabeled data scale

This setting changes the amount of unlabeled source data used to pretrain the SSL encoder. The downstream classifier still uses the full labeled source split unless CLASSIFIER_TRAIN_SUBSET_RATE is changed.

Use TRAIN_SUBSET_RATE to control the fraction of source training data used for pretraining:

MODE=BioBankSSL \
ENCODER_BACKBONE=transformer \
CLASSIFIER_PREFIX=cnn \
INPUT_CHANNELS=6 \
TRAIN_SUBSET_RATE=0.25 \
CLASSIFIER_TRAIN_SUBSET_RATE=1.0 \
bash scripts/run_dataset_5fold.sh

Impact of labeled data scale

This setting evaluates how much labeled source data is needed after SSL pretraining. The encoder is pretrained with the full unlabeled source split, while the downstream classifier uses only a fraction of labeled source embeddings.

Use CLASSIFIER_TRAIN_SUBSET_RATE to control the labeled classifier training fraction:

MODE=BioBankSSL \
ENCODER_BACKBONE=transformer \
CLASSIFIER_PREFIX=cnn \
INPUT_CHANNELS=6 \
TRAIN_SUBSET_RATE=1.0 \
CLASSIFIER_TRAIN_SUBSET_RATE=0.1 \
bash scripts/run_dataset_5fold.sh

Impact of irrelevant data scale

This setting studies whether adding extra unlabeled data from activity categories that are not part of the main benchmark helps or hurts representation learning.

Use DATA_OTHER_SUBSET_RATE to control the fraction of data_other/ added to encoder pretraining.

MODE=BioBankSSL \
ENCODER_BACKBONE=transformer \
CLASSIFIER_PREFIX=cnn \
INPUT_CHANNELS=6 \
TRAIN_SUBSET_RATE=1.0 \
CLASSIFIER_TRAIN_SUBSET_RATE=1.0 \
DATA_OTHER_ROOT=./data_other \
DATA_OTHER_SUBSET_RATE=0.5 \
bash scripts/run_dataset_5fold.sh

Domain-shift factors

# Subject shift: train and test subjects are disjoint.
bash scripts/run_cross_subject.sh

# Device shift: custom-grade devices vs. research-grade devices.
bash scripts/run_cross_device.sh

# Body-location shift: train on one location group and test on another.
TRAIN_LOCATION=trunk \
TEST_LOCATION=upper \
bash scripts/run_cross_location.sh

About

This is the code repository for the paper "BenHAR: Benchmarking Self-Supervised Learning for Generalizable Sensor-based Activity Recognition".

Topics

Resources

Stars

7 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages