Detection Node

3D object detection node using PointNet for classification, PCA for oriented bounding boxes, and Kalman filtering for tracking.

Overview

The detection node performs real-time 3D object detection on point clouds using the following pipeline:

1. Euclidean Clustering - Groups nearby points into clusters
2. Geometry Filter - Filters clusters based on size constraints (height, width, length, volume)
3. PointNet Classification - Uses PointNet deep learning model to classify objects (car, pedestrian, cyclist, truck, bus, unknown)
4. PCA OBB - Computes Oriented Bounding Boxes using Principal Component Analysis
5. Kalman Tracking - Tracks objects across frames using Kalman filtering
6. Detection3DArray - Publishes detection results as ROS2 messages

Installation

Prerequisites

• ROS2 (tested with Jazzy)
• PCL (Point Cloud Library)
• Eigen3
• libtorch (PyTorch C++ API)
• PointNet model file (.pt TorchScript format)

Install libtorch

# Download libtorch (CPU version)
cd ~
wget https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-2.1.0%2Bcpu.zip
unzip libtorch-cxx11-abi-shared-with-deps-2.1.0+cpu.zip

# Or use the install script if available
# ./install_libtorch.sh

Build Package

cd ~/ros2_ws
colcon build --packages-select detection --cmake-args -DTORCH_DIR=$HOME/libtorch
source install/setup.bash

Set Library Path

Add to your ~/.bashrc or run before each session:

export LD_LIBRARY_PATH=$HOME/libtorch/lib:$LD_LIBRARY_PATH

Usage

Launch File (Recommended)

cd ~/ros2_ws
source install/setup.bash
export LD_LIBRARY_PATH=$HOME/libtorch/lib:$LD_LIBRARY_PATH

ros2 launch detection detection.launch.py use_rviz:=true

Direct Node Command

cd ~/ros2_ws
source install/setup.bash
export LD_LIBRARY_PATH=$HOME/libtorch/lib:$LD_LIBRARY_PATH

ros2 run detection detection_node \
  --ros-args \
  -p input_topic:=/patchwork/nonground \
  -p model_path:=/home/metrolla/Development/pointnet/checkpoints/best.pt \
  -p clusters_topic:=/detection/clusters \
  -p markers_topic:=/detection/markers \
  -p detections_topic:=/detection/detections

Parameters

Topics

Parameter	Type	Default	Description
input_topic	string	/patchwork/nonground	Input point cloud topic (non-ground points)
clusters_topic	string	/detection/clusters	Output clustered point cloud topic
markers_topic	string	/detection/markers	Output visualization markers (bounding boxes) topic
detections_topic	string	/detection/detections	Output Detection3DArray topic (as PoseArray)

Clustering Parameters

Parameter	Type	Default	Description
cluster_tolerance	double	0.5	Distance tolerance for clustering in meters
min_cluster_size	int	10	Minimum number of points in a cluster
max_cluster_size	int	25000	Maximum number of points in a cluster

Geometry Filter Parameters

Parameter	Type	Default	Description
min_height	double	0.5	Minimum cluster height (meters)
max_height	double	3.0	Maximum cluster height (meters)
min_width	double	0.3	Minimum cluster width (meters)
max_width	double	5.0	Maximum cluster width (meters)
min_length	double	0.1	Minimum cluster length (meters)
max_length	double	10.0	Maximum cluster length (meters)
min_volume	double	0.01	Minimum cluster volume (cubic meters)
max_volume	double	200.0	Maximum cluster volume (cubic meters)

PointNet Parameters

Parameter	Type	Default	Description
model_path	string	/home/metrolla/Development/pointnet/checkpoints/best.pt	Path to PointNet TorchScript model file (.pt) - REQUIRED
pointnet_input_size	int	1024	Number of points for PointNet input (will be resampled/padded)
pointnet_threshold	double	0.3	Minimum confidence score for detection (0.0-1.0)

Tracking Parameters

Parameter	Type	Default	Description
tracking_association_threshold	double	2.0	Distance threshold for associating detections with tracks (meters)
process_noise	double	0.1	Process noise for Kalman filter
measurement_noise	double	0.5	Measurement noise for Kalman filter
max_track_age	int	5	Maximum frames without update before track deletion
min_track_hits	int	1	Minimum detections needed to become a confirmed track

Class Names

Parameter	Type	Default	Description
class_names	string[]	['car', 'pedestrian', 'cyclist', 'truck', 'bus', 'unknown']	List of detection class names

Output Topics

/detection/clusters (sensor_msgs/PointCloud2)

Clustered point clouds with intensity values set to class IDs for visualization.

/detection/markers (visualization_msgs/MarkerArray)

3D bounding boxes (OBB) and text labels for each tracked object. Markers include:

• OBB markers (namespace: detections) - Oriented bounding boxes with class-based colors
• Text labels (namespace: detection_labels) - Class name, track ID, and confidence score

/detection/detections (geometry_msgs/PoseArray)

Detection results as PoseArray (placeholder for Detection3DArray). Each pose represents:

• Position: Object center (x, y, z)
• Orientation: Object yaw angle (quaternion)

Visualization

The node includes an RViz configuration file at rviz/detection.rviz that displays:

• Clustered point clouds (color-coded by class)
• 3D bounding boxes (OBB)
• Text labels with class names and confidence scores

To use RViz:

ros2 launch detection detection.launch.py use_rviz:=true

Troubleshooting

"PointNet is required but libtorch is not available"

Solution: Rebuild with libtorch path:

colcon build --packages-select detection --cmake-args -DTORCH_DIR=$HOME/libtorch
export LD_LIBRARY_PATH=$HOME/libtorch/lib:$LD_LIBRARY_PATH

"Failed to load PointNet model"

Solution:

• Check that the model path is correct
• Ensure the model file is a TorchScript .pt file
• Verify file permissions

No detections appearing

Solution:

• Check that input topic has data: ros2 topic echo /patchwork/nonground
• Lower pointnet_threshold parameter
• Adjust geometry filter parameters to match your objects
• Verify PointNet model is loaded (check node logs)

Dependencies

• rclcpp
• sensor_msgs
• visualization_msgs
• geometry_msgs
• std_msgs
• pcl_ros
• pcl_conversions
• tf2_ros
• libtorch (PyTorch C++)