Coordinate Transformation in Nuscenes Dataset

Recently I’ve used nuScenes Dataset to train a mesurement model for extended target. The idea is originally form [1-4]. NuSenes dataset is the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view[5]. However, the radar points in nuScenes dataset is very sparse compared to other datasets like the RadarSecenes.

The main difficulty in dealing with the nuScenes dataset is the transformation of the coordinates. There are three coordinates in nuScenes dataset: ego-car coordinate in which the sensor position is recorded, sensor coordinate where the radar points position is recorded and global coordinate where all the annotation is recorded. To accumulate the radar points for certain “vehicle.car” annotaion, we need to tranform the radar points position from sensor coordinate to global coordinate.

In nuScenes dataset, the position $\mathbf{x} = (x,y,z)$ is recorded by attribute “translation” and the orientation is recorded by attribute “rotation” which is a quarternion. Luckily, the nuScenes devkit provides method transform matrix in module nuscenes.utils.geometry_utils which we can use to convert the quarternion to transform matrix. For coordinate transformation, an interesting equation is $\mathbf{x} = T * \mathbf{x}_{\text{new}}$ which means that we can get the position in new coordinate by dot the inverse of the transform matrix. Some good reference about transform matrix and quarternion:

https://zhuanlan.zhihu.com/p/79894982

https://cubox.pro/my/card?id=ff808081856269e401856d7d1f412b2f

[1] H. Kaulbersch, J. Honer, and M. Baum, ‘EM-based Extended Target Tracking with Automotive Radar using Learned Spatial Distribution Models’, in 2019 22th International Conference on Information Fusion (FUSION), Jul. 2019, pp. 1–8. doi: 10.23919/FUSION43075.2019.9011179.

[2] Y. Xia, P. Wang, K. Berntorp, H. Mansour, P. Boufounos, and P. V. Orlik, ‘Extended object tracking using hierarchical truncation model with partial-view measurements’, presented at the Proceedings of the IEEE Sensor Array and Multichannel Signal Processing Workshop, 2020, vol. 2020-June. doi: 10.1109/SAM48682.2020.9104388.

[3] Y. Xia et al., ‘Extended Object Tracking with Automotive Radar Using Learned Structural Measurement Model’, in 2020 IEEE Radar Conference (RadarConf20), Sep. 2020, pp. 1–6. doi: 10.1109/RadarConf2043947.2020.9266598.

[4] Y. Xia et al., ‘Learning-Based Extended Object Tracking Using Hierarchical Truncation Measurement Model with Automotive Radar’, IEEE Journal on Selected Topics in Signal Processing, vol. 15, no. 4, pp. 1013–1029, 2021, doi: 10.1109/JSTSP.2021.3058062.

[5] H. Caesar et al., ‘nuScenes: A multimodal dataset for autonomous driving’. arXiv, May 05, 2020. doi: 10.48550/arXiv.1903.11027.