当前位置：首页 > article >正文

nuScenes数据集使用的相机的外参和内参

article 2024/10/19 12:56:17

因为需要用不同数据集测试对比效果，而一般的模型代码里实现的检测结果可视化都是使用open3d的Visualizer在点云上画的3d框，展示出来的可视化效果很差，可能是偷懒，没有实现将检测结果投影到各相机的图像上，所以检测效果可视化对核对效果根本没多大用处，除非自己把检测结果和点云写如rosbag包并且基于ros写程序画出box并发布然后去rviz里看效果展示，对于基于nuScenes数据集的检测结果在图像上做可视化其实比较简单，因为它使用的相机是普通无畸变平面相机，直接将模型检测输出的3dbox(一般是基于激光雷达点云坐标系的)的8个顶点的坐标组成的矩阵左乘以6个相机的外参和内参转换到像素坐标系下，得到8个顶点的像素坐标，然后在相机图像上用OpenCV画出各个顶点和按一定顺序画出顶点之间的连线，就完成了在图像上画出3d box可视化效果。记录一下整理出来的nuScenes数据集所用的6个相机的内外参数据矩阵，需要时可以直接拷贝使用：

{
"CAM_FRONT" : {

  "extrinsic" : 
 [[ 0.99988014,  0.01223258, -0.00949188, -0.00230111],
  [-0.0101382,   0.05390463, -0.9984946,  -0.35509583], 
  [-0.0117025,   0.99847114,  0.05402219, -0.42520362],
  [ 0,          0,          0,          1        ]],

  "intrinsic": 
  [[1.25281310e+03, 0.00000000e+00, 8.26588115e+02, 0.00000000e+00],
   [0.00000000e+00, 1.25281310e+03, 4.69984663e+02, 0.00000000e+00],
   [0.00000000e+00, 0.00000000e+00, 1.00000000e+00, 0.00000000e+00],
   [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00]],

  "distortion": []

},
 

"CAM_FRONT_RIGHT" : {
  "extrinsic" : 
  [[ 0.5372737,  -0.8417395,  -0.05302503,  0.02984875],
   [-0.00136775,  0.06200032, -0.9980752,  -0.33156687], 
   [ 0.84340686,  0.53631204,  0.03215985, -0.6103719 ], 
   [ 0,          0,          0,          1        ]],
	  
  "intrinsic" : 
  [[1.25674851e+03, 0.00000000e+00, 8.17788757e+02, 0.00000000e+00],
   [0.00000000e+00, 1.25674851e+03, 4.51954178e+02, 0.00000000e+00], 
   [0.00000000e+00, 0.00000000e+00, 1.00000000e+00, 0.00000000e+00], 
   [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00]],

  "distortion": []
},

         
"CAM_FRONT_LEFT" : {
  "extrinsic" : 
 [[ 0.5672582,   0.8228128,   0.034602,    0.10790747],
  [-0.01433343,  0.05187402, -0.9985508,  -0.35026494],
  [-0.8234153,   0.5659401,   0.04121969, -0.52967703],
  [ 0,          0,          0,          1        ]],
        
  "intrinsic" : 
 [[1.25786253e+03, 0.00000000e+00, 8.27241063e+02, 0.00000000e+00],
  [0.00000000e+00, 1.25786253e+03, 4.50915498e+02, 0.00000000e+00],
  [0.00000000e+00, 0.00000000e+00, 1.00000000e+00, 0.00000000e+00],
  [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00]],

  "distortion": []
},


"CAM_BACK" : {
  "extrinsic" : 
 [[-0.99992836,  0.00799071,  0.00891444,  0.00137395],
  [-0.00859485,  0.03917429, -0.99919546, -0.2810333 ],
  [-0.0083335,  -0.99920046, -0.0391028,  -1.0164535 ],
  [ 0,          0,          0,          1        ]],
         
  "intrinsic" : 
 [[796.89106345,   0,         857.77743269,   0        ],
  [  0,         796.89106345, 476.88489884,   0        ],
  [  0,           0,           1,           0        ],
  [  0,           0,           0,           1        ]],

  "distortion": []
},


"CAM_BACK_LEFT" : {
  "extrinsic" : 
[[-0.31910315,  0.9468608,   0.04034692, -0.2123741 ],
 [-0.01589121,  0.03722081, -0.99918073, -0.28666893],
 [-0.9475868,  -0.31948286,  0.00316949, -0.43252733],
 [ 0,          0,          0,          1        ]],
         
  "intrinsic" : 
[[1.25498606e+03, 0.00000000e+00, 8.29576933e+02, 0.00000000e+00],
 [0.00000000e+00, 1.25498606e+03, 4.67168056e+02, 0.00000000e+00],
 [0.00000000e+00, 0.00000000e+00, 1.00000000e+00, 0.00000000e+00],
 [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00]],

 "distortion": []
},
         

"CAM_BACK_RIGHT" : {
  "extrinsic" : 
[[-0.3820134,  -0.92305064, -0.04520244,  0.08872822],
 [ 0.01385406,  0.04318667, -0.998971,   -0.29866916],
 [ 0.92405295, -0.38224655, -0.00370989, -0.46464512],
 [ 0,          0,          0,          1        ]],

  "intrinsic" : 
[[1.24996293e+03, 0.00000000e+00, 8.25376805e+02, 0.00000000e+00],
 [0.00000000e+00, 1.24996293e+03, 4.62548164e+02, 0.00000000e+00],
 [0.00000000e+00, 0.00000000e+00, 1.00000000e+00, 0.00000000e+00],
 [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00]],
 "distortion": []
}
}

把nuScense的3d标注在图像上做可视化发现nuScenes的标注也做得不好好些目标漏标了！只是目前没有更好的公开数据集出现，没银子自己采集和标注数据集的尤其是做学校研究发论文的就只能继续在nuScenes这样的数据集上干活了，尽管现在的车载激光雷达已经比nuScenes数据集采用的32线雷达好了几代了，还是不得不死磕nuScenes 32线雷达的稀疏点云用sweep帧对齐和叠加来模拟现在雷达的稠密点云。