Event-based cameras are predestined for Intelligent
Transportation Systems (ITS). They provide very high temporal
resolution and dynamic range, which can eliminate motion blur
and improve detection performance at night. However, eventbased
images lack color and texture compared to images from a
conventional RGB camera. Considering that, data fusion between
event-based and conventional cameras can combine the strengths
of both modalities. For this purpose, extrinsic calibration is
necessary. To the best of our knowledge, no targetless calibration
between event-based and RGB cameras can handle multiple
moving objects, nor does data fusion optimized for the domain
of roadside ITS exist. Furthermore, synchronized event-based
and RGB camera datasets considering roadside perspective are
not yet published. To fill these research gaps, based on our
previous work, we extended our targetless calibration approach
with clustering methods to handle multiple moving objects.
Furthermore, we developed an early fusion, simple late fusion,
and a novel spatiotemporal late fusion method. Lastly, we
published the TUMTraf Event Dataset, which contains more
than 4,111 synchronized event-based and RGB images with
50,496 labeled 2D boxes. During our extensive experiments,
we verified the effectiveness of our calibration method with
multiple moving objects. Furthermore, compared to a single RGB
camera, we increased the detection performance of up to +9%
mAP in the day and up to +13% mAP during the challenging
night with our presented event-based sensor fusion methods.
The TUMTraf Event Dataset is available at https://innovationmobility.
com/tumtraf-dataset.
«
Event-based cameras are predestined for Intelligent
Transportation Systems (ITS). They provide very high temporal
resolution and dynamic range, which can eliminate motion blur
and improve detection performance at night. However, eventbased
images lack color and texture compared to images from a
conventional RGB camera. Considering that, data fusion between
event-based and conventional cameras can combine the strengths
of both modalities. For this purpose, extrinsic calibration is
necessa...
»