SeaClips: A Video Dataset for Maritime Object Detection

Maritime computer vision is a requirement for autonomous surface vehicles and can improve maritime safety if a high level of robustness is achieved. As deep learning dominates the computer vision community, domain-specific datasets are required to obtain well-generalizing and reliable models. However, maritime datasets, especially those containing videos and temporally dense annotations, are still small compared to other domains, such as autonomous driving or generic computer vision datasets. This paper introduces SeaClips, a new maritime video dataset containing 74 videos with an average duration of 14 seconds and with 31k frames in total. Videos were recorded under varying conditions, with three cameras mounted on shore and on boats. SeaClips provides frame-by-frame annotations, encompassing 129k bounding boxes of seven categories, containing vessel and non-vessel classes. SeaClips contributes to a broader coverage of maritime scenarios, leading to more robust computer vision models. Baseline results on the dataset are established by evaluating six image-based models and three models using temporal context, ranging from lightweight models to heavy transformer architectures. It is found that the different scales and shapes at which objects appear in SeaClips pose a challenge to state-of-the-art detectors. SeaClips is accessible for research on maritime obstacle detection at: https://huggingface.co/datasets/SEA-AI/SeaClips.