TY - JOUR

T1 - NOVA

T2 - Rendering Virtual Worlds with Humans for Computer Vision Tasks

AU - Kerim, A.

AU - Aslan, C.

AU - Celikcan, U.

AU - Erdem, E.

AU - Erdem, A.

N1 - This is the peer reviewed version of the following article: Kerim, A., Aslan, C., Celikcan, U., Erdem, E. and Erdem, A. (2021), NOVA: Rendering Virtual Worlds with Humans for Computer Vision Tasks. Computer Graphics Forum, 40: 258-272. doi: 10.1111/cgf.14271 which has been published in final form at https://onlinelibrary.wiley.com/doi/10.1111/cgf.14271 This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for self-archiving.

PY - 2021/9/30

Y1 - 2021/9/30

N2 - Today, the cutting edge of computer vision research greatly depends on the availability of large datasets, which are critical for effectively training and testing new methods. Manually annotating visual data, however, is not only a labor-intensive process but also prone to errors. In this study, we present NOVA, a versatile framework to create realistic-looking 3D rendered worlds containing procedurally generated humans with rich pixel-level ground truth annotations. NOVA can simulate various environmental factors such as weather conditions or different times of day, and bring an exceptionally diverse set of humans to life, each having a distinct body shape, gender and age. To demonstrate NOVA's capabilities, we generate two synthetic datasets for person tracking. The first one includes 108 sequences, each with different levels of difficulty like tracking in crowded scenes or at nighttime and aims for testing the limits of current state-of-the-art trackers. A second dataset of 97 sequences with normal weather conditions is used to show how our synthetic sequences can be utilized to train and boost the performance of deep-learning based trackers. Our results indicate that the synthetic data generated by NOVA represents a good proxy of the real-world and can be exploited for computer vision tasks.

AB - Today, the cutting edge of computer vision research greatly depends on the availability of large datasets, which are critical for effectively training and testing new methods. Manually annotating visual data, however, is not only a labor-intensive process but also prone to errors. In this study, we present NOVA, a versatile framework to create realistic-looking 3D rendered worlds containing procedurally generated humans with rich pixel-level ground truth annotations. NOVA can simulate various environmental factors such as weather conditions or different times of day, and bring an exceptionally diverse set of humans to life, each having a distinct body shape, gender and age. To demonstrate NOVA's capabilities, we generate two synthetic datasets for person tracking. The first one includes 108 sequences, each with different levels of difficulty like tracking in crowded scenes or at nighttime and aims for testing the limits of current state-of-the-art trackers. A second dataset of 97 sequences with normal weather conditions is used to show how our synthetic sequences can be utilized to train and boost the performance of deep-learning based trackers. Our results indicate that the synthetic data generated by NOVA represents a good proxy of the real-world and can be exploited for computer vision tasks.

KW - procedural content generation

KW - synthetic-data for learning

KW - visual tracking

KW - Deep learning

KW - Large dataset

KW - Meteorology

KW - Virtual reality

KW - Environmental factors

KW - Labor intensive process

KW - Person tracking

KW - State of the art

KW - Synthetic data

KW - Synthetic datasets

KW - Synthetic sequence

KW - Training and testing

KW - Computer vision

U2 - 10.1111/cgf.14271

DO - 10.1111/cgf.14271

M3 - Journal article

VL - 40

SP - 258

EP - 272

JO - Computer Graphics Forum

JF - Computer Graphics Forum

SN - 0167-7055

IS - 6

ER -