Recently, in-depth studies on sensors of autonomous vehicles have been conducted. In particular, the trend to pursue only camera-based autonomous driving is progressing. Studies on object detection using IR (Infrared) cameras is essential in overcoming the limitations of the VIS (Visible) camera environment. Deep learning-based object detection technology requires sufficient data, and data augmentation can make the object detection network more robust and improve performance. In this paper, a method to increase the performance of object detection by generating and learning a high-resolution image of an infrared dataset, based on a data augmentation method based on a Generative Adversarial Network (GAN) was studied. We collected data from VIS and IR cameras under severe conditions such as snowfall, fog, and heavy rain. The infrared data images from KAIST were used for data learning and verification. We confirmed that the proposed data augmentation method improved the object detection performance, by applying generated dataset to various object detection networks. Based on the study results, we plan on developing object detection technology using only cameras, by creating IR datasets from numerous VIS camera data to be secured in the future and fusion with VIS cameras.