In this paper, we present a framework for collision avoidance of UGVs by vision-based control. On the image plane which is created by perspective camera rigidly attached to UAV hovering stationarily, image features of UGVs are to be controlled by our control framework so that they proceed to desired locations while avoiding collision. UGVs are assumed as unicycle wheeled mobile robots with nonholonomic constraint and they follow the image feature’ movement on the ground plane with low-level controller. We used potential function method to guarantee collision prevention, and showed its stability. Simulation results are presented to validate capability and stability of the proposed framework.