CityCraft: 3D virtual city creation from a single image

Abstract

This paper introduces a method to generate a three-dimensional (3D) virtual model of an imaginary city from a single street-view image to represent the appearance of the city in a given input photograph. The proposed approach differs from reconstruction approaches, which generate a city model by guessing the city name from the input photograph. In contrast, we use machine learning to identify where to generate the city, what to allocate in the city, and how to arrange the components. We employ generative adversarial networks and convolutional neural networks to create a terrain map and identify the components and styles that represent the virtual city appearance. We demonstrate that our system creates 3D virtual cities that are visually similar in terms of plausibility and naturalness to actual cities corresponding to input photographs from around the world. To the best of our knowledge, this is the first work to generate a city model including all general city components, including streets, buildings, and vegetation, to match the style of a single input image.