Vacant lots are municipally-owned land parcels which were acquired post-abandonment or due to tax foreclosures. With time, failure to sell or find alternate uses for vacant lots results in them causing adverse effects on the health and safety of residents, and cost the city both directly and indirectly. Although existing research has tried to define these impacts, cities need quantifiable evidence from within the city to make planning decisions based on these studies. Moreover, trying to understand the impact of vacant lots in an uncontrolled setting makes it difficult to perform A key problem with existing methodologies is that they tend to look at the city as a whole, while ignoring the diverse socioeconomic factors at play. Altogether, city planners are left with little or no actionable information to prioritize conversion of vacant lots. In contrast, for our research we try to model the city as blocks, census tracts and neighborhoods while using relevant features to capture key demographic, economic and geographic characteristics. In addition, we build a deep learning model to quantify the impact of vacant lots on changing property values so as to recommend conversions that yields the maximum benefit through property value tax increase. Our results indicate that our model is able to capture the relationship between vacant lots and property values better than conventionally used algorithms and data models. Further, our model specifically caters to small and mid size cities, which are often neglected in the mainstream urban computing research.