Urban Data Analytics: (3) Analytics and Visualization
1. Introduction
Many of the social and environmental phenomena have spatial characteristics. Geo-spatial methods, such as, maps, GIS, and etc, are excellent ways to visualize and understand these characteristics. In this post, I will introduce some of the urban analytics and geospatial modelling techniques that I use frequently.
For visualization of spatial data, I frequently use Leaflet.js. It is a javascript-based web mapping engine that allows for creating a web map with a breeze. For my recent research project, entitled "Noise 311 research", I used Leaflet.js to create a dot density map of noise complaints from Vancouver's public 3-1-1 call data in 2016. Then I used kernel density function to show spatiotemporal patterns of major construction activities and noise complaints in Vancouver.
2. Basic web mapping
For basic web mapping, I used the R leaflet package to implement this the maps in R. All you have to do is load the leaflet package and apply publicly available map tiles.
# Load the leaflet package
library(leaflet)
# Initiate leaflet m1 = leaflet()
# Now we have our loop - each time through the loop, it is adding our markers to the map object: for (i in 1:length(data_filterlist)){ m1 = addCircleMarkers(m1, lng=data_filterlist[[i]]$lon_offset, lat=data_filterlist[[i]]$lat_offset, popup=data_filterlist[[i]]$Case_Type, radius=2.5, stroke = FALSE, fillOpacity = 0.75, color = colorFactors(data_filterlist[[i]]$Month), group = layerlist[i] ) }
# Now let's insert the radio button functionality
m1 = addTiles(m1, group = "OSM (default)") m1 = addProviderTiles(m1,"Stamen.Toner", group = "Toner") m1 = addProviderTiles(m1, "Stamen.TonerLite", group = "Toner Lite") m1 = addLayersControl(m1, overlayGroups = c("Toner Lite"), baseGroups = layerlist)
# Show the dot density map m1
The code above produces this interactive web map shown below (Figure 1). The map shows the dot density map of noise complaints in Vancouver, BC from January, 2016 to December, 2016. Use the pull-down menu on the upper right corner of the map to toggle on/off spatial distribution of noise complaints from January to December.
Figure 1: Interactive web mapping of noise complaints
3. Temporal trends
With this initial exploration, I explored temporal patterns of noise complaints and major development projects by year. Major development projects are any projects with the estimated capital costs over $20 million CAD.
# Assign colors
blue = "#00BFC4"
red = "#F8766D"
# Create default theme p.theme = theme_bw(base_size=12) +
theme(axis.line = element_blank(), panel.grid.major = element_line(color="gray80"), panel.grid.minor = element_blank(), panel.border = element_blank(), panel.background = element_rect(fill="gray95"), plot.background = element_rect(fill="gray95"), axis.title.x = element_blank(), plot.margin=margin(t=1, r=1, b=1, l=1, unit="cm"), plot.title = element_text(size = 12, face = "bold"))
# Temporal trend of major projects
p1 = mpidata2 %>% group_by(year) %>% filter(year > 2009) %>% # summarise(total.mpi = sum(mpi.total, na.rm=T)) %>% ggplot(data=., aes(year, mpi)) + stat_summary(fun.y = sum, geom = "line") + stat_summary(fun.y = sum, geom = "point", shape=1, size=2) + stat_summary(fun.y = sum, geom = "area", fill=blue, alpha=0.3) + # stat_summary(fun.y = sum, geom = "bar", width=0.05, fill="red", alpha=0.5) + scale_x_continuous(breaks=c(2010:2016), labels=c("'10","'11","'12","'13","'14","'15","'16")) + ggtitle("Total Cases of Major Development by Year") + ylab("Total cases of construction projects") + p.theme
# Temporal trend of noise complaints
p2 = calldata3 %>% group_by(year) %>% # summarise(total.call = sum(call.total, na.rm=T)) %>% ggplot(data=., aes(year, call)) + stat_summary(fun.y = sum, geom = "line") + stat_summary(fun.y = sum, geom = "point", shape=1, size=2) + stat_summary(fun.y = sum, geom = "area", fill=red, alpha=0.3) + # stat_summary(fun.y = sum, geom = "bar", width=0.05, fill="red", alpha=0.5) + scale_x_continuous(breaks=c(2009:2016), labels=c("'09","'10","'11","'12","'13","'14","'15","'16")) + ggtitle("Total Noise Complaints by Year") + ylab("Total cases of noise complaint") + p.theme
# Show the combined map
show(grid.arrange(p1, p2, nrow=1))
The result of the above code is shown below (Figure 2). The graphs show that the total volume of both major development projects and noise complaints have been increasing. The graphs also show that these two data seem to be temporally related.
Figure 2: Temporal trends of major projects and noise complaints
4. Spatiotemporal patterns
For spatial patterns, I put together the noise complaints data for all years from 2010 to 2016. Then, I created a heatmap using a kernel density estimation (KDE) to show clustering patterns of the noise complaints in the City of Vancouver.
# Kernel density estimation of major projects
mpidata2 %>% filter(!is.na(mpi) & year>2009 & mpi>0) %>% qmplot(lon, lat, data=., geom = "blank", maptype = "toner-background", darken = .7, size=I(1)) + stat_density_2d(aes(fill = ..level..), geom = "polygon", alpha = 0.3, color = NA) + scale_fill_gradient2(low = "white", mid = blue, high = "blue", midpoint=800) + facet_wrap(~year) + ggtitle("Spatial Patterns of Major Development by Year") + theme(plot.title = element_text(size = 16, face = "bold")) + theme(strip.text = element_text(size = 14, face = "bold", hjust=0)) + theme(strip.background = element_blank()) + theme(panel.spacing = unit(1, "lines"))
The KDE is implemented in gmap package, and the below one line of code performs the KDE process.
stat_density_2d(aes(fill = ..level..), geom = "polygon", alpha = 0.3, color = NA)
The resulting visualization is shown below.
Figure 3: Spatiotemporal pattern of major projects
Next, I pulled geo-coded information of all the major development projects that exceed 20 million in capital costs from the Province of British Columbia. The data ranges from 2010 to 2016. With the same KDE approach as the noise complaints data, I created a heatmap of the major development projects in the City of Vancouver.
# Kernel density estimation of noise complaints
calldata3 %>% filter(!is.na(call)) %>% qmplot(lon, lat, data=., geom = "blank", maptype = "toner-background", darken = .7, size=I(1)) + stat_density_2d(aes(fill = ..level..), geom = "polygon", alpha = 0.3, color = NA) + scale_fill_gradient2(low = "white", mid = red, high = "red", midpoint=300) + facet_wrap(~year) + ggtitle("Spatial Patterns of Noise Complaints by Year") + theme(plot.title = element_text(size = 16, face = "bold")) + theme(strip.text = element_text(size = 14, face = "bold", hjust=0)) + theme(strip.background = element_blank()) + theme(panel.spacing = unit(1, "lines"))
And the resulting graph is shown below.
Figure 4: Spatiotemporal pattern of noise complaints
Both heatmaps show general patterns of noise complaints and major development projects expanding toward southeast direction. Downtown shows the highest clustering patterns for both noise complaints and major development projects.
5. Conclusion
With these visualizations, I developed a mixed-effects Poisson model estimating the number of nose complaints as a function of the number of major development projects. The panel data required the model to be built using dissemination area as a random clustering factor. Later, I will present the model results and random effects diagnostics as I move forward with the analysis.