Somerville ‘data farm’ highlights city’s daily challenges

Mass Innovations, From The Beacon, September 2016

A new, dynamic website in Somerville shares key city data, such as the city’s top service calls and crime reports, in an easy-to-read graphical format.

City officials say the new “data farm,” updated daily, increases government transparency while helping departments quickly identify and address emerging issues.

The daily dashboard, built in-house using free, open-source tools, builds off a pre-existing “key systems indicator” dashboard that presents longer-term data. Michael Mastrobuoni, the city’s budget manager, said former Chief of Staff Daniel Hadley was the “chief architect” of the project.

What makes the automated daily dashboard possible is a unique algorithm that takes the form of two specialized scripts, Mastrobuoni said. The first script grabs all the data from various databases within the city’s network, such as the 311 Constituent Services line and the Police Department’s server, and “cleans” the data, geocodes it, and determines what’s increasing in importance. That last part is based on how much a type of incident has increased above the average, Mastrobuoni said.

“You can’t compare 20 more potholes out of 3,000 per day versus a couple of extra rat sightings,” he said. “You need to look at what’s changed the most from where it usually is.”

The second script uses the cleaned, geocoded and ranked data to build the website each morning.

For instance, the “significant calls” section highlights incidents that the first script ranked as important. The site reads, “During the last seven days, there was an increase in calls for [X, Y and Z].” Those letters are filled in with the types of calls that increased the most compared to their usual averages, which are also represented in a line graph to show the level of increase and plotted out on an interactive map of the city.

Among other data sets, the site charts all 311 calls from the previous day; the proportion of closed work orders versus open orders for significant calls; and quality-of-life calls over the previous week such as graffiti, rats and potholes, which are mapped. The data farm provides maps and charts about crimes such as burglaries, motor vehicle thefts and robberies over the previous 14 days.

The portal makes it easy for the public to see what’s happening in Somerville, but it has also proved useful for officials. Mayor Joseph Curtatone’s staff now checks the daily dashboard during morning staff meetings to see what issues are emerging and need to be addressed, Mastrobuoni said, adding that the issues are sometimes surprising.

“This isn’t super-flattering, but we were having trouble with our parking permits,” he acknowledged. “We saw a significant increase in calls about the permits: ‘Where are they?’ and ‘When are we going to get them?’ The dashboard helps us define what issues we need to be focusing on. And it’s automatic, so we can just pop in and see what happened yesterday and what’s been going on over the last weeks and months, too.”

The data farm, of which the daily dashboard is one part, joins the city’s previously existing data-based websites, such as the open data portal that contains the city’s raw data sets, and the “key systems indicators” dashboard, which displays long-term data such as results of the city’s happiness survey, annual crime trends, property tax figures and more.

The data farm also has a “311 Explorer” that allows users to explore service requests either in charts or by searching for requests by address, and a budget visualization site seen in other communities such as Arlington.

“We felt like people didn’t have a great grasp of what we were doing in all these little things,” Mastrobuoni said. “We wanted to get everything together in one place and give them that one portal.”

While the Somerville Data Farm presents the city’s data in easy-to-read maps, graphs and charts, it also opens a doorway to the raw data and invites members of the public to dive into the data themselves, said Mastrobuoni, who works in the Mayor’s Office of Innovation and Analytics, known as SomerStat.

For more information, contact Somerville Budget Manager Michael Mastrobuoni at (617) 625-6600, ext. 2112.