Big Data Comes to Boston


“Here in Boston, we want to look at these data resources and we want to know how to do things better.” Dan O’Brien, BARI Project Manager

Lindsay Overhage, Article ImageIn the perfect sociological experiment, conditions would be constant, with the experimenter acting as nothing more than an omniscient fly on the wall. At the Boston Area Research Initiative (BARI), a new scientific method comes startlingly close to this ideal. Without knowing it, Boston residents have been showing that fly around their homes, streets, and neighborhoods for months—each time they make an address-based request for city services.  Perhaps the only deviation is that the fly is sitting behind a computer screen full of numbers, and his name is Big Data.

Ten years ago, a proposal to physically map “disorder” for an entire city at one point in time required a van full of cameras to drive five mph, backed by a small army of students who analyzed the footage. The only city to manage this feat twice was Chicago. Now, Ecometrics in the Age of Big Data offers a new approach. Big Data, a buzzword for experimental methods that analyze existing large sets of data currently being underutilized, is catalyzing a movement away from data collection and towards analysis of existing pools. Using existing administrative data, a research team led by Professor Dan O’Brien of the Radcliffe Institute for Advanced Study created a comprehensive map of disorder in Boston that can be updated every two months.  The study is supported by the interdisciplinary and inter-university BARI, and aims to give City Hall real-time data on its constituency.

The starting point was computer data from 300,000 address-based requests for city services (graffiti removal, trash pickup, etc.) that were reported via phone calls, online entries, and smartphone applications. In open-source calculations now available to the public, the team divided the data into census block groups averaging 1,000 citizens, a quarter the size of those used in previous studies, and then divided it into 178 case types and five key issues: Housing issues, “uncivil” use of space, big building complaints, graffiti, and trash. After correcting for how likely each neighborhood was to report a problem, they created a layered, color-coded Google Map.

Available on BARI’s website, the map shows seven measures of physical disorder along with demographic information, such as high school dropout rate, and median income. The map makes surprising correlations, expected or not: civic engagement, for example, is not necessarily higher in more affluent areas.

BARI goes further than reaching out to others in academia. Research accounts for only half of its mission—the other half is ensuring that data makes its way into the hands of decision makers and anyone else who might make use of it. O’Brien says, “[BARI aims to] bring together all of the scholarly expertise in the area and all of the really great energy going on in the public sector right now.” O’Brien believes it works because, “here in Boston…we want to look at these data resources and we want to know how to do things better.”

Now, O’Brien says, BARI hopes to use the system in place to test hypotheses and old theories, working with policymakers to see how policy influences outcomes. One simple example of this response was the use of flyers encouraging residents to use reporting services. Flyers that asked residents “to help clean up Boston” were equivalent to not even putting up flyers, while those that exchanged “Boston” for the name of a specific neighborhood (e.g. Dudley Square) did increase response rates. While this alone may not seem remarkable, it’s a first step to test a change at the policy level. O’Brien says, “we’re thinking small instead of thinking big. What’s really fun about this project is it’s small research projects leading to kind of small ideas in terms of how to implement these ideas and how to redesign how the city interfaces with its constituents. And at the end of the day, these little changes start to build up into a greater volume of long term collaboration.”

Lindsay Overhage is a Brevia staff writer. She can be reached at