When the California Health Care Foundation launched the California Health Data Project last spring, it made a smart decision to create a new role in the civic innovation space. The foundation brought on California Health Data Ambassadors to connect the supply side of the open data equation, in this case the California Health and Human Services Agency’s (CHHSA’s) open data portal — to the demand side of the equation — or potential users of that portal.CHCF_CMYK_Horizontal

As two of these Health Data Ambassadors, we focus primarily on creating and nourishing feedback loops between the data owners and users, with the goal of making the state’s data as useful as possible. One of our primary methods of collecting user feedback is via regularly convening local health data stakeholders to identify needs and opportunities, and distilling those insights into actionable intelligence for the CHHSA open data team. Similar meetings are held by ambassadors in Fresno, Los Angeles, San Jose and San Diego, and they’ve proven invaluable in setting priorities for improvements to and extensions of the open data portal based on user needs.

The California Health Data Ambassador's first project, AsthmaStoryCA.org, used data from the California Department of Health and Human Services' Open Data Portal to tell new stories and promote access to information.
The California Health Data Ambassador’s first project, AsthmaStoryCA.org, used data from the California Department of Health and Human Services’ Open Data Portal to tell new stories and promote access to information.

In the first phase of this work, user feedback led us to partner with IDMLOCO to create AsthmaStoryCA.org, which visualizes asthma severity with interactive maps and presents a narrative that tells the story behind the data. Throughout the site’s development we used agile methodologies, including working in sprints, delivering prototypes and a working site early in the process, and keeping up a dialogue with users. And when, late in the process, a user suggested placing more emphasis on telling a story with the data, we redesigned the site. The result was a site that pleased the client, provides useful data for multiple types of users and was lauded as a key takeaway from the 2015 Code for America Summit.As we begin the second phase, we’re again leaning heavily on user feedback and focusing on an issue raised by many stakeholders statewide: The social safety net is hard to navigate. That is, information on the human services available to Californians is held by multiple referral services that collect and update data independently and on different schedules, and that aren’t interoperable. This results in a landscape of “fragmented and redundant silos” that fails to effectively serve people in need, service providers or decision-makers, said Greg Bloom, founder of Open Referral.

So how might we create a more robust, interoperable human services data ecosystem, using agile methodologies?

We begin with the hypothesis that the cause of pain in this system can be traced to the end-to-end, centralized approach to collect, store and update this information. The irony of the current fragmented referral system is that many of the existing referral services were conceived as ways to centralize previously fragmented data.

But closed systems and centralized structures don’t reflect the distributed way human services are delivered to clients. For example, while counties are agents of the state in administrating human assistance benefits, most direct services are contracted out to clinics and other community service providers.

The system’s distributed nature suggests that no proposed “one-stop shop” would likely gain much traction. Instead, we believe introducing a modular, interoperable, federated system has the highest probability of success, allowing existing referrers to continue using their current systems for as long as they need to, and requiring them only to make their information available in a common format. This will be the framework on which all of their data can be combined, compared, redistributed and built on, benefiting all users.

Modularity and agility are related concepts. While agility provides guidelines for nimble project development, modularity describes a distributed system design, enabling loosely coupled components that deliver value while minimizing complex dependencies. In fact, distributed systems are inherently modular.

As we explore the possibilities of a federated system of human services data collection and referral, we’re using another methodology closely aligned with agile development: discovery-driven planning.

Such planning is more flexible than a traditional process that sets out requirements, timelines and well defined project specifications before beginning work, and is well suited to planning for work in a new venture with which there’s little experience.

Through this scoping process, we’ll be scanning the existing environment, interviewing users, sketching out user journeys, and drafting a high-level system design and implementation plan, all while documenting our assumptions. After the scoping process, we’ll have a framework that provides enough information to make an informed decision about whether to proceed to project development and that defines what project success will look like, but is loose enough to let us pivot on the specifics at any point.

The promise of agile development and its related methodologies is project development that’s not bound by assumptions early in the planning process that may turn out to be erroneous. This should allow for projects that better meet user needs. But it requires a willingness to cede upfront project control to our future, better-informed selves. Our work is demonstrating, on a small scale at least, that this philosophy can pay dividends.

This commentary originally was published in the Spring 2016 issue of TechWire magazine.