Reaping the rewards and measuring results -- Habitat for Humanity Int'l 1

Reaping the rewards and measuring results

By Jose Quinonez

Like many organizations, Habitat is trying to advance a particular social agenda: in this case, housing. To this end, like many other organizations, we have to couple our direct services (building houses) with public policy research and advocacy efforts (on affordable and adequate housing issues). And, like many organizations, after thousands of dollars are spent, we have to consider the question: Did the time and resources invested on advocacy have an impact?

Measuring advocacy/policy change is a difficult task, since all the external factors in a policy reform process cannot be controlled. In some cases, evaluators use control groups to isolate external forces in order to understand if the implemented program itself is the only reason that change occurred. But in terms of policy research, reform and advocacy, this simple way of looking at control groups represents its own set of difficulties and can limit evaluation of impact.

While it may be difficult to prove immediate impact in advocacy, progress can certainly be measured, and more importantly, we can definitely show long-term impact. I suggest that depending on the advocacy initiative—and the conceptual model in which a set of advocacy tactics are grounded—the impact of advocacy can be both short-term (expected outcomes) and long-term (expected results).

After reviewing a broad range of impact categories and sample indicators of progress from other organizations with strong advocacy records (World Vision, Oxfam America, CARE and Save the Children), the same types of desired outcomes emerge. Grouped into four categories, I am sharing these not to duplicate or correct any other existing sources, but rather to consolidate, emphasize and confirm what appear to be the important areas of measuring change in advocacy and policy work.

  • Social indicators: These are used to measure changes in the social level of the people affected by the intervention, i.e., access to services, affordable housing, education, secure tenure, health, gender equality, adoption of new practices, etc. Social-change outcomes are most likely to occur after several other changes have happened, such as public awareness, policy adoption, etc.
  • Economic indicators: These are often a subproduct of the social indicators and are used to measure changes in the economic level of the targeted population (often as a direct result of a change in the social level), i.e., access to credit and loans. As with social indicators, economic outcomes occur after other changes have happened as a result of the advocacy initiative.
  • Organizational capacity indicators: These are used to measure changes in the skill set, staffing and leadership, organizational structure and systems, finances and strategic planning among nonprofit organizations and formal coalitions that plan and carry out advocacy and policy work. Organizational capacity outcomes describe the level of self-organization, social control, and distribution of benefits, decision-making processes and leadership levels that the targeted population and the implementing agency may be acquiring as a result of the advocacy initiative.
  • Policy change indicators: These are used to measure changes in policies, decision-making processes, social participation, attitudes and systems as a direct result of advocacy and lobbying activities, i.e., changes in adequate and affordable housing laws, changes in rent control systems, secure tenure, etc. Even if there is not a change in a particular policy as a result of the advocacy initiative, there are many intermediate results and outcomes that can be measured. Despite whether there is a change in a particular policy or not, there can be changes in awareness and increased agreement on the definition of a problem (i.e., common lingo). There can also be changes in beliefs and attitudes and, more importantly, changes in attitudes and values (social and economic outcomes).

An intermediate result of any advocacy initiative, regardless of the result in policy change, can be an improvement of the organizational management of organizations carrying out advocacy and policy work, as well as an improved ability to form strategic alliances. This is often coupled with improved organizational stability (organizational capacity outcomes).

Finally, advocacy initiatives often involve policy research that most times results in the development of “white papers” that tend to expose problems with a current policy, often providing a tool for educating policy-makers and in some cases creating a watchdog function (policy change outcomes).

Jose Quinonez is associate director of advocacy training at HFHI’s Government Relations and Advocacy office in Washington. As a part of the Organizational Learning team, Jose has cross-functional responsibilities in advocacy planning, training and facilitation in the United States and internationally. He has a master’s degree in international administration from the School for International Training in Brattleboro, Vermont. He has extensive experience in monitoring and evaluation; short- and long-term participatory training; and organizational and community capacity-building.

His e-mail address is JQuinonez@habitat.org.