Ports in the Storm

DISCUSSION

Availability of Resources & Reports

A fundamental difficulty with this study is that it relies on the toolkits, guidelines, reports and other documents publicly accessible from the organization websites. This introduces the very real possibility (indeed, likelihood) that these documents are not a representative sample of the work being done by these organizations. However, one could reasonably assume that the documents publicly accessible (and therefore included in this study) were selected to be posted online based on their quality, their relevance to current humanitarian crises, or their treatment of current issues in humanitarian aid, and are therefore worthy of consideration here.

Two types of documents were sought for inclusion in this study: (see Chart 1)

1. Documents such as guidelines or toolkits which outline prescribed practices to be used in the field when carrying out a particular stage of the program cycle; and
2. reports, such as assessments, appraisals, or evaluations of actual aid responses or programs carried out by the organization.

Chart 1: Documents Reviewed by Type

Continue reading →

Resources for Qualitative Data-Gathering & Research Design

Berg, B. (2008) Qualitative Research Methods for the Social Sciences (7th Edition). Boston, MA: Allyn & Bacon.

Bernard, H. R. (2006) Research Methods in Anthropology: Qualitative and Quantitative Approaches (4th Edition). Lanham, MD: AltaMira Press.

Perecman, E. & Curran, S., eds. (2006) A Handbook for Social Science Field Research: Essays & Bibliographic Sources on Research Design and Methods. London, UK: SAGE Publications.

Trochim, W. M. K. Research Methods Knowledge Base. (online resource)

Additional Resources for Specific Techniques

Emergency Capacity Building Project (ECB). The Good Enough Guide: Impact Measurement and Accountability in Emergencies. Published by Oxfam GB and World Vision International. 2007. Accessible here.

Academy for Educational Development, Population Communication Services (AED/PCS). CAFS Handbook: Participatory Techniques. 2002. Accessible here.

Centers for Disease Control & Prevention (CDC), Evaluation Research Team. Evaluation Brief No. 13: Data Collection Methods for Program Evaluation: Focus Groups. July 1998. Accessible here.

CRS. Maj, M. et al. Guidance for Implementing Station Days: A Child-Centered Monitoring & Evaluation Tool. Catholic Relief Services. Baltimore, MD. 2009. Accessible here.

PolicyLink. Community Mapping – What is it? (online resource) 2002. Accessible here.

Spiers, J.A. “Tech Tips: Using Video Management/Analysis Technology in Qualitative Research.” International Journal of Qualitative Methods, Vol. 3, No. 1, April 2004. Accessible here.

STC. “Participatory Video: A Qualitative Method of Monitoring & Evaluation.” Design, Monitoring & Evaluation – Save the Children (blog). Posted 20 October 2009. Accessible here.

UCLA Center for Health Policy Research. Health DATA Program. Section 4: Key Informant Interviews. Accessible here.

US Agency for International Development (USAID). Performance Monitoring & Evaluation TIPS: Conducting Key Informant Interviews. Washington, DC. 1996. Number 2. Accessible here.

World Bank. Transect Walk and Diagramming: Procedures and Examples. (online resource) Accessible here.

PRELIMINARY FINDINGS

Based on a review of the available documents and the methods discussed therein, here are some initial findings:

METHODS FOUND BY PROGRAM CYCLE STAGE

The stages of the program cycle where data collection methodology is most emphasized, both generally and for qualitative methods in particular, are assessment, monitoring & evaluation. For other stages of the program cycle, there were some documents that addressed data collection for the appraisal stage, but very few that discussed data collection methods for program design or baseline determination. Not one of the organizations studied had documents that covered all stages of the program cycle. (see Table 1)

Table 1: Eight Organizations & Methods Found, by Stage of Program Cycle

Continue reading →

POCKET CHART

A pocket chart consists of rows of paper or cloth pockets, usually four to six horizontally and six to ten vertically.  A set of pictures is attached above the top row of pockets.  These pictures represent areas in which data are needed, such as different people’s participation in decision-making.  Each of these pictures is placed at the head of a vertical column.  If desired, pictures can also be attached down the left-hand side to indicate other variables, such as what kinds of decisions are being made by community members. In order to avoid confusion, the facilitators should use only one variable on a column at a time. (CARE 1998)

WHY-WHY TREE

The why-why tree allows for a participatory debate on the underlying causes of a specific problem. It also facilitates the prioritization of such causes. It clearly illustrates the relationship between the cause and effect. By going to the roots of an issue, this tool makes it possible to better define objectives and to choose appropriate intervention strategies. (IFRC 2008)

SWOL ANALYSIS

SWOL analysis is a powerful tool for group assessment of the issues of concern, particularly interventions or different potential courses of action.  It is based on a structured brainstorming aimed at eliciting group perceptions of the positive factors (strengths), the negative factors (weaknesses), the possible improvements (opportunities) and the constraints (limitations) related to the issue. (CARE 1998)

Continue reading →

STAKEHOLDER ANALYSIS

Stakeholder Identification and Analysis gives a comprehensive picture of all persons, groups or institutions that: i) have an interest in the operation’s success or failure; ii) may hinder its smooth implementation; iii) contribute to or are affected by the objectives of the operation, positively or negatively; or iv) can influence the situation. (WFP M&E)

Humanitarian action evolves in a diversity of contexts in terms of vulnerability to crises, social and functional dimensions, and diverging interests and issues at stake. By looking at these different factors, it is possible to identify the different stakeholders in a given environment. The general objective of stakeholder analysis is to ensure that operations take place in the best possible conditions. To this end, the interests, activities and needs of stakeholders need to be identified and taken into account in dialogue with them, so that mutually beneficial arrangements can be reached. (IFRC 2008)

On a practical level, this involves: identifying the affected people and groups in a specific environment; defining who does what, when, how, where and why; identifying individual interests; understanding power relations; defining the need for assistance; understanding operational strengths and opportunities. (IFRC 2008)

Stakeholders can be: individuals; interest groups; local authorities; services.

Continue reading →

PILE SORTING

A systematic data collection technique used to further explore a topic of interest by allowing informants to group together items according to their own system of categorization. Informants are asked to sort cards on which relevant items are written, drawn or attached. They are then asked to explain the basis on which they sorted the cards. (WV 2000)

Pile sorting is a technique aimed at studying the relations among items within a domain (which have been identified through free listing or some other method).  It is useful for discovering informants’ perceptions of the similarities and differences among items, and to look at intra-cultural variation in how informants define domains. In a pile sort, informants are asked to sort cards with names or symbols of items written on them (or actual items, such as foods or medicines), into piles or groups according to whatever criteria makes sense to them. (CARE 1998)

PROPORTIONAL PILING

Proportional Piling allows people to express their perspective of quantity by piling “counters” such as stones or beans that can then be put into percentages. (WFP M&E)

This is useful for estimating quantities and proportions, especially when working with people who are not used to quantifying data. For example, to discover the proportions of a livelihood group’s annual income to come from different sources, the procedure is as follows: (WFP 2009)

1. Collect 100 dried beans, pebbles or anything similar that are all more or less the same size.
2. Working with a focus group drawn from a specific livelihood group, ask the informants to divide the beans into piles relative to the income received from each source.
3. Count the number of beans in each pile; this number is equivalent to the percentage of annual income to come from that source.

MATRIX RANKING & SCORING

Matrix Ranking and Scoring is a way to structure the perceptions and opinions of informants so that individual or group qualities can be ranked in order of importance and the reasons for this ranking is discussed. (WFP M&E)

Continue reading →

DAILY CALENDARS

Informants are asked to describe a typical day, giving as much detail as possible about the activities that they carry out and the amount of time each takes. (WFP 2009)

Example of Daily Calendar (AED-PCS):

SEASONAL CALENDARS

Informants are asked to identify events that take place at particular times of a normal year. These include climatic events such as rains or cold weather, livelihood activities such as planting, harvesting or labour migration, cultural events such as religious festivals, and other events that are significant to the community. These are plotted on a calendar, and unusual events resulting from the current crisis are superimposed on this. (WFP 2009)

Seasonal calendars are able to diagrammatically show some of the main local activities, problems and opportunities throughout the annual cycle.  In the usual presentation, seasonal calendars are actually a series of different diagrams shown on a single sheet.  The emerging patterns of seasonally-related constraints and opportunities can help to identify the months of greatest difficulty and vulnerability, or other significant variances which have an impact on people’s lives.  (CARE 1998)

Continue reading →

PARTICIPATORY MAPPING

Participatory mapping relies on indigenous knowledge and perceptions about the physical and social environment in a community.  Respondents from a community are asked to draw or build maps of their community, using flipcharts and markers or locally available materials (including sticks, stones, grass, wood, leaves, sand, etc.).   Different interest groups (men, women, young, old, poor, wealthy) can work in teams to illustrate their particular viewpoints and interests about the local distribution of demographic, social, economic, or environmental features of their community. (CARE 1998)

MOBILITY/SERVICE MAPPING

This form of participatory mapping starts with collective discussions among groups of community members and then proceeds to drawing maps of their perceptions about the distribution of services (of any specific kind) available to residents inside and outside their specific community.  The participants are usually requested to draw their own map, e.g., on a flip-chart or on the ground, plotting features with symbols that are understood and accepted by all members of the group, regardless of literacy. (CARE 1998)

HISTORICAL MAPPING

Historical mapping uses a series of participatory mapping exercises to portray the demographic and natural resources situation of the community at different moments of its history.  Up to three maps can be drawn showing the situation as it existed at a specific time in the past (e.g., five years ago, one generation ago), at the present time, and what is expected after some time in the future (e.g., five years, or one generation). (CARE 1998)

SOCIAL NETWORK MAPPING

Social Network Mapping shows the economic, social and cultural ties and relationships that people have within a community or that exist between people from different communities. Maps of social networks can indicate ways in which different social groups benefit from these linkages. (WFP M&E)

Continue reading →

METHOD OVERVIEW

The areas in which development practitioners usually have most experience of working in a participatory way are at the stages of needs assessment, and to a lesser extent, project design. It is comparatively easy to use participatory methods for the purpose of needs assessment since the exercise is usually of a short and fixed duration, yields obvious benefits in being able to persuade donors that the needs the project is addressing are indeed the priorities of the intended beneficiary populations concerned, and does not necessarily commit the emergent project to continuing to work in a participatory manner. In short, the benefits are clear, whilst overall control over project activities – and the needs assessment process itself – need not be diminished. (CARE 1999)

Participatory appraisal is the term used to describe a process and a set of techniques for the collection and analysis of qualitative data. If a rapid appraisal is a discrete study, participatory appraisal is an extended process that can last for months or years as communities develop their own skills needed to address issues, analyse options, and carry out activities. The emphasis is often not so much on the information as it is on the process, and on seeking ways to involve the community in planning and decision-making. The key feature of participatory tools is their emphasis on participatory decision-making – enabling beneficiaries and stakeholders to analyse their own situation, rather than have it analysed by outsiders. (IFRC 2002)

Continue reading →