analysis and bias

analysis The method of analysing the information collected can be simple, such as writing a report based on discussion within the team; or more complex, such as structuring and quantifying results. The purpose of analysis is to form an understanding based on a large amount of information which can be communicated clearly, and which may be useful to planning.

This section describes three general approaches to understanding and communicating the information:

• quantitative statistical analysis

• drawings and diagrams

• weighted matrix.

quantitative statistical analysis

It is common for statistics to be misinterpreted or misused. Quantitative statistics should not be used or reported unless there is an understanding of what the value actually means in practical terms, and the assumptions behind the value. To find a typical, or average, value of a variable, it is often wrongly assumed that the arithmetical mean is appropriate. The arithmetical mean sums up all the values and divides by the number of values, thereby averaging the value. Instead, the median should be used to find a typical or representative value.

values mean median comment

2, 2, 4, 7 3.75 3 both forms of analysis are reasonable

2, 2, 4,16 6 3 the median is a more representative value

drawings and diagrams

Drawings and diagrams are powerful quantitative methods for discovering patterns and trends, and for communicating them clearly.

Maps are useful for finding out information about an area and how

different groups use it, and for quickly communicating large amounts of information.

Mobility maps record, compare, and analyse the movements of

different groups in a community, and are a useful indicator of social networks.

Transects are diagrams of the main land-use zones in an area. They

compare the main features, resources, uses, and problems of different zones. Transects can be constructed by walking in a line through an area, talking to people or observing.

Seasonal calendars are ways of representing seasonal variations,

4 assessment

and income-generating activities. They can help to identify times when resources will be scarce, and the best times of the year for certain activities.

Graphs compare how one variable changes with respect to another:

for example the number of displaced people living with host families over time. Time graphs are particularly useful for showing the rate and direction of change of many variables, such as crop yields, livestock population, prices, and population size.

Time lines are flow charts, showing the sequence of different events. Daily routine diagrams compare the daily routines of different

groups of people, and seasonal changes in the routines. They can help to identify suitable times for meetings, or changes in household roles over time.

Livelihood analysis diagrams interpret the behaviour, decisions, and

coping strategies of households with different socio-economic characteristics. Variables for livelihood analysis may include the following:

• household size and composition

• number of labour migrants in the household

• livestock and land ownership

• proportion of income by source and credit and debt

• expenditures

• seasonality.

Flow diagrams are charts showing causes, effects, and relationships

between key variables; for example:

• relationships between economic, political, cultural, and climatic

factors that cause environmental degradation

• flow of commodities and cash in a marketing system

• effects of major changes or innovations (impact diagrams)

• organisation chart.

Venn diagrams are graphic representations, usually circles, showing

key institutions and individuals in a community, and their relationships and importance for decision making. When the circles are separate, no contact exists between them. When they touch,

information passes between them. The extent of the overlap indicates, for example, the extent of co-operation in decision making.

weighted matrix

A weighted matrix is used to compare options for which the same criteria have been measured. The steps for creating a weighted matrix are as follows.

• Choose the criteria to be used.

• Assign each criterion a weighting of relative importance: for

example, between 1 and 10, with 10 being highly important and 1 being irrelevant.

• Examine each criterion and assign it a value on the same scale,

say between 1 and 100: high values indicate that the criterion is positive for the option, while low values indicate that the criterion is negative.

• Multiply each criterion value by its weighting to produce an

overall ‘score’ for that criterion.

• Add all the scores together.

• Compare the final score for each option. The option with the

highest final score wins.

Some considerations to be borne in mind when developing a weighted matrix:

• Some criteria may be veto criteria: for example, if selecting sites

for a refugee camp, any site within 50 km of a border might be excluded.

• Negative criteria could be given a negative score or a negative

weighting.

• If some options yield scores that are similar, judging which

option is best becomes more subjective.

• A lack of information could mean that values or weightings

have to be estimated.

Weighted matrices are proposed as part of site selection in other guidelines (Lambert and Davis 2002). An example matrix from a fictitious situation is presented in table 4(k).

4 assessment

criterion W option A option B option C option D

V M V M V M V M legal issues 5.5 85 467.5 28 154.0 52 286.0 potential for growing food 7.5 45 337.5 60 450.0 81 607.5 access 2.0 35 70.0 45 90.0 60 120.0 environmental health 6.0 21 126.0 45 270.0 92 552.0 fuel-wood availability 6.0 59 354.0 89 534.0 48 288.0 security 8.5 36 306.0 12 veto 37 314.5 60 510.0 water sources 9.0 85 765.0 70 630.0 12 108.0 robust environment 3.0 15 45.0 11 33.0 0 0 .0 flora and fauna 4.0 35 140.0 35 140.0 80 320.0 topography 6.5 28 182,0 84 546.0 65 422.5 natural hazard safety 8.0 19 152.0 60 480.0 63 504.0

total (summing up values 2945 vetoed 3641.5 3718 in M column)

V=value; W=weighting (same for all options because it refers to each criterion); M= VxW; veto criteria: security below 25, water resources below 10

table 4(k): example of a weighted matrix, selecting a site for a refugee camp

In table 4(k), Option A is clearly inferior to options C and D. More detailed analyses or sensitivity checking might be prudent for options C and D; or, if there are no other concerns, then either could be chosen.

bias It is important to make clear to each member of the assessment,

monitoring, or evaluation team that assumptions and uncertainties are inevitable, but must be recorded. The intention of the activity is to find out what is not known, and to question what is not normal- ly questioned, rather than to confirm what is known. Ignoring or suppressing results that are not understood completely under- mines the purpose of the activity. The International Federation of the Red Cross and Red Crescent has identified the following forms of bias in assessment.

spatial bias: issues of comfort and ease for the assessors determine the

assessment site. Rather than travel into an area, the assessors conduct a ‘windshield’ survey, never leaving the comfort of their truck.

project bias: the assessor is drawn towards sites where contacts and

information are readily available and which may have been assessed before by many others.

person bias: key informants tend to be those who are in a high social

position and have the ability to communicate in a language known to the assessor. They may or may not be conscientious, insightful, or respected by those whom they purport to represent.

season bias: assessments are conducted during periods of pleasant

weather; or areas cut off by bad weather are not assessed. Thus, many typical problems go unnoticed.

mandate or specialism bias: the specialism or mandate of the

assessor blinds him or her to needs beyond his or her specialty. For example, a shelter specialist may focus on assessing needs for shelter, neglecting needs for nutrition and water.

political bias: informants present information that is distorted by

their political agenda. Assessors look for information which fits their own political or personal agenda.

cultural bias: incorrect assumptions are made, based on the

assessor’s own cultural norms. Assessors do not understand the cultural practices of the affected populations.

class/ethnic bias: the needs and resources of different classes of

people or different ethnic groups are not included in the assessment. This bias can be in the minds of local assessors; or the key informants may represent only one social class or ethnic group.

interviewer or investigator bias: assessors may have a tendency to

concentrate on information which confirms preconceived notions and hypotheses, causing them to seek consistency too early and overlook evidence that is inconsistent with earlier findings. Assessors may also tend to favour the opinions of elite key informants.

key informant bias: biases of key informants are carried into

5 la

bour