Process Monitoring Consultancy Services for SIPP

(1)

Social Investment Program Project (SIPP)

Report on Training

on Process Monitoring

in association with

ITAD

Information, Training And Development

12, English Business Park

English Close

Hove

BN3 7ET

U.K.

Telephone: +44 1273 7654 250

Fax: +44 1272 7653 251

e-mail: mail@itad.com

CNRS

Center for Natural Resource Studies

House # 14 (2

nd

Floor), Road # 13/C

Block # E, Banani

Dhaka-1213

Bangladesh

Telephone: +880-2-9886700

Fax: +880-2-9886700

email: cnrs@dominox.com

June 2005

(2)

SIPP – Process Monitoring

Training Report

Glossary of Abbreviations and Acronyms

CF Community Facilitator (of CSO)

CIW Community Infrastructure Works

CSO Community Support Organisation

FF Field Facilitator

FGD Focus Group Discussion

NGO Non Governmental Organisation M&E Monitoring and Evaluation MEL Monitoring, Evaluation and Learning

MIS Management Information System

NFR Note For the Record

PAST Project Appraisal and Supervision Team

PM Process Monitoring

PMA Process Monitoring Agency PMC Project Management Committee

PO Participating Organisation

PPF Pilot Private Financing RO Research Officer (of PMA) SAP Social Assistance Program SDF Social Development Foundation SIPP Social Investment Program Project SRO Senior Research Officer (of PMA)

SW Social Worker

VDC Village Development Committee

Acknowledgements:

This report was written and edited by Julian Barr and M. Anisul Islam. The Training Course Outline and Trainers’ Notes were written by Abigail Mulhall.

(3)

SIPP – Process Monitoring

Training Report

Contents

i

1.

Executive Summary

1

2.

Introduction 2

3.

Details of Trainers and Participants

4

4.

Participant Evaluations of the Course

6

4.1. ACHIEVEMENT OF COURSE OBJECTIVES 7

4.2. QUALITY OF TRAINING 9

4.3. BUILDING ON THIS COURSE 11

5.

A Selection of Course Outputs

13

6.

Annex 1. Powerpoint Slides Used in the Course

17

7.

Annex 2. Training Course Outline (for Participants)

18

(4)

Training Report

1

1.

Executive Summary

This report provides feedback on two short courses on process monitoring, run for

organisations involved in the implementation of the SIPP.

A half day course was run for the various partner organisation, covering a general

introduction to process monitoring and use of the process monitoring tools, notably Report

Cards, that they would have to use.

A one and a half day course was run for SDF staff, covering in more detail the other process

monitoring tools in use, and the outputs from the monitoring, especially the Notes for The

Record.

Evaluation results are presented showing that the course was rated as

very good

on average,

by the participants. They liked the format of the course, but considered that a larger venue and

a slightly longer course would have been an improvement. Consideration may need to be

given to delivering a refresher once the Report Cards tool is in extensive use.

(5)

SIPP – Process Monitoring

Training Report

2

2.

Introduction

This report summarises the results of a two training courses in Process Monitoring, delivered

by ITAD and CNRS at the offices of SDF on 17

th

and 18

th

of January 2005.

In summary, the two courses covered:

•

Course 1:

A half-day course, covering introduction to process monitoring and the

role of the process monitoring agency, Wealth Ranking, and Report

Cards

•

Course 2:

A one and a half day course, covering other process monitoring tools in

detail: field assessment, focus group discussion, and case studies

Full details of the participants is given in Section 3. However, Course 1 was primarily

designed for staff from SDF, plus staff from the partner organisations implementing SIPP

(CSO, PO, PAST, etc), while Course two was design for SDF staff only.

Course details are as follows:

Course 1

Title

Wealth Ranking and Report Cards

Duration

4 hours

Learning

Objectives

Î

Participants gain skills that enable them to carryout a wealth

ranking

Î

Participants can use report cards for facilitating

self-assessment of performance and progress by VDCs or other

community groups

Participants

Staff from SDF< Community Support Organisations, Participating

Organisations, and other project partner organisations

Content

Introduction to the course; participant

introductions

10 mins

Outline of SIPP and the role of the Process

Monitoring Agency

15 mins

Introduction to Process Monitoring

15 mins

Introduction to Wealth Ranking

90 mins

Break

(6)

Training Report

3

Course 2

Title

Process Monitoring

Duration

1.5 days

Learning

Objectives

By the end of the course participants are able to:

Î

Use the process monitoring tools

Î

Implement the process monitoring system used in SIPP

Participants

Staff from SDF

Content

Day 1

Course 1

4 hours

Lunch

60 mins

Introduction to Part 2 of the course

10 mins

Field Assessment

90 mins

Break

15 mins

Field Assessment (

continued

) 120

mins

Day 2

Summary of Day 1

30 mins

Note For the Record (NFR)

90 mins

Break

15 mins

Note For the Record (NFR) (

continued

)

60 mins

Focus Group Discussions (FGDs)

30 mins

Lunch

60 mins

Case Studies

90 mins

Break

15 mins

Wrap-up/Summary 30

mins

(7)

Training Report

4

3.

Details of Trainers and Participants

The training course was delivered by Julian Barr from ITAD. Julian led the design of the

process monitoring system for SIPP, and is both an experienced M&E specialist and trainer.

He was the process monitoring specialist for the World Bank District Poverty Initiatives

Project in Andhra Pradesh, and has been directing the Rural Livelihoods Evaluation

Partnership, which is contracted to make annual evaluations of all DFID’s rural livelihoods

projects in Bangladesh.

The course was facilitated with M. Anisul Islam, Ashitava Halder and Md Alam Hossian from

CNRS. M. Anisul Islam is the Local Coordinator of SIPP’s process monitoring, and was

closely involved in developing the design system for SIPP. CNRS and Anisul both have long

experience in implementing CDD projects in Bangladesh. Ashitava Halder has been leading

the Jamalpur district SIPP process monitoring team. He was directly involved in the process

mapping exercises and report card development process for SIPP. Ashitava is experienced in

M&E systems and worked as Monitoring Officer in the UNDP supported SEMP project. Md.

Alam Hossain has been leading the Gaibandha district SIPP process monitoring team. Alam,

being a PDO, has long experience in monitoring and evaluation of different

CARE-Bangladesh projects such as NOPEST and LIFE-NOPEST.

The following participants attended the two courses:

Sl #

Name of Participant

Designation

Name of

Organization

Participants for Course 1 and Course 2

1.

M. Hafizuddin Khan

Managing Director SDF

2.

AKM. Rahmat Ullah

GM

SDF

3.

Muhammad Abu Taher Khan

GM

SDF

4.

Masud Al Mamun

Manager

SDF

5.

Syed Md. Mosuddeque Hossain

TM

SDF

6.

M. Shahjahan Muntu

SAM

SDF

7.

A. K.M. Mahbubur Rahman

SAM

SDF

8.

F. U. Ahmed Mia

TA (water)

SDF

9.

Shams Uddin Md. Rafi

CDM

SDF

10.

Md. Kamal Bashar

TM

SDF

11.

Ireena Jahan

Manager, Procur.

& Adm.

SDF

12.

Morshed Chowdhury

TA (H)

SDF

13.

Md. Lutfar Rahman

Manager, MFL

SDF

14.

Nazrul Alam Sarder

CDM

SDF

15.

Md. Abdul Kayum

Accounts Officer

SDF

16.

Md. Shoharab Ali Khan

Finance Manager

SDF

17.

M. I. M. Zulfiqar

MIS Specialist

SDF

18.

Md. Kamal Basher

TM

SDF

19.

Ashitava Halder

S. Research

Officer

(8)

Training Report

5

20.

Md. Alam Hossain

S. Research

Officer

CNRS

21.

Md. Touhidul Islam

Research Officer

CNRS

22.

Maheen Newaz Chowdhury

Research Officer

CNRS

Participants for Course 1 only

23.

K.N. Sarker

Team Coordinator

ESDO

24.

Md. Abu Hanif

Field Supervisor

ESDO

25.

Maruf Ahmed

F.S

ESDO

26.

M. Zakir Hossain

F.S

DORP

27.

Foyzoon Nahar

TC (SAP)

TMSS

28.

Ekramul Haque

ATO (SAP)

TMSS

29.

Ms. Shahzadi Begum

PC. (SAP)

TMSS

30.

Amir Hossain Mollah

PC (DAM)

DAM

31.

Kakhal Chandra Das

TC

DAM

32.

Banari Saha

Sr. Program

Officer

READ

33.

Fahmida Karim

D.D. Research

READ

(9)

Training Report

6

4.

Participant Evaluations of the Course

Participants were requested to complete course evaluation sheets anonymously, rating their

satisfaction with the course and providing comments on areas of particular success or where

improvements could be made.

Tables 1 and 2 summarise the participant responses. These tables show the number of

participants applying which rating to each criterion – for example, 2 (out of 12, = 17%) of

participants rated as

excellent

their ability by the end of the course to explain what is meant

by process monitoring, while 7 (out of 12, = 58%) rated their ability as

very good

). Table 1 is

from SDF staff, and Table 2 is from participants from all other organisations.

Table 1: Summary of course evaluations from SDF staff (n = 12)

How do you rate the training you have received? Against each of the performance criteria, indicate your rating of the course by ticking the relevant cell in the right hand column

Excellent (5) Very good (4) Satisfactory (3) Unsatisfactory

(2) _Very

Unsatisfactory

(1)

Average

By the end of the course, participants will be better able to:

1. Describe what is meant by process monitoring

2 7 3 3.9

2. Use the tools of process monitoring 2 5 5 3.8 Achievement of training objectives The extent to which the training objectives have been achieved…

3. Support communities, especially

VDCs, to use the PM tools 4 4 4 4.0

• Visual aids: the quality and the

appropriateness of slides and handouts

3 7 2 4.1

• Exercises: the relevance and

practicality of the exercises

4 3 5 3.9

Quality of training materials

• Participation: the extent to which

all trainees were valued, and able to participate

3 3 6 3.8

• Trainers: the conduct of the

trainers in delivering the materials and interacting with the

participants

5 5 2 4.3

Quality of training delivery

• Time-keeping: the time allowed

for delivery of materials and the completion of exercises

3 4 5 3.8

Quality of

training venue • appropriateness of the training The training room: the room to the type of training provided

(10)

SIPP – Process Monitoring

Training Report

7

Table 2: Summary of course evaluations from staff of other organisations involved in

SIPP (n = 10)

Attributes

Excellent (5) _{Very good (4)} _{Satisfactory (3)} _{Unsatisfactory}

(2) _Very

Unsatisfactory

(1)

Average

By the end of the course, participants will be better able to:

1. Describe what is meant by process monitoring

6 4 4.6

2. Use the tools of process monitoring 4 2 4 4.0 Achievement of training objectives The extent to which the training objectives have been achieved…

3. Support communities, especially

VDCs, to use the PM tools 1 5 4 3.7

• Visual aids: the quality and the

appropriateness of slides and handouts

4 5 1 4.2

• Exercises: the relevance and

practicality of the exercises

4 4 2 4.3

Quality of training materials

• Participation: the extent to which

all trainees were valued, and able to participate

4 3 3 4.1

• Trainers: the conduct of the

trainers in delivering the materials and interacting with the

participants

4 6 4.4

Quality of training delivery

• Time-keeping: the time allowed

for delivery of materials and the completion of exercises

1 7 1 1 3.8

Quality of

training venue • appropriateness of the training The training room: the room to the type of training provided

2 2 4 1 1 3.3

4.1.

Achievement of Course Objectives

Figure 1 below shows that the participants rated all the training objectives as having been

achieved, to at least a satisfactory degree. Participants’ ability to describe what is meant by

process monitoring was achieved most successfully (achieving an average rating of 4.2, =

84.5%), while participants’ ability to use the tools of process monitoring, and participants

ability to support communities, especially VDCs, to use PM tools, both received average

rating of 3.9, = 77.3%.

(11)

Training Report

8

Figure 1: Participants’ Percentage Rating of Achievement of Training Objectives

During the introductory session of the training, the trainer asked participants to give their

expectations of what the course would explain. This list is as follows:

Participants’ expectations:

•

Why monitoring?

•

What is PM

•

Why PM is needed?

•

What is monitoring indicators?

•

How it be implemented?

•

PM tools numbers and use

•

Scope and limitations

•

Who will monitor?

•

Difference between Process monitoring and Progress Monitoring

•

Impact and output of PM

•

Monitoring process of PM

•

Advantages and disadvantages of PM

•

Reporting of PM

•

What benefit derived from PM (for a short duration project)

•

How it support in decision making process?

•

Difference between participatory monitoring and process monitoring.

0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% 50.0% % r e sp o n se Exce llen t Very g ood Satisf actory Unsa tisfac tory Very U nsat isfa ctor y Participant rating Describe what is meant by process monitoring Use the tools of process monitoring

Support communities, especially VDCs, to use the PM tools

(12)

SIPP – Process Monitoring

Training Report

9

During the final session of the course, participants looked at this list again. It was agreed that

the training course had addressed their expectations.

4.2.

Quality of Training

Figure 2 below shows that most aspects of training quality were rated as

very good

(average

rating 4.0, = 79%). The most successful aspects were the training quality of the visual aids

(rating 4.2, = 84%), and the conduct of the trainers in delivering the materials and interacting

with participants (average rating 4.3, = 86%).

Figure 2: Participants’ Percentage Rating of Quality of Training

Although the average ratings were good across the board (Table 3), there were two noticeable

areas which received some low ratings; this can be seen in Figure 2. These areas were

time-keeping, and the training room. As can be seen from the participants’ feedback comments,

there was a feeling that the training would have benefited from being a little longer, probably

3 days in total for both courses:

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% % r e s pons e Exce llent Ver y goo d Sati sfact ory Unsa tisfa ctor y Very U nsati sfac tory Participants' ratings

Visual aids: the quality and the

appropriateness of slides and handouts Exercises: the relevance and practicality of the exercises

Participation: the extent to which all trainees were valued, and able to participate

Trainers: the conduct of the trainers in delivering the materials and interacting with the participants

Time-keeping: the time allowed for delivery of materials and the completion of exercises The training room: the appropriateness of the training room to the type of training provided

In summary, the course successfully delivered an understanding of process monitoring and

its application in the Social Investment Program Project. It was most successful in ensuring

the participants were able to describe what is menat by process monitoring.

(13)

Training Report

10

Increase time for each section Time will extended to the process

There should be enough time for the training course Should provide more time

Time of exercise should be increased Time is not sufficient

Should organize 3 days training

Training duration must be extended and provide training certificate.

The duration of training is always problematical as short courses result in not covering

material in sufficient depth, but long courses conflict with people’s other commitments and

often result in lower attendance rates. Nonetheless, the comment about duration has been

noted by the trainers.

Several of the participants expressed dissatisfaction with the training venue. Eg: “

Venue or training is more closely. So, we are not scope to face or eye contact. I think in this training would be arrange in ideal training centre/room/venue

.” The two problems related to this are

the lack of space for the interactive/participatory elements of the training, and the long narrow

aspect of the room, which puts some participants a long way away from anything projected on

the screen at the front. It would be useful for SDF to note this for future training.

Table 3. Participants’ Average Rating of Quality of Training

Average

rating Average %

Visual aids: the quality and the appropriateness of

slides and handouts 4.2 84%

Exercises: the relevance and practicality of the

exercises 4.0 81%

Participation: the extent to which all trainees were

valued, and able to participate 3.9 78% Trainers: the conduct of the trainers in delivering the

materials and interacting with the participants 4.3 86% Time-keeping: the time allowed for delivery of

materials and the completion of exercises 3.8 76% The training room: the appropriateness of the training

room to the type of training provided 3.5 70%

In summary, training quality was rated very highly. The training powerpoint presentation

and handouts and the interaction of the trainers/facilitators with the participants was very

much appreciated. However, participants felt that a 3 day course may have been warranted,

and they believed that a room better designed for training would have been beneficial.

(14)

Training Report

11

4.3.

Building on this Course

The course evaluation asked participants which parts of the training they like best, and why;

which they considered could be improved, and how; and which parts of the training did not

meet their requirements. The responses to the evaluation questionnaires are summarised

below:

Table 4. Summary of evaluation questionnaire responses.

Best parts of the training

Sessions which could be improved

Course design

Course outline and concept of process monitoring, implementation process Description of Process of Monitoring.

Due to set up an simple example and use projector

Process monitoring tools

Use of PM tools because we will be able to monitor the process

Process monitoring tools technique. Tools of PM

PM tools, to be elaborated all the

process

Field assessment tool and quality criteria Use the tools of PM

I like report card because the report format is very easy.

I like report because the format easy

NFRS

NFR issues - SDF can execute their

opinion on formats and contents

The issue related to NFR. Because it is the output of the whole process

Structure of NFR

Process of developing NFR and flow

charts

Comments on NFR part

Minute taking

Minute writing or taking notes for

minutes, may by through exercise The last part that is note writing

Group work & role playing

Group work. As I have the opportunity of communicating by feelings & experience

Group work- learning by doing Role play parts on wealth ranking Group exercise

Group work, reading the materials and time consideration for conceptual direction

The group exercise session should be improved. It could be the four members group.

Role play ranking of report card of VDC FGD. They improved by facilitation VDC scoring sessions improved Should be improved VDC

(15)

SIPP – Process Monitoring

Training Report

12

From this summary it can be seen that the participants were in general very positive about the

course, however it would appear that from the trainers’ perspective, more attention needs to

be paid to two areas. While there was a mixed response to these two areas (some positive

feedback, and some constructive criticism), the trainers have noted that some changes could

be made to suit all participants. The two areas were: (i) training in understanding the

development and use of the NFR. NFRs are only written by PMA staff, thus the issue for

trainees is mainly in understanding how to use the recommendation contained therein. And

(ii) running the group work sessions, and especially the role playing of process monitoring a

VDC. As noted above, more space in the training venue may have helped here.

One participant commented “

The training is very much helpful. Refresher is needed.

” It is thus

worth SDF considering if follow training is indicated at some point in the future, and if so,

whether SDF’s in-house training section could provide this.

In summary, the overall course design, and the parts dealing with the process monitoring

tools were well received. Some participants considered the group work/role play and the

section on NFRs could have been improved. As commented by one participant, a

refresher may be indicated, particularly once the various organisations have started to

implement the report cards.

(16)

Training Report

13

5.

A Selection of Course Outputs

The following are examples of outputs produced by the participants during exercises in the

course:

1. Outputs from the

wealth ranking

exercise; from two different groups. Criteria used to

differentiate between four wealth groups:

Rich Middle

Criteria Persons Criteria Persons

1.Reside in building/half-building.

2.Cultivable land more than 250 decimal.

3.Regular service/ large business. 4.Have a lot of assets 5.Have access to education

facility.

6.Can maintain regular savings. 7.have other sources of income such as poultry, cow rearing, fishery etc.

8.have electricity facility at home. 1. Parimal Das 2. Bulbul Islam 3. Zahurul Islam 4. Ois uddin 5. Rahmat Ullah 6. Karim Uddin

1.Reside in tin shade house. 2.Cultivable land 50 to 250

decimal.

3.Low paid service/ small business

4.Have assets such as radio, bi-cycle, TV etc.

5.Access to intermediate level education facility. 6.Have some sort of savings.

1. Bulbul Ahmed 2. Salma Begum 3. Azharul Islam 4. Monwar Hossain

Poor Very Poor

1. Land Hold less than 50 decimal

2. Get employed more than 6 months.

3. have domestic animal 4. Have meal twice a day.

1. Karim Uddin 2. Resma Bewya 3. Amir Ali 4. Hafizur Rahman 5. Zahidul Islam 1. No house/homestead 2. No Land

3. No access to education facility 4. Get employed less than 6

months.

5. Woman headed family or disable person in family. 6. No access to micro credit

facilities.

7. Have meal only once in a day.

1. Zaheda begum 2. Arif Mia 3. Rahim Uddin 4. Rokeya Begum 5. Alam Mia Rich Middle

Criteria Persons Criteria Persons

1.Have more than 250 decimal land

2.Monthly income more than Tk. 2000

3.Have enough resources 4.Have access to higher level

education facility

1.Monwar Hossain

2.Rahmat Ullah

3.Zahrul Islam

1.Have less than 250 decimal land

2.Monthly income not more than Tk. 2000

3.Have some significant resources

4.Have access to high level education facility 1.Parimal das 2.Hafizur Rahman 3.Ois Uddin 4.Azharul Islam 5.Salma Begum 6.Bulbul Ahmed

Poor Very Poor

1.Have not more than 50 decimal land

2.Monthly income less than Tk. 1500.

3.Have no resources 4.Have little access to

education facility 5.Reside in tin shade house.

1.Zahidul Islam 2.Reshma Bewya 3.Amir Ali 4.Bulbul Islam

1.Have only homestead. 2.Monthly income less than

Tk. 800. 3.Have no resources 4.Have no access to

education facility 5.Reside in tin shade/straw

made house. 1.Zaheda Begum 2.Alam Mia 3.Arif Mia 4.Milon Rani 5.Rokeya Begum 6.Rahim Uddin

(17)

SIPP – Process Monitoring

Training Report

14

2. Results of 4 groups’ (VDCs 1 – 4) scoring of the hypothetical Shapla village VDC, using

the

Report Cards

tool:

Sl. No. Indicator VDC-1 VDC-2 VDC-3 VDC-4 1 Regular Meeting 4 3 2 5 2 Record Keeping 3 1 1 3 3 Involvement of Poor 5 4 3 3 4 Benefit of Poor 5 3 4 2 5 Involvement of Woman 5 5 4 5 6 Benefit of Woman 5 3 3 3 7 Leadership 5 5 4 5 8 Social Norms 3 2 3 2 9 Linkages 2 1 4 2 10 Maintenance 2 1 2 2 11 Active Planning 4 3 2 5 12 Unity 4 2 3 3 13 Information 5 2 4 3

3. Summary of participants’ criteria for process quality, to be used when undertaking the

Field Assessment

:

Active community Participation

Awareness about process of the stakeholder

Ensure participatory decision making

Quality of leadership

Need and expectation of poor and hardcore reflected in the plan To ensure bottom up

–planning

Clear understanding about the implementation process

Active participation with equal authority and rights

Felt needs are

prioritized in sub-project Maximum presence and active participation assured in PRA session

Villagers know about objective, goal, approach and strategy of SIPP

Proper information

Sharing

4. Results from

Field Assessment

exercise. Participants had to work in groups to produce a

list of questions that they would ask in s

emi-structured interviews

(CAP and pilot private

utilities programme), or factors/issues they would aim to look out for in

participant

observation

:

Semi-structured interview (CAP):

i.

Are/How the villagers involve in prioritization of needs/CAP?

ii.

How the people know about their prioritized sub-project?

iii.

Did the villagers contribute their portion?

iv.

How the community will be benefited from this road sub-project?

v.

Are the PMC and purchase committee formed properly to implement the sub-project?

vi.

Do the people know the cost of the sub-project? Was it shared with the community?

vii.

How the project will be maintained afterward?

viii.

Do you think the quality and cost is reasonable in compare with others?

ix.

What is the feeling about the sub-project?

(18)

SIPP – Process Monitoring

Training Report

15

Participant Observation (CAP formation)

i.

Representation from all para/every corner of the village.

ii.

Male/female participants were present and participating?

iii.

All/majority participants expressed their opinion/comment

iv.

women's opinion are accepted

v.

Whether all participants stayed during the whole PRA?

vi.

Facilitator lead the PRA session directly

vii.

felt needs identification and prioritization

viii.

Opportunity to express opinion

ix.

Timely start and timely end

x.

Sharing of collected information in the planning.

Pilot Private Utility program (using semi-structured interview)

i.

How the information was disseminated to you?

ii.

Why you have decided for rural piped water supply?

iii.

Why you are ready to share the cost?

iv.

What should be your share?

v.

How you will collect tour share/deposit?

vi.

What benefit do you expect?

vii.

How you will maintain the water supply system after the sponsor withdraws?

viii.

How you will bear the maintenance and operation cost?

ix.

Do you consider that the total cost is affordable to you?

x.

Do you think that water will be Arsenic and bacteria free?

5. One group’s assessment of the strengths and weaknesses in content and format of two

sample

NFRs

:

NFR Group Work (NFR -1)

Strength

Contents Format In content women participation focused.

Comments are specific and clear regarding joint meeting of VDC & PMC and female

respondents.

Financial problem identified specifically. Villagers are aware about sub-project

status/progress.

Comments are placed in bold form.

Weakness

Contents Format Misinformation about the roles of VDC & PMC.

Contents not pin pointed.

Specific recommendation is absent.

Comments are not provided under way heading. Date and period of reporting are not mentioned. Contents of NFR-1 is not clearly described eg .

Villagers informed but not quantify how many villagers.

Misinformation in NFR 1(2&3). No recommendations.

Place/date are not specified in NFR. Did not follow the proper R- structure.

Not organized. Format is absent.

(19)

SIPP – Process Monitoring

Training Report

16

NFR Group Work (NFR - 2)

Strength Contents Format

Process recommendations are well defined (eg 2 & 3).

Specific statement.

Process recommendation is specific under SL-2. Field assessment exercise was done.

Quality of process is well identified. Helpful in future planning.

Structure of the report is good. Language is very simple and

understandable.

Format seems to be better. Content described under specific

reading.

Weakness

Contents Format

Process observation wrongly described.

Irrelevant issues were raised. eg . Office bearers. Contains of NFR are not reflected the critical

path.

Date /places are not specified.

Contents with some misinformation and

incomplete information.

Process recommendation is wrongly suggested

under SL.1.

Under SL.3 Wrongly observed the process.

Absent of matrix/matrix are not used.

Summary matrix is absent.

6. Plenary recommendations from

NFR

exercise, for strengthening

NFRs:

Factual information is good

Key success should be clearly highlighted

Date and place should be clearly mentioned

Critical path-project cycle should be followed (VDC not PMC, submit the final

sub-project report)

Language should be specific and clear

Managers like to have specific recommendation

NFR should be submitted with summery matrix

NFRs issues should be discussed in the monthly meeting at the district level and

decisions should be incorporated in the next months report/NFR

Recommendation should be supported by reasons /rational

Critical issues shall be submitted to MD, SDF directly

Follow up of implementation of recommendation by quantity and shared in the

quarterly meeting.

(20)

Training Report

17

(21)

ITAD and CNRS

December 2004

SIPP Process Monitoring Training

1

Social Investment Program Project (SIPP)

Training in Process Monitoring

17-18 January 2005 SDF

SIPP Process Monitoring TRAINING 2

Overview

• COURSE 1[0.5 day; 17 Jan]

– An introduction to wealth ranking and report cards

– Process tools used/facilitated by project partners

• COURSE 2[1.5 days; 17 – 18 Jan]

– An overview & training in the main process tools, used by the PMA

Course 1: Outline

• Introductions (course & participants) • What is Process Monitoring

• Wealth Ranking • Report Cards

House keeping

• Timing for the training sessions • Evaluating the training

• Refreshments • Toilet facilities

•

ªªª

•

Course 1: Learning Objectives

• Your expectations…

• Participants gain skills that enable them to carryout a locally relevant Wealth Ranking

• Participants can use Report Cards for facilitating self-assessment of

performance and progress by VDCs or other community groups

What is Process Monitoring?

• Processes are “coherent sets of actions that produce

outcomes”

• Process Monitoring = “observing and analysing how activities are done”

• PM = a management tool to generate information for institutional learning and taking corrective action in innovative and adaptive projects that involve a high level of community participation

(22)

ITAD and CNRS

December 2004

2

Why do process monitoring?

• To learn about how to improve the way things are done in projects

• To generate information for institutional learning

Why do process monitoring? –

The Role of the PMA

To provide:

• SDF management a more direct and objective communication of the processes and qualitative changes taking place throughout the project, so that corrective measures can be taken where necessary.

• an entry point for scaling up good practice lessons • a feedback mechanism from the communities on how to

do things better, as well as to get a clear assessment of hard-to-measure shifts in community attitudes and practices.

PM Tools

Documentation of better practices and lessons learnt

TOOL 6: Case Studies

Monitor the processes involved in implementation of the utilities programme

TOOL 5: Field Assessment (Utilities)

Verify issues raised during the Field Assessments

TOOL 4: FGDs

Routine monitoring of project processes in sample villages TOOL 3: Field

Assessments

Self-assessment of VDCs performance TOOL 2: Report Cards

Establish poverty status of households in project villages (baseline) TOOL 1: Wealth Ranking

Purpose

Tool

TOOL 1: Wealth Ranking

Wealth Ranking - purpose

• Establish the main criteria of poverty in different villages, as seen by villagers

• Establish in which poverty categories different households in a village are placed (by villagers)

• Verify who are the poor and very poor • Monitor whether HHs move between wealth

categories (due to SIPP)

Wealth Ranking in SIPP

• When is it used?

– Start of programme – Two further intervals

• Who uses it?

– CSO

– Field Facilitators of the POs – Impact Evaluation Agency

(23)

ITAD and CNRS

December 2004

3

Wealth categories

Very / hardcore poor Poor Middle Rich Household 1 Household 2 Household 3 Household 4

Wealth ranking - criteria

Full-time, salary Rich Middle Day labour Poor V.Poor Criteria occasional employment Condition of dress Food availability Land ownership

TOOL 2: Report Cards

Report Cards

• A tool for reflection and learning • For self-assessment of performance • To enhance transparency, encourage

debate and discussion

• To improve process and practice

How to monitor change?

• SIPP is a complex project

Indicators

• Signs we look for to tell us if change is happening

• Signs that tell us not only ‘how much’ change is happening, but also ‘how well’ it is happening

(24)

ITAD and CNRS

December 2004

4

Report Card

Holding regular meetings

Report Cards - Scoring

• Scoring: 5 = best, 1 = worst • Collating scores

Score for each indicator (taken from individual quarterly score sheets) Date Quarte r 1 2 3 4 5 6 7 8 9 10 11 12 13 Total Q1 Q2 Q3 Q4 Total

SDF’s use of Report Cards

Indicators Village Development

Committee Regular meeting keeping Record Involvement

of poor

Benefit of poor Involvement of

women Benefit of women Leade rship Observance of social norms Linkages Maintenanc e of sub-projects Active planning Unity and conflict

resolution Information collection and dissemination Total VDC-1 VDC-2 VDC-3 VDC-..

(25)

ITAD and CNRS

December 2004

1

Social Investment Program Project (SIPP)

Training in Process Monitoring

17-18 January 2005 SDF

Course 2: Learning Objectives

• Participants are able to use the full set of process monitoring tools

• Participants understand and can implement the process monitoring system for SIPP

PM Tools

Documentation of better practices and lessons learnt

TOOL 6: Case Studies

Monitor the processes involved in implementation of the utilities programme

TOOL 5: Field Assessment for Utilities program

Verify issues raised during the Field Assessments

TOOL 4: Focus Group Discussions (FGDs)

Routine monitoring of project processes in sample villages TOOL 3: Field

Assessments

Self-assessment of VDCs performance TOOL 2: Report Cards

Establish poverty status of households in project villages (baseline) TOOL 1: Wealth Ranking

Purpose

Tool

TOOL 3: Field Assessments

Field Assessment Tool

• Main tool used in PM

• Comprises: Participant Observation and Semi-Structured Interviews

• Assesses project processes against indicators of process quality

• The end product is a Draft Note for the Record (NFR)

Field Assessment Tool

• Is not rocket science • Requires:

– good understanding of project objectives and working practices

– good understanding of social norms in rural areas – strong skills in participatory / sociological

techniques

– good enquiry / detective skills – very good analytical and writing skills

(26)

ITAD and CNRS

December 2004

2

Quality Criteria

• Inclusiveness • Transparency

• Governance and empowerment • Sustainability

• Cost effectiveness • Information sharing

Participant observation

• Gathering data (quantitative and qualitative) through observation and enquiry

• About learning through watching and listening

Semi-Structured Interviews

• Guided interview

• Topics for discussion are predetermined • Questions used are flexible and change

according to responses

• Open ended questions are used

Field Assessment Exercise

• Three groups

– Participant observation – SSI

– SSI on pilot private utilities program

Field Assessment Exercise

• Participant observation

– 10 points to look for on process qualityduring PRA exercises for CAP

• SSI

– 10 questions to ask on process quality in road sub projects

• SSI on pilot private utilities program

– 10 questions to ask on process quality in community preparation for involvement in PPUP

What is Field Assessed?

(27)

ITAD and CNRS

December 2004

3

SIPP Process Monitoring TRAINING 13 Steps How the step was/will be_intervened Indicators

IC campaign (village level)(2)

8-10 village was allocated to every CF for IC campaign The IC campaign was done through door to door visit, informal meeting in market, shops, announcement through loud speaker of local mosque, leaflet/poster distribution, mass gathering etc. Sometimes discussion was held in the marriage ceremony and quarrel gathering.

Village survey

Developed data collection form Collected data on general issues, infrastructure,

education, health, occupation, sanitation, water, land resource, cooperative, income, social bondage, crops etc. through FGD in the villages and collected from union council, bureau of statistics. Conducted 3-12 FDGs in every village and calculated

mean for different values Enthusiasm test and

village selection

Forms are supplied by SDF 6 indicators were evaluated; 3 through group

discussion and other 3 through interviewing 20% HH of the village (every interviewed HH are followed by some un-interviewed HH) Summing the scores for ranking the villages Villages with higher scores are selected PRA (3) At first the CF goes to the village to select the venuedate and time

Villagers of all levels participated The CF facilitated the sessions Villagers themselves proposed to prepare the map Firstly, drawing several maps (boundary, direction and

mapping) on earth in small groups Later the best drawing on earth was copied to the large

brown sheet

The session was held at morning or evening for 3-3.5 hours

People have build up attitude for wo in unity People's knowledge about the projec Distributed posters leaflets

Mapping

Raised voice and knowledge of the people about participatory planning processes Villagers endorsement in the prepare

materials and plan

• Process maps - critical processes

PM Outputs:

Notes for the Record

(NFRs)

PM Outputs

• Draft Note For the Record (draft NFR) • Final Note For the Record (NFR) • Summary Matrix of NFRs

NFR

Purpose

• analysis, summary and presentation of findings from F.A.s

• Used to:

– Report to management how processes have been implemented

– Indicate to management where programme design may require modification

Structure of the NFR

• Several processes / groups of processes covered in each NFR

• Each process/group of processes is reported in three parts:

– Observations about the process of SIPP implementation at the field level

– Any issues about how the processes are carried out

– Recommendations that relate to the purpose

Process for developing NFRs

1. Field Assessment

2. Collate, summarise, analyse Field Assessments – using 3-part format

3. (FGDs at field level to explore F.A. issues arising) 4. Write Draft NFR

5. Review Draft NFR within PMA

6. (May circulate Draft NFR to SDF (and partners)) 7. Hold FGD(s)with project partners & SDF 8. Finalise NFR, considering PMA QA & FGD

comments

9. Submit Final NFR to MEL Division, SDF

10. Summarise NFR issues & recommendations into a matrix(monthly)

(28)

ITAD and CNRS

December 2004

4

NFR group work

NFR-1 NFR-2 Strengths Weaknesses Content • • • Content • • • Format • • • Format • • • Strengths Weaknesses Content • • • Content • • • Format • • • Format • • •

TOOL 4: Focused Group

Discussion

FGDs

• Feedback findings from Field Assessments • Seek clarification on confused / complex

issues from Field Assessments

• Triangulation / validation of Field Assessment findings

• Discuss Field Assessment findings and reach decisions on issues raised

• To use the information to inform planning and future implementation

BACK

TOOL 6: Case Studies

Case Studies

Purpose: To help SIPP to:

• Enhance performance through

analysing & disseminating good practice • Establish benchmarks of good practice • Provide structured documentation of the

lessons

Case Studies

• Documenting good practice is only half the job

• The other half is to communicate the lesson to other relevant partners • How does the learning from Case

(29)

ITAD and CNRS

December 2004

5

Case Studies

Group work

• Group 1 – List 10 possible topics for future

Case Studies, and explain the process lessons they demonstrate, which are important for the whole of SIPP

• Group 2 – examine the proposed information

flow for PM information in SIPP. Suggest and justify any modifications, and explain the practical steps for implementing this flow as part of a SIPP learning system

Other resources

Notes for Facilitators • Holding a meeting • Taking notes • Documentation

(30)

Training Report

18

(31)

SIPP Process Monitoring

Notes for Facilitators

January 2005

(32)

(33)

CF Community Facilitator (of CSO)

CIW Community Infrastructure Works

CSO Community Support Organisation

FF Field Facilitator

FGD Focus Group Discussion

NGO Non Governmental Organisation M&E Monitoring and Evaluation MEL Monitoring, Evaluation and Learning

MIS Management Information System

NFR Note For the Record

PAST Project Appraisal and Supervision Team

PM Process Monitoring

PMA Process Monitoring Agency PMC Project Management Committee

PO Participating Organisation

PPF Pilot Private Financing RO Research Officer (of PMA) SAP Social Assistance Program SDF Social Development Foundation SIPP Social Investment Program Project SRO Senior Research Officer (of PMA)

SW Social Worker

(34)

(35)

SIPP process monitoring November 2004

FACILITATION NOTES 1 – Introduction

INTRODUCTION

These Facilitation Notes provide supporting Reference Material for the SIPP Process Monitoring Training Course. Each set of Notes is designed for use as a stand‐alone step‐by‐step guide to using a process monitoring tool. In addition to the guidance on using the process monitoring tools, guidance is provided on three other facets of process monitoring in which process monitors and facilitators ought to be well versed: holding a meeting, talking notes and understanding different types of documentation (which are important for recording how processes have occurred).

The Facilitation Notes comprise the following materials:

Name of Section

Content

Facilitation Notes 1 – Introduction

An

introduction

to

Process

Monitoring

and

the

PM

tools

Facilitation

Notes

2

‐

Wealth

Ranking

A

step

‐

by

‐

step

guide

to

using

A

step

‐

by

‐

step

guide

to

using

TOOL

1:

Wealth

Ranking

Facilitation

Notes

3

‐

Report

Cards

A

step

‐

by

‐

step

guide

to

using

TOOL

2:

Report

Cards

Facilitation

Notes

4

‐

Field

Assessments

A

step

‐

by

‐

step

guide

to

using

TOOL

3:

Field

Assessments

Facilitation

Notes

5

‐

Note

for

the

Record

Guidance

on

how

to

complete

a

‘Note

for

the

Record’

Facilitation

Notes

6

‐

Focus

Group

Discussions

A

step

‐

by

‐

step

guide

to

using

TOOL

5:

Focus

Group

Discussions

Facilitation

Notes

7

‐

Case

Studies

A

step

‐

by

‐

step

guide

to

using

TOOL

6:

Case

Studies

Facilitation

Notes

8

‐

Holding

a

Meeting

An

overview

to

the

principles

of

holding

an

informal

or

formal

meeting

Facilitation

Notes

9

‐

Taking

Notes

Guidance

on

how

to

take

meeting

notes

and

how

to

write

reports

Facilitation

Notes

10

‐

Documentation

Note

on

types

of

possible

documentation

(e.g.

for

process

documentation)

(36)

FACILITATION NOTES 1 – Introduction

What

is

Process

Monitoring?

Process Monitoring is about looking at how activities are done. It concerns:

• Consciously selecting processes, systematically observing them, comparing them with others and communicating this to learn how to better steer and shape the processes

• A management tool to generate information for institutional learning and taking corrective action in projects that involved a high level of community

participation.

Processes are sets of actions that produce outcomes. Processes are essentially

activities, but the process monitoring is concerned with how the activity is done.

A number of tools are used in Process Monitoring of the SIPP. These are outlined in the table below. The set of facilitation notes provides more detail about each tool and provide detailed steps for using the tools.

Tool Purpose When is it used Who uses the

tool? TOOL 1: Wealth

Ranking

To monitor changes in levels of poverty

At the start of CSO involvement in the village, and two repeats

CSO, then PO, then READ

TOOL 2: Report Cards Self-assessment of VDCs

performance Quarterly VDC, facilitated by CSO/PO; summaries used by SDF MEL (MIS) TOOL 3: Field Assessments Routine monitoring of project processes in sample villages. Quarterly PMA

TOOL 4: FGDs Verification of issues

raised during the Field Assessment visits. Monthly (District) & Quarterly (Dhaka) PMA TOOL 5: Field Assessments (Utilities)

To monitor the processes involved implementation of the utilities

programme

Quarterly PMA

TOOL 6: Case Studies Documentation of better

practices and lessons being learnt during the implementation of the SIPP. Quarterly PMA

(37)

FACILITATION NOTES 2 – Wealth Ranking

WEALTH

RANKING

Wealth Ranking is a tool that is used to identify different socio‐economic groups in a

community. The following table summarises how wealth ranking is used in SIPP

Implementation and Process Monitoring. Detailed steps for conducting a wealth

ranking are provided after the table.

Tool Wealth Ranking

Purpose • To identify the different wealth categories in a village

• To monitor whether there is any change in the

number of poor and very poor households

• To verify that the poor and very poor are the main

beneficiaries of the project

When is it used? At the start of the programme (when first working in the

village) after VDC formation

At two further intervals during involvement with the

programme (steps 2 and 3 only)

How long does it take? 4‐5 hours plus additional time for reviewing/checking

Who uses this tool? The Community Facilitators (CF) of the Community

Support Organisations (CSOs) is responsible for carrying

out a wealth ranking at the start and during the

programme.

The Field Facilitators / Social Workers of the Participating

Organisation (POs) are responsible for verifying wealth

rankings in areas where the CSOs have withdrawn.

The Impact Evaluation Agency may use this tool later in

the process to evaluate the impact of different projects. It

may be used to see how many poor people have moved

between different wealth categories (to a high category or

in some cases they may have dropped to a lower

category).

(38)

FACILITATION NOTES 2 – Wealth Ranking

Method

Step 1 Identifying characteristics of poverty / wealth

People from the community undertake the wealth ranking exercise, with outside

facilitation. This is achieved by first holding a mass gathering of village people (this

normally achieves a 10‐20% attendance; approx 1 person per household). At this

gathering, the facilitator divides the gathering according to different geographical

locations or paras. The number of groups depends on the size of the village and

number of paras. The facilitator should try to ensure that each group include a

diverse mixture of people (e.g. different socio‐economic status, clan, occupation,

gender, etc.). In order to achieve a balanced representation, there would ideally be a

minimum of 5 people per para group. Note to facilitator: You may want to discuss with the group what criteria they use to identify who are the heads of households. Such criteria may include the earning capacity of the person, age, and sex.

Ask each group to identify definitions for the following four categories:

Rich

Middle

Poor

Very poor / hardcore poor

In the SIPP context, most groups will differentiate levels of poverty between

households using factors such as:

• Amount of land owned;

• Food availability (how many days in a year the household takes 1, 2 or 3 meals a

day);

• Condition of dress;

• Availability and type of employment;

• Type of house;

• Level of indebtedness

• ….

Other factors may also be mentioned depending upon the village context and this

should be encouraged. All the factors which are used to identify which households

fall in which of the four categories should be documented.

Note to facilitator:

four categories are identified as this conforms with the Operational Manual of SIPP and allows for easier recording and consolidation by the SIPP MIS.

(39)

FACILITATION NOTES 2 – Wealth Ranking

By whatever criteria the villagers define poverty their definitions should be recorded and kept with the VDC. These definitions remain the same throughout the life of the project, even though the definition of poverty nationally or internationally may change. This is important if like is to be compared with like.

Step 2 Classification of all village households by poverty status

Each group should write the name of each household in the village on a card [note: Symbols or objects can be used if literacy levels are low. For example, household x could be discussed and the main characteristics of the household illustrated in a picture. E.g. they may have a pond with ducks and this could represent that particular household.]

Then place each household into one of the four groups according to their wealth

status. To do this, take a sheet of paper and separate it into four sections. Write a

wealth category in each section as illustrated in the table below. Now place a card

with the name of each household head in the relevant box. Once the group has

placed all the cards in the boxes complete FORM 1. The Facilitator should also note

down the criteria that were used to assess the wealth ranks.

Very poor / hardcore poor Poor Middle Rich

Step 3 Verification of wealth ranks

After the formats have been completed, the para groups come together, and with the

CF of the CSO debate and validate the lists of wealth ranks in a plenary session.

Validation entails reading out the para lists and the rank for each household, and

offering people the opportunity to comment on the validity of these ranks. The

information is then consolidated at the village level into one table of the four socio‐

economic categories (FORM 1).

Step 4 Monitoring changes in wealth/poverty

The original forms should be kept by the villagers then by the VDCs once formed.

(40)

FACILITATION NOTES 2 – Wealth Ranking

The lists of households will then be used to verify whether the very poor and poor

are the major beneficiaries of project activities.

Once the CSO completes its work in any village, and the PO takes over, the PO

should review and revalidate the wealth ranking to ensure that it is working only

with very/ hardcore poor households. This would not necessarily be a repeat of Steps

2 and 3, but smaller PRA exercises and one‐to‐one checks with households. The

result may be some non‐poor households being exclude and some previously missed

very poor households now being included.

Towards the end of the project period, Steps 2 and 3 may repeated by the Impact

Evaluation Agency with a sample selection of villages to assess how many

households from the very poor and poor categories have moved up the poverty

(41)

FORM 1 – Wealth Ranking

Wealth Ranking Format

Name of facilitator: Date of ranking:

_ _ /_ _ /_ _ _ _

Village name: Upazilla:

Total number of households in the village:

Wealth category (place a √ in the appropriate box)

No. Name of household _head Very Poor Poor Middle Rich

1. 2. 3. 4. 5. 6. 7. 8.

Insert more rows as necessary ….. TOTALS

The following table should also be completed, to record the poverty criteria used in the particular village:

Very Poor Poor Middle Rich

List criteria here

(42)

FACILITATION NOTES 3 – Report Cards

REPORT

CARDS

A Report Card is a tool for reflection and learning by VDCs to help them assess their

performance and so improve processes and practice. The report cards enhance

transparency and encourage debate and discussion to further the development of the

VDCs. By assessing their own performance, VDCs can identify areas of weakness

and strengths that will help the VDC to develop.

The report card also acts as a monitoring tool for SIPP, to identify areas where CSOs

or POs may need help, or to take initiative to improve the situation.

Tool Report Card

Purpose • For VDCs or community groups to assess their own

performance against criteria that they identify

themselves

• For VDCs to learn through practice

When is it used? Î Quarterly, during a VDC meeting

How long does it take? 1 hour

Who uses this tool? VDC facilitated by CSO / PO (completing FORM 2)

SDF and PMA at quarterly District Coordination

Meetings (only FORM 3)

SDF at quarterly project meetings in Dhaka (only FORM

3)

Method

Step 1 Introduce Indicators

The Report Card comprises 13 indicators. For each indicator an illustrated card is

available to enable non‐literates to understand the meaning of the indicators (the list

of cards, and the accompanying illustrations, that are available is given at the end of

this note ‐ Facilitation Notes 3). Briefly discuss each indicator to ensure that group

members understand each indicator. Ask the group if the indicators are appropriate

to their VDC. Allow some flexibility so that 1‐2 indicators can be added.

(43)