M-Tool SET CONTENTS 1 / 35

(1)

Monitoring is the regular observation and recording of activities taking place in a project or programme. It is a process of routinely gathering information on all aspects of the project. To monitor is to check on how project activities are progressing. It is observation; systematic and purposeful observation. Monitoring also involves giving feedback about the progress of the project to the donors, implementers and beneficiaries of the project. Reporting enables the gathered information to be used in making decisions for improving project performance. in “LLP 2007-2013 Guide for Applicants”

Follow-up of performance and realisations of a beneficiary and provision of feedback and guidance in view of improving the supported activity and management of the grant as needed. in “Guide for National Agencies implementing the LLP”

Monitoring (at project level): The process involves continuous and systematic control of the project’s progress. The intention is manage

and if necessary correct any deviation from the operational objectives and thus improve the performance. Every project should be monitored throughout its duration in order to ensure its success. Monitoring consists of supervision of activities, comparison with the work plan and using the information obtained for the improvement of the project. During the monitoring process dissemination and exploitation activities must be carefully checked, verified and, if necessary - reoriented and adapted. in “LLP 2007-2013 Guide for Applicants”

Thematic Monitoring: Thematic Monitoring is a qualitative process put in place to increase the impact of the LLP programme. The

main elements are: 1) Clustering of projects into thematic groups to gain overview on specific contents and outcomes; 2) Facilitating exchange of experience between project actors with a view to improving quality and impact at individual project level; 3) Facilitating the networking of projects, practitioners and decision makers with a view to the future orientation of political priority and strategy. in “LLP 2007-2013 Guide for Applicants”

(3)

2. Establish a national monitoring strategy according to the EC orientations:

2.1. Structured approach considering, at national level, the objectives and target groups of the LLP sectoral programme (national

criteria to select projects for monitoring). 2.2. NA Purpose:

• Collecting information on the quality in management of the LLP Sectorial Programme and the efficiency of the implementation of the project in the institution / organization and in the educational community.

2.3. NA Aims:

• To collect information regarding the quality in management of the projects. • To support the beneficiaries in project implementation.

• To lead the beneficiaries to possible solutions of the problems detected in the implementation. • To be familiar with the real resources (human, material and institutional) applied to the projects. • To assess the progress of the projects.

• To disseminate examples of good practices.

• To establish a close relation between the beneficiaries and the NA by recognising their project work. 2.4. Use of the monitoring results (NA Reports, Institution Follow-up Reports and NA staff experience):

• To enlarge and improve the pedagogical and technical support given by the NA to beneficiaries and potential project makers in order to enhance the partnerships’ quality.

• To identify success stories and examples of best practices useful for valorisation purposes (dissemination and exploitation of results).

(4)

Section II.

M-TOOL STRATEGY FOR MONITORING EDUCATIONAL PARTNERSHIPS

Section II. A

M-TOOL STRATEGY - METHODOLOGY

WORK PHASE PROCEDURE TOOL / OUTCOME

1 Memo with information about the monitoring strategy. NA Memo

Preparation of the field monitoring process

2 Letter to the institution

(At least one week before the visit) to inform about the aim and methodology of the visit.

Letter to the institution

Field work / Collecting data

3 Monitoring visit in agreement with the principles of qualitative techniques for

collecting data (interview and observation).

Using the General Interview Guide during the visit and carrying out direct observation of documents and premises.

One day or half-day visit.

General Interview Guide

Data analysis and interpretation

4 Guide for analysis/interpretation of monitoring data:

Record the collected data in written and organize them subsequent to the visit.

Employ a common method to analyse and to interpret the data (Content Analysis).

Guide for analysis of monitoring data

Reporting 5 NA Report to the institution with feedback on the monitoring visit. NA Monitoring Report

(5)

Section II.B

M-TOOL STRATEGY - TOOLS

1. NA MEMO

TOPICS

• General selection criteria of the institution (regarding the NA Guide orientations).

• Specific selection criteria regarding a concrete project:

o Defining the purpose of the visit (due to difficulties detected, using the criterion ‘visits to coordinators’, to choose a

specific thematic, etc).

o Selecting specific project local activities and mobilities emphasized in the work plan / contract (one or several main

activities; e.g. the more recent, the ones that involved more persons, the one with more visibility, the only ones implemented…), by using a principle of purposive sampling.

• Basic Data about the project:

o Institution data (identification, type, level, nature, contacts); o Project basics (code, type, title, countries);

o Project content (concrete aims, target-groups, thematic areas); o Quality assessment (score and overall comments).

• Definition of NA staff. • Letter to the institution.

(6)

2. LETTER TO THE INSTITUTION

TOPICS

• Communicating the objectives, both general (regarding the General Interview Guide) and specific (concerning the specific selection criteria related to the “case study” i.e. this concrete project).

• Informing about the method of the visit, regarding the use of the group interview.

• Defining the participants in the monitoring visit and their role (institution management, project coordinator, other staff directly involved, pupils/learners directly involved).

• Request of documentation for observation regarding one or several finalized activities of the work plan (local activities and

agenda of the mobilities).

• Guidance to gather quantifiable data about the level of participation in activities (local, mobilities and

dissemination/valorisation).

• Request for a written confirmation of the availability of the participants on the day of the visit (to ensure the presence of the

relevant participants involved in the project activities and to have an agreement in advance on the agenda and conditions of the visit).

(7)

3. QUALITATIVE TECHNIQUES FOR COLLECTING DATA

TOOL FOR COLLECTION DATA INTERVIEW

Type of interview

Face-to-face verbal interchange by group interviewing.

Semi-structured (using a general interview guide in a flexible approach):

- Interview questions are planned but not standardized in advance because specific questions will emerge during the interview;

- There are no pre-set response categories. Group interview:

- Several people simultaneously (management, project coordinator, project team, pupils/learners, other participant staff);

- Definition of who will be interviewed, why and how many persons.

Purpose

Understanding the project progress.

Gathering cumulative data with the group situation.

Stimulating people to make their views, motives and reasons explicit.

Strategy

One-time interview (variable duration). Use of the General Interview Guide:

- Use the pre-set questions (Summary of the Interview Guide) to know how to ask and gather relevant information (related to the objectives);

- Flexible use and order of questions, adapted to the monitoring objectives and project life cycle;

- Make use of the Guiding Topics to manage the answer and avoid deviations from the objectives.

The role of NA staff: interviewer as a facilitator, who supplies the questions and topics for answers and group interaction.

Managing the interview

(8)

TOOL FOR COLLECTION DATA INTERVIEW

preparing and scheduling the interview;

reading the application form, the contract, reports (if any) or other documents related to the project.

- In the institution:

having the application form and the contract available during the visit; beginning the interview with their stated objectives;

giving oral feed-back during the visit;

closing the interview with an overall view of the strong and weak points, including recommendations;

writing down notes about the participants' speech (during and after the interview).

Data from the group interviews Written records of the NA staff

TOOL FOR COLLECTION OF DATA OBSERVATION

Approach Qualitative approach, not using predetermined categories and classifications.

Observation in a natural open-ended way.

Purpose

Triangulation between observational data and the interviewees’ discourse to ensure different angles.

Making wider links related to the project implementation.

Scope

Direct observation of:

- Institution premises;

- Space used for project activities and dissemination;

(9)

TOOL FOR COLLECTION OF DATA OBSERVATION

Strategy

Observing and listening to the institution’s environment. Reading different types of documents:

- of different authorship (teachers/trainers, pupils/learners);

- and different availability (closed, restricted or openly published).

Definition of what will be observed and why, taking into account the objectives of the monitoring visit.

Categories for describing and analysing the observable data (descriptive and focus categories).

Key categories to analyse documentary data: credibility, representativeness and meaning (who writes them, for what purpose, who read them, what is omitted, what is

accomplished).

(10)

4. SUMMARY OF THE INTERVIEW GUIDE

CATEGORIES QUESTIONS

How did you know about Comenius/Grundtvig/Leonardo Programmes? How did you come across the partners for your project?

Was there a project team to begin the work?

How did you collaborate within the partnership to conceive the common project?

Conception

When and how did you inform the institution about the project? How do you communicate among partners?

How do you share the information with all partners? What tools do you use to communicate?

Which are the communication languages used during the project? Was there any linguistic preparation needed?

Communication in the partnership

How did you satisfy those needs?

How is the project included in the institution’s work plan?

Which project activities are included in the curriculum of pupils/learners? Which activities have you already carried out?

Presently, in what activity are you working?

What was the most crowded/popular project activity? What changed in the partnership planned activities?

Partnership Local activities

(11)

What kind of common activities have you already carried out with your partners? Which mobilities have you already carried out?

What changed in your mobility plan? Can you complete the planned mobilities?

What kind of preparatory activities were developed before the mobility? Could you quantify the participating staff?

How did you choose staff for mobility?

Could you quantify the participant pupils/learners? How did you choose pupils/learners for mobility?

How did you twin pupils during their foreign stay/ when hosting partners?

How did you involve your pupils/learners in the implementation of the activities during mobility and /or when hosting partners?

Which was the contribution given by the parents/families to the mobility activities? Which was the contribution given by the Associated Partners to the mobility activities? Which activities were developed during mobility and/or partner’s hosting?

How much time was spent in each kind of activity during mobility and/or partner’s hosting?

Mobilities

What kind of follow-up activities were developed after mobility?

How many persons are presently involved in the project, including the project team? How is the project team formed at the moment?

Institution’s involvement

(12)

Which channels does the project team use to inform / guide the other institution’s participants? How is the institution’s management involved in the project?

Which responsibilities were taken by the relevant staff regarding the project? In which tasks were pupils/learners involved?

In which tasks were parents / families involved in the project?

Have you involved other institution(s)/organization(s) from the local community? In which tasks were the institution(s)/organization(s) of the local community involved? Did they assume the role of associated partners?

Community involvement

Are you planning to involve other institution(s)/organization(s) from the local community? Which final products have you already finalized?

In what stage are you, concerning the common final product(s)?

Outcomes

What are the tasks and roles for each partner institution concerning the final common product(s)? What knowledge did you gain through the project regarding its thematic area?

What knowledge did you achieve through the project regarding European issues? What changed in your language skills due to the project?

What changed in your ICT skills due to the project?

What changed concerning the implementation of pedagogical methods due to the project? What changed in the organization of the institution due to the project?

What changed concerning motivation in learning / teaching?

Project results

(13)

What changed in the establishment of links with other institutions of the local community due to the project? What changed concerning pupils/learners school/training success motivated by the project?

How do you evaluate the progress of the project? Who participates in the evaluation process? What kind of feedback reached the participants? Which are the results from your evaluation?

Evaluation

What have you changed as a consequence of your evaluation results?

In which events / initiatives did/will you participate to present /make known your project activities/ experiences and or results?

What kind of information was produced to use in the dissemination activities?

What activities did/will you start and implement, on your own initiative, to present / make public the project? Have you developed any activity in common with other institutions to present/publicize the project?

How many participants were present in each dissemination activity?

Dissemination

How do you intend to continue to spread/share information about your project?

Which activities have you developed to promote the application of your project results by other institutions / individuals?

Which project results were applied by other institutions and /or individuals?

What kind of institutions / group of individuals have directly benefited from your project results?

What project result(s) turned to be the most significant one(s) in terms of being applicable in similar educational contexts?

Valorisation

(14)

Which problems/obstacles did you encounter during the project? Did you share the problems with all the people involved in the project?

How did you solve the problems detected during the project’s implementation? How did the mentioned problems change, in some way, the original project?

Project influences

Which were the most favourable conditions to the project implementation? Are you satisfied with the contacts you had with your NA?

Contacts with the

(15)

5. GENERAL INTERVIEW GUIDE

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS

Legitimating the interview

To explain the process of a monitoring visit.

To expose the aims of the visit.

To describe the

methodology used in the monitoring visit.

To reinforce the importance of the institution’s collaboration. To motivate the

participants in the project on the importance of the monitoring process. To inform about the feed-back process regarding the monitoring results.

Conception To recognise how the _{public has access to the}

Sectorial Programme.

To identify the channel(s) used to obtain information about the Programme

How did you know about Comenius/Grundtvig/Leonardo Programmes? Internet / NA website Personnal / Professional contacts Previous participation in other European Programmes Participation in NA events

(16)

To be familiar with the partner finding procedure.

To understand how the contacts are established to begin a partnership. To comprehend the criteria to select the partners.

How did you come across the partners for your project?

Participation in a VP or Contact Seminar Use of an website for partner finding Networks

Previous partners in a different project

To recognize how the institution organized internally the process of project conception.

To understand the initial motivation for the project. To know who is the staff involved in the project team.

Was there a project team to begin

the work? How, whom

To be aware of the common conception of the partnership’s project.

To identify the tasks of each partner institution during the conception process. To know the measures taken by the partnership members to elaborate the project.

To identify the articulation between partner

institutions.

How did you collaborate within the partnership to conceive the common project?

Common fill-in of the joint sections of the application form

(17)

To know the awareness of the institution community about the project.

To identify the opportunities given to the institution community to be involved in the project conception. To recognise the

involvement of other persons besides the project team.

When and how did you inform the institution about the project?

In teacher/trainer meetings, through institution’s posters or newspaper articles, on a website, etc To identify a flowing communication throughout the project. To identify a regular communication throughout the project.

How do you communicate among partners?

When (every day, every week, every month)

Whom (project team, teachers/trainers, management, pupils/learners, other staff)

To identify the measures undertaken to share information among the partnership.

How do you share the information with all partners?

Written confirmation of activities and decisions

To know the tools used to communicate with the partners.

What tools do you use to

communicate? Fax, phone, e-mail, regular mail, etc

Communication in the partnership

To be acquainted with the communication strategy implemented among partners.

To be acquainted with the languages used to

communicate with the partners.

Which are the communication languages used during the project?

Curricular, non-curricular

(18)

To recognize the need for linguistic preparation.

Was there any linguistic preparation needed? Why Who (teachers/trainers, management, pupils/learners, other staff, parents/families) To identify resources used

for linguistic preparation. How did you satisfy those needs?

Using audiovisual materials, making a language course, hosting a Comenius Assistant, etc To be aware of the ways to

integrate the project in the institution’s work plan.

How is the project included in the institution’s work plan?

To recognize the

integration of the project in the institution.

To identify the measures taken to integrate the project activities in the curriculum of

pupils/learners.

Which project activities are included in the curriculum of pupils/learners?

Which subjects

Which activities have you already carried out?

Partnership Local activities

To verify the

implementation of the local activities at the institution and/or in the local community.

To identify the degree of accomplishment of the

planned activities. Presently, in what activity are you

working?

Where, when and with whom (teachers/trainers, management, pupils/learners, parents/families, associated partners, other participants)

(19)

To be aware of the most visible project activity.

What was the most

crowded/popular project activity?

How many participants of each kind (teachers/trainers, management, pupils/learners, parents/families, associated partners, other participants) To identify changes in the

partnership’s work plan.

What changed in the partnership

planned activities? Why

To be aware of the

capacity to accomplish the partnership’s work plan. To identify the conditions that influence the project’s implementation.

Will you be able to complete the

planned activities? Why (enough resources, time, etc)

To understand the consistency of the

cooperative work done in the partnership.

To identify the common activities in the partnership.

What kind of common activities have you already carried out with your partners?

When and how

To identify the degree of accomplishment of the mobility plan.

Which mobilities have you already carried out?

When, where and who (teachers/trainers, management, pupils/learners, parents/families, associated partners, other participants), how many Mobilities To verify the implementation of the mobility activities.

To identify changes in the

(20)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To be aware of the

capacity to accomplish the mobility plan.

To be aware of the conditions that influence the implementation of mobilities.

Can you complete the planned

mobilities? Why (enough resources, time, etc)

To be aware of the

planning process regarding mobility activities.

What kind of preparatory activities were developed before the mobility?

Who To know about the

composition of the participant staff in mobilities.

Could you quantify the participating staff? Number of teachers/trainers, management, associated partners, other participants To identify the criteria for

choosing the staff for mobility.

How did you choose staff for mobility?

Staff from different subjects, staff with disabilities, etc To know about the

composition of the

participant pupils/learners in mobilities.

Could you quantify the participant pupils/learners?

Number of pupils/learners, school level/type of training, age To identify the criteria for

choosing the

pupils/learners for mobility.

How did you choose pupils/learners for mobility?

Pupils/learners age, school level/type of training,

pupils/learners with disability, etc To understand the strategy

to implement mobility activities.

To be familiar with the criteria to twin pupils.

How did you twin pupils during their foreign stay/ when hosting partners?

Applicable to Comenius Bilateral Partnerships Host families and reciprocal twining

(21)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To know the direct

contribution of pupils/learners to the implementation of mobility activities.

How did you involve your pupils/learners in the

implementation of the activities during mobility and /or when hosting partners?

To know the direct contribution of

parents/families to the implementation of mobility activities.

Which was the contribution given by the parents/families to the mobility activities?

To know the direct contribution of the

Associated Partners to the implementation of mobility activities.

Which was the contribution given by the Associated Partners to the mobility activities?

To know the kind of

activities developed during mobility.

Which activities were developed during mobility and/or partner’s hosting?

Who

Visiting other

institutions, time spent as trainer or trainee, attending lessons, working together on the project, time spent for leisure To be aware of the

coherence of the work agenda of the mobility activities.

How much time was spent in each kind of activity during mobility and/or partner’s hosting? To recognize the

consistency of the mobility planned activities.

To identify the follow-up activities consequently to the mobility.

What kind of follow-up activities were developed after mobility?

(22)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To verify the degree of

involvement in project activities.

To spot the volume of institution’s participants.

How many persons are presently involved in the project, including the project team?

Teachers/trainers, management, pupils/learners, other staff participants. To know the composition of

the institution’s project team.

How is the project team formed at the moment?

Who How many What subjects To know the work regularity

of the project team. How often does the project team meet?

To recognize the dynamics of the institution’s project team.

To identify the channels used to inform / guide the institution’s participants about the project.

Which channels does the project team use to inform / guide the other institution’s participants?

“Comenius Corner” E-mail Meetings To be aware of the management’s support to the project. To identify measures regarding management support.

How is the institution’s

management involved in the project?

Visit other partner institutions Links to other institutions (local community, educational authorities, etc) To be acquainted with the

staff’s involvement in the project.

To categorize the main responsibilities of the staff involved in the project.

Which responsibilities were taken by the relevant staff regarding the project?

Teachers/trainers and other institution staff

Institution’s involvement

To be aware of the pupils’/learners’

involvement in the project.

To classify the main tasks of the pupils/learners involved in the project.

In which tasks were pupils/learners

involved? Pupils association

Community involvement

To recognize the parents’/families’

involvement in the project.

To identify synergies

between the institution and families, accomplished through the project.

In which tasks were parents /

(23)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To identify synergies

between the institution and the local community, accomplished through the project.

Have you involved other

institution(s)/organization(s) of the local community?

Type of institutions/ organizations To categorize the main

tasks of the local

community’s involvement.

In which tasks were the

institution(s)/organization(s) of the local community involved? To identify the existence of

an associated partners’ network.

Did they assume the role of

associated partners? Named in the application form

To recognize the official relationships between the institution and the local community.

To know about the possible enlargement of community involvement.

Are you planning to involve other institution(s)/organization(s) from the local community?

To identify the degree of accomplishment of the planned outcomes.

Which final products have you already finalized?

Final products with European Logo To collect evidence of the

existence of the common final product(s).

In what stage are you, concerning the common final product(s)?

What Whom Outcomes To verify the implementation of the project outcomes. To recognize the complementary of tasks within the partnership.

What are the tasks and roles for each partner institution

concerning the final common product(s)?

Project results To understand the results of the project’s implementation.

To know the improvements in knowledge in the thematic area of the project.

What knowledge did you gain through the project regarding its thematic area? Teachers/trainers, pupils/learners, management, other institution staff, parents/families, etc

(24)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To recognize the

improvements made in the European Dimension of Education/Training.

What knowledge did you achieve through the project regarding European issues? Teachers/trainers, pupils/learners, management, other institution staff, parents/families, etc To know the improvements

in language skills achieved through the project’s implementation.

What changed in your language skills due to the project?

Teachers/trainers, pupils/learners, management, other institution staff, parents/families, etc To identify the improvements

accomplished in ICT skills through the project’s implementation.

What changed in your ICT skills due to the project?

Teachers/trainers, pupils/learners, management, other institution staff, parents/families, etc To know the improvements

in pedagogical methods with the project’s

implementation. To identify the

implementation of new pedagogical methods due to the project.

What changed concerning the implementation of pedagogical methods due to the project?

Teachers/trainers, management, other institution staff To identify the improvements in the management approach. To know the new

organization strategies implemented due to the project.

What changed in the organization of the institution due to the

project?

Teachers/trainers, management, other institution staff

(25)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To identify the

improvements on

motivation of the project participants.

What changed concerning motivation in learning / teaching?

Pupils/learners / teachers/trainers

To know the improvements in interpersonal relationships stimulated by the project. To identify the reinforcement of the institution’s internal cohesion.

What changed in the relationships within the institution’s participants due to the project?

Between teachers/trainers and pupils/learners, among pupils/learners, among teachers/trainers, between staff in general, between teachers/trainers and parents/families To recognize the improvements on linking the institution to the local community due to the project.

What changed in the

establishment of links with other institutions of the local community due to the project?

To know the improvements in pupils/learners

school/training

performance due to the project.

What changed concerning pupils/learners school/training success motivated by the project?

(26)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To distinguish the

implementation of a two-level evaluation process (combining the institution level and the partnership level).

To identify the evaluation phases.

To recognize the existence of an ongoing evaluation process.

To know the methods used to collect, analyse and interpret evaluation data.

How do you evaluate the progress of the project?

At institution level and at the partnership level Phases Méthodes Techniques (interviews, questionnaires, observation, etc) What

To identify the level of participation in the evaluation process.

Who participates in the evaluation process? Teachers/trainers, management, pupils/learners, other staff, parents/families, etc How many Evaluation To be aware of the evaluation strategy adopted in the project.

To identify the procedure to report on evaluation results.

What kind of feedback reached the participants?

At institution level and at the partnership level

(27)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To know the institution’s

evaluation results (self- evaluation).

To be familiar with the strong/weak points of the project according to the institution’s view.

To know the

recommendations resulting from the self-evaluation process.

Which are the results from your evaluation?

To be aware of the self-evaluation results.

To identify the adoption of recommendations made at evaluation reporting stage.

What have you changed as a consequence of your evaluation results?

To identify the degree of accomplishment of the planned dissemination activities.

In which events / initiatives did/will you participate to present /make known your project activities/ experiences and or results?

Conferences, seminars, meetings, exhibitions

Where, when, how, to whom Who presented Dissemination To recognize the development of the dissemination strategy.

To know the tools used to disseminate the project. To be familiar with the criteria used to select the information for

dissemination purposes.

What kind of information was produced to use in the dissemination activities? Content of: a webpage, a newspaper, a PowerPoint presentation, an exhibition, a performance, etc

(28)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To distinguish the nature of

the initiative regarding the dissemination activities. To identify the measures taken to disseminate the project.

What activities did/will you start and implement, on your own initiative, to present / make public the project?

Auto-initiative Co-authorship with other institutions (who)

To identify the existence of cooperative work

regarding the implementation of dissemination activities.

Have you developed any activity in common with other institutions to present/publicize the project?

Common organization at partnership level Shared organization with the associated partners or other organizations

To be aware of the level of public knowledge about the project.

How many participants were present in each dissemination activity?

Teachers/trainers, management, pupils/learners, other staff, representatives from the educational authorities, from the associated partners, etc

To identify the channels used to disseminate information about the project.

To know the kind of institution’s participants taking part in the dissemination activities.

How do you intend to continue to spread/share information about your project?

Spread experiences, activities and project results

What, when, where, how, by who and to whom

(29)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To identify the degree of

accomplishment of the planned valorisation activities.

Which activities have you developed to promote the application of your project results by other institutions / individuals?

Training courses, seminars, conferences, pedagogical materials’ offers To identify the kind of

project results more valuable to be transferred to other educational contexts.

To know the extent of the project results at national / international scale.

Which project results were applied by other institutions and /or

individuals?

When, where, how and by whom At individual, local, regional or national level.

To understand the transfer possibilities of the project results (apprenticeships and/or final products).

To identify the target group(s) that could benefit from the project results.

What kind of institutions / group of individuals have directly benefited from your project results?

Distinction between the participants (internal target group) and the external individuals How many

To recognize how the project results are linked to the expected impact.

What project result(s) turned to be the most significant one(s) in terms of being applicable in similar educational contexts?

Target group(s) of the project

Valorisation

To be acquainted with the contribution of the project results to achieve Lisbon objectives and/or the operational objectives and priorities of the LLP

Programme.

To identify the

achievement of the project results in face of the

objectives and priorities of the LLP Programme

Do you consider that your project results are in harmony with the LLP Programme? How Why Project influences To be aware of the difficulties in project development.

To identify specific factors that negatively influenced the project’s

implementation.

Which problems/obstacles did you encounter during the project?

At the institution and at the partnership level

(30)

CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS To know the solving

problem procedures implemented in the partnership.

To spot the effectiveness of the communication

strategy.

To identify the level of cooperation between the project participants.

Did you share the problems with all the people involved in the

project?

Who (project team, management, partnership coordinator, all partners, NA, authorities, etc) To identify concrete solutions in face of the project’s difficulties.

How did you solve the problems detected during the project’s implementation?

Who, when To be aware of the level of

deviation from the initial project plan.

How did the mentioned problems change, in some way, the original

project? Application form

To be familiar with the factors in favour of the project development.

To know specific factors that positively contribute to the project

implementation.

Which were the most favourable conditions to the project

implementation?

At the institution and at the partnership level

To spot the strong /weak points in the support given by the NA to the

beneficiary institution.

Are you satisfied with the contacts you had with your NA?

Contacts with the NA

To be acquainted with the degree of satisfaction of the institution concerning

the NA. To identify ways to improve

NA best support and guidance.

How do you think they could be improved?

(31)

6. GUIDE FOR ANALYSYS OF MONITORING DATA

METHODOLOGICAL FRAMEWORK TO QUALITATIVE ANALYSIS CONTENT ANALYSIS Purpose

Establishing how the NA reaches the findings about the monitoring visit.

Organization, categorization and interpretation of the information collected (through interview and observation).

Strategy

Coding the data collected in the interview to do their labelling:

- The pre-coded categories will be used to validate the interview responses, because these categories are based on a clear definition of aims (included in the General Interview Guide).

- For complementing the use of the categories pre-specified (on the General Interview Guide), other data categories (either descriptive or higher–inference) will be developed if necessary.

- The aim of organizing the data by categories is to reduce data without significant loss of information, summarizing the data by pulling together themes and identifying patterns and regularities.

- The data collected is organized by small or large sentences labelled by the categories and objectives (units of sense), filled in the column “Monitoring Record” of the Analysis Grid.

Developing propositions based on the patterns and connections evidenced with the coding, requiring some degree of inference beyond the data.

- Drawing the findings to establish a meaningful and coherent picture of the project implementation, putting in evidence the strong and weak points and defining

recommendations (filled in the column “Comments for the Report” of the Analysis Grid according to the items of the NA Monitoring Report).

- Crossing the data collected in the interview with the observation data to create a

correspondence between verbal responses (what people say) and what they do and what they say they do.

- The findings will be supported by crossing the observation evidence (from direct observation and from the interview) with the specific objectives established for each category in the General Interview Guide.

(32)

EXAMPLE OF THE ANALYSIS GRID FOR THE INTERVIEW

Categories Objectives Monitoring Record Comments for the Report

To recognise how the public has access to the Sectorial

Programme.

Filled in with topics constituted by “units of sense” from the notes

collected by the NA staff, during and after the monitoring visit.

Filled in with inferences (findings) obtained by crossing the

observation evidence with the specific objectives defined in the General Interview Guide.

Conception

To be familiar with the partner

finding procedure. (…) (…)

Communication in

the Partnership (…) (…) (…)

EXAMPLE OF THE ANALYSIS GRID FOR OBSERVATION

(33)

7. NA MONITORING REPORT

• Combination of results from data collected directly (observation) and indirectly (interview).

• Possibility given to the institution to refute matters of fact or interpretations stated in the conclusions of the NA Report (existence of

a 1st_draft).

Report structure (final version)

Identification of the institution and the project Objectives of the report phase:

o Good practices emphasis; o Counselling and advice.

Monitoring visit:

Introduction (about the visit):

o Date of visit, identification of the persons interviewed (management, coordination,

other staff and pupils/learners), place;

o Context and other relevant circumstantial information; o Identification of the NA monitoring team.

Observed data:

o Documents, materials and other aspects observed in situ;

o Evidence of project activities (artefacts, reports, booklets, log books, multimedia

products, etc). Findings:

o Results of the content analysis of the interview(s);

(34)

Report structure (final version)

weaknesses identified with reference to the pre-defined parameters/categories;

o Identifying problems and their causes;

o Identifying “seeds” of innovation or excellence;

o Conclusion (general appreciation of the state of project implementation; lessons

learned). Recommendations

o What shall be done to solve the identified problems? To adjust activities and/or

resources? To improve action? To make the project better?

Contribution of the project to the LLP Programme (objectives and priorities).

Institution comments about NA conclusions (in case there are any regarding the 1st_draft).

Request for a Follow-up Report:

o Information regarding its minimum requirements; o Establishing a deadline to its submission to the NA.

(35)

8. FOLLOW-UP REPORT

• Aims:

o To ensure that the recommendations have practical consequences in project implementation. o To make certain that recommendations are implement during the project lifecycle.

o To guarantee improvements in the quality of the project.

• Content:

o Different from institution for institution; o Open format, chosen by each institution. o Minimum requirements:

The institution has to propose concrete follow-up measures focused on the NA recommendations;

M-Tool SET CONTENTS 1 / 35

CONTENTS

Section I

Section II

Section II. A

Section II. B

Section I

NA CONTEXT TO DEVELOP MONITORING VISITS TO LLP PARTNERSHIPS

Section II.

M-TOOL STRATEGY FOR MONITORING EDUCATIONAL PARTNERSHIPS

Section II. A

M-TOOL STRATEGY - METHODOLOGY

Section II.B

M-TOOL STRATEGY - TOOLS

1.

NA MEMO

2.

LETTER TO THE INSTITUTION

3.

QUALITATIVE TECHNIQUES FOR COLLECTING DATA

4.

SUMMARY OF THE INTERVIEW GUIDE

5.

GENERAL INTERVIEW GUIDE

6.

GUIDE FOR ANALYSYS OF MONITORING DATA

EXAMPLE OF THE ANALYSIS GRID FOR THE INTERVIEW

EXAMPLE OF THE ANALYSIS GRID FOR OBSERVATION

7.

NA MONITORING REPORT

8.

FOLLOW-UP REPORT