Instituto Tecnológico de Santo Domingo (INTEC)

Global Partnership for social accountability (GPSA)

1. Background

The Global Partnership for Social Accountability (GPSA) wishes to contract an external consultant (individual, team or consultancy firm) to undertake various monitoring, evaluation, research and learning related assignments (henceforth “MERL Consultant”). The consultancy relates to a project called “Collaborative Social Accountability for Improved Governance in Protecting Biodiversity Hotspots Project”, GPSA Grant No. TF0B5264 – hereafter “the project”.

Signed on April 19, 2021, this GPSA funded project is implemented in Antigua & Barbuda, Saint Lucia, Jamaica and the Dominican Republic, and focusses on improving biodiversity conservation in key biodiversity areas located in the Caribbean biodiversity Hotspot, through collaborative social accountability mechanisms between governments, citizens and civil society organizations (CSOs). The project is implemented by Instituto Tecnológico de Santo Domingo (INTEC), based in Dominican Republic, in partnership with Integrated Health Outreach (IHO), based in Antigua and Barbuda. Please, find further project information and information about the sector context here.

The GPSA provides sustained and tailored support to civil society-led partnerships, including INTEC and IHO, to implement projects using collaborative social accountability processes. These processes are implemented by multi-stakeholder compacts that engage citizens, civil society groups, public sector institutions in joint, iterative problem solving for more effective policy responses and improved service delivery in areas such as biodiversity protection and conservation, Protected Areas management, climate action, ecosystem services, and community participation in conservation. As projects are implemented, the GPSA’s Theory of Action expects these relationships to be strengthened and capacities increase for meaningful engagement in public sector policy making, implementation and service delivery systems.

2. Objectives of the consultancy

Instituto Tecnológico de Santo Domingo (INTEC) is required by the GPSA Secretariat to hire MEL consultancy services from an independent consultant or consulting firm throughout the project from May 2023 to August 2025.

The purpose of this MEL consultancy is to provide expert technical assistance to develop and implement the project’s MEL system, strengthening the project’s Theory of Action and associated Results Framework, supervising data collection, reporting on select evaluative indicators, and preparation of the baseline and mid-term assessment, and undertaking the final evaluation of the project.

The specific goals are to:

1. Ensure MEL deliverables, capacity building support and advice to INTEC and project stakeholders encourages learning, which is reflected in project course corrections to meet intended outcomes described in the project-level Theory of Action and Results Framework.
2. To contribute to learning and accountability by explaining how, if at all, the GPSA project contributed to results brought about by collaborative social accountability processes, and what the conditions were for this contribution to take place.  
3. Key expected activities, outputs and timelines[1]

As soon as the MERL Consultant has been hired, the GPSA Secretariat will ‘onboard’ the consultant by providing the assistance and documents needed to fulfil the deliverables below and will work closely with the consultant throughout the consultancy.

• Monitoring data analysis and reporting: The consultant is responsible for carrying out the analysis of mandatory evaluative indicators included in the Results Framework. This data is submitted yearly by the grantee in its technical reports. In addition, the consultant will support analysis of other data collection during project implementation, including relevant feedback to biannual technical reports and data that may support the evaluations, as per the research design and MERL system/plan.
• A mid-term assessment of the GPSA project with report (In Year 2/3): The consultant will agree with INTEC, the project TTL, and the GPSA on the focus on a rapid mid-term assessment of that are deemed critical for the project’s strategic course-correction at this point, given results to-date as well as emerging circumstances that create opportunities and constraints with an eye towards supporting sustainability and scale up (as per the GPSA’s Theory of Action). The goal is to produce timely feedback, operationally feasible guidance, recommendations about allocations of limited resources, and contribute to reflection and adaptation. This mid-term assessment should be coordinated to coincide -to the extent possible- with the World Bank’s mid-term supervision mission to assess the project’s progress. While the mid-term assessment can be articulated with the final evaluation, their goals are distinct.

• A final independent evaluation of the GPSA project (in the final year of the project): The evaluation should answer the key evaluation questions included in this TOR (see below) with causal analysis, informed by the GPSA’s and Project’s Theory of Action as well as analysis of the project-level Results Framework, qualitative and quantitative data captured in the project baseline report, project monitoring and mid-term assessment data and the final evaluation to provide an overall picture of causality.

The evaluation must show results made in the project in a credible and transparent. To do so, the consultant will update the research design for the final evaluation and will develop additional appropriate data collection tools and protocols, which will be cleared prior to implementation. The consultant will collect data, data analysis (theory-informed and causal), share and validate findings to ensure transparency and results are credible, as well as draft and finalize the report.

The final evaluation report will provide findings, lessons learnt, conclusions and recommendations, including learning and knowledge about the key evaluation questions. One goal of the final evaluation is to inform improvements to collaborative social accountability strategies for those working in the sector to utilize. The final evaluation is considered the grantee contribution to the GPSA global knowledge platform. The final evaluation report should be of a high enough quality to share with donors, international agencies, and interested third parties engaged in collaborative social accountability processes. A summary of the final evaluation report will be presented to the grantee/CSO partners, WB TTL and GPSA Secretariat in an online PowerPoint presentation.

All draft deliverables shall be submitted in advance to allow sufficient time for consultations and feedback and approval by the GPSA Secretariat.[2]

• Provide continuous MEL capacity building support to INTEC: This includes support to providing project monitoring, reporting as well as making updates and adaptations to the MEL system through reflection and learning as the project is implemented. (continuous)[3].

The consultant will also support the INTEC to use learning from MEL as well as address capacity needs by:

1. Ensuring the project-level Results Framework indicators remains relevant as well as the MEL system.
2. Monitoring data collection is reliable.
3. Monitoring data analysis is accurate.
4. MEL data is interpreted and validated by key stakeholders.
5. Feedback and support on project reporting is provided.
6. MEL data can be communicated and used appropriately for adaptive learning purposes.

All reports provided should be of high enough quality to share with donors, international agencies, and interested third parties. It should provide substantive evaluation against indicators as outlined in the project Results Framework, results model and work plans.

The final evaluation report must:

• Contain an executive summary (mandatory);
• Be analytical in nature (both quantitative and qualitative), causal rather than descriptive and engage the growing body of lessons and evaluations about collaborative social accountability;
• Be structured around issues and related findings/lessons learned;
• Include conclusions; and
• Include recommendations.

4. Key Evaluation Questions^[4] and Methodology

Below are the key evaluation questions the consultancy is expected to address in the final evaluation to take place in 2025.[5]

1. To what extent, in what circumstances, and for whom did the Project contributeto the intended medium and long-term results as per its Theory of Action and Results Framework?  
2. Did the results from the Project include any unintended results (positive and negative)? If so, what were the unintended results and what effect, if any, did they have on the Project Theory of Action and assumptions?
3. Turning to the GPSA’s Theory of Action, to what extent and how did the process of implementing the Project and achieving or contributing to the identified results align with the hypothesized path as described in the flower illustration of the GPSA’s Theory of Action? Are there instances of this Project’s journey – from receiving the grant to contributing to long-term results – that do not align with the path described in the GPSA’s Theory of Action and related assumptions? If so, what accounts for the divergence?
4. In what ways, if any, is the Project or any of its components likely to be sustainable? As discussed in the GPSA’s MERL Guide, the GPSA is particularly interested in sustainability through partial uptake of lessons and Project approaches by public sector institutions, WBG operations and strategies, development partners, among others (see GPSA Results Framework outcome indicator 4).

• Clearly articulates a research design which uses theory-informed, causal analysis and appropriate qualitative methodologies in line with the purpose and specific goals of the assignment. Guidance on this is available here.
• Shows alignment with the adaptive learning approach - specifically how from project MEL activities that took place considers causal logic from project inputs to expected outputs and will lead to a substantive evaluation against all (functional equivalent) indicators as outlined in the project-level Results Framework and a final analysis of the project outcomes against the Project’s Theory of Action.
• Articulates concrete processes to build stakeholders’ key capacities, including but not limited to inputs to help structure and facilitate the creation of the project-level Theory of Action and Results Framework, as well as how adaptive learning will be used to course correct project implementation and update the MEL system.
• Discusses the potential risks and challenges for the evaluation and how these will be managed.

The successful tenderer will then refine this proposal, in consultation with the World Bank/GPSA, INTEC and IHO.

5. Users and intended uses of MEL products and evaluations.

Project Partners

• To guide improvement of other (ongoing and future) programs and the needed adjustments to improve quality, effectiveness, and efficiency of overall program operations, strategies, and policies.

Donors (World Bank/GPSA)

• To inform and support the GPSA’s portfolio level evaluation, including bounded comparisons among sub-sets of projects seeking to strengthen service delivery systems.
• To guide improvement of other (future) programs and the needed adjustments to improve overall program operations, strategies, and policies.

Roles and Responsibilities

The overall responsibility for managing the evaluation logistics and planning will sit with INTEC including:

• Facilitate contact with key partners and stakeholders of this project;
• To facilitate the access of the evaluator to relevant data and documents;
• To arrange the logistics of travel, interviews and focus groups;
• To manage the contract/evaluation process and serve as a liaison with the evaluator;
• Provide comments on the draft and final reports and approve them; and,
• Participate in briefing and debriefing process for evaluation consultant on the objectives of the evaluation.

GPSA/World Bank will:

• Provide comments and feedback on (i) Theory of Action, updated Results Framework and Research Design with MERL system/MERL Plan; (ii) Baseline Analysis Report (iii) Mid-term Assessment Report; and (iv) Draft and Final Evaluation Reports.
• Share key GPSA Program documents, and provide clarification as needed.

Consultant will:                            

• Develop and elaborate a detailed research design considering the GPSA’s lessons to-date and needs;
• Develop a fit-for purpose detailed methodology for the evaluations in response to the key evaluation questions;
• Collect and analyze data and conduct activities outlined in this TOR;
• Map and review project, sectoral GPSA, and other relevant documentation;
• Keep abreast of adaptations to GPSA’s MERL guidance and project lessons;
• Deliver all products in a timely fashion, considering stakeholders’ bandwidth;
• Be responsible for the quality of all evaluative exercises;
• Develop and present all drafts and final evaluation reports (baseline, mid-term assessment and final evaluation); and,
• Present key findings of the final evaluation to stakeholders.

6. Milestone Payments

The payment structure for the work of the consultant is contingent upon the satisfactory delivery of key milestones and outputs rather than on a set number of days. In reference to the lump sum amount specified, payment will be in four tranches:

1. First payment (40%): upon submission and approval of Research Design containing localized Theory of Action, Results Framework, and methodology for delivering the consultancy’s tasks; and project MERL system and MERL plan and upon completion of supervision of data collection and submission of baseline report.
2. Third payment (20%): upon submission and satisfactory review of mid-term assessment
3. Forth payment (10%): upon submission and satisfactory review of the final draft evaluation report.
4. Fifth payment (30%): upon submission and satisfactory review of the final evaluation report.

7. Consultant Qualifications

This assignment will be contracted to an independent expert evaluator consultant or consulting firm with international experience and knowledge in monitoring and evaluation of social accountability projects. The consultant (could be a team) must meet the following qualifications:

• A post-graduate university degree in the social sciences.
• A minimum of 7-10 years of collecting and analyzing qualitative and quantitative data, including conducting interviews, development of monitoring indicators, and using data for causal claim-making in evaluative studies;
• Solid understanding of social accountability, citizen participation and public governance. Previous experience evaluating social accountability and citizen participation projects is preferred.
• Proven work experience in assessing measurable target indicators
• Proven competence in sample survey techniques
• Knowledge of theory of action, political economy and adaptive management approaches will be considered an asset.
• Demonstrated ability to assess complex situations in order to succinctly and clearly distil critical issues.
• Excellent analytical and report writing skills
• Able to work independently with excellent demonstrated teamwork, good communications and facilitation skills
• Experience in leading multi-disciplinary teams to deliver quality products in high stress and short deadline situations.
• Previous experience in WB/GPSA evaluations is an asset.

8. Application Process

The Consultant should express their interest in this call by providing a curriculum vitae with referees and samples of recent work to INTEC by electronic mail to:

Manuel Hernández, INTEC Project Manager    

Esta dirección de correo electrónico está siendo protegida contra los robots de spam. Necesita tener JavaScript habilitado para poder verlo.

Jhoan Almonte, INTEC M&E Officer

Esta dirección de correo electrónico está siendo protegida contra los robots de spam. Necesita tener JavaScript habilitado para poder verlo.

The documents should be submitted by April 24^th, 2023. Only selected candidates will be contacted.

9. Annexes

Annex 1: Monitoring and Learning within the Project

M&E Principles: The consultant should operate on principles that are in line with the principles of evaluation of the World Bank/GPSA. That is the evaluation must be:

• Adaptive: The project seeks to contribute to governance and development outcomes, through adaptive principles. Project partners and the World Bank will monitor PDO level indicators, as well as implementation and outcomes of activities supported by each Project component.  GPSA projects favor flexibility and adaptation in the design of its interventions, with a view towards scaling up support where the likelihood of sustainable reforms is greatest.  

It is in the scaling of collaborative social accountability mechanisms and processes developed under GPSA grants that success is demonstrated.  As information is gathered during implementation, about what is happening and how we measure it, the project indicators may change. GPSA projects are based on a solid understanding of the technical challenges of implementing collaborative social accountability mechanisms, but they also strive to understand and leverage local contexts. As such, it is important that project course corrections and adaptive management is encouraged and rewarded based on effective supervision and implementation facilitation.

Therefore, as INTEC and the World Bank monitor progress towards the achievement of PDO level indicators and learn how the project’s interventions contribute to it, some component-level indicators may have to be revised or changed and budgets reallocated. Adaptive learning and politically informed action by all stakeholders, including the GPSA, during the project cycle (from inception to evaluation to sustainability beyond the lifetime of the intervention) are important towards the effectiveness, sustainability and coherence of collaborative social accountability.

• Useful: for the evaluation to influence decision making, the findings must be perceived as useful and as geared to current operational concerns. Participation of a range of stakeholders at all levels is an important part of this assessment. The consultant should consider the audience of the evaluation and ways to maximize learning through this process.

• Independent: For the evaluation to be impartial, it must be free from bias in findings, analysis and conclusions. In turn, this means independence from contracting firm or consultant.
• Credible: the evaluation must be perceived as objective, rigorous and impartial.
• Transparent: credibility and usefulness will also depend on the transparency of the evaluation – the ready availability of findings to all stakeholders.

M&E Arrangements: In this context, INTEC will appoint an M&E officer at the onset of the project who along with the MERL Consultant will help assess the Project’s performance, build stakeholders’ capacities and help stakeholders to reflect, learn and adjust from their experiences. In addition, INTEC will submit bi-annual technical self-assessment progress reports.[6]

INTEC plans to devise additional monitoring tools-with support from the independent evaluator- that will enable INTEC its partners and the GPSA-World Bank to keep better track of the Project’s progress, identify any obstacles and generate information that can allow adjustments in planning in a dynamic and flexible way.

The World Bank will also produce Implementation Status and Results Reports (ISR) and one final Implementation Completion Report (ICR) that will seek to examine if adaptive approaches taken helped to solve the problems targeted -service delivery and otherwise-, and whether the capacity of individuals, beneficiaries and implementing partners improved under the project. GPSA projects are not assessed solely on how project implementation followed its original design. For projects where the GPSA and its implementing partners expect to learn what works and doesn’t work through collaborative social accountability, an attempt to formalize strict milestones and indicators at the start of the project can risk derailing the iterative approach needed for success. 

The INTEC M&E officer will be key to ensuring the MEL consultant (or firm) is supported in the planning and implementation of evaluations as well as the necessary logistics (e.g., arranging interviews, transportation, and accommodation) related to collecting baseline, mid-term assessment and final evaluation. In addition, the INTEC will also provide the MEL consultant the appropriate reporting formats to use for completing all MEL deliverables to meet WB requirements for professionalism for publication, stylistic consistency throughout the project and across the GPSA portfolio. This is to also ensure efficiency for the WB in having to address stylistic issues for final evaluation publications.

[1] Please note: this section provides the minimum list of expected activities, outputs and estimated timeline. The list of detailed activities is subject to revisions based on the individual proposal of the Consultant.

[2] Please note: the consultant will be required to provide a final evaluation plan by the project’s end date or before. This is a requirement for the project to receive the final disbursement tranche from the GPSA. Data for the final evaluation should take advantage of that collected by the monitoring system during the lifetime of the project. Additional data may be collected after the project’s official closing date, given that WB-GPSA projects have a six-month grace period after the project’s closing date to furnish the project’s final evaluation to the WB-GPSA. Please note, however, that the project’s TTL is also required to prepare an Implementation Completion Report (ICR) which will partly draw from the project’s independent final evaluation report. Hence, the final evaluation report will need to be completed with sufficient time before the end of the 6-month grace period. These arrangements must be discussed with the CSO, GPSA and WB TTL. 

[3] NB: The deliverables will be negotiated based on project timeframes and budget.

[4] These are subject to refining based on the Theory of Action workshop to localize the GPSA’s Theory of Action.

[5] These should be confirmed based on discussions during the development and updates of the Research Design and with the GPSA Secretariat. Furthermore, questions for the baseline and mid-term assessment must be discussed and agreed during the research design phase.

[6] GPSA requires all grant projects to submit two technical progress reports throughout the year: 1) A midyear progress report by February 15th^th covering July-December and focused on the project’s adaptive management and learning during that period, and 2) An annual progress report covering the whole fiscal year (July-June) by August 15^th.