Final Evaluation of the Support of sustainable and fair tuna fisheries in two of the most important fishing grounds in the Philippines - establishing of a MAP-structure project

Projects: Support of sustainable and fair tuna fisheries in two of the most important fishing grounds in the Philippines - establishing of a MAP-structure (henceforth ‘the project’)
Project Location: Philippines, Mindoro Strait and Lagonoy Gulf
Reports to: WWF Philippines (Project Manager: Joann Binondo, Overall Project Manager Sustainable Tuna Partnership)
Project Duration: Dec. 1, 2017 – Dec. 31, 2020
Project Evaluation Period: Entire project duration
Names of Implementing Partners: Tambuyog Development Center

PROJECT/PROGRAMME OVERVIEW AND STARTING SITUATION

A significant share of Filipino fisherfolk continue to live below the poverty line. The project areas of Lagonoy Gulf and Mindoro Strait are among the country’s major tuna fishing grounds. For the majority of handline fishers in the coastal communities, tuna is the main source of income. Communication and collaboration among the relevant stakeholders of small scale and industrial fisheries, public authorities, science and the public are insufficient and co-management bodies are not operational. Because of conflicting interest, fisheries stakeholders solely aim at increasing their catches and the holistic management approach needed for sustainable yellowfin tuna fisheries cannot be implemented. A lack of confidence and cooperation among supply chain actors and missing business and financial expertise as well as the absence of funding alternatives limit the options of small-scale fishers and their families to negotiate prices in a fair and transparent manner. This leads to low income margins as well as dependency and indebtedness to the local traders. Overfishing and the resulting decline in productivity of the tuna stocks and the indebtedness of tuna fishers increase their poverty risk. In cooperation with WWF DE the WWF PH has previously implemented a public-private partnership (PPP) project to promote sustainable tuna fisheries in the project sites since 2011. This cooperation has been closely collaborating with the small-scale tuna fishers in the site and other relevant supply chain actors. It facilitated the organization of 21 Municipal Tuna Fishers Associations. Based on that, the present project, which is the object of this evaluation, has been developed together with the handline fishers as a multi-actor partnership project to address the economic and living situation of fisherfolk as well as the unsustainable use of yellowfin resources in this important area. The purpose of this end-of-project evaluation is described in the following section.

EVALUATION PURPOSE AND USE, OBJECTIVES, AND SCOPE

The main justification for this final evaluation is the verification of successes, completeness and context of the project by assessing the relevance, efficiency, effectiveness, adaptive capacity and sustainability of the project strategies over the entire implementation period. It represents a cornerstone for cooperation between WWF Germany and WWF Philippines in fisheries-related conservation work. At the time of commissioning this evaluation, a follow-up project is being planned and under discussion with the donor BMZ and its executing agency Engagement Global. This new project, which is planned to start in May 2021, intends to broaden the scope beyond the Philippines to make tuna fisheries on shared stocks in the Western and Central Pacific more sustainable, which particular focus on Indonesia, as Philippines and Indonesia share important spawning and juvenile grounds for tuna in the region. It will require a feasibility study and the results of this evaluation shall therefore be used as an input for guiding the preparations of the planned new project.

EVALUATION CRITERIA AND GUIDING QUESTIONS

Criterion 1: Relevance and Quality of Design

Relevance and quality of design is a measure of the extent to which the conservation project design represents a necessary, sufficient, and appropriate approach to achieving changes in key factors (e.g. direct and indirect threats, opportunities, stakeholder positions, enabling conditions) necessary to bring about positive changes in targeted elements (in this case inter alia income generation and management participation of fisherfolk, empowerment to governance of tuna stocks to improve responsible yellowfin fisheries and associated ecosystem services that support human wellbeing).

Key Questions to Assess Relevance and Quality of Design

• Focal conservation targets and related goals (tuna stocks in the project area, as well as ecosystem services that support human wellbeing): Is there a clear and relevant definition of ultimate conservation success in terms of improved status of conservation targets? Has the underlying assumption of the project, i.e. contributing to socio-economic development of tuna fishers through strengthening multi-level resource governance, as well as fisherfolk’s representation in it, as well as capacity building to develop social enterprises, been fulfilled?
• Relevance to context, priorities of stakeholders, and objectives: Does the project address the right aspects of tuna fisheries governance to achieve the intended outcomes? Has the project focused on and does it remain relevant to issues of highest priority? Does a multi-stakeholder partnership support the right processes towards the intended outcomes and outputs?
• Suitability of strategic approach: Has the project taken the , most efficient strategic approach? Given the complexities of multi-actor coordination, could the project have taken more efficient ways to reach the intended outcomes?
• Coherence and sufficiency of project portfolio: Does the project ‘add up’ to the necessities of WWF Germany’s and WWF Philippines’ programmatic and strategic goals?
• Relevance to WWF and donor priorities: Given WWF’s priorities and what it is most needed to do, is the project doing what it should do? Does the project contribute sufficiently to the fulfilment of the WWF global tuna strategy? How does the project contribute to the priorities of the donor BMZ and to the UN Sustainable Development Goals?
• Adherence to WWF social policies. How well has the social context been understood by the project team? How were the cross-cutting principles of the WWF Environmental and Social Safeguards Framework (ESSF) considered in the project, i.e. human rights, indigenous people´s rights, gender, poverty and inclusion? Particularly regarding the social sector most affected by the project, the fisheries sector, how well has WWF reacted to the need for participation in project design and implementation?

Criterion 2: Efficiency

Efficiency is a measure of the relationship between outputs (i.e. the products or services of an intervention) and inputs (i.e. the resources that it uses) and may include a measure of ‘value for money.’

Key Questions to Assess Efficiency

• Financial & Administrative Resources Are the budget of the project and the planned outputs and outcomes consistent with one another (i.e. have sufficient financial resources been budgeted to support planned activities)? What could have been done better in order to balance budget allocation and output/outcome achievement?

Are WWF Philippines’ administrative structures and capacities sufficient to implement a project of this size or even larger, meet the requirements of the donor agency and ensure the quality that is needed? What are the decisive factors?

Did WWF Philippines provide timely and sufficient flows of match funds to the project?

Is actual spend in line with the budget?

Are there savings that could have been made without compromising the quality of results delivered?

• Use of Time: Are there thorough, well founded work plans being implemented according to plan, monitored, and adapted as necessary?
• Human Resources: Are human resources (i.e. WWF Philippines, Tambuyog, WWF Germany) appropriate, adequate, efficiently organized and operating effectively (e.g. include considerations of capacity needs and gaps, communications, stakeholder engagement, division and clarity of roles and responsibilities, processes for evaluation and improvement)?
• Resource use: Is the project delivering value for money in that costs are reasonable given the outputs and outcomes generated?

Criterion 3: Effectiveness

Effectiveness is a measure of the extent to which the intervention’s intended outcomes—its specific objectives or intermediate results—have been achieved.

Key Questions to Assess Effectiveness

• Planned results verses achievement: Focusing on stated objectives, desired outcomes, and intermediate results (as opposed to delivery of activities and outputs), what has and has not been achieved (both intended and unintended)? In how far have unplanned achievements been a direct result of the project or external influences? What are the main achievements that should be carried forward into a new project?
• Factors Affecting Effectiveness: Which strategies, such as the dialogic change process applied, are proving to be effective, and which are not? What anticipated and unanticipated external factors have promoted or impeded the projects progress? What supporting or impeding factors might affect successful implementation in future projects? What technical, organizational and financial factors within WWF Philippines have helped or limited effectiveness?
• Coordination & Communication: To what extent has coordination/communication been effective within and between the implementation team (WWF Germany, WWF Philippines and Tambuyog), stakeholders, partners and participants, as well as BMZ? Are there well developed internal and external communications strategies being implemented to good effect (e.g. providing reach and/or spread)? What factors have hindered good communication and coordination? What could be done differently to improve this?
• Improving Effectiveness: What lessons can be taken and applied to improve effectiveness in the coming project?
• To what degree were outputs and newly created capacities used and is there an inclusive access (physical, non-discriminatory, affordable) to them in place?

Criterion 4: Impact

Impact is a measure of all significant effects of the conservation intervention, positive or negative, expected or unforeseen, on targeted biodiversity/footprint issues – e.g. species, habitats, and ecological processes (including those associated ecosystem services that support human wellbeing).

Key Questions to Assess Impact

• Evidence of Change: To what extent has the project attained its stated vision and goals, in terms of outcomes and outputs (impact matrix) effecting positive change in human wellbeing, strengthening of fisheries governance, and sustainability of small-scale tuna fisheries in the project region? Discuss observed impacts at all appropriate scales—local, landscape, national, regional, global, and present evidence.
• Contribution: How confident can we be that the project´s activities contributed to perceived changes? What is the likelihood that these changes would have occurred in the absence of the project?
• Unforeseen consequences: Were there any unforeseen impacts (whether positive or negative) on the level of the target group and on an overall scale (social, ecologic, economic)? Could anything have been done differently to repeat or avoid these unforeseen consequences and to have acknowledged them earlier as emerging consequences?
• Increasing impact: How might the project increase its impact and what would be the associated human and financial capacity needs? How was the process of increasing impact understood at the design stage (e.g. project replication, good practice guidelines through policy change, multi-stakeholder processes)? What assumptions were made at the beginning of this project about a follow-up project and were they correct?

Criterion 5: Sustainability

Sustainability is a measure of whether the benefits of a conservation intervention are likely to continue after external support has ended.

Key Questions to Assess Sustainability

• Evidence for Sustainability: Is there evidence that the following key ingredients are being established or exist to the extent necessary to ensure the desired long-term positive impacts of the project?
  • Necessary policy support measures including resource use regulations, such as continued implementation and monitoring of the National and local Tuna Management Plans, and policy changes and implementation related to preventing harmful subsidies into the project fisheries and promoting investments into sustainable fisheries management, continuation of multi-stakeholder partnership etc.
  • Adequate socio-cultural integration, including no negative impact on affected groups (e.g. by gender, religion, ethnicity, economic class) and/or on benefits realized by them, as well as ensuring necessary motivation, support, ownership and leadership by relevant individuals and groups.
  • Did the project sufficiently address fundamental rights of resource users in the project area (i.e. access rights of fisherfolk, gender inclusion etc.)? What are the key strategies to ensure their proper consideration and how can they become continuously guaranteed?
  • Adequate institutional and organisational capacity and clear distribution of responsibilities among those organisations or individuals necessary to ensure continuity of project activities or impacts. For example, .
  • Technical and economic viability and financial sustainability, f.e. of fishers “incubation funds” and social enterprises to be enabled by the project.
  • Technology, such as the Electronic Catch Documentation systems, Vessel tracking and Municipal Fisheries Management Database that is appropriate to existing conditions and capacity.
• Risk and Mitigation: What external factors could have a high or medium likelihood of undoing or undermining the future sustainability of project positive impacts? (e.g. political stability, economic crises and shocks including health crises such as the current pandemic, overall level of development, natural disasters, climate change – especially related to the most vulnerable groups). Did the project adequately anticipate and take measures to ensure resilience to these? Particularly with regards to the COVID-19 pandemic, did the project prove to be sufficiently prepared? What are lessons for a future project?
• Exit—Phase Out Plan: Based upon existing plans and observations made during the evaluation, what are the key strategic options for the future project (e.g. exit, scale down, replicate, scale-up, continue business-as-usual, major changes to approach)? Has the project conducted appropriate knowledge management? Are the results, learnings and products of the project easily available to all relevant stakeholders? What are recommendations for knowledge management?

Criterion 6: Adaptive Capacity

Adaptive Capacity is a measure of the extent to which the project or programme regularly assesses and adapts its work, and thereby ensures continued relevance in changing contexts, strong performance, and learning.

Key Questions to Assess Adaptive Capacity

• Applying Good Practice: Did the team examine good practice lessons from other conservation/ development experiences and consider these experiences in the project design?
• Monitoring of status: Did the project establish a baseline status of conservation and development targets and key contextual factors (e.g. income levels of fisher households, yellowfin stock health etc.)? Is there ongoing systematic monitoring of these?
• Monitoring of efficiency, effectiveness, impact: Did the project track intermediate results that are part of a theory of change (including results chains) that clearly lay out anticipated cause-effect relationships and enable definition of appropriate indicators? Has there been ongoing, systematic, rigorous monitoring of output delivery, outcome attainment, and impact measurement? What percentage of overall staff time and funding has been dedicated to project monitoring, adaptation, and learning? Are there any staff positions dedicated more than half-time or full time to support these efforts?
• Learning: Identify any exceptional experiences that should be highlighted regarding what worked and didn’t work (e.g. case-studies, stories, good practices)?
• Risk Assessment: How often were the original risks and assumptions revisited during the intervention cycle? Were the risks assessed adequately enough and were external assumptions identified realistically? How were mitigation strategies identified and responded to by the intervention team to optimize?

METHODOLOGY CONSIDERATIONS

The final evaluation of the project shall apply a mixed-method and in-depth approach, including a desk analysis of existing documentation, new information collection via phone interviews, surveys etc. .Depending on travel restrictions in place due to the COVID-19 pandemic during the period of evaluation, a visit to the project sites may not be feasible. Should the situation on the ground improve, however, it cannot be ruled out. The detailed methods for gathering additional information shall be discussed under the light of COVID-restrictions with WWF at the start of the work. On this basis a specified clear plan for information collection will be prepared. WWF will provide support to ensure information gathering is facilitated to the best means possible.

Core documents the evaluation should consult include:

• The original project proposal to BMZ
• Progress reports submitted to WWF Germany and all related documents provided for evidence (these include baseline studies
• Progress reports submitted to BMZ (‘Zwischennachweis’)
• Reports and presentations on traceability work and related pilots, work plans, monitoring and Evaluation plans, process monitoring for the MAP-dialogue process, risk registers, project agreements, project amendments.

Key external partners and stakeholders to be consulted:

• IFARMC (representatives from fisherfolk org., academe, national agencies and Local government units)
• BFAR Regional Offices of Region 5 (BFAR-Bicol) and Region 4-B(BFAR-MIMAROPA)
• 21 Local Government Units (Municipal Agriculture Office/ Local Chief Executives)
• Tuna Fishers Federation Leaders and 21 Municipal Tuna Fishers Officers and members
• Additional tuna fishers in the target group that participated the socio-economic activities with Tambuyog
• MSC Client Group Members (PATPI, GLTFFI and OMFTFA)
• Members of PaNaGaT Civil Society Organization Network
• BFAR Central Office (Capture Fisheries Division Chief), relevant trade (e.g. DTI) and finance administrative bodies
• Local traders, tuna exporters and Swiss Coop

PROFILE OF EVALUATOR(S) AND WWF SUPPORTING RESPONSIBILITIES

Evaluators: WWF is looking for individual consultants or consulting teams to perform this service. Experience in impact evaluation of fisheries and sustainability, socio-economic development and complex stakeholder processes is a must. Strong English language skills are a prerequisite, as the report is expected to be delivered in English. CV, track record and other evidence of relevant experience to handed with the application.

The evaluation mission must sure that the following principles for quality evaluations are adhered to:

• Useful: Evaluations must be tailored to respond to the questions and key information needs of those most likely to utilize the evaluation results.
• Independent: For an evaluation to be impartial and therefore more likely to be objective, respected, and accepted, it must be free from bias in findings, analysis, and conclusions.
• Inclusive: Evaluators and evaluation managers (those overseeing the evaluation process) must design and conduct the process with an eye to promoting project/program team self analysis, learning, and constructive participation.
• Timely: Evaluation should be timed to inform key decision-making processes, such as planning a second program phase.
• Evaluations and evaluators must respect the project/programme team and their key stakeholders and supporters.
• Credible: To be viewed as credible, evaluations must be rigorous, impartial, and conducted by a well-qualified evaluator.
• Transparent: Findings must be readily available to all stakeholders, relevant stakeholders should have the opportunity to comment on the draft evaluation products.
• Ethical: Evaluations must adhere to relevant professional and ethical guidelines, be undertaken with integrity and honesty, and be respectful of human rights, differences in culture, customs, and practices of all stakeholders.

EVALUATION PROCESS, DELIVERABLES, AND TIMELINE

WWF Support: Main support and coordination will be provided by the Overall Project Manager of the Sustainable Tuna Partnership (STP) of WWF Philippines. Other key support will be provided by the responsible Project Coordinator at WWF Germany.

FURTHER PROVISIONS

The scope of tasks is not claimed to be exhaustive. It is the consultant’s responsibility to critically verify the scope of services indicated and amend it wherever deemed necessary according to the consultant’s own professional judgment, the knowledge acquired during the preparation of the proposal and after consultations with the contractor. The consultant is obliged to perform all work as necessary to fulfil the objectives of the feasibility study.

All consultant services as described shall be performed in close cooperation with the responsible institutions and administrations. Furthermore, the consultant shall meet and coordinate activities with the operations of other institutions relevant to the Project.

Please bear in mind:

Tenderers themselves will bear the costs of drawing up their tenders and WWF will not be liable to pay any compensation if a tender is rejected or if it decides not to select any tender.

Annex 1: Evaluation Reports

To support more systematic recording of evaluation findings to advance WWF’s broader organisational learning, all evaluators should follow, to the extent possible, the evaluation report structure below and complete the following table (Part B), to be attached to the evaluation report.

Part A - Report Table of Contents Template

The following provides a basic outline for an evaluation report. While this should be easily applied to evaluations of simpler projects or programmes, adaptation will be needed to ensure reports of more complex programmes (e.g. Country Offices, multi-country regions, eco-regions, Network Initiatives) are well organised, easy to read and navigate, and not too lengthy.

Title Page

• Report title, project or programme title, and contract number (if appropriate), Date of report, Authors and their affiliation, Locator map (if appropriate)

Executive Summary (between 2 to 4 pages)

• Principal findings and recommendations, organised by the six core evaluation criteria
• Summary of lessons learned

Acknowledgements

Table of Contents

List of Acronyms and Abbreviations

Body of the report (no more than 25 pages)

1. Introduction (max 3 pages)
   • Concise presentation of the project/programme characteristics
   • Purpose, objectives, and intended use of the evaluation (reference and attach the ToR as an annex)
   • Evaluation methodology and rationale for approach (reference and attach as annexes the mission itinerary; names of key informants; a list of consulted documents; and any synthesis tables containing project/programme information used in the exercise)
   • Composition of the evaluation team, including any specific roles of team members
2. Project/Programme Overview (max 5 pages)
   • Concise summary of the project or programme’s history, evolution, purpose, objectives, and strategies to achieve conservation goals (attach theory of change including conceptual model, results chain or logical framework and project monitoring system as annexes)
   • Essential characteristics: context, underlying rationale, stakeholders and beneficiaries
   • Summarise WWF’s main interest in this project or programme
3. Evaluation Findings (3-5 pages)
   • Findings organised by each of the six core evaluation criteria, including sufficient but concise rationale.
   • Tables, graphics, and other figures to help convey key findings
4. Recommendations (3-5pages)
   • Recommendation organised each of the six core evaluation criteria, including sufficient but concise rationale – recommendations should be specific, actionable and numbered.
   • Project/programme performance rating tables to provide a quick summary of performance and to facilitate comparison with other projects/programmes (see the Summary Table Part B, below).
5. Overall Lessons Learned (max 3 pages)
   • Lessons learned regarding what worked, what didn’t work, and why
   • Lessons learned with wider relevance, that can be generalised beyond the project
6. Conclusions
   • General summation of key findings and recommendations

Annexes

• Terms of Reference
• Evaluation methodology detail
• Itinerary with key informants
• Documents consulted
• Project/programme theory of change/ logical framework/ conceptual model/ list of primary goals and objectives
• Specific project/programme and monitoring data, as appropriate
• Summary tables of progress towards outputs, objectives, and goals
• Maps
• Recommendations summary table