The end of project evaluation of the project titled as ‘Post-emergency and recovery response: Provision of Health, protection, and MHPSS activities th

TERMS OF REFERENCE

EXTERNAL END OF PROJECT EVALUATION

SUBJECT: The end of project evaluation of the project titled as ‘Post-emergency and recovery response: Provision of Health, protection, and MHPSS activities through mobile medical teams and safe spaces in Hatay’

COUNTRY: Turkiye

AUTHOR(S) OF THE TOR: Steering Committee Members

DRAFTED ON: March 2025

MR No.: Financing Agreement CTR11285 (AFD)

CONTEXT OF THE EVALUATION

ORIGIN OF THE REQUEST OF SERVICE

On February 6, 2023, a magnitude 7.8 earthquake struck southern and central Turkey. It occurred 34 km west of the city of Gaziantep at 04:17 (local time), causing extensive damage and casualties in the southeastern Turkey and northwestern Syria. The earthquake was followed by numerous aftershocks, the strongest of which had a magnitude of 7.5. This aftershock occurred 9 hours later, 4 km northeast of Elbistan city in the province of Kahramanmaras at 13:24 (local time).

The earthquake caused the collapse of approximately 6,200 buildings in 10 provinces of Turkey. Also in Syria, many buildings have collapsed, mainly in the Aleppo and Hama governorates. More than 53,000 people have lost their lives. WHO estimates the population affected by the earthquake to be 23 million. Hatay province was one of the worst hit, with over 24,000 casualties. Most of the province’s buildings were destroyed or severely damaged, including public infrastructure and health centers.

The member of the Médecins du Monde network in Turkey, Dunya Doktorlari Dernegi (MdM Türkiye, MdM-T), is established at and registered since 2015, has been operational on the ground since the first day of the earthquake thanks to fundraising organized by the Médecins du Monde network and to own funds released specifically but which do not cover all the needs. MdM-T has had a cross-border coordination office in Antakya since 2017 for its operations in northern Syria, which was heavily damaged but is now functional and enables post-emergency care activities to be carried out in the province.

The project, that is subject of the external evaluation study, aims to provide assistance to the EQ affected populations in Hatay Province, and responds to the emergency needs of the most vulnerable populations. The project is funded by the Agence Française de Développemente (AfD) and proposed the delivery of the emergency response, which consists of primary health care, mental health care and psychosocial support services (MHPSS) through mobile teams and creating “safe spaces” to ensure the confidentiality of consultations. It has been proposed that the mobility of these teams would ensure that the project activities serve to those who are difficult to reach, and thus the most vulnerable and isolated earthquake victims. In the second phase of project implementation, the project aimed transition into post-emergency response integrating capacity-building activities to transfer its services to local authorities.

PRESENTATION OF THE PROJECT

The project aimed to address the following humanitarian needs of the EQ affected communities, the lack of public health care providers due to the damaged hospitals and primary health care level service providers, family health centers, and migrant health centers, in the context where the damaged infrastructure has limited access to clean water in both urban settings and the settings within the Temporary Accommodation Centers. The critical hygiene and sanitation conditions within the overcrowding in temporary settlements are likely to increase the occurrence and spread of diseases linked to poor hygiene and contaminated water, as well as other illnesses such as measles, chickenpox, meningitis and polio. In addition, respiratory diseases are frequently linked to poor air quality and hygiene conditions, combined with poor nutrition. Thus, MdM-T responded by mobilizing a medical team, consisting of a general practitioner (doctor) with a nurse and a midwife to screen the medical conditions of the target communities, and provide medication where possible.[1] The project aimed to improve the ability of the EQ-affected communities to access basic services, such as healthcare, education, and other sector services. Thus, the mobile team also included social workers towards those in need, including vulnerable groups such as the elderly, people with disabilities (PwDs), and those requiring specialized care are linked with existing service providers[2]. The project aimed to serve both host and migrant communities and set the target of the migrant community members among all beneficiaries to be 65%.[3]

EVALUATION OBJECTIVE AND TARGET AUDIENCE

OBJECTIVE AND AIM

The general objective of this independent external evaluation is to capitalize on the emergency and post-emergency responses provided by MdM-T, identify strengths and shortcomings, collaboration and coordination between the various actors and identify corrective measures to be adopted in future programming.

It is expected that the evaluation study will identify areas for improvement, adjustments as lessons learned and good practices within the implementation of the project that will be used for informed decision-making to improve and adapt the program interventions in further programming.

Moreover, the evaluation will also ensure accountability towards the donor and the project beneficiaries.

The timeline of the evaluation will not allow application of all lessons learned in the current project, yet they will remain relevant and useful for future projects in the intervention areas, hopefully,

but not limited to the future projects under generous funding of AfD.

SCOPE OF THE EVALUATION

Geographical perimeter

Areas covered by the project : Hatay Province, Türkiye

Facilities to be visited : Temporary Accommodation Centers in the certain districts of Hatay province

Time frame months

The period covered by the evaluation is between February 1st, 2024 to May 31st, 2025 for a total of 16 months. This period covers the entire implementation of the project from its start until the time of the evaluation data collection activities.

Program scope

All the components of the project will be considered, at the level of its objectives, results and activities, as well as from its methodologies and work approaches.

TARGETED AUDIENCE

The results of the evaluation will be/could be used by:

• MdM Chapters – field and HQ – directly involved in the implementation of the project
• The donor of the project, Agence Française de Développement (AFD) for accountability.
• Local Health authorities.
• Other humanitarian actors in the implementation areas of the project.

EVALUATION CRITERIA

This evaluation mission relies on standards evaluation criteria as defined by the Development. Assistance Committee of the Organisation of Economic Cooperation and Development (OECD/DAC):

relevance, coherence, effectiveness, sustainability/connectedness. To be noticed that the efficiency and impact criteria have been considered but not retained due to both required complex methodologies which are supposed to be designed beforehand.

For each criterion, evaluative questions will have to be responded with evidence to be captured primary and secondary data collection and review activities. Where appropriate, the evaluator should highlight any differences in the various districts of Hatay and the type of communities, as host and migrant communities.

Relevance

1. How needs-based, context-adapted, and capacity-conscious was the design of the project?
2. How appropriate was MdM-T’s programme and implementation structure and modality, as mobile team, consists of medical, social work, MHPSS staff members?
3. How relevant were the response modalities or the identified needs and the environment, and how relevant the capacity building for the identified gaps?
4. How appropriate were MdM-T’s beneficiary participation and complaints and feedback mechanisms to the context and the identified needs?

Coherence

1. To what extent did the existing network of service providers (State and non-State) (State and non-St inform the project and how was it coordinated with other existing networks to improve complementarity and coverage?

Effectiveness

1. How effective was MdM-T’s response to the humanitarian needs towards addressing the health care, social and MHPSS related needs
2. How effective was the referral mechanism and pathways in linking those in need of multi-sectorial assistance to the existing specialized service providers?

Sustainability / Connectedness

1. What mechanisms are effectively put in place and utilized to sustain the project services and to implement the proposed transition strategy?
2. To what extent did MdM-T coordination meetings support the transition process and the coordination of participating key stakeholders for the sustainability of services?

Efficiency

1. Were adequate human and financial sources applied to deliver project outcomes?
2. How efficient were the management structures and the implementation modalities in terms of timeliness of delivery and cost-effectiveness of the interventions?

Impact

1. What was the impact of the project on beneficiaries?
2. To what extent does MdM-T training for its staff and other local governmental and non-governmental stakeholders support the recovery of the health system in Hatay?

Cross-cutting

1. Was the gender approach, including gender differences in childhood, included in all phases of the project? How could this be better considered in future programming?
2. Have mechanisms, put in place to consider the cultural diversity to adapt and respond to the needs? How could this be better considered in future programming?
3. Has the project addressed the intersection of disability with gender, age, and cultural background to ensure that people with disabilities received tailored support? How could this be better considered in future programming?
4. To what extent did the project consider environmental impacts in its activities?
5. Were human rights and protection principles, such as dignity, safety, and privacy, embedded in all stages of the project?

OTHER RELEVANT CRITERIA

Coverage and equity

1. What measures were in place and implemented effectively to improve the geographical and vulnerable group coverage of relevant project activities in line with the objective of the project?
2. To what extent has the project involved local communities and stakeholders in identifying gaps in this coverage?
3. How did the project ensure that all population groups had equitable access to quality care?

METHODOLOGICAL PREFERENCES

The expected evaluation approach should (ideally and not exhaustively):

1. Provide evidence-based information on performance of the project against the intervention logic and existing project indicators;
2. Identify and assess key internal and external factors (positive and negative) that have contributed, affected or impeded the achievement, and how DDD handled these factors;
3. Assess the project design and operational approach in terms of evaluation criteria: relevance/coherence, effectiveness, sustainability/connectedness, coverage and equity;
4. Assess the replicability of the interventions and sub-sequent outcomes in other similar settings, as response to earthquake disasters;
5. Document lessons learned.

QUANTITATIVES / QUALITATIVES METHODS

The evaluator(s) will develop the methodology and the necessary tools, which will be included in the inception report. The data collection should include the use of several approaches mainly a combination of quantitative and qualitative methods to gain a deeper understanding of the project interventions, such as:

• Desk review of background documents (project proposals, donor reports, project indicators tracking and monitoring data system project report, M&E reports, etc.)
• Visits to the activities delivered in the safe space and through outreach;
• Observation of project activities;
• Individual and group interviews with beneficiaries, other stakeholders.

Fieldwork will be carried out in the selected project sites, as the Temporary Accommodation Centers, towns and villages, and the methodology will be adapted to these situations, respecting the required quality standards.

MdM-T will make available to the evaluator(s) the sources of verification of the project and will support the evaluator(s) to contact the different actors and the affected populations.

KEYS DOCUMENTS

The evaluator(s) is expected to consult the following documents related to the process:

• Project proposal submitted to the donor.
• AFD IR reports
• Internal M&E reports
• MdM-T Code of Conduct (CoC), Prevention of Sexual Exploitation and Abuse (PSEA) policy and Ethical clauses that applies for all MdM –T staff, partners and suppliers.

KEY PERSONS / INSTITUTIONS

The evaluator(s) is expected to consult the relevant MdM Project staff, key persons from the national and local partner and key persons from the project target beneficiaries.

This includes:

• MdM Türkiye Project staff at field level – face to face
• MdM Türkiye Project staff at HQ level – remotely
• Other relevant national and/or international humanitarian agencies, their representatives
• Local authorities of relevant ministries, i.e. health ministry
• Inter-Agency Coordination representatives – remotely
• Project beneficiaries – face to face

The detailed list of these individuals/authorities will be given to the evaluator(s) during the initial briefing.

EXPECTED DELIVERABLES AND REPORTING

DELIVERABLES

Inception report

An inception report will be produced at the start of the mission by the evaluator(s) after having reviewed the key documents and the initial interviews. Amongst other things, the inception report defines and formalises the planned methodology and the evaluation matrix, sets out the necessary data collection instruments (interview guides, etc.) and suggests a realistic work plan with a detailed and final calendar. The inception report will be submitted to the Steering Committee for comments and validation during the inception meeting.

Evaluation report

A final report will be produced, taking into account the comments made by the Steering Committee on the interim report.

The body of the evaluation report, in Word format with pages numbered, must be between 40 and 50 pages (excluding the annexes), font size 11 and simple line spacing, and must include the following:

• Executive summary (5 pages maximum)
• Introduction
• List of acronyms
• Context (description of the project)
• Objectives of the evaluation and criteria retained
• Methodology and limits
• Main results and analysis
• Conclusions and recommendations (ranked in accordance with the evaluation criteria)
• Annexes: terms of reference, list of individuals interviewed, calendar, questionnaires, interview guidelines, observation grids, documentary review, etc.

The report shall be written and submitted in English.

Other deliverables

Raw data of interviews conducted with staff members, stakeholders and beneficiaries.

RESTITUTIONS / FEEDBACK

A face to face restitution workshop with MdM Turkiye, MdM France, the donor, AfD and other implementing partners of AfD shall be organised. The aim of the workshop is to co-develop the programmatic recommendations by MdM-Türkiye and other AfD implementing partners based on the evaluation findings. Thus, the bid should budget the related costs, such as travel and accommodation, for the delivery of the workshop. The duration of the workshop is anticipated to be one day.

ORGANISATION OF THE EVALUATION MISSION

LOGISTICAL AND ADMINISTRATIVE ORGANISATION

Equipement

• Computer: MdM will not provide a computer. The evaluator(s) will rely on their own equipment.
• Printer: Printers will be available in the field offices in Hatay for the need of any print-out. The final report will be printed by the evaluator(s), if applicable.
• Office: an office shared with MdM colleagues (Istanbul/Hatay) will be available for the evaluator(s). A room for interviews will be available if needed.

Communication

• Internet: Internet is reliable in the Istanbul office and Hatay office, where frequent cuts in the connection occur on a daily basis.
• Phone: the evaluator(s) will have to manage their smartphone and SIM card for national and international calls.

Travel /Accommodation

Travels should be included in the evaluator(s) budget and organised by the evaluator(s) from:

• The evaluator(s) place to MdM’s office (Istanbul/Hatay)
• From Istanbul to Hatay

Accommodation and food are not covered by MdM.

The accommodation options in Hatay that the MdM-T team utilizes are containers, which can be arranged for the evaluation team members.

Administrative aspects

Visa to Türkiye – Responsibility of the evaluator(s). Depends on country of the visitor, but a tourist visa from 30 to 90 days is applicable.

The evaluator(s) are responsible for ensuring that they have all visas and insurance cover necessary, for finding out how to obtain these, for taking the necessary measures and paying the associated costs.

Other

Translator: if needed the evaluator(s) need to manage to find a translator English-Turkish and English-Arabic for the whole duration of the mission and the costs have to be included in the budget proposal.

SAFETY / SECURITY

The safety / security situation is stable in the respective province where the project is implemented. In case the situation evolves and security measure need to be put in place, a briefing will be provided to the auditors.

STEERING AND REPORTING

Steering Committee

A steering Committee is formed with the representatives of MdM-T and MdM-F for commissioning and management stages of the evaluation study.

Its functions are as follows:

• Design and approve the evaluation terms of reference.
• Review the expressions of interest
• Shortlist the applicants to who will be sent the ToRs
• Review the proposal (Technical and Financial) and evaluate the proposal based on the evaluation criteria.
• Select the evaluator(s).
• Review and validate the inception report.
• Provide the evaluator(s) with access to all relevant information.
• Maintain permanent contact with the evaluator(s).
• Ensure compliance with deadlines.
• Review the different products of the evaluation and validate them.
• Facilitate the dissemination of evaluation results.
• Facilitate the development of the management in response to recommendations.
• Follow-up on implementing the recommendations.

Reporting / Milestones

• Weekly update on the progress of the work to be shared in English with the members of the Steering Committee (short summary email).
• Debriefing shall be organized with technical and activity implementation staff (might also include other stakeholders’ staff) at the end of each field visit.

In case of difficulties, reorientations shall be considered during the course of the evaluation, but only after written approval of the Steering Committee.

CALENDAR

The mission should take place during May and June 2025 and the tentative schedule is as follows:

• Preparation phase (document review, briefings, finalization of inception report) : 30/04/2025 to 07/05/2025 (6 days).
• Field work phase (data collection, analysis) : 12/05/2025 to 28/05/2025 (13 days).
• Debrief/Restitution Workshop 30/05/2025 to 30/05/2025 (1 days).
• Writing up the report (draft and final reports) :02/06/2025 to 11/06/2025 (7 days).

The schedule is indicative and subject to change according to proposed data collection modalities with the project beneficiaries. The data collection activities should be completed by the end of May 2025. To note that the working days in Türkiye are from Monday to Friday.

SKILLS REQUIRED FOR THE MISSION

The evaluator/team of evaluators will be in charge of setting out an appropriate approach and methodology, supporting and carrying out the evaluation process, collecting and analysing the necessary data, producing and delivering the inception report and any other productions anticipated, and also providing feedback as stated in these Terms of Reference.

Qualifications and experience requested:

• Track record of conducting evaluation studies – with at least 3 years’ experience in conducting evaluations or at least conducted 3 evaluations, preferably about the health sector programmes integrating MHPSS and protection interventions.
• Relevant technical profiles of the evaluator(s) University degree.
• In the case of a team of evaluators proposed, presence of a gender-balanced team to ensure gender sensitivity is assured for the interviews to be conducted with project beneficiaries, especially adolescent girls and women.
• Fluency in English and Turkish among the evaluator(s) or the evaluation team for stakeholder interviews and additionally Arabic for the interviews with the project beneficiaries, which would include Turkish speaking host community and Arabic speaking migrant community members
• Experience in participatory/qualitative information collection methodologies
• Experience within evaluations of complex strategies or projects in international development cooperation
• Demonstrated experience in cultural sensitivity
• Excellent writing skills.
• Not have been linked to design, management, or execution of the intervention to be evaluated.
• Ability to be mobilized and access project sites with limited logistical and administrative (mandatory support documents for authorities) support of MdM.
• Full understanding and respect of humanitarian principles, MdM network CoC, and PSEA policies.

Preferable

• Professional experience within an NGO, with better understanding of the constraints in the field and credibility with teams, knowledge of MdM a plus.
• Experience of working in Türkiye and for the Earthquake Response programmes.

[1] By the end of January 2025, out of 13,716 medical screening consultations, 2,201 included the provision of medication, as well. Among all medical screening consultations, 3788 are provided by the midwifes, which includes 328 pregnant women.

[1] By the end of January 2025, the protection team provided 3,307 services, conducted 2,051 information sessions, facilitated 968 referrals, and supported 288 individuals with specific needs (SNF).

How to apply

APPLICATION FILE

HOW TO SUBMIT AN APPLICATION

Applicants are invited to submit a complete file of between 10 and 15 pages (excluding CVs) via email to: <tender@dunyadoktorlari.org.tr**>** with the subject line “AFD Project – Türkiye Final Evaluation”. Applications filed in any other manner will not be taken into consideration. The file is considered complete if it includes the following 3 elements:

1. Technical proposalincluding:

• An understanding of the terms of reference;
• The technical approach developed and the detailed methodology;
• Details regarding evaluator(s), how responsibilities are shared between them, the CVs proposed evaluation team members and the availability of the evaluator(s);
• The provisional calendar for the mission and an estimate of charges per person, per day;
• References from 2 similar projects previously completed and sample of work of two previously conducted evaluation studies on the relevant sector projects.

2. Financial proposalincluding:

• The total budget (inc. VAT);
• A detailed itemised breakdown (inc. VAT) of the budget (fees, living expenses, transport, interpreters, etc.).

3. Sworn statement confirming (the absence of) any conflicts of interest.

Thedeadline for submitting an application is 25/04/2025 at 17.30 istanbul, Turkey time.

The criteria to be utilized for the selection of successful bidder is presented within below table:

Evaluation Criteria

The selection of the successful bidder will be based on the following criteria:

1. Proven record of delivery of timely and quality final evaluation services, preferably in post-disaster contexts (20%)

• The bidder must demonstrate experience through three previous contracts for final evaluations of humanitarian programs, clearly stated in the tender submission.
• A sample report of a final evaluation must be submitted. The report must be authored or co-authored by the same staff who will directly support this consultancy and contribute to the deliverables.
• Three validated references must be provided (to be verified by the logistics team).

2. Technical skills, knowledge, and experience of nominated personnel (15%)

• The CVs and bios of the nominated personnel who will directly conduct the evaluation must be submitted.
• Personnel must demonstrate the ability to conduct field-level data collection with Turkish and migrant community members, particularly Syrians, in both Turkish and Arabic languages.

3. Quality of the proposed methodology (25%)

• A provisional methodology (maximum 8 pages) must be submitted. It should demonstrate:
  • A clear understanding of the Terms of Reference (ToR)
  • Appropriate research methods for each evaluation question or theme
  • Access strategies to the relevant geographic areas
  • Proficiency in both quantitative and qualitative data collection methods
  • Clear measures for data protection, safety, and ethical standards
  • Inclusion of a gender-balanced data collection team

4. The amount of the submitted financial offer (40%)

• A detailed financial proposal must be submitted in Euro, including VAT. The cost-effectiveness and clarity of the offer will be evaluated.

INSTRUCTIONS FOR BIDDERS

In order to be authorised to apply, bidders must be able to establish, at the discretion of MdM-F, that they meet the legal, technical and financial conditions applicable and that they have the necessary capacity and sufficient resources to carry out the mission. The following rules are aimed at helping the bidder prepare a complete document that meets the requirements of MdM-F.

Content of the offer of services

The bidder must provide all necessary and sufficient information to enable the technical and budgetary proposal to be correctly evaluated. The information provided must be succinct and cover all aspects. The bidder must be able to show how it holds the required qualifications to carry out the mission. The bidder may add any information considered relevant.

Exclusion criteria

MdM-T and MdM-F will exclude from the contract procedure any bidders to which any of the following applies:

• Bankrupt or the subject of bankruptcy or liquidation proceedings, court-ordered restructuring or preventive arrangements, having ceased trading or in a comparable situation as a result of proceedings of the same kind under domestic legislation or regulations;
• Convicted via a judgment having res judicata (i.e. no longer subject to appeal in any manner) of any offence impacting their professional morality;
• Guilty of serious professional misconduct established via any means by MdM-F ;
• Having failed to comply with their obligations regarding the payment of social security contributions or their tax obligations under the relevant provisions of law;
• Convicted via a judgment having res judicata on charges relating to fraud, corruption, participation in organised crime or any other unlawful activity;
• Which, following the signature of another contract, was declared to have committed a serious breach of contract due to failure to comply with their contractual obligations.

Bidders must be able to certify via any appropriate means that they are not currently in any of the situations described above.

Ineligibility clauses

Contracts are not awarded to bidders who, during the bidding phase:

• Find themselves in a conflict of interest with or have a specific link to any other bidders or parties to the project; any attempt made by a bidder to obtain confidential information, to enter into unlawful arrangements with its competitors or the influence the Steering Committee for the mission or MdM-F during the course of the examination, clarification, evaluation and comparison of the bids will lead to the rejection of the bid;
• Make inaccurate declarations when submitting the information required by MdM-F in order to take part in the contract or fail to provide such information.
• Being blacklisted by UN, European commission or US agency

Grounds for rejection

MdM-F and the bidders shall comply with the highest possible ethical standards when negotiating and performing contracts.

MdM-F will reject any proposal submitted by a bidder and terminate the relevant contract if it is established that this bidder has engaged in acts of corruption, fraud, collusion or coercion. The administrative or financial sanctions applied must be in proportion to the size of the contract and the seriousness of the misconduct established.

MdM-F and has a duty to ensure that all bidders and applicants have no involvement whatsoever in child labour and respect all basic employment rights and rules on working conditions. They must themselves make a commitment not to purchase goods from suppliers involved in child labour or violating basic employment rights and/or rules on working conditions.