World Vision International İnsanî Yardim Kuruluşu Türkiye – Gaziantep
WORLD VISION INTERNATIONAL IS LOOKING FOR AN EXTERNAL CONSULTANT FOR END OF PROJECT EVALUATION FOR OFDA
ALL PROPOSALS SHOULD BE SUBMITTED TO: Gaziantep_Procurement@wvi.org BY THE 29TH of SEPTEMBER, 2017.
ALL RELATED QUESTIONS SHOULD BE SENT TO : Kubra_Ozel@wvi.org
PLEASE INDICATE “PR 1035 EOP EVALUATION FOR OFDA” AS THE SUBJECT OF YOUR E-MAILS.
Terms of Reference End-of- Grant Evaluation
Supporting Affected Communities for Life-saving Action (SACLA)
OFDA Funded Project
|IDPs||Internally Displaced Persons|
|IYCF||Maternal, Newborn, Child Health and Nutrition|
|OFDA||Office of U.S. Foreign Disaster Assistance|
|OECD-DAC||Organisation for Economic Co-operation and Development – Development Assistance Committee|
|PLW||Pregnant and lactating women|
|USAID||United States Agency for International Development|
|MNCHN||Maternal, Newborn, Child Health & Nutrition|
World Vision is seeking an external consultant to undertake an end-of-grant evaluation for the project titled “Supporting Affected Communities for Life-saving Action (SACLA)”implemented in cooperation with Syria Relief (SR) and Insani Yardim Dernegi (IYD) in Idleb and Aleppo governorates with funding from OFDA between September 1, 2016- November 30, 2017. The main objective of the evaluation is measure the relevance, effectiveness, timelinessand efficiency of the interventions applied in this grant to address the needs in target communities.
|Project||Supporting Affected Communities for Life-saving Action (SACLA)|
|Grant agreement number||AID-OFDA-A-16-00037|
|Implementers||World Vision International in TurkeySyria Relief (SR)|
Insani Yardim Dernegi (IYD)
|Evaluation type and purpose||Summative end-of-project|
|Methodology||Quantitative: Household surveyQualitative: Key informant interviews (KIIs), focus group discussions (FGDs)|
Desk review of secondary data and sources
|Evaluation dates||Oct 9- Nov 30, 2017|
|Anticipated draft evaluation report release date||November 20, 2017|
- Background information
With the ongoing conflict in Syria, it is estimated that 11.5% of the population within Syria have been killed or injured, more than 4.5 million people have fled to neighboring countries and over 6.5 million are internally displaced. Aleppo and Idleb governorates have been disproportionately affected by the crisis, in particular by intensifying Russian airstrikes over the last six months. As a result, there are now 1,951,479 IDPs in these governorates, 50% of whom are children. The majority of these IDPs live in crowded temporary shelters that lack privacy and access to electricity, heating, water and sanitation. Such conditions have increased the risk of disease and malnutrition—with cholera, typhoid, hepatitis A, measles, and leishmaniosis reported with increasing frequency. Maternal, Newborn, Child Health and Nutrition (MNCHN) services have collapsed leaving children under five and pregnant and lactating women (PLW) at risk. 
To address these needs, World Vision implements a twelve-month project to improve WASH, shelter, health, nutrition, and protection conditions for conflict affected population in Aleppo and Idleb governorates. The proposed activities are in line with the sector objectives of the 2016 Humanitarian Response Plan (HRP), as they restore sustainable water and sanitation systems, improve housing and community/public infrastructure, provide life-saving and sustaining humanitarian health assistance, and strengthen appropriate and integrated nutrition response.
- Summary overview of the Action implemented
|Duration||September 1, 2016- November 30, 2017|
|Primary sector of intervention||Protection, Health, Nutrition, WASH, Shelter,|
|Implementing partner||Syria Relief (SR) – Health, Nutrition, and ProtectionInsani Yardim Dernegi (IYD) – WASH|
World Vision – Shelter
|Project Locations||Aleppo and Idleb Governorates|
|Goal||Promote resilience of crisis affected Syrians in Idleb and Aleppo governorates through life-saving interventions in WASH, Shelter, Health, Nutrition and Protection.|
|Outcome||Crisis-affected communities, families including children have improved living conditions and access to basic services|
|Outputs||1. Men, women and children in the targeted areas have increased access to quality and life-saving health and nutrition support, PSS services, and gender based violence care.2. Improve family and community capacities through the identification and promotion of appropriate feeding practices to prevent infant malnutrition in the crisis affected areas|
3. Provide protection support, services and referral pathways to conflict affected and displaced women and children
4. Strengthen the capacity of displaced women, girls, boys and men and hosting families to cope with threats to personal safety and human dignity through improved shelter
5. Dignity of women, girls, boys and men is enhanced through the provision of clean drinking water, improved sanitation facilities and a hygienic living environment.
Table 1. Target Group and Number of Beneficiaries Targeted
|Health and Nutrition||Idleb||Harim||Dana||61200|
|Al Ma’ra||Kafr Nobol|
Table 2. Outcome Indicators and Activities implemented
Activities implemented to achieve the desired results include:
Water Supply Infrastructures
Rehabilitate water infrastructure within the Idleb Governorate
Conduct water testing and water management
Operation and maintenance support to water infrastructure
Rehabilitate of water services in schools
Training of local authorities in water quality and tariff system for the long-term sustainability of water facilities.
Rehabilitate sewage infrastructure
Rehabilitation of sanitation services in schools
Removal of garbage from the streets around the towns, clearing of the access road to the landfill
Supporting the establishment of a solid waste management system in the communities
Rehabilitate municipal vehicles to ensure that the municipality units have capacity to continue removal of waste
Training of local authorities in tariff system for the long-term sustainability of waste management
Conduct hygiene promotion events/activities in schools and communities
Distribute gender sensitive personal hygiene kits
Rehabilitation of damaged residences (for families who are living in one room, overcrowded living spaces), affected by the conflict
Provide heaters and coal for collective centers and households
Install solar electricity panels in collective centers
Rehabilitation of collective centers
Health Systems and Clinical Support
Rehabilitate and operate 2 maternal and pediatric hospitals and one primary health care clinic
Train nurses, doctors and midwives working at the health facilities on integrated management of childhood illness (IMCI), IYCF, CMAM, RH
Support reproductive health services in two hospitals and one health clinic
Train midwives working on Reproductive Health
Promote family planning and sexual reproductive health in the women’s centers located in each of the targeted health facilities
Community Health Education/Behavior Change
Community Health Workers provide ante-and post-natal referrals
Community-based screening and referral for malnutrition (IYCF)
Community Health Workers lead protection awareness sessions in communities and schools (CP sub-sector)
Medical Commodities, including Pharmaceuticals
Provide medical supplies, consumables and medicines to targeted health facilities
Train distributors of pharmaceuticals in proper use and disposal of commodities and pharmaceuticals
Community health workers lead child protection awareness session in communities and schools
Provide case management for children with counselling needs, who are identified by schools and project personnel
Train nurses and CHW on CP topics
Prevention and Response to Gender Based Violence
Train the project staff at the women’s center on the subject of Gender Based Violence prevention and response
Promote GBV Awareness in the women’s centers located in each of the proposed health facilities
Provide GBV Case Management and Counselling through the women’s centers within the supported medical facilities.
IYCF & BCC
Train and support community health workers to conduct community nutrition awareness and malnutrition screening
Train and support caregivers and mothers of infants and young children (0-23 months) on infant/child care and feeding practices
Provide IYCF support and promote breastfeeding through home visits during the postnatal period.
Community-based screening and referral for malnutrition (IYCF)
- Evaluation purpose, questions and intended usage
The evaluation will be based on the four specific evaluation criteria used by World Vision’s Humanitarian and Emergency Affairs (HEA) which is also based on OECD-DAC criteria: relevance, timeliness, efficiency and effectiveness/impact. The four criteria, as they relate to achieving the outcomes and goal of the project, will also serve as the main framework for presenting the evaluation findings. The proposed evaluation criteria are:
- Relevance– How well did the project correspond to local needs and priorities? How did WV target the most vulnerable populations, and what were the challenges of reaching vulnerable, affected populations? Did the project address priority problems faced by the target areas and communities and was the project consistent with policies of both donors and recipient governments (or agencies)?
- Timeliness– To what degree did the project implement timely Response activities? Did the project respond in a timely manner to the affected community’s needs? What affected timely delivery of assistance (including coordination issues), and what measures were taken by WV to address the issues?
- Efficiency– To what degree did the project provide efficient support project implementation? Were inputs (staff, time, money, equipment, relief goods) procured and/or used in the best possible way to achieve outputs and could implementation have improved? Which main factors influenced the efficiency of implementation such as supply chains, and how did this affect the delivery of aid?
- Effectiveness/Impact– To what extent did the project activities meet their intended purpose? Were outcomes achieved? What is the evidence of both the impacts of the Response as well as the unintended positive and negatives effects on the affected population since the conflict? What real difference has the activity made to the beneficiaries? What has happened as the result of the project?
Developing the analytical framework, i.e. refining and operationalizing the questions posted above, will be done in close cooperation between World Vision and the consultant.
The evaluation will be used to:
- Support accountability of the implementing partners vis-à-vis the donor on the outcomes of the Action and the usage of the funds provided through the provision of the evaluation report.
- Support learning between the implementing partners on the management of their cooperation through the process of the evaluation and the discussion of the evaluation results.
- Evaluation target audiences
The targeted audiences for this evaluation are:
- World Vision
Primary data collection
Primary data collection is expected to comply with recognized ethical principles. Data are expected to be collected on smartphones or tablets, using ODK, Kobo, SMAP or similar applications, and processed electronically. The consultant will provide the cleaned primary data set in Excel to World Vision and grants World Vision rights of storage and further usage. The consultant is expected to:
- Provide a time schedule, a breakdown by location and a breakdown of resources needed for primary data collection in the locations covered under this project;
- Hire, train and supervise enumerators, which must include both men and women, and care for their provisions and security;
- Obtain the necessary permissions for gathering primary data from the relevant authorities.
HH survey sampling: For this study, two-phased proportional stratified random sampling will be used, with the primary sampling unit as the household and sample frame of total population of the targeted communities per project/sector. The first phase, involves randomly selecting communities within the targeted communities within the subdistrict per sector. And the second phase, will involve randomly selecting respondents among these selected communities.
Sample size: The below sample size equation is hereby suggested to calculate the required sample size for each strata.
n = [ ] where:
Z = The z-score corresponding to the degree of confidence with which it is desired to be able to conclude that an observed change of size for p and q would not have occurred by chance (Z = 1.96)
p = Estimated prevalence of indicator in the population33 (p = 50%)
q = 1-p, population not presenting with indicator
D= Design effect for a complex, clustered, sample (D = 2.0)
d = Precision/margin of error, as fraction of 1 (d = 5%)
The estimated level of an indicator (to be determined at endline) to be measured as a proportion is 0.5 (P1 =0.5), which is the most conservative assumption. A confidence level of 95% (Z = 1.96) and a design effect of 2.0 is used, with a margin of error of 5% incorporated. Using the sample size formula above, these parameter values result in a sample size of 382 respondents per location plus with 10% oversample, would yield to 420 respondents. Using the three strata listed below, the baseline activity required a sample size of 1260 respondents.
FGD sampling: Focus group discussions will be used to gather qualitative information access to health and nutrition support, PSS services, access to WASH services, protection services and referral pathways and access to shelter . XX communities will be purposively selected from the sampling frame to cover a range of geographic locations, including rural and urban, as well as host communities and IDPs. All participants will be randomly selected by doing a village walk. The anticipated minimum number of FGDs is 4 per location (men, women, girls and boys). For Protection in Schools, as the activity deals with children, FGDs with boys and girls (6-12 years old) will be conducted per location.
KII sampling: Key informant interviews shall be undertaken with Water Authority and Local Council for WASH, Local council for shelter, Health Staff and Local Council for Health, Nutrition and Protection and school authorities for activities relating to children.
Mixed methods approach: The report should triangulate data from primary and secondary sources. The primary data are from quantitative and qualitative sources (listed above) that provide information from various beneficiary perspectives; the secondary data will be drawn from a review of key project documents.
|Sector||Gov||District||Subdistrict||Target Communities||Target||Sample size for HH survey||FGDs||KIIs|
|WASH||Idleb||Ariha||Ariha||Maarzaf, Majdaliya, Mseibin, Maarbalit, Moataf, Kadoura, Mar Tebi||46, 379||420||4-8||6|
|Idleb||Ariha||Mhambal||Mhambal, Alyeh, Mhambal- Alqart, Baqlid||4-8||6|
|Health, Nutrition & Protection||Idleb||Harim||Dana||Sarmada||61,200||420||3||2|
|Protection in Schools||Aleppo||Jebel Saman||Atareb||Kafr Karmin, Halazon, Batbu||7,776||–||2||2|
|Idleb||Idleb||Maaret Tamsrin||Zardana Mashehad||2||2|
|Idleb||Al Ma’ra||Kafr Nobol||Hazarin||2||2|
The proposed methodological approach and the tools will be developed in close cooperation between World Vision and the consultant and include field testing with a sample set of beneficiaries.
Based on the agreed upon analytical framework, WVI, SY and IYD will provide the consultant with access to project documents relevant for the evaluation. The consultant is expected to research and include additional secondary sources in the analysis as needed. World Vision can support with relevant additional secondary sources as available.
The timeline is expected to be as follows:
- Oct 9-20, 2017: Finalization of analytical framework, methodological approach, sampling aspects, primary data collection tools
- Oct 23- Nov 3, 2017 : Primary data collection (including obtaining government permissions and training enumerators) and data cleaning
- Nov 6- 17, 2017: Report writing
- Nov 20-30, 2017: Report review, feedback and finalization
- Expected outputs and deliverables
The evaluation should result in the following outputs and deliverables:
- Inception meetings to finalize scope of work, expectations, process / work plan, and brief on logistics and technical aspects
- Final analytical framework, methodological approach including required secondary sources from the project, approach to sampling, primary data collection tools
- Training of enumerators, government permissions to implement primary data collection
- Raw data (all data sets in excel and SPSS, notes from KIIs and FGDs)
- Draft & final evaluation report and summary fact sheet. The consultant will produce a draft report to be submitted to WVI on the agreed date for review. A final report (MS Word, Excel files, PDF), including feedback from WVI reflected in, will then be submitted to WVI. The report needs to follow the provided template / guidance.
All deliverables (except for the evaluation report) need to be provided in English and Arabic as where appropriate. The evaluation report needs to be provided only in English.
- Profile of the evaluation team
The consultant (team) must have proven expertise and experience in social research, protection in humanitarian emergency responses, and be able to implement the research in Syria following the required procedures and in the required languages.
Proof of these is to be provided by submitting, together with the application:
- An overview of relevant works
- Working samples
- Contact details for references
- The proposed research team´s CVs.
Requirements in detail:
Have excellent team leading skills, with the ability to work with a varied team.
- Have expertise and experience in social science research, including mixed methods and notably quantitative and qualitative primary data collection methods
- Have expertise and experience in research, monitoring and evaluation in humanitarian emergency response contexts, preferably related to the Syria crisis, as well as related technical standards
- Have a good understanding of the Syria crisis
- Have permission to work, travel and undertake primary data collection in the project target locations and be able to do so in the language of the primary target groups (Arabic)
- Have proven experience in conducting participatory qualitative and quantitative evaluation studies with superior analytical skills
- Have a good knowledge of data collection software (such as Kobo, SMAP) and statistical packages (such as SPSS)
- Have excellent report writing skills
- Have excellent written and spoken English and Arabic
- Hold a minimum educational qualification equivalent to a master´s degree in a relevant field.
The consultant should submit a proposal comprising the following:
- A technical proposal which indicates how the consultant is going to undertake the activities highlighted in the ToR, indicative budget with initial indication of the enumerators and timeframe required
- A capacity statement detailing the consultant´s ability to deliver a quality evaluation report within the given timeframe, including an overview of relevant work and technical experience
- At least 2 samples of previous relevant works undertaken, including at least 1 evaluation report that was 100% led by the lead consultant
- Contact details for 2 references for similar assignments done not more than 2 years ago
- CVs of the key personnel on the evaluation team
- A financial budget with explanations about the line items
- Any appendices the consultant sees as relevant to the application.
Applications should be submitted electronically to email@example.com by 29thSeptember 2017.
- Child protection & data confidentiality
The external evaluator, along with all enumerators, will be required to sign and follow World Vision International´s child protection standards and protocols of behavior, which will be provided to the research team selected.
All primary data collected by this evaluation process is to remain confidential and is not to be shared with third parties.
 The degree to which the program addressed needs related to the recovery of typhoon-affected communities
 The degree to which to program produced its intended consequences
 The degree to which the program responded to critical needs in time, and consequences of delays
 The degree to which the leadership, support services and program systems functioned to enable successful programming
 Impact of Syria Crisis Report, Syrian Centre for Policy Research (SCPR), Feb 2016.
 Humanitarian Needs Overview (HNO) 2016, OCHA, October 2015
 OCHA Humanitarian Snapshot Jan 2016
 Syrian Center for Policy Research (SCPR) 2016
 Humanotarian Response Plan (HRP) 2016
 This only refers to Health Target. Nutrition targets 108000 individuals, however, there is overlap between these sectors.