BHA End of Project Consultant At Relief International

Position: BHA end of Project Evaluation Consultant**
Location: Yemen
Duration: 2 months
Reports to: BHA Program Manager
Start Date: April 1, 2021
About RI:
Relief International (RI) is a leading nonprofit organization working in 16 countries to relieve poverty, ensure well-being and advance dignity. We specialize in fragile settings, responding to natural disasters, humanitarian crises and chronic poverty.
Relief International combines humanitarian and development approaches to provide immediate services while laying the groundwork for long-term impact. Our signature approach — which we call the RI Way—emphasizes local participation, an integration of services, strategic partnerships and a focus on civic skills. In this way, we empower communities to find, design and implement the solutions that work best for them.
RI includes the four corporate members of the RI Alliance: RI-US, RI-UK, MRCA/RI-France and RI-Europe. Under our alliance agreement, we operate as a single, shared management structure.
RI is implementing 19-month action to address the Nutrition, Health, and WASH needs for the affected population in Al Mahweet and Al Tawila district in Al Mahweet governorate and Ain, Bayhan and Ataq districts in Shabwah governorate.
The project has three sectors i.e. Health and nutrition, WASH and food security. Under health and nutrition, RI is supporting primary health care including mental health, management of acute malnutrition through CMAM approach, infant and young child feeding activities and referral to secondary healthcare. In food security, RI is providing food assistance through cash distribution for IDPs and households affected with severe acute malnutrition.
Ten health facilities and their catchment population in five districts in Shabwah and Al Mahweet were targeted in the intervention with primary healthcare activities implemented in community management of acute malnutrition activities including food assistance through cash distribution while six health facilities were supported for primary healthcare activities in Shabwah.
The end of project evaluation report will be shared with BHA and relevant stakeholders to give a clear vision of the project achievements, immediate impact, lessons learnt about the implemented modality, and challenges. The main objectives of the evaluation are:
To measure the immediate and long term impact of the project and solicit lessons to inform further action;
To assess the achievement made in the program implementation and the progress made in achieving the overall goal of the project.
The evaluation will identify the best practices, challenges, recommendations and lessons learned related to projects in the targeted areas. The information obtained will enable RI to use the knowledge attained for future proposals.
The evaluation will be conducted in project location shown in Shabwah and Al Mahweet governorates, considering the project’s indicators as indicated in the proposal.
Specifically, RI wishes to confirm the following general points:
Patients were receiving consultation, services and medication at target HFs
Activities, in the opinion of direct beneficiaries and local stakeholders, were responding to the needs of the beneficiaries;
Evaluate the modality of Health and Nutrition support – its relevance, effectiveness, impact on access to health and nutrition services, impact on the communities in the longer term;
Evaluate the functioning and effectiveness of the complaints response mechanism (CRM) implemented at the HFs in terms of accessibility and usage by the beneficiaries.
The beneficiaies targetted for of cash for nutrition and RRM on the supported families received the assistance and was beneficial in improving household food insecurtiy.
The WASH facilities rehabilitated in the supported health facilities are functional and in use by the healthcare workers and the patients.
In relation to the above objectives, RI requests the External Firm to apply the five OECD/DAC evaluation criteria of Relevance, Effectiveness, Efficiency, Impact and Sustainability of the project.[1] The intervention logic (i.e. logframe) must also be properly taken into consideration. The External Firm needs to make sure that both qualitative and quantitative methods are used to gather data in order to investigate the components and answer the questions in the OECD criterion mentioned in Table 1.
Table 1: Assessment criteria for the External Evaluation
Evaluation Criteria
Questions to be considered
Were the provided services and program objectives within the prioritized needs of the community and aligned with the targeted response of the clusters (Health/Nutrition/WASH/FSAC_)?
Were the outputs delivered within the agreed budget and timeline?
To what extent have beneficiaries been satisfied with the program?
What real difference has the program made to the target groups (and beyond)?
What is the likelihood that the benefits from the program will be maintained for a reasonable period of time if the program were to cease?
The methodology for this evaluation should be based on using mixed-methods research. The sample questions and topics to be included in the data collection provided below, as well as the components of the tools are tentative and could be further developed by the external Firm in consultation with RI. However, the external Firm should have the capacity to conduct the number of surveys in compliance with the minimum required sample size both at the HFs targeted areas, HFs and at the community level. The external Firm recommended to use the following qualitative and quantitative methodologies to collect relevant data:
Literature Review and Secondary Data Analysis: to get facts and overview of the project context; this includes an analysis of past project reports, monitoring reports, knowledge, attitudes and practices (KAP) survey results, and program reports from RI.
Primary Data Collection: will include field visits, observations, surveys, participatory and focus group discussion with all beneficiary groups, not limited to patients but including, community representatives, and staff worked with HFs, governorate and districts health directorates, and local councils where possible and applicable.
Quantitative data collection will include access and coverage surveys to be conducted at the household and health facility level (having as a specific group of beneficiaries also patients and direct beneficiaries of the project); a questionnaire will be designed in consultation with RI and the external Firm will have access complaints and feedback mechanism.
Qualitative data collection will include observations, focus group discussions (FGDs) and key informant interviews (KIIs) (see below) and narrative observation reports whereas necessary
a. Quantitative Methods
Household Survey including Access and Coverage sections as well as the direct beneficiary satisfaction survey:
This survey will be developed between the external Firm and RI in order to be conducted at the household level to measure the level of accessibility of the beneficiaries residing in the catchment/target areas which the health facilities are serving.
Specifically, this survey will aim at:
Reaching out to the target population households and learning about their trends in seeking healthcare services when in need;
Measuring the level of 1) awareness and 2) physical, social and financial accessibility to the health facilities and offered services;
Identifying the most important barriers to accessing health facilities and services;
Identify the barriers to services by CHVs or through other modalities;
Understanding whether they are receiving referrals services and the reasons for not receiving the referral services, if any.
Identifying the significance of the food assistance to household dietary diversity.
Measuring their knowledge on good hygiene practices
This survey should also aim at assessing the knowledge, attitudes and practices of the communities located in and around the HFs targeted areas on Health and Nutrition-related topics. Specifically, the households should be assessed to see if they have been visited by a CHV, if they are aware of some of the Health and Nutrition topics and to measure their knowledge, and see if they have adapted any positive health and hygiene practices as a result of the awareness sessions.
The survey will include the section of questions directed to beneficiaries who confirm that they were targeted by services provided by the teams supported within this project. This patient/beneficiary part of the survey will cover both Health and WASH components in all targeted HFs, the targets for the survey will be patients who have visited the HFs. It will aim to assess the following:
The general health service provision they received, specifically focusing on their level of satisfaction with the services offered and the attitude/behavior of staff
Did the consultation meet the needs of the patient? (Chief complaint was addressed? Access to prescribed medication? Laboratory tests or services were available? Timescale for improvement?)
Was there any payment for services and medication?
Do beneficiaries know about the complaints and feedback mechanism? (What channels can they use to make a complaint? In which circumstances should they make a complaint or suggestion?)
How long did they have to wait for a consultation? (Less than thirty minutes? Thirty minutes to one hour? Over one hour?)
Was patient privacy maintained throughout the consultation?
If needed, will they come back to this facility for another consultation? Would they recommend this facility to others in need?
Has the patient or household ever received any informational session from the community health workers?
How they were first informed about the existence of this service?
Did they get referred to another health facility to access certain services? How far is the health facility that they were referred to? Did they/will they go to the health facility that they were referred to? If not, what were the reasons?
What was the nutrition component of the project implemented to their child? Were all the children targeted by screening services and those in need admitted to proper treatment?
What was the subjective opinion of the impact on the beneficiaries?
b. Qualitative Methods
Key Informant Interviews
Health and Nutrition Component: For this evaluation, for each HF, ideally four personnel should be selected for interviews: physician, nurse, midwife; and an administrative staff member. Staff members to interview could be chosen at random by position from the HF staff roster provided by RI. Another interview should be conducted with one of the following positions as well: project coordinators, health facility management board members, medical focal points, Field Assistants, CHWs HFs team leader.
WASH Component: The evaluation should include KIIs with two selected key persons including one RI staff member (WASH engineer), one community hygiene promotion volunteer and health facility in charge for the supported HFs.
The objective of these KIIs should be to analyze the quality of RI’s implementation at the health facilities and community. In particular, the surveys will provide an overview of whether standard operating procedures set by RI in relation to WASH Health and food assistance are being followed, and whether the intervention meets specific quality standards such as those provided by SPHERE or WHO. Additional sample questions to guide the external evaluation in developing interview tools should consider;
Do the staff feel as though they have sufficient resources to perform their jobs?
Is the HFs effectively equipped to address the immediate needs of the community?
What are the challenges related to the referral system? What is working and what is not?
Focus Group Discussions (FGDs)
Health and Nutrition Component: FGDs will be conducted with males and female patients separately in each of the targeted communities (At least one FGD in each mobile team site). Each FGD will cover at least 8-10 participants. The FGD will mostly aim to capture qualitative data with some semi-quantitative data (i.e. approximates of the total community who have access to services; who use services; who have awareness of specific issues relating to health programming at the facilities; the involvement of women and children; challenges preventing part [or all] of the community from benefiting from services; why the community is not following referrals etc.).
WASH Component: The consultant should conduct two FGDs at each project site, one with male and one with female beneficiaries. The FGDs should be conducted at household level in the communities where there are beneficiaries targeted for hygiene promotion.
RI Responsibilities:
RI will be equally responsible for any support relevant to review and developed of the data collection tools.
Background materials (project proposal, meeting notes, reports, and Meetings)
Quantitative and qualitative documentation
Interviewees (and their contact information including RI field staff) so that there will be no problems or challenges during the evaluation team’s visits.
Collaborate with the consultancy firm in reviewing and approving the CV of the selected expert consultant.
Evaluation firm Responsibilities:
The firm will be responsible for coordination with the local authorities for permissions to conduct the evaluation.
The consultancy firm will be responsible for recruitment of an expert consultant with experience evaluating integrated projects preferably funded by US-BHA
The firm will be responsible for logistics, administrative support, telecommunications, printing of documentation, and logistics for fieldwork, etc. The Firm is also responsible for the dissemination of all methodological tools such as surveys;
Mobilize and deploy the survey teams and enumerators in the field to ensure quality of data collected according to the samples.
The firm should budget for all necessary expenses in their Financial Proposal.
The Firm will be leading and managing the data collection and their field team on their own and RI MEAL staff should ensure that the selected consultant/firm is conducting the evaluation accurately as they report in terms of quality controlled.
Data collection and analysis will be the sole responsibility of the selected consultant/firm however, RI will be equally responsible in reviewing the draft report and will be included in the final reporting stage, especially in case some of the findings could be explained further and reported accordingly.
All external consultants/firms involved will commit to strictly complying with the confidentiality of information obtained during the evaluation process (that includes but is not limited to: program documents including datasets, reports and annexes and technical proposal), given the sensitivity of the program/context.
Three Major Deliverables that will be requested are:
An inception report which contains the intervention logic of the program (based on desk study), an evaluation plan, and a list of reviewed documents. The evaluation plan should contain the proposed data collection methods and data sources to be used for addressing each evaluation question/dimension, taking the above methodology section into consideration.
A draft evaluation report: a stakeholder workshop should be facilitated between the evaluation team and RI to discuss preliminary evaluation findings and conclusions at this stage. The timeline of the evaluation should allow sufficient time for stakeholders to review and discuss the draft report. Holistic and final sector-specific analysis and feedback will be provided to enable the technical units to analyze the findings before completion of analysis and validation.
The consultant will share the final draft report with RI to be revised into the final report. (The final evaluation report, and a final findings presentation in RI office. (The final evaluation report should strictly be written in English language and should not exceed 30 pages (excluding annexes). It should be submitted electronically in an MS-Word document. It may include:
Cover page
Executive Summary of key findings and recommendations;
Introduction, including brief context description
Evaluation findings, analysis and conclusions with associated evidence and data clearly illustrated. The findings section should be sub-divided as sub-chapters according to the evaluation criteria.
Recommendations for the future, which should be practical and linked directly to conclusions; and
Appendices, including evaluation tools, questionnaire, and brief biography of Firm.
RI aims to adhere to the following tentative timeframe for some components feeding in to the deliverables per each evaluation to take place:
Deliverables/ Components
Deliverable #1: Inception report for project
In a week after Signing the contract
Deliverable #2: Primary data collection: interviews with stakeholders (key informants, RI staff members), interviews with beneficiaries and vendors (including FGDs); household and patient surveys and observation checklists
Raw data for all of the conducted activities including paper consent forms and any other voice or video recordings.
During the third week of delivered the Inception report
Deliverable #3: Draft report to RI (including raw data)
Three weeks after data collection
Deliverable #4: Final report & presentation
Two weeks after the responses on the draft report.
The consultant is also expected to present the final report to RI’s Country Director, M&E Manager and the program team as per availability. In addition, raw data from field interviews, surveys, or focus groups should be provided to RI. The reports will highlight good practices, areas of improvement, and recommendations. Draft versions of the reports will be submitted to RI for review before final versions are submitted.
Annex 1: Project Locations of HF
Uphold and promote RI’s commitment to ensuring the safeguarding and safety of the vulnerable communities we serve.
Consistent with RI’s safeguarding and protection policies, ensure all people who come into contact with Relief International are as safe as possible.
Relief International’s Values:
We uphold the Humanitarian Principles: humanity, neutrality, impartiality and operational independence. We affirmatively engage the most vulnerable communities.
We value:
Transparency and accountability
Agility and innovation
How to apply
To apply for this post, click on…
You will be asked to upload a CV and Cover Letter. The cover letter should be no more than 2 pages long and explain why you are interested in this post with Relief International and how your skills and experience make you a good fit.
Closing date. Please apply immediately we will be reviewing applicants on a rolling basis, therefore may withdraw the position for the job board closing date.
Due to limited resources, only short-listed candidates will be contacted.
Note to external agencies, we will not be accepting CVs from third parties.
Relief International is committed to protecting our staff and the communities we work with from abuse and harm including sexual exploitation, sexual abuse and sexual harassment.
All staff are expected to abide by our Code of Conduct.
Recruitment to all roles in Relief International include a criminal records self-declaration, references and other pre-employment checks, which may include police and qualifications checks.

Search Jobs By Country

List of Countries

ArabicChinese (Simplified)DutchEnglishFrenchGermanItalianPortugueseRussianSpanish