Final Independent Evaluation of the Project “Supporting the Government of the Republic of Kazakhstan in the Identification and Application of Best International Practices in Migration Policy-Making through the GCM and the Almaty Process”

Introduction

Established in 1951, IOM is a Related Organization of the United Nations and the leading UN agency in the field of migration. Working closely with governmental, intergovernmental and non-governmental partners, IOM promotes humane and orderly migration for the benefit of all. It saves lives and protects people on the move, drives solutions to displacement, and facilitates pathways for regular migration, while providing services and advice to governments and migrants.

IOM is committed to fostering a respectful, inclusive and supportive workplace where all employees can thrive professionally and feel valued. By creating such an environment, IOM aims to better harness the full potential of migration and strengthen its support to people on the move.

The project "Supporting the Government of the Republic of Kazakhstan in the Identification and Application of Best International Practices in Migration Policy-Making through the GCM and the Almaty Process" supports the Government of Kazakhstan in strengthening regional cooperation by maintaining the regular functions of the Almaty Process (AP) and advancing coordinated GCM implementation to enhance migration policy development, through:

• strengthening Kazakhstan's intergovernmental dialogue efforts to support the achievement of GCM objectives and the commitments of the International Migration Review Forum (IMRF) Progress Declaration in Kazakhstan; and
• continuous operations of the Inter-State Consultation Mechanism (ISCM) AP. Established in 2013, the AP is a regional ISCM focused on refugee protection and international migration to, from, and within Central Asia. It was designed to address the complex dynamics of mixed migration and to foster regional cooperation and coordination. The AP promotes sustained dialogue and information exchange on migration and refugee protection among its member and observer states, operating through Technical Expert Group meetings, Senior Officials Meetings, Ministerial Conferences, and regional thematic discussions. The unique nature of the AP lays in geographical proximity of its Member States (Azerbaijan, Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, Türkiye) and visa free regimes between most of the Member States, similarity of history, cultures, traditions, and language, as well as willingness to build continuous dialogue and exchange of experience. The strategic directions and aims of the AP are closely aligned with the GCM and in particular its Objective 23 - Strengthen international cooperation and global partnerships for safe, orderly and regular migration. The project sought to build on these strengths by leveraging bilateral and multilateral consultations among AP Member States, with the support of relevant field missions, and by advancing GCM implementation initiatives.

Evaluation purpose and objective

The purpose of this final evaluation is to identify what changes occurred and why they occurred, including whether and to what extent changes were achieved and the contribution made by the project to those changes (impact). The evaluation also aims at determining performance (efficiency and effectiveness) - how, why and how well the project was implemented in terms of its objectives, outcomes and outputs, as well as determining how meaningful the results of the interventions implemented within the project are, what are the strengths and weaknesses here, why some efforts worked better than others. In addition, the evaluation aims to assess relevance and coherence of the project, in terms of whether the intervention and its approach were the most appropriate to address the problem and considering, as well as the further impact and sustainability prospects.

The evaluation should identify best practices and aspects that could be improved in similar projects which will be implemented by IOM Kazakhstan, as well as lessons learned to be taken into consideration.

This evaluation is being conducted in accordance with IOM’s Evaluation Policy and Guidelines which sets out a number of guiding principles and key norms for evaluation in the organization following the Norms and Standards of the United Nations Evaluation Group (UNEG).

The final evaluation aims to comprehensively assess and demonstrate progress achieved towards the expected results against those stated in the Project Documents and identify the lessons learned and recommendations relevant to the planning, preparation and implementation of activities under the subsequent phase. The purpose of the external final evaluation is to provide a summative assessment encompassing all six core OECD-DAC criteria: relevance, coherence, effectiveness, efficiency, sustainability, and likely impact (including early signs of results, whether intended or unintended, positive or negative) as well as cross-cutting issues. The intended evaluation users are IOM project staff and project partners. The evaluation should provide recommendations for implementation of future activities and identify other complementary or additional priority areas for IOM interventions. Moreover, the findings of the evaluation will be shared with the donor, MFA of Kazakhstan, to assess value for money for the funded activities.

Evaluation scope

The evaluation will cover the entire project implementation period of 02 January 2023 – 1 June 2026 and include during the data collection phase as minimum interviews with IOM project staff, project partners (Government of Kazakhstan; Government Members of the Almaty consultative process; UNMN Secretariat; IOM Regional Office in Vienna; CSOs including NGOS), in Kazakhstan as well as Focus Group Discussion (FGDs) and other methods as required; this will be spelled out further during inception stage and reflected in the inception report to be delivered by the evaluator.

Evaluation criteria

The following evaluation questions will guide the evaluation. The questions may be revised during the inception period in agreement between the evaluator and the evaluation manager, and the evaluator will propose relevant sub-questions as needed as well during the inception phase.

Relevance

• Is the project aligned with and supportive of national strategies?
• Does the project respond to the needs of the target groups and stakeholders, both as originally planned and in the current context?
• To what extent is the project’s design (and theory of change) relevant to its intended results?

Coherence

• To what extent is the project consistent with other actors’ interventions in the same context?
• To what extent does the intervention add value/avoid duplication in the given context?

Efficiency

• Were the project activities undertaken and were the outputs delivered on time?
• How well are the resources (funds, expertise, time) being converted into results?

Effectiveness

• Have the outputs been achieved in accordance with the stated plans? To what extent has the project contributed towards progress to stated outcomes?
• To what extent has the project adapted or is able to adapt to changing external conditions in order to ensure delivery of project results?

Impact

• What significant changes can be observed, whether positive or negative, intended or unintended?
• To what extent has the project likely contributed to the changes, considering also other factors?
• Did the intervention take timely measures for mitigating any unplanned negative impacts?

Sustainability

• To what extent have target groups, and possibly other relevant interest groups and stakeholders, been involved in discussions about sustainability?
• Do the project partners have adequate capacities (technical, financial, managerial) for ensuring that the benefits are retained in the long run, and are they committed to do so?

Evaluation questions and methodology

The evaluation should be transparent, inclusive, participatory and utilization focused. The overall methodology should be implemented following a theory of change approach, framed by the UN/OECD DAC evaluation criteria drawing upon mixed methods (quantitative and qualitative) data to capture direct project results as well as (likely) contributions.

The Evaluator will ensure triangulation of the various data source. Data and evidence will be triangulated with multiple sources to address evaluation questions. The final methodological approach including interview schedule and data to be used in the evaluation should be clearly outlined in the inception report and fully discussed and agreed between IOM, stakeholders and the Evaluation Team.

The proposed format of the project evaluation and the number of representatives of government agencies, civil society organizations and NGOs involved in its implementation do not allow the results obtained to be presented in a quantitative form that would be representative. Quantitative analysis is applied to selected indicators specifically highlighted within the project.

Cross-cutting themes

Stakeholder participation

All major stakeholder groups will be interviewed during the evaluation. The sample of stakeholders to be consulted will take into account gender, country, and place of employment.

In-depth online interviews will be conducted with IOM mission staff and representatives of government agencies.

Monitoring data and related documents will be provided by IOM. At output level, IOM records the number and profile of all beneficiaries trained and supported with technical assistance along with pre/post tests, disaggregated by gender, as well as the number of produced and disseminated materials within workshops. At outcome level, IOM assesses the progress in achieving the intended results using information from partners at the central and local levels. Specifically, IOM uses pre and post-event perception based assessments, legislative review and analysis, interviews with stakeholders, and minutes of meetings of the national consultation networks.

Ethics, norms and standards for evaluation

[1] Evaluation Criteria// OECD Official Site// https://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm[2] See guidance available within the international development evaluation community on selecting appropriate evaluation methods to answer different types of evaluation questions, such as https://www.betterevaluation.org/en/approaches or https://www.bond.org.uk/resources/evaluation-methods-tool [3] Norms and Standards for Evaluation (2016)// United Nations Evaluation Grope// http://www.unevaluation.org/document/detail/1914[4] IOM Monitoring and Evaluation Guidelines. – Geneva: IOM, 2020. – P. 24-26.

In close coordination with IOM Kazakhstan, evaluator will develop and Inception report with an evaluation matrix and a draft interview guide, along with any other needed data collection tools, to describe understanding of the TOR, including any revisions to the methodology (at minimum it should include the evaluation matrix and tools; evaluator may choose to develop other aspects of inception report per the IOM template as needed). This should be submitted to IOM, for comments and discussion, to finalize plans prior to data collection activities.

Based on findings received and discussions with the project team during the presentation of preliminary findings, a draft evaluation report should be shared for the IOM Project Manager’s feedback.

The Evaluator will develop final evaluation report that includes an executive summary, list of acronyms, introduction, evaluation context and purpose, evaluation framework and methodology, findings, conclusions and recommendations. The length of the final evaluation report should be approximately 20 pages including annexes.

The evaluator should also produce a two-page evaluation brief following the IOM template, including: Identification of audience at start of the brief; Project information (project title, countries covered, project type and code, project duration, project period, donor(s), and budget); Evaluation background (purpose, team, timeframe, type of evaluation, and methodology); Brief description of the project; the evaluation results: Key findings and/or conclusions, best practices and lessons learned , and key recommendations.

The final report should meet the standards laid out in the UNEG Quality Checklist for Evaluation Reports.

Evaluator will prepare a draft management response matrix by inserting the recommendations as well as an indicative timeframe or deadline for implementation, using the IOM template for management response. The management response matrix will then be completed by the Evaluation Manager.

The evaluator will be responsible for:

• Preparing for and carrying out data collection and analysis and delivering the products outlined above and producing the deliverables.
• Providing periodic feedback as needed to the Evaluation Manager on progress and any challenges.
• Providing a debrief at the end of the data collection (in country or a few days after the field visit) to present on initial findings and tentative conclusions. This will allow for any obvious oversights, misinterpretations, or information gaps to be identified and addressed before drafting the report.
• Drafting and revising the report and evaluation brief, in coordination with Evaluation Manager.

The Evaluation Manager will be responsible for:

• Providing evaluators with timely access to all relevant project documents, and facilitating the initial preparations including identifying relevant stakeholders and organizing the agenda;
• Arranging meetings with the stakeholders for data collection and lessons learnt debriefing;
• Organizing logistics during the visit, including transportation, and interpretation services.
• Managing the evaluation process including feedback and comments to the evaluation matrix and draft evaluation report, including coordination with RO Vienna Regional M&E Officer.
• Complete the management response to address issues or challenges flagged by the evaluator.

Specifications of roles

The External Evaluator (consultant) will be responsible for the independent design and implementation of the evaluation, including preparing the inception report with methodology and tools, conducting a comprehensive desk review, and carrying out mixed-method data collection (interviews, focus groups, surveys) with full triangulation of evidence. The evaluator will lead and coordinate all interviews and meetings (with IOM support), ensure compliance with UNEG norms and IOM evaluation standards, and provide regular progress updates to the Evaluation Manager. The consultant will present preliminary findings through a debriefing session, prepare the draft evaluation report, revise it based on feedback, and submit the final report and evaluation brief. Additionally, the evaluator will develop a draft management response matrix and ensure methodological rigor, impartiality, and ethical compliance throughout the process.

IOM Kazakhstan (Evaluation Manager and Project Team) will oversee and facilitate the evaluation process, acting as the main coordination and quality assurance body. IOM will provide timely access to all relevant documents, data, and stakeholders, and support the evaluator in organizing interviews and meetings, including logistical arrangements such as scheduling, transport, and interpretation. The Evaluation Manager will review and provide technical feedback on all deliverables (inception report, draft and final reports), coordinate inputs with the IOM Regional Office in Vienna, and ensure alignment with IOM evaluation policies and UNEG standards. IOM will also consolidate comments, approve key deliverables at each stage, and complete the management response matrix to follow up on the evaluation’s recommendations.

Time schedule

A total of 30 working days are foreseen for this evaluation assignment. The following table outlines the number of days per task and the timeline for each. The workplan will be reviewed and potentially adjusted with the selected evaluator during the inception phase.

ActivityResponsible partyNumber of daysDeadline Project documents review and preparation of a detailed inception report, including the evaluation matrix and the data collection toolsEvaluator5 days Planning and facilitation of field visits and/or virtual/e-communication tools and agenda of online meetings, focus groups, phone calls, logistical arrangementsIOM Kazakhstan14 days Field visits for face-to-face meetings (including travel time) or Data collection through virtual/e-communication tools, online meetings, focus groups, phone callsEvaluator6 days De-brief to project team on the initial findings and tentative conclusionsEvaluator1 day Draft evaluation report following received findings and observationsEvaluator5 days Review and comment on the draft reportIOM Kazakhstan, RO Vienna14 days Finalizing report, presentation and Evaluation BriefEvaluator1 days

Required Qualifications and Experience

• The consultant is expected to meet the following requirements:
• Advanced university degree (Bachelor’s or higher) in Sociology, Anthropology, Economics, Development Studies, Migration Studies, or a related field from a recognized institution;
• Demonstrated knowledge of national and international frameworks related to migration governance, including labour migration, human rights, counter-trafficking, and relevant legal and policy frameworks;
• At least 5 years of professional experience in migration-related fields, including proven experience in conducting evaluations, research or analytical studies on labour migration or related topics; prior experience in developing standard operating procedures (SOPs) is an asset;
• Strong experience in engaging with migrants, government institutions, and other relevant stakeholders in the migration sector;
• Demonstrated knowledge of international standards on migration data, statistics, and monitoring and evaluation practices, with the ability to apply these in different contexts;
• Excellent command of English, Russian, and Kazakh;
• Strong analytical, communication, and report-writing skills, with the ability to present findings in a clear and concise manner;
• Proven organizational skills, attention to detail, and ability to work independently and effectively within multicultural and multidisciplinary environments.

Required Competencies

Notes

IOM covers Consultants against occupational accidents and illnesses under the Compensation Plan (CP), free of charge, for the duration of the consultancy. IOM does not provide evacuation or medical insurance for reasons related to non-occupational accidents and illnesses. Consultants are responsible for their own medical insurance for non-occupational accident or illness and will be required to provide written proof of such coverage before commencing work.

Any offer made to the candidate in relation to this vacancy notice is subject to funding confirmation.

Appointment will be subject to certification that the candidate is medically fit for appointment, accreditation, any residency or visa requirements, security clearances.

IOM has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and IOM, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination based on gender, nationality, age, race, sexual orientation, religious or ethnic background or disabilities.

IOM does not charge a fee at any stage of its recruitment process (application, interview, processing, training or other fee). IOM does not request any information related to bank accounts.

No late applications will be accepted. Only shortlisted candidates will be contacted.