Safe Agriculture/Food Export (SAFE) Final Performance Evaluation Request for Proposal (RFP)
Safe Agriculture/Food Export (SAFE) Final Performance Evaluation: Request for Proposal (RFP)
1. Statement of Work Detail
This Request for Proposals (RFP) is aimed at soliciting proposals from qualified individuals/organizations interested in implementing a final evaluation of the “Safe Agriculture / Food Export (SAFE) Project”, funded by the United States Department of Agriculture (USDA) and implemented by NCBA CLUSA. This request calls for a technical firm with demonstrated abilities to design and implement evaluation and research studies and collect data from multiple stakeholders along the Dominican beef and dairy value chains. The purpose of the final evaluation is to assess 1) project accomplishments in relation to its goals and objectives,
2) relevance of project activities, 3) signs of project effectiveness and impact, 4) efficiency of systems and processes, 5) sustainability efforts; and recommendations for USDA, project participants, and other key stakeholders for future food assistance and capacity building programs.
|Activity Title||SAFE Final Performance Evaluation|
|Project Name||USDA Safe Agriculture / Food Export (SAFE) Project|
|Donor||United States Department of Agriculture (USDA)|
|Project Duration||October 2015-March 2021|
|USD $16,212,120.85 resulting from the monetization of 16,100 MT of Crude Degummed Soybean Oil (CDSO) and of 4,320 MT of Yellow Grease Tallow|
|Implementing Organization||National Cooperative Business Association’s (NCBA) Cooperative League of the USA (CLUSA)|
|Junta Agroempresarial Dominicana (JAD) GENEX Cooperative (GENEX)
Texas Agricultural and Mechanical University (TAMU) AgriLife Research
|Puerto Plata, Duarte, María Trinidad Sánchez, Monte Plata, El Seibo, Hato Mayor, La Altagracia, San Juan, Independencia, Dajabón, Santiago Rodríguez and Santo Domingo provinces|
1. Project Background
The SAFE project, known locally as PROGANA, is being implemented over a period of five years, in the provinces of: Santiago Rodríguez, Dajabón, Independencia, San Juan, Monte Plata, La Altagracia, Hato Mayor, El Seibo, María Trinidad Sánchez, Duarte and Puerto Plata.
Prior to the implementation of the Safe Agriculture/Food Export (SAFE) project, NCBA CLUSA conducted a baseline study about the Dominican Republic Livestock value chain and production systems and a mid- term evaluation of the project, focusing on the eleven target provinces. The Project seeks to work with 14,407 individuals, out of which 13,200 are farmers and 1,207 are public and private extension workers, beef and dairy processors, decision makers, public and private stakeholder representatives, and 60 producer organizations.
The following are the specific objectives of the project:
- Improve agricultural productivity in the livestock (beef and dairy) value chain by increasing the use of improved techniques and technologies; improving farm management; increasing the availability of improved inputs and use of financial services; strengthen the capacity of government institutions and key groups; and increasing the leverage of private sector
- Expand trade of beef and dairy products by adding value to post-production; increasing the adoption of established standards; increasing access to markets; building linkages between
buyers and sellers; improving post-harvest infrastructure; increasing the use and efficiency of post- production processes; improving the policy and regulatory framework; and strengthening the capacity of key organizations in the trade sector.
Project Results Frameworks 1 and 2
To achieve the two main objectives of the Project, twelve essential activities were considered in its design to address all the links of the Dominican livestock value chain. Each activity is disaggregated into different tasks and specific actions that together contribute to accomplishing the 57 established indicators to be achieved during the five years of the Project, and their end results. Below is the project’s results framework representing the theory of change and strategic objectives linked to Food for Progress’ expected results.
The Theory of Change dictates the following 12 project activities:
ACTIVITY 1: TRAINING ON IMPROVED AGRICULTURAL TECHNIQUES
The project focuses on the adoption of improved agricultural techniques through training and technical assistance on the planting of improved pastures and their pasture management techniques with training sessions on the following: animal nutrition, animal feed; milking and milk-handling practices, animal health (use of pharmaceuticals, herd vaccinations, ticks prevention and animal handling), sanitary standards for dairy and beef processing facilities (Good Manufacturing Practices (GMP), Hazard Analysis and Critical Control Points (HACCP), sanitary registration of products) and training on the implementation of sustainable livestock management.
ACTIVITY 2: TRAINING ON FARM MANAGEMENT
This activity refers to the implementation of training on agribusiness and training and application of tools and techniques to improve farmers’ capacity in terms of farm administration and management in an orderly and documented way by recording the relevant information of the farms in terms of production (quantity of milk, number of cows, reproductive records), management of personnel and payroll, and farm economic management (income and expenses records). The documentation and registration of activities on the farm is a tool for making administrative and productive decisions that allow for a business approach to farm management thereby improving the levels of herd profitability.
ACTIVITY 3: FACILITATE AGRODEALERS AND OTHER INPUT SUPPLIERS
The project coordinates technical assistance activities with input suppliers (Semillas Latinas, ABODOM, Ferquido, Vitrogan, Enda Caribe, Nestle, Rica and others) to design and establish demonstration farms for management, milking parlors, and other modules of improved practices. Inputs include seeds, fertilizers, pharmaceuticals, etc. The project links input suppliers which are interested in supplying inputs to producers in the target provinces, providing technical training to extension agents on the use of their inputs, discounting and sharing the cost of the required inputs for establishing demonstration plots and providing specific technical assistance (TA) to farmers.
ACTIVITY 4: IMPROVE THE BEEF AND DAIRY VALUE CHAIN
The project provides matching grants for dairy processors and Producer Organizations (POs) to strengthen milk collection centers (MCCs) and beef processors supporting the Equivalence process. Infrastructure improvements for dairy include milk handling equipment for farmers, chilling equipment for MCCs and laboratory equipment for milk testing facilities. The infrastructure improvements in beef processing includes Sanitary Performance Standards control systems identified by the regulatory assessment and agreed to by the GoDR and private sector partners. SAFE works with the Ministry of Agriculture, Ministry of Public Health, the Administrative Ministry of the Presidency, laboratories, the DR Beef Producers Association (ASOCARNE), and slaughterhouses as private sector partners to achieve Equivalence.
ACTIVITY 5: CAPACITY BUILDING OF PRIVATE AND PUBLIC SECTOR AGRICULTURAL EXTENSION AGENTS/ SERVICES
The project will provide more support to key private sector value chain actors to establish and improve private extension systems. To date, 38 students were trained at the Technological Institute of San Ignacio de Loyola (ITESIL) as new extension agents, of which two are working on farms around the area. Also,
SAFE worked with project partners to provide training to private and public agricultural extension workers. Private sector companies included ABODOM, HESATECH, NUGEGA, MULTILAB, Nestle, and Rica. Public sector organizations included Ministry of Agriculture, Vitrogan, and the Autonomous University of Santo Domingo (UASD).
ACTIVITY 6: CAPACITY BUILDING OF PRODUCER GROUPS
There are five livestock federations collaborating with the project (FEDEGANO, FEDEGANORTE, FEGACIBAO, FEDAGARES, and FEGASUR). The project has signed coordination agreements with the first two federations and with many of the cattlemen’s associations geographic zone of intervention. The five federations aggregate around 90% of associated livestock farmers within the eleven target provinces. The obligations of both parties are to share resources, mainly technical support and expertise to contribute to the Project´s goals. One of the SAFE Project’s goals is to strengthen the capacities of livestock POs focusing on organizational, administrative, financial and technical aspects.
ACTIVITY 7: CAPACITY BUILDING FOR PROMOTING IMPROVED POLICY AND REGULATORY FRAMEWORK
Under this activity, the project is supporting the development of necessary infrastructure overseeing the control and implementation of the sanitary standards equivalent to those demanded in the U.S. for the beef sector that are required to export beef to the U.S. market in Puerto Rico. Three main actors of the beef value chain are essential to attaining Equivalence: DIGEMAPS (General Directorate of Medicines, Food and Health Products of the MoH), the official public agency responsible for the sanitary inspection system in the Dominican Republic; the reference laboratories to be audited by the FSIS; and, the three preselected slaughterhouses (Mercarne , Agrocarne, and Suplidora de Carne A&B); at least one of which will be selected and audited by FSIS, to determine whether the MoH is operating a beef inspection system for beef that is equivalent to the U.S. inspection system. SAFE is providing technical assistance to slaughterhouses and beef farmers supplying slaughterhouses on a proposed course of action needed to achieve those standards.
ACTIVITY 8: FINANCIAL SERVICES – FACILITATE AGRICULTURAL LENDING AND PRIVATE AND PUBLIC- SECTOR INVESTMENT
The project provides training to financial institutions (FIs) on the financing needs of dairy farmers and processors. The SAFE project collaborates with FIs to conduct outreach to identify credit-worthy dairy producers and processors. Investment plans for small farms are developed as part of the support of the project. The project links these small farmers with financial agencies to facilitate producers’ and processors’ access to credit based on the result of the small investment plans supported by the project. Additionally, the SAFE project facilitates the participation of farmers and processors at exposition, trade, and finance fairs to promote available financial products.
ACTIVITY 9: TRAINING – BEEF AND DAIRY QUALITY STANDARDS
SAFE provides training to small and medium dairy manufacturers throughout the Dominican Republic. These trainings guide processors on how to improve the quality of cheese and other dairy products by applying GMP such as the use of sanitation materials, avoidance of adulterating milk with peroxide, the use of proper manufacturing clothes (hats, boots, etc.), and avoiding product cross contamination. For beef, the project trains beef processing staff on GMP, specifically related to SPS gaps. Additional training on HACCP will be aimed at beef processors and slaughterhouses.
ACTIVITY 10: MARKET ACCESS: FACILITATE PRIVATE AND PUBLIC PARTNERSHIPS
SAFE signed a general agreement with DIGEGA that was made operational by the articulation of 26 MEGALECHE agents responsible for providing TA to targeted farmers and POs through the public extension system. In addition, SAFE works with a Consultative Technical Council (CTC) and the Equivalence Group to achieve the Equivalence certification to export beef to the USA (Puerto Rico). The approach establishes public-private partnerships to focus on the Equivalence certification (SAFE’s primary result) with FSIS standards.
ACTIVITY 11: CAPACITY BUILDING – DEVELOP PUBLIC INFORMATION CAMPAIGN AND IMPROVE MARKETING AND BRANDING
The project implements a comprehensive approach to develop the use of the Seal of Quality in dairy products. To improve the quality of milk at the consumer level, the project works from the beginning of the milk value chain, training milk producers on hygienic and safe milking techniques that will improve the quality of the milk, seeking to produce Grade “A”. Next, the Project trains milk processors on the requirements for the attainment of the Seal of Quality. Finally, the Project accompanies milk processors in the attainment of the sanitary registration from the Ministry of Health and the posterior attainment of the Seal of Quality through technical assistance. Several visual materials are developed and distributed along the value chain with messages about adequate production and processing practices.
ACTIVITY 12: MARKET ACCESS – FACILITATE BUYER-SELLER RELATIONSHIPS
In the dairy sub-sector, the SAFE project aims to facilitate linkages at four levels: 1) between POs/MCCs and processors; 2) between processors and individual farmers; 3) between equipment and service providers and processors; and 4) between producers and equipment and service providers. The project organizes meetings between MCCs and processors with nearby independent dairy farmers. The following interventions are currently part of the project’s interventions.
Table 1: Value Chain Partners and Intervention
|Value Chain Partner||Intervention|
|Producers||NCBA CLUSA’s interventions with producers include training 3,271 dairy and beef producers on improved agricultural techniques and technologies via workshops, demonstration plots, training in farm management and the
development of 3,750 small business plans for small farmers.
|Processors||NCBA CLUSA’s interventions with processors include: training approximately 300 dairy processing staff on milk quality and best manufacturing practices, training 140 dairy producer association members on milk collection center management, and investments in quality and production improving infrastructure. For beef, technical assistance and three audits about Sanitation Performance Standard (SPS) measures for slaughterhouses supporting the
Equivalence process and training 24 beef processors staff on SPS measures.
|Coops & POs||NCBA CLUSA’s interventions with Co-ops and POs include the signing of coordination agreements, the development of building strategies for the federations and diagnostics of the federations, development of 24 assessments for cooperatives to determine organizational capacity and sustainability as
businesses; the establishment of linkages with Peace Corps volunteers,
|multiple USAID Farmer-to-Farmer Volunteer experts, and exchanges of best practices with U.S. dairy cooperatives such as Organic Valley and Cabot
Creamery to provide administrative and management support.
|Input Suppliers||NCBA CLUSA’s interventions with input suppliers include: organizing exchange visits for project partner suppliers with demonstration plots and training on
|Buyers – Sellers||NCBA CLUSA’s interventions to connect buyers and sellers include: facilitating the linkages between international buyers and local sellers, sponsoring the participation of local sellers at trade expos, participating in trade shows, supporting 3 beef processing companies and two dairy processors for export. Additionally, promoting a Seal of Quality for final consumers to increase demand for beef and dairy products that meet GoDR and USDA food safety
|Financial Services Institutions||NCBA CLUSA’s interventions for Financial Services Institutions include organizing intermediation workshops between input suppliers, MFIs, and producers, sponsoring participation at credit fairs, and creating 3,750 business plans for producers and processing facilities, and providing TA to POs to
develop micro-credit programs for their members to invest in beef and dairy production and quality improving technologies and practices.
|Government Institutions||NCBA CLUSA’s interventions for government institutions include: developing a training program for government extension agents on the dairy value chain, training on extension methodology to the government extension service, training and provision of M&E tools, 30 trainings on beef and dairy farm management, 24 trainings on artificial insemination, 24 trainings to public credit agents on farm valuation. Also, training 25 newly hired slaughterhouse inspectors from the Ministry of Health (DIGEMAPS) on the necessary measures and controls for an adequate beef inspection system and three laboratory analysts from the Dr. Defillo National Laboratory staff on how to evaluate the supporting third-party laboratories and the prevention of conflicts of interest ; the creation and follow up of the Technical Consultative Committee (CTC), strengthening of the Technical Equivalency Group and the Presidential
Commission for Beef Export.
2. Purpose and Scope of the Final Evaluation
Purpose of this final evaluation is to assess the extent to which the objectives and results of the project have been achieved compared to the baseline and midterm results, specifically to what extent the activities have contributed to increased agricultural productivity and marketing of agricultural products. Additional objectives include 1) to provide an independent, third party assessment of the project’s results; 2) assessing to what extent midterm recommendations were implemented; 3) to document lessons learned; and 4) to assess the potential for sustainability.
The evaluation will aim to measure the relevance, efficiency, effectiveness, impact and sustainability of the project activities. To do this, the evaluators will address the following questions:
Table 2: Evaluation Questions:
|Relevance||· To what extent has the project responded to the actual needs and interests of the target farmers, associations, groups, and other stakeholders?
· To what extent does the project integrate or strengthen the national sectoral strategy to improve production and market access?
· To what extent does the project fit into the strategies of the DR government?
· What was the added value of the project considering perspectives from key stakeholders such as USDA WA, USDA DR, POs, and implementing partners?
|Effectiveness||· To what extent have the expected results of the project been achieved?
· What factors have been critical to the achievement or non- achievement of project objectives?
· Has the project led the planned activities? Has it achieved the expected results?
· What adaptations, if any, were made to the project’s implementation process in order to achieve project objectives and targets based on mid-term evaluation findings?
· To what extent the activity changes after midterm evaluation results and recommendations were successful in improving effectiveness and ability of the project to achieve its expected results?
· What were the planned and unplanned outcomes that can be attributed to the project?
· What are the evolutions of the project indicators?
· Has the implementation process (approach – methodology) been effective?
· What is the evidence and estimate by how much agricultural productivity and the marketing of agricultural products have been improved through project actions?
· To what extent did the Livestock Farmer Field School methodology contribute to improved agrobusiness management, production and quality compared to traditional adult education methodologies?
· To what extent did project activities prepare stakeholders to participate in virtual activities versus in-person as a result of the global pandemic from the novel coronavirus COVID-19?
· To what extent did stakeholders benefit from the project’s attempts to provide virtual trainings as a result of the pandemic? To what extent did the technical assistance received by beneficiaries improve the management of their agrobusinesses, production and quality?
· To what extent did beneficiaries adopt the project’s improved practices and technologies and improve their agrobusiness’s
management, production, and quality?
|Efficiency||· To what extent, appropriateness, and functionality have the various units / levels of management and coordination (grants, project staff, M & E, local committees, USDA/W, USDA/Santo Domingo and others) supported the implementation of the project?
· To what extent were the size and composition of the teams appropriate to the strategies?
· To what extent do the various project activities support each other?
· To what extent has the process of project implementation including finance and administration optimized time and resources?
|Impact||· What was the impact of the project on:
Ø Increased household income
Ø Increased agricultural productivity
Ø Increased market access and expanded trade
Ø Increased quality of agricultural products
Ø Increased public and private extension services
· How does the beneficiaries’ perception of the project’s achievements, especially the increase in productivity and income vary across value chains and regions?
· What is the legacy of the project? Organize achievement categories from most significant impact to least significant impact.
· Which activities were the most successful and the least successful? Why? Breakdown by intervention / activity.
· To what extent have beef and dairy ranchers’ capacity to respond to on-farm shocks and daily activities improved as a result of the project, and why?
· How has the project’s activities strengthened local producer organization’s governance, member services and equity, as well as incomes?
|Sustainability||· To what extent have local actors (JAD, cooperatives and farmers’ organizations, MCCs, Slaughterhouses, local government and civil society groups) been involved in the management of the project?
· Which local counterparts have increased their capabilities to continue with the project’s actions, and why?
· Which partnerships were the strongest and why?
· What is the probability that these actors will be able to provide succession?
· Are the techniques and technologies used in the project easy to maintain locally? To what extent are the conditions for local control of these techniques and technologies guaranteed? Are people facing any additional challenges in light of the pandemic? Prioritize activities to continue after the project closes from most likely to least likely.
· To what extent do the project interventions support and stimulate the local economy? When possible, consider adding tools to get insight on quantifying the project’s impact on the local economy.
· Which of the following project impacts are likely to be sustained and/or scaled-up after the project closes?
o Increased household income
|o Increased agricultural productivity
o Increased market access and trade
o Increased quality of products
· Identify and explain short and mid-term challenges to sustain program results, and what actions could be taken before the end of the project to mitigate those challenges, if any.
· To what extent are local government actors likely to achieve FSIS equivalence and continue strengthening the food safety inspection system?
· To what extent did modifying the traditional training methodology to the methodology in the “Collective Efficiency Schools” permit improved learning and application of the concepts?
The successful evaluator will lead the entire evaluation process from design to report writing. NCBA CLUSA staff will provide logistical and other administrative support. Key activities will include:
- Review project documents and other published and grey literature related to the This includes the project’s internal documents e.g., work-plan and strategies as well as relevant external documents including, but not limited to, national policies and regulations, special studies carried out by other agencies, etc.
- Develop study design and methods that will have a balance of quantitative and qualitative techniques, analysis of secondary data, observations, and document
- Identify and select study participants from the sampling frame provided by the project staff. The sampling frame will include all the beneficiaries of both value chains from all the
- Develop, edit and finalize data collection tools e.g., household survey questionnaire, PO survey questionnaire, and key informant interview and focus group
- Develop data collection guide specifying data collection and management structure, field schedules and data quality assurance
- Train field supervisors, key informant interviewers and focus-group facilitator on the methods and processes.
- Perform quality assurance during design, testing, fieldwork, data entry, data analysis and reporting;
- Prepare study report
- Present findings, conclusions and recommendations (in both written report and PowerPoint formats).
Project evaluation activities must not threaten the health of the evaluation team, project staff, participants, or the public. Furthermore, evaluators must comply with local health ordinances to protect public health (for example, wearing face masks that cover their nose and mouth, practice social distancing, etc.).
In adherence to RFP requirements, the final evaluation will be conducted by an independent external evaluator that will be selected by a procurement committee. The project’s M&E Specialist will be the lead NCBA CLUSA representative, however all project personnel will be expected to contribute to the
evaluation by providing information, assisting wherever feasible, and helping to determine any appropriate project corrections to be made and implemented as determined by the evaluator.
The evaluation team will propose a robust study design that will involve both quantitative and qualitative methods to collect and synthesize information in providing detailed answers to the specified evaluation questions related to relevance, effectiveness, efficiency, and sustainability dimensions. Achievement in key project outcomes will be determined by comparing target value with actual accomplishment. Change in key outcome indicators will be assessed by comparing baseline values with the findings of the final evaluation. This will be a non-experimental pre-post design. Baseline and midterm information on outcome indicators will be compared to final information to assess the extent and direction of change. The baseline study did not include a control group, and therefore it will not be feasible to apply experimental or quasi-experimental designs.
The baseline study was conducted in July-August 2016 before the start of the project activities. The objectives of the baseline study were primarily to establish baseline values for the outcome indicators and identification of activities and strategies for dairy and beef value chains. Methods included a survey of a representative sample of 505 farms, structured and semi-structured interviews with dairy distributors, small beef processors, financial institutions, public regulatory agencies, private input suppliers, and farmers and PO officials. The midterm evaluation was conducted between January and March 2019 to determine the effectiveness and impact of the project to date and provide recommendation for any necessary implementation corrections to ensure the project achieves its stated objectives.
Data Collection: Data will be collected from organizational and individual level beneficiaries and market participants e.g., producer organizations, individual and organizational level processors, buyers, individual beneficiaries, government officials, financial institutions and so forth. NCBA CLUSA will provide sampling frame and other necessary support to the evaluation team e.g., literature to be reviewed; project briefing that will include context, activities and indicators; and briefing on stakeholders and beneficiary groups. There will be time spent in the SAFE project office in Santo Domingo and in the field. The evaluator and SAFE management staff will develop a more detailed travel schedule based on the evaluator’s meeting requirements prior to arrival in country, if international. Data collection strategy and logistics will be adjusted considering the evolution of COVID-19 at the time of this evaluation. The safety of the evaluation team, beneficiaries and all staff should always be prioritized by complying with the recommendation of health authorities, respecting physical distancing and wearing masks as appropriate.
- Sampling: Individual farmers will be selected using multi-stage cluster sampling methods. Proportionate random technique will be used to select males and females from each sub-cluster. Beef and dairy will be the first level cluster. Second and tertiary level cluster will be determined based on the geographic spread. The sample selected for the final evaluation will closely match the study samples of baseline and midline. For the quantitative data collection e.g., focus group discussion and key informant interviews purposeful/availability/ snow-ball sampling methods will be
|Table 3: Illustrative Sampling and Data Collection Method|
|Value Chain Actor/Partner||Selection Method||Data Collection Method|
|Direct and indirect beneficiaries||· Use project’s beneficiary listing as a sampling frame
· Treat each region as a stratum.
· Determine the sample size for each region utilizing appropriate sample size estimation formula.
· Select sample randomly from each stratum.
|Household survey to capture quantitative information
M&E report and other project document review
|Federations, Associations, Cooperatives, and POs||Quantitative Data
· Use project’s associations and POs listing as a sample frame
· Treat each region as a stratum
· Determine the sample size utilizing appropriate sample size estimation formula.
· Apply stratified random sampling methods to select participants from each region.
· Use of availability and snow-ball sampling method to select focus group
Structured surveys to capture qualitative data
Focus group discussion to capture qualitative information.
|Agro-dealer and input suppliers||Quantitative Data
· Census instead of sample as the number of is manageable.
· Use of availability and snow-ball sampling method to select focus group participants.
Structured surveys to capture qualitative data
Focus group discussion to capture qualitative information.
(local and national)
|Purposeful and availability sampling method||Document reviews
Key informant interviews
|MCCs, Beef & Dairy Processors||Quantitative data
· Census instead of sample as the number of is manageable.
· Availability and snow-ball sampling method to select focus group participants.
Structured survey to collect quantitative data
Focus group discussion to capture qualitative information
Data Analysis: Data will be analysed at the regional and project level. The regional level analysis will include exploration as well as explaining of the emerging themes and concepts within each value chain. Project level analysis will involve putting the emerging themes and concepts and explanatory factors from all the regions together.
Quantitative data will be analysed using SPSS or similar statistical solutions. The analysis will involve descriptive analysis that will be presented in tabular/chart formats. Within region and across region analysis will be conducted for each indicator and participant category. The evaluation team will develop a code book that will include codes for location, participant type, questions and response categories.
Analysis of qualitative data will follow the framework of data reduction, data visualization, and conclusion drawing and verification. Data reduction phase will include selecting, focusing, simplifying, abstracting, and transforming the data that appear in written-up field notes or transcripts. All the field notes will be transcribed first in the local language and then will be translated in English. Findings from the document review will be noted in the document summary forms. Data will be presented in visual format that will help conclusion drawing and verifications. Finally, the team will draw conclusions based on regularities, patterns, explanations, causal flows, and propositions.
4. Addressing Limitations
Due to the complexity of different components of the project and the geographical location of its activities, opportunities for errors exist. The evaluation team is expected to be aware of such complexity and develop strategies to minimize opportunities for the errors. The evaluation team will develop quality control systems, including developing detailed data collection guides and overseeing field data collection.
This evaluation will heavily rely on secondary performance information reported in quarterly and annual reports. The quality of performance data will affect the accuracy of projected trends. The evaluation team should check the validity and the reliability of performance data before analyzing it.
Primary data collected from beneficiaries may reflect the opinions of the most dominant groups without capturing the perceptions of less vocal groups. The evaluation team should take this into account and make sure that all parties are freely expressing their views. The evaluation team will mitigate this potential oversight by organizing focus groups based on the need to ensure that participants speak freely during discussions.
- Inception report that will include methodology, sampling approach, data analysis, field procedures. USDA will publish evaluation reports in the future and final copies of reports should be free of PII and otherwise ready to publish. The evaluator will be asked to sign the attached waiver with the final report.
- Data quality assurance plan and
- Data collection and analysis tools: Set of questionnaires, formats, and Excel spreadsheets used to collect and analyze data, and their implementation
- Data files: raw field notes, transcribed notes, tabulated
- Weekly Progress Reports: A written and/or electronic report of the evaluation’s progress made in the field covering key scheduled activities, completion status and found constraints regarding data collection.
- Initial presentation: A PowerPoint presentation on results and conclusion of the evaluation. The presentation should not be more than 15
- Submission of Draft Evaluation Report: The team will submit a draft report to the NCBA CLUSA key staff (Project COP and M&E Specialist; and Program Manager at HQ) and USDA Analyst, who will provide comments for
- Final Report: A written and electronic document that includes an executive summary, table of contents, methodology, findings, conclusions, lessons learned and recommendations. The evaluator will collect and report data on all relevant performance The report will also include annexes that will have all custom and standard indicators with disaggregates and updated values in comparison to baseline values. All the personal identifying information must be scrubbed in the final version of the report. The report will be submitted in English with an executive summary in Spanish and should include a database with all collected information and an analytical framework. The evaluation should assess:
- A 2-3 page stand-alone brief describing the evaluation design, key findings and other relevant considerations. It will serve to inform any interested stakeholders of the final evaluation and should be written in language easy to understand by non-evaluators and with appropriate graphics and tables.
- Final presentation in Spanish: A PowerPoint presentation on definitive results and conclusion of the evaluation to the project staff and select local
Evaluation Management and Coordination
Per the requirement of USDA policy, this evaluation will be conducted by independent third-party evaluators. NCBA CLUSA will provide logistical support that will also include inviting stakeholders for the meetings. Evaluators will present methodology and findings to USDA officials and NCBA CLUSA team. Draft report will be shared with USDA for their review and comments. The report will be finalized after the approval from USDA.
All final versions of international food assistance evaluation reports will be made publicly available. Evaluators shall provide a copy of the evaluation reports that is free of personally identifiable information (PII) and proprietary information. Final versions of evaluation reports ready for publication should be accessible to persons with disabilities. For guidance on creating documents accessible to persons with disabilities, please see the following resources: https://www.section508.gov/create/documents
6. Audience and Intended Use
This evaluation will help measure project performance, effectiveness, efficiency, impact, sustainability and relevance of the project’s approach in achieving its stated goals. Performance results, lessons learned, and the success of the supported activity are to be shared with USDA to inform future activity design. The audience will include all key stakeholders e.g., USDA, NCBA CLUSA, partner organizations e.g., Texas Agricultural and Mechanical University (Texas A&M), GENEX Cooperative, Dominican Agricultural Business
Board (JAD), project participants, the MEGALECHE extension program from the General Livestock Directorate (DIGEGA) of the Ministry of Agriculture, the General Directorate for Medicines, Food, and Sanitary Products (DIGEMAPS) from the Ministry of Health, and the National Council for the Support of Dairy Production (CONALECHE), private sector partners, and other non-governmental organizations working in synergy with the SAFE project. Other audiences may include individuals and groups who are not necessarily engaged in the project activities but may benefit from the findings that may be published and disseminated through CLUSA website and/or presentation in national and international conferences.
7. Selection of the Evaluation Team
An outside firm/external evaluator will be selected for the final evaluation. Through this competitive process NCBA CLUSA will select an evaluation firm who has local expertise and experience evaluating agricultural and trade initiatives with similar target audiences. Major requirements for the evaluator will include:
- Demonstrated knowledge, analytical capability, language skills and experience in conducting evaluations of development programs and/or projects involving agriculture and ideally livestock, trade, and marketing in
- Technical skills and capacity in the application of analytical frameworks such as conducting qualitative and quantitative surveys, involvement of stakeholders in the evaluation, and statistical
- Have knowledge of US Government projects including either USDA or
- Have experience managing or evaluating agricultural value chain
- Ability to travel to rural areas of DR on difficult roads to meet with stakeholder
- Budget Competitiveness.
- Fluent English and Spanish language and writing
- Competency to write detailed, concise and coherent final evaluation
- Gender balance and cultural
8. Conditions of application
Consultants and companies wishing to carry out this mission must send the following items (12-point font) in the order specified below to be considered during selection:
1. Technical proposal
The technical proposal (12 pages max.) must reflect how the offeror will carry out the tasks included in the work. Candidate companies will provide a detailed plan of the specific activities, the timetable for carrying out the mission, as well as the data collection and analysis activities. In addition, it will include a proposal for the statistical approach.
2. Financial application
The candidate companies/consultants will propose a realistic estimate of the cost of this mission, including a detailed budget and a justification of the expenditure. The budget will only contain costs that can be directly attributed to the proposed activities, with an explanation of the line items. All training costs, such as rental of premises, advance transport, etc. must be clearly articulated for each training. Applicants must present adequate administrative and financial systems to manage the funds covered by this agreement. If the candidate companies charge overhead, they will need to provide their NICRA. The maximum amount available is $100,000.
The candidate companies/consultants must briefly list and describe their history, vision, objectives, legal status, ownership and management structure, current projects/services, current clients/assignors, current geographic scope and experience.
The candidate companies/consultants must briefly list and describe the names and qualifications of the key personnel assigned to the assignment (the CVs of the proposed staff must be included in the Annex). The proposed team should preferably be multidisciplinary and include at least (A) a team leader for monitoring and evaluation, with extensive experience designing and implementing evaluations and analyzing the results (ex: a statistician and / or analyst familiar with the analysis of quantitative and qualitative data); Experience in the target regions is desirable; (B) an agricultural economist, a rural development specialist or a person with an equivalent level of competence, specializing in economic development and data collection activities; (C) a database and information system specialist; (d) a data entry team; and (e) enumerators.
5. Proof of Experience
Applicant companies/consultants will have to prove their experience in similar missions by providing a list of all contracts and/or cooperative agreements involving similar or related work during the last two years. Additionally, the applicants must submit a work sample and at least three reference letters from previous consultancies.
6. Legal registration
Candidate companies/consultants’ engagement is subject to the consultant obtaining necessary visas and work authorization.
10. Criteria for the evaluation of the proposal
The following criteria are those under which all proposals will be judged:
- Quality of technical approach and methodology (30%)
- Experience in agriculture and/or livestock economics (20%)
- Past experience of individual and/or company with USDA and/or USAID or other International Development Organizations (10%)
- Demonstrated experience and technical skills of the team/reference letters (10%)
- Completeness of proposal, including schedules, total budget, employee CVs, etc. (10%)
- Cost realism, budget justification and effectiveness. Given it meets these standards, competitive budgets will receive a higher score. (20%)
11. Application Deadline
Applications must be in English and with single spacing. The pages must be numbered, and each page must contain the name of the company. Proposals must be sent by e-mail with Subject line “SAFE Final Evaluation Proposal” to NCBA CLUSA at the following address: firstname.lastname@example.org and/or email@example.com
Please include the name of the person in your organization sending the application, as well as the phone
number, fax number and e-mail address. Applications must be submitted by December 30th, 2020 at 5:00 pm EST at the latest.
If you need more details, please contact our offices by email at FinalEvalSAFE@ncba.coop and/or firstname.lastname@example.org. NCBA CLUSA reserves the right to subsidize all or none of the applications submitted and / or to modify the terms of reference / geographical areas before the project begins.
Table 4: Approximate Timeline of Events
|Develop a scope of work for the evaluation and list of required qualifications for the external evaluation
team, publish a request for proposals
|Three months prior to evaluation (November 2020)|
|Review applications||Two months prior to evaluation (December
|Select external evaluation team/consultants||One/Two months prior to evaluation
(December 2020/January 2021)
|Work with the external evaluation team to finalize
project evaluation TORs and agree on evaluation methodology
|One month prior to evaluation (January 2021)|
|Field work – data collection (interview, focus groups)||3-4 weeks in February/March 2021|
|External evaluators prepare analysis documents and
|One month after the evaluation activities
|Project team (COP, senior leadership in HQ, M&E Specialist, technical teams where appropriate) review evaluation report and analysis, as well as lessons
learned and other documentation
|One month after evaluation (April 2021)|
|Submit final evaluation report to USDA||May 2021|
|Presentation of findings to stakeholders||One month after evaluation (May 2021)|