Imani Development recently designed a Monitoring and Evaluation (M&E) System for the Economic Partnership Agreement (EPA) between six SADC member states and the EU, known as the SADC-EU EPA. Elena Mancebo, M&E specialist at Imani Development, sat down with Dr Nick Charalambides to reflect on what were our learnings and findings from this and other projects dealing with the monitoring and evaluation of trade agreements.
The SADC-EU EPA is a trade and development agreement that gives asymmetric access to parties in the SADC EPA group. The agreement grants SADC EPA member states market access to the EU and improves trading terms for signatories mainly for agricultural goods and fisheries, as well as wine, sugar, flowers and canned fruits. The EU also benefits from new market access into the region. From a development perspective, this trade agreement also focuses on sustainable development, by seeking to address social and environmental issues in the region through trade.
Elena Mancebo: To begin with, can you please explain why trade agreements are monitored? What exactly needs to be monitored?
Nick Charalambides: There are three key aspects to trade agreement monitoring:
- Compliance with the agreement. When monitoring a particular trade agreement, we want to check whether the provisions of the agreement are actually being implemented by signatories. Have tariffs been reduced? Do border authorities actually recognise the food safety standards of trade partners?
- Direct beneficiaries. Put simply, we want to see whether the agreement was worth bothering about in the first place. Are those, who should be benefitting from the agreement, indeed benefitting? When monitoring a specific agreement, we would expect for instance to see importers and exporters trading tariff free.
- Overall development impact. Later in the M&E process, the evaluation of the agreement will be able to answer critical questions such as, is the agreement leading to employment creation? Is it leading to economic growth? In the case of SADC region, where industrialisation is a key strategic priority for the region, the evaluation should ask: is the agreement contributing to industrialisation?
A particular difficulty in the monitoring of trade agreements pertains to what implementation really is. What implementation actually looks like in practice, is often not sufficiently clear for the Government authorities responsible for implementation. Trade agreements are legal documents that represent a commitment by their signatories to a set of principles, actions and provisions. However, these documents are not yet actionable but need to be translated: what laws need to be changed? and how? What institutions need to be established, and how should staff be trained to ensure they correctly apply the agreement?
The intended beneficiaries of the agreement – businesses on the ground – also face a steep learning curve when they first start to “use” – take advantage of the benefits offered by – the agreement. Businesses must establish, for example, what the new qualifying criteria are for accessing new markets, which is likely to take time and money. De facto legal market access is different to real market access, which requires a certain business size and skill level. A monitoring system must alert implementers as to whether users are actually taking advantage of the agreement provisions and if not, why not. If this is not picked up and addressed early enough in the implementation process (through the provision of advisory services and support) the agreement is likely to fail.
EM: This would explain why monitoring whether the private is taking advantage of the benefits of trade agreements is so important. Monitoring this “usage” is a pre-requisite to evaluating its impact.
NC: Absolutely. However, evaluation is also particularly difficult even when we have properly monitored whether the private sector is “using” the agreement. In my experience, what can make evaluation of trade agreements an effective process is to capture the hearts and minds of those responsible for evaluation by, focusing on the intended social impact of the trade deal. Monitoring and evaluation of trade agreements needs to move beyond the technocratic aspects of compliance to understanding the power of an agreement. It needs to act as a tool for continuous improvement of implementation plans for the benefit of a country and its citizens. This requires those developing the M&E framework to set out clearly, in a theory of change, how the agreement will make a difference to people’s lives.
The theory of change allows stakeholders to trace the results of implementation of EPA commitments, to its use, to progress towards the desired outcomes and impact. A theory of change enables us to see if the EPA is delivering and, if not, why not. It also enables us to identify appropriate indicators for the different stages (refer to table below).
EM: Are there any additional success factors for trade monitoring to be effective?
NC: There is no brief list of items to answer that question. The entire approach to implementation plans needs to change, because they are in fact ‘the heart beat’ of an agreement. It must be carefully understood that an effective implementation plan is a clear, precise and practical system, and not a bureaucratic checklist as we have become accustomed to. For instance, the SADC region has inherited an approach to trade negotiation and agreement, which suits other contexts, such as America, China or Europe, where trade law, technical specification and standards have a long history. This is not the case in SADC, where basic clauses of signed agreements have not been implemented 15 – 20 years into the agreement. For example, ‘legislation harmonisation’ as it appears in trade agreements, usually involves a plethora of activities! This level of complexity requires that experts with a range of technical skill-sets and expertise are engaged in the process. Fortunately, the pool of homegrown talent and specialisation in the region is increasing rapidly; but, more emphasis and resources need to be invested in the design and operationalisation of trade agreements.
However, once an effective implementation plan is in place, monitoring is essential to ensure impact. We don’t know what progress looks like, which means that monitoring is part of an iterative process of trial, error and improvement. Getting it wrong is part of the process, but we need to get past this phase into review and improvement.
Lastly, ensuring stakeholder buy-in into the process cannot be over emphasised. Realising the economic and social potential of trade agreements is entirely dependent on the level of engagement of those in charge of usage, implementation and monitoring. Ongoing and targeted sensitisation, education and motivation of private sector, civil society and legislators is key.
Imani Development has extensive experience in trade facilitation, policy development and implementation support, and monitoring, evaluation, and learning. Recent examples of our work include the design of a Monitoring and Evaluation System for the SADC EPA Member States to track the operationalisation and impact of the SADC-EU Economic Partnership Agreement (2017 – 2018), undertaking Mid-Term (2014 – 2015) and Final Evaluations (2016 – 2017) of the ITC Horticulture Productivity and Trade Development Programme in Lesotho, the design of TradeMark East Africa’s New Strategy 2 (2017 – 2021) for the elimination of Non-Tariff Barriers (2015 – 2016), and the provision capacity building and implementation support to the Ministry of East African Community Affairs in Uganda (2011 – 2014) to further their EAC integration agenda.