EUROPEAN COURT OF AUDITORS

12, rue Alcide De Gasperi

1615 Luxembourg

LUXEMBOURG

Tel. +352 4398-1

Email: eca-info@eca.europa.eu

Internet: http://eca.europa.eu

Twitter: @EUAuditorsECA

YouTube: EUAuditorsECA

More information on the European Union is available on the Internet (http://europa.eu).

Luxembourg: Publications Office of the European Union, 2015

Print | ISBN 978-92-872-3002-7 | ISSN 1831-0834 | doi:10.2865/77994 | QJ-AB-15-013-EN-C
PDF | ISBN 978-92-872-2998-4 | ISSN 1977-5679 | doi:10.2865/588946 | QJ-AB-15-013-EN-N
EPUB | ISBN 978-92-872-2983-0 | ISSN 1977-5679 | doi:10.2865/94712 | QJ-AB-15-013-EN-E

© European Union, 2015

Reproduction is authorised provided the source is acknowledged.

Copyright: Permission must be sought directly from the copyright holder of the photos "Training for farm women" and "Farmers' working group (field visit)" on p. 10, in order to use or reproduce them.

HOW TO OBTAIN EU PUBLICATIONS

Free publications:

•one copy:
via EU Bookshop (http://bookshop.europa.eu);

•more than one copy or posters/maps:
from the European Union’s representations (http://ec.europa.eu/represent_en.htm);
from the delegations in non-EU countries (http://eeas.europa.eu/delegations/index_en.htm);
by contacting the Europe Direct service (http://europa.eu/europedirect/index_en.htm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) (*).

(*)The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you).

Priced publications:

•via EU Bookshop (http://bookshop.europa.eu).

EN
2015

NO 12

Special Report

The EU priority of
promoting a
knowledge-based rural
economy has been
affected by poor
management of
knowledge-transfer and
advisory measures

(pursuant to Article 287(4), second subparagraph, TFEU)

Audit team

The ECA’s special reports set out the results of its performance and compliance audits of specific budgetary areas or management topics. The ECA selects and designs these audit tasks to be of maximum impact by considering the risks to performance or compliance, the level of income or spending involved, forthcoming developments and political and public interest.

This performance audit was produced by Audit Chamber I — headed by ECA Member Augustyn Kubik — which specialises in preservation and management of natural resources spending areas. The audit was led by ECA Member Jan Kinšt, supported by Alejandro Ballester Gallardo, head of private office; Michael Bain, head of unit; Klaus Stern, team leader and Els Brems, deputy team leader. The audit team consisted of Stuart Ballantine; Malgorzata Frydel; Athanasios Koustoulidis; Lorenzo Pirelli; Maria Eulàlia Reverté i Casas; Michael Spang and Diana Voinea. Secretarial assistance was provided by Terje Teppan-Niesen and language support by Milosz Aponowicz, Cathryn Lindsay, Michał Machowski, Paulina Pruszko and Charlotta Törneling.

Contents

Glossary

Executive summary

Introduction

Investment in knowledge and skills in the European Union

Rural-development support for knowledge-transfer and advisory services

Coordination with other EU support for knowledge-transfer and advisory services

Audit scope and approach

Observations

Part I — The procedures to identify and select the most effective knowledge-transfer and advisory services

Member States identified the skills needs of rural operators in such broad terms that almost any kind of activity could be funded

Member States do not always verify the qualifications and experience of providers

The selection of activities and providers suffers from a lack of fair and transparent competition

Commission guidance to the Member States on knowledge-transfer and advisory measures was limited

Part II — Procedures to ensure cost-effective and coordinated delivery of knowledge-transfer and advisory services

Member States approved training activities without carrying out effective checks that their costs were reasonable …

… and there are indications that they sometimes paid too much for training or advice services which were insufficiently justified

The Commission’s audit coverage did not provide sufficient information on the weaknesses found by the Court

EU funds and other sources of financing are not coordinated to provide efficient delivery of knowledge-transfer and advisory services

Part III — Procedures to monitor and evaluate the results of the knowledge-transfer and advisory activities

Direct feedback from training participants is often collected by training providers but seldom used by Member State authorities to improve subsequent actions

Some examples of good practice exist, with Member States making efforts to collect useful information regarding the impact of knowledge-transfer and advisory activities

The data collected to show the aggregated results of knowledge-transfer and advisory services at EU level are not reliable

Conclusions and recommendations

Annex I | — | Implementation of knowledge-transfer and advisory measures in the Member States audited
Annex II | — | The ‘Kirkpatrick four levels’

Reply of the Commission

Glossary

CAP: common agricultural policy: the set of legislation and practices adopted by the European Union to provide a common, unified policy on agriculture.

CMEF: EU-wide common monitoring and evaluation framework for rural development in the 2007-2013 programming period.

CMES: EU-wide common monitoring and evaluation system for rural development in the 2014-2020 programming period.

Deadweight effect: A situation where a subsidised activity or project would have been wholly or partly undertaken without the grant aid.

European Structural and Investment Funds (ESI Funds, ESIF): A new term for five EU funds providing support under cohesion policy in the 2014-2020 programming period.
These five funds are: the European Regional Development Fund (ERDF); the European Social Fund (ESF); the Cohesion Fund (CF); the European Agricultural Fund for Rural Development (EAFRD); the European Maritime and Fisheries Fund (EMFF).

Framework agreement: An agreement between one or more contracting authorities and one or more economic operators, the purpose of which is to establish the terms governing contracts to be awarded during a given period, in particular with regard to price and, where appropriate, the quantity envisaged¹.

Managing authority: A national or regional body designated by the Member State to manage a rural-development programme (RDP).

Measure: An aid scheme for implementing a policy. Each measure sets out specific rules to be complied with by the projects or actions that can be financed. There are two main types of measures: investment measures and area-related aid.

Programming period: Multiannual framework to plan and implement EU policies such as rural-development policy.

Public procurement: Tender process to be followed by public bodies when purchasing goods, works and services above a certain price threshold. The aim is to obtain the best value offer by creating sufficient competition between suppliers and to ensure that contracts are awarded fairly, transparently and without discrimination. Directive 2004/18/EC and Directive 2004/17/EC set out the legal framework for public procurement which needs to be implemented by the national authorities.

Rural-development programme (RDP): A document prepared by a Member State or region, and approved by the Commission, to plan and monitor the implementation of the rural-development policy.

Rural operators: For the purpose of this report, farmers, forest holders and other rural businesses and stakeholders are referred to as ‘rural operators’, unless otherwise indicated.

Shared management: Method of implementation of the EU budget where implementation tasks are delegated to the Member States². To that end, Member State authorities designate bodies responsible for the management and control of EU funds.

1Article 1(5) of Directive 2004/18/EC of the European Parliament and of the Council of 31 March 2004 on the coordination of procedures for the award of public works contracts, public supply contracts and public service contracts (OJ L 134, 30.4.2004, p. 114).

2Article 59 of Regulation (EU, Euratom) No 966/2012 of the European Parliament and of the Council of 25 October 2012 on the financial rules applicable to the general budget of the Union and repealing Council Regulation (EC, Euratom) No 1605/2002 (OJ L 298, 26.10.2012, p. 1).

Executive summary

I

Knowledge-transfer and advisory activities have been upgraded to a high-level priority, both for the EU as a whole and for rural development in particular. The importance of vocational education and training, skills development, lifelong learning and knowledge transfer has been underlined by the European Parliament and successive EU agriculture commissioners, and is reflected in the EU’s legal framework.

II

This audit examined the provision of knowledge-transfer and advisory activities co-funded through the EU budget for Rural Development (EAFRD) and Member States’ own budgets for the period 2007-2013. Total EU funding allocated to these activities for that period amounts to 1.3 billion euro; co-financing will bring total public support to 2.2 billion euro.

III

In order to be effective and to have the desired impact the funds allocated to knowledge-transfer and advisory activities must be spent in response to identified needs, be provided at a reasonable cost and by appropriately qualified and experienced providers. The results should be measurable and assessed to provide information on what has actually been achieved. However, the Court’s audit revealed that, because of weaknesses in the Member States’ and the Commission’s management of the activities, this was not the case.

IV

The audit found that the delivery of the knowledge-transfer and advisory activities was not sufficient relative to the measures’ importance and the expected outcomes for such activities. Too often Member States relied on the providers’ proposals and any type of training was seen as ‘good’ and eligible to receive public support; only infrequently was there proper analysis made of whether the training could make a real impact.

V

The procedures in place did not always ensure fair and transparent competition to promote the selection of high-quality providers and knowledge-transfer and advisory activities at the best price.

VI

Cost-effective delivery of knowledge-transfer and advisory services is essential to protect the public purse in respect of both EU and national budgets. The Court found weaknesses in the checks Member States must carry out, both at application stage and at payment stage. There are indications that Member States paid too much for certain services and that related costs were insufficiently justified.

VII

The audit further revealed that a considerable number of similar services are financed by different EU funds (e.g. from the ESF as well as through the EAFRD). This implies the risk of double-financing and requires the duplication of costly management structures, which the Court considers to be uneconomic. For the 2014-2020 programming period complementarity between EU funds is stressed above the previous policy of demarcation; this implies a greater need for coordination to avoid the risks outlined above.

VIII

There was a lack of detailed evaluation of what was actually achieved with the public funds and only rather simplistic indicators such as number of persons trained or number of training days financed were collected. In addition the Court found problems with the reliability of the data that is collected.

IX

The weaknesses set out in this report mainly concern the Member State management of the measures. However, given the importance attached to the knowledge-transfer and advisory activities, the Commission’s role in guiding and supervising the Member States was also found to be inadequate and should be improved.

X

On the basis of these findings the Court makes a series of recommendations aimed at improving the situation for the current rural-development programme period (2014-2020) and ensuring that the ambitious objectives set for the activities stand a greater chance of being achieved.

XI

Specifically the Court recommends that:

οThe Member States should have procedures in place to analyse the knowledge and skills needs of rural operators, that go beyond the setting of broad themes, notably for the calls for proposals or tender periods. The Member States should ensure, as part of recurrent procedures, that relevant knowledge-transfer and advisory services are specified and selected on the basis of such an analysis and avoid the risk of the process becoming provider-driven.

οThe Commission should provide additional guidance on how Member States should carry out such recurrent analyses and encourage Member States to formulate these analyses in specific rather than general terms.

οThe Member States should ensure that support for the setting-up of new advisory services is granted only where there is a demonstrated deficit in relevant services in the area concerned and where the need for financing new staff, facilities and/or equipment exists.

οMember States should select the service providers to receive public funds through fair and transparent competition, regardless of whether they use calls for proposals or formal public procurement procedures. In particular, Member States should ensure that their selection or award criteria do not favour certain providers or types of providers.

οThe Commission should provide additional specific guidance on in-house delivery, subcontracting and the assessment of service delivery by consortiums. Furthermore, the Commission should adequately monitor the Member States’ procedures to ensure that the selection of knowledge-transfer and advisory activities is competitive, fair and transparent.

οWith regard to effective checks of cost-reasonableness, the Commission and Member States should implement the recommendations in the Court’s Special Report on this subject³. In particular for the knowledge-transfer and advisory activities, the Member States should assess the need to support activities, which are readily available on the market at a reasonable price. When this need is justified, Member States should ensure that the costs of the supported activities do not exceed the costs of similar activities offered by the market.

οThe Commission should build on the first steps taken to ensure complementarity between EU funds, for example through specific inter-service working groups, to carry out a thorough assessment of the complementarity between different EU funds proposed by the Member States for the 2014-2020 programming period. This assessment should result in a coordinated approach to supporting knowledge-transfer activities in order to mitigate the risk of double-funding and duplication of administration.

οMember States should establish feedback systems that use monitoring and evaluation information to improve upcoming calls for proposals or tendering procedures. These systems should require providers to furnish information not only on the participants’ satisfaction with the services, but also to test whether they have learned what they were supposed to.

οThe Commission should provide guidance on how Member States may execute such recurrent feedback procedures and monitor that Member States have them in place. The results of such feedback may also be used by evaluators to concentrate the evaluation work on the analyses of the activities at result and impact level.

οFinally, the Commission should increase, without delay, the risk profile of knowledge-transfer and advisory measures and enhance its supervision and management accordingly, to provide greater assurance that the Member States deliver the respective services effectively.

3See Special Report No 22/2014 ‘Achieving economy: keeping the costs of EU-financed rural development project grants under control’ (http://eca.europa.eu).

Introduction

Investment in knowledge and skills in the European Union

01

The importance of vocational education and training, skills development, lifelong learning and knowledge transfer has been voiced by numerous politicians and underlined in several high-level documents, including the Europe 2020 strategy for smart, sustainable and inclusive growth⁴.

‘Boosting the knowledge base of our farm sector is an important aspect of the rural-development programmes’⁵

Phil Hogan (European Commissioner for Agriculture and Rural Development)

‘The agriculture of the future has to be knowledge-based’⁶

Dacian Cioloș (former European Commissioner for Agriculture and Rural Development)

02

The Treaty on the Functioning of the European Union (TFEU) explicitly refers to ‘an effective coordination of efforts in the spheres of vocational training, of research and of the dissemination of agricultural knowledge’ in order for the objectives of the common agricultural policy (CAP)⁷ to be attained.

03

The EU legislation for the 2014-2020 programming period⁸, places particular emphasis on knowledge-transfer and advisory services as core activities for achieving sustainable development in rural areas. ‘Fostering knowledge transfer and innovation in agriculture, forestry, and rural areas’ is a horizontal priority in the field of rural development, i.e. it is considered to be an important tool to improve the implementation of other priorities and measures in the field of rural development.

Rural-development support for knowledge-transfer and advisory services

04

Financial support for knowledge-transfer and advisory services in rural areas is granted under the European Agricultural Fund for Rural Development (EAFRD). In the 2007-2013 programming period this support was granted through the following four measures⁹.

οMeasure 111 — Vocational training and information actions.

οMeasure 114 — Use of advisory services.

οMeasure 115 — Setting up of management, relief and advisory services.

οMeasure 331 — Training and information.

05

The aim of the first three measures was to enhance the competitiveness of farmers and forest holders. Measure 331 was directed at other rural businesses and stakeholders with the aim of both increasing their ability to contribute to diversifying the rural economy and improving the quality of life in rural areas. Measures 111 and 331 covered training courses, workshops, farm visits, coaching, etc. and are referred to as knowledge-transfer measures. Measures 114 and 115 were advisory measures that were intended to provide guidance to individuals or groups of farmers and forest holders. The individual knowledge-transfer and advisory activities were offered to the participants either for free or at a reduced fee.

Farmers, forest holders and other rural businesses and stakeholders are hereafter referred to as ‘rural operators’.

Box 1

Examples of knowledge-transfer and advisory services funded by the EAFRD

Training for farm women

© ZAMm/Gerald Lechner

Training for farmers

© ECA

Farmers’ working group (field visit)

© BMLFUW/LFI/weinfranz

06

Within the framework of rural-development policy, knowledge-transfer and advisory measures are subject to shared management by the Commission and the Member States. Rural- development programmes (RDPs) are proposed by the Member States and approved by the Commission. The Member States then select the projects to which funding is to be allocated, based on the programmes submitted. Figure 1 shows how funding is allocated to relevant projects under the regulatory framework.

Figure 1

Regulatory framework for allocating EAFRD funding to projects

Source: European Court of Auditors.

07

As far as the 2007-2013 period is concerned, approximately 1.3 billion euro, i.e. 1.4 % of EAFRD funding, was set aside for knowledge-transfer and advisory measures. By 31 December 2013 some 744 million euro, or 57 % of the planned EAFRD funding, had been spent by Member States. All measures are co-financed by the Member States (for details, see Table 1). However, as already indicated, the overall importance attached to knowledge-transfer and advisory measures far exceeds their value in purely monetary terms.

08

Member States do not necessarily implement all four EAFRD knowledge-transfer and advisory measures. An analysis of 2012 data received from the Commission revealed that, out of a total of 88 RDPs, measures 111, 114, 331 and 115 were not utilised in 19 %, 52 %, 66 % and 74 % of the RDPs respectively.

09

For the future the four individual measures have now been merged into two for the 2014-2020 rural-development programming period.

οMeasure 01 — Knowledge-transfer and information actions.

οMeasure 02 — Advisory services, farm management and farm relief services.

10

Planned EAFRD expenditure for measure 01 amounts to 1.16 billion euro and that for measure 02 to 742 million euro, i.e. 1.9 billion in all¹⁰. When combined with national co-financing the total public spend may exceed 4 billion euro. This constitutes an increase of around46 % compared to the expenditure planned in the previous programming period.

Table 1

Total public (EAFRD plus national co-financing) and EAFRD expenditure by measure

Measure | Description of the measure | Planned for 2007-2013 | Incurred between 2007 and 2013
EAFRD (euro) | Total public (euro) | EAFRD (euro) | Total public (euro)
111 | Vocational training and information actions | 947 013 157 | 1 599 928 390 | 556 561 212 | 974 023 090
114 | Use of advisory services | 197 958 811 | 338 640 512 | 98 320 876 | 169 488 261
115 | Setting-up of management, relief and advisory services | 50 307 450 | 76 068 793 | 26 540 966 | 43 419 212
331 | Training and information | 115 516 980 | 226 388 227 | 62 498 154 | 122 427 257
Knowledge-transfer and advisory measures | 1 310 796 398 | 2 241 025 922 | 743 921 208 | 1 309 357 820

Source: Data from the European Network for Rural Development (updated in February 2014).

11

Whereas participants in training and recipients of advice, as well as service providers, were potential beneficiaries of the measures in the 2007-2013 programming period, this has now changed. In the 2014-2020 period, the beneficiaries of financial support under measures 01 and 02 will be service providers, namely the entities providing advice or training to the final recipients (rural operators).

12

Although not compulsory, some Member States were already using public procurement procedures to select providers of knowledge-transfer and advisory services in the 2007-2013 programming period. Under the new legislation, public procurement rules must be applied before service contracts are concluded with providers of services under measure 02. In the case of measure 01, public procurement is not compulsory for selecting service providers.

Coordination with other EU support for knowledge-transfer and advisory services

13

EU support for knowledge-transfer and advisory services is not limited to the area of rural-development policy. At a very general level, and taking into account the mandate of each fund, training projects are supported via the European Social Fund (ESF) and the European Regional Development Fund (ERDF), the objective of which is to contribute to growth and jobs. These funds, along with the EAFRD and two other funds, are referred to as the European Structural and Investment Funds (ESI Funds) The legislative framework for 2014 to 2020 requires better coordination between the ESI Funds than was previously the case at both EU and Member State level. The new framework is illustrated in Figure 2.

14

The ESI Funds are a means of implementing the Europe 2020 strategy. In order to provide effective support, the ESI Funds should focus on contributing to 11 thematic objectives¹¹. The Fund-specific regulations provide a further breakdown of these objectives into investment priorities.

15

The thematic objective relevant to this audit is No 10, which comprises ‘investing in education, training and vocational training for skills and lifelong learning’. Further detailed investment priorities are set out in the specific regulations of the ESF, the EFRD and the EAFRD¹² (see Figure 2).

Figure 2

Legal framework for the 2014-2020 programming period

Source: European Court of Auditors.

4COM(2010) 2020 final of 3 March 2010 ‘EUROPE 2020 — A strategy for smart, sustainable and inclusive growth’.

5European Commission — press release: ‘A further 18 rural development programmes get green light’, Brussels, 13 February 2015 (http://europa.eu/rapid/press-release_IP-15-4424_en.htm).

6Statement on knowledge-based farming 12 October 2011 (https://www.youtube.com/watch?v=QqOqVed_atA).

7Article 41 of the TFEU.

8Regulation (EU) No 1305/2013 of the European Parliament and of the Council of 17 December 2013 on support for rural development by the European Agricultural Fund for Rural Development (EAFRD) and repealing Council Regulation (EC) No 1698/2005 (OJ L 347, 20.12.2013, p. 487).

9Measure 341 (Skills acquisition and animation with a view to preparing and implementing a local development strategy) and Measure 431 (Running the local action group, skills acquisition, animation) also include training, but are relevant only to Leader and similar groups under Axis 3 and are intended to help such groups implement their local strategies. They are therefore not considered any further in the context of this audit.

10The figures are based on information provided by the European Commission. They include both the figures provided in the 56 of the 118 RDPs approved and those still under discussion. The figures are therefore indicative at the present time (10/07/2015).

11As set out in Article 9 of Regulation (EU) No 1303/2013 of the European Parliament and of the Council of 17 December 2013 laying down common provisions on the European Regional Development Fund, the European Social Fund, the Cohesion Fund, the European Agricultural Fund for Rural Development and the European Maritime and Fisheries Fund and laying down general provisions on the European Regional Development Fund, the European Social Fund, the Cohesion Fund and the European Maritime and Fisheries Fund and repealing Council Regulation (EC) No 1083/2006 (OJ L 347, 20.12.2013, p. 320).

12The Fund-specific investment priorities are set out in the following documents. — ESF: Article 3 of Regulation (EU) No 1303/2013. — EFRD: Article 5 of Regulation (EU) No 1301/2013 of the European Parliament and of the Council of 17 December 2013 on the European Regional Development Fund and on specific provisions concerning the Investment for growth and jobs goal and repealing Regulation (EC) No 1080/2006 (OJ L 347, 20.12.2013, p. 289). — EAFRD: Article 5 of Regulation (EU) No 1305/2013.

Audit scope and approach

16

The Court conducted an audit of the procedures for the management and control of the spend on knowledge-transfer and advisory measures in the 2007-2013 programming period (see also paragraph 4) with a view to providing meaningful recommendations regarding the new programming period. This is so that Member States can take them into account when developing the implementation rules and procedures for their programmes and the Commission can use them in particular to improve its monitoring of the Member States’ management and control systems.

17

The audit focused on answering the following question:

Are there management and control systems in place to deliver rural-development knowledge-transfer and advisory measures effectively?

18

With regard to this, the Court’s report provides answers to the following sub-questions.

οAre there procedures in place to identify and select the most relevant knowledge-transfer and advisory services for rural operators?

οAre procedures in place to ensure cost-effective and coordinated delivery of knowledge-transfer and advisory services?

οAre procedures in place to monitor and evaluate the results of the knowledge-transfer and advisory activities?

19

The audit covered the 2007-2013 programming period and made the link, insofar as possible, with the next programming period, in which only entities providing training and advice to final recipients (rural operators) will benefit from the measures. The audit therefore focused on this type of beneficiary over the 2007-2013 period.

20

The audit was carried out between July 2014 and February 2015. Audit evidence was collected through documentary reviews, audit visits to the managing authorities and to service providers in five Member States¹³ (Spain (Galicia), Austria, Poland, Sweden and the United Kingdom (England)) and a survey¹⁴. The situation with regard to the various measures implemented in the Member States visited is illustrated in Annex I.

21

The expenditure planned and incurred in respect of knowledge-transfer and advisory measures was high in these Member States (see Figure 3). The Court’s choice of Member States was determined by its objective of covering all four measures (see paragraph 4) and having a balanced geographical spread. The audit visits and the survey covered the Member States responsible for more than 65 % of the expenditure of knowledge-transfer and advisory measures.

Figure 3

2007-2013 rural-development period - Knowledge-transfer and advisory measures: the 20 RDPs with the highest expenditure as at December 2012

Source: Data from the European Network for Rural Development.

13For the purposes of this report, ‘Member State’ may be taken to mean the country or a region or autonomous community in that country.

14The Court conducted a survey of 19 Member States, covering a sample of 25 RDPs in order to ascertain the reasons why Member States had not made full use of all the measures available.

Observations

Part I — The procedures to identify and select the most relevant knowledge-transfer and advisory services

22

In order to assess whether the most relevant knowledge-transfer and advisory activities were selected, the Court applied the following criteria.

οThe EAFRD should only be used for financing activities that help to attain the Member States’ specific objectives set for the measures in order to avoid arbitrary financing of any kind of training or advice to rural operators. A prerequisite for this is an analysis of the knowledge and skills needs of rural operators which is sufficiently specific to allow Member States to target the aid at concrete activities.

οThe EAFRD should finance only high-quality training and advice. High-quality providers are of key importance, and Member State authorities should therefore have relevant requirements and procedures in place.

οThe EAFRD should finance only high-quality training and advice. High-quality providers are of key importance, and Member State authorities should therefore have relevant requirements and procedures in place.

οProviders and activities should be selected through fair and transparent competition.

οThe Commission should approve only RDPs which clearly identify the knowledge and skills needs of rural operators. If necessary, the Commission should provide guidance to Member States to help them establish better needs assessment procedures.

Member States identified the skills needs of rural operators in such broad terms that almost any kind of activity could be funded

23

The selection and implementation of needs-driven activities requires Member States to analyse and determine rural operators’ individual preferences and identify the objective knowledge and skills needs of the rural economy and society before entering into the planning and/or selection of training activities. These needs may change in the short or medium term. The analytical and planning procedures should therefore be flexible enough to be able to respond to these changes.

24

Consequently, the Court would expect Member States to examine the knowledge and skills needs over a longer time frame for strategic purposes (for practical reasons, the programming period) and to have recurrent procedures for determining short- and medium-term needs. Following such analyses, the Member States should have sufficient information at their disposal to specify the services required for calls proposals or tenders. This is illustrated in Figure 4.

Figure 4

Recurrent needs analysis for the selection of rural-development knowledge-transfer and advisory activities

Source: European Court of Auditors.

25

The Court reviewed the national strategy plans and RDPs and other strategic documents, and also available studies, implementing rules and the service specifications related to the specific procurement of these types of activities. The Court found that the Member States audited had identified and defined the knowledge and skills needs of rural operators only in broad terms, which serves the strategic purpose. However, none of the Member States audited went on further to identify the most relevant activities (i.e. concrete training courses) within these areas. This implies the risk that generic and provider-driven activities, such as basic computer courses, are financed, which may not be based on identified needs (see Box 2).

Box 2

Examples of broad definitions of knowledge and skill needs not further specified for procurement purposes

United Kingdom (England)

The United Kingdom (England) used procedures at several management levels to identify the knowledge and skills needs of rural operators. However, this did not result in the detailed specification of knowledge-transfer services at procurement level. As a result, the authorities relied on the detailed proposals from the providers.

The RDP refers generally to skills deficits in the farm, food and forestry sectors and training as a priority to improve their competitiveness and to contribute to a better environment and healthy and prosperous communities. With support from an assembly of stakeholders, a broad framework of six groups of courses was developed, which constituted the basis for any tendering procedure conducted by the authorities. A series of mini-competitions was then launched to procure the actual activities. However, the Court found that the service specifications for the mini-competitions remained very general and the authorities relied on the providers’ delivery plans, which had been submitted with each offer.

Austria

Similarly in Austria, the RDP referred to relevant EU priorities and to the low level of formal education in the primary sector. The thematic areas for the measures concerned were defined in the relevant implementing rules¹⁵ but remained quite broad (e.g. ‘Improving the efficiency and the competitiveness of agricultural and forestry businesses’). The implementing rules set out that a dedicated conference, including stakeholders, would propose further specifications and priorities to the Ministry of Agriculture (BMLFUW), which had to be taken into account by the approving authorities for project selection. Despite ten meetings having taken place, no further specification was provided and, consequently, the project selection was made only on the basis of the broad thematic areas mentioned in the implementing rules.

26

The Member States audited involved providers and stakeholders in the process of defining priorities or thematic areas for knowledge-transfer and advisory services, either through meetings and working groups, or by giving them the opportunity to submit written proposals. Provider and stakeholder involvement should lead to services that better respond to the needs of the target groups. However, stakeholders may also have an interest in promoting services that are popular or already readily available, instead of addressing the actual knowledge and skills gaps of rural operators. Member States should therefore consider external opinions and proposals, but should eventually take an impartial decision on whether or not the activities proposed by the service providers actually address the real needs of rural operators and are easily available in the market at a reasonable price. The Court found no evidence that this was the case. As a result, the authorities relied on what is essentially a bottom-up procedure in which providers propose courses that should actually be procured, rather than providers responding to clearly defined needs expressed by the authorities themselves.

27

An insufficient analysis of the proposals from service providers implies not only the risk of financing irrelevant activities, but also the risk of duplicating training activities that are already readily available on the market and of paying above the market price (see paragraphs 54 to 55). This constitutes unnecessary public expenditure and may distort existing service markets.

28

Another example of weak assessments in this respect refers to measure 115. Through measure 115 the EAFRD should finance the costs incurred in setting up services (such as staff costs, costs for new premises, equipment, etc.)¹⁶. However, the Court found that measure 115 had sometimes been used to finance long-standing advisory bodies for the setting-up of services that were in fact not new. They concerned the inclusion of new subjects (deriving from new regulation requirements) in the advice. While in the United Kingdom (England), potential providers of new services had to demonstrate the deficit in the provision of appropriate advice in a specific area, this was not the case in Spain (Galicia) (see Box 3). In the other audited Member States measure 115 was not used.

Box 3

Spain (Galicia) — Funding the setting-up of ‘new’ services that were in fact not new

Spain (Galicia) grants funding for the setting-up of supposedly ‘new’ advisory services to providers that in fact already existed before the 2007-2013 RDP was implemented. The funding is for advice topics that are considered to be new, irrespective of whether or not there is a demonstrated need or demand from the rural operators and/or whether the existing structures are sufficient to absorb these new topics (i.e. whether providing advice on new topics requires the recruitment of new staff, renting additional facilities and/or buying new equipment).

In light of the fact that it reduced its staff at the end of the five-year support period, the service provider audited also seems to depend on EAFRD funding to cover its running costs.

Apart from financing existing service providers, there is also a risk of over-financing of providers’ staff costs. Staff costs are covered by measure 115, but they are also financed via measure 114, which subsidises the fees farmers pay to providers for advice.

29

The audit showed that, in spite of insufficient needs analysis, relevant knowledge-transfer and advisory actions are also being co-financed by the EAFRD. However, this was not the result of the identification and targeting of specific needs (see Box 4).

Box 4

Examples of relevant and innovative projects funded

Sweden — ‘Generationsskifte’

‘Generationsskifte’ (Generation shift) concerns the handover of a farm from the older to the younger generation. This is increasingly important, given the age trend in the population actively involved in farming (more that 50 % of farmers in Sweden and the EU are 55 and older¹⁷). The one-day training course examined in Sweden primarily provides information on economic and financial issues (particularly taxation) and matters of civil law. Psychological and emotional aspects, which are also significant in the handover of a family farm, are touched upon, but are dealt with in greater detail in coaching sessions that are also available.

Sweden — ‘Grön Arena’

‘Grön Arena’ is a project that aims to support farms that provide services relating to social care, education, health and recreation, e.g. providing on-farm activities for elderly people or children after school, ‘feel good activities’ in the countryside or with animals, etc. This project serves societal needs and provides possible income diversification for farmers. After being tested in one region, the project was expanded to the whole country.

Austria — Management course for women in rural areas

The objective of this management course was to encourage and enable women to take over responsible positions in rural business and societal groups, and thereby also serve as role models and encourage other women to become more active in rural development. The relevance of this course derives clearly from the Community Strategic Guidelines for the 2007-2013 programming period¹⁸, which set out the explicit objectives to encourage the entry of women into the labour market and to consider the training needs of women (for additional information on this course, see also Box 9).

Member States do not always verify the qualifications and experience of providers

30

With a view to ensuring the quality and effectiveness of the knowledge-transfer and advisory services that are on offer, bodies providing such services should be competent as regards both staff qualifications, experience and regular training, and administrative and technical capabilities.

31

With regard to advisory services, the Court found that the expected standards¹⁹ had been met by all Member States, which allowed farmers and forest holders to choose from a list of authorised providers. In Sweden, however, the requirements for qualification and experience concerned only advice on cross compliance and occupational safety, because other types of advice to farmers were offered under measure 111, for which no such requirements had been set.

32

In the absence of relevant EU rules, Sweden had not defined any requirements concerning knowledge-transfer measures. Other Member States audited had set out relevant eligibility or award criteria, although, despite some good practices encountered, the audit revealed that the procedures to assess the qualifications of providers were rather ineffective.

33

Austria and the United Kingdom (England) have strong requirements at provider level. In Austria, providers of knowledge-transfer services are only eligible if they hold a quality-management certificate that is valid for adult training institutions, and in the United Kingdom (England), interested service providers must prove their financial, technical or professional abilities at the first stage of a three-stage tendering procedure. However, in both cases the audit revealed that the authorities did not systematically check the qualification and experience of the firms that deliver courses and their trainers (see also Box 5).

Box 5

Austria: Quality-management certificates of training providers not checked

In order to check the eligibility criterion that providers of knowledge-transfer services hold a quality-management certificate, the Austrian authorities have included relevant points for checking in their checklists. However, the Court found that in four out of 30 cases, the authorities confirmed the respective point in their checklist even though no quality-management certificate existed.

The total public expenditure paid to these four providers amounted to 3.6 million euro (EAFRD: 1.8 million euro), of which approximately 3.1 million euro (EAFRD: 1.6 million euro) related to one provider alone. In October 2014 the same provider announced that it had now obtained the quality-management certificate, whereas it had been providing training since 2007.

34

Similar problems of insufficient checks on providers’ qualifications were found in Poland and Spain (Galicia). The authorities relied on the providers’ information, without checking it on the basis of supporting documents. As soon as the authorities in Spain (Galicia) delivered training courses themselves, the qualifications and experience of its staff and its technical and administrative capacities were taken for granted and no checks were made.

The selection of activities and providers suffers from a lack of fair and transparent competition

35

Competition encourages providers to improve the quality of their services, offer good prices and propose needs-driven services and should thus provide greater value for the EU taxpayer’s money. The basic principles of competition are fairness and transparency. Furthermore, as already stated in paragraph 24, the authorities responsible should have a clear idea of the services needed in order to be able to specify the services required in calls for either proposals or tenders, or establish pertinent criteria for the selection of relevant activities.

36

With regard to knowledge-transfer measures, the Member States audited have different ways of selecting providers and services, ranging from no competition to in-house provision by the authorities or affiliated bodies, single calls for proposals and the application of public procurement procedures. However, despite the wide range of procedures used, the Court found that, in general, it was the proposals of long-standing and well-established providers of knowledge-transfer services that were recurrently selected and received most of the EAFRD funding. The Court considers that this was because of a lack of competition and/or because of a lack of fairness and transparency.

37

In the following paragraphs the Court presents the procedures encountered in the different Member States audited and certain risks and findings related to these procedures:

No competitive selection of providers and services — Austria and Sweden

38

Project proposals may be sought or submitted at any time throughout the programming period, i.e. without any fixed periods or deadlines. As a consequence, a competitive assessment and selection of projects and providers is not possible and providers have no incentive to propose particularly needs-oriented and well-designed services, i.e. the risk that the bulk of the financial support is spent on provider-driven services is high.

39

At the time of the audit a group of public body training institutes in Austria had received 77.3 % of the EAFRD funding available for measure 111 and 54.3 % of that for measure 331. These training institutes have a competitive advantage over other providers, because no other providers had the opportunity to participate in official meetings organised by the authorities, where new courses and implementing rules were discussed. An evaluation report commissioned by the Austrian authorities found that the provinces [Bundesländer] approved, almost exclusively, training courses already offered by these institutes, with potentially high deadweight effects²⁰.

In-house provision of services – Spain (Galicia) and Sweden

40

In-house delivery of services was extensive in Spain (Galicia) (89 % of the budget for measure 111) and in Sweden (61 % of the budget for measure 111 and 35 % for measure 331). Private providers were not given the opportunity to carry out such projects, which were instead approved directly by the Member State authorities. The Court found no documented decision-making procedure in which a financial ceiling had been fixed and/or in-house delivery had been justified (e.g. the presence of in-house expertise and capacity, skills needs not covered by the market).

41

In Sweden, in-house projects were processed in the same way as projects carried out by external training providers, i.e. a different organisational unit under the same authority approved the activity. However, projects were not ranked and the Court found that several checks were only pro forma (see paragraph 53).

42

In-house projects were also outsourced, but the cost of the associated administrative activities (e.g. drafting a tender notice, checking incoming offers), which actually comprised part of the authorities’ normal duties, were charged to the project. Furthermore, the audit found instances of in-house projects delivery by the authorities that were significantly more expensive than those already available in the market without EAFRD support (see paragraph 55).

Calls for proposals — Spain (Galicia)

43

In Spain (Galicia), only 11 % of the budget for measure 111 was implemented through a competitive selection procedure, i.e. the annual calls for proposals. The proposals received were assessed on the basis of a system of merit points for different criteria, which were communicated to potential providers. However, the audit revealed that there was no objective method of awarding the merit points. This resulted in applications being granted either 0 points (when the information required in the application is missing) or the maximum possible points (when the related fields are filled in, even with non-specific information). As a result the selection procedure was not transparent and gave no assurance that the best providers and services were selected.

Public procurement — United Kingdom (England) and Poland

44

Public procurement or tendering is a competitive selection procedure that follows formalised rules. If correctly applied, it should provide reasonable assurance of fairness and transparency and good results in terms of the selection of providers and projects. However, the Court found several issues that give rise to concerns.

45

An issue encountered in the United Kingdom (England) relates to subcontracting. The multistage procurement procedure and size of the lots favour the major providers. Smaller providers can only become part of a delivery structure through subcontracting. The English authorities require tenderers to provide relevant information, e.g. their agreements with subcontractors, contract management, and the communication planned to ensure the desired outcome is achieved. On the other hand, they have no direct control over subcontracting and rely on the main provider as regards both the selection of subcontractors and the invoicing of services delivered. The latter resulted in excessive profits paid by the authorities to the main contractor, an aspect which is further developed in Part 2 of this report (see Box 8).

46

In Poland, the procurement procedure was jeopardised by a poorly-designed award system that favours well-established consortiums over new providers. The Court’s findings also show that it is not sufficient for the authorities to carry out a global assessment of the consortiums applying, but the capacities and roles of the individual members must also be assessed (see Box 6).

Box 6

The award system in Poland is poorly designed and implemented

Newly-established providers with less than two years’ experience can never achieve the minimum number of points for organisational experience and are a priori excluded from financial support, regardless of the quality of their proposed activities.

Furthermore, the Polish implementing regulation allows three points to be awarded for ‘sharing experience in organising training (…)’ in cases where an agricultural organisation is a member of a consortium that is tendering. The Court found that the authorities had awarded three points to such a consortium, though the agricultural organisation’s role in delivering a service was minor and did not even give rise to a payment. Analysis of several successful tenders revealed that these three points may be crucial in winning a competition.

Commission guidance to the Member States on knowledge-transfer and advisory measures was limited

47

The Commission approved the RDPs at the beginning of the programming period and did not require Member States to clearly identify the knowledge and skills needs of rural operators in their programming documents or describe their selection procedures. Furthermore, the Commission provided no appropriate guidance in the 2007-2013 programming period that might have helped Member States establish better procedures. As regards the new programming period, the Commission prepared more extensive guidance on the measures, without going into detail on how to identify knowledge and skills needs (e.g. through flexible analytical and planning procedures). Based on the programming documents that were available at the time of the audit, the Court did not find evidence that Member States have better analysed the knowledge and skills needs (see paragraphs 23 to 29).

48

The Commission did not issue guidelines to the Member States, in addition to the existing legal provision for advisory services in 2007-2013, on ensuring that providers of knowledge-transfer and advisory services are reliable, appropriately qualified and experienced, and have the necessary administrative and technical capabilities. With regard to the 2014-2020 programming period, certain improvements may materialise, in particular because the EU provisions concerning the providers’ capacities and qualifications have been extended to cover knowledge-transfer services²¹ (see paragraphs 30 to 34).

49

For the 2007-2013 programming period, the Commission was not always aware of the problems related to the procedures in place for selecting providers and services and thus, could not require the Member States to apply corrective actions (see paragraphs 35 to 46).

50

As regards the 2014-2020 programming period, the legislation requires the procedure for selecting providers of advisory services to be governed by public procurement law²². There is therefore a risk that the procurement results in only one winner per geographical and/or thematic area and reduces the choice for rural operators. This risk could be mitigated by the fact that Member States may sign framework agreements with several providers²³, thus allowing rural operators to choose from amongst a number of service provider(s).

51

As regards the selection of knowledge-transfer services and providers, there is no explicit provision in the relevant legislation, but Member States ‘can choose between various procedures according to their national law, including tenders, (…) and the in-house procedure’²⁴. In the absence of further specific guidance by the Commission, the problems identified by the Court in respect of the 2007-2013 programming period, such as in-house delivery without transparent decision-making procedures or justification, subcontracting that is not subject to administrative checks and the same standards as normal contracts, and award procedures that are not transparent and/or fair, may well persist.

Part II — procedures to ensure cost-effective and coordinated delivery of knowledge-transfer and advisory services

Member States approved training activities without carrying out effective checks that their costs were reasonable …

52

Ensuring that the cost of rural-development grants is kept under control is a key element of the ‘sound financial management’ required by the EU’s financial regulations, whereby all managers of the EU budget are expected to apply the principle of economy, i.e. support the right things at the right price. This should be done by checking costs ex ante, i.e. prior to the grant approval, and before the payment.

53

As the Court highlighted in its Special Report No 22/2014²⁵, Member State authorities do not do enough to ensure that the costs underlying the grants approved for rural development are reasonable. In the case of knowledge-transfer measures, project applications often lack sufficient detail to allow meaningful assessment of costs in relation to the activities planned. Nevertheless, the authorities in Austria and Sweden approved such general project applications and indicated on their checklists that the reasonableness of costs had indeed been checked. Other examples illustrating weaknesses in ex ante checks in Member States are given in Box 7.

… and there are indications that they sometimes paid too much for training or advice services which were insufficiently justified

54

As described above, Member State authorities are required to check that the grants they approve are based on reasonable costs. However, it is not unusual for the costs incurred to differ from the costs approved at application stage: both the specifications and prices may change. Member State authorities therefore need rules and systems in place to detect significant deviations from approved project costs and to check that the revised costs are still reasonable before paying the grant.

Box 7

Examples illustrating weaknesses in ex ante checks on reasonableness of costs

United Kingdom (England)

The authorities received only one offer from a training provider in response to a call for tenders. They accepted the offer without comparing the prices with historical costs or other benchmarks. The Court found that the price per trainee claimed from the authorities was between twice and eight times as high as the standard price advertised in the provider’s leaflet.

Spain (Galicia)

An increase in the budget of a training project was accepted without any supporting documents having been requested to justify the proposed increase.

Austria

The budget of one big project was increased twice, resulting in an overall grant of 2.7 million euro. The authorities approved the budget increase with the argument that the submitted cost calculations were based on the actual payments made for similar activities in the previous years. However, the Court found no evidence of appropriate checks.

55

The Court found several weaknesses in procedures at the payment stage which resulted in incorrect payments, overpayment or uncertainty as to the correctness of the amount paid, as illustrated below.

οCompulsory training for farmers, e.g. on the handling of phytosanitary products and animal welfare, delivered directly by the regional authorities for 2 680 and 2 030 euro respectively, was available in the area at a much lower price (150 and 200 euro respectively) (Spain — Galicia).

οPayments were not adjusted on the basis of participants’ actual attendance of training, even though the project was to be charged on this basis (Spain — Galicia).

οTraining-attendance lists were unreliable: they were either pre-signed, showed signs of having been changed by a person other than the participant, or could not be interpreted because of the different signs and codes used (Poland, Spain — Galicia).

οThe costs charged to the Member State authority for training courses were very much higher than the amounts actually paid to the subcontractors (United Kingdom — England, see Box 8).

οIn the case of projects carried out by the Member State authorities themselves, it was not possible to establish the link between the costs of an individual training activity and the corresponding payment claim submitted to the paying agency within a reasonable time frame (Sweden).

οThere was either no obligation to keep time sheets, or time-recording rules were unclear or did not allow for correct accounting of actual time spent on training activities by either providers or the public authorities themselves (Sweden, Poland, Spain). In the case of public authorities, this means that there is a risk that activities that fall within the normal remit of the authorities will be charged to projects co-financed by the EAFRD (see also paragraph 42).

οSome advisory services were fully financed (Sweden), contrary to the legal provisions, which limit the financing rate to 80 %.

οClear agreements with external trainers or speakers do not always exist and their invoices are not sufficiently detailed to allow both the correctness and reasonableness of the costs and the link with the project to be established (Austria, Sweden).

Box 8

Excessively high profits in the United Kingdom (England)

In the United Kingdom (England), despite the application of a public procurement procedure, the Member State authorities paid excessively high prices for several training courses. A payment claim for three courses submitted by one provider showed costs that were 13.8 times higher than the direct training costs this provider had paid to its subcontractors. The lack of value for money of this contract had also been pointed out by an inspector from the paying agency (prior to the Court’s audit), but with no consequence as far as the payment made was concerned. As a consequence, the United Kingdom authorities envisage putting in place a more rigorous delivery plan monitoring regime.

1 Few other costs would be advertising and administrative costs.

2 According to payment claim.

Source: European Court of Auditors.

The Commission’s audit coverage did not provide sufficient information on the weaknesses found by the Court

56

The Commission, through its audits, must monitor the management and control systems established in the Member States²⁶. Knowledge-transfer and advisory services are considered to be of high importance to the further development of rural society and its economy. However the Commission’s audit coverage to monitor the management of these measures did not provide sufficient relevant information on the Member State implementation and control procedures and did not detect or remedy any of the weaknesses highlighted above.

57

The Commission has not conducted any specific audits of knowledge-transfer and advisory measures in 2007-2013 on the grounds that the amount of funding involved is relatively low. In 2010, one wider audit covered measure 111 as part of its scope. It revealed several weaknesses in the management and control system (key controls) and gave rise to a financial correction²⁷.

EU funds and other sources of financing are not coordinated to provide efficient delivery of knowledge-transfer and advisory services

58

As already indicated in paragraph 13, it is possible for knowledge-transfer and advisory activities for rural operators to be financed by sources other than the EAFRD. During the audit visits and on the basis of its survey (see paragraph 20) the Court found that other EU funds (in particular the ESF), as well as national and regional funding sources, provide such financing.

59

In order to avoid double-financing of the same activities by different funds, Member States are required to draw demarcation lines between the various funds in their RDPs²⁸ and to implement effective management procedures²⁹.

60

Demarcation between the different EU funds was not only the responsibility of the Member States, however. It should have started at Commission level with proper coordination between the Directorates-General responsible for the different funds. The Commission should have assessed the appropriateness of the demarcation lines and approved them as part of the national strategy plans for rural development, issued guidance and checked measure-implementation in the Member States.

61

For the 2007-2013 programming period the Commission did not issue any guidance on demarcation or avoiding the risk of double-funding and approved the Member States’ RDPs without requiring more detail on the mechanisms applied to avoid double-funding. The Court identified several cases where similar activities had been funded by different funds and where the demarcation between the funds had been vague. The obvious disadvantage of this situation is that management structures may be duplicated. The greater the lack of clarity of demarcation when combined with management procedures that do not work well, the greater the risk of double-financing.

62

Nonetheless, the Court found that some Member States employed good practices to mitigate the risk of double-financing, such as the stamping of invoices (Spain (Galicia), Austria, Poland), declarations by beneficiaries (Spain (Galicia)) and consultations between competent authorities and mandatory checks at the application stage (Sweden).

63

As regards demarcation between the different EU Funds, the Court checked the Member States’ national strategy plans and RDPs for the 2007-2013 programming period and found the following weaknesses.

οIn Spain (Galicia) training actions outside ‘normal education’ in rural areas could be supported by either the EAFRD, ESF or ERDF. No relevant demarcation criteria had been defined, however. Examples were found of both information technology (IT) courses in rural areas and courses on cooperative entrepreneurship in rural areas that had been financed by either the EAFRD (measure 111), the ERDF or the ESF.

οIn the United Kingdom (England), ESF-funded courses on leadership and management, IT and office skills were open to all businesses, including rural enterprises, even though the demarcation criteria seemed to exclude them.

οIn Poland similar training activities had been financed under the ESF Human Capital Operational Programme and measure 111.

οIn Belgium (Wallonia) similar activities could be financed under measure 331 and the ESF. This measure was started with rural-development funding but was then discontinued. The Belgian authorities explained that training providers preferred ESF financing, because the ESF cost-reimbursement rules are, in their opinion, less stringent that the EAFRD rules.

64

In Spain (Galicia), the audit revealed that the system had run the risk of double-funding by measure 115 and the ESF in that staff costs for newly recruited staff were covered by both Funds. A case was detected where a service provider had been a beneficiary of both the ESF (122 400 euro) in connection with job creation and measure 115 (124 700 euro).

65

In the new 2014-2020 programming period, the relevant regulations highlight the importance of greater coherence between the EU Funds. The new approach focuses less on demarcation and more on complementarity. There was a lack of operational guidance from the Commission, however. The Court has analysed the partnership agreements and it is not clear in the case of several Member States how the coordination weaknesses that occurred in the 2007-2013 programming period will be overcome in the new period.

Part III — procedures to monitor and evaluate the results of the knowledge-transfer and advisory activities

66

A key element in ensuring that the EU budget is well spent is monitoring and evaluation³⁰. As illustrated in Figure 4 and paragraph 1, monitoring and evaluation can provide input for short-term analyses and the specifications of the programme within the 7-year programming period and when designing longer term programmes for the future.

67

In the absence of a model used by the Commission for the purpose, the Court makes reference to a recognised model, the ‘Kirkpatrick four levels’ model, for the evaluation of training programmes. The principles of this model are applicable to any training programme (see Annex II). Kirkpatrick’s four levels are designed as a sequence of ways to evaluate training programmes. When moving from one level to the next, the process becomes more difficult and time-consuming, but each subsequent level also provides more valuable information³¹.

68

The information obtained through levels 1 and 2 (reaction and learning) is primarily useful to identify issues linked to the quality of the service provided, the scope of the activity, etc. This information can serve as an input to refine the specifications of subsequent actions, or to improve the selection of providers in subsequent calls for tender or calls for proposals. Levels 3 and 4 (behaviour and results) can be used to assess the impacts and outcomes of the actions and serve more strategic purposes such as re-assessing the needs, identifying new needs, etc.

Direct feedback from training participants is often collected by training providers but seldom used by Member State authorities to improve subsequent actions

69

The Court found that, in the case of EAFRD knowledge-transfer measures, most of the Member States collect immediate feedback from training participants but they do not use it to the extent possible for assessing the quality of the training or for modifying courses. Few Member States monitor information about actual learning (level 2).

70

In most of the Member States audited, training participants fill in feedback forms (also called ‘happiness sheets’ or ‘satisfaction sheets’) immediately after having finished training. This is mostly done on the initiative of the service provider but some Member State authorities have developed guidelines or standardised feedback forms (see also Box 9).

71

In many cases however, the Member State authorities do not fully use the information collected. In Poland, consolidated results of only one qualitative question are forwarded to the authorities in the final report at payment stage. In Austria, neither the feedback forms nor their analyses are forwarded to the authorities. In Spain (Galicia), in the case of training actions managed through direct intervention, the central services of the authorities do not receive the completed forms from the project coordinators.

Box 9

Examples of good practice in collecting information for project evaluation

The Polish authorities provide standardised feedback forms to the service providers containing 11 qualitative questions.

The authorities in the United Kingdom (England) allow providers to use their own evaluation forms but they ensure, through guidelines, that the forms correspond to the minimum requirements. Three or more options (e.g. very satisfied, satisfied or dissatisfied) are given to the trainees with regard to six different elements; pace and content of the course, materials, venue and facilities, level of support by the tutor, overall quality and skills relevance. The participants are also required to confirm the relevance of the skills and the likelihood that they will be put into practice. Some space is available for comments and suggestions for improvement.

Still in the United Kingdom (England), the authorities conduct a telephone survey of randomly selected trainees. The primary aim of the telephone survey is to check attendance, but it is also a useful mechanism to obtain feedback.

In Austria, the Court saw a good example, where evaluation was integrated in the concept of an innovative management course for women who are active in rural development. The course included an assessment of qualitative elements per module, such as the learning atmosphere, course content and practical relevance, and the quality of the trainers and the learning documents. The trainers were required to do a self-assessment, which was compared with the participants’ evaluation, and finally the provider had a feedback discussion with the Austrian authorities. The participants were also required to present an examination document, on which they worked continuously during the course, and the provider envisages conducting a survey after 1-2 years to enquire whether the participants are still using the knowledge and skills obtained. Participant interviews, which confirm the positive results of the evaluation, are also provided on the internet.

72

Most of the Member State authorities visited during the audit do not monitor to what extent participants actually acquire the intended knowledge, skills, and attitudes based on their participation in the learning event (equivalent to Kirkpatrick level 2 (learning)). For example, in Spain the authorities did not follow up how many of the training participants actually took the exam for which the preparatory course was offered. In the United Kingdom (England), the authorities do not monitor the pass rates for certificate courses and the Court found examples where even the main contractor did not know the pass rates of courses given by its subcontractor. In Poland, there are no tests or interviews before and after the training event to evaluate what the trainee has learned. The certificate participants receive at the end of the training event is merely a confirmation of their participation and not a confirmation that they have successfully passed a test.

73

In Austria, on the contrary, in the case of certificate courses, participants are required to pass an examination. The providers have to monitor and evaluate the courses in accordance with certain criteria and submit a summary of the results to the national chamber of agriculture. According to the data provided, more than 850 certificates are issued per year, which represents a success rate of approximately 95 %.

Some examples of good practice exist, with Member States making efforts to collect useful information regarding the impact of knowledge-transfer and advisory activities

74

As set out in the previous section, Member States do not fully exploit the possibilities of project evaluation. Some Member States commissioned surveys or evaluation studies at the level of training programmes with the objective of obtaining information on behavioural changes (Kirkpatrick level 3) and impact (Kirkpatrick level 4). These studies must start by collecting basic evaluation data, sometimes a long time after the courses have ended, which makes it even more difficult and time-consuming to draw meaningful conclusions.

75

In Sweden, regular country-wide postal surveys carried out by the Swedish statistical service provide feedback on all four evaluation levels. The latest survey was carried out in spring 2010 on 8000 individuals who had participated in rural-development knowledge-transfer or advisory activities. Similar studies were carried out in 1999 and 2006 and a new survey is planned for 2015.³²

76

In Austria, the authorities commissioned a specific evaluation of measures 111 and 331. The authors of the study use the following four methods: analysis of national and international studies, reports and other documents; online survey of rural operators (both participants and non-participants in training measures), training providers and officials; ‘dialogue interviews’ of 1-2 hours, and ‘evaluation cafés’. The authors provide a lot of detailed information, e.g. on the estimated benefits of the training measures in general and of different types of actions, or the obstacles to participation. On the other hand, they also point to weaknesses in the Austrian system and suggest the introduction of competitive project selection by using specific success indicators per project³³.

77

Data collected through external studies in the United Kingdom (England) and Sweden also allow conclusions to be drawn on the benefits of training and advice (see Box 10).

Box 10

Examples of studies on the impact of RD-financed knowledge-transfer and advice

United Kingdom (England)

In 2013, the Countryside and Community Research Institute (University of Gloucestershire) completed a report³⁴ in which it attempted to assess the social return on investment (SROI) of axes 1 and 3 of the RDP for England by using four case study areas.

The report concluded, inter alia, that to a large extent the benefits observed (e.g. improved on-farm sustainability, engagement of the livestock industry in relation to animal health and skills, improvements to soil and land management practices, development of local capacity in value added in Axis 1) reflect the levels of investment in training and advice.

Sweden

A pesticide monitoring study in a small agricultural catchment in Southern Sweden (Vemmenhög river), started in 1990, shows that the occurrence of pesticide residues in watercourses decreased by 90 %, despite almost no changes being made to the amounts of crop protection chemicals used. Analysis shows that this is largely the result of increased awareness amongst the farmers of safe handling of pesticides following the start of advisory services in that domain in 1994.

78

In Sweden, since 2001, an initiative called Greppa Näringen (Focus on Nutrients)³⁵ has been providing free environmental advice to farms of more than 50 hectares and/or more than 25 animal units. Funding is obtained through the EAFRD and re-invested environmental taxes. The initiative offers, inter alia, courses for farmers and for advisors and provides tools for calculating nutrient balances. Evaluation of the farm advice is integrated through follow-up visits and quarterly surveys. The farmers’ satisfaction with the advisory visits is quite high and, an even more important indication of success; the nitrogen and phosphorus surpluses and ammonia emissions per farm have been reduced.

The data collected to show the aggregated results of knowledge-transfer and advisory services at EU level are not reliable

79

The previous sections refer to monitoring and evaluation mainly as a tool to improve the specifications and implementation of knowledge-transfer and advisory services at Member State level. At EU level, the rural-development regulation established a Common Monitoring and Evaluation Framework (CMEF) for the 2007–2013 period³⁶. This was a new system, introduced in 2007, with common indicators intended to capture the progress towards rural-development objectives in a way that allows each of the RDPs to be compared and the data to be aggregated. A Commission regulation³⁷ sets more specific requirements and lists common baseline, output, result and impact indicators. As common indicators cannot fully capture all the effects of individual RDPs, Member States could also set a limited number of additional indicators. However, none of the Member States audited defined additional indicators for knowledge-transfer and advisory measures.

80

The Court’s audit confirmed the serious weaknesses in the availability and quality of the monitoring and evaluation information relating to spending on rural development, as highlighted in Special Report No 12/2013³⁸.

81

In the course of this audit, the Court found the following weaknesses.

οIn Poland, the calculation method used was not in line with the CMEF guidelines, which resulted in huge differences for the number of actual training days³⁹.

οThe United Kingdom (England) and Spain (Galicia) could not provide the requested data in full and Austria allocated projects to the wrong CMEF types (resulting in an unreliable number of participants and training days for different types of activities).

82

On a positive note, in Sweden, providers and authorities are required to register the participants of each individual training or advisory activity online in a common database. Financial data and information on administrative project checks are recorded in another database. For the purpose of reporting monitoring data to the Commission, both databases must be linked through the project number. However, the separation into two databases means no financial information at the level of individual training or advisory activities can be provided, e.g. costs per participant and/or training day. Moreover, the links between the databases are problematic and the Court observed problems concerning data reliability.

83

In addition, the analysis of aggregated data available from the Commission is evidently misleading for several Member States (see Box 11).

84

The Commission coordinates the CMEF, as Commission staff review the Annual Progress Reports and aggregate the monitoring data. They may participate as observers in the Member States’ monitoring committees and hold an annual meeting with the managing authorities to examine the main results of the previous year for each RDP. The Commission is also required to produce a number of reports on the implementation and achievements of the rural-development policy.

85

However, the Commission is not always aware of the anomalies and data collection problems in the Member States and has published aggregated data without ensuring their consistency. In the absence of reliable data, the Commission is, therefore, not in a position to assess whether the EAFRD funds for knowledge-transfer and advisory actions are well spent or achieving the targets set for the actions.

Box 11

Examples of inaccurate CMEF indicator data

Measure 111

The Court found that the CMEF indicated unrealistically high figures for Denmark. Output data indicated, for example, 7 770 euro in EAFRD expenditure per training participant and 123 708 euro per training day in the 2007-2011 period. The data for the period 2007-2012 still shows comparable values.

Similarly, output data for France (Martinique, Guadeloupe and Reunion) for the 2007-2012 period resulted in unrealistically high values for EAFRD expenditure per training day or per participant (e.g. 382 333 euro/training day in Martinique or 31 498 euro/participant in Guadeloupe) due to the fact that the monetary amounts were encoded for the complete measure whereas the training days or participants were encoded by sub-measure.

Measure 115

Output data indicated that 373 new advisory services were supported through measure 115 in Spain (i.e. more than 50 % of all new advisory services supported in the EU). However, it appeared that all the annual payments to the same providers were counted as ‘new services’ and therefore the figure does not correspond to the actual number of new advisory services set up with the support of the measure, which is much lower.

86

The legislation provides for a new common monitoring and evaluation system (CMES) concerning rural development in the 2014-2020 period⁴⁰. It incorporates a number of changes with respect to the previous one, such as the collection of less data and intermediate evaluation according to the needs of the Member States. The new output indicators are:

οmeasure 01: number of operations supported, number of training days given, number of participants in training.

οmeasure 02: number of operations supported, number of beneficiaries advised, number of advisors trained⁴¹.

87

For measure 01, reporting will be made per type of operation, which should avoid the problem that activities with a small number of participants are reported under the same indicator as events with a large number of participants. With regard to knowledge-transfer and advisory measures, the common evaluation question reads as follows: ‘To what extent have RDP interventions supported lifelong learning and vocational training in the agriculture and forestry sectors?’⁴²

88

Evidently, the Court could not assess at the time of the audit to what extent the new CMES will remedy the weaknesses highlighted above. However, assessing the success of the measures only on the basis of the number of participants or the number of training days given or on the basis of a broad evaluation question will not provide detailed feedback on the success or otherwise of the funds allocated to the measures.

15Sonderrichtlinie des Bundesministers für Land- und Forstwirtschaft, Umwelt und Wasserwirtschaft zur Umsetzung von Maßnahmen im Rahmen des Österreichischen Programms für die Entwicklung des ländlichen Raums 2007 – 2013 ‘Sonstige Maßnahmen’.

16Article 25 of Council Regulation (EC) No 1698/2005 of 20 September 2005 on support for rural development by the European Agricultural Fund for Rural Development (EAFRD) (OJ L 277, 21.10.2005, p. 1): ‘support (…) shall be granted in order to cover costs arising from the setting up of farm management, farm relief and farm advisory services as well as forestry advisory services and shall be degressive over a maximum period of five years from setting up’.

17European Commission (2015): Member States factsheets (Sweden, p. 13, and European Union, p. 17) (http://ec.europa.eu/agriculture/statistics/factsheets/index_en.htm).

18Council Decision 2006/144/EC of 20 February 2006 on Community strategic guidelines for rural development (programming period 2007 to 2013) (OJ L 55, 25.2.2006, p. 20).

19See Article 15(2) of Commission Regulation (EC) No 1974/2006 of 15 December 2006 laying down detailed rules for the application of Council Regulation (EC) No 1698/2005 on support for rural development by EAFRD (OJ L 368, 23.12.2006, p. 15):
‘2. The authorities and bodies selected to provide advisory services to farmers shall have appropriate resources in the form of qualified staff, administrative and technical facilities and advisory experience and reliability with respect to the requirements, conditions and standards referred to in points (a) and (b) of the second subparagraph of Article 24(1) of Regulation (EC) No 1698/2005’.

20Mandl, C. and Kuttner, T., ‘Bildungsevaluierung Ländliche Entwicklung LE 07-13 — Endbericht’, Vienna, 2013, p. 107.
Deadweight effect: A situation where a subsidised activity or project would have been wholly or partly undertaken without the grant aid.

21Articles 14(3) and 15(3) of Regulation (EU) No 1305/2013.

22Article 15(3) of Regulation (EU) No 1305/2013.

23In accordance with Article 32 of Directive 2004/18/EC.

24Measure fiche ‘Knowledge transfer and information actions’, October 2014 version.

25See Special Report No 22/2014 ‘Achieving economy: keeping the costs of EU-financed rural development project grants under control’ (http://eca.europa.eu).

26Article 59(2) of Regulation (EU, EURATOM) No 966/2012.

27Financial corrections may be made by the Commission where expenditure, transactions or practices are irregular and have not been corrected by the Member State. Financial corrections may also be made where there is a serious deficiency in the management and control system which has put at risk the EU contribution already paid to a programme, or where a Member State has not investigated irregularities and made the corrections required.

28Article 60 of Regulation (EC) No 1698/2005.

29Article 24(5) of Commission Regulation (EC) No 65/2011 of 27 January 2011 laying down detailed rules for the implementation of Council Regulation (EC) No 1698/2005, as regards the implementation of control procedures as well as cross-compliance in respect of rural-development support measures (OJ L 25, 28.1.2011, p. 8).

30Monitoring is the regular examination of expenditure, outputs and results, which provides up-to-date information on whether programmes are progressing as intended. Evaluation is a periodic collection and analysis of evidence — including the information produced for monitoring — to form conclusions on the effectiveness and efficiency of a programme or policy.

31Kirkpatrick, Donald L., Kirkpatrick, James D., Evaluating training programs: The four levels, 3rd edition, 2006, p. 21.

32Jordbruksverket: Kompetensutveckling inom landsbygdsprogrammet — Rapport från en statistisk undersökning genomförd. Rapport 2010:30.
Jordbruksverket: Analys av kompetensutvecklingen inom landsbygdsprogrammet — Fördjupning av rapport 2010:30. Rapport 2011:39.

33Mandl, C. and Kuttner, T., ‘Bildungsevaluierung Ländliche Entwicklung LE 07-13 — Endbericht’, Vienna, 2013.

34An assessment of the social return on investment of axes 1 and 3 of the rural development programme for England, Final report, by the Countryside and Community Research Institute on behalf of the Department for Environment Food and Rural Affairs, 1 November 2013.

35http://www.greppa.nu

36Article 80 of Regulation (EC) No 1698/2005.

37Regulation (EC) No 1974/2006. The complete list of common indicators is given in Annex VIII.

38Special Report No 12/2013 ‘Can the Commission and the Member States show that the EU budget allocated to rural development is well spent?’ (http://www.eca.europa.eu).

39According to the CMEF guidelines the indicator ‘Number of training days received’ is to be calculated as the sum of number of hours of training received by all applicants, divided by eight [hours]. The Polish authorities, however, simply indicated in their 2013 CMEF statistics the number of training days that took place.

40Articles 14 to 16 of Commission Implementing Regulation (EU) No 808/2014 of 17 July 2014 laying down rules for the application of Regulation (EU) No 1305/2013 of the European Parliament and of the Council on support for rural development by the European Agricultural Fund for Rural Development (EAFRD) (OJ L 227, 31.7.2014, p. 18).

41Output indicators 11-14; see Annex IV of Commission Implementing Regulation (EU) No 808/2014.

42Annex V of Commission Implementing Regulation (EU) No 808/2014.

Conclusions and recommendations

89

Knowledge-transfer and advisory activities have been upgraded to a high-level priority for rural development. The importance of vocational education and training, skills development, lifelong learning and knowledge transfer has been underlined by the European Parliament and EU agriculture Commissioners, and is reflected in the EU’s legal framework.

90

The Court’s audit posed the following question:

Are there management and control systems in place to deliver rural-development knowledge-transfer and advisory measures effectively?

91

In order to be effective and to have the desired impact the funds allocated to training and knowledge-transfer measures must be spent in response to identified knowledge and skills needs, which may change during the course of a programming period. The activities selected to address these needs must be provided by appropriately qualified and experienced providers, in a cost-effective manner and the results should be measurable and assessed to provide information on what has actually been achieved. However, the Court’s audit revealed that, because of weaknesses in the Member States’ management and control systems and the Commission’s supervision of the measures, this was not the case.

92

Overall, the management and control of the knowledge-transfer and advisory activities was not sufficient relative to the measures’ importance and the expected outcome for such activities. Too often any type of training was seen as ‘good’ and eligible to receive public support; only infrequently was a proper analysis made and training activities targeted at areas where they could make a real impact. The weaknesses set out in this report mainly concern the Member States‘ management of the measures. However, the Commission’s role in guiding and supervising the Member States should not be underestimated.

93

The Court found that the Member States’ procedures for the identification and selection of relevant service providers were flawed. Although they implemented procedures to identify the knowledge and skills needs of rural operators, the identified needs were defined in terms which were too broad. They did not go further to identify the most relevant activities (i.e. concrete training courses) within these areas and relied mainly on the providers’ proposals. In the Court’s view, provider and stakeholder involvement can potentially lead to better measure delivery, but the ultimate responsibility remains with the Member State authorities (paragraphs 23 to 27).

94

With specific regard to EAFRD support for the setting-up of advisory services (former measure 115), support had sometimes been used to finance long-standing advisory bodies for the setting-up of services that were in fact not new (paragraph 28 and Box 3).

Recommendation 1
Analysis of needs

(a)The Member States should have procedures in place to analyse the knowledge and skills needs of rural operators that go beyond the setting of broad themes, notably for the calls for proposals or tender periods. The Member States should ensure, as part of recurrent procedures (see paragraph 24 and Figure 4), that relevant knowledge-transfer and advisory services are specified and selected on the basis of such an analysis and avoid the risk of the process becoming provider-driven.

(b)The Commission should provide additional guidance on how Member States should carry out such recurrent analyses and encourage Member States to formulate these analyses in specific rather than general terms.

(c)With specific regard to the setting-up of advisory services (former measure 115), the Member States should ensure that for the 2014-2020 programming period EAFRD support should only be granted where there is a demonstrated deficit in relevant services in the area concerned and where the need for financing new staff, facilities and/or equipment exists.

(d)The Commission should provide, without delay, further specific guidance to the Member States concerning the setting-up of advisory services, and monitor the Member States’ subsequent compliance.

95

In the 2007-2013 programming period Member States were legally required to ensure that only high-quality providers of advisory services were selected⁴³. Such a provision did not exist for training providers and, with regard to this, the audit revealed several weaknesses in the Member States’ management. The introduction of legal requirements in the new programming period concerning appropriate capacities of training providers may help to remedy these problems⁴⁴ (paragraphs 30 to 34).

96

The Court found, however, that the Member States did not always ensure fair and transparent competition when selecting the knowledge-transfer activities and service providers. Merely referring to public procurement does not ensure competitive selection, as there is, in particular, no clear guidance provided with regard to in-house delivery, subcontracting and the assessment of service delivery by consortiums, despite these practices being widely used. Most of the Member States audited did not ensure fair or transparent competition when selecting the service providers (paragraphs 35 to 46).

Recommendation 2
Selection of providers

(a)For the new 2014-2020 programming period, Member States should select knowledge-transfer activities and service providers to receive public funds through fair and transparent competition, regardless of whether they use calls for proposals or formal public procurement. In particular, Member States should improve their assessment of the qualifications and experience of training providers in accordance with the new legislation and ensure that their selection or award criteria do not favour certain providers or types of providers.

(b)The Commission should provide, without delay, additional specific guidance on in-house delivery, subcontracting and the assessment of service delivery by consortiums. Furthermore, the Commission should ensure that the Member States’ procedures are adequately monitored and that the selection of knowledge-transfer and advisory activities is competitive, fair and transparent.

97

The Court also found that Member States did not ensure the cost-effectiveness of the activities funded or that these were appropriately coordinated with other EU funds. Cost-effective delivery of knowledge-transfer and advisory services is essential to protect the public purse in respect of both EU and national budgets. The Court found weaknesses in the checks Member States must carry out, both at application stage and at payment stage. There are indications that Member States paid too much for certain services and that related costs were insufficiently justified (paragraphs 52 to 55).

98

The audit further revealed that a considerable number of similar services are financed by different EU funds (e.g. from the ESF as well as through the EAFRD). This implies the risk of double-financing and requires the duplication of costly management structures, which the Court considers to be uneconomic. For the 2014-2020 programming period complementarity between EU funds is stressed above the previous policy of demarcation, this implies a greater need for coordination to avoid the risks outlined above (paragraphs 58 to 64).

Recommendation 3
Cost-effectiveness

(a)With regard to effective checks of cost-reasonableness, the Commission and Member States should implement the recommendations in the Court’s Special Report on this subject⁴⁵. More concretely, the Member States should assess the need to support activities which are readily available on the market at a reasonable price. When this need is justified, Member States should ensure that the costs of the supported activities do not exceed the costs of similar activities offered by the market.

(b)The Commission should build on the first steps taken to ensure complementarity between EU funds, for example through specific inter-service working groups, to carry out a thorough assessment of the complementarity between different EU funds proposed by the Member States for the 2014-2020 programming period. This assessment should result in a coordinated approach to support knowledge-transfer activities, in order to mitigate the risk of double-funding and duplication of administration in the Member States.

99

As regards monitoring and evaluation of the results of knowledge-transfer and advisory activities, the Court concludes that the Commission and the Member States do not have effective and efficient procedures in place. The Court found no examples of systems in place where results from previous calls for proposals or tenders feed back into the knowledge and skills analyses and from there to the design of upcoming selection procedures (paragraphs 66 to 78).

100

Furthermore, the Court found additional confirmation that the CMEF data are not reliable⁴⁶. The counting of beneficiaries only provides, in the Court’s view, information on the popularity of certain services but not on their quality and effectiveness. The result is that despite many millions of euro being spent and several million training days being organised the Commission and Member States are not aware of the impact of the measures provided nor which activities contribute best to the objectives set (paragraphs 79 to 83 and 86 to 88).

Recommendation 4
Monitoring, evaluation and feedback

(a)The Member States should implement feedback systems that use monitoring and evaluation information to improve upcoming calls for proposals or tendering procedures (see paragraph 24 and Figure 4). Providers should be required to provide information not only on the participants’ satisfaction with the services, but also to test whether they have learned what they were supposed to. Such results may also be used by evaluators to allow them to concentrate their evaluation work on the analyses of the activities at result and impact level.

(b)The Commission should provide guidance on how Member States may execute such recurrent feedback procedures and monitor that Member States have them in place. In addition, the Commission should ensure that Member States provide relevant and reliable information on the quality and effectiveness of their knowledge-transfer and advisory services.

101

The Commission‘s management of the measures does not match up to the ambitions set for the activities or the expected contribution to EU priorities. The Court is of the opinion that the Commission did not sufficiently fulfil its responsibilities within the shared management arrangements, in particular as regards the monitoring of the management and control systems established in the Member States (paragraphs 47 to 51; paragraphs 56 to 57; paragraph 65; paragraphs 84 to 88).

102

The scale of spending on knowledge-transfer and advisory measures, in comparison to other rural-development measures, is not high in relative terms. However, the potential multiplier effect of knowledge-transfer and advisory activities means that they merit better management and monitoring.

Recommendation 5
Commission supervision

The Commission should increase, without delay, the risk profile of knowledge-transfer and advisory measures and enhance its supervision and management accordingly, to provide greater assurance that the Member States deliver the respective services effectively.

This Report was adopted by Chamber I, headed by Mr Augustyn KUBIK, Member of the Court of Auditors, in Luxembourg at its meeting of 22 July 2015.

For the Court of Auditors

Vítor Manuel da SILVA CALDEIRA
President

43Article 15(2) of Regulation (EC) No 1974/2006.

44Article 14(3) of Regulation (EU) No 1305/2013.

45See Special Report No 22/2014 ‘Achieving economy: keeping the costs of EU-financed rural development project grants under control’ (http://eca.europa.eu).

46See also Special Report No 12/2013 ‘Can the Commission and the Member States show that the EU budget allocated to rural development is well spent?’ (http://www.eca.europa.eu).

Annexes

Annex I

Implementation of knowledge-transfer and advisory measures in the Member States audited

Member State | Measure 111 | Measure 114 | Measure 115 | Measure 331
Austria | implemented | not implemented | not implemented | implemented
Spain (Galicia) | implemented | implemented | implemented | not implemented
United Kingdom (England) | implemented | implemented | implemented | implemented
Poland | implemented | implemented | not implemented | not implemented
Sweden¹ | implemented | implemented | not implemented | implemented

1 In Sweden, advisory services were generally financed under measures 111 and 331.
Only advice on cross compliance and occupational safety was financed under measure 114.

Annex II

The ‘Kirkpatrick four levels’¹

In the absence of a model used by the Commission or the Member States to analyse knowledge and skills needs and to evaluate knowledge-transfer and advisory services, the Court sought expert support. This led to the application of the ‘Kirkpatrick four levels’ model, a highly influential evaluation methodology for training programmes in the business world. The model can also be applied in reverse order for providing useful guidance for analysing the knowledge and skills needs of the trainees (‘The end is the beginning.’)

(a) Evaluation

Level 1 — Reaction: to what degree do participants react favourably to the learning event? This level is usually evaluated through feedback forms, verbal reaction, post-training surveys or questionnaires.

Level 2 — Learning: to what degree do participants acquire the intended knowledge, skills, and attitudes based on their participation in the learning event? This level is typically assessed through tests before and after the training event. Interviews or observation can also be used.

Level 3 — Behaviour: to what degree do participants apply what they learned during training when they are back on the job? For evaluating this level, observation and interviews over time are required to assess change, and the relevance and sustainability of change.

Level 4 — Results: to what degree do targeted outcomes occur, as a result of the learning event(s) and subsequent reinforcement? For evaluating this level, measures are usually already in place via normal management systems and reporting - the challenge is to relate those changes to the trainee.

(b) Analysis of needs and design of actions

Question 1: What are the specific problems and challenges of the Member States’ or regions’ rural areas and do knowledge-transfer and advisory activities contribute to their solution, i.e. are they relevant?

Question 2: In which way should rural operators perform (‘behave’) to contribute to the solution of the identified problems and challenges?

Question 3: What are the concrete learning, knowledge and skills required to enable rural operators to perform in the desired way, as identified in question 2?

Question 4: What are the preferences of rural operators as regards learning environment, conditions, methods and tools?

The principles of the Kirkpatrick model are worth considering when analysing knowledge and skills needs of rural operators and evaluating knowledge-transfer and advisory services. However, the Court wishes to underline that it does not explicitly favour or recommend its exclusive use. Other models for evaluating training programmes exist, such as the Phillips ROI methodologyTM2 or the Framework for designing and analyzing agricultural advisory services, published by the International Food Policy Research Institute (IFPRI)³.

1Kirkpatrick, J. and Kirkpatrick, W. Kayser, The Kirkpatrick four levels: A fresh look after 50 years (1959 – 2009), April 2009.

2See http://www.roiinstitute.net/

3Regina Birner, Kristin Davis, John Pender, Ephraim Nkonya, Ponniah Anandajayasekeram, Javier Ekboir, Adiel Mbabu, David J. Spielman, Daniela Horna, Samuel Benin, and Marc Cohen (2006): From ‘Best practice’ to ‘best fit’: A framework for analyzing pluralistic agricultural advisory services worldwide. Washington DC (see http://www.ifpri.org/publication/best-practice-best-fit-0).

Reply of the Commission

III

The Member States and regions select the measures and operations to be implemented through the RDPs according to the needs identified in the programming area. These needs should be further detailed to the level of concrete training actions to be implemented. The Commission ensures that the RDPs are consistent with the policy objectives and legal requirements. The actual implementation falls under the responsibility of the Member States. The Commission is informed in the framework of monitoring committee meetings and the implementing reports provided by the Member States.

IV

Training and other knowledge-transfer activities are needed to improve technical and soft skills. In this way they contribute to enhancing the performance of the rural operators. Service providers respond to calls for tenders or proposals, which are prepared on the basis of the needs identified following the Member States’ SWOT analysis. However, the Commission is of the opinion that there is room for improvement in the ex ante analysis carried out by the Member States.

V

The Commission recognises that in order to choose the best service providers the selection should be based on an open and fair competition, measured as a ratio price/quality. Ensuring that these procedures are in place falls within the responsibility of Member States.

VI

Member States have to ensure that the costs related to the implementation of the measures under the RDPs are reasonable. Where shortcomings are detected during conformity audits, financial corrections apply.

VII

The issue of demarcation was raised in the RDPs. For the 2014-2020 period, the ‘Commission services’ position papers’ drawn up for each Member State provided guidance on the coordinated interventions of the ESIF. Member States had to ensure the complementarity, consistency and conformity with other EU instruments in order to avoid double funding¹.

The regulatory framework for the 2014-2020 programming period aims to ensure greater complementarity and better coordination between the funds with a view to avoiding overlapping activities through the partnership agreements, where the Member States have to describe the use of ESIF in order to ensure complementarity and synergies of activities.

In addition, in the area of rural development, the issue of complementarity has been addressed in chapter 14 of the RDPs Information on complementarity.

Moreover, the monitoring committees in the Member States are in charge of ensuring the correct implementation of the programmes and complementarity between all EU funds.

Finally, Commission services work together to ensure a coordinated approach towards ESIF complementarity during the phase of approval of the respective programmes.

VIII

The evaluation of RDPs is the responsibility of the Member States. Assessing the effectiveness of measures against the programmes’ objectives is a complex task, which has to be proportionate to the use of the evaluation. This task is carried out by independent professional evaluators. The latter draw their conclusions on a number of factors and methodological approaches, including a thorough assessment of the intervention logic of the programmes, case studies and analysis on the ground, analysis of interactions and spill-over effects between different measures etc. The CMEF set of indicators are just one tool to address the evaluation questions. The latter aim at structuring the evaluation reports in a uniform way, while addressing in a comprehensive way strategic priorities at the level of the Union. The Commission encourages the exchange of good practices in evaluation which can be done through the European Network for Rural Development (ENRD).

IX

The Commission has always assumed its obligations as regards the monitoring of management and control systems established in the Member States. The Commission supervises the implementation of the measures through the process of approval of the RDPs, within the monitoring committees involving the stakeholders and bilateral annual review meetings with the management authorities.

XI — First indent

This recommendation is addressed to the Member States.

XI — Second indent

The Commission accepts this recommendation.

The Commission agrees that recurrent analyses of needs for training activities by the Member States should be encouraged as a matter of good practice and will update the measure fiche on knowledge transfer and carry out information activities in this respect. The Commission will also promote exchange of good practices on methodological approaches in the context of networking activities.

XI — Third indent

This recommendation is addressed to the Member States.

XI — Fourth indent

This recommendation is addressed to the Member States.

XI — Fifth indent

The Commission accepts this recommendation and has started to implement it.

Guidance on in-house delivery, subcontracting and service delivery has already been provided in the form of a measure fiche on advisory services, farm management and farm relief services. This has been presented and discussed on several occasions with Member States in the Rural development committee. Moreover, the Commission and the European Network for Rural Development held specific training for national and regional authorities on ‘Reasonableness of costs and public procurement’ in Brussels in March 2015.

Regarding the monitoring of the Member States’ procedures, the Commission will take due account of the risk profile of knowledge-transfer and advisory measures in establishing its audit planning.

XI — Sixth indent

The Commission accepts this recommendation.

The Commission is implementing the recommendations made in Special Report No 22/2014² by delivering specific training for the managing authorities and paying agencies, enhanced assessment of the verifiability and controllability of the measures, and promotion of the use of simplified cost options. Furthermore, the systems in place for ensuring that the costs of the projects are reasonable will be assessed in the framework of conformity audits. Financial corrections will be applied in cases of non-compliance.

XI — Seventh indent

The Commission accepts this recommendation and has started to implement it.

The Commission already established working procedures and relevant inter-service groups to assess the issues of complementarity and double funding between different EU funds (e.g. the ESIF interpretation network, other networks on specific elements of the regulation such as the simplified cost working group, thematic networks, or the Financial instruments compass platform).

The Commission will further reflect whether another structure (e.g. a specific inter-service group) is needed.

XI — Eighth indent

This recommendation is addressed to the Member States.

XI — Ninth indent

The Commission partially accepts this recommendation.

The Commission will continue to provide guidance on the implementation of the CMES established in the relevant regulations.

There are standard methodologies to ensure the use of feedback procedures, monitoring and evaluations of results as well as impact of training activities. The Commission will encourage the exchange of good practices in the framework of the European Network for Rural Development.

However, according to the legal framework, the Commission is not responsible for monitoring that such feedback procedures are in place.

XI — Tenth indent

The Commission accepts the recommendation.

The Commission will take due account of the risk profile of knowledge-transfer and advisory measures in establishing its audit planning.

Observations

25

Training and advice actions financed by EAFRD respond to the needs identified in the programming area. These needs are assessed and described in the RDP and then concrete, specific training and advice activities should be planned in the framework of measure implementation arrangements.

Taking into account that training needs evolve, the managing authorities and service providers should adapt their offers to the changing demands of the rural operators.

26

According to the partnership principle, close consultation shall take place with economic and social partners and other stakeholders, notably rural operators, on the elaboration and implementation of the RDP. Through monitoring committees as well as through informal mechanisms of participation and consultation, rural operators have the opportunity to manifest their needs, which should be taken into account by the managing authorities, which are finally responsible for defining the programme’s strategy and implementation, including the selection of relevant activities that meet the programme’s strategic objectives.

28

The setting up of an advisory service must respond to a specific need identified in the programming area.

A new service can be understood as body/entity created ex novo to provide service on certain themes/areas of expertise needed in the region. However, the setting up of a new body/department in a pre-existing advisory service which allows it to enlarge the scope of services provided before can also be considered a new service. The premise is that the new service should be able to provide advice in areas where it would not be competent without the creation of the new service.

Common reply to paragraphs 35 and 36

Member States have to ensure an open, fair and transparent selection of service providers according to the EU and national legal frameworks. The requirement of greater transparency has been reinforced in the 2014-2020 programming period. In particular, article 15 of Regulation 1305/2013 imposes the requirement of having calls for tender for the selection of beneficiaries. The selection of beneficiaries has to be done in accordance with the defined criteria. The bodies providing knowledge-transfer and information services shall have the appropriate capacities in the form of staff qualifications and must carry out regular training activities.

See also reply to paragraph 40.

38

This issue has also been identified as a weakness by the Commission. The guidelines on eligibility and selection explicitly recommend avoiding the submission of proposals at any time throughout the programming period.

40

Member States have the option to deliver in-house services as long as they fulfil the eligibility condition set for the measure. Situations where the administration would be the unique beneficiary of a measure should be duly justified.

Following the assessment of the 2014-2020 RDP, the Commission sent an observation letter to Galicia requesting information about actions which would be implemented directly by the administration and how the best value for money would be ensured.

Common reply to paragraphs 47 and 48

In accordance with the legislation applicable to the 2007–2013 programming period, the RDPs approved by the Commission included information regarding the identification of needs and the procedures for the selection of beneficiaries.

The requirement for carrying out a SWOT analysis of the situation and an identification of the needs that have to be addressed in the geographical area covered by the programme is embedded in the RD legislation of both programming periods. This analysis is complemented by the programme’s ex ante evaluation and includes knowledge and skill needs of rural operators. The programme SWOT analysis, identification of needs and strategic orientation of the programme towards those needs are thoroughly checked by the Commission during the phase of approval of the programmes.

The Commission issued basic guidance for the 2007-2013 programming period to help Member States to prepare their RDPs.

As regards the 2014-2020 programming period, the Commission has prepared more extensive and reinforced guidelines and measure fiches³ which were discussed several times with the Member States in the Rural development committee. The Commission has also provided answers to questions submitted by Member States as regards the preparation of the programmes.

49

The Commission is aware that public procurement is one of the main root causes of error rates. In this context, and in accordance with the principle of subsidiarity, the Commission has taken measures to identify and prevent or correct the corresponding risks. For instance, information regarding the risks related to the implementation of the measure and mitigating actions have to be included in the RDP. For the 2014-2020 programming period both the legal provisions and the guidance by the Commission on public procurement have been strengthened, in comparison with the previous programming period.

The Commission monitors the implementation by enforcing the reporting obligations, through the annual review meetings and monitoring committees. Where conformity audits are carried out, the selection of providers is assessed.

See also replies to paragraphs 38, 40 and 43.

51

For the 2014-2020 programming period the specific measure fiche on knowledge-transfer is much more detailed than the one developed in the previous programming period and includes information regarding the procedures for the selection of beneficiaries. It is complemented by a set of comprehensive relevant guidance documents of a horizontal nature (e.g. on programming and public procurement).

Furthermore, the Commission has extensively discussed this issue with the Member States in the context of the Rural development committee and bilaterally and has provided replies to the questions asked. Moreover, the Commission and the European Network for Rural Development held a specific training for national and regional authorities on ‘Reasonableness of costs and public procurement’ in Brussels in March 2015.

Moreover, even in cases where the Member States do not apply ublic procurement procedures, they must make sure that the selection of beneficiaries is done in an open, fair and transparent way.

The in-house provision of the service is a possibility foreseen in the public procurement legislation in cases where the contracting authority has the appropriate resources to implement the operation internally.

See also replies to paragraphs 47-48, 49 and 50.

55

See reply to paragraph 40.

55 — Fifth indent

The Board of agriculture does not report expenditures on the activity level. All expenses should however be linked to the implementation of the project in order to be considered eligible for support. If the expenditure of the administrative control is not considered to be part of the implementation of the project, costs are deducted.

55 — Sixth indent

With regard to the enhanced scrutiny of EU support for rural development in Sweden in 2013, the Board of agriculture reported improvements of the controls. Further developments are foreseen as regards the new programming period.

Common reply to paragraphs 56 and 57

In accordance with the requirements of Article 59 (2) of the financial regulation, the Commission monitors the management and controls systems established in the Member States respecting the principle of proportionality and taking into account the level of assessed risk. The audit work programme is determined on the basis of a risk assessment, the most important element of which is the level of expenditure. Since the amount of funding involved is relatively low, only a limited number of specific conformity audits have been carried out of the knowledge-transfer or advisory services.

58

Knowledge-transfer and advice activities can be supported by different sources depending on the target group and the themes covered. For instance, EAFRD supports training and advice for people active in the agricultural and forest sectors and since 2014 it also targets SMEs in rural areas, while support under ESF focusses on the unemployed.

The Member States and regions have to ensure the complementarity of the instruments, prevent overlapping and double funding.

Common reply to paragraphs 60 and 61

As knowledge-transfer-related activities are eligible under different streams, the responsibility for managing such activities should stay within the relevant authority. Similar operations can be supported by more than one fund, in a complementary way. The objective of demarcation is to ensure that the implementation of various policies generates synergies, overlaps are avoided and double funding is prevented. During the approval process, programme modifications and regular bilateral meetings, the Commission has requested more information on demarcation, where relevant.

The Commission has assessed the appropriateness of the demarcation line with other funds during the approval process and the assessment of subsequent programme modifications. All RDPs were also assessed in inter-service consultations, where the DGs responsible for the different funds aim at preventing possible inconsistences or overlaps. Moreover, the monitoring Committee, which includes members representing other EU funds, plays an important role in ensuring demarcation.

See also reply to paragraph 58.

64

The Commission’s observation letter sent to Galicia following the assessment of the 2014-2020 RDP highlighted the need for avoiding double funding. Where relevant, the risk of double funding should be identified in the RDP and mitigating actions proposed.

65

The legislation for the 2014-2020 programming period sets up a series of rules to ensure the coordination of interventions supported by EU funds and national resources.

Complementarity is firstly addressed in the partnership agreements (PA) where the Member States define the mechanisms to coordinate support received under ESIFs. This is subsequently specified in the RDPs, where a specific section on complementarity has been included (chapter 14 of the RDP).

Extensive guidance documents for the programming at both levels (PA and programmes) have been developed by the Commission. In this respect, a set of common legal provisions (the Common provision regulation) and a Common Strategic Framework for all the ESI Funds have also been established to reinforce coordination, complementarities and synergies between the funds.

See also replies to paragraphs 60 and 61.

67

The Evaluation of the RDPs is performed by external evaluators under the responsibility of the Member States. It is the professional responsibility of the evaluator to propose the best methodology for the evaluation task.

80

The 2007-2013 CMEF was the first integrated monitoring and evaluation framework. In terms of monitoring, it showed some limitations. The 2014-2020 CMES has been improved by introducing for instance a clear separation between training and communication events.

Box 11 — First indent

As regards Denmark, measure 111 includes a sub-measure which integrates measures 111 and 123 (demonstration projects). Consequently, a simple division of the total support by the number of persons trained cannot be used to calculate the cost of training per person or per training day.

85

The Commission, which remains primarily dependent on the data collected, aggregated and submitted by the Member States, was aware of limitations stemming from the quality of such data. As a consequence, the Commission has improved the 2014-2020 CMES by introducing a clear separation between training and communication events which caused the data collection issues.

Monitoring data is one of the elements that is used for evaluation. The assessment of whether EAFRD support has been well spent will be done in the framework of the ex post evaluation of the RDPs.

88

The evaluation of RDPs is the responsibility of the Member States. Assessing the effectiveness of measures against the programme objectives is a complex task, which has to be proportionate to the use of the evaluation. This task is carried out by independent professional evaluators. The latter draw their conclusions on a number of factors and methodological approaches, including a thorough assessment of the intervention logic of the programmes, case studies and analysis on the ground, analysis of interactions and spill-over effects between different measures etc. The CMEF set of indicators is just one tool to address the evaluation questions. The latter aim at structuring the evaluation reports in a uniform way, while addressing in a comprehensive way strategic priorities at the level of the Union.

Conclusions and recommendations

91

The Member States and regions select the measures and operations to be implemented through the RDPs according to the needs identified in the programming area. These needs should be further detailed to the level of concrete training actions to be implemented. The Commission ensures that the RDPs are consistent with the policy objectives and legal requirements. The actual implementation falls under the responsibility of the Member States. The Commission is informed in the framework of monitoring committee meetings and the implementing reports provided by the Member States.

92

Training and other knowledge-transfer activities are needed to improve technical and soft skills. In this way they contribute to enhancing the performance of the rural operators. Service providers respond to calls for tenders or proposals, which are prepared on the basis of the needs identified following the Member States’ SWOT analysis. However, the Commission is of the opinion that there is room for improvement in the ex ante analysis carried out by the Member States.

93

The Commission recognises that in order to choose the best service providers the selection should be based on an open and fair competition measured as a ratio price/quality. Ensuring that these procedures are in place falls within the responsibility of Member States.

See also reply to paragraph 92.

94

The setting up of an advisory service must respond to a specific need identified in the programming area.

A new service can be understood as body/entity created ex novo to provide service on certain themes/areas of expertise needed in the region. However, the setting up of a new body/department in a pre-existing advisory service, which allows it to enlarge the scope of services it provided before, can also be considered a new service. The premise is that the new service should be able to provide advice in areas where it would not be competent without the creation of the new service.

Recommendation 1 (a)

This recommendation is addressed to the Member States.

Recommendation 1 (b)

The Commission accepts this recommendation.

The Commission agrees that recurrent analyses of needs for training activities by the Member States should be encouraged as a matter of good practice and will update the measure fiche on knowledge-transfer and carry out information activities in this respect. The Commission will also promote exchange of good practices on methodological approaches in the context of networking activities.

Recommendation 1 (c)

This recommendation is addressed to the Member States.

Recommendation 1 (d)

The Commission accepts this recommendation and has started to implement it.

The Commission has issued guidance on Article 15 of Regulation (EU) No1305/2013 to help the Member States or regions to implement the measures. However, guidance is not legally binding and its application depends on the Member States. The Commission has also replied to the numerous questions raised by the Member States.

The implementation of RDPs, including the setting-up of advisory services, is discussed within the monitoring committees, the annual review meetings and conformity audits.

Where necessary, the Commission will continue to update and extend the current guidance.

96

Guidance on public procurement in the context of rural development has been provided by the Commission. In particular, guidance on Article 15 of Regulation 1305/2013 explains the requirements for the in-house provision of the service.

Recommendation 2 (a)

This recommendation is addressed to the Member States.

Recommendation 2 (b)

The Commission accepts this recommendation and has started to implement it.

Guidance on in-house delivery, subcontracting and service delivery has already been provided in the form of a measure fiche on advisory services, farm management and farm relief services. This has been presented and discussed on several occasions with Member States in the Rural development committee. Moreover, the Commission and the European Network for Rural Development held a specific training for national and regional authorities on ‘Reasonableness of costs and public procurement’ in Brussels in March 2015.

Regarding the monitoring of the Member States’ procedures, the Commission will take due account of the risk profile of knowledge-transfer and advisory measures in establishing its audit planning.

97

The Member States have to ensure that the costs related to the implementation of the measures under the RDPs are reasonable. Where shortcomings are detected during conformity audits, financial corrections apply.

98

The issue of demarcation was raised in the RDPs.

For the 2014-2020 period, the ‘Commission services’ position papers’ drawn up for each Member State provided guidance on the coordinated interventions of the ESIF. The Member States had to ensure the complementarity, consistency and conformity with other EU instruments in order to avoid double funding⁴.

The regulatory framework for the 2014-2020 programming period aims to ensure greater complementarity and better coordination between the funds with a view to avoiding overlapping activities through the partnership agreements where the Member States have to describe the use of ESIF in order to ensure complementarity and synergies of activities.

In addition, in the area of rural development, the issue of complementarity has been addressed in chapter 14 of the RDPs ‘Information on complementarity’.

Moreover, the monitoring committees in the Member States are in charge of ensuring the correct implementation of the programmes and complementarity between all EU funds.

Finally, Commission services work together to ensure a coordinated approach towards ESIF complementarity during the phase of approval of the respective programmes.

Recommendation 3 (a)

The Commission accepts this recommendation.

The Commission is implementing the recommendations made in Special Report No 22/2014⁵ by delivering specific trainings for the managing authorities and paying agencies, enhanced assessment of the verifiability and controllability of the measures, and promotion of the use of simplified cost options. Furthermore, the systems in place for ensuring that the costs of the projects are reasonable will be assessed in the framework of conformity audits. Financial corrections will be applied in cases of non-compliance.

Recommendation 3 (b)

The Commission accepts this recommendation and has started to implement it.

The Commission already established working procedures and relevant inter-service groups to assess the issues of complementarity and double funding between different EU funds (e.g. the ESIF interpretation network, other networks on specific elements of the regulation such as simplified cost working group, thematic networks, or Financial instruments compass platform).

The Commission will further reflect whether another structure (e.g. a specific inter-service group) is needed.

Common reply to paragraphs 99

and 100

The evaluation of RDPs is the responsibility of the Member States. Assessing the effectiveness of measures against the programmes’ objectives is a complex task, which has to be proportionate to the use of the evaluation. This task is carried out by independent professional evaluators. The latter draw their conclusions on a number of factors and methodological approaches, including a thorough assessment of the intervention logic of the programmes, case studies and analysis on the ground, analysis of interactions and spill-over effects between different measures etc. The CMEF set of indicators are just one tool to address the evaluation questions. The latter aim at structuring the evaluation reports in a uniform way, while addressing in a comprehensive way strategic priorities at the level of the Union. The Commission encourages the exchange of good practices in evaluation which can be done through the ENRD.

Recommendation 4 (a)

This recommendation is addressed to the Member States.

Recommendation 4 (b)

The Commission partially accepts this recommendation.

The Commission will continue to provide guidance on the implementation of the CMES established in the relevant regulations.

There are standard methodologies to ensure the use of feedback procedures, monitoring and evaluations of results as well as impact of training activities. The Commission will encourage the exchange of good practices in the framework of the European Network for Rural Development.

However, according to the legal framework, the Commission is not responsible for monitoring that such feedback procedures are in place.

101

The Commission has always assumed its obligations as regards the monitoring of management and control systems established in the Member States. The Commission supervises the implementation of the measures through the process of approval of the RDPs, within the monitoring committees involving the stakeholders and bilateral annual review meetings with the management authorities.

The audit work programme is determined on the basis of a risk analysis, the most important element of which is the level of expenditure. Since the amount of funding involved is relatively low, only a limited number of specific compliance audits have been carried out on the expenditure of the knowledge-transfer or advisory services.

Recommendation 5

The Commission accepts the recommendation.

The Commission will take due account of the risk profile of knowledge-transfer and advisory measures in establishing its audit planning.

1Art. 5 Regulation 1698/2005.

2Special Report No 22/2014 ‘Achieving economy: keeping the costs of EU-financed rural development project grants under control’ (http://eca.europa.eu).

3Guidance documents on: 1)SWOT Analysis, 2)Public procurement in rural development 2014-2020, 3) Eligibility conditions and selection criteria 2014-2020, 4) Elements for strategic programme, 5) Verifiability and controllability (Art 62), 6) Measure fiche Art 14 Knowledge transfer, 7) Measure fiche_Art.15 advisory services.

4Art. 5 Regulation 1698/2005

5See Special Report No 22/2014 ‘Achieving economy: keeping the costs of EU-financed rural development project grants under control’ (http://eca.europa.eu).

QJ-AB-15-013-EN-E  ISSN 977-5679

The Court examined whether the high priority of lifelong learning and knowledge transfer within the EU’s rural-development policy was matched by the Commission’s and Member States’ efforts to deliver high-quality knowledge-transfer and advisory activities. The audit found that this was not the case. Too often Member States relied on the proposals coming from the training providers and any type of training was seen as ‘good’ and eligible to receive public support. Only infrequently was there proper analysis made of whether such activities could make a real impact. Member States did not always ensure fair and transparent competition when selecting the training activities and paid too much for certain services. In terms of follow-up, the audit found a lack of detailed evaluation of what was actually achieved with the public funds. In its recommendations, the Court particularly encourages the Member States to adapt the knowledge-transfer and advisory activities they choose to the evolving needs of the rural operators through recurrent procedures of analysis and evaluation.