The PricewaterhouseCoopers report

Review of the implementation of the Theatre for Young People strategy [Executive summary]

.1 In accordance with our terms of reference we summarise in this section our views as to 'what went wrong' with the implementation of the TYP element of the drama strategy.

2.2 We have set out in section VIII of this report a number of recommendations that, if adopted, we believe would, mitigate the risk of some of the process failures that our review has identified.

Lack of a formal risk assessment process for the implementation of a major new strategy
2.3 We consider the most important factor that could have prevented the problems with the implementation of the TYP strategy is that there was no formal assessment of the risks that might prevent its successful implementation and how those risks might have been mitigated. We note there was no formal requirement for ACW to have in place a risk management strategy at the time of the development of the TYP strategy. However, the assessment and subsequent mitigation of the risks impacting on an organisation achieving its corporate objectives is best practice in the private sector and increasingly becoming part of the corporate governance process of public sector organisations.

2.4 In section VIII we recommend that ACW adopts a risk management process as soon as possible.

The consultation process and development of the TYP proposals
2.5 The consultation process appears to have failed in two main areas:

It did not identify what were the objectives of TYP in Wales
2.6 We can find no evidence that the consultation process identified and concluded upon the following, which were key questions in the TYP consultation process:

(a) What do young people want
(b) What are the views of the specialist theatre practitioners and teachers
(c) How and where might theatre happen
(d) What resources might be available

2.7 We found no evidence that these had been specifically addressed in the analysis of responses or in the draft drama strategy. The consultation process appears to have focused solely on the 'model' of delivery.

2.8 We understand that the funding allocated to the TYP franchises of £720,000 was not based on an assessment of the funds required to deliver specific objectives but as part of the allocation of the overall drama budget.

2.9 Failure to agree the objectives of TYP led to what we consider to be an inadequate tender document given the appearance that ACW did not know what they wanted from TYP. Specifically, the tender document asked bidders to consider 'building new artistic collaborations and celebrate inventiveness, imagination and originality in serving young audiences in Wales'.

It failed to win support and acceptance of ACW's proposed strategy
2.10 As a funding body, ACW assesses competing demands for limited resources. From time to time it may therefore need to make difficult decisions on its funding priorities, recognising that some decisions may not receive universal support, particularly from those organisations and communities whose funding is withdrawn.

2.11 ACW must therefore ensure that its strategic consultation processes achieve the following:

(a) The objectives of strategy are identified and clearly communicated
(b) A range of realistic options to achieve that strategy are considered
(c) The funding context is identified and clearly explained
(d) It is seen to be consultative and not merely a public statement of intent
(e) The final decision of ACW is dearly explained as are the reasons for rejecting alternative proposals

2.12 The consultation process for TYP and the drama strategy failed to achieve these objectives because;

(a) The objectives of the strategy were not explicitly stated
(b) The basis for the funding was not explained

(c) The timetables adopted and the limited nature of the feedback on the consultation responses were considered by some of the consultees to be inappropriate and selective, although we note that ACW received legal advice on the consultation timetable
(d) Key messages from the feedback process do not appear to have been taken into account
(e) The Local Authorities as key partners were not convinced by, or accepted, ACW's proposals

2.13 We have made ~a number of recommendations in section VIII that will address some of these issues for future consultation processes.

The franchise process

2.14 The franchise process was affected at the outset by a lack of clear objectives for the TYP strategy. This led to a tender document with no specification of what should be delivered and too many criteria with no clear indication of how each would be assessed.

2.15 ACW had no established procedures for the form of competitive franchise award process adopted.
2.16 A number of changes were subsequently made to the evaluation process including;

(a) reducing the number of criteria that would be used for evaluation,
(b) the introduction of weightings to the assessment criteria,
(c) the proposal of the evaluation panel to award areas on the basis of the five 'best' scores, and
(d) the proposal of the evaluation panel change in the areas and level of grants to be awarded,

2.17 We believe such changes in the selection criteria after bids have been received are inappropriate, even if the intention was to correct perceived weaknesses or deficiencies in the process.

2.18 Because of this, we consider Council, on the advice of the Chief Executive, was correct to reject the recommendation of the evaluation panel. However, having reverted to the original 'best in each area' approach for four of the franchises, Council was itself inconsistent in asking for alternative options to be considered for the fifth franchise.

2.19 In our opinion, the company that bid for the Central, West and East Valleys areas did not supply compliant bids for the West and East Valleys areas.

2.20 The failure to retain score sheets and other documentation to support the evaluation process was a major weakness. We are also concerned that there appears to be no evidence of how the 'value for money' of the proposed outputs of the bids were assessed.

2.21 Given the nature of the exercise that ACW were trying to undertake, we do question whether the process adopted was the correct procurement route. Should similar circumstances occur clear procedures for the process should be documented and agreed prior to commencing the process.

Concluding comments

2.22 This executive summary should be considered in conjunction with the detailed findings in this report.

2.23 We provided management with a draft copy of this report and have, where we consider appropriate, taken account of their observations in this final report.

2.24 .We have set out in section VIII of this report our detailed recommendations for improvements in practices adopted by ACW. Included in this section is the management response detailing the proposed action to be taken against each recommendation, the officer responsible and the timescale for implementation. We would suggest that this action plan is reviewed and if appropriate endorsed by the Council. The action plan should then be monitored by the Audit Committee on behalf of Council to ensure the agreed action is taken.


Internal audit report 1/2001 PricewaterhouseCoopers - August 2000

author:

original source: Arts Council of Wales
06 October 2000