Archived decisions
County Council Response to the Second Joint Inspectorate Consultation on CAA (Comprehensive Area Assessment)
`Comprehensive Area Assessment, Joint Inspectorate Proposals for Consultation - Summer 2008'
1.1 Hampshire County Council welcomes the opportunity to comment on this consultation document. As a CAA Trial Site we reserve judgement on the emerging CAA model as the process continues and additional issues come to light. With this accepted, please find below some overall comments as well as responses in appendix one to the individual questions raised in the consultation document. This constitutes our formal response.
2.0 Overall key points
2.1 We are pleased that the revised CAA proposals have removed the separate Direction of Travel assessment in order to streamline CAA into one Organisational Assessment and one Area Assessment, as urged by the County Council and others earlier this year. There are a number of other positive changes in these proposals which we welcome including the dropping of the publicly reported `Risk Assessment' label; a commitment to focus on outcomes over process; a commitment to be proportionate in assessment activity; and no automatic `Red Flags' (if the issues are known and action is being taken).
2.2 Having said that, we still have some issues and concerns that we consider important to raise. Our principle concern remains that the legitimate role of elected Members and local community priorities is still not recognised strongly enough in much of the CAA framework - understanding this will be a critical success factor for CAA. Also, despite the rhetoric, there is no obvious reduction in burden in terms of expectations or frequency of assessment. In addition, CAA needs to be a catalyst for improved outcomes through more effective partnership working. We would welcome the opportunity to discuss these further:
3.0 Councillors & local priorities
3.1 The central and important role of elected Members is not sufficiently recognised in the CAA process or key questions. For instance, of the three headline Area Assessment questions1, the latter two make no mention of local priorities or reference to the fact that they will be looking at these questions in relation to these local priorities.
3.2 In the consultation document the inspectorates propose to highlight whether they believe local priorities "address the needs and aspirations of local people"2. Inspectorates should instead recognise when Members and partners have legitimately decided something is not a priority - the emphasis for CAA should be on whether that active decision has been made rather than second guessing the role of Members and partners to decide local priorities.
3.3 The CAA model needs to recognise the unique and democratic nature of councils and the democratic mandate and legitimacy of Members to lead, prioritise, scrutinise, make decisions and budget on behalf of the local community.
4.0 `Red and Green Flags'
4.1 Red flags have the potential to be an extremely effective catalyst for improvement but they are equally in danger of being erroneous or distorting. It will be important to select, word and present any red flags sensitively and in a non-alarmist way, providing the wider context, whilst being clear which partner is underperforming.
4.2 We question whether the simplicity of the red flag system could adequately measure outcomes in a local area. It is important that any suggestion of a red flag should trigger a dialogue with the relevant audited bodies with the process focussing on `round-table discussions' between the inspectorates and audited bodies.
4.3 In addition, we suggest that a `yellow card/flag' warning system is used privately to alert audited bodies of the inspectorates' concern of an issue ahead of and in advance of any public red flag. This would have the benefit of allowing issues to be addressed before they become major problems and would also be a strong catalyst for improvement.
4.4 It is welcomed that red flags will not be used where serious concerns are recognised and plans in hand to address. It will also be important to ensure clear and transparent moderation to ensure that over zealous local inspection teams do not apply red flags for relatively minor issues.
4.5 We would also like to see a greater emphasis put on green flags which could helpful be expanded so that they do not simply concern the `exceptional' but a wider variety of successes and achievements. We would like to see proportionately more green flags to celebrate innovation.
5.0 Proportionality
5.1 We welcome the inspectorates' adoption of the principle of proportionality but we do not feel this is adequately reflected in the proposed model. There is no obvious reduction in the weight or frequency of CAA for those that demonstrate competence and good performance (ie lower risk).
5.2 The passing reference to CAA being `inherently proportionate'3 lacks transparency and is unconvincing.
5.3 The CAA model does not seem sufficiently flexible to recognise the different levels of assurance or intervention needed for top performing authorities and poorer performing authorities. In CPA, the higher the score the longer the period between Corporate Assessments. We would like to see something similar in CAA.
6.0 Inspectorate gatekeeper
6.1 The institution of a single inspectorate `gatekeeper', as previously called for by the County Council and others, is strongly supported. However, it is felt the role remains too weakly defined by the passing reference that the work of the inspectorates "will be coordinated by the Audit Commission to reduce demands on the council"4.
6.2 We believe that the Audit Commission's role as the single inspectorate gatekeeper should be made more explicit both in terms of coordinating the work of all the inspectorates, escalating any concerns if an inspectorate is not meeting timetables and providing local arbitration for overall CAA judgements.
7.0 Single inspection trigger
7.1 The CAA assessments is supposed to be the sole trigger for further assessment or inspection activity outside of the Ministerially approved rolling programmes5. This seems to be at odds with the separate recent CSCI (Commission for Social Care Inspection) consultation published in June 20086. This consultation simple states that "inspection will continue to be an integral and essential element of performance assessment" and makes no mention of how this may be triggered only as part of the CAA process.
7.2 In addition, the CAA consultation document says that Ofsted (Office for Standards in Education) proposes to carry out "limited annual focussed fieldwork" which suggests another programme of work outside of CAA.
7.3 It is important that CAA is reasserted as the single inspection trigger and the inspectorates and their various current performance frameworks are more tightly aligned.
8.0 Transition
8.1 It is important to recognise the scale of the task to shift from CPA to CAA and not to underestimate the change in skill set, maturity and capacity needed by the inspectorates. The recruitment by the Audit Commission of local CAA Leads is helpful as part of this.
8.2 We still have concerns around the interaction and coordination between inspectorates and believe that the inspectorates do not yet seem to have the capacity locally to undertake a truly comprehensive area assessment.
8.3 We feel there ought to be a period of transition to phase in the CAA arrangements in recognition of the challenge of shifting from CPA to CAA, and the significant implications for the way that inspectorates operate.
9.0 Moderation
9.1 It is important that any moderation process is transparent, simple and open. This is particularly important in the first year of CAA to ensure the assessment model is applied evenly.
10.0 Response to CAA consultation detailed questions
10.1 Please see appendix one for the response to the specific questions raised in the consultation.
Appendix One - Response to the specific questions
1.0 Do you broadly agree with our proposals for the overall CAA framework? (Q1)
1.1 Overall yes, but there remains questions in some areas and more detail is needed. Outstanding issues include:
1. The consultation document states it recognises "political choice" but then goes on to state that performance in "important outcomes" will be followed up whether or not identified as a local priority7. Challenging Members and partners over whether they have made active policy decisions or overlooked issues as well as requiring minimum standards for vulnerable groups is accepted. Challenging the electoral mandate of Members to decide local priorities and budget on behalf of the local community is not the role of the inspectorates.
2. Despite the rhetoric, there is no obvious reduction in burden in terms of expectation and requests on the audited organisation, particularly as we move from some elements of CPA being periodic and most elements of CAA being annual.
3. The CAA proposals lack any mention of rewards for stronger performance in terms of potential freedoms and flexibilities - this was an important element of the original CPA which encourages high performance and can encourage partnerships to work towards agreed outcomes. This principle should be incorporated in the new framework in a way that will encourage and support partnership working. We would like to see `inspection holidays' and greater freedoms over services and funding for those who perform well and demonstrate lower risk.
4. The reference to two-tier areas in the consultation document is welcome8. The inherent complexities in two-tier areas needs to be recognised, in terms of the different partnerships and separate Sustainable Community Strategies. In addition, the difference in scale between upper and other tiers is something that needs to be taken into account when undertaking CAA.
5. We welcome that CAA talks seriously about sustainability9.
2.0 Are the area assessment and the organisational assessment, as the two key elements of the framework, clearly explained? (Q2)
21. Broadly speaking there is a good understanding of the aim of the two assessments, there is less certainty as to how exactly they will be carried out - the interaction with individual audited bodies and jointly across partnerships. Also it is uncertain how the inspectorates will work together to formulate joint judgements and how audited bodies will be consulted with on this collectively.
2.2 It is understood and felt useful that the inspectorates will jointly make the Area Assessment. However, it is also proposed that the Organisation Assessment and specifically the Managing Performance assessment will be a joint inspectorate venture10. We would question how, practically, this will be done and how we prevent this becoming a huge separate exercise (which was the main reason the Direction of Travel assessment in the earlier consultation was opposed)? We believe it makes more sense for the Audit Commission to do conduct the Managing Performance assessment on their own - of course this can be informed by the views and evidence from other inspectorates, as relevant. The Audit Commission leading this makes more sense with their stronger organisational specific understanding through their wider Use of Resources and other audit work.
3.0 Is the link between these two assessments clear? (Q3)
3.1 The principle of the linkages are clear but at what level of significance one would start to have an impact on the other and how this would work in practice is less clear.
4.0 Do you agree that the three questions and supporting issues (see Appendix1) proposed for the area assessment are the right ones? If not, please suggest alternative questions and/or issues. (Q4)
4.1 We would like to see the second and third headline Area Assessment questions11 mention local priorities and reference the fact that the inspectorates will look at these questions in relation to these local priorities.
4.2 In terms of the questions to be asked as to whether further work by the inspectorates is required as part of the Area Assessment, we would like to see a consideration of the value for money/added value of further investigative work included in these questions12. Also, when an area of concern is identified we would like to see a consideration of how important/level of priority this is locally - otherwise this is again in danger of second guessing legitimate elected Member and partner decided local priorities13.
4.3 The list of supporting issues topics under the third question are clearly a list of sub-questions14. We feel that inspectorates should use the LAA themes/priorities in the first instance and then only consider outside of that areas of specific concern using this list of sub-questions as a short hand checklist rather than follow each up in detail separately.
4.4 In addition, there is a danger that the supporting issues under the third question become process rather than outcome focussed, for instance where it is proposed that inspectorates would need to be "satisfied that partnership working is effective"15. This can lead to a preconceived view of what best practice partnership working looks like rather than a focus on outcomes and what works locally.
5.0 Do you agree that we should use the green and red flag approach for reporting the area assessment? If not, please suggest an alternative approach. (Q5)
5.1 Red flags have the potential to be an extremely effective catalyst for improvement but they are equally in danger of being erroneous, distorting and a distraction. It will be important to select, word and present any red flags sensitively and in a non-alarmist way, providing the wider context, whilst being clear which partner is under performing. It is welcomed that red flags will not be used where serious concerns are recognised and plans in hand to address. We feel that more emphasis should be placed on green flags to ensure the CAA process does not become a negative deficit model. We would like to see proportionately more green flags to celebrate innovation. Green flags could helpful be expanded so that they did not simply concern the `exceptional' but a wider variety of successes and achievements.
5.2 The consultation proposes that a red flag could be given for concern over `poor performance'16. We believe a distinction should be made, and the difference reflected in CAA reports, between poor performance where targets are not being met and strong performance but challenging targets not quite being met. There also ought to be some qualification in relation to for how long performance has been `poor, slipping or not improving' as well as how serious or important this is. This is important to guard against disproportionate responses to short, perhaps even managed, dips in performance as well as measured responses to areas of lesser importance locally.
5.3 Overall red flagging areas because they feel they may become a problem in the future is, by its nature, subjective - emphasis needs to be given to round table discussions in these instances as well as moderation (the risk of inconsistency between different CAA areas, particularly in the first year, is great) and ultimately appeal processes.
5.4 In addition, we suggest that a `yellow card/flag' warning system is used privately to alert audited bodies of the inspectorates' concern of an issue ahead of and in advance of any public red flag. This would have the benefit of allowing issues to be addressed before they become major problems and would also be a strong catalyst for improvement.
5.5 We are keen to see the appeal process for these red flags and reserve judgement until the details of this are published - currently, expected "at a later date"17.
6.0 Do you agree that we should have one overall organisational effectiveness judgement, drawn from integrating the managing performance theme and the use of resources themes? (Q6)
6.1 We support an overall Organisation Assessment judgement which we feel will be simpler and easier for the public to understand.
7.0 Do you agree with our proposals for the key questions and focus for the managing performance theme of the organisational assessment for: a) councils? b) fire and rescue services?
If not, please suggest alternative questions and/or focus. (Q7)
7.1 For Councils, broadly speaking these look reasonable. Our only concern is that they should be treated with the importance they deserve rather than as an afterthought to the larger Use of Resources assessment.
8.0 Which of the three options for scoring the organisational assessment should we adopt? If you disagree with all the options, please propose an alternative approach to scoring. (Q8)
8.1 We prefer the second option where there would be a degree of local discretion in the overall Organisation Assessment judgement rather than a rigid wholly rule based system with individual scores restricting overall scores. This discretion should be used as part of round table discussions with the audited bodies. There should also be option to appeal if the judgement given is disputed.
9.0 Do the proposals provide for an appropriate focus to be given to people in vulnerable circumstances? (Q9)
9.1 In June 2008 the IDeA (Improvement and Development Agency) published consultation on an updated Equality Standard for local government18. On the face of it there appears to be many similarities between the proposed new Equality Standard and CAA - certainly in terms of a focus on the vulnerable and hard to reach groups - so we would like to see greater alignment of the two processes.
9.2 We suggest that the CAA inspectorates should make closer links with the IDeA and make it an integral part of the CAA framework to avoid separate and duplicate activity.
9.3 You state in the Equalities Impact Assessment that you need to find better ways to assess progress on diversity19. This could be done by integrating the main principles of the proposed Equality Framework into CAA and then when inspections take place an IDeA approved peer assessor becomes part of the team. This would have a dual effect not only on the audited bodies but also on the other inspectors, encouraging them to be more equality aware. In any case, compliance with the main elements of the Equality Standard ought to be considered part of the inspectorates public duty - integration within CAA is a natural progression.
10.0 Do you agree that CAA should evolve over time? (Q10)
10.1 We are a little concerned about this expression. If this means that the CAA process is continually improved, enhanced and streamlined then this is supported. If, however, it means that the bar will constantly be raised and the burden increased this would not be supported.
10.2 It is important to recognise the scale of the task to shift from CPA to CAA and not to underestimate the change in skill set, maturity and capacity needed by the inspectorates. We feel there ought to be a period of transition to phase in the CAA arrangements in recognition of the challenge of shifting from CPA to CAA, and the implications for the way that inspectorates operate.
10.3 Either way, we would like consideration given to `the extent CAA is proportionate and represents value for money' added to the areas of focus for the evaluation of the CAA model20.
11.0 Do you broadly agree with the way we are proposing to use the National Indicator Set within the CAA framework? (Q11)
11.1 The focus must remain on the locally agreed priority areas as represented in the performance indicators within the LAA. Whilst it is recognised that the wider NIS of 198 may provide useful additional information, it will be important not to skew assessments to NI themes that we do not consider a priority - or even relevant - locally.
11.2 The consultation document proposes using `comparator groups'21 for each NI. This will need to be considered in the context of different base positions and the locally agreed priorities. It will also be important to get the right comparator group - each NI is likely to be different.
12.0 Do you support our proposals to report the assessments as set out in our prototype CAA reporting tool? If not, please suggest alternative proposals for reporting. (Q12)
12.1 It is proposed that CAA assessments will be updated quarterly22. Such updates should only be made in cases of significant change and in consultation with the relevant audited body.
12.2 It will be important to select, word and present any red flags sensitively and in a non-alarmist way, providing the wider context, whilst being clear which partner is under performing. We also feel that more emphasis should be placed on green flags to ensure the CAA process does not become a negative deficit model. We would like to see proportionately more green flags to celebrate innovation.
13.0 Do you agree with our proposals for peer involvement? If you have other suggestions about this, please outline your ideas. (Q13)
13.1 Where peer involvement means adding value by introducing additional insight to advocate the reality of the operational context of local government we would support a degree of peer officer and Member involvement in CAA and would like to know more - there is not enough detail in the current proposals23. We believe peer involvement was something that was a strength of Corporate Assessments and could usefully be incorporated as part of the joint inspectorate Area Assessment team for the annual judgements.
14.0 Do you agree with our approach to self-evaluation? (Q14)
14.1 We support the non-prescriptive approach24.
15.0 Do you agree with our approach to gather relevant information from sources such as Citizen's Advice Bureaux and Regional Business Forums? Are there any other sources we should consider? (Q15)
15.1 Although citizen and users perspectives25 would be useful inspectorates need to be clear about the questions being asked and if these reflect the evidence of engagement. Very often difficult decisions have to be made to balance private benefit with public good. There is a danger of conflicting interests from certain users or organisations that have recently been refused wider services or funding - this may lead them to have an unrepresentative/skewed view. Consideration needs to be given to how this will be overcome.
15.2 In a similar vein, thought needs to be given to how vested interests or contested decisions will be managed26. For instance if the County Council or partner makes a strategic decision based on the evidence (including comprehensive consultation) and long term planning this may not be a popular decision. Following this, inspectorates may pick up dissatisfaction with consultation and involvement but this should be weighed against the actual process of engagement that took place.
15.3 Finally, when seeking views from others it will be important for the inspectorates to make clear which (geographic) `area' they are assessing - the public and other groups have a more fluid interpretation of an area that are not constrained to top-tier administrative or LAA boundaries. It will also be important to make clear the range of organisations within that area that are being assessed.