Why do we evaluate cultural education, and how can we improve?

8 July 2014

(Photo credit: Simon Callaghan)

A little over a month ago, Culture 24’s Anra Kennedy and I did a joint presentation at the Museums & Heritage show about how those of us in museums education (and the broader cultural learning sphere) measure and communicate the impact of our work.

The session ranged quite broadly, also taking in the value of accreditation, and the question how we respond to and support the needs of schools. However, for this blog it’s the big, long-standing and perennially unresolved issue of evaluation and impact that I’d like to look at, and – rather than claiming to have definitive answers of my own – I’d like to share with you some of the questions that Anra and I raised with the audience. This blog will also have a sequel next week, summarising some of the current developments in the field.

Why?

Firstly – and it’s perhaps not a question that we ask often enough – WHY do we feel the need to evaluate?

1. Is it for organisations’ own, self-driven performance objectives, self-understanding & improvement?

2. Is it to satisfy the requirements of schools & other external partners, or funders?

3. Additionally, are we asking about/evaluating against the needs of our audiences (both current users and communities, and potential new ones)?

4. Or is it as ‘just something that has to be done’, without a clear purpose – a sort of ‘Zombie evaluation’?

My sense – and the room seemed to share this view – is that, whilst some combination of (1), (2) & (3) should be what we aim for, all too often we can end up a little lost in the limbo of number (4)...Think about your own practice – where do you fall currently? Where would you like to get to, and what changes, resources and support would be required?

What?

We also reiterated Francois Matarrasso’s astute distinctions between the different types of exercise that we as a sector sometimes muddle together when we talk of evaluation:

Monitoring – whether intended outcomes are being/have been achieved
Impact measurement – what specific changes were brought about as a consequence of an intervention
Evaluation – which elements succeeded, which failed, and what could be done differently in future
Advocacy – promoting successes externally

I sometimes fear that advocacy is often the overriding motivation for many so-called evaluations, but this is dangerous for all kinds of reasons.

If we are driven primarily by a pre-emptive desire to celebrate our successes, can this limit our investment in really challenging or innovative types of projects?

Could it also lead us to ‘spin’ the results of our work, overlooking results that are weak or unwelcome, and cherry-picking positives that may look a little lonely or insubstantial when held up alone?

How can we learn from our work, individually, organisationally and as a sector, if we can’t honestly contemplate and usefully scrutinise what hasn’t worked? Could our fear of failure paradoxically prevent us from really striving for, and achieving, the fullest contribution that we can make to our communities, and to society more widely?

What next?

I’m intrigued by the role of funders in this. For children and young people’s programmes particularly, the shift from ‘core’ funding (eg long-term funding from ACE or local government) to project funding we’re more afraid than ever that a failed project now could signal a future loss of funding.

If we were to re-imagine a more progressive system, what could it look like? What if funders provided support in evaluation, and required us to share both our successful and failed projects with the wider sector – a huge research database to allow us to continuously improve our work nationally and internationally? Where the learning from a single disappointing project isn’t a secret shame to be buried in the vaults, but there to be shared for the advancement of the cultural education community?

How do you evaluate your work? Please let us know via Twitter, or using the comments below.

A New Direction will be hosting an event based around the theme of Value in arts and culture on January 28th 2015 – booking details to follow soon, sign up for our Twitter and newsletter to be first ‘in the know’!

Like this blog?

We send out a monthly digest of all our blogs, news and updates - sign up here to receive straight to your inbox - www.anewdirection.org.uk/newsletter

You can also follow us at @A_New_Direction