Are we measuring what matters in the Arts?

19 November 2013

(Image source)

Arts Inform working with A New Direction, the Roundhouse, The Royal Opera House, the Lyric Hammersmith, Spitalfields Music, Dance United, London Transport Museumand Horniman Museum, have been exploring how organisations can respond to this need for measurement. People and organisations see the various models of measurement in different ways; this post suggests some (not all) of the ways of seeing.

Sir Peter Bazalgette and Sir Alan Davey express their view in the foreword to the recently published document Towards Plan A: a new political economy for arts and culture.

"At present, you could liken our case to that of the Higgs boson, the elusive particle that gives others their mass. The arts are essential, but so embedded in our lives that their presence often goes unacknowledged. Pinning this down requires specifics. It needs more measurement, more evaluation, and a new language of value to communicate across different sectors of influence. We need to be able to make this holistic case as clearly as others in the ‘third sector’ do."

The Holistic case steps over any divide between ‘intrinsic ‘and ‘instrumental’ value of arts and culture and places the artistic role as central.

Grayson Perry, in his recent Reith lectures, affirmed the intrinsic value of art, in the way it makes life meaningful,

"….art’s primary role is not as an asset group and it’s not necessarily about urban regeneration, but its most important role is probably meaning making."

Speaking about the particular role of art in the lives of children and young people, he says,

"art, like play, can have a very serious purpose because I think one of the most serious purposes that art can have is helping children deal with the difficulties in their lives."

Which performance, event, participatory experience meant the most to your community? And on what evidence do you base your answer?

This was one of the challenges thrown down by Diane Ragsdale, an American commentator currently attending the Erasmus University in Rotterdam, at a recent talk to a group of arts organisations entitled Defining Value.

Yes, we can say which had the highest attendance; yes, we can say which one had the best reviews; and yes, we can probably say which meant the most to us personally. But which had the most value to our community? And on what evidence is this based? This is not so easy.

Why measure at all?

Diane answered this via Andrew Taylor, the Artful Manager, as being:

1. Because you need to account for your progress toward a goal that outside stakeholders (a funder, for instance) care about
2. Or because you want to measure your progress toward a goal that matters to the organisation.

However, often when we work to evaluate for funders, we can lose sight of our own organisational goals and priorities. And to bring a focus on measurement does not mean that we are looking at the arts for their instrumental or monetary value, Diane went on to outline how valuing the ‘intrinsic’ was to be welcomed in an increasingly market-driven society.

Are we measuring what matters?

Diane outlined how in a relatively short period of time the narrative in the UK seems to have shifted from culture for the culture’s sake; to culture for innovation’s sake (to support the creative economy); and, more recently, to culture for the economy’s sake with Maria Miller asking the sector to help her reframe the case for the arts entirely in economic terms.

This shifting narrative is not limited to the UK. We’ve shifted out of an era in which the value of the arts was taken-for-granted. Diane used the phrase of Michael Sandel, we’ve shifted from having a market economy to living in a market society. He describes this development in his book “What Money Can’t Buy” writing:

"The reach of markets, and market-oriented thinking, into aspects of life traditionally governed by non-market norms is one of the most significant developments of our time."

At the Defining Value event meeting, Project Phakama gave an example of the work they do.

The mission of the organisation is driven by the intrinsic value of their ‘give and gain’ ethos. In this process, learning becomes two-way; everyone has something they can give to the project and everyone has something to gain. And through this interchange of skills, knowledge, information and ideas, everyone becomes both student and teacher.

What was interesting was to hear Fabio Santos, Artistic Director, articulate how the ‘give and gain’ ethos could combine with a fundraising agenda.

Young people on Phakama programmes were able to approach fundraising as a social activity and a way of delivering personal development; in this way the ‘intrinsic’ and ‘intrumental’ values are combined. It is an example of the benefits of linking the central mission of an organisation with a fundraising agenda.

Another example of re-examining value is the Happy Museum project that advocates for well-being and sustainability and whose manifesto principles include ‘measure what matters.’

A View from the United States – the Architecture of Value

Diane went on to offer an approach that focuses on measuring what matters. She described it as a way of reversing the spiral that places economic impact as the main driver.

"How do organisations go from being valued for economic impact, or a form of entertainment, or a driver of innovation, to being valued for contributing to a sense of identity and place? Being valued for helping people to learn to adapt to a changing world and lead better lives? Being valued for bringing people together on equal terms?"

Diane introduced one of the leading researchers in the area of measuring impact in the US, Alan Brown.

He began his research in response to a report by the Rand Corporation that said, in essence,

"After years of encouraging arts organisations to measure their instrumental impacts we’ve determined that what they really need to measure are their intrinsic impacts."

In his book Architecture of Value he outlines the following model.

This framework identifies three types of value (at the individual level, shared between people, and at the community level) and three time frames for when this value is realized (at the time of the event, in the period following and the cumulative effect of experiences over time). So, in the circle “imprint of arts experience” (experienced at the individual level, during the event) you would see the value “flow”; in the circle personal development (experienced by the individual over time), you would find “aesthetic growth” (stretching beyond one’s comfort level to experience something new); under human interaction you would find the “capacity for empathy”; under Communal meaning you will find examples arts and culture bringing identity to a place.

There are many examples of this ’place-making’ agenda, for example, Turner Contemporary in Margate, Guggenheim in Bibao, Nós do Morro in Vidigal and is relevant to the Arts Council’s Creative People and Places programme. The economic and social benefits are pretty self-explanatory.

A view from Arts Council England – the Holistic Case

How we visualise this ‘architecture of value’, this model from the US, is particularly relevant to how Arts Council views the Holistic Case. There is a clear commitment to putting the ‘intrinsic value’ of arts/culture at the centre of the architecture, the base of empathy, identity and collective memory which supports the other values (the economy, education and society). In this way, we can see that the Holistic Case as a model has many similarities with the model proposed by Alan Brown in the Architecture of Value model.

The Arts Council has a Quality Metrics Pilot that aims to capture the quality and reach of arts and cultural productions, led by 13 cultural organisations in Manchester.

What will be interesting is how data can be aggregated to show the impact of an arts experience at an individual level and at a community level. What will be harder is how these assessments can measure the cumulative effect of experiences over time.

The Arts council has also developed seven principles of quality to be used in the context of work with children and young people. These principles include ‘excellence’ but also elements that offer a sense of development over time, for example, ‘personal progression’ and ‘ownership and belonging.’

In the context of holistic development, the child and the young person is where it all begins and ACE have re-structured their 10-year strategic framework with the goal dedicated to children and young people being at the centre of the architecture.

A View from Ofsted – the social, moral, spiritual and cultural case

In the formal education sector, one of the criteria through which schools are assessed is OFSTED’s report on social, moral, spiritual and cultural education, a measure that has an increasingly important value amid all the changes that are happening in the education landscape. The statutory requirement that schools should encourage pupils’ SMSC development was first included in the Education Reform Act 1988. The Act began as follows: ‘The curriculum for a maintained school (must be) a balanced and broadly based curriculum which —

(a) promotes the spiritual, moral, cultural, mental and physical development of pupils at the school and of society; and

(b) prepares such pupils for the opportunities, responsibilities and experiences of adult life.’

An idea of what legislators had in mind when legislating for the inspection of pupils’ SMSC development can be gained from a debate in the House of Lords in July 1996. This emphasised the need to establish the values schools should impart to pupils. It was clearly recognised that there is more to life than achieving high standards in academic subjects. The task was described as:

‘…the training of good human beings, purposeful and wise, themselves with a vision of what it is to be human and the kind of society that makes that possible’.

Susan Hodgetts speaking at the Westminster Cultural Education Forum, noted the importance of this measurement.

"I never thought I would say the next line, but thank God for Ofsted, and their insistence on looking for elements of social, moral, spiritual and cultural aspects of the school. It’s a pity, however, that it forms such a small part of their inspections, because in my opinion SMSC is the fundamental building block of a good education. If you truly want to know why a school doesn’t make the grade, then don’t bother looking at the exam results, look at the school’s SMSC, because you can bet your bottom dollar that this is where things are going wrong."

Arts and cultural organisations play a big part in developing in young people ‘a vision of what it is to be human’ and partnerships with schools to help deliver this are part of the architecture of place-making.

A view from the organisations – the collaborative case

The Tate is an example of an organisation that has developed a values-based approach to their work. Trust, Generosity, Desire, Risk and Thoughtfulness underpin how the organisation assesses their work with schools.

The challenge for organisations is to embed the principles and values in their work and to measure if it is making a difference to the world outside them.

At the above mentioned Westminster Cultural Education Forum, Purni Morell, Artistic Director of the Unicorn Theatre for young audiences, spoke about the intrinsic value of theatre for children and young people that lies outside of formal education.

"It's a visceral, emotional, rather than intellectual, process of communication that enables us to unlock processes, conscious and subconscious, which help us live, and those things have been done by humans in theatres since the beginning of time as far as we can tell."

But how do we actually measure this process with children and young people?

In Live theatre, the audience response is an excellent measure. Not how many are watching but how are they watching? Studying children’s faces watching a play, particularly in the daylight of a school hall is a good source of evidence. Are their eyes engaged in the action? Is there a collective gasp at a reveal in the plot? Are they laughing at moments you expect, or at other moments? Are they talking about the show afterwards? And if you are interested in how the performance has impacted on the child over time, a piece of coursework, a review or a conversation, may evidence how they have been prompted to think of themselves and the world in a different way and how they may develop an opinion about aspects of their society and their community.

In this way the imprint of the arts experience impacts on personal development and communal meaning. The quality of the artistic experience is at the centre; the social impacts are secondary. As Tim Joss from the Rayne Foundation says, “you don’t join a choir for the purposes of social cohesion…”

And how should organisations work together to address this holistic case?

Kelly Lean from the Royal Opera House Bridge makes the following point.

"The most significant thing for me is the confirmation that this is a collective challenge. Colleagues across the arts sector are grappling with the same issues/challenges/barriers. It feels like we are all well versed in talking through the limitations of our evaluation models but find it much harder to find better approaches to evaluating and measuring impact. It’s a work in progress but having space to give this more thought and talk to professionals in the arts sector and beyond has been a great way of starting the dialogue around the impact of the arts."

Other colleagues from the Impact Measurement project have wrestled with these challenges and compared practice and thinking across organisations.

'It is hard, it requires time, commitment, buy-in and resource,' says Kathryn Simpson from Whitechapel Gallery.

‘We’ve acknowledged that it’s messy and that’s the point' says Clare Lovett from Spitalfields Music.

The project web-site brings together the learning from the group and signposts colleagues to further material. We encourage you to visit the site, give feedback and spread the word.