The beauty of behaviour indicators

9 October 2015

...In this capacity, I was approached by Project Oracle in June 2015 to join their Arts Cohort – a group of arts organisations, like us, brought together to improve the evidence capabilities of our specific part of the youth sector.

Now I know that some people find evaluation problematic or a necessary evil in order to secure funding. But surely it’s about getting your arts project right; feeling confident that it achieves what you want it to achieve?

I must admit I had become a little set in my ways when it came to evaluating and collecting that mysterious thing called ‘data’. As a facilitator of arts projects, I am very aware of group dynamics, participant interactions and behaviours; I relied on instinct to gauge - in that moment of delivery - whether or not an exercise or arts activity was working. I never thought about taking a step back and using those skills to create more meaningful questions and to think about the longer term.

So there I was in a meeting with Project Oracle, with my project outcomes and aim, feeling all smug about the impact of our project, when they asked a simple question that suddenly became very difficult for me to answer: ‘'What does successful behaviour look like?’ Instinct led me in search of a few stock answers: participants will feel more confident, and I’ll know this because they will have ticked a box in a questionnaire or answered a question on their favourite moment in the project.

And that’s when I realised our approach to evaluation wasn’t robust or thought through.

So how do you go about designing a rigorous evaluation that really puts the participants at the centre of the process?

In order to understand our impact, I had to think about what success looks like in terms of the way people behave. In other words, I needed to ask myself: ‘what are the behaviour indicators?’ With this one simple question the idea behind evaluation shifted from an external focus to one where my skills really lay, in encouraging and guiding participant interactions and behaviours.

At Drake Music we deliver music-making workshops. A simple objective might be that we want people to feel more confident using music technology. To evaluate this we could simply ask: ‘Do you feel more confident with music now?’ But what does this really tell us beyond a yes or no answer, on a scale of say 1 to 5?

Instead, if we think about the behaviour indicator - what confidence using music technology looks like - we can begin to get a deeper understanding and map a participant’s progress in more detail. Confidence in music technology may therefore look like someone using music technology more often or teaching their friends what they have learnt about it.

I now have three indicators, three new and more interesting questions, and much more information about the behaviour habits of the individuals taking part. The questions I ask at the start and end of a project are: ‘How often have you used music technology over the last 3 months?’ or ‘How often have you taught your friend about Music Technology in the last 3 months?’ and ‘How often have you shared your music online in the last 3 months?’

Most arts facilitators will be able to tell you a 1000 stories about the participants they work with - about how they see them change over time. For me, a meaningful evaluation is one where you can harness that strength to evaluate and make a robust and meaningful case that demonstrates the impact of the arts on young people.

We know it’s true and it’s our role to share it. The Arts Cohort has allowed members to come together to share problems, ideas and - more importantly - creative solutions that will help us learn from our experiences and make our case stronger. I look forward to the months ahead.

Daryl Beeton is the London Regional Programme Manager at Drake Music, the leading national organisation working in music, disability and technology. The organisation is expert in special educational needs and disability (SEND) music delivery and in using technology to break down disabling barriers to music-making.

If you like this blog, have a read of 'The cohort model: A new direction for Project Oracle'