Understanding your audiences through Evaluation

Understanding who is coming to your museum means you’re better informed to develop your content, interpretation, marketing, strategy, planning – you name it. Knowledge is key as it gives you: 

• a better understanding of your audiences and their likes/needs 
• the ability to identify conclusions and recommendations based on fact – not just anecdotes 
• the opportunity to identify what you are doing well and where there is room for improvement 
• evidence that you are using resources (time and money!) effectively 
• the chance, through using the information you have captured, to make a case with future funders, advocating to decision makers and local government 

That’s why evaluation makes so much sense! We know that evaluation often only takes place when requested by a funder or stakeholder but the output provides you with so much insight it’s invaluable so, where possible, try to embed it into your work. At the beginning be sure of your aims and objectives – they will guide everything you do throughout the evaluation process. 

Take a look at what others are doing and see if you can tweak this to suit your needs. Here’s some suggestions and, as you can see, all agree that there are benefits in undertaking evaluation!   

• Visitor Studies Group
• Learning and Interpretation: Evaluation at the V&A  
• Natural History Museum, Visitor Research Evaluation  
• Heritage Fund, Evaluation of your National Lottery funded project (whilst this is linked specifically to Heritage Fund requirements it is a useful guide for anyone conducting evaluation) 
• Sheffield University, Evaluation Toolkit  
• British Museum, Visitor Research: Evaluating Exhibitions  

In an ‘ideal’ world, evaluation has three key stages: 

• Front-end evaluation which is carried out before a project begins. 
• Formative evaluation which is conducted before the project is completely finalised – ie do some research six months into the project with audiences to see if you are still on track or if anything needs refining 
• Summative evaluation which is conducted at the end of a project to assess to what extent the original objectives have been met.  

These are detailed in the links outlined above with some featured case studies. Unfortunately, because evaluation is not always embedded in a project at the outset all too often it is just summative evaluation that is completed, meaning there is no time left to address any issues which have arisen during the previous two stages. Make sure evaluation is not an afterthought! 

Sometimes there will be a need to commission external consultants with some or all elements of the evaluation. If this is the case make sure you write a clear, concise brief which will elicit the proposals you need. Take a look at this document in the Visitor Studies Group toolkit: 

Visitor Studies Group, Audience Research Brief  

Where possible include an element of skills sharing, i.e. how to write a survey,  in the brief so that when the project has finished you have acquired some new skills which you can use to good practice in the future. 

Qualitative research is best used when we are concerned with understanding things (i.e. views, perceptions, behaviours etc) rather than with measuring them. It also assists with the identification of how people think and feel and is used to capture this information rather than gather statistical data. 

We will use qualitative research to: 

• Explore issues in detail 
• Discuss and debate topics – ‘why do you say that’, ‘why do you think and/or feel that way?’ 
• Bring people together to discuss a specific topic or subject 

It will most often take the form of focus groups (in person or online and usually 8-10 people), mini groups (approximately five to six people), in-depth interviews (i.e. one-to-one interviews conducted over the telephone or in-person). Participants are given an incentive for taking part and this can sometimes be as much as £30 per participant BUT offering refreshments i.e. coffee and cake during the day, and maybe a glass of wine in the evening, can also be appealing!    

These discussions use a topic guide to ensure that the output meets with the aims and objectives ie you have explored in detail the key issues and topics.   

The topic guide – or discussion guide – is the equivalent to the quantitative researcher’s questionnaire and is designed to encourage debate and discussion whilst ensuring there is a structure to the session. The facilitator must be adept at engaging with people, listening to what they are being told and creating a discussion rather than a yes/no response.  

In contrast, quantitative research is performed on a larger scale than qualitative research and is about gathering accurate statistical data from which conclusions can be drawn. 

‘It is a systematic approach to investigations during which numerical data is collected and/or the researcher transforms what is collected or observed into numerical data.’ (LearnHigher, Manchester Metropolitan University) 

We will use quantitative research, therefore, to: 

• Provide statistical evidence – it’s about numbers and percentages! 
• Gather feedback from a wider population than qualitative research 

Surveys can be delivered online and face-to-face and when designing a survey consider: 

• Importance of the introduction – why are these questions being asked and for whom 
• First question – should be answerable by everyone 
• Should move from general topics to specifics 
• Keep ‘sensitive’ questions to the end – build up trust 
• Ensure the length is not prohibitive – people will tire and not complete 
• Avoid leading questions 
• Look out for areas that are dominated by social expectations eg ‘how many times a day do you brush your teeth?’ 
• Consider the order in which questions are asked – the flow is important  

When choosing methodologies, consider do we want to look in detail at the way people think/feel about something, do we want to encourage debate and discussion or do we want to be able to say 40% of people said such and such? Once you have considered that – along with your aims and objectives – you will have a steer as to whether qualitative or quantitative research is your best approach. 

Sometimes we do both – if time and budget allow. There is real value to doing both – but which do we do first? There is no definitive answer to that. Sometimes doing qualitative first means we gain a better understanding (ie why people think the way they do) which helps us to develop a survey which is good if we don’t know what perceptions people may have. Sometimes we do quantitative first so that any issues which emerge can be explored further in qualitative research.  

Whilst surveys are the means by which most museums capture visitor data, if you don’t have the resources, or skillset, to use this methodology don’t be put off. There are other ways you can consult with visitors. The list below is by no means exhaustive but gives you an idea of the range of methodologies at your disposal. When selecting a methodology reflect on who your visitors are (from what you have observed) and consider: 

• Their age 
• Whether they are accompanied by children 
• How much time they have spent onsite 
• What do you know about them already and what do you want to know now? 
• How you will analyse the findings  
• Who will use and how you will use the information you capture? 

Potential methodologies: 

• Observation – Watch how your visitors move around a space, engage with each other, engage with content, how long they spend in specific areas, etc. In this way you can count the number of people walking through a space or engaging at a specific point. This can be a useful starting point for your research because you understand where they enjoy spending time, who enjoys spending time where, what bottlenecks are created, which interactives are particularly popular and so on. V&A used observational research to understand how its visitors engaged with digital exhibits: https://www.vam.ac.uk/blog/digital/digital-exhibits-observational-research 
• Comments books – Might sound old-fashioned but sometimes the old ways are good ways. By leaving out comment books in strategic places it provides visitors with the opportunity to note down feedback on their experience. You can also ask visitors to note down where they have come from (local or further afield) and whether they are a first time/repeat visitor. 
• Interviews – It doesn’t always need to be a survey but there are times it is useful in capturing feedback. Remember the survey needs to be well-designed and delivered by someone who is good at engaging with visitors. When designing the content remember to be guided by the aims and objectives of the study. All too often we are tempted to add in ‘just one more question’ but keep it focused so the length is not a barrier. https://www2.open.ac.uk/students/skillsforstudy/conducting-an-interview.php 
• Online surveys – sometimes visitors don’t like to spend too much time during their visit providing feedback so why not ask for an email to which you can send an online survey. Offer an incentive to boost completion – it can be something simple like a £25 High Street voucher – but it’s a thank you for taking the time. Survey Monkey, co.uk, is a cost-effective way survey tool and will also provide you with tables of analysis which are very user-friendly. They also provide advice on survey design: https://www2.open.ac.uk/students/skillsforstudy/conducting-an-interview.php# 

• Focus groups/mini groups – a group of people interviewed on the day or at a later time. The facilitator is key here – someone who can listen and engage well with people. A group of six to eight for a focus group and four to five for a mini group is the norm. Use a topic guide to facilitate the discussions making sure that you are receiving feedback on the points important to you. 
• Postcodes – capture postcodes and then you’ll know where your visitors are coming from – check against a drive time map and if you have additional budget think about using Mosaic to gain greater insight to the visitor. 

Make feedback fun for the visitor. If it’s bright and colourful visitors will gravitate towards the feedback mechanism, sometimes without the need to approach them.

Use coloured post-it notes or pieces of paper to create a graffiti board or washing line with a key question inviting feedback.  Set out flip chart paper on different tables which invites visitors to respond to specific questions/stimulus material.  Use sentence completion such as ‘What I enjoyed most about my visit today was…’, ‘What I discovered during my visit today was…’, ‘One thing that would have made my visitor experience better would have been…’.      

What is useful and interesting about this type of approach is that often people read what others have written and either agree with what has been said or offer a different opinion. They may feel happier about doing this in written format than if they were taking part in a discussion. 

Why not ask children to draw pictures about what they enjoyed most providing space and crayons for them to do so. Use the washing line approach to hang up those pictures and/more comments – it looks attractive and inviting. 

These options provide rich qualitative feedback which will enhance any visitor studies report. 

It is important that when you capture details from visitors that you are complying with GDPR. GDPR it is a practical approach to safeguarding personal data. Make sure you refer to GDPR when you are asking for names, emails, telephone numbers and postcodes.

For more on the GDPR, make the Office of the Information Commissioner your first port-of-call. 

Having acquired all this knowledge about visitors, how do we analyse the findings so that they are meaningful. Consider what you want to know and how you will analyse it at the project’s outset making sure you have the time required. 

When analysing quantitative research: 

• Use a system with which you are familiar ie SurveyMonkey, SPSS, Snap, Excel etc 
• Identify ways in which you want to analyse the data eg age, gender, first time/repeat visitor, place of residence, whether visiting with children under 16 etc. This will help you to identify, for instance, that visitors aged 17-24 are most likely first time visitors, local people are most likely visiting with children etc. 
• If more than 50 people have been included in the study use percentages. If less than 50 then state ‘number of respondents’. If you use percentages for a small sample it can be misleading   

With qualitative data: 

• Keep an open mind – don’t jump to conclusions. It’s all too easy to think you know everything after only one focus group! 
• Highlight areas of agreement and disagreement between participants within your notes 
• Identify significant words and phrases 
• Identify emerging themes – what are the common points across the discussions
• Mark anything which stands out from the rest of the data 
• Link the findings back to the aims and objectives 
• Select quotes to illustrate points 
• Do not attribute quotes to individuals by name but by visitor profile eg female, 19, student, Aberdeen 

So now you have analysed the data and you want to share your knowledge with others. As with analysis, reporting should be thought about at the outset as it will influence your approach and analysis. 

There are a number of options but whatever route you take remember to clearly present the findings and avoid jargon!  Use tables, charts and graphs to show data visually, use quotes that have been captured through qualitative research and text to explain the key points arising from the study. Potential reporting formats included: written, detailed reports (usually in Word), topline reports (topline report with key takeaways), verbal debriefs, a PowerPoint presentation. Sometime it might be one or more of these options.         

In summary… 

• Evaluation is not an afterthought! 
• Identify aims and objectives at the outset and let them guide all elements and stages of the project. 
• It doesn’t always have to be a survey – consider the best way to consult with your visitors and the resources you have available. 
• Where relevant, collaborate with others (within and out-with your organisation). 
• Share your findings.