Understanding your audiences through Evaluation

Understanding who comes to your museum means you’re better informed to develop your content, interpretation, marketing, strategy, planning. Knowledge is key as it gives you:

• a better understanding of your audiences and their likes/needs
• the ability to identify conclusions and recommendations based on fact – not just anecdotes
• the opportunity to identify what you are doing well and where there is room for improvement
• evidence that you are using resources (time and money!) effectively
• the chance, through using the information you have captured, to make a case with future funders, advocating to decision makers and local government

Sometimes evaluation can be overlooked throughout a project, but the insight it provides is invaluable. Where possible, try to embed evaluation into your work. Set aims and goals at the beginning or any work project. They will guide you throughout the evaluation process.

Basic evaluation has three key stages:

• Front-end evaluation which is carried out before a project begins.
• Formative evaluation which is carried out before the project is completely finalised – i.e. do some research six months into the project with audiences to see if you are still on track or if anything needs to be changed.
• Summative evaluation which is conducted at the end of a project to assess to what extent the original objectives have been met.

Summative evaluation is often undertaken as evaluation isn’t embedded in a project from the start. Leaving evaluation until the end of project means there’s no time left to address any issues from the previous two stages.

It’s helpful to look at how other museums and galleries undertake evaluation. The following resources include a mixture of different kinds of evaluation.

• Visitor Studies Group
• Natural History Museum: Visitor Research Evaluation
• Heritage Fund: Evaluation Good Practice
• British Museum, Visitor Research: Evaluating Exhibitions

There will be times you may need to work with external consultants to undertake some or all elements of an evaluation. Writing a clear, concise brief of your requirement will help encourage consultants to take on your commission. The Visitor Studies Group have created a template brief for audience research that you can adjust to meet your project needs.

Qualitative research is best used when we want to understand things (i.e. views, feelings, behaviours etc) more meaningfully than measuring them. It also helps identify how people think and feel.

We use qualitative research to:

• Explore issues in detail
• Discuss and debate topics – ‘why do you say that’, ‘why do you think and/or feel that way?’
• Bring people together to discuss a specific topic or subject

This research is often gathered through focus groups (in person or online and usually 8-10 people), mini groups (approximately five to six people), in-depth interviews (i.e. one-to-one interviews conducted over the telephone or in-person). Participants are given an incentive for taking part. This can sometimes a financial incentive per participant but offering refreshments i.e. coffee and cake during the day, and maybe a glass of wine in the evening, can also be appealing.

These discussions are facilitated using a topic or discussion guide. This helps keep them on track and ensures that the output meets with the aims and objectives. i.e. you have explored in detail the key issues and topics.

The topic guide is designed to encourage debate and discussion whilst ensuring there is a structure to the session. The facilitator must be adept at engaging with people, listening to what they are being told and creating a discussion rather than a yes/no response.

In contrast, quantitative research is performed on a larger scale than qualitative research. It gathers accurate statistical data from which conclusions can be drawn.

We will use quantitative research to:

• Provide statistical evidence – such as numbers and percentages
• Gather feedback from a wider population than qualitative research

Surveys can take place remotely and face-to-face. When designing a survey consider:

• Importance of the introduction – why are these questions being asked and for whom
• First question – should be answerable by everyone
• Should move from general topics to specifics
• Keep ‘sensitive’ questions to the end – build up trust
• Ensure the length is not prohibitive – people will tire and not complete
• Avoid leading questions
• Look out for areas that are dominated by social expectations eg ‘how many times a day do you brush your teeth?’
• Consider the order in which questions are asked – the flow is important

It’s important to think about the outcome you want to achieve when choosing methodologies. Do you want to look in detail at the way people think/feel about something? Do you want to encourage debate and discussion? Or do you need statistics for reports? Considering this as well as your aims and objectives will steer you towards a qualitative or quantitative research approach. Or sometimes both if budgets allow.

Surveys are the most common museums capture visitor data. This can be resource intensive and there are other ways you can consult with visitors. The list below gives several basic suggestions of methodologies you can use.

Potential methodologies:  

• Observation – Watch how your visitors move around a space, engage with each other, engage with content, how long they spend in specific areas, etc. In this way you can count the number of people walking through a space or engaging at a specific point. This can be a useful starting point for your research because you understand where they enjoy spending time, who enjoys spending time where, what bottlenecks are created, which interactives are particularly popular and so on. The V&A used observational research to understand how its visitors engaged with digital exhibits.

• Comments books – By leaving out comment books in strategic places it provides visitors with the opportunity to note down feedback on their experience. You can also ask visitors to note down where they have come from (local or further afield) and whether they are a first time/repeat visitor. 

• Interviews – It doesn’t always need to be a survey but there are times it is useful in capturing feedback. Remember the survey needs to be well-designed and delivered by someone who is good at engaging with visitors. When designing the content remember to be guided by the aims and objectives of the study. You may wish to add in ‘just one more question’ but keep it focused so the length is not a  

• Online surveys – sometimes visitors don’t like to spend too much time during their visit providing feedback so why not ask for an email to which you can send an online survey. Survey Monkey is a cost-effective survey tool. It will also provide you with user friendly tables of analysis. They also provide advice on survey

• Focus groups/mini groups – a group of people interviewed on the day or later. The facilitator is key here. Someone who can listen and engage well with people. A group of six to eight for a focus group and four to five for a mini group is the norm. Use a topic guide to facilitate the discussions making sure that you are receiving feedback on the points important to you. 

• Postcodes – capture postcodes and then you’ll know where your visitors are coming from – check against a drive time map and if you have additional budget think about using Mosaic to gain greater insight to the visitor.

When selecting a methodology reflect on who your visitors are (from what you have observed) and consider:

• Their age and life stage.  
• Who they are with – are they accompanied by children. 
• How much time they have spent
• What do you know about them already and what do you want to know now?
• How you will analyse the findings.  
• Who will use and how you will use the information you capture?

There are ways to make feedback fun for your visitors. If it’s bright and colourful visitors will gravitate towards the feedback mechanism, sometimes without the need to approach them.

Use coloured post-it notes or pieces of paper to create a graffiti board or washing line with a key question inviting feedback. Set out flip chart paper on different tables which invites visitors to respond to specific questions/stimulus material. Use sentence completion such as

• ‘What I enjoyed most about my visit today was…
• ‘What I discovered during my visit today was…
• ‘One thing that would have made my experience better would have been…

Having transparent feedback that everyone can see lets visitors validate what others have said or offer an entirely different opinion. They may feel happier about doing this anonymously in written format than if they were taking part in a discussion.

You can ask children to draw pictures about what they enjoyed most about their visit, providing space and crayons for them to do so. Use the washing line approach to hang up those pictures and/more comments – it looks attractive and inviting.

These options provide rich qualitative feedback which will enhance any visitor studies report.

It is important that you follow the law when gathering and keeping details from visitors.

The UK General Data Protection Regulation (GDPR) it is a practical approach to safeguarding personal data. These regulations combine with data protection principles to make up a set of rules and responsibilities that organisations must follow when gathering personal data.

While data protection can feel intimidating The Information Commissioner’s Office has created simple guides to help organisations understand their responsibilities. The Association of Independent Museums (AIM) has also created a Success Guide to help small museums navigate privacy and data regulations.

Once you have data on your visitors, you need to analyse the findings so that they are meaningful. Consider what you want to know and how you will analyse it at the start of the project.

When analysing quantitative research:

• Use a system with which you are familiar ie SurveyMonkey, SPSS, Snap, Excel etc
• Find ways in which you want to analyse the data eg age, gender, first time/repeat visitor, place of residence, whether visiting with children under 16 etc. This will help you to identify, for instance, that visitors aged 17-24 are most likely first time visitors, local people are most likely visiting with children etc.
• If more than 50 people have been included in the study use percentages. If less than 50 then state ‘number of respondents. If you use percentages for a small sample it can be misleading

With qualitative data:

• Keep an open mind – don’t jump to conclusions. It’s all too easy to think you know everything after only one focus group!
• Highlight areas of agreement and disagreement between participants within your notes
• Identify significant words and phrases
• Identify emerging themes – what are the common points across the discussions
• Mark anything which stands out from the rest of the data
• Link the findings back to the aims and objectives
• Select quotes to illustrate points
• Do not attribute quotes to individuals by name but by visitor profile eg female, 19, student, Aberdeen

There are a number of options to share your findings at the end of your evaluation. Whatever route you take remember to clearly present the findings and avoid jargon. Use tables, charts and graphs to show data visually, use quotes that have been captured through qualitative research and text to explain the key points arising from the study. Potential reporting formats included: written, detailed reports (usually in Word), topline reports (topline report with key takeaways), verbal debriefs, a PowerPoint presentation. Sometimes it might be one or more of these options.

In summary

• Evaluation should be planned from the start and performed throughout the project at agreed stages.
• Identify aims and objectives at the start and let them guide all elements and stages of the project.
• It doesn’t always have to be a survey – consider the best way to consult with your visitors and the resources you have available.
• Where relevant, collaborate with others (inside and outside your organisation).
• Share your findings and use them to make changes to improve.