Reflections on the #MGAConf2018 for Audience Research and Evaluation


2018 conf image

@feraldata #meetday2108

By Abbie McPhie, Audience Research Manager, National Museum of Australia

As the new Audience Research Manager at the National Museum of Australia and being (relatively) new to working in the museums sector, the MGA 2018 conference and the MEET day held prior to the conference represented a significant opportunity for me to reflect on the issues impacting the sector and what these mean for audience research and evaluation.

A few common themes stand out, and pose some questions for the industry:

  • We need to understand who is not currently attending our institutions, and why. Are they being excluded from our institutions in some way? In what ways? And why does our content appeal to our current audience – might this partially explain why it doesn’t appeal to, or even actively excludes, other audiences.
  • For smaller museums and galleries, audience research is front and centre as they receive feedback directly and often, and can see the immediate impact of changes made on the basis of this feedback. For larger institutions, budgets will usually allow for some form of audience research and evaluation (even if this isn’t always done!). But for mid-sized organisations (under 100,000 visitors / under 50 staff), audience research and evaluation is often unable to be budgeted and not front and centre in day to day work. How do we support mid-sized museums and galleries in understanding their audiences and measuring success.
  • And finally, there are still a number of institutions and museum professionals for whom measuring success is in itself still a new concept – how do we support the introduction of a culture of evaluation throughout the industry?

I have a few immediate thoughts: getting institutions in the habit of understanding what their objectives are in any given piece of work (the ‘why’ factor) will help with understanding how to measure their success, and example template visitor surveys could be developed that would help those who are unsure where to start. But I’d also welcome your thoughts on this – please leave your comments below or contact me on

[Note: Abbie was the recipient of the Evaluation and Visitor Research National Network 2018 conference bursary – thanks for the post!]


‘Ten things for my museum colleagues working in digital’: a response #MGAConf2018 #musetech @sebchan

Chairing the digital trends session at the recent #MGAConf2018 and listening to Seb’s 10 provocations got me thinking. And since Seb was interested in my response, here goes (at least for those I have opinions on).

2. You need to stop comparing your museum to a US museum.

We are delivering to an increasingly international audience – tourists visiting Australia or locals that have travelled and have been to the big ‘uns, whether in US, UK, Europe or, increasingly, Asia. As many museums audience researchers will tell you over the years we are finding audiences comparing what they’re seeing in Australia to what they have visited overseas, so in that sense we do need to compare, yet we do need to be different. I realise this point was in the context of issues other than visitors but still worth reminding ourselves that our “competition”, for want of a better word, includes overseas institutions, so naturally some of their conversations are our conversations.

3. You almost certainly can’t afford the digital unicorns.

No, but you could find new ways to work with these unicorns and tap into their expertise through partnerships (think Google Cultural Institute) or co-working spaces (think ACMI-X, DX Lab, Science Museum’s Digital Lab, etc). Tech companies want content, and guess what, we have that in spades.

4. Your most successful digital initiatives are marketing campaigns.

So why is the relationship between digital teams and marketing teams still so fractured?

5. An app still won’t save you although it might make your board happy for another quarter.

Well, maybe an app won’t save you, but what it will do is give you some digital skills:

  • Working with external vendors
  • Thinking about audience, user-testing, etc
  • Writing content for mobile
  • Working with different platforms and digital tools
  • Working as a team
  • Working out ways to re-purpose content, use objects in a different way, etc
  • How to roll-out and market a digital product

So, while an app these days may be seen as a ‘vanity’ project with little reach, use it as a learning opportunity. After all, how many exhibitions we develop are the same?

6.-8. Privacy etc

I’m not even going to go there, only to say that issues around copyright and IP are not excuses to “do nothing”.

9. Digitisation is not the solution, it’s just a new set of problems.

See point 5 (kinda).

10. What’s your institutional capacity building strategy?

Yes, let’s all have one of these, but in the absence find ways to take small steps and build capacity from the ground-up, especially if the upper echelons are still having the “physical vs. digital debate”. Some good ideas emerged from the #GettingDigitalDone session at the conference and if you’d like to sign up to that group for more sharing go here (and yes, we are gathering your email address, we are complicit, but you’re either part of the conversation and learning, or you’re not).

Thanks Seb, always a pleasure to see what you’re up to and where you’re thinking is and look forward to more when we next meet – at the COMPASS conference (Conference on Mobile Position Awareness Systems and Solutions) in San Francisco I believe (and a shameless plug here!).

#MEETDAY2018 – an event not to miss!


We welcome all species to #MEETDAY2018!

So, I know that you are:

  • Wanting to know how to create a 360 video easily 
  • Interested in how to co-design 
  • Dying to find out what’s happening in the education, evaluation and tech fields across the cultural sector 
  • Keen to hear Genevieve Bell in an intimate setting 
  • Interested in networking √

So, #MEETDAY2018 is for you! And, conveniently, just the day before the #MGA2018 conference.

To view the program and activities go to the #MEETDAY2018 website. We have an exciting group of lightning talks so far from museums, galleries and academia as follows:

  • Using feedback to create a successful digital excursion (MoAD)
  • Re-imagining the little Leaellynasaura polar dinosaur: a co-creative tactile Mixed Reality museum experience (Deakin University)
  • Testing audience response to increasing temporary exhibition prices (NMA)
  • Deepening engagement with students (MCA)
  • Educators as curators of student learning (University of Melbourne)
  • Big ideas, little people and craft: PlayUP (MoAD)
  • “This is our voice” – inviting visitor voices into learning programs (MAAS)

Go to the links below to book your place (and once you have added your order to your cart, please go up to the top menu to check out – just some helpful advice!):

If you have any questions please contact me as the EVRNN Convenor:

See you there – it’s gonna be great!

If the museum was a person #TBT


The pop-up museum

Gotta love the web. Just when you’re in the thick of something up pops a gem from the past that causes you to stop, think and reflect. For this #throwbackthursday post I came across some work we did back in 2008 with students asking them to describe the museum as a person. As usual, when working with young people their responses are not only insightful, but cute and hilarious at the same time! I was working at the Australian Museum then, so their answers very much reflect their thoughts within the context of a natural history museum, but they also give a whole lot more.

So, here we go – If the museum was a person:

  • If the Museum were a person, it would have been around 200 years old. Also it would have huge brain containing all of the evidence about dinosaurs and animals. This person would have a huge heart because it also helps people discover or learn something they haven’t seen or heard before. It is a female, because not everything smart can be a man. It tells us about things we didn’t know existed.
  • It would be an historian because it’s mainly about things from many years ago.
  • It would be a very nice person and smart to tell people about the world and it is very old.
  • If the Museum were a person it would be wise and know a lot about the past. It would be an outgoing person because it always lets people in.
  • It will be a famous kind because the museum is famous and it is a bit like a castle because of its shape. He is a good actor because the museum has a lot of different things. He knows a lot of information because the Museum is full of information.
  • I think if the museum was a person he/she would be old and full of knowledge. I think this, because of all the exhibits in the museum are full of the knowledge and the museum looks very old.
  • If the Museum was a person it would be a palaeontologist because the museum is full of dinosaurs.
  • Old Grandpa. Knows a lot about history and what happened around the world. Nice, funny and smart.
  • Smart, interesting, lots of stories to tell. Funny, bad at sport. Nice
  • It would be a very knowledgeable person, because it has loads of information about the past.
  • I think it would be a massive brick monster because it is huge and has a lot of bricks in the building. But it tells a lot of stories.

The original post appeared on my old (and first!) blog- The Audience Research blog.

Other readings on student learning in museums

  • Kelly, L. (2013). 21st Century Learning – A Students’ Perspective.
  • Kelly, L. (2014). Student learning in museums – what do we know?
  • Kelly, L. and Fitzgerald, P. (2011). Cooperation, collaboration, challenge: how to work with the changing nature of educational audiences in museums. In Rethinking Educational Practice Through Reflexive Inquiry. Ed. N. Mockler and J. Sachs. 77-88. Springer: London.
  • Kelly, L. and Groundwater-Smith, S. (2009). Revisioning the Physical and On-line Museum: A Partnership with the Coalition of Knowledge Building Schools. Journal of Museum Education. 34(4), 55-68.
  • Kelly L. and Russo, A. (2010). From Communities of Practice to Value Networks: Engaging Museums in Web 2.0. In Hot Topics, Public Culture, Museums. Ed. F. Cameron and L. Kelly 281-298. Cambridge Scholars Publishing: London.
  • Groundwater-Smith, S. (2002). Evidence Based Practice in School Education. Paper presented at the Why Learning? Seminar. Australian Museum.
  • Groundwater-Smith, S. and Kelly, L. (2009). Learning outside the classroom: A partnership with a difference. In Connecting Inquiry and Professional Learning in Education. Ed. A. Campbell and S. Groundwater-Smith. 179-191. London: Routledge.

#MEETDAY2018 #MGA2018

Created with Nokia Smart CamMEET is the annual gathering of educators, evaluators and technologists working across the cultural sector, held in conjunction with the MGA conference. The 2018 MEETDAY will be held on Monday 4 June, 9.30-5pm at the National Sports Museum, Melbourne Cricket Ground, with a morning session of keynote speaker and lightning talks, followed by an optional afternoon session of workshops and other goodies.

MEET 2018The overall theme of the day is making connections, and each Network is focusing on one of the other conference sub-themes:

  • Education – breaking barriers
  • Evaluation – championing innovation
  • Technology – places of engagement


Registration from 9am for 9.30am start

9.30 Welcome and introduction to MEET

9.35-10.15 Keynote Speaker Professor Genevieve Bell, ANU (includes question time)

Knowing our audience, knowing our tools and guiding our communities: given the current pace of change, how are we going to shape our future within the contexts of evaluation, technology and education?

10.15 Morning tea (provided)

10.45 Lightning Talks

There will around nine Lightning Talks – three from each Network. If you would like to propose a talk, complete this form by Monday 9 April. Please note that submission doesn’t necessarily mean acceptance.

12.00 Panel / wrap-up / Q&A

12.30 Lunch (provided, including tours of the Melbourne Cricket Ground and the opportunity to visit the National Sports Museum)

2.00-5.00 Optional afternoon workshops:

  1. 2-3pm Easy 360 video creation – make your own 360 video in one hour! Jonny Brownbill, Museum Victoria
  2. 3-4pm Co-design – what it is and how to do it: case studies from the field Lynda Kelly, LyndaKellyNetworks / Andrew Hiskens and Linda Angeloni, State Library Victoria
  3. 4-5pm Voice to text chat interfaces – how can they be used in our institutions? Jonny Brownbill, Museum Victoria

Tours of National Sports Museum will run through the afternoon. There is a café downstairs for those who just want to chat.


To cover fees and expenses, we have priced MEET 2018 at:

  • $125 for MGA conference attendees
  • $150 non-conference attendees

If you have registered for the MGA conference go here to register and pay

If you are only attending MEETDAY go here to register and pay


  • Contact Dr Lynda Kelly, Convenor, Evaluation and Visitor Research National Network,
  • Updates will be provided on the MEET website
  • Follow us on Twitter: #MEETDAY2018
  • And, go here to propose a Lightning Talk


NSM logo

MEETDAY2018 is supported by the National Sports Museum


Transformative Learning in #STEM at #ASTEN2017

TL group 1As part of the ASTEN conference in October, 2017, we had a session devoted to (you guessed it!) Transformative Learning (TL) with a focus on STEM. This post summarises the outcomes of the workshop, giving a science flavour to our discussion so far on TL.

At the beginning of the session, participants completed a feedback form asking them to describe a TL experience, then to tease out indicators they look for when visitors are having a TL experience – similar to the work we did with teachers and staff at MoAD.

As ASTEN participants are primarily based in science communication and science learning, their take on TL addressed mostly science experiences, with some synergies with the literature review of TL and STEM.

A TL STEM experience – some examples

  • The first time I saw a Bernoulli Blower exhibit in my first visit to a science centre.
  • Attending the Milan Furniture Fair and learning about European design / manufacturing … engaging with a manufacturer to develop one of my designs / products … innovative product recognised, feelings of empowerment, change, recognition.
  • Finally working out how an Archimedes screw worked … felt very much an “aha” moment when I could see exactly how it worked, and that I could make one.
  • First visit to Exploratorium, SF. Late in the afternoon, ran out of time. Lost track of my children because I was so busy, last to leave.
  • Making a water heater in our Maker Space – the heat lamp melted the tubing, water went everywhere but helped show me why hands-on is so important when forming ideas.
  • Primary school visit to Questacon – hands-on activities, I remember quite clearly visiting. One of the few excursion memories that I have.
  • Learning about air pressure and that vacuum cleaners don’t suck; the air is pushed into a space of low pressure. Changed the way I understand physics – helped with cleaning! – but the experience gave me a mindset to stop, step back, observe and see if I could explain phenomena differently.
  • Presenting shows and workshops. Became aware of how the presentation style and language used allows for an engaging, transformative experience. Learned how important story-telling was to communicate an idea.
  • Te Papa Bug Lab. Totally changed the way I saw and valued the physical / sculptural environment to immerse people in a completely unfamiliar world. WETA workshop expertise transformed an often icky topic (insects) into a magical wonderland where the humans were smaller than the bugs. Interactivity, conversation, innovation and mood creation deepened the experience for visitors of all ages.
  • Following the Copenhagen conference on climate was really influential. At the time I was working in science and really believed that in time science would steer the conversations – seeing the interplay between science, politics and culture during that time was powerful.
  • Time with a major Architect discussing informally the practice of how it connected to all the disciplines.
  • Learning the history of computer development and realising it was aligned with war (Computer History Museum, 2015). Felt shocked, looked around to see if others were experiencing the same thing.
  • Experiencing the southern night skies at Tekapo, NZ.
  • New job – steep learning curve.
  • When I was 3-4 years old visiting Queensland Museum and the turtle diorama had an interactive component (still does!), where you touched eggs o feel different temps to understand role t has in determining sex.

Indicators of a TL experience

From the feedback forms the following indicators were mentioned:

  • Making connections – cognitive, social/personal, identity, physical
  • Link it to their lives or translate into their life experiences
  • Change in behaviours or attitudes
  • Emotive response
  • Engagement – a ‘light bulb’ moment
  • “Ow wow!” – when you hear children calling out
  • Smiling, ‘wonderment’ on face, facial emotions
  • Having fun and engaging / interacting with objects
  • Noises of struggle but continued persistence
  • Lose track of time
  • Increase in dwell time
  • Focussed, moving slowly through a gallery space
  • Excitement
  • Exclamations – “that’s cool”, “come and look at this”, “I hadn’t thought of that”, “wow”
  • Anger at information presented – views and beliefs being challenged, shock
  • Open body language
  • Taking photos
  • Conversations, sharing experiences at the time or post-visit
  • Telling someone else about the changes
  • When a child and family are working together intensely at an exhibit, kids want to show parents, adult/child interaction
  • Test new learning in real life
  • Asking questions to further understand concepts
  • Come back for more

In the group report-back session we classified the indicators into a series of categories –  cognitive, behavioural, cognitive, emotional and social – as in the images below.

Thanks for participating in the session and I think we teased out some really good ideas, with some new and different takes on the subject, complementing the outcomes from the MoAD staff workshop and Canberra/ACT-based teachers.

Specific audience studies re-visited

As part of some work I’m doing am uploading a series of research reports here for easy access.

Older Audiences and Museums

IMG_20180226_114232.jpgOlder people are an important and increasing group of museum visitors, particularly in Australia. The proportion of the Australian population aged 65 years and over grew steadily during the twentieth century and is projected to grow further during this century. In 1901 there were 151,000 people aged 65 years and over living in Australia , or 4% of the population. By 1998 this number had increased to 2.3 million, or 12% of the total population. It is projected that by 2051 this will have grown to between 6 to 6.3 million, or around 24-26% of Australians.

During 2002 a collaborative study was developed and managed by the National Museum of Australia, Canberra, the Australian Museum, Sydney and Environmetrics, a private market research company in Sydney, into the needs and expectations of older visitors in order to recommend ways that museums can respond to these. The research targeted Australians aged 65 years and over and included those who currently visited and engaged with museums and those who didn’t. There were three phases to the project: an extensive literature review; a quantitative study via a telephone survey of Sydney and Canberra adults; and several qualitative projects consisting of depth interviews and discussion groups.

The published report , Energised, engaged, everywhere: Older Australians and Museums by Lynda Kelly, Gillian Savage, Peta Landman and Susan Tonkin (ISBN 0 7347 2311 3), provides comprehensive statistical and qualitative information about this group, specifically focussing on leisure habits and museum visiting. As well, a set of recommendations were made for museums to consider when programming for older audiences, listing over forty achievable things that museums can do to attract and satisfy older people that are universally applicable.

Download the report here: Older Audiences full report

Family Audiences and Museums

IMG_20180226_114254.jpgAcross the world, museums have paid considerable attention to the needs of children and families through the development of targeted exhibitions, activities and programs and as special-purpose areas. In 2004 a joint study was undertaken between the Australian Museum and the National Museum of Australia This study brought together current information about family visit experiences and made recommendations to enhance these.

There were two parts to this study: a literature review and field research. The detailed literature review includes studies spanning the past 70 years and was commissioned to obtain an up-to-date information about families; how they are defined, how they use museums and the ways that they learn. The field research included a total of 29 case study families in Sydney and Canberra who had visited either the Australian Museum or National Museum of Australia. The sample included a variety of family types with children under 12 years of age.

The report, Knowledge Quest: Australian Families Visit Museums, can be downloaded here: Knowledge Quest full report

Museum Audiences with Disabilities

IMG_20180226_114303.jpgOne in five Australians has some type of disability. People with disabilities represent a large, diverse and important audience for museums and galleries. The aim of this study, a collaboration between the Australian Museum, the National Museum of Australia and Accessible Arts was to give voice to the views of visitors with disabilities and suggest ways in which museums and galleries can better support their access needs.

Overall, it was found that people with a disability are supportive of museums and galleries, are motivated visitors and see museums and galleries as excellent environments for learning, education and social interaction. People with a disability have strong networks and often visit in groups or with friends and family. Ensuring their access needs are met offers the potential for museums to increase both visitor numbers and the diversity of their audience. In addition, by finding better ways to meet the needs of visitors with disabilities, museums and galleries will automatically be improving the visiting experience for all visitor groups.

The published report from the study, Many Voices Making Choices: museum audiences with disabilities by Peta Landman, Kiersten Fishburn, Lynda Kelly, and Susan Tonkin (ISBN 0 7347 2318 0), provides information about this audience, specifically focussing on their views about museums and museum visiting. As well, a set of recommendations were made for museums to consider when programming for these audiences, and a range of available resources are listed in the publication, which can be downloaded here: Many Voices full report

An audio transcript from the seminar held to launch the report (with instructions) can be accessed here.

The Young Adult Audience

The Pre-school audience




6P modelMy 2007 doctoral thesis investigated the question: What are the interrelationships between adult visitors’ views of learning and their learning experiences at a museum? I’m publishing it here for easy access and download (as well, I’m re-visiting some of this work for a range of different projects so wanted to have it to hand!).


Many museums around the world are reviewing the ways they are thinking about visitors and learning. Current theories of learning focus on the meaning individuals make based on their experiences — alone, within a social context and as part of a community. A critical aspect in better understanding the process of learning for individuals is to find out how people view themselves as learners across the rich array of available formal and informal learning experiences. Research has shown that when asked why they visit museums people often say “to learn” but there has been little exploration into what this means. What do museum visitors think learning is? How do visitors view themselves as learners within the context of a museum visit and does this change during and after their visit?

The research question investigated in this doctoral study was What are the interrelationships between adult visitors’ views of learning and their learning experiences at a museum? A key focus of the study was on how adults describe learning, the place of learning in their lives and where museums are situated. Other areas examined included the relationship between learning, education and entertainment, as well as the roles visitors play during a museum visit. The framework of learning identity was used to characterise how individuals describes themselves as learners within a sociocultural context, including their future views of learning and the roles learning plays in their lives.

The study was undertaken in two parts – Stage One investigated individuals’ personal philosophies and views about learning, and Stage Two explored how a museum exhibition experience provided insights into visitors’ learning identities.

It was found that participants in the study described learning in very rich and detailed ways, yet there were also a number of common ideas that emerged. It is proposed that museum learning can be framed under six interrelated categories – person, purpose, process, people, place and product – called the 6P model of museum learning. The literature review showed that visitors learn a great deal from museums across a diverse range of content areas and at many different levels. However, the method used in this study also revealed that visitors could learn more about the concept of learning as well as their own learning processes – likes, dislikes, preferred strategies – if they are encouraged to think about themselves as a learner before they engage with an exhibition.


The entire thesis can be downloaded here: FINAL THESIS FOR GRADUATION_KELLY

There are also individual chapters for download:

Co-design, co-curation, participation, crowdsourcing, etc – working with our audiences


My Cultural Object Program @austmus

As part of the Questacon work we are also looking at the issue of co-design – how to work with a variety of audiences, communities, etc in designing exhibitions and programs in a participatory way, encouraging a two-way relationship that generates positive benefits for all. These ideas (and a bit of history too!)are summarised in this post:  Ruminations on #crowdsourcing, participation and museums.

So again in the spirit of sharing, here are a range of co-design examples (in no particular order):


My Cultural Object Program @austmus

And, some co-design publications to read later:

Beyond facts and phenomena: Teaching and learning about science #TBT

heureka 2

Heureka, Finland

As part of this recent series on transformative learning and #STEM, this #throwbackthursday post revisits two articles that inspired me when undertaking my thesis (all those years ago!), while thinking about what they mean for transformative learning in 2018.

As I re-read my, somewhat battered, copy of a 1998 paper by James Bradburne, Dinosaurs and White Elephants: the Science Centre in the 21st Century, was reminded of how far we have come, yet also how so little has changed. Bradburne’s premise in this paper was that science museums need to fundamentally shift from being ‘fonts of scientific knowledge’ to showing visitors how to inquire. He identified a key number of trends impacting on science centres (and museums) and suggested that these institutions need to become a hybrid, or new learning platforms where users are considered as the starting point for effective learning. One measure of success suggested was that visitors leave “… not saying ‘I know’ but rather ‘I know how to know’” (p.119, emphasis in original).

There is an abundance of riches in this paper, so for the sake of brevity I’ll just summarise his conclusions (p.132-133):

  • Stress acquisition of new skills, not information: “These skills are largely shared by art, science and technology alike – creativity, collaboration, abstraction, thinking in terms of systems” and include finding, using, understanding and the ability to add to information
  • Turn visitors into users – find ways to embed our institutions within communities and encourage repeat use (visitation, whether physical or online) using lessons from how libraries operate
  • Think about “high value, not high volume” – find out how users learn and use this information in creating new experiences in an iterative (and design-thinking?) loop
  • Actively research visitors and share that knowledge, as well as using this knowledge to develop “effective new tools for teacher training”
  • Think global, act local – new learning platforms need to be based on what cannot be done somewhere else, using virtual communities to input to local communities and vice versa

Bradburne concluded by stating that the key to institutional survival is “… having the flexibility to respond to the needs of a wide variety of users” (p.133), and taking a leading role in informal learning by drawing on knowledge gained via research.

This led me to another paper, Perspectives on Learning Through Research on Critical Issues-Based Science Centre Exhibitions, by Erminia Pedretti (2004). In this the author talks about the new directions in science centres that go beyond ‘science as wonder’ and ‘objects as curiosities’ to an “emphasis on involvement, activity and ideas … [that include] social responsibility and the raising of social consciousness” (p.S35).

Pedretti outlines three types of science centre exhibits:

  • Experiential that enable visitors to experience phenomena
  • Pedagogical that set out to teach something
  • Critical that critically explore the nature of science, being: “… issues-based, inviting visitors to consider socioscientific material from a variety of perspectives, engage in decision-making and healthy debate of complex issues, and critique the nature and practice of science and technology … [emphasising] learning about science” (p.S36)

Pedretti concluded that critical exhibitions challenge visitors in different ways, appealing to a person’s intellect and emotions (sensibilities), enhancing learning by “… personalising subject matter, evoking emotion, stimulating dialogue and debate, and promoting reflectivity” (p.S45).

However, we know that visitors enjoy experiential exhibits, are naturally curious, like to learn (from simple facts to deep change), as well as engaging with exhibitions that make them think, so using the best elements from all three of these approaches may be the best way to develop and deliver transformative learning experiences.