Measuring Transformative Learning Part 2: Checklists and Rating scales

dilbert

© dilbert.com

Who doesn’t love a good checklist or rating scale? Have a read and tick a response – easy to complete, easy to analyse and easy to report!

 

While qualitative methods have been used widely in measuring transformative learning (TL), I’m wanting to look at more quantitative approaches at this stage (as referenced here). In this, the fifth post in a series looking at TL, I’m gathering checklists and statements from a variety of sources that we may use in the future, so am popping them on this blog both to share and for easy access. The following are some examples of, what I think, could form the basis of a potential survey (and are in no particular order). Formal references are at the end of the post.

MARVEL Museums Actively Researching Visitor Experiences and Learning (Griffin, Kelly, Savage and Hatherley, 2005)

The MARVEL research project used a series of categories in a checklist format to self-evaluate how an exhibition fosters learning:

  1. Content
  2. Comfort
  3. Coherence
  4. Challenge
  5. Control
  6. Choice
  7. Collaboration
  8. Curiosity
  9. Continuing learning

Each item has a sub-set of “measures” that unpack each concept. The detailed form explaining these in more detail can be downloaded here and is a useful tool, especially for project teams to self-assess an exhibition before, during and after development.

Visitors and Learners: Adult Museum Visitors’ Learning Identities (Kelly, 2007)

In my doctoral work I used a range of rating scales to investigate museum visitors’ thoughts about learning. In Stage One of the study, visitors to the Australian Museum were asked to rate the following 11 statements that described various aspects of learning based on how important they were to them as a learner:

  1. Learning in a physical, “hands-on” way
  2. Learning when the information provided is of immediate interest to me
  3. Learning that builds on what I already know
  4. Learning that specifically fits with how I like to learn
  5. Teacher-led learning at school/other formal place
  6. Being told what to learn
  7. Constructing meaning based on my own experiences
  8. Changing how I see myself
  9. Seeing something in a different way
  10. Learning with and through others
  11. Learning new facts

In Stage Two, visitors to a particular exhibition were asked to rate a set of statements in order to find out how they felt they learned in the exhibition, rating the following ten statements on a four-point scale of Yes / a lot; Yes / somewhat; No / not really; Not at all:

  1. I discovered things I didn’t know
  2. I learned more about things I already knew
  3. I remembered things I hadn’t thought of for awhile
  4. I shared some of my knowledge with other people
  5. I got curious about finding out more about some things
  6. I was reminded of the importance of some issues
  7. I got a real buzz out of what I learned
  8. It was pleasant to be reminded and to learn more
  9. It was all very familiar to me
  10. Some of the things I learned will be very useful to me

Note that many of these statements were derived from the literature review and the MARVEL project referenced above.

Australian Museum: Visitor Motivations (Chris Lang)

The Australian Museum has surveyed visitors since 2012 using a set of statements uncovering visit motivations as follows:

  1. It’s an Australian / Sydney attraction / ‘must do’ activity
  2. I am drawn to interesting buildings and places
  3. It’s an enjoyable way to pass the time
  4. It is a nice place to spend time with my friends and family
  5. To encourage children’s interest in a subject at the Museum
  6. To improve my own knowledge or experience of a subject at the Museum
  7. I have a personal interest in a subject at the Museum
  8. I have an academic or professional interest in a subject at the Museum
  9. For a strong sense of personal connection or identity
  10. To see fascinating, awe inspiring things
  11. To see beautiful things in an attractive setting
  12. To stimulate my own creativity
  13. For peaceful, quiet contemplation
  14. To escape and to recharge my batteries

Chris then grouped responses into what he calls a “hierarchy of engagement”:

  • Social (a-d)
  • Intellectual (e-h)
  • Emotional (i-k)
  • Spiritual (l-n)

More details about this idea and the results from a range of exhibition evaluations are available on the Audience Research Unit Evaluation Reports section of the Australian Museum’s website.

Motivational Factors and the Experience of Learning in Educational Leisure Settings (Packer, 2004)

Jan Packer’s doctoral work investigated the impact of a range of motivational factors on both how and what visitors learned in museums and other leisure spaces. Jan used a number of rating scales in her work, with the following one I think is most relevant (2004, p.207):

  • The information was presented in an interesting way
  • I was reminded of something I already knew or had experienced
  • I had the opportunity to participate actively
  • The topic “clicked” with some of my personal interests
  • The information was relevant to my life
  • It provided “food for thought”
  • The information was surprising or unexpected
  • The information was new to me
  • I was able to discuss the information with a companion
  • The information appealed to my emotions
  • The information appealed to my imagination
  • I had the opportunity to ask questions
  • I was able to see the real things or places the information referred to

These were rated by participants along a scale of -3 to +3, but I’ll think I’ll stick to a four or five point scale.

Exploring visitors’ perceptions of the value and benefits of museum experiences (Packer, 2008)

While technically not a rating scale, these questions from Jan’s 2008 research (below) also offer a good starting point when thinking about transformative learning experiences in museums:

  1. What do you feel you have gained from the visit?
  2. If you think about yourself now, and yourself when you first arrived, what would you say has changed? (“I am more ……; I am less ……”)
  3. Would you say your mood has changed at all?
  4. Did you learn anything about yourself during the visit?
  5. Has it changed the way you feel or think about yourself?
  6. Has it changed the way you understand your place within the world?
  7. Was there anything about the visit that made you feel good (or bad) about yourself or about the world?
  8. Did the visit enhance your relationship with your companion(s) in any way?
  9. Was there any part of your museum visit that spoiled your experience in any way?
  10. In general, what do you value most about visiting museums? Why is this important to you? (prompting on entertainment, relaxation, discovery, and social interaction aspects of the visit)

I think these questions would also work on a follow-up basis, i.e. for interviewing visitors down the track.

Exploring Satisfying Experiences in Museums (Pekarik, Doering and Karns, 1999)

An oldie but a goodie, written by authors that have influenced many of us. This scale I feel will be useful for both on-site and follow-up surveys (p.167):

Object experiences

  • Seeing the real thing
  • Seeing rare things
  • Being moved by beauty
  • Thinking about owning such things
  • Continuing professional development

Cognitive experiences

  • Gaining information or knowledge
  • Enriching my understanding

Introspective experiences

  • Imagining other times or places
  • Reflecting on meaning
  • Recalling travels / other memories
  • Feeling a sense of connectedness
  • Feeling a spiritual connection

Social experiences

  • Spending time with friends / family
  • Seeing my children learning

This work was continued and built into a theory of experience preferences (IPOP), through four typologies (Pekarik, et al, 2014) distilling visitors’ primary interests under the following categories:

  • I = ideas
  • P = people
  • O = objects
  • P = physical

Finally: I’ll continue gathering away, in the meantime if anyone has any other scales or questions that may be relevant please feel free to share them here.

REFERENCES

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s