Research on ‘Quality’ in Online Experiences for Museum Users

Introduction

Potential 'Best Practices' in Developing Quality in Online VMC Experiences

When VMC products are successful in having quality in online experiences, what should be considered in the development process? Based on the interview sample, the best practices that seemed most important to project managers and multimedia included:

• Common vision, clear goals and objectives, consistency, and a solid structure.
• Topics chosen that have clear and simple, yet emotional and powerful stories.
• Dedicated individuals with a love and passion for the product they are developing, and collaborative partnerships.
• Adequate time and budget to develop as high a quality product as possible.
• Navigation and access to provide a clear organization of information and images.
• Thinking in virtual rather than physical medium.
• Products that are user-focused, useful, and used by online visitors.
• Leading edge technology sympathetic to the content and messages of partners.
• Production process considerations that ensure accessibility.
• Maintenance of sites and after launch follow-up:
• Evaluation of the process, team, user statistics, and Feedback messages.
• Flexibility in design to make changes based on feedback from partners and users.
• Building on learning for future collaborative projects.
• Sharing lessons learned with partner heritage organizations and museums and multimedia companies.

Project Concept

Common vision, clear goals, objectives, consistency:

• A common vision at the outset, which is very clearly defined.
• Stronger producers with a vision, or an auteur model with a visionary is the answer to making better sites where there are no boundaries (e.g., a project driven by an individual with a great idea who finds people to help build it and gets funding from corporations and government to push forward a small number of national projects).
• The framework is in place very early in the project, and all decisions are made around learning objectives.
• Partners identify a theme, organize information, and create a schema.
• Being fastidious in the development process, not taking anything for granted, not cutting corners, and doing what it takes to get the job done.
• Consistency, a solid structure in place, and a Web site for collaborators on which all the documents are provided at all times (i.e., eliminating problems like Internet communication breaks, and facilitating more content contribution and sharing of documents).
• Partner organizations' primary goals are to learn as much as they can about components, design, and content, share knowledge, and take ownership of their product. One of the important outcomes should be that partners learn how to create more accessible online as well as on-site exhibits for their visitors.

Topics with clear and simple yet emotional and powerful stories:

• Making the story clear, simple, and compelling. A good story and theme are very important (e.g., the stories and images on the Yukon Photographers site are very emotional, powerful, and touching -- photos indicate that after men found gold their dress became richer and there was a look of hope on their faces).
• Giving users a chance for personal expression (i.e., the development team going a little bit further than just coming up with a theme, following a framework, establishing structure, obtaining collections, and putting the collections online).
• Providing credibility of online information, and developing sites that are not too narrow, academic, or institutional focused.
• Choosing VMC products that are more issues-based and asking more in-depth questions with content that is politically and socially meaningful.

Dedicated individuals and collaborative partnerships:

• The will, love, and passion of the people working on the product.
• Partnership as the key to success, and having people want to go beyond a sense of duty.
• Including on a team international as well as national partners, and dedicated individuals (e.g., huge consortiums have produced excellent sites when there is a rich group with lots of ideas).
• A collaborative approach to determine the development and production path.
• Lots of discussion early in the creative process and the team meeting together, preferably in person, and working things out (i.e., the development team accomplishes in one or two days what they can in conference calls over one month).
• Develop learning objectives as a project team (i.e., specific things to get across), and sites that are not purely entertainment.

Planning

Adequate time and budget:

• A comfortable budget and more time to develop and produce VMC products (i.e., to build something that is attractive, include the research needed to develop the content, develop the design, and find new ways of developing an architecture that is interesting and lively).
• A longer development cycle and production window in large projects where more partners are involved.
• Allowing time and budget to prototype and do focus group testing.
• Discussion and consensus-building between and among partners with face-to-face meetings where the team works things out together early in the creative process.
• Multimedia companies, as well as project managers, managing the scope of a project, timelines, and the company's efforts. Multimedia developers tend to have more of a technological advisory role, and partners take the lead in conceptualizing and developing the approach to a VMC product.

Design

Navigation and access with clear organization of information and images:

• Clear, easy to understand, and logical navigation. If users need to step back to figure out how to move forward, you lose them.
• Simplicity often works very well.
• Think about best access to content for target audiences.
• Information has to be broken up so it does not overwhelm users (e.g., using chronology or metaphor). It is better to fragment text to make it more readable.
• Offer a variety of choices of multimedia files.
• Determine how users can best access sites with large amounts of images and content.
• Organize information, create a schema, and develop clear, easy to understand, and logical navigation.
• Break up information so it does not overwhelm users (e.g., using chronology, or powerful personal stories or metaphors).

Thinking in virtual medium:

• Partners think in the virtual medium rather than think in terms of a physical exhibition. They learn how to create a museum experience for visitors in a virtual, online medium, and push the boundaries of traditional curatorial practice.
• Experiment with things that no one has done to help advance the medium (e.g., new technology, such as holography to move from a 2-D to a 3-D experience).
• Leverage 'dynamic content management technologies' to get users coming back. When all the time and investment are put into producing VMC products, Virtual Exhibits should not be built to be one-off sites with a shelf life of a year after they are built. Historical, informative types of sites tend to slowly retire with utilization dropping off (e.g., in a case like the Butterflies site, new species and new facts are always being discovered). Different generations of Web sites offer different levels of designs.
• Develop the VMC product as a virtual medium, which is user-oriented rather than a physical exhibit, which is object-oriented.

Products that are user-focused, useful, and used:

• Establish user requirements at the beginning of a project with a clear idea of whom the development team is trying to reach, what they are trying to give to target users, and what users might be trying to get from the VMC product. Questions to consider include:
• How are users going to use the site?
• What will they try to get out of it?
• What are users going to be trying to do?
• What is the development and production team trying to do?
• Offer a viewpoint and a product that is more user-oriented than object-oriented.
• Develop unique sites and experiences that individual users choose to have based on their interests. Multiple types of VMC products are needed within each site because users have different moods when they are surfing the Web at different times (e.g., escapism, learning, or entertainment). Different things reach different people in different ways at different times.
• Create something dynamic and fluid, which relates to a sense of season and time period as well as being connected to an institutional exhibit. The experiences users could have need to be identified at the outset, as well as opportunities that will give users a reason to come back to further explore the site.
• Provide intellectually satisfying online challenges for users (e.g., a quiz). Individuals will take time to work on the challenge, and if they invest time they will want a reward (e.g., to learn something).
• Consult target users to find out their user requirements (rather than guessing what they might be) and check regularly with users before the site is done (i.e., more front end and formative evaluation by launching an idea, seeing what users want, and observing their reaction).
• Measure success by determining whether a product has been deemed useful and continues to be used by users.

Leading edge technology sympathetic to content:

• Use technology that is sympathetic to content and the messages that partners are trying to convey (e.g., using QuickTime to display a transformation mask on Haida Spirits of the Sea with time lapse sequencing to open the mask and reveal the transformation).
• Take advantage of leading edge technology but be careful not to be exclusionary by making content broadly available to as vast an audience as possible.
• Develop a technical plan to manage the content and apply the technologies.
• Assess technology capabilities of partner heritage organizations and museums, and resources required where partners have limited technological capability.

Production

The production process:

• W3C accessibility standards for persons with disabilities are a set of regulations that developers need to follow for VMC products (http://www.w3.org/WAI/). Target browsers standards and limitations of the VMC server's technology as the host site tend to limit the pushing of conceptual boundaries. The CHIN server can limit technology possibilities for VMC products, which inhibits the user experience.
• Plug-ins need to be available for users to enjoy all the elements of a site. Giving users the technology that they prefer (e.g., QuickTime for Mac users, and Windows Media Player for PC users) provides online visitors with choice of technology and reduces initial frustration accessing content.
• For school presentations of an interactive game (e.g., Transport Canada was planning to use Safe Trax as a visual aid when they went into schools to talk about rules, regulations, safety, and trespassing), the content could be tailored to be more effective in a classroom environment rather than a one-on-one experience.
• Involving young people in the creation and development process requires finding ways to keep them motivated, stimulated, and interested in working on the project. Taking ownership of their section of the project helps them to feel proud of the product they have helped to produce.
• Integrating Interns into the project is an important opportunity for the Interns but can be challenging for multimedia developers. Interns tend to have a limited level of experience and require more time for mentoring.

Maintenance

After launch follow-up:

Evaluation of the process, team, user statistics, and Feedback messages:

• A feeling that partners have accomplished something on many fronts -- they got the story out, promoted the museum community, and showcased the partners' collections.
• Evaluation of the team, and bringing in new team members for follow-up projects (e.g., the development of the Yukon group's VMC products).
• Examination of Feedback messages over multiple exhibits to see where gaps are and what people are asking for. Email is a powerful feedback source that isn't being used enough. When evaluating Feedback messages, if users accessed files 1,000 times and there are only 10 emails, there probably are not a lot of problems.

Flexibility in design to make changes based on feedback from partners and users:

• Build in flexibility in design, and the ability for staff to quickly make revisions. Although the content is accurate at the time the site is launched, it is open for different interpretations, ongoing debates, and information sharing. User feedback may require correcting content when users report inaccuracies.

Building on learnings for future collaborative projects:

• The success of Haida Spirits of the Sea led the way for Our World - Our Way of Life. Indian and Northern Affairs point to the first Web site as one of their exemplary sites and use the materials for their outreach programs.
• The Yukon group's next Web sites will use the search model the VMC has developed. An image will come up after a search request with a short description of the site, what type of plug-in programs users need to explore them (e.g., Flash), and links to related Yukon sites

Sharing lessons learned with partner heritage organizations and museums:

• Share lessons learned with the community by writing up the lessons, explaining what was learned, and recommending what to avoid.

After VMC product launch:

After a site has been delivered there is very little communication between partners and the multimedia company, other then tweaking, enhancements, or regular warranty work when the project terminates. The relationship with the partners tends to drop off and the multimedia company ceases to track their satisfaction, concerns, and successes with the product. How to track users and build a site that meets their interests and changing needs is critical to a project's lasting impact for partner museums and heritage organizations.

After the launch of VMC products, project managers should maintain the content, or ask the multimedia company that produced the site to maintain the site (i.e., updating content rather than re-developing the site). Fourth and fifth generation sites today are doing that, although the sites are not as engaging from a graphical or interactive perspective. 'Dynamic content management technologies' can ensure that users come back to a site.