Producing Online Heritage Projects

2. Project Development

2.5 Evaluating Your Product in Development

Sample Versions and Prototypes

A sample version shows samples of content, graphics and navigation. It may not have the pages linked and functions working as a prototype would.

A prototype is a working model that incorporates the look and feel of the product as a whole ³³.

A prototype includes:

• look and feel of the product and the navigation
• graphic style
• samples of the content
• samples of the flow of the story through branching points
• samples of other unique and special features that will be part of the Web product

The earlier in your production schedule that you can create a prototype to use for testing, the better. It can be much more cost-effective to implement recommendations while the production is still in progress than having to modify the completed work.

Testing and Quality Assurance

As the project manager, you set quality standards for the production team, and team members accept responsibility for ensuring their work meets those standards. All work should be tested before proceeding to the next steps.

Some ways of testing your Web product:

• Test on both IBM-compatible and Macintosh platforms.
• View the completed work in Netscape and Internet Explorer browsers.
• View the work in older and newer versions on both types of browsers.
• Do spelling and grammar checks.
• Do a links check.
• Test e-mail addresses.
• Test functionality of everything that has been developed.
• Check that the contents of Web pages load within ten seconds or less.
• Test your online product on various computers without a high-speed connection and on old hardware. Consider testing your Web product on a computer at a local library terminal or at home.

Once each team member has tested his or her own work and believes it is ready, the project manager or quality assurance manager should review and test the work again. This second test should take place when the files have been uploaded to the final hosting server where the product will reside. Differing versions of server software and hardware can have an effect on the end product. For instance, if a form or database was programmed in a specific programming language, you must ensure that the host server supports the same version of that language.

Doing the Research—Focus Groups and Surveys

If your schedule and budget permit, it is recommended that you use either a focus group or an online survey to test your product with your target audience(s). This allows you to gather information about your target market, as well as specific feedback on your Web product while it is still in development. The hands-on experience immerses your audience in the product and is more likely to elicit more detailed feedback than an abstract interface or concept test.

Questions to ask testers of the prototype:

• Do you find the design pleasing? Why?
• If so, why? If not, why not? Why?
• Do you find the navigation clear and easy? Why?
• Would you use the Search function of this Web product?
• What keywords would you search on?
• Would you wait for the multimedia file to download?
• What do you find compelling about the layout?
• Would you change anything about the content?
• What would you add in the way of features or content? What would you change about the content?
• Do you find this easy to use? Why?
• Would you find this product useful (or valuable)? Why?

Usability testing

Usability testing is another method of gathering information on your developing product. This involves having a user move around the tested Web product while under observation. In this case, you need to strike a balance between allowing the user a completely natural experience and moving them through the Web product to ensure critical areas are reviewed. To gather worthwhile data from a usability test, you need to do this with several participants.

Online research

Because your product will be on the World Wide Web, the users testing it do not all have to be in the same place as they are for a focus group. Testers should be recruited by similar means to those used for the focus group. Because there is no moderator to take them through your Web product and record their responses, an online questionnaire or survey must be prepared.

To create a survey for online testers, compile a set of questions that require several "yes/no" and "more/less" type of responses to make it easy for respondents to make their selections, and to facilitate the compilation of results. You should also include questions that allow the user to write comments in an unrestricted comment box. A programmer should be able to devise an electronic survey form to capture the responses of the participants.

For online testing, a longer period of time should be allowed for completing the online survey. Five days or a week should be enough for the testers that have been recruited to find the time needed to test the prototype. In the introduction to the survey, communicate how long you think the user will need to spend reviewing the prototype and responding to the online survey.

An incentive is another option to be considered. For example, all completed surveys received before a certain date could be eligible for a free pass or some other thank-you gift.

If it were a large sample, then a database to capture all of the results would be most effective. Determine whether your programmer is experienced in database design and programming. If the sample is small, less than thirty for example, then receiving the survey results in e-mail form will permit you to review the comments and tally the results manually.