2004 Survey of Visitors to Museums’ Web Space and Physical Space: Survey Documentation and Findings
1.1 Population
This survey targeted visitors to Canadian museums, of which there are about 2600. Because of operational limitations, especially since it was the first time that CHIN undertook a survey of that magnitude, the survey population was limited to museums that were CHIN members and that were in operation during the survey period (July and September 2004). As of May 2004, there were 991 of those museums. The museums were classified by size based on the number of employees according to the information on CHIN’s member database. The museums were classified as either large (50 or more employees), medium (3 to 49 employees) or small (2 or fewer, including volunteer-run and size unknown). The museums were also divided by region. The following table presents the breakdown of museums by category:
Table 45: CHIN Member Museums as of May 2004, by Region and Size
1.2 Sample Plan
Based on the population figures presented in Table 45, the required sample size in terms of museums was determined in order to achieve the required precision for survey results at the national level. The assumptions used in determining the sample size were:
- 80 completed interviews would be required from large and medium museums;
- 40 completed interviews would be required from small museums;
- About half of the visitors would have access to the internet;
- 30% of those would have used the internet to plan their visit for large and medium-sized museums and 15% for small museums;
- A margin of error of ± 7.5% was required for a 95% confidence interval based on questions answered only by those who used the internet to plan their visit (the questions with the smallest number of respondents).
Based on those assumptions, it was determined that a sample of 80 museums (5000 visitors) would be necessary. This total sample size was allocated to each size and then region in order to get adequate estimates by region and by size (but not by combination of size and region). Medium and large museums were over sampled, because of the assumption that more internet use for planning would be observed in those museums. The sample sizes obtained were as follows:
Table 46: Projected Museum Sample Size, by Region and Size
According to the assumptions listed above, this would lead to the following sample sizes in terms of visitors, as well as in terms of visitors who used to internet to plan their visit.
Table 47: Projected Number of Visitors to be Interviewed, by Region and Size
Table 48: Expected Sample Size in Terms of Visitors Who Used the Internet to Plan Their Visit, by Region and Size
For each selected museum, interviews were conducted during two weeks in July (July 17 to 30) and two weeks in September (September 11 to 24). The opening days and hours, as well as the expected number of visitors, were obtained directly from museums for each of the survey weeks to determine the interview periods and sample. Two days per survey week were then randomly selected and for each of those days, a two-hour period was also randomly selected (the selection of museums in each stratum, as well as the selection of days and hours were all done using simple random sampling). A systematic sample of visitors was then to be selected for each time period chosen for the survey. For example, the interviewer was instructed to interview the second visitor exiting the museum after the start of the interview period (say 2:00 PM), and then every third visitor after that. Those numbers were based on expected number of visitors provided by museums.
1.3 Survey Questionnaire
The questionnaire used for the survey is given in Appendix B. Since the Survey of Visitors to Museums is an exit survey, the goal was to keep it short, while still providing the required information. The questionnaire was submitted to some museums for comments. It was then tested in a museum and also reviewed by Statistics Canada’s Questionnaire Design Resource Centre. The goal was to test it in three museums, but it was difficult to find museums available that had staff available for the test at the specified time.
1.4 Data Collection
Interviews took place for 16 hours in each museum, during the weeks mentioned and according to the sample plan described in section 1.2 (4 weeks x 2 days per week x 2 hours per day). CHIN provided funding for each museum to hire someone to do the interviews (generally someone who was already a part-time employee or volunteer for the museum). A package was sent to each museum, containing an Interviewer Guide (detailed instructions on how to conduct the interviews), an Interview Control Form and enough copies of the questionnaires for the whole survey period. Each museum was asked to return the questionnaires used at the end of each two-week period.
Although museums are responsible for providing updated information for the CHIN member database, changes that occurred during the time the survey was being planned would have an effect the accuracy of the information. Also, the interest and availability of museums to participate in the study was not known in advance. For those reasons, more museums had to be contacted, in order to get the targeted number of participating museums for each stratum (combination of region and size). In spite of those efforts, the achieved sample sizes were lower than anticipated. The following table gives for each stratum the number of museums contacted, with the status of those museums.
Table 49: Final Results for the Sample of Museums
Of the 104 museums that were considered in-scope for the survey, 60 agreed to participate, a participation rate of 58%. Some museums chose not to participate in the survey because they were running their own surveys at the same time or because they didn’t have the resources to dedicate to that task, among other reasons.
As mentioned in section 1.2, 40 completed interviews were expected from small museums and 80 for medium and large museums. Based on that requirement and the achieved sample sizes, 3880 completed interviews were expected (down from the 5000 originally expected). This target was not achieved, mainly because the actual number of visitors was lower than expected during some survey periods and also because some museums missed one or more days of collection. The final number of completed interviews for each stratum is as follows:
Table 50: Number of Completed Interviews, by Region and Size
1.5 Data Capture and Data Processing
CHIN captured the completed questionnaires in an Excel spreadsheet. Questionnaires were batched in groups of about 20 and quality control of the capture was done using acceptance sampling (where a sample of questionnaires from each batch was verified to determine if the batch was to be accepted or rejected. For batches that failed this first verification, 100% verification was performed). The Statistical Consultation Group (SCG) was in charge of processing the data. Minimal edit and imputation rules were applied.
1.6 Estimation
Survey weights were calculated to represent the multiple stages of the sample design. First, a museum weight was calculated, based on the numbers presented in Table 49. Then weights were added to take into account the random selection of days in each week, hours in each selected day, and visitors within each time period. The final survey weights, obtained by combining the weights at each stage mentioned above while taking non-response into account, were used to calculate survey estimates. The detailed survey results are provided in Appendix A for each question. Two quantities are provided: the total number of visitors in the category and the proportion of visitors with that characteristic. For example, looking at Question 1, an estimated total of 613,234 visitors to museums during the survey period were internet users. They make up 81% of all the visitors in the period. Because of the sampling design used, estimated proportions are considered more stable than estimated volumes.
Estimates of sampling variability were obtained using the bootstrap method. This method consists of selecting several sub-samples (i.e. museums) from the survey data, with replacement (meaning that the same museum could be selected more than once), each time duplicating the original sampling process. This way, several estimates can be obtained for each quantity estimated by the survey, and the sampling variability can be estimated by looking at the variation between the estimates from each bootstrap sample. Two hundred bootstrap samples were selected for this purpose.
Estimates of sampling variability are provided with the data in Appendix A. In each table, the margin of error of a 95% confidence interval is provided for the largest estimate from each row. For example, looking at Question 11, 53% of visitors to small museums said they visit museums a few times a year. The margin of error (provided on the left, with the row title) for this estimate is +/- 7%. Estimates of sampling variability were calculated for each survey estimate, but only one per row (the one for the largest category) is provided. The others are available, if required.
2.1 Population, Sampling and Data Collection
The target population for this survey was all visitors to the Virtual Museum of Canada and to the web space of Canadian museums. Because of the challenges of building an intercept survey on all those different sites, and the limited resources to implement the survey, it was decided to put a link on each site and offer visitors the opportunity to participate in the survey. CHIN member museums were offered information on how to add a link to the survey questionnaire on their site and 35 museums accepted the invitation. The survey was available from mid-July to mid-September. A total of 567 respondents completed a survey questionnaire (57 from the link on the VMC site and 510 from various museums’ pages).
Although it is known that voluntary, self-selected surveys do not necessarily yield results that are representative of the overall population (in this case, the whole population of visitors to the VMC and to museum’s sites), it was the only possible option in this case. The survey results are only representative of the opinions and behaviors of the sampled individuals (who could be people who have more interest in the topic or who are more frequent visitors, since they chose to participate in the survey); nonetheless, it provides valuable insights as to what people expect when they visit such sites.
2.2 Survey Questionnaire
The questionnaire used to conduct the survey on museum’s web spaces is provided in Appendix B. At the end, a few minor differences in wording between that questionnaire and the one available on the VMC site are mentioned. The questionnaire was tested with people who were not involved in the survey process and was also reviewed by Statistics Canada’s Questionnaire Design Resource Centre.
2.3 Data Capture and Data Processing
Since the Survey of Visitors to Museums’ Web Space was an online survey, the results were entered directly in the database as respondents were answering the questions. The Statistical Consultation Group (SCG) was in charge of processing the data. Minimal edit and imputation rules were applied.
2.4 Estimation
Because of the reasons mentioned in section 2.1, only the sample results are presented, with no attempt to generalize those results to a broader population.
