The Trouble With Numbers

Earlier this month, the provincial Department of Tourism, Culture and Recreation (TCR) posted the results of their 2011 Visitor Exit Survey. According to TCR’s press release, the survey is the most detailed study of its kind the province has ever produced and is intended to provide a picture of who is visiting our province, what they are doing while they’re here, and how satisfied they are with those experiences. My first breeze through the document left me with the impression that it provides some good and not-so-good news for the heritage sector. A more thorough reading left me with more questions than answers about how visitation data is collected, how it’s interpreted, and what we might do in response. But first, a little bit of background. 

Survey Methodology 

The 2011 survey was given to individuals leaving the Province through four major exit points: the Port Aux Basques and Argentia ferry terminals and the St. John’s and Deerlake airports. Travellers at each of these locations were asked to complete an initial short survey. It collected information relating to trip purpose, party size and composition, length of stay, and point of origin. This helped identify members of the survey’s target group - non-resident visitors who were leaving the province. 

Once identified, members of the target group were asked to complete a longer survey which collected data on destinations visited, participation in activities, expenditures and demographics. Participants were given the option of returning the long survey via drop boxes at each of the four locations, by mail, or responding online. A total of 17,279 short surveys were conducted with non-residents leaving the province at the four exit points. 4,083 long surveys were completed and returned. 

TCR has published the survey results in a series of short profiles (you can find them here). Each one provides insight into visitor characteristics based on one of a variety of variables ... things like a visitor’s point of origin, or the purpose of their visit, or the activities they participated in. The following profiles are of specific interest to the heritage sector:

  • Profile of Non-residents Visiting Historic Sites
  • Profile of Non-residents Visiting Archaeological Sites
  • Profile of Non-residents Participating in Pleasure Walking In and Around Communities

The Good News

According to survey results, 59% of non-resident parties visiting Newfoundland and Labrador during the period May-October, 2011 reported they visited at least one of the province’s historic sites. That’s a total of 200,775 non-resident visitors. Woot! Woot! Other heritage-related activities reported by non-resident visitors included:

  • Visiting lighthouses (44%)
  • Visiting museums/archives (37%)
  • Visiting galleries and exhibits (34%)
  • Visiting interpretation/science centers (25%)
  • Visiting archaeological sites (15%)

For comparison, the following are the reported participation rates of non-resident visitors in nature/outdoor activities:

  • Pleasure walking in/around communities (67%)
  • Trail Hiking (32%)
  • Whale Watching (28%)
  • Sightseeing Boat Tour (27%)
  • Iceberg Viewing (21%)
  • Bird Watching (21%)

Clearly, cultural heritage is an important component of non-residents’ experiences in this province. 

And The Not-So-Good News

According to the results presented in the Profile of Non-Residents Visiting Historic Sites, 51% of travel parties reported visiting a National Historic Site, 38% reported visiting a Provincial Historic Site and just 15% reported a visit to other historic sites. My first reaction to those statistics was “Man, community-based heritage is really under-performing!”

A Closer Look

Because I am a heritage wonk, I thought it would be interesting to use the figures provided in the Visitor Exit Survey to compare those percentages with the province’s overall visitor stats for Provincial and Federal Historic Sites. Here are my results:

 

 

 

 

 

table.jpg

Notice anything strange? Remember, the column on the far right represents total visitation - that’s non-resident visitors + resident visitors. 

So, what’s going on? Here are my thoughts. First, we need to recognize that “total visitation” represents the total number of visitors who pay admission, view our exhibits, and/or participate in an organized program or activity. Essentially, these are the visitors who enter our facilities, encounter a staff person or volunteer, and are counted. For sites like Signal Hill, Cape Spear, Cape Bonavista and others, a very significant percentage (perhaps even a majority) of their visitors do not purchase admission, view exhibits, and/or participate in an organized activity. These visitors engage with the site for reasons other than heritage. They walk the trails, admire the views, nature watch and picnic, sometimes before or after opening hours, and/or outside the official operating season of the site. My hunch is that a significant number of the exit survey’s respondents fall into this category. 

That said, I don’t think this completely accounts for the wonkiness of the numbers. If this were the case, the relationship between resident and non-resident visitation at our National Historic Sites would look more like the Provincial Historic Sites numbers ... probably even more so. Let’s face it, who visits St. John’s and doesn’t head for the top of Signal Hill?  No, I think there’s something else going on. 

You say Potato, I say Potato

According to TCR staff, participants in the Exit Survey were asked to respond to the following question:

Which of the following did you participate/visit:

  • National Historic Sites
  • Provincial Historic Sites
  • Other Historic Sites
  • Galleries/Exhibits
  • Museums/Archives
  • Interpretation/Science Centres
  • Archaeological sites

Specific sites were not identified and definitions or examples were not provided. Simply put, visitors were left to their own devices to decide how to classify their experiences. For National Historic Sites, that’s pretty straight forward. Parks Canada is a nation-wide organization with a long history and a brand that’s widely recognized, particularly among fellow Canadians. But can the same be said of our Provincial Historic Sites? I don’t have any hard evidence for this, but my suspicion is that many visitors do not distinguish between Provincial Historic Sites and our province’s largest, independently-operated historic sites ... places like the Colony of Avalon, the Cupid’s Legacy Centre, and Trinity’s Historic Sites. And let’s face it, the division isn’t always clear. At Cupids, the archaeological site is a Provincial Historic Site. The neighboring Legacy Centre is not. At Trinity, visitors can purchase a single pass that provides entry to both the town’s Provincial Historic Sites and sites operated by the Trinity Historical Society.

And what exactly is a “historic site” anyway? How does one distinguish it from a museum? Come to think of it, how do you distinguish between a museum, an interpretation centre and a gallery? And don’t they all have exhibits? As a heritage professional, I have my own definition for each of these based on things like mobility/immobility of heritage resources and the role of collections, but does anyone honestly believe that visitors think about our sites this way? 

Professional Help

I’ll be the first to admit it ... I’m no statistician. So, I contacted the professionals at TCR’s Planning and Research Division. To their credit, they patiently answered all my questions and reviewed an earlier draft of this post. Here, in a nutshell, is what they said. Yes, my numbers look wonky, but that’s because I’m asking the Exit Survey’s results to do something they were never intended to do. The survey was designed to to provide insights into the characteristics of non-resident visitors to the province, not to estimate visitation levels to sites, museums, galleries, or other attractions. As a result, the survey’s methodology does not include "processes" which would allow an estimation of visitor numbers to specific sites and attractions. In other words, the statement “15% of non-resident parties reported visiting a historic site operated by others” does not mean that the actual non-resident visitation to non-government operated historic sites in 2011 was 15% of the total non-resident visitation to historic sites in the province for that year (Warning: You may have to read that sentence multiple times before it makes sense).

Why You Should Care

Does any of this really matter? I think so. Regardless of their intention, the numbers presented in the Visitor Exit Survey give the impression that it’s our National and Provincial historic sites that are really doing the heavy lifting when it comes to attracting visitors. Compared to them, the rest of the historic sites in this province appear to be under performing, attracting just 15% of our non-resident visitors. Using these numbers, it’s hard to paint a picture of community-based historic sites as attractive investments for either public or private funding, or as important engines of rural economic development. 

I totally get what the pros at TCR are saying. Really, I do. But I can guarantee that in the near future, I will be sitting in a meeting and someone - a politician, a bureaucrat, or potential funder - will pull out that 15% and lob it at me as evidence of our poor performance. At least now I have something to throw back.

Next Steps

I see this survey as a wake up call to our sector. The time has come for all of us to take visitor tracking more seriously. Yes, most of us count overall visitation to our sites. In fact, organizations that receive operational funding under CEDP are required to submit their total annual attendance as part of their final report. But a closer look at those numbers shows that there’s more than one way to count a visitor. Take the couple who stop by your site’s cafe for a cup of tea, but do not enter your museum. Are they “visitors”? What about the locals who walk your site’s trails every morning? Or the seniors’ bus tour who are only there to use your washrooms? Or the fish plant employees who park in your site’s parking lot because the one next to the plant is full? Where you draw the line can make a huge difference (for an eye opening look at this issue, check out this article from the St. Louis Post Dispatch).

Is it time we created and implemented a sector-wide standard for tracking visitor attendance? Should our sector establish our own internal collection and distribution point for these statistics? Let me know what you think.