The NCS Report - Charlottesville, VA 2022 - Flipbook - Page 4
Methods
Selecting survey recipients
All households within the City of Charlottesville were eligible to participate in the survey. A list of all households within the zip
codes serving Charlottesville was purchased from Go-Dog Direct based on updated listings from the United States Postal Service.
Since some of the zip codes that serve the City of Charlottesville households may also serve addresses that lie outside of the
community, the exact geographic location of each housing unit was compared to community boundaries using the most current
municipal boundary file. Addresses located outside of the City of Charlottesville boundaries were removed from the list of
potential households to survey. Each address identified as being within city boundaries was further identified as being within
one of the 6 areas. From that list, addresses were randomly selected as survey recipients, with multi-family housing units
(defined as those with a unit number) sampled at a rate of 11 compared to single family housing units.
An individual within each household was selected using the birthday method. The birthday method selects a person within the
household by asking the “person whose birthday has most recently passed” to complete the questionnaire. The underlying
assumption in this method is that day of birth has no relationship to the way people respond to surveys. This instruction was
contained in the introduction of the survey.
Conducting the survey
The 2,800 randomly selected households received mailings beginning on November 9th, 2022 and the survey remained open for 7
weeks. The first mailing was a postcard inviting the household to participate in the survey. The next mailing contained a cover
letter with instructions, the survey questionnaire, and a postage-paid return envelope. All mailings included a web link to give
residents the opportunity to respond to the survey online. All follow-up mailings asked those who had not completed the survey
to do so and those who had already done so to refrain from completing the survey again.
About 4% of the 2,800 mailed invitations or surveys were returned because the household address was vacant or the postal
service was unable to deliver the survey as addressed. Of the remaining 2,682 households that received the invitations to
participate, 371 completed the survey, providing an overall response rate of 14%. The response rate was calculated using
AAPOR’s response rate #2* for mailed surveys of unnamed persons.
It is customary to describe the precision of estimates made from surveys by a “level of confidence” and accompanying
“confidence interval” (or margin of error). A traditional level of confidence, and the one used here, is 95%. The 95% confidence
interval can be any size and quantifies the sampling error or imprecision of the survey results because some residents’ opinions
are relied on to estimate all residents’ opinions. The margin of error for the City of Charlottesville survey is no greater than plus
or minus 5.1 percentage points around any given percent reported for all respondents (371 completed surveys).
In addition to the randomly selected “probability sample” of households, a link to an online open participation survey was
publicized by the City of Charlottesville. The open participation survey was identical to the probability sample survey with two
small updates; it included a map at the beginning asking where the respondent lives and a question about where they heard
about the survey. The open participation survey was open to all city residents and became available on December 7th, 2022. The
survey remained open for 2 weeks. The data presented in the following tabs exclude the open participation survey data, but a tab
at the end provides the complete frequency of responses to questions by the open participation respondents.
3
Analyzing the data