WLA 2007: Applying Survey Methodology in the Real World

Applying Survey Methodology in the Real World
a presentation by Thomas Walker, Associate Dean, UW-Milwaukee, SOIS

Introduction to surveys:

1. What are they?
2. How to plan a survey
3. How to collect data
4. Sampling
5. Questionnaires

I. What are surveys?

• a formalized method of gathering info about a group of people through a sample
• a carefully chosen sample can be used to project results to a larger population

Surveys are not...

• collected from 100% of a population
• collected entirely from a self-selected group
• collected from a group just because that "sample" is easy to get data from

Data gathered systematically:

• standardized procedures
• not data from individuals -- should be anonymous
• data should form a composite profile of the whole

Library surveys:

• usually to assist in the planning process
• assess community needs
• assess community perceptions of...
• - what libraries are
  
• - what libraries should be

Surveys and methods of collection:

• telephone
• mail
• in-person at the library or other location
• web site or email
• while methods of data collection can be used to describe the type of survey, methods should not be the main reason a sample is chosen

II. Planning a survey

1. Development
2. Pre-test
3. Final draft of plan and survey
4. Implementation
5. Coding
6. Analysis and reporting

Development:

• define budget, staffing, and time
• define outcomes
• broadly define population and sample
• draft data collection method

Pre-test -- allows you to try things out; a dry run

• more clearly define population and sample
• refine questionnaire
• pre-tests(s) of revised questionnaire
• evaluate pre-tests(s) and contine or pre-test again

Final draft of plan and survey:

• finalize population and sample
• prpare final questionnaire
  
• organize logistics of implementation -- will the survey be distributed at a service desk; given to every 10th person, etc.

Implementation:

• select sample

Coding:

• evaluate validity of data -- remove invalid responses and otherwise clean up
• prepare data for analysis (code)

Analysis:

• prepare data sets and subsets
• analyze data

Final reporting

• contextualize data in pre-established framework of survey plan
• prepare report

Two crucial tasks:

1. questionnaire design
2. sampling

V. Questionnaires:

• define:
• - what kind of information is required?
• - from whom do you need data?
• write questions at a 5th grade level; keep things simple & direct
• break down complex problems into very simple ones
• create clear simple questions

The questionnaire:

• may be self-administered or done by an interviewer
• should be introduced to let the respondent know what the purpose is, who will analyze it, and whether the results will be made public
• should conclude by expressing appreciation
• should be designed at a 5th grade reading level

Confidentiality

• statements assuring confidentiality are desirable and may be required
• inform respondents that thier responses are voluntary and that their anonymity is assured
• if children are involved in any way, extra precautions must be taken

Clear, simple questions

• scales may be useful ("on a scale of 1-5...")
• multiple choices may be clear, if all possible choices have been anticipated
• open-ended questions can yield rich data, but are difficult to analyze or quantify
• questions should be pre-ested
• special terms should be defined (acronyms like OPAC, jargon, etc.)

IV. How to sample

• define overall population
• determine ways to accurately sample that population

Examples of samples -- public library service populations subgroups:

• users
• non-users
• children in school
• visually impaired

Sample size:

• there is not one magic formula for determining sample size
• factors:
• - degree of exactitude needed
• - budget available
• - staff time
• - ease of administration of survey
  

Sample may be small

• if well-chosen, a sample may be just a small percent of the whole population
• it's better to spend one's time focusing on the design of a survey and the sampling of a population than to blanket a larger percent of a population

Confidence level

• expressed as a percentage for how frequently the true percentage of a population would answer the question
• the number of percentage points
• it is most common to express confidence more fully by including both the confidence level and the interval

Sample size calculator: http://www.surveysystem.com/sscalc.htm
"You can use it to determine how many people you need to interview in order to get results that reflect the target population as precisely as needed. You can also find the level of precision you have in an existing sample."

Finding sample surveys

• same time by replicating other surveys
• published articles for similar institutions or types of surveys, some of which may even include a copy of the original survey instrument

The Rodski Survey: http://www.newcastle.edu.au/service/library/rodski/