Wednesday, October 17, 2007

WLA 2007: Applying Survey Methodology in the Real World

Applying Survey Methodology in the Real World
a presentation by Thomas Walker, Associate Dean, UW-Milwaukee, SOIS

Introduction to surveys:
  1. What are they?
  2. How to plan a survey
  3. How to collect data
  4. Sampling
  5. Questionnaires
I. What are surveys?
  • a formalized method of gathering info about a group of people through a sample
  • a carefully chosen sample can be used to project results to a larger population
Surveys are not...
  • collected from 100% of a population
  • collected entirely from a self-selected group
  • collected from a group just because that "sample" is easy to get data from
Data gathered systematically:
  • standardized procedures
  • not data from individuals -- should be anonymous
  • data should form a composite profile of the whole
Library surveys:
  • usually to assist in the planning process
  • assess community needs
  • assess community perceptions of...
  • - what libraries are
  • - what libraries should be
Surveys and methods of collection:
  • telephone
  • mail
  • in-person at the library or other location
  • web site or email
  • while methods of data collection can be used to describe the type of survey, methods should not be the main reason a sample is chosen
II. Planning a survey
  1. Development
  2. Pre-test
  3. Final draft of plan and survey
  4. Implementation
  5. Coding
  6. Analysis and reporting
Development:
  • define budget, staffing, and time
  • define outcomes
  • broadly define population and sample
  • draft data collection method
Pre-test -- allows you to try things out; a dry run
  • more clearly define population and sample
  • refine questionnaire
  • pre-tests(s) of revised questionnaire
  • evaluate pre-tests(s) and contine or pre-test again
Final draft of plan and survey:
  • finalize population and sample
  • prpare final questionnaire
  • organize logistics of implementation -- will the survey be distributed at a service desk; given to every 10th person, etc.
Implementation:
  • select sample
Coding:
  • evaluate validity of data -- remove invalid responses and otherwise clean up
  • prepare data for analysis (code)
Analysis:
  • prepare data sets and subsets
  • analyze data
Final reporting
  • contextualize data in pre-established framework of survey plan
  • prepare report
Two crucial tasks:
  1. questionnaire design
  2. sampling
V. Questionnaires:
  • define:
  • - what kind of information is required?
  • - from whom do you need data?
  • write questions at a 5th grade level; keep things simple & direct
  • break down complex problems into very simple ones
  • create clear simple questions
The questionnaire:
  • may be self-administered or done by an interviewer
  • should be introduced to let the respondent know what the purpose is, who will analyze it, and whether the results will be made public
  • should conclude by expressing appreciation
  • should be designed at a 5th grade reading level
Confidentiality
  • statements assuring confidentiality are desirable and may be required
  • inform respondents that thier responses are voluntary and that their anonymity is assured
  • if children are involved in any way, extra precautions must be taken
Clear, simple questions
  • scales may be useful ("on a scale of 1-5...")
  • multiple choices may be clear, if all possible choices have been anticipated
  • open-ended questions can yield rich data, but are difficult to analyze or quantify
  • questions should be pre-ested
  • special terms should be defined (acronyms like OPAC, jargon, etc.)
IV. How to sample
  • define overall population
  • determine ways to accurately sample that population
Examples of samples -- public library service populations subgroups:
  • users
  • non-users
  • children in school
  • visually impaired
Sample size:
  • there is not one magic formula for determining sample size
  • factors:
  • - degree of exactitude needed
  • - budget available
  • - staff time
  • - ease of administration of survey
Sample may be small
  • if well-chosen, a sample may be just a small percent of the whole population
  • it's better to spend one's time focusing on the design of a survey and the sampling of a population than to blanket a larger percent of a population
Confidence level
  • expressed as a percentage for how frequently the true percentage of a population would answer the question
  • the number of percentage points
  • it is most common to express confidence more fully by including both the confidence level and the interval
Sample size calculator: http://www.surveysystem.com/sscalc.htm
"You can use it to determine how many people you need to interview in order to get results that reflect the target population as precisely as needed. You can also find the level of precision you have in an existing sample."

Finding sample surveys
  • same time by replicating other surveys
  • published articles for similar institutions or types of surveys, some of which may even include a copy of the original survey instrument
The Rodski Survey: http://www.newcastle.edu.au/service/library/rodski/

No comments: