WAAL 2008: User Surveys: Cost-effective Marketing Strategies

• Larry Duerr and Dolores Skowronek, Alverno College

Cost-effective Marketing Strategies

• Both of us are on campus library marketing committee
• Previous committee members were more interested in promoting internally through bulletin boards, etc.
• International Encyclopedia of Information and Library Science: "Marketing represents an organized way of offering library services cost-effectively and efficiently, baesd upon user interests, communication methods, imaginative design of service and products, and feedback that improves what the library is doing."
• Rather than pulling an idea out of a hat and saying, "let's try this" - we needed data on our users
• Last time we'd gathered data like this was 10 years ago
• Info we wanted to gather: frequency of library use, demographics, reality of student tech proficiency vs. their self-assessment, satisfaction level with library

Data gathering & statistical analysis

• Probability sampling - better option but requires more understanding of statistics, and more funds - everyone has same chance of being surveyed - yields statistically measurable results
• Non-probability sampling - tends to be biased and not representative of whole population - not generalizable - convenience sampling (try to get as many people as possible who come into library on certain day/time to take survey) - handy when you aren't sure if there's a problem or not
• Most librarians don't have the research methodology skills to do probability sampling
• We had both taken a research methods class, but had never used in practice - we chose non-probability sampling
• Used same categories in our demographic survey questions, as those used by college - could then compare
• Educational institutions have tons of data on students - very helpful
• Attempted to minimize bias when distributing surveys - handed out to absolutely everyone who passed by, whether or not we knew if they were students
• Larger the sample size, the better - our goal was 331 (minimum sample size for our overall study body, if we'd used probability sampling)
• You're welcome to use our survey questions - contact us

Our questions + results

• We adopted a definition of a non-user: "somebody who uses the library once a year or less" - turned out that it wasn't good definition - very few people fit into that category - but it was good that we didn't seem to have the problem of non-use that we suspected
• Identify and separate 1st-year students, because of course they didn't use library last year...
• When analyze open-ended questions, need to go through "open coding" process - group into categories and assigned code to enter into analysis software - librarians are good at this!
• Try to reduce bias by having more than one person work on open coding, separately
• SPSS is great qualitative analysis software, but not intuitive - seek someone out with know-how
• 377 usable surveys, 322 were students, 8 nonusable, 124 chose not to answer survey [a data point most people don't collect]
• Happy to get small differences in percentage between our sample, and overall population by year in school, etc.
• Grad students - higher non-use of physical facility, but higher use of website; don't necessarily know why
• "Liked best about the library?" - generating talking points we can use with the administration - why have a library, why continue to staff it - leverage value-added component - high = library staff, quiet space
• "Liked least?" - too few computers (we had anecdotal evidence about this, but now real data to use in requests for funding); too few hours (have made some changes)
• "Where do you go first?" - internet (no surprise, not trying to compete there); library building 21%; library website; library databases

What changes have we made?

• 1st/2nd year students don't use website - more focus by Librarians in Residence, and tutorial videos
• Students say staff are friendly and easy to work with - confidence for librarians to serve on more curriculum committees, extend staff outside the building
• Want more computers - budget request, opened nearby computer lab for drop-ins from 12-1pm during week
• Focus group: more classes in how to do research - offering more workshops on APA citation format - got 5 people at each of 3 workshops [hmmm... not sure this is the right tactic]
• Low use of web2.0 (blogs, wikis, rss) - continuing workshops on learning technologies, want to purchase Captivate license
• High use of Facebook - created page, want to create link from website - communicating resources/services/events - gray area about crossing the line into student online communities - but we know it's being used, so good place to spend library time
• Faculty focus groups: expect library to keep them informed about new technologies and research

Future

• Do more frequent surveys - not wait 10 years between
• Target grad students
• Short-term goals: more laptops, more embedded librarians for specific courses - "Librarian in Residence," promote website to freshmen
• Long-term: new library building 4-5 years (ours is from 1950's) - get librarians involved, include Information Commons; create information literacy general education course taught by librarians (ACRL IL competencies); staff training - using technology

Lessons learned

• Find and use experts on campus (ex: SPSS experts)
• Redefine non-user = someone who uses library once a semester or less
• Verify college data - use master set/codes, rather than info on the open web which might not match (oops!)
• Detailed timelines would have been more helpful - tried to go back to analyze data after a semester, had lost momentum - this is common - loss of morale for staff as well as users who have answered survey
• Offered online survey version after refused paper survey - not many takers - has to be exactly the same
• Followed up survey with one focus group with students, one with faculty - wanted more, but very hard to schedule - good information - wanted to know what librarians "really do"
• Need IRB approval, admin buy-in, above-board
• Without funds for outside expert consultant, wanted to become good researchers - wonderful experience - will understand the literature better now - positive change - total cost was under $600

Questions/comments

• Let users know results? Haven't yet much; some info in annual report. Will make formal report to admin. Notepad at next "Institute" event where faculty meet - giveaway with facts about library.
• Memorial Library at UW-Madison did a "know your librarian" display - what they do when they are and aren't on the job
• Non-usable surveys? Filled out front and not back of survey.
• Amanda's reaction: Yay! Assessment as part of marketing!

WAAL 2008: Integrating and assessing information literacy

WAAL Information Literacy Award presentation

• Dave Dettman, Coordinator of IL and Outreach, UW-Green Bay
• UW System now looking at student learning assessment tools: iSkills, Project SAILS, ILT
• Assessing learning outcomes is more difficult for IL than subject areas
• How well is IL integrated into the curriculum at your campus?
• "Assessment Loop" - want to complete all steps (goals, measurement, etc.), and close the loop (set new goals) - funders want to see this
• Accreditation agencies starting to look at IL-related skills
• Faculty had been afraid of IL - confusing it with technology literacy - can student use x piece of software/hardware?
• We've been doing "one-shot" instruction sessions for a long time - faculty are used to this
• Needed to document students' poor understanding of IL concepts for faculty who think that students have these skills
• Need to approach library databases differently than the Wild West Web, not in same way that you'd ask one another a question
• Action plan: Make failure visible, in order to move on to more effective approaches
• Beyond one shot: librarian meets with students 2 times in computer labs where they can continue to research topics
• Standardized tests: can be powerful for communicating a need for more integration
• ETS iSkills ICTL test (Scale: 0-300) used at UWGB
• No library instruction = 156.5
• One-shot lecture = 164.4
• Lecture + ICT literacy exercises = 171.2
• National recommendation = at least 162
• What does this score mean? Don't know, but he started to get invited by admin to serve on many instruction committees
• UWGB revived freshman seminar concept, after ~15 years - beyond composition class
• 1st assignment: Watch movie, pick 5 themes, find scholarly article related to each
• Library e-guide - but faculty didn't promote, and complained about predominant website citations
• 1st year (2006) = Failure
• Would like to parse out data to an instructor level, but my data analyst isn't comfortable with that
• Self-assessment of learning: increased on most measures (exception: "how to use" information)
• People upstairs love this: "If the librarian's not doing much teaching right now, imagine the results if librarian was doing a bit more!"
• Unfortunately, some students get to see me multiple times in their classes, others don't see me at all
• Perception of what a librarian is the same, regardless of where they work - think the academic librarian is the same as their old kindergarten librarian
• Really opening doors to faculty, institutional admin
• Successfully advocated to expand freshman seminar in 2008-09 with IL to about 1/2 students
• Planning to integrate more with 8 instructors
• Want to convince 4 more instructors to require a similar assignment with one-shot, so not comparing apples to oranges
• Many librarians find that they aren't able to use the data they collect in the way that they think, because the wind up trying to compare very different experiences
• Now possible to make connections between IL scores (higher-level thinking) and post-grad hiring
• Students finally get something back from assessment - see real-time results = higher motivation
• We promoted campus interest in results, by some staff promising to shave their heads if score was higher than a certain level
• Need to build IL assessment into a course, with class time
• A different campus was offering $25 for students to take IL test - a student said "I'd rather give blood"
• Awarded an IL "Lesson Study" grant - seed money is powerful on campus
• Pre/post surveys, focus groups coordinated by Education students
• Now working on MLLO Project: Assessing mission level student learning objectives
• Faculty development - monetary rewards - make yourself visible there - with data, you'll become more welcome
• LibQual at UWGB in 2004, again in 2008 - can show gains
• UW System-wide assessment interest - we've been working in our own backyards for too long
• Dave: Better to organically roll into an existing class, than to have a stand-alone "library class" for credit
• Audience: Disagreement - our credit class is very popular among everyone

[Image: http://www.uwgb.edu/univcomm/news/insidearchive/04nov1.htm]