Evidence-Based Survey Design Presentations

Co-presented evidence-based survey design principles to a variety of audiences including:

Evidence-Based Survey Design Principles, National Institute of Health Strategic Change Management Community of Practice Meeting (Co-Presenter) (2021)
Evidence-Based Survey Design Principles, International Society for Performance Improvement Atlanta Chapter Meeting (Co-Presenter) (2021)
Survey Design 101: Crafting Effective Response Scales, International Society for Performance Improvement Bay Area/Boise State Chapter Meeting (Co-Presenter) (2020)
Bring Me Data! Evidence-Based Survey Design, International Society for Performance Improvement Annual Conference, New Orleans, LA (Co-Presenter) (2019)
Write a Strong Survey Instrument Using Evidence-Based Practices, International Society for Performance Improvement Bay Area/Boise State Chapter Meeting (Co-Presenter) (2017)

The Challenge

Self-administered survey questionnaires are one of the most popular data collection tools in use today. When designing surveys, it is important to pair the closed-end statement/question with an appropriate response scale to improve effectiveness. What are the parts of a survey item (anatomy)? What types of response scales should you use? How might various design decisions like the use of positively/negatively worded statements or questions, horizontal/vertical response scale presentation, ascending/descending response scales, or balanced/unbalanced response scale presentations affect your survey design?

Work Description

A collaborative team approach was used to design and develop a 1-hour synchronous Zoom presentation. This presentation included an overview of basic survey design concepts such as survey item terminology, the wording of survey statements or questions, types of response scales (nominal, ordinal, interval, ratio, Likert, etc.), alignment of survey purpose and response scale selection, evidence-based best practices in survey design, and research on more nuanced survey design concepts like ceiling effects and design strategies to minimize them.

Participants are provided with a job aid and practice opportunity to redesign a survey applying the things they have learned in the presentation with recommended amendments.

Methods/Technology Employed

MS PowerPoint
Zoom

What I Learned

Less is more!

When presenting evidence-based survey design, it is important to present the “big picture” to participants and provide resources for the details. All participants have some experience with surveys whether it is consumer surveys, net promotor score surveys, or more detailed surveys in instructional design or performance improvement initiatives. However, participants usually vary wildly in their knowledge about the subtle details. After the presentation, participants often remark “I didn’t realize there was so much involved in designing a survey!”

However, when trying to highlight too much information, it is easy to leave participants lost and disinterested. The team chose a few survey design concepts, simplified those concepts, and linked them to relevant examples from a variety of domains that would be immediately applicable in the workplace for novice and expert participants. Participants were provided with a practice opportunity via handout and corresponding recommendations to support application.

Bring Me Data! Evidence-Based Survey Design presented at ISPI Annual Conference – New Orleans, LA (2019)