Do Your Surveys Suck? 5 Tips For Creating Better Surveys

Over the past seven months, school districts are using surveys more than ever to gather stakeholder preferences and feedback for their COVID-19 beginning of the school year planning and implementation of some form of in-person or hybrid reopening plans over the next few months.

While surveys are and can be a useful tool, a survey’s utility is only as beneficial as its design.  In addition, survey results can have some intended consequences (providing the information being sought), and some very unintended ones, as well (stakeholders holding the results and positioning them as “votes” over the district’s head to pressure their leadership to take certain positions).

One of the biggest mistakes I am seeing right now is that school districts are using the term “survey” to define a number of things, from collecting information about learning preferences, feedback about distance learning, and more.  The problem is, these are not all surveys.  For example, I am seeing a number of “surveys” that are collecting information from parents about whether their child will be continuing to learn in a distance learning or will be transitioning to in-person (hybrid or otherwise) learning. These “surveys” collect student name, school, and contact information data, so that the districts know which students are returning to in person learning and which ones are remaining in distance learning. In reality, parents are not completing a survey–they are completing a registration form. Likewise, if you’re looking for the answer to one specific question, a poll might be a better choice, and if you’re looking for general feedback about your stakeholders’ overall experience with a program or approach to learning, asking for a review could be most effective.

Another mistake I am seeing is that surveys are asking for learning model preferences without providing all of the details to stakeholders.  These questions fail to include all of the known information, and stakeholders may be opting for certain responses without understanding the impacts of those choices.

For example, a five day a week schedule in your district may mean that student cohorts will be split into morning and afternoon schedules, and will spend fewer hours in the classroom.  Learning in person may mean that students will be required to remain with their cohort, will encounter restrictions during recess, will eat lunch in their classroom, and socialization will be greatly limited.

This is not to say that in person learning is filled with insurmountable challenges, but in parents’ excitement  to return their children to school, they also may have certain assumptions of what that will look like, and similarly staff and teachers may also have certain assumptions as they complete their surveys.

Without painting a clear picture of what each survey choice means, stakeholders may be selecting certain options based on expectations and assumptions, and if those are not met, then your district risks creating confusion and disappointment, being accused of lacking transparency and losing trust in the process.  This is why providing clear information to stakeholders in advance of sending surveys is so critically important–you don’t want to create more confusion when you send your surveys.

Maybe it’s been a decade since you took your research methods class in grad school–or maybe you never took a course at all. In any case, anytime you want to create a survey, you need to be thoughtful and thorough.  So, here are five ways to improve your surveys so that your stakeholders feel valued, you will collect the data you need, and you’ll prevent confusion and unintended consequences.

 

5 Ways to Improve Your Surveys

1. Be prepared. First, establish your survey’s goals and objectives. What information do you want to gather–preferences, taking the temperature of your stakeholders, obtaining registrations?  How do you want to use this information? Would your survey be best suited to be split up into two surveys?  If you have vastly different goals, you may want to consider sending two different surveys so that you aren’t confusing your stakeholders.  Lastly, determine whether a survey is the best approach for gathering the information you are seeking, or if focus groups or an advisory committee would be more effective.  
2. Be informative. Provide clear information in advance of sending your survey–on your website, through video, and through stakeholder meetings.  Your survey should not be the first (or only) place where information is conveyed.  Make sure you are preparing your stakeholders with the most up to date information before asking them to take a survey, otherwise your survey results will not be accurate, and your stakeholders will accuse your district of lacking transparency.
3. Be clear.  Are you collecting information, do you want people’s commitments, are you taking a poll or wanting a review?  A survey collects data. Registrations collect commitments.  Polls collect preferences, and reviews collect feedback. So, be clear with your audience about how you will be using the information you are collecting, and clearly and accurately title the survey.  Additionally, avoid jargon, keep questions clear and concise, and don’t ask loaded questions.
4. Be thoughtful. Based on your survey goals and objectives, develop easy to understand, bias-free survey questions. Make sure that you are not asking leading questions.  Do a test run with friends and family outside of your organization to catch any problems or issues with your questions or survey design.
5. Be varied. Provide a balance of different types of survey questions being asked, and based on the goals and objectives of your survey, you approach to survey questions should support those goals and objectives.  Hubspot provides a great summary of the different types of survey questions, and how to use them, but here is their easy-to-understand summary of the question types in case you needed a refresher: 
   • Multiple Choice: Multiple choice survey questions are questions that offer respondents a variety of different responses to choose from. 
   • Rating Scale: Rating scale questions (also known as ordinal questions) ask respondents to rate something on a numerical scale assigned to sentiment. 
   • Likert Scale: Likert scale survey questions evaluate if a respondent agrees or disagrees with a question. 
   • Ranking: Ranking survey questions ask respondents to rank a variety of different answer options in terms of relative priority or importance to them.
   • Semantic Differential: Semantic differential survey questions also ask for respondents to rate something on a scale, but each end of the scale is a different, opposing statement. Use semantic differential questions to get clear-cut qualitative feedback from your stakeholders.
   • Dichotomous: Dichotomous survey questions offer only two responses that respondents must choose between. 
   • Close-Ended: Close ended survey questions are questions that have a set number of answers that respondents must choose from. 
   • Open-Ended: Where the survey types above all have closed-ended answers that you input as different options to choose from, open-ended questions are usually accompanied by an empty text box, where the respondent can write a customer answer to the question.

The bottom line: Sending a survey is a lot like Thanksgiving dinner. You’re gathering the family around the table and feeding them the opportunity to share their thoughts. Remember to set your table with care before feeding your family. Your stakeholders will thank you for it.