CC Daily’s recent piece on jargon and first-generation students (“Breaking a different language barrier”) named something most of us working in higher education already sense but rarely act on. We know our language is confusing and can be a barrier, as L. Dean Fisher mentioned. Knowing that isn’t the same as fixing it.
Fisher’s practical list of fixes included testing comprehension with actual first-generation students. That’s the right instinct, and it’s also where most institutions need more instruction. Here’s some direction on how to test content for understanding.
Skip the survey
Do not send a survey. Surveys are easy to send out but are a terrible method for content testing. You’re not asking students whether your language is clear. You’re watching what they can do with it, because what people say and what they can act on are two different things. As Fisher implied, a student can read a phrase like “satisfactory academic progress” without a problem. That tells you nothing about whether they know it affects their financial aid.
Here are three low-effort ways to test content with students:
Three ways to test with students:
1. A highlighter test. Get a clipboard and print out a piece of information with key details that students need to know. For example, find a real document, an enrollment email, a financial aid letter, text from a web page or a syllabus policy. Find a student, hand them the clipboard and paper, and two different colored highlighters. One color marks any word or phrase they don’t understand. (Note that students don’t often want to admit not understanding, so one strategy is ask them to highlight anything that they think is confusing or “that others might find confusing.”)
The other highlighter marks anything where the words make sense, but it’s still not clear what they are supposed to do. Vocabulary and instruction are two separate problems, and this test shows which one is causing misunderstanding.
2. Teach-back. This method comes from health literacy work, and it works well on campus. After a student reads something once, ask them to explain back to you, in their own words, what it’s telling them to do. You can present the task as, “Read this please, and then would you explain it to me as if you were talking to a friend about what they need to do?”
If they can’t restate the action accurately, the document has failed. Teach-back catches the gap between recognizing words and understanding what’s expected of you.
3. The real-world task. Give a student a real scenario with information, either on paper or online (tablets or light laptops are easy to hand to someone else to view), and watch them work through it live. For example, “Find your census date and tell me what happens if you miss it.” Then stay quiet and watch where they get stuck, on the portal, in the email or reading the policy. Avoid answering questions they may direct at you. Instead, reply “Well, what do you think?” to learn more about their mental model. Every place they hesitate, backtrack or guess is on your revision list. You don’t need to ask them what confused them because you’ll see it.
Some other do’s:
- Make it clear you are testing the content and not them. Explain “We are trying to see what, if anything, needs to change about this information so it works well for students.” And at the end, say “Thanks for helping us make this better.”
- Offer students a $5 coffee card for their time. This is a good way to get them to stop and give you five or 10 minutes of their time. When I worked at a university, we would intercept people in a high-traffic area near the library and the attached cafe. I’ve also had good results intercepting students in student centers on community college campuses.
- Fix one item at a time. If you test and fix one thing at a time regularly, you can make significant progress in one school year. Could you test once a month, once a quarter or once a semester? Ongoing testing is needed to iterate into understandable content.
- Be intentional about who you test with. Test with your first-generation students, your first-year students, your most shy or anxious students. They’re the ones your language really needs to work for.
- Plan to test again. After you’ve improved the information, re-test. It’s easy to think we fixed something when we changed it, but you should confirm.
What not to do
A few ways content testing goes awry, despite good intentions:
- Don’t test with your colleagues, even young ones. Once you have been a part of a system for a few weeks, you are an insider and already know too much. Staff are not your audience, and their comprehension tells you nothing about a student’s understanding.
- Don’t ask “Does this make sense?” That’s a leading question, and most students will say yes to be agreeable, even when they’re lost.
- Don’t skip the retest. A first rewrite doesn’t always achieve the fix. Test again after you revise, the same way you tested the original. Iterate your way to plain and accessible language.
Just start this semester
You don’t need a research department or a budget line to do this work. You need one piece of information, four to six students and an hour of time from one to two staff members to execute. You’ll see patterns quickly.
Pick the information piece that generates the most confused phone calls or emails to your area. Run one of these tests this semester. See what you learn, then do it again. Think about how improving communication might avoid these repetitive time-wasting questions for both students and staff.
Fisher is right that language is inside our control to fix. Testing it with real students is how we find out whether we actually did.
