Question 1

What's the smallest sample for reliable UX success-rate estimates?

Accepted Answer

For a reliable task success rate in UX (user experience) testing, you typically need a few dozen users rather than just a handful. With only 5 or 10 users, the success percentage you observe can vary hugely just by chance (say 3 out of 5 vs. 4 out of 5 users succeeding is a 20-point swing), so it's not a stable metric. By around 25-30 users, the results start to stabilize and give you a decent ballpark figure. Many teams actually aim for about 40 participants for important quantitative UX tests, because that sample size tends to narrow the margin of error to a reasonable range (so your measured 75% success isn't actually anywhere from 50% to 90% in reality). In short, if you need confidence in the exact success rate or want to compare designs statistically, recruit on the order of dozens of users. Smaller samples are fine for finding obvious usability issues, but not for precise success metrics.

Question 2

How can we increase team engagement with CX dashboards short of more reminders?

Accepted Answer

Instead of sending more nagging reminders, improve the intrinsic pull of your customer experience (CX) dashboards by making them indispensable in the team's daily work. One approach is to ensure the dashboards answer the "so what?" at a glance - highlight one or two key metrics or alerts (for example, a spike in detractors or a trending new customer complaint) so that team members naturally want to log in and see what needs attention. You can also integrate the dashboards into existing workflows: embed snapshots or automated summaries into weekly team emails or Slack updates, so engagement happens organically rather than relying on memory. Another tactic is to personalize or role-tailor the data: when a salesperson sees their own client NPS score or a product manager sees the latest user feedback on their feature, the dashboard instantly becomes more relevant. In short, people use a dashboard when it consistently provides value with minimal effort - by surfacing timely, actionable insights and being present where the team already communicates, you'll see engagement rise without having to constantly chase everyone down.

Question 3

Can push-to-web ABS replace phone for hard-to-reach groups?

Accepted Answer

Push-to-web using Address-Based Sampling (ABS) is emerging as a strong alternative to phone surveys for hard-to-reach groups, though it often works best in a mixed approach. By sending mail invitations with unique survey URLs, researchers can reach households that phone polls miss — including those without landlines or who ignore unknown calls — and encourage them to respond online at their convenience. This method has improved response from demographics like rural residents and younger adults who rarely pick up the phone. However, it may not completely replace phone outreach: some people still won't go online even when mailed a link, so researchers often follow up with a paper questionnaire or phone call for non-responders. In practice, a hybrid strategy is common: ABS push-to-web for broad coverage, complemented by phone (or mail) follow-ups to ensure no important segment is left out.

Question 4

When should we add qual to explain borderline concept screening scores?

Accepted Answer

Any time a concept's quantitative test scores come back inconclusive or borderline, it's wise to add qualitative research to learn the reasons behind the lukewarm reception. Borderline scores — ideas that test in the gray area between clearly good or clearly poor — often mean some people saw potential while others had reservations. Follow-up interviews or focus groups with participants can reveal exactly what was confusing, unappealing, or missing in the concept. Maybe the value proposition wasn't clear, or a particular feature excited some users but turned off others; qualitative discussions help pinpoint those issues. Bringing in qual at this stage allows the team to decide whether the concept can be tweaked and improved or if the lukewarm score is a sign of a deeper flaw. In short, whenever numbers alone leave you on the fence about a concept, qualitative context will illuminate the path forward.

Question 5

How can we get topic models to recognize minority or emergent themes?

Accepted Answer

To get topic models to recognize minority or emergent themes, adjust your modeling approach to amplify those rare signals and incorporate some guidance. Standard topic models like LDA can overlook infrequent themes because the most common topics dominate, so one solution is to tweak parameters (for instance, using a higher number of topics or adjusting the priors) to give smaller clusters a chance to surface. You can also seed the model with keywords related to the minority theme or use a semi-supervised approach, where you provide a few examples of the emergent topic to guide the algorithm. Another tactic is to run an initial topic analysis to capture the obvious themes, then remove or downweight those dominant topics and re-run the model on the residual data to reveal subtler ones. Iterative human-in-the-loop modeling — where an analyst reviews interim results and nudges the model (by adjusting settings or adding domain knowledge) — is often effective to ensure niche but important themes are detected by the topic model.

Glaut

Why choose Us

Features and attributes

What is Glaut?

Research Software and Platform Details

Glaut Services and Expertise

Webinars

Market Research FAQs with Expert Insights

What's the smallest sample for reliable UX success-rate estimates?

How can we increase team engagement with CX dashboards short of more reminders?

Can push-to-web ABS replace phone for hard-to-reach groups?

When should we add qual to explain borderline concept screening scores?

How can we get topic models to recognize minority or emergent themes?