Listening at Scale: Rebuilding the Sitewide Provider Experience Survey
Survey DesignQuantitative ResearchThematic Coding
~20,000Valid provider responses in the first cycle
37%Ranked Benefits & Eligibility as their #1 improvement priority
2 → 1Two provider portals, one streamlined instrument
Problem: The legacy provider survey had been triggered so often, per session, that it frustrated providers enough to get disabled entirely, leaving Digital and Line-of-Business partners without the data they needed to justify feature funding.
Outcome: Redesigned it as a role-personalized, bi-annual instrument across both provider portals, explicitly positioned as one input in a broader research stack. The first cycle immediately surfaced a clear, prioritized signal for where the product roadmap should focus next.
Read the full case study
Project Brief
The Digital team inherited ownership of an existing sitewide survey running across two provider portals. That survey had been triggered so frequently, per session, that it frustrated providers, enough that the team disabled it entirely while figuring out how a sitewide instrument could actually serve Digital and Line-of-Business (LOB) partners going forward. At the same time, Digital and LOB stakeholders needed actionable data to justify funding for both net-new features and enhancements, particularly anything that could reduce inbound call volume. I was asked to draft the question set and design principles for a redesigned, far less intrusive version of the survey.
The old survey's unpredictable frequency was the direct cause of the fatigue that got it disabled.
My Role
I led the end-to-end redesign: auditing the legacy instrument, defining the new measurement model, writing and structuring the survey itself, and coordinating with design, analytics, and platform partners to launch it across both portals.
Approach
Replaced the old session-triggered pop-up (the direct cause of the fatigue that got the survey disabled) with a predictable, time-boxed window that runs twice a year for four weeks, keeping data comparable cycle over cycle
Positioned the survey as one input in a broader research stack (alongside call/digital analytics, session replay, and interviews), best suited for tracking broad signals and surfacing future research topics, not for deep root-cause insight on its own
Structured the question set around Jobs-to-be-Done: which task a respondent is responsible for, which channel they use to accomplish it and why, effort, relevance to their role, frequency, and trust/confidence in the information
Built role-personalized task lists so a Behavioral Health provider, a Medical provider, and a Dental provider each see questions relevant to their own portal tasks, not a generic list
Applied a "minimize frustration, maximize relevance" design principle throughout: optional and easily dismissible, auto-saved progress, and a clear exit at any point
Designed the instrument to be a repeatable asset: built in a defined retrospective cadence (response rate, drop-off, theme saturation) so the survey itself gets evaluated and refined each cycle, not just the data it produces
The survey was deliberately positioned as one signal among several, not a stand-alone source of truth.
Key Findings
~20,000 valid responses came in during the survey's first live cycle, strong participation for a redesigned instrument.
37% of respondents named Benefits & Eligibility information as their top improvement priority: the clearest single signal to come out of the first cycle.
Nearly 2,200 open-ended responses across the two new qualitative questions surfaced recurring language patterns pointing to specific friction points the closed-ended questions alone couldn't show.
A known limitation: the survey only reaches providers who use the portal in the first place. Those who don't (or whose portal experience is limited enough that they avoid it) are systematically underrepresented, a selection bias built into any portal-delivered instrument.
Recommendations & What's Next
A focused deep-dive study on Benefits & Eligibility, the highest-priority theme from the survey, to turn the "what" into a concrete "why" and "how to fix it"
Thematic coding of the full open-end dataset to convert raw provider language into a structured, prioritized backlog for usability testing
A living retrospective process for the survey itself, so it keeps measuring what actually matters as the portals evolve