Sarah Williams | Design Leader

Associate Listening

AI-powered feedback insights to improve store performance.

The objective

Create a suite of intelligent tools in the Me@Walmart app that enables store leaders to review and act on associate feedback, quickly, to improve engagement and performance.

Company
Walmart

My Role
Project design lead working closely with a Sr Designer and Design Manager; in collaboration with research and content partners.

Timeline
Q1 – Q2 2024

The opportunity

Empowering store managers to impact business outcomes

In order for Walmart to be the best place to shop, it needs to be the best place to work.

Associate Listening is a central part of Walmart’s vision to be a premier workplace that attracts, retains, and nurtures top talent to drive business growth.

By equipping store managers and leaders with real-time insights from associate feedback, we can empower them to take action and positively impact Walmart’s culture and business outcomes.

DESIRED OUTCOMES

Reduce time on data interpretation

Make it easier for managers to find actionable insights from associate feedback.

Expand access to insights

Enable more leaders to access feedback data.

Facilitate action planning

Help managers create plans to improve team experiences based on feedback.

Discovery

Designing best-in-class feedback tools

Through primary and secondary research, we uncovered that the associate feedback process is driven by a flywheel model.

Associates: The more associates share their thoughts and experiences, the more opportunities managers have to learn and respond effectively.

Managers: When managers address feedback in transparent and meaningful ways, it encourages even more associates to share their insights, creating a continuous cycle of improvement and engagement.

We used this to define the user journey and communicate the relationship and dynamics of the holistic program to our stakeholders.

Illustration of the associate/manager feedback flywheel

Focusing on the manager experience, we identified current state challenges and user needs to create our guiding principles.

Current state insights

Low visibility
Store leaders lack visibility into surveys and feedback their store associates submit today.

Highly manual process
The feedback process is manual and requires store leaders to be experts in interpreting survey data.

Low impact perception
Store associates don’t feel their feedback is acknowledged or leads to change.

Desire for control
Store leaders want better digital tools but still want to be decision-makers.

Guiding principles

Optimize for digital
Provide a digital-first experience to improve access to feedback insights and streamline actioning.

Show insights over data
Leverage technology to distill complex data into clear, actionable insights to save managers time and support effective decision-making.

Engage the collective
Foster a collaborative environment where everyone is involved, has a voice, and shares ownership in driving store success.

Respect human expertise
Empower managers to make informed decisions that best align with their goals and contexts.

Concepting

Using design to align stakeholders

Early high-fidelity designs centered on the manager journey helped communicate bold ideas and spark crucial discussions.

This approach was vital as we were developing a proprietary AI system, and navigating many unknowns. Because of this, there was a big opportunity for design to play a significant role in shaping the development of the system itself along with its features.

Documentation of early stage concepts for the project.
User testing

Validating ideas with store leaders

We wanted to get feedback from end users early to validate our concepts were valuable and usable.

We collaborated with a research partner to define our research goals and develop a test plan.

Research approach
Remote 1-on-1 interviews with Store Managers (x2), Store Leads (x2), Store Coaches (x2), and one Market Manager across the U.S.

Research goals

Uncover confusion about the new feature.

Identify additional expectations, unmet needs or concerns.

Evaluate the usability of key steps in the workflow.

AL user flow for research

User feedback confirmed the value of these new experiences, but identified areas to improve clarity and reduce complexity.

Key takeaways

Participants need more education on store outcomes.

Participants have a desire to see more data.

Some features were unclear or less intuitive (i.e. regenerate, share).

The experiences were seen as simple, clear, and intuitive overall.

Participants felt data and insights were easy to comprehend.

Participants liked the amount of content presented, despite some concerns of scrolling.

Iteration

Refining based on insights

Equipped with a new set of user inputs, we facilitated weekly discussions with stakeholders to manage changes in scope, priorities and requirements.

We collaborated with a large internal and external stakeholder group to align on key decisions that were core to the experience:

  • App architecture
  • Content organization
  • Integrating Machine Learning and AI
App architecture
Tab navigation

  • Tab navigation has reduced visibility and usability with our users.
  • Hard to scale when adding additional feedback methods and features.
  • Keeps survey data front and center which is what we are trying to avoid.
Design showing tab navigation
Page navigation

  • “Hub” model emulates large scale navigation in a mobile page environment.
  • Improves future proofing and scalability.
  • Increases store leaders’ focus on the appropriate task at hand.
Design showing hub navigation
Content organization
By outcomes

  • Increases cognitive load between survey themes and outcome metrics.
  • Increases the amount of content and scrolling.
  • Separates recommended actions across outcomes.
  • Creates confusion as to the origin of the recommended actions.
Content organized by outcomes
By things to celebrate vs improve

  • Prioritizes what’s important for store leaders to know, holistically.
  • Improves scalability as more feedback methods are added.
  • Centralizes all actions in one section to get store leaders to act, faster.
  • Creates a strong relationship between insights and recommended actions.
Design organized by things to celebrate vs improve
Integrating Machine Learning and AI
Defining the system

  • Defines what a “high quality” AI system looks like.
  • Leverages implicit and explicit feedback mechanisms.
  • Provides detailed frameworks for insights and recommended actions.
Documentation for the AI system
Final design

Delivering an MVP

We prioritized an initial experience that would be valuable enough to encourage store leaders to use these new features and build new behaviors.

Notifications and nudges

Increases awareness and encourages new feedback management behaviors.

prototype of push notification for AES survey insights
prototype of feedback hub
Feedback hub

Centralized location for reviewing and acting on feedback.

Intelligent insights

AI-driven analysis of feedback, providing natural language insights.

prototype of viewing feedback insights
prototype of adding recommended actions to action plan
Recommended actions

AI-generated recommendations for increased speed to action.

Flexible action plans

Easy management and tracking of store action plans, fostering collaboration and communication.

prototype of action plans
Impact

Expected impact after launch

0 %

Increased manager access to insights

0 x

Increased speed to insight

$ 0 million

Cost savings for Walmart

More work

Next project
Paladin

Pro Bono Engagement

Back