Back to projects

RALLYUXR

Turning Rally's most-requested feature into its enterprise unlock

Rally is a participant recruitment platform for UX researchers.

Study quotas was Rally's most-requested feature since launch. Without it, customers absorbed hours of extra work, and enterprise deals stalled on a competitor who had it.

Study Quotas Dashboard

Role

Lead Product Designer

(PM-less)

Timeline

3 months

0 → 1

Team

1 Engineer

Partnered with Leadership + GTM

Outcomes + Impact

Researchers got relief, the business got results.

MetricBeforeWith QuotasChange
Study Setup Time2-3 hours15 minutes↓ 90%
Studies Required3 separate studies1 study↓ 67%
Feature Adoption7% of studies3 weeks
Revenue ImpactDeals stalled$57K+ ARRWon

*Based on customer interviews and post-launch metrics

THE PROBLEM

A workaround that slowed recruitment and cost us deals

Recruiting balanced participant groups meant creating three separate studies and tracking progress in spreadsheets. Duplicate setup, scattered data, manual counting. One researcher described spending 2-3 hours per study on the workaround alone.

Before: three studies to recruit one balanced group

Before: three studies to recruit one balanced group

THE STAKES

A problem about to get worse

Rally had just launched Teams, enabling ResOps to democratize research across their orgs. More users in the product meant more people hitting the same friction. Without quotas, every new team member inherited the same workaround: three studies, a spreadsheet, hours of overhead.

Meanwhile, enterprise prospects kept asking "Do you support cohorts?" One fintech prospect was willing to stay with a clunkier competitor rather than switch to Rally without this capability. The feature gap was costing deals.

Signal was clear: quotas were blocking adoption

Signal was clear: quotas were blocking adoption

"I duplicate the study 3x to manage different roles. It's a nightmare."

– Research Ops Lead, customer survey

THE BET

Customers told us what, discovery told us how

I kicked off discovery by talking to 40+ customers about their quota workflows. I learned how they currently managed cohorts, what workarounds they'd built, and what would actually move the needle.

In parallel, I ran competitive analysis on tools that already had this feature. I documented what worked, what confused users, and where the gaps were. This wasn't about copying—it was about understanding the problem space and learning from others' mistakes.

🤠 Wore the PM hat: drove stakeholder alignment, roadmap vision, and GTM enablement. Partnered with engineering to scope V1 and make prioritization calls. Ran a 2-week beta with 4 customers before GA.

Competitive analysis of quota features

Analyzed how competitors handled quotas to identify patterns and gaps

Customer survey data and insights

Survey data revealed clear patterns in how researchers think about quotas

STRATEGY

Insights → Design Strategy

V1 focused on simple, flexible quotas that match common researcher workflows: 2 to 6 groups, segmented by screener or import.

Now

Foundation

Create and track participant groups within a single study, eliminating the need for multiple studies.

  • Create quota in studies
  • Basic assignment via screener or manual
  • Track progress by quotas
  • Auto-close quotas when filled

Next

Advanced

Includes templates, complex logic conditions, and advanced rich quota progress tracking

  • Templates & complex conditions logic
  • Multi-criteria & nested logic
  • Matrix views & cross-study metrics
  • Template library & advanced viz

Later

Automation

Intelligent research engine that automates balancing quotas, and optimize the entire research process.

  • Automated recruitment flows
  • Intelligent quota balancing
  • AI-driven participant matching
  • End-to-end research automation

THE DECISIONS

What we prioritized for V1

Focused on the most common quota needs, laying a strong foundation for more advanced logic.

1. Align with user mental model — Setup quotas in study builder

Setup quotas in study builder

Create status and set up recruiting cards per quota. Auto-close groups when filled.

2. Foundation to automation — Allow screener questions to automatically move participants to quotas

Conditional logic for automatic quota assignment

Maps screener responses to groups automatically. Reduces setup from 15 minutes to 2 minutes.

3. Baseline functionality — Allow manual distribution of participants to quotas

Manual participant distribution

ResOps needed control for edge cases. Move participants between groups, handle exceptions, adjust mid-study.

4. Better tracking and monitoring visibility — Real time tracking of quota progress visible

Real-time quota tracking

Progress bars and status tags sit directly in Overview. No digging into reports.

DEFERRED FEATURES

What we didn't focus on for V1

These ideas were explored but intentionally deferred to ensure V1 stayed focused on delivering the core foundations for Study Quotas and setting up the infrastructure needed to support more sophisticated workflows and automations down the line.

1. Complex logic — Allow deeper segmentation within a quota group (e.g., Android dev in the EU)

Complex logic for deeper segmentation

Deferred: Engineering complexity. Foundation logic covers most use cases.

2. Centralize management of quotas — Managing outside of the study and enable cross-study quotas

Centralized quota management

Deferred: Needed V1 to prove value first before investing in platform changes.

CUSTOMER ALIGNMENT

Building conviction, not just validation

Research Goals

  • Validate how quota setup fits into the existing product
  • Test the line between control (ResOps) and convenience (Researchers)
  • Prioritize what's essential for V1

What we tested

  • Quota group setup
  • Dynamic and manual participant groups
  • Quota management UI
  • Complex nested groups + auto-close logic

Structure & Participants

  • 8 customer + 3 prospects interviews
  • Topics: April 7-17
  • Fast iteration loop (long alignment by 4/19)
Customer research interview session

Leveraged Observer Rooms to involve the entire organization in the research sessions.

"Auto-close is non-negotiable. If we can't stop over-filling a group, it's unusable."

– UX Researcher, customer interview

"This would save me so much time — I wouldn't need 3 separate studies."

– Res Ops Lead, customer interview

"...of what I'm seeing right now — would really cover the big pain point, and then everything past that would be a bonus."

– Lead Researcher, prospect interview

Key learnings:

  • Manual assignment to quota groups is foundational.
  • Complex logic is ideal; supporting manual and screener-based grouping would solve 90% of workflow pain.
  • Visibility into quota progress would eliminate much of the spreadsheet tracking customers rely on today.
  • Users mental model aligned with setting up quota groups in the plan step of the study builder

LAUNCH

Shipped and celebrated

Launched June 2025 · View changelog →

Team celebration on Slack

"...your work has a DIRECT and measurable revenue impact."

– Rally leadership

See it in action

Video credit: Engineering partner in crime on this project