Measuring UX capacity
Measuring a team’s capacity and velocity is nothing new in the engineering world, but we haven’t quite cracked the code when it comes to doing the same in the UX space.
Building upon traditional agile methodologies, and taking into account unique challenges that UX team’s face, I’ve successfully developed a light weight way to track a multi-disciplinary UX team’s capacity which has aided in tradeoff and resourcing discussions with managers and partners alike.
The problems
After being in the industry for a number of years, I started to notice that engineering teams were able to provide extremely accurate estimates of work, push back on unreasonable timelines, discuss tradeoffs, and very clearly communicate resourcing needs. For UX teams that I’ve been a part of, we were unable to do any of this well. After talking with multiple UX team members across a number of organizations, the key problem areas were typically very similar:
Inability for individual contributors to manage bandwidth, resulting in burnout
Feeling unable to communicate tradeoffs due to lack of enablement & support
Decreased visibility of the team’s workload for managers and leadership
Misaligned priorities across lines of business and disciplines
Lack of education and alignment around deliverables and timelines, resulting in inaccurate estimates and missed deadline
Proof of concept
I started with a very simple proof of concept, with the help of an amazing product owner and scrum master who worked closely with the development team I supported.
In this 5 month experiment, I tracked my work in the same way the dev. team did, by estimating story points for UX tasks and tracking completion at the end of the sprint. My product owner and I tracked the time I spent in agile ceremonies, repetitive meetings, and activities like training and PTO to understand the actual time I had available for doing work.
After about a few months of tracking, some interesting trends began to surface:
My original capacity estimate for a 2 week sprint was 25 story points. In actuality, I was only able to take on 5.9 story points on a regular basis.
When I started managing an intern, my capacity dropped to 3.6 story points per sprint due to the amount of support I was providing.
Of my time allotted to the team I was spending 41% on actual design work (roughly 4.5 days out of a 2 week sprint)
Using this data, my product owner and I were able to identify needs for additional UX support by calculating my current capacity and measuring the rate by which new tasks were added to my backlog. The data spoke for itself – In order to simply keep up with UX needs for the team, ~1.5 additional designers were needed.
Scaling the process
When starting to think about how to scale the process, I had to take a step back and look at root causes of the problems that I wanted to solve.
Speaking the same language
First and foremost, UX, Product, and Engineering needed to speak the same language. At the time, there were a few different terms for deliverables that the UX team used, which caused a lot of confusion for product and engineering teams. Additionally, the time it took to complete those deliverables were a mystery outside of the UX org, making it hard to estimate project roadmaps and when engineering could expect handoff from UX.
I took the UX team through a card-sorting exercise, detailing out different types of tasks or deliverables we created on a regular basis and grouping them by the amount of time each took to complete. From there, we were able to assign a size to each of the categories, which provided us with a baseline for sizing other UX deliverables and allowed us to set expectations of estimated UX effort with product owners and dev teams.
Standardizing an intake process
Often times, UX resources were treated as supporters rather than partners, not being included in project roadmap discussions and being given a concept to execute against with little time to complete it in. A lot of times the projects and initiatives met a lot of business needs, but failed to take the customer into account, resulting in additional upfront research or questions which ate away at the already short timeline. To solve for this, I enlisted a small working group who created a comprehensive intake form which was required prior to the start of any project. We sent this to product owners to fill out and worked with them to refine the concept, keeping the customer top of mind. Much to our surprise, Project Managers had no problem filling out the form and loved seeing the team jump into action faster!
Adopting agile processes
Understanding priorities
Okay. So now we had a way to become informed of projects ahead of time, but now how do we know if the request is something we should be working on at all, let alone prioritize them across members of the team?
I enlisted the help of a Scrum Coach to help set the UX team up with a JIRA board and again, with the help of the working group, filled it with active and upcoming epics and related stories and tasks. We then took the board to Senior Leadership and asked them to prioritize the epics to ensure the team was working on the most critical items. Once finished, we finally had a common understanding of organizational priorities and were able to push back when a request came in that didn’t align.
Increasing communication
Like I had done in my original proof of concept, the UX team agreed to become more involved in our engineering partner’s existing dev ceremonies and, with the help of some awesome Project Managers, were able to groom, plan, and execute upon UX stories during the same sprint cycle as the dev team.
Very quickly, the team saw a reduction in rushed UX requests, questions about execution coming from the development team, and complaints about UX being a bottleneck.
Education & evangelism
With all of the knowledge the working group and I had gained, we needed to ensure all other members of the UX team as well as partners were aware of all we had been working on to further scale this process to the rest of the team. We developed an educational series for each audience:
Education for the UX team:
Agile Basics
Learn what Agile means and the ceremonies, processes, and lingo usedTooling 101 & Best Practices
Learn how to use JIRA (in this case) to ensure the team is using it in the same wayBi-weekly Food for Thought Discussions
A fun lunch time get together with the UX team to watch inspiring industry-related videos, discuss articles, and align on deliverable templatesTeam Peer Review
A time to review and get peer feedback on work in progress (Can you believe that we didn’t have one before this!?)
Education & evangelism for Partners:
UX 101
In addition to giving an introduction about what UX is, we explained the purpose of typical UX deliverables and the time it takes to complete themBrown bags with product & development teams
To inform partners about this process, answer questions, and build upon UX 101 sessionsLeadership evangelism
We needed leadership to have our back so we could feel enabled to manage our own backlogs and push-back if needed.
Rollout plan
To ensure success, I didn’t want to just roll out this process to everyone. Not only were some teams not as diligent in their Agile processes, the working group and I got some push back from middle-management and even individual contributors in the UX team so we needed to ease them into the idea of working differently.
I formulated a roll out plan as follows:
Phase 1: POC (Already done)
Timing: 5 months
Teams involved: 1
Inspect & adapt
Draft process documentation
Gather data to support process change
Gain leadership support
Phase 2: Working group (In progress)
Timing: 5 months
Teams involved: 5
Inspect & adapt
Gather input from leadership & partners
Gather data from multiple teams
Identify Agile UX champions
Form UX team training & workshop content
Evangelize within company
Phase 3: 100% rollout
Timing: Iterate going forward
Teams involved: All teams
Inspect & adapt
Organization-wide adoption
Report on capacity & velocity
Gather input from leadership, partners & ICs
Ongoing education within UX team
Ongoing company-wide evangelism
Iterating
Unfortunately I left Concur before I could see the entire plan come to fruition, but started building upon this process in the next companies I found myself at. I found that the process was very easy to adapt to other company cultures and ways of working.
Capital One
I contracted for a short time at Capital One as a UX Program Manager, where I noticed immediately how differently this team worked:
The team was already using a distilled version of Scrum-ban (a combination of Kanban with the addition of team-wide stand-ups and grooming sessions)
This UX team was smaller, and definitely less excited about “official” process and definitely against using JIRA, though they did acknowledge the same problems I identified at Concur.
There was an Agile Coach team, but they really didn’t interface with UX at all.
So…
Instead of using JIRA, I enhanced the Kanban board on the wall and included engineering and project manager partners to the daily stand-ups to increase communication. Daily, we took time to write tasks on sticky notes and talk about them to achieve common understanding.
We were working on a REALLY tight deadline, so I also printed out a physical calendar each month and added more sticky notes (color coded by discipline) to achieve alignment on key milestones across design, content strategy, UX research, and partners.
Because everything was moving so fast, at the end of the month, I posted all of the finished sticky notes on another wall and we took time to celebrate the massive amount of work the team has accomplished. Doing this not only really boosted morale, it solidified the need for some process in the minds of certain naysayers.
I formed a tight partnership with the Agile Coach team, and enlisted their help to keep the process going and to iterate on it once my contract ended.
Expedia Group
I joined Expedia Group as a contractor and again saw the same issues there! My manager was ecstatic when I told her about this process, so I started a pilot on my team. Again, the company used different tooling and I had to once again shift this model to fit company culture.
Expedia used a lot of different tools but the organization had primarily settled between simple spreadsheets, JIRA, and Trello for tracking purposes. Airtable had just recently become popular in the UX team, but licensing was expensive.
My team worked in a squad model, where multi-disciplinary teams organized around a specific customer problem for 6-12 weeks.
This particular engineering and product team had never worked with UX before
The UX team was in the process of switching from Sketch to Figma, a highly collaborative tool that reduced the need to redline, if engineering teams knew how to use it.
My manager wanted to understand the workload associated with our problem area to prove that full-time resources were needed
So…
The most immediate need was education. My teammate and I put together UX 101 & Figma 101 presentations together to educate our product and engineering partners about User Experience Disciplines and how to use the tooling to simplify handoffs.
I started tracking capacity and workload for the team using a google sheet, taking into account recurring meetings, time off and holidays.
Tracking in a spreadsheet got pretty tedious so, I then moved to Trello and Airtable. All methods had their pros and cons, but we managed to make a case for headcount and became full time 6 months later.
I used a variety of Trello Power-Ups, including Screenful, CardSync, Custom Fields, and Butler to help automate some of the reporting that my manager then used to request headcount.
I also modified the story estimations from high level t-shirt sizing to story points to better align with the engineering team’s method of estimation. This estimation scale included high-level timing so Product Managers could create more accurate roadmaps without needing additional meetings.
The outcome
Because of months of tracking, I was able to put together the first truly data-driven UX staffing plan at Expedia, that ultimately resulted in my colleague and I transitioning to full-time rolls!
2 additional work-streams in Expedia’s UX team have been on-boarded are still using this framework today.
Based on this process, colleagues on a different work-stream created an Airtable base template to extend this process to teams outside of Expedia.