ROLLING RESEARCH AT SERVICENOW

Supporting an agile team with ongoing user research through launch of an Incident Response SAAS product

Overview

Users Incident responders, people who respond to incidents and issues with a company’s software services e.g. Site Reliability Engineer, DevOps Engineer

Method Remote interviews and concept testing

Impact De-risked launch of the product and provided user-data-driven answers to the team’s questions

Context

My project team was quickly driving toward a product launch of a new incident response tool in October 2021. As a point-solution, it was very different from other ServiceNow products. This was a new landscape, a new user, new competitors, and a new way of going to market. The entire cross-functional team transitioned to Scaled Agile Framework (SAFe) at the beginning of the year and product development had been moving at a rapid pace ever since.

Problem

To stand out in this market, products compete to be the best-in-class. They must be highly usable, efficient, and effective to win and retain customers. The plan was to launch a MVP (minimum viable product) incident response tool with an ambitious roadmap for releasing new features and functionality. As a UX Researcher, I helped the team understand the user so that they could build a successful, user-centered product in these high-stakes circumstances. 

Process

I addressed the highest priority and most timely research requests of PM, Design, and SWE by running five rounds of rolling research about six months prior to the planned launch date. 

I knew that recruiting for this user persona had proven difficult. The discipline is relatively young, and there can be wide variability in the use of titles and job responsibilities. The best way to ensure quality participants was to maintain ongoing connections with those that we had found in previous research to be a good fit. I was able to invite back many of the same participants with which the previous researcher had spoken in the last year. This saved time on recruiting and ensured that I already had important context and background information for each participant. 

I compressed the research timeline in order to deliver the most feedback possible and to keep up with the team’s sprint schedule. I collaborated with stakeholders from project management and design to identify questions about user needs and organize the most urgent topics into rounds of rolling research.

The team was encouraged to attend sessions in a virtual backroom. After running sessions and analyzing the data, I shared detailed findings with stakeholders and high-level findings with the entire cross-functional development team. 

Impact

Each round of research impacted product plans or designs. I uncovered users’ current work processes, expectations, and attitudes around incident response tools. I collected users’ perceptions of key page designs and recommended design changes based on the feedback. I discovered users’ mental models and recommended the most complimentary conceptual model for the product. I provided guidelines for building effective user onboarding and explained user expectations for integrations with communication tools and command line interface. Each of these contributions helped to de-risk the product before launch and inform the future roadmap.

Surprises

On round three, I initiated a new recruiting process to attempt to get fresh participants into the rotation. I was working with a vendor who screened participants with the screener that I provided. My first interview with a new participant did not go well, as the participant was not a good fit for the study. I circled back with the vendor to review the participant’s screener answers. I learned that even though the screener was clear and specific, there was still a chance for people to pass the screener and not be a good fit for the study. In such a fast-paced environment, we didn’t have time for more misses like this. I quickly pivoted the recruit to invite back previous participants and worked out a plan with the vendor to double-screen future participants. I also piloted a new process with a different recruiting vendor who specializes in speedy recruits and double-screening capabilities. 

Learnings

This rolling research succeeded in providing timely insight to the cross-functional team. I was able to iterate on this research program to do another round of rolling research around the time of launch. The second time around, I was able to improve the synchronization of the rolling research with the agile schedule, set clearer expectations with stakeholders around the opportunities and limitations of the method. I also increased access to quality participants by utilizing the second vendor mentioned previously.