Reimagining the onboarding experience at CircleCI to help developers get to CI value seamlessly.
Skills in use Collaborators
Design Product Management
UX Research Engineering team
Facilitation UX team
Content Design
Challenge
After joining CircleCI's growth team as a product designer, I collaborated with a research colleague to facilitate over 30 discovery interviews with new users. We found that many first-time users struggled to set up a meaningful project, leading to lack of adoption. We identified the largest customer problems:
» Configuring a project in CircleCI requires users to write code in YAML, which can be painful and error-prone, even for experienced developers. New users wer spending hours combing through documentation and forums just to get a simple use-case off the ground.
» The previous onboarding flow lacked sufficient signposting, context, and feedback, leaving users disoriented. Research participants said it was difficult to discern what happened as a result of their actions in the system, and why.
» A recent initiative to automate the project setup process resulted in users perceiving the experience was a black box, where they didn't have control over their own CI (continuous integration) configuration.
Solution
Our team conducted an in-person design sprint, iterated to create a spectrum of low-fidelity prototypes that we tested through usability research, and arrived at a new onboarding flow that:
» Uses our fine-tuned LLM to automatically generate YAML configuration for our users, as well as convert configuration files from our competitors to ease the process of migrating tools.
» Provides ample signposting and context to help users understand what happened, why, and what's coming next.
» Allows users to choose from a few different project setup methods based on experience level, preferences, and security requirements, creating a sense of autonomy missing from previous versions.
Learnings
Speed up early value.
Although we worked backwards from our ideal-state prototype to release parts of the new experience rapidly and get in-production user feedback faster, the first round launch took longer than expected. In the upcoming release phases, we plan to release smaller — but complete — pieces of the experience so we can learn from real user behavior.
We started this project with an in-person design sprint involving colleagues across product management, product engineering, and architecture. From there, members of the sprint team iterated on prototypes individually, resulting in 36 different options in total. I took on the responsibility of distilling our team's strongest ideas into six unique interactive prototypes.
These prototypes were intended to help us test hypotheses and assumptions we outlined as a part of our research plan. I worked at a medium fidelity when producing the prototypes to allow for lightweight iterations, and I made them as different as possible to ensure coverage of all the hypotheses in research.
Low-Fi Prototype 1: 'Fastest'
Tested the hypothesis: Despite different levels of experience, users would prefer onboarding to be more direct vs. more educational
Low-fi Prototype 2: 'Build as you go'
Tested the hypothesis: Signposting and context are key — Users want to know what’s coming next and why they’re being asked to take actions
Low-fi Prototype 3: 'Conversational CI'
Tested the hypothesis: Users would prefer to describe their pipeline in plain text and let us figure out how to write their YAML configuration
Low-fi Prototype 4: 'Vision'
Tested the hypothesis: Users would accept a longer onboarding process if it provided more educational content and guidance.
Low-fi Prototype 5: 'Template framing'
In research on the first four prototypes, templates came up frequently as something participants wanted, liked, or expected. This prototype helped us understand:
Would being able to try the product before account creation make users feel more confident?
Do users prefer an onboarding approach with more control, even if it means more required decisions?
Do users prefer opt-in over automagic?
Low-fi Prototype 6: 'Fast-Dash'
Combined the most successful aspects of two of the first prototypes. Questions it answers:
Do users prefer an onboarding approach with fewer required decisions, even if that means less control?
Does being able to view the workflow map + logs provide users enough confidence after the first pipeline runs?
I facilitated and analyzed interviews in the first round of prototype usability studies and synthesized the data, noting findings related to the hypotheses we set out to test, as well as unexpected themes that arose. The synthesis allowed us to rapidly iterate, cutting out prototype concepts and hypotheses that were proven ineffective, and evolving the most successful approaches to create new prototypes.
After conducting another round of usability research to test our new prototypes, we identified two core experience principles for onboarding.
Save user decisions for when they really matter.
Too many decisions are taxing and caused evaluators to abandon the flow.
Too few decisions erode confidence and negate autonomy. We found that users didn't want our platform tools doing arbitrary and potentially dangerous things without asking. Getting this balance right is the foundation of the new onboarding experience.
Facilitate confidence & autonomy.
We drive engagement when users feel ready to bring on business-critical projects and/or invite additional team members to further their evaluation. Based on our research, confidence and autonomy were the sentiments most commonly correlated to readiness.
Equipped with our experience principles, I created a high-fidelity prototype of an onboarding flow that could automate project configuration for users who required more guidance, while allowing more experienced users the flexibility of configuring their projects from scratch or from a template.
Our team presented the flow to the CEO, CTO, and VP of Product, who validated that we were on the right path. With alignment on our vision state, we were able to define an MVP to deliver the benefits quickly to our customers.
The MVP features automated project setup powered by our LLM, improved signposting and interface feedback, and a streamlined journey from signup to meaningful project.
Subsequent rollouts will encompass a more robust template experience, a diagram representation of the YAML configuration, one-click platform migration assistance, changes to the visual language, and more.
Though we're still in the early stages of MVP release, we've already seen a 23% lift in our user engagement metric. We plan to continue validating through both quantitative data and qualitative user research.
