When I was at Microsoft, one of the Azure Machine Learning team’s strategic initiatives was “five minutes to wow.” Our aim was to ensure that a developer new to Azure ML should, within 5 minutes, achieve such a surprising/useful/novel outcome with Azure ML that they say “wow!” Inspired by the goal, I worked on a feature in my product for real-time inference called “no-code deploy (NCD).” With NCD, a developer new to Azure ML could click on a button in a GitHub repo containing a machine learning model, fill out some fields in a form, and deploy that model as a REST endpoint in Azure. While NCD really impressed my leadership, it failed to make a significant impact on the top-line metrics we cared about: number of active users and retention.
It was literally impossible to make my product any easier to use, and yet users still weren’t sticking around. How could I explain this conundrum? As I talked with churning users I realized that I had simply replaced their “day 1” questions with “day 2” questions. Users went from “how do I deploy a model with Azure ML?” to “how do I deploy my model with Azure ML?” Because we had obscured all the details about what happens when deploying a model, like defining a Docker container, uploading the model to storage, or running the model in an entry script, as soon as users wanted to actually use the product, they were actually worse off than they had been before, because they had the same knowledge of my product but higher expectations of what it should do.
Armed with this new insight, I focused on refactoring our tutorial to introduce concepts rather than to hide them. Start by creating an entry script with no model, then add in your model, then modify the Docker container. The quickstart actually got harder, but more users were successful when using it. Our support team saw an immediate reduction in basic conceptual questions from users.
My personal experience was only one example of a frequent misconception I see among developer-first companies that their quickstarts need to be as easy to use as possible. Taking this idea to an extreme, some developer-first companies will actually build quickstarts similar to mine where all the code has been written in a code sandbox, and all that remains for developers to do is to click buttons in the correct order. Unfortunately, because there is no meaningful cognitive engagement on the part of the user, the developer can very easily zone out and learn nothing during the quickstart.
Csikszentmihalyi’s concept of “flow” captures roughly what I’m trying to say about developer quickstarts:
You don’t want your quickstart to be like vim’s, where the learning curve is so steep that developers enter the “anxiety” section and avoid learning the tool at all. But you also don’t want a highly guardrailed experience like AWS Amplify’s that removes all agency from the user and relegates them to “boredom.” You want the developer to find a “flow,” the happy medium that gives them opportunities to think about real problems and how your product can solve those problems.
The Python Django quickstart is a great example that gets “developer flow” right. This tutorial goes so far as to intentionally introduce a bug for a developer to resolve. Because this quickstart requires developers to take non-trivial actions to succeed, they leave feeling not just motivated to do more but equipped to do so.
When you’re building onboarding experiences for developers and make a decision to hide something, ask yourself: is this step I just hid something that most developers will probably need to learn at some point? If the answer is yes, find a way to reintroduce that concept into the quickstart in a way that makes sense. Change your goal from “5 minutes to wow” to “10 minutes to understanding.”
Learn more about how API usability impacts your business
Make your quickstart harder
When I was at Microsoft, one of the Azure Machine Learning team’s strategic initiatives was “five minutes to wow.” Our aim was to ensure that a developer new to Azure ML should, within 5 minutes, achieve such a surprising/useful/novel outcome with Azure ML that they say “wow!” Inspired by the goal, I worked on a […]
How Courier is using Usabl to unlock developer virality for its multi-channel notification API
“Videos from Usabl are timely, informative, and actionable. They help us uncover previously unknown issues in our developer experience and build internal consensus around real solutions” – Donnie Wang, Growth PM @ Courier Courier’s API makes it easy to send multi-channel product notifications reliably and in a way that is respectful to the end user. […]
We evaluated the usability of 4 leading eSignature APIs
Most were surprisingly bad eSignature as a space has been heating up for a while now. Look at Dropbox’s $230 million acquisition of HelloSign in 2019 or PandaDoc’s recent unicorn status for evidence of the growth in interest in the sector. The high valuations reflect the fact that signatures are involved in some of the […]
Why you, a UI-first business, should care about developer experience
What do the video game Doom, shopping malls, and Salesforce have in common? (No, it’s not that they all are fun ways to spend an afternoon). Doom was remarkable as one of the first PC games to support “modding.” Shopping malls are only as successful as the tenants that rent space within them. And Salesforce […]
Tag yourself: a framework for developer experience maturity
As we’ve built out Usabl’s platform for getting feedback from real developers, we’ve spoken to dozens of companies building API- and developer-first products for everything from payments processing to security scanning. In each conversation, we’ve consistently seen patterns in the way these businesses engage with developers. In this piece I propose that developer-first businesses can […]
Understanding the developer activation funnel
Your developer experience should be your best marketing tool As we’ve interviewed companies building developer-facing SaaS products, I’ve seen some common patterns emerging in the way companies build for developers. In this blog post, I’ll explain at a high level the journey that a developer will typically take with a SaaS product. To the best […]
API usability for developer-first products
It’s 2021, and developer experience is coming more and more into focus. In fact, companies are hiring product managers specifically with the goal of improving their developer experience. Staying close to your customer matters in every industry. Though it feels harder to get feedback as developers are expensive and specialized, Usabl is focusing on helping […]
Writing your first task
Writing your a task for an API usability test Let’s say you’ve gone through the steps here and chosen a well-sized task that tests your riskiest hypothesis about the usability of your API. You’re now ready to actually write your first task. How should you go about doing this? Minimize configuration time Remember, your goal […]
What part of your API should you test for usability?
So you’ve decided you want to run an usability test to discover ways in which you could improve the developer experience with your APIs. Congratulations! The next step is to decide what exactly to test. You might be thinking of simply taking one of your public samples, removing some of the code, and asking the […]