How we rebuilt Shopify’s developer docs
A detailed look at what we did to create a better experience for developers
Today, Shopify relaunched its developer documentation. This was a long, complex project that required pulling on every UX and development discipline working inside the company, and it’s great to see it finally out in the world. I had the unique privilege of working on this project from beginning to end, so I wanted to share some of the lessons we learned on the way.
The final verdict is, of course, up to the community of thousands of developers who turn to our docs every day. We won’t know for some time whether we’ve truly hit the mark, and as we continue to gather feedback, we’d love to hear from you.
I can, however, provide some background on the origins of this change: why we decided it needed to be done, and how we decided on the changes we’ve made.
As with just about everything we launch at Shopify, our developer docs will continue to evolve as we learn more about how people use them. But the core of this project was about trying to understand what makes a great developer experience.
So how do we know? Well, we asked.
Why change the docs?
First, let’s rewind. What’s the problem we were trying to solve here?
More than 1,000,000 businesses now run on Shopify, and each one has unique needs. Shopify solves many of their hardest commerce problems out of the box — but it can’t be all things to all people. One of Shopify’s secret weapons has always been the large community of third-party developers who build apps, themes, and other integrations to solve problems for merchants. Their hard work makes Shopify flexible and adaptable to just about any special case you can think of.
Shopify has grown a lot over the last few years, and we were finding that our developer documentation wasn’t scaling at the same pace. The growing pains showed up in subtle but increasingly troublesome ways:
- Tutorial and how-to content had become mixed in with reference documentation. My analogy would be like using a recipe that doesn’t separate the list of ingredients from the directions. Sometimes this is the right choice! But in our case, it was leading to growing complexity that was making the docs harder to maintain. We haven’t totally solved this problem yet, but we have a clear path forward with this new site.
- The old site structure was very deeply nested, which made it hard for developers to browse and discover new and useful features spontaneously. Take something like Kit, the conversational “virtual employee” that merchants can install on their stores to automate common tasks. This is a powerful feature of Shopify and developers can extend it in all sorts of ways. But if you didn’t already know about Kit, you were fairly unlikely to stumble over it, since it only showed up three levels deep in the “Embedded Apps” menu. (Why was it put there? No one could quite remember.)
- The app and theme developer docs were struggling in the Shopify Help Center.
The Help Center’s primary job is to assist business owners who are running Shopify stores. Developer content, focused on interacting with the underlying Shopify APIs, was sometimes an awkward fit. Shopify merchants and developers sometimes have related needs — and plenty of merchants do their own coding! — but there are genuine differences between the types of documentation each audience needs. We wanted to set up our developer documentation platform for future growth and evolution by separating these concerns.
Some of these pain points were visible when browsing the docs. Others were internal organizational issues — the little behind-the-scenes dramas and stresses of any fast-growing company. Both types were making it harder and harder to maintain the docs, let alone grow them for the future.
Our developer docs were, in fact, rated fairly well in our annual surveys of the community. There was no crisis that demanded an instant fix. But we could see trouble on the horizon. Documentation that is hard to maintain is documentation that soon gets outdated, duplicated, and disorganized. So we knew we wanted to address these problems sooner rather than later. We knew we needed to do…something. But what, exactly?
The content audit
I started by simply trying to get an idea of how much stuff we were dealing with. Our goal was to get the lay of the land, understand the potential scope of the work, and start narrowing the focus to a manageable, shippable project.
I pulled out my dog-eared copy of Content Strategy for the Web, by Kristina Halvorson and Melissa Rach. It’s one of the definitive books about the craft, and it contains great advice on how to do a thorough content audit.
There’s not enough space here to get into all the gory details of my audit methodology, but it was a combination of manually combing through our codebase, some brute-force web and API scraping, and trawling through our historical analytics. It took a few weeks, but in the end I had an enormous spreadsheet that could answer many of our quantitative content questions quickly.
In all, my initial audit turned up 2,868 pages of material that would be relevant to Shopify’s developer community. It included things like:
- Hundreds of pages of instructional documentation in our Help Center
- Hundreds more pages of API reference docs. (For the most part, this type of page, describing a particular API resource in detail, is automatically generated straight from the actual API, not manually authored by our technical writing teams)
- More than 800 posts on our Partner Blog, many of them covering development topics
- Shopify’s Partner Academy, the online learning platform that offers courses and certification, and which contains hundreds of pages of educational material
- Polaris, Shopify’s open-source design system
- Around 500 open-source GitHub repos.
By correlating individual URLs with analytics data and git commit histories, I was able to fill out the spreadsheet with more detail, and we began to see some pretty clear patterns: which parts of our docs were being revised most actively; which sections were most popular with readers; which ones were performing well in search. I also categorized pages by type (auto-generated API reference pages; human-authored instructional content) and by broad topic (themes, apps, custom storefronts).
At this point, we felt like we had a good map of the territory, but a quantitative content audit can only really give you a snapshot of how things stand today. To figure out what changes we needed to make, we’d need to talk to users.
The research phase
To puzzle through our questions in a more structured way, I turned to our User Experience Research team. One of the great privileges of working at Shopify is having access to world-class UX researchers who love untangling exactly these kinds of thorny, ambiguous questions.
For this project we did two types of research: one-on-one interviews, and a moderated card-sort exercise. In both cases, the goal was to figure out a common mental model for the Shopify ecosystem. If we know how people think about our platform, we can design an information architecture that matches people’s understanding, reinforce the things they already know, and, hopefully, fill any knowledge gaps.
User interviews
Andrew Rajaram, our UX researcher on this project, designed the program and conducted most of the interviews. We recruited two panels: existing Shopify partners who were already experienced with the platform; and developers who had never used Shopify before. We also balanced the list for gender representation, a mix of international markets, and business size, from solo freelancers to people who worked at large firms or agencies.
Andrew drew up a script to run through with each interviewee, so that we’d ask all the key questions consistently. We scheduled the interviews for an hour each by video chat. In each one, there was the interviewee, the interviewer, and a note-taker to observe.
The interviews were instructive. There were a few clear patterns that came through:
1. Most developers wanted to learn by doing
While there’s definitely a segment of developers who thoroughly read over documentation before starting their development process, a majority of the ones we interviewed like to start something quickly and see how far they can get, consulting references and tutorials as they go.
There’s fascinating pre-existing research on this topic, identifying developers as taking either “systematic” or “opportunistic” approaches to navigating software documentation. The systematic ones like to read first, then code. The opportunists start writing code as quickly as they can. It’s worth noting that neither style is associated with “better” code or a more successful outcome; but it does mean that documentation needs to be accessible and useful to both types of learners.
2. Interest in legal documentation scaled with firm size
At mid-sized agencies and up, the interviewees typically asked more about the API terms of service, privacy policies, service-level agreements, and other legal or contractual documentation. This is especially important for those types of companies, because they’re often working on behalf of clients who have their own requirements around licensing and user privacy.
This was one of the valuable new insights we learned from our interview process — one that hadn’t been suggested by earlier research. Once we heard it, of course, it made perfect sense. As a result, the new developer docs foreground things like Shopify’s API Terms of Service, and concepts around user privacy and legislation.
3. Documentation needs change over time
This may sound obvious, but this insight is what steered us away from organizing our documentation around user personas. Often, UX practitioners bucket users by common personas: for instance, if you were building an app for running a car dealership, you’d probably design different activity flows for the people selling the cars out on the lot (a “salesperson persona”), compared to the mechanics working in the service bay (a “mechanic persona”). This technique allows you to make informed generalizations that address common workflows and user needs.
This clear-cut division of labor is less prevalent in software development. At a larger shop, there may be a vice-president who needs to know about the business opportunity, a chief product officer who needs to know about the platform’s high-level technical capabilities, and a team of devs who need to know the API references in detail. But often, those information needs overlap and blur; sometimes, all those “personas” are, in fact, a single individual — just one who’s progressing through the different stages in their development process.
This was what convinced us that organizing docs around personas was a dead end. People aren’t “beginners” or “experts” — they’re people with a problem to solve. We can’t read their minds or pigeonhole them based on (likely erroneous) assumptions about what they want to do at any given time.
This finding led us toward a flatter information hierarchy, increasing the number of topics at higher levels of navigation. Our hypothesis is that this makes it easier for casual users to browse and discover, while motivated users can more quickly drill down to answer their specific questions.
Card-sorting
Our next step was a moderated card-sort exercise. This involved giving developers a stack of about 50 “cards”, each with the name of a documentation concept or topic on it. We asked them to sort these cards into piles that made sense to them, and then label the piles. We wanted to see if we could find common mental models in how people arranged information about Shopify’s platform.
What emerged was surprisingly consistent. Very often the arrangement of piles came out looking something like this:
- Basic information about the platform
- Legal requirements and responsibilities
- Thorough reference documentation
- How-to guides, instructional material, tutorials
- Software tools, libraries, and SDKs
What was really interesting about this result is that it matched some other research we’d been doing in parallel about DITA, which is an XML-based format for managing documentation, originally developed by IBM in the early 2000s and now an open standard. DITA categorizes instructional material into three broad categories:
- Concepts (contextual information you need to know)
- References (a detailed description of the thing you’re working with)
- Tasks (actions you can take, based on your knowledge of 1 and 2)
The overlap between theory and behavior was striking. People were consistently looking for:
- Background information and context (Concepts)
- A comprehensive “dictionary” of Shopify’s capabilities (References)
- Guides to implementing common workflows (Tasks)
- Tools to help them do their work faster
DITA’s core specification doesn’t address “tools”, but the categories were clear enough that we decided we could start prototyping and try revamping the information architecture around these ideas. The result, many months later, is what you see today.
The result
As you can see, the architecture of the site pretty closely mirrors what we learned from the user research. The five top-level “zones” are Concepts, Docs, Tutorials, Tools, and Community (the last of which, for now, only links out to other parts of Shopify, like our developer forums and Partner Academy).
With today’s launch, Shopify.dev has become the primary home of all our developer-facing documentation and educational content, and we see lots of room for further growth and innovation in Shopify’s developer experience.
We feel pretty confident that this new information architecture sets us up to continue growing in an organized and scalable way. But whether it’s useful? Intuitive? Quick to navigate? Clear? That judgement is ultimately up to you. We’d love to hear what you think in the comments below.
