Notes for Measuring Developer Relations on swyx.io
Your North Star metric is “monthly active developers”. If growth is accelerating, good. If growth is constant, fine. If growth is 0% or worse then whatever you are doing isn’t working.
MAD is multi-causal and a lagging indicator so you need leading indicators which you have more direct control over.
Unhappiness arises when there is an expectations mismatch between what the company wants out of DevRel and what each DevRel is naturally good at.
There are three emerging sub-specialities of developer relations: community-focused, content-focused, and product-focused.
the ratio of this two way street traffic is 99% outbound and 1% inbound, because the product/engineering orgs haven’t set aside any bandwidth for “shadow PM-ing” from devrel and all of devrel’s metrics are outbound focused
a typical developer content creator’s CPM is $5, a normal devrel’s CPM is $50-$ 100. In other words you are paying 10-20x more to make content “in-house” than just paying a “professional” to make something about you.
you have to provide some other form of value than raw anonymous reach. This can be: depth, breadth, consistency, access, insight, community, email list building, etc.
you have to sell features, not benefits
Try the Sean Ellis question instead: “How would you feel if you could no longer use the product/be part of the community/read this content?” and use it as a way to segment/understand your audience
You are hereby banned from suggesting to A/B test anything if you do not have the traffic or the infrastructure to easily implement an A/B test like 90% of devtools startups
People are waking up to the lasting power of community and now Technical Community Builder is the Hottest New Job in Tech.
Community metrics that I like (pick 1-3):
- Number of active members in Slack/Discord
- Number of weekly new topics in Discourse/StackOverflow/GitHub Discussions
- Number of user contributions (whether it is PRs, questions, answers, blogposts, meetup talks, etc)
- Number of Orbit level 1 users
- Number of events and number of attendees
- Number of engaged “superusers”
Beware the optics of regularly asking your users to do free work for you,
…most company “communities” are really just support forums
Having great content is often the “minimal viable community”.
One definition of a good community is that people expect their association with your project/product to outlast their current employment
“forums and chats are “cold”, events are “hot”
We had a Principal DevRel whose primary job was to help organize our 3 conferences (together with Marketing) and their associated community. If he did nothing else but that, he would already have been worth his weight in gold to us.
develop a sense of what kinds of things win hackathons — anything realtime (because a live demo can include the audience), anything phone based (everyone has a phone), anything AI based (people like seeing machines act like people), anything visual design
A small group of highly engaged fans can make a community feel more alive than a large group of passive users.
Runa Capital has a definition of Active Contributors, and the big dogs go up to 200 regular contributors a year.
Devrel scales by how it enables users to tell (even brag to) each other about their usage.
How do you bring your community closer in touch with your engineers?
“How can you help your users hire each other?
I recommend producing at least 1 piece of content on your company a week
“Content metrics that I like (in rough order):
- Number of Newsletter subscribers
- Number of YouTube subscribers
- Number of Twitter follows
- Number of Workshop completions
- Number of Conference/meetup appearances
- SEO Domain Authority
As a content creator you have the choice between TOFU, MOFU, and BOFU content:
- Top of Funnel: never heard of you (Awareness)
- Middle of Funnel: comparing you to others and learning basic features/concepts (Evaluation)
- Bottom of Funnel: deciding to buy and put you into production (Conversion)
If you only measure website traffic then you naturally incentivize DevRel to create clickbait TOFU that doesn’t have to convert at all and alienate your biggest fans.
Newsletter signup (incentivized by great company blogposts and updates) keeps you honest.
Most marketers and professional creators view 1 email subscriber to be worth between 100-1000 social media followers.
“The more seriously you view yourself as ”building a media-company-within-a-company” rather than “doing some content marketing”, the better you will do.
How long until a piece of content gets half of the views it will ever get in its lifetime. Twitter’s half-life is hours, YouTube’s half-life is months, blog is years. Create accordingly.
“Test things on more ephemeral media before developing them further on more permanent media.
Managers think they are helping by establishing content calendars. I have never seen a DevRel operation stick to these beyond month 1
…for YouTube, the recommendation algorithm does reward weekly output
5-10 million developers on YouTube (my estimate). There are between 1-2 million developers on Twitter (my estimate). There are about 6 million developers on Hacker News.
“…best way to get started is just default to consistency
Workshops are extremely underrated forms of content for achieving depth
It’s well worth producing 1-2 great conference talks a year at high profile events, and sometimes to get there, you need to practice and iterate at 5-10 smaller venues, but anything beyond that probably has diminished returns compared to anything else you could be doing.
Being able to give the same elevator pitch over and over again at different meetups can be great for capturing the attention of a small set of developers.
Raw # of meetup appearances is probably a good enough metric here.
Most developers discover solutions more by hearing about them from friends and thoughtleaders than by searching
Auth0 and Digital Ocean famously prioritized an SEO based approach, however your devtool/brand should be general enough for this to work.
who owns example repos and code samples and keeps them up to date?
are you creating content because your docs aren’t clear enough? are you creating docs because your product isn’t intuitive? All content ultimately has a half-life — sometimes you have to go downstream to fix root cause
Product metrics that I like:
- Number of launch day users, and positive launch day mentions
- Number of prioritized user issues from DevRel
- Number of monthly users of integrations/tooling
- the Seanl Ellis question (over NPS) for integrations/tooling managed by devrel
Gustaf Alstromer’s PMF measure: Value metric, and % retention at ideal recurring frequency
Devrels can provide tremendous value in the lead up to any product launch by beta testing (either personally or with users), or creating eyecatching/inspiring demos, blogposts, and videos for launch day/annual conference.
Devrel are also often responsible for maintaining non-core integrations and helper tooling.
Netlify has an entire Integrations Engineering team.
Solutions Engineering: where you help out with customer-specific custom demos and integration work. For large enough customers and small enough startups this is fine, but the bulk of the DevRel effort should focus on work that scales.
many CTOs and VP Engs recommend a multiyear adoption path especially for core tech. Traditional marketing advice says people have to hear about you at least 7 times before they decide to buy; for developer tools, Matt Biilmann from Netlify says it’s more like 14.