Reimagining ReadMe's Markdown Engine

Migration Guides & Documentation→

If you’ve worked on any technical docs recently, you’ve almost definitely written Markdown. Love it or hate it, it’s the closest thing to a standard in terms of how we document our code. Jeff Atwood explains its dominance succinctly:

Markdown is the worst form of markup, except for all the other forms of markup.

From newcomers like Notion, to establishment names like GitHub, everyone uses Markdown. But not at ReadMe. Well, not entirely at least...

Magic Blocks

ReadMe’s docs are a strange hybrid: part Markdown, part proprietary format that we call “magic blocks.” These JSON-based blocks represent rich components, and can be mixed in directly alongside plain-text Markdown. Seven years ago—before the proliferation of MDX, or StoryBook—this truly did make for a magical documentation experience. But the web matured, ReadMe grew, and our bespoke syntax increasingly strained to keep up.

Features were missing. Bugs needed quashing. All this magic started to get in the way of development of features like GitHub sync, and PDF exports. And it was adding friction to the onboarding process and authoring workflows, especially for users who had come to expect standard Markdown.

Presto Change-O!

This got us thinking: Were our custom magic blocks really much more than stylized versions of any old Markdown component? Could we get a conventional Markdown processor to read and recognize this JSON-based format? And would we be able to refactor each magic block into some sort of Markdown-y syntax?

The first step was to punch up the Markdown standard “to taste”, so we started to spec out possible syntax extensions. As we mapped these to our magic blocks, we found that many of them—such as headings and images—were already covered by standard Markdown. Where there wasn’t a clear one-to-one mapping, we tried to extend existing constructs and hew closely to the contours of the original syntax.

Next, we needed our engine to parse the magic block code itself. This would instantly give us broad backward compatibility with nearly any ReadMe doc ever written. Finally, we wrote out the actual parsing and transform logic for each of these extension, as well as a set of React components to render an mdast tree in the browser.

ReadMe-Flavored Markdown

Docs & Guides→ The result of this work is our ReadMe-flavored syntax extension and a brand spanking new Markdown engine to go with it! Let’s look a bit deeper at how this works in practice, and why it’s so useful.

Say we had some docs we were syncing to ReadMe from our repo. In one of them, we might want to display a set of tabbed code blocks. Sure, we could handwrite the magic block, but honestly even I wouldn’t know how to do that off the top of my head! Let's take the lazy route, and rip the generated source code for this component directly out of ReadMe’s raw editor:

[block:code]
{
  "codes": [
    {
      "code": "console.log('Code Tab A');",
      "language": "javascript",
      "name": "Tab 1"
    },
    {
      "code": "console.log('Code Tab B');",
      "language": "javascript",
      "name": "Tab 2"
    }
  ]
}
[/block]

Now we can paste this back in to our doc, sync it to our ReadMe project, and we’d get a nice tabbed set of code blocks:

But just imagining what this will look like in the GitHub repo physically pains me... Much less thinking about how annoying it would be to edit this as a plain-text document. Or worse, actually having to read this and make sense of it, especially if you were just jumping in to a new code base. Oy vey!

What if we tried writing it out in our new, flavored markdown syntax instead? This should feel instantly familiar, and it’s certainly a hell of a lot more readable:

```javascript src/a.js
console.log('Code Tab A');
```
```javascript src/b.js
console.log('Code Tab B');
```

Save a missing line break and some extra metadata, this looks almost identical to two conventional Markdown code blocks. In fact, if you copied and pasted this snippet into any decent renderer, that’s exactly what you’d see. But render it with the ReadMe engine, and bingo bongo—you get the same, sleek little tabs component as a magic block, without any of the syntactical bloat or baggage.

This gets at the real raison d’être behind the new engine: To modernize ReadMe’s documentation experience and, simultaneously, to seamlessly bridge the gap between our old “magic” format and the evolving expectations of the developer community at large when it comes to authoring our docs.

Docs & Guides→ Today we’re releasing an initial public preview of our new Markdown engine. Eventually we’ll turn it on by default across our platform, but for now you can check out a live preview for any doc in your project. There are likely some kinks and rough edges to iron out, but we’re excited for you to give it a try! And don’t hesitate to drop us a line with any feedback or questions. We’d love to get your take.