Custom JavaScript parser vs Jison – Our experience

We recently announced QuickDBD, a simple product we made for drawing database diagrams by typing. If you take a look at the QuickDBD app you’ll see it converts source code into a diagram. What we needed to make this work was obviously a parser.

After a bit of research on how to approach this problem, we knew that we would have to use either an existing parser generator or build a custom parser ourselves. After narrowing the choices down a bit, PEG.js and Jison emerged as the two most popular JavaScript parser generators at the moment. Out of these two, Jison seemed to have slightly bigger community – a bit more GitHub followers, more StackOverflow questions and a slightly better documentation. It seemed like a better bet so we decided to spend a bit of time playing with it and to try to make it parse the QuickDBD syntax.

We managed to make it parse the first version of our syntax we had a few months back pretty fast. But since the language we came up with for QuickDBD is closer to a data description language than what most people would consider a programming language, we started hitting bumps in the road pretty quickly as well. We soon ended up having to handle multiple edge cases we weren’t able to with just Jison and what that meant was overriding Jison behaviour and injecting custom bits of JavaScript into it.

That kind of felt pretty messy so we talked a bit about it and made a decision to go with our own custom JavaScript parser for several reasons:

• we would have complete control over how the parser works
• everyone here is very well versed in JS
• Jison was new to everyone and there is a bit of a learning curve in being able to do stuff with it efficiently
• it felt as if we were fighting Jison to make it work something it wasn’t supposed to more than it felt it was this great tool that was would empower us to do things better and faster
• a couple of times it was pretty hard to get information on how to do something with Jison so we had to fall back to reading it’s source code to figure things out
• it didn’t feel like the right tool for the job

We however did pick up some ideas from trying it out and I believe it made the custom parser we came up with that much better. We wrote a parser that’s fairly small, fast and easy to read, expand and fix – which is ultimately what we needed.

I still think Jison is a great tool but it just wasn’t a very good fit four for our needs. If you’re considering using it, perhaps try it out on a smaller subset of features of your language first and see how you like it before committing to it. You can always go back to writing something custom after you tried it out.

I also recommend you read this very good parser generators vs custom parsers SO thread with pros and cons for both sides.

Hope this helped!