Introducing Scribble Generator
Some vendors use an effective sales strategy called “lock-in”. Once you’re in the vendors’ ecosystem, it can be challenging to get out. You’ll have made a big investment into various products that work great together but suck once you use them with anything else.
A similar thing can happen (often unintentionally) with tech stacks. You invest big-time in tool X and are then locked into using it because you don’t want to start from scratch with a new tool Y. We’ve seen this happen for fuzzing and formal verification tools in the Ethereum space. Writing properties and fuzz tests isn’t trivial, and you’ll have to do it for every tool that you want to use.
We think this is too bad because it’s often a good idea to apply multiple tools and techniques to finding bugs.
We’re building Scribble a specification language, and one of our goals is to make it as easy as possible to use fuzzing and formal verification tools. To get closer to that goal, we’ve built a tool for making fuzz test cases more universal!
Scribble Generator
In this post, I want to talk about our first big step in the direction of verification compatibility: Scribble Generator. We’ve taken a look at the formats currently being used for writing fuzz tests. In this space, you’ll see that every tool has its own format. As a result, you’ll regularly only be able to use a single tool to check your properties unless you’re willing to re-write your property tests in several different formats.
Scribble Generator changes that by adding Scribble annotations to fuzz test cases, enabling any Scribble compatible tool to check these properties.
💡 Almost any tool is compatible with Scribble, as it comes with runtime assertion instrumentation out of the box!
An example DSTest fuzz test before Scribbling.
Scribble Generator is the simplest command-line utility that you’ll ever use! You can install it super easy through npm:
npm i -g scribble-generator
Then all you do is go to a directory with smart contracts in there and run:
scribble-generate
It will go through all your smart contracts and look for fuzz test-cases written for either Echidna or Dapptools, and it will add Scribble annotations for each test case!
Are you using another tool for your fuzz cases? Let me know! We’ll try to include it ASAP 🚀.
A DS fuzz test after automatic Scribbling.
Expressiveness
It’s great to re-use fuzz tests between tools, but there is another benefit: You can mix and match different property writing approaches.
We’re working hard on the Scribble specification language, making it super easy, concise and practical to write understandable specifications, and we’ve made great progress towards this goal!
That said, we recognise that while there is a lot of overlap, property writing methods can be complementary. By enabling cross-functionality between tools, we allow you to reach for the right tool for the job when you need it!
Conclusion
While we still have a long way to go, this is a simple but effective first step towards unifying smart contract verification and testing.
So what’s the next step? Well, we’re looking towards current and future verification backends. We’d love to be able to translate to and from (or integrate with) these backends so that you really can use any tool on any specification 🚀.
Interested in collaborating on universifying smart contract verification? Reach out to us!
Are you interested in doing extensive fuzzing on smart contracts? We’re opening up access to our Fuzzing platform! Give us a nudge if you’re interested in early access!