Announcing caut erl ref; a "new" Cauterize decoder for Erlang - 11 Jan 2017


I just tagged 1.0.0 of caut-erl-ref which is a Cauterize encoder/decoder implementation for Erlang. This isn’t actually a ‘new’ library, it is almost a year old, but it has been in use for most of that time and I finally took the time to clean up some stuff and add some documentation.

“What the heck is Cauterize” I hear you cry, dear reader. Cauterize is yet another serialization format, like msgpack, thrift, protocol buffers, etc. Cauterize, however, is targeted at hard real-time embedded systems. This means that it focuses heavily on things like predictable memory usage, small overhead and simplicity. At Helium we use Cauterize extensively to shuttle our data around, especially on the wireless side, where smaller packets mean less transmit power used and more transmit range (because you can operate at a lower bitrate). Cauterize is an invention of my colleague, John Van Enk, and he’s provided implementations for C and Haskell. Another Helium colleague, Jay Kickliter has a Rust implementation.

John and I, last February at a Helium meetup in Denver, implemented the first versions of the Erlang implementation in about 4 hours. Since then I’ve been tweaking and refining it to better suit my usage. It is a little different than the other implementations, because the Cauterize code generator doesn’t generate an encoder/decoder directly, it generates an abstract representation of the schema and uses a generic library (cauterize.erl) for the encoding/decoding. This probably means it is not the fastest implementation, but it did keep the code generator simple and I’ve mostly focused on making the library very powerful and easy to use.


In addition to being able to (obviously) encode/decode Cauterize, the Erlang implementation has a couple neat features:

Key value coding

The library is compatible with Bob Ippolito’s kvc library, which provides key-value coding for Erlang. This makes it very easy to traverse decoded Cauterize structures, rather than writing complicated pattern matching expressions.

Decode stack traces

When a Cauterize decode fails, erl-caut-ref will show you how far it managed to get before the parsing hit an error. This has been helpful in chasing down some packet corruption issues we’ve seen. This was quite a bit trickier than I expected to implement.

Lots of testing

The library has been in use for almost a year, it has a pretty comprehensive unit test suite and it’s also been checked with Crucible which generates random schemas and random messages based on that schema and checks they can be decoded.


Cauterize is pretty neat, it just gives you a very tiny serialization format. There’s no RPC bullshit, there’s no fancy, brittle pieces, you can probably make it work anywhere (we use it on a bare-metal Cortex M0) and you can probably implement it for your own pet language yourself.

Hot takes on Elixir - 03 Jan 2017

So, Elixir has been a thing for a while now, and on the whole it seems like a great thing. People who get all hung up on Erlang’s syntax have an alternative, we have Hex for Erlang packages in rebar 3 and they’ve come up with some cool syntax like the pipe operator that might make it back into Erlang one day.

However, I do have a bit of a problem with Elixir: people are using my Erlang libraries from Elixir. I’m the author of 2 fairly popular libraries for Erlang; lager for logging and gen_smtp for SMTP. Both have become arguably the de-facto libraries for those tasks in Erlang. Obviously the Elixir community would use that battle tested code in their own ecosystem as well, and they do. This is all fine and well, and I’m very happy my code is making the world a better place. The problems are two fold: support and credit.

I’ve been getting enough Elixir GitHub issues filed that it is getting annoying. Almost always it has to do with incorrectly invoking my Erlang code from Elixir. When it is a legitimate bug I’m stuck trying to understand what the hell the Elixir code actually is doing (I don’t use Elixir and so I’m not very familiar with it). Essentially every time I see a Github email come in and it mentions Elixir, my heart sinks. I’m already neglecting my open source maintainerships (free code doesn’t pay well), and this isn’t helping.

The second issue is credit. Some of the Elixir wrappers for my libraries don’t actually acknowledge they’re wrappers around my code. There’s nothing in the license that requires that, but it feels a bit… icky. Whenever I wrap code, or use some code to derive something, I try to give credit. Open source as a resume booster is a thing (it’s happened to me), but also if you don’t actually know what code you’re using in your project, because the wrapper hid it from you, you have no way to know if a security vulnerability or a bugfix applies to you.

I’m sure people who write Java or .NET libraries see the same problems with Clojure/Scala/F# etc. It is just interesting to see it play out in Erlang land.

Encryption: you can't put the Genie back into the bottle - 14 Dec 2015

I’ve been hearing a lot of noise in the media about the strong encryption on ‘social media’ and on phones has been shielding and enabling terrorists and criminals to communicate securely. As someone who has at least some familiarity with encryption and security (although I am by no means an expert), this really sounds like a lot of nonsense.

Leaving aside the politics of it all, and focusing on this as a purely technical issue. The simple fact of the matter is that all the legislation and pressure on tech companies in the world isn’t going to put the encryption genie back in the bottle.

We now have, between Diffie-Hellman, RSA and Elliptic Curve Cryptography (ECC) (as well as the new crop of ciphers like AES-GCM and ChaCha/Salsa20) a pretty formidable set of tools for doing strong cryptography. There’s also a pretty wide array of hardware based key storage like YubiKeys, trust stores built into CPUs, etc.

Putting all this together with the wide array of places on the internet you can use as a ‘dead drop’ for publishing messages, governments and ‘experts’ can call for banning strong encryption all they want, but even if they succeeded in rolling back some of the recent advances in cryptography in consumer devices and services, there’s nothing stopping people from using one of the many open source libraries like libsodium, LibreSSL, OpenSSL or GNUTLS of trivially rolling their own.

This whole proposal of regulating strong encryption is basically a flawed idea. We’ve known how to do strong encryption for ~40 years and, simply by bumping the key size, even those venerable systems are pretty hard to break. The more modern, and freely available, stuff is (probably) even harder to break.

If someone dropped me on a desert island and told me to build a ‘secure’, end-to-end encryption scheme, using only the software installed on my laptop right now, I suspect I could design a system that would be pretty tough to detect, let alone break. It isn’t that hard to put the cryptographic building blocks together (heck, that is the whole point of the aforementioned libraries) and build something like Apple’s iMessage encryption, or go even further and use file hosting or image sharing websites to publish public keys and encrypted messages. Nobody sane invents their own cryptography, they all use the same well-scrutinized building blocks (although some blocks are better than others).

Now, one could argue for the idea of ‘key escrow’, where every good citizen shares their private key with the government, who stores it securely until a warrant to intercept their secure communications is signed by a judge, etc. Leaving aside the sheer administrative overhead of that scheme, what is to stop me generating some other key to use, or using someone else’s ‘unofficial’ key. It’s madness, akin to when they tried to ban the DVD decryption key (which was just a big number). You can’t really regulate information on the internet, people will always find a way. Furthermore, what about things like Ephemeral Diffie-Hellman, where the keys used are thrown away as soon as they serve their purpose, are we going to ban that too (because even nobody involved in that communication can decrypt it afterwards)? I will also leave aside the whole notion of trusting a government to keep sensitive information secure, and to resist the temptation to use that information without a warrant and due process.

In fact, this while obsession with centralizing the internet to make it easier to monitor and record is actually harming the robustness of the internet. I’ve heard people simultaneously decry the use of strong encryption while also prophesying doom due to ‘cyber attacks’. You can’t have it both ways. If you weaken the security and the decentralization of the internet to increase your surveillance capabilities, you also make yourself a more tempting target for the dreaded ‘cyber warfare’.

Until recently we were living in a golden age of surveillance, while strong crypto was available, it wasn’t widely considered a requirement, nor was it particularly easy to use. Things have changed, and the people watching us no longer know what we are saying and doing. This is not, historically speaking, a big change, but rather a return to the norm. I understand that this is hard for the people who have grown used to being able to see into people’s lives but, for better or for worse, that time is ending and new strategies will have to be developed to respond to that fact.

When politicians or pundits call on ‘silicon valley’ or ‘tech companies’ to ‘disrupt’ terrorists or criminals, or they ask for a discussion about ‘golden keys’ they are asking the wrong questions and are merely betraying a way of thinking that no longer applies to the modern internet or technology.

TL;DR - you can ban strong crypto all you want, but in doing so you’re not going to prevent anyone who really cares about secure communication from using it, you’re just validating the need for it.