Of communities and bikesheds - 12 Feb 2018

So, this morning a new Erlang package building tool was announced. I happened to be reading the erlag-questions mailing list (a fairly rare occurrence, as we’ll get into) and I saw the announcement. As soon as I saw the name of the project, I decided to ignore the thread. However, that thread soon re-connected with me via 2 IRC channels, a Slack channel and Twitter. The project’s name? Coon.

Now, having grown up in Ireland, I was unfamiliar with the word, or the racist connotations. Only since moving back to the US have I been introduced to the surprisingly large lexicon of American racism that was not mentioned in ‘To Kill a Mockingbird’ or ‘Huckleberry Finn’. Thus, given that the author didn’t seem to be a native English speaker, and certainly not someone expected to be familiar with derogatory American slang, I expected someone to politely point this out and for the author to realize they’d made a terrible mistake and rename it.

Well, at least the first part happened.

About now is the time to mention why I don’t regularly follow the erlang-questions mailing list anymore. Many years ago, when I was new to Erlang, I was an avid reader of the mailing list. However, over time something changed. I’m not sure if I simply became proficient enough with the language or if the tone of the mailing list changed as the community grew, but I began to lose patience with the threads on naming and API design that would always grow out of all proportion to their importance while deep, technical discussions would often be overshadowed. For the most part this was just annoying, but harmless and I gradually drifted away from paying close attention to it.

Today however, things are a little different. There’s yet another naming discussion, and people are adding their opinions to a dog-pile of a thread faster than you can read the responses, but this time it’s about the accidental use of a racist slur as a project name.

Now, let’s remember, this is a programming language community. These communities are supposed to help practitioners of the language, advocate for its use and generally be a marketing and outreach platform to encourage people to use it. There are a lot of programming languages these days and developer mindshare is valuable, especially for an oddball language like Erlang. And while it is true that communities are not always (or maybe even often) inclusive or welcoming, surely programming communities should be.

Instead the thread (and I confess to having not read the bulk of it) devolved into arguments around intent vs effect and appeals that other problematic project names had flown under the radar in the past. I’m sorry, but this is not how it works. When you create something and release it into the world, you lose control of the interpretation that thing takes on. I’ve seen cases of authors, reviewing their work in a school curriculum where their work is analyzed vehemently disagree with the interpretation of their creation. It’s easy to forget that building things, naming things, etc are as much, if not more, about the effect produced in the consumer of that work as it is about the author’s intent. You don’t get to say “That’s not what I meant” when someone points out a problem with what you’ve done; you need to examine the effect and determine if you feel you should correct it. This is your responsibility as a member of a community and if you’re hurting inclusively or diversity then you are not being a good member of that community.

When I visited ‘coonhub’, the associated website for the tool that lists available packages, I saw one of my own projects prominently featured. Given that I am not a member of a group to which the derisory term applies, I didn’t expect to feel anything, but instead I felt ashamed that I, however indirectly and involuntarily was lending support to this. I can’t imagine what it feels like for someone to whom the slur has been applied, but the faint echo I encountered was unpleasant enough to give me pause.

Long story short, I hope the Erlang community can pull its head out of its ass long enough to realize that bikeshedding about something like this is bordering on the obscene and should shut that shit down. The original author should recognize their mistake, sacrifice their beloved ‘coonfig.json’ pun, rename the project and everyone should move on. A 50 email thread on the matter is ridiculous and is not appropriate.


Announcing caut erl ref; a "new" Cauterize decoder for Erlang - 11 Jan 2017

What?

I just tagged 1.0.0 of caut-erl-ref which is a Cauterize encoder/decoder implementation for Erlang. This isn’t actually a ‘new’ library, it is almost a year old, but it has been in use for most of that time and I finally took the time to clean up some stuff and add some documentation.

“What the heck is Cauterize” I hear you cry, dear reader. Cauterize is yet another serialization format, like msgpack, thrift, protocol buffers, etc. Cauterize, however, is targeted at hard real-time embedded systems. This means that it focuses heavily on things like predictable memory usage, small overhead and simplicity. At Helium we use Cauterize extensively to shuttle our data around, especially on the wireless side, where smaller packets mean less transmit power used and more transmit range (because you can operate at a lower bitrate). Cauterize is an invention of my colleague, John Van Enk, and he’s provided implementations for C and Haskell. Another Helium colleague, Jay Kickliter has a Rust implementation.

John and I, last February at a Helium meetup in Denver, implemented the first versions of the Erlang implementation in about 4 hours. Since then I’ve been tweaking and refining it to better suit my usage. It is a little different than the other implementations, because the Cauterize code generator doesn’t generate an encoder/decoder directly, it generates an abstract representation of the schema and uses a generic library (cauterize.erl) for the encoding/decoding. This probably means it is not the fastest implementation, but it did keep the code generator simple and I’ve mostly focused on making the library very powerful and easy to use.

Features

In addition to being able to (obviously) encode/decode Cauterize, the Erlang implementation has a couple neat features:

Key value coding

The library is compatible with Bob Ippolito’s kvc library, which provides key-value coding for Erlang. This makes it very easy to traverse decoded Cauterize structures, rather than writing complicated pattern matching expressions.

Decode stack traces

When a Cauterize decode fails, erl-caut-ref will show you how far it managed to get before the parsing hit an error. This has been helpful in chasing down some packet corruption issues we’ve seen. This was quite a bit trickier than I expected to implement.

Lots of testing

The library has been in use for almost a year, it has a pretty comprehensive unit test suite and it’s also been checked with Crucible which generates random schemas and random messages based on that schema and checks they can be decoded.

Conclusion

Cauterize is pretty neat, it just gives you a very tiny serialization format. There’s no RPC bullshit, there’s no fancy, brittle pieces, you can probably make it work anywhere (we use it on a bare-metal Cortex M0) and you can probably implement it for your own pet language yourself.


Hot takes on Elixir - 03 Jan 2017

So, Elixir has been a thing for a while now, and on the whole it seems like a great thing. People who get all hung up on Erlang’s syntax have an alternative, we have Hex for Erlang packages in rebar 3 and they’ve come up with some cool syntax like the pipe operator that might make it back into Erlang one day.

However, I do have a bit of a problem with Elixir: people are using my Erlang libraries from Elixir. I’m the author of 2 fairly popular libraries for Erlang; lager for logging and gen_smtp for SMTP. Both have become arguably the de-facto libraries for those tasks in Erlang. Obviously the Elixir community would use that battle tested code in their own ecosystem as well, and they do. This is all fine and well, and I’m very happy my code is making the world a better place. The problems are two fold: support and credit.

I’ve been getting enough Elixir GitHub issues filed that it is getting annoying. Almost always it has to do with incorrectly invoking my Erlang code from Elixir. When it is a legitimate bug I’m stuck trying to understand what the hell the Elixir code actually is doing (I don’t use Elixir and so I’m not very familiar with it). Essentially every time I see a Github email come in and it mentions Elixir, my heart sinks. I’m already neglecting my open source maintainerships (free code doesn’t pay well), and this isn’t helping.

The second issue is credit. Some of the Elixir wrappers for my libraries don’t actually acknowledge they’re wrappers around my code. There’s nothing in the license that requires that, but it feels a bit… icky. Whenever I wrap code, or use some code to derive something, I try to give credit. Open source as a resume booster is a thing (it’s happened to me), but also if you don’t actually know what code you’re using in your project, because the wrapper hid it from you, you have no way to know if a security vulnerability or a bugfix applies to you.

I’m sure people who write Java or .NET libraries see the same problems with Clojure/Scala/F# etc. It is just interesting to see it play out in Erlang land.



Archive