Director of Platform Engineering
We at Smartcar want to make it easy and secure for developers to integrate applications with cars. Our API allows apps to communicate with connected vehicles across car brands without the need for any additional hardware. Recently, our team overhauled the internal interface that connects our developer-facing API to each of our car manufacturer (OEM) integrations. Rather than calling the respective JavaScript functions directly, we introduced the Avro protocol. Why, you might ask?
Before switching to Avro, our internal interface posed three fundamental problems that made it hard for us to scale our product:
When trying to solve these three challenges, we decided to tackle the serialization problem first. Soon, we ended up diving head over heels into the world of serialization formats. We spent many a day researching and trying out formats like Google’s Protocol Buffers, Apache Thrift, Apache Avro, MessagePack, and others. Some of them, like MessagePack, function as serialization formats only. Others, like Avro and Protocol Buffers, act as combinations of serialization formats and Remote Procedure Call (RPC) frameworks.
Our guess was that an RPC framework could help us decouple the two sides of the interface and thus solve our second problem as well. This narrowed our choices down to the formats that came with an RPC framework: Avro, Thrift, and Protocol Buffers.
Looking more into those three formats, we found that Avro’s features would solve all three of our challenges and make our product scalable in other ways too:
Once we knew that Avro was the format to go with, it was time to start building!
By “building” I mean implementing the Avro protocol as a copy of Smartcar’s internal interface. Let’s recall the two sides of the internal interface: the developer-facing API and the OEM integrations. As both sides were written in JavaScript, we needed to use an npm library to implement the Avro protocol. By a fluke, we stumbled upon avsc — an npm library that implements Avro in pure JavaScript.
When we started implementing, we found that both creating Avro schema files and following the avsc instructions were relatively easy. We also had to face some challenges:
# Schema for Smartcar's odometer record in Avro's IDL
record OdometerDistance {
// 2,000,000km ~ 1,242,742 miles
@logicalType("range") @min(0) @max(2000000)
float value;
}
Once we had successfully replicated our internal interface in Avro, the last step was to replace our old interface with the new Avro interface.
Did I mention that our internal interface is on the critical path of almost every request to Smartcar? We couldn’t just go on a feature freeze and halt all work until the migration was complete. Thankfully, we were able to use some avsc features alongside some carefully designed shims — small chunks of temporary code that wrapped our old methods and made them compatible with our new interface. All this allowed developers to keep using the Smartcar API, while we integrated and deployed the new interface at the same time.
The work is done! Smartcar now has a brand new internal interface that uses Avro to connect the developer-facing side of our API with our OEM integrations. This new interface allows our team to scale the Smartcar platform more easily. It ensures that we can:
Thank you, Avro and avsc! 🚀
Smartcar is the connected car API that allows mobile and web apps to communicate with connected vehicles across brands (think “check odometer” or “unlock doors.”)
Want to take our API for a spin? Check out our and get started with our ! Have any questions or feedback? Shoot us an !
Smartcar is the easiest way to integrate mobility apps and services with cars.