Introduction The Golang package Shopify/sarama introduced transactional API for Kafka in release v1.37.2. These implement the Kafka transactional API and enable developing more consistent workflows in our applications. In short, if you are a sarama user you can now leverage concepts like idempotent producers, satisfy the exactly-once practice for your consumers and generally improve how you handle producing a batch of messages synchronously with no headaches. I work in the Development Experience organisation at Cloudflare and, between other things I do, I maintain a package that utilises sarama to interact with Kafka....
Introduction Recently I’ve been working on a new project, and I’ve decided to build the backend in Go leveraging a microservice architecture. I’ve also decided to use GRPC as transport for service-to-service synchronous communication as well as Kafka for asynchronous communication. GRPC leverages protocol buffers (known as protobuf) to compile API contracts to a variety of languages and easily generate stubs for clients and servers. Protobuf enables writing strongly typed contracts defining how to serialise and deserialize bytes into a given struct or class....