-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #42 from rmoff/joining-decodable
New blog: Joining Decodable
- Loading branch information
Showing
12 changed files
with
58 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
--- | ||
draft: false | ||
title: 'An Itch That Just Has to Be Scratched… (Or, Why Am I Joining Decodable?)' | ||
date: "2023-09-21T14:25:39Z" | ||
image: "/images/2023/09/h_IMG_5397.jpeg" | ||
thumbnail: "/images/2023/09/t_IMG_8746.jpeg" | ||
credit: "https://twitter.com/rmoff/" | ||
categories: | ||
- Streaming | ||
- Career | ||
- DevEx | ||
--- | ||
|
||
This week I joined [Decodable](https://decodable.co). I'm grateful to my former colleagues at Treeverse for allowing me to [join them](/2022/12/09/looking-forwards-and-looking-backwards/) on the journey with [lakeFS](https://lakefs.io) - but something about the streaming world was too strong to resist 😁. | ||
|
||
<!--more--> | ||
|
||
I spent several years previously at Confluent bringing my data engineer's view of the world to help [advocate](https://rmoff.net/categories/kafka-connect/) [for](http://youtube.com/rmoff) what was being built within Confluent as [ksqlDB](https://ksqldb.io/)—and the broader Apache Kafka community in [Kafka Connect](https://kafka.apache.org/documentation.html#connect)—as a coherent platform on which analytics and data integration solutions could be built. | ||
|
||
|
||
<video autoplay="true" loop="true" width=800 src="/images/2023/09/just-when-i-thought-i-was-out.mp4">Just when I thought I was out / They Pull Me Back In - Al Pacino - The Godfather</video> | ||
|
||
Turns out, stream processing was only really just getting started. In the last 12-18 months a veritable plethora of stream processing projects and companies have shown up, demonstrating that there is not only demand for it but also different approaches to take too. | ||
|
||
The advent of real-time data stores such as [Apache Druid](https://druid.apache.org/), [Apache Pinot](https://pinot.apache.org/), and [Clickhouse](https://clickhouse.com/) further complicates the picture (or makes it more interesting, depending on your view). How much do you serve from your operational application, your streams, or your dedicated realtime store? What's driving the need for the data, and what kind of access patterns do you have? | ||
|
||
In short, I have unfinished business with streaming. | ||
|
||
<video autoplay="true" loop="true" height=400 src="/images/2023/09/im.mp4">I Am Not Finished! - Yara Greyjoy - Game of Thrones</video> | ||
|
||
Unsuprising really, since event streams _are_ unbounded 🤓 `<groan/>`. | ||
|
||
|
||
## A.B.L. (Always Be Learning) | ||
|
||
One of the many things that excites me about joining Decodable is the opportunity to dive in and learn [Apache Flink](https://flink.apache.org/). I'm planning to blog my journey with it (similar to how I did as [a complete n00b going into Go](https://rmoff.net/2020/06/25/learning-golang-some-rough-notes-s01e00/)), so if you have any burning questions about it already then make sure to [drop me](https://twitter.com/rmoff/) [a line](https://www.linkedin.com/in/robinmoffatt) and I'll be happy to use it as an excuse to go and find out the answer. And find out the answer I shall, because one of the *other* many reasons that I'm so excited about Decodable is my colleagues here: | ||
|
||
## Never Be The Smartest in the Room | ||
|
||
_Well, there's never been any danger of that at any of my previous gigs—I've been fortunate to work with some excellent people—and the same holds true for Decodable_. | ||
|
||
One of the things that really sealed the deal for me during the interview process was just how damn smart the folk were. Not in a "*I ask you stupidly pointless question to highlight how smart I am, you wriggle for an answer that I'll not really listen to, rinse & repeat*" kind of way. But in a focussed and probing and inquisitive way, with true dialogue and listening. That's the kind of stuff you can't LeetCode for, and is a hugely positive indicator. | ||
|
||
As well as being lovely people to chat to and sharp as a sharp thing that's just been sharpened, there's some extensive and deep knowledge about the stream processing world in general and the projects which Decodable uses. These include folk such as [Eric Sammer](https://www.linkedin.com/in/esammer/), [Sharon Xie](https://www.linkedin.com/in/sharonxr/), Flink PMC Chair [Robert Metzger](https://home.apache.org/phonebook.html?uid=rmetzger), and former lead on the [Debezium](https://debezium.io/) project [Gunnar Morling](https://www.morling.dev/). | ||
|
||
## What Will I Be Doing at Decodable? | ||
|
||
Who knows?! That's the fun part of joining a 30-person startup 😉. | ||
|
||
![Conference badge that reads "Robin Moffatt: Principal Shitposter and Meme Artist"](/images/2023/09/IMG_5699.jpeg) | ||
|
||
Srsly tho: my starting point for this answer is my blog from earlier this year [_What Does This DevEx Engineer Do?_](/2023/05/23/what-does-this-devex-engineer-do/). But very shortly after that my answer is: getting stuck in and helping build a really awesome streaming platform. Friction logs, blogs, <del>shitposting</del>, community, docs, making videos…whatever needs doing to make the experience for developers the very best 🏆. | ||
|
||
## Stream Processing - We've Only Just Got Started | ||
|
||
So, time for me to get back into stream processing land. I'll save you the fluff and gumpf about the business value of stream processing, the use cases, all the evils and perils of batch, yada yada. If you know, you know. And if you don't, well, join me for the ride 🙃. | ||
|
||
<video autoplay="true" loop="true" height=300 src="/images/2023/09/saddle-up-partner-dumb-and-dumber.mp4">Saddle up, partner! Jim Carrey - Dumb & Dumber</video> |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.