Bytewax is a curious stream processing tool that blends a Python surface with a Rust core to produce something that’s in a similar vein to Kafka Streams or Apache Flink, but with a fundamentally different implementation. This week we’re going to take a look at what it does, how it works in theory, and how the marriage of Python and Rust works in practice…
–
The original Naiad Paper: https://dl.acm.org/doi/10.1145/2517349.2522738)
Timely Dataflow: https://github.com/TimelyDataflow/timely-dataflow)
Bytewax the Library: https://github.com/bytewax/bytewax)
Bytewax the Service: https://bytewax.io/)
PyO3, for calling Rust from Python: https://pyo3.rs/v0.21.2/)
Kris on Mastodon: http://mastodon.social/@krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/)
Kris on Twitter: https://twitter.com/krisajenkins)
--#softwaredevelopment #dataengineering #apachekafka #timelydataflow
