Batch processing is dead? Maybe not, but streaming is king for our use case.
We process 1M+ events/sec using Kafka + Flink on Kubernetes.
Architecture: Producer -> Kafka Topic (Partitioned) -> Flink Job -> ClickHouse.
Latency is under 200ms end-to-end.