Skip to main content


Showing posts from 2016


Kafka MirrorMaker in Kafka

Check MirrorMaker.scala for more details.
Target cluster setupDownload and install Kafka (target cluster). Select appropriate version and download its tgz from Kafka Downloads page.tar -zxf kafka_2.11- cd kafka_2.11- Configure Target Kafka cluster's ZooKeepervim ./config/ # the directory where the snapshot is stored. dataDir=/work/kafka_2.11- # the port at which the clients will connect clientPort=2181 # disable the per-ip limit on thseparatedof connections since this is a non-production config maxClientCnxns=0 Start Target Kafka cluster's ZooKeeper./bin/ config/ Configure Target Kafka cluster's Servervim ./config/ # The id of the broker. This must be set to a unique integer for each broker. # The number of threads handling network requests # The number of threads doing disk I/O # The send buffer (SO_…

Kafka MirrorMaker in Kafka

ElasticSearch pipeline bucket selector aggregation