storm trident 一个batch多大

标签：related   example   kafka   ota   ide   ber   而且   部分   sum   

You can increase the batch size by changing "tridentKafkaConfig.fetchSizeBytes" property.

Also, batch size is related with number of brokers and number of partitions.

For example, if you have 2 brokers and 3 partitions for each broker that means the total count of partition is 6.

y this way, the batch size equals to tridentKafkaConfig.fetchSizeBytes X total partition count.

if we assume that the tridentKafkaConfig.fetchSizeBytes is 1024X1024, the batch size equals to 6 MB.(3x2x1024x1024)bytes

相当于从kafka里面是批量读取的，然后形成storm中的一个batch
而且spout里面有一个buffer，应该也算是一个缓存，将正在处理的一部分数据，存在内存中，直到处理完

storm trident 一个batch多大

标签：related   example   kafka   ota   ide   ber   而且   部分   sum   

原文地址：https://www.cnblogs.com/brainstorm/p/9094797.html