本文根据flume官网提供的Flume 1.8.0 User Guide整理得出的,主要内容是总结了flume1.8.0支持的各种sink,见下表。
sink名称 | type | 数据目的地 | 工作方式 | 备注 |
HDFS Sink | hdfs | HDFS | ||
Hive Sink | hive | hive | ||
Logger Sink | logger | 自定义 | 日志级别为INFO,主要用于调试 | |
Avro Sink | avro | 外部avro客户端 |
将接收到的events转换成avro events 并发送给指定地址的avro客户端 |
|
Thrift Sink | thrift | 外部thrift客户端 |
将接收到的events转换成thrift events 并发送给指定地址的avro客户端 |
|
IRC Sink | irc | IRC目的地 | ||
File Roll Sink | file_roll | 本地文件系统 | ||
Null Sink | null | 无 | 丢弃所有数据 | |
Hbase Sink | hbase | Hbase | 这两个sink同属一个大类:HbaseSinks | |
AsyncHbaseSink | asynchbase | Hbase(异步模式) | ||
MorphlineSolrSink |
org.apache.flume. sink.solr.morphline. MorphlineSolrSink |
Apache Solr servers |
This sink extracts data from Flume events, transforms it, and loads it in near-real-time into Apache Solr servers, which in turn serve queries to end users or search applications. |
|
ElasticSearchSink |
org.apache.flume. sink.elasticsearch. ElasticSearchSink |
elasticsearch cluster | ||
Kite Dataset Sink |
org.apache.flume. sink.kite.DatasetSink |
Kite Dataset | ||
Kafka Sink |
org.apache.flume. sink.kafka.KafkaSink |
Kafka topic | ||
HTTP Sink |
http |
一个URI地址(POST) | ||
Custom Sink |
自定义FQCN |
自定义 |