码迷,mamicode.com
首页 > Web开发 > 详细

flume 集群安装监听测试

时间:2020-03-18 23:45:55      阅读:95      评论:0      收藏:0      [点我收藏+]

标签:col   home   basis   exec   ecs   bsp   code   may   under   

1、在已经搭建好集群基础上,配置监听机器配置,主机名 hadoop1,flume-conf.properties配置文件

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#  http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.


# The configuration file needs to define the sources, 
# the channels and the sinks.
# Sources, channels and sinks are defined per agent, 
# in this case called ‘agent‘

agent.sources = avroSource
agent.channels = memoryChannel
agent.sinks = loggerSink

# For each one of the sources, the type is defined
agent.sources.avroSource.type = avro
agent.sources.avroSource.channels = memoryChannel
agent.sources.avroSource.bind = 0.0.0.0
agent.sources.avroSource.port = 1234

# Each channel‘s type is defined.
agent.channels.memoryChannel.type = memory
agent.channels.memoryChannel.capacity = 100

# Each sink‘s type must be defined
agent.sinks.loggerSink.type = logger
agent.sinks.loggerSink.channel = memoryChannel

被监听两台机器配置文件:flume-conf.properties

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#  http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.


# The configuration file needs to define the sources, 
# the channels and the sinks.
# Sources, channels and sinks are defined per agent, 
# in this case called ‘agent‘

agent.sources = execSource
agent.channels = memoryChannel
agent.sinks = avroSink

# For each one of the sources, the type is defined
agent.sources.execSource.type = exec
agent.sources.execSource.channels = memoryChannel
agent.sources.execSource.command = tail -F /home/hadoop/data/flume/logs/test.log

# Each channel‘s type is defined.
agent.channels.memoryChannel.type = memory
agent.channels.memoryChannel.capacity = 100

# Each sink‘s type must be defined
agent.sinks.avroSink.type = avro
agent.sinks.avroSink.channel = memoryChannel
agent.sinks.avroSink.hostname = hadoop1
agent.sinks.avroSink.port = 1234

 技术图片

 

flume 集群安装监听测试

标签:col   home   basis   exec   ecs   bsp   code   may   under   

原文地址:https://www.cnblogs.com/ptbx1t/p/12521145.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!