双层flume搭建过程中遇到的坑

2020-06-27  本文已影响0人  bigdata张凯翔
INFO - Creating /user/hive/warehouse/ods.db/origin_user_behavior/19-04-19/events.1555671386480.tmp
ERROR - Avro source r2: Unable to process event batch. Exception follows.
org.apache.flume.ChannelFullException: Space for commit to queue couldn't be acquired. Sinks are likely not keeping up with sources, or the buffer size is 

解决方法:
先将flume的JVM内存改为512M

capacity = 1000
transactionCapacity = 1000
keep-alive = 30
ERROR - Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to send events
    at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:389)
    at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
    at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoop104, port: 4545 }: Failed to send batch
    at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:314)
    at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:373)
    ... 3 more
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoop104, port: 4545 }: Avro RPC call returned Status: FAILED
    at org.apache.flume.api.NettyAvroRpcClient.waitForStatusOK(NettyAvroRpcClient.java:390)
    at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:373)
    at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:302)
    ... 4 more
image.png

请求超时,导致发送event失败
解决:
设置request-timeout长一点,默认20秒

上一篇下一篇

猜你喜欢

热点阅读