解决：flink 日志不打印 flink打印中文日志出现乱码

flink版本 1.14
安装使用CDH 6.3

解决问题1：flink 日志不打印 flink启动日志 flink web ui日志 均不打印
解决问题2：flink打印中文日志出现乱码 中文日志变成了 ？？？？
解决问题3：flink打印日志输出到kafka

flink启动日志 flink web ui日志 均不打印

fix1: ${FLINK_HOME}/lib 项目jar包移除

查看日志：
SLF4J: Found binding in [jar:file:****/filecache/21/flink-spark.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file::****/application_1686032130046_6235/filecache/21/flink-rongshu.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file::****/application_1686032130046_6235/filecache/21/flink-stream.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file::****/application_1686032130046_6235/filecache/11/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:****/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]

发现lib下有大量项目相关jar包 每个jar包都包含log4j的依赖

移除${FLINK_HOME}/lib 下的项目jar包到其他路径

fix2: 排除项目代码中log4j1依赖

发现项目依赖的log4j1，${FLINK_HOME}/lib下代码为log4j2

    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>1.7.25</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
        <version>1.7.25</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-to-slf4j</artifactId>
        <version>2.14.0</version>
        <scope>provided</scope>
    </dependency>

项目包内的日志相关jar包全部排除 使用${FLINK_HOME}/lib目录下自带的log4j2 jar包做日志打印
问题解决 出现日志

中文日志出现乱码

fix1: 新增 flink.yaml 指定UTF-8

env.java.opts: "-Dfile.encoding=UTF-8"
web.character.encoding: UTF-8
问题未解决

fix2: 新增 提交脚本 指定jvm编码

-Dfile.encoding=UTF-8
-Dsun.jnu.encoding=UTF-8
问题未解决

fix3: 新增 ${FLINK_HOME}/conf/log4j.properties

log4j.appender.YourAppender.Encoding=UTF-8
问题未解决

fix4: 新增 ${FLINK_HOME}/conf/log4j.properties

appender.main.layout.charset = UTF-8
问题解决

虽然flink日志配置叫log4j.properties 但实际上使用的log4j2 所以指定编码需要使用log4j2的语法

最终日志

# Allows this configuration to be modified at runtime. The file will be checked every 30 seconds.
monitorInterval=30

# This affects logging for both user code and Flink
rootLogger.level = INFO
rootLogger.appenderRef.file.ref = MainAppender


# Uncomment this if you want to _only_ change Flink's logging
#logger.flink.name = org.apache.flink
#logger.flink.level = INFO

# The following lines keep the log level of common libraries/connectors on
# log level INFO. The root logger does not override this. You have to manually
# change the log levels here.
logger.akka.name = akka
logger.akka.level = INFO
logger.kafka.name= org.apache.kafka
logger.kafka.level = INFO
logger.hadoop.name = org.apache.hadoop
logger.hadoop.level = INFO
logger.zookeeper.name = org.apache.zookeeper
logger.zookeeper.level = INFO
logger.shaded_zookeeper.name = org.apache.flink.shaded.zookeeper3
logger.shaded_zookeeper.level = INFO

# Log all infos in the given file
appender.main.name = MainAppender
appender.main.type = RollingFile
appender.main.append = true
appender.main.fileName = ${sys:log.file}
appender.main.filePattern = ${sys:log.file}.%i
appender.main.layout.type = PatternLayout
# appender.main.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} [] %-5p %-60c %x - %m%n
appender.main.layout.pattern = %-4r [%t] %-5p[%d{yyyy-MM-dd HH:mm:ss}][%c] %x-%m%n
appender.main.policies.type = Policies
appender.main.policies.size.type = SizeBasedTriggeringPolicy
appender.main.policies.size.size = 100MB
appender.main.policies.startup.type = OnStartupTriggeringPolicy
appender.main.strategy.type = DefaultRolloverStrategy
appender.main.strategy.max = ${env:MAX_LOG_FILE_NUMBER:-10}
# 2023.07.13 add encoding
appender.main.layout.charset = UTF-8


# 新增 Kafka appender
rootLogger.appenderRef.kafka.ref=kafka

appender.kafka.type = Kafka
appender.kafka.name = kafka
appender.kafka.topic = FLINK_APPLICATION_LOG

appender.kafka.property.type=Property
appender.kafka.property.name=bootstrap.servers
appender.kafka.property.value=kafka:9092
appender.kafka.layout.charset = UTF-8

appender.kafka.layout.type = PatternLayout
appender.kafka.layout.pattern = %d{yyyy-MM-dd HH:mm:ss}@|$%p@|$[%c{3}]:%L@|$${flink.job.name}@|$%m%n


# Suppress the irrelevant (wrong) warnings from the Netty channel handler
logger.netty.name = org.jboss.netty.channel.DefaultChannelPipeline
logger.netty.level = OFF