spark-streaming
2017-10-30 本文已影响8人
焉知非鱼
.set("spark.streaming.receiver.writeAheadLog.enable", "true") //
To ensure zero data loss, enable the Write Ahead Logs (introduced in Spark 1.2). This synchronously saves all the received Kafka data into write ahead logs on a distributed file system (e.g HDFS), so that all the data can be recovered on failure. See Deploying section in the streaming programming guide for more details on Write Ahead Logs.
