Fluentd Blog

Fluentd v0.12.31 has been released

Hi users!

We have released Fluentd version 0.12.31. Here are the changes:

New features / Enhancement

• output: Add slow_flush_log_threshold parameter: #1366
• formatter_csv: Change fields parameter to required. Now accepts both a,b and ["a", "b"]: #1361
• in_syslog: Add priority_key and facility_key parameters: #1351

Add slow_flush_log_threshold parameter

We introduce slow_flush_log_threshold parameter to investigate flush performance issue. Fluentd users sometimes hit BufferQueueLimitError without errors. In such case, users hard to find the cause, traffic is high or output destination becomes slow?

With this parameter, users can know output destination has a problem or not. If you see following message in the fluentd log, your output destination or network has a problem and it causes slow chunk flush.

2016-12-19 12:00:00 +0000 [warn]: buffer flush took longer time than slow_flush_log_threshold: elapsed_time=15.0031226690043695 slow_flush_log_threshold=10.0 plugin_id="es_output"

In this example, slow_flush_log_threshold is 10.0 but chunk flush takes 15 seconds.

<match app.**>
  @type elasticsearch
  @id es_output
  slow_flush_log_threshold 10.0
  # ...
</filter>

The default value is 20.0. If your buffer chunk is small and network latency is low, set smaller value for better monitoring.

Lastly, v0.12.31 docker image has also been available on Docker Hub.

Happy logging!