Fluentd Blog

Fluentd v0.12.32 has been released

Hi users!

We have released Fluentd version 0.12.32. Here are the changes:

New features / Enhancement

• formatter: Add add_newline parameter to remove \n from the result https://github.com/fluent/fluentd/pull/1447
• in_tail: Capture unmatched lines https://github.com/fluent/fluentd/pull/1421
• in_monitor_agent: enable to select instance variables of plugins https://github.com/fluent/fluentd/pull/1402
• in_monitor_agent: Port #1387 to v0.12 for retry field https://github.com/fluent/fluentd/pull/1387
• Allow msgpack-ruby v1

Bug fixes

• in_exec: Ensure to sleep for prevending fork bomb https://github.com/fluent/fluentd/pull/1348
• record_transformer: remove_keys processing should be last https://github.com/fluent/fluentd/pull/1433

formatter: Add add_newline parameter to remove '\n' from the result

json, ltsv, csv and hash formatters now support add_newline parameter. If you set add_newline false, the formatted result doesn't contain \n. This is useful for Queue / KVS plugins.

in_tail: Capture unmatched lines

v0.12.32 introduces emit_unmatched_lines parameter to in_tail plugin. This option enables you to get unparseable line as an event.

For example, if you have following configuration and file:

• fluent.conf

<source>
  @type tail
  path /path/to/json.log
  format json
  emit_unmatched_lines true
  # ...
</source>

• json.log

{"message":"Hello!"}
This is broken line!
{"message":"Yay :)"}

Then, the following events are generated from this tail plugin.

{"message":"Hello!"}
{"unmatched_line":"This is broken line!"}
{" message":"Yay :)"}

Unmatched line is assigned to unmatched_line field.

This option works with both single-line and multi-lines.

in_monitor_agent: Add retry field

Fluentd v0.14 adds retry field to /api/plugins.json response. The retry field contains detailed information of buffer's retry. For API consistency, v0.12's in_monitor_agent also provides same field. Here is retry field example:

{
  // other monitor_agent keys
  "retry": {
    "steps":1, // same as retry_count
    "next_time":"2016-12-23 01:37:16 +0700"
  }
}

Unlike v0.14, start field doesn't exist because v0.12's buffer doesn't have its information.

We recommend to use retry's steps instead of retry_count for monitoring.

Lastly, v0.12.32 docker image has also been available on Docker Hub.

Happy logging!