Fluentd Blog

Fluentd v0.14.13 has been released

Hi users!

We have released v0.14.13 with several bug fixes and new utilities.

Here are the changes (full ChangeLog is available here):

We say again, fluentd v0.14 is still development version. We need your feedback seriously! If you try to use v0.14, check your configuration and plugins carefully.

Add 'fluent-plugin-generate' command

fluent-plugin-generate helps you to create new plugin for v0.14. It generates skelton code and rubygems files. Here is the step for new plugin release:

# Create plugin files. generate command takes two arguments, plugin type and name
% fluent-plugin-generate output test
License: Apache-2.0
create Gemfile
create README.md
create Rakefile
create fluent-plugin-test.gemspec
create lib/fluent/plugin/out_test.rb
create test/helper.rb
create test/plugin/test_out_test.rb
Initialized empty Git repository in /Users/repeatedly/tmp/fluent-plugin-test/.git/

# Set up develop environment
% gem install bundler # if you don't install bundler yet
% edit fluent-plugin-test.gemspec # need to update summary, description and homepage to use bundle
% bundle install --path vendor/bundle
% bundle exec rake test # check running test

# edit plugin code and test
% edit lib/fluent/plugin/out_test.rb
% edit test/plugin/test_out_test.rb

# Update README and publish gem
% edit README.md
% bundle exec rake release

Currently, our plugin development article for v0.14 is incomplete. We will write new article with v0.14 API soon.

Skip record with Output#format API

In v0.12, Output plugin can skip invalid / unexpected record by using Fluent::BufferedOutput#format_stream API. But v0.14 API doesn't provide it so hard to migrate v0.12 plugin into v0.14.

To avoid this problem, Fluent::Plugin::Output#format API can return nil to skip the record. Here is an example:

helpers :event_emitter # need this code to use router in Output plugin

def format(tag, time, record)
  if record['field'] == 'foo'
    record.to_json + "\n"
  else
    router.emit_error_event(tag, time, record, ArgumentError.new("received event is invalid in xxx plugin"))
    nil
  end
end

With above code, if the value of field is 'foo', its events are stored into buffer. On the other hand, other events are routed to error stream and sucn events are not stored in buffer.

Major bug fixes

• output: Secondary calculation should consider retry_max_times
• parser: Fix missing parser_regex require
• in_tail: Untracked files should be removed from watching list to avoid memory bloat
• record_transformer: Revert "Use BasicObject for cleanroom" for enable_ruby regression.

Enjoy logging!