fluent-package v5.0.3 has been released

Hi users!

We have released fluent-package v5.0.3. fluent-package is a stable distribution package of Fluentd. (successor of td-agent v4)

This is a maintenance release of v5.0.x LTS series. As significant slow starting service and crash issues during startup on Windows were fixed, we recommend upgrading to fluent-package v5.0.3!

Changes from fluent-package v5.0.2

  • Update fluentd to 1.16.5. See Fluentd v1.16.5 has been released blog article about details.
  • Update bundled plugins
    • e.g. fluent-diagtool v1.0.5. It supports to collect list of plugins on Windows.
  • msi: fixed wrong environment path for Fluent Package Prompt (#606)
    • It breaks fluent-diagtool behavior to launch fluent-gem correctly.
  • msi: removed unnecessary path delimiter (#607)
    • It doesn't cause any problem yet, but it should treat %~dp0 correctly.
  • rpm: fixed to take over enabled state of systemd service from td-agent v4 (#613)
  • deb rpm: fixed to quote target files correctly not to cause migration failures (#615)
  • msi: added a patch for RubyInstaller to avoid crash on start up (#620)
  • msi: fixed slow start issue on Windows (#631)

About next LTS schedule

We plan to ship next LTS version of fluent-package v5.0.4 on June, 2024. The content of updates are still in T.B.D.

About td-agent v4.5.2 and v4.5.3 (Windows)

As it was already announced Drop schedule announcement about EOL of Treasure Agent (td-agent) 4, td-agent v4 was reached EOL in Dec, 2023.

There is a exceptional maintenance release for v4.5.3 on Windows because there was a crash bug during startup on Windows. It was backported fix from fluent-package v5 as it is critical in some case.

We strongly recommend migrating from td-agent v4 to fluent-package v5 (LTS). See Upgrade to fluent-package v5

Download

Please see the download page.

Read More

Fluentd v1.16.5 has been released

Hi users!

We have released v1.16.5 on 2024-03-27. ChangeLog is here.

The previous version v1.16.4 has a serious Buffer bug when it processes large data exceeding chunk size limit. So, we have released v1.16.5 urgently, and have fixed the bug. So, please don't use v1.16.4.

Sorry for troubling.

Bug Fixes

Buffer: Fix emit error of v1.16.4 sometimes failing to process large data exceeding chunk size limit

In the previous version v1.16.4, we fixed a Buffer bug (Please see Fluentd v1.16.4 has been released for details).

There was a race condition problem with that fix, and similar Buffer errors explained in Fluentd v1.16.4 has been released could occur.

  • emit transaction failed: ...
  • send an error event stream to @ERROR: ...

These errors are the same as the Buffer error of Fluentd v1.16.4 has been released, but it would be more likely to happen. The cause is a race condition when processing large data exceeding chunk size limit. So, in that case, these errors can occur depending on the timing.

So, please don't use v1.16.4.

See #4447 for more information.

Sorry for troubling.

Read More

Fluentd v1.16.4 has been released

Hi users!

We have released v1.16.4 on 2024-03-14. ChangeLog is here.

In this release, we fixed several bugs and improved performance.

Bug Fixes

Buffer: Fix emit error sometimes caused by failing to process large data exceeding chunk size limit

In previous versions, when Buffer processes large data exceeding the chunk size limit, it sometimes fails and causes an emit error.

It is usually unlikely, but it may occur when Buffer receives data with extremely large records that are distributed unevenly in the data.

If you set @ERROR label, the data is routed to that label, and Fluentd outputs a warning log message: send an error event stream to @ERROR: ....

However, if you don't set @ERROR label, the data is discarded, and Fluentd outputs a warning log message: emit transaction failed: .... In addition, Input plugins that do not consider an emit error may stop working, such as in_windows_eventlog2.

Here is an example of the warning message:

2024-03-22 14:13:35 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="closed stream" location="/path/to/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:82:in `pos'" tag="test"

If you see these warnings, please update Fluentd.

See #4342 for more information.

Others

  • in_tail: Fix tail watchers in rotate_wait state not being managed. #4334
    • This problem should not have any actual negative impact on the operation.
  • Buffer: Avoid unnecessary log processing. It will improve performance. #4331

Enjoy logging!

Read More

Open Source Summit Japan 2023

Hi users and developers!

We would like to share the information about an event appearance of Fluentd maintainers.

Daijiro Fukuda and Kentaro Hayashi, Fluentd Maintainers from ClearCode Inc., are going to talk at the Open Source Summit Japan on December 5th

The event has been taken thousands of attendees all over the world and one of the leading Open Source event in Japan. We are honored to take this opportunity and very excited to share about Fluentd project at the conference!

Everyone attending the event, please join us!

About the Talk

Date: December 5th 14:00-14:40
Speakers: Daijiro Fukuda and Kentaro Hayashi from ClearCode Inc.
Summary:
Fluentd, the CNCF graduated project, starts a new chapter as the OSS project. Fluentd community released the new package fluent-package as the successor of td-agent and started providing a new release cycle to fit better with needs. This talk is going to share the latest news of Fluentd, including its updates, package, rebranding and so on, with people who are using Fluentd or struggling to manage their logs. Fluentd is an open source data collector that unifies data collection and consumption. Because of its flexible plugin system and scalability, Fluentd is adopted in various systems and has become a de facto standard, especially for scale logging in distributed systems.

In the new release cycle, the package applies only bug fixes and security fixes of Fluentd over a long term, about two years. We are confident that this will meet the demands of users who want to use Fluentd stably over the long term. In this session, we will talk about (a) the history of Fluentd v1 and td-agent v4, (b) why Fluentd community decided to rebrand existing td-agent to fluent-package, (c) what the benefits and impacts of the new release cycle are, and (d) how to upgrade to fluent-package from td-agent.

Detail info: Link to event page

Overall Event Information

Name: Open Source Summit Japan 2023
Date: 2023-12-04 to 2023-12-06
Host: The Linux Foundation
Description:

Open Source Summit Japan is the leading conference in Japan connecting the open source ecosystem under one roof, providing a forum for technologists and open source industry leaders to collaborate and share information, learn about the latest in open source technologies and find out how to gain a competitive advantage by using innovative open solutions.

Open Source Summit is a conference umbrella, composed of a collection of events covering the most important technologies, topics, and issues affecting open source today.

Quote from the event web page : https://events.linuxfoundation.org/open-source-summit-japan/

Read More

fluent-package v5.0.2 has been released

Hi users!

We have released fluent-package v5.0.2 and td-agent v4.5.2. fluent-package is a stable distribution package of Fluentd.

This is a maintenance release of v5.0.x LTS series. As significant in_tail bugs (wrongly stopping tailing logs) were fixed in latest release, we recommend upgrading to fluent-package v5.0.2!

Changes from fluent-package v5.0.1

  • Update fluentd to 1.16.3 which contains significant bug fixes about in_tail. See Fluentd v1.16.3 and v1.16.2 have been released blog article about details.
  • Update plugins
    • fluent-diagtool v1.0.3. It supports fluent-package and can collect information about locally installed gems. It may help to migrate from td-agent v4 a bit. See Upgrade to fluent-package v5 for migration.
  • msi: support path which contains space or parenthesis (#589)
  • deb: fixed system user/group name in logrotate config (#592,#594)
    • It fixes a bug that unknown user error was reported during log rotation.
  • rpm: fixed to create fluentd user as system account (#596)
    • It fixes a bug that /var/lib/fluent directory was created unexpectedly. It doesn't affect the fluentd service behavior, but it is desirable one.
  • rpm: changed to keep system account after removing fluent-package. (#598)
    • In the previous versions, there was a bug that group was not cleanly removed when the package was upgraded from td-agent v4. This change makes reinstall/downgrade friendly.

About next LTS schedule

We plan to ship next LTS version of fluent-package v5.0.3 on Feb, 2024. The content of updates are still in T.B.D.

About td-agent v4.5.2

This is a exceptional maintenance release of v4.5.x series. Fluentd was updated to 1.16.3 because it contains significant bug fixes about in_tail. Note that td-agent will not be updated anymore. See Drop schedule announcement about EOL of Treasure Agent (td-agent) 4.

Download

Please see the download page.

Read More

About Fluentd

Fluentd is an open source data collector to simplify log management.

Learn

Want to learn the basics of Fluentd? Check out these pages.

Ask the Community

Couldn't find enough information? Let's ask the community!

Ask the Experts

You need commercial-grade support from Fluentd committers and experts?

Follow Us!