Fluentd

Ruby

The fluent-logger-ruby library is used to post records from Ruby applications to Fluentd.

This article explains how to use it.

Prerequisites

• Basic knowledge of Ruby
• Basic knowledge of Fluentd
• Ruby 2.1 or later

Installing Fluentd

Please refer to the following document to install Fluentd:

• Installation

Modifying the Config File

Configure Fluentd to use the forward input plugin as its data source:

<source>
  @type forward
  port 24224
</source>
<match fluentd.test.**>
  @type stdout
</match>

Restart agent after configuring.

# for rpm/deb only
$ sudo /etc/init.d/td-agent restart

# or systemd
$ sudo systemctl restart td-agent.service

Using fluent-logger-ruby

Add 'fluent-logger' gem to your Gemfile:

gem 'fluent-logger', "~> 0.7.1"

Initialize and post the records as shown below:

require 'fluent-logger'
Fluent::Logger::FluentLogger.open(nil, :host=>'localhost', :port=>24224)
Fluent::Logger.post("fluentd.test.follow", {"from"=>"userA", "to"=>"userB"})

Executing the script will send the logs to Fluentd:

$ ruby test.rb

The logs should be output to /var/log/td-agent/td-agent.log or the standard output of the Fluentd process via the stdout output plugin.

Production Deployments

Output Plugins

Various output plugins are available for writing records to other destinations:

• Examples
  • Store Apache Logs into Amazon S3
  • Store Apache Logs into MongoDB
  • Data Collection into HDFS
• List of Plugin References
  • Output to Another Fluentd
  • Output to MongoDB or MongoDB ReplicaSet
  • Output to Hadoop
  • Output to File
  • etc...

High-Availability Configurations of Fluentd

For high-traffic websites (more than 5 application nodes), we recommend using the high-availability configuration for td-agent. This will improve the reliability of data transfer and query performance.

• High-Availability Configurations of Fluentd

Monitoring

Monitoring Fluentd itself is also important. The article below describes the general monitoring methods for td-agent.

• Monitoring Fluentd