Installing Fluentd

Step 1:

Install ruby

(Fluentd is written in ruby)

ruby 2.0.0 

Open a terminal and type in

     tar -zxvf Downloads/ruby-2.0.0-p247.tar.gz 

     cd ruby-2.0.0-p247

     ./configure

     sudo make install

Step 2:

Install rake

(Shortcut to install everything you need as this would also install rubygems)

    

    sudo apt-get install rake

Step 3:

Install fluentd 

         sudo gem install fluentd --no-ri --no-rdoc  

Step 4:

Running fluentd  

      #Move to directory where you wish to setup fluentd
        fluentd --setup ./fluent
        fluentd -c ./fluent/fluent.conf

Intro to Fluentd

Official Website: fluentd.org

A brief description-

                   Fluentd is an open source tool for log aggregation.
It can be used in myriad ways... You can customize it according to your needs...

Its default input plugins include options to...

           1. Tail file
           2. Listen on port
           3. Retreive records form HTTP post messages
           4. read syslogs directly
           5. execute external programs to receive or pull event logs

Its default output plugins include options to...

           1.Output to file
           2. Forward to other Fluentd nodes
           3. pass events to an external program as a tab-separated value (TSV) file
           4. Output to MongoDB

                    You can also buffer the data using the buffer plugins.

The basic configuration has three parts:

           1. Source directive    :     Specify sources for data

           2. Match directive     :     Filters the input data based on matching tags
                                                   and takes the action corresponding to the tag.
                                             
                                                   e.g. If the system requires access logs to be
                                                          forwarded to other Fluentd node then we
                                                          can make the configuration as:
                                                                   
                                                                    Tail apache access log file using in_tail
                                                          plugin. Give the tag as something like
                                                          apache.access. In the match directive
                                                          (match apache.access) write the output
                                                          configuration for out_forward plugin.

           3. Include directive    :    Directives in separate configuration files can be
                                                   imported using the include directive

You can even write your own regex in the pattern field of source directive

So... go ahead and enjoy logging.....