Présentation :

Cette suite (il y a aussi d'autres logiciels qui peuvent s'y greffer comme par exemple redis, flume) se décompose comme cela :

  • Elasticsearch : l'indexer (Base de données en NoSQL)
  • Logstash : Parser qui va permettre d'alimenter Elasticsearch
  • Kibana : la WebUI qui permet de générer des dashboards, recherches de donnée sur Elasticsearch

Contexte :

Nous allons installer cette solution à partir d'une CentOS 6.5 avec SELinux en permissif.
Pour l'installation des packages il y a 2 possibilités les RPMs ou les repositories.

Les fondations :

Le pré-requis minimum sont un serveur Web et Java.

Apache

yum install httpd
chkconfig --level 2345 httpd on
service httpd start

Java

yum install java-1.7.0-openjdk

Elasticsearch :

RPM

cd /usr/src
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.2.noarch.rpm
yum localinstall elasticsearch-1.3.2.noarch.rpm
#préférer yum localinstall à rpm -ivh car il n'y a pas de corruption de la DB yum
chkconfig --level 2345 elasticsearch on
service elasticsearch start

Repository

rpm --import http://packages.elasticsearch.org/GPG-KEY-elasticsearch
[elasticsearch-1.3]
name=Elasticsearch repository for 1.3.x packages
baseurl=http://packages.elasticsearch.org/elasticsearch/1.3/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1
yum install elasticsearch
chkconfig --level 2345 elasticsearch on
service elasticsearch start

Test de bon fonctionnement

curl http://127.0.0.1:9200
{
  "status" : 200,
  "name" : "John Falsworth",
  "version" : {
    "number" : "1.3.2",
    "build_hash" : "dee175dbe2f254f3f26992f5d7591939aaefd12f",
    "build_timestamp" : "2014-08-13T14:29:30Z",
    "build_snapshot" : false,
    "lucene_version" : "4.9"
  },
  "tagline" : "You Know, for Search"
}

Logstash :

RPM

wget https://download.elasticsearch.org/logstash/logstash/packages/centos/logstash-1.4.2-1_2c0f5a1.noarch.rpm
wget https://download.elasticsearch.org/logstash/logstash/packages/centos/logstash-contrib-1.4.2-1_efd53ef.noarch.rpm
yum localinstall logstash*

Repository

[logstash-1.4]
name=logstash repository for 1.4.x packages
baseurl=http://packages.elasticsearch.org/logstash/1.4/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1
yum -y install logstash*

Configuration de base :

Notre fichier de configuration de base permettra de rediriger les messages syslog dans Elasticsearch.

input {
  #Ouverture en entrée d'un port d'écoute utilisant le protocol syslog
  tcp {
	port => 5544
	type => syslog
  }
  udp {
	port => 5544
	type => syslog
  }
}

filter {
  # Traitement type syslog, le type étant marqué sur les données entrant par nos ports de type syslog
  if [type] == "syslog" {
	grok {
  	# Si on ne veut pas garder le message non traité
  	overwrite => "message"
  	match => {
    	# rsyslong envoi des messages de type : <Numero>Ligne Syslog avec le message
    	"message" => "^(?:<%{NONNEGINT:syslog_pri}>)?%{SYSLOGBASE2} %{GREEDYDATA:message}"
    	"message" => "%{SYSLOG5424PRI}%{SYSLOGBASE2} %{DATA:message}"
  	}
  	# on ajoute des tags perso, pratique pour filtrer dans l'interface kibana
  	add_tag => [ "syslog", "grokked" ]
	}
    }
}


output {
# on stock dans elasticsearch
  elasticsearch {
	host => "localhost"
  }
}

Finalisation de Logstash

chkconfig --level 2345 logstash on
service logstash start

Rsyslog

Voici un fichier de configuration pour accépter les messages syslog du réseau et forwarder à logstash ces messages

# rsyslog v5 configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)
#$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
#on accepte les messages en UDP
$ModLoad imudp
$UDPServerRun 514

# Provides TCP syslog reception
#on accepte les messages en TCP
$ModLoad imtcp
$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure
# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$WorkDirectory /var/lib/rsyslog # where to place spool files
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
#Forward a logstash
*.* @@127.0.0.1:5544
# ### end of the forwarding rule ###
service rsyslog restart

Kibana

cd /var/www/html
wget https://download.elasticsearch.org/kibana/kibana/kibana-3.1.0.tar.gz
tar xzvf kibana-3.1.0.tar.gz
chown -R apache:apache /var/www/html/kibana-3.1.0

Il faut savoir qu'Elasticsearch utilise le port 9200, donc deux solutions s'offre à nous, utiliser le port standard 9200 mais il faut que le flux soit ouvert à travers les différents du réseau ou utiliser les directives de mod_proxy qui va permettre de proxifier via le port 9200 via le port 80 et/ou 443 ce qui permet via HTTPS de sécuriser le flux.

Configuration Apache avec mod_proxy et config kibana

<LocationMatch "^/+$">
    Options -Indexes
    ErrorDocument 403 /error/noindex.html
</LocationMatch>

ProxyPass /my_elasticsearch http://127.0.0.1:9200/
ProxyPassReverse  /my_elasticsearch http://127.0.0.1:9200/
service httpd graceful
/** @scratch /configuration/config.js/1
 *
 * == Configuration
 * config.js is where you will find the core Kibana configuration. This file contains parameter that
 * must be set before kibana is run for the first time.
 */
define(['settings'],
function (Settings) {


  /** @scratch /configuration/config.js/2
   *
   * === Parameters
   */
  return new Settings({

    /** @scratch /configuration/config.js/5
     *
     * ==== elasticsearch
     *
     * The URL to your elasticsearch server. You almost certainly don't
     * want +http://localhost:9200+ here. Even if Kibana and Elasticsearch are on
     * the same host. By default this will attempt to reach ES at the same host you have
     * kibana installed on. You probably want to set it to the FQDN of your
     * elasticsearch host
     *
     * Note: this can also be an object if you want to pass options to the http client. For example:
     *
     *  +elasticsearch: {server: "http://localhost:9200", withCredentials: true}+
     *
     */
    /**elasticsearch: "http://"+window.location.hostname+":9200",**/
    elasticsearch: "//"+window.location.hostname+"/my_elasticsearch",
    /** @scratch /configuration/config.js/5
     *
     * ==== default_route
     *
     * This is the default landing page when you don't specify a dashboard to load. You can specify
     * files, scripts or saved dashboards here. For example, if you had saved a dashboard called
     * `WebLogs' to elasticsearch you might use:
     *
     * default_route: '/dashboard/elasticsearch/WebLogs',
     */
    default_route     : '/dashboard/file/default.json',

    /** @scratch /configuration/config.js/5
     *
     * ==== kibana-int
     *
     * The default ES index to use for storing Kibana specific object
     * such as stored dashboards
     */
    kibana_index: "kibana-int",

    /** @scratch /configuration/config.js/5
     *
     * ==== panel_name
     *
     * An array of panel modules available. Panels will only be loaded when they are defined in the
     * dashboard, but this list is used in the "add panel" interface.
     */
    panel_names: [
      'histogram',
      'map',
      'goal',
      'table',
      'filtering',
      'timepicker',
      'text',
      'hits',
      'column',
      'trends',
      'bettermap',
      'query',
      'terms',
      'stats',
      'sparklines'
    ]
  });
});

Test de kibana :

Dans un shell du serveur taper la commande suivante :

logger "Test syslog pour kibana"

Cette commande va envoyer un texte au syslog local du serveur et l'on pourra vérifier dans kibana si cela s'affiche bien (via *Test* dans le champs Query en haut du dashboard).

Utilisez son navigateur et aller à l'adresse http://_Adresse_IP_du_serveur_/kibana-3.1.0/ et cliquer sur le lien "Sample Dashboard"
Il devrait apparaître dans la partie basse les messages syslog.

Voici le résultat :
Kibana Sample Dashboard

Un prochain article traitera d'autres sujets sur ELK.

Bonus :

Les packages sur ma ressource ICI en cas de besoins ou sur GitHub.

Logs Elasticsearch : /var/log/elasticsearch/elasticsearch.log
Logs Logstash : /var/log/logstash/logstash.log