Index


Overview

Splunk is a great tool for managing and interrogating data from multiple data-sources. A Splunk installation includes the ability to create and distribute instances of the Splunk Universal Forwarder, a tool that can be deployed to a data-source server and used to collect data and send data through to your Splunk server.

This document details the process required to configure a Splunk Universal Forwarder to send your Statseeker log data through to a Splunk server deployed within your environment.

[top]


Prerequisites

  • A Statseeker server
  • A Splunk server, configured to received data from a Splunk Universal Forwarder
  • A Splunk Universal Forwarder for FreeBSD, match the forwarder version to the FreeBSD version of your Statseeker server
    Note: to confirm the FreeBSD version of your Statseeker server run the following on the server's CLI:

    # freebsd-version

For details on acquiring, installing, and configuring a Splunk server see https://www.splunk.com

Note: this process was created and confirmed using:

  • Statseeker v5.6.2
  • Splunk Universal Forwarder v9.4.0 for 64bit FreeBSD v13

[top]


Configure the Splunk Forwarder

  • SSH into your Statseeker server as the statseeker user and then change to the root user
  • Install the forwarder:
    pkg install splunkforwarder-<version>-freebsd-<version>-amd64.txz



    Example:

    # su root
# pkg install splunkforwarder-9.4.0-freebsd13-amd64.txz
Note: if your Statseeker server cannot access the internet:

  • Download the Splunk Universal Forwarder from the Splunk website
  • Upload the forwarder to your Statseeker server
  • Navigate to the directory containing the uploaded package add the package to the server

    Example:

    # su root
# cd /tmp/
# pkg add splunkforwarder-9.4.0-251651b291fe-freebsd13-amd64.txz
  • If you receive a message that an updated version of teh forwarder is available


  • Change directory to the binary location of the installed Splunk agent
  • Start the Splunk forwarder
Note: the forwarder will present a software license for review on first run, you can automate the acceptance of this license with the --accept-license argument

Example:

# cd /opt/splunkforwarder/bin
# ./splunk start --accept-license

If this is the first time that the package has been run it will prompt you to create an administration account.


  • Configure the Splunk forwarder to start whenever the Statseeker server is restarted

Example:

#./splunk enable boot-start


  • Configure the forwarder to send to the Splunk server

    Syntax: splunk add forward-server [splunk_server]:[port]
Note: the default port on the Splunk server to receive data from the forwarder is 9997. Additional ports can be configured from the Splunk web interface or by editing the inputs.conf, see Splunk's documentation for details.

Example:

#./splunk add forward-server test122.statseeker.com:9997

OR

#./splunk add forward-server 10.1.44.3:9997


  • Add a monitor for each log you want to send through to Splunk

    Syntax: add monitor [source]

Example:

#./splunk add monitor /home/statseeker/base/logs/base-logd.log
#./splunk add monitor /home/statseeker/base/logs/httpd.errors
#./splunk add monitor /home/statseeker/base/logs/httpd.access
#./splunk add monitor /var/log/maillog


You can add additional parameters to make it easier to group and identify data sources, see Splunk's documentation library ( http://docs.splunk.com/Documentation) for details.

You can view details on all currently configured datasources with:

#./splunk list monitor

[top]


Log Files

There are a range of logs available on a Statseeker server:
Statseeker specific logs:

  • /home/statseeker/base/logs/base-logd.log - Statseeker process logs
  • /home/statseeker/base/logs/dashboards.log - Statseeker dashboard logs
  • /home/statseeker/base/logs/httpd.error - Statseeker web-server errors (http)
  • /home/statseeker/base/logs/httpd.access - Statseeker web-server access logs(http)
  • /home/statseeker/base/logs/httpsd.error - Statseeker web-server errors (https)
  • /home/statseeker/base/logs/httpsd.access - Statseeker web-server access logs(https)
  • /home/statseeker/base/logs/python.log - Statseeker's custom reporting error and log messages
  • /home/statseeker/base/logs/perl-logd.log - Statseeker legacy services error log

FreeBSD logs:

  • /var/log/maillog - mail server log
  • /var/log/cron - system-wide cron activity
  • var/log/messages - generic system activity (non-kernel boot errors, application/service errors)
  • /var/log/auth.log - server authentication logs