Skip to content

Elasticsearch

We got a script!!!! 🎉

FortiDragon provides an automated script to set up all necessary Elasticsearch components.

Prerequisites

Installation Steps

  1. Clone the repository:
git clone https://github.com/enotspe/fortinet-2-elasticsearch.git
cd fortinet-2-elasticsearch
  1. Make the script executable:
chmod +x ELK/load.sh

  1. Modify variables according to your environment: Either on

    1. Script itself
    2. Via environment variables

  2. Run the installation script:

cd ELK
./load.sh

Configuring Variables

Set Elasticsearch

Define Elasticsearch endpoint

# Elasticsearch connection settings
ES_URL="${ES_URL:-https://localhost:9200}"

For local deployments, with autogenerated certificates, set INSECURE flag to true

# SSL certificate validation: "true" to bypass SSL validation, "false" to validate
INSECURE="${INSECURE:-false}"

Define user/pass

# User/Password authentication (used when AUTH_METHOD="user")
ES_USERNAME="${ES_USERNAME:-elastic}"
ES_PASSWORD="${ES_PASSWORD:-changeme}"

Or API key

# API Key authentication (used when AUTH_METHOD="apikey")
ES_API_KEY="${ES_API_KEY:-}"

Set LOAD

Via LOAD_* variables, we control what modules are loaded to Elasticsearch. By default, only index related configuration is loaded.

LOAD_ECS="${LOAD_ECS:-true}"
LOAD_COMPONENT="${LOAD_COMPONENT:-true}"
LOAD_ILM="${LOAD_ILM:-true}"
LOAD_INDEX_TEMPLATES="${LOAD_INDEX_TEMPLATES:-true}"
LOAD_INGEST_PIPELINES="${LOAD_INGEST_PIPELINES:-false}"
LOAD_TRANSFORMS="${LOAD_TRANSFORMS:-false}"

Ingest Pipelines

❌ Ingest Pipelines had been deprecated in favor of Vector

✅ Deploy Vector instead

Transforms

❌ Transforms are not stable

😬 We are probably not using them as they are intended to be

What gets installed

The script automatically creates:

Index Templates

  • logs-fortinet.fortigate.traffic
  • logs-fortinet.fortigate.utm
  • logs-fortinet.fortigate.event
  • logs-fortinet.forticlient
  • logs-fortinet.fortiedr
  • logs-fortinet.fortimail

Index Templates

You can manually add/remove component templates to the new created index templates

Component Templates

  • ECS field mappings
  • Fortinet specific mappings
  • ILM policies
  • Index settings (refresh intervals, field limits, etc.)

ILM Policies

Automated lifecycle management for each datastream type:

  • logs-fortinet.fortigate.traffic
  • logs-fortinet.fortigate.utm
  • logs-fortinet.fortigate.event
  • logs-fortinet.forticlient
  • logs-fortinet.fortiedr
  • logs-fortinet.fortimail

Ingest Pipelines

  • logs-fortinet.fortigate
  • logs-fortinet.forticlient
  • logs-fortinet.fortiedr
  • logs-fortinet.fortimail
  • logs-fortinet.fortiweb

Ingest Pipelines

Will only load if LOAD_INGEST_PIPELINES is set to true

Data Retention

Elasticsearch can manage data lifecycle either via:

Data stream lifecycle vs ILM Policies

Both options are set on created index templates

ILM has precedence over Data stream lifecycle, so be sure to remove them from your index template if you don't intended to use them.

Data stream lifecycle

Use them when you do not have a data tiered architecture

ILM Policy

Use them when you do have a data tiered architecture

Next Steps

Once Elasticsearch is configured:

  1. Import dashboards in Kibana

  2. Start dancing with your logs!