Skip to main content
Version: Next

Logstash

Elastic Logstash is Centralize, transform & stash your data, used for Data Orchestration and Integration . Logstash is a free and open server-side data processing pipeline that ingests data from a multitude of sources, transforms it, and then sends it to your favorite "stash."

dashboard

Prepare​

When referring to this document to use Elastic Logstash, please read and ensure the following points:

  • Login to Websoft9 Console and find or install Elastic Logstash:

    • Go to My Apps listing applications
    • Go to App Store installing target application

  • This application is installed by Websoft9 console.

  • The purpose of this application complies with the SSPL-v1 open source license agreement.

  • Configure the domain name or server security group opens external network ports for application access.

Getting started​

Initial setup​

  1. When completing the installation of Logstash in the Websoft9 Console, get the applicaiton's Overview and Access information from My Apps

Capturing (Input) Data​

In Logstash's configuration file, specify one or more input plugins to define the source of your data. Logstash provides a variety of built-in input plugins, such as File, TCP, UDP, HTTP, and more. Choose the appropriate input plugin based on your needs and configure its parameters.

Filtering data​

For data processing, configure filtering plugins (for example, grok, mutate, date) to parse, modify, or enrich data based on its structure and your requirements.

Exporting data​

To Elasticsearch​

Refer to: Elasticsearch Connection Logstash

To Kafka​

  1. Install the Kafka plugin.
  2. Add the connection configuration to the configuration file. 
    output {
       kafka {
         bootstrap_servers => "kafka_host:port" # Kafka server host and port
         topic_id => "your_topic" # Name of the Kafka topic
       }
     }

Configuration options​

  • Logstash configuration file (mounted): usr/share/logstash/pipeline/logstash.conf

Administer​

Troubleshooting​