How to install ELK on Debian 10 or Debian 11?

Overview

The Elastic Stack or ELK Stack is a collection of open-source software tools developed by Elastic that offer centralized logging, analytics, research and logs visualization gathered from any source and in any format. In order to find issues with your servers or apps, centralized logging might be helpful as it enables you to browse through all of your logs in a single location. Additionally, by comparing the logs of such servers over a predetermined period of time, you can pinpoint problems that affect multiple servers simultaneously.

In this tutorial, we'll see how to install the ELK stack, which is a is collection of three open-source tools, Elasticsearch, Kibana, and Logstash.

Prerequisites

There are certain prerequisites that need to be met before you begin.

• Server running Debian 10 or 11

• Access to SSH connected text editor

• User account with root or sudo access

• Internet connection

Get Started

Step 1: Java Installation:

Let's first update the system and install Java, which is a pre-requisite for Elasticsearch.

Oracle Java and OpenJDK are both supported by Elasticsearch, but since ELK deployment requires Java 8 or 11, we're installing OpenJDK instead.

sudo apt-get update && sudo apt-get install openjdk-8-jdk 

You may verify the installation with the following command:

 java -version 

Step 2: Public signing key:

The GPG key (GNU Privacy Guard) should be added to assure the security and legitimacy of the download source. Download the public signing key using the command below:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - 

Note: You may encounter the following highlighted error while running the above command.

Run the following command to install the required packages for adding the GPG key and resolve the above said error.

apt-get update && apt-get install gnupg2 

Step 3: Transport-https package installation:

APT transport permits access of repositories via HTTP Secure protocol (HTTPS). Your Debian installation may also need to have apt-transport-https after adding the GPG key. You can obtain it by using the following command:

sudo apt-get install apt-transport-https –y 

Step 4: Saving directory definitions:

Use the following command to save the repository once apt-transport-https is available for your Debian 11 system at /etc/apt/sources.list.d/elastic-7.x.list.

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list 

Step 5: ELK installation:

Use the following command to collectively install Elasticsearch, Logstash and Kibana:

sudo apt-get update && sudo apt-get install Elasticsearch && sudo apt-get install Logstash && sudo apt-get install Kibana 

Step 6: Elasticsearch configuration:

Elasticsearch takes relatively minimal configuration changes, as it comes with decent default settings. Meanwhile, one needs to put the node-specific details such as node name, cluster name, network host IP and port number etc in the yaml file.

For setting up the configurations for Elasticsearch, make the following changes in the /etc/elasticsearch/elasticsearch.yml script.

• Uncomment the highlighted code lines.

• Give suitable name to your cluster and node.

• Put your network host to “0.0.0.0” (this will allow traffic from each port).

• Set your http port number to access.

• Paste < discovery.type: single-node> at the end of the script.

Step 7: Elasticsearch service startup:

Systemctl can be used to launch the Elasticsearch service. Wait a short while for Elasticsearch to launch. Errors about being unable to connect can appear if you don't.

sudo systemctl start elasticsearch

Step 8: Services heath checkup:

By making an HTTP request, you can check if your Elasticsearch service is active:

curl -XGET http://localhost:9200/_cluster/health?pretty 

Step 9: Kibana Configuration:

We need to set the configurations for kibana, using the following commands:

nano /etc/kibana/kibana.yml 

Make the following changes in the /etc/kibana/kibana.yml script.

• Uncomment the highlighted lines in white.

• Set the server base URL as your localhost IP address with the input port number (here we have taken 5601).

• Set your server's name.

• Set the address for Elasticsearch host.

Step 10: Kibana Startup:

systemctl start kibana 

Step 11: Logstash configuration:

We need to create a configuration file for logstash, using the command below:

sudo nano /etc/logstash/conf.d/30-elasticsearch-output.conf

Paste the following lines of code into the file, and make sure to put your localhost IP address, as shown in the image below.

Step 12: Logstash Startup:

Start Logstash and enable it if your configuration test is successful to apply the configuration changes:

systemctl start logstash 

Step 13: Services Startup:

If all your configuration tests are successful, enable all the three installed services to start and put the configuration changes into effect:

1. systemctl enable elasticsearch

2. systemctl enable kibana

3. systemctl enable logstash

As a next step, visit http:localhost/5601 in your browser and you’ll be prompted to the Elasticsearch home page, from where you can navigate and explore different windows including kibana, logstash and other integrations.