October 21, 2021
How to install Apache Flink On CentOS 8

How To Install Apache Flink On CentOS 8

On this tutorial we will discuss how to install Apache Flink version 1.13.2 on CentOS 8 operating system and run a simple example of how Apache Flink is running.

Introduction

Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments perform computations at in-memory speed and at any scale. Apache Flink is executing arbitrary dataflow programs in a data-parallel and pipelined (hence task parallel) manner. Flink’s pipelined runtime system enables the execution of bulk/batch and stream processing programs. Flink is written in Java and Scala. On this article we will learn how to install Apache Flink version 1.13.2 on CentOS 8 operating system.



Apache Flink Installation On CentOS 8

On this tutorial we will examine how to install Apache Flink on local installation. Apache Flink requires Java 8 or 11 to run smoothly on the system. The Apache Flink installation on CentOS 8 will be consist of several steps, namely :

  1. Prerequisites
  2. Download Flink binary file
  3. Unpack the downloaded file
  4. Testing Apache Flink On Local Cluster
    • Start a Local Flink Cluster
    • Submit The Job
    • Stop the Local Cluster

1. Prerequisites

On this tutorial, we will deploy the newest stable of Apache Flink, version 1.13.2. To be able to run Apache Flink, the only requirement is to have a working Java 8.x or higher installed on the system. Before starting the installation process, we have to make sure ifJava is installed on our system first. To verify this requirement we will query the Java version on our system by submitting command line : java --version.

[ramans@diginetapp01 ~]$ java -version
openjdk version "1.8.0_302"
OpenJDK Runtime Environment (build 1.8.0_302-b08)
OpenJDK 64-Bit Server VM (build 25.302-b08, mixed mode)

We already had Java version 8 installed on the system.

2. Download Apache Flink Binary File

We will download the latest stabel of Apache Flink, version 1.13.2. The official website for latest Apache Flink located on https://flink.apache.org/downloads.html#apache-flink-1132. For this purpose we will type the command line :

 wget https://downloads.apache.org/flink/flink-1.13.2/flink-1.13.2-bin-scala_2.12.tgz
[ramans@diginetapp01 ~]$ wget https://downloads.apache.org/flink/flink-1.13.2/flink-1.13.2-bin-scala_2.12.tgz
--2021-09-10 17:36:52--  https://downloads.apache.org/flink/flink-1.13.2/flink-1.13.2-bin-scala_2.12.tgz
Resolving downloads.apache.org (downloads.apache.org)... 88.99.95.219, 135.181.214.104, 135.181.209.10, ...
Connecting to downloads.apache.org (downloads.apache.org)|88.99.95.219|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 304614530 (291M) [application/x-gzip]
Saving to: ‘flink-1.13.2-bin-scala_2.12.tgz’

flink-1.13.2-bin-scala_2.12.t 100%[===============================================>] 290.50M   826KB/s    in 6m 5s   

2021-09-10 17:42:58 (815 KB/s) - ‘flink-1.13.2-bin-scala_2.12.tgz’ saved [304614530/304614530]

3. Unpack Apache Flink Source

After downloading was completed done, than we will exctrac the file inside current directory by submitting command line :

tar -xzf flink-1.13.2-bin-scala_2.12.tgz
[ramans@diginetapp01 ~]$ tar -xzf flink-1.13.2-bin-scala_2.12.tgz
[ramans@diginetapp01 ~]$ ls -ltr
total 297480
drwxr-xr-x. 10 ramans ramans       156 Jul 23 04:06 flink-1.13.2
-rw-rw-r--.  1 ramans ramans 304614530 Jul 23 06:14 flink-1.13.2-bin-scala_2.12.tgz
[ramans@diginetapp01 ~]$ cd flink-1.13.2/

4. Testing Apache Flink On Local Custer

After all are set, the we will test the Apache Flink on our system. For this purpose we will start Flink, submit a simple job and stop the Flink.

Starting Flink Cluster

Form the flink-1.13.2/bin directory we will submi the command line :

 ./start-cluster.sh
[ramans@diginetapp01 bin]$ ./start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host diginetapp01.
Starting taskexecutor daemon on host diginetapp01.

The Apache Flink web frontend can be accessed at http://<ip-address_or_hostname>:8081. We can monitor and make sure everything is up and running. The web front end should report a single available TaskManager instance.

Apache Flink front end website
Apache Flink front end website

Submit Job

We will submit a simple job for testing Apache Link function. For this purpose we will type the command line :

./bin/flink run examples/streaming/WordCount.jar
[ramans@diginetapp01 flink-1.13.2]$ ./bin/flink run examples/streaming/WordCount.jar
Executing WordCount example with default input data set.
Use --input to specify file input.
Printing result to stdout. Use --output to specify output path.
Job has been submitted with JobID d5210027f89d443e4b62fdad2a5713a8
Program execution finished
Job with JobID d5210027f89d443e4b62fdad2a5713a8 has finished.
Job Runtime: 2360 ms

The log file output should be :

[ramans@diginetapp01 flink-1.13.2]$  tail log/flink-*-taskexecutor-*.out
(nymph,1)
(in,3)
(thy,1)
(orisons,1)
(be,4)
(all,2)
(my,1)
(sins,1)
(remember,1)
(d,4)

And the information on the front end website will be as follow :

Apache Flink | Submitting a simpe job (word count)
Apache Flink | Submitting a simpe job (word count)

Stop Apache Flink Local Cluster

Then we will stop the Apache Flink local cluster to complete our testing on running Apache Flink on CentOS 8. For this purpose we will type the command line :

./bin/stop-cluster.sh
[ramans@diginetapp01 flink-1.13.2]$ ./bin/stop-cluster.sh
Stopping taskexecutor daemon (pid: 5396) on host diginetapp01.
Stopping standalonesession daemon (pid: 5129) on host diginetapp01.

Conclusion

On this article we have shown you how to install Apache Flink on CentOS 8 operating system. Take a simple test for starting apache Flink, submit a job and stopping Apache Flink local cluster. I hope this article is useful for anyone who needs it.

Share this article via :

Leave a Reply

Your email address will not be published. Required fields are marked *