On this tutorial we will discuss how to install Apache Flink version 1.13.2 on CentOS 8 operating system and run a simple example of how Apache Flink is running.
Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments perform computations at in-memory speed and at any scale. Apache Flink is executing arbitrary dataflow programs in a data-parallel and pipelined (hence task parallel) manner. Flink’s pipelined runtime system enables the execution of bulk/batch and stream processing programs. Flink is written in Java and Scala. On this article we will learn how to install Apache Flink version 1.13.2 on CentOS 8 operating system.
Apache Flink Installation On CentOS 8
On this tutorial we will examine how to install Apache Flink on local installation. Apache Flink requires Java 8 or 11 to run smoothly on the system. The Apache Flink installation on CentOS 8 will be consist of several steps, namely :
- Download Flink binary file
- Unpack the downloaded file
- Testing Apache Flink On Local Cluster
- Start a Local Flink Cluster
- Submit The Job
- Stop the Local Cluster
On this tutorial, we will deploy the newest stable of Apache Flink, version 1.13.2. To be able to run Apache Flink, the only requirement is to have a working Java 8.x or higher installed on the system. Before starting the installation process, we have to make sure ifJava is installed on our system first. To verify this requirement we will query the Java version on our system by submitting command line :
[ramans@diginetapp01 ~]$ java -version openjdk version "1.8.0_302" OpenJDK Runtime Environment (build 1.8.0_302-b08) OpenJDK 64-Bit Server VM (build 25.302-b08, mixed mode)
We already had Java version 8 installed on the system.
2. Download Apache Flink Binary File
We will download the latest stabel of Apache Flink, version 1.13.2. The official website for latest Apache Flink located on https://flink.apache.org/downloads.html#apache-flink-1132. For this purpose we will type the command line :
[ramans@diginetapp01 ~]$ wget https://downloads.apache.org/flink/flink-1.13.2/flink-1.13.2-bin-scala_2.12.tgz --2021-09-10 17:36:52-- https://downloads.apache.org/flink/flink-1.13.2/flink-1.13.2-bin-scala_2.12.tgz Resolving downloads.apache.org (downloads.apache.org)... 184.108.40.206, 220.127.116.11, 18.104.22.168, ... Connecting to downloads.apache.org (downloads.apache.org)|22.214.171.124|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 304614530 (291M) [application/x-gzip] Saving to: ‘flink-1.13.2-bin-scala_2.12.tgz’ flink-1.13.2-bin-scala_2.12.t 100%[===============================================>] 290.50M 826KB/s in 6m 5s 2021-09-10 17:42:58 (815 KB/s) - ‘flink-1.13.2-bin-scala_2.12.tgz’ saved [304614530/304614530]
3. Unpack Apache Flink Source
After downloading was completed done, than we will exctrac the file inside current directory by submitting command line :
tar -xzf flink-1.13.2-bin-scala_2.12.tgz
[ramans@diginetapp01 ~]$ tar -xzf flink-1.13.2-bin-scala_2.12.tgz [ramans@diginetapp01 ~]$ ls -ltr total 297480 drwxr-xr-x. 10 ramans ramans 156 Jul 23 04:06 flink-1.13.2 -rw-rw-r--. 1 ramans ramans 304614530 Jul 23 06:14 flink-1.13.2-bin-scala_2.12.tgz [ramans@diginetapp01 ~]$ cd flink-1.13.2/
4. Testing Apache Flink On Local Custer
After all are set, the we will test the Apache Flink on our system. For this purpose we will start Flink, submit a simple job and stop the Flink.
Starting Flink Cluster
flink-1.13.2/bin directory we will submi the command line :
[ramans@diginetapp01 bin]$ ./start-cluster.sh Starting cluster. Starting standalonesession daemon on host diginetapp01. Starting taskexecutor daemon on host diginetapp01.
The Apache Flink web frontend can be accessed at
http://<ip-address_or_hostname>:8081. We can monitor and make sure everything is up and running. The web front end should report a single available TaskManager instance.
We will submit a simple job for testing Apache Link function. For this purpose we will type the command line :
./bin/flink run examples/streaming/WordCount.jar
[ramans@diginetapp01 flink-1.13.2]$ ./bin/flink run examples/streaming/WordCount.jar Executing WordCount example with default input data set. Use --input to specify file input. Printing result to stdout. Use --output to specify output path. Job has been submitted with JobID d5210027f89d443e4b62fdad2a5713a8 Program execution finished Job with JobID d5210027f89d443e4b62fdad2a5713a8 has finished. Job Runtime: 2360 ms
The log file output should be :
[ramans@diginetapp01 flink-1.13.2]$ tail log/flink-*-taskexecutor-*.out (nymph,1) (in,3) (thy,1) (orisons,1) (be,4) (all,2) (my,1) (sins,1) (remember,1) (d,4)
And the information on the front end website will be as follow :
Stop Apache Flink Local Cluster
Then we will stop the Apache Flink local cluster to complete our testing on running Apache Flink on CentOS 8. For this purpose we will type the command line :
[ramans@diginetapp01 flink-1.13.2]$ ./bin/stop-cluster.sh Stopping taskexecutor daemon (pid: 5396) on host diginetapp01. Stopping standalonesession daemon (pid: 5129) on host diginetapp01.
On this article we have shown you how to install Apache Flink on CentOS 8 operating system. Take a simple test for starting apache Flink, submit a job and stopping Apache Flink local cluster. I hope this article is useful for anyone who needs it.