Ansible Playbook To Configure Hadoop And Start Cluster Service

For configuring hadoop cluster we need a name node and a data node. Data node will share its storage to name node to store data For donig this we will use ansible playbook.

Configuration of hadoop cluster consist of several steps:

Firstly we have to configure Name Node:

  1. We have to send hadoop and jdk(java) software to the target node using ansible for this these two software or rpm we should have in our system.
tasks:    
- name: "copying hadoop software"
copy:
src: "/root/hadoop-1.2.1-1.x86_64.rpm"
dest: "/root/"
- name: "copying jdk"
copy:
src: "/root/jdk-8u171-linux-x64.rpm"
dest: "/root/"

2. We will install the software in target node

- name: “installing software” 
shell: “rpm -ivh hadoop-1.2.1–1.x86_64.rpm — force”
— name: “installing jdk”
shell: “rpm -ivh jdk-8u171-linux-x64.rpm”
#Instead of shell command here we can use "command" module also.

3. Now we will configure Hdfs file and core file

file: 
state: directory
path: “{{namenode_dir}}”
— name: “hdfs file configuration”
blockinfile:
path: “/etc/hadoop/hdfs-site.xml”
insertafter: “<configuration>”
block:
<property>
<name>dfs.name.dir</name>
<value>{{namenode_dir}}</value>
</property>
— name: “core file configuration”
blockinfile:
path: “/etc/hadoop/core-site.xml”
insertafter: “<configuration>”
block:
<property>
<name>fs.default.name</name>
<value>hdfs://0.0.0.0:9001</value>
</property>

4.Formating Name node

- name: “formating namenode”
shell: "echo Y | hadoop namenode -format"

5.Starting services

- name: “starting namenode service”   s
shell: "hadoop-daemon.sh start namenode"

Same kind of setup we have to do for Data node

Name Node

When we run These ansible playbook our hadoop setup will configured

VEDIO LINK

https://youtu.be/o7BDwNi53r0

ENJOY LEARNING…..