Ansible Playbook To Configure Hadoop And Start Cluster Service
2 min readMar 22, 2021
For configuring hadoop cluster we need a name node and a data node. Data node will share its storage to name node to store data For donig this we will use ansible playbook.
Configuration of hadoop cluster consist of several steps:
Firstly we have to configure Name Node:
- We have to send hadoop and jdk(java) software to the target node using ansible for this these two software or rpm we should have in our system.
tasks:
- name: "copying hadoop software"
copy:
src: "/root/hadoop-1.2.1-1.x86_64.rpm"
dest: "/root/"
- name: "copying jdk"
copy:
src: "/root/jdk-8u171-linux-x64.rpm"
dest: "/root/"2. We will install the software in target node
- name: “installing software”
shell: “rpm -ivh hadoop-1.2.1–1.x86_64.rpm — force”
— name: “installing jdk”
shell: “rpm -ivh jdk-8u171-linux-x64.rpm”
#Instead of shell command here we can use "command" module also.3. Now we will configure Hdfs file and core file
file:
state: directory
path: “{{namenode_dir}}”
— name: “hdfs file configuration”
blockinfile:
path: “/etc/hadoop/hdfs-site.xml”
insertafter: “<configuration>”
block:
<property>
<name>dfs.name.dir</name>
<value>{{namenode_dir}}</value>
</property>
— name: “core file configuration”
blockinfile:
path: “/etc/hadoop/core-site.xml”
insertafter: “<configuration>”
block:
<property>
<name>fs.default.name</name>
<value>hdfs://0.0.0.0:9001</value>
</property>4.Formating Name node
- name: “formating namenode”
shell: "echo Y | hadoop namenode -format"5.Starting services
- name: “starting namenode service” s
shell: "hadoop-daemon.sh start namenode"Same kind of setup we have to do for Data node
Name Node
When we run These ansible playbook our hadoop setup will configured
VEDIO LINK
