Saturday, September 3, 2016

Step By Step Mapreduce Example run on hadoop 2.7.1 with video




Step 1: open terminal using ctrl+alt+T

Step 2: go to hadoop user using
hadoopashish@ubuntu:su hadoop-user

Step 3: Start all Hadoop using below command
hadoop-user@ubantu : start-all.sh

Step 4: Make new Data Directory on Desktop and create on text file in side Data folder which is on desktop
Desktop/data/text.txt

Step 5:After start hadoop just type below text
hadoop-user@ubantu : jps
6385 ResourceManager
6146 SecondaryNameNode
5637 NameNode
6696 NodeManager
5866 DataNode
12510 Jps

Step 6:after doing this You have to go on below path
hadoop-user@ubantu : cd   /usr/local/hadoop/
or which path inside your Directory Mine is
hadoop-user@ubantu : cd   /usr/local/hadoop-2.7.1/

Step 7 :after going in this folder you have to type
hadoop-user@ubantu : cd /usr/local/hadoop-2.7.1 $ bin/hdfs dfs -mkdir /user
after this press enter
hadoop-user@ubantu : cd /usr/local/hadoop-2.7.1 $ bin/hdfs dfs -mkdir /user/ashish/
after this press enter

Step 8 : next step to open web browser to veryfying this user will be created or not so just open web browser and type in navigation bar
localhost:50070/
then press enter.You are seeing hadoop window inside that you have to go utilities-->Browse the file System, In side that you can see Browse Directory list.just click on GO Button and You can see that below user directory then select that user directory u can see another directory which one we have created ahead,In my case i see ashish directory name I am seeing.
then verification is done.

Step 9: now we are give the input which is stored on desktop/data/text.txt
hadoop-user@ubuntu:/usr/local/hadoop-2.7.1$ hdfs dfs -put /home/hadoopashish/Desktop/data input
so here hadoop system is getting input directory file

Step 10: now this is main step to run map-reduce.
Here first you have to type hadoop which is internal command.
the jar which is java archive file extensionw which will be run our hadoop-mapreduce-example-2.7.1.jar
the wordcount which is program name wordcount.java which is inbuilt store inside hadoop-mapreduce-example-2.7.1.jar file
then our input file name input and output which is going to stored output of next after running this command.
hadoop-user@ubuntu:/usr/local/hadoop-2.7.1$ hadoop  jar  share/hadoop/mapreduce/hadoop-mapreduce-example-2.7.1.jar  wordcount  input  output

After putting this step just press enter and your mapreduce job is going to run and u can see that process in terminal window

Step 11: after completing this step u have to go once again in the browser and open localhost:50070 then go in utilities à browse the file system inside that click on GO button and go inside user the inside user goes in hadoop-user (My Directory where stored file)
You can see too file input and output.
You can see there r-0000 file and success file in output folder.open r-0000 file yes this is our output file.
Open it and u can see the output of this file




1 comment: