How to Install Hadoop

This updated guide examines How to Install Hadoop and organizes the essential facts, background, and practical takeaways in clear American English.

Method 1

Preparing to Install Hadoop

1. Install WinRARwith TAR support. Since you'll need to extract a TAR file to install Hadoop, make sure that you check the "TAR" box when installing WinRAR.
2. Install Java. To do so:
   • Go to https://www.java.com/en/download/ and click Free Java Download .
   • Click Agree and Start Free Download .
   • Double-click the downloaded Java file.
   • Click Yes when prompted.
   • Click Install , then follow any on-screen prompts.
3. Open the Hadoop page. Go to https://hadoop.apache.org/ in your computer's web browser.
4. ClickDownload. It's at the top of the page.
5. Clickbinarynext to the top result. This will take you to the Binary mirror link page for the most recent version of Hadoop.
6. Click the recommended link. It's the link below the "We suggest the following mirror site for your download:" line of text near the top of the page. Doing so prompts your computer to begin downloading the Hadoop TAR file.
7. Open the Java Development Kit (JDK) site. Go to http://www.oracle.com/technetwork/java/javase/downloads/index.html in your browser.
8. ClickDOWNLOAD. You'll find this under the "JDK" heading on the right side of the page.
9. Check the "Accept License Agreement" box. It's near the bottom of the page.
10. Click the Windows "Download" link. This is the link under the "Downloads" heading and to the right of the "Windows" category near the bottom of the page. Doing so prompts the JDK setup file to download onto your computer.
11. Install the JDK. Double-click the JDK's setup file, then do the following:
    • Click Yes when prompted.
    • Click Next twice.
    • Allow the JDK to install, and follow any on-screen prompts if necessary.

Method 2

Adding Hadoop to Your Hard Drive

1. Open the Hadoop TAR file. Double-click the downloaded Hadoop TAR file to do so. It should open in WinRAR.
2. ClickExtract To. It's at the top of the WinRAR window. Doing so opens a pop-up window.
3. Select your hard drive as the extraction location. In most cases, your hard drive should be labeled OS (C:) ; click it in the right-hand pane of the pop-up window to select it.
4. ClickOK. It's at the bottom of the window. Doing so will extract the Hadoop folder to your computer's "C" drive. Once the extraction is complete, you can proceed.

Method 3

Adding Hadoop and Java Environment Variables

1. Open This PC. Double-click the This PC app icon, which resembles a computer monitor.
   • If you don't have This PC on your desktop, click File Explorer and then click This PC on the left side of the window.
2. Open your hard drive. Double-click your hard drive's name (again, usually OS (C:) ) below the "Devices and drives" heading.
3. Open the Hadoop installation folder. Double-click the "hadoop" folder in the list of folders on your hard drive.
   • You'll see the Hadoop version number after "hadoop" in the folder's name.
4. Open the "bin" folder. Double-click the "bin" folder to do so.
5. Copy the address. Click the address bar at the top of the File Explorer to highlight the address, then pressCtrl+Cto copy it.
6. Open Start. Click the Windows logo in the bottom-left corner of the screen.
7. Open the System Variables window. Type inedit the system, then click Edit the system environment variables at the top of the Start menu.
8. ClickEnvironment Variables…. It's at the bottom of the System Variables window. A new window will appear.
9. ClickNew…under the "User variables" section. Doing so opens yet another window.
10. Name the variable. TypeHADOOP_HOMEinto the "Variable name" text box.
11. Enter the path to the Hadoop folder. Click the "Variable value" text box, then pressCtrl+V.
12. ClickOK. It's at the bottom of the window. You should now have a "HADOOP_HOME" variable listed in the top pane.
13. Add Java to the "User variables" section. Go to your hard drive, open the "Program Files" folder, open the "Java" folder, open the "jdk" folder (this folder will have a number after it), and open the "bin" folder, then copy the address. You can then add Java to the environment by doing the following:
    • Click New... under the "User variables" section.
    • TypeJAVA_HOMEinto the "Name" text box.
    • Click the "Variable value" text box, then pressCtrl+V.
    • Click OK .
14. Add both addresses to the PATH variable. You can add both the Hadoop address and the Java address to the PATH variable by doing the following:
    • Click the "Path" heading in the "System variables" section.
    • Click Edit... .
    • Click New... on the right side of the window.
    • Paste in the Java address by pressingCtrl+V, then press↵Enter.
    • Go back and copy the Hadoop "bin" folder's address, then click New... , paste it in, and press↵Enter.
15. Save your changes to the environment variables. Click OK on all three open System Environment Variables windows to do so.

Method 4

Configuring Hadoop

1. Open the Hadoop installation folder. Go to your hard drive in This PC, then double-click the "hadoop" folder.
2. Open the "etc" folder. Double-click it to do so.
3. Open the "hadoop" folder. This should be the only folder in the "etc" folder.
4. Open the "core-site" file in Notepad. To do so:
   • Right-click the "core-site" file.
   • Select Open with in the drop-down menu.
   • Click Notepad .
5. Add the proper configuration. Delete the "" tags at the end of the document, then copy and paste in the following code:

   fs.defaultFShdfs://localhost:9000

6. Save the document. PressCtrl+S, then click OK if prompted. You can then close the document.
7. Open the "mapred-site" file in Notepad. You should see a file similar to the "core-site" file open.
8. Add the proper configuration. Delete the "" tags at the end of the document, then copy and paste in the following code:

   mapreduce.framework.nameyarn

9. Save the document. PressCtrl+S, then click OK if prompted. You can then close the document.
10. Go back to the "hadoop" installation folder. Click the "hadoop" folder which includes the Hadoop version number in the address bar.
11. Create a new folder named "data". To do this, right-click a blank space in the window, select New , click Folder , type indata, and press↵Enter.
12. Open the "data" folder. Double-click it to do so.
13. Create two new folders named "datanode" and "namenode". When you're done, you should have a "data" folder that has one folder named "datanode" and one folder named "namenode".
14. Open the "hdfs-site" file in Notepad. You'll find this file in the "hadoop" folder that's inside the "etc" folder from earlier.
15. Add the proper configuration. Delete the "" tags at the end of the document, then copy and paste in the following code:

    dfs.replication1dfs.namenode.name.dirC:hadoop-2.8.0datanamenodedfs.datanode.data.dirC:hadoop-2.8.0datadatanode

16. Save the document. PressCtrl+S, then click OK if prompted. You can then close the document.
17. Open the "yarn-site" file in Notepad. This file is also in the "hadoop" folder.
18. Add the proper configuration. Delete the "" tags at the end of the document, then copy and paste in the following code:

    yarn.nodemanager.aux-servicesmapreduce_shuffleyarn.nodemanager.auxservices.mapreduce.shuffle.classorg.apache.hadoop.mapred. ShuffleHandler

19. Save the document. PressCtrl+S, then click OK if prompted. You can then close the document.
20. Open the "hadoop-env" file in Notepad. Simply right-click this file and then click Edit to do so.
21. Update the Java path. Find theset JAVA_HOME=%JAVA_HOME%line, type@remin front of it, and then type below it the phrase "set JAVA_HOME=" followed by your Java JDK folder's address (not the "bin" folder's address). For example, your line might resemble the following:

    @rem set JAVA_HOME=%JAVA_HOME%setJAVA_HOME=C:Program FilesJavajdk-10.0.2

Method 5

Applying the Hadoop Fix

1. Download and extract the Hadoop fix file. Open the fix file's Google Drive page and click Download , then extract it by doing the following:
   • Double-click the fix file.
   • Click Extract To .
   • Click Desktop .
   • Click OK .
2. Open the extracted fix file. Once the file finishes extracting, double-click it to open it.
3. Copy the "bin" folder. Click once the "bin" folder to select it, then pressCtrl+Cto copy it.
4. Open the Hadoop installation folder. Go to your hard drive's folder, then double-click the "hadoop" folder there.
5. Paste in the "bin" folder. PressCtrl+Vto do so.
6. ClickReplace the files in the destinationwhen prompted. Once this process is complete, Hadoop should be installed on your computer.