02. Openedx Analytics - Hadoop/Hive/Sqoob

Configure hadoop, hive and sqoop

sudo su - hadoop 

cd /edx/app/hadoop




Please check Hadoop 

hadoop jar hadoop-2.7.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar pi 2 100

Please check Hive is working

/edx/app/hadoop/hive/bin/hive







Test Dashboard is up and running




http://localhost:18110 

make suer hadoop dashboards are working

http://localhost:8088/cluster

hadoop job history

http://localhost:19888

hadoop node description

http://localhost:8042

Troubleshooting
01. Hadoop dashboard not working

hadoop/sbin/stop-all.sh

hadoop/sbin/start-all.sh

Determine Port is listening

telnet localhost 9000

lsof -i :9000

nc -l -p 9000

Test HDFS System

hadoop fs -ls /

Create Directory 

hadoop fs -mkdir /data

Put file

hadoop fs -put /home/file.txt /data