Discover Top Posts Tagged with #quickstart vm

RHadoop Installation on Cloudera QuickStart VM

Download 3 packages (rmr,rhdfs,rhbase) for RHadoop from https://github.com/RevolutionAnalytics/RHadoop/wiki/Downloads

According 2 the installation manual rmr must be installed on all the hadoop nodes, rhdfs to be installed on NameNode.

But i installed all the packages and modified /etc/profile on all the nodes in order to avoid errors...

#update the source to fedoraproject and get R installed rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm yum install R #run commands in R to install the following #packages related to rhdfs and rmr > install.packages("rJava") > install.packages("reshape2") > install.packages("Rcpp") > install.packages("iterators") > install.packages("itertools") > install.packages("digest") > install.packages("RJSONIO") > install.packages("functional") > install.packages("bitops") > install.packages("caTools")

#find the working DIR for /bin/hadoop and #find the working DIR /contrib/streaming/hadoop-streaming*.jar locate /bin/hadoop locate /contrib/streaming #add HADOOP_CMD path and HADOOP_STREAMING path #to the environment profile (vi /etc/profile) export HADOOP_CMD=/usr/lib/hadoop/bin/hadoop export HADOOP_STREAMING=/usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.4.0.jar export HADOOP_HOME=/usr/lib/hadoop export HADOOP_CONF_DIR=/etc/hadoop/conf #enable the modify without reboot source /etc/profile #install rmr and rhdfs in the terminal R CMD INSTALL "rmr2_3.0.0.tar.gz" R CMD INSTALL "rhdfs_1.0.8.tar.gz"

#run commands in R > library(rhdfs) HADOOP_CMD=/usr/lib/hadoop/bin/hadoop Be sure to run hdfs.init() > hdfs.init() 14/02/17 09:29:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable #enter the DIR of /usr/lib/hadoop/lib/native #and copy libhadoop.so and libhadoop.so.1.0.0 to /usr/lib64 cd /usr/lib/hadoop/lib/native cp libhadoop.so /usr/lib64 cp libhadoop.so.1.0.0 /usr/lib64 #add JAVA_LIBRARY_PATH to profile to solve the problem #vi /etc/profile export JAVA_LIBRARY_PATH=/usr/lib/hadoop/lib/native #enable the ENV modify once more and re-config R java source /etc/profile R CMD javareconf #run commands in R and try 2 load rhdfs and rmr2 again #generally speaking the problem could be solved > library(rhdfs) > library(rmr2)