[huseyinkuyucu]

HOW TO RESET ANDROID STUDIO MASTER PASSWORD

If you forgot your master password for Android Studio, follow the following steps:

open Android Studio > File > Settings > Appearance & Behavior > System Settings > Passwords

Click on Master Passwords then you’ll see “Change Master Password” box like following.Click on reset 

You can set your new password and confirm it.That’s all and don’t forget again, please :)

[huseyinkuyucu]

DATA TRANSFER FROM MYSQL TO HDFS

Prerequisites

MySQL 

Sqoop 1.4.6 

Hadoop 2.6.0

IMPORTANT:Configure the MySQL Service and Connector mysql-connector-java-5.1.36.jar file and copy it to $SQOOP_HOME/lib directory.

First connect to MySQL

 mysql -u root -p  //Enter password 

Create database(my database name:userdata) and use userdata database

mysql>create database userdata;

mysql>use userdata;

now create table “student”

mysql>create table student(id integer, name char(40));

Add following 4 records to the table

mysql> insert into student values(1, 'Huseyin');

mysql> insert into student values(2, 'Volkan'); 

mysql> insert into student values(3, 'Alptekin');

mysql> insert into student values(4, 'Van Persie');

Exit from MySQL

mysql> exit; (if you want, write quit instead of exit.it does not matter)

Now we will see, importing a table into hdfs

General:

sqoop import –connect jdbc:mysql://localhost/ –username –password –table –m 1

MyExample:

sqoop import –connect jdbc:mysql://localhost/userdata –username root –password 1234 –table student –m 1

and then

hadoop dfs -cat /user/huseyin/student/part-m-00000 

I hope, this example be useful for you if you have a issue, you can write me your question(s) ;)

[huseyinkuyucu]

MYSQL INSTALLATION ON UBUNTU 14.04

First  update your system

sudo apt-get update

sudo apt-get dist-upgrade

Install the MySQL server and client packages:

   sudo apt-get install mysql-server mysql-client

during the installation,  MYSQL will ask you to set a root password and you can access your Mysql server

mysql -u root -p

[huseyinkuyucu]

SQOOP INSTALLATION ON UBUNTU 14.04

Download sqoop 1.4.6

http://mirror.idealhosting.net.tr/Apache/sqoop/1.4.6/

Extract here and move to /usr/local

 sudo mv '/home/huseyin/Desktop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha' /usr/local/sqoop        //Enter your password

 sudo gedit ~/.bashrc

   add following lines

 export SQOOP_HOME=/usr/local/sqoop

 export PATH=$PATH:$SQOOP_HOME/bin 

 source ~/.bashrc

 cd /usr/local/hadoop

sqoop help

and you have to see following commands

Available commands:

 codegen         Generate code to interact with database records

 create-hive-table Import a table definition into Hive

 eval               Evaluate a SQL statement and display the results

 export             Export an HDFS directory to a database table

 help               List available commands

 import             Import a table from a database to HDFS

 import-all-tables Import tables from a database to HDFS

 import-mainframe   Import datasets from a mainframe server to HDFS

 job               Work with saved jobs

 list-databases     List available databases on a server

 list-tables       List available tables in a database

 merge             Merge results of incremental imports

 metastore         Run a standalone Sqoop metastore

 version           Display version information

[huseyinkuyucu]

How to Install Hadoop 2.6.0

First we must install jdk. If you want to make sure, you can use following command.

NOTE-1:sometimes when you copy, you may get errors because quote marks.

NOTE-2:I use ubuntu 14.04 and jdk 1.8

huseyin@huseyin:~$ java -version

java version “1.8.0_66”

Java™ SE Runtime Environment (build 1.8.0_66-b17)

Java HotSpot™ 64-Bit Server VM (build 25.66-b17, mixed mode)

 1) JDK Installation

 $ sudo add-apt-repository ppa:webupd8team/java

 $ sudo apt-get update

 $ sudo apt-get install oracle-java8-installer

 **to verify Java version, write following command

   $ java -version  

    java version “1.8.0_20”

 Java™ SE Runtime Environment (build 1.8.0_20-b13)

 Java HotSpot™ Client VM (build 25.0-b70, mixed mode)

2)Installing SSH

~$ sudo apt-get install openssh-server

~$ ssh-keygen -t rsa -P “”

~$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

Disabling IPv6

nano /etc/sysctl.conf

net.ipv6.conf.all.disable_ipv6 = 1

net.ipv6.conf.default.disable_ipv6 = 1

net.ipv6.conf.lo.disable_ipv6 = 1

Installation Steps

 We’ll setup in /usr/local/

1) Download

huseyin@huseyin-pc:~$ cd Downloads/

huseyin@huseyin-pc:~/Downloads$ sudo cp hadoop-2.6.0.tar.gz /usr/local/ 

huseyin@huseyin-pc:/usr/local$ sudo tar -xzvf hadoop-2.6.0.tar.gz 

huseyin@huseyin:/usr/local$ mkdir hadoop

huseyin@huseyin-pc:/usr/local$ sudo mv hadoop-2.6.0/* /usr/local/hadoop     

huseyin@huseyin-pc:/usr/local$ cd hadoop

huseyin@huseyin-pc:/usr/local/hadoop$ sudo chown huseyin:huseyin -R /usr/local/hadoop

huseyin@huseyin-pc:/usr/local$  sudo chown huseyin:huseyin -R /usr/local

huseyin@huseyin-pc:/usr/local$ cd hadoop

huseyin@huseyin-pc:/usr/local/hadoop$ sudo mkdir -p /usr/local/hadoop_tmp/hdfs/namenode

huseyin@huseyin-pc:/usr/local/hadoop$ sudo mkdir -p /usr/local/hadoop_tmp/hdfs/datanode 

huseyin@huseyin-pc:/usr/local$ sudo chown huseyin:huseyin -R /usr/local/hadoop_tmp/

2) Update Hadoop configuration files

huseyin@huseyin:/$ sudo gedit .bashrc

 #– HADOOP ENVIRONMENT VARIABLES START – # export JAVA_HOME=/usr/lib/jvm/java-8-oracle export HADOOP_HOME=/usr/local/hadoop export PATH=$PATH:$HADOOP_HOME/bin export PATH=$PATH:$HADOOP_HOME/sbin export HADOOP_MAPRED_HOME=$HADOOP_HOME export HADOOP_COMMON_HOME=$HADOOP_HOME export HADOOP_HDFS_HOME=$HADOOP_HOME export YARN_HOME=$HADOOP_HOME export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export HADOOP_OPTS=“-Djava.library.path=$HADOOP_HOME/lib” # – HADOOP ENVIRONMENT VARIABLES END – #

 huseyin@huseyin: source ~/.bashrc

huseyin@huseyin:/usr/local/hadoop/etc/hadoop$ sudo nano hadoop-env.sh

add  following command

export JAVA_HOME=/usr/lib/jvm/java-8-oracle

 huseyin@huseyin:/usr/local/hadoop/etc/hadoop$ sudo nano core-site.xml

# paste following command inside <configuration> tags

<property>   

   <name>fs.default.name</name>    

   <value>hdfs://localhost:9000</value>

  </property>

 huseyin@huseyin:/usr/local/hadoop/etc/hadoop$ sudo nano hdfs-site.xml

# paste following command inside <configuration> tags

<property>

      <name>dfs.replication</name>

      <value>1</value>

  </property>

   <property>

     <name>dfs.namenode.name.dir</name>

      <value>file:/usr/local/hadoop_tmp/hdfs/namenode</value>

  </property>

  <property>

     <name>dfs.datanode.data.dir</name>

      <value>file:/usr/local/hadoop_tmp/hdfs/datanode</value>

 </property>

huseyin@huseyin:/usr/local/hadoop/etc/hadoop$ sudo nano yarn-site.xml

# paste following command inside <configuration> tags

<property>        

      <name>yarn.nodemanager.aux-services</name>

      <value>mapreduce_shuffle</value>

   </property>

   <property>

     <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>

     <value>org.apache.hadoop.mapred.ShuffleHandler</value>

   </property>

 huseyin@huseyin:cd /usr/local/hadoop/etc/hadoop

huseyin@huseyin:/usr/local/hadoop/etc/hadoop$ sudo cp /usr/local/hadoop/etc/hadoop/mapred-site.xml.template /usr/local/hadoop/etc/hadoop/mapred-site.xml  

huseyin@huseyin:/usr/local/hadoop/etc/hadoop$ sudo gedit mapred-site.xml

# paste following command inside <configuration> tags

<property>      

      <name>mapreduce.framework.name</name>

      <value>yarn</value>

 </property>

3) Format Namenode

huseyin@huseyin:/usr/local/hadoop$ bin/hdfs namenode -format

4) Start all Hadoop daemons

huseyin@huseyin:/usr/local/hadoop$ start-dfs.sh     #Start hdfs daemons

huseyin@huseyin:/usr/local/hadoop$ start-yarn.sh   #Start MapReduce daemons

5) Track/Monitor/Verify

huseyin@huseyin:/usr/local/hadoop$ jps

27861 ResourceManager

27383 NameNode

28281 Jps

27530 DataNode

28174 NodeManager

27711 SecondaryNameNode

6)Now, you can access hadoop service from localhost:50070 on your browser

[huseyinkuyucu]

Merhaba, bu yazı Tumblr da ki ilk paylaşımım ve aynı zamanda ilk gezi yazım 😊