HFaber

For my four readers

[hfaber-blog1]

OpenRefine - for cleaning dirty data

Some useful tutorial and presentation on OpenRefine. How many times, clients use Excel for business, so many inconsistency in data.

http://openrefine.org/

http://enipedia.tudelft.nl/wiki/OpenRefine_Tutorial

https://evanwill.github.io/clean-your-data/1-about.html

[hfaber-blog1]

Google Maps and geoJSON

Some try: https://github.com/MassimoCappellano/learn_geoJSON

[hfaber-blog1]

Redis: Zero to Master in 30 minutes

http://openmymind.net/2011/11/8/Redis-Zero-To-Master-In-30-Minutes-Part-1/

To use for session management with hapi.js and JWT

[hfaber-blog1]

The First 5 Imports

Essential Node/JavaScript Libraries I Can't Live With Out

https://rclayton.silvrback.com/my-first-5-imports

[hfaber-blog1]

Repo Badges

https://github.com/dwyl/repo-badges

[hfaber-blog1]

Calculating Work Days In Java

From this: https://pardoe.io/2008/10/12/calculating-work-days-in-java/

http://www.java2s.com/Code/Java/Data-Type/CalculateHolidays.htm

[hfaber-blog1]

YAML Cookbook

YAML cookbook: http://yaml.org/YAML_for_ruby.html

YAML a primer: https://github.com/darvid/trine/wiki/YAML-Primer

YAML: YAML Ain't Markup Language

What It Is: YAML is a human friendly data serialization standard for all programming languages.

For java - jackson: https://github.com/FasterXML/jackson-dataformat-yaml

[hfaber-blog1]

Cassandra Db && node js easy on the cloud for Free

Thanks to  Branson Gitomeh and his example of using nodejs to connect to cassandra, using openshift plattorm: https://github.com/BransonGitomeh/prium

[hfaber-blog1]

wkhtmltopdf on OpenShift

Here https://github.com/ormanli/openshift-html2pdf

[hfaber-blog1]

How buil a cassandra cartridge

Start from here: https://github.com/rajatchopra/openshift-origin-cartridge-cassandra

On my repository to add an environment variable: OPENSHIFT_CASSANDRA_NATIVE_TRANSPORT_PORT

https://github.com/MassimoCappellano/openshift-origin-cartridge-cassandra

[hfaber-blog1]

How-To: Keep a github repository and an OpenShift repository in sync

Read here a simple explanation: https://forums.openshift.com/how-to-keep-a-github-repository-and-an-openshift-repository-in-sync

[hfaber-blog1]

Sharing database across applications in Openshift

Started from

https://blog.openshift.com/sharing-database-across-applications/

First try

Same problem trying from one java application on a diy gear to connect to another application with a mysqldb.

The same error: no route to host

Even connecting with ssh to the classcardapp (java application) and then doing telnet to the application mysql, the same error:

APPLICATION WITH DB

[classcardappdb-simplethings.rhcloud.com 566e8d057628e1f9d900004c]\> env | grep MYSQL OPENSHIFT_MYSQL_DIR=/var/lib/openshift/566e8d057628e1f9d900004c/mysql/ OPENSHIFT_MYSQL_DB_PORT=3306 OPENSHIFT_MYSQL_DB_HOST=127.6.59.130 OPENSHIFT_MYSQL_DB_PASSWORD=?????????? OPENSHIFT_MYSQL_IDENT=redhat:mysql:5.5:0.2.20 OPENSHIFT_MYSQL_DB_USERNAME=???????? OPENSHIFT_MYSQL_DB_SOCKET=/var/lib/openshift/566e8d057628e1f9d900004c/mysql//socket/mysql.sock OPENSHIFT_MYSQL_DB_URL=mysql://??????????:***********@127.6.59.130:3306/ OPENSHIFT_MYSQL_VERSION=5.5

CONNECTING FROM OTHER APPLICATION AND TRYING TELNET TO MYSQL

[classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> telnet $OPENSHIFT_EXTMYSQL_DB_HOST $OPENSHIFT_EXTMYSQL_DB_PORT Trying 127.6.59.130... telnet: connect to address 127.6.59.130: No route to host [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\>

Strange, should be a normal use case.

Solutions

The problem was that I haven’t created the db application to be scalable. In the tutorial says:

Step 1: Create an application with a database

We will create a scalable PHP application using a MySQL database cartridge. In non-scalable applications, the database will be installed in the same gear as the application. In this case we want the database to be accessible from other gears. So creating a scalable application ensures that the database runs in its own gear that can be accessed from other gears.

This was the explanation on previous failures.

So creating db application, use the -s option for scalable applications:

$ rhc app create -s classcard2db php-5.4 mysql-5.5

Other resources:

https://blog.openshift.com/cloud-connections-how-to-use-openshift-with-external-databases/

https://blog.openshift.com/getting-started-with-port-forwarding-on-openshift/

[hfaber-blog1]

Problem using maven with java8 on cartridge ‘diy’

My application is depeloped using a simple frameworl Dropwizard. Here a mini tutorial: https://blog.openshift.com/day-13-dropwizard-the-awesome-java-rest-server-stack/

Now I've to deploy application on OpenShift. Here some tips during deployment.

1° PROBLEM

By default ‘diy’ cartridge uses java 7. I’ve to use java 8 in my application so I’ve found this hit for installing java 8 on my application: 

http://stackoverflow.com/questions/23894323/jdk-8-support-at-diy-cartridge-in-openshift

It’s works well.

Now following these instructions: http://dmly.github.io/blog/2013/05/01/diy-java-app-server-on-openshift-so-far-so-good/

But,

2° PROBLEM

Running maven, It can’t write in defaul repository for missing permission, so I’ve found a solution by specifing in sessings.xml file the repisitory location in my $OPENSHIFT_DATA_DIR

Here the explanation, how to run maven locally: 

https://forums.openshift.com/maven-could-not-create-local-repository

But,

3° PROBLEM

Even when I specify JAVA_HOME to my java 8 installation, the default maven, yet installed by default on Openshift, uses java 7, so It gives problems compiling source, executing maven task:

+ mvn package -s /var/lib/openshift/566e8b000c1e66efec00002f/app-root/runtime/repo//.openshift/settings.xml -q clean package [debug] execute contextualize [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile) on project classcard_gest: Fatal error compiling: invalid target release: 1.8 -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

In fact from command line on connecting to my gear:

[classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> export JAVA_HOME="${OPENSHIFT_DATA_DIR}jdk1.8.0_05" [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> export PATH=${OPENSHIFT_DATA_DIR}jdk1.8.0_05/bin:$PATH [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> java -version java version "1.8.0_05" Java(TM) SE Runtime Environment (build 1.8.0_05-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode) [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> javac -version javac 1.8.0_05 [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\> mvn -version Apache Maven 3.0.4 (r1232336; 2012-12-18 14:36:37-0500) Maven home: /usr/share/java/apache-maven-3.0.4 Java version: 1.7.0_91, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.91/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "2.6.32-504.34.1.el6.x86_64", arch: "i386", family: "unix" [classcardapp-simplethings.rhcloud.com 566e8b000c1e66efec00002f]\>

Here some explanation:

http://stackoverflow.com/questions/33460703/openshift-maven-project-fails-to-find-java-8-compiler

https://bugzilla.redhat.com/show_bug.cgi?id=1109980

https://developer.jboss.org/people/fjuma/blog/2014/06/03/wildfly-810final-on-openshift--with-jdk-8-support

So I’ve to installa a mvn in my $OPENSHIFT_DATA_DIR and uses this one.

An example hook for installing maven

https://github.com/shekhargulati/spark-openshift-quickstart/blob/master/.openshift/action_hooks/deploy

[hfaber-blog1]

HowTo, fatal: The remote end hung up unexpectedly

Configuring a cassandra instance on Openshift, when updating the configurations with git push command, I’ve this error:

Counting objects: 3, done. Delta compression using up to 8 threads. Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 284 bytes | 0 bytes/s, done. Total 3 (delta 2), reused 0 (delta 0) remote: Stopping DIY cartridge remote: ++ grep cassandra remote: ++ ps -ef remote: ++ grep -v grep remote: ++ awk '{ print $2 }' fatal: The remote end hung up unexpectedly fatal: The remote end hung up unexpectedly

Searching solutions

https://forums.openshift.com/diy-the-remote-end-hung-up-unexpectedly

seams the stop script in .openshift/action_hooks kill even the connection stopping the deployment.

The stop script was

#!/bin/bash # The logic to stop your application should be put in this script. set -x kill `ps -ef | grep cassandra | grep -v grep | awk '{ print $2 }'` > /dev/null 2>&1 exit 0

Mybe is due to the fact that my application is named cassandra2classcard, so grep cassandra on stop script mybe kill even the process that is redeploing the application.

The solution was to change in the stop script the grep expression with java in this way:

#!/bin/bash # The logic to stop your application should be put in this script. set -x kill `ps -ef | grep java | grep -v grep | awk '{ print $2 }'` > /dev/null 2 >&1 exit 0

It function even is a bit more dangerous! Mybe better could be to change the name of the application, not having cassandra in it.

Here deleted `cassandra2classcard` witch has problem and used name `classcarddbsessions` for cassandra application.

[hfaber-blog1]

How To Configure and Run Cassandra on OpenShift

Start from here,

https://blog.openshift.com/cassandra-on-openshift/

but problem when starting cassandra (java memory allocation) described in: 

https://github.com/shekhargulati/cassandra-openshift-quickstart/issues/2

working on,

I’ve done this patch that seems to work: https://github.com/MassimoCappellano/cassandra-openshift-quickstart

untill now.

[hfaber-blog1]

Joda time

Now included in java 8. Useful start http://joda-time.sourceforge.net/quickstart.html

[hfaber-blog1]

reverting files

This is a topic that is a constant source of confusion for many git users, basically because there’s more than one way to skin the proverbial cat. Let’s go over some of the basic commands that you’ll need to undo your work.

So, you just want to revert one file back to its original state:

git checkout <file>

One problem with this is that you may have is that a file and branch named the same. Since the checkout command is used for both reverting files and swapping out to a different branch, you’ll need to use this syntax (thanks, Norbauer)

git checkout -- <file>

If you want to throw out all of the changes you’ve been working on, there’s two ways to do that.

git checkout -f or git reset --HARD

Once these commands are run you’ll lose all of the work that isn’t committed in your directory, so make sure to take caution when using them.

Also, be aware that ‘git revert’ is not equivalent to ‘svn revert’! git-revert is used to reverse commits, something another tip will cover in the future.