one-liners in linux and more

[recordcodes-blog]

Numpy quick stats

import numpy as np

dat=[some array of numbers]

np.percentile(dat,10)

np.percentile(dat, 90)

np.mean(dat)

np.median(dat)

np.std(dat)

[recordcodes-blog]

Compiling C programs

To compile a C or program (extension .c), use the command 

gcc programname.c

Use the -o option to specify the output name

gcc -o newname programname.c

If the extension is .cpp, then it is a C++ program and you can use

g++ programname.cpp

The result by default is a script called “a.out”. You can run a.out directly.

[recordcodes-blog]

Numpy loadtxt

All parameters are

numpy.loadtxt(fname, dtype=<type 'float'>, comments='#', delimiter=None, converters=None, skiprows=0, usecols=None, unpack=False, ndmin=0)

fname is the filename ‘./sample.txt’), where the text needs to be columns of the same length, ie no missing values (for missing values look into genfromtxt)

dtype is the data type, which is automatically set to floating numbers (includes decimals). To download the list as a string, you would type in  dtype=‘str’

[recordcodes-blog]

AWK print rounded numbers

To get the format XXX.XX for a column of numbers, try:

awk ‘{printf “%3.2f\n”, $1}’

the \n indicates a new line

[recordcodes-blog]

AWK printing single and double quotes

for the double quote “ awk ‘BEGIN{print “\””}’

for the single quote ‘ awk ‘BEGIN{print “\x27″}’

[recordcodes-blog]

Specify field separator for sort in Unix

surprisingly easy :) and listed in the manual for ‘sort’ command (>> man sort)

as an example, say you have a commas as the FS. to sort, use the option

sort --field-separator=,

that is a double dash - -

[recordcodes-blog]

Remove empty files

find . -type f -empty -delete

[recordcodes-blog]

Using argparse

One use of argparse in Python that I use often is writing a script that takes an input from the command line. This way, you can run the script iteratively over multiple files. 

import argparse parser = argparse.ArgumentParser() parser.add_argument(”--inf”) args = parser.parse_args() data = np.loadtxt(args.inf)

[recordcodes-blog]

Python Imports

Some good/common things to import on python

import numpy as np import matplotlib as mpl mpl.use(’Agg’) from scipy import stats from matplotlib import pyplot as plt import argparse

[recordcodes-blog]

Tranpose matrix with Python and Numpy

Given a text file that formed like a complete matrix, you can use python and numpy to get the transpose of it as needed by using the following lines:

import numpy as np origdata=np.loadtxt(’./datafile’) transposedata=np.array([list(i) for i in zip(*origdata)])

[recordcodes-blog]

Ipython

use the command to bring up the ipython gui

ipython notebook

and use the command to see the figure without on the same page of results

matplotlib inline

[recordcodes-blog]

Changing the color bar for gmt plot

use the command 

makecpt -T1/3/0.5 -Z > ppp.cpt

to create a color bar from value of 1 to value of 3 in 0.5 intervals stored in a file called ppp.cpt. This ppp.cpt file is used in the script that makes the plot within the line

cpt=“ppp.cpt”

An example (as of the day this is posted) is in the space I have on hydra, I have a file called .plot.temp.scr -- it is a script that was originally used to create a plot of temperature. I use the command

./plt.temp.scr precip.climatology

to create a plot called precip.climatology.png

[recordcodes-blog]

Suppressing and saving a figure in python

Useful for when figures/graphics are not supported and you want to save the figure directly without it popping up

import matplotlib as mpl import matplotlib.pyplot as plt mpl.use(’Agg’)

<<code>>

plt.savefig(’./title.png’)

[recordcodes-blog]

using AWK to find min, max

Use awk to find the minimum and maximum value of column one in some file ‘data.txt’

awk ‘{if(min==“”){min=max=$1};if($1>max){max=$1};if($1<min){min=$1}}END{print min, max}’ data.txt