HungryCat Projects

[fborja]

5.4 - Hobby Servo Calibration

In this last post of this series, we will see the results for the calibration. We have now a file with X - value in microseconds the signal is up for a cycle, and Y - angle measured for the servo (using artificial vision). It is a csv file, that we can open with any software like LibreOffice, for example. And represent the data in a chart:

Here we can see the relation is linear between these two magnitudes, and the trend line give us the calibration. For this Pan Servo we get a 10.594 us/degree as B value. For the Tilt servo the B value is 10.739 us/degree. Each servo has to be calibrated independently, to get a more precise relation between angle and duty cycle, so we can reduce the error in positioning for certain applications.

[fborja]

5.3 - Hobby Servo Calibration

Lets continue with the code for moving a small step the servo, and take a photo and save it to disk. 

5.3.1 Moving and saving servo position image

Would have been better to use artificial vision at the same time, to get the line and its angle, but the library we are using BoofCV, is for some reason working really slow for this particular method in the BeagleBone Black... so we will do it in next step.

The code is quite simple ServoCalibation.java

public class ServoCalibration {         //variables     private final Board board; private URL url; private ServoHS422 servo;     //constructor public ServoCalibration(){     //get access to GPIO in BBB this.board = Platform.createBoard(); //initialize camera url=null; try {  //change here the ip that appears in the camera url = new URL("http://192.168.1.116:8888/camera&timestamp"); } catch (MalformedURLException e1) { // TODO Auto-generated catch block e1.printStackTrace(); }     servo=new ServoHS422(board, BBBNames.EHRPWM0B_P9_22,50.0); servo.setCalibration(1500.0, 10.0); servo.startPosition(0.0); }     //method to calibrate the servo using the camera public void calibrateServoUsingCamera(){ //we work directly with the duty cicle... and we will keep increasing with a certain step //take image //detect the line and angle on the servo using BoofCV double dc=500.0; //starting value double step=1.0; //increasing step for duty cycle double maxdc=2300.0; BufferedImage input=null; System.out.println("Starting movement..."); for(double dci=dc;dci<maxdc;dci+=step){    //move the servo servo.setDutyCycle(dci); //getting and saving image try { input = ImageIO.read(url); ImageIO.write(input, "jpg",new File("./images/"+dci+".jpg")); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } System.out.println("Image ok..."); } } }

We just keep increasing from 1500 us with steps of 1us. The ServoHS422 code will get this value and calculate duty cycle and pass it to the pin that controls the servo position. The images we get from the camera we are saving to disk in “./images” directory. 

5.3.2 Processing images using BoofCV java library

This is the link to BoofCV library: http://boofcv.org/ We are using this library to detect the line and get its angle. The method to detect the line was working really slow in the BBB as I told you before, so I have downloaded the images from the ./images directory to my laptop, and I can continue processing the data there. 

We are using one class for detecting the line given an image. This code is from the example we get from boofcv web page, detecting lines. You have three different types of detector for lines. I have tried for the images I got from the camera, and the HoughPolar method is the one that better result has given.

Here you have the code for the findLine.java class:

import georegression.struct.line.LineParametric2D_F32; import java.awt.Dimension; import java.awt.image.BufferedImage; import java.util.List; import boofcv.abst.feature.detect.line.DetectLineHoughFoot; import boofcv.abst.feature.detect.line.DetectLineHoughFootSubimage; import boofcv.abst.feature.detect.line.DetectLineHoughPolar; import boofcv.core.image.ConvertBufferedImage; import boofcv.factory.feature.detect.line.ConfigHoughFoot; import boofcv.factory.feature.detect.line.ConfigHoughFootSubimage; import boofcv.factory.feature.detect.line.ConfigHoughPolar; import boofcv.factory.feature.detect.line.FactoryDetectLineAlgs; import boofcv.gui.feature.ImageLinePanel; import boofcv.gui.image.ShowImages; import boofcv.struct.image.ImageSingleBand; public class findLine { // adjusts edge threshold for identifying pixels belonging to a line private static final float edgeThreshold = 10; // adjust the maximum number of found lines in the image private static final int maxLines = 1; //result angle from line detection public double LineAngle=0.0; //constructor public findLine(){ //do nothing... } @SuppressWarnings("rawtypes") public boolean detectLines1( BufferedImage image , Class imageType ,Class derivType ){ // convert the line into a single band image T input = ConvertBufferedImage.convertFromSingle(image, null, imageType ); // line detector DetectLineHoughPolar detector = FactoryDetectLineAlgs.houghPolar( new ConfigHoughPolar(3, 30, 2, Math.PI / 1800,edgeThreshold, maxLines), imageType, derivType); List found = detector.detect(input); if(found.size()<1) return false; LineAngle=(double)found.get(0).getAngle(); LineAngle=Math.toDegrees(LineAngle); System.out.println("FINDLINE:: Angle "+LineAngle); /* // display the results ImageLinePanel gui = new ImageLinePanel(); gui.setBackground(image); gui.setLines(found); gui.setPreferredSize(new Dimension(image.getWidth(),image.getHeight())); double angle=(double)found.get(0).getAngle(); ShowImages.showWindow(gui,"Found Lines 1 "+Math.toDegrees(angle)); */ return true; } }

The maxlines variable is set to 1, because we just have a line to detect... and there is another variable you can play with, edgeThreshold=10. You will have to modify to get maximum of good detection rate and get good calibration values.

We are getting the angles in degrees. The line keep changing its position, rotating with the servo. We will have to give to the angle the right sign. When the angle goes from -91º to -89º or passing through, the calculation can give you a positive value and increase it... so from -91, -90, +91,+92... and you have to identify those transitions and fix the value so it makes sense... Remember than the tangent for an angle will be same if you add or reduce the angle 180º... and this is what happens with the angle detection for a line.  

Here is the code for the main class that will pick up image by image, and will apply the findLine class to get an angle. The image name give us the x value. Here the source code for CalibrateServoUsingCamera.java

import java.awt.image.BufferedImage; import java.io.BufferedWriter; import java.io.File; import java.io.FileWriter; import java.io.IOException; import boofcv.io.image.UtilImageIO; import boofcv.struct.image.ImageSInt16; import boofcv.struct.image.ImageUInt8; import com.sensorlab.image.findLine; public class CalibrateServoUsingCamera { public static void main( String args[] ) { //write results to file File file = new File("/Users/borja/Desktop/BeagleBone/images/ExampleResultTiltCalibration.csv"); FileWriter fw = null; try { if (!file.exists()) { file.createNewFile(); } fw = new FileWriter(file.getAbsoluteFile()); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } BufferedWriter bw = new BufferedWriter(fw); double auxAngle=0.0; for(double dci=500.0;dci<2300.0;dci+=1.0){ BufferedImage input = UtilImageIO.loadImage("/Users/borja/Desktop/BeagleBone/images/"+dci+".jpg"); findLine fl=new findLine(); fl.detectLines1(input,ImageUInt8.class,ImageSInt16.class); try { if((fl.LineAngle>0)&&(dci<1612.0)){ fl.LineAngle=fl.LineAngle-180.0; }else if(dci>1612.0){ fl.LineAngle=180.0+fl.LineAngle; } if(Math.abs(fl.LineAngle-auxAngle)<0.8){ bw.write(dci+","+fl.LineAngle+"\n"); System.out.println(dci+","+fl.LineAngle); } } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } auxAngle=fl.LineAngle; } try { bw.close(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } } }

It is a loop loading image by image. See the angle change for x=1612.0 This is the x value for which in the list, in a first program run, i have seen the angle goes from -90 to +90, and I have enter the correction, so we have the angle between -180º to +180º range. Change 1602 for the value you find in your results there is that transition.

Results for the calibration in next post.

[fborja]

5.2 - Hobby Servo Calibration

Here you have an image of how the cardboard is attached, and the camera pointing at it.

We put the camera close to it, so we just see the line on the image. If we have more elements, it is more difficult later to recognize just the line we want using the artificial vision. We are going to take like 1000 images... we can not go later just telling the program the right line to identify...

5.2.1 Getting images from the camera

We are using a very useful application called Wifi Camera Plus, this allow us to broadcast the image from the camera to another device. What we will do is to take snapshots of the camera automatically for every servo position, and save those images to disk for processing.

Install the application, and click start. Will give you the http address from where to take the images. In our case will be http://192.168.1.116:8888

Stick the cardboard to the servo, so it moves with it. And put the camera close to it, so we can see just the cardboard in the image.

This is an example of the cardboard with the line on it from the camera:

5.2.2 Moving the servo with the BeagleBoneBlack and Java

A servo has minimum 3 wires. Red and Black are for power supply (5V for red and Ground for Black). And it has a yellow cable (sometimes another color) for the PWM signal to position the servo. Check your servo datasheet to make sure about the input voltage and color code for each one.

We are powering the servo using an external power supply (do not power the servo using the 5V outputs from the BeagleBone Black). Just remember to connect the grounds together (servo and BBB). And for PWM output se are using the pin 22 from P9 in BBB.

And now let’s go for the code in java to move the servo. The best is to make a class to manage the servo... so later we can just command something like “move one degree left” or “set the duty cycle to 1500 microseconds”, depending on the situation...

the file it is called ServoHS422.java :

import org.bulldog.core.gpio.Pwm; import org.bulldog.core.platform.Board; public class ServoHS422 { public String address; public double freq; final Board board; public Pwm pwm; //calibration public double a; public double b; //moving range public double maxAngle; public double minAngle; //constructor public ServoHS422(Board board, String address,double freq){ this.board=board; this.address=address; this.freq=freq; this.pwm = board.getPin(address).as(Pwm.class); pwm.disable(); } //angle in degrees public boolean startPosition(double angle){ boolean task=true; pwm.setFrequency(50.0f); // in Hz pwm.setDuty(getDutyCycle(angle)); // % duty cycle pwm.enable(); return task; } //we send the angle in degrees public boolean moveToAngle(double angle){ boolean task=true; if((angle>maxAngle)||(angle1.0)||(duty<0.0)){ return false; } pwm.setDuty(duty); return task; } //method to set the relation between the duty cycle and the angle //t=a+bx - t - time in us (microseconds) signal up // x - angle in degrees public void setCalibration(double a, double b){ this.a=a; this.b=b; } //method to get duty cycle from angle (in degrees) public double getDutyCycle(double angle){ double t=a+b*angle; double dc=(t/1000.0)*(freq/1000.0); //System.out.println("ServoHS422 duty Cycle: "+dc+ " angle: "+angle); return dc; } //method to close down the use of the servo public void shutdownServo(){ pwm.disable(); } //method to set moving range public void setRange(double maxAngle, double minAngle){ this.maxAngle=maxAngle; this.minAngle=minAngle; } //method to set position directly with duty cycle public void setDutyCycle(double t){ double dc=(t/1000.0)*(freq/1000.0); pwm.setDuty(dc); } }

Variables that we provide in the constructor are the Board (is an instance of the LibBulldog java library that represents the BBB GPIO system), the pin address that will command the servo position (the PWM output) and the frequency we are working.

There is also a method called setCalibration(double a,double b). Here we provide the relation between the angle and the duty cycle. So, once we have the servo calibrated, we can just send a command mentioning the angle, and using this relation we can get the duty cycle for that angle. Easy, yes?

Another useful method is the last one, setDutyCycle(double t). This is the one we are going to use for our calibration task. We will send the time the signal is up during the cycle, in microseconds, then we calculate the duty cycle (dc) that is the parameter that needs the libbulldog library.

We will use this last method, sending a command to move to successive positions, and take an image of the cardboard for each one of it. Code in next post!

[fborja]

5.1 - Hobby Servo Calibration

I have started to play with a PanTilt System. It has two servos and allow you to move 180º each one. Here you have an image:

It is from robotshop in France. The servos are HS422 from Hitec. They work really well, and are very easy to manage. I am working in a system, that using a camera, we can point a laser, mounted on the panTilt system, to any position we want on the floor or on another surface. 

For moving the servos, I am using a BeagleBone Black. Using the PWM outputs, we can set the angle position for each servo. Position change with the duty cycle we apply to the PWM. Frequency is 50 Hz.

First step for doing that is to get a proper and nice relation between the duty cycle we apply and the angle we get as result. We can find a lot of documentation about it, telling us that the 0º is around 1500 us (microseconds) signal on. -90º is around 500 us and +90º in 2500. 

The relation between the time the signal is on (in microseconds) in a cycle (of frequency 50Hz) and the angle (in degrees) is like this:

t = A + B · angle

and A is around 1500 us and B around 10 us/degrees. The problem is that if we want precision, we have to adjust these values... B in this case is the most critical. The error in B multiplies for the angle value... so bigger the angle, bigger the error. 

There is a very interesting article (by Ethan Tira-Thomson) about the calibration of the servos, that goes deep into the matter. We are using one of the techniques that shows in the article.

This is how we are going to get angle for each t value: We are going to use the camera of an android phone, and we will put a cardboard on the moving part of the servo. This cardboard has a line on it. With the camera, very close to the cardboard we are going to take an image for every small step movement of the servo. Then, with artificial vision, we will get the angle for the line. This will allow us to get very precise data from the movement, and a good calibration trend line.

[fborja]

The GatuHomeLab (4.4) - The CatEyeDetector

Now is time to test our neural network! We got the master images for every situation we want to identify, trained the network, and we have save it to disk to use it with new images from the webcam or camera.

So we can make a java class to do the job. We will give the name of the neural network to load from disk, where are the images to analyze (directory path), and the downdimensions for the images (the size we want to reduce the original images for proper use with the neural network). Then will give us the result.

We will call the class UseNeuralNetwork, and these are the variables and constructors we need.

public class UseNeuralNetwork { public int downdimensionH=120; public int downdimensionW=200; public String workingdirectory="/home/ftpimages/fb/"; public String networkname="MotiNetwork"; public String prefix="ts"; public List NNoutput; //results public int[] ivalues; public String[] svalues; //constructor public UseMotiNetwork(){ //nothing... use default parameters } //constructor public UseMotiNetwork(int ddh,int ddw,String workdirec,String neuralnet){ this.downdimensionH=ddh; this.downdimensionW=ddw; this.workingdirectory=workdirec; this.networkname=neuralnet; } (...)

And next is the method that will do the job. These are the steps we are going to follow:

Load the neural network from disk (the one we have trained)

Load the images we want to evaluate

Compute and get the results in a List

This method we have called evaluateNN(). Let’s see how it works. First we have the load the serialized object from disk:

File netfile=new File(networkname); BasicNetwork network=null; try { network=(BasicNetwork)SerializeObject.load(netfile); } catch (ClassNotFoundException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); }

Then we have to prepare the dataset where we are going to load the images to evaluate. 

//Datasets RGBDownsample downsample=new RGBDownsample(); ImageMLDataSet testset=new ImageMLDataSet(downsample,false,1,-1);

Again we will use the ImageLoader class for loading the names of the images and put it inside the dataset, do the downsampling and get it ready for the neural network.

ImageLoader test=new ImageLoader(workingdirectory,prefix); test.loadFileNames(); for(int j=0;j<test.fileNames.size();j++){ //load images File file=new File(workingdirectory+test.fileNames.get(j)); Image img=null; try { img=ImageIO.read(file); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } ImageMLData data = new ImageMLData(img); testset.add(data); } testset.downsample(downdimensionH, downdimensionW);

We have to create a list where to store the results for each image evaluated, then get the result. We have follow the criteria of saying that if the output of one of the neurons (output layer) is bigger than 0.5, we consider that an Ok for that situation. Again here you can change this part and make it as per your own criteria.

NNoutput=new ArrayList(); //enter data in network and get output for(int j=0;j0.5){ if(result[0]>0.5){ nnresult="CatEats"; }else if(result[1]>0.5){ nnresult="CatDrinks"; }else if(result[2]>0.5){ nnresult="FoodBowlEmpty"; }else if(result[3]>0.5){ nnresult="FoodBowlFull"; } //System.out.println("Result "+nnresult); NNoutput.add(nnresult); File oldfile =new File(workingdirectory+test.fileNames.get(j)); File newfile =new File(workingdirectory+nnresult+"_"+test.fileNames.get(j)); oldfile.renameTo(newfile); } Encog.getInstance().shutdown();

What we also do is to change the name of the images we have used, so next time we do not process it again. And we give a prefix to the image, with the result from the neural network. Then it is easy later to get the images and see if the computer brain got it right or was something else completely. You will get surprised of how many get it right...

[fborja]

The GatuHomeLab (4.3) - The CatEyeDetector

A Neural Network is normally used to recognize patterns. It is not use for cases where we have an input and we can determine the output using an equation, simple or complex... is for cases where we do not have means to get to this kind of equation, that should be extremely extremely complicated to find, and with multiple variables that we don’t even know how they behave... And one of the most remarkable features from Neural Networks is their ability to learn, adapting the output to different inputs. 

Brain Neurons, (link) 

See how easy it is for a computer to calculate 3256x8921... and how long will take for us to get the answer... but show a person a symbol, a face, a painting, and ask later to recognize it among others similar... and that is something that for a computer is not so easy to achieve. The neural network, like our brain, processes the information in parallel, and with many neurons working together, we can obtain really powerful results.

(Link)

Basically, the neurons are connected with the previous and next layer, and these connections has a certain weight, that is decided when the Neural Network is trained, in order to obtain a certain result at the end. That gives the power to recognize patterns, and give same outputs for different inputs, that share same pattern and is recognized by the Neural Network.

Now is time to configure the Neural Network. Will have three layers. And every layer has a number of neurons. The entry layer will have 120x200 neurons, that is the number of pixels for each image. You can change this to adapt to your image size.

The second layer is the hidden layer, and will have 100 neurons. Is not much advisable to use more hidden layers, but again I recommend you to check about neural networks and play with the size and number of hidden layers. 

The third layer is the output, and will have four neurons, one for each estate we want to identify. The code is as follows:

// configure the neural network BasicNetwork network = new BasicNetwork(); int hiddenLayerNeuronsCount = 100; network.addLayer(new BasicLayer(null, true, downdimensionH*downdimensionW)); network.addLayer(new BasicLayer(new ActivationElliott(), true, hiddenLayerNeuronsCount)); network.addLayer(new BasicLayer(new ActivationElliott(), false, 4)); network.getStructure().finalizeStructure(); network.reset();

You have noticed the ActivationElliot function. This is a neural activation function, similar to Sigmoid or Hyperbolic Tangent, but more convenient because of its computation time, and that is an advantage to make the neural network converge faster during training. 

We now train the neural network, using the images we have add to the training set, and we make iterations until we reach a certain error level. Then we have the neural network ready to use.

// train the neural network final Propagation train = new ResilientPropagation(network, training); int epoch = 0; do{ train.iteration(); System.out.println("Epoch("+epoch+") error: "+train.getError()); epoch++; }while(train.getError()>error); //close Encog Encog.getInstance().shutdown();

Why have we used Resilient Propagation to train the network? performance. It converges towards the solution faster than other methods. It is a good training method to use, and will work most of the times.  

Last step will be to serialize the trained network and save it to disk. 

//serialize Network and save to disk File networkfile=new File(workingdirectory+networkname); try { SerializeObject.save(networkfile,network); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } System.out.println("Network saved ok: "+networkname);

In next post we will test the NN, how to evaluate an image and get the output from the Neural Network.

[fborja]

The GatuHomeLab (4.2) - The CatEyeDetector

Niceee!!! first we need to get Encog for Java. You can get it here. I recommend you to read the great guide for Java to understand neural networks and how Encog works. It is really amazing the work and dedication that Mr. Jeff Heaton has put in this project. And I was using for this project the code in the great post about using Encog with images, from Pavel Surmenok, that you can find here. Please, read it, because it is really interesting.

This is the script of what we are going to do:

load the images for each situation

create and configure the neural network

train the neural network using the images

serialize and save to disk the neural network

We will use very useful class for loading the images. We have the images in four different directories, and inside each one of them, a variable number of images, that doesn’t have to be same for each one. We will give as parameters in the constructor the directory and the pattern the files follow, and the class will return the list of names of each file inside the directory. Here is the code:

public class ImageLoader { String directory; String filepattern; public List fileNames; //constructor public ImageLoader(String directory,String filepattern){ this.directory=directory; this.filepattern=filepattern; } //methods //load filenames in directory for analisys public boolean loadFileNames(){ boolean result=true; fileNames = new ArrayList(); File folder = new File(directory); for (final File fileEntry : folder.listFiles()) { if(fileEntry.getName().startsWith(filepattern)){ System.out.println("Found File: "+fileEntry.getName()); fileNames.add(fileEntry.getName()); } } return result; } }

The pattern for the file names is how the files start, like “cat_eating_” and after that a number, date, time, whatever... 

These are the global variables we are using:

static int downdimensionH=120; static int downdimensionW=200; static String workingdirectory="/Users/borja/sensorlabdata/ftpimagesfb/"; static String networkname="MotiNetwork_wzfb_02"; static float error=0.001f;

You can get images 320x240 pixels for example, and then reduce the size when you are entering the images in the training set for the neural network. That is done by “downdimensionH/W” variables. Is good also to give a name to the neural network, so you can evaluate and write down the performance for that particular version.

Let’s start creating the Training set:

//Datasets RGBDownsample downsample=new RGBDownsample(); ImageMLDataSet training=new ImageMLDataSet(downsample,false,1,-1);

And we have to decide for the output of the Neural Network. We have four possible situations: 

cat eating

cat drinking

food bowl empty

food bowl full

And we define the outputs for these situations as follow:

//output array double[] outputmoti_food={1.0,0.0,0.0,0.0}; //this is for cat eating - 1000 double[] outputmoti_water={0.0,1.0,0.0,0.0}; //this is for cat drinking - 010 double[] outputbowl_empty={0.0,0.0,1.0,0.0}; //this is for bowl empty - 0010 double[] outputbowl_full={0.0,0.0,0.0,1.0}; //this is for bowl full - 0001

Then we proceed to load the images:

//load images from folder ImageLoader moti_food=new ImageLoader(workingdirectory+"2fbmoti_food/","kf"); ImageLoader moti_water=new ImageLoader(workingdirectory+"2fbmoti_water/","kw"); ImageLoader bowl_empty=new ImageLoader(workingdirectory+"2fbbowlempty/","be"); ImageLoader bowl_full=new ImageLoader(workingdirectory+"2fbbowlfull/","bf"); moti_food.loadFileNames(); moti_water.loadFileNames(); bowl_empty.loadFileNames(); bowl_full.loadFileNames();

And we load the images one by one, and we put it inside the Training Set, along with the output that we want for these images:

for(int j=0;j<moti_food.fileNames.size();j++){ //load images File file=new File(workingdirectory+moti_food.fileNames.get(j)); Image img=null; try { img=ImageIO.read(file); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } ImageMLData data = new ImageMLData(img); BasicMLData idealData=new BasicMLData(outputmoti_food); training.add(data, idealData); } for(int j=0;j<moti_water.fileNames.size();j++){ //load images File file=new File(workingdirectory+moti_water.fileNames.get(j)); Image img=null; try { img=ImageIO.read(file); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } ImageMLData data = new ImageMLData(img); BasicMLData idealData=new BasicMLData(outputmoti_water); training.add(data, idealData); } for(int j=0;j<bowl_empty.fileNames.size();j++){ //load images File file=new File(workingdirectory+bowl_empty.fileNames.get(j)); Image img=null; try { img=ImageIO.read(file); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } ImageMLData data = new ImageMLData(img); BasicMLData idealData=new BasicMLData(outputbowl_empty); training.add(data, idealData); } for(int j=0;j<bowl_full.fileNames.size();j++){ //load images File file=new File(workingdirectory+bowl_full.fileNames.get(j)); Image img=null; try { img=ImageIO.read(file); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } ImageMLData data = new ImageMLData(img); BasicMLData idealData=new BasicMLData(outputbowl_full); training.add(data,idealData); }

We will make now the downsampling for the images. Normally we will use smaller images than the ones we take with the webcam or another camera, so it is necessary to do this step. We are going to use an imput layer for the Neural Network as big as pixels we have in the image. If the image is too big, this layer is going to be same size... this will lead to problems of speed for computing the images. But I encourage you to try different sizes and see your system performance. But the idea behind is that you have in your image elements, patterns that the neural network can recognize and associate with the correct output we want... most of the time we do not need a high resolution image for that matter, right?

//downsampling training.downsample(downdimensionH, downdimensionW);

And we will continue in next post creating the Neural Network structure.

[fborja]

The GatuHomeLab (4.1) - The CatEyeDetector

I have been working in a small project for fun, with many elements I just go crazy with: 

CubieTruck Debian Linux Server

And old webcam / old android smart phone

Neural Networks

Moti the cat

For very long I have been reading about neural networks, uses and possibilities. I found an amazing java framework to play with, Encog, and a project to put everything together: The GatuHomeLab!

So this is the project: we put a webcam over moti’s foodbowl, and we have to determine if she is drinking, eating, the foodbowl is empy or full. Then we will publish the status with a tweet. So if I’m not home, I just get some info about moti’s activities...

As any project with artificial vision, the system is quite sensible with light, but with good light and good material to train the neural network, results are very good (more than 80% ok hits...)

The Set Up

We have use an old webcam. Try any you may have at home, many chances it will work fine. We are using a Cubietruck with Debian Linux, so will work with Raspberry Pi or with BeagleBone Black also. 

For setting up the camera, there are two great software applications we can use. First one is MJPG-Streamer. And here you have a great tutorial about how to install and use with the webcam in the cubietruck. 

When you connect the camera, play with the online controls in MJPG-Streamer, with the flicker and other parameters to get the best image quality possible. 

Once this part is done, we will use a very useful piece of software, called “motion”. This application will use the webcam, and once detects movement (some amount of pixels change) will save snapshots in a directory of your choosing... And from them we will analyze it using the neural network.

Here are the instructions for installing motion in the Cubietruck. You will have to go through the motion.conf file, to configure the application. There you can define the directory where you want the images to be saved, the size of the image, etc.

Getting the right images

Place the camera right over the foodbowl. Moti’s have a double one, where we can put water and food. We need images to process. Bigger the images they are, more computation time we will need. We are using images RGB, so we are using the color. Why using color and not going for gray scale? I think is interesting for the neural network, to “see” the cat color, and “learn” about, so later does not get confused with our hand or feet in the image when we are going to refill the bowl. RGB images are three times bigger than grey scale images. 

In this example we are going to take images 320x240 pixels (width x height) and select a part inside of 200x120 pixels. And this subimage we will use to feed the neural network.

It is interesting to somehow try to fix the bowl to the floor, so we have always the bowl in the same position... to make it easy for training and do its job to the neural network. Chances to not identify the right situation increases if the image is not always taken in same conditions.

The objective is to get these kind of subimages:

or this one 

Getting and classifying images

Put the webcam, connect and configure motion in your Debian Linux, and let it work during some hours, so you have material to train the network. You need good images, more quantity, the better. Try also the images not to be ambiguous, because we are going to get image sets for training, and the performance of the neural network will depend on this.

You can then use FileZilla from your computer. It is a secure ftp client, that we can use to connect to our Cubietruck, and download the images. Then we can classify them in four different folders, one for every status we want the neural network to identify.

- foodbowl_empty - foodbowl_full - cat_eats - cat_drinks

When you have like minimum 50 images for each folder, then we can start training the neural network. You can train again and again, adding more images, increasing the training sets as much as you want. 

In the next post, we will train the neural network using Encog!!

[fborja]

The GatuHomeLab (3.3) - CubieTruck and BlueTooth Bee

In this post we will discuss the Arduino code for the BTbee.

We have a protocol to follow when we are speaking with the CubieTruck Linux server, and here we will see the implementation for this protocol. At the end is quite simple: the Linux server starts speaking, sending an array of characters that we have to identify in the BTBee, take out the information encapsulated inside, and answer back in same way.

Here you have the -- source code -- complete. And we can comment the interesting parts of the code.

//Sensor Dataint AnalogIn=5; //SCL pin is Analog Input 5 boolean conn=false; char btstr[80];//100 char btinfo;char trama[50]; //100 char info[40]; //80 char aux[20]; char tramainit[]={'S','T','R','\0'}; char tramaend[]={'E','T','R','\0'}; int btstrlen; boolean foodbowl=false; boolean tramafound=false; int sensingvalue=100; void loop() // run over and over { char aux[15]; //read analog signal int value=analogRead(AnalogIn); Serial.print("Analog Value: "); Serial.println(value); if(value>sensingvalue) foodbowl=true; //read available while(blueToothSerial.available()>0){ btinfo=blueToothSerial.read(); strcat(btstr,&btinfo); //btstr.concat(btinfo); //Serial.print(F("Receive BT trama: ")); //Serial.println(btstr); delay(100); }

This loop() part is listening for data (characters) arriving to the BT serial. Keeps the characters in an array, waiting 100ms between every read. You can reduce this if you need more speed.

So we are saving the characters in "btstr" (that has a length of 80 characters). This is our read buffer. Now we have to look for the subarray that stars with "STR" and ends with "ETR".

//STRIDETR //01234567 btstrlen=strlen(btstr); if(btstrlen>5){ Serial.print("--debug trama length: "); Serial.println(btstrlen); int auxindex=0; char *s; s = strstr(btstr, "STR"); if (s != NULL){ Serial.print("Found Trama at index "); auxindex=s-btstr; Serial.println(auxindex); Serial.println(&btstr[auxindex]); tramafound=true; //getting data char *p=&btstr[auxindex]; char *str; str=strtok_r(p,"#",&p); str=strtok_r(NULL,"#",&p); sensingvalue=atoi(str); Serial.print("Sensing Value: "); Serial.println(sensingvalue); }else{ Serial.println("Trama not found"); // `strstr` returns NULL if search string not found }

with the "strstr(btstr,"STR")" we get the index where this substring is located inside the buffer btstr. Remember that we have put the data between # characters. So next is to extract the characters between them, and read it as an integer. (That part of the code is under //getting data).

And now we have to answer back the Linux Server. We will put the answer in a similar array of characters, and send it, and continue the process in the loop 

if(tramafound){ //command - ID - send the sensor ID strcpy(info,"FD"); if(foodbowl==true){ ObjectID[0]='1'; strcat(info,ObjectID); }else{ //ObjectID[0]='0'; sprintf(aux,"%d",value); strcat(info,aux); } //strcat(info,ObjectID); Serial.println(F("Sending Result... ")); BTsend(); foodbowl=false; tramafound=false; } //Serial.println("Poniendo a cero btstr"); strcpy(btstr,"\0"); delay(300); //300

And finally we have the functions we have been calling to send and read data from the BlueTooth serial.

void BTsend(){ //Serial.print(F("Sending info: ")); strcpy(trama,"STR"); strcat(trama,info); strcat(trama,"ETR"); blueToothSerial.println(trama); strcpy(trama,"\0"); } //Checks if the response "OK" is received void CheckOK() { char a,b; while(1) { if(blueToothSerial.available()){ a = blueToothSerial.read(); if('O' == a){ // Wait for next character K. available() is //required in some cases, as K is not immediately available. while(blueToothSerial.available()) { b = blueToothSerial.read(); break; }if('K' == b){ break; } } } while( (a = blueToothSerial.read()) != -1){ //Wait until all other response chars are received }

In BTsend() we can see how we put the STR and ETR to the info we are sending to the Linux server.

Have fun!!!

[fborja]

The GatuHomeLab (3.2) - CubieTruck and BlueTooth Bee

To pair with the BlueTooth Bee, we can use this script

#!/bin/bash bluez-simple-agent hci0 00:13:EF:00:09:78 remove >>> '0000' bluez-simple-agent hci0 00:13:EF:00:09:78 >>> '0000'

then, when it is run, we make sure we have paired correctly the BTbee with the CubieTruck.

Communication Protocol between BTbee and CubieTruck

We need a protocol to make both parts speak. The CubieTruck, the java program, will send a command, with some information for the BTbee to process. This is interesting, because we can send for example the threshold level for the BTbee to give an alert, that we can modify at any time, in case conditions change for the detection we want to perform.

So we make the command look like this:

So we have the information in the green area, and we have a three characters start and 3 end. So it is easy to detect the piece of information we want on the other side, and extract that information.

We encapsulate the information between the three initial and final characters. We can just send a command, form by two characters, or we can send a command with a value after it.

We have to think about the protocol to follow between the two parts, so we know what are we receiving, and what is the other part expecting later.

STR#123#ETR

STR#0#ETR or STR#1#ETR

You can make this dialog as complicated as you want, sending different parameters from one to another device. 

This is the Java Class that helps to implement a protocol using this kind of information bits. To use it, we need to open  input and output stream with the other part, and give it as parameter to this class: -- source code --

import java.io.IOException;import java.io.InputStream; import java.io.OutputStream; public class LinuxBTCommunicationTrama { //variables private InputStream is; private OutputStream os; byte[] start; byte[] end; byte[] tramareceived; String sttramareceived; byte[] tramasent; int trindex=0; //constructor public LinuxBTCommunicationTrama(InputStream is, OutputStream os, byte[] start, byte[] end){ this.is=is; this.os=os; this.start=start; this.end=end; } //get/identify and return trama from inputstream public byte[] getTrama(){ //System.out.println("BTHttpTask - BTCOMM - getTrama"); sttramareceived=new String(); tramareceived=new byte[1025]; trindex=0; boolean findtrama=false; while(!findtrama){ if(!getBTbytes()) return null; //analize data String staux=new String(start); String edaux=new String(end); if(((sttramareceived.indexOf(staux)+1)*(sttramareceived.indexOf(edaux)))>1){ //WE ADD +1 BECAUSE INDEX IS 0 IS OK findtrama=true; } //System.out.println("BTHttpTask - BTCOMM - getTrama "+(sttramareceived.indexOf(staux))); } return tramareceived; } //get bytes from BT input stream private boolean getBTbytes(){ try{ while(is.available()>0){ byte[] btinfo = new byte[is.available()]; is.read(btinfo); for(int i=0;i<btinfo.length;i++){ sttramareceived+=(char)btinfo[i]; tramareceived[trindex+i]=btinfo[i]; } trindex+=btinfo.length; //System.out.println("BTHttpTask - BTCOMM - trindex-length "+trindex); //System.out.println("BTHttpTask - BTCOMM - getTrama "+sttramareceived.toString()); } } catch (IOException e) { System.out.println("-"); return false; } return true; } //send trama to outputstream - command/info public boolean sendTrama(byte[] command, byte[] info){ //lets construct the trama int Tlength=start.length+command.length+info.length+end.length; tramasent=new byte[Tlength]; System.arraycopy(start, 0, tramasent, 0, start.length); System.arraycopy(command,0,tramasent, start.length, command.length); System.arraycopy(info, 0, tramasent, (start.length+command.length), info.length); System.arraycopy(end, 0, tramasent, (Tlength-end.length), end.length); //send trama try { os.write(tramasent); os.flush(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); return false; } //System.out.println("BTHttpTask - BTCOMM - sendTrama ok: "+tramasent.toString()); return true; } //send trama only with command // STR AA ETR public boolean sendTrama(byte[] command){ tramasent=new byte[start.length+end.length+command.length]; System.arraycopy(start, 0, tramasent, 0, start.length); System.arraycopy(command, 0, tramasent, start.length, command.length); System.arraycopy(end,0,tramasent,(start.length+command.length),end.length); //send trama try { os.write(tramasent); os.flush(); }catch (IOException e) { e.printStackTrace(); return false; } return true; } //get command from current trama available //command are 2 bytes after start array public String getTramaCommand(){ return sttramareceived.substring(start.length, start.length+2); } //get info from current trama available   // STRID3ETR // 012345678 public byte[] getTramaInfo(){ byte[] info=new byte[trindex-start.length-end.length-2-2]; //we have to add -2 because of end of line characters       System.out.println("BTHttpTask - BTCOMM - tramaInfo "+trindex); System.out.println("BTHttpTask - BTCOMM - tramaInfo "+sttramareceived); System.out.println("BTHttpTask - BTCOMM - infoLength "+info.length); System.arraycopy(tramareceived, (start.length+2), info, 0, info.length); return info; } public String getTramaInfoSt(){ return new String(getTramaInfo()); } //get data received in the form // STR#12#345#ABCD#789#ETR public String[] getTramaMultipleInfoSt(){ String tramainfo=getTramaInfoSt(); String[] data=tramainfo.split("#"); //System.out.println("BTHttpTask - BTCOMM - MultipleInfo "+data.length); return data; } }

Java code for the implementation of the communication 

System.out.println("Opening Connection..."); StreamConnection conn = null; try { conn = (StreamConnection)          //Connector.open("btspp://"+log.btaddress.get(pos)+":1");  Connector.open("btspp://"+log.btaddress.get(pos)+":1",Connector.READ_WRITE,true); } catch (Exception e1) { // TODO Auto-generated catch block e1.printStackTrace(); return; } System.out.println("Opening Input/Output Streams..."); OutputStream out = null; InputStream in=null; try { out = conn.openOutputStream(); in = conn.openInputStream(); } catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); System.out.println("Error opening streams "+e.toString()); return; } System.out.println("Connection with BTBee OK!"); System.out.println("Sending TramaID..."); LinuxBTCommunicationTrama lbt=new LinuxBTCommunicationTrama(in,out,start,end); String minValue=((Integer)(log.sensingvalue.get(pos))).toString(); minValue='#'+minValue+'#'; //lbt.sendTrama(tramaID); lbt.sendTrama(minValue.getBytes()); lbt.getTrama(); String stat=lbt.getTramaInfoSt(); System.out.println("Received from BTBee -- sensor Status: "+stat); if(stat.equals("0")){ System.out.println("No interaction"); log.createDBreport(log.sensorid.get(pos), "report"); }else if(stat.equals("1")){ System.out.println("Cat Detected!!!!"); log.createDBreport(log.sensorid.get(pos), log.sensorname.get(pos)); }else{ System.out.println("No interaction"); log.createDBreport(log.sensorid.get(pos), "report"); } try { in.close(); out.close(); conn.close(); } catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); return; } System.out.println("Connection End");

In next post we will see how it is the Arduino programm to speak with this code from the BTbee.

[fborja]

The GatuHomeLab (3.1) - CubieTruck and BlueTooth Bee

Let’s use the software installed to detect the BTBee. #hcitool scanScanning ...        00:13:EF:00:09:6D       BTB01-st    <string.h>I think it is interesting the idea of a system of small sensors reporting to a central Hub, analyzing and taking actions with the information collected. 

I have a couple of BlueTooth Bee (BTBee) Standalone from SeeedStudio. This is a nice device, has bluetooth connectivity and an ATMega168. 

Here we can put a small program for getting data and communicate with the CubieTruck. The CubieTruck is a development board with Wifi, 2 USB ports, and BlueTooth onboard chip. Is perfect to be the “brain” for this project, where different sensors will go and report data to it. We have install a Debian server and works really nice.

We need BlueCove for Linux. You can find info of how to install it here . This software allow us to use bluetooth in Linux. Later we will also need native libraries for our java program.

First lets install the packages for linux (you have to bee root and with internet connection to download and install the packages):

#sudo apt-get update #sudo apt-get install bluetooth bluez-utils blueman

BTBee first configuration sketch

We will use the the BlueTooth in slave configuration, with pin. Here is the code:

#include <SoftwareSerial.h> //Software Serial Port #include <string.h> #define RxD 2 #define TxD 3 SoftwareSerial blueToothSerial(RxD,TxD); int AnalogIn=5; //SCL pin is Analog Input 5 void setup(){ Serial.begin(9600); //BlueTooth pinMode(RxD, INPUT); pinMode(TxD, OUTPUT); //Serial.println("Starting BT..."); setupBlueToothConnection(); } void setupBlueToothConnection() { //Serial.println(F("..")); blueToothSerial.begin(38400); //Set BluetoothBee BaudRate to default baud rate 38400 delay(1000); sendBlueToothCommand("\r\n+STWMOD=0\r\n"); //device working as slave (0) master (1) sendBlueToothCommand("\r\n+STNA=BTB01-st\r\n"); //device name sendBlueToothCommand("\r\n+STAUTO=0\r\n"); // 0 // permit auto connect (1) sendBlueToothCommand("\r\n+STOAUT=1\r\n"); // not permit paired device to connect me (0) sendBlueToothCommand("\r\n+STPIN=0000\r\n"); // pin code //sendBlueToothCommand("\r\n+LOSSRECONN=0\r\n"); // permit auto-reconnecting delay(2000); // This delay is required. sendBlueToothCommand("\r\n+INQ=1\r\n"); // enable been inquired (1) when slave by master delay(2000); // This delay is required. Serial.println(F("setupBTconn: listening...")); while(!conn){ if(blueToothSerial.available()>0){ CheckOK(); conn=true; Serial.println(F("SetupBTconn: ok")); } } } void loop() // run over and over { //read analog signal int value=analogRead(AnalogIn); Serial.print("Analog Value: "); Serial.println(value); delay(300); //300 }

Load the program in the BTBee. When the configuration arrives to INQ=1, the BTBee will start blinking with red and blue leds, indicating that is waiting for bluetooth connection.

Getting the BTBee bluetooth address and pairing the device

Let’s use the software installed to detect the BTBee.

#hcitool scan Scanning ...        00:13:EF:00:09:6D       BTB01-st    

There we have the address. The name we have given to the BTBee will help us identify the device, even we have more BlueTooth devices presents in discovery mode. And next is to pair the Cubietruck with the BTBee, so we can later make the connection using the java program.

root@donpimpon:/dev# bluez-simple-agent hci0 00:13:EF:00:09:6D RequestPinCode (/org/bluez/2274/hci0/dev_00_13_EF_00_09_6D) Enter PIN Code: 0000 Release New device (/org/bluez/2274/hci0/dev_00_13_EF_00_09_6D)

It will ask for the PIN code, that is ‘0000’ as we have indicated in the arduino sketch. It is also useful the command 

root@donpimpon:/dev# bluez-simple-agent hci0 00:13:EF:00:09:6D remove

to release the device, and then pair it again with the first command. This would be necessary in case the BTBee has restart for any reason.

[fborja]

The GatuHomeLab (2.2) - BeagleBone Black I2C Communication in Java

In this post we will make an executable jar file with Eclipse, and execute it in the BeagleBone Black. 

Create a java project in Eclipse, and remember to add the LibBulldog jar to to the project as an external jar.

We have to create a class with a public static void main method, that is the entry point for the executable jar. From there we call our classes (the ones created in the previous post).

This is code for this last class  -- source code --

package linux.java.process; import com.sensorlab.devices.ADCi2c_Accelerometer; public class AccelerometerBBB { public static void main(String[] args) { ADCi2c_Accelerometer acc=new ADCi2c_Accelerometer(0x50,"I2C_1"); acc.init(); for(int i=0;i<100;i++){ System.out.println("Accelerometer Value: "+acc.getValue()); } } }

And we have to click now in the Run button:

Nothing will happen, but it is important later to create the executable jar file. Make right click on the file name on the left, and select Export... There choose the Runnable Jar File. In Launch Configuration you have to select the AccelerometerBBB (is the name of the main class we have create) - this option appears when we have click run in the main screen - 

Select the option to copy the libraries to a sub-folder. 

We have to transfer the jar file and the directory created to the BBB.

Attention!!: Using the bulldog.beagleboneblack.hardfp-0.1.0 library for Debian, include the file libbulldog-linux.so in the directory in the BBB. These are the native libraries that the libbulldog will call in the BBB to access the GPIO and the I2C bus.

You will have this files structure in your BBB

accelerometerBBB.jar accelerometerBBB_lib/ (directory)       bulldog.beagleboneblack.hardfp.jar       libbulldog-linux.so

then just execute the program as

# java -jar accelerometerBBB.jar

[fborja]

The GatuHomeLab (2.1) - BeagleBone Black I2C Communication in Java

The BeagleBone Black maximum voltage we can apply to the pins for analog to digital converter is 1.8V. As many outputs from analog devices are 3 to 5V, I think it is better, also to protect the board, to use an I2C ADC converter. 

This is what we will use for our example.

- a BeagleBoneBlack Rev C. - An I2C ADC 12 bits (Seeedstudio link) - An Accelerometer / or another analog device to measure

For the example we will use a three axis accelerometer, but any device with analog output will work fine in this case. 

BeagleBone Black Operating System

We are using Debian, with latest version from http://beagleboard.org/latest-images

BeagleBone Black I2C

We have an explanation in this web page (http://datko.net/). Basically we have to identify the port and the linux address...

To do that we have to take into consideration that the BBB has three I2C buses:

i2c0: Not exposed in the expansion headers / 0x44E0_B000

i2c1: pins P9 17,18 (and 24,26) / 0x4802_A000

i2c2: pins P9 19,20 (and 21,22) / 0x4819_C000

Let's check which one we can use...

root@beaglebone:~# ls -l /sys/bus/i2c/devices/i2c-* lrwxrwxrwx 1 root root 0 Jan 1 2000 /sys/bus/i2c/devices/i2c-0 -> ../../../devices/ocp.2/44e0b000.i2c/i2c-0 lrwxrwxrwx 1 root root 0 Jan 1 2000 /sys/bus/i2c/devices/i2c-1 -> ../../../devices/ocp.2/4819c000.i2c/i2c-1

Here we can see that the 4819c address, that correspond to the I2c2 bus (using pins P9 19 and 20), it is mapped in our Linux BBB as i2c-1.

Connect the ADC as per the previous image, taking care to connect properly SCL and SDA to P9-19 and P9-20 respectively.

Then we can check with the command i2cdetect if we can see the ADC (in the command we indicate "1" because we are using the i2c-1 in linux)

root@espinete:/home/borja# i2cdetect -r 1 WARNING! This program can confuse your I2C bus, cause data loss and worse! I will probe file /dev/i2c-1 using read byte commands. I will probe address range 0x03-0x77. Continue? [Y/n] Y 0 1 2 3 4 5 6 7 8 9 a b c d e f 00: — — — — — — — — — — — — — 10: — — — — — — — — — — — — — — — — 20: — — — — — — — — — — — — — — — — 30: — — — — — — — — — — — — — — — — 40: — — — — — — — — — — — — — — — — 50: 50 — — — UU UU UU UU — — — — — — — — 60: — — — — — — — — — — — — — — — — 70: — — — — — — — —

Attention: The I2C ADC that we are using has the default address of 0x55. The BeagleBone Black has reserved this address for some EEPROM communication... so we can not use it. We had to change the I2C address to another one outside that range, to 0x50. Then we can see it answering in the previous example ("50"). 

The Java GPIO Library

To access the pins and the I2C bus from our java code, we will use LibBullDog. This is a wonderful library, that will allow us to interact with the devices attached to the BBB, and give us the power of java programming language in a Debian Linux environment.

We will make our own package with wrappers to use the library and adapt it to our code. To speak with an I2C device, we need to follow a protocol. That means that many times we have to write commands in certain registers, and configure the device to ask for readings. Then it is very useful to make these wrappers, where you put all this work, as a black box, and later you just take care of the project you are working in. 

The ADC is based in the ADC121C021, so we have to find the datasheet and read it to find out the protocol. We have also the code for Arduino in the Seedstudio page where it also shows how it works, and we can use it to make our own in Java.

Here the link to the files for this post -- source code --

public class ADCi2c extends I2cDevice{ byte REG_ADDR_RESULT=0x00; byte REG_ADDR_CONFIG=0x02; //constructor public ADCi2c(I2cBus bus, int address) { super(bus, address); } //constructor public ADCi2c(I2cConnection connection) { super(connection); } //methods public void initMode() { try { this.open(); System.out.println("ADCi2c::initMode:: open ok"); this.writeByte(REG_ADDR_CONFIG); //select the CONFIGURATION REGISTER OF THE ADC System.out.println("ADCi2c::initMode:: write byte 0x02 ok"); this.writeByte(0x20);//set sampling rate to 27ksps System.out.println("ADCi2c::initMode:: write byte 0x20 ok"); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); System.out.println("ADCi2c::initMode::error "+e.toString()); } } public void closeMode() { try { this.close(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); System.out.println("ADCi2c::closeMode::error "+e.toString()); } System.out.println("ADCi2c::closeMode:: close ok"); } public int readIntValue() { byte[] buffer = new byte[2]; try { this.writeByte(REG_ADDR_RESULT); this.readBytes(buffer); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } int intValue = (((BulldogUtil.getUnsignedByte(buffer[0]) << 8)) | BulldogUtil.getUnsignedByte(buffer[1])); return intValue; } }

The class extends the I2cDevice from LibBulldog. The construction it is done telling which bus we are using, and the address for the device. Then we use the initMode() to configure the device, and we can then start reading values. When we are done, we just close with closeMode().

Using the wrapper

We are connecting an accelerometer to the ADC. So we can make now a class to use the previous one, and get the data as we please, the ADCi2c_Accelerometer. 

Here the link to the files for this post -- source code --

public class ADCi2c_Accelerometer{ I2cBus bus; int address; String I2Cbus; ADCi2c adc; String I2C; public ADCi2c_Accelerometer(int address, String I2C) { this.address=address; this.I2C=I2C; } /* * Starts the communication with the ADC * getting ready for reading from the Accelerometer * */ public void init() { final Board board = Platform.createBoard(); I2cBus bus = board.getI2cBus(I2C); adc=new ADCi2c(bus,address); adc.initMode(); } /* * Close the communication with the ADC * */ public void close(){ adc.closeMode(); } /* * Get Accelerometer value from ADC * */ public int getValue(){ return adc.readIntValue(); } }

In the next post we will see how to make the executable jar file and put it in the BeagleBone Black.

[fborja]

The GatuHomeLab (1)

I want to check on Moti when I go out. Because I keep thinking "what she must be doing?" "Has enough water and food?" "Keeps singing whole night waking up the neighbors?" "Has she finished the water already, or thrown it away just playing?"

"I'm ok when you go. just remember to leave the computer on so I can order some chinese noodles!"

yeah... sure, Moti... you keep scratching the bill notes and you don't know yet how to open the main door... 

Let's make a system to detect the cat movements around the house... and we can start by the Food Bowl! We can use a sensor to detect the cat there, and then give an alert... like a Twitter message!!!! ooooh... this is a good and funny idea! 

We will use the new BeagleBone Black just got for Christmas, and the Cubietruck as a server. There we will have a Java Tomcat server with MySQL, and we can control the cat access to the Food Bowl at any time... 

is going to be just AMAZING!!

"I think this goes against cat privacy, and I will not stop fighting till the GatuHomeLab is away from my fellow cats..."

Here some screenshots for the web application... 

[fborja]

The Overheated Router Dilema - Part 4

Let's continue commenting about the code for the wifi bee.

Here the next part of the code:

//-------------------- //variables by default. will be initialize uint8 PBip[4] = {192,168,1,1}; char PBhostName[] = "192.168.1.105"; char PBurl[] = "/sensorlab/slwifimicro"; //vssensorlab POSTrequest postCollectivity(CollADDip, 8080, PBhostName, PBurl, feedData);

PostRequest is the function we call when we want to communicate with the server. Here we have declared the function and pass the parameters. 

feedData, the last parameter, is the function we use to make this communication. Will be clearer in the following lines of code:

char trama1[200]; char aux[15]; char data1[200]; char tramaok[]="mssg=ok"; char trama66[]="mssg=66"; char trama33[]="mssg=33"; char tramakl[]="mssg=kl"; char tramaon[]="mssg=on"; char tramaoff[]="mssg=off"; //to report object data to server void feedData(){ Serial.println(trama1); WiServer.print(trama1); }

here we have char arrays for the communication with the server. The different tramaXX arrays are to identify the answers from server and change between different states. This is the function that receives the message from the server, and has to process to move the wifibee from one state to the next.

void printData(char* data, int len) { data[len]='\0'; Serial.println(data); //process status if(estado==8){ if(strncmp(tramaok,&data[len-7],7)==0) estado=1; } else if(estado==7){ if(strncmp(tramaok,&data[len-7],7)==0) estado=7; //ok if(strncmp(tramakl,&data[len-7],7)==0) estado=1; //kill measuring if(strncmp(tramaon,&data[len-7],7)==0) digitalWrite(pd6, LOW); //switch on if(strncmp(tramaoff,&data[len-8],8)==0) digitalWrite(pd6, HIGH); //switch off } else if(estado==5){ char *p=data; char *str; str=strtok_r(p,"#",&p); str=strtok_r(NULL,"#",&p); sTime=atol(str); str=strtok_r(NULL,"#",&p); duration=atol(str); str=strtok_r(NULL,"#",&p); samples=atoi(str); if(sTime*duration*samples>0) estado=6; } else if(estado==4){ char *p=data; char *str; str=strtok_r(p,"#",&p); str=strtok_r(NULL,"#",&p); unsigned long t=atol(str); setTime(t); Serial.print("Time: "); Serial.println(t); if(t>0)estado=5; } else if(estado==1){ if(strncmp(trama66,&data[len-7],7)==0) estado=4; if(strncmp(trama33,&data[len-7],7)==0) estado=3; } }

This is the setup() part

void setup() { // Enable Serial output and ask WiServer to generate log messages (optional) Serial.begin(9600); WiServer.init(NULL); //function is called when data is returned by the server postCollectivity.setReturnFunc(printData); pinMode(pd6, OUTPUT); digitalWrite(pd6, HIGH); estado=1; }

In the setup we make printData the function to take care of the answers from the server.

And this is the function we use to read the temperature from the serial connection.

float readTemperature() { byte i; byte present = 0; byte data[12]; byte addr[8]; if ( !ds.search(addr)) { ds.reset_search(); delay(250); } ds.reset(); ds.select(addr); ds.write(0x44,1); // start conversion, with parasite power on at the end delay(1000); // maybe 750ms is enough, maybe not // we might do a ds.depower() here, but the reset will take care of it. present = ds.reset(); ds.select(addr); ds.write(0xBE); // Read Scratchpad for ( i = 0; i < 9; i++) { // we need 9 bytes data[i] = ds.read(); } byte MSB = data[1]; byte LSB = data[0]; float tempRead = ((MSB << 8) | LSB); //using two's compliment float TemperatureSum = tempRead / 16; return TemperatureSum; }

And finally here we have the loop() part

//--------------------------------------------------------------------- //--------------------------------------------------------------------- time_t t1=0; time_t t2=0; int iteration=0; //--------------------------------------------------------------------- //--------------------------------------------------------------------- //--------------------------------------------------------------------- void loop(){ Serial.print(F("ESTADO=")); Serial.println(estado); if(estado==1){ strcpy(trama1,"sid="); strcat(trama1,ObjectID); strcat(trama1,"&cod=55"); postCollectivity.submit(); delay(20); } if(estado==3){ delay(5000); //5000 estado=1; } if(estado==4){ strcpy(trama1,"sid="); strcat(trama1,ObjectID); strcat(trama1,"&cod=77"); //connect to Collectivity postCollectivity.submit(); delay(20); } if(estado==5){ strcpy(trama1,"sid="); strcat(trama1,ObjectID); strcat(trama1,"&cod=88"); //connect to Collectivity postCollectivity.submit(); delay(20); } if(estado==6){ if(now()>sTime){ estado=7; t1=now(); } } if(estado==7){ if((now()-t1)>(duration/samples)){ strcpy(trama1,"sid="); strcat(trama1,ObjectID); strcat(trama1,"&cod=99&samp="); //code 99 - value is directly magnitude value float temp=readTemperature(); int tempdec=(temp-(int)temp)*100; sprintf(aux,"%0d.%d",(int)temp,tempdec); strcat(trama1,aux); Serial.println(trama1); //connect to Collectivity postCollectivity.submit(); t1=now(); iteration+=1; } if((iteration-1)==samples) { estado=8; iteration=0; } } if(estado==8){ strcpy(trama1,"sid="); strcat(trama1,ObjectID); strcat(trama1,"&cod=AA"); //connect to Collectivity postCollectivity.submit(); delay(20); } WiServer.server_task(); delay(20); }

As we can see, we move through different states while we speak with the server. In state=7 is where we are measuring the temperature and sending to the server. If server detects it is over the threshold temperature, will send the order to activate the output to close, and when the temperature has gone down enough, will give order to deactivate the output again...

Are you happy, Mothi? did you like the gadget? "is quite silly..." 

[fborja]

The Overheated Router Dilema - Part 3

To make a program for the Arduino that checks temperature, and just activate and deactivate the relays when temperature is over certain value and below another one, is quite easy. We want to go beyond... 

Remember that we have a Wifi Bee working here... because we want to have the device connected to our home network, and we want to be able to monitor the temperature, and set the max and min temperature for switching on and off the router and the fan.

So this is the system that we will use, with Tomcat and MySQL. I love this combination... is really powerful. and you can make it work everywhere... 

So we have a servlet listening, and we have to send using a POST, information to be process by the system, and send back an answer for the device to act accordingly.

- The device reads temperature and sends to servlet - Servlet answers ok, or ask to activate the fan or cut the power line to the router

Let's go step by step - 1. The Wifi Bee

Better we start from the software for the Wifi Bee then we go for the servlet, other useful classes and the database.

This is the Arduino program for the Wifi Bee (http://www.seeedstudio.com/wiki/Wifi_Bee) that we can comment.

Here you have the headers and the wireless configuration parameters. Just as they told us in seeedstudio how to do it. 

#include <WiServer.h> #include <Time.h> #include <OneWire.h> #define WIRELESS_MODE_INFRA 1 #define WIRELESS_MODE_ADHOC 2 // ---------------------------------Wireless configuration parameters ---------------------------------------- unsigned char local_ip[] = {192,168,1,150}; // IP address of WiShield unsigned char gateway_ip[] = {192,168,1,1}; // router or gateway IP address unsigned char subnet_mask[] = {255,255,255,0}; // subnet mask for the local network const prog_char ssid[] PROGMEM = {"HomeNet"}; // max 32 bytes unsigned char security_type = 2; // 0 - open; 1 - WEP; 2 - WPA; 3 - WPA2 // WPA/WPA2 passphrase const prog_char security_passphrase[] PROGMEM = {"XXXXXX"}; // max 64 characters // WEP 128-bit keys // sample HEX keys prog_uchar wep_keys[] PROGMEM = {0x01, 0x02}; // setup the wireless mode // infrastructure - connect to AP // adhoc - connect to another WiFi device unsigned char wireless_mode = WIRELESS_MODE_INFRA; unsigned char ssid_len; unsigned char security_passphrase_len; // ----------------------End of wireless configuration parameters ----------------------------------------

Next we have to give an ObjectID to the sensor, so the server can differentiate between different sensors working in different tasks

//--------------------------------------- char ObjectID[]="1"; //1 - temperature serial dallas char CollectivityADD[]="192.168.1.104"; uint8 CollADDip[4]={192,168,1,104}; //--------------------------------- //Working program status int estado=0; //-------------------- //Sensor tasks unsigned long duration=0; int samples=0; //---------------------------------------

CollectivityADD is the IP address of the server, where the wifibee will go to send and receive commands.

estado=0 is the state in which the program is at certain point of times. We are going to use state machine paradigm to work with the wifibee. The idea is that we can not go to the next state till certain conditions are not accomplished, or tasks performed. Then we change to the new state, and so on, till we come back to the zero or initial state. 

In our system, the sensor is the one starting the communication, and the server will answer with tasks or commands to perform. This makes the system easy to scale up, adding many sensors and manage them all at the same time...

Sensor tasks has to variables, duration and samples, that will save the duration in milliseconds of the data acquisition, and the number of samples to take during that time

/* DS18S20 Temperature chip i/o */ OneWire ds(5); // on pin PD5 int pd6=6; //relay digital output

To speak with the Digital Temp Sensor (DS18B20, from Dallas) we use the pin 5, and to control the digital relay we use the pin 6 (we will configure the pin properly later on in the code)

We can continue in a next Post!!!

[fborja]

The Overheated Router Dilema - Part 2

Let's make the schematic!!!! what we will use!!!???

This is the list

Wifi Bee (http://www.seeedstudio.com/wiki/Wifi_Bee)

Power Supply with 3.3V and 5V outputs

Relay (http://www.seeedstudio.com/wiki/Grove_-_Relay) 

Digital Temp Sensor (DS18B20, from Dallas)

READING TEMPERATURE

<<why don't you use something like a thermistor or something like that!! is very complicated this digital temperature sensor!!>> What is this!!?? a cat that discuss my components choosing? where did you study electronics, Mothi? on BBC? watching Open University episodes in the morning, with people with strange-funny hair and horrible ties???

<<Borja, you are an idiot. I'm going to sharp my nails in the sofa...>> The Dallas one wire temperature sensor is a wonder!! and you do not need much code to make it work... is very stable, robust, looks nice and not very expensive. I just hate thermistors... anyway.

This is how we should make the connection... the communication line of the device with the Wifi Bee has to be using a resistance with the main voltage. In our case 3.3V (because the Wifi Bee works with 3.3V).

Then you can check this place to get the OneWire Library for the Arduino. 

I assume you know how to add a library to Arduino... but you can find info here if you need it. 

#include <OneWire.h> float readTemperature() { byte i; byte present = 0; byte data[12]; byte addr[8]; if ( !ds.search(addr)) { ds.reset_search(); delay(250); } if ( OneWire::crc8( addr, 7) != addr[7]) { Serial.print("CRC is not valid!\n"); return; } ds.reset(); ds.select(addr); ds.write(0x44,1); delay(1000); // maybe 750ms is enough, maybe not present = ds.reset(); ds.select(addr); ds.write(0xBE); // Read Scratchpad for ( i = 0; i < 9; i++) { // we need 9 bytes data[i] = ds.read(); } byte MSB = data[1]; byte LSB = data[0]; float tempRead = ((MSB << 8) | LSB); //using two's compliment float TemperatureSum = tempRead / 16; return TemperatureSum; }

With this code you can get easy the temperature from any point in the loop() process.

USING THE RELAY

This Relay is working at 5V, that's why wee need a power supply in the project that can give 5V also. We feed the relay, and then for the signal we will use one digital output from the Wifi Bee, the PD6.

You know how a Relay works... it has two terminals (the green ones) that will be connected when we rise the PD6 to HIGH (3.3V).

ATTENTION!! - To work, we have to connect the Ground for the 3.3V and the 5V power sources, if not, will obviously not work the gadget... Referenceeeee is the key!!!

image is from SeeedStudio 

What else we neeed???? just to put everything together. MOTHI!!! what are you doing with the curtains??

Sorry, I didnt found the power supply in Fritzing that we are using, so I have put some batteries... just details... mmmm. 

<<Borja, you are sooo lazy sometimes...>> And you, Mothi, so annoying singing at 3am in the morning...