If I havenāt already wrote this before or announced it yet, then Iāll first like to start this post by announcing that Iāll be heading back to Singapore some time next month for about 2.5 weeks. The (return) flight ticket has already been booked for just a mere Ā£500 (it was ~Ā£900-Ā£1100 when I checked last November/December!), but I have to deal with a 7 hours transit at Doha airport. But thatās fine, because Iāll have my laptop with me and Iāll be busy using the (overall) 40 hours to-and-from journey to write my dissertation report, which is also the topic of this post. Efficient time management much?
The weather is starting to become cooler this week, but I donāt think itāll start to actually turn cold until mid-August, or early September. Iāll still be coming back here for a couple of months to carry on with my job hunt ā so far, Iāve sent out my CVs to various parts of the world ā like Denmark, Japan, Germany and even in Thailand.Ā The process to get a working visa is hard, but Iām somewhat confident that with my newfound technical skills and expertise, I should be able to get what I want.
14th Jul Sun
Groceries: Ā£5
Total spending: Ā£2538.85
15th Jul Mon
Total spending: Ā£2538.85
16th Jul Tue
Groceries: Ā£12
Total spending: Ā£2550.85
17th Jul Wed
Total spending: Ā£2550.85
18th Jul Thu
Lunch: Ā£4.70
Dinner: Ā£10.40
Total spending: Ā£2565.95
19th Jul Fri
Groceries: Ā£6
Total spending: Ā£2571.95
20th Jul Sat
Dinner: Ā£9
Total spending: Ā£2580.95
I actually wanted to write a short introduction about Neural Networks (which is highly related to my dissertation project), but at the moment, I donāt understand it enough to write much about it. In fact, I donāt even think Iām qualified to write about it at the moment, and Iām not sure how long itāll take for me to understand it to a āpassableā level.
Anyway, the main task of my dissertation project ā and what I was working on for the past month and the coming 1.5 months ā is to implement an algorithm that can detect and draw bounding boxes around all the insects in a given image. The insects in these images can be sparsely distributed or densely packed together. For the former, itās not too hard to detect with the right parameters and pre-processing ā but of course itās not so simple either, like just using template matching. For the latter, I totally have no idea how to approach the problem.
To give a better idea, for the image below, itās easy for me to make the computer detect the exact location of where the insects are because itās so obvious. A simple find edges and contours algorithm will get me an accuracy of >50%, with a few false positives here and there on the wooden frame, and on the labels. Just last week, I finally found a way to measure the accuracy of the algorithm. Now, the challenge for me is to pump it up by 90% by finding the right hyperparameters (I hope Iām using the term correctly here).
Of course, I wish that my dissertation were that easy. The same algorithm that works on the above image will not work on the images below. The caveat is that Iām only allowed to use automatic methods only ā that means that itās not like I can just make a GUI to get the results that I want. So far, Iāve just started having all my packages and Iāve got a few algorithms on my list to try. Implementing them, however, is another problem.
Either way, Iām sorry that I cannot even provide a simple explanation of how Neural Networks work at the moment, but I can give a very brief overview on the image classification/Computer Vision process. This is by no means a detailed explanation of the overall process, but rather, what I understand from the process off the top of my head.
Computer Vision is a really vast topic and there are so many things that you can explore at every step of the way. Hell, I can even spend 3 months on a research topic about the best pre-processing methods and write a 30-50 page report about that.
Pre-processing
In any image analysis/computer vision project, the first step is to pre-process the image. There are a lot of things that you should do to an image first ā like resizing it (simply because a 256Ć256 image will process much faster than a 3200Ć3200 image), removing the noise (usually, this means smoothing the image using some kind of blurring filter), and segmentation (the simplest being thresholding, to create a āmaskā).
Typically, this step isnāt very hard. You donāt even need to apply any coding at this stage ā even using Adobe photoshop or Microsoft Paint to crop the image or re-colorize some parts is already considered āpre-processingā. I tend to use ImageJ to try different filters on the image first to check the result, and then deploy the code on Python to get the same results.Ā
Algorithms
This part is the reason why Iām posting this so late ā it takes FOREVER to train the CNN model, but I think itās important to include this since itās the whole point of this post.Ā
By far, the gold standard for any image classification/detection problem is the use of Neural Networks, which Iām trying to implement on my project (and Iām pleased to say that after many months of getting stuck in the installation process, I finally got TensorFlow and Keras up on Python. You see, Iāve been using Python 3.7 all these while and itās not compatible with those packages for some reason. Then, I downloaded Python 3.5, and while I could make TensorFlow run, I had some problems with OpenCV. I was stranded there until just yesterday where I downloaded Python 3.6 and shifted all my packages and lo-and-behold, it works!).
If youāve been following the news recently, thereās this āFaceAppā application thatās been going around that makes your face look older than you currently are. The magic behind it is something called āGenerative Adversarial Networksā (GAN), which are a type of neural network. Of course, Iām not using GAN in my project, so thereās no need for me to find out more about it yet. Anyway, the key thing here is that Neural Networks are like, the hottest thing in data science right now. In the last 5 years or so, almost every image classification problem will first approach the problem using some form of neural network algorithm, and the results are very promising ā with about an 85-95% accuracy.
Anyway, I copy-pasted a code from the TensorFlow tutorial website and trained it on a built-in sample dataset. The training took ~5 minutes (may be faster if I just ran it from the console) and the result isā¦wrong. It classified the image as a Coat with 100% probability, when it actually is an Ankle Boot. If you are interested to go through the whole process yourself, you are welcomed to try your luck at https://www.tensorflow.org/tutorials/keras/basic_classification.
BUT, as my professor told me, the problem with these deep learning techniques are that the āhard partā ā the intermediate step of actually processing and applying the algorithm to the image ā are already done for you by the experts. These days, you just need to install the packages, feed the correct input to the pre-written algorithm, train the model (which is automatic), and finally get the output. Then you just check if the output is correct or not and get some sort of a truth table.
Unfortunately, for a graded project, itās hard to give good marks for this because you are not actually āthinkingā or coming up with your own solutions, but just piggy-backing on the works of others. Thereās no merit to give for just downloading images and packages!
Computer Vision is something that Iām really interested in, but unfortunately, I did not have the chance to take up a module on this in my undergraduate or postgraduate education ā other than a small, small part in my Biomedical Imaging module about simple thresholding. Everything that I have learned about this topic is purely from online sources, A LOT OF TRIAL AND ERROR, and a bit of luck (but Iāve been very lucky with my results so far).
On the other hand, seeing that I didnāt do too well for my signals and controls modules, the nitty-gritty details in Computer Vision might just be too maths-intensive for me to comprehend.
Last but not least, although Iām still stuck at Andrew Ngās Machine Learning course on Coursera, I think I managed to learn some similar content in my Database Systems module, which takes a more theoratical approach, rather than focusing on the mathematical details. I highly recommend anyone who has a month or so to enrol in this course because itās perfect for absolute beginners ā you just need the basic discipline and time to get through it, especially so if you are bad at maths.
Results
Finally, the last part of any image classification problem is obtaining the results of the classifier ā that is, how well or not well did the algorithm perform. At first, I thought it was something so simple like just getting a truth table by counting the number of algorithm-generated matches and comparing if it matches with the ground truth rectangles, but it was more complicated than I assumed that it took me 2 weeks to figure it out!
Even at this step, there is no one fixed way to get the accuracy of the algorithm ā and this all depends on the images that you are dealing with, like how many samples are there in an image? Say you have a dataset of 50 cat images and 50 dog images ā each image contains either only 1 cat or 1 dog.
You have an algorithm that determines if a cat is present in the image. You apply your algorithm and you will have a list of cat images that are correctly identified as cat (true positive), cat images that are incorrectly identified as not cat (false negative), not cat incorrectly identified as cat (false positive) and not cat correctly identified as not cat (true negative).
Okay, this is still a little confusing to me, even if I have been dealing with a question like this for every year since polytechnic, but I think I got it correctly (pardon me if I made a mistake ā how shameful!). Anyway, you sum up all these numbers and you get some sort of a final value from 0 to 1 that tells you how good your model is ā in the best case scenario, all of your 50 cat images are correctly determined as cat images, and all of your 50 dog images are correctly determined as not cat images. TP = 50, FP = 0, TN = 50, FN = 0.
You can just find the accuracy using the simple formula of TP + TN / (TP + FP + TN + FN), which you will get 1.
In my case, itās not so simple because I have many different kinds of boxes in one image ā how do I calculate the accuracy of my algorithm?Ā
The quick answer is that I used something called the āJaccard Indexā to get the overlapping areas, and from there, get the number of True Positives, False Positives and False Negatives (True Negatives are not taken into account because there is simply no way to get them, unless perhaps you count it at a pixel-level). Then, I find the Precision and Recall for the algorithm and finally calculate the F1 score, which is the measure of how āgoodā my algorithm is.
Iām going to stop here because I really didnāt think that Iāll be writing about something so technical, and besides, I have better use for my time anyway, like ACTUALLY working on my project instead of writing a blog post about it. On the side, Iām also watching 1 or 2 lessons on YouTube every day about Neural Networks, and I highly recommend this as an advance course after taking Andrew Ngās Machine Learning course.
1 YEAR IN CARDIFF ā WEEK 43 āĀ PROJECT If I haven't already wrote this before or announced it yet, then I'll first like to start this post by announcing that I'll be heading back to Singapore some time next month for about 2.5 weeks.
