Future Vision

Greetings ! My name is Andrew Mendez and this blog is about my work in designing intelligent, interactive technology. My interests are in Computer Vision, Augmented Reality, and Artificial Intelligence.

[interactivetech-blog]

Never Give Up

Greetings world, much has happened since I last posted and I apologize for not keeping you all up to speed. It seems right now I have hit an impasse, I have reached a huge roadblock. I applied to attend MIT for graduate school, and I was rejected. I know attempting to go there was an almost impossible feat, but that dream enabled me to push myself to end up working at MIT for two summers. During those summers, I worked with the brightest minds in the fields of augmented reality, and the internships gave me such a strong chance to go there. At first I was really bitter, however around this time, another friend notified me of his rejection to the summer research program at MIT, the one I have attended these past two summers. 

I really thought about what to say to him in this feeling of rejection, and it took me to deal with a really hard rejection to determine what to say to him. Ultimately I said this: 

"Things unfortunately don't go our way, and it sucks. But don't let this make you think your not good enough or not smart enough to be there. You are, the oppurtunity is just not there at the moment. In spite of my rejection, I am not going to let this stop my dream of going there, nor looking if there are any other opportunities that are just as good or even better. Don't ever give up, nor ever stop pushing yourself. It is that motto that has gotten me where I am, and it is going to take me where I want to be."

As I do not know where I will end up or where I will be, I will never forget my moto: "Never stop pushing myself, and never give up." I still have this vision of developing tools to augment the human intellect, and I am going to pursue it. I will give it my all and make it a reality.

[interactivetech-blog]

Yes We Can. But Should We?

The unintended consequences of the maker movement

A reason why design is so relevant. Instead of asking what can I build, ask  ask why are you building it.

[interactivetech-blog]

Fearless Genius

[interactivetech-blog]

Created by Nadezda Suvorova, MIKMA an exploration game for iOS that uses computer vision and the sensors embedded in the device to call for new visual perspectives of world around us using scientific imaging.

[interactivetech-blog]

My Intel Make It Wearable DREAM IT Visionary Submission. It is about my concept of the Wearable Intelligent Interface.

Wearable computing is a new tool that can amplify our human vision. By incorporating machine vision systems into wearable computers, it can help us gain a better perception of our world and a more insightful way to interact within it.

Enjoy and comment to let me know what you think.

[interactivetech-blog]

The Dilemma / Future Vision of Google Glass

Greetings World,

Today I wish to discuss the current standings of google glass as well as the biggest problem that Glass is facing. Google Glass has been at the top of monitored tech coverage and consumer hype because of the radical shift it carries. A device that has the same functionality and processing power as your smartphone and designed to be worn on your face opens new possibilities to explore new applications and really define the future of wearable computing. It has captivated people's curiosity as it raises new possibilities as well as new concerns of privacy.

Though Google Glass has captivated the tech world in excitement of its potential, why is there a standstill? It seems that less exciting applications are being built and more controversy is rising about Glass. My friends who have google glass have stopped building applications and are even starting to wear the device less! I can even admit I am not wearing is as much as I use too. This epidemic is growing to other Glass Explorers and I am afraid that this dilemma will affect the future of google glass. So the question is what is the problem?

This dilemma can be classified as a two sided problem: the dilemma of developers and the dilemma of users/consumers. The problem that regular users and enthusiasts are starting to use the innovative device less is simply because, there are no innovative or even useful apps for the device. A Google Glass explorer and CEO of PRServe Chris Barret explains his dilemma in an interview saying "There's just no apps on Google Glass that makes me want to wear it every day." From my experience, I can concur with chris that there are no great apps that I am wanting to use that makes me want to wear Google Glass a lot. All of the apps that I played with are apps that are simple to use and are not that engaging. Even the mini games app that Google developed (which I think is pretty cool) exemplify the other problem that engaging game apps are uncomfortable to play on Glass because of  the eye strain that occurs when looking at the screen to long.

This problem could be fixed once developers build innovative, meaningful applications right? So why aren't there any developers publishing innovative apps for Glass? That's cause its so damn hard to build good applications for Glass. Just look at the differences between a regular smartphone and Google Glass. The touchscreen input on a smartphone slimmed down to a little sliver on Glass and the rich, high definition screen is now a 640x480 frame on your right eye. Plain and simply the devices are so different that the interaction and interface for Google Glass is nothing any normal developer has seen before. No wonder Google released this to hundreds of thousands of people, how would they alone be able to solve this huge problem. Will us developers even be able to solve it.

Unfortunately, I have discussed several big problems of Google Glass but I have never divulged into the real underlying problem. To really understand the problem, I need to explain what the actual underlying vision this product manifests. In a book called Design Driven Innovation, they note all of the greatest products in the past 20 years have all a common theme: They were all technological epiphanies. A technological epiphany is the moment when a new technology is developed with an underlying new meaning. When I say meaning, I mean a new vision that embodies the soul of the device. So back to Glass, what is the underlying problem. Glass is essentially a new technology that has an unwritten meaning.

When I realized this problem, I was truly stunned. It makes a lot of sense why they would send what seems a half thought of product. That is because it is actually a half completed device! So here is my newest and biggest ambition: decipher and develop the true meaning of Google Glass. Once this is achieved, the true vision of Glass will be fulfilled, and the future of wearable computing will truly unfold. 

These next few weeks I will be posting analysis of several interesting projects that impact the meaning of glass as well as discuss my vision for glass I have developed these past few months. So it begins the conquest. As my mentor, Professor Hiroshi Ishii, once told me, to developing meaningful innovative technologies relies on two words. Envision and Embody. Let the envisioning begin!

Until next time, stay tuned!

[interactivetech-blog]

2014 update - Schoolwork, Interviews, and Google Glass... Oh my!

Greetings World.

My life never seems to slow down, but that is cause I always keep sprinting. My last post was about my research at MIT and this blog has been dormant since then. So much has happened since then, my goal now is to start delivering weekly content about my ambitions, projects, and tech rants as a way to keep myself and you all up to date with my work.

To recap the end of 2013, my GPA did not improve like I hoped it would. As I felt like I accomplished nothing this past semester, something amazing happened. I received the amazing opportunity to be a Google Glass Explorer!

Google Glass is a wearable computer that originated from Google X lab and is now out for developers to test and develop apps. I remember two years ago when they were first announced, I was extremely fascinated by this device and the huge potential it had as it initiated the dawn of wearable computing. I kept up with all the latest technology articles regarding the device and I even set out a plan to develop my own wearable computer.

I made great strides to develop my own wearable computer with the beagle bone black, however the 1Ghz processor was just not enough power to develop the computer vision applications that I envisioned. Well rest assure, from the past two months that i have had glass, it has a lot of potential for computer vision applications. 

I have developed a demo app of using OpenCV4Android 2.4.7.1 and the dual core OMAP 4460 processor looks extremely promising.

In other news, last week I has three interviews and I resubmitted my MIT application to go back and research at the Media Lab this summer. As I anxiously wait to see what happens, I plan on refocusing my efforts in developing innovative applications for Google Glass.

I will post a video of my google glass demo app as well as begin a series of my plan for google glass and my mission to develop a wearable intelligence interface! The future looks bright and I will keep you all informed! Stay Tuned!!!

[interactivetech-blog]

AnnoScape: Use of a Shared 3D Virtual Workspace for Remote Collaborative Review.

In many remote collaboration systems, sharing information over a distance is discontinuous to the physical workspace. This discontinuity limits professional designers, architects and engineers to utilizing basic skills such as sketching, using hand gestures, and collaborative feedback. Further, sharing and manipulating content becomes hindered. AnnoScape is a system that shares information on a 3D virtual workspace, allowing users to collaboratively review 2D and 3D content with a physical workspace, thus allowing users to sketch and use gestures for remote collaboration. My work consists of redesigning the AnnoScape system to be integrated into the Apple iPad. By redesigning the AnnoScape system, the remote collaboration system can be expandable to research new tangible interactions for better collaborative review. 

[interactivetech-blog]

Officially a co-author of a research paper!

This is so exciting! My mentor and I just submitted a paper to TEI 2014. The paper is about a tangible actuated ruler called Vpointer. I hope it will get published!

[interactivetech-blog]

Week 5 at MIT Media Lab!

Greetings world,

My apologies for such sparse activity. My research internship at the MIT Media Lab has consumed so much of my time! It is amazing that I am actually working here. Here is a summary of what has happened these past few weeks.

I arrived on the day of Sunday, June 6th at the Burton-Connor dorm at MIT. After that first step out of the cab, everything changed. A way to describe that first week was I was  running across a river on nothing but a thin wire. That week, I met everyone in my research group, getting use to walking around Cambridge, and starting my first week at the Media Lab. I was placed as an intern working at the Tangible Media Group. I remember the first day I met the faculty advisor of the group, Prof. Hiroshi Ishii. Ya I met all the post docs and graduate students at the lab, but meeting him was a league of its own. Here I stand in front of the father of human-computer interaction and the inventor of the tangible-user interface. I was so amazed that this man was so excited to meet me and work in the lab this summer. I felt that he had high hopes for me and knew I would do a great job this summer.

I felt so excited to work. Nothing was going to bring my mood down. Then I realized what my research project was about. I am currently working on the AnnoScape project, which is a system that allows remote collaboration system for designers and architects to annotate 3D and 2D content. A better way to understand is imagine a system that connected a network of devices. This network allowed people to draw on 3D models and the drawing was broadcasted and received in real time. This system sounded amazing and its implications could lead to a huge impact in the design world. The only problem was that I was designated to actually implement an improved AnnoScape system. One that had the synchronous network and a new software architecture that can be integrated on the iPad. So my job here is to devise this network system on both a desktop and iPad. 

This seemed like no big deal, except I knew nothing about network programming, software development, nor iOS programming. I was really just a dude who new how to program in c++ and opencv. Now I see where MIT gets the saying of going to school here is like drinking water from a fire hose. The first three weeks felt EXACTLY like that. With a lot of surveying, hacking, and praying, I was able to manage the basic client-server network model and integrated the system on a desktop, an iPad, and my laptop. Now I am actually designing the UI for the new AnnoScape system! 

Only being here for 5 weeks, I have accomplished and learned so much. I believe that my current success on this project has really shown people in my research group that I am at the top of my game and I have what it takes to be a graduate student at the Media Lab. I have learned a lot about creativity and design and I plan on continuing to learn about design. I plan on also building iOS apps and really learning about mobile application development. There is just so much to learn and I have barely scratched the surface. 

I promise to post more frequently and show you all updates of my work.

Stay tuned!

[interactivetech-blog]

Vision Glasses 2.0 is underway.

This semester I have been searching for a development board like an Arduino but powerful enough to handle computer vision algorithms. I looked at the Arduino Mega, Gumstix, BeagleBoard, Raspberry Pi, and the CMUcam. Every board I researched just was not able to even handle capturing a video feed at a decent resolution or a high enough framerate. I have seen several projects online where people are incorporating opencv to the raspberry pi. The best frame rate aquired was only 5 frames per second! I could of just saved money and built the system from my netbook, but being able to create a computer vision system on a board that is as small as an Arduino but the capability of an embedded system would be signifigant. Figuring out if this feat can be accomplished would be a huge start to actually building a product that can handle my idea of an wearable intelligent interface.

After days of researching, I found my solution. I discovered the BeagleBone Black. This board features a 1Ghz ARM Cortex-A8 processor, 512 Mb of RAM, 64 GPIO pins, SPI and IC2 connection,  USB client and host, micro-hmdi port, ethernet port, 2Gb eMMC flash memory to run the Angstrom linux operating system, and it can fit into an Altoid tin can! BeagleBoard developed one of the most powerful single-board computers I have seen and I believe the BeagleBone Black will be the hardware platform I need to build my wearable-intelligent interface. My next step is to get one and see what this thing can do! I hope learning linux and the Angstrom wont be too challenging!

I posted pictures of the BeagleBone Black I purchased as well as a comparison chart of the board to the Arduino Mega and the Raspberry Pi,

Enjoy!

[interactivetech-blog]

Wearable Computing: A New Frontier

Greetings World,

It has been a while since I last wrote. Classes this semester became extremely difficult to manage and all of my time these past months have been dedicated to surviving and finshing strong. In the end I did not do as well as I wanted to, but I am proud that I completed probably my hardest semester and still kept my grades well. 

Now the semester is over and the summer has begun, it is time to dive into my projects.  In these past few months, wearable computing has popularized from the huge events that occurred with Google's latest innovation, Google Glass. This device has taken the technology world by storm as it shows the possibilities and excitement for augmented reality and wearable computing. Glass  currently takes photos and videos, sends text messages, engages in FaceTime-like Google Hangouts, makes phone calls, searches Google, and gets turn-by-turn navigation with maps. Google has shipped the Glass Explorer kits to many developers now so developers are devising apps for this new platform.

As Google Glass is the start of a new trend in the technology of wearable computing, the device is not this grand augmented reality system that people imagine it is. The current set up of Glass,the hardware limitations and the simple user interface makes Glass not designed for an immersive experience. The device is focused on giving you information on-the-go and when you need it. This is nice for people who don't want to grab there phone out of their pocket and waste their time getting the information they need from there smartphone, but I see a huge opportunity in wearable computers augmented our senses and capabilities.

When I started my Vision Glasses Project, I wanted to build a special interface that incorporated contextual scene analysis for intuitive learning and understanding of a person's physical environment. I see wearable computers as a gateway to a new realm of computing that will heighten our sense of sight and knowledge by incorporating computer vision to analyze the world we see and extract and supplement the information we desire. Using the power of computers and the vast source of knowledge available on the internet, wearable computing will open the doors to a new perspective  of our world that humans have never fathomed before. 

This is the vision I have and the opportunity is mine to seize. I have thought and pondered on this idea and now I have the plan to execute this mission. This summer I shall begin my new project, Vision Glasses 2.0. This project will be based on new hardware and software that will serve as my true platform for my wearable intelligent interface. A long road is ahead of me but I know with enough determination I can accomplish this huge feat.

[interactivetech-blog]

Successful Installation of OpenNI onto OpenCV 2.4.4 on Mac OSX!

From my past difficult experiences, I think I understand how to compile any OpenCV library and actually configure all of the header and library paths onto a C/C++ compiler. So far I have installed OpenCV in both Windows and MacOSX. I have always wanted to incorporate the Xbox Kinect into my projects however it has always been difficult to install the right libraries and build them correctly. The Kinect SDK for windows makes it easy to build applications with the Kinect, however there wasnt an easy way to incoperate the Kinect onto OpenCV. OpenCV has amazing image processing and computer vision functions that would really bring out the potential of using the Kinect. OpenNI is a better library to handle the Kinect because OpenCV library is compatible with OpenNI and can support handling the Kinect device using the VideoCapture class.

In order to support the Kinect device, you have to configure OpenCV, when building the library, with OpenNI support by setting the WITH_OPENNI flag. This can be done using a cross-platform, open-source build system called CMake. 

A few months ago, I posted my successfully installing OpenNI onto MacOSX 10.6.8. Once I realized that I can build CMake on my Mac, building OpenCV with OpenNI support was really simple since I already installed OpenNI! 

Here are the links that I found very useful  to build OpenCV with OpenNI support. Feel free to drop a comment if you have any questions on how to use the Kinect in your OpenCV projects! 

Installing OpenNI to Mac OSX: link

Install OpenCV on Mac with OpenNI support: link

[interactivetech-blog]

Summer Research at MIT

Sorry for the delay folks, college has kept me busy. Today seemed like the first break I have so I decided to post the big news now. 

 A few weeks ago I was accepted into the 2013 MIT Summer Research Program! Even now it seems too good to be true! MSRP is a 9 week program long where I will be researching under supervision by an MIT faculty member or postdoctoral fellow. During my research, I will be exposed to state of the art research facilities in MIT departments and have opportunity to possibly co-author a scientific paper with the faculty mentor. 

It's crazy to think that I was offered to participate in a research internship of this caliber. Not only  will I conduct research at MIT, I will be working with the brightest researchers in the field and contribute to work that will impact society. From my research at the CRCV lab at UCF and my summer internship at the ISU VRAC lab, a lot of strenuous work and mental exertion has lead to where I am and now has helped me get this opportunity. MIT is doing some amazing work and I am excited to collaborate in this scientific community. 

When I applied to the research program, I had to request several faculty advisors I would like to work with. Though this was a tough decision, I knew I wanted to work in the MIT Media Lab and collaborate in either two research groups. One was the Tangible Media group and the other is the Fluid Interface group. If I work in the Tangible Media group, I would work with Dr. Hiroshi Ishii, the director of the Media Lab. If I work in the Fluid Interfaces group I would work with Dr. Pattie Maes. Dr. Hiroshi Ishii is the pioneer of the Tangible Bits Vision, which is to seamlessly combine the two worlds of bits and atoms and redefine tangible information by giving a physical form to digital information. Dr. Pattie Maes and the The Fluid Interfaces group radically rethinks human-computer interaction with the aim of making the user experience more seamless, natural and integrated in our physical lives. 

I am super excited for this summer. It has been my dream to research in the MIT Media Lab and now by being apart of the MSRP program, my dream might become a reality.

[interactivetech-blog]

Found an interesting article earlier this week. MIT Graduate Students ,Moritz Kassner and William Patera, developed a low-cost eye tracking system called PUPIL that originated as their thesis project. PUPIL consists of both eye tracking software and a inexpensive  3D-printed headset. The software , written in python and using OpenCV,  is now open-source and you can make you own eye-tracking system under 100 dollars. Their vision is to create a tool kit for a diverse group of people interested in learning about eye tracking and conducting their eye tracking projects. 

I was amazed when I found this awesome project. Eye tracking can be useful for many applications and it is really simple to implement. What really caught my attention was seeing how these headsets are easily affordable and easy to produce with 3D printing. This headset looks similar to the Google Glass headset and now I am interested in using their headset to improve my Vision Glasses. It looks super cool and more practical than taping a webcam to a pair of sunglasses. Another thing that interested me about this project is how eye tracking could be useful with wearable intelligent interfaces. I have been thinking about a more practical way for my system to interact with digital information in physical environments. My current method of recognizing gestures is using color tracking. Color tracking was good enough for me to build by gesture recognition system, however its only effective if that color I am tracking is the only color in the scene. This is not practical and I need a better way for my system to recognize gestures.

I believe eye tracking can solve this problem because this method is invariant to many environmental conditions and accurate enough to illustrate complex gestures that can be mapped in the user's environment. Also I have an idea using eye tracking to localize object recognition. I believe localizing objects of interest, an intelligent interface can focus on what the user is looking at and can utilize powerful scene analysis algorithms without processing the entire frame. 

I will definitely look more into this project. In the meantime here is the link to the project page. 

PUPIL link

[interactivetech-blog]

In relation to that last question, tracking the relative location/movement of two people's lips could be used to compile a list of possible words being spoken. Checking a string of possible words for grammar and choosing the most grammatically-correct word string could then result in successful lip-reading. This could be used to covertly record conversations or help the impaired somehow. Google is working on enhancing their search technology to target searches based on grammar. So it's possible.

I think to detect whether a person is actually nodding, you can just detect the person's face, then compute the movement of the face within the bounding using optical flow to see if the head is moving up and down. But tracking the relative location and movement of a person's lip would be better and can be used for lip recognition. Great comment! 

[interactivetech-blog]

How big does the red point being tracked by your webcam need to be? If you painted small red dots on your fingernails, would it be able to track all of them for complex gestures? Also, how robust does a face have to be to be recognized by your software? Can smileys (or very simple hand-drawn faces) be recognized? Can facial recognition be used in conjunction with gesture tracking to perhaps register someone nodding their head?

To Anonymous:

The red point can be any size just as long as it is the only red color object in the scene. The way the red point is being tracked is converting each frame from RGB to HSV(Hue-Saturation-Value) color space and then finding the centroid position using HuMoments of all the red pixels in the HSV image space. You can probably draw red dots on your fingernails and it will be tracked as long as the red is distinguishable in the HSV color space.

The face detection in OpenCV is very robust for human faces. The way the program can detect so well is it was trained using thousands of human face images. I have seen it detect faces of some cartoon faces but I am not sure if it would detect a smileys or hand drawn faces. On the bright side, if you trained the face detector with enough smileys, you can essentially make a smiley detector. For more information about the face detection, google how to train face detector in OpenCV.