Wetware – Rise of the Machines
My observations on how Machine Learning and AI has evolved over the years
Over the past decade cloud computing has evolved leaps and bounds in terms of not just availability but in terms of the services and functionality that is available for us to consume. When design patterns around Service Oriented Architecture emerged several years ago, the tech world and more importantly the business world had just recovered from the dot com bubble and was reconstructing its models from a pure ecommerce model to a model where both the brick & mortar and ecommerce would coexist. The existing "legacy" systems were to be "web enabled" so as to start serving the growing internet traffic. Business's that relied on FoxPro, VB and Mainframe systems had to build web front ends to cater to this demand. The extended use of XML and the XMLHttpRequest were being realized by the developers to cater to this need.
In early 2000, our small team was trying to solve a complex request – to mimic the Foxpro interface on the web browser. On these thick client systems, typing in data and hitting tab would populate the rest of fields in an instant. The client's users were accustomed to that type of user interaction and wanted to replicate it on a web application albeit without refreshing the page at each tab press. At around this time, at one of the Microsoft Dev conferences a discussion around the technology behind Outlook Web Access was being held and it was a Eureka moment for us. XHR or AJAX has contributed immensely in the growth of technologies around us.
ASP, not Active Server Pages, but Application Service Providers were the new breed of startups that relied heavily on these core concepts of loosely coupled data exchange formats. Systems and processes that organizations built from ground up and maintained, such as CRM or Payment Processing, could now be consumed as a service. This model helped organizations clear up space and save time on maintaining the hardware and software. But this model was more beneficial to larger organizations primarily because most ASPs single tenancy in the deployment of their software which meant that costs were high for smaller organizations. As the demand for such services increased from smaller organizations, ASPs began offering multi-tenancy services which gave rise to the "cloud" service model.
Grid computing, that originated in early 1990s, gained traction and started going mainstream in early 2000s. Large organizations and institutions began to tap the power of the networked systems to perform CPU hungry tasks. The Berkeley Open Infrastructure for Network Computing was one of the early adopters of this model that utilized the power of the internet to perform such tasks. A simple program, that when installed in any computer, could help contribute to scientific computing especially when in an unused state. BOINC had about 238,507 volunteers and 770,875 computers (hosts) worldwide processing on average 10.712 PetaFLOPS as of this writing.
As more people started consuming data on remote servers and more information was being consumed via the internet, gathering of data for analytical purposes for all type of actions and interactions started evolving rapidly. In-application analytics captured data not only about user interactions but also the behavior of the application itself. In parallel more and more organizations started adopting the ASP model especially the CRM and ERP systems. Business Intelligence gained from CRM and ERP systems were easy in the earlier days as the amount of data was small and the processing power needed to compute was less. The analytical models were also limited to the type of data that was being gathered. Most models relied on just the empirical aspects of the data. As the volume of data that was being captured through end user application interactions as well as through CRM systems, the traditional data processing applications were being rendered inadequate.
With the rise of Web 2.0, social media platforms and other businesses had started acquiring information from internet users at a very personal level. People started posting all their emotions using words such as tweets, reviews, opinions and recommendations. What simple words meant earlier had different meanings when put in context to the subject being discussed. Organizations started to mine this huge volume of data to better understand the sentiments behind these words. Emojis when used in context to a sentence also proved extremely valuable to better understand the overall sentiment of the phrase or comment. Computational models were getting better trained with the huge volume of data which in turn led to better Sentiment Analysis models. Microsoft Azure, HP Haven On Demand (formerly Idol On Demand), Yahoo, IBM Watson all provide Sentiment Analysis for data ranging from Tweets to Whitepapers. Facebook introduced text based emotions such as “feeling happy with” and “feeling excited at”. These provide significant value to the models that are crunching and learning from all these social interactions coupled with real world news and information.
On the other hand Amazon and Netflix have volumes of data on usage and preferences either provided directly by the users or by inferring based on patterns. For example, while on Amazon you search for a product, rate a product, add to your list are implicit ways that the site collects information about you. Also, as you browse through the catalog, add and remove from your cart and revisiting the site are explicit ways data is collected. The engines or more specifically the algorithms that site behind and analyze this data have been hone greatly over the years. This leads to personalized recommendations to the user. A combination of such engines can clearly understand what you would need based on various data points surrounding you. AI like Alexa, Cortana, Siri, Google Now and the likes are already functioning in a similar capacity.
As machines start beating humans at complex games like Go and understand better about our behavior and the sentiments that surround all this, it won't be very far before simple back-office functions will be handled by these systems followed by the important activities. Skynet make become a reality after all :)
