A maverick neuroscientist believes he has deciphered the code by which the brain forms long-term memories.
Theodore Berger, a biomedical engineer and neuroscientist at the University of Southern California in Los Angeles, envisions a day in the not too distant future when a patient with severe memory loss can get help from an electronic implant. In people whose brains have suffered damage from Alzheimerâs, stroke, or injury, disrupted neuronal networks often prevent long-term memories from forming. For more than two decades, Berger has designed silicon chips to mimic the signal processing that those neurons do when theyâre functioning properlyâthe work that allows us to recall experiences and knowledge for more than a minute. Ultimately, Berger wants to restore the ability to create long-term memories by implanting chips like these in the brain.
It's another victory for the machines: a robotic surgical system outperformed humans and robot-assisted human operators in a soft tissue procedure, bringing..
Itâs another victory for the machines: a robotic surgical system outperformed humans and robot-assisted human operators in a soft tissue procedure, bringing us that much closer to automated medical care (and the robocalypse).
Just imagine all the things we don't want robots to figure out how to use!
Robot hands are amazing, but compared to human hands, theyâre junk. A robot has real trouble catching things, or letting go of an object in order to shift its grip. You and I can reposition a pencil in our fingers with no effort at all. A robot will probably drop it.
Scientists have grown a human embryo for two weeks in a petri dish.
For the first time, scientists have managed to grow human embryos to at least 14 days old inside a petri dish. Their work has the potential to revolutionize our understanding of what makes us humanâand itâs the next step on the road to completely artificial, womb-free reproduction. In fact, itâs possible that these embryos might even have kept growing past two weeks, if the scientists did not have to terminate them for ethical reasons.
Several factors have contributed to the sudden expansion of connected car services available or coming to the market, most notably the expansion of mobile broadband networks, high penetration of smartphones in the consumer market, and auto manufacturers' re-evaluation of connected services as a competitive advantage and means to generate new revenues.
While the connected car and smart home ecosystems haven't yet entered the mainstream, neither is in its infancy. Crossover between the two markets is evident and offers a unique opportunity for the ecosystem players.
Automotive OEMs
Connected vehicle data presents an opportunity and a challenge to automotive original equipment manufacturers. They can sell access to vehicle performance and driver behavioral data, as well as leverage collected data to improve product designs. With better insight into driver behavioral data, manufacturers ultimately can create unique and personalized experiences and interfaces.
With the Internet of Things expanding, auto manufacturers must expand their connected car strategies to consider developments in adjacent ecosystems, such as the connected home space. Several considerations are paramount:
Differentiating the car connectivity platform with unique app experiences;
Creating a superior in-vehicle experience for apps and services that are not native to the car ecosystem;
Preventing distracted driving; and
Addressing data security and privacy concerns.
Aftermarket Device Manufacturers
Some 225.6 million consumer vehicles in the U.S. don't have the ability to connect to the Internet. Owners of these vehicles don't need to wait until their next vehicle purchase to take advantage of new connected features. Several manufacturers offer connected aftermarket devices, typically in the form of head units or OBD-II dongles.
Aftermarket device makers are forming key partnerships with established smart home device manufacturers and startups, smart home hub suppliers, and insurers with interests in both the vehicle and home markets.
As the consumer vehicle fleet becomes more connected, the market for OBD-II telematics devices will shrink. Current market players will then switch to a software-first strategy, leveraging their development platforms as their key products.
Software and Platform Developers
Most services and initiatives that cross the car and home ecosystems are the result of direct partnerships between industry players.
Automakers' desire for a proprietary app development ecosystem inhibits innovation in the space, because developers can't write codes once and run them on all car models.
However, as the mobile app industry demonstrates, consumers will gravitate toward connected solutions that enhance their lifestyles wherever they are; solutions that are closed, device-specific, or otherwise do not play well with others will struggle to retain consumer loyalty in the long term.
Insurance Providers
Traditional auto insurance models determine premiums based on factors such as a driver's area of residence, the vehicle make and model, demographic profiles, and claims history.
Usage-based insurance, or UBI, leverages consumers' actual driving behavior to best match each driver's risk profile with an appropriate insurance premiums.
It allows insurance companies to create more accurate risk assessment profiles of drivers. Additionally, drivers are provided with real-time feedback regarding their driving patterns. A reduced insurance premium is a powerful motivator for safer driving, which ultimately results in reduced costs for insurers.
As UBI has gained popularity in the automotive sector, providers seek to apply a similar approach to the smart home.
Mobile and Broadband Service Providers
Mobile network operators, or MNOs, and broadband service providers have a natural interest in the crossover between the smart home and the connected car space. This is because they both deliver value-added services and premium content that act as additional revenue streams and ward off commoditization of their core businesses.
MNOs have assets in both the connected car and smart home ecosystems. As such, they are major players at the intersection of these markets and have an advantage over other service providers that operate in just one market.
Broadband service providers also have assets and incentives to seek opportunities at the intersection of the connected car and smart home markets.
Several Internet service providers already offer pay-TV services and aim to extend their value in the home further with smart home and security services. Providers in the video space face increasing pressure to diversify their home offerings as a growing segment of consumers shave or cut the cord. From this perspective, expanding services beyond the home to the connected car space further expands the functionality and value of their platforms.
Consumers' desire for their connected solutions to work together in a simple, easy-to-manage way will drive crossover opportunities in the connected car and smart home ecosystems. Companies with assets in both ecosystems, such as mobile network operators and insurance companies, stand to benefit from their convergence and will push the markets closer together
Microsoft last week revealed that it will release the core components of its Chakra JavaScript engine to open source as ChakraCore.
It will be available next month on GitHub under the MIT license.
ChakraCore offers best-in-class JavaScript execution with the broadest set of ES2015 feature coverage and dependable performance, reliability and scalability, Microsoft said. It will target cloud-based services, the Internet of Things and beyond.
Microsoft will work with Intel, AMD and NodeSource to develop the ChakraCore community.
"Microsoft, once the undisputed champion of de facto Web standards, now looks to open source as the key to regaining that crown for JavaScript," commented Bill Weinberg, president and principal analyst at LinuxPundit.
"Like other core infrastructure open source software products, Chakra has the potential to turn the standards world on its head, putting de facto implementation ahead of de jure standardization," he told LinuxInsider.
"Releasing Chakra as open source paves the way for adoption by other software products and projects across the ecosystem, from the cloud to desktop to IoT," Weinberg said.
More About Chakra
Chakra powers Universal Windows applications across all form factors supporting Windows 10.
It's optimized for TypeScript, and Node.js runs with it.
Chakra supports most of the ECMAScript 2015 features, and it supports some future ECMAScript proposals, such as Async Functions. It supports asm.js and is a key player in helping evolve WebAssembly and its associated infrastructure, Microsoft said.
ChakraCore is a fully fledged, self-contained JavaScript virtual machine that can be embedded in derivative products and can power applications such as NoSQL databases, productivity software and game engines that need scriptability, the company said.
It can be used to extend the reach of JavaScript on the server with platforms such as Node.js and cloud-based services.
Unlike Chakra in Microsoft Edge, ChakraCore doesn't expose Chakra's private bindings to the browser or the Universal Windows Platform. Further, ChakraCore will support a new set of modern diagnostic APIs, which will be platform-agnostic and could be standardized or made interoperable across different implementations.
The initial release of ChakraCore will be for Windows only, but Microsoft will bring it to other platforms.
"Developers should find advantages in using this toolset" because "it's focused on developers, and Microsoft makes a decent set of tools," remarked Rob Enderle, principal analyst at the Enderle Group.
There's little risk to Microsoft in going this route, and, at worst, its tool would be trivialized, he told LinuxInsider.
Why ChakraCore Has the Juice
ChakraCore enables the "ultrapopular" Node.js on both Windows 10 and Windows 10 IoT Core and has a pre-existing end-user and interoperability base from Chakra's proprietary database, LinuxPundit's Weinberg said. It also has strong benchmarking performance from day one of its open source release and has early, selective implementation of ECMAScript 2015 capabilities.
Microsoft is likely to benefit most from open sourcing Chakra by "regaining leadership in a key standard, making interoperability with Chakra's JavaScript dialect de rigueur," he suggested.
"If Microsoft marketing is skilled enough to tease apart the JavaScript from the rest of users' Web experience, they'll regain their reputation as a technology leader," Weinberg added.
Platforms adopting ChakraCore as their JavaScript engines also will benefit, and ECMA "will benefit by having a forward-looking, thorough-going implementation of its standard -- and features from it -- that have a greater potential to propagate across today's Internet and the emerging IoT," he said.
ECMA standards have been a battleground, with Adobe, Mozilla, Opera and Google primarily developing ECMAScript 4 specs. Microsoft and Yahoo led another working group to implement some changes and bug fixes in ECMAScript 3 while retaining it as a subset of ECMAScript 4.
Nobody was happy with the result, and ECMAScript 4 was abandoned. Work then began on ECMAScript 5, dubbed "Harmony."
Los Angeles, Chicago could be the next cities to get Google Fiber
The second and third most populous cities in America could be the next municipalities to host Googleâs gigabit Internet service provider, Fiber. The company announced that they want to invite the cities of Chicago and Los Angeles to explore the possibility of getting the hottest Internet service around. Theyâre by far the largest cities yet considered by the fledgling ISP, having previously come to just a single city with more than a million residents (San Jose). The idea is likely to be received in Chicago and LA much as it has been elsewhere: very, very well.
This is an important moment for the service, since itâs the first time Google has publicly considered going for the countryâs most important urban centers. Not only does this represent a more powerful attack on the customer-base of competing ISPs, but it positions Fiber much more as a service to be desired. Itâs one thing to not have some experimental thing you heard theyâre getting out in Salt Lake City or something (no offense to anyone there), but quite another to have worse service than the residents of Americaâs big, trend-setting cities.
Los Angeles, in particular, seems to pose a particularly strong opportunity for Fiber to prove itself. It combines incredible population density in some areas with enormous urban sprawl in others. Itâs got a wide variety of income levels, cultures, and languages. If Fiber can roll out in Los Angeles, it should be able to do so just about anywhere.
Some have questioned whether Google (or Alphabet? I guess is still âGoogle Fiberâ for nowâŠ) will be able to be successfully capture the market on a national scale. The idea is that since existing ISPs already have the people and businesses in place to roll out a fiber Internet service, and in many cases actually have fiber in the ground already, they will be able to adapt and out-price Googleâs offering, or simply provide gigabit Internet more quickly as Google runs around ripping up miles and miles of urban pavement.
The problem with this thinking is that it ignores how widely Google seems to be interpreting the win conditions in the online space. Obviously, the company would like to make oodles of money off of selling you the Internet, and theyâd like even more to control both the physical lines of communication and the communication that goes on over those lines. However, Google didnât decide to get into the ISP game because it saw the space as wide open and ripe for the picking, but because it saw the space as decadent, uncompetitive, and totally unprepared to deliver the services Google want stop deliver over the next decade or two.
In a very real way, if AT&T or anybody else does go on to âbeatâ fiber at its own game, Google will have won. The individual investors and stake holders who bet on Fiber in a particular city would be devastated, but the larger entity that is Google will have affected the change it was aiming to.
Google developing AI-based messaging service: Report
A couple of news items from Google today: First, the company is reportedly working on a new kind of artificial intelligence-based messaging service, in an attempt to compete with Facebookâs WhatsApp and Messenger, according to the Wall Street Journal. The report said that Google plans to integrate chatbots, which answer user queries from within a messaging app after searching the Web for information. The idea is to provide the answer from within the context of a conversation without having to exit that conversation at any point.
âAll users care about is a convenient way to find what they are looking for and if Google isnât in front of the consumer that is a problem for them,â said Scott Stanford, co-founder of venture-capital firm Sherpa Capital, in the article. âMessaging is a subset of the Internet where Google is not strong. They have to win and be the dominant player in messaging.â
Google has tried and failed to reinvent messaging, email, and social networking several times in the past decade, including Google+, Google Wave, Google Buzz, Google Messenger, and Google Hangouts. Google has been struggling to capture users that have moved on to other services like the aforementioned Facebook Messenger and WhatsApp, particularly in other countries besides the US.
Back in January 2014, Google acquired the British artificial intelligence company DeepMind for a reported $400 million, after hiring futurist and inventor Ray Kurzweil and acquiring eight robotics firms in the year prior. DeepMind aims to help researches mine big data in an effort to boost discoveries in medicine, genomics, and more. And in November of this year, Google unveiled TensorFlow, an open-source machine learning platform that could boost its deployment of artificial neural networks. At the time, Google called Tensorflow the companyâs âsecond generationâ machine learning platform, successor to the older DistBelief platform that has led to many of the companyâs current AI-infused products.
Meanwhile, a separate Android Police report said Google has confirmed itâs testing a new way to log into your account from multiple devices just by using your phone, without having to provide a password or use two-step authentication:
âWeâve invited a small group of users to help test a new way to sign in to their Google accounts, no password required. âPizzaâ, âpasswordâ and â123456ââyour days are numbered.â
That report stemmed from a Reddit user who claimed to be testing the new feature on his Nexus 6P.
Thereâs no word when Google plans to unveil its new messaging service or roll out the phone-based login feature to more users.
How Technology Could Prevent Another Paris-Like Attack
What I find fascinating is that with all of the focus members of the intelligence community place on violating our privacy, they still aren't able to stop attacks like the one in Paris. Currently they are complaining that it is our fault for implementing encryption that blocks their often-illegal views into citizens' personal lives.
I think that even if encryption didn't exist, they still would be ineffective. Looking back at 911, there was no lack of intelligence indicating an attack was imminent. That intelligence was clearly in evidence to show guilt after the fact, but there was an inability to get that intelligence to a decision maker who could -- or would -- make a timely decision to keep the event from happening.
Our system is set up largely to punish people who commit crimes after the fact -- not to keep a crime from happening in the first place. That mindset will need to change if we actually are to become safer. Something else that will need to change is the separation between citizens and enforcement. Rather than being treated like we are part of the problem, we -- the folks supposedly being protected -- need to be part of the solution, and I think there could be an app for that.
I'll go deeper into that this week and close with my product of the week: IBM Watson, which could be at the core of not only this effort, but also of efforts to make us healthier and happier.
Minority Report vs. Current Law Enforcement
Restraining orders don't work. Law enforcement largely is based on the concept of punishment as a deterrent. If it doesn't work, then it is about catching and punishing the perpetrator to serve as an example in order to prevent the next crime. This works to a degree for financially motivated crimes like theft and illicit drug sales, because there is a risk-reward balance when criminals think strategically. They have to weigh the reward of the crime against the cost and risk of going to jail.
With crimes of passion, folks tend not to think strategically. They are more in the moment, and while they certainly may regret their violent action, the punishment plays little role in the decision process, because they are living in the moment and not considering it until after the act is committed.
With suicide bombers, the plan is to be dead. Even if they think strategically, at least for now, we don't have anything we can do to punish people who are dead. In fact, the punishment may help ensure the outcome, because the only folks who are punished in these instances are those whose bombs fail to detonate. In a situation that might lead to torture to extract information about other parts of a terrorist plan, the punishment simply would ensure the outcome society wants to prevent.
That is the concept behind the movie Minority Report -- stopping crimes before they are committed, which supposedly is the goal of most antiterrorist activity. Since the psychic approach probably won't work, that movie isn't much help. However, we have tons of people out on the streets with smartphones, and we have cognitive computing solutions like IBM's Watson. That combination actually could be better than the psychic approach. (If you recall, the entire movie was about how it failed.)
There Is an App for That - or Should Be
The idea for this came from a note I got from John Byrnes, writing for theCenter for Aggression Management, which has a scientifically backed process that reliably identifies people who are about to do something violent. This truly drifts into Minority Report territory, because it is about violence in general. It would include those planning to shoot up schools, places of employment (or ex-employment), or spouses.
The process looks for behaviors that would be evident leading up to an attack or crime -- while criminals are gathering information and selecting targets and planning -- and even early in the execution process.
The thing is, to make this work, we need folks who know what to look for, and a way for those folks to alert the authorities to take action.
There could be an app for that. People just use their cellphones to take pictures of people they think are acting questionably as part of an app that sends the pictures to a central service, with an AI, such as Watson, that uses facial recognition to identify and profile them. If the behavior captured triggers a violent profile, law enforcement would be notified, and the individual put under formal surveillance, with a flag level ranging from questionable to the possibility of imminent violence.
At the end of the year, the person who was instrumental in preventing the most crimes would get recognized -- maybe a presidential citation or a medal, and a cash award to be donated to charity of the person's choice. A job offer from the FBI might even make sense, because a top skillset like that could be valuable full time.
Wrapping Up
I think the problem with the current approach to attacks like the one in Paris is that it treats us all like criminals who need to be monitored. Our privacy is violated as a consequence of a process -- based on post penalties -- that doesn't work for this kind of crime.
Instead, I think we could use technology to make people a part of the solution. Focus on those who fit a scientific profile for people intending violence, which would give law enforcers a better chance of stopping crimes before they're committed.
Backed by something like Watson, I think there could be an app for that -- and it could do a better job of catching upset, crazy or depressed people who are planning violence against schools, employers or government facilities. I don't know about you, but I'm tired of being part of the problem and actually would like to be a bigger part of the solution.
Product of the Week: IBM Watson
A couple of weeks ago, I was at IBM getting an update on Watson, and I heard how it now can capture, classify, and even direct programs that can modify behavior at a national scale.
The information was presented in a way that suggested marketing, and it would be incredibly useful to move product or even change the outcome for elections.
It hit me at the time that IBM should be using it to change perceptions that surround IBM and Ginny Rometty -- the company's CEO -- and help get people to see IBM for what it is becoming, not what it was. One of the historic problems with firms like IBM is that they don't use their own technology aggressively enough, even when it could improve the value of every employee's stock options.
Here is a system -- currently unique in the market -- that once trained could do things like identify terrorists or other criminals early on, and maybe help influence them not to take the steps that end up with them and a lot of us dead.
This is an incredibly powerful cognitive computing tool, and every major CEO in the world has gone to see it, is in the process of going to see it, or planning on going to see it, according to IBM, and every one of has been amazed at what this early step into artificial intelligence could do. The system apparently has advanced a great deal from when it won Jeopardy! and could end up saving the world. As a result, IBM's Watson is my product of the week.
Silicon 2.0 promises superpowered chips and solar cells
ITâS a material so good they named a valley after it. And no wonder. Todayâs connected society would be impossible without silicon. Chips made from it run everything from smartphones to pacemakers, with some 6.5 million square meters of the stuff rolled out every year. And the solar industry relies on vast quantities of silicon to make the photo voltaic cells that convert light into electricity.
Silicon is in such demand that youâd be forgiven for thinking its position at the top of the pile was untouchable. But its status owes more to the fact that it is the second most abundant element on the planet than to its performance. Crucially, siliconâs atomic structure limits its ability to conduct electricity. And that holds back computer processing speeds and the efficiency of solar panels. If electronic devices are to get faster, cheaper and more compact at the rate weâve come to expect, silicon as we know it needs to be shown the door.
So the hunt is on for a replacement. Many elements and compounds have been proposed over the years, but it is starting to look like the solution might be closer to home. Ordinary silicon, imbued with certain superpowers, might be able to replace itself.
Silicon belongs to the semiconductor family of materials, whose ability to carry an electric current lies somewhere between that of a metallic conductor and an insulator. In a computer chip, applying a small voltage is enough to flip siliconâs state between conducting and insulating, producing the binary 1s and 0s of digital information.
Software better than humans at guessing how you feel from speech
Chalk up another win for computers. Software developed at the University of Rochester in New York has outstripped humans in its ability to identify emotions in speech. The researchers plan to use it to understand the effects of emotion in parent-child interactions.
The software, developed by Rochester graduate students Na Yang and Emre Eskimez, is not the first to recognize the feelings behind human utterances, but it is the first to outstrip humans in a robust comparison. When classifying 700 audio samples, the system got their emotional sense right 72 per cent of the time. A group of 138 people working on Amazonâs Mechanical Turk platform who were paid 50 cents for every 10 samples they classified, averaged 60 per cent accuracy on the same clips. Yangâs software was also much better than the humans at recognizing which clips it couldnât classify accurately. When allowed to skip these clips, the accuracy rose to 85 per cent.
The software recognizes emotions on the fly without having to record any of the speech â an advantage when people are already wary of letting researchers into their homes. Instead it identifies vocal tract shape, frequency, brightness, flatness, roughness and energy from speech, and uses those properties to model the emotion behind it. The systemâs only output is the emotions it identifies â levels of happiness, sadness, anger, disgust and fear, or neutrality.
Melissa Sturge-Apple â the psychologist on the Rochester team â plans to use the software to record parentsâ emotions when they interact with their children, and to understand how those emotions impact a childâs development. She has already started recording family conversations in the lab. If the software can classify those conversations accurately, she plans to take it into peopleâs houses, placing emotion-recognizing Bluetooth microphones to listen in on conversations.
Ani Nenkova, a computer scientist at University of Pennsylvania, is not surprised that Yangâs software beat the humans. She says people are just not very good at identifying emotions from speech alone. Her recent research has shown that people do better when they have both video and audio cues. âDetection of emotion is a multi modal business for people,â she says.
No-touch smartwatch scans the skin to see the world around you
What are you doing now? Your next watch may know the answer.
Two new smartwatch prototypes developed at Carnegie Mellon University in Pittsburgh, Pennsylvania, can guess what their wearer is up to by tracking subtle signals in their skin and muscles. The technology could allow the owner to answer calls, track activities, and more â all without needing to be touched.
The first, named EM-Sense, can figure out what object its owner is touching. By fitting the smartwatch with a radio receiver, EM-Sense can use its wearer like a living antenna, picking up on the electromagnetic ânoiseâ that travels through the human body when emitted by electrical objects.
So far, the system has been trained to recognize the unique electromagnetic signals of 23 common items, including desk lamps, refrigerators and computer trackpads.
Developed in the lab of Chris Harrison, a professor of human-computer interactions, EM-Sense is envisioned as a tool for people to augment their everyday activities. If the watch senses that youâve jumped on your motorcycle, it might open up a map to guide you to your next destination or display a stored reminder to pick up milk from a shop. Or if it knows you have stepped on some scales, it might automatically log your weight.
Tomorrowâs world
A separate prototype, named Tomo, tracks the wearerâs hand gestures in real time. This relies on an imaging technique called electrical impedance tomography to see inside the arm. The watch band is studded with copper electrodes that can bounce electrical signals between each other, building a picture of the muscles inside the wrist.
To test their invention, Harrison and graduate student Yang Zhang got 10 people to wear Tomo armbands and wristbands, and collected data on their hand motions. A machine learning algorithm then learned to accurately recognize 13 gestures: a thumbs up, a fist, a pinch between the thumb and different fingers, and so on.
In one demonstration, the researchers hooked Tomo up to a Samsung Galaxy smartwatch. This allowed the wearer to flip through new messages with a flick of the hand to the right or left, or answer incoming phone calls by making a fist.
âSmartwatch capabilities in general are super-limited right now,â says Gierad Laput, who led one of the projects. âWeâre just starting to tap into the full potential of having a computer on your wrist.â
Both projects were presented on Monday at the Symposium on User Interface Software and Technology in Charlotte, North Carolina.
Iâm going to make Facebookâs AI predict what happens in videos
This week, Facebook unveiled several artificial intelligence projects. Yann Lecun, the company's director of AI, reveals what this technology can do
What are the big challenges ahead for you?
The big challenge is unsupervised learning: the ability of machines to acquire common sense by just observing the world. And we donât have the algorithms for this yet.
Why should AI researchers be concerned about common sense and unsupervised learning?
Because thatâs the type of learning that humans and animals do mostly. Almost all of our learning is unsupervised. We learn about how the world works by observing it and living in it without other people telling us the name of everything. So how do we get machines to learn like in an unsupervised way like animals and humans?
This week, Facebook demonstrated a system that can answer simple questions about whatâs happening in a picture. Is that trained by annotations made by humans?
Itâs a combination of human annotation and artificially generated questions and answers. The images already have either lists of objects they contain or descriptions of themselves. From those lists or descriptions, we can generate questions and answers about the objects that are in the picture, and then train a system to use the answer when you ask the question. Thatâs pretty much how itâs trained.
Are there certain types of questions your AI system struggles with?
Yes. If you ask things that are conceptual then itâs not going to be able to do a good job. It is trained on certain types of questions like the presence or absence of objects, or the relationship between objects, but thereâs a lot of things it cannot do. So itâs not a perfect system.
Is this system something that could be used for Facebook or Instagram to automatically caption pictures?
Captioning uses a slightly different method, but itâs similar. Of course, this is very useful for the visually impaired who use Facebook. Or, say youâre driving around and someone sends you a picture and you donât want to look at your phone, so you could ask âWhatâs in the picture?â
Right now the system just tells you the type of image it is  â if itâs outdoors or indoors, if thereâs a sunset or whatever. It then gives you a list of the things thatâs found in it, but itâs not like full sentences. Itâs just a list of words.
It doesnât know the relationships between these things?
Right, and so the next generation that we have working in the lab is more like prose.
What other potential uses do you envisage for such artificial neural networks?
In biology and genomics, there is a lot of interesting work. For example, Brendan Frey at the University of Toronto has shown that you can train a deep-learning system to emulate the biochemical machinery that reads the DNA and produces proteins. With that you can figure out the relationship between multiple particular changes in the genome and particular diseases, which are not really traceable to a single mutation but can be an assembly of things. There is going to be a lot of progress in medicine because of this kind of stuff.
Are there problems that you think deep learning or the image-sensing convolutional neural nets you use canât solve?
There are things that we cannot do today, but who knows? For example, if you had asked me like 10 years ago, âShould we use convolution nets or deep learning for face recognition?â, I would have said thereâs no way itâs going to work. And it actually works really well.
Why did you think that neural nets werenât capable of this?
At that time, neural nets were really good at recognizing general categories. So hereâs a car, it doesnât matter what car it is or what position it is. Or thereâs a chair, there are lots of different possible chairs and those networks are good at extracting the âchair-nessâ or the âcar-nessâ, independently of the particular instance and the pose.
But for things like recognizing species of birds or breeds of dogs or plants or faces, you need fine-grained recognition, where you might have thousands or millions of categories, and the differences between the different categories is very minute. I would have thought deep learning was not the best approach for this  â that something else would work better. I was wrong. I underestimated the power of my own technique. Thereâs a lot of things that now I might think are difficult, but, once we scale up, are going to work.
Facebook recently unveiled an experiment in which engineers gave a computer a passage from Lord of the Rings and then asked it to answer questions about the story. Is this an example of Facebookâs new intelligence test for machines?
Itâs a follow-up of that work, using the same techniques that underlie it. The group thatâs working on this has come up with a series of questions that a machine should be able to answer. Here is a story, answer questions about this story. Some of them are just a simple fact. If I say âAri picks up his phoneâ and then asked the question where is Ariâs phone? The system should say that itâs in Ariâs hands.
But what about a whole story where people move around? I can ask, âAre those two people in the same place?â and you have to know what the physical world looks like if you want to be able to answer these questions. If you want to be able to answer questions, like âHow many people are in the room now?â, for example, you have to remember how many people came into this room from all the sentences. To answer those questions, you require reasoning.
Do we need to teach machines common sense before we can get them to predict the future?
No, we can do this at the same time. If we can train a system for prediction, it can essentially infer the structure of the world itâs looking at by doing this prediction. A particular embodiment of this thatâs cool is this thing called Eyescream. Itâs a neural net that you feed random numbers and it produces natural-looking images at the other end. You can tell it to draw an airplane or a church tower, and for things that itâs been trained on, it can generate images that look sort of convincing.
So thatâs a piece of puzzle, to be able to generate images  â because if you want to predict what happens next in videos, you must first have a model that can generate images.
What kind of things could a model predict?
If you show a video to a system and ask, âWhatâs the next frame in the video going to look like?â itâs not that complicated. There are several things that can happen, but moving objects are probably going to keep moving in the same direction. But if you ask what the video will look like a second from now, there are a lot of things that can happen that you just canât predict, so there the system will have a hard time making a good prediction. Thatâs the problem weâre facing that we donât know how to handle properly.
And what if youâre watching a Hitchcock movie and I ask, â15 minutes from now, what is it going to look like in the movie?â You have to figure out who the murderer is. Solving this problem completely will require knowing everything about the world and human nature. Thatâs whatâs interesting about it.
Five years from now, how will deep learning have changed our lives?
One of the things weâre exploring is the idea of the personal butler, the digital butler. There isnât really a name for this, but at Facebook itâs called Project M. A digital butler is the long-term sci-fi version of M â like in the movie Her.
