Speak With Me

[spkwme]

You who are on the road, Must have a code, that you can live by. And so, become yourself, Because the past is just a good bye. Teach your children well. Their father's hell, did slowly go by, And feed them on your dreams The one they picked, the one you'll know by.

Crosby, Stills, Nash & Young.

[spkwme]

Speak With Me now supported on the Nvidia Tegra K1, X1, Jetson Pro, and DriveCX platforms

This is something we started working on years ago, and kept under wraps for quite some time. Speak With Me supports offline and client-server hybrid modes on the Nvidia Tegra K1, Tegra X1, Jetson Pro (Automotive Tegra K1), and DriveCX (Automotive Tegra X1) platforms!

Our performance numbers (0.4s-0.75s latency) were announced at the AutoTech council meetup in November, and we demonstrated the system at the AutoTech council meetup in December.

And just like every other platform we support, Speak With Me’s advanced conversation management features, including conflict resolution, disambiguation, simultaneous multi-domain context management with a memory, and changing one’s mind in the middle of the conversation -- are all supported. We have emotional expressivity coming soon too. The Tegra X1/DriveCX also gets emotional understanding and voice biometrics in the coming months. 

If you’d like to create an AI personality for your product, and connect it to apps and services so that your product can have engaging, funny, and useful conversations with your users, send us a note at:

info AT speakwithme DOT com

[spkwme]

Autonomous Driving should be about Safety, not replacing humans at the wheel

I did some math recently. In an average Bay Area rush hour commute period, there are 4 accidents with 1.44M cars on the road. That means, humans are 99.99972% accurate behind the wheel.

How many artificial intelligence systems achieve that level of accuracy today? None. 

How many will achieve that in 30 years? I still predict, none.

Autonomous cars are at about 90% accuracy today. Getting to 99.99972% or higher to beat a human shouldn’t be our immediate goal.

However, reducing the number of accidents by 90% should be our immediate goal. If that’s all autonomous vehicles achieve, then they are a stunning success, even if by Silicon Valley standards it means they can’t lounge and be on their iPad, not pay attention to anything, and let the computer do all the work.

Switch the conversation to safety, not replacing humans.

[spkwme]

What Can You Do With Speak With Me?

Speak With Me enables rich, deep conversations with a user and a device.

What is a rich, deep, conversation?

Rich conversations understand who you are.

Rich conversations can resolve conflicts.

Rich conversations can disambiguate.

Rich conversations allow you to change your mind.

Rich conversations have a memory.

Rich conversations are deeply connected to the services you use.

Rich conversations are scalable into more advanced features we have on our roadmap including:

--Emotional Expressivity.

--Emotional Understanding.

--Use of Computer Vision with Voice together.

--Voice based e-commerce.

[spkwme]

Self Driving Car Policy - one that works

When I speak to consumers, they frequently dream of having a self-driving car take them around, while they tend to other things.

The lazy consumer instinct is to treat a self-driving car as a taxi-cab, and abdicate all responsibility for their own safety to the machine.

Unfortunately, this does not work.

Because what happens when a normal car purposely hits a self driving car, and steals from the victim, or injures them? I grew up in Philadelphia where car-jacking is unfortunately common. 

In America’s litigious society, it is unfortunately probable that the manufacturer of the self-driving car would get sued even though they are clearly not at fault in the above scenario.

So, if you want a self driving car on the road, you will have to accept terms similar to pilots:

1. You must know how to take over controls of the vehicle.

2. You are still responsible for what happens in the car and to the car, and the events happening around you.

3. If you do not take over the car in the designated time the car requests you to re-take control, the car may decide what to do and likely will pull over to a safe area and stop.

----

If you have ideas for other policies that work, please do share them!

[spkwme]

The Apple Watch will disrupt the Internet of Things significantly

Speak With Me sells into automotive and the internet of things, and something we have realized is that literally hundreds of internet of things companies are doing things....

that can be displaced by an app on the Apple Watch.

Many, many investors, and many kickstarter projects and many internet of things companies are going to go belly up, because the Apple Watch, will displace them and get far greater distribution.

It’s a wireless microphone on your wrist. You won’t need another mic in your house to control things.

So if you want a voice activated Apple Watch app, let us know, we are happy to help you make one!

[spkwme]

The Interconnectedness of our Things

Imagine that you have a smart watch that knows your pulse rate, and how you are feeling (happy, sad, annoyed, delighted, etc.) and when you are getting ready for work, it communicates with your Spotify-connected radio, and turns on the right playlist or radio station just for your mood at this moment. 

Let's say you have now gotten ready for work, and you are in the middle of listening to Terry Gross on NPR. The radio knows the moment you left home, and communicates with your car, so that when you get into your car, the interconnected dialogue system asks you if you want to resume listening to Terry Gross, and it automatically starts 30 seconds before where you left off.

It happens to be your spouse's birthday, and when the radio show ends, your car says to you: 

"It's your wife's birthday today, and Miete Pattissere is offering you 6 free cupcakes if you buy six cupcakes, and it's across the street from your usual morning favorite coffee shop, Blue Bottle. Would you like to order cupcakes for your wife today?"

"Yes I would."

"That will be $16.00. Please say 'I would like these cupcakes' to confirm your order, or 'cancel my order' to cancel your order."

"I would like these cupcakes."

Using voice biometrics, your order is placed. Using your current position and speed of travel, the cupcake shop knows your estimated time of arrival.

------- At Speak With Me, we are building out this vision, in a helpful manner for consumers that respects a users' privacy and we understand that our customers need to be brought to there in stages.

Let's deconstruct this vision:

It knows who you are. It knows what devices and products are yours. It remembers where you like to go. It only offers you something of high-value in a thoughtful, non-distracting manner, at the moment of it's need.

Thoughtful and compassionate, contextual user interfaces, centered around voice. That is what Speak With Me is all about.

[spkwme]

Google's getting it all wrong. Do you want a self driving car that obeys all traffic laws, never breaks the speed limit, is silent, has no steering wheel and no personality? This is Google's vision of the future. or Do you want a self driving car that smartly makes decisions to suit your life (including breaking the speed limit if you are running late assuming it doesn't jeopardize your life or those of the ones around you), lets you take control, makes you a better driver through a built in track day driving assistant with laser projection and haptic steering feedback, and has a witty personality? This is Speak With Me, Inc.'s vision of the future (and for that matter Knight Rider's too. This is why our tagline is "Make your products come alive." Who wants a boring robot?

Speak With Me interviews Ferrari drivers, not Prius drivers.

[spkwme]

Empathic Products - Bringing Back Our Humanity.

This past several months, we’ve been overwhelmed in a great way, with customer requests. I feel like a German Shepard is bowling me over every day right now. I personally have been overwhelmed with the amount of meeting requests - and this weekend I am so looking forward to disconnecting - spending time with awesome friends, and relishing in being a car nut.

And this has given us greater insight - into something that’s sorely lacking in today’s products.

Empathy.

This started to take hold, when Facebook met the iPhone (Thank you Steve Waterhouse for crystalizing that moment over lunch today). Suddenly, human society actually started behaving worse.

We interrupted our in-the-moment dinner conversations with the people closest to us, and replaced that with staring at a screen, “liking” posts from people that are - not present in the moment at that dinner table.

As humans, we lost a lot from that. Communicating with too many people - is emotionally exhausting. A ton of research has been shown that humans cannot have close emotional ties with more than 150 people. This is known as the "Dunbar Number."

The moment we stare at a smartphone screen - is the moment we don’t notice our significant other smiling at us - it’s the moment we don't notice someone brought a gun onto the San Francisco Muni - it’s the moment we don’t notice that cute girl or guy at the bar that was flirting with you. The loss of too many of these moments - is damaging to the core of our humanity.

What’s cool about voice - is that it’s a partial attention user interface and - when there’s no screen with “suggested posts” - it enables - and requires - each person to focus on what is truly most important to them. The only people who you’ll ask about the status of from Facebook - are the people you have mindful thought of. And while Facebook would like us to believe that every person you have ever met is important - they just aren’t.

Being overwhelmed with information makes us grumpy. A hug from a close friend - makes us smile. Celebrities intimately understand this. The demands for them to be “on” are unnatural. Celebrities are humans too - and they have meltdowns and attack the paparazzi - because - too much unwanted attention -- actually makes us unhappy.

The internet of things, wearable computing, is all about computers getting out of the way, so we can get what we need done, out of the way - more quickly - and spend more quality time with our dear family, closest friends, and favorite furry and feathered friends.

Voice done right - which Speak With Me provides as an affordable service - is the key to being more present in our every day lives, and focusing on what’s most important to us, and stripping away the noise.

The smartphone and Facebook - are both revolutionary - but also, very narcissistic products. They demand our full attention.

Human society needs empathic products to move forward.

Nissan used to have an awesome tagline in the 80’s and early 90’s:

“Built for the human race.”

It’s about time we start designing products that way again.

Voice, Artificial Intelligence, and Machine Learning done right - enables us to filter out the noise and once again focus on what’s *truly important to us.*

Because that cute guy or girl that’s flirting with you at the bar - might be the person you end up marrying one day. And if you miss out on that opportunity because you were lost in Facebook liking someone’s post who lives a thousand miles away from you - you just replaced a meaningful social interaction with a meaningless one.

Speak With Me’s mission - is to bring back our humanity. By building products with empathy at their core - and enabling developers to build products with empathic artificial intelligence at their core - we can re-educate the planet - to be kinder, to be more present, to be more thoughtful, and to be more self-aware.

[spkwme]

What is Artificial Intelligence?

So a lot of people in the past couple of weeks have asked me a lot about AI and what is it. This is my take on it (outside of the video game realm):

AI is in the interface: voice recognition emotional understanding emotional responsiveness object recognition contextual awareness situational awareness partial attention user-interfaces predictive modeling

AI is in data: big data statistical models understanding of the data make decisions based on data. predictive modeling

AI is in learning: sparse data unsupervised machine learning big data unsupervised machine learning big data supervised machine learning sparse data supervised machine learning

AI is in creativity: audio musicality intonation

AI is in empathy: emotional understanding emotional expressivity situational awareness contextual awareness ---------------

All of these things are a phD unto themselves, so this is why we need so many people from so many different backgrounds to work on this problem together. In addition - we want AI to help humans and help humanity - and wrongly using the data could be bad for our planet. So we have to follow ethics statements on our use of data for these systems.

Do you have additions to this list?

[spkwme]

The Consumer (User) vs. The Researcher

Consumers (Users) and Researchers speak different languages.

Here is an anecdote from when we first started demoing to consumers (2004-2007):

Consumers were fascinated that "voice recognition" actually could work.

Immediately, their expectations went through the roof.

Show them a music system with voice control, changing your mind in the middle of the conversation, and advanced "smart" seamless error correction, and suddenly they "expect" to be able to ask for any song in the universe, rather than just songs that they own. It doesn't matter that their music player of that era didn't even support that functionality (enter Spotify today...)

What if you don't own it? "Well the system should ask me to buy it!"

Consumers felt at first, slightly let down by only being able to ask for music that they owned, but once they started using the system they were extremely happy with it.

The Consumer vs.The Researcher

In a consumer's mind, "voice recognition" included what those of us in research world call a "spoken dialogue system + voice recognizer + text to speech + information retrieval + the ability to run any imaginable service and execute the command." In other words, everything we've all been working on for 40+ years, *in one integrated system.* The classic researcher definition of "voice recognition" - is audio -> text on screen, not the tightly integrated system that the consumer really wants.

So the whole time consumers were telling researchers they wanted "voice recognition" but because researchers didn't understand the consumer definition of "voice recognition," we had decades of poor products in "voice recognition."

Sometimes the consumer does know what they want, but the technology to deliver the experience the consumer wants, doesn't exist yet. Consumers, in turn, take these colossal technical hurdles that need to be overcome, for granted.

The rabbit is found... by getting the researchers outside of the research bubble, and talking to consumers (users).

Get out of the lab. Make sure normal people understand what you are doing in such a way, that normal people can give you feedback. That, is a recipe for changing the world. I call it, research driven product design.

The alternative to getting out of the lab, is bringing people into your lab, and feeding them chocolate, while you demo the cool new stuff you're working on!

[spkwme]

Design Harmony

"Just like a piece of music, designing a car is about harmonizing the whole as much as you can. People will not rationally understand it, but they will sense it." - Giorgetto Giugiaro –

Giorgetto Giugaro is one of our favorite designers. Speak With Me follows a similar philosophy. -- our user experience harmonizes the whole. 

[spkwme]

Always Find The Rabbit

The Eureka Moment

Thinking back in time, Voice User Interfaces have been one of the key areas of computer science that has always had the potential to move the human race forward... so what happens when... it's your 21st birthday at 4am and you finally have a system that works the way we all wanted them to for your entire known life, since the days of Star Trek, Knight Rider, and 2001 a Space Odyssey?

December 5th, 2001, was that moment for us.

Randy Pausch's lab was the one next door to the lab we were in at CMU, that of the Carnegie Mellon Interact Lab.

To no surprise, his lab was the only other one with people in it at that time in the morning.

I showed the system working with a demo that lasted for an hour.. and I have a giant 4 FT tall Scooby Doo commemorating that moment. It used up more than a gigabyte of ram back then and needed a $9000 dual processor computer to run, and crashed every hour, but it worked... the way a voice user interface should.

The age of context, begins here.

The uniqueness of this dialogue system, as we call it, is that it used Context, State, and Sentence Structure to reduce the search space of the speech recognizer. We looked at speech as a contextual sensitive search problem, whereas everyone else was looking at it as an audio and acoustics problem.

I walked back home to  5302 Beeler Street with a huge grin and Scooby in hand.

Seeing the Future.

On the floor below us, there was a car that could drive itself, called the NavLab; funded by Bosch; who was also one of the main funders of our research.

The NavLab was the predecessor to Red Team Racing (CMU), Stanley (Stanford). Our other funder was DARPA. In fact, the same DARPA grants were funding a team at SRI. Today you might know them as Siri. The NavLab project you might know now as the Google and Audi self driving cars.

Grad School or Start a Company?

Bosch graciously offered to fund a PhD (Thank you Jeffrey Donne and Jill Wooster for your help in making that happen), but I chose to start a company instead, Speak With Me.

Everyone will have a 1 GHZ smartphone by 2007, right?

My thinking (in 2003) was that everyone would have a gigahertz smartphone by 2007. Turns out, that was a few years too early, but I reasoned that the PhD was going to take too long (7-13 years) if we wanted to make our market window, it was better to get the core of the dialogue system in the hands of consumers, and then the stuff that was going to be in my PhD (a Situationally Adaptive Dialogue Manager for Driving Applications, aka let's make KITT from Knight Rider for real) could be done later on after the core technology had some distribution.

Speak With Me, Inc.

So we negotiated a license with CMU and Speak With Me started in July 2005.

Fundamentally, we at Speak With Me made a different bet than the rest of the industry. I personally was hell bent on running things client-side because, for a voice interface, speed is one of the four main things for a system to feel human like. Today we have a system that runs majority client side and responds to your requests in a tenth of a second. It's 50 times faster than Siri. And, it doesn't need a network connection to work.

Blissful Optimism

Our first project was the most fabulous Tom Tom GPS ever during 2007-2009, when Tom Tom was selling 12M units a year. It did way more than what Siri does in iOS 6, and the CTO of Tom Tom said we gave them a Rolls Royce, and all he wanted was a Fiat. I responded that consumers today want the Rolls Royce for the price of the Fiat. We were performing driver assistance tasks, like, "Take me to San Francisco using 380 to 280 to 101." Today's GPS systems STILL don't do that.

Hitting The Wall

Two weeks after we shipped the product to TomTom, Google gave away navigation for free on Android and Tom Tom sales dropped 80% overnight (and their stock price dropped 25% in one day). Today, Tom Tom is the map supplier for Apple's take on a product we had in mind to ship 3 years before them.

We had spent over a million dollars of our funding making that product, and I thought it would take Google at least 3 more years to map the whole planet. You know, cause mapping the whole planet is kinda a big task. Well, they actually hadn't mapped the whole planet. Apparently they only had mapped the USA at that point, but that was enough to kill the $7 billion GPS market overnight.

Unfortunately,  smartphones were still not yet fast enough to run a client side dialogue manager, the first one able to do so was the iPhone 4 which didn't come out till June 2010.

Where's the next rabbit?

In April 2010, we spent 6 hours with the Siri team on the day they were getting bought by Apple. Adam Cheyer, Tom Gruber, and I all felt the speed of the client side system was a big deal. And the ideal system was a hybrid client/server approach.

But none of us knew how to combine the two.

There he is. Nope, not there. Look over there.

This is where your passions are really helpful things. The answer to this question came from the world of music. When you paired signal-to-noise with speech recognition confidence scores from the on-device speech recognizer, we now had a low compute power way of determining what utterances to run device-side vs. server side.

This is one of our patents pending, and it came about through thinking about things from the world of music, rather than the world of computer science.

Meet Aria.

We're working on finishing this next generation dialogue manager now, I call her Aria. It's a play on our research name of Ariadne. It works both offline and online. And it helps you integrate context with your big data with your sensor data with your user interface, to power what Robert Scoble coins as The Age of Context, and Wearable Computing.

So here is what we learned along the way:

Always trust your gut instincts.

Your competitors are your best friends.

They know your space as well as you do, they struggle with the same things as you. Remember that there is plenty of pie for all of you to eat, and make sure to invite them to your parties, and go to theirs.

You each will learn to make better pie if you try one another's recipes :)

Your true friends are also your best friends.

When your back is against the wall, move your arm around the hat until you find the next rabbit.

The rabbit isn't always where you expect it to be.

Trust that there will always be a rabbit, if you are a good person and treat people kindly.

It takes a village to start a company

When you screw up, cash in your built up goodwill and faith and try again.

Remember your core motivations for why you got started down your path.

Be gracious and humble at all times.

Trust that when you give everyone in your life a hug, that the world will give you one back, as long as you keep giving hugs.

When in doubt, keep searching for the next rabbit. There is always another one.

Thank you to all of our investors, customers, advisors, staff, consultants, and all of our wonderful friends and competitors. We can't wait for you to try our next pie recipe. It's our best yet.