Talking digital assistants have long been featured in fiction, from HAL to Samantha, but they've only been available in real life since Apple introduced Siri four years ago. Google Now followed for Android, then Microsoft's Cortana in Windows Phone and, as of this year, Windows 10. (Curiously, Mac OS X does not yet support Siri, but if you install Windows 10 via a virtual machine such as Parallels, Cortana will work on Mac.)
Digital assistants are increasingly integrated with your OS and apps, and they've become more than a way to search Google by voice. However, their skill sets vary, so we decided to interview the three assistants to find out which get the job done best.
We didn't want to ask gotcha questions or use trickery to find faults; our goal was to engage each assistant for help with everyday tasks to see how well they performed. For our tests, Joshused Siri on his iPhone 6S. Cliff used Google Now on his Samsung Galaxy S6. Tom used Cortana on a Nokia Lumina 830 and double-checked some results on his Acer Aspire S7 running Windows 10.
Everyday questions
We began with general queries we'd normally use a search engine for, such as questions about movies, TV shows, sporting events, and locations.
"When does The X-Files come out?"
The truth is out there, so we wanted to see how well the assistants could distinguish between the earlier edition of the popular science fiction series and the upcoming, much-anticipated miniseries.
Siri struggled with this question, displaying information on the original series and offering to search Josh's iTunes library for episodes of the X-Files, even after he refined his question by asking specifically about the new series.
Google Now pulled up a card displaying information about the upcoming shows, plus related search results.
Cortana showed relevant search results from Bing.
"When does Star Wars come out?"
We next asked about Star Wars to see if the assistants could distinguish between the six olderStar Wars movies and the upcoming Star Wars: Episode VII the Force Awakens.
Siri wrestled with this one, showing results for earlier movies. Josh again tried to fine-tune his query and eventually got a relevant answer.
Google Now displayed a card with the correct date and then listed related search results.
Cortana also displayed a page of correct and related search results.
"When does The Martian play next?"
Here in the Bay Area, The Martian seems to be playing everywhere. We wanted to find the next screening.
Siri displayed by default an alphabetical list of theaters showing the movie, offering a theater across the bay first. Scrolling down, Josh found a list of theaters screening the movie, sorted by how near they were.
Google Now listed the next showtimes for theaters closest to us.
Cortana also returned a chart of times for the closest theaters.
"Where is Sansome Street?"
Moving on to locations, we wanted to try a spot with a name that is easy to slur to see how the assistants responded.
Siri correctly displayed a map of the San Francisco street but referred to it as "San-som-ee" (it's pronounced "san-sum").
Google Now also showed a map of Sansome Street, pronouncing the name correctly.
Cortana gave Tom a map of the street with several more accurate results.
"Where is Walnut Creek?"
A location question using two common nouns. We didn't specify which mode of transportation we wanted to use.
Siri brought up Apple Maps and offered accurate directions via public transit.
Google Now showed a driving route to the East Bay town and a time estimate for driving.
Cortana also displayed a map to the city, with turn-by-turn directions.
"Where is the nearest McDonald's?"
Acknowledging Josh's fascination with fast food, we sought the closest McDonald's.
Siri gave the closest restaurant.
Google Now provided a listing of the closest locations.
Cortana told us it was listing the 10 closest McDonald's locations.
"Do you have a sense of humor?"
While you want your assistant to assist you, you might like to chitchat, too. We wanted to see how personable the digital assistants were and requested jokes, asked if they thought they were funny, and inquired if they could marry.
Siri was charming and displayed a nice sense of humor in response to Josh's questions.
Google Now was humorless and returned search results based strictly on the query instead of responding to the intent behind the question.
Cortana was unexpectedly funny and engaging, responding "I'm hilarious" when Tom asked if it had a sense of humor.
"What are the lyrics to 'Renegade'?"
With so many lyrics websites out there, we figured that searching for lyrics might be a popular activity. Josh, fearing the jig might be up, wanted the lyrics to "Renegade."
Siri (to Josh's relief) returned the Styx version of the song.
Google Now went current and displayed the complete lyrics to X Ambassadors' "Renegades."
Cortana broke the tie and returned a link to "Renegade" lyrics by Styx.
"When is the next general election?"
To be topical, we wanted to find out about the upcoming US presidential election. But our assistants went global with our broad question.
Siri offered a Wikipedia page on the topic of general elections.
Google Now returned information on the next UK general election.
Cortana also returned information about the upcoming UK general election.
"When is the next US general election?"
We narrowed our query to learn the US date.
Siri struggled with the question and offered again to show us a Wikipedia page.
Google Now gave us the correct date.
Cortana also returned the correct date.
"When is the next World Series game?"
Without a Bay Area team in the postseason, we weren't paying close attention to game times.
Siri gave Josh the correct time.
Google Now provided the score of the preceding game and then the time of the next game.
Cortana relied on Bing results to display the correct answer.
Outcome
Cortana and Google Now each successfully answered questions we'd use a search engine for -- in large part, we suspect, because each relied on its native search engine, Bing for Cortana and Google Search for Google Now. And the two did a good job of understanding the intent of our questions, even when we didn't provide a lot of detail. Siri usually returned relevant results but struggled a few times to provide helpful answers; we had to fine-tune questions to get usable results.
Having seen the recent iPhone 6S commercials, we knew Siri had personality. We weren't expecting Cortana to display character, too, but it did, offering humorous responses to playful questions. Google Now, on the other hand, gave "just the facts" answers.
Personal assistance
Josh observed that both Google Now and Cortana handled some search queries better than Siri did and wanted to see how the assistants would handle activities. So we agreed our final tests would cover common tasks.
"What do I have planned tomorrow?"
First we asked the assistants to check our calendars.
Siri showed Josh his schedule.
Google Now showed Cliff's first event of the day.
Cortana showed Tom's schedule and offered to add an event.
"Read me my emails from today."
We've seen the Bill Hader iPhone 6S commercial, but we wanted to see if the assistants could read our email to us.
Siri quickly retrieved the eight emails Josh received that day and started reading the latest three, going from sender to subject to body.
Google Now was able to locate and display Cliff's email messages from a specific day and person but didn't seem able to read unread messages.
Cortana only performed a Bing search using the phrase.
"Call someone."
This is a useful hands-free task while driving, for example, so we asked the assistants to call a few people in our contacts.
Siri called the people Josh intended to call -- if he had the contact stored in his phone. If Siri didn't have access to the contact, it suggested similar names to call.
Google Now found contact info for people Cliff wanted to call. When it found several numbers for a contact (mobile and home, for example), Google Now asked which number it should use.
Cortana requires Tom to log in to a Microsoft account, which it uses to fetch contacts. So if you use a Google service like Tom does to store that info, no names or numbers will show up. The Windows Phone and Windows 10 don't have an official Gmail app, so Cortana can't go through there either.
"Send an instant message to someone."
We asked the assistants to compose and send instant messages to some folks in our contacts.
Siri brought up Messages with the appropriate contact in the To field. After Josh spoke his message and confirmed sending, Siri sent the message as expected.
Google Now brought up the Hangouts app and asked to send an instant message to a person. After Cliff dictated the message, Google Now asked if he wanted to send it.
Cortana still requires a Microsoft account, and unfortunately it could not fetch Tom's Skype contacts either, despite the fact that Microsoft makes both products.
"What are the current traffic conditions on the Bay Bridge?"
This may be the most commonly asked Bay Area commuter question after "What's wrong with BART today?"
Siri opened Maps and showed live traffic conditions on the Bay Bridge, along with the time it would take to cross it.
Google Now presented Cliff with a page of search results for live traffic reports. Even asking to have Waze show Bay Bridge traffic just got search results.
Cortana asked Tom to specify which Bay Bridge he was referring to, by selecting from a list that was sorted by distance from his perceived location. The Maps app then asked for permission to use Tom's location info, and it produced driving directions from downtown San Francisco to a location on Treasure Island, which is at the midpoint of the Bay Bridge. These directions didn't indicate traffic conditions.
"When does the next ferry leave for Sausalito?"
Although we have our individual public-transit schedules memorized, it's always good to double-check.
Siri pulled up results that matched our query, with the top result being Sausalito Ferry Schedules from goldengateferry.org.
Google Now presented a Google Search page of related results, including one for the ferry schedule. When Cliff asked when his next commuter bus would leave, Google Now offered a bus schedule for a similarly named route in Columbus, Ohio, which is kind of odd, because every afternoon Google Now displays a card showing when his next bus departs San Francisco.
Cortana's speech recognition was unable to correctly hear the name "Sausalito" after three attempts. When Tom manually typed in the town name, Cortana created a Bing search using the phrase. The search results did not include an answer.
"Play a song."
You can have way more than a thousand songs in your pocket now, so we asked the assistants to play some.
Siri responded by saying that it could not find Josh's requested song in his music. It asked if it should play a radio station based on that song instead. Josh said yes, and Siri claimed to have found one. When he clicked Open Radio, he was taken to an Apple Music sign-up page. We've seen reports that Siri will not return music results for users without Apple Music accounts, and this test seems to confirm it.
Google Now found the songs Cliff requested and played most of them without assistance, either through YouTube or, if Cliff had the song in his Google Play Music library, from the music app. For a few songs, Google Now sent Cliff to Spotify with a list of possibilities. Google Now can also identify a song playing when you ask it, "What is this song?"
Cortana attempted to queue up music, found none on the device (which was correct), and responded with "I'm sorry, there's no music in your collection." It offered to do a Bing search for the phrase "play a song."
Outcome
As much as Cortana cruised through our first set of tests, it stumbled through our second round, at times unable to provide useful results or connect the dots in Tom's account. Siri did much better, ably helping Josh with everything but songs -- which seemed odd, given Apple's hooks into music. Google Now was mostly serviceable but wobbled around transportation assistance.
While the three digital assistants often surprised us with their ability to parse our requests and return meaningful results, they stumbled over other requests we thought were in their wheelhouse -- music for Siri, for example. We also were frustrated by how often the assistants returned search-result pages instead of meaningful and contextual answers. At times, the assistants behaved more like hands-free search engines instead of helpful sidekicks.
Posts
No comments :
Post a Comment