AI could predict your next move from watching your eye gaze

  • Written by Eduardo Velloso, Lecturer in Human-Computer Interaction, ARC DECRA Fellow, University of Melbourne

Our eyes often betray our intentions. Think of poker players hiding their “tells” behind sunglasses or goalkeepers monitoring the gaze of the striker to predict where they’ll shoot.

In sports, board games, and card games, players can see each other, which creates an additional layer of social gameplay based on gaze, body language and other nonverbal signals.

Digital games completely lack these signals. Even when we play against others, there are few means of conveying implicit information without words.

Read more: What eye tracking tells us about the way we watch films

However, the recent increase in the availability of commercial eye trackers may change this. Eye trackers use an infra-red camera and infra-red LEDs to estimate where the user is looking on the screen. Nowadays, it is possible to buy accurate and robust eye trackers for as little as A$125.

Eye tracking for gaming

Eye trackers are also sold built into laptops and VR headsets, opening up many opportunities for incorporating eye tracking into video games. In a recent review article, we offered a catalogue of the wide range of game mechanics made possible by eye tracking.

This paved the way for us to investigate how social signals emitted by our eyes can be incorporated into games against other players and artificial intelligence.

To explore this, we used the digital version of the board game Ticket to Ride. In the game, players must build tracks between specific cities on the board. However, because opponents might block your way, you must do your best to keep your intentions hidden.

Our studies using Ticket to Ride to explore the roles of social gaze in online gameplay.

In a tabletop setting, if you are not careful, your opponent might figure out your plan based on how you look at the board. For example, imagine that your goal is to build a route between Santa Fe and Seattle. Our natural tendency is to look back and forth between those cities, considering alternative routes and the resources that you have in the cards in your hands.

Read more: A sixth sense? How we can tell that eyes are watching us

In our recent paper, we found that when humans can see where their opponents are looking, they can infer some of their goals – but only if that opponent does not know that their eyes are being monitored. Otherwise, they start employing different strategies to try to deceive their opponent, including looking at a decoy route or looking all over the board.

Can AI use this information?

We wanted to see if a game AI could use this information to better predict the future moves of other players, building upon previous models of intention recognition in AI.

Most game AIs use the player’s actions to predict what they may do next. For example, in the figure below on the left, imagine a player is claiming routes to go from Sante Fe to some unknown destination on the map. The AI’s task is to determine which city is the destination.

When at Santa Fe, all of the possible destinations are equally likely. After getting to Denver, it becomes less likely that they want to go to Oklahoma City, because they could have taken a much more direct route. If they then travel from Denver to Helena, then Salt Lake city becomes much less likely, and Oklahoma City even less.

AI could predict your next move from watching your eye gaze Left: without gaze information, it is difficult to tell where your opponent is going next. Right: by determining that your opponent keeps looking at Helena and Seattle, the AI can make better predictions of the routes the opponent might take.

In our model, we augmented this basic model to also consider where this player is looking.

The idea is simple: if the player is looking at a certain route, the more likely the player will try to claim that route. As an example, consider the right side of the figure. After going to Denver, our eye-tracking system knows that the player has been looking at the route between Seattle and Helena, while ignoring other parts of the map. This tells us that it is more likely that they take this route and end up in Seattle.

Our AI increases the relative likelihood of this action, while decreasing others. As such, its prediction is that the next move will be to Helena, rather than to Salt Lake City. You can read more about the specifics in our paper.

Experimentation

We evaluated how well our AI could predict the next move in 20 Ticket To Ride two-player games. We measured the accuracy of our predictions and how early in the game they could be made.

Read more: Eye tracking is the next frontier of human-computer interaction

The results show that the basic model of intention recognition correctly predicted the next move 23% of the time. However, when we added gaze to the mix, the accuracy more than doubled, increasing to 55%.

Further, the gaze model was able to predict the correct destination city earlier than the basic model, with the AI that used gaze recognising intentions a minute and a half earlier than the one without gaze. These results demonstrate that using gaze can be used to predict action much better and faster than just using past actions alone.

Recent unpublished results show that the gaze model also works if the person being observed knows that they are being observed. We have found that the deception strategies that players employ to make it more difficult for other players to determine their intentions do not fool AIs as well as they fool humans.

Where to next?

This idea can be applied in contexts other than games. For example, collaborative assembly between robots and humans in a factory.

In these scenarios, a person’s gaze will naturally lead to earlier and more accurate prediction by the robot, potentially increasing safety and leading to better coordination.

Authors: Eduardo Velloso, Lecturer in Human-Computer Interaction, ARC DECRA Fellow, University of Melbourne

Read more http://theconversation.com/looks-arent-so-deceiving-ai-could-predict-your-next-move-from-watching-your-eye-gaze-97927

Australia Today

Why we don't need to prepare young people for the 'future of work'

Shirley Jackson, PhD Candidate in Economic Sociology, University of Melbourne

Ultimately, the future of work depends on what we want our future society to look like.ShutterstockWhile there is little consensus about the “future of work”, one thing is certain – young people are at the coalface. Young workers experience insufficient opportunities for work exp...

Read more

We don’t own data like we own a car – which is why we find data harder to protect

Vincent Mitchell, Professor of Marketing, University of Sydney

ShutterstockIt’s known as the “privacy paradox”: people say they want to protect their data privacy online, but often do little to keep it safe.Why?We propose that it’s because people find data difficult to own – and things we don’t own, we tend not to protect. Th...

Read more

How eye disorders may have influenced the work of famous painters

Andrew Anderson, Associate Professor, Department of Optometry & Vision Sciences, University of Melbourne

It's been argued the Impressionists were short sighted.The Boulevard Montmartre at Night, Camille Pissarro/Wikimedia CommonsVision is an important tool when creating a painted artwork. Vision is used to survey a scene, guide the artist’s movements over the canvas and provide feedback on the co...

Read more

Take heart, charity stunts can make CEOs better people

Jacki Montgomery, Lecturer, Advertising and Media, Western Sydney University

Thousands of CEOs across the country will sleep “rough” tonight to raise money for homelessness. But is one night of sleeping rough enough to truly empathise with those who do it every night? And are CEOs who take part in the Vinnies CEO Sleepout just doing it as a self-serving PR stunt?...

Read more

Bangarra's Dark Emu is beautiful, but lacks the punch of its source material

Justine Shih Pearson, Honorary Associate, Department of Theatre and Performance Studies, University of Sydney

Dancers perform in Bangarra's premiere production of Dark Emu. Daniel BoudReview: Dark EmuLights come up on an orb receding back in space, an echo perhaps taking us back in time, or into the cosmos, into the dreaming to the Emu in the Sky. Dancers, in groupings of three or five, move stage right to ...

Read more

Holidays & Travel

Dredging works underway Plantation Island

Tourists and workers travelling to popular tourist destination Plantation Island, Fiji, are set to benefit from safer access when marine channel dredging is completed at the end of June 2018...

Top 5 Tips for Traveling With Dogs

Travelling with your dog can be thrilling, but sometimes it can turn into a real nuisance. Unless you know how to properly restrain the pooch, whether it’ll be able to...

Top Island Destinations for 2018

When we picture island destinations in our minds, we often imagine a tropical beach of white sand, a sun lounger laid out in the shade of a line of palm...

Why Car Rental Excess Insurance is a Smart Choice

Some car rental agencies include basic damage cover in the standard rental fee, with the option to add on liability reduction, or excess insurance. The addition of excess insurance normally...

More Than a Third of Australians Feel 'Too Busy' With Work to Take a Holiday

New research from InterContinental Hotel & Resorts has unveiled a large number of Australians (32 percent) are simply too busy with work to take some much-needed rest and relaxation. Almost...

Matakauri Lodge and Mahu Whenua Lodge

TRIED, TRUE AND SOMETHING NEW  A luxurious accommodation package at Matakauri Lodge + Mahu Whenua Lodge South Island, New Zealand  Available 1 May, 2018 - 30 September, 2018   Central Otago, NEW ZEALAND –...

Heaven is the Adelaide Hills

Looking for a quick getaway? Enjoy food and wine? How about stunning countryside? Look no further! The Adelaide Hills has something for everyone. We flew to Adelaide for a weekend-getaway...

5 Amazing Australian Holiday Ideas

Australia is one of the most fantastic places in the world to travel, explore, and witness all of the beauty that Mother Nature has to offer. With so much rich...

Top Amenities You Enjoy by Renting Luxury Caravans for Road Trips

Do you love to undertake road trips? What the luxury cruises on the ocean are for ocean travelers, luxury caravanning is the same for land-lubbers. Traveling in a luxury caravan...

Fashion & Beauty

Simple Secrets To A More Confident Smile

We all know just how important first impressions are. After a handshake, what’s one of the first things people will notice about you? It’s your smile. A friendly smile is...

Improve Your Body Confidence and Feel Great

No matter how good you feel about yourself, sometimes you just can’t help but see a few little flaws that seem gigantic to you and make you feel self-conscious. Whether...

First-Timer Tips For Your First Lip Filler Appointment

Getting injectable lip fillers can be a quick and effective way to plump up your smile with minimal drama. They’re a cost-effective and safe way to boost the natural shape...

The Most Common Skin Conditions and How to Deal with Them

We often underestimate the importance of our skin, but as the largest organ in our body, and our primary layer of defense, it is constantly affected by many environmental factors...

Beach Babe Basics: Everything You Need to Look Stylish at the Beach

Summer is right around the corner, and this year, we all should make the best of it. Right now, it is not just about looking stylish on the streets, it...

Easy Dress-Up Tips for Lazy Girls

Look, getting all dolled up is fun, but you may not always be in the mood for it. Unfortunately, though, this is exactly when you are asked to go to...

HEALTH

My Health Yoga

Carrie-Anne Fields of My Health Yoga has taken her Gold Coast yoga teacher training worldwide. She started her health business 20 years ago aged 22 and has grown into a...

Fit Body 101 - Increase Your Fitness Effectiveness

You have to find some space for physical exercise in your weekly schedule in order to live a good life. Embracing this way of life is very beneficial because it...

The World of Dentistry: 5 Most Common Dental Procedures in Australia

Are you dissatisfied with the state of your pearly whites? Do you often experience discomfort and pain while eating and drinking? Well, be sure that you’re not the only one...

How You Can Conquer Your Dental Anxiety

Does the thought of going to the dentist fill you with dread? Well, you will be pleased to know that you are not the only one. A large number of...

Up and away: the impact of flying on your health

These school holidays, many people will take to the skies, unaware of the risks flying can pose to their health. Brisbane based house call doctor Ryan Harvey said it was important...

How to Lose Weight After 40

As you age, your metabolism slows down, which makes losing weight much harder. Additionally, your hormones start to fluctuate significantly, causing easier weight gain. Sticking to the same tricks you...

LIFE STYLE

Dry Carpet Cleaning Vs Steam Carpet Cleaning: Which one to Choose?

A clean carpet not only improve your interiors, increases your comfort around the office and home but also beneficial for your health. This is why it is necessary to choose...

Carpet cleaning: Things you need to know

Carpets are one of the most essential room interiors, and the kind of maintenance it requires is nothing worth comparison. We understand the importance of good health and hygiene at...

Get The Look: How to build a beautiful weatherboard home

Coastal, Queenslander and Contemporary Australia’s love affair with the weatherboard look can be seen in many of our historic and most popular styles – from the Queenslander to coastal and new...

The Fabrics That Make the Simple Indian Saree Sizzle

6 Yards of Sophistication and Sensuality Saree – an attire that can take you from boardroom to cruises; from kitchen to red-carpet; from lounging to marathon; from weddings to vacation –...

6 Tips for Mixing Design Styles Like a Pro

Each interior style is unique in its own way, creating a specific ambiance in our homes. While some accentuate simplicity and formality, others embrace playfulness and authenticity. Thus, you’re left...

Deborah Hutton: Designing My Dream Hamptons Inspired Home

Television presenter, magazine editor and media personality, Deborah Hutton is renovating her charming three bedroom house in Bronte, into her own Hamptons inspired forever home, with work due to be...

FOOD & DINING

Morning greens: the breakfast skincare regime

It’s safe to assume that most Aussies fail to eat enough greens. Although this was momentarily resolved with the rise and popularity of the green juice movement, the lack of...

Best Hidden Gem Hong Kong Restaurants

You finally find yourself enjoying the endless beauties of China and this fascinating city. In a place with so much variety, all your senses will be wide awake, and your...

JOHNNY DI FRANCESCO NAMED OFFICIAL AMBASSADOR OF EAT STREET MELBOURNE

Johnny Di Francesco, world champion pizzaiolo and owner of Melbourne’s Gradi Group of restaurants, is excited to announce he has been named official ambassador for Eat Street Melbourne 2018. As Ambassador...

The best Mother's Day gift EVER!

Whether it’s to celebrate a special occasion, or just wind down at the end of a long day, there’s nothing much better than pouring yourself a glass (or two…) of...

Sing for Your Supper at Chargrill Charlie’s

To celebrate the final curtain call of the Barden Bellas, Chargrill Charlie’s is giving you the chance to Sing for your Supper! Show off your best acapella vocals to win...

MAKE IT WINE TIME, ANY TIME

Whether it’s to celebrate a special occasion, or just wind down at the end of a long day, there’s nothing much better than pouring yourself a glass (or two…) of...