Friday, August 4, 2017

Microsoft is testing 'eye control' feature


Microsoft is testing a futuristic feature that will allow users to control their computers using only their eyes. 
The program - which is in beta - makes it possible to operate an on-screen mouse, keyboard, and text-to-speech using just users' eyes.
Windows 10 users with the compatible eye-tracking device already on their computer can try out the feature.



We are excited to release this experience as a beta and would love your feedback,' reads the Microsoft blog post announcing testing of the feature.

The experience requires only requires a compatible eye tracker like the Tobii Eye Tracker 4C, which unlocks access to the Windows operating system to be able to do the tasks one could previously accomplish with a physical mouse and keyboard.

The reason for the feature is not just to create a new high-tech way to use computers - Microsoft is trying to 'empower people with disabilities.'

ALS and other neuromuscular diseases that make it difficult to use your hands or prevent you from using them all together make using computers nearly impossible.

The ability to control a PC with just slight eye movements would finally make it possible for people with disabilities to easily use ordinary computers for the first time.


In 2014, former NFL player Steve Gleason - who had been stricken with ALS - sent an email to Microsoft asking the company to develop technology to address some of constraints of diseases like his, in which the death of neurons causes difficulty with muscle movements, speaking, swallowing and eventually breathing. 

For most people with the disease, the eyes are the only body part that can be moved. 

'I realized pretty quickly after my diagnosis that technology would have to become an extension of myself,' Gleason said.

'Until there is a medical cure for ALS, technology will be that cure.'


Technologists at the company's first hackathon - which encouraged employees to focus on passion projects that advance the company’s mission of empowering people around the planet - worked on developing a tool for his request.

The Ability Eye Gaze team developed the Eye Gaze Wheelchair, which allowed Gleason to drive his wheelchair using only his eyes. 


It won the grand prize out of 3,000 projects and inspired the creation of an entirely new Microsoft Research to work on eye tracking technology.

The team worked closely with Gleason's nonprofit, Team Gleason, and the ALS community - eventually, it developed the eye control for PCs. 

At last week's hackathon, Microsoft CEO Satya Nadella shared projects that inspired him over the year and announced Windows 10 will include built-in eye tracking and eye control. 

To get it working, users have to download the update to Tobii that allows it to run the program and then calibrate it. Then to enable it, go to Settings > Ease of Access > Other Options > Eye control

To get it working, users have to download the update to Tobii that allows it to run the program and then calibrate it.

Then to enable it, go to Settings > Ease of Access > Other Options > Eye control.

A launchpad that allows you to control the computer appears on the screen 


Then all you have to do is look with your eyes.

To control the mouse, just select it from the launchpad and position your eyes wherever on the screen you want the cursor to be placed. 

Then you can left click, right click double click or cancel to perform an action. 


To control the keyboard, select the keyboard from the launchpad and move your eyes to type letters, number and symbols. 

There are two specific techniques - you could either hover over each individual letter or type faster using the shape-writing (referred to as Swype on many phones).

With this method you dwell on the first and last letter of a word and simply glance at the letters in between. 

To make things even easier, there's a word prediction feature as well.


To control text-to-speech, you select the function and type using the aforementioned methods.

Your sentences will immediately be spoken out loud.
You can even change the voice and language. 

There is a lot about eye control that can be personalized - in settings, you can also adjust dwell times for typing, turn on/off shape-writing, and turn on/off the gaze cursor used to test hardware calibration.

There is a lot about eye control that can be personalized - in settings, you can also adjust dwell times for typing, turn on/off shape-writing, and turn on/off the gaze cursor used to test hardware calibration
The feature is  in beta testing, and there are already a few known issues an limitations.
The program can't function in too much sunlight and may also require new calibration if the computer is moved into a different lighting condition.
Also, the launchpad partially blocks the Tobii UI during device calibration; however, you can get around this by turning off Eye Control during calibration and turning it back on when you are done. 
You can also use touch or mouse to reposition the UI during calibration.
Shape-writing can get stuck, which you can fix by turning the feature off and on again.
Additionally, the reposition UI icon in the launchpad can take focus after exiting text-to-speech






No comments:

Post a Comment