The SixthSense device is an advanced technological invention that could irreversibly change our outlook on the future.
This technology, first put forward by MIT student Pranav Mistry, is a portable gestural interface that enriches the world we live in by making digital information accessible through simple hand gestures.
Mistry is breaking down the barriers between our physical world and the online universe through a very simple device that consists of a tiny camera carried around the neck, a small-sized projector and a band wrapped around the fingers, all connected to a laptop or a mobile phone on which an advanced software has been installed.
Mistry’s mission of enabling anyone to make his or her own device — the price of which should not exceed US$300 — goes along with his open-source software policy.
Free access to this new technology and the possibilities at hand got the attention of students from the Engineering Faculty at Cairo University, who centered their graduation project on the SixthSense device.
Mahmoud Khairy, a tutor at Cairo University’s Engineering Faculty, said he and his colleagues chose this project in particular because they found it a challenge to develop this very advanced technology in Egypt for the first time.
With this new technology, all the information available online is brought to us through hand gestures, thanks to little bands wrapped around the fingers.
This device, as Mistry says in a TED conference, will help humans exit our attitude of sitting in front of machines all day to actually indulge in a much more creative way of relating to technology. It can project images on any type of surface — a wall, a hand, the floor — then instantly becomes an interface where digital and physical worlds interact in a very inspiring way.
A picture of a garden, for example, can be taken just by forming the shape of the frame with both hands. Then the picture can be picked by the fingers and transferred on a wall before being sent to a friend, without ever approaching a computer screen.
Some of the SixthSense applications defy common understanding. For example, instead of checking the departure time of a plane on the computer, merely screening the boarding pass with the device will suffice to know if there is a 30-minutes delay or not. This info is projected directly on the boarding pass in glowing white lights.
There are also potential uses of the device in museums, where you can instantly access a building map and information about the museum. Also, as Mistry demonstrated in one of his videos, going to a bookshop can be an entirely different experience. Picking up a book from the shelves, Mistry opens it on the first blank page and, immediately, critics from The New York Times appear, as well as videos related to the same topic.
In order to make a phone call, the only interface you need is your hand — the number will appear on your fingers so you can dial. A video can be displayed on the newspaper you are reading to add images to the article. Mistry explains during the conference that some people have approached him, asking about how his idea could be tailored for people who lack one of the five human senses. How can this device help the mute communicate? Can it help blind people see?
But, more concretely, how does the SixthSense device manage to connect the physical and digital worlds? Understanding the type of the application the person is doing thanks to specific hand gestures, the device researches the internet, which is connected to the device through the mobile or laptop.
The camera recognizes and tracks the user’s hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data computerized by the camera and tracks the locations of the colored markers at the tip of the user’s fingers. The movements and the arrangements of these markers are interpreted into gestures that act as interaction instructions for the projected application interfaces.
Khairy from Cairo University is proud his students managed to implement the program on their own without having to refer to the original inventor of the device. Khairy said the design of the software is the core of the SixthSense system.
“The software is the operating brain of the device’s technological system,” Khairy explains.
While making the program, the student’s main aim was making a website where the technology used to create the device would be clearly displayed and explained to encourage others to make the SixthSense program. They have added a new application to the program, and they are now considering making a robot-like application that would receive orders from the user and implement them to take photos, write messages and so on.
Khairy and his colleagues hope that the device can be industrialized on a wide scale.
All the device’s hardware parts — the camera and the projector — are not unfortunately not available in Egypt. Khairy blames Egyptian media for the lack of attention to this unprecedented device.
For its superiority, the SixthSense device was granted the first award by Nahdet El Mahrousa Foundation. This contest awards innovative graduation projects from governmental universities, and each winning project is granted LE6,000. According to Mohamed El Zarkani, the foundation’s director of young innovator awards, “The program aims at boosting scientific research and creating connections between university research and industrial applications.”
A promising step in the future of the SixthSense device is that some companies, such as Nokia, Samsung and Microsoft have started manufacturing it in accordance with Mistry.
Dr. Rabe’i Ramadan, an assistant computer engineering professor at Cairo University, is currently contacting the concerned ministries to adopt the SixthSense project so that Egypt can catch up with the fast-moving train of technology.
