“I’m going to try Hololens next Friday,” I told a friend in one of my computer science classes.
“To try what?”
“I have an appointment at the Microsoft Store today,” I announce to some friends at breakfast. A few raise their eyebrows.
“I’m going to try Hololens.”
“Is it like, holograph?” another pipes in.
I do not roll my eyes, but I get very close to doing so.
Apparently nobody knows what Hololens is.
(image credit: Microsoft)
In a nutshell, Hololens is Microsoft’s take on Augmented Reality. That is, Hololens integrates virtual three dimensional objects with your physical surroundings which yields a “mixed reality” of physical and virtual objects.
Last Friday, I demoed the Hololens at Microsoft’s fancy new store on 5th Avenue.
I walked into the store and asked where I should be for a Hololens Demo. They told me to go upstairs. I went upstairs and found two Microsoft personnel waiting for me at the elevator. They didn’t ask for my ID, even though my event reservation confirmation email had indicated that I would need it. After a brief wait until the demo’s scheduled time, it was time to begin and neither of the other two demo-ers had shown up. I was escorted into the elevator and up from the 2nd floor to the 6th.
Like the rest of the store downstairs, this area was beautifully designed. There were three cushioned cubic wooden seats in the middle of the room, and doors opening off to the sides. Three chairs at a table on the opposite wall each had their own device, some sort of Surface tablet.
I was instructed to sit on one of the cube-chairs and face a huge flat screen for a several minute introduction video. The video basically explained how to put on and take off the Hololens, as well as how to do an air tap: Hold your hand in a fist except for your index finger pointing upwards, then move it down quickly and up again.
After the video, a woman taught me the other gestures: air tap and drag, which is like air tap, but you hold your index finger down and move your hand, and bloom, in which you hold your palm up, fingers closed, and open them
as if they were a blooming flower. This gesture, she explained, is used to return to the default desktop or to open the start menu. After a quick measure of my interpupillary distance, the Microsoft employee handed me off to the next, and I was escorted into the first of three demo rooms.
“Welcome to my office,” he said with a smile. And what an office it was! The small room was basically a black box with a light pattern painted in lighter color. It did feel very futuristic. One wall was entirely black, to serve as a screen, and there was a small round pedestal opposite that wall with two Hololens devices on it. Finally, it was time for me to don the Hololens.
Since I was wearing glasses, he told me to loosen the Hololens and wear it around the glasses, with the front of the Hololens hanging in front of them. To be sure it was on properly, I needed to be able to see all four blue edges of a rectangle. It was a little tricky to get it right, but I got it on approximately correctly. My attention was then called to the center of the pedestal, where a bright light quickly expanded to a high definition Volvo logo.
“Welcome to the Volvo Microsoft mixed reality experience.” This part sort of felt like an ad for Volvo’s new line of cars. First, a blue, two-dimensional outline of a car showed on the pedestal. When I walked around it, it maintained its position. Then the Microsoft guy did an air tap, and the blue outline expanded to show the Volvo vehicle in its full three dimensions. Then he did another air tap, and the car burst into rays of light and headed into the blank wall on the opposite side of the room for a full-sized look at its interior. At this point it became impossible to ignore the extremely limited field of view, as the light just sort of stopped way before the edge of my vision. I turned to the wall and witnessed a demo of three different modes of the Volvo vehicle’s acceleration. Then the car returned to the pedestal to give me the chance to do some air taps to customize it, and then a graphic about the vehicle’s different sensors displayed. I walked around it as the Microsoft guy was lecturing, and was amazed as the vehicle maintained its position. The sensor labels also updated to reflect my position – there were two types of sensors, and the two closest to my position were labeled at all times. Finally, the demo returned to the wall for a highway demonstration of the vehicle’s sensors in use. I saw the car slow down to keep a safe distance behind a truck in some sort of auto-pilot mode, and quickly stop to avoid hitting a stopped vehicle when the truck suddenly switched lanes. Let me tell you, the particle smoke coming out of that van was beautiful. More beautiful than any smoke I have ever seen… The demo concluded with a life-size preview of the car and the Microsoft guy’s statement that this sort of technology could be used to demo vehicles or other products in a mall or in the comfort of your own home before buying.
I took off the headset, and was led out into the other room. I asked about the field of view, as it felt severely limited. The response was “Yeah, a lot of people have been mentioning that.” I was then escorted into the second room…
The second demo was a game, presumably some version of RoboRaid. After putting on a different Hololens, I started the game by anchoring the menu with an air tap and spinning in a circle to allow the device to do spacial mapping. This means that the device scans your space and determines where walls and objects are, as well as if they are hard or soft. Another air tap started the game, and an alien robot-ey thing made a quick crash landing in front of me. Of course, because of the limited field of view, this means that it jumped into the middle of my vision, went down a little bit, and then disappeared somewhere in the middle of my vision. I looked down, and saw the mech-ish white and blue robot or alien thing. One of his legs appeared stubbed, and there were blue particles leaking from it. He quickly recovered, hovered, and addressed me.
“Don’t be afraid,” he said. “They’re coming. They’ll kill us all. But If I give you my powers, maybe we can stop them. Come closer…”
I stepped closer, and suddenly the being erupted in a flash of blue light, which channeled its way into my body. Or rather, into about the middle of my view, where it disappeared, but it was supposed to represent light going into my body. Then a crack began appearing in the wall, and a metal thing broke through. The first wave of enemies had arrived. They looked sort of like robotic flying lobsters. The game was played by aiming with my head and shooting with the clicker, which basically just bypassed the need to air tap, to make it a bit easier. It took a little getting used to aiming with my head – not my eyes – and clicking with the clicker but not aiming with it at all. But there was plenty of moving around, ducking out from under fireballs and lasers, running around to get a better angle at the different enemies. After every few robots were killed, more would break in from a different area, and I would have to find them. While I would have much preferred finding them with peripheral vision, there were actually arrows in the limited field of view pointing towards any enemies outside my view window, as well as audio cues such as “To your right!” Three dimensional sound effects also added to the experience. I won the game with 47 kills and scored 8900 points, the highest ever on that particular device.
I came out from that demo sweaty and happy. I had just played a game in augmented reality, and it felt great. Granted, the field of view was bad, but they used what limited field of view was available pretty well, and the graphics within that tiny box had been convincing enough. I came out really excited for augmented reality gaming of the future.
The third and final demo was held in the third and final room (Surprise!). The room was small, with a couch in one corner, and a windowsill next to it. Besides a few hanging paintings, the rest of the room was basically blank. This demo was the “desktop” of the Hololens – the basic setup that that will be shown when you aren’t using any more immersive 3D app. The Microsoft employee had, according to her story, arranged the room as she would her living room. Thus, a hologram of her dog rested near a lamp in the corner, a hologram of her cat was on the couch where I had been sitting, and a hologram of a hamster in a hamster wheel was resting on the windowsill with another hologram of a record player with the ActionGram app logo above it. Her hamster, she told me, had died two years ago, and this was her way of remembering it. I hope that was made up. There was also a picture between two picture frames on one wall, and a skiing video with a play button in the top left corner of the space between the door and another picture on the wall. Finally, a globe hung in the corner in front of the door, and a bunch of planets and stars hung out in a dark patch above the couch. “This,” she said pointing to the planetary diorama, “is my space.” I didn’t comment.
She guided me through the process of resizing the video. I was instructed to say resize, and then air tap and drag the bottom right corner of the video to resize it to my liking. When I was done, I wasn’t sure how to stop resizing it, so I did a bloom and the video stopped following my air tap and drag. That threw her off, as that bloom wasn’t part of the script. She told me to say “done,” but I didn’t have to because the bloom had exited the resize and left the video in its proper size. But she couldn’t see that, as she wasn’t wearing a Hololens, so she made me say “done,” and then told me to watch the video from the couch. It was an infinite loop, so after the first run-through, she told me to say “stop” to stop it. Next, we tortured a zombie (I think his name was Fred?) by turning on the record player under his feet, and dragged a T-Rex from the ActionGram app to the corner by the door. Saying “Full Size” caused the T-Rex to grow significantly, and an air-tap played a roaring animation, which was beautiful. I played it twice and walked around to get different views of the T-Rex, and it worked very well. Except, of course, for the parts that were cut off by being outside the view area…
And that was it for the demo. I did a quick feedback survey on one of those Surfaces I mentioned before, in which I told Microsoft what kinds of apps I’d like to develop and that I need a larger field of view, and then took the elevator down to my relatively boring reality.
So that was my Friday.
The technology behind the Microsoft Hololens is absolutely amazing. The holograms were consistent with my movements and didn’t cause any sort of dizziness.
Although I’d heard some complaints about the weight, the headset felt like a feather to me. The gaming was immersive, and I could see my life benefiting from augmented reality in terms of additional freedom with the Windows 10 Desktop, gaming in which I’m forced to move, and virtual product demos. I definitely worked some muscles that are out of practice with that RoboRaid game, and that type of game will be far healthier than the games of today and the intense thumb exercises they entail. However, the field of view issue is huge. I mean, really, really big. Or perhaps I should say, really, really small. Microsoft has been going around with these fancy cameras demoing in front of large crowds at conferences etc; and showing the holographic world as seen by a guy wearing a Hololens, and they’ve been getting tons of support. The crowds are wowed, but at what cost? It all comes from the little white lie that this is actually what he is seeing – because the guy demoing it only sees a fraction of what is displayed by the fancy camera, as the device is actually aware of more than fits into the field of view… Perhaps it would have been a bit more comfortable and taken up just a bit more of my view if I were not wearing glasses, but Microsoft has advertised that the Hololens works with your glasses on, so it shouldn’t make such a difference.
In conclusion, the technology works, the holograms are real, but the field of view is very limited. If you’re a developer, you can work with it. If you’re a consumer, you shouldn’t have to. As of now, this is all fine, as the Hololens isn’t being marketed to consumers. It’s just out there as a proof of concept and to see how enterprises and independent developers use it to make different kinds of mixed reality apps. But there have been no promises from Microsoft to increase the field of view before a consumer release. In fact, Microsoft told techradar that they will not be increasing the field of view:
“We have nothing to announce with regard to a new HoloLens today,” Terry Myerson, Executive View President of Windows & Devices at Microsoft, told me at Computex 2016.
“But there will be partners who build products with wider field of views with Windows Holographic. I’m certain of that.”
If you’re a consumer, you can look forward to some other company’s devices built for Windows Holographic, say, two to five years into the future.
If you’re a developer, especially one in control of a business, you can (and should!) consider dishing out $3k to toy with this emerging tech.
If you work at Microsoft on the Hololens team, or know someone who does, please push for an increased field of view. If not that, eye tracking – as opposed to just head – would also make a world of difference.
If you’re a gamer, you’re better off investing in virtual reality such as the HTC Vive or Oculus Rift for now, but there will be some awesome mixed reality games available in the next few years.
Finally, if you are close enough to NYC or Sydney, I strongly recommend you try Hololens for yourself. While they do advertise the demo is for “developers” specifically, nobody is quizzing you, and anyone 13 or older can register for a Hololens demo.