Since back in 2014 when we got the Not-For-Sale developer model of Samsung Gear VR Innovator Edition which only works with Samsung Note 4, we started to set our eyes on the development of immersive digital media experience of virtual reality.
The original Gear VR in 2014. Feels so ancient 🙂
Along the way, we have researched on enterprise VR use cases, device platform, and development technologies. We created virtual gadget lab as soon as A-Frame (WebVR) made first release; and then tried to capture Oracle offices around different continents in 360 views to create a virtual office tour, in both A-Frame build for delivering over the Internet, and Unity built on Oculus Rift for better immersion in the Gadget lab at Oracle HQ. We discussed about virtual “First day at Oracle” as HCM Onboarding demo use case. And finally, we captured 2017 OAUX Exchange event in VR, where you can wander around the conference stations and watch the video presentation.
Comparing to the hyper activities in VR, the AR space was relatively quiet in past years, without major devices or development platform support. Project Tango provided earlier peek, followed by Microsoft releasing its Hololens headset, with development platform support such as Unity plugin. We also played with Holocube from Merge Cube, which provides a sense of you holding a virtual object in your hand. And now, with AR support built-in on mobile devices such as Android phones and iPhone, it seems that the time is finally coming to build compelling AR use cases that can be showcased on readily available devices.
Being one of the first well-built AR headsets, Hololens brought the AR to main focus
During our research on AR/VR, we concluded that AR has much broader applications in enterprise environment, where VR is more suited to entertainment and gaming. We will investigate and prototype AR use cases as main focus this year.
I have been looking into ARKit and Unity/Vuforia development platform, to address the elements of making an AR application.
1. trigger mechanism – to recognize environment and objects
There are two big categories of marker based, and markerless approach to trigger something happen. Pokemon Go is an example of markerless AR which is mostly based on GPS to determine what Pokemons are in the surrounding. While for many of indoor or limited space environment, marker based approach may provide more “spot-on” triggering and anchoring. Any identifiable thing can be a marker, from simple QR sticker, printed-out image, to a rigid object having been scanned and registered, or object recognized by trained ML/AI model.
I just finished exploring the image marker with Unity/Vuforia last week, and heard the passing of the great mind Stephen Hawking, so I made a VR video to bid farewell and send him off to the journey of Universe.
2. anchor point – to overlay contextual virtual content in real surrounding
When we detect a marker or object, we need to position a corresponding virtual object in surrounding environment, so that it fits nicely. For example, display nutrition information next to a food item, or overlay a internal view of machinery on top of that exact machine location (called superimposition). It is not always easy to anchor a virtual object onto physical environment at development time, as this photos showing the misalignment in final rendering.
Toy Jeep with virtual car – misaligned anchor point
In this case, end user should be able to move around the virtual objects to make them fitting better to the surroundings. That involves the end user interaction and changes the anchor point of virtual objects into the real environment.
3. human interaction
In addition to look at Augmented Reality environment where virtual objects blends into real surrounding, end users may try to interact with virtual objects in the scene, such as moving around virtual furniture, or even operating an equipment. This type of interaction is very close to game development, which the game platform/engine such as Unity comes in handy.
We are ready to create our first AR prototype for an enterprise setup, likely settling on Unity with Vuforia plugin, for the advantages of making builds for Android, iOS and potentially Hololens. Unity also makes virtual environment creation and object interaction much easier because of the effects and physics from its game engine.
Here is our latest prototype without art work yet
We will post updates once we completed the prototype.