In this article we are going to look at Augmented Reality and the various AR platforms available to create amazing AR experiences with.
If you are new to AR then please check out our AR guide first to learn more about this exciting new technology.
AR Technology Features
Before we go further, lets bust the jargon and explain the different features commonly found in augmented reality platforms.
Image Tracking
This is the ability to recognise defined flat images and then track them as you move about. It used to require special markers or QR codes to work, but now we can use almost any image (called Natural Image Markers). Image Markers can be almost anything distinctive – an image that is not part of repeating pattern, has sufficient detail and contrast.
Some platforms allow you to keep tracking the marker when it moves outside the view of the camera (called Extended Tracking), if not it means your virtual objects can disappear if the camera cannot recognise the image any longer which is jarring.
Use case: Point the camera at a poster and it comes to life with a beautiful animation.
Plane Tracking
Is the ability to detect pretty much any flat surfaces (usually horizontal ones). The simplest and most useful example is the floor. So you can place virtual objects on the floor without them appearing to ‘float’ above it – even casting shadows.
Some SDKs can track multiple flat surfaces like table tops and walls. Note there are two things going on here – one is detecting that there is a flat surface, and second is knowing when that surface ends (where the edges are). Most SDKs know where the floor is, but calculating table edges is harder and is less reliable and it can take time for the application to work this out.
Use case:
Simple – You place a dinosaur on the floor of your office and it roars at you.
Advanced – a little character is walking on the table in-front of you, gets to the edge and jumps off.
Lighting Estimation
The ability to know how much light there is in the environment. This allows us to place the same lighting to the virtual object as in the real world thus making it blend in the scene more realistically.
Note this is usually quite limited, usually it knows the amount of light but not the direction (so you can’t calculate the direction of shadows etc..). ARKit has a cool feature that allows you to build reflection probes up from the camera, so shiny objects can actually reflect their environment – which we used in our AR dance project 0AR.
Use case: A robot walks around in the office, but because it is indoors he isn’t overly bright and looks more natural. Outside the robot is brighter.
Shared Worlds
This is quite a new feature allowing multiple devices to share an understanding of the environment – enabling multi user experiences. This means each user can be looking at the same objects located in the same space and even interact with it.
Use case: AR users play a game of virtual ping-pong against each other, they can see the ball move between them.
Face Tracking
The ability to recognise and track features on human faces (and occasionally pets too!) like the eyes and mouth. This allows you to augment people’s face with virtual items or manipulate how people look – like all those funny camera effects in Snap and Facebook.
Use Case: User looks in a magic mirror and they have metamorphosed into an alien monster
Body Tracking
The ability to track a person’s body – e.g. arms, legs and hands. Although the technology has been around for a while (e.g. Xbox Kinect) – this is rare at the moment on mobile devices. Facebook can track hands allowing you to perform simple gestures and the application to understand it. Obviously this works better on wearable devices when your hands aren’t holding the phone.
Use Case: User uses their hands to cast a magic spell that sends off a fireball.
3D Object Tracking
The ability to recognise 3D objects e.g. cups, toys etc.. so it doesn’t matter what direction you are looking at the item from. This is harder problem than detecting just a flat image, so not all SDKs support it yet. You have to tell the app what the 3D object looks like as well, usually involving uploading a 3D model of it or scanning it with another app.
Use Case: User scans a toy car, in augmented reality, the car grows rocket launchers out of the side and fires them.
Occlusion
By default all virtual objects in AR are drawn on top or in front of the real world. This is fine if you have a virtual teapot on top of a real table. The problem comes when you want a virtual object to move behind a real object, because if it drawn on top the illusion is broken. Occlusion in AR is quite rare as it is technical hard to solve, but we will start to see this in future iterations and headsets like Magic Leap do this already. Spark AR also does this but only in the specific case of faces.
Use Case: Robby the robot walks behind the desk, and the desk occludes Robby allowing him to hide behind it.
Cloud Database
Some SDKs provide a way to update the images and objects that the app can recognise online. Although convenient for the developer, it usually isn’t that hard to build this functionality on top of SDKs that don’t – so its not a game changer.
Use Case: App users can tag and create their own markers and share with others
Geo-tagging
Placing augmented reality content in real locations on a map, like in Nintendo’s popular Pokemon Go. This uses GPS to work out where you are, so that accuracy of the tracking can be variable. In the future, AR devices will recognize where you are by seeing and understanding the environment, which will provide more accurate tracking, but that’s one for the future.
AR SDK Comparison
I’ve created a table of what features each of the AR technology providers offer. I’ve not included wearable AR (like Hololens and Magic Leap) for now, because availability of these devices are so limited currently so it’s not a mass consumer proposition yet. However we will update in the near future.
Please note these are always changing, so check back regularly.
ARKit vs ARCore
ARKit is Apple’s solution. It comes free as part of the Apple developer kit. This is probably the most complete solution on the market right now. The main caveat is that it only runs on Apple devices. Integration is done predominantly via app, so if you want your audience to access your AR content then they will need to download your app from the AppStore.
However Apple have also integrated AR into a lot of their apps including Safari, Messenger and Mail via their AR Quick Look feature. This means that you integrate simple AR content directly into websites and mails. You can only see simple animations, there is no support for image markers or custom interactivity. But still an exciting development to bring AR to a mass audience.
ARCore is created by Google and again comes free. It actually runs on Android and iOS, though it essentially wraps ARKit on iOS. The feature set is not quite as rich as ARKit yet, but has many of the most commonly used AR features built in. So a good option for developers.
Both provide integration into Unity – a popular 3D application development tool.
Spark AR & Lens Studio
Spark AR (formally called Facebook AR Studio) and Lens Studio are Facebook’s and Snapchats solution for building fun and engaging AR experiences right into the camera of their apps.
The big advantage of developing AR experiences for these platforms are that your audience doesn’t need to download a seperate app to access your content. If they have Facebook or Snapchat then they just need to click on a web link which will open the app directly into the camera.
Both platforms are very easily and quick to develop for, reducing cost, and they are surprisingly powerful. These are best suited for small bite-sized content – like face effects, as the overall content size allowed is very small (e.g. 2MB on Facebook).
Third Party AR APIs
There are a multitude of third party API’s out there for developers to integrate into. The most popular one is Vuforia which has been around for many years. But there is also Easy AR, Blippar, Kudan, Wikitude and others. We find Vuforia very powerful, however their license structure is opaque and if you are working on enterprise applications you can be left with a very nasty bill which is why we no longer recommend it for clients.
Instead our current pick is Easy AR – it has a very similar feature set with a clear license policy: Free for the basic version and $400 single payment for Pro. Their documentation and community are not as strong as Vuforia, but for experienced developers this shouldn’t be too much of an issue.
Conclusion
Augmented Reality is a very exciting area with rapid advances in technology. However it’s still best to work with developers with a proven track record, as there are lots of cavets and gotchas around this technology that can affect the end result of your campaign.
In the next article we will cover the mixed reality and various wearable headsets that are coming out.
If you have any questions or would like to explore what AR can do for your business, check out our latest work or please do get in touch.