A new ARKit version released. Will Apple lead or will Apple follow, in this war on Augmented Reality platforms?
Apple’s ARKit revealed to the public at the World Wide Developers Conference in June (WWDC 2017), last year. Since then, our developers community has been increasingly busy in developing interactive Augmented Reality apps.
Recently, Apple introduced ARKit version 1.5 and we could not ask for a better augmented reality mobile platform but Apple, at best, is known for making hero devices. What is with the attention on Augmented Reality?
APPLE’S FOCUS ON AUGMENTED REALITY
When Tim cook was asked the very question, he deflected of course, but answered that Augmented Reality at present is at a stage that is very fulfilling and opens many doors to uncharted territories. He further added that they have many fantastic people in the Augmented Reality team, lead by Mike Rockwell, pioneering on new AR concepts, which puts Apple in a unique position to lead in Augmented Reality.
“I view AR as profound. Not today, not the app you’ll see on the App Store today, but what it will be, what it can be, I think it’s profound, and I think Apple is in a really unique position to lead in this area.”
Tim also shared how Augmented Reality adds value to their customer’s experience. According to him, AR enhances user experience like nothing before. It brings people back to the world, instead of separating them. His exact words were,
“Augmented Reality amplifies human performance instead of isolating humans, It’s a help for humanity, not an isolation kind of thing for humanity.”
Before we move on to the next section, I want to tell you what Augmented Reality is. In simple words, Augmented Reality is a combination of the digital world and the physical world. With the help of AR platforms and AR integrated devices, you can place digital objects in the real world and interact with them through your phones and tablets. Apple developed their ARKit for this very purpose.
WHAT IS APPLE’S ARKIT, EXACTLY?
ARKit is an Augmented Reality platform for iOS devices introduced by Apple. With the recent ARKit version release, Apple integrated it as part of the iOS 11. So in simpler words, it’s a developer’s tool that allows you to create and run ARKit apps.
Let’s use some ARKit examples to better understand it.
This is an ARKit app. A very basic but equally immersive and interactive ARKit example. Did you notice the shades and reflections on the car’s surface? That is how real ARKit apps feel.
Looking at another ARKit example.
Apart from dancing along with the video, how amazing was the part when the user stepped inside the A-ha dancing animation world and could actually view real people as part of the animation and vice versa?
Eager to know how this Augmented Reality platform works?
HOW DOES ARKIT WORK?
Fundamentally, ARKit works by using the iPhone’s camera to map out the environment. One of the most core functionalities is recognizing where the walls and floors are. It helps in establishing the geometry of the space.
The real interesting part is how ARKit analyzes and processes our environment. There are three main features.
WORLD TRACKING
ARKit understands the world by using Visual Inertial Odometry, VIO.
This is a feature responsible for understanding the device’s location and orientation relative to the physical world.
That is computed by
1: Taking in images from the TRUE DEPTH CAMERA and combining that with
2: The data received from the device’s motion sensor.
This is what allows ARKit to dock digital objects in one place as well, so when you move around, they stay in place.
SCENE UNDERSTANDING
Simply put, this is plane detection. This is what allows ARKit to determine the attributes and properties of the environment. So it accurately understands and maps out planes like recognizing the surface of floors and tabletops.
LIGHT ESTIMATION
Applying the right amount of light is crucial to the level of reality of digital objects. ARKit estimates light application down to the very precise level. Whatever lighting conditions are present in the existing environment, ARKit applies the same level of light on the digital objects you place in the physical world.
HIGH PERFORMANCE HARDWARE RENDERING OPTIMIZATIONS
The good thing about Apple’s devices is, they are optimized to work with augmented reality apps. You do not need to setup anything externally to get the best Augmented reality experience. AR supported devices (covered later in the article) come with hardware that does the heavy processing for ARKit apps.
A CPU, central processing unit, for processing World Tracking and Scenes.
A GPU, graphics processing unit, that renders realistic immersive graphics up to 60 fps.
An ISP, Image signal processor, for processing real-time lighting estimation.
These processors deliver revolutionary performance that enables fast scene processing and lets you build detailed and interactive virtual content on top of real-world scenes.
These are just the processors; the speakers play a special part in our Augmented reality experience as well.
The speakers can imitate SPATIAL AUDIO.
They adjust the sound volume with respect to your distance to the virtual plane. So, for example, if you are leaning in on the plane where all your virtual content is present, you will hear louder more clearer sound. If you move far away, the sound will adjust accordingly. Even if there is a large object in your field of view, the speakers will imitate a muffled sound.
APPLE ARKIT SUPPORTED DEVICES
AH! To the biggest question, Can I use ARKit?
Well, yes you can IF you own an IPhone 6s or above and a MacBook.
Mobile devices with bionic processor A9, A10 or A11 are compatible with Apple’s ARKit. Here is a list
- iPhone SE
- iPhone 6s
- iPhone 6s plus
- iPhone 7
- iPhone 7 plus
- iPhone 8
- iPhone 8 plus
- iPhone X
- iPad pro models (9.5, 10.5, 12.9)
- 7 inch iPad (2017)
Now for the list of devices, Mac computers, compatible with ARKit through which you code for AR apps include:
- MacBook (Late 2009 or newer)
- MacBook Pro (Mid 2010 or newer)
- MacBook Air (Late 2010 or newer)
- Mac mini (Mid 2010 or newer)
- iMac (Late 2009 or newer)
- Mac Pro (Mid 2010 or newer)
This is basically all Apple devices with the OS Sierra 10.12.4. So aim for the OS.
HOW TO CREATE AR APPS USING ARKIT
First Step: Download iOS 11 on your mobile device, if you don’t already have it.
Second Step: You need to register for an Apple developer account. You can also upgrade your regular consumer account to a free/paid developer account. Choose for yourself Here.
Third &Final Step: Download Xcode version 9 on your Mac computer. Fair warning Xcode requires a lot of space. 4.9 GB on download and 9.76 GB once unzipped.
Why did you download all of these?
- Xcode will be the coding environment in which you code.
- iOS 11 is the required mobile OS to run ARKit apps.
So when you are done, simply open up Xcode, plug in your mobile device into your Mac device and start creating ARKit apps.
One last thing, there will be an option named Content Technology. That just means what graphics renderer would you like to choose.
You will have three options.
SceneKit – Use if you are creating 3D objects
SpriteKit– Use if you are creating 2D objects
Metal – Use if you are creating complex scenes
You can also use Other third-party ARKit app development tools like Unity and Unreal Engine.
HOW IS APPLE’S ARKIT DIFFERENT?
Google’s ARCore VS Apple’s ARKit, is a debate that will take some more time to breed viable results.
Since ARKit came first and Google had to follow up with their ARCore, Apple’s ARKit does have the advantage of maturing more over time. In addition, while Google is working to make ARCore available for millions, Apple’s ARKit is already available to millions of iPhone and iPad users.
So what is the difference between ARCore and ARKit?
Not much, at least at the moment.
Both Augmented reality platforms account for environment sensing, light estimation and motion tracking but when it comes to AR hardware Apple really out performs.
iPhone cameras are exceptionally great in capturing high detail images. Not only that, the front cams are first to incorporate depth sensing as well, while Google’s compatible devices are a close match.
It’s too early to say who will win the ARKit vs ARCore war but un till the time Google’s ARCore suffers from the fragmentation in the Android ecosystem, Apple’s ARKit will definitely stay in the lead.
SOME ARKIT LIMITATIONS
ARKit is heavy on the battery. A lot of ARKit developers have complained that it drains the battery very quickly. So you cant use it for long hours.
Also, playing a full ARKit game will require more space. You can’t play a high level game in a cramped up room. Other than that, everything is good.
ARKIT APP CATEGORIES
BEST ARKIT APPS
1: The IKEA App
Place and test virtual furniture in your home with this AR app. That not only saves a tiring back and forth trip to the store but you get to involve all your family members in the decision. Everyone is happy!
App Available: iOS
2: The Machines
An entirely immersive game, that takes you in an engaging battle. Setup your base and ready up for an all out attack. Experience Augmented Reality like never before with incorporated spatial audio features, highly detailed virtual content and deadly enemies.
App Available: iOS
3: STACK
A simple and fun game of stacking 3D boxes on top. Find a flat surface and challenge yourself for a high score.
App Available: iOS
4: MagicPlan
ARKit apps are not all fun and games. Using this app, you can create professional floor plans by mapping out a house room and so much more.
App Available: iOS
5: THE OFFICIAL STAR WARS APP
Okay, This one is for all the Star wars fan. This Augmented reality app lets you scan star wars posters and bring the very characters to life.! Play around with hologram modes or pose with your favorite characters.
App Available: iOS
6: Kings OF Pool
Pool games in AR will beat any time killing app. Play by yourself or with your friends, Kings of Pool is an amazing AR experience.
App Available: iOS
7: AR Runner
Perfect for some physical exercise. AR runner is a running game. You choose from different modes, play with friends or by yourself. Very addictive and fun, but you must be careful. Play this game in a safe place.
App Available: iOS
8: AR Dragon
If you are bad at keeping pets, you can practice with this AR dragon app and raise a baby dragon. AR dragon gives a very detailed virtual content and an immersive experience.
App Available: iOS
9: Air measure
The simplest things are the most over looked sometimes. This AR measuring app lets you measure anything anywhere. It might not sound that cool, but imagine having to measure how far you traveled, or the length of any door, or how tall a building is?
App Available: iOS
10: Carrot Weather
This weather app has a little AR robot that follows you around and updates you with weather reports, snarky remarks and much more. Do try it.
App Available: iOS
CONCLUSION
Apple’s vast user base of a million people is a great advantage for battling the augmented reality war with its ARKit platform. There are already so many high quality ARKit apps on the app store. As other tech companies join in with their own versions of augmented reality SDKs, it will be a close match with whoever competes with Apple’s ARKit.