While everyone is talking about next-gen smartphones like the very impressive Galaxy Note 8 and the highly-anticipated iPhone X, I'm more interested in what's being built on top of smartphones that will bridge us to the next big thing.¹

Let's talk about Augmented Reality.

What is AR?

You've actually seen it before, you just don't realize it:

• If you've ever watched a game of football on TV, the yellow line on the field that constantly moves to indicate the first-down…that's augmented reality.

More recently:

• If you've ever went outside hunting for Pokémon characters…that was augmented reality.

• If you use Snapchat to turn your face into a dog and send it to all your friends…that's augmented reality.

Not to be confused with Virtual Reality, Augmented Reality is about overlaying digital information on everything you see. To put it another way:

Virtual Reality is about teleporting you to a new world. Augmented Reality is about enhancing the world around you.

AR Today: Incubation Phase

Here's the current landscape of AR summed up by Timothy Buck:

Google's Project Tango has been around since 2014, and it is almost universally considered a flop. Project Tango only works on specialized hardware that relatively few people own, and because of that, the software landscape is bleak.

Update: Google's ARCore, which is basically Project Tango minus the required specialized hardware, was announced just a few weeks ago. When it launches (release date TBA), it will be limited to Google Pixel (1+ million units shipped) & Galaxy S8 (20+ million).

Microsoft's HoloLens was announced in early 2015, and it's really quite impressive by all accounts. But it's not really a consumer product. The HoloLens starts at $3,000 and is marketed as a developer edition. At this point, they have too few users to truly attract developers in large numbers. Obviously, this could change dramatically if they announce a truly revolutionary consumer device.

Facebook's AR Studio is only a few months older than AR Kit, and Facebook has 2 billion users. But at this point, Facebook isn't offering a way for developers to monetize their AR Studio creations. This means it will be filled with AR "apps" that are essentially ads for companies that monetize in other ways.

In the fall, Apple will update their iPhone line and hundreds of millions of iOS devices being used today will be updated to iOS 11 and capable of running ARKit apps. This is serious incentive [for app developers], and I expect to see a cascade of AR-enabled apps in the App Store at the end of the year.

When iOS 11 launches later this month, Augmented Reality will be supported on iPhone SE, iPhone 6s, iPhone 7, all iPad Pros, the 2017 iPad, and all new iPhones launching this month.

How many devices is that? Some number crunching by David Barnard:

326m iPhones have been sold since the iPhone 6S was released. Some were iPhone 6/6 Plus, but 350m total compatible devices is realistic.

78m iPhones and 13m iPads shipped Q1 (Oct-Dec) last year. And quite a few 7/7 Plus in Q4. iPhone 8 will push it to 450m by the end of 2017.

To put it another way:

Overnight, iOS becomes the largest Augmented Reality platform in the world.

While everyone else gets bragging rights for demoing AR years ago, Apple hits the ground running in the race that matters — first to critical mass.²

AR 1.0: Coming Soon

This past June, Apple opened its doors to developers with the release of ARKit. In just three months, developer adoption has been phenomenal. Here are a few demos I want to highlight:

• Imagine going to a concert, fair, or festival and finding out your friends are there too:

  🤳 1. Hold up your iPhone → 2. Find your festival friends → 3. Rock on 🤟 Courtesy of ARKit 💖 Demo = @neonapp by @mrdavidurbina 🦄 pic.twitter.com/S9mp5Mjlal

  — ARKit Weekly (@ARKitweekly) August 15, 2017

• Imagine navigating around a new city like this:

  ARKit + CoreLocation, part 2 pic.twitter.com/AyQiFyzlj3

  — Andrew Hart (@AndrewProjDent) July 21, 2017

• Imagine going to a new restaurant and picking out the perfect meal:

  Apple's AR kit is MENTAL! Imagine sitting down in a restaurant and being able to physically see your food on your plate before you order?! pic.twitter.com/XDjCfMu9Co

  — Phillip Nunn (@PhillipNunnUK) August 26, 2017

• Imagine moving into a new home and planning out your interior design:

• Imagine your kid playing games like this in the living room:

• Imagine trying on new clothes, makeup, and hairstyles like this:

• Imagine floating a little virtual charm above your head as a status message for the real world. Share your mood with an emoji. Rep your hometown with your favorite sports team logo. Put up a little Fuck Off DND sign when you want to be alone.

• These two are just flat out cool:

• For 40+ more demos that I've handpicked, check out all the videos I've posted under Made with ARKit.

Yes, these demos are more on the gimmicky side. But let's put it in perspective:

When the iTunes App Store opened in 2008, novelties like Super Monkey Ball and fart apps were all the rage. They weren't game-changing themselves by any means…but getting developers to invest into the platform was paramount.

Over the years, app development matured and the world was introduced to cultural phenomenons and world-changers like Twitter, Instagram, Snapchat, and Uber.

AR 2.0: Beyond Smartphones

Today's AR comes in the form of Pokémon Go & Snapchat filters. Tomorrow's AR will come in the form of fun & gimmicky demos on next-gen smartphones. But what will AR be like in the years ahead?

• What happens when we move AR from our smartphone screens to glasses that are comfortable, stylish, and affordable? What happens to smartphones & tablets when all we need is wireless voice-enabled earbuds, AR glasses, and a powerful smartwatch?

• What happens when multiple people in the same room can share the same augmented experience? What about entire sports arenas? Entire cities?

• What happens to child development when groups of kids grow up together building & interacting with the same elaborate fantasy worlds right in your living room?

• What happens when we can replace web browsers & URLs with AR glasses & QR codes in the real world?

• What happens to online shopping when you can augment any product imaginable onto your body, your driveway, or your home?

• What happens to offices & conference rooms when we don’t need monitors or TV screens? What happens to interiors of cars when we don’t need physical dashboards or navigation consoles?

• What happens to international travel when every sign is automagically translated for you? What happens when you can talk to anyone in the world and their words are subtitled for you in real-time?

• What happens when we can hold up signs anywhere but only allow certain people to see them?

• What happens to parties & socialization when virtual name tags can only be seen by those with mutual friends? What happens to clubs, concerts, festivals, and dating when singles can display their statuses above their head to only compatible matches?

In Closing

There's so much more to innovation than showing off a flashy demo, impressing loyal fans, or being first to market. Innovation is more than just improving the status quo.

Innovation is about challenging the status quo.

Innovation is about getting new technologies into the hands of millions, driving people to break old habits & create new ones. Innovation is about disrupting the way companies do business, forcing entire industries to rethink their product strategies, operations, and business models.

The next big thing isn't a better-looking smartphone with beefier hardware specs that one-ups the competition. The next big thing will be new AR software & services in the real world...and smartphone AR will be the bridge that takes us there.

It will take years for AR to mature into a world-changing technology. Up until now, Augmented Reality has been limited to early-adopting developers & big-spending enthusiasts. AR has yet to prove compelling, convenient, and affordable enough for mainstream adoption. As the saying goes: "The future is already here — it’s just not evenly distributed."

With the upcoming launch of iOS 11 in the next couple weeks — which will unlock Augmented Reality for 450 million iOS devices this year — AR finally makes its first real step for the masses.³

The iPad Pro (with the Smart Keyboard) cannibalizes my MacBook Pro the same way the iPhone 7 Plus cannibalized my old iPad mini.

I'm blown away by how many non-work related tasks I can accomplish on the iPad Pro. And for some tasks, I can even do it faster on the iPad Pro.

My biggest discovery is split-screen mode works brilliantly because many apps are designed to work on iPhone-sized screen. Browsing and clicking on a link in a left-side app can open in Safari on the right. Apps on the desktop were not designed to work on iPhone-sized spaces.

Some of my favorite use-cases for split-screen mode are:

• catching up on RSS feeds with Reeder + Safari
• researching reviews with YouTube + Safari
• catching up on email newsletters with Newton + Safari
• catching up on computer tasks with Things + Safari

There's also something special about having a general-purpose computer that can turn on & off instantly compared to a laptop, yet having all the benefits of keyboard shortcuts.

This seriously makes me wish Apple would resurrect the iBook brand in the form of an iOS-driven laptop with built-in LTE.

Vic Gundotra, former SVP at Google:

Here is the problem: It's Android. Android is an open source (mostly) operating system that has to be neutral to all parties. This sounds good until you get into the details. Ever wonder why a Samsung phone has a confused and bewildering array of photo options? Should I use the Samsung Camera? Or the Android Camera? Samsung gallery or Google Photos?

It's because when Samsung innovates with the underlying hardware (like a better camera) they have to convince Google to allow that innovation to be surfaced to other applications via the appropriate API. That can take YEARS.

Also the greatest innovation isn't even happening at the hardware level - it's happening at the computational photography level. (Google was crushing this 5 years ago - they had had "auto awesome" that used AI techniques to automatically remove wrinkles, whiten teeth, add vignetting, etc... but recently Google has fallen back).

Apple doesn't have all these constraints. They innovate in the underlying hardware, and just simply update the software with their latest innovations (like portrait mode) and ship it.

Bottom line: If you truly care about great photography, you own an iPhone. If you don't mind being a few years behind, buy an Android.

And for clarification on his role at Google:

By the way, I ran all of Google's mobile efforts from 2007-2010. I was SVP of engineering. So I understand this topic reasonably well. I would NEVER buy an Android phone again if I cared about photography.

As the smartphone category hits maturity and physical hardware innovations become more incremental (aka "boring"), we're going to see companies put more focus on tighter software integration with custom silicon.

This just happen to be one of Apple's greatest, most underrated strengths.

As KGI reports the iPhone 8 will no longer have TouchID, Mark Gurman gives some insider info on Apple's development of 3D Facial Recognition:

For its redesigned iPhone, set to go on sale later this year, Apple is testing an improved security system that allows users to log in, authenticate payments, and launch secure apps by scanning their face, according to people familiar with the product. This is powered by a new 3-D sensor, added the people, who asked not to be identified discussing technology that’s still in development. The company is also testing eye scanning to augment the system, one of the people said.

The sensor’s speed and accuracy are focal points of the feature. It can scan a user’s face and unlock the iPhone within a few hundred milliseconds, the person said. It is designed to work even if the device is laying flat on a table, rather than just close up to the face. The feature is still being tested and may not appear with the new device. However, the intent is for it to replace the Touch ID fingerprint scanner, according to the person.

The intention sounds promising, but I have concerns about this. If true, Apple would be putting their entire security technology in one basket and security reputation on the line. There would be a lot riding on this, especially with their Apple Pay initiative.

So far, facial recognition has been proven insecure and easily fooled by printed selfies. While Samsung S8's Iris Scanner is in improvement in technology, the requirement of facing the phone at a very specific angle and distance weakens the user experience.

Hopefully Apple can pull it off.

Fraser Speirs:

The huge issue with the MacBook Pro is its form factor. The fact that the keyboard and screen are limited to being held in an L-shaped configuration seriously limits its flexibility. It is basically impossible to use a MacBook pro while standing up and downright dangerous to use when walking around. Your computing is limited to times when you are able to find somewhere to sit down.

Not that you would want to use a MacBook Pro while standing anyway. The sheer weight of these devices means that your shoulder is going to take a beating if you switch from iOS to OS X. The current 15" MacBook Pro tips the scales at 4.49 pounds - or three iPad Pros - despite having a lower-resolution screen and one less hour of battery life.

A brilliant alternative view of the MacBook Pro if judged by a tablet-first user.

What us older people forget is kids these days have literally grown up with multi-touch screen devices for 10 years. Kids who were 12 years old when the iPhone was introduced in 2007 will be graduating college and entering the workforce.

They will build their workflows and solve problems with multi-touch devices, not with keyboards & mice like we did.

To the touchscreen generation, the tablet is a real computer.

Timothy Buck:

I see a common obstacle for Microsoft, Google and Facebook's entrances into AR. I expect they will all struggle to incentivize enough developers to build a vibrant ecosystem.

Google's Project Tango has been around since 2014, and it is almost universally considered a flop. Project Tango only works on specialized hardware that relatively few people own, and because of that, the software landscape is bleak.

Microsoft's HoloLens was announced in early 2015, and it's really quite impressive by all accounts. But it's not really a consumer product. The HoloLens starts at $3,000 and is marketed as a developer edition. At this point, they have too few users to truly attract developers in large numbers. Obviously, this could change dramatically if they announce a truly revolutionary consumer device.

Facebook's AR Studio is only a few months older than AR Kit, and Facebook has 2 billion users. But at this point, Facebook isn't offering a way for developers to monetize their AR Studio creations. This means it will be filled with AR "apps" that are essentially ads for companies that monetize in other ways.

In the fall, Apple will update their iPhone line and hundreds of millions of iOS devices being used today will be updated to iOS 11 and capable of running ARKit apps. This is serious incentive, and I expect to see a cascade of AR-enabled apps in the App Store at the end of the year.

This is exactly why Apple's stubborn approach to top-to-bottom proprietary/integrated technology pays off.

While everyone else gets bragging rights for beating Apple to market, Apple focuses on becoming first to mainstream adoption.

TechCrunch:

MIT’s Computer Science and Artificial Intelligence Lab has figured out a way of measuring walking speed to within 95 and 99 percent accuracy — all without requiring a wearable or other on-body measurement device. The tech uses wireless signals, dubbed “WiGait” by the research team, sent out by router-like devices within the home to track walking speed and stride length over time.

Based on my theory that Apple will use its iBeacons and Indoor Mapping to supplement Augmented Reality in the real world, I could see Apple acquiring this tech.

Kyle Russell of TechCrunch theorizes that Workflow, an app for iOS power users that was recently acquired by Apple, is a hint at the future of Apple Watch apps:

Instead of providing an interface with options to pick from a menu or icons representing actions, Workflow on the Apple Watch has been stripped down to verbs. I want an Uber home, or to the next meeting in my calendar. I’m walking home and want to send an ETA to my roommates. Maybe I’m on BART and it’s just too tightly packed to read on my phone — no worries, I can pick a Pocket article to be read over the headphones plugged into the iPhone in my back pocket.

There are no gestures to remember or content to download to fill a feed. It’s the perfect application for the WatchKit app paradigm, with a single tap executing multiple instructions on the phone. And if, say, a destination or article needs to be picked, the pre-made workflows in the app’s gallery will serve up a few options that users are likely to choose.

Over the coming months, most developers will figure out that the best question to ask themselves when designing smart watch apps is, “What can I help users do with a single tap?” With cameras, LTE, GPS, screen size, and battery life keeping the smartphone relevant for the foreseeable future, developers should assume that users will always have a phone on them for any action that takes longer than raising your wrist, swiping once or twice, and tapping a button or two.

Another supporting argument for Apple breaking down traditional apps into its smallest, simplest actions to make more things possible on the wrist and with your voice.

If the 5.8" iPhone Pro rumors are to be true, I can see Apple embracing multi-tasking in the form of dockable widgets. Not to be confused with Android's split-screen apps that are literally apps chopped in half, but the ability to dock widget-versions of apps.