Under The Bridge Under The Bridge

Tag: ARKit
AR Reality Check: RealityKit

So it’s been a while since ARKit came out, and the App Store was briefly flooded with novelty apps, and we went off giving speeches on it and all, and that was all very exciting wasn’t it? Until the excitement of watching floating sharks and all died down, and interest is at a pretty low level these days. Incorrectly, we think, you should be keeping an eye on it, as Apple’s ongoing investment in is just too big to be purely a side hobby — as was perceptively noted here by Mr. Dent Reality:

Apple Restarted Their AR Platform Strategy, and Nobody Noticed

It’s a significant move for Apple to leave behind a two-year old platform, and replace it with something completely new. They’ve obviously made that decision with a long-term view — SceneKit had some downsides, and they thought a new framework would give them a better foundation for the future.

With that in mind, it’s interesting to think about what RealityKit brings — an AR-only framework, with high-performance, and a modern Swift-only API that regular developers can use. I don’t like to speculate too much, but I’d imagine that this is Apple’s path towards wearable AR. And features that we’re seeing emphasised today, such as anchored content, shared experiences and Quick Look — all now supported through RealityKit — could be a primary part of a future wearable product…

Well, he’s certainly doing his part … and much more than his part, check out

How to Build Really Compelling AR Experiences

Complete with open source MuseumAR – “A museum AR experience and simulator.”

(Museum apps are a commonly mooted use case — also check out Building a Museum App with ARKit 2!)

The most comprehensive overview of the new stuff we’ve seen is this series, annotating the RealityKit-CardFlip example game:

Also rather interesting is the effort Apple’s going to to make this technology user accessible. Remember last year,

How USDZ Enables New Experiences for ARKit

Creating an Animoji-Style 3D Character to Use With TrueDepth

All very interesting, but a lot of work … until now:

Introducing Reality Converter

The new Reality Converter app makes it easy to convert, view, and customize USDZ 3D objects on Mac. Simply drag-and-drop common 3D file formats, such as .obj, .gltf and .usd, to view the converted USDZ result, customize material properties with your own textures, and edit file metadata…

Which makes it even easier to create experiences like discussed here:

How to make an augmented reality decorating experience app with AR Quick Look

Here’s some more interesting articles on using CoreML and Vision to make your AR apps cleverer:

Enhancing ARKit Image Detection with CoreML

Face Detection and Recognition With CoreML and ARKit

How to Detect and Track the User’s Face Using ARKit

And although it could use some up to dating, check out Awesome-ARKit as well!

Reality and AReality: The Russia Files

Been a while since we got around to making a post here — these days, if you want to keep up with what we’re doing you have a much better chance of doing it over at our travel blog everywhs.com … or following @everywhs on Twitter … or subscribing to us on Youtube … or Instagram … or Facebook … or even Pinterest. Yep, we’re giving a full on shot at this “travel influencer” thing.

We have started to get a bit of the hang of what we’re doing over there though, so we will be posting here a bit more frequently. For starters, let’s catch up with where we left off telling you we’d been invited to Moscow for MBLTDev! We are fairly blasé about the regular run of shows … but one in Moscow? Now that’s fun! And it went pretty well considering, we talked up on it a bit here over on the new blog:

MBLTdev 2017, Moscow The Reality Side of AR

Yep, there we are with our audience participation ARKit demos in progress, wearing our new signature Indonesian batik dragon presentation shirt. Want another look?

MBLTdev 2017

More photos in this Facebook album, and if you’d like to check out the slides, they’re the first download here on Google Drive; the video itself appears to be restricted to participants, or at least people with an MBLTdev account, but if you do, hey check it out.

If you’d like to take a look at our demos, they’re up on Github:

ARSelfie: Demonstrates adding an SKLabelNode to a tapped ARAnchor in an SKScene with the current top result from a classification MLModel

ARWorld: Demonstrates adding an SCNNodev collection to an ARSCNView representing the results of an MKLocalSearch for nearby POI.

ARCloud: This was the … overenthusiastic … stretch goal here, we’d planned some Really Cool persistent presence demos using the Estimote gear, as iBeacons are one of those things we’ve never quite had the opportunity to get into as much as we wanted to, so this was a good excuse! Well, that didn’t quite work out; we went to our Plan B of just shoving some ARKit glitter on top of a stock proximity detection sample pretty much. But it all came off amusing the audience to all appearances, so hey we’ll call that a win.

So that was a great change in every way from the glorified product demos that have tended to mark our speaking career so far, and it was definitely a good conference all around; if you have a hankering to visit Moscow, definitely recommend you plan it around an MBLTdev conference!

UPDATES:

You can read their report on the talk at The Reality Side of AR — and check out the whole talk here on YouTube!

ARKit And Kaboodle

The other obviously transformational technology introduced at WWDC this year was Apple’s sudden leap from

One thing is clear: Apple needs to get moving soon.

to, a mere 5 days later — how’s that for “soon?” —

If Apple gets this right, they will own the hardware market for years to come.

and what we’re sure must be by far the quickest ever adoption of an Apple-only technology outside the computer industry,

Ikea’s plans for ARKit revealed, virtual shopping tool will launch in fall with iOS 11

so let’s start collecting links on that shall we?

Introducing ARKit

With ARKit, iPhone and iPad can analyze the scene presented by the camera view and find horizontal planes in the room. ARKit can detect horizontal planes like tables and floors, and can track and place objects on smaller feature points as well. ARKit also makes use of the camera sensor to estimate the total amount of light available in a scene and applies the correct amount of lighting to virtual objects…

… yeah ok that’s pretty cool.

There’s already signup open for ARKit Weekly to keep abreast of news here, and the same obviously extra-keen folk have also started

Made With ARKit: Hand-picked curation of the coolest stuff made with #ARKit

That’s some pretty darn nifty videos there for, like two weeks at this!

To grab some source code examples and interesting commentary, check out

Apple ARKit by Example

  1. Getting setup, draw a cube in virtual reality
  2. Plane Detection and Visualization
  3. Adding geometry and physics fun
  4. Physically Based Rendering

ARShooter – An Example Shooter Created Using iOS 11’s ARKit

ViewAR does first tests with ARKit

3 Things To Know About Apple’s ARKit

Apple’s new augmented reality platform may be its next game-changer

Pretty cool huh? It’s like the future’s so bright we have to wear shades … except the whole point of Apple AR (so far…) is that we don’t!

UPDATES:

ARBrush: “Quick demo of 3d drawing in ARKit using metal + SceneKit.”

ARKit Tutorial in Swift 4 for Xcode 9 using SceneKit

Watch a Tesla Model 3 come to life

ARTetris: “Augmented Reality Tetris made with ARKit and SceneKit”

Getting Started with ARKit: Waypoints

ARKit introduction

Wenderlich Introduction to ARKit:

  1. Getting Started
  2. Adding 3D Models
  3. Measuring Distances

I’m presenting the ARKit workshop at RWDevCon 2018!

ARKit-CoreLocation: “Combines the high accuracy of AR with the scale of GPS data.”

Inside iOS 11: The coolest Apple ARKit demos created so far

Using ARKit with Metal

ARKit and Autism: New Futures

Adventures in ARKit (with magic)

ARPaint: “Draw with bare fingers in the air using ARKit.”

ARKit for iOS Developers

Why is ARKit better than the alternatives?; Why Apple’s glasses won’t include ARKit ; Q&A on ARKit

FaceRecognition-in-ARKit: “Detects faces using the Vision-API and runs the extracted face through a CoreML-model to identiy the specific persons”

Latest demos with iOS 11 ARKit show plated food, 3D sculpting with Apple Pencil

New Augmented Reality Resources Now AvailableARKit Human Interface Guidelines

Apple invites developers, media to Cupertino HQ to showcase ARKit apps

arkit-occlusion: “A demonstration of vertical planes “tracking” and occlusions with ARKit+Scenekit”

How is ARCore better than ARKit?

Using ARKit with Metal and Using ARKit with Metal part 2

Why ARKit will be Apple’s biggest innovation in years

Animating a 3D model in AR with ARKit and Mixamo

ARKit and Core Location: Part One

IKEA App

How Is ARCore Better Than ARKit? + The Potential of Apple’s ARKit

How Apple’s iPhone X TrueDepth AR waltzed ahead of Google’s Tango

ARKit Tutorial: Understanding and Implementing 3D Objects

ARKit 1.5 examples show how vertical surface mapping upgrades augmented reality in iOS 11.3

ARKit Tutorial: Light Estimation with Ambient Intensity and Color Temperature

A Quick Guide to Designing for Augmented Reality on Mobile (Part 1) and (Part 2)

ARVideoKit: “enables developers to capture videos 📹, photos 🌄, Live Photos 🎇, and GIFs 🎆 with ARKit content.”

The AR Location Toolkit: “uses Computer Vision technology to identify the surrounding landscape, and then precisely overlays interactive information about different points of interest.”

Introducing Urban Visual Positioning — a breakthrough in location-based AR;  Blippar Launches Landmark Recognition using Deep Learning and Augmented Reality

ARKit Tutorial: Working with 2D Image Recognition

Awesome-ARKit: “A curated list of awesome ARKit projects and resources. Feel free to contribute!”

iOS Week: ARKit and ARReferenceImage

ARKit 2: Bringing richer experiences through collaboration, enhanced detection, and greater realism

AR Face Tracking Tutorial for iOS: Getting Started

An Introduction to AR Quick Look in iOS 12

Saving and Restoring World-mapping Data to Create a Persistence AR Experience

What I Learned Making 5 ARKit Prototypes

Asteroid: “A graphical tool for designers and JS developers to make ARKit apps”

ARKit-SCNPath: “Create paths for your Augmented Reality environments using just points to represent the centre of the path.”

Building voice responsive AR apps

Realistic Rendering of 3D Photogrammetry Model in ARKit

Designing 3D models for AR apps (ARKit)

6 Interesting Concepts for AR Experiences

Using ARKit and Image Tracking to Augment a Postcard

Getting Started with ARKit & Adding the finishing touches to 3D models in Xcode

Building a Museum App with ARKit 2

How To Build A Super Quick Smile Tracking App