Under The Bridge Under The Bridge

Category: Programming
Summer of Code 2018

Well, here’s some news that’s interesting and also apropos to our Letter To A Young Developer from last time: 

  1. Are you a student developer? (If you’re not, but you know any, keep reading)
  2. Are you interested in an open source internship? (Correct answer is: YES)
  3. Are you interested in SWIFT? (If you’re not, ok wrong blog you may go now)

OK then, here’s an opportunity for you to jump on:

Google Summer of Code 2018

2018SummerOfCode

13,000+ STUDENTS, 108 COUNTRIES
13 YEARS, 608 OPEN SOURCE ORGANIZATIONS

33,000,000+ LINES OF CODE

Google Summer of Code is a global program focused on bringing more student developers into open source software development. Students work with an open source organization on a 3 month programming project during their break from school…

Which is an interesting opportunity every year yes, but this year is particularly interesting if you have designs on the Apple ecosystem:

Swift to participate in GSoC 2018!

I am thrilled to announced that Swift has been selected as a participating open source project in Google Summer of Code 2018!

…  Student applications start on March 12, 2018

The official application to GSoC starts on March 12. Before then, we encourage all prospective participates to socialize their ideas on the respective Development forum categories. Please tag all posts with the GSoC 2018 tag so that they are easily discoverable by others (e.g., students and mentors) interested in GSoC.

There are several open project ideas available on Swift.org — but potential projects do not need to be restricted to these ideas. Regardless if you are looking into working on a suggested project or one of your own creation, you are strongly encouraged to engage on the forums before applications are accepted to help shape your idea and ensure a good alignment with a mentor.

… Key Dates
March 12 (16:00 UTC) – Applications open
March 27 (16:00 UTC) – Deadline to file your application
April 23 (16:00 UTC) – Accepted student proposals are announced
May 14 – Coding begins

Personally, we think a particularly useful place to help out would be the Swift Package Manager, and if that sounds interesting here is your mentor:

So, student developers, if you haven’t got a firm résumé-enhancing plan already in place for the summer, we strongly recommend you give very serious thought to the Summer of Code in general, and making Swift better for us all in particular!

API 4 IAP PIA: SwiftyStoreKit Seriously Simplifies Subscriptions

There’s been quite a bit of angst and snark recently about what a pain in the fundament it is dealing with IAP subscriptions, you may have noticed:

Testing In-App Purchases Ruins Your Phone

Getting in-app purchases right is a goal that every app developer should have. Your customer’s experience when handing over their money should be seamless. To guarantee a good experience you must rigorously test. However, there are many pain points involved in doing this. Each individual issue is only irritating, but together they create enough friction that properly testing in-app purchases is a miserable experience…

My personal least favorite when working with recurring subscriptions: If you make a test purchase of a recurring subscription, you will be forever presented with a looping App Store login modal, usually when unlocking your phone. This renders any personal device you are testing on ruined… This is a common problem … which some people give short shrift:

My thoughts on the trending iOS subscriptions loop login complaints: read the documentation carefully and follow @Apple best practices. You’ll never encounter this problem.

Well … they’re not quite as clear as all that, really. Here is an excellent guide to the travails you’ll face:

Testing Auto-Renewable Subscriptions on iOS

Subscriptions work differently in TestFlight, the sandbox, and live on the App Store, so testing them requires knowledge of those differences. This is my attempt to help sort out the mess and document it for others to reference…

You definitely want to Read The Whole Thing™ on this one, but we’ll highlight this hack we did not know for testing live subscriptions on the App Store before release! No, really, you can do that:

For an app that has yet to be released on the App Store, getting an early version of the app approved is a great way to test subscriptions:

1. Submit a beta version of the app to App Review. Make sure to set “Version Release” to “Manually release this version” so that the app is not released on the App Store. 2. Generate promo codes for the app. This can be done for free apps that are approved, but not yet on the App Store. 3. Download the app from the App Store using a promo code. 4. Subscribe.

Since this app has gone through approval, subscriptions will perform exactly as they will when the app is live on the App Store, including charging testers who subscribe and allowing testers to manage the subscription in the App Store app. Promo codes can be given to testers so that they can test the app for free. Subscriptions paid for via promo code work exactly like paid subscriptions, except that they don’t auto-renew… Definitely want to make sure you do that. Surprises with your monetization are the worst kind of surprise, amirite?

Now, on the programming side, we feel there’s a clear best practice currently for implementing your StoreKit support in general, but particularly for subscriptions:

SwiftyStoreKit: “Lightweight In App Purchases Swift framework for iOS 8.0+, tvOS 9.0+ and macOS 10.10+”

SwiftyStoreKit Preview

Features:

  • Super easy to use block based API
  • Support for consumable, non-consumable in-app purchases
  • Support for free, auto renewable and non renewing subscriptions
  • Receipt verification
  • iOS, tvOS and macOS compatible

It’s been actively maintained up to 2 days ago as we write this, with 2733 stars on Github and 26,536 installs on CocoaPods.org — it’s healthy to be leery of introducing third party dependencies to essential parts of your app flow, but this one is downright mainstream at this point. And it comes with a complete background education:

Essential Reading

  • Apple – WWDC16, Session 702: Using Store Kit for In-app Purchases with Swift 3
  • Apple – TN2387: In-App Purchase Best Practices
  • Apple – TN2413: In-App Purchase FAQ (also see Cannot connect to iTunes Store)
  • Apple – TN2259: Adding In-App Purchase to Your Applications
  • iTunes Connect Developer Help – Workflow for configuring in-app purchases
  • Apple – About Receipt Validation
  • Apple – Receipt Validation Programming Guide
  • Apple – Validating Receipts Locally
  • Apple – Working with Subscriptions
  • Apple – Offering Subscriptions
  • Apple – Restoring Purchased Products
  • Apple – Testing In-App Purchase Products: includes info on duration of subscriptions in sandbox mode
  • objc.io – Receipt Validation

I have also written about building SwiftyStoreKit on Medium:

  • How I got 1000 ⭐ on my GitHub Project
  • Maintaining a Growing Open Source Project

So there you go, that should be all the resources you need for as painless an IAP implementation as possible!

And while we’re talking about IAP, there is a most excellent article here on design, psychology and strategy:

The Ultimate Guide To Mobile In-App Purchases Optimization

Now you can’t just create in-app purchases in your app haphazardly and let it run itself. It’s a long process that requires blood, sweet, and tears. You need to optimize your in-app purchases to make sure that you’re getting the most that you can out of them.

There are many different models, techniques, and strategies that the top-grossing apps and industry leaders use to optimize their in-app purchases and increase their revenue and we’ll be discussing all of the main and most popular ones… There’s not much here that will be completely new to you, especially if you play those pay-to-win games that have taken over the App Store these days; but it’s nice to have a concise compendium of best practices that’s up to date with iOS 11’s App Store IAP promotions to refer to. Our fave tip:

Purchases IRL

If your mobile app or game becomes big and popular enough, you can use that to offer real-life merchandise through your app. This had been the case with hit mobile games like Halfbrick’s Fruit Ninja and Jetpack Joyride that sell t-shirts and mugs through their games and Rovio’s Angry Birds that sells t-shirts, hats, toys, LEGO sets, backpacks, and more, and even went on to make an animated series and a full feature movie… That’s an interesting idea actually; if you have all this game art, why not make swag out of it? We do every so often get asked if we have shirts with the Trollsheep on them … maybe we’ll look into that. Especially now that we live in Southeast Asia and all where swag production is a thing!

UPDATES:

Apple Will Reject Your Subscription App if You Don’t Include This Disclosure iOS Subscription Groups Explained Castro 3’s Business Model

AppReceiptValidator: “Parse and validate App Store receipt files.”

iOS Introductory Prices

App Store Subscriptions And You 

IOS In-App Purchases and Subscriptions

iPhone X Notch-alance

Remember those innocent days a few short months ago when we all spotted the new safeAreaLayoutGuide and thought

Our constraints are now with the top and bottom anchors of the safe area layout guide. This is not a big change…

Oh, sweet naiveté. Did anybody actually call out in advance that subtle little change was foreshadowing THE NOTCHALYPSE? Yes, yes, every time something changes it brings out the fear and loathing no matter what it is, but the level of OMG NEVER BUY APPLE AGAIN … we saw this time, that was unprecedented. Personally, we think a phone with a bat-eared screen is kinda neat, but we’re weird. 

There’s a certain level of enthusiasm out there for clipping and masking and otherwise working around the new roundedness and notchedness … but we think Marco has it right here:

This is the new shape of the iPhone. As long as the notch is clearly present and of approximately these proportions, it’s unique, simple, and recognizable.

It’s probably not going to significantly change for a long time, and Apple needs to make sure that the entire world recognizes it as well as we could recognize previous iPhones.

That’s why Apple has made no effort to hide the notch in software, and why app developers are being told to embrace it in our designs…

So, we’ve got some rewriting to do in the next few weeks to have our apps fit nicely into The Brave New Notched World, and you probably do too we imagine; let’s see what resources are out there!

First off, go over the relevant stuff over at the mothership:

Building Apps for iPhone X

Designing for iPhone X

Updating Your App for iOS 11

Human Interface Guidelines — iPhone X 

Positioning Content Relative to the Safe Area

The great PaintCode people have updated their invaluable 

The Ultimate Guide To iPhone Resolutions

and also have specs for all the Notch attributes at

iPhone X Screen Demystified

More good reads on upgrading UI for the X at

Design for iPhone X

Supporting iPhone X

iPhone X: Designing for the Notch

iPhone X: Dealing with Home Indicator

UI Design for iPhone X: Bottom Elements

How iOS Apps Adapt to the iPhone X Screen Size — and the Adaptivity app showcased here looks like a good purchase!

Adaptivity is an app for developers and designers to visualize how iOS’s Size Classes and margins for layout and readable content look on real devices and how they change with respect to orientation, iPad Slide Over/Split View and Dynamic Type size changes…

Templates to help you out at

Apply Pixel’s iPhone X and iOS 11 UI Kit

iOS Design Kit’s Free iOS 11 GUI for Sketch

If you’re concerned with how your web sites will deal with the Notch, check out

Designing Websites for iPhone X 

Understanding the WebView Viewport in iOS 11

And let’s finish off with the niftiest idea we’ve seen for adapting in a non-approved way with the Notch:

ScrollSnake: “What if scroll bars on the iPhone X worked like the game “Snake”?”

UPDATES:

UI Design for iPhone X: Top Elements and the Notch

Designing for iPhone X: Guidelines to designing for iOS 11

Challenges of Supporting iPhone X

Fix for the missing network activity indicator on iPhone X

Finding the Real iPhone X Corner Radius

Validate My Experience

So we’re looking at updating one of our apps here for iOS 11 and joining the move to subscriptions which it appears is the only reasonable path to sustaining a productivity app on the horizon, so we’re seeing what’s new in receipt validation to support that, as last time we checked into it was four years ago and all and RMStore we picked then has languished since.

Let’s look around a bit … hmmm … hmmm … oh, what’s this? A comprehensive series walking through the whole process, updated for Swift 3? Well, bit late to the party, but looks like it’s all still relevant, and receipt validation code is something we really ought to write ourselves:

Preparing to Test Receipt Validation for iOS

After having to piece together each step along the path of preparing to test receipt validation for iOS apps, I’ve decided to combine everything into the following guide. Whether you’re working to implement receipt validation for a new iOS app, or for an existing one, this walk-through should provide guidance to get you ready to work with receipts in your iOS application…

Loading a Receipt for Validation with Swift

There are at least 5 steps to validate a receipt, as the Receipt Validation Programming Guide outlines…

Extracting a PKCS7 Container for Receipt Validation with Swift

Before attempting to work with OpenSSL’s PKCS7 functions, you’ve got to do a little prep work to get the functions to play nicely with Swift. Unfortunately, Swift doesn’t work well with C union types. It simply can’t see things defined with a C union…

Receipt Validation – Verifying a Receipt Signature in Swift

The aim of this guide is to help you take a look inside the PKCS #7 container, and verify the presence and authenticity of the signature on the receipt…

Receipt Validation – Parse and Decode a Receipt with Swift

The aim of this guide is to help you parse a receipt and decode it so that you have readable pieces of metadata to inspect and finalize all of the receipt validation steps…

Finalizing Receipt Validation in Swift – Computing a GUID Hash

After finishing this guide, you’ll simply need to use the parsed receipt data to perform any app-specific enabling/disabling of features based on the data within a valid receipt. If the receipt is invalid, you’ll need to handle that as well. But all of the relatively difficult work of working with the Open SSL crypto library will be DONE after this guide….

… The bottom line is that from this point on, you no longer need Open SSL or any additional cryptic, low-level, unsafe pointer-type stuff to finish things out. I hope this series has been helpful in setting you up to validate receipts locally on a user’s device!

That … is a veritably monumental guide to the validation process. And conveniently finished up right when we need it too, thanks ever so much @andrewcbancroft!

UPDATE:

We just beat by a day Messr. Bancroft pulling together his own collection guide, now that you’ve read ours go to the real thing!

Local Receipt Validation for iOS in Swift From Start to Finish

and code at SwiftyLocalReceiptValidator!

RELATED UPDATES:

InAppPurchase – A Simple Swift Library For In-App Purchases With Support For iOS 11’s Promotion Features

YapDatabase: Ditch Core Data

The choice of database technology for our apps has been something of a question for us the last while, where by “last while” we mean since iCloud Core Data was deprecated, which miffed us no end since we’d rather like our apps’ initial releases to Just Work™ across devices for iOS users without jumping through any great hoops or introducing any dependencies on non-Apple services.

So let’s see what’s new in Core Data this year in the iOS 11 release notes! Errr… nothing?

Hmmm. Let’s check out the What’s New In Core Data WWDC 2017 session

kinda thin, don’t you think?

Hmmm, hmmm. Sorta looking like it’s time to seriously evaluate other alternatives, it is.

We liked the looks of RocketData, very Swifty and all, but it … appears to be languishing.

Soooooo, what to do? Well, here is an article that presents an alternative of interest:

Ditching Core Data

I have been using Core Data since iOS 3.0. With few exceptions, it has served entirely as a network cache. Take data from a network response, turn it into a set of objects, save those objects, and update the user interface. Rinse and repeat.

The stack is pretty simple. A set of nested contexts to keep object building off of the main thread, a couple of models that have init(dictionary: NSDictionary), and some find-or-create helpers can get you a long way. The problem is that the apparent simplicity comes with an incredible amount of baggage. Instability, buggy classes, and outdated APIs make building modern apps difficult.

As a community, we have tried to solve these problems by building wrappers around Core Data. These wrappers add their own issues and none of them hide the fact that Core Data was simply not meant to be used like this…

Nodding along here? Yep, us too.

After some research and planning, I landed on a set of features that I wanted in a persistence framework. The list is pretty short:

  • My objects must be plain objects. Requiring model objects to extend a certain base class is painful; especially when you cannot see the source for those classes. I want to know that I can create objects however I like, use them wherever I like, and build my own functionality around them without worrying about what sandbox they belong to.
  • I should not have to worry about concurrency. Tools like closures, Grand Central Dispatch, and NSOperationQueue make it easy to write code that spans many threads. Unpacking objects on background threads that are later used to update interface elements is a common operation that should be simple and easy to reason about.
  • I should not have to write migration code. Frameworks and libraries can and should be smarter than this. Let me describe how I want the database to look. You take care of the rest.
  • I should be able to track and respond to atomic changes. Elements in my application should be able to reload themselves if a single object, or a collection of objects changes. Otherwise it may spend too much time updating things that may not need to be updated, which could have a negative impact on the user experience.

Sounds good. Only thing we’d add as a base requirement is “CloudKit sync” to make the cross-device experience seamless.

And the winner was:

I decided to use YapDatabase. It came highly recommended and it offers support for all of my requirements. YapDatabase is, at its core, a key-value store build on top of SQLite. It has a simple API and is incredibly flexible…

You will notice a few things about my model objects right away: they are completely immutable, they have no knowledge of the persistent store, and no magic superclass. These objects can be passed around the application (and across thread boundaries) without requiring knowledge of the database or how these objects are to be handled…

All database operations in YapDatabase are handled through database connections. YapDatabaseConnection has functions for reading from, writing to, and responding to individual changes in the database. It does all of this in a thread safe manner. One less thing to worry about!

OK, flipping down the feature list there, looks like a solid solution to entry level needs, got a track record and is being easily maintained, just that one little extra base requirement…

• Sync. Support for syncing with Apple’s CloudKit is available out of the box. There’s even a fully functioning example project that demonstrates writing a syncing Todo app.

… well, there we go then. Unless some dramatically superior alternative comes to our attention, next data-storing app we ship will use YapDatabase too, farewell Core Data!

This iOS Goes To 11

Been enough of a while since WWDC ’17 for people to sort out what they find interesting about iOS 11 and all now, so let’s take a look shall we?

The canonical references up at the mothership are

A good TL;DR while you bookmark this for later is What’s new in iOS 11 for developers

iOS 11 By Examples has, surprise, examples of using new iOS 11 APIs:

  • Core ML: Image classification demo using Core ML framework
  • Vision: Face detection, landmarks, and object tracking
  • ARKit: Augmented reality experiences in your app or game
  • Core NFC: Reading of NFC tag payloads.
  • IdentityLookup: SMS and MMS filtering using IdentityLookup framework
  • DeviceCheck: Identifying devices that have used a promo, flagging fraudsters
  • Blogs/Newsletter: Other places that mentioned this list — probably mention more good stuff too!

Personally we’d already been planning to try out iPad-only travel soon, and iOS 11 + new iPad Pro looks like MASSIVE WIN on both fronts — seriously, did anybody at all predict the super duper ProMotion screen? — and the mutitasking stuff is a good bit better than we’d expected, and easy to use, see:

CoreNFC is one of those what took you so long things, but hey better late than never: CoreNFC tutorial

Now that Metal is even more metal to support 120 fps displays and all, they’re Introducing Metal 2

If you do anything with file names in iOS or macOS, make sure you read APFS Native Normalization

One thing worthy of noting as removing what we’d found the major annoyance with UIStackView and likely you too: Stack View Custom Spacing

Also top and layout guides are simplified to Safe Area Layout Guide

Some more little bites, to coin a phrase, at #309: UIFontMetrics and #310: Screen Edges in iOS 11 and #311: Round Corner Improvements

Now, to step back from the API level, some people are very excited about this Business Chat thing:

… I believe that Apple Pay Cash, and the ability to send low transaction amounts with no cost to the sender or the receiver (the business) will be one of the most transformative elements to the impact of Business Chat. This can bring rise to a number of new businesses and applications that may be similar to the “gig economy” of Fivver and other systems…

Business Chat can not only spell the end of the POS system and payment systems as we know it in retail sales, it may also spell the end of the shopping carts and payment systems in online sales. Some may argue this is bombastic…

Yes, maybe a touch. Doesn’t mean it’s wrong though; read the whole thing, and the Business Chat info at the mothership, and see what you think —or read this one that figures it’s not just shopping carts in the crosshairs: Apple Bank here we come

Last one we’ll call out here — some good thoughts on the new evolutions of the interface design language: Think Bigger: Design Changes in iOS 11

Need more? Check out your veritably canonical reference to everything new over at New stuff from WWDC 2017!

UPDATES:

Changes to location tracking in iOS 11; Location Permissions in iOS 11 and avoiding the Blue Bar of Shame

iOS 11, Privacy and Single Sign On

What’s New In UIKit Animations In Swift 4

Working with CoreNFC in iOS 11

Apple Platform SDK API Differences for iOS 11+

Working with Drag and Drop APIs in iOS 11

Working with Core Bluetooth in iOS 11 > Introduction to HealthKit with Core Bluetooth

Touch ID and Face ID on iOS

ARKit And Kaboodle

The other obviously transformational technology introduced at WWDC this year was Apple’s sudden leap from

One thing is clear: Apple needs to get moving soon.

to, a mere 5 days later — how’s that for “soon?” —

If Apple gets this right, they will own the hardware market for years to come.

and what we’re sure must be by far the quickest ever adoption of an Apple-only technology outside the computer industry,

Ikea’s plans for ARKit revealed, virtual shopping tool will launch in fall with iOS 11

so let’s start collecting links on that shall we?

Introducing ARKit

With ARKit, iPhone and iPad can analyze the scene presented by the camera view and find horizontal planes in the room. ARKit can detect horizontal planes like tables and floors, and can track and place objects on smaller feature points as well. ARKit also makes use of the camera sensor to estimate the total amount of light available in a scene and applies the correct amount of lighting to virtual objects…

… yeah ok that’s pretty cool.

There’s already signup open for ARKit Weekly to keep abreast of news here, and the same obviously extra-keen folk have also started

Made With ARKit: Hand-picked curation of the coolest stuff made with #ARKit

That’s some pretty darn nifty videos there for, like two weeks at this!

To grab some source code examples and interesting commentary, check out

Apple ARKit by Example

  1. Getting setup, draw a cube in virtual reality
  2. Plane Detection and Visualization
  3. Adding geometry and physics fun
  4. Physically Based Rendering

ARShooter – An Example Shooter Created Using iOS 11’s ARKit

ViewAR does first tests with ARKit

3 Things To Know About Apple’s ARKit

Apple’s new augmented reality platform may be its next game-changer

Pretty cool huh? It’s like the future’s so bright we have to wear shades … except the whole point of Apple AR (so far…) is that we don’t!

UPDATES:

ARBrush: “Quick demo of 3d drawing in ARKit using metal + SceneKit.”

ARKit Tutorial in Swift 4 for Xcode 9 using SceneKit

Watch a Tesla Model 3 come to life

ARTetris: “Augmented Reality Tetris made with ARKit and SceneKit”

Getting Started with ARKit: Waypoints

ARKit introduction

Wenderlich Introduction to ARKit:

  1. Getting Started
  2. Adding 3D Models
  3. Measuring Distances

I’m presenting the ARKit workshop at RWDevCon 2018!

ARKit-CoreLocation: “Combines the high accuracy of AR with the scale of GPS data.”

Inside iOS 11: The coolest Apple ARKit demos created so far

Using ARKit with Metal

ARKit and Autism: New Futures

Adventures in ARKit (with magic)

ARPaint: “Draw with bare fingers in the air using ARKit.”

ARKit for iOS Developers

Why is ARKit better than the alternatives?; Why Apple’s glasses won’t include ARKit ; Q&A on ARKit

FaceRecognition-in-ARKit: “Detects faces using the Vision-API and runs the extracted face through a CoreML-model to identiy the specific persons”

Latest demos with iOS 11 ARKit show plated food, 3D sculpting with Apple Pencil

New Augmented Reality Resources Now AvailableARKit Human Interface Guidelines

Apple invites developers, media to Cupertino HQ to showcase ARKit apps

arkit-occlusion: “A demonstration of vertical planes “tracking” and occlusions with ARKit+Scenekit”

How is ARCore better than ARKit?

Using ARKit with Metal and Using ARKit with Metal part 2

Why ARKit will be Apple’s biggest innovation in years

Animating a 3D model in AR with ARKit and Mixamo

ARKit and Core Location: Part One

IKEA App

How Is ARCore Better Than ARKit? + The Potential of Apple’s ARKit

How Apple’s iPhone X TrueDepth AR waltzed ahead of Google’s Tango

ARKit Tutorial: Understanding and Implementing 3D Objects

ARKit 1.5 examples show how vertical surface mapping upgrades augmented reality in iOS 11.3

ARKit Tutorial: Light Estimation with Ambient Intensity and Color Temperature

A Quick Guide to Designing for Augmented Reality on Mobile (Part 1) and (Part 2)

ARVideoKit: “enables developers to capture videos 📹, photos 🌄, Live Photos 🎇, and GIFs 🎆 with ARKit content.”

The AR Location Toolkit: “uses Computer Vision technology to identify the surrounding landscape, and then precisely overlays interactive information about different points of interest.”

Introducing Urban Visual Positioning — a breakthrough in location-based AR;  Blippar Launches Landmark Recognition using Deep Learning and Augmented Reality

ARKit Tutorial: Working with 2D Image Recognition

Awesome-ARKit: “A curated list of awesome ARKit projects and resources. Feel free to contribute!”

iOS Week: ARKit and ARReferenceImage

ARKit 2: Bringing richer experiences through collaboration, enhanced detection, and greater realism

AR Face Tracking Tutorial for iOS: Getting Started

An Introduction to AR Quick Look in iOS 12

Saving and Restoring World-mapping Data to Create a Persistence AR Experience

What I Learned Making 5 ARKit Prototypes

Asteroid: “A graphical tool for designers and JS developers to make ARKit apps”

ARKit-SCNPath: “Create paths for your Augmented Reality environments using just points to represent the centre of the path.”

Building voice responsive AR apps

Realistic Rendering of 3D Photogrammetry Model in ARKit

Designing 3D models for AR apps (ARKit)

6 Interesting Concepts for AR Experiences

Using ARKit and Image Tracking to Augment a Postcard

Getting Started with ARKit & Adding the finishing touches to 3D models in Xcode

Building a Museum App with ARKit 2

How To Build A Super Quick Smile Tracking App

Core MLagueña

Checked out the WW17 Roundup yet? OK then, let’s start digging into this new stuff a little deeper. And we’ll start with the one with the most buzz around the web,

Introducing Core ML

Machine learning opens up opportunities for creating new and engaging experiences.

Core ML

is a new framework which you can use to easily

integrate machine learning models into your app

. See how Xcode and Core ML can help you make your app more intelligent with just a few lines of code.

Vision Framework: Building on Core ML

Vision

is a new, powerful, and easy-to-use framework that provides solutions to computer vision challenges through a consistent interface. Understand how to use the Vision API to detect faces, compute facial landmarks, track objects, and more. Learn how to take things even further by providing custom machine learning models for Vision tasks using CoreML.

By “more intelligent” what do we mean exactly here? Why, check out

iOS 11: Machine Learning for everyone

The API is pretty simple. The only things you can do are:

  1. loading a trained model
  2. making predictions
  3. profit!!!

This may sound limited but in practice loading a model and making predictions is usually all you’d want to do in your app anyway…

Yep, probably. Some people are very excited about that approach:

Apple Introduces Core ML

When was the last time you opened up a PDF file and edited the design of the document directly?

You don’t.

PDF is not about making a document. PDF is about being able to easily view a document.

With Core ML, Apple has managed to achieve an equivalent of PDF for machine learning. With their .mlmodel format, the company is not venturing into the business of training models (at least not yet). Instead, they have rolled out a meticulously crafted red carpet for models that are already trained. It’s a carpet that deploys across their entire lineup of hardware.

As a business strategy, it’s shrewd. As a technical achievement, it’s stunning. It moves complex machine learning technology within reach of the average developer… Well, speaking as that Average Developer here, yep this sure sounds like a great way to dip a toe into $CURRENT_BUZZWORD without, y’know, having to actually work at it. Great stuff!

Here’s some more reactions worth reading:

Here’s some models to try it out with, or you can convert your own built with XGBoost, Caffe, LibSVM, scikit-learn, and Keras :

  • Places205-GoogLeNet CoreML (Detects the scene of an image from 205 categories such as an airport terminal, bedroom, forest, coast, and more.)
  • ResNet50 CoreML (Detects the dominant objects present in an image from a set of 1000 categories such as trees, animals, food, vehicles, people, and more.)
  • Inception v3 CoreML (Detects the dominant objects present in an image from a set of 1000 categories such as trees, animals, food, vehicles, people, and more.)
  • VGG16 CoreML (Detects the dominant objects present in an image from a set of 1000 categories such as trees, animals, food, vehicles, people, and more.)

And some samples and tutorials:

Also note NSLinguisticTagger that’s part of the new ML family here too.

For further updates we miss, check out awesome-core-ml and Machine Learning for iOS !

UPDATES:

YOLO: Core ML versus MPSNNGraph

Why Core ML will not work for your app (most likely)

Blog: Getting Started with Vision MLCamera – Vision & Core ML with AVCaptureSession Inceptionv3 model

Can Core ML in iOS Really Do Hot Dog Detection Without Server-side Processing?

Bringing Machine Learning to your iOS Apps 🤖📲

Creating A Simple Game With CoreML In Swift 4

Custom classifiers in iOS11 using CoreML and Vision

Using Vision Framework for Text Detection in iOS 11

Smart Gesture Recognition in iOS 11 with Core ML and TensorFlow

Awesome-CoreML-Models: “Largest list of models for Core ML (for iOS 11+)”

fantastic-machine-learning: “A curated list of machine learning resources, preferably CoreML”

Announcing Core ML support in TensorFlow Lite

Creating a Custom Core ML Model Using Python and Turi Create

DIY Prisma, Fast Style Transfer app — with CoreML and TensorFlow

Core ML Simplified with Lumina

Style Art: “is a library that process images using COREML with a set of pre trained machine learning models and convert them to Art style.”

Beginning Machine Learning with Keras & Core ML

How I Shipped a Neural Network on iOS with CoreML, PyTorch, and React Native

TensorFlow on Mobile: Tutorial

FastPhotoStyle: “Style transfer, deep learning, feature transform”

Netron: “is a viewer for neural network and machine learning models.”

Leveraging Machine Learning in iOS For Improved Accessibility

CoreMLHelpers: “Types and functions that make it a little easier to work with Core ML in Swift.”

Benchmarking Core ML — Estimating model runtimes on iOS

Machine Learning in iOS: IBM Watson and CoreML

Detecting Whisky brands with Core ML and IBM Watson services

Swift for TensorFlow, Released & Why data scientists should start learning Swift

Build a Taylor Swift detector with the TensorFlow Object Detection API, ML Engine, and Swift

Lumina: “A camera designed in Swift for easily integrating CoreML models – as well as image streaming, QR/Barcode detection, …”

Lobe: Visual tool for deep learning models

Building a Neural Style Transfer app on iOS with PyTorch and CoreML

What’s New in Core ML 2

Create ML: How to Train Your Own Machine Learning Model in Xcode 10

Create ML Tutorial: Getting Started

Training a Text Classifier with Create ML and the Natural Language Framework

Natural Language in iOS 12: Customizing tag schemes and named entity recognition

NSHipster’s NLLanguage​Recognizer

Creating a Prisma-like App with Core ML, Style Transfer and Turi Create

 NSFWDetector: “A NSFW (aka porn) detector with CoreML”

Running Keras models on iOS with CoreML

Skafos “is machine learning for iOS developers”

WWDC 2017 Roundup

That was a particularly good WWDC this year wasn’t it? Lots of possibly transformational technologies introduced it’s going to be very interesting indeed to see how they play out!

We’ll explore those in more depth later on, but for now let’s sort out some resource links and initial reactions.

First up, you want to go download The Unofficial WWDC app for macOS:

4a5b0d11384fd46acdbe8f552a5b85a7

Isn’t that pretty? Definitely your WWDC video viewing app of choice, we say!

Next, address this problem

687474703a2f2f63646e2e6d656d6567656e657261746f722e6e65742f696e7374616e6365732f343030782f32313932383230372e6a7067.jpeg

with the WWDC-Downloader script over at Github.

(And while that’s all downloading for your next transcontinental plane flight or whatever, have your heart warmed with

WWDC17 Scholarship winner Kenny Batista shared his experiences at Apple’s big week for developers !)

If you want to prioritize a bit, here’s a WWDC 2017 Viewing Guide that looks solid.

There’s enough interesting new technologies that they deserve roundup posts each as there’s going to be a lot of figuring out there to keep track of, so for now we’ll just link a couple overviews hitting the highest points:

The 2017 Apple Design Award Winners

WWDC 2017 Initial Impressions from the Wenderlich team

Another take from the Big Nerd Ranch crew at WWDC 2017: Helping You Get Things Done

Also check out their article New HEVC & HEIF Media Formats: What You Need to Know and Begun, The Codec War Has for background on the fur that’s no doubt going to be flying all over about that soon.

Oh, and we’ll put An In Depth Look At the New App Store and New rules following WWDC 2017 here too — there’s some interesting changes there you want to be aware of. Most notably to us, that finally The new iOS App Store lets devs choose whether or not to reset ratings when updating!! Woo-woo-woohoo!

And as always, over at mjtsai.com you can find a near-canonical list of WWDC 2017 Links for deeper diving!

UPDATES:

HEIF: A First Nail in JPEG’s Coffin?

Swift Witness For Sourcery

Yep, think we’ve outdone ourselves with the obscure title references this time. If you knew it was from Malachi 3:5, our congratulations!

What we’re talking about today is what you might have noticed when it was named Insanity, but these days is called Sourcery:

What is Sourcery?

Sourcery scans your source code, applies your personal templates and generates Swift code for you, allowing you to use meta-programming techniques to save time and decrease potential mistakes.

Using it offers many benefits:

  • Write less repetitive code and make it easy to adhere to DRY principle.
  • It allows you to create better code, one that would be hard to maintain without it, e.g. performing automatic property level difference in tests
  • Limits the risk of introducing human error when refactoring.
  • Sourcery doesn’t use runtime tricks, in fact, it allows you to leverage compiler, even more, creating more safety.
  • Immediate feedback: Sourcery features built-in daemon support, enabling you to write your templates in real-time side-by-side with generated code.

Sourcery is so meta that it is used to code-generate its boilerplate code

We’re pretty leery of tools like this in general as they end up being yet another cesspool of technical debt, but that this one is based on SourceKitten is interesting enough to at least pay some attention to it. And it’s catching a bit of buzz:

How to Automate Swift Boilerplate Code with Sourcery

Switching between our Android projects written in Java (or more recently, Kotlin) and iOS projects in Swift makes me sorely miss annotation processing. Annotations in Java can be used to generate common code and things that Swift developers would have to hand write. Hand written code has to be bug free and consistently updated and maintained. Things like implementing equals and parsing JSON typically require boilerplate code, unique enough that you cannot abstract it away.

Although Swift does not have annotation processing, we can get pretty close with Sourcery. Let’s take a look at how it all works in this quick Sourcery tutorial…

AutoEquatable and AutoHashable by Sourcery

I’m really interested in Sourcery, so I installed it in my one of repositories. In my case, I generated AutoEquatable and AutoHashable at first because I felt these implementations contain really a lot of boilerplate codes…

If I add a new property or a new API, it prevents a human error! Though it looks trivial, Sourcery uses a safe Hashable as I mentioned in Safe Hashable in Swift. Isn’t it cool?

The Magic of Sourcery

Sourcery is a code generation tool for Swift. It is particularly well suited for auto-generating code that is very repetitious; code most developers refer to as “boilerplate”. Some examples that come to mind:

  • Equatable implementations
  • Hashable implementations
  • struct initializers
  • Lens implementations …

Sourcery — Meta-programing in Swift

Think of the potential : you can basically generate all the boilerplates of your architecture be it CLEAN or VIPER, you can generate all the protocol implementation or even the Realm data object, or even a view based on the variables that are being declared ( Sourcery do support inline code generate ). A declarative view ! The sky is the limit !

Meta Programming Swift with Sourcery

#292: Metaprogramming with Sourcery 🔮

#294: Annotations with Sourcery 🔮

#295: Building an API client with Sourcery Key/Value Annotations 🔮

If you’re looking for a good project to try this out with, a decent set of VIPER templates would be eagerly welcomed. Let us know if you come up with something!

UPDATES:

Sourcery Tutorial: Generating Swift code for iOS

Sourcery In Practice

How to supercharge Swift enum-based states with Sourcery

Generating Swift Types with Sourcery

Immutable Model Mutability

Introduction to Sourcery

Codable in practice — “let Sourcery to generate boilerplate Codablecode instead of compiler.”