Under The BridgeUnder The Bridge

Musings
Paleo App Dieting

No doubt you’re aware that bigger is not better for downloads on the App Store, especially if you hit the dreaded 100 MB cellular download limit, but were you aware that even if you haven’t there’s a marked disadvantage to download size increasing? Well, here it’s quantified by way of Actual Real Life Experiment:

We bought a successful app, loaded it with extras and watched it fail

… we estimate a linear change in install conversion rate below the 100MB cutoff of -0.45 percent install rate per MB. Above the 100MB cutoff, we estimate a linear change in install rate of -0.32 percent per MB. To our best estimate, the gap between the two lines is covered by a 10 percent instantaneous install rate drop across the cellular download limit.

Although Apple says the cellular download limit is 100MB, we found in practice that a 101MB IPA did not trigger the cellular download block. The actual limit was somewhere between 101MB and 123MB, and it varied depending on the exact build.

Increasing the size of our app from 3MB to 99MB reduced installs by 43 percent, and the increase to 150MB reduced installs by 66 percent in total…

They’ve put together a handy App Readiness Guide with an especially handy App Size Calculator to let you check out your favourite apps.

Sooo, how do we get our size down? If you’re shipping to iOS 9 and later, there’s great options:

App Thinning (iOS, tvOS, watchOS)

  • Slicing is the process of creating and delivering variants of the app bundle for different target devices
  • Bitcode is an intermediate representation of a compiled program. Apps you upload to iTunes Connect that contain bitcode will be compiled and linked on the store…
  • On-demand resources are resources—such as images and sounds—that you can tag with keywords and request in groups, by tag. The store hosts the resources on Apple servers and manages the downloads for you…

Those are generally pretty straightforward to adopt, although you may have occasion to refer to

But if you’re still shipping for iOS 8, none of those help your iOS 8 users at all. Ah well.

First thing is to make sure you’re stripping debugging symbols and dead code with Xcode build settings. Also, make sure that Xcode isn’t bloating your asset catalog behind your back. The setting for that is “Optimization” aka ASSETCATALOG_COMPILER_OPTIMIZATION — and if you haven’t set anything, apparently it defaults to “time” which undoes all your compression and bloats up wildly besides. So set it to “space” instead.

If you need to dig deeper than that, here’s some useful tools for messing with your binary and asset catalog to figure out just what’s going on:

Bloaty McBloatface: a size profiler for binaries

Crunch: “Extract resources from iOS apps. Make iOS icons.”

cartool: “Export images from OS X / iOS .car CoreUI archives.”

AssetCatalogTinkerer: “An app that lets you open .car files and browse/extract their images.”

iOS-Asset-Extractor: “A tool to extract image assets from the iOS SDK.”

And if you need to go still deeper into smallerizing those graphics, well then you’ll have to go to some work. Possibly useful approaches include:

PaintCode is your goto tool for converting vectors into code — and it now supports Swift 3, Android, and JavaScript!

Or you could use SVG files as resourcesMacaw is a new library that looks particularly good for that.

TexturePacker is your goto tool for creating sprite sheets, if that suits your resource usage profile.

ImageAlpha + ImageOptim is an effective strategy for reducing PNG sizes.

JPEGmini is particularly good at making JPEGs smaller.

Or, you could go with the generally well regarded WebP image format, supported by iOS-WebP for instance.

Any more tips? Share and enjoy!

Consistently Immutable

Here’s a good read for when you’re planning a data caching strategy for your Next Great Project:

Managing Consistency of Immutable Models

Immutable models have many advantages, but managing consistency can be difficult. At LinkedIn, we’ve open sourced two Swift libraries to help manage consistency and persistence of immutable models. Unlike Core Data, the API is non-blocking, exception-free, and scales well with large numbers of models and view controllers…

The natural reflex of course is Core Data, and these were their issues:

  • The first problem we ran into was stability…
  • The other issue we ran into was performance…
  • Another issue is there are no eviction strategies…
  • Another point was that migrations are necessary…
  • The final thing for us was scalability…
  • Even if we were to solve all of these problems, we didn’t like the programming model that Core Data provided. One of the things we didn’t like is that all the models were mutable, and they’re not thread-safe, so when you pass it to a different thread, you need to reread from the database…

We could nitpick over how serious these problems actually are if you know what you’re doing, but yeah that last one is kinda a deal. So what would a modern architecture look like, you ask?

What we really wanted to try was to use immutable models for all the data in our application. Immutable models have long been a cornerstone of functional programming languages, and with the introduction of Swift to iOS, I think that they’ve been pushing this idea of immutability. Swift has a bunch of “immutable by default” features, and personally, I’ve been learning a lot about immutability because of Swift. I’m going to list a few of the examples of why immutable models are beneficial:

We’ll assume you know those benefits already, but if not TL;DR they’re easier.

  • We wanted immutable, thread safe models for the reasons which I just went over.
  • We wanted consistency, so a change in one place would automatically be reflected elsewhere in the application.
  • We wanted non-blocking access on all reads and writes. There wouldn’t be any disk access on the main thread.
  • We wanted a simple eviction strategy. Looking at some of our peers, we saw that a lot of apps were using 200, 300, 400 megabytes of disk space, and ideally, for us, we wanted to limit it below 50 megabytes.
  • We want it to scale well. Again, as I said, we have hundreds of models and view controllers, and we wanted easy migrations. Actually, we wanted to write no migration code at all, ever.

To accomplish all these things we wrote RocketData.

RocketData is a caching and consistency solution for immutable models. It’s intended to replace Core Data or at least fulfill that role in an application. It’s written 100% in Swift, and we really like it…

Definitely read it all, but here’s the secret sauce:

The Consistency Manager, as I said, is truly the engine which drives everything. It’s an entirely separate open-source library that RocketData depends on. It’s written completely in Swift, and you can use it independently.

and here’s the executive summary:

Core Data Comparison

There is no tool which will solve every problem. For many use cases, we believe Rocket Data is a better option to Core Data because of the speed and stability guarantees as well as working with immutable instead of mutable models. However, Core Data is useful for other use cases, and it’s important to consider the differences before you make a decision for your app…

Always good to have options … and this looks like a pretty sweet one!

UPDATES:

Speaking of options: Core Data or Realm

Make Collections Great Again

Pretty good chance you use UICollectionView pretty widely in your apps, right? And pretty much exactly the same chance you’d like some extra performance and enhanced reusability, no doubt? Ch-ch-check out

Open Sourcing IGListKit

Today, we are excited to announce that we’re open sourcing one of Instagram’s core frameworks: IGListKit. This framework powers how we take data from the server and turn it into fast and flexible lists.

To do this, we combined a familiar data-driven UICollectionView architecture with a state-of-the-art diffing algorithm. With this setup, we created a tool that lets engineers with varying levels of experience work quickly and safely on Instagram. Outside of Instagram, IGListKit can help anyone who is building lists to skip tedious and error-prone setup, and utilize one of the fastest diffing algorithms available for iOS…

Better Architectures

IGListKit reduces the chance of having “massive view controllers” by dividing responsibilities into multiple layers: the view controller, list adapter, section controller, and the cell.

This design has a positive side effect of building your lists with independent components, meaning you end up with reusable section controllers and cells. The result is a one-way data flow, where each component has no knowledge of its parent…

Diffing

In Instagram, our data can change a lot. For example, new data arrives from the server each time you like a photo or you receive a direct message in real-time.

UICollectionView can handle all types of updates (deletes, reloads, moves, and inserts), but performing those updates without crashing can be a little tricky.

We built a lightning-fast diffing algorithm based on a paper published back in 1978 by Paul Heckel. This algorithm finds all of the possible updates that UICollectionView needs in linear time (that’s O(n))…

That’s a pretty serious level of win … especially if you’ve got some experience with that “little tricky” task of managing updates elegantly, in the same way that liquid nitrogen is a “little chilly”. So if you’ve got a big collection of data to manage, we wholeheartedly recommend you check that out!

ADDENDA:

And while you’re looking at libraries to make your UI nicer, check out this most excellent list:

39 Open Source Swift UI Libraries For iOS App Development

and of course there’s always

Awesome iOS: A curated list of awesome iOS libraries, including Objective-C and Swift Projects

Rollout The Red Carpet

So you might have heard of Rollout.io before:

Update Live Mobile Apps

Instantly Fix Bugs|

Deploy code-level changes to native iOS apps, without waiting on the App Store.

Which works by remotely swizzling your methods up to and including completely replacing with JavaScriptCore-executed code, which is 100% legit as a delivery method, no really it is, here check the latest rules:

3.3.2 … an Application may not download or install executable code. Interpreted code may only be used in an Application if all scripts, code and interpreters are packaged in the Application and not downloaded. The only exceptions to the foregoing are scripts and code downloaded and run by Apple’s built-in WebKit framework or JavascriptCore, provided that such scripts and code do not change the primary purpose of the Application by providing features or functionality that are inconsistent with the intended and advertised purpose of the Application as submitted to the App Store.

All well and good then … except that swizzling only works in Objective-C and it’s $CURRENT_YEAR, who still writes Objective-C? Cha, we’d spend our ever so valuable blogging space on that right after an in-depth article on the state of 3270 emulation, amirite? (With all due respect to the author of that no doubt invaluable if you need it fine bit of work, mind you, just that most of the world has moved on a bit!) Well, look at what the boffins over Rollout way have come up with now:

Hi Everyone,

I’m delighted to announce that Rollout now supports Swift apps!

After months of development, testing and tweaking we’ve just released our official Swift support.

If you’ve been waiting for Swift support in order to try out Rollout, either create a new app and install the SDK or upgrade your existing app with the latest SDK version…

Uh … yeah? And just how exactly did you get that to work?

The heart of Rollout’s Swift patching magic lies in a technique we call Pre-SIL instrumentation.

SIL stands for Swift Intermediate Language, it is generated from the Abstract Syntax Tree. This intermediate form is used by the Swift optimizer to perform Swift language specific optimizations prior to generating the LLVM IR.

Rollout instrumentation needs to happen before the optimization phase so it will work even if the method was inlined, the dispatching was de-virtualized and more (I’ll write more about optimizations in Swift in the near future).

To add this code instrumentation Rollout runs the following flow:

  1. Replace the swiftc compiler with a proxy script
  2. When swiftc compiles a file, intercept the file and instrument it
  3. Identify all methods in file
  4. Instrument all methods with the patching mechanism

All of the above needs to happen without a cost in compile time, runtime and most importantly maintaining a consistent debugging experience (as if Rollout didn’t touch your code)…

Well, brush our teeth and call us pearly. If that does indeed work as advertised, that is quite the leap forward for application maintenance, isn’t it? Looks like the free tier doesn’t support Swift so it’s not a completely trivial cost to adopt, but hey, it’s pretty cheap compared to the damage a broken app could do you!

BoardingBot Beta Beneficence

Been a while since we had any app landing page tools worth noting for you Dear Readers, but here’s a new one BoardingBot that looks quite interesting indeed, it lets you sign up beta testers directly … even via Facebook Messenger no less. That a great way to get the buzz buzzing buzzingly, or what?

BoardingBot.png

BoardingBot will create a site for your app, with screenshots taken from iTunes and an option for beta testers to request a TestFlight invite..

BoardingBot can answer chat messages from your Facebook fans and send them Testflight invites…

BoardingBot upcoming directory of beta apps enables you to find more beta testers. This feature is still under development – Become one of the first apps to be featured in our beta directory.

Pretty cool sounding, huh? If you’ve got an app ready for an open beta, give them a try and let us know it goes!

Wages of App Is Life

So it’s been a very long time indeed since we had occasion to note any new developments on the App Store sales tracking front — but what ho?

Well isn’t that just an awesome burger with awesome sauce? For years we’ve been not quite annoyed enough with the vagaries of getting reports split out by revenue share with appropriate currency conversions to do it ourselves, and now here’s a nice looking tool all Open Sourced for those final tweaks!

  • SALES REPORTS: Quickly view your sales reports for multiple products and days. Access information such as the number of downloads, in-app purchases, promo codes and refunds.
  • REVIEWS: The key to a successful product is keeping your customers happy. Track reviews from each and every country, and even have them translated into your local.
  • RANKINGS: Making changes to your keywords? Doing some marketing? AppWage tracks your rankings allowing you to see where and when your apps change position in charts.
  • TRACK COMPETITION: Need to see the results for multiple tables at once? You can execute multiple queries and have the results for each displayed at the same time, including any errors and messages.

*does a little happy dance*

Speaking of nice looking tools, there’s a new multi platform review tracker out, Review Command:

Review Command gathers your ratings & reviews for all of your apps, from all countries, and all major app stores, and displays them in 1 simple feed.

Supports iOS App Store, Mac App Store, Google Play, Amazon App Store and Windows Store.

We bought that just now because it looked ever so pretty and hey it’s launch pricing of $19 right now so why not, and yes so far looks like it works well and is very pretty indeed. Responsive developer too — had a setup question because my iTunes account is weird and got a reply within a couple hours, on a Sunday afternoon no less. So if you have multi platform review management needs, or even if you don’t, we recommend checking that out too!

Notification Tokenism

So, did you catch WWDC 724 yet? It’s less than 15 minutes, check it out:

What’s New in the Apple Push Notification Service (transcript)

Starting with a review of the HTTP/2 based provider API, you will learn about an important new feature: Token Based Authentication. Learn to connect to APNs using authentication tokens for sending pushes via the HTTP/2 API, relieving you of the overhead associated with maintaining valid certificates.

Ah, yes. Longtime readers will remember various instances of untrammelled joy (where by “joy” we mean “near-homicidal frenzy”) APNs has provided over the years, so this is a very interesting indeed development. Full details were published September 20 at

Local and Remote Notification Programming Guide / Apple Push Notification Service

and announced September 22. So let’s check this out, shall we? Especially as we got the annually dreaded “This certificate will no longer be valid in 30 days.” email day before yesterday, which makes the timing entirely apropos!

Step 1: Oh look, now under certs in the Dev Portal we have this new “APNs Auth Key” option! So add one, and…

Step 2: Read the instructions:

Download, Install and Backup

Download your Authentication Key to your Mac, then double click the .key file to install in Keychain Access. Make sure to save a back up of your key in a secure place. It will not be presented again and cannot be retrieved at a later time.

Step 3: Be confused as the download gives you a .p8 file, not a .key file, and it’s not evident how you get that into the Keychain.

Step 4: Check out the discussion on the dev boards:

It’s a PEM-encoded, unencrypted PKCS#8 file. You can inspect it with `openssl pkcs8 -nocrypt -in <your_file.p8>`, but you’ll need a newer version of OpenSSL than the one that ships with El Capitan (I’m not sure which version comes with Sierra). I’m not sure how to import it into the Keychain, but I also haven’t run into a situation where having it in the Keychain would be helpful (other than just for storage).

Step 5: OK then, let’s find some service that can use this shiny new .p8 file then…

Step 6: ¯\_(ツ)_/¯ Not much out there yet!

Alright, so we’ll keep working with a cert-based process for the moment then, check for UPDATES! below once we find some. In the meantime, development tools that can work with the shiny newness? Hmmmm … looks like you’re pretty much on your own there too as we write this. Right then, if you know of any token-supporting services or tools, hit the comments section!

If you’re proactive enough to want to help move the development tools scene along, our goto for APNs tinkering for a while now has been Knuff: “The debug application for Apple Push Notification Service (APNs).” along with Knuff-Framework “just add it to your application and the phone will be visible under “Devices”” and Knuff – The APNs Debug Tool on the App Store! Pretty complete ecosystem there and we strongly encourage helping out with it if you’re interested in this space.

If you’re new to this APNs thing, or If you’d simply like a straightforward HTTP page to use for testing, check out this new site PushTry.com to, eponymously enough, try pushing:

pushtry.jpeg

Nicely done tutorials for both iOS and Android, clean and functional; definitely recommend you give that a pushtry. And they’re working on token support too!

Should you be looking to get this on your server side ASAP, a quick look around for projects at least using HTTP/2 APNs, that being kinda the floor for ‘actively maintained’ currently, in a variety of environments turns up these:

VaporAPNS “is a simple, yet elegant, Swift library that allows you to send Apple Push Notifications using HTTP/2 protocol in Linux & macOS. It has support for the brand-new Token Based Authentication but if you need it, the traditional certificate authentication method is ready for you to use as well. Choose whatever you like!”

swift-apns: “Swift Framework for sending Apple Push Notification over HTTP/2 API”

node-apn: “A Node.js module for interfacing with the Apple Push Notification service.”

ApnsPHP: “Apple Push Notification & Feedback Provider”

APNS/2 “is a go package designed for simple, flexible and fast Apple Push Notifications on iOS, OSX and Safari using the new HTTP/2 Push provider API.”

And again, if you have good … or bad … experiences with any particular piece of APNs server kit, let us know!

iMessage In A Bottle

So you jumped on board the iMessage train yet? It’s important enough the mothership gives it a top level website folder no less,

iMessage + Apps

Check that out for links to WWDC videos, sample codes, yada yada yada for apps and stickers. Never really got the whole sticker thing until moving here to Thailand where Line is the preferred means of communication to the tune of 90% of mobile Internet users; but yeah, they send a couple billion a day. If you’ve got anything resembling artistic ability, might want to look into that, hey it’s a seriously good paying market for clip art!

More interesting to most of you Dear Readers I imagine would be the opportunities to write apps, and here is an excellent article to get started on that:

Introducing the Airbnb iMessage App

We started out looking to define the grand vision of planning your Airbnb trip within Messages — what would that look like on this new platform? Over the course of just a few weeks before the iOS 10 release, our team rapidly prototyped on a variety of ideas. We started with countless drawings and mockups of possible features, and as we became more familiar with the possibilities and limitations of the Messages Framework, a product began to form. Once we had something we could use, a group iMessage thread between engineers, designers, and product managers allowed us to quickly discover the features we wanted — a rather meta approach. At this point, we had an iMessage app where people could share homes between friends and view key details about the accommodation…

Speaking of collaboration, an iMessage app wouldn’t be complete without an additional social feature: voting. When sharing a message with friends, each collaborator can vote on homes they like and see who else has voted on them. This gives the trip organizer a bird’s-eye indication of the most popular listing, greatly simplifying the decision of which one to book…

Goes on in detail about features, limitations, and process; even if you’re not currently planning an iMessage app, in which case you probably should be, this’ll get you up to speed with the state of the framework nicely!

Machines Of Learning Grace

Our nominee for Most UnderDiscussed Shiny New Stuff with this year’s Apple crop definitely has to be BNNS. What, not ringing a bell? Ch-ch-check it out:

The Accelerate framework’s new basic neural network subroutines (BNNS) is a collection of functions that you can use to construct neural networks. It is supported in macOS, iOS, tvOS, and watchOS, and is optimized for all CPUs supported on those platforms.

Of course, there’s a perfectly good reason it hasn’t made more of a splash, which is

BNNS supports implementation and operation of neural networks for inference, using input data previously derived from training. BNNS does not do training, however. Its purpose is to provide very high performance inference on already trained neural networks.

That’s a bit too high of a bar to jump, right? Well, not quite as much as you’d think! See, if you even noticed Google open sourcing TensorFlow at all, you probably figured it was a bit academic for any real world application you were working on, right? Well, here’s the inestimable Aaron Hillegass to explain the synergy here:

Use TensorFlow and BNNS to Add Machine Learning to your Mac or iOS App

Thus, there are two ways to get deep learning into your Mac or iOS application:

Solution 1: Do all the neural net work on a server using TensorFlow. You must be certain that all your users always have a good internet connection and that the data you are sending/receiving is not too voluminous.

Solution 2: Train the neural net using TensorFlow and export the weights. Then, when you write the iOS or Mac application, recreate the the neural net using BNNS and import the weights.

Google would love to talk to you about the first solution, so the rest of this posting will be about the second. I’ve used TensorFlow’s MNIST example: handwritten digit recognition with just an input and an output layer, fully connected. The input layer has 784 nodes (one for each pixel) and the output has 10 nodes (one for each digit). Each output node gets a bias added before it is run through the softmax algorithm. Here is the source, which you will get automatically when you install TensorFlow.

My sample code is posted on GitHub

So right now you can ship trainees that won’t learn any more — and feel free to fill in a simile of your choice here for a laugh — if you have a static problem space in your app that could be enhanced with a bit of clever; or you could just get up to speed on the technology now and wait for the almost certainly inevitable addition of backpropagation in future releases! Here’s some more light reading and viewing to help you out with that:

WWDC16 715 – Neural Networks and Accelerate

Basic Neural Network Subroutines: Using The Accelerate Framework’s Convolution Filters

Neural Networks in iOS 10 and macOS

The “hello world” of neural networks

The Perfect 10

Yep, it’s that time of year again, where we all panic about getting new iOS features into our apps before ship date. Know the feeling? Thought so, yep. We’ve got a better excuse than usual this year though; this last month we’ve been settling into the electric city of Bangkok to work at agoda.com. Which we’d thoroughly recommend based on our experience so far; check out all the great jobs if you’re looking for an awesome gig! Any-ways, if you’re looking to get up to speed on iOS 10, here’s some great resources to put the pedal down:

iOS 10 Day by Day

This series has been a great resource for several OS versions now, and this year sure doesn’t look like a letdown so far:

Day 1 :: Messages

One of the new features was the ability for 3rd-party developers to create their own message extensions that can be used from within the Messages application… To demonstrate this exciting new extension type, we’ll take a look at a project that allows two players to play a simplified version of the popular game, Battleships,

Day 2 :: Thread Sanitizer

Xcode has shipped with a new debugging tool called the Thread Sanitizer which can help identify [threading] issues before you even notice them. We’ll create a simple application which enables us to deposit and withdraw money in $100 denominations…

Day 3 :: Xcode Source Editor Extensions

We’ll take a look at building our own extension that replaces ASCII characters with their Emoji equivalents…

Day 4 :: UIViewPropertyAnimator

To explore this new class, we’ll go through a few examples where we animate an image across the screen…

Of course, the Mother of All Tutorial Sites aka raywenderlich.com has their (6th!) annual opus proceeding nicely:

iOS 10 by Tutorials: First 6 Chapters Now Available!

Also note the iOS 10 + watchOS 2 bundle, and check out the latest tutorials/screencasts.

And one particular to finish off with; our entire career, we’ve had requests for voice dictation apps that we’ve always begged off with “Dude, that’s hard. And expensive.” Well, LOOK HOW EASY IT IS NOW:

Building a Speech-to-Text App Using Speech Framework in iOS 10

Truly, we live in the age of marvels.

UPDATES:

A List of All Current iOS 10 UI Kits for Sketch

iOS 10 Sampler – A Set Of Examples For Many Of The New Features Added With The iOS 10 SDK

Getting Ready for ATS Enforcement in 2017

How to generate haptic feedback with UIFeedbackGenerator

Tutorial: Using iOS 10’s UIViewPropertyAnimator Class