Cocoa at Tumblr

Chronicling the fun (and not so fun) parts of building iOS and OS X applications. Brought to you by the iOS developers at Tumblr. hljs.initHighlightingOnLoad();

[staff]

It’s the Great Tumblr Bug Hunt: iOS Edition

Help find bugs lurking in the Tumblr iOS app before they get released to the bug-hating public.

Who’s eligible: The first 10,000 people to register. You need to be an iOS user, obviously. And you should use Tumblr every day—because bugs are natural sneaks. You won’t find them if you’re not using the app all the time.

How it works: You’ll get access to each new release about a week before it goes public. When you see a bug, take a screenshot and email it to to [email protected] with a description of what happened and what you were doing at the time. (Some legal stuff: Any feedback that you send in is completely voluntarily but it will be exclusively owned by Tumblr so we can use it to make Tumblr better—even the mean stuff).

What you should know: You’ll be looking for bugs, which means there are bugs to look for. Don’t register if you’re expecting a bug-free experience. But if you get tired of bugs, you can leave the beta program at any time. Also, even though it’s a beta, it’s still the Tumblr app and the same Tumblr Terms of Service and Privacy Policy applies.

How to register: Join this Google group with your email. In a couple days you’ll get an email from TestFlight with instructions on how to install the beta.

What else you should know: Is in this FAQ.

[engineering]

Rapid View Prototyping Using FBSnapshotTestCase

Paul Rehkugler gave this talk at the Brooklyn Swift Meetup on Apr 12, 2016.

FBSnapshotTestCase is a powerful tool that was built to unit test the visual appearance of UIViews, but that’s not all it can do. This talk will explore how you can leverage FBSnapshotTestCase to simulate the visual feedback of Interface Builder when programmatically laying out views.

You can find the code used for this exercise on Paul’s GitHub.

Paul Rehkugler (pr) is a Staff Engineer at Tumblr. If he’s not writing software, he’s probably hiking or playing guitar.

[cocoa]

MFMailComposeViewController Unexpectedly Returns Nil When MFMailComposeViewController.canSendMail is False

When compiling with the iOS 10 SDK and running on an iOS 10 device, calling MFMailComposeViewController() can return nil when MFMailComposeViewController.canSendMail is false, despite the fact that its initializer is not marked as failable.

We’ve filed a radar about this issue (OpenRadar).

[cocoa]

Time for a pop quiz. Given that NSFetchedResultsChangeType is defined as follows:

@jasdev and @pr tracked down a tricky Core Data bug related to Open Enumerations.

[engineering]

Mozart

When Tumblr announced its Spring ’16 Hack Day, we decided to build an audio composer into the app. We codenamed it Mozart.

Giving people “tools” to create something - in this case a musical composition - in a way that feels effortless and powerful felt greatly rewarding. A huge source of inspiration for us was the Byte app. We were also excited because this was going to be in the Tumblr post form.

Keep reading

[cocoa]

WWDC 2016 has come and passed, but we wanted to take the time to call out the new ideas that Apple unveiled which we think are important as developers, as well as things to make our product teams aware of for future launches.

WWDC has slowly returning back to a software and developer focused event over these passed few years, and this year was no exception. Many new technologies, tools, and ideas introduced for developers to plug into to enrich both their applications as well as the Apple ecosystem in general. So let’s get into what we saw and enjoyed.

Reinvestment into developer tooling

This year Apple will introduce Xcode 8 to all developers. With this major version update comes a multitude of things designed to help everyone make better apps, find problems before they are in the wild, and alleviate common headaches.

The thread sanitizer and memory debugger are huge additions to a growing box of tools that developers can grab from in order to track down problems. The sanitizer is actually an addition to a new version of Clang that Apple has integrated into Xcode which can help identify race conditions and threading problems before you ship into the wild. This can be a boon for developers as these specific problems are almost always tricky to track down and resolve, providing static analysis around this lets developers fix their problems and continue back to working on their application.

The memory debugger added to Xcode 8 seems like it could be one of the most powerful debugging tools added in years. Visualizing the somewhat abstract concept of reference counting and object ownership allows for people of all levels to understand where a problem is happening and easily address it. While using & understanding the tools provided in Instruments is still very valuable, this single tool allows for folks to tackle some of the most dangerous issues in their application with relative ease.

Finally, Apple “Fixed It”, code signing has been redesigned to be less opaque for everyone! The first part of this solution is the redesigned signing UI which introduces simplified and customized signing flows. These allow you to work with sane defaults, or specify your own information (which Xcode will no longer modify :) ). The second part is you can finally have multiple development certificates! For folks that have multiple computers this is fantastic, and a very reasonable choice for Apple to allow since this is a predominant development pattern for some teams.

Simple and to the point code-signing settings.

Extensibility

This year Apple introduced a slew of new extension points for iOS and macOS. It seems that Apple has doubled down on the places you can extend your application into the iOS ecosystem even further than previous years. There are NINETEEN(!) different extension points currently available in the Xcode 8 beta, some of which allow for deep integration within iOS.

If you’re a VoIP application, messaging application, or ride sharing application there are serious advancements for integrating into the Messages application, Siri, and the built in Phone application. All through different extension points without the need to ship multiple apps in order gain this added functionality.

One of the extension points that we were most happy about was the Xcode extension point. Finally there is an approved plugins API for developers to extend the functionality of Xcode without having to swizzle things! However, this API seems somewhat limited for the time being, but the Apple engineering teams encouraged everyone to file bugs about what features they’d like to see in the future. The major downside here is that it seems that this move will likely kill Alcatraz, which is unfortunate.

As always, there’s a ton of other fantastic things that came out, but for more information you should head on over to https://developer.apple.com/videos/wwdc2016/ and soak it all in.

— @brianmichel

P.S. here’s a photo of people that could not be torn away from their phones 😁

P.P.S. Good Charlotte is still around!

[ziba]

Ran into Craig walking around Moscone :) #wwdc16 (at Moscone Center)

[engineering]

Tumblr engineers are at WWDC again this year!

[engineering]

Peeking into a Black Box - The Effect of Dependency Structure on iOS Launch Times

Abstract

We’ve experienced issues with iOS’s dynamic linker increasing our app’s launch time to unacceptable levels. To learn more about the linker’s inner workings and explore possible solutions, we created a Ruby script that generates dependency hierarchies of arbitrary complexity and tested their effect on launch times.

Keep reading

[engineering]

Swift Compilation Reporting at Tumblr

Doing a clean build of the Tumblr iOS app takes…a while, and we have a bunch of developers on the team, assuming each person does at least two clean builds a day1 that adds up to, well, too much time. As we march forward with Swift for new features, we noticed our compilations times increasing at surprising rates. There have been a few blog posts outlining expressions that the Swift compiler has trouble on. For instance, type inference of nested literal expressions2 and operator overloads are expensive to resolve.

To address this, we decided to automate monitoring of compilation performance. The goal was to create a weekly job that would compile our project with specific debug flags3, process the results, and email out the slowest compilation paths.

The result is a Swift script, called SwiftCompilationPerformanceReporter (nicknamed SwiftCPR), that we use to generate our weekly compilation report. Below are the steps SwiftCPR takes:

Runs a clean build with the following command4

xcodebuild -workspace workspacePath -scheme scheme clean build OTHER_SWIFT_FLAGS="-Xfrontend -debug-time-function-bodies" | grep [1-9].[0-9]ms | sort -nr > buildOutputDirectory

where workspacePath, scheme, and buildOutputDirectory are the workspace file, scheme, and output directory for the raw compilation logs, respectively. These can be specified in the config.json file.

Processes the raw compilation logs and merges duplicate entires.

Sample raw logs:

5992.1ms /Users/tumblr/workspace/SwiftCPR/orangina/Classes/PerformanceLoggingEvent.swift:267:37 final get {} 5718.3ms /Users/tumblr/workspace/SwiftCPR/orangina/Classes/PerformanceLoggingEvent.swift:267:37 final get {} 4376.1ms /Users/tumblr/workspace/SwiftCPR/orangina/Classes/UniversalLink.swift:127:25 private final class func dictionaryOfAppArgumentsFromQueryString(string: String) -> [NSObject : AnyObject]? ...

Outputs a final report with the total compilation time and the slowest limit compilation paths, where limit can be configured in the config.json file.

Sample report:

Total build time: 1115.24661797285 11.7104 /Users/tumblr/workspace/SwiftCPR/orangina/Classes/PerformanceLoggingEvent.swift:267:37 final get {} 8.5783 /Users/tumblr/workspace/SwiftCPR/orangina/Classes/UniversalLink.swift:127:25 private final class func dictionaryOfAppArgumentsFromQueryString(string: String) -> [NSObject : AnyObject]? ...

Once the above steps are complete, our job emails the report to the team! From these insights, we have been able to refactor functions that took over 10 seconds to compile to roughly a tenth of a second. Hope this script can help your team better profile Swift compilation times!

Jasdev Singh

1: Majority of our builds are incremental.

2: This has been resolved and the fix will ship with Swift 3!

3: Specifically, -debug-time-function-bodies

4: Thanks to Michael Skiba!

[yahooeng]

Using NSURLProtocol for Testing

By Roberto Osorio-Goenaga, iOS Developer

Unit testing networking code can be problematic, due mostly to its asynchronous nature. Using a staging server introduces lag and external factors that can cause your tests to run slowly, or not at all. Frameworks like OCMock exist to specify how an object responds to a specific query to address this behavior, but a mock object must still be set up for each type of behavior being mocked.

Using Apple’s NSURLProtocol, we can create a test suite that eschews these problems by mocking the response to our network calls centrally, essentially letting your test focus only on business logic. This protocol can be used not only with the built-in NSURLSession class, but can also be used to test classes and structs written with modern third party networking libraries, such as the popular Alamofire. In this article, we will look at mocking network responses in Swift for requests made using Alamofire. The sample project can be found on github.

NSURLProtocol’s main purpose is to extend URL loading to incorporate custom schemes or enhance existing ones. A secondary, yet extremely powerful, use of NSURLProtocol is to mock a server by sending canned responses back to callbacks and delegates. Say we have a very simple struct that uses Alamofire to make an HTTP GET request.

Fig 1 - A simple struct that serves as a REST client

The sample in Figure 1 creates a struct with an NSURL as an init parameter, and a sole method, getAvailableItems(), taking in a completion block as an argument, making a rest call to the NSURL and populating an array of MyItem in the block sent into it. From a testing perspective, we’d like to have a JSON response that matches the expected response, containing an object called items whose value pair is an array of strings. In order to make our tests as thorough and robust as possible, we’d also include at least two other mock responses: a JSON response that does not match this expectation, to test the else clause, and a garbage or erroneous response to check our error handling.

  Fig 2 - A valid response

  Fig 3 - A non-valid response

  Fig 4 - A throw-away garbage response  

Figures 2, 3 and 4 show a valid response for our purposes, a non-valid yet correct JSON response, and a throw-away string that isn’t even valid JSON, respectively. Without having to make a full-blown staging server, let’s see how we could go about testing these using NSURLProtocol.

To understand where NSURLProtocol fits into this problem, it’s important to look at a bit of the architecture Alamofire employs. Alamofire works as a singleton, as one can see from the above example. There is no instantiation required. Just feed a URL in, and make a request. Under the hood, the entity making the request is called the Manager. Manager is the entity that actually stores the URL and parameters, and is responsible for firing off an NSURLSession request abstracted from the caller class.

The manager for Alamofire can be initialized with a custom configuration of type NSURLSessionConfiguration, which has a property called protocolClasses, an array of NSURLProtocol members. By creating a new protocol that defines what happens when NSURLSession tries to reach a certain type of endpoint, loading it into the protocol array of a new configuration at index 0 (the default configuration), and initializing a new Manager object with this configuration, we can inject Alamofire with a simple, local mock server that will return whatever we want, given any request. Let’s start setting up a test class for our REST client by extending NSURLProtocol to respond to GET requests, and creating an Alamofire.Manager object with a custom NSURLSessionConfiguration that employs our protocol.

Fig 5 - Setting up a testing class for our client

Great, now we have an NSURLProtocol class that takes a GET request, checks the URL, and returns either a valid JSON response, or a simple “GARBAGE” response. This should allow us to test how our client responds. We still haven’t written any cases. We have a MyRESTClient property, as well as a Manager property. We also have a setup initial method that instantiates and loads our custom protocol into the manager instance. We now need a way to inject this manager instance into our Alamofire singleton. Let’s extend our client to the following.

Fig 6 - The REST client with an injectable “manager” parameter

We’ve added an initializer to our struct that allows us to send either a custom manager or nil into Alamofire. When the parameter is nil, the manager will load with its standard configuration. We also edited the request execution to be called via the manager we selected instead of directly through Alamofire. We can now add the following test case to our test class.

Fig 7 - Our first test case

In this test case, we create a new client, and give it our custom manager through the new initializer. We set a testing expectation, since the result comes back on a closure, and, after loading our itemsArray inside, fulfill the expectation. We tell the test case to wait for said expectation to be fulfilled, and, once it is, we make sure the itemsArray contains three items. If so, our test is successful, and our business logic is tested for getAvailableItems. Notice that we have used a bogus URL of “http://notNil”, which we have defined in the protocol to be selected in the conditional for populating the response correctly. To test the “garbage” case, we could write a test like the following.

Fig 8 - A test case for verifying a garbage response

In this second test case, the mocked URL of “http://nil” is not recognized, and the protocol responds by returning “GARBAGE”, thus not populating the response array. If our method is written correctly, it will call the closure with a nil array.

Fig 9 - A test case for verifying an incorrect response

In the third and final test case, our protocol class will return a “concepts” array instead of an “items” array, so the end result should still be a nil array in the closure.

As you can see, using NSURLProtocol we have created what amounts to a tiny server that responds to our requests and replies as specified, perfect for testing our asynchronous net calls. Now, go forth and test!

[cocoa]

Build Speed

Curious about build speeds? Turn this on and get that information in your progress HUD in Xcode.

defaults write com.apple.dt.Xcode ShowBuildOperationDuration YES

[cocoa]

NSURLSession does not copy its configuration on iOS 8

It retains it, despite being marked as copy. As such, you can still mutate the reference after setting it.

This is fixed in iOS 9 so make sure you weren’t relying on it.

[cocoa]

Conforming to Objective-C protocols with custom getters in Swift

@protocol Loadable <NSObject> @property (nonatomic, readonly, getter = isLoaded) BOOL loaded; @end

class Photo: Loadable { @objc(isLoaded) let loaded = true }

[cocoa]

Swift properties need to be `dynamic` to be KVO’able

[cocoa]

Beware of Hash Collisions

Consider this code. What do you think the values of numberArrayHashesEqual and stringArrayHashesEqual are?

let numberArray1 = [1, 2, 3] let numberArray2 = [2, 3, 4] let numberArray1Hash = (numberArray1 as NSArray).hash let numberArray2Hash = (numberArray2 as NSArray).hash let numberArrayHashesEqual = numberArray1Hash == numberArray2Hash let stringArray1 = ["like", "reblog"] let stringArray2 = ["follow", "post"] let stringArray1Hash = (stringArray1 as NSArray).hash let stringArray2Hash = (stringArray2 as NSArray).hash let stringArrayHashesEqual = stringArray1Hash == stringArray2Hash

Watch out, because NSArray.hash() is a shallow hash, and only reflects the number of items in the array:

let numberArray1 = [1, 2, 3] let numberArray2 = [2, 3, 4] let numberArray1Hash = (numberArray1 as NSArray).hash  // 3 let numberArray2Hash = (numberArray2 as NSArray).hash  // 3 let numberArrayHashesEqual = numberArray1Hash == numberArray2Hash  // true let stringArray1 = ["like", "reblog"] let stringArray2 = ["follow", "post"] let stringArray1Hash = (stringArray1 as NSArray).hash  // 2 let stringArray2Hash = (stringArray2 as NSArray).hash  // 2 let stringArrayHashesEqual = stringArray1Hash == stringArray2Hash  // true

[cocoa]

Extensions’ frameworks should be included in the application itself

When working with frameworks and application extensions in iOS, make sure to not copy your frameworks into the extension. Every application extension by default has a Runpath Search Path configured to look in the main application’s Frameworks folder for everything.

This is not what Apple mentions in their programming guide, but when you submit you will likely get rejected for ITMS-90205 & ITMS-90206 errors. Here’s some discussion on SO, and here’s Apple’s official guide describing how to handle frameworks and extensions. Best of luck, YMMV.

[brianmichel]

Adventures in making a CocoaPods Plugin

So, there I was, trying to comprehend why I couldn’t grab some dependencies that had been resolved by including a specific pod only to realize that it was a problem with not being able to clone in a specific way. In my case it was a specification that had a source URL over SSH as opposed to HTTP. An hour or two later I figured out a workaround by replacing the specific source URL with an HTTPS URL instead of using the specified git URL. Meaning I would transform something like this:

[email protected]:Organization/repo.git

into something like this:

https://[email protected]/Organization/repo.git

Let’s talk about this transformed structure for a second first. It’s almost a standard https URL that you’ve seen dozens of time on GitHub and other places. But what’s this access-token-here bit? It’s a GitHub personal access token!1

We now know the structure of the transform we need to create, all we need is a way for us to tell CocoaPods to run code that will perform the transform. We’re in luck, since CocoaPods 0.28, there has been a plugins interface! Check out their blog announcement here.

After reading the blog post, I was sure this was the right way forward, and there were even linked examples, and an informal interface, which was all good stuff. The following made me very happy to see they had thought about this.

Plugin support allows one to tweak the internals of CocoaPods and to insert additional commands.

Here’s where things get a bit murky. The listed example only covers 1/3rd of the extensibility options offered by this plugin interface. If you need to make a plugin that patches internals (dangerous), or uses a pre or post-install hook (a sanctioned CocoaPods interface) you’re on your own. Thankfully, due to the informal interface conventions, every other plugin must begin with cocoapods- so let’s take a look at RubyGems.org and find some other plugins.

I ended up checking out the cocoapods-src plugin which will checkout all of the sources for a pod after you’ve finished running pod install. I chose this because it utilized a post-install hook as it’s trigger, and figured this would closely mirror what a pre-install hook plugin would do. Here’s the bit of code in the gem that helped me get going in the right direction:

Pod::HooksManager.register(:post_install) do |installer_context| Pod::Src::Downloader.new(installer_context).download end

Great, I’m off and running by changing that to this:

Pod::HooksManager.register(:pre_install) do |installer_context| configuration = SourceURLRewriter::Configuration.new(installer_context) SourceURLRewriter::Rewriter.new(installer_context, configuration).rewrite! end

I installed my gem locally, ran pod install, and…nothing happened.

At this point the effort of this project hockeysticked. Without much documentation to go off of, I setup my machine as if I was going to contribute to the CocoaPods project (instructions here). My plan was to search through the code for some symbols I’d been using such as :pre_install and work my way through what was going on.

I found that, in Pod::HooksManager::Hook, the use of hooks without specifying a plugin name has been deprecated. Okay, so now I just need to specify a name for this plugin, no problem. In the same file there is the signature for the register function. I can use that to figure out how to call this function with a name:

Pod::HooksManager.register('url_rewriter_plugin', :pre_install) do |installer_context| configuration = SourceURLRewriter::Configuration.new(installer_context) SourceURLRewriter::Rewriter.new(installer_context, configuration).rewrite! end

I rebuilt my gem, installed it locally, ran pod install, and…nothing happened, again.

At this point, I’m losing the desire to even want to try and build a plugin, and start entertaining the idea in my mind that I can figure out a different way to solve this problem.

After reading more of the source, and reaching out for help from the masterful @segiddins, he explained that the name of plugin you register must be exactly the same as the gem you publish as your plugin. In retrospect this totally makes sense, and I don’t know of another way this could work.

So I renamed the pre-installation hook that I’m registering, built my gem, installed it locally, and ran pod install, and…stuff happened!

However, at this point I realized that my problem was a bit deeper than simply grabbing URLs of the dependencies specified in my Podfile and changing them. Since after dependency resolution there are likely many other dependencies that you need to build, but are not directly specified, I would need to find a way to rewrite these URLs at the time of downloading vs. just changing them in memory on the Podfile object.

WARNING: This gem was designed to be a stop gap solution and monkey patching is dangerous, YMMV!

I worked with Sam a bit more, and we decided that a solution, albeit a dangerous one, would be to patch the Git downloader object in CocoaPods. Since this was designed to be a temporary solution, I felt comfortable doing this and then working other channels to resolve the actual network issue. Ultimately, I ended up with a CocoaPods plugin that leveraged something like the following code in my cocoapods_plugin.rb file to deliver a solution:

module Pod module Downloader # Reopening the Git downloader class class Git alias_method :git_url_rewriter_url, :url def url source_url_rewriter.url_for(git_url_rewriter_url) end def source_url_rewriter @source_url_rewriter ||= SourceURLRewriter::Rewriter.new end end end end

I can’t stress how dangerous this is, and ultimately not guaranteed to last forever. I am willingly taking advantage of the nature of Ruby to add behavior into the CocoaPods library that suits my needs in the short term. The decision to patch any library in this fashion should not be taken lightly with the expectation that this will break in the future.

Now that I had this working, I wanted to be able to provide these rewrite options within my plugin declaration so that it was highly visible in the Podfile what was going to happen. For this I looked at the cocoapods-keys plugin to see how they configured their plugin. Our goal is to be able to provide syntax like this:

plugin 'cocoapods-git_url_rewriter', { '[email protected]:' => 'https://[email protected]/' }

Which leads us towards an internal method that looks like this:

def user_options @options ||= podfile.plugins['cocoapods-git_url_rewriter'] Pod::UI.notice('No options have been specified for rewriting') unless @options @options end

On the Podfile object, there exists a hash of plugin which can be referenced by the name of your plugin. The value returned from this is the hash you passed in when you declared the plugin usage in your Podfile. This allows us to keep sensitive info out of the plugin, while being very descriptive of what will happen to our dependencies.

Once this had been done, I could test my gem and see that the resources I previously was unable to reach are now being resolved, and downloaded correctly. Additionally, I was able to provide a simple solution that was transparent to my fellow developers and extensible for future use (but hopefully not needed). I ran into some problems that I feel like others developing plugins might hit which is why I felt the need to document them here.

Please feel free to reach out if you have questions I’m @brianmichel and I couldn’t have done this without help from the awesome @segiddins, seriously, he’s great.

1. Personal access tokens are a solution provided by GitHub and GitHub Enterprise that allow you to specify them instead of OAuth tokens or the basic username:password combination which will grant the caller access to a specific resource. These tokens are the kind of thing that you create once, get one time to look at it, and then it just becomes an opaque resource on your account. Additionally, they can be configured with different limitations (check out the options in the screenshot).