#xcode 5 bots

[redgreenrefactor-eu]

Developer Diaries: fixing provisioning and code signing problems on Mac OS X Server Xcode service

Apple made a gigantic step to make unit testing and continuous integration more enjoyable by making testing a first class citizen in Xcode 5. When you watch WWDC 2013 videos, you may be truly inspired by how easy it is to test and integrate using Mac OS X Xcode Service.

We still have to wait before Apple fixes a couple of bugs here and there, but not taking advantage of this great native support for testing and continuous integration would be rather hard to defend.

Ok, shortly, what does not work as it should and what can we do about it.

Signing Identities and Provisioning profiles

If your scheme supports creating an archive and you checked the Perform archive action in your bot settings (see the picture below) then you expect a ready to use, properly signed archive at the end of each successful integration.

Instead you may get two errors similar to these:

No matching provisioning profile found: Your build settings specify a provisioning profile with the UUID “9F2F3156-6380-4A91-AFA5-A68BD5781949”, however, no such provisioning profile was found.

CodeSign error: code signing is required for product type 'Application' in SDK 'iOS 7.0'

Apparently, the Xcode service on the Mac OS X server has big troubles retrieving the provisioning profiles from Member Center. Improving profile management was really Apple's big selling point for the Xcode service, so I am quite disappointed that this thing is not finished with the expected level of quality.

But first things first. This is what I see on my development machine when I look into the Apple ID account details in the Xcode Preferences:

My Mac OS X Server is running on another machine that I used earlier for development. I tried to configure both machines so that the machine on which runs Mac OS X Server mirrors as closely as possible my development machine. Unfortunately, Xcode had big troubles fetching all the Signing Identities and Provisioning Profiles from my member account. In particular, I could not make it fetching the iOS Distribution signing identity and my Ad Hoc provisioning profile. Creating an example project in Xcode and asking it to "Fix it" a couple of times also did not help. Only after I first exported Development Profile from my development machine and then imported it on the server machine, things started to look better. This operation also put all the required by Xcode keys into my keychain.

I hoped that if I now restart the server (by rebooting the computer) it will magically find all certificates and profiles and copy them intelligently to its secret locations. It did not.

In searching for an answer I found this: great blog post. I was put off at first with a post-build action script - if you still have to do this, then what's the point of using bots to create an archive - but still I managed to extracted some very useful information for which I am truly grateful.

Step 1 - fixing missing provisioning profiles

Copy your provisioning profile (it was an ad-hoc provisioning profile in my case) - the file with mobileprovision extension - to the following location:

/Library/Server/Xcode/Data/ProvisioningProfiles

I have downloaded my provisioning profile from the member center:

The location where you need to copy this file to is highly protected, so you will need to use sudo:

sudo cp your_profile.mobileprovision \ /Library/Server/Xcode/Data/ProvisioningProfiles/

This will fix the first error on our error list. If you run the integration again, you may still get the following errors:

No codesigning identities found: No codesigning identities (i.e. certificate and private key pairs) that match the provisioning profile specified in your build settings (“... Ad Hoc Provisioning Profile”) were found.

CodeSign error: code signing is required for product type 'Application' in SDK 'iOS 7.0'

The second one is our good friend now (well, it comes as a consequence of the first one), but we have to do something about new number 1.

Step 2 - fixing missing code signing identities

As described in the above mentioned article, you have to copy your iPhone Distribution certificate (see the picture below) to the server machine:

This is simple: just right (ctrl) click and choose Export... from the context menu.

> It may seem obvious, but make sure that your certificate contains the private key. There should be a small arrow on the left. When you click it you should see your private key unfolding (there will be a picture of a key on the left this time). If this is not the case for you, you have to find the complete certificate on some other machine.

Once you have it, you will need to import this certificate into your system chain on the server.

> If your Development Profile on the server machine is in sync with the Development Profile on your Development Machine (see the description earlier in this article), you already have this certificate but in the login chain. The Xcode service is not clever enough to find it there so you must copy it to the system chain.

This should get rid of our new error number 1 on the list. It also takes care for the error number 2, but as a gift it gives as a totally new error:

Command /usr/bin/codesign failed with exit code 1

It is not very verbal, but if you look into your log for the details you will find out this as part of the error description:

User interaction is not allowed.

What is this? Well, during code signing, Keychain Access is asking for confirmation and Xcode service does not find it convenient - of course.

Step 3 - fixing "User interaction is not allowed" codesign error

This issue is not directly described in the the article I mentioned at the beginning, but fortunately, there is a ready to use solution in the comment from one user:

> As per http://stackoverflow.com/a/14761060/1252368 the issue can be solved without scripting if you have access to the server’s UI – which you most certainly have when you use the Keychain Access application to install the certificate and identity.

> After you put the certificate and key into the system keychain, select the key and shows its properties via the context menu’s “Get Info” option. Then add /usr/bin/codesign to the list of allowed applications that can use the key without asking for permission first. You will have to enter your password one or two times to confirm, but once that is done, codesigning will work (just did it :) )

This should not be too hard:

Make your usr folder visible so that you can navigate to /usr/bin/codesign with Finder. You can use this command:

sudo chflags nohidden /usr

In Keychain Access, double click on the private key (the one with the picture of a key), select the Access Control tab and add /usr/bin/codesign to the list of "Allways allow access by these applications". Save the changes. You may be asked for your credentials more than once, but after successful authentication all should be fine.

> After adding codesign to the list you should see it on the list. However if you switch the tabs to Attributes and then back to Access Control, you will be surprised that codesign entry is no longer there. The same will happen if you close the property window and reopen it again. Don't worry. It is there, it just does not show. If you close Keychain Access completely and then start it again, the codesign entry will appear. It is all good.

Hide your usr folder from Finder if you want:

sudo chflags hidden /usr

This is it. Now Xcode service should be able to successfully create your archive.

Happy developing!

[redgreenrefactor-eu]

Developer Diaries: using remote repositories with Xcode 5 bots

> If you just need to know how to quickly setup your remote repository to use with bots with Mac OS X Server Xcode process and Xcode 5, you may jump directly to Section How to setup your external repository. If you want to know why, keep reading.

Background

Git with ssh follows the standard ssh URI format:

git clone ssh://[email protected]/home/username/repository.git

Following the RFC3986, a non-empty path must begin with the slash '/' character (see Section 3.3 of the RFC3986 document).

When used with the ssh transport protocol, git also allows for the scp-like syntax, which uses a colon ':' to separate the authority component from the path component of the URI. Using the shorter scp-like syntax, the above URI could be specified as follows:

git clone [email protected]:/home/username/repository.git

or, assuming that /home/username is a home directory of a user named username, we could write:

git clone [email protected]:repository.git

Notice that in the scp-like syntax you don't keep the scheme name at the beginning of the URI (no ssh:// in the example above). Git will use the ssh transport protocol by default.

Standard URIs also provide a way to shorten the path when referring to the user's home directory. The tilde '~' unreserved character (see Section 2.3 of the RFC3986 standard) can be used for this purpose. The actual meaning of this character is not standardised, but I can't recall ever seeing this character interpreted differently by any URI parser.

Using the tilde '~' character, we can shorten our full ssh URI as follows:

git clone ssh://[email protected]/home/username/repository.git git clone ssh://[email protected]/~/repository.git

The scp-like syntax also supports the tilde '~' character with exactly same meaning. The two URIs below are therefore equivalent:

git clone [email protected]:repository.git git clone [email protected]:~/repository.git

In each case you can skip the .git extension and achieve the same effect:

git clone ssh://[email protected]/home/username/repository git clone ssh://[email protected]/~/repository git clone [email protected]:/home/username/repository git clone [email protected]:repository git clone [email protected]:~/repository

The problem with the scp-like syntax for URIs is that it is not really standardised and as such does not have to be supported everywhere - it turns out that at the time of this writing Apple's Xcode 5 is one such place.

> What I am describing here is my experience. Your experience may be different depending on the version of the tool you are using. Apple may also change the they implement their git client in the nearest future. What I describe may therefore be quite specific, still I believe it may point you to the right direction.

The scenario

I have an iOS project. The project is under version control. It uses git. It only has one remote:

$ git remote --verbose origin [email protected]:repos/my_cool_project (fetch) origin [email protected]:repos/my_cool_project (push)

As you see, in the remote's URI I am using the scp-like syntax.

I open the project with Xcode 5 and it recognises that the project is under version control. It also seems to recognise my repository remotes without any issues (this dialog window can be accessed in Xcode 5 via Source Control/my_cool_project - master/Configure... menu):

You can see on the picture that the repository Origin is:

[email protected]:repos/my_cool_project

However, if you look into the Remotes tab of the same dialog, you will find something slightly different:

What we see in the picture is:

ssh://[email protected]/repos/my_cool_project

Well, it looks like Xcode decided to convert the scp-like syntax into a more standard ssh format, but did not realise that my scp-like URI points to the home folder of user steve. The closest ssh URI for the above scp-like URI would be:

ssh://[email protected]/~/repos/my_cool_project

It seems that Xcode 5 blindly replaces the colon ':' character from the scp-like syntax with the slash '/' character in the ssh URI. Therefore, in order to get a correct URI after Xcode's manipulation we should use the following scp-like URI as the input:

[email protected]:~/repos/my_cool_project

Xcode 5 will translate this URI into the following one:

ssh://[email protected]/~/repos/my_cool_project

> If you want to replicate this, use a real repository. Surprisingly enough, Xcode 5 does not bother to touch your URI if it is fake or when it does not align with what Xcode may find in the .git folder.

Why does it all matter?

It matters because your Mac OS X Server Xcode service may end up using an incorrect URI for the repository. In our example, if the remote URI is set to:

[email protected]:repos/my_cool_project

your bot will not be able to connect to the remote repository:

If you configure the remote URI as described in the next section, you will be able to connect to the remote repository:

How to setup your external repository

On your Mac OS X Server in the Xcode service add a new remote repository. Make sure that your repository uses format that is correct, i.e. if you use an scp-like syntax for the repository URI, make sure that you use the tilde '~' character after the colon if your intention is to point to the user's home directory.

So instead of:

[email protected]:repos/my_cool_project

use

[email protected]:~/repos/my_cool_project

Of course you can always use a valid ssh URI directly:

ssh://[email protected]/~/repos/my_cool_project

or even an absolute path:

ssh://[email protected]/home/steve/repos/my_cool_project

On your development machine, in Xcode, make sure that your remote is the same as the one you used on the server. You can view and change your remote settings through the Source Control/my_cool_project - master/Configure... menu.

Create a new bot. If your remote updated in the previous step does not match any remote repository URI on the Mac OS X Server, you will be asked for your repository credentials:

As you see, this is not very useful, as this dialog does not let you use the public key authentication. Cancel, align your remote with your server, and try again. If everything is correct, you won't be asked to provide any further details about your remote.

Nice to know

Xcode 5 shows the bots that are relevant to the project in the log navigator:

How does Xcode know which bots are relevant? Xcode looks at your repository remote URI. If it finds a bot associated with your remote URI, it will display the bot in the log. You can change the remote URI in Xcode and you will see that your bot disappears from the log navigator. In a similar way, when you change the URI of the remote repository on the server, the bot in Xcode will disappear as soon as you restart Xcode.

> Hint: if your bot is not visible after changing the URI on the server, try restarting Xcode 5 (do not only reopen the project: restart the whole IDE).

When changing URI of the remote repository on the server you may need to update your private key as well. It seems that the server wipes the private key each time you change the repository URI treating it as a new repository.