Caching audio streamed using AVPlayer
AVPlayer is quite convenient for playing audio/video files from the disk or the network. However, you may want to cache the audio that was played by AVPlayer but it is not straightforward. A good alternative for this used to be the AudioStreamer library written by Matt Gallagher. I've been using the library for sometime now, and although it's a fantastic library, there are some quirks that I've had to deal with especially around race conditions that are hard to reproduce and an 'hwiu' exception. Nonetheless, this library had served us well. Moving on..
I was looking for solutions to get AVPlayer to play an audio file from the network but also cache the downloaded data so the next time I can play it locally. Unfortunately that's not a very straightforward thing to do. I found a couple of helpful hints that tapped into the audio processing and re-recorded the stream as it was being played back by the system. This article from venodesigns.net had been tremendously helpful in implementing that. It worked great for the most part but it was unsuited for my purposes because I wanted the entire audio cached without interruptions. If the user seeked anywhere in the audio, my cached audio had gaps in it.
So I started looking around digging through the APIs and came across the resourceLoader object in AVURLAsset. This is actually an amazing API using which you can provide controlled access to a remote audio file to AVPlayer. This works like a local HTTP proxy but without all the hassles.
The most import thing to remember about this resourceLoader is that it will only come into play when AVPlayer does NOT know how to load a resource. It is quite equipped to deal with the "http" protocol so as long as you have http(s) URLs, no callbacks will be made. The trick here is to change the protocol so AVPlayer is forced to defer the loading of the resource to our application. The resource loader has two methods we need to deal with. We'll store the requests that are still pending in an array called pendingRequests. Then we'll start off an NSURLConnection to fetch the audio file, and as and when we retrieve more data, we will try to process the pendingRequests, if possible. The two delegate methods of AVAssetResourceLoaderDelegate we have to implement are:
// Called when the resource loader needs data or information about the resource. To take over the resource loading, we'll return YES from this method. - (BOOL)resourceLoader:(AVAssetResourceLoader *)resourceLoader shouldWaitForLoadingOfRequestedResource:(AVAssetResourceLoadingRequest *)loadingRequest // Called when a load request is being cancelled. - (void)resourceLoader:(AVAssetResourceLoader *)resourceLoader didCancelLoadingRequest:(AVAssetResourceLoadingRequest *)loadingRequest
To start off, we'll create an AVURLAsset object, change the URLs protocol to a custom one, and set the resource loader's delegate to be our class.
// Change protocol to streaming from http AVURLAsset *asset = [AVURLAsset URLAssetWithURL:@"streaming://sampleswap.org/mp3/artist/earthling/Chuck-Silva_Ninety-Nine-Percent-320.mp3"]; // Set resource loader on the asset so that we can control the loading process [asset.resourceLoader setDelegate:self queue:dispatch_get_main_queue()]; // This tracks all pending AVAssetResourceLoadingRequest objects we have not fulfilled yet self.pendingRequests = [NSMutableArray array]; AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:asset]; self.player = [[AVPlayer alloc] initWithPlayerItem:playerItem]; [playerItem addObserver:self forKeyPath:@"status" options:NSKeyValueObservingOptionNew context:NULL];
The delegate methods are basically responsible for adding and removing a request from the pendingRequests array. When the first request is received and no NSURLConnection exists, we'll create one and start fetching the audio file.
- (BOOL)resourceLoader:(AVAssetResourceLoader *)resourceLoader shouldWaitForLoadingOfRequestedResource:(AVAssetResourceLoadingRequest *)loadingRequest { if (self.connection == nil) { NSURL *interceptedURL = [loadingRequest.request URL]; NSURLComponents *actualURLComponents = [[NSURLComponents alloc] initWithURL:interceptedURL resolvingAgainstBaseURL:NO]; actualURLComponents.scheme = @"http"; NSURLRequest *request = [NSURLRequest requestWithURL:[actualURLComponents URL]]; self.connection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:NO]; [self.connection setDelegateQueue:[NSOperationQueue mainQueue]]; [self.connection start]; } [self.pendingRequests addObject:loadingRequest]; return YES; } - (void)resourceLoader:(AVAssetResourceLoader *)resourceLoader didCancelLoadingRequest:(AVAssetResourceLoadingRequest *)loadingRequest { [self.pendingRequests removeObject:loadingRequest]; }
The NSURLConnection delegate callbacks try to process any pendingRequests in all callbacks.
- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response { self.songData = [NSMutableData data]; self.response = (NSHTTPURLResponse *)response; [self processPendingRequests]; } - (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data { [self.songData appendData:data]; [self processPendingRequests]; } - (void)connectionDidFinishLoading:(NSURLConnection *)connection { [self processPendingRequests]; // Done loading, cache the file locally NSString *cachedFilePath = [NSTemporaryDirectory() stringByAppendingPathComponent:@"cached.mp3"]; [self.songData writeToFile:cachedFilePath atomically:YES]; }
The request handling is the crucial part. An AVAssetResourceLoadingRequest has two parts to it - contentInformationRequest, and dataRequest. A contentInformationRequest is a request to identify the content length, content type, and whether the resource supports byte range requests. With byte range requests, AVPlayer can get fancy and apply various optimizations. The first request I've seen asks for the first two bytes of data. Here is an implementation of the request handling part:
- (void)processPendingRequests { NSMutableArray *requestsCompleted = [NSMutableArray array]; for (AVAssetResourceLoadingRequest *loadingRequest in self.pendingRequests) { [self fillInContentInformation:loadingRequest.contentInformationRequest]; BOOL didRespondCompletely = [self respondWithDataForRequest:loadingRequest.dataRequest]; if (didRespondCompletely) { [requestsCompleted addObject:loadingRequest]; [loadingRequest finishLoading]; } } [self.pendingRequests removeObjectsInArray:requestsCompleted]; } - (void)fillInContentInformation:(AVAssetResourceLoadingContentInformationRequest *)contentInformationRequest { if (contentInformationRequest == nil || self.response == nil) { return; } NSString *mimeType = [self.response MIMEType]; CFStringRef contentType = UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType, (__bridge CFStringRef)(mimeType), NULL); contentInformationRequest.byteRangeAccessSupported = YES; contentInformationRequest.contentType = CFBridgingRelease(contentType); contentInformationRequest.contentLength = [self.response expectedContentLength]; } - (BOOL)respondWithDataForRequest:(AVAssetResourceLoadingDataRequest *)dataRequest { long long startOffset = dataRequest.requestedOffset; if (dataRequest.currentOffset != 0) { startOffset = dataRequest.currentOffset; } // Don't have any data at all for this request if (self.songData.length < startOffset) { return NO; } // This is the total data we have from startOffset to whatever has been downloaded so far NSUInteger unreadBytes = self.songData.length - (NSUInteger)startOffset; // Respond with whatever is available if we can't satisfy the request fully yet NSUInteger numberOfBytesToRespondWith = MIN((NSUInteger)dataRequest.requestedLength, unreadBytes); [dataRequest respondWithData:[self.songData subdataWithRange:NSMakeRange((NSUInteger)startOffset, numberOfBytesToRespondWith)]]; long long endOffset = startOffset + dataRequest.requestedLength; BOOL didRespondFully = self.songData.length >= endOffset; return didRespondFully; }
