#imageio

[arcanelab]

SwiftyImageIO - A swift wrapper around ImageIO framework

[sktenu]

Apple Lovers Get Patching – Just One Text Can Steal Your iPhone Passwords

Just a year ago vulnerabilities in Android allowed hackers to quietly spy on nearly a billion phones with one specially-crafted text. iPhone owners should now take note: a security researcher today…

[theanimatron]

A Fix for ImageIO: Making Animated GIFs From Streaming Images

Animated GIF images are widespread on the Internet, despite the fact that most modern browsers can play video on a web page with no additional software installed. GIF images have lots of drawbacks compared to video files - they’re much bigger, don’t support true-colors (GIF palette is limited to 256 distinct colors), can’t include sound - but they’re still a popular way to distribute short animations or even video clips. Even in 2014.

Animatron supports publishing an animation as a GIF file though the excellent library Gif4J. This awesome library does a great job of optimizing frames, compression and color palettes for resulting GIF files. From my point of view, it provides the best results of GIF encoding out of all the tools available for Java in a server-side environment. Since it heavily uses various optimization tricks and tries to be abstract of the input source, it uses BufferedImage as the source of image data. So, it doesn’t care what you’re using as source - a set of JPEG images, PNG or whatever else, even frames generated by Java2D - as soon as you convert it to an instance of BufferedImage, Gif4J can deal with it.

Note: we love Scala at Animatron. At least, some of us do, and I’m the spiritual leader of this sect. So all the following examples will be in Scala.

Now, everything is fine if you have set of frames. The code to create an animated GIF looks like:

val writer = new GifImage() writer.setDefaultDelay(2) writer.setLoopNumber(0) val frames = … // get a sequence of BufferedImage instances for (frame <- frames) { writer.addGifFrame(new GifFrame(frame)) } GifEncoder.encode(writer,new File(“animated.gif”))

So the question is - how to get a list of BufferedImages? Well, it depends. If you’ve got bunch of, say, PNG files - then you can use ImageIO and its helper methods.

val writer = new GifImage() writer.setDefaultDelay(2) writer.setLoopNumber(0) val frames = new File("path/to/pngs").listFiles(new FilenameFilter { def accept(dir: File, name: String): Boolean = name.endsWith(".png") }).toStream.map(ImageIO.read) for (frame <- frames) { writer.addGifFrame(new GifFrame(frame)) } GifEncoder.encode(writer,new File(“animated.gif”))

The frames variable would be of type Stream[BufferedImage] and we can iterate on it and feed the Gif4J with the data it needs in order to create an animated GIF image.

So far, so good. If you have a set of JPG/PNG/other image files - you can create an animated GIF out of them. But here, in Animatron, we don’t have files on filesystem - instead we have to deal with a stream of PNG files, produced by PhantomJS and written to the standard output of the PhantomJS process. At first glance, it doesn’t seem to be a problem - just follow the same approach as with the set of files - but use stream as source of data to be read by ImageIO. Let’s try this.

import _root_.javax.imageio.ImageIO import com.gif4j.{GifEncoder, GifFrame, GifImage} import java.io._ object Sample { def createAnimation(is: InputStream, os: OutputStream) = { val writer = new GifImage() writer.setDefaultDelay(2) writer.setLoopNumber(0) var finish = false var framez = 0 while (!finish) { val frame = ImageIO.read(is) writer.addGifFrame(new GifFrame(frame)) finish = frame != null framez = framez + 1 } GifEncoder.encode(writer, os) framez } def main(args: Array[String]) { val os = new FileOutputStream(args(1)) val rendered = createAnimation(new FileInputStream(args(0)), os) os.flush() os.close() println("Rendered framez: " + rendered) } }

As the source, we could use the Animatron logo (save it as 1.png). Now we’re going to emulate a stream from single file - just create another file with multiple copies of the logo (some shell-scripting):

for i in 1 2 3; do cat 1.png; done >> out.png

The command above will create file “out.png” with content of 1.png, copied 3 times.

Now, let’s try to read images from the file. The result on my system is:

Rendered framez: 1

So ... only one frame is read? WTF? If you try to google the problem, you’ll find posts on stackoverflow like this. So in short - ImageIO cannot read more than one image from the stream.

I wanted to understand what was wrong, so I did some slight modification of the source code to see what was going on with the image streams inside. I’d like to say “many thanks” to the guys at Oracle, who bundled ImageIO classes into the “rt.jar” with no debugging info - that obviously added some fun to my life that I couldn’t live without.

So I added some “tracing” to the image generation code. I wanted to see how many bytes have been read from the stream, and what are the first 20 bytes from the remaining stream, after ImageIO finished with an image creation.

def createAnimation(is: InputStream, os: OutputStream) = { val src = new CountingInputStream(is) val writer = new GifImage() writer.setDefaultDelay(2) writer.setLoopNumber(0) var finish = false var framez = 0 while (!finish) { val frame = ImageIO.read(src) writer.addGifFrame(new GifFrame(frame)) finish = frame != null framez = framez + 1 } println("Read bytes:" + src.getByteCount.toHexString) val data = Array.ofDim[Byte](20) is.read(data) println(data.map(x => String.format("%1$x", java.lang.Byte.valueOf(x))).mkString(":")) GifEncoder.encode(writer, os) framez }

I used CountingInputStream from Commons IO, which is a great collection of tools and classes that have been missing from Java core for decades. The results now are:

Read bytes:2826 a7:dd:6a:20:0:0:0:0:49:45:4e:44:ae:42:60:82:89:50:4e:47 Rendered framez: 1

So 2826 bytes were read (in HEX) and 20 bytes from the remaining stream were printed.

Okay, let’s look at what’s going on inside the out.png. I recalled the good old hiew program and found the port of it for Arch Linux. So open the file out.png, change the mode to “Hexadecimal mode” by pressing F2 and then go to to the offset 2826 (it’s in HEX!) by pressing F5. You’ll see something like this:

Notice that the output of our Scala program

a7:dd:6a:20:0:0:0:0:49:45:4e:44:ae:42:60:82:89:50:4e:47

matches with the first line in the HIEW, so we seem to be on the right track. Let’s look at the brief definition of PNG image format. I was especially interested in finding the boundaries of PNG file in a stream. So, according to the PNG format specification: a PNG file starts with an 8-byte signature and IEND marks the image end.

So, looking back into the HIEW - the first line contains both IEND block, which marks the end of the first PNG image, and beginning of next PNG image (as marked by the sequence of bytes 89:50:4e:47). This leads us to 2 important conclusions:

1. The authors of ImageIO’s PNG reader didn’t pay much attention to reading a PNG file from a stream correctly. So, when it’s applied to reading from a stream, ImageIO leaves some parts of a PNG file in the remaining stream, and any subsequent call to ImageIO.read method on the same stream will lead to no image being read - because there’s no Image header available. Worse, if by some accident a set of bytes looks like a valid image header - the remaining stream will be read completely wrong.

2. It’s quite easy to fix this behavior by skipping the content of the stream until the next PNG image is found.

Keeping this in mind, the code which will correctly read a set of images from a stream and create an animated GIF may look like this:

import _root_.javax.imageio.ImageIO import com.gif4j.{GifEncoder, GifFrame, GifImage} import java.io._ import org.apache.commons.io.IOUtils import java.awt.image.BufferedImage object Sample { private def b(x: Int): Byte = x.toByte private final val PNGHDR = Array[Byte](b(137), b(80), b(78), b(71), b(13), b(10), b(26), b(10)) private val MAX_IMG = 1024 def createAnimation(is: InputStream, os: OutputStream) = { val data = Array.ofDim[Byte](MAX_IMG) val stream = new BufferedInputStream(is) val writer = new GifImage() writer.setDefaultDelay(2) writer.setLoopNumber(0) var finish = false var framez = 0 while (!finish) { stream.mark(MAX_IMG) val dataRead = IOUtils.read(stream, data) finish = if (dataRead > 0) { data.sliding(PNGHDR.length, 1).toStream.take(dataRead - PNGHDR.length).zipWithIndex.dropWhile { case (a, _) => !a.sameElements(PNGHDR) }.headOption match { case Some((_, idx)) => stream.reset(); stream.skip(idx); false case None => stream.reset(); true } } else true if (!finish) { val imageData: BufferedImage = ImageIO.read(stream) finish = imageData == null writer.addGifFrame(new GifFrame(imageData)) framez = framez + 1 } } GifEncoder.encode(writer, os) framez } def main(args: Array[String]) { val os = new FileOutputStream(args(1)) val rendered = createAnimation(new FileInputStream(args(0)), os) os.flush() os.close() println("Rendered framez: " + rendered) } }

The code above is pretty close to what we’re using in Animatron.

The moral of the story is that sometimes you shouldn’t just blindly trust code “proven by decades of usage,” but rather investigate what’s going on under the hood. It’s also important to understand the nature and protocols of software and files. And: good old tools from the ancient times of MS-DOS are still useful in the daily lives of Java/Scala developers :)

Eugene Dzhurinsky

Dark Lord of Serverside Orcs

[evadnelt]

Acquiring proper UIImage decoding performance

`UIImage` is lazy, and lazy is good. Lazy is only bad when it’s late to play catch up on required work. Since it never does any decoding unless somebody actually needed the bytes it represents, caching `NSData` objects read from flat files on the disk then spitting out new `UIImage` objects obviously doesn’t help presentation at all. The scrolling is often sluggish because it can’t reach 60 FPS. It can’t reach 60 FPS, because for every 100 pixels or so scrolled, there’s a new image needing to be decoded into something that the GPU understands, on the CPU, and it is being decoded on the main thread, while Core Animation, the cornerstone of iOS UI, is inherently heavily concurrent. If you’re planning to show any large image in a table view or anything that scrolls on demand, the real way to do it is to trigger a background decode, so `ImageIO` churns happily out of your users’ reach, and your interface is snappy. This snippet does the decoding, assuming it’s in something like `-[UIImage irDecodedImage]`: CGImageRef cgImage = [self CGImage]; size_t width = CGImageGetWidth(cgImage); size_t height = CGImageGetHeight(cgImage); CGContextRef context = CGBitmapContextCreate(NULL, width, height, 8, width * 4, CGImageGetColorSpace(cgImage), kCGImageAlphaNoneSkipFirst); CGContextDrawImage(context, CGRectMake(0, 0, width, height), cgImage); CGContextRelease(context); As long as the image is decoded (you can surely do it in a background thread, or in GCD terms a global queue with lower priority), you can safely use it in the UI without fearing that it would gunk up the scrolling… with alacrity. Keep it off the main thread. Now we have GCD, there is no excuse not to. :) *Bonus:* Don’t use Quartz to draw the entire table view cell as one big image. (Or, use it judiciously.) It might be tempting, but leave the compositing to the GPU. If you ever used any stretchable image, you’ll feel the pain of not abusing the hell out of `contentsCenter`. And you’ll feel the pain of hemorrhaging video memory for many one-use bitmaps of the entire cells; and it is pointless work when there is `shouldRasterize`. And it’s never easy to understand drawing code, and if there is any interaction you’re screwed… blah.