#java nio

[bhaskarp]

Java NIO: Getting Started (Part II)

Stream based IO model of Java

Before we take up NIO its important that we refresh our understanding of the stream based IO model. It forms the core to java.io - the fundamental package which "provides for system input and output through data streams and serialization."

Let us analyse the model with focus on data streams as this is where the need of an improved treatment of IO in Java was felt.

Data Stream is a sequence of data units that can be accessed in order. The unit of data is fundamentally either a byte or a UTF-8 encoded character. Now we can either read from a stream or write to it. Any application that requires to read or write data would ultimately function at the level of data streams. A typical program reads data from a "SOURCE" in the form of an "INPUT" stream and delivers it to the "DESTINATION" as an "OUTPUT" stream. This approach generally translates into a loop that reads the input stream and writes to the output stream byte by byte or character by character. The following diagram attempts to elucidate the same:

Implications of this "stream oriented" approach:

1.) We cannot move back and forth. Data stream is actually a continuous flow of data. Unlike an array it is not indexed and the data units (bytes or characters) are not cached anywhere. So we simply cannot traverse the stream at will. If while writing an output stream you suddenly have to rewrite a portion of it, it becomes virtually impossible. Similarly while reading an input stream if you wish to reread a portion of the former half, you are hopelessly stuck. All that you can do to circumvent this is to cache your data in a buffer and then move back and forth. 2.) The thread remains blocked. In java.io the streams are "blocking". That is, if a thread is performing a read or a write operation it shall remain blocked until there is some new data to read or it is finished with the writing of data. More so, that if two threads attempt to read from a single input stream ,one beginning its read() after another the second thread will be forced to begin from where the first one leaves. 3.) Impedance mismatch. As has already been discussed in the previous blog, "impedance mismatch" is linked to the byte-by-byte or character-by-character treatment of data streams by the JVMs. To put it succinctly, the OS carts data in large chunks. These chunks need to be broken down by java.io before they can be used. As a result there is a huge mismatch in the speed with which data can be delivered by the OS to java.io and the speed with which java.io actually processes it further. To quote,Ron Hitchens from Java NIO, "The operating system wants to deliver data by the truckload. The java.io classes want to process data by the shovelful." This indeed appropriately summarises the performance spoiler java.io can sometimes be. 4.) Every connection has its own thread. In the Stream based model each connection (i.e., your input sources or output destinations) will require a separate thread. It is a direct fallout of the blocking nature "java.io" streams. So to deal with multiple connections at a time you shall need multiple threads. One for each. It turns out be a cumbersome when the amount of data in consideration is small and the connections are in thousands. 5.) Overhead generated by each IO request. Well this is one of the factors that result in the aforementioned mismatch of impedance. While the program loops the data byte-by-byte (or character-by-character) each such read or write request often triggers expensive operations like disk access or network activity. Again if we need to minimize these overheads we need to buffer the data first.

With the vision to rectify the inefficiencies of the stream oriented approach an improved model was developed and packaged as Java NIO!

[References:

https://blogs.oracle.com/slc/entry/javanio_vs_javaio http://docs.oracle.com/javase/tutorial/essential/io/bytestreams.html http://tutorials.jenkov.com/java-nio/nio-vs-io.html ]

[bhaskarp]

Java NIO : Getting Started (Part I)

Reconsidering IO

Friends,

Input Output, IO for short, hardly ever generates the excitement it deserves. Though indispensable, we often prefer not getting our hands dirty with the guts of the subject. It hardly ever appeals to us, and we wonder if its worth our effort. When Java so beautifully abstracts all the messy details, it appears wiser to quickly wrap it all up in few API calls.

We prefer to focus more on improving the processing of the application, to make it smarter by the day. IO naturally gets pushed away from receiving any serious attention. Given our time restraints, our mind just revolts against any idea of prioritizing this particular aspect.

With our concerns(or rather obsession) for process-optimization we often end up with lapses which have serious performance implications.

IO versus Process Time :

No matter how much effort is invested in optimizing the code, all of it may be belittled if the application suffers from IO inefficiencies. Contrary to the "process-optimization will decide it all" perception, which many of us carry, IO is actually a significant determinant in deciding the overall performance of our application. The singular fact that performing IO operations usually takes many times longer than in-memory processing, calls for its careful evaluation. The following analysis will probably elucidate with greater clarity as to how much I/O is crucial in our application. [The figures have been borrowed from the book "JAVA NIO” by Ron Hitchens]

Consider the case(hypothetical) where the in-memory processing of 1 unit of data requires 5 ms and the IO for the same consumes 100 ms. So the throughput of the application turns out to be 9.52 units/second.

[ Formula: (1/(process time + IO time))X 1000].

Now if we manage to decrease the processing time (keeping the IO expense constant) to 1 ms the gain in throughput will be around 3.96%.

However if the IO consumption is reduced to 75 ms (keeping process time fixed at 5 ms) the throughput gain is a whooping 31.25%.

Even at a slight 10% decrease in IO time (90 ms) the gain jumps considerably to 10.53%

The substantial throughput gains are enough to spur us into action and reconsider our IO approach.

Why this neglect?

However its not all our fault that IO takes a back seat! The ignorance stems from an old habit that should have, but did not die with time. In the early days of java JVMs offered little runtime optimization while interpreting the bytecode. They ran slower than their natively compiled counterparts. So there would have not been any specific benefit in working on IO performance. The latency in in-memory operations had diverted the attention from the need to ensure efficiency in IO. That is, the process was too slow and improving IO would not have resulted in any substantial throughput gain. Programmers instinctively remained preoccupied in reducing the processing time.

Nevertheless with time the runtime optimization improved and the processing time dropped. JVMs performed at speeds approaching that of the natively compiled code.IO obviously got pushed to the center stage in determining application performance.

How could Java help?

It was never the case that the OS failed to transmit data fast enough. It was actually the JVM that lagged behind. There was a huge "impedance-mismatch". Let us understand how.

The stream based model of java.io packages treats data byte by byte or character by character. In contrast the OS carts data in large chunks. These chunks need to be broken down by java.io before they can be used. Consequently there is a huge mismatch in the speed with which data is delivered by the OS to java.io and the speed with which java.io processes it further.

This very impedance-mismatch adversely affected the application performance.

Though java.io increases the probability that our application will be kept waiting for I/O, it is not that notorious! There are implementations that have bridged the lacuna to some extent, but still a paradigm shift in the approach was long overdue.

And then Java NIO emerged!

[Reference : "JAVA NIO” by Ron Hitchens]

[iteratia]

Netty, And A Fresh View Of Web Development

Rather a long time ago, Iteratia Corporation received an order to develop an unusual client-server application. It was assumed that the server would hold a huge number of simultaneous connections with desktop clients and perform exchanges of data between them, and later in this article, we'll talk about the issue of the server. Our company has extensive experience in writing classic Web applications; however, in this case, we realized that the traditional approach, which we regularly use for solving such kind of tasks, would be inappropriate.

The traditional approach involves using Java Servlet-containers suitable for working with HTTP-requests, which are certainly very comfortable, but not always highly efficient. The first thought that comes to mind when you need to increase the performance of client-server's interaction is to completely abandon the use of HTTP. In our case, the client communicates with the server to exchange small amounts of data, and our main priorities are maintaining performance and stability. For such tasks, the use of HTTP does not make sense because of its redundancy. For this reason, we decided to use another communication protocol of a lower level—TCP/IP.

At first glance, Java NIO is the best solution from the viewpoint of performance, but this framework requires the implementation of low-level code for working with sockets and data transformation, which entail significant developmental costs. Therefore, we needed a framework that would "hide" the work of Java NIO but with a versatile and convenient API.

Unfortunately, to date, there are few such ready-made frameworks - according to articles on the Internet, discussions on IT forums, and test reports - and the invariable leaders in this area for quite some time have been Apache Mina and Netty. And, if the first framework, we have already had an opportunity to try out, then about Netty, we have only heard positive reviews and intriguingly high benchmarks figures. As it turned out, Netty is better documented than Mina and is accompanied by detailed practical examples of how to implement a particular feature - that is why it was simple enough to master this framework. Overall, the success of these two frameworks was probably justified by the fact that they were both founded by developer, Trustin Lee. Here are his own words about these two frameworks.

Netty is a framework that supports multiple simultaneous connections and processes them asynchronously. The major advantages of this framework are its high performance, open source, and versatility, which means it supports most protocols of the client-server connection “out of the box," ranging from low-level protocols TCP / IP and UDP, up to contemporary WebSocket. The work with this framework gave the developers complete freedom, allowing us to solve transport problems of any complexity. No wonder that, in the end, our choice fell on Netty.

General information about Netty

For our customer's web application, we have developed our own communication protocol, trying to minimize the size of each message exchanged between the client and the server. And the most resource-demanding task was the writing of a class of encoders and decoders for the handlers of messages on both the server and client. Only some time after publishing the web application, we came across Google Protobuf framework, which solves exactly this issue, that is, it encodes and decodes the messages without assistance; you just need to describe the structure of all the messages in a special format and generate message classes for your programming language. Eventually, we decided not to move our application to Protobuf, but we nevertheless noted that in subsequent similar projects, Protobuf would certainly save us a considerable amount of time, especially as Netty supports Protobuf "out of the box".

Speaking about the productivity of solutions based on Netty, it should be mentioned that Netty channel handlers should perform only the logic that is directly connected with encoding-decoding the data. When the handler produces a message, all further actions on it should already be made in a separate flow - any blocking of the handler's flow not only inhibits execution of the handlers of other channels, but could also potentially lead to a loss of control over the channel. This simple rule will help you to get maximum performance results when utilizing Netty.

As a result of using Netty in our application, we have achieved an impressive performance; on the server with a 4-core processor and 16 GB of RAM, Netty easily handles thousands of simultaneous open connections, loading the processor by not more than 50% at its peak. This has not only saved us from the problems with the support of the live application, but has also reduced the costs of the hardware.

So, as Netty is so good, why do not we also apply this framework to our web development? Instead of the traditional web server, we could implement the server based on Netty, which is ideal for fast work with a large number of asynchronous requests to the server, and, as it supports them “out of the box”, we could use both AJAX and WebSocket requests right away, resulting in a more powerful server than the traditional one. So what do you think about this suggestion? Is it something you would consider? We’d be interested to hear your thoughts!

Source: Iteratia.com

Twitter

Facebook

Google+ Page

[gokulvanan]

Java NIO - Part 1

Java NIO in building Non blocking networking apps. This post does not describe the Java NIO packages completely. It just focuses on a few key components.

The main components of Java NIO packages are:

Channels 

Buffers

Selectors

Channels and Buffer will be covered in this Post, Selectors will be in a separate Post.

Channels These are analogus to Streams with the difference that they are bidirectional and you can directly read data from them. Developer can only read data from Channel using a buffer. (More of Buffer will be discussed in Buffer section below) Data is read from channel to buffer and written from buffer to channel. Channel is the interface and there are different implentations for different entities such as Files, Sockets etc.. similar to streams. eg of opening Channels:

RandomAccessFile fromFile = new RandomAccessFile("fromFile.txt", "rw"); FileChannel fromChannel = fromFile.getChannel(); FileChannel ch = new FileInputStream(new File("file.txt")).getChannel(); DatagramChannel dh = DatagramChannel.open(); ServerSocketChannel ch = ServerSocketChannel.open(); SocketChannel sh = SocketChannel.open();

Types of channels:

Selectable - can be used with Selectors. have a method to register the selector. eg.. SockeChannel

NonSelectable - cant be used with Selectors. eg.. FileChannel

Ways to configure channel in Non blocking mode:

ch.configureBlocking(false); // Non blocking

Ways to read data from a channel and write data to a channel

int bytesRead = ch.read(buff); int bytesWriten = ch.write(buff); int bytesWriten = ch.write(buff,offset,limit);// offset and limit of the ByteBuffer array (more like starting and end point of the array of bytes

Transfer data between channels:

You can transfer data between an input channel and an output channel directly via:

public abstract long transferFrom(ReadableByteChannel source, long position, long count) public abstract long transferTo(long position, long count, WritableByteChannel target)

Buffers: Buffer is an abstraction to the block of bytes transfered (read/write) to and from a channel. Buffer is the interface and as its implementations for each data type such ByteBuffer for bytes, CharBuffer for char etc.. ByteBuffer is special implemenation and has few extra utility methods that are not found in other buffers.

ByteBuffer is an abstraction to an array of bytes. Three methods used to control navigation in this array are:

capacity() - total capacity of the ByteBuffer array

limit() - limit of the ByteBuffer - should be equal to capacity from empty read buffer at start.

position() - current position of pointer when data is being read and written.

To create a buffer:

ByteBuffer buff = ByteBuffer.allocate(1024) // creates a buffer of size 1024 ByteBuffer buff = ByteBuffer.wrap("test data".getBytes());

To read from a channel to a buffer:

int bytesRead = ch.read(buff); while(bytesRead != -1){ // eof file case buff.flip(); -- flips limit() to position() at the end of read (could be capacity in full capacity is read) and position to 0 index while(buff.hasRemaining()) // checks if position != limit() { System.out.print(buff.get());// gets the element from current position and moves the position forward by one. } buff.clear();// resets buffer by clearing data and moving position to 0 and limit to capacity //buff.compact(); can be used alternatively if some data is not read from buffer and you dont want to loose unread data i.e. data from position to limit }

To write to a channel from a buffer:

ByteBuffer bf = ByteBuffer.wrap("test".getBytes()); // created and bytes written ByteBuffer bf = ByteBuffer.allocate(1023); bf.put("new data".getBytes()); // there is also putChar putLong etc.. int bytes written = ch.write(bf); //writes to buffer // Note in nonblocking mode to write to buffer // while(bf.hasReamining()) ch.write(bf); // nonblocking call;

Buffer manipulation:

Slice: Slice is a sub-buffer of buffer. However, slice and buffer share the same underlying data array. Slice buffers are excellent for facilitating abstraction. You can write your functions to process an entire buffer, and if you find you want to apply that process to a sub-buffer, you can just take a slice of the main buffer and pass that to your function. This is easier than writing your functions to take additional parameters specifying what portion of the buffer should be acted upon. eg code:

buffer.position( 3 ); buffer.limit( 7 ); ByteBuffer slice = buffer.slice();

Mark and Reset: Used to mark a position in buffer while processing and call reset latter to jump back to the marked position eg:

while(buff.hasRemaining()){

  char ch = (char) buff.get();

  if(ch == 'A') buff.mark();

  else if (ch == 'B') buff.reset();

}

ReadOnlyBuffers: Useful to clone immutable copies of a buffer: eg:

ByteBuffer clone = buff.asReadOnlyBuffer();

Direct/Indirect buffers: Another useful kind of ByteBuffer is the direct buffer. A direct buffer is one whose memory is allocated in a special way to increase I/O speed. Actually, the exact definition of a direct buffer is implementation-dependent. Sun's documentation has this to say about direct buffers: Given a direct byte buffer, the Java virtual machine will make a best effort to perform native I/O operations directly upon it. That is, it will attempt to avoid copying the buffer's content to (or from) an intermediate buffer before (or after) each invocation of one of the underlying operating system's native I/O operations. Note, this also makes using Direct buffers risky as the in case of writing to FileChannel file can be corrupted.

Direct buffers have a number of advantages The avoid an extra copy of data passed between Java and native memory. If they are re-used, only the page used are turning into real memory. This means you can make them much larger than they need to me and they only waste virtual memory. You can access multi-byte primitives in native byte order efficiently. (Basically one machine code instruction) Way to create Direct buffer

ByteBuffer buff = ByteBuffer.allocateDirect(1024);

More info on NIO, Buffer @: http://www.youtube.com/watch?v=rTa68v4Gfrk http://blog.vlad1.com/2011/10/05/looking-at-java-nio-buffer-performance/ http://www.ntu.edu.sg/home/ehchua/programming/java/J5b_IO_advanced.html