2 Bytes @twobytes - Tumblr Blog

PebbleNotes

Managing and keeping track of daily events and tasks are not easy. It requires commitment. It is easy to forget what you did yesterday, so how can someone keep track of what happened last year or five years ago. I think it is important to keep track of daily events in a form of journal, private blogs or simple notes if someone wants a little sanity in today's busy life. Journal taking has its own rewards. You see day to day progress of your life. It gives you a chance to reflect back and see how life passed by.

PebbleNotes allows anyone to keep track of their daily notes, events, TODOs and progress reports. PebbleNotes represents small notes like stone pebbles, and it is not any way related to Pebble watches.

PebbleNotes is very simple to use. Signing up takes less than 20 seconds. Once login, the user sees a welcome note explaining how PebbleNotes can be used to manage daily notes or tasks. PebbleNotes itself uses notes to communicate with its users. The Frequently Asked Questions (FAQs) are also in form of notes. It is refreshing to see unified user interface where everything falls in its place without any clutter.

One of the powerful features of PebbleNotes is support of tags. A user can tag his/her notes and organize them.

Another neat feature of PebbleNotes is OneView, which allows the user to view all notes on a single page.

In today's world where everything is exposed and privacy is an issue, PebbleNotes offers a home where people can maintain their own content.

#PebbleNotes

Java Set Map List - Differences

Okay, this is quite confusing but important to remember!

Set does not allow duplicate values

Map stores key-value pairs. It allows duplicate values but keys are always unique

List allows duplicate values

Set allows one null value

Hashtable does not allow null key or value

HashMap allows one null key and allows null values

staff

A new row being installed at our East Coast datacenter

2,560 newer faster cores for things like image processing and animating all those GIFs. 3 petabytes of storage and 40 terabytes of memory to remember how beautiful you look.

Java Midnight Date

Recently I encountered a strange and silly problem with Java Calendar class. I spent more than 5 minutes to fix the issue, so I figured let me write about it.

I wanted to create today’s date with time at midnight. I went ahead and used java.util.Calendar class to create today’s date and set hour, minute and seconds as all zeros. At later point in time, when I compared newly created today’s date with another today’s date (midnight also), it did not match. When I printed both dates, it showed identical strings (e.g. Tue Apr 30 00:00:00 EDT 2013). I was puzzled.

It turned out I forgot to set milliseconds to zero when creating today’s date. If you do not set milliseconds to zero then a timer running in the background will keep updating milliseconds behind the scene. This is obvious when you print time in milliseconds.

Interestingly the following calendar.set method does not take parameter to set milliseconds. I guess they do not want too many parameters to be passing in method calls :)

public final void set(int year, int month, int date, int hourOfDay, int minute, int second)

You have to call another set method to set milliseconds.

public void set(int field, int value)

https://gist.github.com/shahjigar/0123e76ef53251c6f4f2

ParallelGC and ParallelGCThreads

When it comes to tuning garbage collection in Java, there are too many options available. There is no single default configuration that works best for your application, so you have to try different options.

When selecting ParallelGC for minor collection, it is important to set right number of garbage collector threads for ParallelGC. If you turn on ParallelGC for your application and do not set –XX:ParallelGCThreads, then the JVM will set number of parallel GC threads based on number of processors (cores) available on a machine where your application is running. The JVM will create one parallel GC thread per processor (core) for young generation.

An example helps. Imagine you have a machine with 16 processors (32 cores). For simplicity sake, you have an application running in 8 clones (JVMs) for load balancing reason. You have decided to turn on ParallelGC for all 8 clones (JVMs) for higher throughput. It means by default each JVM will create 32 garbage collector threads (one thread per core). You have 8 clones (JVMs), so you will end up with 32 x 8 = 256 garbage collector threads in your system. This is not good. You obviously do not need 32 garbage collector threads per JVM – it is overdone and a waste of system resources. In this case, you must set –XX:ParallelGCThreads property and set low number of threads (e.g. 4 or 6 threads per JVM rather than keeping it default).

Remember, more garbage collector threads can take system resources and more importantly it can increase fragmentation in old generation. There is a space reserved in old generation for each young generation's garbage collector thread for promotions. The division in space can cause fragmentation and young generation garbage collector threads may not be able to work efficiently when copying large amount of survived data from young generation to old generation. In order to fix fragmentation, the old generation will have to go through compaction process, which can increase major collection time.

If you want to make sure number of parallel GC threads is running as expected, then take a thread dump of your application and use an analysis tool to check whether you have correct number of parallel GC threads running.

So remember to set –XX:ParallelGCThreads=N whenever you use -XX:+UseParallelGC in a large multi-processor system.

#parallelgc #gc

Java Exceptions - How did that happen?

When you are working on large scale Java application, you will encounter one or many exceptions like below. This blog summarizes what I have experienced so far and may not cover all scenarios.

java.lang.NoSuchMethodException

The exception happens when you are trying to invoke a method that is not present in target class. The exception is prominent if your application is using reflection API. If reflection was not used, then it could be that partial build was deployed. You had compiled your Java class with a bunch of dependent classes but at some point in time dependent classes were modified (methods removed or changed) and deployed independently.

java.lang.reflect.InvocationTargetException

It happens when you are directly or indirectly using reflection in your application. This exception is more visible if you are using Axis or any Web Services where bean marshalling and unmarshalling happen via reflection API. The exception can be tricky to resolve because the original exception is wrapped in InvocationTargetException. You probably want to get handle of target exception by calling getTargetException() on InvocationTargetException object and analyze it.

java.lang.IllegalAccessException

Again it happens when you are using reflection in your application at runtime. Your code is trying to access method or member variable that are out of reach (e.g. private, protected) and should not be allowed to access. I think this class should be part of reflect package along with InvocationTargetException.

java.lang.ClassNotFoundException

This one can be really easy or extremely tricky to figure out. The classloader cannot find the class it is looking for. You need to first check the obvious places like your classpath and jar files where your class supposed to reside. Check your reflection code as well.

The trickiness comes if you are in J2EE environment with multiple classloaders. For example you have parent-first classloader setting in your appserver configuration. Your EAR file has class A which is referenced by class B in parent classloader, then class A cannot be found and ClassNotFoundException will be thrown. So in this case, even if you have class A present in your classpath, it cannot be found by classloader.

java.lang.NoClassDefFoundError

Technically this is an error not an exception. At runtime, the class you are trying to load is not found or class definition is not correct (similar to linkage error in C/C++). It could be you have given invalid class name or the class itself is not there. You should check your classpath settings. It could also be triggered by ClassNotFoundException meaning if Test.java was using Test2.java and once compiled if you remove Test2.class at runtime, then Test.class would complain the following

Exception in thread "main" java.lang.NoClassDefFoundError: Test2 at Test.main(Test.java:7) Caused by: java.lang.ClassNotFoundException: Test2 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)

The difference between ClassNotFoundException and NoClassDefFoundError is very confusing because there is no clear separation between the two. If you read various posts online, you will find ClassNotFoundEception and NoClassDefFoundError clear in its own domain but they can be quite confusing when you think about them on your own. If someone asks you the difference between the two in an interview, then consider yourself in trouble.

#java exception

Enterprise Document Management

Can you imagine how many paper documents a medium or large size organization has to keep? The answer is a lot - depending on the sector. Obviously the financial, insurance, and public (government) sectors are overloaded with documents. Moving from paper documents to digital form has great advantages, but the cost can add up even in digital format if not done right.

Basically the documents have their own lifecycle within an organization. How documents are transformed from paper to digital format and live in the storage happily ever after (ok, may be until they are purged) and transformed depending on the clients’ need - this is called Enterprise Document Management (Process). All the systems or software that are part of this process is called Enterprise Document Management Systems (EDMS for short).

The life cycle of a document has the following states: captured, indexed, archived, stored, transformed, retrieved, and printed.

Figure 1: Life Cycle of Documents

Most companies use high-speed scanners to capture documents. These scanners are not your average scanners at home. They can scan pretty huge papers that are more than 6 feet wide. The capturing part is usually the hardest part because some manual intervention is almost always necessary. If you are dealing with different kinds of documents, then first someone has to identify which part of the document will become the search key(s) or meta-data. For example, when a paper invoice is scanned, usually invoice number is used as an index key. This invoice number may not be obvious to scanners, so manual intervention is necessary. We have to tell the scanner that once the item is scanned, what will be the index key(s) or search key(s) for that document. So later, when someone wants to retrieve the same document, he or she has to use the exact key(s) to retrieve the document in digital format.

Once the documents are captured, they reside in storage whether you want to call it DASD, hard disk, or a tape. Usually, storage software is used like IBM OnDemand or FileNet or Documentum to store large number of documents. These software will manage how documents are organized and stored. They are pretty expensive software but they can also manage millions of documents. Imagine how many credit card statements a bank has to store in their storage. If the bank has 50 million active customers, then 50 million x 12 months of statements. Plus, sometimes customers would like to go back to get their previous years’ statements. This can happen if the customer is audited for tax reason or what not. By law, if I am not mistaken, most banks keep up to 7 years of credit card or bank statements in the storage. So the storage cost adds up.

Now imagine how fast a system should perform when a customer requests to view a statement online. This is a crucial and heart pounding moment for EDMS. Going through millions of statements and finding the right one is not an easy task. You should not be anxious if it takes more than 5 seconds to download your credit card statement online. I am sure you will argue that if Google can search billions of web pages then why can’t my bank do it? It is a valid point but most banks do not have a huge infrastructure like Google has.

Usually, a middle tier is built between front-end application and backend system (i.e. OnDemand, FileNet), which can search and retrieve documents from backend systems. This middle-tier application can be in Java, .Net, or in any language where integration toolkits for backend systems are available.

In case of OnDemand or FileNet, such APIs are available in Java/C++, and the programmers can integrate their middle-tier applications to retrieve documents. Once the documents are retrieved, they are usually transformed into different formats based on customers needs. For example, most of the time your bank statements are stored in AFP (Advanced Function Printing) format in storage. The AFP format is used to print documents by high speed printers. Once printed they are usually delivered by mail. If the customer wants to view the documents online, they must be transformed from AFP to PDF or similar web view-able format. Xenos Document Transformation is an example of an engine that can transform documents from AFP to PDF or AFP to TIFF. This way you keep documents in one format (e.g. AFP) in archive and have transformation engine in EDMS layer that can transform documents based on customers needs.

Hopefully next time when you see your bank statements online, you will know what is happening behind the scene.

#tech #edms #document management

Trending Blogs

Recently Viewed Blogs

2 Bytes