When I saw Sencha Touch in action for first time, I thought „Wow, it's a really powerful tool!” Some time later I came here (to Codeworx) and I was really excited about working in project that uses Sencha Touch (Appyourself). The more I was getting familiar with Sencha, the more I was able to see some weaknesses of this framework. For example, Sencha Touch 1 was sometimes having troubles with a correct access to DOM elements and we had to do it manually.
Another thing was when apps became bigger, loading time wasn't satisfying at all. Especially in this case, when user could have many modules (views). All this made us to start looking for solution which would improve the performance.
Fortunately we didn't have to wait long, new Sencha Touch was released. Quick preview and I thought „Looks promising!” Performance boost, nice handling with DOM, etc.
Rebuild whole app structure.
Rewrite components to use Sencha Touch 2 syntax.
Connect output files with backend.
Rebuilding apps structure
When the project was initiated (before I came to Codeworx), guys just started learning coding in Sencha Touch 1, since they never had a chance to get to know this framework. The output code was messy, they didn’t use MVC and all other features. When performance boost issue grew bigger, I started to look for an opportunity to rebuild structure and bring MVC in. I finally got the chance when we decided to upgrade Sencha Touch to newest version.
Sencha Touch 2 fully supports MVC, as a descendant of ExtJS4, so the task was easy. For those of you who just started programming and wonder what’s the meaning behind MVC. It is a well known pattern of building an application. Model-View-Controller basically splits our app interaction between three roles: Model (business logic), View (user interface) and Controller (user input).
Rewriting part
First & most important thing: always keep API documentation opened! Guys from sencha made some really big changes while comparing both versions. Don’t think that migration to the latest Sencha is fast, it takes some time to rewrite code. Also take into account that all the API changes can force you to rebuild old functions, write new ones, listen for different events. It’s pointless to write down here all the small changes. I just named the most significant ones, in my opinion of course.
First big thing (maybe not in syntax) is fact that we don’t need extra panel to hold the list any longer. No more looking for a fix to eliminate stupid visual bug. Selection on screenshot shows what I’m talking about. Last item wasn't reachable.
Why do I mention that? Because finally it works like it should. And it applies not only to this situation. Most visual bugs has been fixed.
Next great feature is the way we define& overwrite elements. It has changed pretty much, and I strongly recommend you using Ext.create() instead of the new. Why? One word: instantiation. Ext.create provides a higher level control for instantiation throughout. During the development the target class will be automatically loaded synchronously the first time it's used, which frees you from having to remember "Ext.require-ing" it before-hand.
Besides, just like Ext.require, Ext.create accepts either a class name or an alias, which makes it extremely convenient to instantiate almost everything in ST2 library by its xtype/type without having to figure out its full class name upfront.
And the last thing that drew my attention is the fact how controllers work now, as well as defining stores and models changed. Instead of using Ext.dispatch on component’s side, now we have selectors (similar to those in CSS) which allow you to select all elements of a given type. It’s a great solution, especially when you have one function used by multiple elements. One reference to those elements solves everything, and we don’t need to worry about forgetting to add this Ext.dispatch method in some places.
Connecting to backend
Since everyone can design backend for their own apps in a different way, I don’t see point to describe it deeply here. I’ll just list out the biggest problems that we have to overcome:
Converting static .js files into .jsp. Builder on our site allows user to dynamically generate app so this has to be done.
After converting to .jsp, we have to resolve caching problems.
Forcing to work old natives (natives built before upgrade).
Rebuilding backend a bit in order for it to work with ST2 and new structure of apps.
Readers interested in details, feel free to leave the questions below.
Summary
The sooner you decide to upgrade your Sencha Touch libraries, the better for you. Why?
Sencha Touch 1 is no longer developed.
Sencha Touch 2 offers very solid MVC architecture. It’s fast, really fast! Speed improvement was clearly visible with a bare eye while we were testing new Sencha (especially with complex, big apps). And last but not least – it offers native packaging (really cool feature!).
Most bugs are fixed, DOM elements are rendered properly.
Sencha Touch 2 offers build in pullRefresh plugin!
Some of us will ask whether it does have any cons?
Of course, it’s time consuming as hell. And, unfortunately, API documentation doesn't help at all (it's still a weak point of Sencha), many things aren't described properly or in the worst case they aren't included in API (then you have to read the source code).
Even so, I encourage you to take the advantage of the new Sencha :)
I hope that this short collection of my thoughts and conclusion was quite helpful to you and your projects. I also highly encourage you to post your own thoughts and conclusions regarding your work with Sencha Touch 2.
I am also very eager to compare your ideas with mines. There is nothing better for developing new ideas than stormy discussion. Remember, we can always learn something new from each other, so please comment and post your opinions.
For some time lately we have been considering how to increase the frequency of new features' updates in one of our projects. It led us to a small investigation, in which we came across the Continuous Deployment. The outcome of our research is very interesting and inspired us to write this article.
What is Continuous Deployment?
In developers’ community, continuous deployment is perceived as very controversial. Developer who faces this topic for the first time thinks:
“that’s encouragement for low quality code writing and uncoordinated development process!”
You cannot be more wrong. Continuous deployment isn’t just an abstract theory. It’s the next step in common releasing (common means released each week or month). Naturally, adopting continuous deployment is not an 1-minute action, it requires investment in reorganization & automation of development process. Fortunately in the end it gives the desired effects and teams can cheaply and safely use innovative features at any stage of the project evolution, mitigating the risk that the solutions won't work, by exposing them to only small group of users.
Eric Ries describes the entire process of transition in a very straightforward way:
Following these five steps will not give you continuous deployment overnight. In its initial stages, most of your root cause analysis will come back to the same problem: "we haven’t invested in preventing that yet." But with patience and hard work, anyone can use these techniques to inexorably drive waste out of their development process.
Continuous integration server.
This is the backbone of continuous deployment. We need a centralized place where all automated tests (unit tests, functional tests, integration tests, everything) can be run and monitored upon every commit. Whatever tool you use, it’s important that it be able to run all the tests your organization writes, in all languages and frameworks.
If you only have a few tests (or even none at all), don’t despair. Simply set up the CI server and agree to one simple rule: we’ll add a new automated test every time we fix a bug. Following that rule will start to immediately get testing where it’s needed most: in the parts of your code that have the most bugs and, therefore, drive the most waste for your developers. Even better, these tests will start to pay immediate dividends by propping up that most-unstable code and freeing up a lot of time that used to be devoted to finding and fixing regressions (aka "firefighting").
If you already have a lot of tests, make sure that the total time the CI server spends on a full run is a small amount of time, 10-30 minutes at the maximum. If that’s not possible, simply partition the tests across multiple machines until you get the time down to something reasonable.
Source control commit check.
The next piece of infrastructure we need is a source control server with a commit-check script. I’ve seen this implemented with CVS, subversion or Perforce and have no reason to believe it isn’t possible in any source control system. The most important thing is that you have the opportunity to run custom code at the moment a new commit is submitted but before it is accepted by the server. Your script should have the power to reject a change and report a message back to the person attempting to check in. This is a very handy place to enforce coding standards, especially those of the mechanical variety.
But its role in continuous deployment is much more important. This is the place you can control what I like to call “the production line” to borrow a metaphor from manufacturing. When something is going wrong with our systems at any place along the line, this script should halt new commits. So if the CI server runs a build and even one test breaks, the commit script should prohibit new code from being added to the repository. In subsequent steps, we'll add additional rules that also "stop the line," and therefore halt new commits.
This sets up the first important feedback loop that you need for continuous deployment. Our goal as a team is to work as fast as we can reliably produce high-quality code - and no faster. Going any “faster” is actually just creating delayed waste that will slow us down later.
Simple deployment script.
Start simple. It’s not even important that you have an automated process, although as you practice you will get more automated over time. Rather, it’s important that you do every deployment the same way and have a clear and published process for how to do it that you can evolve over time.
For most websites, I recommend starting with a simple script that just rsync’s code to a version-specific directory on each target machine. If you are facile with unix symlinks, you can pretty easily set this up so that advancing to a new version (and, hence, rolling back) is as easy as switching a single symlink on each server. But even if that’s not appropriate for your setup, have a single script that does a deployment directly from source control.
When you want to push new code to production, require that everyone use this one mechanism. Keep it manual, but simple, so that everyone knows how to use it. And, most importantly, have it obey the same “production line” halting rules as the commit script. That is, make it impossible to do a deployment for a given revision if the CI server hasn’t yet run and had all tests pass for that revision.
Real-time alerting.
No matter how good your deployment process, bugs can still get through. The most annoying variety are bugs that don’t manifest until hours or days after the code that caused them is deployed. To catch those nasty bugs, you need a monitoring platform that can let you know when things have gone awry, and get a human being involved in debugging them.
To start, I recommend a system like the open source nagios. Out of the box, it can monitor basic system stats like load average and disk utilization. For continuous deployment purposes, we want to be able to have it monitor business metrics like simultaneous users or revenue per unit time. At the beginning, simply pick one or two of these metrics to use. Anything is fine to start, and it’s important not to choose too many. The goal should be to wire the nagios alerts up to a pager, cell phone, or high-priority email list that will wake someone up in the middle of the night if one of these metrics goes out of bounds. If the pager goes off too often, it won’t get the attention it deserves, so start simple.
Follow this simple rule: every time the pager goes off, halt the production line (which will prevent checkings and deployments). Fix the urgent problem, and don’t resume the production line until you’ve had a chance to schedule a five whys meeting for root cause analysis, which we’ll discuss next.
Root cause analysis (five whys).
So far, we’ve talked about making modest investments in tools and infrastructure and adding a couple of simple rules to our development process. Most teams should be able to do everything we’ve talked about in a week or two, at the most, since most of the work is installing and configuring off-the-shelf software. Five whys is not something you can get in a box. It’s a powerful practice that is the motive force that will drive major improvements in development process incrementally, one step at a time.
The idea is to always get to the root cause of any unexpected failure in the system. A test failing, a nagios alert firing, or a customer seeing a new bug are all sufficient triggers for root cause analysis. That’s why we always shut down the production line for problems of this kind - it signals the need for root cause analysis and also creates the time and space for it to happen (since it deliberately slows the whole team down).
Five whys gets its name from the process of asking “why” recursively to uncover the true source of a given problem. The way five whys works to enable continuous deployment is when you add this rule: every time you do a root cause analysis, make a proportional investment in prevention at each of the five levels you uncover. Proportional means that the solution shouldn’t be more expensive than the problem you’re analyzing; a minor inconvenience for only a few customers should merit a much smaller investment than a multi-hour outage.
But no matter how small the problem, always make some investments, and always make them at each level. Since our focus in this post is deployment that means always asking the question “why was this problem not caught earlier in our deployment pipeline?” So if a customer experienced a bug, why didn’t nagios alert us? Why didn’t our deployment process catch it? Why didn’t our continuous integration server catch it? For each question, make a small improvement.
What is interesting, there is another term called ‘Continuous Delivery’ that sounds similar to ‘Continuous Deployment’, but has slightly different meaning. It is related to speeding up software development, but is also more focused on having prepared package to deploy (after every check-in to source control), than on the final action of deploying it to the production.
We can look on Continuous Delivery as a subset of Continuous Deployment. To introduce Continuous Delivery we need to make similar changes as in the Continuous Deployment, but at the end we decide when to deploy delivered software to the production instead of automatic publication few times per day. If you begin your journey with Continuous Deployment I would recommend you to start from Continuous Delivery.
Once you have mastered the first step, the next one will be piece of cake!
For whom Continuous Deployment/Delivery is intended?
Continuous Deployment is dedicated for agile teams that want to deliver high quality software (which is the first of the Twelve Principles standing behind the Agile Manifesto) to the production much more frequently, provide a richer and faster feedback cycle from users’ feedback.
It is also intended for teams that want to reduce stress connected with releases & ‘checking twice’ behaviour before deployment (because small changes reduce the risk during deployment).
What tools are out there?
What is needed to start Continuous Deployment/Delivery?
As I mentioned above - there are no special tools to implement the Continuous Deployment/Delivery approach. The main change is more HOW we use this existing tools.
It means that we can still use well known tools like:
Subversion or Github for source code versioning
Liquibase for database changes management liquibase.org
Puppet puppetlabs.com or Chef opscode.com for infrastructure management
Nagios nagios.org for infrastructure monitoring
Maven for building automation and dependency management
Jenkins or Hudson for Continuous Integration
JUnit, DBUnit, Mockito or Selenium for testing
Fitness fitnesse.org, Concordion concordion.org , Easyb easyb.org or JBehave jbehave.org for acceptance tests
JMeter for load testing
IMO Eric's advices, quoted above, very accurately navigate beginners into required package of changes in order to adopt Continues Deployment/Delivery – and as he wrote in one of his articles:
… learn from our mistakes.
If you read these pieces, you'll get a visceral sense for just how painful it was.
But it worked. We made it. So can you.
Are we going to use it in AppYourself?
Yeah, naturally we should start with Continuous Delivery as AppYourself is a startup & we expect quick user’s feedback and results. Of course first question is what is needed, Continuous Delivery or Continuous Deployment? Beginning of its usage is not an one-time event, so we will take some time to design and introduce new processes in order to deliver software that fits our requirements, project and the whole organization continuously. When we will finish working with Continuous Delivery approach then we should try to introduce Continuous Deployment to introduce software to the production more frequently.
Now that we are used to the Continuous Integration approach with Jenkins, which provides us with ready-to-use test environment and ready-to-deploy application package.
What needs to be improved in order to acquire Continuous Delivery in AppYourself is:
Shortening of the delivery time of ready-to-deploy software.
Currently we have such software every 1-2 weeks. The main goal is to have prepared package after developing or solving every single issue (few minutes to max. few hours). And this requires a different testing strategy, source control management and development model.
If we need next step in delivering our software - Continuous Deployment - many technical problems will have to be solved, f.eg. hot deployments on tomcat or changing database structure with each release.
It requires fully automated tests and high test coverage on different levels: like unit tests, integration tests, functional/acceptance tests, performance/load tests, penetration tests for security and explanatory tests. This will enable us to automatically prove that software still works properly and is deployable.
Introduce artefacts repository (like Nexus http://www.sonatype.org/nexus/) in order to store and manage many more versions of our software and their modules which are being used for delivery and deployment to the production.
Detach features deployments from the sprints or iterations.
Introduce Feature Toggle pattern martinfowler.com for detachment of unfinished features from deployment requirements.
Agile approach for releasing software seems to be as natural as agile approach for developing it. That is why we should improve our whole development lifecycle with testing, versioning and deploying at the head.
From now on, we can assure you that this topic will be in constant motion in terms of AppYourself. We plan to devote ourselves fully to these ideas and give you first-hand reports from our progress. Stay tuned for more!
by K.Kornijenko, M.Muras, M.Kaiser
edited P.Chaberski
Resources
We strongly recommend you this great book which describes Continuous Delivery approach in details Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation
And for those of you who will be even more curious in this area we have this set of lectures concerning Continuous Delivery/Deployment. Of course we are always eager to answer any of your question so don’t be shy and write to us as well in case of doubts and questions.
Five Whys
Continuous Deployment in 5 easy steps
Analysis for Continuous Delivery: Five Core Practices
Lately I had to execute a Perl script from java project and I want to share my experience with you. It might be useful in your future projects as it was for me in this particular work. I based my job on usage of WWW::Mechanize library for automating interaction with websites. There were some issues to run Perl from java project, but I managed to deal with them in my own way. And that way of mine I want to share with today in this small article.
Problems:
Where to put your perl script file in javaEE Spring project.
Define absolute path to your script.
Define correct bash command.
Turn on the logs to log the messages at the debug level.
Define a process.
How to use Runtime to execute your script.
How to log message from output and error stream of the process.
First of all you have to decide where to store your perl script files. I chose the location where I keep my resources (src/main/resources). This location is added to classpath by default. The output files will be in target/classes folder.
Here is entry from eclipse project .classpath file:
At this moment we have ready-to-use command as a string, so we can trigger a script using Runtime.exec().
But wait! What if the script is not working correctly or even it does not work at all? What if we run the script and nothing happens?
To obtain the information about what the script is doing, we can define a process:
Process process = Runtime.getRuntime().exec(bashCommand);
“ Runtime.exec method create a native process and return an instance of a subclass of Process that can be used to control the process and obtain information about it. The class Process provides methods for performing input from the process, performing output to the process, waiting for the process to complete, checking the exit status of the process, and destroying (killing) the process.” [1]
At this moment we can get information about the process:
Input stream of the subprocess obtains data from the standard output stream of the process
process.getInputStream()
Error stream of the subprocess obtains data from the error output stream of the process
Of course first thing that most of us will think of is the backend solutions. Trying to improve java code, optimize database queries, setup scalable solutions etc. Of course that is one way of getting things done in this department. However my simple-minded friend once said to me that such solutions are like stirring the soup by turning the pot around the spoon, that is being hold still, so basically not very efficient and rather time-consuming.
Unfortunately we all know what time-consuming will mean in the end: missing deadlines and necessity of re-doing the cost estimation that will collapse due to the mentioned lag. Of course these are not the only flaws that may occur. We cannot forget about the customer dissatisfaction concerning the delay in delivery, this can ruin your relationship with the buyers.
What can you do?
First thing that comes to mind is checking the backend. That is normal and nothing bad. There's the complex business logic, the database as a typical bottle-neck, etc. People tend to make their life more and more complicated, so they seek solution of the problems in the most complicated part of their creations and thus frontend seems to be out of focus for them. But does it make sense? It was proved many years ago that the simplest solutions are in many cases the best one, like in the case of Gordian knot which was solved by a simple cut. Why not use this way with this problem too?
Start Analyzing.
Use tools such as YSlow which tests your web pages "based on a set of rules for high performance web pages". YSlow is available as plugin for Firefox, Chrome, Safari and Opera making it really easy to use. YSlow provides a quick overview where the time got lost and what you can do to improve the performance. With this data you can easily find the very root of your problems. This way it is easier to act on the real source of the problem and not just “stir the pot around the spoon”.
Picture 1. Grade before optimization in YSlow.
Picture 2. Grade after optimization in YSlow.
Picture 3. Statistics before optimization.
Picture 4. Statistics after optimization.
The best thing is that many improvements require just a small implementation effort, in many cases it can be done via Apache Webserver configuration!
For example:
Gzip Components - use and configure mod_deflate in your Apache Server
Add an Expires or a Cache - Control Header - use mod_expires
Minify your JavaScript - use JSMin or use already minified versions of 3rd-party-libraries, e.g. jquery-1.7.1.min.js rather than jquery-1.7.1.js
We recommend you some further readings concerning this topic and encourage you to ask us questions as well, we are always open for an interesting discussion:
