Chintana Wilamuna's blog

[chintanawilamuna]

SAML2 SSO for Ruby on Rails apps with Identity Server

Identity Server document has samples on how to configure SAML2 SSO for a Java webapp. SAML2 allows decoupling service providers from identity providers. Only requirement is ability to create/process SAML2 messages which is XML. In an identity management scenario, service provider (referred to as SP) is typically a web application. Identity provider (or IdP) is any system that provide user repository, authentication and user profile details to the service provider application.

TLDR;

If you want to skip rest of the post and go right then follow these steps,

Download and start Identity Server

Clone the modified rails project from here - https://github.com/chintana/ruby-saml-example

Create a cert keypair and update the SP cert settings in rails app

Upload public cert to Identity Server

Add an SP entry for the rails app in Identity Server (Issuer is already mentioned in app/models/account.rb

Login to Rails app - use admin/admin default credentials. You can add more users and use their accounts to login as well

Changes to the rails app

Rest of the post cover details and changes I had to do to get single sign in as well as single sign out working.

I’m using the sample skeleton rails project that’s integrated with ruby-saml. We need to configure details on connecting to Identity Server. SAML settings are in app/models/account.rb.

First up we need to change IdP settings to match what we have in Identity Server.

settings.issuer = "railsapp" settings.idp_entity_id = "localhost" settings.idp_sso_target_url = "https://localhost:9443/samlsso" settings.idp_slo_target_url = "https://localhost:9443/samlsso" settings.idp_cert = "-----BEGIN CERTIFICATE----- MIICNTCCAZ6gAwIBAg ... -----END CERTIFICATE-----"

Certificate here is default certificate that comes with Identity Server 5.1.0. I've truncated it for brevity. Then we need the following 2 entries for decrypting the encrypting SAML assertions. Certs are truncated.

settings.certificate = "-----BEGIN CERTIFICATE----- MIICfjCCAeegAwIBAgIEFFIb3D ... -----END CERTIFICATE-----" settings.private_key = "-----BEGIN PRIVATE KEY----- MIICdgIBADANBgkqhkiG9w0BAQEFAA ... -----END PRIVATE KEY-----"

When we're sending logout requests we need to include the session index with the request. So that Identity Server will logout the correct user. Rails sample doesn't send the session index by default so you'll see an exception when you do single logout saying session index cannot be null. Changes can be found here.

Then we need to make saml/logout route accessible through HTTP POST. Need to add an additional route,

post :logout

Service provider configuration in Identity Server

At the Identity Server we need to register the rails app as a service provider.

Then click on Inbound Authentication Configuration, click SAML2 Web SSO Configuration.

In the above configuration http://localhost:3000/saml/acs is the assertion consumer URL. Certificate is the certs created earlier. Also we need to configure single logout URL - http://localhost:3000/saml/logout

Tracing SAML messages

I’m using SAML Chrome extension to monitor SAML messages. First SAML call is for authentication. This is the message sent from rails app to Identity Server.

https://gist.github.com/chintana/4c1f8a090edf38a6d9ce9af7ce59a1b7

Second message is SAML response message sent from Identity Server to rails application. As you can see in the below screenshot SAML response is encrypted.

https://gist.github.com/chintana/1981b433b1e3c17983b20b4638ecc23c

Let’s do single logout from rails app. Here’s the logout request with session index.

https://gist.github.com/chintana/e5dfc587039b77ee1a2ea51b03499ff7

Then the logout response we get from Identity Server.

https://gist.github.com/chintana/0ea75ba3c326970ba85d36e66dfe6e81

With a library that supports processing SAML requests almost any web app can be integrated into Identity Server using SAML2.

[chintanawilamuna]

WSO2 API Manager - Switching to WSClient for key validation - Why/What/How?

WSO2 API Manager has 3 main components. API Publisher/Subscriber UI apps, API gateway and API key manager.

API Component Overview - What is key manager?

Publisher/Subscriber UI - Used for publishing and consuming APIs. These 2 are portal type webapps that give full control over creation/subscription through a nice UI. Bonus point - There’s a REST API too for completely automating everything. See Publisher API and Store API.

API Gateway - Expose a given API definition through the gateway. For API consumers, they see the API Gateway endpoint for all APIs from UI. Runtime API calls hit gateway directly.

API Key Manager - Handle OAuth key validation

For all API calls at runtime, API gateway calls key manager to do OAuth key validation. For performance you can enable caching at the gateway level as well as the key manager level. Caching at the gateway prevents HTTPS calls to key manager until cache expiry for a cache token. Caching at key manager prevent DB calls for cached tokens.

Why change to WSClient?

Since API gateway and key manager are 2 separate components (truly distributed) they talk to each other via API calls. Out of the box this API call happens via Thrift. When API Manager starts up you’ll see there’s a Thrift server that listens to requests. See below,

[2016-02-168 12:15:52,219]  INFO - APIKeyMgtServiceComponent Setting default carbon host for thrift key management service: 192.168.50.1 [2016-02-16 12:15:52,289]  INFO - APIKeyMgtServiceComponent Started thrift key mgt service at port:10397

This is fine when the server is running on a single machine. Also translate to you’re testing/playing around. On a production environment with high availability, it’s recommended to switch to WSClient for key validation. In an HA setup you’ll have at least 2 nodes for each component. Why do you want to switch?

Load balancing is difficult to do with Thrift. Need to TCP passthrough to begin with

WSClient (through web service calls) performance is in par with Thrift API calls

How to change?

In <API Manager>/repository/conf/api-manager.xml look for <APIKeyValidator>. Find following two entries. Here are the default values for those,

<KeyValidatorClientType>ThriftClient</KeyValidatorClientType> <EnableThriftServer>true</EnableThriftServer>

Change the above values to WSClient and false. Entries should look like following after the change

<KeyValidatorClientType>WSClient</KeyValidatorClientType> <EnableThriftServer>false</EnableThriftServer>

In api-manager.xml, under <APIKeyValidator> element, change ServerURL to point to key manager URL. Usually this URL point to the load balancer that’s responsible for distributing load among key manager nodes

[chintanawilamuna]

View from Kerry Park, NYE 2015.

[chintanawilamuna]

Using a local CA for certificate management in a multi tenant PaaS

On a multi tenant middleware PaaS a shared JVM can host multiple tenants. A tenant can do pretty much anything (within the security constraints of course) inside his tenant. While a PaaS can be used in different scenarios and use cases this discussion concentrate on using a PaaS inside an enterprise. Although the ideal scenario is every application is developed and deployed on top of the common platform reality is different. There are applications that’s very expensive to rewrite. Integration first and refactoring later is a much more practical approach.

Simple solution for this certificate management problem is to have a local CA. You can then sign external systems having self signed certificates with your root certificate and only have the root certificate on server’s trust store. This is an example diagram that shows the scenario we’ll be discussing.

On a PaaS there are many components that can make an outgoing HTTPS calls. Application Servers, API Manager, ESB and so on. For external HTTPS calls to external systems, public certificate of the external system should be installed in trust store of whatever the JVM running the service. If a web app is making an external HTTPS call, public certificate of the external system should exist in trust store of the Application Server. That’s regardless of what tenant that application is deployed into. As of this writing, Application Server (version 5.2.1 and before) use a common trust store for that running JVM instance. Not all systems inside an enterprise use a valid CA signed certificate. Specially if it’s a system designed for internal consumption.

By having a self service PaaS you allow different groups inside the organization to create/deploy applications without much supervision. That’s the goal anyway because you need the teams to move at their pace without getting in their way to do devops tasks.

When tenants make external HTTPS calls to systems with self signed certs, you need to import those certs to Application Server’s trust store. This creates an administrative problem if you have a self service PaaS running. Every time you update a certificate or try to integrate with a system with self signed certs you have to rely on an administrator to install certificates into server’s trust store.

In my case, I generated a custom keystore by going through this article and configured it in AppServer called MyCompany.jks. I’m using this instance as my external system with a custom self signed cert. I’m following this article to have a local CA cert and have that installed into the trust store. Here’s what the keystore configuration look like,

As in the article I created a root key with a password and self signed that cert,

$ openssl req -x509 -new -nodes -key rootCA.key -days 1024 -out rootCA.pem

$ openssl genrsa -des3 -out rootCA.key 2048

Now we have a PEM file, we need to import this to the keystore of 3rd party system because we’re using Application Server for testing and once we import the signed cert it should be able to validate that. Otherwise you'll get an error like keytool error: java.lang.Exception: Failed to establish chain from reply. Before importing we need to convert it to DER format

$ openssl x509 -outform der -in rootCA.pem -out rootCA.der

Import this cert to the keystore

$ keytool -import -alias rootca -keystore MyCompany.jks -file rootCA.der

You should also import this cert to the client-truststore where you're hosting the application that make the external service call

$ keytool -import -alias rootca -keystore client-truststore.jks -file rootCA.der

Now we need to create a certificate signing request. You can find details of how to generate a certificate signing request and then installing the signed cert to the keystore. Use the following command to generate a CSR

$ keytool -certreq -alias mycompany -keyalg RSA -keystore MyCompany.jks

This will generate an output like below. I'm going to save below to a file named mycompany.csr

-----BEGIN NEW CERTIFICATE REQUEST----- MIIC4jCCAcoCAQAwbTELMAkGA1UEBhMCVVMxEzA… ... -----END NEW CERTIFICATE REQUEST-----

Let's sign this using our root cert

$ openssl x509 -req -in mycompany.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out mycompany.crt -days 500

We have our CA signed certificate, we need to import this to the keystore

$ keytool -importcert -alias mycompany -keystore MyCompany.jks -file mycompany.crt

After keystore updates it should look like the following diagram,

Now mycompany.crt is signed by rootCA.der, we can have one cert in the trust store and talk to external systems that’s signed by the same certificate. This avoid having to import multiple keys into the trust store every time a tenant want to integrate into an external system with a self signed cert.

[chintanawilamuna]

WSO2 is nowhere near the goal I set out to do for us - take over the world (of middleware!). But 10 years later, we're now on a solid foundation to build WSO2 into a much stronger position in the next 10 years

@Sanjiva

[chintanawilamuna]

Walking in SFO during WSO2Con 2013

[chintanawilamuna]

Scalable deployments - Choosing middleware components

Different user groups have different performance expectations from a system they're involved with. Given a platform, be it a middleware/application development or integration, there are 3 sets of stakeholders we can consider,

Developers always want the platform/tools they're working with to be robust. Some developer expectations include,

A robust platform applications can be developed and deployed with minimum effort and time.

Use of different 3rd party libraries to ease development as much as possible. When library usage increase, available resources for shared tenants goes down. No amount of autoscaling can solve uncontrolled library usage

Fast service calls regardless of business logic complexities in services. Sort of ignoring the network boundaries/external DB and system calls. Another aspect is when more stricter security measures are utilized, more time spent for processing security related headers, thus reducing response times.

Fast application loading times when the applications grow in size and complexity. Without designing the application to keep a low footprint, or process large data chunks efficiently, underlying platform cannot solve this. When size and complexity of applications grow, autoscaling times of instances will also increase.

Operations or deveops folks have a different set of performance expectations from a platform.

They want instant instance spawning and start up times.

Fast log collection. This can be collecting platform level logs to identify performance problems as well as collecting application level logs for  developers.

No sane devops folks want to manually deploy a complex system. The entire deployment should be automated.

Seamless autoscaling of services from the deployment itself without having to wait until one server gets bogged down and do some manual patching.

Available resources should be equally distributed among active nodes for efficient utilization of shared resources.

Customers are probably the most difficult lot to please. Quoting from The Guardian article based on Mozilla blog of metrics,

32% of consumers will start abandoning slow sites between one and five seconds

So applications exposed to customers should be fast and have short response times.

Users expect application developers to write fast responsive applications. App developers look at devops/operations to provide a scalable platform for their apps. Devops in turn rely on architects to provide a robust deployment architecture that scales based on application requirements.

Expectations on architects are,

Identify and use correct reference architecture/s

Use correct deployment architecture based on application requirements as well as non functional requirements

Deployment architecture should be aligned with business expectations of the system

Should take SLAs into account

Identify correct SOA/middleware components for the business use cases involved

Getting the right architecture

Identifying the right SOA components is an important step. In a typical middleware stack there are multiple components to choose from. Usually each component have very different performance characteristics. There are many ways to implement the same scenario.

Let's see a simple example. Scenario - withdrawing money from a bank. This can be implemented is several different ways.

Example - Method 1

Here this can be implemented as a web service or web application which talks to a DB and host it in an application container

Example - Method 2

This example expose DB operations through Data Services Server, use an ESB for doing mediation and then expose an API for withdrawing money.

Example - Method 3

In this example, there's a Business Process Server orchestrating the process. There are Business Activity Monitor for monitoring business transactions and Governance Registry as a repository.

A given scenario can be implemented in many ways and also the complexity and use of components will vary. Based on the scenario in hand, right components should be selected.

[chintanawilamuna]

Reliable messaging pattern

Reliable messaging involve sending a message successfully from one system to another over unreliable protocols. Although TCP/IP gives you reliability at a lower level, reliable messaging provide delivery guarantees at a higher level. If the recipient is unavailable, messages will be retransmitted over a defined period until it's successfully delivered.

WS-ReliableMessaging

Traditional SOA method of handling reliable messaging is through a framework/library that implements WS-ReliableMessaging specification. The pattern is illustrated here. A framework like Apache Sandesha provide reliable delivery guarantees according to the specification. From the reliable message specification (PDF) message exchange sequence is like this,

At each step there will be an XML message going back an forth the wire. This create a lot of additional overhead and as a result performance suffer.

Alternative approach using JMS

Looking at the communication overhead and complexity involved with creating/maintaining ReliableMessaging capable clients, an alternative approach using JMS is very popular. Simplicity of JMS and easy maintainability are key factors for JMS's success as a defacto solution used for reliable message delivery. You put messages to a queue and process messages from a queue.

Messaging with WSO2 platform

WSO2 ESB have a concept of messages stores and message processors. Message stores does what you expect. They store messages. Default implementation of message store is an in memory implementation. Also you have the option of pointing the message store to a queue in an external broker. Also there are standard extension points to extend the functionality and write a custom message store implementation.

Message processors are responsible for reading messages from a message store and sending it to another system. You can configure parameters at the message processor to have a flexible interval to poll for new messages, retry interval, delivery attempts and so on.

Advantages of using WSO2 for reliable messaging

ESB follows a configuration driven approach. Easy to configure, don't need to write code for large set of integration patterns

Protocol conversion comes naturally and don't have to do any extra work - Accepting an HTTP request and sending it to a JMS queue require you to specify the JMS endpoint only

Can take advantage of a large set of enterprise integration patterns

Simple config and deployment for simple scenarios. Complex scenarios are possible just be extending/integrating external brokers

Production deployment flexibility (single node, multiple nodes, on-prem, cloud, hybrid deployments)

[chintanawilamuna]

What is microservices architecture?

Microservice architecture to me is a term that doesn't convey anything new. Martin Fowler's article about microservices compare it with a monolithic application. Breaking a monolithic application into a set of services comes with several benefits. That's true regardless of the term being used, microservices or SOA. Quoting from article,

In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies

Also the article goes on to explain a set of characteristics of microservices based architecture. These characteristics are general to any architecture that's based on services regardless of the implementation technology used. Association of ESBs with SOA and labeling it as heavyweight is not right. It's a flimsy argument against widely available high quality open source tools we have today. We can certainly do SOA without an ESB. A lot of the bad rap for SOA stems from trying to use expensive and bloated tools to do the wrong job. Compared with open source middleware tools these days we can implement distributed services with ease. A lot of these tools doesn't require developers to lock down on a particular messaging format. It's about understanding the customers' problems and using the right technology and toolset to solve the problems with minimum effort and time. So microservices doesn't give you anything new. You need to understand the problems you have and decide what's the best possible path to take. After that a proof of concept can be done in weeks. Once you can prove your architecture, progressing towards broader goals become easy.

Challenges and possible solutions

I really like this article on highscalability.com about microservices. This again is not specific to "microservices". Those challenges are there in every software system that's based on services. Let's see whether we can address some of the challenges.

Significant Operations Overhead

Any moderately large distributed deployment with 10+ servers (with HA) will have a significant operational overhead. There are multiple ways to reduce this complexity. This will increase when there are multiple environments for develop, test and production.

Deployment automation - Tools like Puppet gives a lot of power to automate deployment of environments as well as artifacts that should be deployed. All the components of the deployment such as load balancers, message brokers and other services can be successfully automated though Puppet

Monitoring running instances - Although there are open source tools like Nagios/Ganglia/ELK they're still a bit behind compared to tools like Amazon CloudWatch IMO.

Real-time alerting - Again there are multiple open source products that you can configure to get alerts on based on disk/CPU/memory utilization

Rolling into production - Open source tools like WSO2 AppFactory plays a significant role here to give developers the ability to rollout to different environments easily

We can utilize some of the above tools to reduce operational overhead.

Substantial DevOps Skills Required

To stand up the environment in a private data center or on top of an IaaS vendor does require some devops skill. You can get around this by using cloud hosted platforms such as WSO2 Public Cloud. Then again there can be organization policies around data locality.

Implicit Interfaces

This is one of the major challenges in any RESTful architecture. If you're using SOAP then the service interface is available through WSDL. Now WADL is becoming popular around HTTP based services for providing a service definition. Even though there are standards coming up still there are challenging aspects when it comes to securing and versioning these services.

Interface changes of services can be solved with a mediation layer when the need arise. This can be an alternative to introducing contracts for services where differences in interfaces are masked at the mediation layer.

Duplication Of Effort

Sanjiva's article about API Management explains the importance of having an API Management strategy for removing duplication of effort as well as service reuse. In order to reuse, there should be an efficient way to find what's available. Having a catalog of APIs help to find what can be reused and reduce duplication of effort.

Distributed System Complexity

Although distributed systems add complexity, flexibility aspects you get by having a set of independently operating services is large. According to the requirements of the application and performance characteristics expected, granularity of service decomposition should be decided.

Asynchronicity Is Difficult!

This IMO is a difficult problem to solve than the other points author has mentioned. Maintaining complex distributed transaction semantics across service boundaries is hard.

Testability Challenges

While individual service testing is important, integration testing of the entire interaction from end to end is more important in a distributed environment. So more emphasis should be given to service integrations that will involve several services being called to get a single result

[chintanawilamuna]

Wandering around streets of Zurich

[chintanawilamuna]

Moving the blog

I've just moved the blog to a new location. I might move the archives a little later. Moved a couple of posts to adjust the look and feel of the blog. Also hoping that this will be a bit more photo friendly.

[chintanawilamuna]

Selecting a Strategy for New IT

There are many advantage that you can gain from moving to new IT that connects business data with customers, partners and general public in general. “New IT” here refers to pretty much any process/mechanisms that you take to move away from siloed applications and having a more integrated experience to make day to day business more efficient. From a business and project management standpoint this can mean gaining more visibility into ongoing projects and the business impact they make. From an engineering stand point this can mean, versions of each project that’s in production, projects that will be going to production soon, critical bugs affecting a release schedule and so forth. This blog tries to explore couple of existing problems and how API management helps to move forward with adopting “new IT” for a connected business.

Existing processes

Everyone has some sort of a process/methodology that works for them. No 2 places or teams are alike. When there are already systems and certain ways of doing things are in place, it’s hard to introduce change. First and foremost the battle seems to be from the developers who are accustomed to doing certain things in certain ways. When introducing something that’s new, it’s always helpful to position it as filling a gap in the current process, doing incremental improvements.

Possible problems

Let’s see what some of the usual problems and how API management can solve those.

Standard method for writing services

If you have a rigid standard when introducing a new service, that might not be optimal. Rigid standards in the sense that tied to particular architectural style like REST or everything being a SOAP service. There are some services and scenarios where a SOAP service make sense and there are other situations where an HTTP service accepting a JSON message would be much simple and efficient. Having a one true standard which forces non-optimal creation of services will lead to unhappy developers/consumers and end up not being reused at all.

Challenges in service versioning/consistent interfaces

When a new version of a service is to be released, there has to be a way of versioning the service. If you can disrupt all the client and force them to upgrade then that’s easy (in case of an API change). Otherwise there has to be a way of deprecating an exising service and moving it to the new one.

Finding an existing service. Encouraging service reuse

If there are services available, then there should be a way to discover existing services. Not having a place to discover existing services is going to make service reuse a nightmare. This is not about holy grail of service reuse most SOA literature talks about. Think about how a developer in another team or a new hire discover services.

Monitoring/Metering/Throttling

When you have a service, you need to find out who’s using that service, what are the usage patterns, should certain clients be throtted out to give priority for other clients and so on. If you don’t have any metering, it’s near impossible to determine what’s going on. Problems (if you have any) service consumers face. It also helps to have historic data for doing projections that’ll result in resources expansions accordingly.

Solution

Here I would like to present a solution and how it will address some of the problems we listed out earlier. Make APIs the interfaces for services. There is a difference between a service and an API. API will be the consistent interface to everything outside (users within the organization/partners/suppliers/customers etc…), anything or anyone who will consume the service.

Having an API façade will be the first step. You can expose a SOAP based service without any problem. Exposing a REST service falls naturally. Exposing a service is not bound by any imposed standard anymore. A service can be written by analysing the best approach based on the problem at hand.

Versioning is enabled at the service definition level. The API layer. Following picture shows how to create a new version of an existing API.

In the picture above it shows the current API version in the top and you can give a version when you’re creating the new one. Once you create the API, at the publishing stage you can choose whether to deprecate the existing API or to have them coexist so that the new version is taken as a separate API. As following picture shows,

Another cool thing is that you can choose whether to revoke existing security tokens, forcing all clients no re-subscribe.

Next up is having a place for developers to find out about existing services. API Manager has a web application named API Store that list out published APIs. Here’s an example for the store that’s developed on top of API Manager. Monitoring is equally simple and powerful.

This is how API Manager helps to make life easy as well as help making the right technical choice. Allowing developers and other stakeholders to choose right service type, message formats encouraging service implementation to be diverse but still, having a consistent interface where monitoring/security/versioning etc… can be applied with ease.

[chintanawilamuna]

Implementing an SOA Reference Architecture

A reference architecture try to capture best practices and common knowledge for a particular subject area or a business vertical. Although you can define a reference architecture at any level, for the purpose of this blog we’ll be talking about an SOA reference architecture. When tasked with implementing an SOA architecture, blindly folloing a reference architecture might not give optimal results. If business requirements are not considered sometimes it will be not the right fit for the issues at hand.

When an SOA architecture is going to be implemented, close attention should be given to business requirements and any other technological constraints. Having to work with a specific ERP system, having to work with a legacy datastore which otherwise, too expensive to replace with a whole new system and so on. Based on these facts a solution architecture should be developed that’s aligned with the business requirements.

Looking at the solution architecture a toolset should be evaluated that maximize ROI and possible future enhancements that might come in the next 1 - 2 years. Evaluating an existing architecture every 1 - 2 years and making small refinements saves the time and effort for doing big bang replacements and modifications for critical components. While selecting a toolset having a complete unified platform helps to build the bits you need right away and still have room for additions later. If you’re looking for a complete platform you probably want to consider WSO2 middleware platform provide a complete open source solution.

I’m biased for open source solutions and having a platform that makes connected business possible is mighty cool.