Sunday, 31 July 2016

Docker makes your life easy !!!

Most of the time we have come across situations to set up a cluster for WSO2 products. With in a product QA cycle it is a very common thing. But as you all know it consumes considerable amount of time to set up the cluster and troubleshoot.

Now, with the use of dockers we can set up a cluster within few seconds and it makes your life easy :)

So let me give a basic knowledge on what is "docker"

What is Docker

In most simplest terms, docker is a platform to containerize software images.

Install Docker  :

What is Docker Compose

Docker compose is used to compose several applications and run those using one single command to initialize in multiple containers.

Install Docker Compose :

For some of the wso2 products there are docker compose images already exists in a private repository.

Main purpose of this blog is to highlight some of the useful docker commands you have to use while working with docker compose images.

To explain some of the usages I will be using ESB 4.9.0 docker compose image.
You can get a clone of the git repository where the docker compose image for ESB 4.9.0 is available. Follow the instructions in the READ.ME to setup the ESB docker cluster.

Start Docker container

docker-compose up

Build the changes and up the docker compose image

docker-compose up --build

Stop docker containers

docker-compose down 

Start docker in demon mode

 Docker-compose up -d

List docker images

 docker images 

List running docker containers

 docker ps 

Login to an active container

docker exec -i -t <container_id> /bin/bash 

Delete/Kill existing containers

 docker rm -f $(docker ps -aq) 

View container logs 

 docker logs <container_id> 

Insert a delay between docker containers

Sample Scenario: When running ESB cluster, first we want to ensure that DB is up and running, Therefore we can introduce a delay and start the ESB nodes. To configure this, you can add below property to the docker-compose.yml file
      - SLEEP=50

Add additional host names

Sample Scenario: Lets assume you want to use a backend service hosted in Application Server in another instance. Host name of the Application Server is "". Docker can not resolve the host name unless you defined the host name in docker-compose.yml file as below.
      - ""

Enable additional ports

Sample Scenario: Each of the ports used for the docker compose should be exposed through the docker-compose.yml file. If you are using inbound HTTP endpoint with port 7373, this port should be exposed as below.
      - "443:443"
      - "80:80"
      - "7373:7373"

Friday, 17 June 2016

Configuring an email notification to define user password

This blog will discuss how to configure an auto generate email to define the password when creating a user via management console. In WSO2 Identity Server there is an inbuilt feature called 'Ask Password' to fulfill this requirement. Lets look at how to implement this in other wso2 products.

'Ask Password' is a feature that comes with wso2 Identity Server. The purpose of this feature is to allow the users to decide there own password rather than defining a password by the server administration and allow the user to change the defined password.

So let me move on to the purpose of writing this blog.

While I was working with WSO2 API Manager, I got a requirement that the APIM administrator wants to create users via APIM management console, but the administrator wants to allow the users to define a password by the user itself. This requirement can be fulfilled using the 'Ask Password' feature available in wso2 Identity Server.


APIM Administrator creates a user by providing a username and a user email through the management console. Then an email will be sent to the defined email address with a redirection URL to define a password for the user account.

I will use APIM 1.10.0 product to explain this.

Steps to configure 'Ask Password' feature in APIM 1.10.0

1. Download APIM server

2. Log in to APIM server as the administrator

When you go to 'Add User' option you can see a window like below.

Now lets look at how to configure auto-email to set user password.

3. Install 'Account Recovery and Credential Management' feature in APIM

Due to some of the limitations in identify server feature activation, you have to install 'Account Recovery and Credential Management' feature in APIM 1.10.0. ( Steps to install a feature in wso2 product can be found from [1]).

4. As the next step, do the configuration changes mentioned here in APIM server.

These configurations are required to enable 'Ask Password' feature.

5. Restart the server after above changes.

When you navigate to 'Add User' option you can see that 'Ask password' feature is installed in UI as below.

6. Now create a user from APIM management console by defining the user email address.

You can verify whether auto generate email is received to the defined user email address and the user can define a password through the redirection screen provided in the auto-generated mail. Then check whether the user can successfully log in to APIM server.

Now APIM administrator can add the users via management console and allow the users to define a password they prefer.


Tuesday, 14 June 2016

Encrypting sensitive information in configuration files

Encrypting information 

I thought to start from basics before dig in to the target topic. So lets look at what is "encrypting".

Encrypting information is converting information in to another format, which is hard to understood. As we all know encrypting information is really useful to secure sensitive data.

In wso2 products, there is an inbuilt 'Secure Vault' implementation to encrypt plain text information in the configuration files to provide more security.

In this post I will not discuss about the secure vault implementation in details. You can refer 'secure vault implementation' to get more insight about it.
In wso2 products based on carbon 4.4.0 or later visions, 'Ciper Tool' feature is installed by default, therefore you can easily use that to encrypt sensitive information in the configuration file. 

Lets move on to the main purpose of this blog.

We already know that we can use ciper tool encrypt the information in configuration files. But can we encrypt the sensitive information in properties files or .json files ??

How to encrypt information when we can't use xpath notation?

Using the ciper tool we can encrypt any information if we can specify the xpath location of the property correctly. So basically if xpath notation can be defined for a certain property we can encrypt that using the ciper tool without much effort. Detailed steps to encrypt information based on an xpath can be found from here.

But in the properties file or .json files we can not define a xpath. Now you might be thinking how can we encrypt the information in these files !!!

To overcome this, we can manually encrypt the sensitive information using the ciper tool. You can refer the detailed steps provided here to manually encrypt the sensitive information in properties file and .json files.

However, I want to point you out to a very important fact. When you encrypt a sensitive information in a properties file or .json file, the product component which reading the encrypted property should have written in a way to call the secure vault to decrypt the value correctly.

Thursday, 21 April 2016

Information filtering using grep commands

While I was working on monitoring the long running test issues, I thought it would be useful to write a post on the usage of 'grep' commands in Linux.

In this article I will be discussing few real examples of using "grep" commands and how to execute grep commands as a shell script.

Search for a given string 

This command is use to search for specific string in a given file.
grep "<Search String>" <File Name>

Ex: In the below example, it will search for the string "HazelcastException" within wso2carbon.log file.
grep "HazelcastException" wso2carbon.log 

Search for a given string and write the results to a text file

This command is use to search for a given string and write the search results to a text file.
grep "<Search String>" <File Name> > <Text File Name>

Ex: In the below example, it will search for the string "HazelcastException" within wso2carbon.log file and write the search results to "hazelcastexception.txt" file.
grep "HazelcastException" wso2carbon.log > hazelcastexceptions.txt

Execute grep commands as a shell script

In some situations it will be useful to execute grep commands as a shell script.
Ex: While I was monitoring the long running test for certain exceptions, I used to search all the target exceptions from wso2carbon.log files and write those to specific files for further reference.

Follow below steps to execute multiple search strings and write those to text files using a shell script.

1) Create a file and add the grep commands to that file as given below and save it as a shell script. (Here I will name this file as "")

grep "HazelcastException" wso2carbon.log* > hazelcastexception.txt
grep "Hazelcast instance is not active" wso2carbon.log* > hazelcastnotactive.txt

2) Now add the shell script to the <Product_HOME>/<repository>/<logs> folder

3) Execute the script file using below command
After you execute the shell script, it will grep all wso2carbon.log files for the given search string and write those to separate text files.

Sunday, 10 January 2016

Allowing empty charachters using regular expression

This post will guide you to configure regular expression, to allow empty characters (spaces) for properties like user name and role name.

Validations for User Name, Role Name and Password are done using the regular expressions provided in <Product_Home>/repository/conf/user-mgt.xml file.

I will be taking EMM product as the example. By default empty characters are not allowed for role names in management console. If you enter a role with empty character (ex: Device Manager) you will get a message as in below image.

 Follow below steps to allow empty characters for role name.

1. Go to <EMM_HOME>/repository/conf/user-mgt.xml file and open the file. Then change <RolenameJavaRegEx> property and <RolenameJavaScriptRegEx> proerty as given below
Property name="RolenameJavaRegEx">[a-zA-Z0-9\s._-|//]{3,30}$</Property>

Property name="RolenameJavaScriptRegEx">^\w+( \w+)*$</Property>

Note -
  • <RolenameJavaScriptRegEx> is used by the front-end componenet for role name validation
  •  <RolenameJavaScriptRegEx> is used for back-end validation

2. Then restart the server

Now you will be able to add role names with empty spaces (ex: Device Manager).