Monitoring Apache Storm using JConsole

When you setup a storm cluster in a production environment it is important to monitor performance and resource usage of storm components to make sure that your cluster is working smoothly without issues.

This post will guide you to enable JMX and monitor storm components using JConsole.
Mainly there are two nodes to monitor in a storm cluster
  • Nimbus Node
  • Supervisor Node

Configure Nimbus Node 

Open storm.yaml file located at Nimbus_HOME>/conf/ directory and add below configurations.
  • Here I will be enabling JMX at port 9997. You can specify a preferred port here.
  • Add the host name of nimbus node
nimbus.childopts: "-Xmx1024m -Dcom.sun.management.jxmremote 
-Dcom.sun.management.jmxremote.local.only=false 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.port=9997 
-Djava.rmi.server.hostname=192.168.22.231"
Start storm nimbus using below command
./storm nimbus

Configure Supervisor Node

Open storm.yaml file located at Supervisor_HOME>/conf/ directory and add below configurations.
  • Here I will be enabling JMX at port 9998. You can specify a preferred port here.
  • Add the host name of supervisor node
supervisor.childopts: "-Xmx1024m -Dcom.sun.management.jxmremote 
 -Dcom.sun.management.jmxremote.local.only=false 
 -Dcom.sun.management.jmxremote.authenticate=false
 -Dcom.sun.management.jmxremote.ssl=false
 -Dcom.sun.management.jmxremote.port=9998
 -Djava.rmi.server.hostname=192.168.22.232"
Start storm supervisor using below command
./storm supervisor

Using JConsole

Open a terminal and type 'jconsole' to open JConsole. To monitor storm nimbus and supervisor running on remote serves you have to select 'Remote Process' and provide the ip and port of the server.


After you connect to JConsole, you might get a message as below. Click 'Insecure' to connect to the servers.
 

Upon successful connection to the storm servers, performance and resource usage can be monitored through JConsole.

Comments

  1. Unknown18 May 2017 at 01:05

    I really appreciate information shared above. It’s of great help. If someone want to learn Online (Virtual) instructor lead live training in Apache Storm , kindly contact us http://www.maxmunus.com/contact
    MaxMunus Offer World Class Virtual Instructor led training on TECHNOLOGY. We have industry expert trainer. We provide Training Material and Software Support. MaxMunus has successfully conducted 100000+ trainings in India, USA, UK, Australlia, Switzerland, Qatar, Saudi Arabia, Bangladesh, Bahrain and UAE etc.
    For Demo Contact us.
    Sangita Mohanty
    MaxMunus
    E-mail: sangita@maxmunus.com
    Skype id: training_maxmunus
    Ph:(0) 9738075708 / 080 - 41103383
    http://www.maxmunus.com/

    ReplyDelete

Post a Comment

Popular posts from this blog

Use ZAP tool to intercept HTTP Traffic

Image
ZAP Tool Zed Attack Proxy is one of the most popular security tool that used to find security vulnerabilities in applications. This blog discuss how we can use the ZAP tool to intercept and modify the HTTP and HTTPS traffic. Intercepting the traffic using the ZAP tool Before we start, lets download and install the ZAP Tool. 1) Start the ZAP tool using / zap.sh 2) Configure local proxy settings  To configure the Local Proxy settings in the ZAP tool go to Tools -> Options -> Local Proxy and provide the port to listen. 3) Configure the browser  Now open your preferred browser and set up the proxy to listen to above configured port. For example: If you are using FireFox browser browser proxy can be configured by navigating to "Edit -> Preferences -> Advanced -> Setting -> Manual Proxy Configuration" and providing the same port configured in the ZAP proxy 4) Recording the scenario Open the website that you want to intercept using t
Read more

Working with WSO2 carbon Admin Services

Image
WSO2 products managed internally using defined SOAP web services named as admin services. This blog will describe how to call the admin services and perform operation with out using the Management Console. Note - I will be using WSO2 Enterprise Integrator to demonstrate this. Lets look at how to access the admin services in wso2 products. By default the admin services are hidden from the user. To enable the admin services, 1. Go to <EI_HOME>/conf/carbon.xml and enable admin services as follows <HideAdminServiceWSDLs>false</HideAdminServiceWSDLs> 2. Now start the EI server using ./integrator.sh -DosgiConsole When the server is started click 'Enter' and you will be directed to the osgi console mode. 3. To search the available admin services, Add 'listAdminServices' in the osgi console.This will list down the available admin services with the URL to access the admin services. Access Admin Service via SOAP UI 4. You can ac
Read more

The 100% Open Source ESB

Image
Open Source ESB Market Today, open source solutions giving a significant competition to the proprietary software solutions. Same applies for the ESB software. There are number of open source ESB solutions available in the market, but the WSO2 ESB is the only ESB solution to be claimed as a 100% Open Source ESB solution in this competitive market. It was licensed under Apache License versions 2.0 . When we say 100% Open Source ESB, there can be several questions coming in to your mind. Below will answer your questions. Is the entire source code available freely? How about the Enterprise edition of WSO2 ESB? There is no such concept called Enterprise WSO2 ESB and Open Source WSO2 ESB as you have come across with other open source ESB solutions. In WSO2 ESB, we provide the entire source code available for the users without restricting the use of any available feature. Managing the success with Open Source Model When it comes to Open Source ESB, you may have some doubts
Read more