My workspace

Every programmer at least develop one Scheduler or Job in their life time of programming. Nowadays writing or developing scheduler to get you job done is very simple, but when you are thinking about high availability or load balancing your scheduler or job it getting some tricky. Even more when you have a few instance of your scheduler but only one can be run at a time also need some tricks to done. A long time ago i used some data base table lock to achieved such a functionality as leader election. Around 2010 when Zookeeper comes into play, i always preferred to use Zookeeper to bring high availability and scalability. For using Zookeeper you have to need Zookeeper cluster with minimum 3 nodes and maintain the cluster. Our new customer denied to use such a open source product in their environment and i was definitely need to find something alternative. Definitely Quartz was the next choose. Quartz makes developing scheduler easy and simple. Quartz clustering feature brings the HA and

UP1: Much more about high performance compution should be found in this book . This year i was invited to HighLoad++ conference in Moscow as a speaker. My session was in 2nd November in hall number 1, you can check the summary of my presentation here . I have very enjoyed my session, there are a lot of specialists came from the different sector and I was pleased to answer their questions. Even I continue my talk with participants after my session. Here you can find my full presentation in slide share . Certainly, i also listen to a few talks and I have to mention some of them. Session from company 2 sigma and Alibaba was very interesting. Company 2 sigma describe how they uses and managed their cluster using apache Mesos. Also 2 days non stop sessions from PostgresSQL, a lot of informations for developer and DBA. I have also learn a few new thing such as "competition" base machine learning which are using Avito. Also company Hawq introduce their new SQL engine for Hadoop

One of my few blog posts i have mentioned how to use and the use cases of using Oracle DataBase changed notification. When you have to need context search or count facts over your datas, you certainly need any modern Lucene based search engine. Elastic search is one of that search engine that can provide those above functionalities. However, from the previous versions of ES, elastic search river is deprecated to ingesting data to ES. Here you can get the whole history about deprecating river in ES. Any way, for now we have two option to ingest or import data from any sources to ES: 1) Implements or modify your DAO services, that can update data in ES and DataBase same time. 2) Polling, implements such a job, which will polling data in some period of time to update data in ES. First approach is the very best option to implements, however if you have any legacy DAO services or 3rd party application that you couldn't make any changes is not for you. Polling to data base frequentl

A few years ago in one of my blog post , i described how to use Oracle database changed notification to update HazelCast cache in application server layer. For now this is one of the finest use case of using Oracle database changed notification, but you can also use this possibilities for solving others problem such as event processing in legacy table. For instance, you are developing dispute system for any Banking system. For banking core system, dispute is as a another banking transaction, when any dispute comes from any client, operator of bank should react on this transaction. Most of the time disputes keeps in same storage (tables) along with another transactions. Such type of tables can keeps billions of rows, and when you would like to get notified when a few of the rows changes, you have a few options in your hand: 1) Poll periodically, schedular which will poll the whole table periodically to get changed data. 2) Using oracle trigger to send some notification (stored procedu

UP1: If you are planning to use Reactive programming, we recommend you to read the book "High performance in-memory computing with Apache Ignite". Nowadays, term Reactive programming and Reactive manifesto is became trend. Blog post, articles and presentations are all over. Why peoples getting crazy with it? or what's the problem it could solve? The easiest way to answer the above questions is to think about the requirements we have building application these days. We have to: 1) Modular - Module can help to maintain the application, it can be go offline and getting back to online without breaking all the system down; 2) Scalable - this way we can scale vertically or horizontally to handle a large amount of data or transactions; 3) Resilient - system can be getting back online and be fault tolerant; 4) Responsive - this means fast and available. Here is the paradigm of reactive manifesto: For most of the part, we already have a few framework such as Akka, Vert.x

UP1: If you are planning to migrate to Microservice, we recommend you to read the book "High performance in-memory computing with Apache Ignite". One of the challenging thing in Microservices world is not the implementation of the services, rather it's monitoring and the management. In the time of writing the blog, we have a few frameworks and tools to implements microservices such as Dropwizard , Spring boot and vertx . Complexity grows when you have a lot of independent micro services deployed and running over cloud infrastructure. It really a pain full task to monitor and manage all the services through 24*7. However Ansible, puppet, docker and logstash tools can help you to build such a management platform for micro services but they are not always sufficient. To solve the above described problem Jboss project release such a management tool for micro services named fabric8 . Fabric8 provides following possibilities such as management, continuous delivery and simple

Last couple of months, most of the conferences and seminar discussing and debuting about the new architecture design named microservice. From the first glance micro service means, developing independent small service which will be high cohesive in functionality and could be changed rapidly. Some of the IT evangelist already starts debating, why micro service is a buzzword. From my point of view word microservice misleading itself. Design model under this concept is as follows: 1. Service should be loosely coupled. 2. High cohesion in functionality. 3. Service should be change with minimal effect on other services. 4. Automated in deployments. 5. Scaling out easily. 6. Can be use polyglot programming language and polyglot persistence store. Most of these requirements are nothing new at all. Developers or IT Architect who already familiar with SOA already know all these above concept very well. From my point of view, we have to clear following three things to giving start coding o

Yesterday i visited Saint Petersburg university after a long period of time as a speaker. Conference held already a weeks and students came here from all over the country and out side of Russia. Audiences was very nice and ask a lot of technical question mostly about Cassandra and we are utilising it in our projects. Community was very curious about NoSQL and wanted find all the answer of their questions about the new trend. Here is the presentation from slide share net. NoSQL - World IT Planet, Saint Petersburg 2015 from Shamim bhuiyan

UP1: If you are interested in in-memory computing for getting High Performance from your system, we recommended you the book " High performance in-memory computing with Apache Ignite ". Today i have provide new presentation for tuning and optimising high load j2EE web application based on developing sberbank portal. Presentation is available here in slide share. Tuning HighLoad J2EE web application from Shamim bhuiyan

One of my friend ask me to develop a course for students in subject "Machine Learning". Course should be very simple to familiar students with machine learning. Main purpose of this course is to explorer the machine learning world to the students and playing over this topics with own their hands. Machine learning is very much matured and a lot of tools and frameworks is available to wet your hands in this topics, however most of the articles or tutorials you could found in the internet will start installing cluster or have to write a bunch of code (even, in site mahout, they are using maven) to start learning. Even more not all students are familiar with hadoop or do not have very powerful notebook to install and run all the components to get test of machine learning. For these reasons i have got the following approach: Standalone Hadoop Standalone Mahout  And a few CSV data files to learn how to works with Predictions Assume you already have java installed in your work s

Recently we start getting a few complains from our client related to connection on DataBase from IBM WAS. First action we have taken to take a look on log which we got from the client and discovered these following errors on application logs: Error 404: Database connection problem: IO Error: Got minus one from a read call DSRA0010E: SQL State = 08006, Error Code = 17,002 java.sql.SQLException: The back-end resource is currently unavailable. Stuck connections have been detected. With a quick search on google i have found  PMR 34250 004 000 on IBM support sites, which is also effect IBM WAS 8.* version. As soon as we are using third party web portal engine ( BackBase ) it was travois to figure out the problem, so we decompiled some code to make sure that all the data source connection closing well. After some research i have asked data base statistics and data source configurations from support team of the production. And i was  surprised with the data base statistics that all connect

In your entire software development, sooner or later you will got a few project with very low budget, where you can't use commercial software because the budget is low and in the long run company also want some profit completing the project. From the begging of the year i have done a few pre-sale for such projects and decided to write down a list of all open source alternatives against commercial product. One thing i have to clear that, i have no religious view over open sources software or vica verse. There a plenty of reasons to use most of the commercial product but most of all time we have to cut our coat according to our cloths. 1) BPM : Most of all vendor like Oracle and IBM have their finest product in BPM, such as Oracle BPM Server, IBM Business process manager. Also you can find a few very good open source product such as JBPM from Jboss and Bonitasoft. But their are another very good open source BPM engine you can try, it's Activity . Spring based (state machine) li

A few years ago (2011) in Java One Conference in Moscow, i participated with presentation about CI. During this time a lot of changes has been made with this fields. By the years many tools, plugins and frameworks has been released to help devOps to solve problems with CI. Now CI is one of the vital part of the development life cycle. With the aggressive use of cloud infrastructure and horizontal scaling of every application, now most of all application deployed in a lot of server (virtuals and dedicated). Moreover, most of the systems are heterogeneous and always need extra care (scripts) to successfully deploy the entire system. Most of the time development environment is very different from the production environment. Here is the common workflow from the development stage to production DEV environment -> Test Environment -> UAT environment -> Production environment. Every environments has their own characteristics, configurations. For example, most of the developers use

Last few months we are developing a portal for 3rd largest bank in Europe. Unique visitor of the bank grows more than 1 million visitor in a day. The main non functional requirements of the project is the high availability of the portal and giving high through output. One of the main feature of the portal is to giving user to customize their pages with widgets and provide different services for targeted auditory. After a long discussion and analysis, bank decided to use java based engine to build up the portal and we have got the following stack: 1) Java 1.7_47 2) IBM WEBSphere 8.5 as Application server 3) Nginx as web server 4) Alteon as load balancer 5) Oracle 11gR2 as DataBase 6) SOLR for content search Main challenge for us to supported legacy browser such as IE8, opera 12 e.t.c and one portal for all device (desktop, smart phone and tablet pc). Java based portal engine generated a lot of java script which didn't give us very good performance. For these above reasons we

Offtopic: My first book "High performance in-memory computing with Apache Ignite" has been released and available at http://leanpub.com/ignite . This post is my review of the Packt Publishing book Cassandra High Availability by Robbie Strickland. As the main title suggest. The book has almost 186 pages covering 9 chapters and must read for Cassandra users. Chapter 1: Cassandra's approach to High Availability The initial chapter of Cassandra High Availability cover some architectural design such as Monolithic Architecture, Master slave Architecture and the Cassandra's approach to achieve high availability. Most of all modern software system requires a non functional requirements such as High availability. In this chapter author briefly describe why RDBMS is the single point of failure and is not suitable for horizontal scaling, also the drawback of the Master Slave architecture. In the next few section Author describe the Cassandra's architecture and it's