My workspace

Recently I was involved into a project for replicating data from PostgreSQL to Datalake. I heard about PostgreSQL logical data replication a few years ago and found a few hours to play with the Change data capture fuction of these popular database. Today's blog is all about PostgreSQL logical replication with a few fragments of code to test the functionality.  Change data capture is an architecture design principle that allows us to capture the recently data changed into the database in realtime. The changes could be any DML operations: Insert, update, delete.  TThere are a few charactaritics of CDC as follow: Logical replication uses publish-subscribe model. Subscriber pulls the data from the publication they subscribe to and manipulate the data. Only commited transactions on WAL archive (transaction log file) will be return by the CDC. CDC will return on orderd form how they commited on the database. Logical CDC doesn't impact on database performance. From the PostgreSQL v15,

Apache Ignite  is built on JVM and not a set-it-and-forget-it system. So, like other distributed systems, it requires monitoring for acting on time. However, Apache Ignite provides a web application named Ignite Web Console to manage and monitor the cluster, but it's not enough for system monitoring. You can also use JConsole/VisualVM for monitoring an individual Ignite node and a small number of Ignite nodes. Monitoring an Ignite cluster over 5 nodes by VisualVM or JConsole is unrealistic and time-consuming. Also, JMX does not provide any historical data. So, it is not recommended for production environments. Nowadays, there are a lot of tools available for system monitoring. The most famous of them are: Nagios Zabbix Grafana In this article, we cover the Grafana for monitoring Ignite clusters and provide step-by-step instructions to install and configure the entire stack technology. Grafana  is an open source graphical tool dedicated to query, visualize, and alert all

Any technology, no matter how advanced it is, will not be able to solve your problems if you implement it improperly. Caching, precisely when it comes to the use of a distributed caching, can only accelerate your application with the proper use and configurations of it. From this point of view, Apache Ignite is no different, and there are a few steps to consider before using it in the production environment. In this article, we describe various technics that can help you to plan and adequately use of Apache Ignite as cutting-edge caching technology. Do proper capacity planning before using Ignite cluster. Do paperwork for understanding the size of the cache, number of CPUs or how many JVMs will be required. Let’s assume that you are using Hibernate as an ORM in 10 application servers and wish to use Ignite as an L2 cache. Calculate the total memory usages and the number of Ignite nodes you have to need for maintaining your SLA. An incorrect number of the Ignite nodes can become a b

If you're just starting with this great open source framework, don't worry, we're here to help. Check out this great resource to help get you going. If you are running Apache Ignite for the first time, you might face some difficulties. You have just downloaded Apache Ignite, run it a few times, and got some issues. Mostly, these problems are solved in a similar fashion. Therefore, I decided to create a checklist, which provides recommendations to help you avoid issues in the development environments. 1. Configuration Files When Ignite starts in standalone mode by executing the ignite.sh|bat file, Ignite uses the $IGNITE_HOME/config/default-config.xml configuration file. In this situation, to connect to the specified node from the Visor command line console, you should choose the default-config.xml file from the configuration file list. Most of the time, the default- config.xml file is the first file in the list. You have to run the following command to execute an Ig

The book review was done by Md Sadruddin and publish on his web site itteratory.com . The part of the review is published here with permission by Md Sadruddin. Last year, as I was working with Apache Ignite, I was desperately in search of a book that I could rely upon in terms of clearing my doubts. A book that would help me learn the product in a holistic manner. And as my search was on, I bumped into the book by   Shamim Ahmed Bhuiyan , Michael Zheludkov, and   Timur Isachenko . I was super impressed with it as I scanned through the book. I also put up a review of that version of the book   here . The product Apache Ignite has been evolving in a rapid speed. It introduced many new features, different architectural revamp etc as the new versions were released. But as the new features got added with the newly released versions, there came a need of the new version of the   book   covering these new features. I found out   Shamim Ahmed Bhuiyan   and Michael Zheludkov are already

From the version 2.4.0, Apache Ignite introduced a new way to connect to the Ignite cluster, which allows communication with the Ignite cluster without starting an Ignite client node. Historically, Apache Ignite provides two notions of client and server nodes. Ignite client node intended as lightweight mode, which does not store data (however, it can store near cache), and does not execute any compute tasks. Mainly, client node used to communicate with the server remotely and allows manipulating the Ignite Caches using the whole set of Ignite API’s. There are two main downsides with the Ignite Client node: Whenever Ignite client node connects to the Ignite cluster, it becomes the part of the cluster topology. The bigger the topology is, the harder it is for maintaining. In the client mode, Apache Ignite node consumes a lot of resources for performing cache operations. To solve the above problems, Apache Ignite provides a new binary client protocol for implementing thin Ignite cl

This week The apache Ignite book becomes one of the top books of leanpub.