Modern use cases are demanding real-time stream processing with Graph and AI for efficient and scalable data analysis for continuous intelligence
Modern use cases are demanding modern ways to deal with data. These use cases have a lot in common. For example, they all have fast-moving data at the core, mostly coming from devices. They all need modern processing data in a lot more contextual and predictive manner which will require data to be linked in graph structure coupled with many AI models working together. The era of assembling a few pieces from open source to build such systems is gone already, most of the tools and systems in the market were created decades ago. Traditional architecture is failing to cope with the requirements increasingly. This is a great time to come up with the architecture and system which is designed from the ground up to address such problems. BangDB is a converged data platform that natively integrates AI, stream, and Graph processing within the NoSQL database.
Please see this blog which covers some of these aspects as well in a nice manner
Let’s look at the simple fact. In just a few years, the majority of the data will be generated by the devices. In less than a decade, device data will grow from being less than 2% to over 50% of the global data sphere [ Ref: IDC]. This change has been extremely rapid compared to the natural growth of the other systems and tools/technologies in the market. This has created an impedance mismatch and a huge gap when it comes to tackling the emerging or even the traditional use cases in the modern context.
By nature, these data move fast and carry lots of valuable insights which should be mined as soon as possible for higher value extraction. Graph structure gives these data much-needed context which could be leveraged in an absolute or probabilistic manner. Native AI provides real-time prediction support for various streaming events. And, since data may have many different formats, therefore, we must have a mechanism in place to deal with unstructured data in a scalable manner.
Some of the use cases for modern data processing would be.
All the above use cases involve a plethora of log/data streaming into a single system (for linking, correlation). They also require real-time stream and Graph processing and analysis. Building several machine learning models for online predictions and mechanisms to take action. On top of it, we must do all these with sub-second latency as these data have quickly diminishing intelligence value.
What are the problems with the existing systems/ architecture?
How does BangDB fit into such a scenario?
BangDB is a converged database that has natively implemented following
From an architectural point of view, BangDB follows the convergence model which means instead of splitting the system into multiple pieces and scaling different pieces separately, it follows space-based architecture where all necessary pieces are always in a single space, and we scale the spaces as we need. In other words, instead of 3-tier or n-tier architecture, in space-based architecture, we always deal with several units of computing (or machines) where each unit of computing contains all necessary components.
This avoids extra copy of data, network hops, distribution overhead for splitting requests, and combining responses which result in higher latency, computational overhead, and complex management apart from high cost and resource guzzling procedure. True convergence allows us to scale the system linearly without much overhead along with hyper-fast processing speed and ease of development and management.
BangDB is a full-fledged database as well, it implements some of the advanced features of a mature database like transaction, write-ahead log, buffer pool/page cache, indexes, persistence, rich query support, etc. BangDB is available free of cost, download it and start building modern apps with ease