bg

BangDB Vs Other DBs – YCSB

chevron

Evaluation of BangDB, MongoDB, Couchbase, Redis and Yugabyte using YCSB framework

10M records, for different workloads (A to F)

Following configurations were used for the evaluation purpose.

Redis Server: 5.07, x86/64 Redis Server: 5.07, x86/64 MongoDB server: 4.4.2, x86_64 YugabyteDB:2.5.0, x86_64 Couchbase2: 7.0 Beta, x86_64 BangDB server: 2.0.0, x86_64 Number of records: 10M RAM: 32GB, Cores: 16 YCSB workloads: see github.com/brianfrankcooper/YCSB/wiki/Core-Workloads
Each of these workload test runs in two steps, 1. Load and 2. Run. Load stage is to load the data and then run stage we run the test. I have run each test with clean database to reflect the numbers in fair manner

Summary

To summarize the test, here is the high-level report of the tests, load and run both.

image

Load is consistent for all dbs for all tests as expected as this phase is to load the data. However, run shows BangDB doing much better than other dbs for all the tests/workloads.

Next, let’s go into details of each test. We will have two charts one depicting throughput and other the depicting rate/speed at which db completes the test.

Workload A: Update heavy workload

This workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions.

image

BangDB throughput is lot higher than the others, even with different number of client connections it was not possible to improve the performance of other dbs The first graph shows the ops/sec (throughput) for the 10M records. However the second chart shows how quickly the test was completed by DBs.

Workload B: Read mostly workload

This workload has a 95/5 reads/write mix. Application example: photo tagging; add a tag is an update, but most operations are to read tags.

image

With less write percentages, performance improves for other dbs, but still BangDB performance is ~2X than the others The latency table shows that 99th percentile latency for Yugabyte is quite high compared to others (lower is better)

Workload C: Read only

This workload is 100% read. Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop).

image

The latency table is following for test C and since it was read only test hence there is no update latency figure here. Again Yugabyte latency is quite high

Workload D: Read latest workload

In this workload, new records are inserted, and the most recently inserted records are the most popular. Application example: user status updates; people want to read the latest.

image

The latency table for test D is as below. Here Redis and Yugabyte have higher latencies, Yugabyte performs bad for both Insert and Read for the test

Workload E: Short ranges

In this workload, short ranges of records are queried, instead of individual records. Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id).

image

This requires a lots of scans, hence the throughput decreases for all dbs, however Redis is slowest, understandably so as it is also reflected in the latency table below. Yugabyte performs really good in this test

Workload F: Read-modify-write

In this workload, the client will read a record, modify it, and write back the changes. Application example: user database, where user records are read and modified by the user or to record user activity.

image

Here Yugabyte has high latency esp for 99th percentile for Update and Read-modify-write. However, Very high Read latency for MonoDB makes it the last db to finish the task In this case records are read and updated. BangDB has 3X more performance than the other dbs.
This concludes the test report.

You may download the detailed test report from bangdb on github

Conclusion

While each database has been designed for different goals and use cases, YCSB test provides somewhat a common ground for the benchmark, therefore the numbers shown in this document can be used by developers or users to help select the db suitable for their requirement. All of these dbs are available free of cost for download / install and it will be fairly straightforward to run these tests in your environment for further analysis. The tests can be modified or added in order to cover a set of specific scenarios as needed.