BangDB API for C++

database

  1. To create database
  2. database(const char *dbName, const char *configPath = NULL, db_transaction_type tranType = DB_TRANSACTION_NONE, const char *dbpath = NULL, const char *dblogpath = NULL);

    The tranType by default is DB_TRANSACTION_NONE which means db is not opened in transaction mode and dbpath by default is NULL which means read the path from the config file. If you wish to override the path provided in the bangdb.config file then pass the path value here

    configPath is also by default NULL which means find the bangdb.config in the local project dir, else provide the full dir path to it

    If successful, db is created and initialized and returned a pointer to database, else program quits with error message. Note that this is a contructor hence use 'new' to create an object

    db_transaction_type has following values;

    • DB_TRANSACTION_NONE
    • DB_OPTIMISTIC_TRANSACTION
    • DB_PESSIMISTIC_TRANSACTION
  3. To get a table
  4. table* gettable(const char *tblName, bangdb_open_type openFlag = OPENCREATE, table_env *tenv=NULL);

    wideTable* getWideTable(const char *name, bangdb_open_type openflag = OPENCREATE, table_env *tenv=NULL);

    table* getPrimitiveTable(const char *name, bangdb_primitive_data_type dataType = PRIMITIVE_LONG, bangdb_open_type openflag = OPENCREATE, table_env *tenv=NULL);

    The bangdb_open_type has following values;

    • OPENCREATE
    • TRUNCOPEN
    • JUSTOPEN

    The table_env is a type which can be created, set with proper values and passed to the gettable. This allows user to override a set of config values as defined in the config file. Note that if NULL is passed for table_env then db config (defined in bangdb.config) values will apply to the table as well, but user can override the values by creating table_env for each table. This allows us to treat each table differently. Following parameters can be set or overridden for the table;

    • bangdb_persist_type
      • INMEM_ONLY
      • INMEM_PERSIST
      • PERSIST_ONLY
    • bangdb_index_type
      • HASH, - deprecated
      • EXTHASH
      • BTREE
      • HEAP – Not supported
    • log (wal) state
      • True
      • False
    • bangdb_log_type (private or shared log)
      • SHARED_LOG
      • PRIVATE_LOG
    • auto commit
      • True
      • False
    • table size hint ( This will be discussed separately, default is good for most cases)
      • TINY_TABLE_SIZE
      • SMALL_TABLE_SIZE
      • NORMAL_TABLE_SIZE
      • BIG_TABLE_SIZE
      • key size in bytes
    • key type
      • NORMAL_KEY
      • COMPOSITE_KEY
      • NORMAL_KEY_LONG
    • allow duplicates (bool)
      • True
      • False
      • log size in MB
    • table type
    • Note that user should never set table type, it's totally for BangDB to set based on API called (gettable, getPrimitiveTable and getWideTable)

      • NORMAL_TABLE
      • WIDE_TABLE
      • INDEX_TABLE
      • PRIMITIVE_TABLE_INT
      • PRIMITIVE_TABLE_LONG
      • PRIMITIVE_TABLE_STRING
    • bangdb_primitive_data_type
      • PRIMITIVE_INT
      • PRIMITIVE_LONG
      • PRIMITIVE_STRING
    • sort method
      • LEXICOGRAPH
      • QUASI_LEXICOGRAPH
    • sort direction
      • SORT_ASCENDING
      • SORT_DESCENDING

    Hence table_env gives us the capability to create multiple tables withing a db with different set of config values. For example, we can create table1 with {inmem_only, hash, log-on..}, table2 with {inmem_persist, Btree, log-off, autocommit-on...} and so on

    When successful, this returns a pointer to the table object else NULL with error message

  5. Close database
  6. void closedatabase(bangdb_close_type dbclose = DEFAULT);

    The possible values for bangdb_close_type are;

    • DEFAULT
    • CONSERVATIVE
    • OPTIMISTIC
    • CLEANCLOSE

    Note that closedatabase is enough to close the database and user need not close each table or connection separately. Closing database closes everything and cleans up all resources

  7. Close table
  8. int closetable(const char *tblName, bangdb_close_type tblclose = DEFAULT);

    int closetable(table *tbl, bangdb_close_type tblclose = DEFAULT);

    int closetable(wideTable *tbl, bangdb_close_type tblclose = DEFAULT);

    Returns 0 on success and -1 for error with message

  9. Beginning a transaction
  10. void* begin_transaction();

    Returns a opaque pointer which should be used in all operations participating in the transaction

  11. Committing a transaction
  12. LONG_T commit_transaction(void **txn_handle);

    When successful, returns a unique transaction id else -1 on error with message

  13. Aborting a transaction
  14. void abort_transaction(void **txn_handle);

    table

  1. Getting a connection
  2. connection* getconnection();

    primConnection* getPrimConnection();

    Returns a pointer to connection object else NULL with error message

    Note that for NORMAL_TABLE user should call getconnection() and for PRIMITIVE TABLE (all types of primitive tables) user should call getPrimConnection. If there is mismatch in calling either of the APIs then it returns NULL with proper error message.

  3. Closing a connection
  4. int closeconnection(connection *conn);

    eturn 0 for successful close else -1 with error message

  5. Closing this table
  6. int closetable(bangdb_close_type tableclose = DEFAULT);

    Return 0 for successful close else -1 with error message

  7. Dumping the data
  8. int dumpdata();

    This is helpful in explicitly dumping the data to disk at any time during the operation. But mostly used when we run the table in inmemory only mode but later during close or any other time we wish to save the data on disk

    Return 0 for successful close else -1 with error message

    wideTable

  1. Getting a connection
  2. wideConnection* getconnection();

    Returns a pointer to connection object else NULL with error message

  3. Closing all connections
  4. int closeAllConnections();

    eturn 0 for successful close else -1 with error message

  5. Adding/creating index
  6. int addIndex_str(const char *idx_name, int idx_size, bool allowDuplicates);

    int addIndex_num(const char *idx_name, bool allowDuplicates);

    int addIndex(const char *idx_name, table_env *tenv);

    int dropIndex(const char *idx_name);

  7. Closing this table
  8. int closetable(bangdb_close_type tableclose = DEFAULT);

    Return 0 for successful close else -1 with error message

  9. Dumping the data
  10. int dumpdata();

    This is helpful in explicitly dumping the data to disk at any time during the operation. But mostly used when we run the table in inmemory only mode but later during close or any other time we wish to save the data on disk

    Return 0 for successful close else -1 with error message

    connection (for Normal Table)

  1. Putting key value without transaction
  2. FILEOFF_T put(const char *key, int keylen, DATA_VAR *val, insert_options flag);

    FILEOFF_T put(const char *key, const char *val, insert_options flag);

    FILEOFF_T put(FDT *key, FDT *val, insert_options flag);

    FILEOFF_T put(LONG_T key, FDT *val, insert_options flag);

    FILEOFF_T put(LONG_T key, const char *val, insert_options flag);

    char* key and values are NULL terminated string of characters and FDT is a type with following structure;

    	struct FDT
    	{
    		void *data;
    		DATALEN_T len;	//DATALEN_T is unsigned int
    		FDT();
    		FDT(void *_data, DATLEN_T _len);
    		void copy(FDT *v);
    		void free();
    		~FDT();
    	};
    	

    The insert_options has following options;

    	enum insert_options
    	{
    		//if non-existing then insert and returns 0 
    		//else returns 1 if already exists else -1 for error
    		INSERT_UNIQUE,	
    		
    		//if existing then update and returns 0 
    		//else returns 1 if does not exist else -1 for error
    		UPDATE_EXISTING,	
    	
    		//insert if non-existing else update, should 
    		//always return 0 for success else -1 for error
    		INSERT_UPDATE,	
    		
    		//delete if existing(for future usage)
    		DELETE_EXISTING,	
    	};
    	
    	struct DATA_VAR
    	{
    		//the pointer to pre allocated buffer. for write db will read from
    		//it and for read db would write to it. User should clear the buffer
    		
    		void *data_buf;							
    						
    		//the allocated buffer length (db doesn't touch this)
    		
    		int data_buf_len;	
    		
    		//length of the read data (returned by db) or length of data to write
    		
    		int data_len;		
    		
    		//for write db takes the bytes from data_buf from the data_offt and
    		//for read db reads data in db from data_offt
    		//data_offt + data_len <= data_buf_len, else data will be curtailed
    		
    		int data_offt; 						
    					
    		//db sets if more data is to come for read(MORE_DATA_TO_READ)
    		
    		int flag;		
    	}
    	

    Return 0 for success else -1 with error message

  3. Putting key value with transaction
  4. FILEOFF_T put(const char *key, int keylen, DATA_VAR *val, insert_options flag, bangdb_txn *txn_handle);

    FILEOFF_T put(const char *key, const char *val, insert_options flag, bangdb_txn *txn_handle);

    FILEOFF_T put(FDT *key, FDT *val, insert_options flag, bangdb_txn *txn_handle);

    FILEOFF_T put(LONG_T key, FDT *val, insert_options flag, bangdb_txn *txn_handle);

    These are similar to above APIs except that they take transaction handle

    Return 0 for success else -1 with error message

  5. Getting value for a key without transaction
  6. char* get(const char *key);

    FDT* get(FDT *key);

    FDT *get_data(LONG_T key);

    int get(const char *key, int keylen, char **val, int *vallen);

    int get(const char *key, int keylen, DATA_VAR *data);

    int get(LONG_T key, DATA_VAR *data);

    returns appropriate values or NULL with error message

  7. Getting value for a key with transaction
  8. char* get(const char *key, void *txn_handle);

    FDT* get(FDT *key, void *txn_handle);

    int get(const char *key, int keylen, char **val, int *vallen, void *txn_handle);

    int get(const char *key, int keylen, DATA_VAR *data, bangdb_txn *txn_handle);

    int get(LONG_T key, DATA_VAR *data, bangdb_txn *txn_handle);

    These are similar to above APIs except that they take transaction handle

    Returns appropriate values or NULL with error message

  9. Deleting a key value without transaction
  10. FILEOFF_T del(char *key);

    FILEOFF_T del(FDT *key);

    FILEOFF_T del(LONG_T key);

    FILEOFF_T del(int key);

    Return 0 for success else -1 with error message

  11. Deleting a key value without transaction
  12. FILEOFF_T del(char *key, void *txn_handle);

    FILEOFF_T del(FDT *key, void *txn_handle);

    FILEOFF_T del(LONG_T key, void *txn_handle);

    Return 0 for success else -1 with error message

  13. Range query or scan without transaction
  14. resultset* scan(const char *skey, const char *ekey, scan_filter *sf = NULL);

    resultset* scan(FDT *skey, FDT *ekey, scan_filter *sf = NULL);

    resultset* scan(LONG_T skey, LONG_T ekey, scan_filter *sf = NULL);

    resultset* scan(int skey, int ekey, scan_filter *sf = NULL);

    resultset* scan(const char *skey, const char *ekey, scan_filter *sf, DATA_VAR *dv);

    resultset* scan(FDT *skey, FDT *ekey, scan_filter *sf, DATA_VAR *dv);

    resultset* scan(LONG_T skey, LONG_T ekey, scan_filter *sf, DATA_VAR *dv);

    Here skey and ekey are start and end keys for range query. Either of these two keys can be NULL, partial, full or some random value keys. When NULL is provided for skey and/or ekey then it means full scan in that direction. For example to get all values for keys greater than equal to key=”mykey” we can use scan(key, (FDT*)NULL) or to get get all values (select * from table) we can use scan((FDT*)NULL, (FDT*)NULL)

    scan_filter is provided to further add operators while doing the query and provide some limits on the returned resultset. The values for scan_filter is as follows;

    • scan_operator skey_op; //default GTE
    • scan_operator ekey_op; //default LTE
    • scan_limit_by limitby; //default LIMIT_RESULT_SIZE
    • int limit; //default 2MB (MAX_RESULTSET_SIZE)

    The scan_operator has following values;

    • GT, //greater than
    • GTE, //greater than equal to - default
    • LT, //less than
    • LTE, //less than equal to – default

    And scan_limitby has following values;

    • LIMIT_RESULT_SIZE, //defines the MB of data that should be returned (max)
    • LIMIT_RESULT_ROW, //number of rows (max) that should be returned

    The resultset is scrollable cursor with methods to iterate over the returned keys and values based in the given parameters by scan.

    Returns resultset on success else NULL for error

  15. Range query or scan with transaction
  16. resultset* scan(const char *skey, const char *ekey, bangdb_txn *txn_handle, scan_filter *sf = NULL);

    resultset* scan(FDT *skey, FDT *ekey, bangdb_txn *txn_handle, scan_filter *sf = NULL);

    resultset* scan(LONG_T skey, LONG_T ekey, bangdb_txn *txn_handle, scan_filter *sf = NULL);

    resultset* scan(const char *skey, const char *ekey, bangdb_txn *txn_handle, scan_filter *sf, DATA_VAR *dv);

    resultset* scan(FDT *skey, FDT *ekey, bangdb_txn *txn_handle, scan_filter *sf, DATA_VAR *dv);

    resultset* scan(LONG_T skey, LONG_T ekey, bangdb_txn *txn_handle, scan_filter *sf, DATA_VAR *dv);

    These are similar to above APIs except that they take transaction handle

    returns appropriate values or NULL with error message

  17. Counting the number of items
  18. FILEOFF_T count(char *skey, char *key, scan_filter *sf = NULL);

    FILEOFF_T count(FDT *skey, FDT *key, scan_filter *sf = NULL);

    FILEOFF_T count();

    LONG_T count(LONG_T skey, LONG_T ekey, scan_filter *sf = NULL);

    LONG_T count(int skey, int ekey, scan_filter *sf = NULL);

    FILEOFF_T count();

    This is very similar to scan except that this returns the count instead of resultset. The skey and ekey behavior is similar

    Third one is convenient method for counting all elements in the table

    Returns count(FILEOFF_T is long long) on success else -1 on error

  19. Setting auto commit
  20. void set_autocommit(bool flag);

    When transaction is OFF, then autocommit by default is ON and all individual operations guarantees ACID. We use non transactional APIs for get, put, del, scan. But when transaction is on then if autocommit is off then we can only use transactional APIs for get, put, del, scan and all non-transactional ones will fail. But to override the scenario, one can call this method and perform transactional or non transaction ops as required.

    For example, if db is opened in transactional mode with autocommit = OFF then following is not allowed;

    	char *v = conn->get(k);		//Not OK
    
    	we will have to either use the transactional API,
    
    	bangdb_txn txn;
    	db->begin_transaction(&txn);
    	char *v = conn->get(k, &txn);
    
    	Or, set the autocommit ON,
    
    	conn->set_autocommit(true);
    	char *v = conn->get(k);		//OK
    	
    	//if you don't wish to bother about all these then simply
    	//set the auto commit in bangdb.config as 1 and then you
    	//will not have to set or unset autocommit using the API
    	
  21. Close this connection
  22. int closeconnection();

    Returns 0 for success else -1 with error message

    primConnection (for primitive table)

  1. put
  2. The supported get and put are given below. The scan, count, del etc... APIs remain same as of normal connection

    FILEOFF_T put(LONG_T key, LONG_T val, insert_options flag);

    FILEOFF_T put(int key, int val, insert_options flag);

    FILEOFF_T put(const char *key, LONG_T val, insert_options flag);

    FILEOFF_T put(FDT *key, LONG_T val, insert_options flag);

  3. get
  4. int get(LONG_T key, LONG_T *val);

    int get(int key, int *val);

    int get(FDT *key, LONG_T *val);

    int get(const char *key, LONG_T *val);

    wideConnection (for wide table)

    Most of the APIs are similar to that of connection, except that it has few more APIs for put and scan and they are;

  1. more put APIs
  2. int put(FDT *key, FDT *val, const char* index_name, FDT *index_val);

    int put(LONG_T key, FDT *val, const char* index_name, FDT *index_val);

    int put_doc(const char *key, const char *json_str, insert_options flag);

    int put_doc(LONG_T key, const char *json_str, insert_options flag);

    LONG_T put_doc(const char *json_str);

  3. more scan APIs
  4. resultset *scan_doc(const char *index_name, FDT *skey, FDT *ekey, scan_filter *sf, DATA_VAR *dv);

    resultset *scan_doc(const char *index_name, LONG_T skey, LONG_T ekey, scan_filter *sf, DATA_VAR *dv);

    resultset *scan_doc(const char *index_name, const char *skey, const char *ekey, scan_filter *sf, DATA_VAR *dv);

    resultset *scan_doc(const char *index_name, FDT *skey, FDT *ekey, scan_filter *sf);

    resultset *scan_doc(const char *index_name, const char *skey, const char *ekey, scan_filter *sf);

    resultset *scan_doc(const char *index_name, LONG_T skey, LONG_T ekey, scan_filter *sf);

    resultset *scan_doc(const char *json_filter, int json_filter_len);

    resultset

  1. Checking if the result has next key, val pair
  2. bool hasNext();

    returns true if yes and false otherwise

  3. Moving to the next key and val pair
  4. void moveNext();

  5. Beginning the iteration
  6. void begin();

  7. To iterate from reverse, call this
  8. void begin_reverse();

    When resultset is returned, it is set at beginning, but if we want to go back to beginning then we call this api

  9. Getting the next key
  10. FDT *getNextKey();

    long getNextKeyLong();

    char * getNextKeyStr();

    returns FDT type for key or NULL

  11. Getting the next val
  12. FDT *getNextVal();

    long getNextValLong();

    char * getNextValStr();

    returns FDT type for val or NULL

  13. Last Evaluated Key
  14. FDT *lastEvaluatedKey();

    long lastEvaluatedKeyLong();

    This returns the last key in the resultset. Please see the documentation for use of this method

    returns the FDT type for key

  15. Check if more data is to come
  16. bool moreDataToCome();

    returns true if more data is expected to come for the query else false

  17. Counting the num of items in the resultset
  18. int count();

    returns the number of items in the resultset

  19. Searching a key in the resultset
  20. FDT *searchVal(FDT *key);

    returns the value for the given key

  21. Freeing the resultset resource
  22. void clear();

    Operations on two resultsets

    void add_doc(resultset *rs, char *orderBy=NULL);

    void add(resultset *rs);

    void append(resultset *rs);

    void intersect(resultset *rs);

    Sliding Table

  1. create table
  2. swTable(database *db, const char *tableName, table_env *tenv, int ttlsec, bool archive = false);

    Creates the sliding window table. The archive option is to tell if you would wish to archive the older data or simply discard them

  3. Initialize
  4. int initialize();

  5. Add Index
  6. void addIndex(const char *idxName, table_env *tenv);

  7. Get Connection
  8. wideConnection *getConnection();

  9. Put
  10. int put(const char *str, int len, insert_options iop);

    int put(const char *str, int len, char *idx, const char *idxkey);

    int put(const char *str, int len, char *idx, const char *idxkey, int idxkeylen);

  11. Scan
  12. resultset *scan(int period);

    resultset *scan(int period, int lag);

  13. Close
  14. void close();

    Counting

  1. Create type (general wrapper for counting)
  2. swEntityCount(const char *countName, int swTime, int swExpiry, int nEntity = 32);

  3. add for counting
  4. int add(const char *entityName, const char *s, int len);

    int add(const char *entityName, int entitylen, const char *s, int len);

    void add_create(const char *entityName, const char *s, int len, bangdb_window_type swType, bangdb_count_type countType);

    void add_create(const char *entityName, int entitylen, const char *s, int len, bangdb_window_type swType, bangdb_count_type countType);

  5. create entity
  6. void createEntity(const char *name, bangdb_window_type swType, bangdb_count_type countType);

    void createEntity(const char *name, int len, bangdb_window_type swType, bangdb_count_type countType);

  7. remove the entity
  8. void removeEntity(const char *name);

  9. list
  10. char* list_count();

    char *list_count2();

  11. count
  12. int count(const char *entityName);

    int count(const char *entityName, int span);

  13. shutdown
  14. void shutdown();

    TopK

  1. create
  2. topk(const char *topkName, int swSizeSec, int k, bool desc, char *uniqueBy);

  3. put
  4. void put(long score, char *data, int datalen, char *uniqueParam);

    void put(long score, char *data, int datalen, char *uniqueParam, int uniqueParamLen);

  5. Get TopK
  6. resultset *getTopK(int k = 0);

    char *getTopKJson(int k = 0);

  7. close
  8. void close();