BangDB API for JAVA

    Database

  1. To create database
  2. DatabaseImpl(String dbName, String configPath, TRANSACTIONTYPE transactionType, String dbPath, String dblogPath)

    TRANSACTIONTYPE has following options

    • DB_MULTIOPS_TRANSACTION_NONE
    • DB_OPTIMISTIC_TRANSACTION
    • DB_PESSIMISTIC_TRANSACTION
  3. To get a table
  4. Table getTable(String name, DBACCESS openflag, TableEnv tenv)

    WideTable getWideTable(String name, DBAccess openflag, TableEnv tenv)

    Table getPrimitiveTable(String name, BangDBPrimitiveDataType dataType, DBAccess openflag, TableEnv tenv)

    DBACCESS has following options

    • OPENCREATE
    • TRUNCOPEN
    • JUSTOPEN

    The TableEnv is a type which can be created, set with proper values and passed to the gettable. This allows user to override a set of config values as defined in the bangdb.config file. Note that if null is passed for TableEnv then db config (defined in bangdb.config) values will apply to the table as well, but user can override some of the values by creating TableEnv for each table. This allows us to treat each table differently.

    Following parameters can be set or overridden for the table;

    • PersistType
      • INMEM_ONLY, //only RAM based, cache enabled (no overflow to disk, ideally overflow to other RAM)
      • INMEM_PERSIST, //disked backed, cache enabled (over flow to disk)
      • PERSIST_ONLY,, //many procs one db file, cache disabled
      • DBTYPE_INVALID,
    • IndexType
      • HASH, - deprecated
      • EXTHASH,
      • BTREE,
      • HEAP, - not supported
      • INDEXTYPE_INVALID,
    • TableSizeHint
      • TINY_TABLE_SIZE
      • SMALL_TABLE_SIZE
      • NORMAL_TABLE_SIZE
      • BIG_TABLE_SIZE
    • key type (BangDBKeyType)
      • NORMAL_KEY,
      • COMPOSITE_KEY,
      • NORMAL_KEY_LONG,
      • KEY_TYPE_INVALID
    • LogType
      • SHARED_LOG,
      • PRIVATE_LOG,
      • LogType_INVALID
    • table type (BangDBTableType, default is NORMAL_TABLE)
    • Note that user should never set table type, it's totally for BangDB to set based on API called (getTable, getPrimitiveTable and getWideTable)

      • NORMAL_TABLE,
      • WIDE_TABLE,
      • INDEX_TABLE,
      • PRIMITIVE_TABLE_INT,
      • PRIMITIVE_TABLE_LONG,
      • PRIMITIVE_TABLE_STRING,
      • TABLE_TYPE_INVALID
    • BangDBPrimitiveDataType
      • PRIMITIVE_INT
      • PRIMITIVE_LONG
      • PRIMITIVE_STRING
      • PRIMITIVE_INVALID
    • sort method (BangDBSortMethod)
      • LEXICOGRAPH,
      • QUASI_LEXICOGRAPH,
      • SORT_METHOD_INVALID
    • sort direction (BangDBSortDirection)
      • SORT_ASCENDING,
      • SORT_DESCENDING,
      • SORT_DIRECTION_INVALID
    • boolean allowDuplicates
    • boolean isLogOn
    • boolean isAutoCommitOn
    • short keySize - in bytes
    • short LogSizeMB - in MBs

    Hence table_env gives us the capability to create multiple tables within a db with different set of config values. For example, we can create table1 with {inmem_only, hash, log-on..}, table2 with {inmem_persist, Btree, log-off, autocommit-on...} and so on

    When successful, this returns a Table object else null with error message

  5. Closing a Table
  6. int closeTable(Table tbl, DBClose dbclose)

    int closeTable(WideTable tbl, DBClose dbclose)

    DBClose has following options;

    • DEFAULT
    • CONSERVATIVE
    • OPTIMISTIC
    • CLEANCLOSE

    returns 0 for success or -1 for error

  7. Closing database
  8. void closeDatabase(DBClose dbclose)

    Note that closedatabase is enough to close the database and user need not close each table or connection separately. Closing database closes everything and cleans up all resources

  9. Beginning a transaction
  10. Transaction beginTransaction()

    returns a Transaction object which should be used in all participating operations

  11. Committing a transaction
  12. long commitTransaction(Transaction txn)

    returns a unique identifier for the committed transaction else -1 on error

  13. Aborting a transaction
  14. void abortTransaction(Transaction txn)

    Table

  1. Getting a connection
  2. Connection getConnection()

    PrimConnection getPrimConnection()

    returns a connection object using which all operations for the table can be done

    Note that for NORMAL_TABLE user should call getConnection() and for PRIMITIVE TABLE (all types of primitive tables) user should call getPrimConnection. If there is mismatch in calling either of the APIs then it returns NULL with proper error message.

  3. Closing a table
  4. int closeTable(DBClose tableClose)

    Returns 0 for success, -1 for error

  5. Dumping data
  6. int dumpData()

    This is helpful in explicitly dumping the data to disk at any time during the operation. But mostly used when we run the table in inmemory only mode but later during close or any other time we wish to save the data on disk

    Returns 0 for success, -1 for error

  7. Closing connection
  8. int closeConnection(Connection conn)

    Returns 0 for success, -1 for error

    WideTable

  1. Getting a connection
  2. WideConnection getConnection()

    returns a connection object using which all operations for the table can be done

  3. Closing a table
  4. int closeTable(DBClose tableClose)

    Returns 0 for success, -1 for error

  5. Adding Indexes
  6. int addIndex_str(String idx_name, int idx_size, boolean allowDuplicates)

    int addIndex_num(String idx_name, boolean allowDuplicates)

    int addIndex(String idx_name, TableEnv tenv)

    int dropIndex(String idxName)

  7. Dumping data
  8. int dumpData()

    This is helpful in explicitly dumping the data to disk at any time during the operation. But mostly used when we run the table in inmemory only mode but later during close or any other time we wish to save the data on disk

    Returns 0 for success, -1 for error

  9. Closing all connections
  10. int closeAllConnections()

    Returns 0 for success, -1 for error

    Connection (for Normal Table)

  1. Putting key value without transaction
  2. public long put(long key, String val, InsertOptions flag);

    public long put(long key, byte[] val, InsertOptions flag);

    public long put(String key, DataVar val, InsertOptions flag);

    public long put(byte[] key, DataVar val, InsertOptions flag);

    public long put(byte[] key, byte[] val, InsertOptions flag);

    public long put(String key, String val, InsertOptions flag);

    public long put(long key, byte[] val, InsertOptions flag, Transaction txn);

    Variety of options for keys and values are provided for user's convenience, user should ensure calling the appropriate apis which ever suitable

    The InsertOptions has following options;

    • INSERT_UNIQUE, //if non-existing then insert else return
    • UPDATE_EXISTING, //if existing then update else return
    • INSERT_UPDATE, //insert if non-existing else update
    • DELETE_EXISTING, //delete if existing(for future usage)

    Return 0 for success else -1 with error message

  3. DataVar is used with pre-allocated buffers
    • create DataVar
    • DataVar(int sizeInBytes)

    • copy buffers
    • void setBuffer(byte[] _buf)

    • Reading the buffer
    • byte[] getBuffer()

  4. Putting key value with transaction
  5. public long put(String key, DataVar val, InsertOptions flag, Transaction txn);

    public long put(byte[] key, DataVar val, InsertOptions flag, Transaction txn);

    public long put(String key, String val, InsertOptions flag, Transaction txn);

    public long put(byte[] key, byte[] val, InsertOptions flag, Transaction txn);

    These are similar to above APIs except that they take transaction handle

    Return 0 for success else -1 with error message

  6. Getting value for a key without transaction
  7. public String getStr(long key);

    public byte[] getByte(long key);

    public int get(String key, DataVar dv);

    public int get(byte[] key, DataVar dv);

    public String get(String key);

    public byte[] get(byte[] key);

    public int get(long key, DataVar dv);

    Failure to retrieve means data may not be there in the db, for other reasons like errors, an error message is also sent

  8. Getting value for a key with transaction
  9. public String get(String key, Transaction txn);

    public byte[] get(byte[] key, Transaction txn);

    public int get(long key, DataVar dv, Transaction txn);

    These are similar to above APIs except that they take transaction handle First three APIs returns true for success and false for failure to retrieve, whereas the last two apis returns the value or null for success and failure respectively. Failure to retrieve means data may not be there in the db, for other reasons like errors, an error message is also sent

  10. Deleting a key value without transaction
  11. long del(String key)

    long del(byte[] key)

    public long del(long key)

    Return 0 for success else -1 with error message

  12. Deleting a key value without transaction
  13. long del(String key, Transaction txn)

    long del(byte[] key, Transaction txn)

    Return 0 for success else -1 with error message

  14. Range query or scan without transaction
  15. ResultSet scan(String skey, String ekey, ScanFilter sf)

    ResultSet scan(byte[] skey, byte[] ekey, ScanFilter sf)

    public ResultSet scan(long skey, long ekey, ScanFilter sf)

    public ResultSet scan(String skey, String ekey, ScanFilter sf, DataVar dv);

    public ResultSet scan(byte[] skey, byte[] ekey, ScanFilter sf, DataVar dv);

    public ResultSet scan(long skey, long ekey, ScanFilter sf, DataVar dv);

    Here skey and ekey are start and end keys for range query. Either of these two keys can be null, partial, full or some random value keys. When null is provided for skey and/or ekey then it means full scan in that direction. For example to get all values for keys greater than equal to key=”mykey” we can use scan(key, null) or to get get all values (select * from table) we can use scan((byte[])null, (byte[])null)

    ScanFilter is provided to further add operators while doing the query and provide some limits on the returned resultset. The values for ScanFilter is as follows;

    • SCANOPERATOR skeyOp; //default GTE
    • ScanOperator ekeyOp; //default LTE
    • SCANLIMITBY limitBy; //default LIMIT_RESULT_SIZE
    • int limit; //default 2MB (MAX_RESULTSET_SIZE)

    The SCANOPERATOR has following values;

    • GT, //greater than
    • GTE,//greater than equal to - default
    • LT, //less than
    • LTE,//less than equal to – default

    And ScanLimitBy has following values;

    • LIMIT_RESULT_SIZE, //defines the bytes of data that should be returned (max)
    • LIMIT_RESULT_ROW, //number of rows (max) that should be returned

    The resultset is scrollable cursor with methods to iterate over the returned keys and values based in the given parameters by scan.

    Returns resultset on success else null for error

  16. Range query or scan with transaction
  17. ResultSet scan(String skey, String ekey, Transaction txn, ScanFilter sf)

    ResultSet scan(byte[] skey, byte[] ekey, Transaction txn, ScanFilter sf)

    public ResultSet scan(long skey, long ekey, Transaction txn, ScanFilter sf);

    public ResultSet scan(String skey, String ekey, Transaction txn, ScanFilter sf, DataVar dv);

    public ResultSet scan(byte[] skey, byte[] ekey, Transaction txn, ScanFilter sf, DataVar dv);

    public ResultSet scan(long skey, long ekey, Transaction txn, ScanFilter sf, DataVar dv);

    These are similar to above APIs except that they take transaction handle

    Returns appropriate values or null with error message

  18. Counting the number of items
  19. long count(String skey, String ekey, ScanFilter sf)

    long count(byte[] skey, byte[] ekey, ScanFilter sf)

    public long count(long skey, long ekey, ScanFilter sf);

    long count()

    public long count(long skey, long ekey, ScanFilter sf)

    This is very similar to scan except that this returns the count instead of resultset. The skey and ekey behaviour is similar

    Third one is convenient method for counting all elements in the table

    Returns count on success else -1 on error

  20. Setting auto commit
  21. void setAutoCommit(boolean flag)

    When transaction is OFF, then autocommit by default is ON and all individual operations guarantees ACID. We use non transactional APIs for get, put, del, scan. But when transaction is on and if autocommit is off then we can only use transactional APIs for get, put, del, scan and all non-transactional ones will fail. But to override the scenario, one can call this method and perform transactional or non transaction ops as required.

    For example, if db is opened in transactional mode with autocommit = OFF then following is not allowed;

    		String v = conn.get(k);		//Not OK
    	
    		we will have to either use the transactional API,
    	
    		Transaction txn = db.beginTransaction();
    		String v = conn.get(k, txn);
    	
    		Or, set the autocommit ON,
    	
    		conn.setAutocommit(true);
    		String v = conn.get(k);		//OK
    		
    		//if you don't wish to bother about all these then simply
    		//set the auto commit in bangdb.config as 1 and then you
    		//will not have to set or unset autocommit using the API
    		
  22. Close this connection
  23. int closeConnection();

    Returns 0 for success else -1 with error message

    primConnection (for primitive table)

  1. put
  2. The supported get and put are given below. The scan, count, del etc... APIs remain same as of normal connection

    public long put(int key, int val, InsertOptions flag);

    public long put(long key, long val, InsertOptions flag);

    public long put(String key, long val, InsertOptions flag);

    public long put(byte[] key, long val, InsertOptions flag);

  3. get
  4. public Integer get(int key);

    public Long get(long key);

    public Long get(String key);

    public Long get(byte[] key);

    wideConnection (for wide table)

    Most of the APIs are similar to that of connection, except that it has few more APIs for put and scan and they are;

  1. more put APIs
  2. public long putDoc(String jsonStr);

    public long putDoc(String key, String jsonStr, InsertOptions flag);

    public long putDoc(long key, String jsonStr, InsertOptions flag);

    public long put(String key, String val, String idxName, String idxVal);

    public long put(String key, byte[] val, String idxName, String idxVal);

    public long put(byte[] key, String val, String idxName, String idxVal);

    public long put(long key, String val, String idxName, String idxVal);

    public long put(long key, byte[] val, String idxName, String idxVal);

    public long put(byte[] key, byte[] val, String index_name, byte[] index_val);

  3. more scan APIs
  4. public ResultSet scanDoc(String idxName, String skey, String ekey, ScanFilter sf);

    public ResultSet scanDoc(String idxName, long skey, long ekey, ScanFilter sf);

    public ResultSet scan(String idxName, String skey, String ekey, ScanFilter sf);

    public ResultSet scan(String idxName, byte[] skey, byte[] ekey, ScanFilter sf);

    Resultset

  1. Checking if the result has next key, val pair
  2. boolean hasNext();

    returns true if yes and false otherwise

  3. Moving to the next key and val pair
  4. void moveNext();

  5. Beginning the iteration
  6. void begin()

    void beginReverse()

    When resultset is returned, it is set at begining, but if we want to go back to beginning then we call this api

  7. Getting the next key
  8. byte[] getNextKey()

    long getNextKeyLong()

    String getNextKeyStr()

  9. Getting the next val
  10. byte[] getNextVal()

    long getNextValLong()

    String getNextValStr()

  11. Last Evaluated Key
  12. byte[] lastEvaluatedKey()

    long lastEvaluatedKeyLong()

    Please see the documentation for use of this method

  13. Check if more data is to come
  14. boolean moreDataToCome()

    returns true if more data is expected to come for the query else false

  15. Counting the num of items in the resultset
  16. int count();

    returns the number of items in the resultset

  17. Searching a key in the resultset
  18. byte[] searchValue(String key)

    byte[] searchValue(byte[] key)

    Returns true for success and false otherwise

  19. Operations of ResultSet
  20. void addDoc(ResultSetImpl rs, String orderBy)

    void add(ResultSetImpl rs)

    void append(ResultSetImpl rs)

    void intersect(ResultSetImpl rs)

  21. Freeing the resultset resource
  22. void clear();

    Sliding Table

  1. Create
  2. SWTableImpl(DatabaseImpl db, String tableName, TableEnv tenv, int ttlsec, boolean archive)

  3. Initialize
  4. int initialize()

  5. Adding index
  6. void addIndex(String idxName, TableEnv tenv)

  7. Getting Connection
  8. WideConnectionImpl getConnection()

  9. put
  10. int put(String str, InsertOptions iop)

    int put(String str, String idx, String idxKey)

  11. scan
  12. ResultSet scan(int period)

    ResultSet scan(int period, int lag)

    ResultSet scanFull()

    ResultSet scanRemaining(long fromTime, int lag)

  13. close
  14. void Close()

    Counting

  1. Create wrapper
  2. SWEntityCountImpl(String entityName, int swTime, int swExpiry, int nEntity)

  3. Create Entity
  4. void createEntity(String name, BangDBWindowType swType, BangDBCountType countType)

    BangDBWindowType

      NON_SLIDING_WINDOW,

      SLIDING_WINDOW_SPAN,

      SLIDING_WINDOW_UNIT,

  5. Add
  6. int add(String entityName, String str)

    void addCreate(String entityName, String str, BangDBWindowType swType, BangDBCountType countType)

  7. Remove Entity
  8. void removeEntity(String name)

  9. List
  10. String listCountJson()

    String listCountStr()

  11. Count
  12. int count(String entityName)

    int count(String entityName, int span)

  13. shutdown
  14. void shutDown()

    TopK

  1. Create
  2. TopKImpl(String topkName, int swSizeSec, int k, boolean desc, String uniqueBy)

  3. Put
  4. void put(long score, String data, int datlen, String uniqueParam)

  5. Get TopK
  6. String getTopKJson(int k)

    ResultSet getTopK(int k)

  7. Close
  8. void close()