All Classes Namespaces Functions Variables Groups Pages
BulkSqlUtil::BulkUpsertOperation Class Reference

base class for bulk DML upsert operations More...

Inheritance diagram for BulkSqlUtil::BulkUpsertOperation:
[legend]

## Public Member Functions

| constructor (SqlUtil::Table target, *hash opts)
| creates the object from the supplied arguments More...
| constructor (SqlUtil::AbstractTable target, *hash opts)
| creates the object from the supplied arguments More...
private | flushImpl ()
| executes bulk DML upserts in the database with internally queued data
private | init (*hash opts)
| common constructor initialization
Public Member Functions inherited from BulkSqlUtil::AbstractBulkOperation
nothing | commit ()
| flushes any queued data and commits the transaction
| constructor (string name, SqlUtil::Table target, *hash opts)
| creates the object from the supplied arguments More...
| constructor (string name, SqlUtil::AbstractTable target, *hash opts)
| creates the object from the supplied arguments More...
| destructor ()
| throws an exception if there is data pending in the internal row data cache; make sure to call flush() or discard() before destroying the object More...
| discard ()
| discards any buffered batched data; this method should be called before destroying the object if an error occurs More...
| flush ()
| flushes any remaining batched data to the database; this method should always be called before committing the transaction or destroying the object More...
private | flushIntern ()
| flushes queued data to the database
Qore::SQL::AbstractDatasource | getDatasource ()
| returns the AbstractDatasource object associated with this object
int | getRowCount ()
| returns the affected row count
SqlUtil::AbstractTable | getTable ()
| returns the underlying SqlUtil::AbstractTable object
string | getTableName ()
| returns the table name
private | init (*hash opts)
| common constructor initialization
| queueData (hash data)
| queues row data in the block buffer; the block buffer is flushed to the DB if the buffer size reaches the limit defined by the `block_size` option; does not commit the transaction More...
| queueData (list l)
| queues row data in the block buffer; the block buffer is flushed to the DB if the buffer size reaches the limit defined by the `block_size` option; does not commit the transaction More...
nothing | rollback ()
| discards any queued data and rolls back the transaction
private | setupInitialRow (hash row)
| sets up the block buffer given the initial template row for inserting
private | setupInitialRowColumns (hash row)
| sets up the block buffer given the initial template hash of lists for inserting

## Additional Inherited Members

Public Attributes inherited from BulkSqlUtil::AbstractBulkOperation
const | OptionDefaults
| default option values
const | OptionKeys
| option keys for this object

## Detailed Description

base class for bulk DML upsert operations

This class assists with bulk upsert (SQL merge) operations into a target table.

Submitting Data
  To use this class, queue data in the form of a hash (a single row or a set of rows) or a list of rows by calling the queueData() method.
  The queueData() method queues data to be written to the database; the queue is flush()ed automatically when `block_size` rows have been queued.

Flushing and Discarding Data
  Each call to flush() (whether implicit or explicit) will cause a single call to be made to the dataserver; all queued rows are sent in a single bulk DML call, which allows for efficient processing of large amounts of data.
  A call to flush() must be made before committing the transaction to ensure that any remaining rows in the internal queue have been written to the database. Because the destructor() will throw an exception if any data is left in the internal queue when the object is destroyed, a call to discard() must be made prior to the destruction of the object in case of errors.

# single commit and rollback
on_success ds.commit();
on_error ds.rollback();
{
BulkUpsertOperation op1(table1);
BulkUpsertOperation op2(table2);
# each operation needs to be flushed or discarded individually
on_success {
op1.flush();
op2.flush();
}
on_error {
op1.discard();
op2.discard();
}
# data is queued and flushed automatically when the buffer is full
map op1.queueData($1), data1.iterator();
map op2.queueData($1), data2.iterator();
}

Note
  Wach bulk DML object must be manually flush()ed before committing or manually discard()ed before rolling back to ensure that all data is managed properly in the same transaction and to ensure that no exception is thrown in the destructor(). See the example above for more information.

## Member Function Documentation

BulkSqlUtil::BulkUpsertOperation::constructor | ( | SqlUtil::Table | target,
|  | *hash | opts
| ) |  |

creates the object from the supplied arguments

Parameters

  target | the target table object

  opts

  an optional hash of options for the object as follows:
  - `"block_size"`: the number of rows executed at once (default: 1000)
  - `"info_log"`: an optional info logging callback; must accept a string format specifier and sprintf()-style arguments
  - `"upsert_strategy"`: the upsert strategy to use; default SqlUtil::AbstractTable::UpsertAuto; see Upsert Strategy Codes for possible values for the upsert strategy

BulkSqlUtil::BulkUpsertOperation::constructor | ( | SqlUtil::AbstractTable | target,
|  | *hash | opts
| ) |  |

creates the object from the supplied arguments

Parameters

  target | the target table object

  opts

  an optional hash of options for the object as follows:
  - `"block_size"`: the number of rows executed at once (default: 1000)
  - `"info_log"`: an optional info logging callback; must accept a string format specifier and sprintf()-style arguments
  - `"upsert_strategy"`: the upsert strategy to use; default SqlUtil::AbstractTable::UpsertAuto; see Upsert Strategy Codes for possible values for the upsert strategy