Loading...
Searching...
No Matches
DataProvider::AbstractDataProcessor Class Referenceabstract

Defines an abstract class for accepting data and outputting optionally transformed or filtered data. More...

`#include <AbstractDataProcessor.qc.dox.h>`

## Public Member Functions

| flush (code enqueue)
| This method is called for each pipeline element to permit flushing of any pending data at the end of processing.
AbstractDataProviderType | getExpectedType ()
| Returns the expected type of data to be submitted, if available.
AbstractDataProviderType | getReturnType ()
| Returns the type of data that will be returned, if available.
| setThreadLocalData (*hash< auto > thread_local_data)
| Sets thread-local data to set before running the processor.
| submit (code enqueue, auto _data)
| Submits the data for processing.
bool | supportsBulkApi ()
| Returns True if the data processor supports bulk operation.

## Protected Member Functions

| flushImpl (code enqueue)
| This method is called for each pipeline element to permit flushing of any pending data at the end of processing.
*AbstractDataProviderType | getExpectedTypeImpl ()
| Returns the expected type of data to be submitted, if available.
*AbstractDataProviderType | getReturnTypeImpl ()
| Returns the type of data that will be returned, if available.
abstract | submitImpl (code enqueue, auto _data)
| Submits the data for processing.
abstract bool | supportsBulkApiImpl ()
| Returns True if the data processor supports bulk operation.

## Protected Attributes

*hash< auto > | thread_local_data
| Thread-local data to set before running the processor.
*list< string > | thread_local_data_keys
| Thread-local data keys from thread_local_data.

## Detailed Description

Defines an abstract class for accepting data and outputting optionally transformed or filtered data.

## Member Function Documentation

## flush()

DataProvider::AbstractDataProcessor::flush | ( | code | enqueue | ) |

This method is called for each pipeline element to permit flushing of any pending data at the end of processing.

Parameters

  enqueue | a closure taking a single argument that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated, then call it once for each generated record; prototype: code enqueue = sub (auto qdata) {}

Note
  - Calls flushImpl() on the data to do the actual processing
  - Accept and return type information is not enforced in this method; it must be enforced in flushImpl()

## flushImpl()

DataProvider::AbstractDataProcessor::flushImpl | ( | code | enqueue | ) |

protected

This method is called for each pipeline element to permit flushing of any pending data at the end of processing.

Parameters

  enqueue | a closure taking a single argument that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated, then call it once for each generated record; prototype: code enqueue = sub (auto qdata) {}

Note
  - The default implementation in the base class does nothing; to submit pending data for processing by subsequent elements in the pipeline when pipeline processing is finished, call enqueue with the pending data
  - No data should be cached or pending after this call; if the object requires closing or purging of internal state to make it reusable, it should be performed in this method

## getExpectedType()

AbstractDataProviderType DataProvider::AbstractDataProcessor::getExpectedType | ( |  | ) |

Returns the expected type of data to be submitted, if available.

Returns
  the expected type of data to be submitted, if available

Note
  Calls getExpectedTypeImpl() to provide the return value

## getExpectedTypeImpl()

*AbstractDataProviderType DataProvider::AbstractDataProcessor::getExpectedTypeImpl | ( |  | ) |

protected

Returns the expected type of data to be submitted, if available.

This base class method returns NOTHING; reimplement in subclasses to provide a type

## getReturnType()

AbstractDataProviderType DataProvider::AbstractDataProcessor::getReturnType | ( |  | ) |

Returns the type of data that will be returned, if available.

Returns
  the type of data that will be returned, if available

Note
  Calls getReturnTypeImpl() to provide the return value

## getReturnTypeImpl()

*AbstractDataProviderType DataProvider::AbstractDataProcessor::getReturnTypeImpl | ( |  | ) |

protected

Returns the type of data that will be returned, if available.

This base class method returns NOTHING; reimplement in subclasses to provide a type

## submit()

DataProvider::AbstractDataProcessor::submit | ( | code | enqueue,
|  | auto | _data
| ) |  |

Submits the data for processing.

Parameters

  enqueue

  a closure taking a single argument that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated from the input data, then call it once for each generated record; prototype:
  code enqueue = sub (auto qdata) {}

  _data | the data to process

Sets thread-local data before running and clears on exit if any is set

Note
  - Calls submitImpl() on the data to do the actual processing
  - Accept and return type information is not enforced in this method; it must be enforced in submitImpl()

## submitImpl()

abstract DataProvider::AbstractDataProcessor::submitImpl | ( | code | enqueue,
|  | auto | _data
| ) |  |

protectedpure virtual

Submits the data for processing.

Parameters

  enqueue

  a closure taking a single argument that enqueues the processed data for the next step in the pipeline; if no data should be processed onwards, do not call enqueue; if only one record should be processed onwards, then enqueue should be called only once; if multiple records are generated from the input data, then call it once for each generated record; prototype:
  code enqueue = sub (auto qdata) {}

  _data | the data to process

## supportsBulkApi()

bool DataProvider::AbstractDataProcessor::supportsBulkApi | ( |  | ) |

Returns True if the data processor supports bulk operation.

Returns
  True if the data processor supports bulk operation

Note
  Calls supportsBulkApiImpl() to return the answer

## supportsBulkApiImpl()

abstract bool DataProvider::AbstractDataProcessor::supportsBulkApiImpl | ( |  | ) |

protectedpure virtual

Returns True if the data processor supports bulk operation.

Returns
  True if the data processor supports bulk operation

See also
  Data Provider Pipeline Bulk Processing