Mapper::Mapper Class Reference

this class is a base class for mapping data; see Mapper Examples for usage examples More...

## Public Member Functions

| constructor (hash mapv, *hash opts)
| builds the object based on a hash providing field mappings, data constraints, and optionally custom mapping logic More...
int | getCount ()
| returns the internal record count More...
string | getFieldName (string fname)
| returns a descriptive name of the given field if possible, otherwise returns the field name itself
*hash | getInputRecord ()
| returns the value of the `"input"` option
*hash | getOutputRecord ()
| returns the value of the `"output"` option
auto | getRuntime (string key)
| get current runtime option value for a key More...
| logOutput (hash h)
| calls the output logging closure or call reference (if any) to log the output record
list< hash > | mapAll (list recs)
| maps all input records and returns the mapped data as a list of output records More...
list< hash > | mapAll (hash recs)
| maps all input records and returns the mapped data as a list of output records More...
hash | mapData (hash rec)
| processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash. More...
hash | optionKeys ()
| returns a list of valid constructor options for this class (can be overridden in subclasses) More...
| replaceRuntime (*hash runtime)
| replaces runtime options More...
| resetCount ()
| resets the internal record count More...
| setRuntime (string key, auto value)
| set the runtime option with "key" to value "value" More...
| setRuntime (hash runtime)
| adds runtime options to the current runtime option hash More...
hash | validKeys ()
| returns a list of valid field keys for this class (can be overridden in subclasses) More...
hash | validTypes ()
| returns a list of valid field types for this class (can be overridden in subclasses) More...

## Public Attributes

const | ConstantConflictList = ("name", "struct", "code", "default")
| field keys that conflict with "constant" and "index"
const | OptionKeys
| constructor option keys (can be extended by subclassing and reimplementing optionKeys())
const | OutputKeys
| output option keys
const | ValidKeys
| default known mapper hash field keys (can be extended by subclassing and reimplementing validKeys())
const | ValidTypes
| default known field types (can be extended by subclassing and reimplementing validTypes() and mapFieldType())

## Private Member Functions

| checkInputField (string k, string name)
| raises an error if an invalid input field name is declared; only call this if "input" is defined
| checkMap ()
| verifies the input map in the constructor
| checkMapField (string k, reference< any > fh)
| perform per-field pre-processing on the passed map in the constructor More...
| checkTimezoneOption (hash opts, string rn)
| verifies a timezone constructor option
| constructor ()
| private constructor for subclasses
| convertToHash (int t, string k, reference< any > fh)
| convert a field definition to a hash if possible
| error (string fmt)
| throws a `MAP-ERROR` exception; prepends the map name to the description if known More...
| error2 (string ex, string fmt)
| throws the given exception; prepends the map name to the description if known
| fieldLengthError (string k, string val, int ix, int sze, int maxlen, hash rc)
| called when a field exceeds its maximum length when processing hashes of lists
hash | mapDataIntern (hash rec)
| processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash. More...
nothing | mapFieldIntern (reference< hash > h, string key, hash rec, bool do_list, int list_size)
| maps a single field to the target More...
| mapFieldType (string key, hash m, reference< any > v, hash rec)
| performs type handling
auto | mapSubclass (hash m, auto v)
| to be overridden as necessary in subclasses
| setup (hash mapv, *hash opts)
| sets up the mapper object before checking the mapper hash
string | truncateField (string k, string val, int ix, int sze, int maxlen)
| called to truncate fields when processing hashes of lists

## Private Attributes

bool | allow_dot = False
| do not assume struct when field names have a `"."` in them; instead allow input field names to have a `"."` in them
bool | allow_output_dot = False
| do not assume structured/hash output when output field names have a `"."` in them; instead allow output field names to have a `"."` in them
hash | consth
| map of constant fields
int | count = 0
| count of records mapped
*string | date_format
| the global date format for parsing dates
string | encoding = "utf-8"
| the output character encoding; if not given then the output encoding is assumed to be UTF-8
hash | identh
| map of fields to be mapped 1:1 input -> output
*list | identl
| list of fields to be mapped 1:1 input -> output
*code | info_log
| an optional info logging callback; must accept a sprintf()-style format specifier and optional arguments
*hash | input
| an optional description of possible input hash keys
*code | input_log
| an optional input data logging callback; must accept a hash giving the input data hash
Qore::TimeZone | input_timezone = TimeZone::get()
| the timezone for input fields in case of parsing text values; if not set defaults to the current TimeZone (see Qore::TimeZone::get())
bool | m_empty_strings_to_nothing = False
| flag to enforce deletion of the empty string in the output record More...
*hash | m_runtime
| current runtime values More...
hash | mapc
| the hash providing output field names and mappings
hash | mapd
| the hash with a subset of the mappings used dynamically
hash | mapo
| the hash of output records for key order
*string | name
| the optional name for the object (for example a table name); will be prepended to field names in error messages
*string | number_format
| the global number format for parsing numbers
*hash | output
| an optional description of the output data structure
*code | output_log
| an optional output data logging callback; must accept a hash giving the output data hash
hash | rconsth
| map of constant runtime fields
*Qore::TimeZone | timezone
| an optional timezone for output date fields
bool | trunc_all = False
| truncate all option

## Detailed Description

this class is a base class for mapping data; see Mapper Examples for usage examples

## Member Function Documentation

## checkMapField()

Mapper::Mapper::checkMapField | ( | string | k,
|  | reference< any > | fh
| ) |  |

private

perform per-field pre-processing on the passed map in the constructor

Parameters

  k | the field name
  fh | a reference to the field's value in the map; will be converted to a hash

## constructor()

Mapper::Mapper::constructor | ( | hash | mapv,
|  | *hash | opts
| ) |  |

builds the object based on a hash providing field mappings, data constraints, and optionally custom mapping logic

Example:
  const DataMap = (
  # output field: "id" mapper from the "Id" element of any "^attributes^" hash in the input record
  "id": "^attributes^.Id",
  # output field: "name": maps from an input field with the same name (no translations are made)
  "name": True,
  # output field: "explicit_count": maps from the input "Count" field, if any value is present then it is converted to an integer
  "explicit_count": ("type": "int", "name": "Count"),
  # output field: "implicit_count": runs the given code on the input record and retuns the result, the code returns the number of "Products" sub-records
  "implicit_count": int sub (any ignored, hash rec) { return rec.Products.size(); },
  # output field: "order_date": converts the "OrderDate" string input field to a date in the specified format
  "order_date": ("name": "OrderDate", "date_format": "DD.MM.YYYY HH:mm:SS.us"),
  );
  Mapper mapv(DataMap);

Parameters

  mapv | a hash providing field mappings; each hash key is the name of the output field; each value is either True (meaning no translations are done; the data is copied 1:1) or a hash describing the mapping; see Mapper Specification Format for detailed documentation for this option
  opts | an optional hash of options for the mapper; see Mapper Options for a description of valid mapper options

Exceptions

  MAP-ERROR | the map hash has a logical error (ex: `"trunc"` key given without `"maxlen"`, invalid map key)

## error()

Mapper::Mapper::error | ( | string | fmt | ) |

private

throws a `MAP-ERROR` exception; prepends the map name to the description if known

if this method is subclassed, it must also cause an exception to be thrown

## getCount()

int Mapper::Mapper::getCount | ( |  | ) |

returns the internal record count

See also
  resetCount()

## getRuntime()

auto Mapper::Mapper::getRuntime | ( | string | key | ) |

get current runtime option value for a key

Parameters

  key | the runtime option key

Returns
  a runtime value if the key exists in the current runtime option hash and is set

See also
  - Mapper Runtime Options
  - replaceRuntime()
  - setRuntime()

Since
  Mapper 1.1

## mapAll() [1/2]

list<hash> Mapper::Mapper::mapAll | ( | list | recs | ) |

maps all input records and returns the mapped data as a list of output records

this method applies the mapData() method to all input records and returns the resulting list

Parameters

  recs | the list of input records

Returns
  the mapped data as a list of output records

Exceptions

  MISSING-INPUT | a field marked mandatory is missing
  STRING-TOO-LONG | a field value exceeds the maximum value and the 'trunc' key is not set
  INVALID-NUMBER | the field is marked as numeric but the input value contains non-numeric data

## mapAll() [2/2]

list<hash> Mapper::Mapper::mapAll | ( | hash | recs | ) |

maps all input records and returns the mapped data as a list of output records

this method applies the mapData() method to all input records and returns the resulting list

Parameters

  recs | a hash of lists of input records

Returns
  the mapped data as a list of output records

Exceptions

  MISSING-INPUT | a field marked mandatory is missing
  STRING-TOO-LONG | a field value exceeds the maximum value and the 'trunc' key is not set
  INVALID-NUMBER | the field is marked as numeric but the input value contains non-numeric data

## mapData()

hash Mapper::Mapper::mapData | ( | hash | rec | ) |

processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash.

Parameters

  rec | the record to translate

Returns
  a hash of field values in the target format based on the input data and processed according to the logic in the map hash

Exceptions

  MISSING-INPUT | a field marked mandatory is missing
  STRING-TOO-LONG | a field value exceeds the maximum value and the 'trunc' key is not set
  INVALID-NUMBER | the field is marked as numeric but the input value contains non-numeric data

Note
  - each time this method is executed successfully, the record count is updated (see getCount() and resetCount())
  - uses mapDataIntern() to map the data, then logOutput() is called for each output row

## mapDataIntern()

hash Mapper::Mapper::mapDataIntern | ( | hash | rec | ) |

private

processes the input record and returns a hash of the mapped values where the keys in the hash returned are the target field names; the order of the fields in the hash returned is the same order as the keys in the map hash.

Parameters

  rec | the record to translate

Returns
  a hash of field values in the target format based on the input data and processed according to the logic in the map hash

Exceptions

  MISSING-INPUT | a field marked mandatory is missing
  STRING-TOO-LONG | a field value exceeds the maximum value and the 'trunc' key is not set
  INVALID-NUMBER | the field is marked as numeric but the input value contains non-numeric data

Note
  - each time this method is executed successfully, the record count is updated (see getCount() and resetCount())
  - this is the same as mapData() except no output logging is performed

## mapFieldIntern()

nothing Mapper::Mapper::mapFieldIntern | ( | reference< hash > | h,
|  | string | key,
|  | hash | rec,
|  | bool | do_list,
|  | int | list_size
| ) |  |

private

maps a single field to the target

Performs the actual mapping

Parameters

  h | the hash to be updated with the mapped key/value pair; Depending on the mapper specification and the input, it can contain a bulk record (hash of lists) or a hash of both list and non-list keys (these are considered a "constants" in the output record.
  key | the column name (hash key) to be mapped (target field)
  rec | input record - either single record of hash of lists (batch); to increase performance, the input type (single record vs. batch) is determined by the do_list parameter. In case of bulk input, all the lists are supposed to have the same length list_size
  do_list | - whether the input record rec is single record or bulk format (hash of lists)
  list_size | - size of the lists in case the input is in bulk format (all lists must have the same length, asserted inside the method).

Note
  it is assumed that list_size > 0 whenever do_list is Qore::True
  if do_list == Qore::True, the 'rec' can contain a mix of lists and non-list values - the non-list values are used as constants in the mapping - as if expanded to lists with all values identical.

## optionKeys()

hash Mapper::Mapper::optionKeys | ( |  | ) |

returns a list of valid constructor options for this class (can be overridden in subclasses)

Returns
  a list of valid constructor options for this class (can be overridden in subclasses)

## replaceRuntime()

Mapper::Mapper::replaceRuntime | ( | *hash | runtime | ) |

replaces runtime options

Parameters

  runtime | a hash of runtime options to use to replace the current runtime option hash

Note
  this method cannot be called once data has been supplied to the Mapper; this method can only be used to initialize the Mapper with constant mappings before the Mapper processes any data

See also
  - Mapper Runtime Options
  - getRuntime()
  - setRuntime()

Since
  Mapper 1.1

Deprecated:
  use Mapper construction options to set the runtime options.

Exceptions

  RUNTIME-OPTION-CHANGED | the option has changed during while already processing the data.

## resetCount()

Mapper::Mapper::resetCount | ( |  | ) |

resets the internal record count

See also
  getCount()

## setRuntime() [1/2]

Mapper::Mapper::setRuntime | ( | string | key,
|  | auto | value
| ) |  |

set the runtime option with "key" to value "value"

Parameters

  key | a string with valid runtime key
  value | anything passed to the current runtime `key`

Note
  this method cannot be called once data has been supplied to the Mapper; this method can only be used to initialize the Mapper with constant mappings before the Mapper processes any data

See also
  - Mapper Runtime Options
  - replaceRuntime()
  - setRuntime()

Since
  Mapper 1.1

Deprecated:
  use Mapper construction options to set the runtime options.

Exceptions

  RUNTIME-OPTION-CHANGED | the option has changed during while already processing the data.

## setRuntime() [2/2]

Mapper::Mapper::setRuntime | ( | hash | runtime | ) |

adds runtime options to the current runtime option hash

Parameters

  runtime | a hash of runtime options to add to the current runtime option hash

Note
  this method cannot be called once data has been supplied to the Mapper; this method can only be used to initialize the Mapper with constant mappings before the Mapper processes any data

See also
  - Mapper Runtime Options
  - replaceRuntime()
  - setRuntime()

Since
  Mapper 1.1

Deprecated:
  use Mapper construction options to set the runtime options.

Exceptions

  RUNTIME-OPTION-CHANGED | the option has changed during while already processing the data.

## validKeys()

hash Mapper::Mapper::validKeys | ( |  | ) |

returns a list of valid field keys for this class (can be overridden in subclasses)

Returns
  a list of valid field keys for this class (can be overridden in subclasses)

## validTypes()

hash Mapper::Mapper::validTypes | ( |  | ) |

returns a list of valid field types for this class (can be overridden in subclasses)

Returns
  a list of valid types for this class (can be overridden in subclasses)

## Member Data Documentation

## m_empty_strings_to_nothing

bool Mapper::Mapper::m_empty_strings_to_nothing = False

private

flag to enforce deletion of the empty string in the output record

Since
  Mapper 1.1

## m_runtime

*hash Mapper::Mapper::m_runtime

private

current runtime values

Since
  Mapper 1.1