FtpPoller::FtpPoller Class Referenceabstract

FtpPoller client class implementation. More...

## Public Member Functions

| constructor (hash< auto > nconf)
| creates the FtpPoller object from the configuration hash argument passed More...
| constructor (Qore::FtpClient n_ftp, hash< auto > nconf)
| creates the FtpPoller object from the FtpClient argument and configuration hash argument passed More...
| destructor ()
| stops the polling operation if in progress and destroys the object
*bool | fileEvent (list l)
| called for each poll with a list of all files matched before transfer; if this method returns False or NOTHING, then the singleFileEvent method is not called
binary | getFile (string path)
| retrieves a binary file and returns the file's contents
list< hash< FtpPollerFileEventInfo > > | getFiles (string subdir, int sort=FtpPoller::SortNone, int order=FtpPoller::OrderAsc)
| returns a list of regular file hashes matching any file name mask set for the object More...
int | getPollCount ()
| returns the current poll count
| getStoreFile (string remote_path, string local_path)
| retrieves a remote file and stores it to a local path More...
string | getTextFile (string path)
| retrieves a text file and returns the file's contents
abstract | postSingleFileEvent (hash< FtpPollerFileEventInfo > fih)
| called after singleFileEvent() for each matching file individually whenever matching files are polled with the list of matching file names; if any error occurs here, the polling operation stops More...
| removeFile (string fn)
| deletes a file on the server
| rename (string old, string nnew)
| renames a file on the server
bool | runOnce ()
| runs a single poll (useful for checking for errors inline before starting a background thread)
abstract | singleFileEvent (hash< FtpPollerFileEventInfo > fih)
| called for each matching file individually whenever matching files are polled with the list of matching file names; if any error occurs here, the error is logged and the polling operation is retried More...
int | start ()
| starts polling in the background; returns the thread ID of the polling thread More...
| startInline ()
| starts the polling operation inline (not in a background thread) More...
| stop ()
| stops the polling operation, returns when the polling operation has been stopped More...
| stopNoWait ()
| stops the polling operation, returns immediately More...
| waitStop ()
| waits indefinitely for the polling operation to stop; if polling was not in progress then this method returns immediately More...

## Static Public Member Functions

static | checkPath (string path, string type, bool write=False)
| checks a path on the local file system More...

## Public Attributes

const | AllKeys = RequiredKeysWithHost + Defaults.keys() + OptionalKeys
| all keys
const | Defaults = ...
| default values for constructor hash argument
const | ErrorDelay = 1m
| pause when Ftp errors are detected
const | OptionalKeys = ...
| optional constructor hash keys
const | OrderAsc = 0
| ascending sort order
const | OrderDesc = 1
| descending sort order
const | RequiredKeys = ...
| minimum required keys for all constructors
const | RequiredKeysWithHost = RequiredKeys + "host"
| RequiredKeys for the constructor(hash) without an Qore::FtpClient argument.
const | SortName = 1
| sort by name
const | SortNone = 0
| no sorting

## Protected Member Functions

| ftpSleep (softint secs)
| sleeps for the specificed number of seconds
| logDebug (string fmt)
| calls the `"log_debug"` closure or call reference with verbose debugging information
| logDetail (string fmt)
| calls the `"log_detail"` closure or call reference with detail information
| logInfo (string fmt)
| calls the `"log_info"` closure or call reference with important information
Mutex | m ()
| start mutex
| run ()
| starts the polling operation
Counter | sc ()
| stop counter
| setMask ()
| converts a glob mask into a regex

## Protected Attributes

bool | binary
| binary transfer flag (for singleFileEvent())
*string | encoding
| file encoding for text files
bool | fatal = False
| internal fatal error flag
Qore::FtpClient | ftp
| FtpClient object.
bool | get_files
| internal "get files" flag
string | host
| host or address name
*code | log_debug
| optional debug log closure
*code | log_detail
| optional detail log closure
*code | log_info
| optional info log closure
*string | mask
| file glob name mask (ignored if `"regex_mask"` also set)
*softint | minage
| minimum file age
*string | pass
| password
softlist< string > | path = "."
| path(s) to poll
int | poll_interval
| poll interval in seconds
int | pollcnt = 0
| internal poll counter
int | port
| port
int | reopts = 0
| file matching regex options
string | rootFtpPath
| path after connecting to Ftp server
bool | runflag = False
| run flag
*code | sleep
| optional sleep closure
*code | start_thread
| optional start thread closure
int | tid
| polling tid
timeout | timeout
| timeout in ms
string | url
| url
hash< UrlInfo > | urlh
| url hash
string | user
| user

## Detailed Description

FtpPoller client class implementation.

## Member Function Documentation

## checkPath()

static FtpPoller::FtpPoller::checkPath | ( | string | path,
|  | string | type,
|  | bool | write = `False`
| ) |  |

static

checks a path on the local file system

Exceptions

  DIR-ERROR | this exception is thrown if the local path does not exist, is not readable, is not a directory, or should be writable and is not

## constructor() [1/2]

FtpPoller::FtpPoller::constructor | ( | hash< auto > | nconf | ) |

creates the FtpPoller object from the configuration hash argument passed

Parameters

  nconf

  a hash with the following keys:
  - `host:` (required) the hostname or address to connect to
  - `port:` the integer port number to connect to (default 21; must be > 0 if given)
  - `user:` the username to use for the connection
  - `pass:` the password to use for the connection
  - `path:` the remote path(s) for retrieving the files
  - `poll_interval:` the integer polling interval in seconds (default: 10 seconds; must be > 0 if given)
  - `mask:` the file glob mask to use (default: `"*"`, ignored if `"regex_mask"` is also present)
  - `regex_mask:` a regular expression to use as a mask (overrides any `"mask"` value)
  - `reopts:` regular expression match options (ex RE_Caseless for case-insensitive matches)
  - `minage:` the minimum file age in seconds before a file will be acquired (default: 0)
  - `encoding:` the encoding for any text files received
  - `log_info:` a closure or call reference for logging important information; must accept a single string giving the log message
  - `log_detail:` a closure or call reference for logging detailed information; must accept a single string giving the log message
  - `log_debug:` a closure or call reference for logging verbose debgugging information; must accept a single string giving the log message
  - `start_thread:` (required when imported into a context where Qore::PO_NO_THREAD_CONTROL is set) a closure or call reference for starting threads; must return the integer thread ID (if not set then Background Operator (background) will be used)
  - `sleep:` (required when imported into a context where Qore::PO_NO_PROCESS_CONTROL is set) a closure or call reference to use instead of Qore::sleep() (if not set then Qore::sleep() will be used)

Exceptions

  FTPPOLLER-CONSTRUCTOR-ERROR | missing required key, invalid port or poll_interval given
  FTPCLIENT-PARAMETER-ERROR | empty hostname passed
  SOCKET-CONNECT-ERROR | error establishing socket connection (no listener, port blocked, etc)
  SSH2CLIENT-CONNECT-ERROR | no user name set; ssh2 or libssh2 error
  SSH2-ERROR | error initializing or establishing ssh2 session
  SSH2CLIENT-AUTH-ERROR | no proper authentication method found
  FTPCLIENT-CONNECT-ERROR | error initializing ftp session or getting remote path

## constructor() [2/2]

FtpPoller::FtpPoller::constructor | ( | Qore::FtpClient | n_ftp,
|  | hash< auto > | nconf
| ) |  |

creates the FtpPoller object from the FtpClient argument and configuration hash argument passed

Parameters

  n_ftp | the new FtpClient object

  nconf

  a hash with the following optional keys:
  - `poll_interval:` the integer polling interval in seconds (default: 10 seconds; must be > 0 if given)
  - `mask:` the file glob mask to use (default: `"*"`, ignored if `"regex_mask"` is also present)
  - `path:` the remote path(s) for retrieving the files; if a list of strings is given then each path will be polled for matching files according to the `"mask"` or `"regex_mask"` option
  - `regex_mask:` a regular expression to use as a mask (overrides any `"mask"` value)
  - `reopts:` regular expression match options (ex RE_Caseless for case-insensitive matches)
  - `minage:` the minimum file age in seconds before a file will be acquired (default: 0)
  - `encoding:` the encoding for any text files received
  - `binary:` if set to True then files are transferred in binary mode by default (with singleFileEvent() usage only), otherwise file data is returned in text format
  - `log_info:` a closure or call reference for logging important information; must accept a single string giving the log message
  - `log_detail:` a closure or call reference for logging detailed information; must accept a single string giving the log message
  - `log_debug:` a closure or call reference for logging verbose debgugging information; must accept a single string giving the log message
  - `start_thread:` (required when imported into a context where Qore::PO_NO_THREAD_CONTROL is set) a closure or call reference for starting threads; must return the integer thread ID (if not set then Background Operator (background) will be used)
  - `sleep:` (required when imported into a context where Qore::PO_NO_PROCESS_CONTROL is set) a closure or call reference to use instead of Qore::sleep() (if not set then Qore::sleep() will be used)

Exceptions

  FTPPOLLER-CONSTRUCTOR-ERROR | missing required key, invalid port or poll_interval given
  FTPCLIENT-PARAMETER-ERROR | empty hostname passed
  SOCKET-CONNECT-ERROR | error establishing socket connection (no listener, port blocked, etc); timeout establishing socket connection
  SSH2CLIENT-CONNECT-ERROR | no user name set; ssh2 or libssh2 error
  SSH2-ERROR | error initializing or establishing ssh2 session
  SSH2CLIENT-AUTH-ERROR | no proper authentication method found
  FTPCLIENT-CONNECT-ERROR | error initializing ftp session or getting remote path

## getFiles()

list<hash<FtpPollerFileEventInfo> > FtpPoller::FtpPoller::getFiles | ( | string | subdir,
|  | int | sort = `FtpPoller::SortNone`,
|  | int | order = `FtpPoller::OrderAsc`
| ) |  |

returns a list of regular file hashes matching any file name mask set for the object

Parameters

  subdir | the directory to check
  sort | the sort option for the list returned
  order | the ordering of sorted data returned

Returns
  a list of regular file hashes with the following keys in each list element:
  - `name:` the name of the file, link, or directory
  - `size:` the size of the file in bytes
  - `mtime:` the last modified date/time of the file

## getStoreFile()

FtpPoller::FtpPoller::getStoreFile | ( | string | remote_path,
|  | string | local_path
| ) |  |

retrieves a remote file and stores it to a local path

Parameters

  remote_path | the remote file path
  local_path | the local file path

## postSingleFileEvent()

abstract FtpPoller::FtpPoller::postSingleFileEvent | ( | hash< FtpPollerFileEventInfo > | fih | ) |

pure virtual

called after singleFileEvent() for each matching file individually whenever matching files are polled with the list of matching file names; if any error occurs here, the polling operation stops

This method would normally delete / rename / move files processed by singleFileEvent() so that they would not be polled a second time. If an error occurs in this operation, then the polling event will stop since continuing after failing to delete, rename, or move a file already processed would cause the file to be processed more than once.

Parameters

  fih

  a hash of file data and information with the following keys:
  - `name:` the name of the file, link, or directory
  - `size:` the size of the file in bytes
  - `mtime:` the last modified date/time of the file
  - `data:` the file's data; this will be a string unless the "binary" option is set to True, in which case this key is assigned to the files binary data
  - `filepath:` the remote filepath relative to FTP root directory

## singleFileEvent()

abstract FtpPoller::FtpPoller::singleFileEvent | ( | hash< FtpPollerFileEventInfo > | fih | ) |

pure virtual

called for each matching file individually whenever matching files are polled with the list of matching file names; if any error occurs here, the error is logged and the polling operation is retried

Parameters

  fih

  a hash of file data and information with the following keys:
  - `name:` the name of the file, link, or directory
  - `size:` the size of the file in bytes
  - `mtime:` the last modified date/time of the file
  - `data:` the file's data; this will be a string unless the "binary" option is set to True, in which case this key is assigned to the files binary data
  - `filepath:` the remote filepath relative to FTP root directory

## start()

int FtpPoller::FtpPoller::start | ( |  | ) |

starts polling in the background; returns the thread ID of the polling thread

if polling had already been started, then the thread ID of the polling thread is returned immediately

## startInline()

FtpPoller::FtpPoller::startInline | ( |  | ) |

starts the polling operation inline (not in a background thread)

Exceptions

  FTPPOLLER-ERROR | this exception is thrown if polling is already in progress

## stop()

FtpPoller::FtpPoller::stop | ( |  | ) |

stops the polling operation, returns when the polling operation has been stopped

if polling was not in progress then this method returns immediately

Exceptions

  THREAD-ERROR | this exception is thrown if this method is called from the event thread since it would result in a deadlock

See also
  stopNoWait()

## stopNoWait()

FtpPoller::FtpPoller::stopNoWait | ( |  | ) |

stops the polling operation, returns immediately

See also
  stop()

## waitStop()

FtpPoller::FtpPoller::waitStop | ( |  | ) |

waits indefinitely for the polling operation to stop; if polling was not in progress then this method returns immediately

Exceptions

  THREAD-ERROR | this exception is thrown if this method is called from the event thread since it would result in a deadlock