// Copyright (C) 2016-2025 Internet Systems Consortium, Inc. ("ISC")
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.

/**

@page libdhcp_legal_log Kea Legal Log Hooks Library

Welcome to Kea Legal Log Hooks Library. This documentation is
addressed at developers who are interested in internal operation of the
libdhcp_legal_log library. This file provides information needed to understand
and perhaps extend this library.

This documentation is stand-alone: you should have read and
understood <a href="https://reports.kea.isc.org/dev_guide/">Kea
Developer's Guide</a> and in particular its section about hooks: <a
href="https://reports.kea.isc.org/dev_guide/df/d46/hooksdgDevelopersGuide.html">
Hooks Developer's Guide</a>.

@section libdhcp_legal_logUser How To Use Libdhcp_legal_log
## Introduction
libdhcp_legal_log is a hooks library which customizes the
DHCP lease management provided by Kea DHCP server modules (kea-dhcp4
and kea-dhcp6).  Specifically it implements recording a detailed log of lease
assignments and renewals as might be required for legal purposes.  It is
also described in the <a href="https://kb.isc.org/docs/kea-administrator-reference-manual">Kea
Administrator Reference Manual</a> instead in the section titled
Forensic Logging Hooks.

This library is designed to help with the requirement
that in many legal jurisdictions companies, especially ISPs, must
record information about the addresses they have leased to
DHCP clients.  If the information that it records is sufficient
it may be used directly. If your jurisdiction requires that
you save a different set of information you may use it as a
template or example and create your own custom logging hooks.

## Log Files
Legal logs are appended into a text file implemented by the class
@c isc::legal_log::RotatingFile. By default, it is rotated daily, but this can
be configured via the ``time-unit`` and ``count`` parameters.

Additional actions can be performed just before closing the old file and after
opening the new file. These actions must point to an external executable or
script and are configured by setting ``prerotate`` and ``postrotate``
parameters.
Kea will instantiate the prerotate action with the name of the file that will be
closed, but does not wait for the process to finish.
Kea will instantiate the postrotate action with the name of the file that had
been opened, but does not wait for the process to finish.

Legal file names, if using ``day``, ``month`` or ``year`` as time unit have the
form:
@verbatim
<path>/<base-name>.<CCYYMMDD>.txt
@endverbatim

where CC represents century, YY represents current year, MM represents current
month and DD represents current day.

Legal file names, if using ``second`` as time unit have the form:
@verbatim
<path>/<base-name>.<TXXXXXXXXXXXXXXXXXXXX>.txt
@endverbatim

where XXXXXXXXXXXXXXXXXXXX represents time in seconds since epoch.

To note that when using ``second`` as time unit, the file will be rotated when
the ``count`` number of seconds pass. In contrast, when using ``day``, ``month``
or ``year`` as time unit, the file will be rotated whenever the ``count``th day,
month or year starts respectively.

## DHCPv4 Log Detailed Structure
For DHCPv4, the library creates entries based on DHCPREQUEST, DHCPDECLINE,
DHCPRELEASE messages et.al. and their responses. The resulting packets and
leases are taken into account, intercepted through the following hook points:
* pkt4_receive
* leases4_committed
* pkt4_send
* lease4_release
* lease4_decline
This is necessary to get the subnet logging information.
Since Kea 1.9.8, the logging action is performed only on the pkt4_send hook
(for request and renew replies), on the lease4_release hook (for release
replies) and on the lease4_decline hook (for declined replies).
This guarantees that nothing is logged if the server or any other hook decides
to drop the packet between lease update and packet send operations.

An entry is a single string with no embedded end-of-line markers,
a prepended timestamp and has the following sections:
@code
<timestamp><address><duration><device-id>{client-info}{relay-info}{user-context}
@endcode
Where:
 - @b timestamp - the current date and time the log entry was written
   in "%Y-%m-%d %H:%M:%S %Z" strftime format.
 - @b address - the leased IPv4 address given out and whether it was
   assigned, renewed or released.
 - @b duration - the lease lifetime expressed in days (if present),
   hours, minutes and seconds.  A lease lifetime of 0xFFFFFFFF will be
   denoted with the text "infinite duration". This information is not
   given when the lease is released.
 - @b device-id - the client's hardware address shown as numerical type
   and hex digit string.
 - @b client-info - the DHCP client id option (61) if present, shown as
   hex digit string. When the id is printable it is added.
 - @b relay-info - for relayed packets the giaddr and the RAI circuit id
   and remote id options (option 82 sub options 1 and 2) if present.
 - @b user-context - the optional user context.

For instance:
@verbatim
2018-01-06 01:02:03 CET Address: 192.2.1.100 has been renewed for 1 hrs 52 min 15 secs to a device with hardware address: hwtype=1 08:00:2b:02:3f:4e, client-id: 17:34:e2:ff:09:92:54 connected via relay at address: 192.2.16.33, identified by circuit-id: 68:6f:77:64:79 (howdy) and remote-id: 87:f6:79:77:ef
@endverbatim
or for a release:
@verbatim
2018-01-06 01:02:03 CET Address: 192.2.1.100 has been released from a device with hardware address: hwtype=1 08:00:2b:02:3f:4e, client-id: 17:34:e2:ff:09:92:54 connected via relay at address: 192.2.16.33, identified by circuit-id: 68:6f:77:64:79 (howdy) and remote-id: 87:f6:79:77:ef
@endverbatim

The ``request-parser-format`` and ``response-parser-format`` can be used to
extract and log data from the incoming packet and server response packet
respectively. The configured value is an evaluated parsed expression returning a
string. This can be used to generate custom log entries.
Use with caution as this might affect server performance.
If any of them is configured, the default logging format is not used.
If both of them are configured, the resulting log message is constructed by
concatenating the logged data extracted from the request and the logged data
extracted from the response.

The ``timestamp-format`` parameter can be used to change the timestamp logged
at the beginning of each line. Permissible formatting is the one supported by
strftime plus the '%Q' extra format which adds the microseconds subunits. The
default is: "%Y-%m-%d %H:%M:%S %Z". This parameter has no effect for
the database backends, where the timestamp is defined at the schema level.

In addition to logging lease activity driven by DHCPv4 client traffic, it also
logs entries for the following lease management control channel commands:
lease4-add, lease4-update, and lease4-del.  Each entry is a single string
with no embedded end-of-line markers and they will typically have the following
forms:

@b lease4-add:
@code
<timestamp> Administrator added a lease of address: <address> to a device with hardware address: <device-id>
@endcode

Dependent on the arguments of the add command, it may also include the
    client-id and duration.

Example:
@verbatim
2018-01-06 01:02:03 CET Administrator added a lease of address: 192.0.2.202 to a device with hardware address: 1a:1b:1c:1d:1e:1f for 1 days 0 hrs 0 mins 0 secs
@endverbatim

@b lease4-update:
@code
<timestamp> Administrator updated information on the lease of address: <address> to a device with hardware address: <device-id>
@endcode

Dependent on the arguments of the update command, it may also include the client-id and lease duration.

Example:
@verbatim
2018-01-06 01:02:03 CET Administrator updated information on the lease of address: 192.0.2.202 to a device with hardware address: 1a:1b:1c:1d:1e:1f, client-id: 1234567890
@endverbatim

@b lease4-del:
Deletes have two forms, one by address and one by identifier and identifier type:

@code
<timestamp> Administrator deleted the lease for address: <address>
@endcode
     or
@code
<timestamp> Administrator deleted a lease for a device identified by: <identifier-type> of <identifier>
@endcode
Currently only a type of @b hw-address (hardware address) is supported.

Examples:
@verbatim
2018-01-06 01:02:03 CET Administrator deleted the lease for address: 192.0.2.202

2018-01-06 01:02:12 CET Administrator deleted a lease for a device identified by: hw-address of 1a:1b:1c:1d:1e:1f
@endverbatim

If High availability module is enabled, the partner will periodically send lease
commands which have a similar format, the only difference is that the issuer of
the command is 'HA partner' instead of 'Administrator'.

Example:
@verbatim

   *timestamp* HA partner added ...

@endverbatim

or

@verbatim
   *timestamp* HA partner updated ...
@endverbatim

or

@verbatim
   *timestamp* HA partner deleted ...
@endverbatim

## DHCPv6 Log Detailed Structure
For DHCPv6, the library creates entries based on REQUEST, RENEW, RELEASE,
DECLINE messages et.al. and their responses. The resulting packets and leases
are taken into account, intercepted through the following hook points:
* pkt6_receive
* leases6_committed
* pkt6_send
* lease6_release
* lease6_decline
This is necessary to get the subnet logging information.
Since Kea 1.9.8, the logging action is performed only on the pkt6_send hook
(for request, renew, rebind, release and declined replies).
This guarantees that nothing is logged if the server or any other hook decides
to drop the packet between lease update and packet send operations.

Note:

When using custom log format, the lease address and other parameters must be
extracted from the information stored in the client request or server response.
In the case of IPv6, the packets can contain multiple IA_NA (3) or IA_PD (25)
options, each containing multiple options, including OPTION_IAADDR (5) or
OPTION_IAPREFIX (25) suboptions.
To be able to print the current lease associated with the log entry, the
forensic log hook library internally isolates the corresponding IA_NA or IA_PD
option and respective suboptions matching the current lease.
The hook library will iterate over all new allocated addresses (stored in the
``leases6`` hook argument/context) and all deleted addresses (stored in the
``deleted_lease6`` hook argument/context), making each address available for
logging as the current lease for the respective logged entry.
It is recommended to extract the data from the server response (if available)
because the server can allocate other leases if the ones requested by the client
are not available. In this case the server must respond and update lifetimes to
0 for such leases, and they can be filtered out by the parser expression.

An entry is a single string with no embedded end-of-line markers,
a prepended timestamp and has the following sections:
@code
<timestamp><address><duration><device-id>{relay-info}*{user-context}*
@endcode
Where:
 - @b timestamp - the current date and time the log entry was written
   in "%Y-%m-%d %H:%M:%S %Z" strftime format.
 - @b address - the leased IPv6 address or prefix given out and whether
   it was assigned, renewed or released.
 - @b duration - the lease lifetime expressed in days (if present),
   hours, minutes and seconds.  A lease lifetime of 0xFFFFFFFF will be
   denoted with the text "infinite duration". This information is not
   given when the lease is released.
 - @b device-id - the client's DUID and hardware address (if present).
 - @b relay-info - for relayed packets the content of relay agent
   messages, remote id and subscriber id options (x and xx) if present.
 - @b user-context - the optional user context.

For instance:
@verbatim
2018-01-06 01:02:03 PST Address:2001:db8:1:: has been assigned for 0 hrs 11 mins 53 secs to a device with DUID: 17:34:e2:ff:09:92:54 and hardware address: hwtype=1 08:00:2b:02:3f:4e (from Raw Socket) connected via relay at address: fe80::abcd for client on link address: 3001::1, hop count: 1, identified by remote-id: 01:02:03:04:0a:0b:0c:0d:0e:0f and subscriber-id: 1a:2b:3c:4d:5e:6f
@endverbatim
or for a release:
@verbatim
2018-01-06 01:02:03 PST Address:2001:db8:1:: has been released from a device with DUID: 17:34:e2:ff:09:92:54 and hardware address: hwtype=1 08:00:2b:02:3f:4e (from Raw Socket) connected via relay at address: fe80::abcd for client on link address: 3001::1, hop count: 1, identified by remote-id: 01:02:03:04:0a:0b:0c:0d:0e:0f and subscriber-id: 1a:2b:3c:4d:5e:6f
@endverbatim

In addition to logging lease activity driven by DHCPv6 client traffic, it also
logs entries for the following lease management control channel commands:
lease6-add, lease6-update, and lease6-del.  Each entry is a single string
with no embedded end-of-line markers and they will typically have the following
forms:

@b lease6-add:
@code
<timestamp> Administrator added a lease of address: <address> to a device with DUID: <DUID>
@endcode

Dependent on the arguments of the add command, it may also include the hardware address and duration.

Example:
@verbatim
2018-01-06 01:02:03 PST Administrator added a lease of address: 2001:db8::3 to a device with DUID: 1a:1b:1c:1d:1e:1f:20:21:22:23:24 for 1 days 0 hrs 0 mins 0 secs
@endverbatim

@b lease6-update:
@code
<timestamp> Administrator updated information on the lease of address: <address> to a device with DUID: <DUID>
@endcode

Dependent on the arguments of the update command, it may also include the hardware address and lease duration.

Example:
@verbatim
2018-01-06 01:02:03 PST Administrator updated information on the lease of address: 2001:db8::3 to a device with DUID: 1a:1b:1c:1d:1e:1f:20:21:22:23:24, hardware address: 1a:1b:1c:1d:1e:1f
@endverbatim

@b lease6-del:
Deletes have two forms, one by address and one by identifier and identifier type:

@code
<timestamp> Administrator deleted the lease for address: <address>
@endcode
     or
@code
<timestamp> Administrator deleted a lease for a device identified by: <identifier-type> of <identifier>
@endcode
Currently only a type of @b DUID is supported.

Examples:
@verbatim
2018-01-06 01:02:03 PST Administrator deleted the lease for address: 2001:db8::3

2018-01-06 01:02:11 PST Administrator deleted a lease for a device identified by: duid of 1a:1b:1c:1d:1e:1f:20:21:22:23:24
@endverbatim

If High availability module is enabled, the partner will periodically send lease
commands which have a similar format, the only difference is that the issuer of
the command is 'HA partner' instead of 'Administrator'.

Example:
@verbatim

   *timestamp* HA partner added ...

@endverbatim

or

@verbatim
   *timestamp* HA partner updated ...
@endverbatim

or

@verbatim
   *timestamp* HA partner deleted ...
@endverbatim

## Configuring the DHCP Modules

It must be configured as a hook library for the desired DHCP server
modules.  Note that the legal_log library is installed alongside the
Kea libraries in "<install-dir>/lib" where <install-dir> is determined
by the --prefix meson setup option.  It defaults to
"/usr/local". Assuming the default value then, configuring kea-dhcp4
to load the legal_log library could be done with the following Kea4
configuration:

@code
"Dhcp4": {
    "hooks-libraries": [
        { "library": "libdhcp_legal_log.so",
          "parameters": {
            "path": "/var/log/kea",
            "base-name": "kea-legal4" } },
        ...
    ]
}
@endcode

To configure it for kea-dhcp6, the commands are simply as shown below:

@code
"Dhcp6": {
    "hooks-libraries": [
        { "library": "libdhcp_legal_log.so",
          "parameters": {
            "path": "/var/log/kea",
            "base-name": "kea-legal6" } },
        ...
    ]
}
@endcode

Two string Hook Library Parameters are supported:
 - @b path - Directory in which the legal file(s) will be written.  The
 default value is "<prefix>/var/log/kea".  The directory must exist.
 - @b base-name - An arbitrary value which is used in conjunction with
 the current system date to form the current legal file name.  It defaults
 to "kea-legal".

When running Kea servers for both DHCPv4 and DHCPv6 the log names must
be distinct, either the path or the base-name (or both) should be
different for DHCP4 vs DHCP6.

@section libdhcp_legal_logDesign Commented Design for Libdhcp_legal_log
## Introduction
Some countries require ISPs to log the connection meta-data for legal purposes.
A legal log hook library demonstrating how to log this information is then
useful as both an example explaining how to program with hooks and for direct
use.

As explained in the tutorial of the Hook Developer's Guide <a
href="https://reports.kea.isc.org/dev_guide/df/d46/hooksdgDevelopersGuide.html#hooksdgTutorial">
Tutorial</a> a hook library is a Dynamic Shared Object with known
entry points for the framework functions and hooks. The DSO is (when
not static linked) dynamically loaded by a Kea server module. Some
parameters can be passed to configure it and DSO entry points with
a well known hook names are auto-magically registered.

The Kea server module invokes the registered callouts during processing
and passes some context arguments.  The returned status is used to
control further processing, e.g., by dropping a message to cancel
the current action.

## Framework
The file version.cc defines the required framework version() function
which protects against Hooks API changes. The file load_unload.cc
defines a load() function which processes the configuration parameters
and creates and opens the legal log file (implemented by class @c
isc::legal_log::RotatingFile in rotating_file.h and
rotating_file.cc). The unload() function destroys the object which by
side effect closes the legal log file. Note the load() function does
not register callouts nor a logger: the Hooks Library Manager takes
care of this (triggered for the logger by legal_log_log.h,
legal_log_log.cc and legal_log_messages.h files).

## Callouts
For DHCPv4 pkt4_receive(), leases4_committed(), pkt4_send(), lease4_release()
and lease4_decline() hook callouts are defined in lease4_callouts.cc file.
For DHCPv6 pkt6_receive(), leases6_committed(), pkt6_send(), lease6_release()
and lease6_decline() hook callouts are defined in lease6_callouts.cc file.

The legalLog4Handler() or legalLog6Handler() is called
with the callout handle (to get the parsed query message and the lease) and an
action type which can either be ``ASSIGN`` or ``RELEASE``.
This requires the receive packet callouts (pkt4_receive and pkt6_receive) to
initialize an empty context which can be used in all following hooks.

When something goes wrong callouts return 1 (next step skip)
which tells the server to skip processing the next step. So
in the server any legal log error is converted to an internal
error and the client is denied service.

The hook callout, command_processed() is defined to log lease management
control channel commands (lease<4/6>-add, -update, -del).  Entries are
only generated for commands that execute successfully.

## Log Generation
For DHCPv4 the only difficulty is to get the relay agent information
option and its two interesting (i.e., to log) circuit ID and remote
ID sub-options for any queries that were relayed.

For DHCPv6 the hardware address is not built-in but can be recovered and
recorded by kea-dhcp6 in many cases. A DHCPv6 query can be relayed
more than once but the current code logs only the first (closest
to the client) relay information with first found remote ID and
subscriber ID relay options.

The lease client context can only be printed using the default format, as this
information is not directly stored in the incoming packet or in the server
response packet.

## Database
As an alternative to files a database backend is available. The code is
based on the lease backend with an automatic timestamp and log text columns in a
new logs table.

The configuration reuses the lease database one with for backward
(i.e. file) compatibility either no "type" entry or "logfile" value.

Since Kea 1.9.6, the libdhcp_legal_log hooks library supports database
connection recovery which can be enabled by setting the ``on-fail`` parameter.
If not specified, the ``on-fail`` parameter defaults to ``serve-retry-continue``
as opposed to the case of lease manager, host manager and config backend where
it defaults to ``stop-retry-exit``.
In this case, the server will continue serving clients and it will not shut down
even if the recovery mechanism fails.
If the ``on-fail`` is set to ``serve-retry-exit``, the server will shut down if
the connection to the database backend is not restored according to the
``max-reconnect-tries`` and ``reconnect-wait-time`` parameters, but it will
continue serving clients while this mechanism is activated.
During server startup, the inability to connect to any of the configured
backends is considered fatal only if ``retry-on-startup`` is set to ``false``
(the default). A fatal error is logged and the server exits, based on the idea
that the configuration should be valid at startup. Exiting to the operating
system allows nanny scripts to detect the problem.
If ``retry-on-startup`` is set to ``true``, the server will start reconnection
attempts even at server startup or on reconfigure events, and will honor the
action specified in the ``on-fail`` parameter.
The ``retry-on-startup`` parameter is supported since Kea 2.5.5.

Design notes:
 - the timestamp should be set by the database server (vs. DHCP server /
  database client). According to documentation it refers to the beginning
  of the transaction.

 - database systems manage time zones: the timestamp is stored in UTC
  and is displayed following options given to query tools.

 - the address (or prefix) column can be null / empty.
  This allows to insert no address related log entries.

## Syslog

Log entries can be inserted into syslog by setting the "type" to "syslog".
When syslog type is configured, the "pattern" parameter specifies the details that
are used for logging. If not configured, it defaults to: "%-5p [%c.%t] %m\n".

The "facility" parameter specifies the syslog facility and it defaults to "local0".

@section libdhcp_legal_logMTCompatibility Multi-Threading Compatibility

The libdhcp_legal_log hooks library is compatible with multi-threading.

*/
