HttpServer Module

# HttpServer Introduction

The HttpServer module provides a multi-threaded HTTP server to Qore programs.

The HttpServer implemented is designed for serving REST, RPC-style, and file-system-based requests as well as other types of traffic with an appropriate handler (for example handlers requiring protocol changes such as with the WebSocket protocol are supported as well).

To use this module, use `"%requires HttpServer"` in your code. See examples/httpserver.q for an example program using this module

All the public symbols in the module are defined in the HttpServer namespace.

The main classes is (see HttpServerUtil for supporting definitions; note that the HttpServerUtil module's definitions are also imported into Program objects requiring this module):

- HttpServer: this class implements the main interface to the HTTP server provided by this module

See also:

- RestHandler: a module providing a handler framework for this module for implementing server-side REST services
- WebSocketHandler: a module providing a handler framework for this module for implementing server-side WebSocket services
- WebUtil: a module providing higher-level HTTP services using this module as a base as well as providing support for dynamic template-based HTML rendering (ie rendering qhtml files which are a mix of HTML and Qore code)
- JsonRpcHandler: provides infrastructure for implementing JSON-RPC server-side services using the HttpServer module
- YamlRpcHandler: provides infrastructure for implementing YAML-RPC server-side services using the HttpServer module
- XmlRpcHandler: provides infrastructure for implementing XML-RPC server-side services using the HttpServer module
- SoapHandler: provides infrastructure for implementing SOAP server-side services using the HttpServer module

Example:
  #!/usr/bin/env qore
  %requires HttpServer
  %requires Mime
  %new-style
  class MyHandler inherits AbstractHttpRequestHandler {
  hash handleRequest(hash<auto> cx, hash<auto> hdr, *data body) {
  # NOTE: change "%y" to "%N" to get a more readable multi-line output format for container values
  log("request received on %s from %s: context: %y hdr: %y body size: %d byte%s", cx."peer-info".desc, cx."socket-info".desc, cx, hdr, body.size(), body.size() == 1 ? "" : "s");
  return {
  "code": 200,
  "body": sprintf("<title>Test Page</title><body><h1>Qore HTTP server v%s</h1><p>%y on %s PID %s TID %d connection %d</p><p>Qore %s</p></body>", HttpServer::Version, now_us(), gethostname(), getpid(), gettid(), cx.id, Qore::VersionString),
  "close": True,
  };
  }
  }
  sub log(string str) {
  printf("%y: %s\n", now_us(), vsprintf(str, argv));
  }
  const MyHttpPort = 19001;
  # create our handler object
  MyHandler myHandler();
  # create the http server object
  HttpServer hs(\log(), \log());
  # add our handler to the server
  hs.setHandler("my-handler", "/", MimeTypeHtml, myHandler);
  # set our handler as the default handler
  hs.setDefaultHandler("my-handler", myHandler);
  # start a listener
  hash lh = hs.addListener(MyHttpPort);
  # output a log message
  log("started listener on %s", lh.desc);

# HttpServer Module Release Notes

## HttpServer 0.9.4

- added support for sending chunked replies from an InputStream (issue 3702)

## HttpServer 0.3.13.1

- enable remote certificates to be retrieved in listeners and handlers by default; added new API entry points for listeners and handlers with more flexible and saner parameters, deprecated old complex methods (issue 3512)
- shut down dedicated socket conenctions last in order to allow for effective keep-alive implementations with WebSocket for example (issue 3488)
- fixed certificate and key errors for HTTPS listeners to generate user-friendly exceptions (issue 3397)

## HttpServer 0.3.13

- added the `"header-info"` hash to the context argument when calling handlers (issue 3026)
- added support for adding new HTTP methods to the server with the HttpServer::addHttpMethod() method (issue 2805)

## HttpServer 0.3.12.1

- fixed `HEAD` and other responses that cannot have a message body (bug 3116)

## HttpServer 0.3.12.1

- added a minimal substring of string bodies received to the log message when logging HTTP requests
- added logic to allow sensitive data to be masked in log messages (issue 1086)
- added support for the HTTP `"PATCH"` method

## HttpServer 0.3.11.1

- fixed a bug where HttpServer::HttpServer::addListener() would not accept port 0 meaning bind on any random open port (bug 1284)

## HttpServer 0.3.11

- fixed a bug setting the response encoding in HttpServer::HttpServer::setReplyHeaders() where the Socket encoding was not set properly and therefore the encoding in the `Content-Type` in the response header did not necessarily match the encoding of the response
- added support for private keys with passwords
- added the PermissiveAuthenticator class
- moved base code to the HttpServerUtil module (imported with the `reexport` option) and santized APIs, made HttpListener class private/internal again
- fixed a socket / connection performance problem with HTTPS listeners where the SSL connection was being negotiated inline with the accept instead of in the connection thread, thereby blocking new connections from being accepted
- new methods implemented in HttpServer:
  - HttpServer::HttpServer::setListenerLogOptions()
  - HttpServer::HttpServer::setListenerLogOptionsID()
- performance improvement of matching handlers to request URIs (not used for RegExp matching)
- changed the order of path-based handler matching:
  - the old implementation selects the first (in order of their registration) handler whose path matches the request URI, regexp and path matching have the same priority
  - the new implementation has two stages:
    - first, handlers without regexp are considered - if there are some handler with their path matching the request, the one with the longest path is selected
    - otherwise, the first (in order of their registration) handler with regexp that matches the request URI is chosen
  - if no path matches the request, the same logic as before applies
- added HttpServer::addListeners(string, hash, *code, *code, *code, *string, int)
- fixed a bug in HttpServer::HttpServer::addListener() with an integer argument; a UNIX socket was opened instead of a wildcard listener on the given port
- added the HttpServer::HttpServer::listenerStarted() method to allow for reporting when listeners are actually running since they are started asynchronously
- added the `"ssl"` key to the listener socket info hash
- fixed typos causing bugs in HTTP error logging (bug 308)
- fixed bugs handling persistent connections

## HttpServer 0.3.10

- if an error occurs receiving a message with chunked transfer encoding, send the response immediately before reading the rest of the chunked transfer

## HttpServer 0.3.9

- when binding a wildcard address with AF_UNSPEC on Windows with HttpServer::HttpServer::addListeners() and both IPv6 and IPv4 addresses are returned, bind both addresses since Windows doesn't direct the IPv4 requests to the wildcard-bound IPv6 listener

## HttpServer 0.3.8

- moved more message handling code to HttpServer::AbstractHttpRequestHandler so that chunked transfers can be efficiently supported by handlers
- added a minimum body size threshold for `Content-Encoding` compression to HttpServer
- added the following classes to facilitate handling chunked requests and responses:
  - HttpServer::AbstractStreamRequestHandler
  - HttpServer::AbstractStreamRequest
- added support for persistent connections to a given handler to the HTTP server

## HttpServer 0.3.7

- fixed a bug in parsing arguments in parse_uri_query()
- updated parse_uri_query() to accept both `"&"` and `";"` as argument separators
- added support for the `PUT` and `DELETE` methods and for optional message bodies independent of the HTTP method
- fixed handler matching to use a score for each handler and to check dynamic handlers also for a match if the fixed handlers don't provide a perfect match
- fixed response handling to not overwrite the `Connection` header if present from the handler
- fixed response handling to only return error reponses with codes >= 400
- implemented the AbstractHttpSocketHandler handler class to support dedicated socket connections when switching protocols in handlers
- do not add HTML to the msg body from explicit error responses from handlers
- improved HTTP server log messages
- added "family" argument to HttpServer::add*Listener*() methods
- turned of `TCP_NODELAY` by default and use socket shutdown when closing sockets instead
- set the listen backlog queue size to 100 by default (previously was hardcoded internally in Qore to 5)
- use the new ThreadPool class introduced in Qore 0.8.8 to pre-allocate threads to reduce latency for new socket connections
- improved handler matching algorithm
- implemented many performance improvements
- implemented support for simple string path matching when finding a handler for a request
- added static HttpServer::AbstractHttpRequestHandler::redirect() to help in generating 301 Moved Permanently messages

## HttpServer 0.3.6

- fixed a race condition in tracking active requests in dynamic HTTP handlers and removing dynamic HTTP handlers

## HttpServer 0.3.5

- fixed bugs in regexes in the HttpServer::addListeners() and HttpServer::addListenersWithHandler() methods
- updated module example code in summary
- added `"listener-id"` to request context hash

## HttpServer 0.3.4

- added public parse_uri_query() function

## HttpServer 0.3.3

- updated to a user module
- added support for custom redirects from user handlers
- use the Mime module for mime definitions

## HttpServer 0.3.2

- added support for listeners with specific handler lists
- added the ability to manage dynamic content handlers
- implemented content-encoding handling flags in AbstractHttpRequestHandler
- implemented support for `"identity"` encoding (if anyone ever sends it)

## HttpServer 0.3.1

- added IPv6 support in qore 0.8.2

## HttpServer 0.3.0

- added the ability to start and stop listeners on demand

## HttpServer 0.2.9

- updates for new SSL and timeout behavior with with Socket class with qore 0.8.1+
- set socket encoding to UTF-8 by default
- add `"charset=utf-8"` to `Content-Type` header if not already present
- add `"text/html"` to `Content-Type` header if no content-type is given by the handler
- fixed setting X.509 certificate and private key for HTTPS listeners
- require qore >= 0.8.1 for new Socket features

## HttpServer 0.2.8

- converted to hard typing for use with require-types
- require qore >= 0.8.0 for new Socket features

## HttpServer 0.2.7

- set `TCP_NODELAY` on all sockets to ensure that clients get all data before closing the socket, especially in case of errors
- require qore >= 0.7.4 for new Socket features

## HttpServer 0.2.6

- minor fixes for SOAP support
- improved handler matching

## HttpServer 0.2.5

- minor fixes for SOAP support

## HttpServer 0.2.4

- improved `Content-Type` handling
- improved URL/path support

## HttpServer 0.2.3

- bzip2 `Content-Encoding` support

## HttpServer 0.2.2

- basic authentication

## HttpServer 0.2.1

- implementing logic to handle `"deflate"` and `"gzip"` content-encoding
- chunked content-encoding supported for POSTs
- Date: header always sent as per HTTP 1.1 spec

## HttpServer 0.2.0

- modular/multiple listener support added
- https support added

now_us
date now_us()
vsprintf
string vsprintf(string fmt, auto varg)
getpid
int getpid()
gettid
int gettid()
sprintf
string sprintf(string fmt,...)
printf
string printf(string fmt,...)
gethostname
string gethostname()