# mamake and the MAM language #

MAM (Make Abstract Machine) is a simple rule-based make language
that is implemented in just seven four-letter commands and five attributes,
yet allows unlimited flexibility as it can execute arbitrary shell code.
The program implementing MAM, `mamake`,
is a portable C90 program written in a single file, `mamake.c`.
This allows ksh 93u+m,
or other programs using this build system,
to be built using only a standard C compiler and utilities installation
without any other dependencies or complications.

MAM was originally designed by Glenn Fowler at AT&T and intended as an
abstraction layer for `make` implementations such as AT&T `nmake`.
The [original documentation](https://web.archive.org/web/20041227143022/http://www2.research.att.com/~gsf/mam/mam.html)
for MAM specified a more extensive and slightly different language
than was actually implemented in `mamake.c`.
Notably, the `bind` command described there is completely different.
This file documents the MAM implementation that is actually in use.

Since fixing and maintaining AT&T `nmake` proved impractical, `mamake` is
used here as a full `make` replacement, gradually adding some features to
the language to facilitate human maintenance of the `Mamfile`s.

## Table of contents ##

* [General overview of the MAM language](#user-content-general-overview-of-the-mam-language)
    * [Strict and legacy modes](#user-content-strict-and-legacy-modes)
* [MAM variables](#user-content-mam-variables)
    * [Special expansion syntax](#user-content-special-expansion-syntax)
    * [Automatic variables](#user-content-automatic-variables)
* [Commands](#user-content-commands)
    * [Comments](#user-content-comments)
    * [Rules](#user-content-rules)
    * [Referencing prerequisites or previously defined rules](#user-content-referencing-prerequisites-or-previously-defined-rules)
    * [Setting MAM variables](#user-content-setting-mam-variables)
    * [Shell actions](#user-content-shell-actions)
        * [Viewpathing](#user-content-viewpathing)
        * [Execution](#user-content-execution)
        * [Strict level 2+ change](#user-content-strict-level-2-change)
    * [Declaring common code for shell actions](#user-content-declaring-common-code-for-shell-actions)
    * [Binding libraries](#user-content-binding-libraries)
        * […while scanning and sorting leaf directories](#user-content-while-scanning-and-sorting-leaf-directories)
        * […while building the current directory](#user-content-while-building-the-current-directory)
    * [Repeatedly iterating through a block](#user-content-repeatedly-iterating-through-a-block)
* [Appendix: Main changes from the AT&T version](#user-content-appendix-main-changes-from-the-att-version)

## General overview of the MAM language ##

MAM is a simple declarative language, easy to parse for machines and easy to
read for humans, in which targets are defined that correspond to files that
need to be generated or updated, or that are prerequisites. `mamake` reads
build scripts from a file, `Mamfile` by default, from start to finish. As it
encounters `make` *target*…`done` blocks, it ‘makes’ (updates) the *target*s
using the instructions within those blocks. Those instructions may declare
dependencies and/or specify shell command actions.

If a *target* is specified on the command line, `mamake` will update the
actions contained within that target and ignore the rest.

### Strict and legacy modes ###

By default, `mamake` remains fully backward compatible with Mamfiles as
originally generated by AT&T `nmake`. If the `MAMAKE_STRICT` variable is
set, some backward incompatible changes and deprecation warnings are
activated to ensure correct operation and to facilitate human maintenance of
the Mamfiles. A numeric value assigned to `MAMAKE_STRICT` indicates the
backward incompatibility level. The empty value is equivalent to level 1.
These are called the "strict levels" and their absence the "legacy mode",
also known as strict level 0.

Each time backward incompatible changes are introduced to `mamake` that would
break previous Mamfiles, those are made subject to a `MAMAKE_STRICT` value of one
higher than the previous highest one; details of those changes are documented
throughout this file and listed in the appendix below.
This makes it possible to test or backport old code using the current build
system. Current Mamfiles should use the highest strict level available.
The current highest available strict level is **3**.

## MAM variables ##

MAM variables are imported from the environment
or set via `setv` (see below).
They are referenced using a `${`...`}` syntax not dissimilar to `sh`(1),
though the braces are *not* optional.
If an undefined variable is expanded and the variable name is valid in `sh`(1)
syntax, the expansion is left in place unexpanded, otherwise it is removed.
At strict level 2 and up, it is left unexpanded even if it is not a valid
`sh`(1) variable name; this allows POSIX shell expansions like `${foo#*bar}`.

By default, the expansion of MAM variable references is recursive,
i.e., the value may itself contain other variable references.
Beware: there is no reference loop detection; any variable referencing
itself directly or indirectly will cause mamake to crash.
At strict level 2 and up, this (mis)feature is disabled and
variables always expand to their literal values,
and variable references in `setv` only work for previously defined variables.

Note that, in shell actions (see `exec` below), MAM variables are expanded
before the script ever reaches the shell. Consequently, the use of single
shell quotes `'`…`'` does not stop their expansion as you might expect;
in fact, they ensure that only MAM variable expansion happens,
avoiding any potential conflicts with the shell expansion syntax.

### Special expansion syntax ###

In `${`*variable*`?`*str*`?`*x*`?`*y*`?}`,
if the string value of the *variable* is identical to *str* or if *c* is `*`,
then the value *x* is substituted, otherwise *y*.
The *x* and *y* values may result from nested variable references.
The last `?` is optional.

In `${`*variable*`-`*x*`}`, the value of *variable* is substituted
if it is defined and non-empty, otherwise the value of *x* is substituted.
In `${`*variable*`+`*x*`}`, *x* is substituted if the value of
*variable* is defined and non-empty, otherwise the reference is removed.
Note that, unlike in `sh`(1), no distinction is made between an undefined
variable and a defined variable with an empty value.

### Automatic variables ###

The following variables are set and updated automatically.
They are inspired by similar variables in `make` implementations,
but since `mamake` is different, so are these variables.

`${@}` is the name of the rule currently being made.

`${<}` is the name of the prerequisite rule (`make`…`done` or `prev`)
that was *last* processed within the current rule.

`${^}` is a space-separated list of names of all the current rule's
previously processed prerequisites.

`${?}` is a space-separate list of the current rule's previously processed
prerequisites that have been updated by a shell action (see `exec` below)
during the current `mamake` run. Prequisites that were already up to date,
or prerequisites that do not contain a shell action, are not included.

## Commands ##

MAM commands have the following basic form:

*command* [ *argument* [ *operand string* ] ]

The *command* name consists of four lower-case letters.
Unrecognized commands or attributes are an error.
The *argument* is a single word.
The *operand string* is any arbitrary text until the end of the line.

### Comments ###

`note` is the comment command and is ignored.
In the legacy mode, `info` and `meta` are also ignored.

### Rules ###

`make` *target* [ *attribute* ... ]    
`done` [ *target* ]

A `make`...`done` block defines the rule named *target* using the other commands described here.
Unless the `virtual` attribute is used, *target* names the pathname of the file generated or referenced by the rule.

`mamake` processes the commands within the block if the *target* is out
of date or if the rule has the `virtual` attribute (see below).

The *target* may be repeated as the operand to the `done` command.
In that case, it is matched against the current `make` *target* and
any mismatch will produce a "mismatched done statement" error.
If it is omitted, the current `make` *target* is assumed.

Dependencies may be defined in two ways:
1. By nesting `make`...`done` blocks:
   the enclosing rule is the parent
   and the enclosed rules are the prerequisites.
2. By using the `prev` command (see **Referencing previously defined rules** below)
   to reference a previous `make`...`done` block.
   The dependency is defined as if that block were repeated at the `prev` command's location.

If the block contains one or more `exec` commands (see **Shell actions** below),
the `done` command executes the shell script defined by them.

A `make`...`done` block may lack any `exec` action and, if it does not have any
dependencies of its own, it may even be empty; this has the effect of merely
declaring a dependency on a prerequisite file, such as a source code file that
comes with the distribution or a file generated by a previously run Mamfile.

Making a prerequisite that is currently being made, or one that has already
been made, produces a warning; at strict level 3 and up, this is an error.

One or more *attribute*s may be specified by appending them to the `make` command.
(At strict levels \< 2, they may also be appended to the `done` command; the effect
is the same either way. At strict level 1, this produces a deprecation warning.)
Attributes apply to the current rule only and do not propagate down to nested rules.
The following *attribute*s are available:
* `dontcare`: Marks files that do not need to exist.
  If the file exists then its last-modified timestamp is checked and propagated,
  otherwise it is silently ignored. 
* `ignore`: The timestamp associated with the *target* is ignored in dependency resolution.
* `implicit`: Marks the current rule as an implicit prerequisite of the enclosing parent rule.
  An implicit prerequisite can make the parent rule out of date without triggering the parent action.
  Implicit prerequisites usually correspond to `#include` prerequisites.
  For example, if `foo.o` is generated from `foo.c` and `foo.c` includes `foo.h`,
  then `foo.h` should be marked as an implicit prerequisite of `foo.c`
  so that touching `foo.h` does not make `foo.c` out of date while making `foo.o` out of date.
* `notrace`: Disables echoing (xtrace) of shell action commands.
  This does not disable the trace header for the containing rule (see *Shell actions* below).
* `virtual`: Marks a rule that is not associated with any file.
  The commands within are executed every time the rule is processed.
  By convention, a virtual rule with target `install` performs pre-installation.

At strict level 1 and up, specifying the following *attribute*s is
deprecated and will produce a warning; at strict level 2 and up,
specifying these is an error.

* `archive`: Ignored.
  Historically used to mark the generation of an `ar`(1) archive.
* `generated`: Marks rules that produce output files generated by a shell action.
  The explicit assignment of this attribute is ignored at strict level 1.
  The `exec` command implicitly assigns this attribute.
  If a rule has this attribute, other rules dependent on this rule
  will avoid applying viewpathing based on this rule.
* `joint`: Ignored.
  Historically used to mark one of a group of rules that are built by a single shell action.

### Referencing prerequisites or previously defined rules ###

`prev` *target* [ *attribute* ... ]

The `prev` command is used in two ways:

1. If *target* matches a previously defined rule, `prev` adds a dependency on that rule to the current rule.
   This can be used to make a rule a prerequisite of multiple `make`...`done` blocks without repeating the rule.
   No attributes should be given for this use of `prev`, because the attributes of the referenced rule are used.
   Superfluous attributes are an error at strict level >= 1 and ignored in the legacy mode.

2. If *target* does not match a previously defined rule, the following applies.
   In the legacy mode, `prev` creates an empty dummy
   rule and ignores the *attribute*s; this is for backward compatibility.
   At strict level 1 and up,
   `prev` creates a rule that declares a dependency on a prerequisite file named by *target*
   in a manner equivalent to an empty `make`...`done` block,
   with the optional *attribute*s applied to the new rule, and
   a nonexistent prerequisite is an error unless a `virtual` or `dontcare` attribute is given.
   Declaring a dependency on a prerequisite that is currently being made
   (i.e.: directly or indirectly within that prerequisite's block) produces
   a warning; at strict level 3 and up, this is an error.

### Setting MAM variables ###

`setv` *variable* [ *defaultvalue* ]

Defines a new MAM *variable*, optionally assigning the initial *defaultvalue*.
If the variable already has a value, the `setv` command is ignored; assigning a new value is not possible.
When `mamake` starts, it imports all environment variables as MAM variables,
so any variable's default value can be overridden by exporting an environment variable by its name.

If the strict level is less than 2 and the *defaultvalue* begins and ends with
double quotes (`"`), those quotes are discarded,
though double quotes elsewhere in the value are not treated specially.
The value is otherwise taken entirely literally.

When the *variable* is `CC`, mamake runs the `mamprobe` script
to probe the C compiler for flags and features,
or uses that script's stored results if not outdated.
The results are stored as a series of `setv` commands
in a file in the directory `${INSTALLROOT}/lib/probe/C/mam`,
the file name being a hash of full path to the compiler indicated by `${CC}`.
That results file is then read and included in the current Mamfile
as if it followed the `setv CC` command.

### Shell actions ###

`exec` `-` *code*

One or more `exec` commands within a `make`...`done` block
define a shell script that is run to generate the *target*.
The argument following `exec` is ignored; by convention it is `-`.
Each `exec` command appends a line of code to the shell script for the current rule.
It is customary for a rule's `exec` commands to be contiguous, but not necessary.

Before adding each line of code to the script,
MAM variable references (see **MAM variables** above)
are expanded; their literal values are inserted into the *code* line
(beware: no quoting is applied!).
Because variables are expanded when the line is encountered, the value
of the automatic variables for any `exec` line depends on the position
of the line in the rule.

#### Viewpathing ####

After MAM variable expansion, *viewpathing* is applied.
The first colon-separated element of `${VPATH}` is considered
the object code directory and the second the source code directory;
viewpathing provides the first with a vew to the second.
Viewpathing applies two transformations.

The first is *prerequisite replacement*.
Each word (separated by whitespace, `;`, `(`, `)`, `` ` ``, `|`, `&` or `=`)
is searched for in the current rule's prerequisites,
and if it matches the name of a non-generated prerequisite,
it is replaced by the canonical path to it in the source directory,
ensuring that things like prerequisite headers are found.

The second is *include flag duplication*.
After every argument that looks like a compiler include directory path (i.e.,
starting with `-I`) with a relative path name (i.e., a directory path that does
*not* start with a `/`), another argument starting with `-I` is inserted with
that path name prefixed by the path to the source directory.  This mechanism
ensures that headers are found both in the object directory and in the source
directory.  It is processed regardless of the command; for example, it also
works for compiler flags passed to `iffe`(1).

Note that shell quotes are *not* treated specially. If an argument starting
with `-I` ends in a shell quote without preceding whitespace, that trailing
quote is repeated along with the prefixed path and causes a syntax error.

#### Execution ####

When `mamake` encounters the `done` command,
the script is executed by the shell whose path is in the `SHELL` environment variable
or, absent that, by `sh`(1).
Each shell action is run in a new instance of the shell.

Before executing the script, an empty line followed by a trace header
in the following format is written to standard error:

    # path/to/Mamfile: startline-endline: rule

During script execution, shell action comands are traced using the
shell's xtrace option, unless the rule has the `notrace` attribute.

#### Strict level 2+ change ####

At strict level 2 and up, `mamake` turns off global pathname expansion
(globbing) using `set -f` to make safer MAM variable expansion and shell
field splitting possible; this avoids unexpected pathname expansion if
a value contains `?`, `*` or `[`.
A shell action can override this using `set +f`;
this should only be done for individual commands in a `(`subshell`)`.

### Declaring common code for shell actions ###

`shim` `-` *code*

One or more `shim` commands declare a ‘shim’: a common section of `sh`(1)
code that will be automatically inserted in front of subsequent shell
actions upon execution. Like `exec`, `shim` combines multiple lines of
*code* into one section, with MAM variables expanded at declaration time and
viewpathing applied at execution time. The effect of `shim` is global.

One use case is defining a shell function that each shell action can call.
For example, such a function might invoke the compiler with a series of
compiler flags common to all compiler invocations, with other flags added via
arguments to the function, so the common flags do not need to be repeated in
every shell action.

Only one shim is active at a time, but it can be redefined. When the next
`exec` command is encountered, the shim is marked ready for use. The next
time a `shim` command is encountered after that, it starts a new shim from
scratch that affects subsequently executed shell actions.
A single `shim -` deactivates the shim.

### Binding libraries ###

`bind` `-l`*libraryname* [ `dontcare` ]

These commands are scanned for while sorting leaf directories for recursive
building, and executed as normal commands while building the current directory.

#### …while scanning and sorting leaf directories ####

Any leaf directories with names that start with `INIT` will always be built before
all others. For all other leaf directories, the presence of any `bind`
command of the form `bind -lfoo` anywhere in a leaf
directory's Mamfile causes the leaf directory named `libfoo` (if it exists)
to be a prerequisite of that leaf directory.
The prerequisite leaf directory does not have to be in the same parent
directory, as long as it is processed as part of the same scan.
At this stage, attributes are ignored.

#### …while building the current directory ####

An argument of `-l`*libraryname*
causes a MAM variable `mam_lib`*libraryname* to be defined (see **MAM variables** above).
The variable will contain either the compiler argument for linking to the library *libraryname*
(either the `-l`*libraryname* flag, or the full path in case of a static library)
or, if the `dontcare` attribute is specified, possibly the empty string.
Any library dependencies are also included (see below).
This can be used both for AST libraries shipped with the distribution and for system libraries.
For each corresponding *.a library archive dependency built previously,
its time stamp is checked and the current target is marked as outdated if it is
newer, as if a `prev` had been executed for it.

The variable set by `bind` is global, but the marking of the target as
outdated applies to the current rule only, so it may be necessary to
repeat a `bind` command when statically linking executables that depend
on a library, otherwise they may not be relinked when the library changes.
The `mam_lib`*libraryname* variable will not be regenerated when repeating a `bind`.

There is also a mechanism to communicate library dependency information across Mamfiles and `mamake` invocations.
If a file named *libraryname*`.req` in the current directory
or an `${INSTALLROOT}/lib/lib/`*libraryname* file
exists, `mamake` processes each of the words in the form `-l`*libraryname* in its contents
as if they were arguments to `bind` commands
and the resulting values are appended to the value of `mam_lib`*libraryname*
as dependencies separated by spaces.
`mamake` does not create these dependency files;
they are expected to be generated by Mamfile shell actions (see **Shell actions** above).

If no such dependency file exists, and the `dontcare` attribute is added,
then `mamake` compiles a small test program on the fly to check if the library exists;
if this fails, the `mam_lib`*libraryname* variable will be emptied.

Any `bind` command whose argument does not start with `-l` is ignored.

### Repeatedly iterating through a block ###

`loop` *variable* *word* [ *word* ... ]    
`done`

`loop` reads the lines contained between it and the corresponding `done`
repeatedly with a named *variable* set to each of the *word*s. The lines
are processed as part of the rule containing the loop.
The *variable* is restored to its previous state after the loop completes.

Note that `loop` causes repeated reading and processing of Mamfile lines,
*not* necessarily repeated execution. For instance, a loop can be used to
consolidate repetitive `make`…`done` rules. However, each rule is only made
once and subsequent rules by the same name are an error at strict level 3
and up, or skipped over at strict \< 3. So it only makes sense to do this
if the contained make target names are modified by the expansion of the
iteration *variable*.

`loop` requires a seekable input file (i.e.: not a pipe).

## Appendix: Main changes from the AT&T version ##

Compared to the original AT&T version, ksh 93u+m made a number of
changes to `mamake` that facilitate correct operation and make it easier to
maintain Mamfiles by hand. The following lists the important changes.
* Introduced the notion of ‘strict mode’ levels that tidy things up by
  activating some backward incompatible changes and deprecation warnings.
* Indentation and word separators may use any whitespace (e.g. tabs), not only spaces.
* Fixed a bug that stopped a rule marked `virtual` (not associated with
  any file) from being executed if a file by that rule's name exists.
* Unrecognized commands and rule attributes throw an error instead of being silently ignored.
* It has been made optional to repeat the `make` target after `done`.
* The `notrace` attribute was added to disable xtrace for a rule's shell action.
* The automatic variables `${@}`, `${<}`, `${^}` and `${?}` have been added.
* An iteration block command, `loop`…`done`, has been added.
* A command to set common code for shell actions, `shim`, has been added.
* Attempting to make a rule that has already been made produces a warning.
* Attempting to declare a dependency on a rule currently being made produces a warning.
* **At strict level 1 and up:**
    * Appending attributes to `done` instead of `make` is deprecated
      and produces a warning.
    * The ignored `archive` and `joint` attributes are deprecated.
    * Explicitly specifying the `generated` attribute is deprecated.
    * The dummy `info` and `meta` commands are unavailable instead of ignored.
    * The `prev` command may be used instead of an empty `make`...`done`
      block to declare a simple prerequisite with possible attributes.
    * When `prev` references a previously processed target,
      attributes are an error instead of being ignored.
    * The legacy `silent` and `ignore` command prefixes are unavailable.
* **At strict level 2 and up:**
    * Appending attributes to `done` instead of `make` is an error.
    * The `archive` and `joint` attributes are unavailable.
    * Explicitly specifying the `generated` attribute is an error.
    * All variable references are expanded to their literal values
      without scanning the values for recursive variable references.
    * `setv` does not remove leading and trailing `"` from the value.
    * Shell actions have pathname expansion (globbing) disabled by default.
* **At strict level 3 and up:**
    * Attempting to make a rule that has already been made is an error.
    * Attempting to declare a dependency on a rule currently being made is an error.
