% -*-LaTeX-*-
\documentclass[10pt]{article}
\usepackage{amsmath,amssymb}
\usepackage{supertabular}
\usepackage{longtable}
\usepackage{geometry}
\usepackage{verbatim}
\usepackage{alltt}
\usepackage{hevea}
\usepackage[usenames]{color}
\usepackage{hyperref}
\usepackage{booktabs}
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}} % For pandoc lists

%\usepackage{ctable} %for pandoc-generated table
\newcommand{\ctable}[1][]{}
\newcommand{\FL}{\begin{center}\begin{tabular}{ll}}
\newcommand{\ML}{\\\hline}
\newcommand{\LL}{\end{tabular}\end{center}}

%\begin{htmlonly}
%\renewcommand{\fbox}[1]
%{\begin{cellstyle}{text}{}#1\end{cellstyle}}
%\end{htmlonly}


\geometry{%
a4paper,
% letterpaper,
dvips,
twoside,
%left=22.5mm, right=22.5mm,
left=25.4mm, right=25.5mm,
top=20mm,  bottom=35mm,
}
\usepackage{parskip}


%\newcommand{\mybf}[1]{{\pmb{\texttt{#1}}}}
\newcommand{\mybf}[1]{{{\texttt{#1}}}}
%\newcommand{\mybackslash}{\mbox{$\backslash$}}
%\newcommand{\mybackslash}{\mbox{\tt\char'134}}
%\newcommand{\mylb}{\mbox{\tt\char'173}}
%\newcommand{\myrb}{\mbox{\tt\char'175}}

%\newcommand{\mykw}[1]{{\color{Mahogany}{\pmb{\texttt{#1}}}}}
\newcommand{\mykw}[1]{{\color{Mahogany}{{\texttt{#1}}}}}
\newcommand{\mysym}[1]{{\color{RoyalBlue}{{\texttt{#1}}}}}
%\newcommand{\mybackslash}{\mbox{$\backslash$}}
\newcommand{\mybackslash}{\mbox{\tt\char'134}}
\newcommand{\mylb}{\mbox{\tt\char'173}}
\newcommand{\myrb}{\mbox{\tt\char'175}}

\ifhevea
\newcommand{\tick}{\mbox{$*$}}
\newcommand{\cross}{\mbox{$-$}}
\else
\newcommand{\tick}{\mbox{$\surd$}}
\newcommand{\cross}{\mbox{$\times$}}
\fi
\newcommand{\fakeindex}{\Box}


\newcommand{\TODO}[1]{\textbf{[TODO{ #1}]}}

%\ifdraftversion
\newcommand{\mlabel}[1]{\label{#1}}
\newcommand{\mref}[1]{\ref{#1}}
\newcommand{\mpageref}[1]{\pageref{#1}}
%\newcommand{\mlabel}[1]{\label{#1}\raisebox{1.85ex}[0pt][0pt]{\makebox[0pt]{}}}
%\newcommand{\mref}[1]{\ref{#1}\raisebox{1.85ex}[0pt][0pt]{\makebox[0pt]{}}}
%\newcommand{\mpageref}[1]{\pageref{#1}\raisebox{1.85ex}[0pt][0pt]{\makebox[0pt]{}}}
%\else
%\newcommand{\mlabel}[1]{\label{#1}\raisebox{1.85ex}[0pt][0pt]{\makebox[0pt]{\scriptsize#1}}}
%\newcommand{\mref}[1]{\ref{#1}\raisebox{1.85ex}[0pt][0pt]{\makebox[0pt]{\fontseries{m}\selectfont\scriptsize#1}}}
%\newcommand{\mpageref}[1]{\pageref{#1}\raisebox{1.85ex}[0pt][0pt]{\makebox[0pt]{\fontseries{m}\selectfont\scriptsize#1}}}
%\fi

\newcommand{\myparagraph}[1]{\subsection{#1}}
%\newcommand{\myparagraph}[1]{\vspace{0.5\baselineskip}\par\noindent{\textbf{#1}}\ }
\newcommand{\mysubparagraph}[1]{\vspace{0.5\baselineskip}\par\noindent{\emph{#1:}}\ }

\newcommand{\lparagraph}[1]{\vspace{0.4\baselineskip}\par\noindent{\normalsize\bfseries\itshape{#1}}\quad}
\newcommand{\myaspect}[1]{\lparagraph{\textit{#1} }}

 
\include{test10}
\include{test10.2}
\include{test10.meta}
\include{test8}
\include{test7}
\include{test13d}
%\include{test13}
\include{test10st}
\include{test17.10}
\include{binding.1}
\include{binding.2}
\include{binding.3}
\include{binding.4}
\include{binding.5}
\include{binding.6b}

\input{../src/version}

\ifhevea
\newcommand{\mydagger}{**}
\newcommand{\myLaTeX}{LaTeX}
\else
\newcommand{\mydagger}{+}
\newcommand{\myLaTeX}{\LaTeX}
\fi

\begin{document}
\title{Ott: Tool Support for
  Semantics \\User Guide\\
\ \mbox{version \ottver{}}
}
\author{Peter Sewell$^*$ \qquad Francesco Zappa Nardelli$^\mydagger$ \qquad Scott Owens $^*$ \\[5mm]
with  Gilles Peskine$^*$, Tom
  Ridge$^*$,\\ Susmit Sarkar$^*$, and Rok Strni\v sa$^*$\\[5mm]
 ${}^*$University of Cambridge \qquad
 ${}^\mydagger$INRIA}
\date{\today{}}
\maketitle

{\small
\tableofcontents
\listoffigures
}
\newpage  
\section{Introduction}\mlabel{a1}%
Ott is a tool for writing definitions of programming languages and
calculi. 
%
It takes as input a definition of a language syntax and semantics, in
a concise and readable ASCII notation that is close to what one would
write in informal mathematics.  It generates output:
\begin{enumerate}
\item a \myLaTeX{} source file that defines commands to build a typeset version of
  the definition;
\item a Coq version of the definition; 
\item a HOL version of the definition;
\item an Isabelle/HOL version of the definition; 
\item a Lem version of the definition; 
\item an OCaml version of the syntax of the definition.
\end{enumerate}
Additionally, it can be run as a filter, taking a
\myLaTeX{}/Coq/Isabelle/HOL/Lem/OCaml  source file
with embedded (symbolic) terms of the defined language, parsing them and replacing
them by typeset terms.


This document is a user guide for the tool.  The papers
\begin{itemize}
\item
\ahref{http://www.cl.cam.ac.uk/users/pes20/ott/ott-jfp.pdf}{Ott: Effective Tool Support for the Working Semanticist}. Peter Sewell, Francesco Zappa Nardelli, Scott Owens, Gilles
Peskine, Thomas Ridge, Susmit Sarkar, Rok Strni\v sa. 
Journal of Functional Programming 20(1):71-122, 2010 \cite{ott-jfp}.

\item
\ahref{http://www.cl.cam.ac.uk/users/pes20/ott/paper.pdf}{Ott: Effective Tool Support for the Working Semanticist}. Peter Sewell, Francesco Zappa Nardelli, Scott Owens, Gilles
Peskine, Thomas Ridge, Susmit Sarkar, Rok Strni\v sa. 
ICFP'07 \cite{ott-sub}.

\end{itemize}
give an overview of the project, including discussion of motivation,
design decisions, and related work, and one should look at that together
with this manual.  The project web page
\begin{quotation}
\ahrefurl{http://www.cl.cam.ac.uk/users/pes20/ott/}
\end{quotation}
links to the github source repository, with a
BSD-style licence. It also has a range of examples, including
untyped and simply typed CBV lambda calculus, ML polymorphism, various
first-order systems from Pierce's TAPL~\cite{Pierce:TypeSystems}, the
POPLmark F$_{<:}$ language~\cite{poplmark}, a module system by
Leroy~\cite[\S4]{Leroy-generativity} (extended with a term language and an
operational semantics), the LJ Java fragment and LJAM Java module
system~\cite{ljam-sub}, and a substantial fragment of OCaml.



Our main goal is to support work on large programming language
definitions, where the scale makes it hard to keep a definition
internally consistent, and hard to keep a tight correspondence between a
definition and implementations.
%
We also wish to ease rapid prototyping work with smaller calculi,
and to make it easier to exchange definitions and definition fragments
between groups.
%
%
Most simply, the tool can be used to aid completely informal \myLaTeX{} mathematics.
Here it permits the definition, and terms within proofs and
exposition, to be written in a clear, editable, ASCII  notation, without \myLaTeX{}
noise. It generates good-quality typeset output. 
By parsing (and so sort-checking) this input, it quickly catches a
range of simple errors, e.g.~inconsistent use of judgement forms or
metavariable naming conventions. 
%
That same input, extended with some additional data, can be used to generate formal definitions for
Coq, HOL, Isabelle, and Lem.  It should thereby enable a smooth transition
between use of informal and formal mathematics.  Further, the
tool can automatically generate definitions of functions for free
variables, single and multiple substitutions, subgrammar checks
(e.g.~for value subgrammars), and binding auxiliary functions.
Ott supports a `fully concrete' representation, sufficient
for many examples but not dealing with general alpha equivalence.  
An experimental Coq backend generates definitions in locally-nameless style for a subset of the Ott metalanguage.
 The OCaml backend 
generates type definitions that may be useful for developing a complete
implementation of the language, together with the functions listed
above. It does not generate anything for inductively defined relations
(the various proof-assistant code extraction facilities can
sometimes be used for that).
%
Our focus here is on the problem of writing and editing language
definitions, not (directly) on aiding mechanized proof of metatheory. If one
is involved in hard proofs about a relatively stable small calculus
then it will aid only a small part of the work (and one might choose
instead to work just within a single proof assistant), but for larger
languages the definition is a more substantial problem --- so much so
that only a handful of full-scale languages have been given complete definitions. We
aim to make this more commonplace, less of a heroic task.

% getting started with Ott
\input{README}


\section{A minimal Ott source file: the untyped CBV lambda calculus}\mlabel{a51}%
Fig.~\ref{a45} shows an Ott source file for an untyped call-by-value
(CBV) lambda calculus.  This section explains the basic features that
appear there, while in the following sections we show what must be
added to generate typeset output, proof assistant definitions, and
other things.
\begin{figure}
\fbox{\small\begin{minipage}{\textwidth}
\input{test10.0.alltt}
\end{minipage}}
\caption{Source: \texttt{test10.0.ott}\protect\label{a45}}
\end{figure}
The figure is colourised,  with Ott keywords like \mykw{this} and Ott
symbols such as \mysym{|} and \mysym{::}.  Other user-specific input
appears like \texttt{this}. Any text between \% and the next newline is
discarded as a comment.

At the top of the figure, the \mykw{metavar} declaration introduces
a sort of \emph{metavariables} \texttt{termvar} (with synonym \texttt{x}), for term
variables.
%
The following \mykw{grammar}  introduces two grammar rules, one for terms, with
 \emph{nonterminal root}
 \texttt{t}, and one for values \texttt{v}.
This specifies the concrete syntax of object-language terms,
the abstract syntax representations for proof-assistant mathematics,
and the syntax of symbolic terms to be used in semantic rules.

Each rule has a rule name prefix (e.g.~\texttt{'t\_'}) and then a list
of productions.  Each production, e.g.
\begin{alltt}
  \mysym{|} \mybackslash{} x . t      \mysym{::}  \mysym{::} Lam
\end{alltt}
specifies a syntactic form as a list of elements, here `\verb+\+',
`\verb+x+', `\verb+.+', and `\verb+t+', each of which is either a
metavariable (the `\verb+x+'), a nonterminal (the `\verb+t+'), or a
terminal 
(\texttt{\mybackslash} \texttt{.} \texttt{(} \texttt{)} \texttt{\mylb} \texttt{\myrb} \texttt{/} \texttt{-->}). 
%
Within productions all elements must be whitespace-separated, so that
the tool can deduce which are terminals.  In the symbolic terms in
the semantic rules below, however, whitespace is required only where necessary.
%Terminals are not required to be declared in the \texttt{terminals}
%grammar (all strings that are not metavariables or nonterminals are
%regarded as terminals) but, if they are, then \myLaTeX{} pretty printing
%information for them can be specified there.  In this example the
%`\verb+\+' is included in the \texttt{terminals} grammar and will be
%\myLaTeX pretty printed as a $\lambda$, whereas the `\verb+.+' is not
%included and will be pretty printed in a default form. 
A few terminals have to be quoted (with \verb+''+) if they appear in a grammar, e.g. to
use \texttt{|} as an object-language token, as they are part of the Ott syntax, but they
do not have to be quoted at usage points. 
%
(If one accidentally omits inter-token whitespace in the grammar, the
output of Ott can be surprising.  This is best diagnosed by looking at
the colourised ASCII or \myLaTeX{} output from Ott.)

Metavariables and nonterminals can be formed from the specified
metavariable and nonterminal roots by appending a suffix, e.g.~the
nonterminal \verb+t'+ in the \verb+App+ and \verb+Tsub+ productions. 




Between the \mysym{::}'s is an optional meta flag \mykw{M} or \mykw{S}.  Non-meta
productions give rise to clauses of datatype definitions in the
Isabelle/Coq/HOL output, whereas meta productions do not.  Later, we
will see how the user can specify how meta syntax should be translated
away when generating proof assistant output.  The two flags \mykw{M}
and \mykw{S} are identical except that productions with the latter are 
admitted when parsing example concrete terms; the \mykw{S} tag is thus
appropriate for lightweight syntactic sugar, such as productions for
parentheses.  One can also use another flag, e.g.~\mykw{X}, along with the command-line option \verb+-tex_suppress_category+,  to suppress 
productions in the generated LaTeX.

Each production has a production name (e.g.~\verb+t_Lam+), composed of
the rule name prefix (here~\verb+t_+) and the production name kernel
that follows the \mysym{::}'s (here~\verb+Lam+).  The production name is
used as a constructor name in the generated Isabelle/Coq/HOL. 

The tool supports arbitrary context-free grammars, extended with
special constructs for list forms (c.f.~\S\mref{a61}). 


Following the \mykw{grammar} in this example is a \mykw{subrule}
declaration
\begin{alltt}
  \mykw{subrules}
    v \mysym{<::} t
\end{alltt}
declaring that the \verb+v+ grammar rule (of values) is a
subgrammar of the \verb+t+ rule (of terms).  The tool checks that
there is in fact a subgrammar relationship, i.e.~that for each
production of the lower rule there exists a production of the higher
rule with corresponding elements (up to the subrule relation).  
The subrule declaration means that, in the semantic rules below, we
will be able to use \verb+v+'s in places where the grammar specifies \verb+t+'s.
In the generated Isabelle/Coq/HOL for this example only one free
datatype will be generated, for the \verb+t+ rule, while for the \verb+v+
rule we generate an \verb+is_v+ predicate over the \verb+t+ type.  Usages of
\verb+v+ nonterminals in the semantic rules will have instances of this
predicate automatically inserted.

Finally, we give a collection of definitions of inductive relations.
In this example there is just one family of definitions (of
operational judgements), introduced by the \mykw{defns} \verb+Jop+; it contains just one
definition of a relation, called \verb+reduce+.  
In general there may be many \mykw{defns} blocks, each of which introduces a
mutually recursive collection of \mykw{defn}s. 
The relation definition
\mykw{defn}\verb+ ...+
also includes a grammar production specifying how elements of the
relation can be written and typeset, here
\begin{verbatim}
  t1 --> t2
\end{verbatim}
As in the main grammar, the tokens of this syntax definition in the
header must be space-separated, but usages of the syntax generally
need not be.
Syntax rules for each family of
judgements, and for their union, are implicitly generated. 
The relation definition is given by a sequence of inference rules,
each with a horizontal line separating a number of premises from a
conclusion, for example as below.
\begin{alltt}
    t1 --> t1'
\mysym{    -------------- ::} ctx_app_arg
    v t1 --> v t1'
\end{alltt}
The conclusion must be a symbolic term of the form of the judgement being
defined.
In simple cases (as here) the premises can be symbolic terms of the
form of any of the defined judgements.  More generally (see
\S\ref{a53}) they can be symbolic terms of a user-defined
\mykw{formula} grammar, or in-line embedded prover code. 
Each rule
 has a name, composed of a definition family prefix
(here empty), a definition prefix (here also empty) and a kernel
(the~\verb+ctx_app_arg+). 
%




The symbolic terms in semantic rules are parsed with a scannerless parser, built
using parser combinators over character-list inputs.  The parser
searches for all parses of the input.  If none are found, the ASCII
and TeX output are annotated \texttt{no parses}, with a copy of the
input with \texttt{***} inserted at the point where the last token was
read.  This is often at the point of the error  (though if, for
example, a putative dot form is read but the two element lists cannot
be anti-unified, it will be after the point of the error). 
If multiple parses are found, the TeX output is annotated
\texttt{multiple parses} and the different parses are output to the
console in detail during the Ott run.  
%
If the option \texttt{picky\_multiple\_parses} is set to
\texttt{true}, multiple parses are always reported.  If it set to
\texttt{false}, a symbolic term is considered ambiguous only if two
different parses compile to different strings (for a target).
%
The parser combinators use memoization and continuation-passing to
achieve reasonable performance on the small symbolic terms that are
typical in semantic rules.  Their performance on large (whole-program
size) examples is untested.
%
To resolve ambiguity one can add metaproductions for parentheses (as
in Fig.~\mref{a45}), or 
production-name annotations in particular symbolic terms,
e.g.~the \verb+:t_tsub:+ in the \verb+AppAbs+ rule of the POPLmark
example, 
\texttt{test7.ott}. %\S\mref{a27}.
  There is currently no support for precedence
or associativity.


This file is included in the distribution as
\texttt{tests/test10.0.ott}. It can be processed by executing
\begin{alltt}
   bin/ott -i tests/test10.0.ott
\end{alltt}
from the main directory.  This simply reads in the file, checking that
it is well-formed.  Adding options:
\begin{alltt}
   bin/ott -show\_sort true -show\_defns true -i tests/test10.0.ott
\end{alltt}
it echos a colourised version to the screen,
with metavariables in red, nonterminals
  in yellow, terminals in green, and object variables in white. 
The colourisation uses vt220 control codes; if they do not work on
your screen add \texttt{-colour false} to the middle of the command
line. To suppress the echo of the definition, add
\texttt{-show\_sort false} and \texttt{-show\_defns false}.


\subsection{Index variables}
In addition to the \mykw{metavar} declarations above, the user can declare any number of distinguished \emph{index}
metavariables, e.g. by:
\begin{alltt}
  \mykw{indexvar} index\mysym{,} i\mysym{,} j\mysym{,} n\mysym{,} m \mysym{::=} \mysym{\mylb\mylb{}} \mykw{isa} num \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{coq} nat \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{hol} num \mysym{\myrb\myrb{}} 
\end{alltt}
Given such a declaration, \verb+index+, \verb+i+, \verb+j+, \verb+n+
and \verb+m+ can be used in suffixes, e.g.~in the production
\begin{alltt}
   \mysym{|}  ( t1 , \mysym{....} , tn )           \mysym{::} \mysym{::} Tuple
\end{alltt}
There is a fixed ad-hoc language of suffixes, including numbers, primes, and index variables (see \S\mref{a17}).
Index metavariables cannot themselves be suffixed.




\section{Generating \myLaTeX}\mlabel{a58}%
The example from the previous section can already be used to generate
\myLaTeX, for example by executing 
\begin{alltt}
   bin/ott -i tests/test10.0.ott -o out.tex 
\end{alltt}
to produce a \myLaTeX{} file \texttt{out.tex}.  One often needs to
fine-tune the default typesetting, as illustrated in 
Figure~\mref{a46} (the Ott source) and Figure~\ref{a49}
(the resulting \myLaTeX{}).  
(The latter was built using the additional option \verb+-tex_show_meta false+, to 
suppress display of the metaproductions.)
\begin{figure}
\fbox{\small\begin{minipage}{\textwidth}
\input{test10.2.alltt}
\end{minipage}}
\caption{Source: \texttt{test10.2.ott}\protect\label{a46}}
\end{figure}
%
\newcommand{\testTenTwoallImage}{
\begin{toimage}
\begin{minipage}{\textwidth}
\testTenTwoall
\end{minipage}
\end{toimage}%
\imageflush}
%
\begin{figure}
\testTenTwoallImage
\ifhevea\else 

\hrule\fi
\caption{Generated \myLaTeX: \texttt{test10.2.tex}\protect\label{a49}}
\end{figure}
%
%  
The source file has three additions to the previous file.  
Firstly, the \mykw{metavar} declaration is annotated with a
specification of how metavariables should be translated to \myLaTeX:
\begin{alltt}
  \mykw{metavar} termvar\mysym{,} x \mysym{::=}   
    \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}mathit\mylb{}\mysym{[[}termvar\mysym{]]}\myrb{} \mysym{\myrb\myrb{}}
\end{alltt}
Inside the   \texttt{\mysym{\mylb\mylb{}} \mykw{tex} }$\ldots$\texttt{ \mysym{\myrb\myrb{}}}
is some \myLaTeX{} code
\texttt{\mybackslash{}mathit\mylb{}\mysym{\$[[}termvar\mysym{]]\$}\myrb{}} 
giving the translation of a \texttt{termvar} or \texttt{x}.  Here they
are typeset in math italic (which in fact is also the default).
Within the translation, the metavariable itself can be mentioned
inside double square brackets \texttt{\mysym{[[} }$\ldots$\texttt{ \mysym{]]}}.

Secondly, there is a grammar for a distinguished nonterminal root
\mykw{terminals}, with a 
\texttt{\mysym{\mylb\mylb{}} \mykw{tex} }$\ldots$\texttt{ \mysym{\myrb\myrb{}}}
translation for each, overriding the default typesetting of some
terminals.  Note that the other terminals 
(\texttt{.} \texttt{(} \texttt{)} \texttt{\mylb} \texttt{\myrb} \texttt{/})
are still given their default typesetting.
\begin{alltt}
  \mykw{terminals} \mysym{::} 'terminals_' \mysym{::=}
    \mysym{|} \mybackslash{}            \mysym{::}  \mysym{::} lambda  \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}lambda \mysym{\myrb\myrb{}}
    \mysym{|} -->          \mysym{::}  \mysym{::} red     \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}longrightarrow \mysym{\myrb\myrb{}}
\end{alltt}
Thirdly, the file has \mykw{com} comments, including
the 
\texttt{\mysym{\mylb\mylb{}} \mykw{com} term    \mysym{\myrb\myrb{}}}
attached to a grammar rule, 
the 
\texttt{\mysym{\mylb\mylb{}} \mykw{com} variable\mysym{\myrb\myrb{}}}
attached to a production, and the
\texttt{\mysym{\mylb\mylb{}} \mykw{com} \mysym{[[}t1\mysym{]]} reduces to \mysym{[[}t2\mysym{]]}\mysym{\myrb\myrb{}}}
attached to a semantic relation. These appear in the \myLaTeX{} output
as shown in Figure~\ref{a49}.



\subsection{Specifying \myLaTeX{} for productions}
One can also specify \mykw{tex} translations for productions, overriding the default
\myLaTeX{} typesetting, e.g.~as in this example of
a type abstraction production.
\begin{alltt}
    \mysym{|} \ X <: T . t   \mysym{::} \mysym{::} TLam   \mysym{\mylb\mylb} \mykw{tex} \mybackslash{}Lambda \mysym{[[}X\mysym{]]} \mysym{[[}<:\mysym{]]} \mysym{[[}T\mysym{]]}. \mybackslash{}, \mysym{[[}t\mysym{]]} \mysym{\myrb\myrb}
\end{alltt}
These \emph{homomorphisms}, or \emph{homs}\footnote{Strictly, clauses
  of primitive recursive function definitions from symbolic terms to strings, here of \myLaTeX{} code.}, can refer to the metavariables and
nonterminals that occur in the production, e.g.~the \texttt{\mysym{[[}X\mysym{]]}},
\texttt{\mysym{[[}T\mysym{]]}}, and \texttt{\mysym{[[}t\mysym{]]}} in the \mykw{tex} hom above,
interleaved with arbitrary strings and with typeset elements of the
\mykw{terminals} grammar, e.g.~the \texttt{\mysym{[[}<:\mysym{]]}}.

Homomorphisms are applied recursively down the structure of symbolic
terms. For example, an F$_{<:}$ term
\begin{verbatim}
  (\X<:T11.t12) [T2]
\end{verbatim}
would be \myLaTeX-pretty-printed, using the \mykw{tex} clause above, as
\begin{verbatim}
( \, \Lambda  \mathit{X} <: \mathit{T_{\mathrm{11}}} . \, \mathit{t_{\mathrm{12}}} \, )
 \, \, [ \, \mathit{T_{\mathrm{2}}} \, ]
\end{verbatim}
which is typeset as below.
\newcommand{\lamA}{
\begin{toimage}
$
( \, \Lambda  \mathit{X}   <:   \mathit{T_{\mathrm{11}}} . \,  \mathit{t_{\mathrm{12}}} \, ) \,  \, [ \, \mathit{T_{\mathrm{2}}} \, ]
$
\end{toimage}\imageflush
}\[\mbox{\lamA}\]
Note the \verb+X+, \verb+T11+ and \verb+t12+ of the symbolic term are
used to instantiate the formal parameters \verb+X+, \verb+T+ and
\verb+t+ of the homomorphism definition clause. 
%
If the \verb+t+ itself had compound term structure, e.g. as below
\begin{verbatim}
  (\X<:T. \X'<:T'.x)
\end{verbatim}
the homomorphism would be applied recursively, producing 
\begin{verbatim}
( \, \Lambda  \mathit{X} <: \mathit{T} . \,  \Lambda  \mathit{X'} <: \mathit{T'} 
. \,  \mathit{x} \,  \, )
\end{verbatim}
typeset as follows.
\newcommand{\lamB}{
\begin{toimage}
$( \, \Lambda  \mathit{X} <: \mathit{T} . \,  \Lambda  \mathit{X'}   <:   \mathit{T'} . \,  \mathit{x} \,  \, )$
\end{toimage}\imageflush
}\[\mbox{\lamB}\]
Where there is no user-supplied homomorphism clause the \myLaTeX{}
pretty-printing defaults to a sequence of the individual items
separated by thin spaces (\verb+\,+),
with reasonable default fonts and making use of the \verb+terminals+ grammar where appropriate.

\subsection{Specifying \myLaTeX{} for grammar rules}
Grammar rules can include a \mykw{tex} hom specifying how all the
  nonterminal roots should be typeset, e.g.
\begin{alltt}
  type\mysym{,} t\mysym{,} s \mysym{::} Typ_ \mysym{::=}  \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}mathsf\mylb{}\mysym{[[}type\mysym{]]}\myrb{} \mysym{\myrb\myrb{}} 
        \mysym{|} unit                  \mysym{::}   \mysym{::} unit   
        \mysym{|} type * type'          \mysym{::}   \mysym{::} pair   
        \mysym{|} type -> type'         \mysym{::}   \mysym{::} fun    
\end{alltt}
%Here that \verb+tex+ hom overrides the default typesetting with
%\verb+\mathsf+.

Alternatively, the individual nonterminal roots can have \mykw{tex}
homs specifying how they should be typeset:
\begin{alltt}
  G \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}Gamma \mysym{\myrb\myrb{}} \mysym{,} D \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}Delta \mysym{\myrb\myrb{}} \mysym{::} 'G_' \mysym{::=} 
        \mysym{|} empty                 \mysym{::}   \mysym{::} empty       
        \mysym{|} G , x : T             \mysym{::}   \mysym{::} term        
\end{alltt}
permitting the user to write \verb+G'+, \verb+D12+ etc.~in symbolic
terms, to be typeset as 
\begin{toimage}
$\Gamma'$,
$\Delta_{12}$,
\end{toimage}
\imageflush
etc.




\subsection{Using the \myLaTeX{} code}\mlabel{a64}%
The generated \myLaTeX{} code can be used in two main ways.  
By default, Ott generates a stand-alone \myLaTeX{} file, 
with a standard wrapper (including a  \verb+\documentclass+, various
macro definitions, and a main body),
that gives the complete system definition.

The default header can be overridden by writing
\verb+ embed {{ tex-wrap-pre ... }} + and the default footer by
writing \verb+embed {{ tex-wrap-post ... }} +. Alternatively, the
program option \verb+-tex_wrap false+ with the \verb+-tex_wrap false+
command-line argument, one can generate a file that can be included in
other \myLaTeX{} files, that just defines macros to typeset various
parts of the system (\verb+-tex_wrap false+ overrides any
\verb+tex-wrap-pre/tex-wrap-post+ embeds).


The generated \myLaTeX{} output is factored into individual \myLaTeX{}
commands: for the metavariable declarations, each rule of the syntax
definition, the collected syntax (\verb+\ottgrammar+), each rule of the inductive relation
definitions, the collected rules for each relation, the collected
rules for each \verb+defns+ block, the union of those
(\verb+\ottdefns+) and the whole (\verb+\ottall+). 
%
This makes it possible to quote individual parts of the definition,
possibly out-of-order, in a paper or technical report. 


If one needs to include more than one system in a single \myLaTeX{}
document, the \verb+ott+ prefix can be replaced using the
\verb+-tex_name_prefix+ command-line argument. 

The generated \myLaTeX{} is factored through some common style macros, 
e.g.~to typeset a comment, a production, and a grammar.  If necessary
these can be redefined in an \mykw{embed} block (see Section~\ref{a60}).  
For example, the file \verb+tests/squishtex.ott+
\input{squishtex.hand.alltt}
defines a more compact style for grammars.  Note that the
\texttt{\mysym{[[}\mykw{TEX\_NAME\_PREFIX}\mysym{]]}} is replaced by whatever prefix is in force,
so such style files can be reused in different contexts.

A more sophisticated \myLaTeX{} package \verb+ottlayout.sty+, providing fine control of how
inference rules and grammars should be typeset, is contained in the
\verb+tex+ directory of the distribution.  It is described in the
manual therein.


\section{Generating proof assistant definitions}\mlabel{a52}%
To generate proof assistant definitions, for Coq, Isabelle, and HOL,
the minimal Ott source file of Section~\ref{a51}/Figure~\ref{a45} must
be extended with a modest amount of additional data, as shown in Figure~\ref{a47}.
Executing 
\begin{alltt}
   bin/ott  -i tests/test10.4.ott  -o out.v  -o out.thy  -o outScript.sml 
\end{alltt}
generates Coq \texttt{out.v}, Isabelle \texttt{out.thy}, and HOL
\texttt{outScript.sml}, shown in Figures~\ref{a6}, \ref{a5}, and \ref{a40}.
The additional data can be combined with the annotations for
\myLaTeX{} of the previous section, but those are omitted here. 
\begin{figure}
\fbox{\small\begin{minipage}{\textwidth}
\input{test10.4.alltt}
\end{minipage}}
\caption{Source: \texttt{test10.4.ott}\protect\label{a47}}
\end{figure}
We add four things.
First, we specify proof assistant types to represent object-language
variables --- in this example, choosing the \texttt{string} type of
Isabelle and HOL, and the \texttt{nat} type for Coq:
\begin{alltt}
  \mykw{metavar} termvar\mysym{,} x \mysym{::=}  
  \mysym{\mylb\mylb{}} \mykw{isa} string\mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{coq} nat\mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{hol} string\mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{coq-equality} \mysym{\myrb\myrb{}}
\end{alltt}
For Coq output, one can specify \verb+{{ coq-equality+ \textit{proof-script} \verb+}}+
to build a decidable equality over the Coq representation type using
the proof \textit{proof-script}.  If the script is omitted, as in this
example, it defaults
to 
\begin{verbatim}
Proof.
  decide equality; auto with ott_coq_equality arith.
Defined.
\end{verbatim}
where the \verb+ott_coq_equality+ database contains the decidable
equalities of the representation types defined in the source.  It is
possible to suppress type generation for specific metavariables or nonterminals, by adding the
declaration \verb+{{ phantom }}+.  This is useful in some cases, for
instance to avoid duplicate definitions of types already defined in an
imported library. Any type homs are  taken into account when
the metavariable or nonterminal root is output as a type.


Second, we specify what the binding is in the object language, with
the \texttt{\mysym{(+} \mykw{bind} x \mykw{in} t \mysym{+)}}
annotation on the \texttt{Lam} production:
\begin{alltt}
  \mysym{|} \mybackslash{} x . t      \mysym{::}  \mysym{::} Lam     \mysym{(+} \mykw{bind} x \mykw{in} t \mysym{+)}
\end{alltt}
Section~\ref{a62} describes the full language of binding specifications.

Third, we add a block
\begin{alltt}
  \mykw{substitutions}
    \mykw{single} t x \mysym{::} tsubst 
\end{alltt}
to cause Ott to generate Coq/Isabelle/HOL definitions of a substitution
function, with name root \texttt{tsubst}, replacing metavariables \texttt{x} by terms \texttt{t}.  This is for single
substitutions; multiple substitution functions (taking lists of
substitutand/substitutee pairs) can also be generated with the keyword
\mykw{multiple}. 
Substitution functions are generated for all rules of the grammar for
which they might be required --- here, just over \verb+t+, with a
function named \verb+tsubst_t+.

Finally, we specify translations for the metaproductions:
\begin{alltt}
  \mysym{|} ( t )        \mysym{::} \mykw{S}\mysym{::} Paren   \mysym{\mylb\mylb{}} \mykw{icho} \mysym{[[}t\mysym{]]}  \mysym{\myrb\myrb{}} 
  \mysym{|} \mylb{} t / x \myrb{} t' \mysym{::} \mykw{M}\mysym{::} Tsub    \mysym{\mylb\mylb{}} \mykw{icho} (tsubst_t \mysym{[[}t\mysym{]]} \mysym{[[}x\mysym{]]} \mysym{[[}t'\mysym{]]})\mysym{\myrb\myrb{}}
\end{alltt}
These specify that \texttt{(t)} should be translated into just the
translation of \texttt{t}, whereas 
 \texttt{\mylb{}t/x\myrb{}t'} should be translated into the
proof-assistant application of \texttt{tsubst\_t} to the translations
of \texttt{t}, \texttt{x}, and \texttt{t'}.
The (admittedly terse) \mykw{icho} specifies that these translations should be
done uniformly for Isabelle, Coq, HOL, and OCaml output.  One can also
specify just one of these, writing
\texttt{\mysym{\mylb\mylb{}} \mykw{coq} }$\ldots$\texttt{\mysym{\myrb\myrb{}}}, 
\texttt{\mysym{\mylb\mylb{}} \mykw{hol} }$\ldots$\texttt{\mysym{\myrb\myrb{}}}, 
\texttt{\mysym{\mylb\mylb{}} \mykw{isa} }$\ldots$\texttt{\mysym{\myrb\myrb{}}}, or 
\texttt{\mysym{\mylb\mylb{}} \mykw{ocaml} }$\ldots$\texttt{\mysym{\myrb\myrb{}}},
or include several, with different translations for each.
There are also abbreviated forms \mykw{ich}, \mykw{ic}, \mykw{ch}, and \mykw{ih}.
The body of a proof assistant hom should normally include outer
parentheses, as in the \texttt{Tsub} hom above, so that it is
parsed correctly by the proof assistant in all contexts.

\begin{figure}
\fbox{\small\begin{minipage}{\textwidth}
\verbatiminput{test10.despaced.v}
%\verbatiminput{test10.hand.edited.v}
\end{minipage}}
\caption{Generated Coq:\texttt{test10.v}\protect\label{a6}}
\end{figure}

\begin{figure}
\fbox{\small\begin{minipage}{\textwidth}
\verbatiminput{test10.thy}
%\verbatiminput{test10.hand.edited.thy}
\end{minipage}}
\caption{Generated Isabelle:\texttt{test10.thy}\protect\label{a5}}
\end{figure}

\begin{figure}
\fbox{\small\begin{minipage}{\textwidth}
\verbatiminput{test10Script.sml}
\end{minipage}}
\caption{Generated HOL:\texttt{test10Script.sml}\protect\label{a40}}
\end{figure}


\subsection{Proof assistant code for grammar rules}
The normal behaviour is to generate a free proof assistant type for
each (non-subrule, non-phantom) grammar rule.  
For example, the Coq compilation for \texttt{t} here generates a free type with three
constructors:
\begin{verbatim}
  Inductive term : Set := 
   | t_var (x:var)
   | t_lam (x:var) (t:term)
   | t_app (t:term) (t':term).
\end{verbatim}
(note that the metaproductions do not give rise to constructors).  

Remark: prior to version 0.20.2, the free type generated for Coq was 
\begin{verbatim}
  Inductive term : Set := 
   | t_var : var -> term
   | t_lam : var -> term -> term
   | t_app : term -> term -> term.
\end{verbatim}
but we found that trying to preserve the names specified by the user
is helpful later, when doing proofs.  Whenever a clash is detected, or
for list forms, the wildcard \texttt{\_} is used.  The old behaviour
can be obtained via the top-level option
\texttt{-coq\_names\_in\_rules false}.

By default the order of the arguments to those constructors follows
the order in which they appear in the production.  That can be
overridden with an \texttt{order} hom.  For example, if for some
reason (perhaps compatibility with other Coq code) one wished the
arguments to \verb+t_Lam+ to be reversed:
\begin{verbatim}
   | t_Lam : t -> termvar -> t
\end{verbatim}
one could add an \mykw{order} hom as below.
\begin{alltt}
  \mysym{|} \mybackslash{} x . t      \mysym{::}  \mysym{::} Lam  \mysym{\mylb\mylb{}} \mykw{order} \mysym{[[}t\mysym{]]} \mysym{[[}x\mysym{]]} \mysym{\myrb\myrb{}}
\end{alltt}


Instead of using the generated free type, one can specify an arbitrary proof assistant representation type,
annotating  the grammar rule with a \mykw{coq}, \mykw{isa}, \mykw{hol}, or \mykw{ocaml} hom ---
for example, in the following grammar for substitutions.
\begin{alltt}
  s \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}sigma \mysym{\myrb\myrb{}} \mysym{::} 'S_' \mysym{::=} \mysym{\mylb\mylb{}} \mykw{com} multiple subst \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{isa} (termvar*t) list \mysym{\myrb\myrb{}}
       \mysym{|} [ x |-> t ]            \mysym{::}   \mysym{::} singleton   \mysym{\mylb\mylb{}} \mykw{isa} [ (\mysym{[[}x\mysym{]]},\mysym{[[}t\mysym{]]}) ]  \mysym{\myrb\myrb{}}
       \mysym{|} s1 , \mysym{..} , sn           \mysym{::}   \mysym{::} list        \mysym{\mylb\mylb{}} \mykw{isa} List.concat \mysym{[[}s1 \mysym{..} sn\mysym{]]} \mysym{\myrb\myrb{}}
\end{alltt}
Here the \texttt{\mysym{\mylb\mylb{}} \mykw{isa} (termvar*t) list \mysym{\myrb\myrb{}}} hom specifies that in
Isabelle output this type be represented as an Isabelle
\verb+(termvar*t) list+ instead of the default free inductive type;
all the productions are metaproductions (tagged \mykw{M}); and \mykw{isa} homs for each production specify how they should be translated into that Isabelle type.
%
This feature must be used with care, as any Ott-generated functions, e.g.~substitution functions, cannot recurse through such user-defined types.  



Grammar rules (whether free or non-free) can also include a \mykw{coq equality} hom, instructing
the Coq code generator to derive a decidable equality for the Coq
representation type.  For example, the ML polymorphism Ott source of
\texttt{test8.ott} %\S\mref{a24} 
includes the following.
\begin{alltt}
  typvar \mysym{::} TV_ \mysym{::=} \mysym{\mylb\mylb{}} \mykw{coq-equality} decide equality. apply eq_value_name_t. \mysym{\myrb\myrb{}}
       \mysym{|} ' ident                \mysym{::}   \mysym{::} ident
\end{alltt}



The Coq/HOL/Isabelle/OCaml type name for a grammar rule, or for a
metavariable declaration, is normally
taken to be just its primary nonterminal root. 
Occasionally it is useful to work around a clash between a
 metavar or nonterminal primary root and a proof assistant symbol,
e.g.~\texttt{T} in HOL or \texttt{value} in Isabelle. 
For this, one can add a \mykw{coq}, \mykw{hol}, \mykw{isa}, or
\mykw{ocaml} hom to the primary nonterminal root.  In the example
below, the user can write \texttt{T}, \texttt{T'} etc. in their Ott
source, but the generated HOL type is \texttt{Typ}.
\begin{alltt}
T \mysym{\mylb\mylb{}} \mykw{hol} Typ \mysym{\myrb\myrb{}}\mysym{,} S\mysym{,} U \mysym{::} 'T_' \mysym{::=}                    \mysym{\mylb\mylb{}} \mykw{com} type  \mysym{\myrb\myrb{}}
  \mysym{|} T -> T'                          \mysym{::} \mysym{::} Fun          \mysym{\mylb\mylb{}} \mykw{com} type of functions \mysym{\myrb\myrb{}}
\end{alltt}

The grammar rules within each \mykw{grammar} block of a syntax definition may depend on each other arbitrarily.
When generating Isabelle/Coq/HOL/OCaml representation types, however, they are
topologically sorted, to simplify the resulting induction
principles.

\subsection{Proof assistant code for inductive definitions}
The semantic relations are defined
with the proof-assistant inductive relations packages, 
\verb+Inductive+, \verb+Hol_reln+, and \verb+inductive_set+ or \verb+inductive+, respectively.
Each  \mykw{defns} block gives rise to a potentially mutually
recursive definition of each \mykw{defn} inside it
(it seems clearer not to do a topological sort here).
%
Definition rules are expressed internally with symbolic terms. 
We give a simplified grammar thereof in Fig.~\mref{a19}, omitting the
symbolic terms for list forms. 
A symbolic term $\mathit{st}$ for a nonterminal root is either an explicit nonterminal or a
node, the latter labelled with a production name and containing a list of
$\mathit{symterm\_element}$s, which in turn are either symbolic terms,
metavariables, or variables.
%
Each definition rule 
gives rise to an implicational clause, essentially
that the premises (Ott symbolic terms of the \mykw{formula} grammar)
imply the conclusion (an Ott symbolic term of whichever judgement is
being defined). 
%
Symbolic terms are compiled in several different ways:
\begin{itemize}
\item Nodes of non-meta productions are output as applications of the appropriate proof-assistant constructor (and, for a subrule, promoted to the corresponding constructor of a maximal rule).
\item Nodes of meta productions are transformed with the user-specified homomorphism.
\item Nodes of judgement forms are represented as applications of the defined relation in Coq and HOL, and as set-membership assertions in Isabelle.
\item Lists of formulae (the \verb+formula_dots+ production, c.f.\S\ref{a61}) are
  special-cased to proof-assistant conjunctions.
\end{itemize}
Further, for each nonterminal of a non-free grammar rule,
e.g.~a usage of \verb+v'+ where \verb+v<::t+, an additional premise
invoking the generated  subrule predicate for the non-free rule is added, e.g.~\verb+is_v v'+.
For Coq and HOL, explicit quantifiers are introduced for all variables
mentioned in the rule.
For HOL, rules are labelled with their rule name and tagged using \verb+clause_name+.


\subsection{Representation of binding}
At present the generated Isabelle/Coq/HOL uses fully concrete
representations of variables in terms, without any notion of alpha
equivalence, as one can see in Fig.~\mref{a5}:
see the \verb+t+ datatype of terms and the \verb+tsubst_t+
substitution function there.  
An experimental Coq backend generates definitions in locally-nameless style for a subset of the Ott metalanguage.  This is work-in-progress, and it is extensively documented in \url{http://moscova.inria.fr/~zappa/projects/ln_ott/}.
We intend in future to generate other representations, and in some
circumstances homs can be used to implement other representations directly.
For a reasonably wide variety of
languages, however, one can capture the intended semantics of whole programs in
this idiom, subject only to the condition that standard library
identifiers are not shadowed within the program, as the operational
semantics does not involve reduction under binders --- so any
substitutions are of terms which (except for standard library
identifiers) are closed.  This includes the ML polymorphism example of
\texttt{test8.ott}. %\S\mref{a23}.  
For languages which require a type
environment with internal dependencies, however, for example F$_{<:}$, this is
no longer the case. The POPLmark F$_{<:}$ example given in \texttt{test7.ott}
has a type system which disallows all shadowing, a property that is
not preserved by reduction.  However, a correct translation of F$_{<:}$ is generated by the Coq locally-nameless backend, and can be found in \url{http://moscova.inria.fr/~zappa/projects/ln_ott/}.

Further discussion of binding representations is in the Ott ICFP 2007
paper and in a working draft 
\begin{quotation}\noindent
Binding and Substitition.  Susmit Sarkar, Peter Sewell, and Francesco
Zappa Nardelli.  August 2007.
\end{quotation}
available from the Ott web page.

\subsection{Helper functions for free variable and substitution functions}

The generated free variable and substitution functions in the Coq output 
(e.g., in Figure~\ref{a6}) often rely on a few standard library functions: 
\verb+list_mem+, \verb+list_assoc+, \verb+list_minus+, \verb+list_minus2+.
In order to avoid dependencies on external libraries for defining those
functions, by default Ott generates the definitions for any such functions it
uses. It is possible to turn off the generation of definitions for 
these such functions by writing the following directive early on in the source file: 

\begin{alltt}
\mykw{embed} \mysym{\mylb\mylb{}} \mykw{coq-lib} list_mem list_minus \mysym{\myrb\myrb{}}
\end{alltt}

This instructs Ott to avoid generating definition for \verb+list_mem+ and
\verb+list_minus+, but to continue generating definitions for other functions
such as \verb+list_assoc+ and \verb+list_minus2+.

\textbf{Note about }\verb+list_minus2+\textbf{:} 
Instead of using the function \verb+list_minus2+, earlier versions of Ott
generated equivalent code based on \verb+list_filter+, which was more
difficult to reason about. For backwards compatibility, however, we provide the
command-line option \verb+-coq_use_filter_fn+ for generating a definition using
the older code pattern.

\subsection{Correctness of the generated proof assistant code}
We have attempted to ensure that the proof assistant definitions
generated by Ott are well-formed and what the user would intend.  This
is not guaranteed, however, for several reasons: (1) There may be name
clashes between Ott-generated identifiers and proof assistant built-in
identifiers (or, in pathological cases, even among different
Ott-generated identifiers).  (2) In some cases we depend on automatic
proof procedures, e.g.~for HOL definitions.  These work in our test
cases, but it is hard to ensure that they will in all cases.  More
importantly, (3) the generation process is complex, so it is quite
possible that there is either a bug in Ott or a mismatch between the
user expectation and what the tool actually does.  Ultimately one has
to read the generated proof assistant definitions to check that they
are as intended --- but typically one would do this in any case, many
times over, in the process of proving metatheoretic results, so we do
not consider it a major issue.

\subsection{Using the generated proof assistant code}
%
Note added 2017-11-30: the following is out of date.  

Ott builds code for 

\begin{tabular}{ll}
Coq 8.3 &  \ahrefurl{http://coq.inria.fr/} \\
HOL 4  (the current svn version) & \ahrefurl{http://hol.sourceforge.net/}\\
Isabelle/HOL (Isabelle 2011) & \ahrefurl{http://isabelle.in.tum.de/}
\end{tabular}

Given proof assistant files in the top-level directory of the
distribution, as produced at the start of this section
(Coq \texttt{out.v}, Isabelle \texttt{out.thy}, and HOL
\texttt{outScript.sml}), the various proof assistants can be invoked as follows.

\subsubsection{Coq}
First run 
\begin{verbatim}
  make
\end{verbatim}
in the \texttt{coq} directory of the distribution, to build the auxiliary files.
These include a core file (\verb+ott_list_core+) of definitions that
are used in Ott-generated output.  
At present these are only required when Coq native lists are used. 
There are also various lemmas (in
\verb+ott_list.v+) which may be useful; they can be made available with
\verb+Require Import ott_list.+ 

For batch mode run
\begin{verbatim}
  coqc -I coq  out.v
\end{verbatim}
where \verb+coq+ is the path to the \verb+coq+ directory of the distribution.

The experimental locally-nameless backend requires the \verb+Metatheory+ library by Arthur Chargueraud, available from the project web page.

\subsubsection{HOL}
First run 
\begin{verbatim}
  Holmake
\end{verbatim}
in the \texttt{hol} directory of the distribution, to build the auxiliary files.

 For
batch mode run
\begin{verbatim}
  Holmake -I hol  outTheory.uo
\end{verbatim}
%
where \verb+hol+ is the path to the \verb+hol+ directory of the distribution.
%
For interactive mode, run 
\begin{verbatim}
  hol -I hol
\end{verbatim}
inside an editor window (where the second \verb+hol+ is again the path
to the \verb+hol+ directory of the distribution), and in
another window view the \verb+outScript.sml+ file.  First
paste in the \verb+app load+ command from a comment at the top of the
file, then paste in the remainder.


\subsubsection{Isabelle}
For batch mode:
\begin{verbatim}
  echo 'ML_command {* (use_thy "Tmp"; OS.Process.exit OS.Process.success) handle e => (OS.Process.exit OS.Process.failure); *}' | /usr/local/Isabelle/bin/isabelle tty 
\end{verbatim}
Interactively, using Proof General:
\begin{verbatim}
  isabelle emacs out.thy
\end{verbatim}

\section{Judgments and formulae}\mlabel{a53}%
In a semantic rule, for example
\begin{alltt}
    t1 --> t1'
\mysym{    -------------- ::} ctx_app_arg
    v t1 --> v t1'
\end{alltt}
the conclusion must be a symbolic term of the form of the judgement being
defined, but in general the premises may be symbolic terms
of a \mykw{formula} grammar or in-line embedded prover code.
By default the formula grammar includes all the defined judgement forms: for the
running example Ott will synthesise grammars as below.

\begin{toimage}
\testTenMetagrammartabular{
%\testTenMetat
%\testTenMetav
%\testTenMetaterminals
\testTenMetaformula\testTenMetainterrule
\testTenMetajudgement\testTenMetainterrule
\testTenMetaJop\testTenMetaafterlastrule
%\testTenMetauserXXsyntax\testTenMetaafterlastrule
}
\end{toimage}%
\imageflush



The user can also define an explicit formula grammar, to let other
forms (not just judgements) appear as rule premises.  Below is a
fragment of the formula grammar from the LJ example on the Ott web page.
\begin{alltt}
  \mykw{formula} \mysym{::} formula_ \mysym{::=}  
   \mysym{|}  judgement                       \mysym{::}   \mysym{::} judgement
   \mysym{|}  formula1 \mysym{..} formulan            \mysym{::}   \mysym{::} dots
   \mysym{|}  not \mykw{formula}                     \mysym{::} \mykw{M} \mysym{::} not
          \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}neg \mysym{[[}\mykw{formula}\mysym{]]} \mysym{\myrb\myrb{}} 
          \mysym{\mylb\mylb{}} \mykw{isa} \mybackslash{}<not> (\mysym{[[}\mykw{formula}\mysym{]]}) \mysym{\myrb\myrb{}}
   \mysym{|}  ( \mykw{formula} )                     \mysym{::} \mykw{M} \mysym{::} brackets
          \mysym{\mylb\mylb{}} \mykw{tex} (\mysym{[[}\mykw{formula}\mysym{]]}\mybackslash{}!) \mysym{\myrb\myrb{}}  
          \mysym{\mylb\mylb{}} \mykw{isa} \mysym{[[}\mykw{formula}\mysym{]]} \mysym{\myrb\myrb{}}
   \mysym{|}  \mykw{formula} \mybackslash{}/ \mykw{formula}'             \mysym{::} \mykw{M} \mysym{::} or
          \mysym{\mylb\mylb{}} \mykw{tex} \mysym{[[}\mykw{formula}\mysym{]]} \mybackslash{}vee \mysym{[[}\mykw{formula}'\mysym{]]} \mysym{\myrb\myrb{}}
          \mysym{\mylb\mylb{}} \mykw{isa} \mysym{[[}\mykw{formula}\mysym{]]} \mybackslash{}<or> \mysym{[[}\mykw{formula}'\mysym{]]} \mysym{\myrb\myrb{}}
   \mysym{|}  \mykw{formula} /\mybackslash{} \mykw{formula}'             \mysym{::} \mykw{M} \mysym{::} and
          \mysym{\mylb\mylb{}} \mykw{tex} \mysym{[[}\mykw{formula}\mysym{]]} \mybackslash{}wedge \mysym{[[}\mykw{formula}'\mysym{]]} \mysym{\myrb\myrb{}}
          \mysym{\mylb\mylb{}} \mykw{isa} \mysym{[[}\mykw{formula}\mysym{]]} \mybackslash{}<and> \mysym{[[}\mykw{formula}'\mysym{]]} \mysym{\myrb\myrb{}}
   \mysym{|}  x = x'                          \mysym{::} \mykw{M} \mysym{::} xali
          \mysym{\mylb\mylb{}} \mykw{isa} \mysym{[[}x\mysym{]]} = \mysym{[[}x'\mysym{]]} \mysym{\myrb\myrb{}}
   \mysym{|}  X = X'                          \mysym{::} \mykw{M} \mysym{::} Xali
          \mysym{\mylb\mylb{}} \mykw{isa} \mysym{[[}X\mysym{]]} = \mysym{[[}X'\mysym{]]} \mysym{\myrb\myrb{}}
\end{alltt}
This example adds (to the judgement forms) syntax for parenthesised
formulae, negation, and, or, and equality testing on two sorts.  For
each, \mykw{tex} and \mykw{isa} homs specify how they should be
typeset and be translated into Isabelle.

If the user defines a \mykw{formula} grammar then (as here) the production
name prefix must be \texttt{formula\_} and the name for the
\texttt{judgement} production must be \texttt{judgement}.

\subsection{Naming of premises for the Coq backend}
It is possible to specify the names of premises of inductive
predicates; these names are then used by the Coq backend, and are
often useful in proofs.  For instance, we can call \texttt{RED} the hypothesis in the rule below
\begin{alltt}
    t1 --> t1'  [[:RED]]
\mysym{    -------------- ::} ctx_app_arg
    v t1 --> v t1'
\end{alltt}
which will then generate the following Coq code:
\begin{alltt}
| ctx_app_arg : forall (v t1 t1':term)
    (RED: reduce t1 t1'),
    is_val_of_term v ->
    reduce (t_app v t1) (t_app v t1').
\end{alltt}
Names of rules cannot contain spaces or other non alpha-numerical
characters, and must begin with a letter.  The name annotation must at
the rightmost place on the hypothesis line, and must be enclosed
(without spaces) between the \texttt{[[:} and \texttt{]]} parentheses.

\subsection{In-line embedded prover code in premises}
Instead of adding a formula production, one can directly embed prover
code as a premise, delimited as below by \texttt{\mysym{\mylb\mylb}} and
\texttt{\mysym{\myrb\myrb}}.  Within that, text will be echoed
directly to a prover (or given a default \LaTeX{} typesetting) except that symbolic terms enclosed within
\texttt{\mysym{[[}} and \texttt{\mysym{]]}} will be processed as in an
\mykw{embed} section. 
\begin{alltt}
  \mysym{\mylb\mylb} type_to_chunk (\mysym{[[}typeof e1\mysym{]]}) = Some \mysym{[[}c\mysym{]]} \mysym{\myrb\myrb}
  \mysym{----------------------------------------------------------- ::} Assign1
  e1=e2 . k |env   --tau-->    lval(e1) . [__=c e2] . k |env
\end{alltt}


\subsection{User syntax}
The tool also synthesises a \texttt{user\_syntax} grammar of all the
user syntax, for example:

\begin{toimage}
\testTenMetagrammartabular{
%\testTenMetat
%\testTenMetav
%\testTenMetaterminals
%\testTenMetaformula\testTenMetainterrule
%\testTenMetajudgement\testTenMetainterrule
%\testTenMetaJop\testTenMetaafterlastrule
\testTenMetauserXXsyntax\testTenMetaafterlastrule
}
\end{toimage}%
\imageflush

This is used for parsing top-level strings, for example when filtering
embedded code (\S\ref{a54}).


\section{Concrete terms and OCaml generation}\mlabel{a57}%
In semantic definitions, one typically never uses concrete variables,
only metavariables that range over them.  
In examples, however, one may need either a mix of concrete variables
and metavariables, or, for strictly concrete terms, to restrict to
just the former  (and also to prohibit symbolic nonterminals).


Figure~\mref{a46} combines the \myLaTeX{} and proof assistant
annotations of Sections \mref{a51} and \mref{a58}, adding a
\texttt{\mysym{\mylb\mylb{}} \mykw{lex} alphanum\mysym{\myrb\myrb{}}}
hom
to the \mykw{metavar} declaration to specify the lexical form of
concrete variables of this sort. 
%
At present a \mykw{lex} homomorphism must have body either
 \mykw{Alphanum}  (standing for \verb+[A-Z]([A-Z]|[a-z]|[0-9]|'|_)*+),
 \mykw{alphanum} (for \verb+([A-Z]|[a-z])([A-Z]|[a-z]|[0-9]|'|_)*+),
 \mykw{alphanum0} (for \verb+[a-z]([A-Z]|[a-z]|[0-9]|'|_)*+), or
 \mykw{numeral} (for \verb+[0-9][0-9]*+); more general regular expressions are not supported.
%
An identifier that can be ambiguously lexed as either a concrete or
symbolic metavariable, e.g.~\texttt{x} in the scope of the above
declaration, will be taken to be symbolic. 
%
To restrict the parser to strictly concrete terms only, one can add a
\mykw{:concrete:} prefix, as shown in Figure~\ref{a56}.

One can also specify how concrete variables should be \myLaTeX'd or
translated into a proof assistant, e.g.~with
homomorphisms 
\texttt{\mysym{\mylb\mylb{}} \mykw{texvar} \mybackslash{}mathrm\mylb{}\mysym{[[}termvar\mysym{]]}\mysym{\myrb\myrb{}}}
and
\texttt{\mysym{\mylb\mylb{}} \mykw{isavar} ''\mysym{[[}termvar\mysym{]]}''\mysym{\myrb\myrb{}}}
(and similarly \mykw{coqvar}, \mykw{holvar}, and \mykw{ocamlvar}).




Figure~\mref{a46} also specifies an OCaml representation type for
variables, with the \mykw{metavar} hom
\texttt{\mysym{\mylb\mylb{}} \mykw{ocaml} int\mysym{\myrb\myrb{}}}.
Executing 
\begin{alltt}
   bin/ott -i tests/test10.ott -o test10.ml  
\end{alltt}
produces the OCaml code shown in Figure~\ref{a59}, including OCaml
types to represent the abstract syntax, and auxiliary functions for
subrules and substitutions.  This does not implement the semantic
rules.  In some cases the various proof assistant code extraction
facilities can be used --- see Section~\ref{a22}.


\begin{figure}
\fbox{\small\begin{minipage}{\textwidth}
\input{test10.7.alltt}
\end{minipage}}
\caption{Source: \texttt{test10.7.ott}\protect\mlabel{a68}}
\end{figure}

\begin{figure}
\fbox{\small\begin{minipage}{\textwidth}
\verbatiminput{test10.ml}
\end{minipage}}
\caption{Generated OCaml code: \texttt{test10.ml}\protect\mlabel{a59}}
\end{figure}





\section{Filtering: Using Ott syntax within \myLaTeX{}, Coq, Isabelle,
  HOL, or OCaml}\mlabel{a54}%

\subsection{Filtering embedded code}\mlabel{a60}%
It is possible to embed arbitrary code in
the Ott source using an \mykw{embed} block, which can contain
\mykw{tex}, 
\mykw{coq}, 
\mykw{hol}, 
\mykw{isa}, or
\mykw{ocaml} homomorphisms, the
bodies of which will appear in the respective output. 
The \verb+embed+ keyword should be on a line by itself). For
example, 
\verb+test8.ott+ contains the following to
define Coq and HOL \verb+remove_duplicates+ functions.
\begin{alltt}
\mykw{embed}
\mysym{\mylb\mylb{}} \mykw{coq}
Fixpoint remove_duplicates (l:list typvar_t) : list typvar_t :=
  match l with
  | nil => nil
  | cons h t => if (list_mem eq_typvar_t h t)  then remove_duplicates t
                else cons h (remove_duplicates t)
end. \mysym{\myrb\myrb{}}

\mysym{\mylb\mylb{}} \mykw{hol}
val _ = Define `
  (remove_duplicates [] = []) /\mybackslash{}
  (remove_duplicates (x::xs) = if (MEM x xs) then remove_duplicates xs 
                               else x::(remove_duplicates xs))
`; \mysym{\myrb\myrb{}}
\end{alltt}
Within the body of an \mykw{embed} homomorphism, any text between 
\mysym{[[} and \mysym{]]} will be parsed as a symbolic term (of the
\verb+user_syntax+ grammar) and pretty printed, so one can use user
syntax within \myLaTeX{} or proof assistant code. An Isabelle example
is below, defining an Isabelle function to calculate the order of a
type with productions \verb+unit+, \verb+t*t'+, and \verb+t->t'+.
\begin{alltt}
\mysym{\mylb\mylb{}} \mykw{isa}
consts
order :: "type => nat"
primrec
"order \mysym{[[}unit\mysym{]]} = 0"
"order \mysym{[[}t*t'\mysym{]]} = max (order \mysym{[[}t\mysym{]]}) (order \mysym{[[}t'\mysym{]]})"
"order \mysym{[[}t->t'\mysym{]]} = max (1+order \mysym{[[}t\mysym{]]}) (order \mysym{[[}t'\mysym{]]})"

\mysym{\myrb\myrb{}}
\end{alltt}
It is often useful to define a proof assistant function, in an
\mykw{embed} section, together with a production of the \mykw{formula}
grammar with a proof assistant hom that uses that function, thereby
introducing syntax that lets the function be used in semantic rules. 

% needed right now:
%
Ott also permits embed blocks with \mykw{tex-preamble},
 homs, whose
contents appear in the generated \LaTeX{} preamble.
Any definitions of \LaTeX{} commands must appear in such a
\mykw{tex-preamble} section.
%  desirable:
%
% Ott also permits embed blocks with \mykw{tex-preamble},
% \mykw{hol-preamble}, or \mykw{isa-preamble} homs. 
% The contents of these appear (respectively) in the \LaTeX{} preamble, 
% just before the HOL \texttt{val \_ = new\_theory}, or between the Isabelle
% \texttt{imports Main Multiset} and \texttt{begin}. 
% Any definitions of \LaTeX{} commands must appear in such a
% \mykw{tex-preamble} section.





\subsection{Filtering files}
Similar processing can be carried out on separate files, using the
command-line options \verb+tex_filter+, \verb+isa_filter+, etc.
Each of these takes two arguments, a source filename and a destination
filename.  In processing the source file, 
any text between 
\mysym{[[} and \mysym{]]} will be parsed as a symbolic term (of the
\verb+user_syntax+ grammar) and pretty printed in the appropriate
style.  All other text is simply echoed.
% (except that strings of $n$
%\verb+[+ or \verb+]+ characters, for $n\geq 3$, are shortened by $1$). 


Typical usage for \myLaTeX{} would be something like this (from the \verb+Makefile+
used to produce this document):
\begin{verbatim}
test7.tex: ../src/ott ../tests/test7.ott ../tests/test7tt.mng
         cd ../src; make tmp_test7_clean.ott
         ../src/ott                                        \
                -i ../src/tmp_test7_clean.ott              \
                -o test7.tex                               \
                -tex_show_meta false                       \
                -tex_wrap false                            \
                -tex_name_prefix testSeven                 \
                -tex_filter ../tests/test7tt.mng test7tt.tex 
\end{verbatim}
The \verb+-tex_wrap false+ turns off output of the default \myLaTeX{}
document preamble, so the generated file \verb+test7.tex+ just contains
\myLaTeX{} definitions.  
The \verb+-tex_name_prefix testSeven+ sets a prefix for the generated
\myLaTeX{} commands
(so the \myLaTeX{} definitions from multiple Ott source files can be
included in a single \myLaTeX{} document).
The \verb+-tex_filter+ argument takes two
filenames, a source and a destination.  It filters the source file,
(roughly) replacing any string found within \verb+[[+ \verb+]]+ by
the tex pretty-print of its parse.  This parsing is done w.r.t.~the generated nonterminal 
\verb+user_syntax+ which is a union of all the user's grammar.  

At present %(unlike our previous munger) 
munged strings are not automatically
put within \verb+$+ \verb+$+, and there is no analogue of the
\verb+<[+ \verb+]>+ of our previous munger.  

%We probably want to tune up the tex to make it more
%convenient, e.g.~to put single rules in math displays (or not);
%definitions in centering environments (or not), etc.

The lexing 
%pays some attention to \myLaTeX{} comments, but may not get it
%right.  It 
turns any sequence of \verb+[+ (resp. of \verb+]+) of
length $n+1$ for $n>2$ into a literal sequence of length $n$.

Figures~\mref{a56} and \mref{a55} show a source file (\texttt{test7tt.mng}) that uses terms of the F$_{<:}$
definition of \texttt{test7.ott}, and the result of filtering it.

Similar filtering can be performed on Coq, Isabelle, HOL, and OCaml files.

To filter files with respect to a relatively stable
system definition, without having to re-process the Ott source files of
that system definition each time, there are command-line options
\begin{verbatim}
  -writesys <filename>                Output system definition
  -readsys <filename>                 Input system definition
\end{verbatim}
to first write the system definition (generated from some source files) to a file, and
then to read one back in (instead of re-reading the Ott source files).  
The saved system definitions are in an internal format, produced using
the OCaml marshaller, and contain OCaml closures.  They therefore will
not be compatible between different Ott versions.  They may also be
quite large. 

\begin{figure}
\begin{alltt}
\input{test7tt.mng.alltt.tex}
\end{alltt}
\caption{F$_{<:}$ Extracts: \myLaTeX{} source file to be filtered (\texttt{test7tt.mng})\mlabel{a56}}
\end{figure}
 


\begin{figure}
\begin{toimage}
\input{test7tt.tex} 
\end{toimage}%
\imageflush
\caption{F$_{<:}$ Extracts: the filtered output (\texttt{test7tt.tex})\mlabel{a55}}
\end{figure}






\section{Binding specifications}\mlabel{a62}%



Our first example involved a production with a single binder:
% HACK TO WORKAROUND ABSENCE OF TEX-PREAMBLE PROCESSING TO GET DOC TO
% BUILD FOR NOW
\renewcommand{\testTengrammartabular}[1]{\begin{tabular}{llcllllll}#1\end{tabular} }
\renewcommand{\testTenmetavartabular}[1]{\begin{tabular}{ll}#1\end{tabular} }

\renewcommand{\bindingOnegrammartabular}[1]{\begin{tabular}{llcllllll}#1\end{tabular} }
\renewcommand{\bindingOnemetavartabular}[1]{\begin{tabular}{ll}#1\end{tabular} }
 

\begin{toimage}
\[
\testTengrammartabular{
\testTenrulehead{\testTennt{t}}{::=}{  }\\ % \testTencom{term}}\\ 
\testTenprodline{|}{\lambda \, \mathit{x} \, . \, \testTennt{t}}{}{\textsf{bind}\; \mathit{x}\; \textsf{in}\; \testTennt{t}}{}{\testTencom{Lam}}\testTenafterlastrule}
\]
\end{toimage}%
\imageflush

specified by the source shown in Figure~\mref{a47}:
\begin{alltt}
  \mysym{|} \mybackslash{} x . t      \mysym{::}  \mysym{::} Lam     \mysym{(+} \mykw{bind} x \mykw{in} t \mysym{+)}
\end{alltt}
in which a single variable binds in a single subterm.
Realistic programming languages often have much more complex binding
structures, e.g.~%
structured patterns, multiple mutually recursive \texttt{let} definitions,
comprehensions, or-patterns, and dependent record patterns.


Ott has a flexible metalanguage for specifying binding structures,
expressive enough to cover these.
It comprises two forms of annotation on productions.
The first, 
\begin{toimage}
$\textsf{bind} \, \testThirteendnt{mse} \, \textsf{in} \, \testThirteendnt{nonterm}$,
\end{toimage}
\imageflush
%$\mybf{bind} \, \testThirteendnt{mse} \, \mybf{in} \, \testThirteendnt{nonterm}$,
%$\texttt{bind} \, \ldots \, \texttt{in} \, \testThirteendnt{nonterm}$,
lets one specify that variables bind in nonterminals of the
production, as in the \texttt{Lam} production above.
Here 
\begin{toimage}
$\testThirteendnt{mse}$
\end{toimage}
\imageflush
 is a \emph{metavariable set expression},
e.g.~in that \testTencom{lambda} production just the singleton
 metavariable 
\begin{toimage}
$\mathit{x}$
\end{toimage}
\imageflush
 of the production.
A variable can bind in multiple nonterminals, as in the example of
a simple recursive \texttt{let} below.

\begin{toimage}
\[
\bindingOnegrammar
\]
\end{toimage}%
\imageflush

More complex examples require one to collect together sets of
variables. For example, the grammar below (shown in Ott source and the
generated \myLaTeX{}) has structured patterns,
with a 
\begin{toimage}
$\bindingTwokw{let} \, \bindingTwont{p} \, = \, \bindingTwont{t} \,
\bindingTwokw{in} \, \bindingTwont{t'}$ 
\end{toimage}%
\imageflush
production in which all the binders of the pattern 
\begin{toimage}
$\bindingTwont{p}$\end{toimage}
\imageflush
bind in the continuation 
\begin{toimage}
$\bindingTwont{t'}$.
\end{toimage}
\imageflush

{\small
\begin{alltt}
t \mysym{::} E_ \mysym{::=}
  \mysym{|} x                        \mysym{::}   \mysym{::} ident
  \mysym{|} ( t1 , t2 )              \mysym{::}   \mysym{::} pair
  \mysym{|} let p = t in t'          \mysym{::}   \mysym{::} letrec      \mysym{(+} \mykw{bind} binders\mysym{(}p\mysym{)} \mykw{in} t' \mysym{+)}


p \mysym{::} P_ \mysym{::=}
  \mysym{|} _                        \mysym{::}   \mysym{::} wildcard    \mysym{(+} binders \mysym{=} \mysym{\mylb\myrb{}} \mysym{+)}
  \mysym{|} x                        \mysym{::}   \mysym{::} ident       \mysym{(+} binders \mysym{=} x \mysym{+)}
  \mysym{|} ( p1 , p2 )              \mysym{::}   \mysym{::} pair        \mysym{(+} binders \mysym{=} binders\mysym{(}p1\mysym{)} \mykw{union} binders\mysym{(}p2\mysym{)} \mysym{+)}
\end{alltt}
}

\begin{toimage}
\[
\bindingTwogrammar
\]
\end{toimage}%
\imageflush

This is expressed with the second form of annotation: user-defined
\emph{auxiliary functions} such as the 
\textrm{binders} above. This is an auxiliary function defined over the
\begin{toimage}
$\bindingTwont{p}$\end{toimage}
\imageflush
 grammar that identifies a set of variables to be
used in the \textsf{bind} annotation on the \begin{toimage}
$\bindingTwokw{let}$\end{toimage}
\imageflush
 production.  There can be any number of such auxiliary functions;
 \textrm{binders} is not a distinguished keyword.

The syntax of a precise fragment of the binding metalanguage is given in
Fig.~\mref{a13},
\renewcommand{\testThirteendmetavars}{
\begin{tabular}{lll}
\textbf{metavars} &
 $ \testThirteendmv{metavarroot} ,\, \testThirteendmv{mvr} $ \qquad &  
 $ \testThirteendmv{nontermroot} ,\, \testThirteendmv{ntr} $   \\
&  $ \testThirteendmv{terminal} ,\, \testThirteendmv{t} $ &  
 $ \testThirteendmv{auxfn} ,\, \testThirteendmv{f} $   \\
& $ \testThirteendmv{prodname} ,\, \testThirteendmv{pn} $ &  
% $ \testThirteendmv{suffix} ,\, \testThirteendmv{suff} $ &  \\
 $ \testThirteendmv{variable} ,\, \testThirteendmv{var} $   \\
% $ \testThirteendmv{index} ,\, \testThirteendmv{i} ,\, \testThirteendmv{j} ,\, \testThirteendmv{n} ,\, \testThirteendmv{m} $ &  \\
% $ \testThirteendmv{defnclassname} ,\, \testThirteendmv{dcn} $ &  \\
% $ \testThirteendmv{defnname} ,\, \testThirteendmv{dn} $ &  \\
% $ \testThirteendmv{defnrulename} ,\, \testThirteendmv{drn} $ &  \\
\end{tabular}}
%
%
%
\renewcommand{\testThirteendgrammar}{\testThirteendgrammartabular{
\testThirteendmetavar\testThirteendinterrule
\testThirteendnonterm\testThirteendinterrule
\testThirteendelement\testThirteendinterrule
\testThirteendmetavarXXsetXXexpression\testThirteendinterrule
\testThirteendbindspec\testThirteendinterrule
\testThirteendprod\testThirteendinterrule
\testThirteendrule\testThirteendinterrule
\testThirteendgrammarXXrules\testThirteendinterrule
%\testThirteendauxfnXXtype\testThirteendinterrule
%\testThirteendauxfnXXtypeXXenv\testThirteendinterrule
%\testThirteendgrammarXXtype\testThirteendinterrule
%\testThirteendsymterm\testThirteendinterrule
%\testThirteendsymtermXXnodeXXbody\testThirteendinterrule
%\testThirteendsymtermXXelement\testThirteendinterrule
%\testThirteendrelations\testThirteendinterrule
%\testThirteenddefnclass\testThirteendinterrule
%\testThirteenddefinition\testThirteendinterrule
%\testThirteenddefnrule\testThirteendafterlastrule
}}%
%
\newcommand{\symtermgrammar}{\testThirteendgrammartabular{
\testThirteendsymterm\testThirteendinterrule
\testThirteendsymtermXXnodeXXbody\testThirteendinterrule
\testThirteendsymtermXXelement\testThirteendinterrule
}}%
%
\newcommand{\castgrammar}{\testThirteendgrammartabular{
\testThirteendconcreteXXabstractXXsyntaxXXterm\testThirteendinterrule
}}%
%
\begin{figure}[t]
\begin{toimage}
\framebox[\columnwidth]{ 
\begin{minipage}{\columnwidth}
\small
\testThirteendmetavars

\testThirteendgrammartabular{
\textbf{grammar}\\}\\ 
\mbox{\quad}
\testThirteendgrammar
\end{minipage}}
\end{toimage}%
\imageflush
\caption{Mini-Ott in Ott: the binding specification metalanguage\label{a13}}
\end{figure}%
%
%\begin{figure}[t]
%\framebox[\columnwidth]{ \begin{minipage}{\columnwidth}
%\small
%\castgrammar
%\end{minipage}}
%\caption{Mini-Ott in Ott: concrete abstract syntax terms\label{a14}}
%\end{figure}%
%
where we have used Ott to define part of the Ott
metalanguage.  A simple type system (not shown) enforces sanity properties,
e.g.~that each auxiliary function is only applied to nonterminals that
it is defined over, and that metavariable set expressions are
well-sorted.

Further to that fragment, the tool supports binding for the list forms
of \S\mref{a61}. 
Metavariable set expressions can include lists of metavariables
and auxiliary functions applied to lists of nonterminals, e.g.~as in
the record patterns below.
% if this produces a bizarre latex error, it might be supertabular's fault...!

\begin{toimage}
\[
\begin{minipage}{\columnwidth}
\bindingThreegrammartabular{
%\bindingThreet%\bindingThreeinterrule
\bindingThreep\bindingThreeafterlastrule
}\end{minipage}
%\bindingThreegrammar
\]
\end{toimage}%
\imageflush

This suffices to express the binding structure of almost all the
natural examples we have come across, including definitions of
mutually recursive functions with multiple clauses for each, Join
calculus definitions~\cite{FGLMR96}, dependent record patterns, and many others.





\section{Generating substitution and free variable functions}\mlabel{a67}%
The tool can generate Isabelle/Coq/HOL/OCaml code for both single and multiple
substitution functions.  For example, the ML polymorphism Ott source
of \texttt{test8.ott} includes the following.
\begin{alltt}
  \mykw{substitutions}
    \mykw{single}   expr value_name \mysym{::} subst  
    \mykw{multiple} typexpr typvar  \mysym{::} tsubst 
\end{alltt}
This causes the generation of two families of substitution
functions, one replacing a single \verb+value_name+ by a \verb+expr+,
the other replacing multiple \verb+typvar+s by \verb+typexpr+s. 

Each family contains a function for each datatype for which it is
required, so in that example there are functions
\verb+subst_expr+ for the first and \verb+tsubst_typexpr+,
\verb+tsubst_typscheme+ and \verb+tsubst_G+ for the second. 

The functions for substitutions declared by
\begin{alltt}
  \mykw{substitutions}
    \mykw{single}   this that \mysym{::} name1 
    \mykw{multiple} this that \mysym{::} name2 
\end{alltt}
replaces terms of productions consisting just of a single \verb+that+ by a
\verb+this+.
Here \verb+this+ must be a nonterminal root, while \verb+that+ can be
either a metavariable root or a nonterminal root  (the latter
possibility allows substitution for compound identifiers, though it is
not clear that this is generally useful enough to be included). 
Substitution functions are generated for each member of each (mutually recursive)
block of grammar rules which either contain such a production or (indirectly)
refer to one that does. 

At present multiple substitutions are represented by Isabelle/Coq/HOL/OCaml
lists, so for the example above we have Isabelle
\begin{verbatim}
  tsubst_typexpr :: "(typvar*typexpr) list => typexpr => typexpr"
  tsubst_typscheme :: "(typvar*typexpr) list => typscheme => typscheme"
  tsubst_G :: "(typvar*typexpr) list => G => G"
\end{verbatim}
The generated functions do not substitute bound things, and recursive
calls under binders are filtered to remove the bound things.



Similarly, the tool can generate Isabelle/Coq/HOL/OCaml to calculate the free
variables of terms. For example, the ML polymorphism Ott source
of \texttt{test8.ott} includes the following.
\begin{alltt}
  \mykw{freevars}
    typexpr typvar \mysym{::} ftv
\end{alltt}
This causes Isabelle functions as below to be generated, calculating
the free \texttt{typvar}s that occur in singleton productions in the
\texttt{typexpr} grammar, within terms of all types.
\begin{verbatim}
  ftv_typexpr :: "typexpr => typvar list"
  ftv_typscheme :: "typscheme => typvar list"
  ftv_G :: "G => typvar list"
\end{verbatim}



\section{Locally-nameless representation}

The Coq backend of Ott includes experimental support for a
locally-nameless representation (and co-finite quantification). 

The user must specify which metavariables require a locally-nameless
representation via the \texttt{\mykw{repr-locally-nameless}} hom,
e.g.:

\begin{alltt}
\mykw{metavar} x \mysym{::=} \mysym{\mylb\mylb{}} \mykw{repr-locally-nameless} \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{com}  term variable  \mysym{\myrb\myrb{}} 
\end{alltt}

As usual, metavariables can be bound in productions, using the
bindspec language, as in the \texttt{lam} production below:
\begin{alltt}
\mykw{grammar}
  t \mysym{::} 't_' \mysym{::=}                                         \mysym{\mylb\mylb{}} \mykw{com} term \mysym{\myrb\myrb{}}
    \mysym{|} x                   \mysym{::}   \mysym{::} Var                     \mysym{\mylb\mylb{}} \mykw{com} variable \mysym{\myrb\myrb{}}         
    \mysym{|} \mybackslash{} x . t             \mysym{::}   \mysym{::} Lam  \mysym{(+} \mykw{bind} x \mykw{in} t \mysym{+)}  \mysym{\mylb\mylb{}} \mykw{com} abstraction \mysym{\myrb\myrb{}}      
    \mysym{|} t t'                \mysym{::}   \mysym{::} App                     \mysym{\mylb\mylb{}} \mykw{com} application \mysym{\myrb\myrb{}}      
    \mysym{|} ( t )               \mysym{::} \mykw{S} \mysym{::} paren   \mysym{\mylb\mylb{}} \mykw{coq} \mysym{[[}t\mysym{]]} \mysym{\myrb\myrb{}} 
    \mysym{|} \mylb{} t / x \myrb{} t'        \mysym{::} \mykw{M} \mysym{::} tsub    \mysym{\mylb\mylb{}} \mykw{coq} (t_subst_t \mysym{[[}t\mysym{]]}\mysym{[[}x  t'\mysym{]]}) \mysym{\myrb\myrb{}}
\end{alltt}

This definition gives rise to the datatype term below (here with option \texttt{-coq\_names\_in\_rules false}):
\begin{alltt}
Inductive term : Set := 
| term_var_b : nat -> term 
| term_var_f : var -> term 
| term_lam : term -> term 
| term_app : term -> term -> term.
\end{alltt}

Remarks:
\begin{enumerate}
\item Productions containing metavariables susceptible to be bound
  (e.g., \texttt{term\_var}) give rise to two distinct constructors, one
  (\texttt{term\_var\_b}) for de Bruijn indices to be used when the metavariable
  is bound, one (\texttt{term\_var\_f}) for "free" variables. The type \texttt{var},
  together with decidable equality and several useful lemmas and
  functions, is defined in the Metatheory library.

In the current implementation, metavariables susceptible to be bound
in a symbolic term (eg.\ the \texttt{x} in the \texttt{term\_var}
production) must be the only element of the production.

\item Binder metavariables are erased from productions
  (eg.\ \texttt{term\_lam}), as in de Bruijn representation.
\end{enumerate}

Ott automatically generates the appropriate \texttt{open} functions
and \texttt{lc} predicates to test if terms are locally-closed. The
other support functions for substitutions and free-variables
(\texttt{subst} and \texttt{fv}) are generated once the user declares
the relevant substitutions and freevars sections.

Ott automatically compiles the symbolic terms that appear in rule
definitions into the appropriate terms in locally-nameless style. For
instance, the typing rule for the simply-typed lambda-calculus:

\begin{alltt}
    E,x:S |- t : T
\mysym{    ------------------} :: lambda
    E |- \mybackslash{}x.t : S->T
\end{alltt}
is compiled into its locally-nameless representation:
\begin{alltt}
Inductive typing : env -> term -> type -> Prop := (* defn typing *) 
| ... 
| typing_lambda : forall (L:vars) (E:env) (t:term) (S T:type), 
   (forall x, x \mybackslash{}notin L -> typing (E & x ~ S) (open_term_wrt_term t (term_var_f x)) T) -> 
   typing E (term_lam t) (type_arrow S T).
\end{alltt}

For that, Ott follows the algorithm below. For each rule,
\begin{enumerate}
\item for each nonterminal that appears in the rule, compute the
  maximal set of binders under which it appears: for example, in the
  rule lambda above, the maximal set of binders for the nonterminal \texttt{t}
  is the singleton \texttt{x}, and it is empty for all the other nonterminals;
\item for each pair nonterminal / maximal binder set collected in
  phase 1., go over all the occurrences of the nonterminal in the rule
  and open them with respect to all the variables in the maximal
  binding set except those under which this particular occurrence is
  bound. In the example, this amounts to opening the occurrence of \texttt{t}
  in the premise with respect to the metavariable \texttt{x};
\item quantify using cofinite-quantification each metavariable that
  has been used to open a nonterminal;
\item add hypothesis about local-closure to guarantee the invariant
  that if a derivation holds, then the top-level terms involved are
  locally-closed.
\end{enumerate}
In some cases the user may want a finer control on which nonterminals
are opened and with respect to which metavariables. Consider for
instance the CBV beta-reduction rule:
\begin{alltt}
\mysym{    --------------------------}  :: ax_app
    (\mybackslash{}x.t1) v2 -->  \mylb{}v2/x\myrb{}t1
\end{alltt}
A naive application of the algorithm described above would open the
right hand side occurrence of \texttt{t1} with respect to a
cofinitely-quantified \texttt{x}. Substitution should then be used to replace
the occurrences of \texttt{x} with \texttt{v2}, resulting in the awkward term
\begin{alltt}
reduce (term_app (term_lam t1) v2) (subst_term v2 x (open_term_wrt_term t1 (term_var_f x)))
\end{alltt}

Instead, an idiomatic translation of CBV beta-reduction rule would
directly rely on the open function to substitute \texttt{v2} for the
bound occurrences of \texttt{x} in \texttt{t1}, as in:
\begin{alltt}
reduce (term_app (term_lam t1) v2) (open_term_wrt_term t1 v2)
\end{alltt}

A special syntax for production homomorphisms allow the user to specify this translation:
\begin{alltt}
    \mysym{|} \mylb{} t / x \myrb{} t'        \mysym{::} \mykw{M} \mysym{::} tsub    \mysym{\mylb\mylb{}} \mykw{coq} (t_subst_t \mysym{[[}t\mysym{]]}\mysym{[[}x t'\mysym{]]}) \mysym{\myrb\myrb{}}
\end{alltt}
In the homomorphism the nonterminal \texttt{t'} is referred to with
\texttt{\mysym{[[}x t'\mysym{]]}} instead of the usual
\texttt{\mysym{[[}t'\mysym{]]}}: the prefixed \texttt{x} specifies
that occurrences of \texttt{t'} should not be opened with respect to
the metavariable \texttt{x}. If this homomorphism is specified, then
the translation of the \texttt{ax\_app} rule is exactly idiomatic Coq
shown above.

\textit{Current limitations: support for single binders only, no auxfn, Coq only.}

\textit{Disclaimer: to compile rule definitions, Ott applies blindly the algorithm described above. Although in most of the cases, this generates a correct and idiomatic representation of the language, some language constructs might not be faithfully translated. Please, let us know if you find one of these cases.
}

If Ott is invoked with the \texttt{-coq\_lngen} option, then the
generated locally-nameless Coq code is compatible with Aydemir's
\textit{lngen} tool (\ahrefurl{http://www.cis.upenn.edu/~sweirich/papers/lngen/}).


\section{List forms}\mlabel{a61}%
Ott has direct support for lists, both as \emph{dot forms} such as 
\begin{toimage}
$t_1,\ldots,t_n$
\end{toimage}
\imageflush
and as \emph{list comprehensions} such as 
\begin{toimage}
$
\testSeventeenTencomplu{\mathit{t_{\testSeventeenTenmv{i}}}}
{\mathit{i}}
{{\mathrm{1}}}
{..}
{\mathit{n}}
$
\end{toimage}
\imageflush.
Figure~\ref{a63} shows an example semantic rule taken from our OCaml
fragment semantics, as both the generated \myLaTeX{} and its Ott
source, that involves several dot forms.
%
Other types commonly used in semantics, e.g.~finite maps or sets, can
often be described with this list syntax in conjunction with type and
metaproduction homs to specify the proof assistant representation.
%
When using list forms, one usually also wants to add a list-of-formula
production to the \mykw{formula} grammar, e.g. (as in
\verb+test17.10.ott+):
\begin{alltt}
  \mykw{formula} \mysym{::} formula_ \mysym{::=}  
   \mysym{|}  judgement                       \mysym{::}   \mysym{::} judgement
   \mysym{|}  formula1 \mysym{..} formulan            \mysym{::}   \mysym{::} dots
\end{alltt}
The proof assistant code generation for such a production (which must
be named \verb+formula_dots+) is special-cased to a list conjunction.


\newcommand{\ocamlruledrule}[4][]{\frac{\begin{array}{l}#2\end{array}}{#3}\quad\ocamlruledrulename{#4}}
\newcommand{\ocamlruleusedrule}[1]{\[#1\]}
\newcommand{\ocamlrulepremise}[1]{ #1 \\}
\newenvironment{ottdefnblock}[2]{ \framebox{\mbox{#1}} \quad #2 \\[0pt]}{}
\newcommand{\ocamlrulent}[1]{\mathit{#1}}
\newcommand{\ocamlrulemv}[1]{\mathit{#1}}
\newcommand{\ocamlrulekw}[1]{\mathbf{#1}}
\newcommand{\ocamlrulecom}[1]{\text{#1}}
\newcommand{\ocamlruledrulename}[1]{\textsc{#1}}
\newcommand{\ocamlrulecomplu}[5]{\overline{#1}^{\,#2\in #3 #4 #5}}
\newcommand{\ocamlrulecompu}[3]{\overline{#1}^{\,#2<#3}}
\newcommand{\ocamlrulecomp}[2]{\overline{#1}^{\,#2}}
\makeatletter
\newcommand{\ocamlruleenvironmentappend}[2]{%
  \begingroup%
  \def\@tempa{#1}\def\@tempb{ \ocamlrulekw{empty} }%
  \ifx\@tempa\@tempb\def\@tempc{}\else\def\@tempc{#1,}\fi%
  \expandafter\endgroup%
  \@tempc#2%
}
\makeatother

\newcommand{\ocamlruledruleJTeXXrecordXXconstr}[1]{\ocamlruledrule[#1]{
  \ocamlrulepremise{\ocamlrulent{E} \, \vdash \, \ocamlrulent{e_{{\mathrm{1}}}} \, : \, \ocamlrulent{t_{{\mathrm{1}}}} \quad ... \quad \ocamlrulent{E} \, \vdash \, \ocamlrulent{e_{\ocamlrulemv{n}}} \, : \, \ocamlrulent{t_{\ocamlrulemv{n}}}}
  \ocamlrulepremise{\ocamlrulent{E} \, \vdash \, \ocamlrulent{field\_name_{{\mathrm{1}}}} \, : \, \ocamlrulent{t} \, \rightarrow \, \ocamlrulent{t_{{\mathrm{1}}}} \quad ... \quad \ocamlrulent{E} \, \vdash \, \ocamlrulent{field\_name_{\ocamlrulemv{n}}} \, : \, \ocamlrulent{t} \, \rightarrow \, \ocamlrulent{t_{\ocamlrulemv{n}}}}
  \ocamlrulepremise{\ocamlrulent{t} \, = \, ( \, \ocamlrulent{t'_{{\mathrm{1}}}} \, , \, ... \, , \, \ocamlrulent{t'_{\ocamlrulemv{l}}} \, ) \, \ocamlrulent{typeconstr\_name}}
  \ocamlrulepremise{\ocamlrulent{E} \, \vdash \, \ocamlrulent{typeconstr\_name} \, \;\vartriangleright\; \, \ocamlrulent{typeconstr\_name} \, : \, \ocamlrulent{kind} \, \ocamlrulekw{\{} \, \ocamlrulent{field\_name'_{{\mathrm{1}}}} \, ; \, ... \, ; \, \ocamlrulent{field\_name'_{\ocamlrulemv{m}}} \, \ocamlrulekw{\}}}
  \ocamlrulepremise{\ocamlrulent{field\_name_{{\mathrm{1}}}} \, ... \, \ocamlrulent{field\_name_{\ocamlrulemv{n}}} \, \ocamlrulekw{PERMUTES} \, \ocamlrulent{field\_name'_{{\mathrm{1}}}} \, ... \, \ocamlrulent{field\_name'_{\ocamlrulemv{m}}}}
  \ocamlrulepremise{\ocamlrulekw{length} \, ( \, \ocamlrulent{e_{{\mathrm{1}}}} \, ) \, ... \, ( \, \ocamlrulent{e_{\ocamlrulemv{n}}} \, ) \, \geq \, 1}}{
\ocamlrulent{E} \, \vdash \, \ocamlrulekw{\{} \, \ocamlrulent{field\_name_{{\mathrm{1}}}} \, = \, \ocamlrulent{e_{{\mathrm{1}}}} \, ; \, ... \, ; \, \ocamlrulent{field\_name_{\ocamlrulemv{n}}} \, = \, \ocamlrulent{e_{\ocamlrulemv{n}}} \, \ocamlrulekw{\}} \, : \, \ocamlrulent{t}}{
{\ocamlruledrulename{JTe\_record\_constr}}{}
}}

\newcommand{\ocamlruledruleJRXXexprXXtupleXXctx}[1]{\ocamlruledrule[#1]{
  \ocamlrulepremise{\vdash \, \ocamlrulent{e} \,  \stackrel{ \ocamlrulent{L} }{\longrightarrow}  \, \ocamlrulent{e'}}}{
\vdash \,  \, ( \, \ocamlrulent{e_{{\mathrm{1}}}} \, ,\, \, .. \, ,\, \, \ocamlrulent{e_{\ocamlrulemv{m}}} \, ,\, \, \ocamlrulent{e} \, ,\, \, \ocamlrulent{v_{{\mathrm{1}}}} \, ,\, \, .. \, ,\, \, \ocamlrulent{v_{\ocamlrulemv{n}}} \, ) \,  \stackrel{ \ocamlrulent{L} }{\longrightarrow}  \,  \, ( \, \ocamlrulent{e_{{\mathrm{1}}}} \, ,\, \, .. \, ,\, \, \ocamlrulent{e_{\ocamlrulemv{m}}} \, ,\, \, \ocamlrulent{e'} \, ,\, \, \ocamlrulent{v_{{\mathrm{1}}}} \, ,\, \, .. \, ,\, \, \ocamlrulent{v_{\ocamlrulemv{n}}} \, )}{
{\ocamlruledrulename{JR\_expr\_tuple\_ctx}}{}
}}

\begin{figure}
% some bizarre latex messes with the font sizes if we put these into a
%  single array
\small
%\[\ocamlruledruleJRXXexprXXtupleXXctx{}\]
\begin{toimage}
\[
\ocamlruledruleJTeXXrecordXXconstr{}
\]
\end{toimage}%
\imageflush
\par\noindent{\begin{alltt}
 E |- e1 : t1 ... E |- en : tn
 E |- field\_name1 : t->t1 ... E |- field\_namen : t->tn
 t = (t1', ..., tl') typeconstr\_name
 E |- typeconstr\_name gives typeconstr\_name:kind \mylb{}field\_name1'; ...; field\_namem'\myrb{}
 field\_name1...field\_namen PERMUTES field\_name1'...field\_namem'
 length (e1)...(en)>=1
 -------------------------------------------------------------------------- :: record\_constr
 E |- \mylb{}field\_name1=e1; ...; field\_namen=en\myrb{} : t
\end{alltt}}\noindent
\caption{A sample OCaml semantic rule, in \myLaTeX{} and Ott
  source forms\protect\mlabel{a63}}
\end{figure}



\subsection{List dot forms}\mlabel{a66}%
%
%used in the \S\mref{a23} type schemes:
%
%{\small\testEightgrammartabular{\testEighttypscheme\testEightafterlastrule}}
Example productions for 
record types, record terms, and record
patterns are shown below, in both  Ott source and \myLaTeX{}, taken
from our F$_{<:}$ example.
\begin{alltt}
T\mysym{,} S\mysym{,} U \mysym{::} 'T_' \mysym{::=}                                               \mysym{\mylb\mylb{}} \mykw{com} type  \mysym{\myrb\myrb{}}
  \mysym{|} \mylb{} l1 : T1 , \mysym{..} , ln : Tn \myrb{}      \mysym{::} \mysym{::} Rec                       \mysym{\mylb\mylb{}} \mykw{com} record \mysym{\myrb\myrb{}}           

t \mysym{::} 't_' \mysym{::=}                                                     \mysym{\mylb\mylb{}} \mykw{com}  term  \mysym{\myrb\myrb{}}
  \mysym{|} \mylb{} l1 = t1 ,  \mysym{..} , ln = tn \myrb{}     \mysym{::} \mysym{::} Rec                       \mysym{\mylb\mylb{}} \mykw{com} record \mysym{\myrb\myrb{}}
  \mysym{|} let p = t in t'                 \mysym{::} \mysym{::} Let \mysym{(+} \mykw{bind} b\mysym{(}p\mysym{)} \mykw{in} t' \mysym{+)} \mysym{\mylb\mylb{}} \mykw{com} pattern binding\mysym{\myrb\myrb{}}

p \mysym{::} 'P_' \mysym{::=}                                                     \mysym{\mylb\mylb{}} \mykw{com}  pattern \mysym{\myrb\myrb{}}
  \mysym{|} x : T                           \mysym{::} \mysym{::} Var \mysym{(+} b \mysym{=} x  \mysym{+)}          \mysym{\mylb\mylb{}} \mykw{com} variable pattern \mysym{\myrb\myrb{}}
  \mysym{|} \mylb{} l1 = p1 , \mysym{..} , ln = pn \myrb{}      \mysym{::} \mysym{::} Rec \mysym{(+} b \mysym{=} b\mysym{(}p1 \mysym{..} pn\mysym{)} \mysym{+)} \mysym{\mylb\mylb{}} \mykw{com}  record pattern  \mysym{\myrb\myrb{}}

\end{alltt}

\begin{toimage}
{\small\testSevengrammartabular{
\testSevenrulehead{\testSevennt{T}  ,\ \testSevennt{S}  ,\ \testSevennt{U}}{::=}{\testSevencom{type}}\\ 
%\testSevenprodline{|}{\ldots}{}{}{}{}\\
\testSevenprodline{|}{\testSevenkw{\{} \, \mathit{l_{{\mathrm{1}}}} \, : \, \testSevennt{T_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l_{\testSevenmv{n}}} \, : \, \testSevennt{T_{\testSevenmv{n}}} \, \testSevenkw{\}}}{}{}{}{\testSevencom{record}}\testSeveninterrule
%
\testSevenrulehead{\testSevennt{t}}{::=}{\testSevencom{term}}\\ 
%\testSevenprodline{|}{\ldots}{}{}{}{}\\
\testSevenprodline{|}{\testSevenkw{\{} \, \mathit{l_{{\mathrm{1}}}} \, \!\! = \!\! \, \testSevennt{t_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l_{\testSevenmv{n}}} \, \!\! = \!\! \, \testSevennt{t_{\testSevenmv{n}}} \, \testSevenkw{\}}}{}{}{}{\testSevencom{record}}\\ 
\testSevenprodline{|}{\testSevenkw{let} \, \testSevennt{p} \, \!\! = \!\! \, \testSevennt{t} \, \testSevenkw{in} \, \testSevennt{t'}}{}{\textsf{bind}\; \textrm{b}(\testSevennt{p})\; \textsf{in}\; \testSevennt{t'}}{}{\testSevencom{pattern binding}}\testSeveninterrule
%
\testSevenrulehead{\testSevennt{p}}{::=}{\testSevencom{pattern}}\\ 
\testSevenprodline{|}{\mathit{x} \, : \, \testSevennt{T}}{}{\textrm{b}=\mathit{x}}{}{\testSevencom{variable pattern}}\\ 
\testSevenprodline{|}{\testSevenkw{\{} \, \mathit{l_{{\mathrm{1}}}} \, \!\! = \!\! \, \testSevennt{p_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l_{\testSevenmv{n}}} \, \!\! = \!\! \, \testSevennt{p_{\testSevenmv{n}}} \, \testSevenkw{\}}}{}{\textrm{b}=\textrm{b}(\testSevennt{p_{{\mathrm{1}}}}..\testSevennt{p_{\testSevenmv{n}}})}{}{\testSevencom{record pattern}}\testSevenafterlastrule
} 
}
\end{toimage}
\imageflush

%A dot form in a production, such as the \texttt{l1 : T1 , \mysym{..} , ln : Tn}
%above, may or may not involve a \emph{separating terminal}.  Here
%there is one, the comma `\texttt{,}'.  The tool finds the longest
%subsequence of elements on either side of the dots that can be made to
%coincide by anti-unifying their suffixes, anti-unifying index
%variables with natural numbers. 

Dot forms can be used in symbolic terms in semantic rules:

\begin{toimage}
{ \[\testSevendruleTyXXRcd{}\]}
\end{toimage}
\imageflush

Individually indexed projections from dot forms can be mentioned, eg
the $l_j$ below:

\begin{toimage}
{ \[\testSevendruleTyXXProj{}\]}
\end{toimage}
\imageflush

Symbolic terms can also include concatenations of two dot forms with a
singleton in between:

\begin{toimage}
{ \[\testSevendrulereduceXXCtxXXrecord{}\]}
\end{toimage}
\imageflush

Multiple dot forms within the same semantic rule can share bounds (e.g.~$1..m$):

\begin{toimage}
{ \[\testSevendruleMXXRcd{}\]}
\end{toimage}
\imageflush


In more detail, productions can have dot tokens interspersed between the elements. 
Dot tokens consist of two, three or four consecutive dots (\mysym{..},
\mysym{...}, or \mysym{....}), indicating lists with minimum lengths $0$,
$1$, and $2$ respectively (these length minimums are respected
 only when parsing concrete lists; they are not present in Isabelle/Coq/HOL
  output). 
The tool identifies the maximal sequence of elements on either side of
the dots that are identical modulo anti-unification of some
index. Optionally, there may also be a single terminal on either side
of the dot token, separating instances of the repeated unit.
For example, in the \texttt{test7.ott} production
\begin{alltt}
  \mysym{\mysym{|}} \mylb{} l1 = t1 ,  \mysym{..} , ln = tn \myrb{}  \mysym{::} \mysym{::} Rec                     
\end{alltt}
there is such a terminal (the `\verb+,+').  The tool identifies
that \verb+l1 = t1+ and \verb+ln = tn+ can be anti-unified as 
(roughly) \verb+l_ = t_+,  taking \verb+_+ to be the bounds \verb+1+ and \verb+n+.
A single production may contain multiple dot forms, but they must not overlap;
nested dot forms (including those with multiple changing indices) are not currently
supported. 

Homomorphisms and binding specifications are generalised to match:  an \verb+mse+ can
involve a dot form of metavariables; 
%(e.g.~the
%$\mathit{typvar_{\mathrm{1}}}..\mathit{typvar_{\mathit{n}}}$ above);
a dot form of nonterminals; or an auxiliary function applied to a dot
form
of nonterminals (e.g.~the
$\textrm{b}(\mathit{p_{\mathrm{1}}}..\mathit{p_{\mathit{n}}})$ above).
Dot forms on the right of a \mykw{bind} are not currently supported.

\myLaTeX{} homomorphisms should not refer to dot forms, as either an error
or bad output will be generated. (For \myLaTeX, there should really be
some means to specify
a homomorphism for the repeated expression, and also data on how any
list separators should be typeset.  This would require more
special-case treatment, which is not currently supported.)



\subsection{List comprehension forms}\mlabel{a44}%
%
Lists can also be expressed as explicit list comprehensions, 
for more concise typesetting.
Three different styles are supported, with no bounds, an upper bound,
or a lower and upper bound.  For example, in a symbolic
term, instead of the dot form 
\par\noindent{\small
\begin{alltt}
  G |- t1:T1  \mysym{..}  G |- tn:Tn
 \end{alltt}
}
one can write any of the following
\par\noindent{\small
\begin{alltt}
   \mysym{</} G |- ti:Ti \mysym{//} i           \mysym{/>}
   \mysym{</} G |- ti:Ti \mysym{//} i \mysym{IN} n      \mysym{/>}
   \mysym{</} G |- ti:Ti \mysym{//} i \mysym{IN} 1 \mysym{..} n \mysym{/>}
 \end{alltt}
}
Similar comprehensions can be used in productions, for example lines
2--4 below.  In addition, comprehensions in productions can specify a
terminal to be used as a separator in concrete lists, as in lines 5--7 below.
(These examples are taken from \verb+test17.10.ott+.)
\par\noindent{\small
\begin{alltt}
  \mysym{|} { l1 = t1 ,  \mysym{..} , ln = tn }           \mysym{::} \mysym{::} Rec              \mysym{\mylb\mylb} \mykw{com} dots \mysym{\myrb\myrb}
  \mysym{|} { \mysym{</} li = ti \mysym{//} i           \mysym{/>} }      \mysym{::} \mysym{::} Rec_comp_none    \mysym{\mylb\mylb} \mykw{com} comp \mysym{\myrb\myrb}
  \mysym{|} { \mysym{</} li = ti \mysym{//} i \mysym{IN} n      \mysym{/>} }      \mysym{::} \mysym{::} Rec_comp_u_none  \mysym{\mylb\mylb} \mykw{com} compu \mysym{\myrb\myrb} 
  \mysym{|} { \mysym{</} li = ti \mysym{//} i \mysym{IN} 1 \mysym{..} n \mysym{/>} }      \mysym{::} \mysym{::} Rec_comp_lu_none \mysym{\mylb\mylb} \mykw{com} complu \mysym{\myrb\myrb}  
  \mysym{|} { \mysym{</} li = ti \mysym{//} , \mysym{//} i           \mysym{/>} } \mysym{::} \mysym{::} Rec_comp_some    \mysym{\mylb\mylb} \mykw{com} comp with terminal \mysym{\myrb\myrb}
  \mysym{|} { \mysym{</} li = ti \mysym{//} , \mysym{//} i \mysym{IN} n      \mysym{/>} } \mysym{::} \mysym{::} Rec_comp_u_some  \mysym{\mylb\mylb} \mykw{com} compu with terminal \mysym{\myrb\myrb} 
  \mysym{|} { \mysym{</} li = ti \mysym{//} , \mysym{//} i \mysym{IN} 1 \mysym{..} n \mysym{/>} } \mysym{::} \mysym{::} Rec_comp_lu_some \mysym{\mylb\mylb} \mykw{com} complu with terminal \mysym{\myrb\myrb}  
 \end{alltt}
}
In Coq, HOL or Isabelle output, list dot forms and the
various list comprehension forms are treated almost identically.
In LaTeX output, comprension forms are default-typeset with overbars. 
For example, the rules below
\par\noindent{\small
\begin{alltt}
 G|- t:{l1:T1,\mysym{..},ln:Tn}
\mysym{ ----------------------- :: }Proj_dotform
 G|- t.lj : Tj

 G|- t: { \mysym{</} li:Ti \mysym{//} i\mysym{/>} }
\mysym{ ---------------------------------- ::} Proj_comp
 G|- t.lj : Tj

 G|- t: { \mysym{</} li:Ti \mysym{//} i \mysym{IN} n\mysym{/>} }
\mysym{ ---------------------------------- ::} Proj_comp_u
 G|- t.lj : Tj

 G|- t: { \mysym{</} li:Ti \mysym{//} i \mysym{IN} 1\mysym{..}n\mysym{/>} }
\mysym{ ---------------------------------- ::} Proj_comp_lu
 G|- t.lj : Tj
 \end{alltt}
}
are typeset as follows.

\begin{toimage}
%\testSeventeenTendruleTyXXRcdXXdotform
\[\testSeventeenTendruleTyXXProjXXdotform{}\]
%\testSeventeenTendruleTyXXRcdXXcomp
\[\testSeventeenTendruleTyXXProjXXcomp{}\]
%\testSeventeenTendruleTyXXRcdXXcompXXu
\[\testSeventeenTendruleTyXXProjXXcompXXu{}\]
%\testSeventeenTendruleTyXXRcdXXcompXXlu
\[\testSeventeenTendruleTyXXProjXXcompXXlu{}\]
\end{toimage}
\imageflush

Upper bounds of the form $n-1$ are also permitted, e.g. with
\par\noindent{\small
\begin{alltt}
 G|- t:{l0:T0,\mysym{..},ln-1:Tn-1}
\mysym{ ----------------------- ::} Proj_dotform_minus
 G|- t.lj : Tj

 G|- t: { \mysym{</} li:Ti \mysym{//} i \mysym{IN} 0\mysym{..}n-1\mysym{/>} }
\mysym{ ---------------------------------- ::} Proj_comp_lu_minus
 G|- t.lj : Tj
 \end{alltt}
}
typeset as below.  More complex arithmetic expressions are not
currently supported.

\begin{toimage}
\[\testSeventeenTendruleTyXXProjXXdotformXXminus{}\]
\[\testSeventeenTendruleTyXXProjXXcompXXluXXminus{}\]
\end{toimage}
\imageflush

A list form used in a symbolic term does not have to be in the same
style as that in the corresponding production. 
However, if a metavariable or nonterminal occurs in multiple different
list forms in the same inference rule, they must all be in the same style and
with the same bounds.  Moreover, in a production, a list form in a bindspec or
homomorphism must be in the same style and with the same bounds as the
corresponding list form in the elements of the production. 

The comprehension form without an upper bound, 
e.g.~\texttt{\mysym{</} G |- ti:Ti \mysym{//} i \mysym{/>}}, 
typeset as 
\begin{toimage}
$\overline{\Gamma \, \vdash \, \mathit{t_{\mathit{i}}} \, :
  \, \mathit{T_{\mathit{i}}}}^{\,\mathit{i}}$,
\end{toimage}%
\imageflush%
is not standard
notation, but is often very useful.  Many semantic rules involve lists
of matched length, e.g.~of the 
\begin{toimage}
$\mathit{t_{\mathit{i}}}$
\end{toimage}%
\imageflush%
 and
\begin{toimage}
$\mathit{T_{\mathit{i}}}$
\end{toimage}%
\imageflush%
 here, but do not need to introduce an
identifier for that length; omitting it keeps them concise.

The default visual style for typesetting list comprehensions can be
overridden by redefining the \myLaTeX{} commands \verb+\ottcomp+,
\verb+\ottcompu+, and \verb+\ottcomplu+ in an \mykw{embed} section, as
in Section~\ref{a64}.

In some cases one could make the typeset notation even less noisy, by
either omitting the superscript $i$ or omitting both the superscript $i$ and
the subscript $i$'s on $t$ and $T$.  The first is unambiguous if there
is at most one index on each element in the comprehension; the second
if all the elements are indexed by the same thing  (not the case for
this example, but common for comprehensions of single elements,
e.g. \verb+<< Ti // i>>+ for $\overline{T}$). It is arguable that that
should be automated in future Ott releases, though it would bring the typeset and ASCII
versions out of step.

%The tokens used for list dot forms and comprehension forms (\verb+</+,
%\verb+//+, \verb+IN+, \verb+/>+, \verb+..+, \verb+...+, and
%\verb+....+) cannot at present be used in the object language. 

List comprehension forms can also be used in bindspecs and in
homomorphisms. 


\subsection{Proof assistant code for list forms}\mlabel{a65}%
\subsubsection{Types}
We have to choose proof assistant representations for productions involving list
forms.  For example, for a language with records one might write 
\input{binding.6.alltt}
%\par\noindent{\small\verbatiminput{../tests/binding.6.ott}
%}\noindent
In HOL and Isabelle we represent these simply with contructors whose
argument types involve proof-assistant native list types, e.g.
the HOL list of pairs of a \verb+label+ and a \verb+t+:
\par\noindent{\small\begin{verbatim}
  val _ = Hol_datatype ` 
  t = E_record of (label#t) list  `;
\end{verbatim}
}\noindent
For Coq we provide two alternatives: one can either use
native lists, or lists can be translated away, depending on taste.
The choice is determined by the \verb+-coq_expand_list_types+
command-line option.
%pros & cons:
% native: can use standard functions and theorems
% translated away: ...how to articulate this...?
In the former case we generate an appropriate induction principle
using nested fixpoints, as
the default principle produced by Coq is too weak to be useful.
In the latter case we
synthesise an additional type for each type of lists-of-tuples that
arises in the grammar.  
In the example, we need a type of lists of
pairs of a \texttt{label} and a \texttt{t}:
\par\noindent{\small\begin{verbatim}
Inductive 
list_label_t : Set := 
   Nil_list_label_t : list_label_t
 | Cons_list_label_t : label -> t -> list_label_t 
     -> list_label_t

with t : Set := 
   E_record : list_label_t -> t .
\end{verbatim}
}\noindent
These are included in the grammar topological sort, and utility functions, e.g.~to 
make and unmake lists, are synthesised. 
%A similar translation will be needed for Twelf, as it has no
%polymorphic list type.
%We also generate, on request, default Coq proofs that there is a decidable equality
%on various types. 


\subsubsection{Terms (in inductive definition rules)}
Supporting list forms in the rules of an inductive definition requires some additional analysis. For example,
consider the record typing rule below.

\begin{toimage}
\[\bindingSixbdruleTyXXRcd{}\]
\end{toimage}%
\imageflush%

We analyse the symbolic terms in the premises and conclusion to
identify lists of nonterminals and metavariables with the same bounds --- here $t_0 .. t_{n-1}$, $T_0 .. T_{n-1}$, and $l_0 .. l_{n-1}$
all have bounds $0..n-1$.
%
To make the fact that they have the same length immediate in the
generated code, we introduce a single proof assistant
variable for each such collection, with appropriate projections and
list maps/foralls at the usage points.
For example, the HOL for the above is essentially as follows, with
an \verb+l_t_Typ_list : (label#t#Typ) list+.
\par\noindent{\small
\begin{verbatim}
(* Ty_Rcd *)  !(l_t_Typ_list:(label#t#Typ) list) (G:G) . 
(EVERY (\b.b) 
  (MAP (\(l_,t_,Typ_). (Ty G t_ Typ_)) l_t_Typ_list))
 ==> 
(Ty 
  G 
  (E_record (MAP (\(l_,t_,Typ_). (l_,t_)) l_t_Typ_list))
  (T_Rec    (MAP (\(l_,t_,Typ_). (l_,Typ_)) l_t_Typ_list)))
\end{verbatim}
}\noindent
This seems to be a better idiom for later proof development than the
alternative of three different list variables coupled with assertions
that they have the same length. 




%================


\newcommand{\oottdrule}[4][]{\frac{\begin{array}{l}#2\end{array}}{#3}\quad\oottdrulename{#4}}
\newcommand{\oottusedrule}[1]{\[#1\]}
\newcommand{\oottpremise}[1]{ #1 \\}
\newenvironment{oottdefnblock}[2]{ \framebox{\mbox{#1}} \quad #2 \\[0pt]}{}
\newcommand{\oottnt}[1]{\mathit{#1}}
\newcommand{\oottmv}[1]{\mathit{#1}}
\newcommand{\oottkw}[1]{\mathbf{#1}}
\newcommand{\oottcom}[1]{\text{#1}}
\newcommand{\oottdrulename}[1]{\textsc{#1}}
\newcommand{\oottcomplu}[5]{\overline{#1}^{\,#2\in #3 #4 #5}}
\newcommand{\oottcompu}[3]{\overline{#1}^{\,#2<#3}}
\newcommand{\oottcomp}[2]{\overline{#1}^{\,#2}}
\makeatletter
\newcommand{\oottenvironmentappend}[2]{%
  \begingroup%
  \def\@tempa{#1}\def\@tempb{ \oottkw{empty} }%
  \ifx\@tempa\@tempb\def\@tempc{}\else\def\@tempc{#1,}\fi%
  \expandafter\endgroup%
  \@tempc#2%
}
\makeatother

\newcommand{\oottdruleTyXXProj}[1]{\oottdrule[#1]{
  \oottpremise{\Gamma \, \vdash \, \oottnt{t} \, : \, \oottkw{\{} \, \mathit{l_{{\mathrm{1}}}} \, : \, \oottnt{T_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l_{\oottmv{n}}} \, : \, \oottnt{T_{\oottmv{n}}} \, \oottkw{\}}}}{
\Gamma \, \vdash \, \oottnt{t} \, . \, \mathit{l_{\oottmv{j}}} \, : \, \oottnt{T_{\oottmv{j}}}}{
{\oottdrulename{Ty\_Proj}}{}
}}
%hand-hacked to rename
\newcommand{\oottdrulereduceXXProjRcd}[1]{\oottdrule[#1]{
}{
\oottkw{\{} \, \mathit{l'_{{\mathrm{1}}}} \, \!\! = \!\! \, \oottnt{v_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l'_{\oottmv{n}}} \, \!\! = \!\! \, \oottnt{v_{\oottmv{n}}} \, \oottkw{\}} \, . \, \mathit{l'_{\oottmv{j}}} \, \longrightarrow \, \oottnt{v_{\oottmv{j}}}}{
{\oottdrulename{Proj}}{}
}}

%hand-hacked to single-column the conclusion and rename
\newcommand{\oottdrulereduceXXCtxXXrecord}[1]{\oottdrule[#1]{
  \oottpremise{\oottnt{t} \, \longrightarrow \, \oottnt{t'}}}{
\begin{array}{l}
\oottkw{\{} \, \mathit{l_{{\mathrm{1}}}} \, \!\! = \!\! \,
  \oottnt{v_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l_{\oottmv{m}}}
  \, \!\! = \!\! \, \oottnt{v_{\oottmv{m}}} \, , \, \mathit{l} \, \!\! =
  \!\! \, \oottnt{t} \, , \, \mathit{l'_{{\mathrm{1}}}} \, \!\! = \!\!
  \, \oottnt{t'_{{\mathrm{1}}}} \, , \, .. \, , \,
  \mathit{l'_{\oottmv{n}}} \, \!\! = \!\! \, \oottnt{t'_{\oottmv{n}}} \,
  \oottkw{\}} \\ \longrightarrow \, \oottkw{\{} \,
  \mathit{l_{{\mathrm{1}}}} \, \!\! = \!\! \, \oottnt{v_{{\mathrm{1}}}}
  \, , \, .. \, , \, \mathit{l_{\oottmv{m}}} \, \!\! = \!\! \,
  \oottnt{v_{\oottmv{m}}} \, , \, \mathit{l} \, \!\! = \!\! \,
  \oottnt{t'} \, , \, \mathit{l'_{{\mathrm{1}}}} \, \!\! = \!\! \,
  \oottnt{t'_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l'_{\oottmv{n}}}
  \, \!\! = \!\! \, \oottnt{t'_{\oottmv{n}}} \, \oottkw{\}} \end{array}}{
{\!\!\!\oottdrulename{Rec}}{}
}}

%testSeventeenTen
\newcommand{\oottdruleTyXXProjXXcompXXluXXminus}[1]{\oottdrule[#1]{
  \oottpremise{\Gamma \, \vdash \, \oottnt{t} \, : \, \oottkw{\{} \, \oottcomplu{\mathit{l_{\oottmv{i}}}:\oottnt{T_{\oottmv{i}}}}{\oottmv{i}}{{\mathrm{0}}}{..}{{\oottmv{n}-1}} \, \oottkw{\}}}}{
\Gamma \, \vdash \, \oottnt{t} \, . \, \mathit{l_{\oottmv{j}}} \, : \, \oottnt{T_{\oottmv{j}}}}{
{\oottdrulename{Proj}}{}
}}


With direct support for lists, we need also direct support for 
symbolic terms involving list projection and concatenation.
For example, the rule
%
%Lastly, one sometimes wants to write list \emph{comprehensions} rather
%than dots, for compactness or as a matter of general style. We support
%comprehensions of several forms, e.g.~with explicit index $i$ and
%bounds $0$ to $n-1$, as below, and with unspecified or upper-only bounds.
%
%
%
%==================
% 
% \[\testSeventeenTendrulereduceXXProjRcd{}\]
% \[\testSeventeenTendrulereduceXXCtxXXrecord{}\]
% %
% Lastly, one sometimes wants to write list \emph{comprehensions} rather
% than dots, for compactness or as a matter of general style. We support
% comprehensions of several forms, e.g.~with explicit index $i$ and
% bounds $0$ to $n-1$, as below, and with unspecified or upper-only bounds.
% \[\testSeventeenTendruleTyXXProjXXcompXXluXXminus{}\]
% 
% ==============
%

\begin{toimage}
\[\oottdrulereduceXXCtxXXrecord{}\]
\end{toimage}%
\imageflush%

gives rise to HOL code as below --- note the list-lifted usage of the
\verb+is_v_of_t+ predicate, and the list appends (\texttt{++}) in the conclusion.
\par\noindent{\small
\begin{verbatim}
(* reduce_Rec *)  !(l'_t'_list:(label#t) list) 
       (l_v_list:(label#t) list) (l:label) (t:t) (t':t) . 
((EVERY (\(l_,v_). is_v_of_t v_) l_v_list) /\
(( reduce t t' )))
 ==> 
(( reduce (t_Rec (l_v_list ++ [(l,t)] ++ l'_t'_list)) 
          (t_Rec (l_v_list ++ [(l,t')] ++ l'_t'_list))))
\end{verbatim}
}\noindent
For the \texttt{Proj} typing rule 

\begin{toimage}
\[\oottdruleTyXXProjXXcompXXluXXminus{}\]
\end{toimage}%
\imageflush%

we need a specific projection (the
HOL \verb+EL+) to
pick out the $j$'th element:
\par\noindent{\small
\begin{verbatim}
(* Ty_Proj *)  !(l_Typ_list:(label#Typ) list) 
       (j:index) (G:G) (t:t) . 
((( Ty G t (T_Rec (l_Typ_list)) )))
 ==> 
(( Ty 
     G 
     (t_Proj t  ((\ (l_,Typ_) . l_) (EL j l_Typ_list)))  
     ((\ (l_,Typ_) . Typ_) (EL  j l_Typ_list))))
\end{verbatim}
}\noindent
For Coq, when translating away lists, we have to introduce 
yet more list types for 
these proof assistant variables, in addition to the obvious
translation of symbolic terms, and, more substantially, to introduce additional inductive relation
definitions to induct over them.



%\[\oottdrulereduceXXProjRcd{}\]

% -------------------
% 
% 
% The generated Isabelle/Coq/HOL/OCaml types for dot forms involve lists of
% tuples, e.g.~for the production above:
% \par\noindent{\small
% \begin{verbatim}
%   datatype 
%   t = 
%    ...
%    | t_Rec "(label*t) list"
%    ...
% \end{verbatim}
% }
% The generated code for a rule involving dot forms involves, for each
% bound that occurs, a generated variable for the list of tuples of
% nonterminals and metavariables that occurred with that bound.  For
% example, 

For similar examples in Isabelle, the generated Isabelle for the first
three rules of \S\ref{a66} is
shown below (lightly hand-edited for format).  The first involves an
Isabelle variable \verb+l_t_T_list+, and list maps and projections
thereof.
\par\noindent{\small
\begin{verbatim}
Ty_RcdI: "
  [|(formula_formuladots ((List.map (%(l_,t_,T_).( ( G , t_ , T_ ) : Ty)) l_t_T_list)))|] 
  ==> 
  ( G , 
    (t_Rec ((List.map (%(l_,t_,T_).(l_,t_)) l_t_T_list))) , 
    (T_Rec ((List.map (%(l_,t_,T_).(l_,T_)) l_t_T_list))) 
  ) : Ty"

Ty_ProjI: "
  [| ( G , t , (T_Rec (l_T_list)) ) : Ty|] ==> 
  ( G , 
    (t_Proj t (%(l_,T_).l_) (List.nth l_T_list (j - 1))) , 
    (%(l_,T_).T_) (List.nth l_T_list (j - 1)) 
  ) : Ty"

E_Ctx_recordI: "
[| List.list_all (%(l_,v_).is_v v_) l_v_list ;
   ( t , t' ) : E|] 
==> 
 ( (t_Rec (l_v_list @ [(l,t)] @ l_'t_'list)) , 
   (t_Rec (l_v_list @ [(l,t')] @ l_'t_'list)) 
 ) : E"
\end{verbatim}
}
The generated code for substitutions and free variables takes account of such list structure.

Note that at present the generated Isabelle code for these functions
does not always build without change, in particular if tuples of size
3 or more are required in patterns.


\subsubsection{List forms in homomorphisms}

Proof assistant homomorphisms in productions can refer to dot-form
metavariables and nonterminals.  For example, the second production
below (taken from \verb+test17.9+) mentions \texttt{\mysym{[[}x1 t1 \mysym{...} xn tn\mysym{]]}} in the \mykw{isa}
homomorphism.  This must exactly match the dot form in the production
except that all terminals must be omitted --- the metavariables and
nonterminals must occur in the same order as in the production, and
the bounds must be the same.  

% \begin{alltt}
% % test17.9.ott  dot form test
% 
% \mykw{metavar} ident\mysym{,} x \mysym{::=} \mysym{\mylb\mylb{}} \mykw{isa} string \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{coq} nat \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{coq-equality} \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{hol} string \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{ocaml} int \mysym{\myrb\myrb{}}
% 
% \mykw{indexvar} index\mysym{,} n \mysym{,} i \mysym{::=} \mysym{\mylb\mylb{}} \mykw{isa} nat \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{coq} nat \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{hol} num \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{ocaml} int \mysym{\myrb\myrb{}}
% 
%   \mykw{grammar}
% 
%     E \mysym{::} 'E_' \mysym{::=} \mysym{\mylb\mylb{}} \mykw{isa} ( ident * t ) list  \mysym{\myrb\myrb{}}
%       \mysym{|} < x1 : t1 , \mysym{..} , xn : tn > \mysym{::} \mysym{::} 2  \mysym{\mylb\mylb{}} \mykw{isa}  List.rev \mysym{[[}x1 t1 \mysym{..} xn tn\mysym{]]} \mysym{\myrb\myrb{}}
%  
%     t \mysym{::} 't_' \mysym{::=}
%       \mysym{|} unit               \mysym{::}   \mysym{::} unit
% 
%     K \mysym{::} 'K_' \mysym{::=}
%       \mysym{|} Type               \mysym{::}   \mysym{::} Type
% 
% 
% \mykw{formula} \mysym{::} formula_ \mysym{::=}          
%   \mysym{|} judgement              \mysym{::} \mysym{::} judgement
%   \mysym{|} formula1 \mysym{..} formulan   \mysym{::} \mysym{::} dots
% %  | formula1 .. formulan   :: :: realdots \mylb{}\mylb{} isa (List.list_all (\mybackslash{}<lambda> b . b) ( [[ formula1 .. formulan ]] ) ) \myrb{}\myrb{}
% 
% \mykw{terminals} \mysym{::} terminals_ \mysym{::=}
%   \mysym{|} |-                     \mysym{::}   \mysym{::} turnstile \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}vdash \mysym{\myrb\myrb{}}
%   \mysym{|} <                      \mysym{::}   \mysym{::} langle \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}langle \mysym{\myrb\myrb{}}
%   \mysym{|} >                      \mysym{::}   \mysym{::} rangle \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}rangle \mysym{\myrb\myrb{}}
% 
% \mykw{defns}
%   Jtype \mysym{::} '' \mysym{::=} 
% 
% \mykw{defn}
% |- E  \mysym{::} \mysym{::} Eok \mysym{::} Eok_ \mykw{by}
% 
% 
% \mysym{---------} :: 1
% |- < >
% 
% \mysym{----------------------------} :: 2
% |- <x1:t1,..,xn:tn> 
% 
% |- t1:K1  .. |- tn:Kn
% \mysym{----------------------------} :: 3
% |- <x1:t1,..,xn:tn> 
% 
% 
% \mykw{defn}|- t : K  \mysym{::} \mysym{::} tK \mysym{::} tK_ \mykw{by}
% 
% \mysym{---------------} :: 1
% |- unit : Type
% 
% \end{alltt}


%\par\noindent{\small
\begin{alltt}
    E \mysym{::} 'E_' \mysym{::=} \mysym{\mylb\mylb{}} \mykw{isa} ( ident * t ) list  \mysym{\myrb\myrb{}}
      \mysym{|} < x1 : t1 , \mysym{..} , xn : tn > \mysym{::} \mysym{::} 2  \mysym{\mylb\mylb{}} \mykw{isa}  List.rev \mysym{[[}x1 t1 \mysym{..} xn tn\mysym{]]} \mysym{\myrb\myrb{}}
    \mykw{formula} \mysym{::} formula_ \mysym{::=}          
      \mysym{|} judgement              \mysym{::} \mysym{::} judgement
      \mysym{|} formula1 \mysym{..} formulan   \mysym{::} \mysym{::} dots
\end{alltt}
%
%
%\begin{verbatim}
%E :: 'E_' ::= {{ isa ( ident * t ) list  }}
%  | < >                         :: :: 1  {{ isa  [] }}
%  | < x1 : t1 , ... , xn : tn > :: :: 2  {{ isa  List.rev [[x1 t1 ... xn tn]] }}
%
%formula :: formula_ ::=          
%  | judgement              :: :: judgement
%  | formula1 .. formulan   :: :: realdots {{ isa (List.list_all (\<lambda> b . b) ( [[ formula1 .. formulan ]] ) ) }}
%
%\end{verbatim}
%}
The generated Isabelle code for symbolic
terms mentioning this production will involve a list of pairs.  For
example, the rules
%\mysym{---------} :: 1
%|- < >
\begin{alltt}
\mykw{defn}
|- E  \mysym{::} \mysym{::} Eok \mysym{::} Eok_ \mykw{by}

\mysym{----------------------------} :: 2
|- <x1:t1,..,xn:tn> 

|- t1:K1  .. |- tn:Kn
\mysym{----------------------------} :: 3
|- <x1:t1,..,xn:tn> 
\end{alltt}
%\par\noindent{\small
%\begin{verbatim}
%defn
%|- E  :: :: Eok :: Eok_ by
%
%---------------------------- :: 2
%|- <x1:t1,...,xn:tn> 
%
%|- t1:K1  ... |- tn:Kn
%---------------------------- :: 3
%|- <x1:t1,...,xn:tn> 
% \end{verbatim}
%}
generate
\par\noindent{\small
\begin{verbatim}
consts
  Eok :: "E set"
inductive Eok tK
intros

(* defn Eok *)

Eok_2I: " ( List.rev  (x_t_list) ) : Eok"

Eok_3I: "[|
(List.list_all (\<lambda> b . b) (  ((List.map (%(x_,t_,K_). ( t_ , K_ ) : tK) x_t_K_list))  ) )|]
 ==> 
 ( List.rev  ((List.map (%(x_,t_,K_).(x_,t_)) x_t_K_list)) ) : Eok"
 \end{verbatim}
}
Note that in the second the list of pairs is projected out from the
\verb+x_t_K_list+ list of triples that is quantified over in the rule.


\section{Subrules}\mlabel{a14}%
Subrule declarations have the form
\begin{alltt}
  \mykw{subrules}
    nt1 \mysym{<::} nt2
\end{alltt}
where \texttt{nt1} and \texttt{nt2} are nonterminal roots. 


Subrules can be chained, i.e.~there can be a pair of
subrule declarations \texttt{nt1 \mysym{<::} nt2} and \texttt{nt2 \mysym{<::} nt3},
and they can form a directed acyclic graph, e.g.~with 
\texttt{nt0 \mysym{<::} nt1}, \texttt{nt0 \mysym{<::} nt2},
\texttt{nt1 \mysym{<::} nt3}, and \texttt{nt2 \mysym{<::} nt3}.  However, there cannot be
cycles, or  nonterminal roots for which there are multiple upper bounds. 
Subrule declarations should not involve nonterminal roots for which
proof-assistant type homs are specified. 

We support the case in which the upper rule is also
non-free, i.e.~it contains productions that mention nonterminals that
occur on the left of a subrule declaration. In the example below
(\texttt{test11.ott}) the
\texttt{t} rule contains a production \texttt{Foo v}.
\input{test11.alltt}
In this case generated Isabelle/Coq/HOL/OCaml will define a single type and both \texttt{is\_v}
and \texttt{is\_t} predicates, and the generated inductive definition
clause for \texttt{ax} uses both predicates.  The Isabelle clause is below.
\begin{verbatim}
  axI: "[|is_t t ; is_v v|] ==>  ( t , v ) : Baz"
\end{verbatim}


\section{Context rules}\mlabel{a69}%
The system supports the definition of single-hole contexts, e.g.~for
evaluation contexts.  For example, suppose one has a term grammar as below:
\begin{alltt}
t \mysym{::} 't_' \mysym{::=}                               \mysym{\mylb\mylb{}} \mykw{com} term    \mysym{\myrb\myrb{}}
  \mysym{|} x                  \mysym{::}  \mysym{::} Var                   \mysym{\mylb\mylb{}} \mykw{com} variable\mysym{\myrb\myrb{}}
  \mysym{|} \mybackslash{} x . t            \mysym{::}  \mysym{::} Lam \mysym{(+} \mykw{bind} x \mykw{in} t \mysym{+)} \mysym{\mylb\mylb{}} \mykw{com} lambda  \mysym{\myrb\myrb{}}
  \mysym{|} t t'               \mysym{::}  \mysym{::} App                   \mysym{\mylb\mylb{}} \mykw{com} app     \mysym{\myrb\myrb{}}
  \mysym{|} ( t1 , \mysym{....} , tn ) \mysym{::}  \mysym{::} Tuple                 \mysym{\mylb\mylb{}} \mykw{com} tuple \mysym{\myrb\myrb{}}
  \mysym{|} ( t )              \mysym{::} \mykw{S}\mysym{::} Paren                 \mysym{\mylb\mylb{}} \mykw{icho} \mysym{[[}t\mysym{]]}  \mysym{\myrb\myrb{}} 
  \mysym{|} \mylb{} t / x \myrb{} t'       \mysym{::} \mykw{M}\mysym{::} Tsub  
                        \mysym{\mylb\mylb{}} \mykw{icho} (tsubst_t \mysym{[[}t\mysym{]]} \mysym{[[}x\mysym{]]} \mysym{[[}t'\mysym{]]})\mysym{\myrb\myrb{}}

  \mysym{|} E . t              \mysym{::} \mykw{M}\mysym{::} Ctx  
                        \mysym{\mylb\mylb{}} \mykw{icho} (appctx_E_t \mysym{[[}E\mysym{]]} \mysym{[[}t\mysym{]]})\mysym{\myrb\myrb{}}
                        \mysym{\mylb\mylb{}} \mykw{tex} \mysym{[[}E\mysym{]]} \mybackslash{}cdot \mysym{[[}t\mysym{]]} \mysym{\myrb\myrb{}}
\end{alltt}
%
A context grammar is declared as a normal grammar but with a single
occurrence of the terminal \verb+__+ in each production, e.g. as in
the grammar for \texttt{E} below (a rather strange evaluation
strategy, admittedly).
%
\begin{alltt}
E \mysym{::} 'E_' \mysym{::=}                               \mysym{\mylb\mylb{}} \mykw{com} evaluation context \mysym{\myrb\myrb{}}
  \mysym{|} __ t         \mysym{::}  \mysym{::} AppL                   \mysym{\mylb\mylb{}} \mykw{com} app L    \mysym{\myrb\myrb{}}
  \mysym{|} v __         \mysym{::}  \mysym{::} AppR                   \mysym{\mylb\mylb{}} \mykw{com} app R    \mysym{\myrb\myrb{}}
  \mysym{|} \mybackslash{} x . __     \mysym{::}  \mysym{::} Lam                    \mysym{\mylb\mylb{}} \mykw{com} reduce under lambda \mysym{\myrb\myrb{}}
  \mysym{|} ( t1 ( __  t2 ) ) \mysym{::} \mysym{::} Nested              \mysym{\mylb\mylb{}} \mykw{com} hole nested \mysym{\myrb\myrb{}}
  \mysym{|} ( v1 , \mysym{..} , vm , __ , t1 , \mysym{..} , tn ) \mysym{::} \mysym{::} Tuple \mysym{\mylb\mylb{}} \mykw{com} tuple \mysym{\myrb\myrb{}}
\end{alltt}
%
A \mykw{contextrules} declaration:
\begin{alltt}
\mykw{contextrules}
  E \mysym{_::} t \mysym{::} t
\end{alltt}
causes Ott to (a) check that each production of the \texttt{E} grammar
is indeed a context for the \texttt{t} grammar, and (b) generates
proof assistant functions, e.g.~\verb+appctx_E_t+, to apply a context
to a term:
\begin{verbatim}
(** context application *)
Definition appctx_E_t (E5:E) (t_6:t) : t :=
  match E5 with
  | (E_AppL t5) => (t_App t_6 t5)
  | (E_AppR v5) => (t_App v5 t_6)
  | (E_Lam x) => (t_Lam x t_6)
  | (E_Nested t1 t2) =>  (t_App t1  (t_App t_6 t2) )
  | (E_Tuple v_list t_list) => (t_Tuple ((app_list_t v_list
     (app_list_t (Cons_list_t t_6 Nil_list_t) (app_list_t t_list Nil_list_t)))))
\end{verbatim}
As the \texttt{Nested} production shows, context productions can
involve nested term structure.

Note also that here the \texttt{E} grammar is not free (it mentions the
subrule nonterminal \texttt{v}) so an isvalue predicate
\verb+is_E_of_E+ is also generated.



In general, context rule declarations have the form
\begin{alltt}
  \mykw{contextrules}
    ntE \mysym{_::} nt1 \mysym{::} nt2
\end{alltt}
where \texttt{ntE}, \texttt{nt1}, and \texttt{nt2} are nonterminal
roots.  This declares contexts \texttt{ntE} for the \texttt{nt1}
grammar, with holes in \texttt{nt2} positions.

Just as for substitutions, the context application function is
typically used by adding a metaproduction to the term grammar. 
Here we add a production \texttt{E.t} to the \texttt{t} grammar with
an \texttt{icho} hom that uses \verb+appctx_E_t+.
\begin{alltt}
t \mysym{::} 't_' \mysym{::=}                               \mysym{\mylb\mylb{}} \mykw{com} term    \mysym{\myrb\myrb{}}
  ...
  \mysym{|} E . t              \mysym{::} \mykw{M}\mysym{::} Ctx  
                        \mysym{\mylb\mylb{}} \mykw{icho} (appctx_E_t \mysym{[[}E\mysym{]]} \mysym{[[}t\mysym{]]})\mysym{\myrb\myrb{}}
                        \mysym{\mylb\mylb{}} \mykw{tex} \mysym{[[}E\mysym{]]} \mybackslash{}cdot \mysym{[[}t\mysym{]]} \mysym{\myrb\myrb{}}
\end{alltt}
That can then be used in relations:
\begin{alltt}
    t --> t'
\mysym{    --------------} :: ctx
    E.t --> E.t'
\end{alltt}

One would typically also define a \texttt{terminals} production for
the hole terminal \verb+__+, e.g.~here we typeset the hole as $[\cdot]$.
\begin{alltt}
\mykw{terminals} \mysym{::} 'terminals_' \mysym{::=}
  \mysym{|} __           \mysym{::}  \mysym{::} hole    \mysym{\mylb\mylb{}} \mykw{tex} [\mybackslash{}cdot] \mysym{\myrb\myrb{}}
\end{alltt}


\section{Auxiliary Rules}
We permit an \texttt{aux} hom on grammar rules. For any rule with such a hom,
we transform that rule by appending an \texttt{\_aux} to its primary nonterminal
root name. We then add a synthesised rule with the original nonterminal
root name and a single production, with a shape described by the body of 
the aux hom, which must be of the form  
\begin{verbatim}
  {{ aux  foo1 foo2 _ bar1 bar2 bar3 }}
\end{verbatim}
with a single \texttt{\_} and any number of strings \texttt{fooi} and \texttt{barj} before and
after.  The \texttt{\_} is replaced by the original
nonterminal root name.  

For example, given a grammar or metavariable \texttt{l} of source locations, one
might say 
\begin{verbatim}
ntr :: 'NTR_'  ::=  {{ aux _ l }}
 | ...
\end{verbatim}
to synthesise grammars \texttt{ntr\_aux} and \texttt{ntr} of unannotated and location-annotated 
terms, the first with all the original productions and the second with a 
single production 
\begin{verbatim}
 | ntr l :: :: NTR_aux. 
\end{verbatim}
If the rule has an empty production name wrapper (eg with \texttt{''} in place
of \texttt{'NTR\_'}) then the production name is based on the original
nonterminal root, capitalised and with \texttt{\_aux} appended (eg \texttt{Ntr\_aux}), to
avoid spurious conflicts.

An additional \texttt{auxparams} hom on rules lets the user add type
parameters to the generated OCaml output. 


Generation of aux rules is controlled by a command-line option
\texttt{-generate\_aux\_rules}, which one might (eg) set to false
for latex output and true for OCaml output.


\section{Functions}

Ott includes experimental support for writing function definitions.
  As a simple example, consider the Ott file below:

\begin{alltt}
    \mykw{grammar}
     n \mysym{::} 'n_' \mysym{::=}
       | 0       \mysym{::} \mysym{::} Zero
       | S n     \mysym{::} \mysym{::} Succ

    \mykw{funs}
      Add \mysym{::=}   \mysym{\mylb\mylb{}} hol-proof ...  \mysym{\myrb\myrb{}}
    \mykw{fun}
      n1 + n2 \mysym{::} n \mysym{::} add    \mysym{\mylb\mylb{}} com a function of type num -> num -> num \mysym{\myrb\myrb{}}
    \mykw{by}
      0 + n2 \mysym{===} n2
      S n1 + n2 \mysym{===} n1 + S n2
\end{alltt}

Here the \verb+add+ function is compiled into the following Coq code:

\begin{alltt}
    Fixpoint add (x1:num) (x2:num) : num:=
      match x1,x2 with
      | n_zero , n2 => n2
      | (n_succ n1) , n2 =>  (add n1 (n_succ n2) ) 
    end.
\end{alltt}

More in detail, the 
    \mykw{fun}
      n1 + n2 \mysym{::} n \mysym{::} add
    \mykw{by}
 declaration specifies:
\begin{itemize}
\item the name of the function: \verb+add+
\item the symbolic term that defines the lhs: \verb-n1 + n2-
\item the non-terminal that defines the rhs: \verb+n+
\end{itemize}

The type of the arguments of the function is defined by the
 non-terminals appearing in the lhs, the return type by the rhs
 non-terminal (so \verb+num+ $\to$ \verb+num+ $\to$ \verb+num+ in the
 above example).  As side-effect, whenever a function of type
 \verb+symb_term+ $\to$ \verb+nt+ is defined, a production
 \verb+nt ::= symb_term+ is added to the definition of the
 non-terminal \verb+nt+ (in the above example, the production
 \verb-n1 + n2- is added to the grammar of \verb+num+).

Functions are then defined by case analysis, where the lhs and the rhs
are separated by the reserved symbol \mysym{===}.

The \mysym{\mylb\mylb{}} \verb+hol-proof+ \mysym{\myrb\myrb{}} hom
 allows the specification of a termination proof, which is required by
 Hol.  Mutually recursive functions can be defined in the same
 \mykw{funs} block, analogously to mutually recursive rule
 definitions.

\emph{Disclaimer:} the different treatment of partial functions by the
 different provers can result in a function definition being compiled
 correctly to one prover but not to others.




\section{Parsing Priorities}

Symbolic terms that can have more than one parse tree are typically considered
erroneous; however, certain classes of parse trees are ignored in order to
support common idioms that are ambiguous.  For example, the production 
%
\[\Gamma ::= \Gamma_1, .., \Gamma_n\]
%
might be used to allow a list of typing contexts to be appended together, but
it is highly ambiguous.  The following restrictions forbid many unwanted parses
that could otherwise occur.
%
\begin{itemize}
\item
All parses in which a nonterminal derives itself without consuming any input
are ignored.  For example, in the production above, the list could otherwise be
of length one so that $\Gamma$ directly derives $\Gamma$ giving rise to a
vacuous cycle, and an infinite forest of parse trees.  This restriction
ensures that only the tree without the vacuous cycle is considered.
\item
The parser for a list form ignores parses that unnecessarily break up the list
due to (direct or indirect) self reference.  For example,
$\Gamma_1,\Gamma_2,\Gamma_3,\Gamma_4$ will not parse as a two element sequence
of two element sequences $(\Gamma_1,\Gamma_2),(\Gamma_3,\Gamma_4)$ given the
production above.
\item
%\textit{\large Experimental Feature:} 
User supplied priority annotations in a \mykw{parsing} section rule out certain
trees as follows:
\begin{itemize}
\item
$\mathit{prodname}_1 \mysym{<=} \mathit{prodname}_2$: Parse trees where a
$\mathit{prodname}_1$ node is a child of a  $\mathit{prodname}_2$ node are
ignored.
\item
$\mathit{prodname}_1~\mykw{right}~\mathit{prodname}_2$: Parse trees where a
$\mathit{prodname}_1$ node is the leftmost child of a  $\mathit{prodname}_1$
node are ignored.
\item
$\mathit{prodname}_1~\mykw{left}~\mathit{prodname}_2$: Parse trees where a
$\mathit{prodname}_2$ node is the rightmost child of a  $\mathit{prodname}_1$
node are ignored.
\end{itemize}
In addition to immediate children, these priority annotations also prohibit
parse trees where the forbidden child node occurs underneath a chain of
derivations from the specified parent when the chain does not consume any
input.
Figure~\ref{parsing_figure} demonstrates a typical use of a \mykw{parsing}
section; the declarations have effect as follows:
\begin{itemize}
\item
Line \#1: \verb|n + n + n| parses as \verb|(n + n) + n|, but not \verb|n + (n + n)|;
\item
Line \#3: \verb|n + n - n| parses as \verb|(n + n) - n|, but not \verb|n + (n - n)|;
\item
Line \#9: \verb|-n + n| parses as \verb|(-n) + n|, but not \verb|-(n + n)|;
\item
Line \#15: \verb|n + n n| parses as \verb|n + (n n)|, but not \verb|(n + n) n|;
\verb|n n + n| parses as \verb|(n n) + n|, but not \verb|n (n + n)|;
\item
Line \#20: \verb|n, n n, n| parses as \verb|n, (n n), n|, but not \verb|(n, n) (n, n)|.
\end{itemize}
Currently, the \mykw{parsing} section supports only these relatively low level
and verbose declarations.
\end{itemize}

\begin{figure}

\begin{alltt}
\input{test21.1.alltt}
\label{parsing_figure}
\end{alltt}

\caption{An Ott source file for basic arithmetic using the typical parsing priorities}
\end{figure}

\section{Combining multiple source files}
Ott can be invoked with multiple source files.
% on the command line,
%either as multiple files at the end of the command line or as multiple
%\texttt{-i <filename>} options. 
%
Input filenames with extensions \texttt{.tex}, \texttt{.v},
\texttt{.thy}, \texttt{.sml}, or \texttt{ml} are simply copied into
the relevant output (\LaTeX, Coq, Isabelle, HOL, or OCaml). 
%
By default the source-file and command-line order of blocks is preserved, for grammar,
embeds, and inductive definitions.

The prover output can be split into multiple output files: each prover
output file specified with \texttt{-o <filename>} will contain the material
from the previous input files specified with \texttt{-i}  (since the last \texttt{-o}
for the same prover).


Alternatively, one can add a \texttt{-merge true} command-line option,
in which case
the productions of multiple grammars that share the same header
are merged into a single grammar, and the rules of multiple
inductive definitions that share the same header are merged into a
single inductive definition. 
%
This rudimentary form of modularity can be very useful, either to
split a language definition into separate features, or to define
reusable Ott components to define standard formulae, \myLaTeX{} pretty
printing of terminals, or \myLaTeX{} styles.  
For example, Figure~\ref{a12} shows the Ott source file for a
\texttt{let} feature in isolation, taken from our Ott development of
some languages from Pierce's TAPL~\cite{Pierce:TypeSystems}.
\begin{figure}
\framebox[\columnwidth]{\hspace{1.7mm}\begin{minipage}{\columnwidth}
%\footnotesize
\small
%\scriptsize
%\verbatiminput{../tests/test10.ott}
\input{let.alltt}\end{minipage}}
\caption{An \texttt{ott} source file for the \texttt{let} fragment of TAPL\protect\label{a12}}
\end{figure}
The original TAPL languages were produced using
TinkerType~\cite{LevinPierce99} to compose features and check for
conflicts. 
In \texttt{examples/tapl} we build a system, similar to the TinkerType
\texttt{sys-fullsimple}, from \texttt{ott} source files that
correspond roughly to the various TinkerType components, each with
syntax and semantic rules for a single feature. 




\section{Hom blocks}
Bindspecs and homomorphisms for productions, and any homomorphisms
for definitions, can appear in an Ott source file either 
attached to the production or definition, as we have shown earlier, or
in separate hom blocks.
For example, one can write
\begin{alltt}
\mykw{homs} 't_'
  \mysym{::} Lam  \mysym{(+} \mykw{bind} x \mykw{in} t \mysym{+)}  

\mykw{homs} 't_'
  \mysym{::} Var     \mysym{\mylb\mylb{}} \mykw{com} variable \mysym{\myrb\myrb{}}         
  \mysym{::} Lam     \mysym{\mylb\mylb{}} \mykw{com} abstraction \mysym{\myrb\myrb{}}      
  \mysym{::} App     \mysym{\mylb\mylb{}} \mykw{com} application \mysym{\myrb\myrb{}}      
  \mysym{::} paren   \mysym{\mylb\mylb{}} \mykw{ich} \mysym{[[}t\mysym{]]} \mysym{\myrb\myrb{}} 
  \mysym{::} tsub    \mysym{\mylb\mylb{}} \mykw{ich} ( tsubst_t \mysym{[[}t\mysym{]]} \mysym{[[}x\mysym{]]} \mysym{[[}t'\mysym{]]} ) \mysym{\myrb\myrb{}}

\mykw{homs} ''
  \mysym{::} reduce  \mysym{\mylb\mylb{}} \mykw{com} \mysym{[[}t1\mysym{]]} reduces to \mysym{[[}t2\mysym{]]} \mysym{\myrb\myrb{}} 
\end{alltt}
%
Each of these begins with a prefix and then has a sequence of
production name or definition name kernels, each followed by a
sequence of bindspecs and then a sequence of homomorphisms.

The \verb+test10_homs.ott+ example, in Fig.~\mref{a38}, shows this. It
is semantically equivalent to the \verb+test10.ott+ example of
Fig.~\mref{a68}, but the homs have been moved into \texttt{hom}
blocks.
\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\input{test10_homs.alltt}
\end{minipage}}
\caption{Hom Sections: \texttt{test10\_homs.ott}\protect\label{a38}}
\end{figure}


\section{Isabelle syntax support}\mlabel{a39}%
Ott has limited facilities to allow the Isabelle mixfix syntax support
and xsymbol to be used.  The example \verb+test10_isasyn.ott+ shows
this in use.  
%

Non-meta productions can be annotated with
\mykw{isasyn} and/or \mykw{isaprec} homomorphisms. 
For example, \verb+test10_isasyn.ott+ contains the production
\begin{alltt}
  \mysym{|} t t'      \mysym{::} \mysym{::} App   \mysym{\mylb\mylb{}} \mykw{isasyn} \mysym{[[}t\mysym{]]}\mybackslash{}<bullet>\mysym{[[}t'\mysym{]]} \mysym{\myrb\myrb{}}  \mysym{\mylb\mylb{}} \mykw{isaprec} 50 \mysym{\myrb\myrb{}}
\end{alltt}
The two homs are used to output the Isabelle syntax annotation in the
\verb+t_App+ clause of the datatype definition below.
\par\noindent{\small
\begin{verbatim}
t = 
   t_Var "termvar"  
 | t_Lam "termvar" "t"  ("\<lambda> _ . _" 60)
 | t_App "t" "t"  ("_\<bullet>_" 50)
\end{verbatim}
}
Definitions can be annotated with
\mykw{isasyn} and/or \mykw{isaprec} homomorphisms similarly, e.g.~as below.
\begin{alltt}
    \mykw{defn}
    t1 --> t2 \mysym{::}  \mysym{::} reduce \mysym{::} '' \mysym{\mylb\mylb{}} \mykw{isasyn} \mysym{[[}t1\mysym{]]} ---> \mysym{[[}t2\mysym{]]} \mysym{\myrb\myrb{}}  \mykw{by} 
\end{alltt}
This generates \verb+syntax+ and \verb+translations+ blocks as below.
\par\noindent{\small
\begin{verbatim}
inductive_set reduce :: "(t*t) set"
and  "reduce'" :: "t => t =>  bool" ("_ ---> _" 50)
where "(t1 ---> t2) ==  ( t1 , t2 ) : reduce"
\end{verbatim}
}
Symbolic terms in definitions are printed using any production or
definition syntax.  This (especially with xsymbol turned on) makes the
current goal state during Isabelle proof development much more
readable. 

Further, there is a command line option \verb+-isa_syntax true+.  If
this is set then the tool generates Isabelle syntax annotations from
the source syntax.  For example, the source file production for the
\verb+t_Lam+ clause is
\begin{alltt}
    \mysym{|} \mybackslash{} x . t             \mysym{::}   \mysym{::} Lam   \mysym{\mylb\mylb{}} \mykw{isaprec} 60 \mysym{\myrb\myrb{}}
\end{alltt}
and the \verb+terminals+ grammar contains a mapping from \verb+\+ to \verb+\<lambda>+:
\begin{alltt}
  \mykw{terminals} \mysym{::} 'terminals_' \mysym{::=}
    \mysym{|} \mybackslash{}                   \mysym{::}   \mysym{::} lambda  \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}lambda \mysym{\myrb\myrb{}}  \mysym{\mylb\mylb{}} \mykw{isa} \mybackslash{}<lambda> \mysym{\myrb\myrb{}}
    \mysym{|} -->                 \mysym{::}   \mysym{::} red     \mysym{\mylb\mylb{}} \mykw{tex} \mybackslash{}longrightarrow \mysym{\myrb\myrb{}} \mysym{\mylb\mylb{}} \mykw{isa} ---> \mysym{\myrb\myrb{}}
\end{alltt}
This is used (just as for \myLaTeX{} homs) to generate the \verb+("\<lambda> _ . _" 60)+
in the datatype definition above.


This functionality is limited in various ways:
(1) the full range of Isabelle precedence and associativity
specifications are not supported;
(2) the automatically generated syntax annotations are somewhat crude,
especially w.r.t.~spacing and parenthesisation;
(3) syntax annotation on meta productions is not propertly supported;
and
(4) it would be desirable to have more fine-grain control of whether
to automatically generate annotations: per-production, per-rule, and
per-file.


%For example, the \verb+out.thy+ produced by
%\verb+make test10_isasyn+ (which includes this option), the annotation
%on 


\section{Isabelle code generation example}\mlabel{a22}%
The Isabelle/Coq/HOL code generation facilities can be sometimes used to
generate (variously) OCaml and SML code from the Isabelle/Coq/HOL
definitions produced by Ott.

For example, the \verb+test10st_codegen.thy+ file uses Isabelle
code generation to produce SML code to calculate the possible
reductions of terms in the \verb+test10st.ott+ simply typed lambda
calculus.

\verbatiminput{../tests/test10st_codegen.thy}

\section{Reference: Command-line usage}\mlabel{a32}%
A good place to get started is one of the test
\texttt{make} targets in the \texttt{ott} directory, e.g.
\begin{verbatim}
test10: tests/test10.ott
           bin/ott                                                \
                  -i tests/test10.ott                             \ 
                  -o out.thy -o out.v -o outScript.sml            \
                  -o out.tex                                      \
                  -parse ":t:  (\z.z z) y"                        \                  
           && ($(LATEX) out; $(DVIPS) out -o)
\end{verbatim}
When \verb+make test10+ is executed, \texttt{ott}:
\begin{itemize}
\item reads the source file \verb+tests/test10.ott+
\item (if one also specifies \texttt{-show\_sort true} and \texttt{-show\_defns
true}) prints on standard output various diagnostic information,
  including ASCII versions of the grammar and
  inductive definitions.
  By default these are coloured (using
  \verb+vt220+ control codes) with metavariables in red, nonterminals
  in yellow, terminals in green, and object variables in white.
  Scanning over this output quickly picks up some common errors.
\item parses the symbolic term \verb+(\z.z z) y+ using the \verb+t+
  grammar and prints the result to standard output
\item generates Isabelle definitions in the file \verb+out.thy+
\item generates Coq definitions in the file \verb+out.v+
\item generates HOL definitions in the file \verb+outScript.sml+
\item generates a \myLaTeX{} document in the file \verb+out.tex+, with a
  standard document preamble to make it self-contained.
\end{itemize}
That \myLaTeX{} document is then compiled and converted to postscript. 

%If multiple source files are specified then they are (after parsing)
%effectively concatenated together.  

For convenience, input files can also be listed at the end of the command line:
\begin{verbatim}
  ott [options] <file1> ... <filen>
\end{verbatim}
is equivalent to
\begin{verbatim}
  ott -i <file1> ... -i <filen> [options]
\end{verbatim}


The~\verb+%.out+ \verb+Makefile+ target runs \verb+ott+ with common
defaults on the file~\verb+%.ott+, so for example executing \verb+make tests/test10.out+
runs \verb+ott+ on \verb+tests/test10.ott+, generating all outputs.  There are
also targets ~\verb+%.coq.out+, ~\verb+%.hol.out+, and
~\verb+%.isa.out+, to generate just LaTeX and the code for one proof
assistant, and ~\verb+%.tex.out+, to generate just LaTeX.



The \texttt{ott} command-line options (with default values where applicable) are
shown below. 
\verbatiminput{options.txt}

%By default the Makefile compiles Ott using \texttt{ocamlopt}.  To force it to
%compile with \texttt{ocamlc} (to get backtraces on exceptions) do \texttt{make opt}.


\section{Reference: The language of symbolic terms}\mlabel{a17}%
%
A syntax definition conceptually defines two different languages: that
of concrete terms of the object language, and that of symbolic terms
over the object language.  
The former includes concrete variables (if nontrivial \verb+lex+ homs
have been specified for metavariables). 
The latter includes the former but also
allows symbolic metavariables and nonterminals.
%
Symbolic terms may also include the %\verb+:deeper:+ and 
production-name
annotations mentioned in \S\mref{a51}.
For a syntax definition with list forms (c.f.~\S\mref{a61}) symbolic
terms also include various list constructs.
A simplified abstract syntax  of symbolic terms is shown in
Figure~\ref{a19}, omitting list forms.  In this section we give an
informal definition of the full concrete syntax of symbolic terms.
\begin{figure}[t]
\begin{toimage}
\framebox[\columnwidth]{ \begin{minipage}{\columnwidth}
\small
\symtermgrammar
\end{minipage}}
\end{toimage}%
\imageflush
\caption{Mini-Ott in Ott: symbolic terms\label{a19}}
\end{figure}%





The premises and conclusions of inductive definition rules are
symbolic terms.  \newcommand{\sem}[1]{[\![#1]\!]}
The language of symbolic terms is defined informally below, with interpretation functions $\sem{\_}$ that map defined entities into grammar clauses.

For a rule $\mathit{rule} \, = $
\[ \mathit{nontermroot}_1 \, , \, .. \, , \, \mathit{nontermroot}_n \texttt{::} \, \texttt{''} \, \texttt{::=} \, \mathit{prod_{{\mathrm{1}}}} \, .. \, \mathit{prod_{\mathit{m}}} \]
we have
\[\begin{array}{lcll}
 \sem{\mathit{rule}} & ::= & \\
& | & \mathit{nontermroot} \mathrm{suffix}   &   (1) \\
& | & \sem{\mathit{prod_{{\mathrm{1}}}}} \\
& | & ..  \\
& | & \sem{\mathit{prod_{\mathit{m}}}} 
  \end{array}\]

(1) for each $\mathit{nontermroot}$ in the set
    $\{\mathit{nontermroot}_1 \, , \, .. \, , \,
    \mathit{nontermroot}_n \}$  
and for each $\mathit{nontermroot}$ defined by any $\mathit{rule}'$
    which is declared as a subrule of this rule. 


For a production $ \mathit{prod} \, = $
\[ \texttt{|} \, \mathit{element_{{\mathrm{1}}}} \, .. \, \mathit{element_{\mathit{m}}} \, \texttt{::} \, \texttt{::} \, \mathit{prodname}\]
we have
\[\begin{array}{lcl}
 \sem{\mathit{prod}} & ::= & \\
& | & \sem{\mathit{element_{{\mathrm{1}}}}} \, .. \, \sem{\mathit{element_{{\mathit{m}}}}} \\
& | & \texttt{:}\mathit{prodname}\texttt{:}\, \sem{\mathit{element_{{\mathrm{1}}}}} \, .. \, \sem{\mathit{element_{{\mathit{m}}}}} 
\end{array}
\]


For an element there are various cases.  
\begin{enumerate}
\item For a terminal $\mathit{terminal}$
\[\begin{array}{lcl}
 \sem{\mathit{terminal}} & ::= & \mathit{terminal}\\
\end{array}\]
\item For a nonterminal $\mathit{nontermroot} \, \mathit{suffix}$
\[\begin{array}{lcl}
 \sem{\mathit{nontermroot} \, \mathit{suffix}} & ::= &  \sem{\mathit{rule}} 
\end{array}\]
where $\mathit{rule}$ includes $\mathit{nontermroot}$ among the
nonterminal roots it defines.  (Note that this does not depend on what $\mathit{suffix}$ was used in the grammar, and similarly for the $\mathit{metavar}$ case below.)

\item For an index variable $\mathit{indexvarroot}$
\[\begin{array}{lcl}
 \sem{\mathit{indexvarroot}} & ::= &  \mathit{indexvarroot}'
  \end{array}\]
for each $\mathit{indexvarroot}'$ defined by the \texttt{indexvar}
definition that defines $\mathit{indexvarroot}$.

\item For a metavariable $\mathit{metavarroot} \, \mathit{suffix}$
\[\begin{array}{lcll}
 \sem{\mathit{metavarroot} \, \mathit{suffix}} & ::= &  \\
& | & \mathit{metavarroot}' \mathrm{suffix} & (1) \\
& | & \mathit{variable} \\
\end{array}\]
(1) for each $\mathit{metavarroot}'$ defined by the \texttt{metavar}
definition that defines $\mathit{metavarroot}$.
(2) where $\mathit{variable}$ ranges over all the strings defined by
the \texttt{lex} regexp of the \texttt{metavar}
definition that defines $\mathit{metavarroot}$, except for any string
which can be parsed as a nonterminal, metavariable or terminal of the syntax
definition.
\item A list form element $\mathit{element}$ could be any of the following, either without a separating terminal:
\[\begin{array}{l}
\mathit{element}_1 .. \mathit{element}_n 
\, \mathit{dots} \,
\mathit{element}'_1 .. \mathit{element}'_n 
\\
\texttt{</} \,\mathit{element}_1 .. \mathit{element}_n \, \texttt{//} \, \mathit{indexvar} \, \texttt{/>} 
\\
\texttt{</} \,\mathit{element}_1 .. \mathit{element}_n \, \texttt{//} \, \mathit{indexvar} \, \texttt{IN} \, \mathit{indexvar}' \, \texttt{/>} 
\\
\texttt{</} \,\mathit{element}_1 .. \mathit{element}_n \, \texttt{//} \, \mathit{indexvar} \, \texttt{IN} \, \mathit{number} \, \mathit{dots} \, \mathit{indexvar}' \, \texttt{/>} 
\\
\texttt{</} \,\mathit{element}_1 .. \mathit{element}_n \, \texttt{//} \, \mathit{indexvar} \, \texttt{IN} \, \mathit{number} \, \mathit{dots} \, \mathit{indexvar}'\texttt{-1} \, \texttt{/>} 
\end{array}\]
or with a separating terminal:
\[\begin{array}{l}
\mathit{element}_1 .. \mathit{element}_n 
\, \mathit{terminal} \, \mathit{dots} \, \mathit{terminal} \,
\mathit{element}'_1 .. \mathit{element}'_n 
\\
\texttt{</} \,\mathit{element}_1 .. \mathit{element}_n \, \texttt{//} \,\mathit{terminal}\,\texttt{//}\, \mathit{indexvar} \, \texttt{/>} 
\\
\texttt{</} \,\mathit{element}_1 .. \mathit{element}_n \, \texttt{//} \,\mathit{terminal}\,\texttt{//}\, \mathit{indexvar} \, \texttt{IN} \, \mathit{indexvar}' \, \texttt{/>} 
\\
\texttt{</} \,\mathit{element}_1 .. \mathit{element}_n \, \texttt{//} \,\mathit{terminal}\,\texttt{//} \, \mathit{indexvar} \, \texttt{IN} \, \mathit{number} \, \mathit{dots} \, \mathit{indexvar}' \, \texttt{/>} 
\\
\texttt{</} \,\mathit{element}_1 .. \mathit{element}_n \, \texttt{//} \,\mathit{terminal}\,\texttt{//} \, \mathit{indexvar} \, \texttt{IN} \, \mathit{number} \, \mathit{dots} \, \mathit{indexvar}'\texttt{-1} \, \texttt{/>} 
\end{array}\]
In any of these cases the interpretation $\sem{\mathit{element}}$ is the lists (separated by the $\mathit{terminal}$ if one was specified) of concrete list entries and of list forms.
Without a separating $\mathit{terminal}$, this is: 
\[\begin{array}{lcll}
 \sem{\mathit{element}} & ::= & (\mathrm{concrete\_list\_entry} | \mathrm{list\_form})^* & (2), (3)\\
\ \\
\mathrm{concrete\_list\_entry} & ::= & \sem{\mathit{element}_1} \, .. \, \sem{\mathit{element}_n} \\
\ \\
\mathrm{list\_form} & ::= & \\
                   &  |  & \sem{\mathit{element}_1} \, .. \, \sem{\mathit{element}_n} 
                           \,\mathit{dots}'\,
                           \sem{\mathit{element}'_1} \, .. \, \sem{\mathit{element}'_n} & (1) \\

                   &  |  & \texttt{</} \,\sem{\mathit{element}_1} .. \sem{\mathit{element}_n} \, \texttt{//}\, \mathit{indexvar}'' \, \texttt{/>} 
\\
                   &  |  & \texttt{</} \,\sem{\mathit{element}_1} .. \sem{\mathit{element}_n}\,\texttt{//}\, \mathit{indexvar}'' \, \texttt{IN} \, \mathit{indexvar}''' \, \texttt{/>} 
\\
                   &  |  & \texttt{</} \,\sem{\mathit{element}_1} .. \sem{\mathit{element}_n}\,\texttt{//} \, \mathit{indexvar}'' \, \texttt{IN} \, \mathit{number}' \, \mathit{dots}' \, \mathit{indexvar}''' \, \texttt{/>} 
\\
                   &  |  & \texttt{</} \,\sem{\mathit{element}_1} .. \sem{\mathit{element}_n}\,\texttt{//} \, \mathit{indexvar}'' \, \texttt{IN} \, \mathit{number}' \, \mathit{dots}' \, \mathit{indexvar}'''\texttt{-1} \, \texttt{/>} 
\end{array}\]
This is subject to constraints: (1) that 
$\sem{\mathit{element}_1} \, .. \, \sem{\mathit{element}_n}$
and
$\sem{\mathit{element}'_1} \, .. \, \sem{\mathit{element}'_n}$ can be
anti-unified with exactly one varying index; (2) if the list has only
concrete entries (i.e., no list forms), its length must meet
the constraint of any $\mathrm{dots}$ in the
$\mathit{element}$.


With a separating $\mathit{terminal}$, we have: 
\[\begin{array}{lcll}
 \sem{\mathit{element}} & ::= & \epsilon | (\mathrm{concrete\_list\_entry} | \mathrm{list\_form})
(\mathit{terminal} (\mathrm{concrete\_list\_entry} | \mathrm{list\_form}))^*
\end{array}\]

\end{enumerate}

In the above 
\[\begin{array}{lcll}
\mathrm{dots} & ::= & \texttt{..} | \texttt{...} | \texttt{....} \\
\mathrm{number} & ::= & \texttt{0} | \texttt{1}  \\
\mathrm{suffix} & ::= & \mathrm{suffix\_item}^* \\
\mathrm{suffix\_item} & ::= & \\
                      &  |  & 
(\texttt{0} | \texttt{1} |\texttt{2} | \texttt{3} |\texttt{4} | \texttt{5} |\texttt{6} | \texttt{7} |\texttt{8} | \texttt{9})^+ & \mbox{(longest match)} \\
& | & \texttt{\_} \\
& | & \texttt{'} \\
& | & \mathit{indexvar} \\
& | & \mathit{indexvar}\texttt{-1} \\
\end{array}\]

Further, whitespace (\verb+' '|'\010'|'\009'|'\013'|'\012'+) is allowed before any token except a those in a suffix, and nonterminals, metavariables, index variables, and terminals that end with an alphanumeric character, must not be followed by an alphanumeric character.



The tool also builds a parser
for concrete terms, with fake nonterminal roots \texttt{concrete\_ntr}
for each primary \texttt{ntr} of the syntax definition.  One can
switch to concrete-term parsing with a \texttt{:concrete:} annotation,
as in the example 
\par\noindent{\small
\begin{verbatim}
\[ [[ :concrete: \Z1<:Top. \x:Z1.x ]]\]
\end{verbatim}
}
shown in Figure~\ref{a56}.  Below such an annotation, only concrete terms
are permitted, with no further annotation, no symbolic nonterminals or
metavariables, no list dot forms or comprehensions, etc.



Parsing of terms is done with a scannerless GLR parser over character-list
inputs.  The parser
searches for all parses of the input.  If none are found, the ASCII
and TeX output are annotated \texttt{no parses}, with a copy of the
input with \texttt{***} inserted at the point where the last token was
read.  This is often at the point of the error  (though if, for
example, a putative dot form is read but the two element lists cannot
be anti-unified, it will be after the point of the error). 
If multiple parses are found, the TeX output is annotated
\texttt{multiple parses} and the different parses are output to the
console in detail during the Ott run.

The GLR parser 
achieves reasonable performance on the small symbolic terms that are
typical in semantic rules.  Its performance on large (whole-program
size) examples is untested.

\section{Reference: Generation of proof assistant definitions}\mlabel{a42}%
This section briefly summarises the steps involved in the generation
of proof assistant definitions from an Ott source file.  For a description of the locally-nameless backend, refer to \ahrefurl{http://moscova.inria.fr/~zappa/projects/ln\_ott/}.
\subsection{Generation of types}
\begin{itemize}   
\item The primary metavariable roots and primary nonterminal roots are used
  directly as the names of proof assistant types, except where they
  have a hom specifying a root-overriding string.
\item Type abbreviation declarations are produced for metavars, in the
  source-file order, skipping metavars or nonterminals defined with \mykw{phantom}.
\item Type generation considers each rule of the user's source grammar
  except those for \verb+formula+ and \verb+terminals+ (or the
  synthesized rules for the syntax of judgements or \verb+user_syntax+).
\item The subrule order is analysed to identify the top elements. For
  each of those, a proof assistant type will be generated --- either a
  free type (\verb+coq+: \verb+inductive+, \verb+isa+:
  \verb+datatype+, \verb+hol+: \verb+Hol_datatype+), or if there is a
  type hom for the proof assistant in question, a type abbreviation.
  No types are generated for the non-top elements, as they will be
  represented as predicates over the top free type above them.
\item For the former, each non-meta production of the rule
  gives rise to a constructor.  The production name (with any per-rule
  prefix already applied) is used directly as the constructor name. 
  The (curried) constructor argument types are taken from the
  types associated with the metavariables and nonterminals mentioned
  in the production body. 
\item Rules (within each \mykw{grammar} block, if \texttt{-merge
false}, or all rules, if \texttt{-merge true}) are topologically sorted according to the dependency order
  (a free-type rule directly depends on another if one of its non-meta
  productions includes a nonterminal of the other; dependencies for  rules with a type-hom
  for the proof assistant in question
  are obtained from a crude lexing of the body of the type hom).
  We then generate mutually recursive type
   definitions for connected components, in an order consistent with
   the dependencies.
\item For productions that involve list dot forms or list comprehension
  forms, for HOL and Isabelle we produce constructors with argument
  types that involve native list types.  For Coq, however, we
  synthesise an additional inductive type for each list-of-tuples that arises
  (both for those that occur in the grammar and for others required in
  the translations of inductive definitions) and include them in the
  topological sort. 
\end{itemize}

\subsection{Generation of functions}
%
A small number of library functions  (\verb+list_mem+,
\verb+list_minus+,...) are included in the output if they are
required.

Several Coq list functions (\verb+map+, \verb+make+, \verb+unmake+, \verb+nth+, \verb+app+) are generated
for each synthesized list type.

The definitions of the more interesting functions (subrule predicates,
binding auxiliaries, free variable functions, and substitutions) are
generated over the free types generated for the maximal elements of the
subrule order  (generation of these functions for rules with type homs
is not supported).  The definitions are by pattern-matching and
recursion.
The patterns are generated by building canonical symbolic terms from
the productions of each relevant rule.
The recursion is essentially primitive recursion:
for Coq we produce \verb+Fixpoint+s or  \verb+Definition+s
(the latter is sometimes needed as the former gives an error in the case where there is no
recursion);
for Isabelle we produce \verb+primrec+s (or, experimentally, \verb+fun+s);
for HOL we use an \verb+ottDefine+ variant of the \verb+Define+
package.
%
In general we have to deal both with the type dependency 
(the topologically sorted mutually recursive types described above)
and with function dependency --- for example, for subrule predicates
and binding auxiliaries we may have multiple mutually recursive
functions over the same type.


For Coq the function generation over productions that involve list
types must mirror that, so we generate auxiliary functions 
that recurse over those list types. 

For Isabelle the \verb+primrec+ package does not support 
definitions involving several mutually recursive functions over the
same type, so for these we generate single functions calculating 
tuples of results, define the intended functions as projections of
these, and generate lemmas (and simple proof scripts) characterising
them in terms of the intended definitions.
%
Further, it does not support pattern matching involving nested
constructors.  We therefore generate auxiliary functions for
productions with embedded list types.  Isabelle tuples are treated as
iterated pairs, so we do the same for productions with tuples of size
3 or more. 
%
Isabelle also requires a function definition for each recursive type.
In the case where there are multiple uses of the same type
(e.g.~several uses of \verb+t list+ in different productions) all the
functions we wish to generate need identical auxiliaries.  As yet, the
tool does not generate the identical copies required.

If the option \verb+-isabelle_primrec+ is set to \verb+false+, then
Ott uses the \verb+fun+ package instead of the \verb+primrec+ package.
Since at the time of writing Isabelle 2008 is not capable of proving
automatically termination of all the \verb+fun+s that Ott generates,
this feature should be considered experimental.

For HOL the standard \verb+Define+ package tries an automatic
termination proof.  For productions that involve list types 
our generated functions involve various list functions which prevent
those proofs working in all cases.  
We therefore use an \verb+ottDefine+ variant (due to Scott Owens),
 with slightly stronger support for proving termination of
definitions involving list operators.



\subsubsection{Subrule predicates}\mlabel{a43}%
We generate subrule predicates to carve out the subsets of each free
proof assistant type (from the maximal elements of the subrule order) 
that represent the rules of the grammar. 
The non-free rules are the least subset of the rules that either
(1) occur on the left of a subrule (\verb+<::+) declaration, or
(2) have a (non-meta) production that mentions a non-free rule.  
Note that these can include rules that are maximal elements of the
subrule order, e.g.~if an expression grammar included a production
involving packaged values. 
The subrule predicate for a type is defined by pattern matching over
constructors of the maximal type above it --- for each non-meta
production of the maximal type it calculates a disjunction
over all the productions of the lower type that are subproductions of
it, invoking other subrule predicates as appropriate. 


\subsubsection{Binding auxiliaries}
The binding auxiliary functions calculate the intuitive semantics of auxiliary functions defined in bindspecs of the Ott source file.  Currently these are represented as proof assistant lists of metavariables or nonterminals (arguably set types should be used instead, at least in Isabelle).


\subsubsection{Free variables}
The free variable functions simply walk over the structure of the free proof assistant types, using any bind specifications (and binding auxiliaries) as appropriate. 
%
For these, and for substitutions, we simplify the generated functions by using the dependency analysis of the syntax to exclude recursive calls where there is no dependency. 


\subsubsection{Substitutions}
The generated substitution functions also walk over the structure of the free proof assistant types.  
For each production, for each occurrence of a nonterminal \verb+nt+ within it, we first calculate the things (of whatever type is in question) binding in that \verb+nt+, i.e.~those that should be removed from the domain of any substitution pushed down into it.  There are two cases:
(1) the \verb+mse'+ from any \verb+bind mse' in nt+;
%(2) the \verb+f(nt)+  which occur in the \verb+mse''+ of any \verb+bind mse'' in nt''+ 
%     in the production, i.e.~any auxiliary-defined things within \verb+nt+ which are used to bind elsewhere; and 
(2) \verb+nt+ itself if it occurs in the \verb+mse''+ of any \verb+bind mse'' in nt''+, i.e.~
\verb+nt+ itself if it is directly used to bind elsewhere.
List forms within bindspecs are dealt with analogously. 
%

The substitution function clause for a production is then of one of two forms:
either (1) the production comprises a single element, of the nonterminal or metavariable that we are substituting for, and this is within the rule of the nonterminal that it is being replaced by, or (2) all other cases. 
For (1) the element is compared with the domain of the substitution, and replaced by the corresponding value from the range if it is found.
For (2) the substitution functions are mapped over the subelements, having first removed any bound things from the domain of the substitution. 

This is all done similarly, but with differences in detail, for single and for multiple substitutions. 


\subsection{Generation of relations}
The semantic relations are defined
with the proof-assistant inductive relations packages
(\verb+coq+: \verb+Inductive+,
\verb+isa+: \verb+inductive+,
\verb+hol+: \verb+Hol_reln+).
%
They use the mutual recursion structure that is given by the user,
with each \verb+defns+ block giving rise to a potentially mutually
recursive definition of each \verb+defn+ inside it.
(It is debatable whether it would be preferable to do an automatic dependency analysis and topological sort, as for the syntax.)
%
Each definition rule gives rise to an implicational clause, essentially
that the premises (Ott \verb+formula+s) imply the conclusion (an Ott symbolic term of whichever judgement is being defined). 
In addition:
\begin{itemize}
\item Symbolic terms are transformed in various different ways:
\begin{itemize}
\item Nodes of non-meta productions are output as applications of the appropriate proof-assistant constructor (and, for a subrule, promoted to the corresponding constructor of a maximal rule).
\item Nodes of meta productions are transformed with the user-specified homomorphism.
\item Nodes of judgement forms are represented as applications of the defined relation in Coq and HOL, and as set-membership assertions in Isabelle.
\item Lists of formulae (the \verb+formula_dots+ production) are special-cased.
\end{itemize}
\item For each nonterminal of a non-free syntax rule (as in \S\mref{a43}) that occurs,
 e.g.~a usage of \verb+v'+ where \verb+v<::t+, an additional premise invoking the subrule predicate for the non-free rule is added, e.g.~\verb+is_v v'+.
\item The set of symbolic terms of the definition rule are analysed together to identify list forms with the same bounds.  A single proof assistant variable is introduced for each such, with appropriate projections and list maps/foralls at the usage points.
\item For Coq, auxiliary defined relations are introduced for list forms.
\item For Coq, as the projections from list forms involve (Ott-generated) \verb+nth+ functions that return option types, for any such projection a pattern-match against \verb+Some+ is introduced as an additional premise.
\item For Coq and HOL, explicit quantifiers are introduced for all variables mentioned in the rule.
\end{itemize}




\section{Reference: Summary of homomorphisms}\mlabel{a41}%
%
Homomorphisms can appear in various positions in an Ott source
file. The table below summarises their meanings.  A \tick{} indicates that arguments are meaningful for that  usage (e.g.~\texttt{\mysym{[[}e1\mysym{]]}} in a production mentioning a nonterminal or metavariable
\verb+e1+).

\begin{tabular}{lcl}
% Hu_root
\multicolumn{3}{l}{a metavar or indexvar declaration, after one of the defined metavar/indexvar roots, or}\\ 
\multicolumn{3}{l}{a rule, after one of the defined nonterminal
  roots}\\ \hline
\mykw{tex} & \tick & \myLaTeX{} typesetting for symbolic variables with that root\\
\mykw{isa}/\mykw{coq}/\mykw{hol}/\mykw{ocaml} &       & Isabelle/Coq/HOL/OCaml root overriding string (1) \\
\mykw{repr-locally-nameless} & & use a locally-nameless representation (Coq backend only) \\
\ \\
%
\end{tabular}


% Hu_metavar
\begin{tabular}{lcl}
\multicolumn{3}{l}{a metavar or indexvar declaration, after the \texttt{::=}}\\ \hline
%
\mykw{isa} &       & Isabelle representation type \\
\mykw{coq} &       & Coq representation type \\
\mykw{hol} &       & HOL representation type \\
\mykw{ocaml} &       & OCaml representation type \\
\mykw{tex} & \tick & \myLaTeX{} typesetting for symbolic variables \\
\mykw{com} &       & comment to appear in \myLaTeX{} syntax definition \\
\mykw{coq-equality} & & Coq proof script to decide equality over the representation type \\
\mykw{coq-notation} & & Coq sentence using \texttt{Notation} for the representation type \\
\mykw{coq-universe} & & Coq universe (e.g.~\texttt{Type}) for the representation type  \\
\mykw{repr-locally-nameless} & & (Coq only) use a locally-nameless representation \\
\mykw{phantom} & & suppress the representation type definition in theorem prover output \\
\mykw{lex} &       & regular expression for lexing concrete variables \\
\mykw{texvar} & \tick & \myLaTeX{} typesetting for concrete variables \\
\mykw{isavar} & \tick & Isabelle output for concrete variables \\
\mykw{holvar} & \tick & HOL output for concrete variables \\
\mykw{ocamlvar} & \tick & OCaml output for concrete variables \\
\ \\
%
\end{tabular}


% Hu_rule
\begin{tabular}{lcl}
%
\multicolumn{3}{l}{a rule, after the \mysym{::=}}\\ \hline
%
\mykw{isa} &       & Isabelle representation type, if a non-free type is required \\
\mykw{coq} &       & Coq representation type, if a non-free type is required \\
\mykw{hol} &       & HOL representation type, if a non-free type is required \\
\mykw{ocaml} &     & OCaml representation type, if a non-free type is required \\
\mykw{tex} & \tick & \myLaTeX{} typesetting for symbolic variables \\
\mykw{com} & \tick & comment to appear in \myLaTeX{} syntax definition \\
\mykw{coq-equality} & & Coq proof script to decide equality over the representation type \\
\mykw{coq-notation} & & Coq sentence using \texttt{Notation} for the representation type \\
\mykw{coq-universe} & & Coq universe (e.g.~\texttt{Type}) for the representation type  \\
\mykw{phantom} & & suppress the representation type definition in theorem prover output \\
\mykw{aux} & (\tick)& construct an auxiliary grammar rule with a single production \\
% \mykw{auxparams} & (\tick)& type parameters for OCaml  \\
\mykw{icho}  & \tick & shorthand for identical \mykw{coq}, \mykw{isa}, \mykw{hol}, and \mykw{ocaml} homs\\
\mykw{ichlo}  & \tick & shorthand for identical \mykw{coq},
\mykw{isa}, \mykw{hol}, \mykw{lem}, and \mykw{ocaml} homs\\
\mykw{ich}  & \tick & shorthand for identical \mykw{coq}, \mykw{isa} and \mykw{hol} homs\\
\mykw{ic}  & \tick & shorthand for identical \mykw{coq} and \mykw{isa} homs\\
\mykw{ch}  & \tick & shorthand for identical \mykw{coq} and \mykw{hol} homs\\
\mykw{ih}  & \tick & shorthand for identical \mykw{isa} and \mykw{hol} homs\\
\ \\
%
\end{tabular}

% Hu_prod
\begin{tabular}{lcl}
\multicolumn{3}{l}{a production}\\ \hline
%
\mykw{isa} & \tick & Isabelle output, for a non-free (meta) production \\
\mykw{coq} & \tick & Coq output, for a non-free (meta) production \\
\mykw{hol} & \tick & HOL output, for a non-free (meta) production \\
\mykw{ocaml} & \tick & OCaml output, for a non-free (meta) production \\
\mykw{tex} & \tick & \myLaTeX{} typesetting for symbolic terms  \\
\mykw{texlong} &  & typeset as long production  \\
\mykw{com} & \tick & comment to appear in \myLaTeX{} syntax definition \\
\mykw{order} & \tick & specify order of arguments to prover or Ocaml constructor\\
\mykw{isasyn} & \tick & Isabelle mixfix syntax output \\
\mykw{isaprec} &      & Isabelle mixfix syntax precedence string \\
\mykw{ich}  & \tick & shorthand for identical \mykw{coq}, \mykw{isa} and \mykw{hol}homs\\
\mykw{ic}  & \tick & shorthand for identical \mykw{coq} and \mykw{isa} homs\\
\mykw{ch}  & \tick & shorthand for identical \mykw{coq} and \mykw{hol} homs\\
\mykw{ih}  & \tick & shorthand for identical \mykw{isa} and \mykw{hol} homs\\
\ \\
%
\end{tabular}

% Hu_prod_tm
%
\begin{tabular}{lcl}
\multicolumn{3}{l}{a production of the \texttt{terminals} grammar}\\ \hline
%
\mykw{isa} &  & Isabelle output, for terminals in default generated Isabelle mixfix declarations \\
\mykw{tex} &  & \myLaTeX{} default typesetting for terms  \\
\mykw{com} & \tick & comment to appear in \myLaTeX{} syntax definition \\
\ \\
%
\end{tabular}

% Hu_defn
%
\begin{tabular}{lcl}
\multicolumn{3}{l}{a \mykw{defn}, before the \mykw{by}}\\ \hline
%
\mykw{tex} & \tick & \myLaTeX{} typesetting for symbolic terms  \\
\mykw{com} & \tick  & comment to appear in \myLaTeX{} syntax definition \\
\mykw{isasyn} & \tick & Isabelle mixfix syntax output \\
\mykw{isaprec} &      & Isabelle mixfix syntax precedence string \\
\mykw{lemwcf} &      & Lem witness, check, and functions spec for indrelns \\
\ \\
%
\multicolumn{3}{l}{a \mykw{homs} section clause (for a production or a definition)}\\ \hline
%
\multicolumn{3}{l}{as in the above production and \mykw{defn} forms}\\
\ \\
%
\end{tabular}

% Hu_defnclass
\begin{tabular}{lcl}
%
\multicolumn{3}{l}{a group of defns, after the \mysym{::=}}\\ \hline
\mykw{coq-universe} & & Coq universe (e.g.~\texttt{Type}) for the representation type  \\
\ \\
\end{tabular}


% Hu_embed
%
\begin{tabular}{lcl}
\multicolumn{3}{l}{an \mykw{embed} section}\\ \hline
\mykw{isa} &  & embedded Isabelle output\\
\mykw{coq} &  & embedded Coq output\\
\mykw{hol} &  & embedded HOL output\\
\mykw{ocaml} &  & embedded OCaml output\\
\mykw{tex} &  & embedded \myLaTeX{} output\\
\mykw{tex-preamble} &  & embedded \myLaTeX{} output, appearing in the \myLaTeX{} preamble\\
\mykw{coq-lib} &  & do not generate definitions for the listed helper functions\\
\mykw{isa-auxfn-proof} &  &  Isabelle proof script  \\
\mykw{isa-subrule-proof} &  &  Isabelle proof script \\
\end{tabular}


% Hu_subrule and Hu_subst and Hu_freevar
%
\begin{tabular}{lcl}
\multicolumn{3}{l}{in a subrule, substitution or function definition}\\ \hline
\mykw{isa-proof} &  & Isabelle proof script \\
\end{tabular}


(1) This is occasionally useful to work around a clash between a
    metavar or nonterminal primary root and a proof assistant symbol,
    e.g.~\texttt{value} in Isabelle or \texttt{T} in HOL.




\section{Reference: The Ott source grammar}
This is automatically generated (by \verb+mly-y2l+) from the
\verb+ocamlyacc+ grammar for Ott.
%\ifhevea
%(At present it does not display properly in html, only in the
%postscript/pdf versions of this manual.)
%\fi

The lexing of Ott source files is context-dependent; this does not
show that.

Not everything in the grammar is fully supported --- in particular,
option element forms, non-dotted element list forms, the three \verb+names+ distinctness forms of
bindspecs, and context rules.


\newcommand{\token}[1]{\texttt{#1}}
\newlength\rulelhs
\newlength\rulemid
\newlength\rulerhs
{\small
\ifhevea
\input{grammar_parser_hevea.tex}
\else
\input{grammar_parser.tex}
\fi}


\section{Reference: Examples}\mlabel{a33}%
The project web page
\begin{quotation}
\ahrefurl{http://www.cl.cam.ac.uk/users/pes20/ott/}
\end{quotation}
gives a variety of examples.  Some of these, and additional
 small examples, are included in the distribution in the \verb+tests+
directory. Typically they can be built using the \verb+Makefile+ in
the \verb+ott+ directory, e.g.~typing \verb+make test10+ or (more generally) \verb+make tests/test10.out+) there.

%  test-j.ott             HAT calculus
%  test1.ott              typing for functions and pairs
%  test2.ott              hom examples
%  test3.ott              typing and reduction for functions, let, ;, if
%  test4.ott              dot form examples
%  test5.ott              reduction for functions and pair patterns, using auxfns
%  test6-tex.ott          typing and reduction for functions and pairs, with com and tex homs
%  test6.ott              typing and reduction for functions and pairs, with com and tex homs
\begin{tabular}{ll}
 \verb+test10.ott+    &         untyped CBV lambda\\
 \verb+test10st.ott+  &         simply typed CBV lambda\\
 \verb+test8.ott+      &        ML polymorphism example\\
 \verb+test7a.ott+     &         POPLmark Fsub example (without records)\\
 \verb+test7b.ott+     &         POPLmark Fsub example (with records)\\
 \verb+leroy-jfp96.ott+&  Leroy module system \\
 \verb+lj.ott+       & LJ: Lightweight Java\\
 \verb+test7t.mng+     &        whole-document tex mng source\\
                       &   (\verb+make test7afilter+ to build) \\
 \verb+test7tt.mng+    &        fragment tex mng source\\
% test8p.ott             ML polymorphism example with record patterns
% test9.ott              notes on multiple language definitions
% test10st_metatheory.thy
% test10st_snapshot_out.thy
 \verb+test11.ott+     &        subrule example\\
 \verb+test12.ott+     &        topological sort example\\
 \verb+test13.ott+     &        small bindspec fragment\\
% test13b.ott            larger bindspec fragment
\verb+test10st_snapshot_out.thy+ & snapshot of generated Isabelle from \verb+test10st.ott+ \\
\verb+test10st_metatheory_autoed.thy+ & Isabelle proof script for type
preservation and progress \\
\verb+test10st_codegen.thy+ & Isabelle code generation script for reduction\\
\verb+test10_isasyn.ott+ & Isabelle mixfix syntax example\\
\verb+test10st_metatheoryScript.sml+ & HOL proof script for type preservation and progress \\
 \verb+test17.10.ott+     &        list comprehension examples\\
\end{tabular}

The \verb+examples/tapl+ directory contains several examples taken from the book `Types and Programming Languages' by Benjamin Pierce.  The \texttt{make} targets, listed below, combine Ott source files
following roughly the TinkerType component structure used in TAPL.

\begin{tabular}{ll}
 \verb+sys-bool+    &         booleans (p34) \\
 \verb+sys-arith+  &         arithmetic expressions (p41) \\
 \verb+sys-untyped+      &   untyped lambda-calculus with booleans \\
 \verb+sys-puresimple+     &   simply-typed lambda-calculus \\
 \verb+sys-tybool+     &    typed booleans \\
% \verb+sys-tuple+      & \\
 \verb+sys-sortoffullsimple+&   \\
 \verb+sys-roughlyfullsimple+&   \\
 \verb+sys-puresub+    &  \\
 \verb+sys-purercdsub+    &      \\
\end{tabular}

Other examples can be found on the locally-nameless backend web page.

\section*{Acknowledgements}
% 
We thank 
the Ott users for their feedback, especially Matthew Parkinson;
the other members of the POPLmark team, especially Benjamin
Pierce, Stephanie Weirich, and Steve Zdancewic, for 
discussions;
and Keith Wansbrough, Matthew Fairbairn, and Tom Wilkie, for their work
on various Ott predecessors.


We acknowledge the support of EPSRC grants
GR/T11715, EP/C510712, and EP/F036345, a Royal Society
University Research Fellowship (Sewell), an EPSRC Leadership
Fellowship (Sewell), 
EPSRC Programme Grant, EP/K008528/1 REMS \emph{Rigorous Engineering of
Mainstrem Systems},
and ANR grant ANR-06-SETI-010-02 (Zappa Nardelli). 


\bibliographystyle{alpha}%abbrv}%alpha}
%\bibliographystyle{abbrvnat}
%\bibliographystyle{plainnat}
\bibliography{shorter}

\end{document}
\clearpage

\section{OLD STUFF}


\begin{figure}
\fbox{\small\begin{minipage}{\textwidth}
\input{test10.7.alltt}
\end{minipage}}
\caption{Source: \texttt{test10.7.ott}\protect\label{a48}}
\end{figure}



-----------------------------



\textbf{This document is a preliminary user guide for the tool.  It does not
include discussion of the design choices, related work, future
directions, or detailed motivation.  }
%At present the tool is in a pre-alpha state: it can be used
%  for some examples, but it is far from polished or complete.}

The current state of some examples  is shown below.  The ``mt'' column indicates whether we have proved the usual type preservation/progress metatheory results for the generated definitions.
%\newcommand{\tick}{\mbox{$\surd$}}
%\newcommand{\cross}{\mbox{$\times$}}
%\newcommand{\myh}[1]{\multicolumn{2}{c}{#1}}

\newcommand{\myh}[1]{\multicolumn{2}{c}{#1}}
\newcommand{\myfile}[1]{\texttt{#1}\ }
\newcommand{\mysz}[2]{#2}
\[
%\hspace{-0.5cm}
{\small%scriptsize
\begin{tabular}{@{\,}l@{}l@{}r@{\,}c@{}c@{}cc@{}cc@{}c@{}c@{}c@{}} \hline 
                        & System                             &rules           & \myLaTeX& \myh{Coq\mbox{\ \ }}               & \myh{HOL}             & \myh{Isabelle}   \\%      & \myh{Twelf}                 \\ 
                        &                                    &               &       & defn     & mt           & defn      & mt        & defn      & mt  \\ \hline %       & defns     & mt              \\ \hline
\myfile{test10.ott}     &  untyped CBV lambda (Fig.~\mref{a10})&\mysz{  5}{  3}& \tick & \tick    &              & \tick     &           & \tick     &            &           &                 \\
\myfile{test10st.ott}   &  simply typed CBV lambda           &\mysz{ 12}{  6}& \tick & \tick    &\tick         & \tick     & \tick     & \tick     & \tick      &           &                  \\
\myfile{test8.ott}      &  ML polymorphism                   &\mysz{ 36}{ 22}& \tick & \tick    &              & \tick     &           & \tick     &            &           &                 \\ \hline
%\myfile{test7a.ott}     &  POPLmark F$_{<:}$                &\mysz{   }{  3}  & \tick &\tick$^1$&              &\tick$^1$  &           &\tick$^1$  &            &           &                  \\
\myfile{sys-roughlyfullsimple}&TAPL full simple              &\mysz{119}{ 63}& \tick & \tick    & \tick          & \tick     & \tick       & \tick     & \tick      &           &                  \\ \hline
%\myfile{sys-puresub}    &  TAPL - subtypes                  &\mysz{   }{  3} & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
%myfile{sys-purercdsub} &  TAPL - record subtypes            &\mysz{   }{  ?}& \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\ \hline
\myfile{test7b.ott}     &  POPLmark F$_{<:}$ with records%
%
    &\mysz{ 99}{ 48} & \tick &         &              &           &           &           &            &           &                  \\ \hline
%   &\mysz{ 99}{ 48} & \tick &  (1)    &              &(1)        &           & (1)       &            &           &                  \\ \hline
%                                                            &\mysz{   }{  3}                                                                                                                    
\myfile{leroy-jfp96.ott}&  Leroy JFP96 module system               &\mysz{142}{ 67}& \tick &          &              & \tick     &           &           &            &           &                  \\ \hline
\myfile{}               &  RG-Sep language                   &\mysz{   }{ 22}& \tick & \tick    & \tick        &           &           &           &            &           &                  \\ \hline
\myfile{}               &  Mini-Ott-in-Ott                   &\mysz{   }{ 55}& \tick &          &              &           &           & \tick     & \tick$^2$   &           &                  \\ \hline
%
\myfile{lj.ott}         &  LJ: Lightweight Java              &\mysz{105}{ 34}& \tick &          &              &           &           & \tick     & (3)        &           &                  \\ 
\myfile{        }         &  LJAM: Java Module System        &\mysz{  ?}{ 140}& \tick &          &              &           &           & \tick     &           &           &                  \\ \hline
%
%\myfile{sys-bool}       &  TAPL - boolean values             & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
%\myfile{sys-arith}      &  TAPL - integers                   & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
%\myfile{sys-puresimple} &  TAPL - simply typed lambda        & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
%\myfile{sys-tybool}     &  TAPL - stl + bool                 & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
%\myfile{sys-tuple}      &  TAPL - tuples                     & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
%\myfile{sys-sortoffullsimple} &  TAPL - ... lots ...         & \tick & \tick    &                & \tick     &             & \tick     &            &           &                  \\ 
%
%\myfile{minicaml\_plain} &  OCaml fragment                   & \tick & \tick    &              & \tick     &           &           &            &           &                  \\
\myfile{minicaml\_typedef} &  OCaml fragment                 &\mysz{612}{271}& \tick & \tick    &              & \tick     & \tick$^1$ &  \tick    &            &           &                  \\[0.5mm]
%
\hline
\multicolumn{8}{l}{\qquad\footnotesize \mbox{}$^1$ see below.  \quad\mbox{}$^2$ hand proofs.  \quad\mbox{}$^3$ in progress.}
\end{tabular}
}
\]

% {\scriptsize
% \begin{tabular}{llccccccccc} \hline 
% name                    & system                             & \myLaTeX& \myh{Coq}               & \myh{HOL}             & \myh{Isabelle}         & \myh{Twelf}                 \\ 
%                         &                                    &       & defns    & mt           & defns     & mt        & defns     & mt         & defns     & mt              \\ \hline
% \texttt{test10.ott}     &  untyped CBV lambda                & \tick & \tick    &              & \tick     &           & \tick     &            &           &                 \\
% \texttt{test10st.ott}   &  simply typed CBV lambda           & \tick & \tick    &\tick$^7$       & \tick     & \tick$^6$ & \tick     & \tick$^5$  &           &                  \\
% \texttt{test8.ott}      &  ML polymorphism                   & \tick & \tick    &              & \tick     &           & \tick     &            &           &                 \\ \hline
% \texttt{test7a.ott}     &  POPLmark F$_{<:}$                  & \tick &\tick$^1$&              &\tick$^1$  &           &\tick$^1$  &            &           &                  \\
% \texttt{test7b.ott}     &  POPLmark F$_{<:}$ with records     & \tick &\tick$^1$&              &\tick$^1$  &           &\tick$^1$  &            &           &                  \\ \hline
% %                                                                                                                                                                                
% \texttt{leroy-jfp96.ott}&  Leroy module system$^3$           & \tick &          &              & \tick     &           &           &            &           &                  \\ \hline
% %
% \texttt{lj.ott}         &  LJ: Lightweight Java$^4$          & \tick &          &              &           &           & \tick     & *$^4$      &           &                  \\ \hline
% %
% \texttt{sys-bool}       &  TAPL - boolean values             & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
% \texttt{sys-arith}      &  TAPL - integers                   & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
% \texttt{sys-puresimple} &  TAPL - simply typed lambda        & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
% \texttt{sys-tybool}     &  TAPL - stl + bool                 & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
% \texttt{sys-tuple}      &  TAPL - tuples                     & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
% \texttt{sys-sortoffullsimple} &  TAPL - ... lots ...         & \tick & \tick    &                & \tick     &             & \tick     &            &           &                  \\ 
% \texttt{sys-nearlyfullsimple}&TAPL - ... lots ...$^2$        & \tick & \tick    & *$^7$          & \tick     & \tick$^6$   & \tick     &            &           &                  \\ 
% \texttt{sys-puresub}    &  TAPL - subtypes                   & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\
% \texttt{sys-purercdsub} &  TAPL - record subtypes            & \tick & \tick    &              & \tick     &           & \tick     &            &           &                  \\ \hline
% %
% \texttt{minicaml\_plain} &  OCaml fragment                   & \tick & \tick    &              & \tick     &           &           &            &           &                  \\
% \texttt{minicaml\_typedef} &  OCaml fragment with type defs  & \tick & \tick    &              & \tick     &           &           &            &           &                  \\
% %
% \hline
% \end{tabular}
% }



\myaspect{TAPL full simple}
This covers most of the simple features, up to variants, from
TAPL~\cite{Pierce:TypeSystems}.
It demonstrates the utility of a very simple form of 
\emph{modularity} provided by \texttt{ott}, allowing clauses of grammars and
semantic relations to be split between files.
The original TAPL languages were produced using
TinkerType~\cite{LevinPierce99} to compose features and check for
conflicts. 
Here we build a system, similar to the TinkerType
\texttt{sys-fullsimple}, from \texttt{ott} source files that
correspond roughly to the various TinkerType components, each with
syntax and semantic rules for a single feature. 
The \texttt{ott} source for \texttt{let} is shown in Fig.~\mref{a12},
to which we add:
\texttt{common}, 
\texttt{common\_index}, 
\texttt{common\_labels}, 
\texttt{common\_typing}, 
\texttt{bool}, 
\texttt{bool\_typing}, 
\texttt{nat}, 
\texttt{nat\_typing}, 
\texttt{arrow\_typing}, 
\texttt{basety}, 
\texttt{unit}, 
\texttt{seq}, 
\texttt{ascribe}, 
%\texttt{let}, 
\texttt{product}, 
\texttt{sum}, 
\texttt{fix}, 
\texttt{tuple}, and
\texttt{variant}.
It also proved easy to largely reproduce the TAPL visual style and
(though we did no proof) to add subtyping.
%
\begin{figure}
\framebox[\columnwidth]{\hspace{1.7mm}\begin{minipage}{\columnwidth}
%\footnotesize
%\small
\scriptsize
%\verbatiminput{../tests/test10.ott}
\input{../examples/tapl/let_alltt.ott}\end{minipage}}
\caption{An \texttt{ott} source file for the \texttt{let} fragment of TAPL\protect\label{a12}}
\end{figure}

%\myaspect{POPLmark F$_{<:}$ with records}
%(1) As discussed in \S\mref{foo}, the generated proof assistant  definitions for F$_{<:}$ are well-formed
%    but the concrete variable representation in the generated code is
%    not satisfactory here --- this version of the type system
%    disallows all shadowing, so typing is not preserved by reduction.

\myaspect{POPLmark F$_{<:}$} 
The generated proof assistant  definitions for F$_{<:}$ are well-formed
    but the concrete variable representation in the generated code is
    not satisfactory here --- this version of the type system
    disallows all shadowing, so typing is not preserved by reduction.



\myaspect{Leroy JFP96 module system} (Owens)
This formalizes Leroy's path-based type system
\cite[\S4]{Leroy-generativity}, extended with a term language and an
operational semantics based on work of Owens~\cite{OF06}.  


\myaspect{RG-Sep language} (Vafeiadis, Parkinson)
This is a concurrent while language used for ongoing work combining
Rely-Guarantee reasoning with Separation Logic, defined and proved
sound by Vafeiadis and Parkinson~\cite{RGSEP}.

\myaspect{Mini-Ott-in-Ott}  
This precisely defines the \texttt{ott} binding specifications
(without list forms) with their fully concrete representation and
alpha equivalence. The metatheory here is a proof that for closed
substitutions the two coincide.  To date only a hand proof has been
completed; we plan to mechanize it in due course.

\myaspect{LJ and LJAM} (Strni\v sa, Parkinson)
LJ     is an imperative fragment of Java. 
LJAM extends that (again using \texttt{ott} modularity) with a
formalization of the core part of JSR-277 and a proposal for JSR-294,
which together form a proposal for a Java module system
\cite{ljam-sub}.

\myaspect{OCaml fragment} (Owens, Peskine, Sewell)
This covers a substantial core of OCaml --- to a first approximation,
all except subtyping, objects, and modules. 
Notable features that are handled are:
ML-style polymorphism; 
% (with the traditional value restriction, not
%      OCaml's relaxation);
pattern matching;
mutable references;
finiteness of the integer type;
%definitions of type aliases;
generative definitions of record and variant types; and
generative exception definitions.
%; and compilation units.
It does not cover much of the standard library, mutable records, arrays,
pattern matching guards, labels, polymorphic variants, objects, or
modules.
%
%
We have tried to make our definition mirror the behaviour of the
OCaml system rather closely.  The OCaml manual~\cite{ocaml2005}
defines the syntax with a BNF; our syntax is based on that.
It describes the semantics in prose; our semantics is based on a
combination of that and our experience with the language. 

\myaspect{Experience}
Our experience with these examples has been very positive.  The tool,
while not perfect, does make it easy to work with these
definitions, allowing one to focus on the content rather than the
proof assistant or \myLaTeX{} markup.  

For our most substantial example, the OCaml fragment, to date we have proved type preservation and
progress for the expression language, all machine-checked in HOL
except for one currently outstanding lemma (type substitution for type
variables) for
which we have a paper proof.  
Alpha equivalence arises only in a lemma showing that typing is preserved by renaming the bound variables of a type scheme in the type environment.
This proof effort has
taken only around 7 man-weeks, and the preceeding definition effort
was only another few man-weeks. 
Compared with our previous experiences this is remarkably lightweight: it
has been possible to develop this as an example, rather than requiring
a major research project in its own right. 
Apart from \texttt{ott}, the work has been aided by HOL's powerful first-order
reasoning automation and its inductive definition package, and by the
use of the fully concrete representation.


We have attempted to ensure that the proof assistant definitions
generated by Ott are well-formed and what the user would intend.  This
is not guaranteed, however, for several reasons: (1) There may be name
clashes between Ott-generated identifiers and proof assistant built-in
identifiers (or, in pathological cases, even among different
Ott-generated identifiers).  (2) In some cases we depend on automatic
proof procedures, e.g.~for HOL definitions.  These work in our test
cases, but it is hard to ensure that they will in all cases.  More
importantly, (3) the generation process is complex, so it is quite
possible that there is either a bug in Ott or a mismatch between the
user expectation and what the tool actually does.  Ultimately one has
to read the generated proof assistant definitions to check that they
are as intended --- but typically one would do this in any case, many
times over, in the process of proving metatheoretic results, so we do
not consider it a major issue.


\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\input{test10.0.alltt}
\end{minipage}}
\caption{Source: \texttt{test10.0.ott}\protect\label{a4}}
\end{figure}

\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\input{test10.2.alltt}
\end{minipage}}
\caption{Source: \texttt{test10.2.ott}\protect\label{a4}}
\end{figure}

\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\input{test10.4.alltt}
\end{minipage}}
\caption{Source: \texttt{test10.4.ott}\protect\label{a4}}
\end{figure}

\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\input{test10.7.alltt}
\end{minipage}}
\caption{Source: \texttt{test10.7.ott}\protect\label{a4}}
\end{figure}



\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\verbatiminput{../tests/test10.ott}
\end{minipage}}
\caption{Source: \texttt{test10.ott}\protect\label{a4}}
\end{figure}

\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\testTenall
\end{minipage}}
\caption{Generated \myLaTeX: \texttt{test10.tex}\protect\label{a3}}
\end{figure}


\section{Example: Untyped CBV Lambda Calculus}\mlabel{a2}%
We begin with a small example, an untyped call-by-value lambda
calculus.  Below we highlight the main points shown in this example
--- which uses many, but not all, of the features of Ott.  We return
to the options in more detail in later sections.

The complete Ott source file is shown in Fig.~\mref{a4}, the typeset
output in Fig.~\mref{a3}, and the generated Isabelle, Coq, and HOL files 
in Figs.~\mref{a5}, \mref{a6}, and \mref{a40}.
%  (the latter have been
%lightly hand-edited to remove redundant definitions so that they each fit
%on a page). 

Looking at the source file in Fig.~\mref{a4}, it begins by declaring a
metavariable form \texttt{x}, giving the Isabelle, Coq, and HOL  types used
(\texttt{string}, \texttt{nat}, and \texttt{string} respectively) to represent concrete
variables, and also specifying how metavariables should be typeset.

The following \texttt{grammar} block contains 4 grammar rules, defining
the (concrete and abstract) syntax for nonterminal roots \texttt{t},
\texttt{v}, \texttt{terminals}.
%
Each rule has a rule name prefix (e.g.~\texttt{t\_}) and then a list
of productions.  Each production, e.g.
\begin{verbatim}
    \mysym{|} \ x . t             ::   :: Lam  (+ bind x in t +)  {{ com abstraction }}      
\end{verbatim}
specifies a syntactic form as a list of elements, here `\verb+\+',
`\verb+x+', `\verb+.+', and `\verb+t+', each of which is either a
metavariable (the `\verb+x+'), a nonterminal (the `\verb+t+'), or a
terminal (the `\verb+\+' and `\verb+.+'). 
Within productions all elements must be whitespace-separated (so that
the tool can deduce which are terminals).  In the symbolic terms in
the rules below, however, whitespace is required only where necessary.


Metavariables and nonterminals can be formed from the specified
metavariable and nonterminal roots by appending a suffix, e.g.~the
nonterminal \verb+t'+ in the \verb+App+ and \verb+tsub+ productions. 

Terminals are not required to be declared in the \texttt{terminals}
grammar (all strings that are not metavariables or nonterminals are
regarded as terminals) but, if they are, then \myLaTeX{} pretty printing
information for them can be specified there.  In this example the
`\verb+\+' is included in the \texttt{terminals} grammar and will be
\myLaTeX pretty printed as a $\lambda$, whereas the `\verb+.+' is not
included and will be pretty printed in a default form. 
A few terminals have to be quoted in the grammar, as they are part of the Ott syntax, but they
do not have to be quoted at usage points. 



Between the \verb+::+s is an optional meta flag \verb+M+.  Non-meta
productions give rise to clauses of datatype definitions in the
Isabelle/Coq/HOL output, whereas meta productions do not. 

Each production has a production name (e.g.~\verb+t_Lam+), composed of
the rule name prefix (e.g.~\verb+t_+) and the production name kernel
that follows the \verb+::+s (e.g.~\verb+Lam+).  The production name is
used as a constructor name in the generated Isabelle/Coq/HOL. 

Following the production name is an optional binding specification
(e.g.~\verb-(+ bind x in t +)-), specifying which metavariables (or
nonterminals) bind in which nonterminals. This is used in the
generation of substitution and free variable functions.

At present the generated Isabelle/Coq/HOL uses completely concrete
representations of variables in terms, without any notion of alpha
equivalence, as one can see in Fig.~\mref{a5}:
see the \verb+t+ datatype of terms and the \verb+tsubst_t+
substitution function there.  
We intend in future to generate other representations.
For a reasonably wide variety of
languages, however, one can capture the intended semantics of whole programs in
this idiom, subject only to the condition that standard library
identifiers are not shadowed within the program, as the operational
semantics does not involve reduction under binders --- so any
substitutions are of terms which (except for standard library
identifiers) are closed.  This includes the ML polymorphism example of
\verb+test8.ott+.%\S\mref{a23}.  
Unfortunately, for languages which require a type
environment with internal dependencies, for example F$_{<:}$, this is
no longer the case, and the POPLmark F$_{<:}$ example given in 
\verb+test7.ott+ %\S\mref{a26}
has a type system which disallows all shadowing, a property that is
not preserved by reduction. 



Finally, there may be a number of homs, defining clauses of
homomorphisms from the grammar to strings.  Certain homs have special
meanings, including:

\begin{tabular}{ll}
\verb+com+ & defining comments used in \myLaTeX{} pretty prints of the grammar\\
\verb+tex+ & defining how this production (and terms formed by it) are \myLaTeX{} pretty printed\\
\verb+isa+ & (for meta productions) defining how the terms formed by
the production are translated into Isabelle\\
\verb+coq+ & similarly for Coq\\
\verb+hol+ & similarly for HOL\\
\verb+ich+  & a shorthand for defining an identical triple of \verb+isa+,
 \verb+coq+, and \verb+hol+ homs
\end{tabular}

Section \mref{a41} gives a summary of all the usable homomorphisms.

Following the \texttt{grammar} in this example is a \texttt{subrule}
declaration
\begin{verbatim}
  subrules
    v <:: t
\end{verbatim}
declaring that the \verb+v+ rule (of values) is a
subgrammar of the \verb+t+ rule (of terms).  The tool checks that
there is in fact a subgrammar relationship, i.e.~that for each
production of the lower rule there exists a production of the higher
rule with corresponding elements (up to the subrule relation).  
In the generated Isabelle/Coq/HOL for this example only one free
datatype is generated, for the \verb+t+ rule, whereas for the \verb+v+
rule we generate an \verb+is_v+ predicate over the \verb+t+ type.  Usages of
\verb+v+ nonterminals in the semantic rules have instances of this
predicate automatically inserted.

We continue with a substitutions declaration
\begin{verbatim}
  substitutions
    single t x :: tsubst 
\end{verbatim}
instructing the tool to generate Isabelle/Coq/HOL for a substitution
function replacing metavariables by terms.  This is for single
substitutions; multiple substitution functions (taking lists of
substitutand/substitutee pairs) can also be generated. 
Substitution functions are generated for all rules of the grammar for
which they might be required --- here, just over \verb+t+, with a
function named \verb+tsubst_t+.  Such functions can be used in the
\verb+isa+/\verb+coq+/\verb+hol+ homomorphisms for meta syntax, e.g. as in the
production 
\begin{verbatim}
    | { t / x } t'    :: M :: tsub    {{ ich ( tsubst_t [[t]] [[x]] [[t']] ) }}
\end{verbatim}
introducing syntax for term substitution.
 

Finally, we give a collection of definitions of inductive relations.
In this example there is just one family of definitions (of
operational judgements), called \verb+Jop+; it contains just one
definition of a relation, called \verb+E+.  The relation definition
also includes a grammar production specifying how elements of the
relation can be written and typeset, here
\begin{verbatim}
  t1 --> t2
\end{verbatim}
Syntax rules for each family of
judgements, and for their union, are implicitly generated. 
The relation definition is given by a sequence of inference rules,
each with a horizontal line separating a number of premises from a
conclusion, for example as below.
\begin{verbatim}
    t1 --> t1'
    -------------- :: ctx_app_arg
    v t1 --> v t1'
\end{verbatim}
The conclusion must be a symbolic term of the form of the judgement being
defined.
In simple cases (as here) the premises can be symbolic terms of the
form of any of the defined judgements.  More generally (see
\S\mref{a13}) they can be symbolic terms of a user-defined
\texttt{formula} grammar. 
Each rule
 has a name, composed of a definition family prefix
(here empty), a definition prefix (here also empty) and a kernel
(the~\verb+ctx_app_arg+). 









\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\verbatiminput{test10.thy}
%\verbatiminput{test10.hand.edited.thy}
\end{minipage}}
\caption{Generated Isabelle:\texttt{test10.thy}\protect\label{a5}}
\end{figure}

\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\verbatiminput{test10.v}
%\verbatiminput{test10.hand.edited.v}
\end{minipage}}
\caption{Generated Coq:\texttt{test10.v}\protect\label{a6}}
\end{figure}

\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\verbatiminput{test10Script.sml}
\end{minipage}}
\caption{Generated HOL:\texttt{test10Script.sml}\protect\label{a40}}
\end{figure}

  
% \section{\texttt{test1}}
% \testOneall
% 
% \section{\texttt{test8}}
% \testEightall

\clearpage

\section{Command-line Usage}\mlabel{a32}%
The best place to get started is probably one of the test
\texttt{make} targets in the \texttt{ott} directory, e.g.
\begin{verbatim}
test10: tests/test10.ott
           bin/ott                                                \
                  -isabelle out.thy -coq out.v -hol outScript.sml \
                  -tex out.tex                                    \
                  -parse ":t:  (\z.z z) y"                        \
                  tests/test10.ott                                \
           && ($(LATEX) out; $(DVIPS) out -o)
\end{verbatim}
When \verb+make test10+ is executed, \texttt{ott}:
\begin{itemize}
\item reads the source file \verb+tests/test10.ott+
\item prints on standard output various diagnostic information,
  including ASCII versions of the grammar and
  inductive definitions.
  By default these are coloured (using
  \verb+vt220+ control codes) with metavariables in red, nonterminals
  in yellow, terminals in green, and object variables in white.
  Scanning over this output quickly picks up some common errors.
\item parses the symbolic term \verb+(\z.z z) y+ using the \verb+t+
  grammar and prints the result to standard output
\item generates Isabelle definitions in the file \verb+out.thy+
\item generates Coq definitions in the file \verb+out.v+
\item generates HOL definitions in the file \verb+outScript.sml+
\item generates a \myLaTeX{} document in the file \verb+out.tex+, with a
  standard document preamble to make it self-contained.
\end{itemize}
That \myLaTeX{} document is then compiled and converted to postscript. 

If multiple source files are specified then they are (after parsing)
concatenated together. 


The~\verb+%.out+ \verb+Makefile+ target runs \verb+ott+ with common
defaults on the file~\verb+%.ott+, so for example executing \verb+make tests/test10.out+
runs \verb+ott+ on \verb+tests/test10.ott+, generating all outputs.  There are
also targets ~\verb+%.coq.out+, ~\verb+%.hol.out+, and
~\verb+%.isa.out+, to generate just LaTeX and the code for one proof
assistant, and ~\verb+%.tex.out+, to generate just LaTeX.



The \texttt{ott} command-line options (with default values where applicable) are
shown below. 
\verbatiminput{options.txt}

%By default the Makefile compiles Ott using \texttt{ocamlopt}.  To force it to
%compile with \texttt{ocamlc} (to get backtraces on exceptions) do \texttt{make opt}.

\section{Examples}\mlabel{a33}%
The distribution contains several small examples in the \verb+tests+
directory. Typically they can be built using the \verb+Makefile+ in
the \verb+ott+ directory, e.g.~typing \verb+make test10+ or (more generally) \verb+make tests/test10.out+) there.

%  test-j.ott             HAT calculus
%  test1.ott              typing for functions and pairs
%  test2.ott              hom examples
%  test3.ott              typing and reduction for functions, let, ;, if
%  test4.ott              dot form examples
%  test5.ott              reduction for functions and pair patterns, using auxfns
%  test6-tex.ott          typing and reduction for functions and pairs, with com and tex homs
%  test6.ott              typing and reduction for functions and pairs, with com and tex homs
\begin{tabular}{ll}
 \verb+test10.ott+    &         untyped CBV lambda\\
 \verb+test10st.ott+  &         simply typed CBV lambda\\
 \verb+test8.ott+      &        ML polymorphism example\\
 \verb+test7a.ott+     &         POPLmark Fsub example (without records)\\
 \verb+test7b.ott+     &         POPLmark Fsub example (with records)\\
 \verb+leroy-jfp96.ott+&  Leroy module system \\
 \verb+lj.ott+       & LJ: Lightweight Java\\
 \verb+test7t.mng+     &        whole-document tex mng source\\
                       &   (\verb+make test7afilter+ to build) \\
 \verb+test7tt.mng+    &        fragment tex mng source\\
% test8p.ott             ML polymorphism example with record patterns
% test9.ott              notes on multiple language definitions
% test10st_metatheory.thy
% test10st_snapshot_out.thy
 \verb+test11.ott+     &        subrule example\\
 \verb+test12.ott+     &        topological sort example\\
 \verb+test13.ott+     &        small bindspec fragment\\
% test13b.ott            larger bindspec fragment
\verb+test10st_snapshot_out.thy+ & snapshot of generated Isabelle from \verb+test10st.ott+ \\
\verb+test10st_metatheory_autoed.thy+ & Isabelle proof script for type
preservation and progress \\
\verb+test10st_codegen.thy+ & Isabelle code generation script for reduction\\
\verb+test10_isasyn.ott+ & Isabelle mixfix syntax example\\
\verb+test10st_metatheoryScript.sml+ & HOL proof script for type preservation and progress \\
 \verb+test17.10.ott+     &        list comprehension examples\\
\end{tabular}

The \verb+examples/tapl+ directory contains several examples taken from the book `Types and Programming Languages' by Benjamin Pierce.  These can be built using the \verb+Makefile+ in
the \verb+ott+ directory.  The targets, listed below, combine \verb+ott+ source files
following roughly the TinkerType component structure used in TAPL.

\begin{tabular}{ll}
 \verb+sys-bool+    &         booleans (p34) \\
 \verb+sys-arith+  &         arithmetic expressions (p41) \\
 \verb+sys-untyped+      &   untyped lambda-calculus with booleans \\
 \verb+sys-puresimple+     &   simply-typed lambda-calculus \\
 \verb+sys-tybool+     &    typed booleans \\
 \verb+sys-tuple+      & \\
 \verb+sys-sortoffullsimple+&   \\
 \verb+sys-roughlyfullsimple+&   \\
 \verb+sys-puresub+    &  \\
 \verb+sys-purercdsub+    &      \\
\end{tabular}



\section{Features}
This section describes the main Ott features.  It assumes the reader
has read \S\mref{a2}.
\myparagraph{Grammar}\mlabel{a7}%
The tool supports arbitrary context-free grammars, extended with
special constructs for dotted list forms (c.f.~\S\mref{a21}). 
%
It builds a recursive descent parser for parsing symbolic terms in
semantic rules, finding all parses up to a certain depth, and reports
an error if there are multiple parses or no parses.
%
To resolve ambiguity one can add metaproductions for parentheses (as
in Fig.~\mref{a4}), or 
production-name annotations in particular symbolic terms,
e.g.~the \verb+:t_tsub:+ in the \verb+AppAbs+ rule of the POPLmark
example, \S\mref{a27}.
%
%The current parser implementation is very naive; its performance on
%larger terms can be a problem.  Occasionally one must add a
%\verb+:deeper:+ annotation to a symbolic term to increase the search
%depth, as in the \verb+ax_app+ rule of Fig.~\mref{a4}.


\myparagraph{Lexing}\mlabel{a8}%
There are fixed lexical conventions.
In a production of the grammar specification, 
the elements must be blank-separated (blanks are
\verb+( |\010|\009|\013|\012)+).

% Nonterminals and metavariables must be idents:
% \begin{verbatim}
%   ident = ([A-Z]|[a-z]|[_]) ([A-Z]|[a-z]|[0-9]|[_]|['])* 
% \end{verbatim}
% Terminals must either be idents or strings that do not contain blanks
% and do not start with \verb+([A-Z]|[a-z])+.
% 
% (These conventions are broadly similar to those of OCaml, except that,
% for ease of \myLaTeX{} output, we disallow accented characters in
% idents.  

(To precisely express the concrete syntax of existing full
languages would typically need more flexibility.)

To allow the Ott tokens to appear in an object language, elements in
a grammar specification may be quoted, \verb+'thus'+.  Any string
surrounded by a pair of single quotes has those quotes removed.



\myparagraph{Metavariables, Nonterminals, and Suffixes}\mlabel{a9}%
Declarations of metavariable roots and rules defining nonterminal
roots can specify any number of synonyms, for example the
\verb+termvar+, \verb+x+, \verb+y+, and \verb+z+ in 
\begin{verbatim}
  metavar termvar , x , y , z ::=  {{ isa string }} {{ coq nat }} {{ hol num }} {{ ocaml int }}
\end{verbatim}
and the \verb+t+ and \verb+u+ in 
\begin{verbatim}
  t , u :: '' ::=                                        
               |  x              :: :: Var                     
               |  \ x : T . t    :: :: Lam  (+ bind x in t +)  
               |  t t'           :: :: App                     
\end{verbatim}
The first such (here \verb+termvar+ and \verb+t+) is taken to be the
primary root, and is used to build the corresponding type name in
Isabelle/Coq/HOL/OCaml output. 

Occurrences of metavariables and nonterminals, both in productions of
the grammar specification and in symbolic terms in semantic rules, 
can be formed by adding a suffix to a metavariable or nonterminal
root, for example the \verb+t'+ in the \verb+App+ rule above.  

The user can declare any number of distinguished `index'
metavariables, e.g. by:
\begin{verbatim}
  indexvar index , i , j , n , m ::= {{ isa num }} {{ coq nat }} {{ hol num }} {{ ocaml int }}
\end{verbatim}
Given such a declaration, \verb+index+, \verb+i+, \verb+j+, \verb+n+
and \verb+m+ can be used in suffixes, e.g.~in the production
\begin{verbatim}
               |  ( t1 , .... , tn )           :: :: Tuple
\end{verbatim}
There is a fixed ad-hoc language of suffixes, including numbers, primes, and index variables (see \S\mref{a17}).
Index metavariables cannot themselves be suffixed.


Metavariable declarations can include a \verb+lex+ homomorphism
specifying the syntactic form of the corresponding concrete
variables, and an \verb+isavar+ homomorphism specifying how such
strings should be translated into Isabelle.  For example, one might write
\begin{verbatim}
  metavar typevar , X ::= {{ isa string }} {{ lex Alphanum }} 
  metavar termvar , x ::= {{ isa string }} {{ lex alphanum }} 
\end{verbatim}
Similarly, one can specify how concrete variables should be \myLaTeX'd, e.g.~with
the homomorphisms \verb+{{ texvar \mathrm{[[typevar]]} }}+ and \verb+{{ isavar ''[[termvar]]'' }}+.

\textbf{At present \texttt{lex} homomorphism must have body either}
 \texttt{Alphanum}  (\verb+[A-Z]([A-Z]|[a-z]|[0-9]|'|_)*+),
 \texttt{alphanum} (\verb+([A-Z]|[a-z])([A-Z]|[a-z]|[0-9]|'|_)*+),
 \texttt{alphanum0} (\verb+[a-z]([A-Z]|[a-z]|[0-9]|'|_)*+), or
 \texttt{numeral} (\verb+[0-9][0-9]*+); \textbf{more general regexps are not supported.}

One can likewise specify how metavariables themselves should be
\myLaTeX'd, e.g.~with a homomorphism
\verb+{{ tex \mathit{[[typevar]]} }}+.

For Coq output, one can specify \verb+{{ coq-equality+ \textit{proof-script} \verb+}}+
to build a decidable equality over the Coq representation type using
the proof \textit{proof-script}.  If the script is omitted it defaults
to 
\begin{verbatim}
Proof.
  decide equality; auto with ott_coq_equality arith.
Defined.
\end{verbatim}
where the \verb+ott_coq_equality+ database contains the decidable
equalities of the representation types defined in the source.

\myparagraph{Grammar Rule Homomorphisms}\mlabel{a11}%
%
Grammar rules can be annotated with various homomorphisms.

First, the individual nonterminal roots can have \verb+tex+
homomorphisms specifying how they should be typeset, e.g.~
\begin{verbatim}
G {{ tex \Gamma }} , D {{ tex \Delta }} :: 'G_' ::= {{ com type environment }}
    | empty                 ::   :: empty       
    | G , x : T             ::   :: term        
\end{verbatim}
permitting the user to write \verb+G'+, \verb+D12+ etc.~in symbolic
terms, to be typeset at $\Gamma'$, $\Delta_{12}$, etc.

Second, the rule can include a \verb+tex+ hom specifying how all the
  nonterminal roots should be typeset, e.g.
\begin{verbatim}
  type , t , s :: Typ_ ::=  {{ tex \mathsf{[[type]]} }} 
        | unit                  ::   :: unit   
        | type * type'          ::   :: pair   
        | type -> type'         ::   :: fun    
\end{verbatim}
Here that \verb+tex+ hom overrides the default typesetting with
\verb+\mathsf+.

Third, the rule can include a \verb+com+ hom giving a comment to
include in typeset versions of the grammar itself, e.g.~the
\verb+{{ com type environment }}+ above.

Fourth, the rule can include a \verb+coq equality+ hom, instructing
the Coq code generator to derive a decidable equality for the Coq
representation type.  For example, the ML polymorphism Ott source of
\texttt{test8.ott} includes the following.
\begin{verbatim}
  typvar :: TV_ ::= {{ coq-equality decide equality. apply eq_value_name_t. }}
       | ' ident                ::   :: ident
\end{verbatim}

Fifth, and most fundamentally, the rule can include \verb+isa+, \verb+coq+, 
\verb+hol+, or \verb+ocaml+ homs specifying a proof-assistant (or OCaml) type, e.g. in the
following rule for substitutions.
\begin{verbatim}
  s {{ tex \sigma }} :: 'S_' ::= {{ com multiple subst }} {{ isa (termvar*t) list }}
       | [ x |-> t ]            ::   :: singleton   {{ isa [ ([[x]],[[t]]) ]  }}
       | s1 , .. , sn           ::   :: list        {{ isa List.concat s_list }}
\end{verbatim}
Here the \verb+{{ isa (termvar*t) list }}+ hom specifies that in
Isabelle output this type be represented as an Isabelle
\verb+(termvar*t) list+ instead of the (default) free grammar. 



\myparagraph{Production Homomorphisms}\mlabel{a12}%
Each production may also have \verb+coq+ and \verb+isa+ homomorphisms,
specifying how symbolic terms should be translated into Isabelle/Coq/HOL/OCaml
expressions, e.g.~as in the last example of the previous
section.  This overrides the default translation into terms of a free
grammar. 

Productions may also have \verb+tex+ homs, overriding the default
\myLaTeX{} typesetting, e.g.~as in this example of
a type abstraction production from \texttt{test7.ott}.
\begin{verbatim}
    | \ X <: T . t   :: :: TLam  (+ bind X in t +)  
                                 {{ com type abstraction }} 
                                 {{ tex \Lambda [[X]] [[<:]] [[T]]. \, [[t]] }}
\end{verbatim}
They may also have a comment, e.g.~the \verb+{{ com type abstraction }}+, to
appear in typeset versions of the grammar itself. 

All these homomorphisms can refer to the metavariables and
nonterminals that occur in the production, e.g.~the \verb+[[X]]+,
\verb+[[T]]+, and \verb+[[t]]+ in the \verb+tex+ hom above,
interleaved with arbitrary strings and with typeset elements of the
\verb+terminals+ grammar, e.g.~the \verb+[[<:]]+.

Homomorphisms are applied recursively down the structure of symbolic
terms. For example, the \texttt{TappTabs} rule of \texttt{test7.ott} has a
conclusion with a subterm
\begin{verbatim}
  (\X<:T11.t12) [T2]
\end{verbatim}
This is \myLaTeX-pretty-printed, using the \verb+tex+ homomorphism
clause above, as
\begin{verbatim}
( \, \Lambda  \mathit{X} <: \mathit{T_{\mathrm{11}}} . \, \mathit{t_{\mathrm{12}}} \, )
 \, \, [ \, \mathit{T_{\mathrm{2}}} \, ]
\end{verbatim}
which is typeset as below.
\[
( \, \Lambda  \mathit{X}   <:   \mathit{T_{\mathrm{11}}} . \,  \mathit{t_{\mathrm{12}}} \, ) \,  \, [ \, \mathit{T_{\mathrm{2}}} \, ]
\]
Note the \verb+X+, \verb+T11+ and \verb+t12+ of the symbolic term are
used to instantiate the formal parameters \verb+X+, \verb+T+ and
\verb+t+ of the homomorphism definition clause. 
%
If the \verb+t+ itself had compound term structure, e.g. as below
\begin{verbatim}
  (\X<:T. \X'<:T'.x)
\end{verbatim}
the homomorphism would be applied recursively, producing 
\begin{verbatim}
( \, \Lambda  \mathit{X} <: \mathit{T} . \,  \Lambda  \mathit{X'} <: \mathit{T'} 
. \,  \mathit{x} \,  \, )
\end{verbatim}
typeset as follows.
\[( \, \Lambda  \mathit{X} <: \mathit{T} . \,  \Lambda  \mathit{X'}   <:   \mathit{T'} . \,  \mathit{x} \,  \, )\]
Where there is no user-supplied homomorphism clause the \myLaTeX{}
pretty-printing defaults to a sequence of the individual items
separated by thin spaces (\verb+\,+),
with reasonable default fonts and making use of the \verb+terminals+ grammar where appropriate.


\myparagraph{Subrules}\mlabel{a14}%
Subrule declarations have the form
\begin{verbatim}
  subrules
    nt1 <:: nt2
\end{verbatim}
where \verb+nt1+ and \verb+nt2+ are nonterminal roots. 


Subrules can be chained, i.e.~there can be a pair of
subrule declarations \verb+nt1 <:: nt2+ and \verb+nt2 <:: nt3+,
and they can form a directed acyclic graph, e.g.~with 
\verb+nt0 <:: nt1+, \verb+nt0 <:: nt2+,
\verb+nt1 <:: nt3+, and \verb+nt2 <:: nt3+.  However, there cannot be
cycles, or  nonterminal roots for which there are multiple upper bounds. 
Subrule declarations should not involve nonterminal roots for which
proof-assistant type homs are specified. 

We support the case in which the upper rule is also
non-free, i.e.~it contains productions that mention nonterminals that
occur on the left of a subrule declaration. In the example below
(\verb+test11.ott+) the
\verb+t+ rule contains a production \verb+Foo v+.
\verbatiminput{../tests/test11.ott}
In this case generated Isabelle/Coq/HOL/OCaml will define a single type and both \verb+is_v+
and \verb+is_t+ predicates, and the generated inductive definition
clause for \verb+ax+ uses both predicates.  The Isabelle clause is below.
\begin{verbatim}
  axI: "[|is_t t ; is_v v|] ==>  ( t , v ) : Baz"
\end{verbatim}




\myparagraph{Topological Sorting}\mlabel{a19}%
The grammar rules of a syntax definition may depend on each other arbitrarily.
When generating Isabelle/Coq/HOL/OCaml representation types, however, they are
topologically sorted (to simplify the resulting induction
principles).  For example, this source file (\verb+test12.ott+) 
\verbatiminput{../tests/test12.ott}
gives rise to the following Coq and Isabelle.
\[\vspace{-0.5cm}
\begin{minipage}{0.43\textwidth}
\verbatiminput{test12.v}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\verbatiminput{test12.thy}
\end{minipage}
\]
In contrast, at present the inductively defined relation definitions
are not topologically sorted --- instead, each \verb+defns+ block
gives rise to a group of mutual definitions, so the user has control.


\myparagraph{Formulae and Judgements}\mlabel{a13}%
The system synthesises a grammar of the syntactic forms of judgements
for each \verb+defns+ block, a grammar \verb+judgement+ of
their union, and a grammar \verb+user_syntax+ of the union of all the
user-defined syntax.  

If there is no \texttt{formula} grammar then the system adds a default
definition, as if the source included the following. 
\begin{verbatim}
  formula :: 'formula_' ::=
    | judgement           ::   :: judgement
\end{verbatim}
If the user defines a \texttt{formula} grammar then the production
name prefix must be \texttt{formula\_} and the name for the
\texttt{judgement} production must be \texttt{judgement}, as above.

For the example of Fig.~\mref{a4} these are as below.
%\testTenMetagrammar

\testTenMetagrammartabular{
%\testTenMetat
%\testTenMetav
%\testTenMetaterminals
\testTenMetaformula\testTenMetainterrule
\testTenMetaJop\testTenMetainterrule
\testTenMetajudgement\testTenMetainterrule
\testTenMetauserXXsyntax\testTenMetaafterlastrule
}

The \verb+formula+ grammar (whether user-defined or the default) defines the syntax of
premises of inductive definition rules.  In simple cases this can just
have a single production allowing arbitrary judgements, as above.  In
richer examples one may need parentheses, conjunction and disjunction,
etc. 
Care is needed with quantifiers ranging over subrule types, as these
will not be automatically handled correctly.








\myparagraph{Binding}\mlabel{a10}%
\begin{figure}[t]
\fbox{\begin{minipage}{\textwidth}
{\small
\testThirteendgrammar
}
\end{minipage}}
\caption{Ott Source Grammar: a Simplified Fragment\protect\label{a31}}
(Omitting dot forms,  non-primary nonterminal and metavariable
roots, meta flags, homomorphism definitions, and non-\verb+''+
production-name prefixes.)
\end{figure}%
%\testThirteengrammartabular{
%\testThirteenbindspec
%}
Ott supports a rich class of binding specifications.
Each production can be annotated with a number of `bindspec's.  
As shown in the grammar in Fig.~\mref{a31} (a simplified fragment of the Ott input
language), these are of two forms.
The first,
\[\texttt{bind} \, \mathit{mse} \, \texttt{in} \, \mathit{nonterm} \]
declares that the metavariables and nonterminals denoted by
the $\mathit{mse}$ expression are binding in the $\mathit{nonterm}$. 
The second, 
\[
\mathit{auxfn} \, = \, \mathit{mse} 
\]
lets the user define \emph{auxiliary functions} to collect
selected metavariables and nonterminals of subterms.
For example, consider the Ott source below for expressions with \texttt{let}
binding of pair patterns (based on \texttt{test8p.ott}).
{\small
\begin{verbatim}
metavar value_name , x ::= {{ isa string }} 

expr , e :: E_ ::=
  | value_name                  :: :: ident
  | ( expr1 ,  expr2 )          :: :: pair
  | expr expr'                  :: :: apply
  | function value_name -> expr :: :: function (+ bind value_name in expr +)
  | let p = expr in expr'       :: :: let      (+ bind binders(p) in expr' +)

pattern , p :: 'P_' ::=                                      
  | value_name                  :: :: ident  (+ binders = value_name +)   
  | ( p1 , p2 )                 :: :: pair   (+ binders = binders(p1) union binders(p2) +)   
\end{verbatim}
}
Here the bindspec for functions simply binds the single
\verb+value_name+ metavariable that occurs in the production.
The bindspec for lets, however, binds all the `\texttt{binders}' of
the pattern \texttt{p}.  Here \texttt{binders} is an auxiliary
function, taking a pattern and returning a set of \verb+value_name+
metavariables.  It is defined by two clauses, one for each production
of the \verb+pattern+ grammar. 
They state that the \verb+binders+ of a \verb+value_name+ are just the
singleton set $\{\verb+value_name+\}$ and that the \verb+binders+ of a
pair pattern are the union of the \verb+binders+ of the two subpatterns.

Each auxiliary function can be defined over multiple rules of a
grammar, but must return sets of a single type (a particular
metavariable root or nonterminal root). 


\myparagraph{Substitutions}\mlabel{a15}%
The tool can generate Isabelle/Coq/HOL/OCaml for both single and multiple
substitution functions.  For example, the ML polymorphism Ott source
of \verb+test8.ott+ 
%\S\mref{a24} 
includes the following.
\begin{verbatim}
  substitutions
    single   expr value_name :: subst  
    multiple typexpr typvar  :: tsubst 
\end{verbatim}
This causes the generation of two families of substitution
functions, one replacing a single \verb+value_name+ by a \verb+expr+,
the other replacing multiple \verb+typvar+s by \verb+typexpr+s. 

Each family contains a function for each datatype for which it is
required, so in that example there are functions
\verb+subst_expr+ for the first and \verb+tsubst_typexpr+,
\verb+tsubst_typscheme+ and \verb+tsubst_G+ for the second. 

The function for substitutions
\begin{verbatim}
  substitutions
    single   this that :: name1 
    multiple this that :: name2 
\end{verbatim}
replaces terms of productions consisting just of a \verb+that+ by a
\verb+this+.
Here \verb+this+ must be a nonterminal root, while \verb+that+ can be
either a metavariable root or a nonterminal root  (the latter
possibility allows substitution for compound identifiers, though it is
not clear that this is generally useful enough to be included). 
Substitution functions are requried for each member of each (mutually recursive)
block of grammar rules which either contain such a production or (indirectly)
refer to one that does. 

At present multiple substitutions are represented by Isabelle/Coq/HOL/OCaml
lists, so for the example above we have Isabelle
\begin{verbatim}
  tsubst_typexpr :: "(typvar*typexpr) list => typexpr => typexpr"
  tsubst_typscheme :: "(typvar*typexpr) list => typscheme => typscheme"
  tsubst_G :: "(typvar*typexpr) list => G => G"
\end{verbatim}
The generated functions do not substitute bound things, and recursive
calls under binders are filtered to remove the bound things.



\myparagraph{Free Variables}\mlabel{a16}%
Similarly, the tool can generate Isabelle/Coq/HOL/OCaml to calculate the free
variables of terms. For example, the ML polymorphism Ott source
of \texttt{test8.ott} includes the following.
\begin{verbatim}
  freevars
    typexpr typvar :: ftv
\end{verbatim}
This causes Isabelle functions as below to be generated, calculating
the free \texttt{typvar}s that occur in singleton productions in the
\texttt{typexpr} grammar within terms of each type.
\begin{verbatim}
  ftv_typexpr :: "typexpr => typvar list"
  ftv_typscheme :: "typscheme => typvar list"
  ftv_G :: "G => typvar list"
\end{verbatim}

\myparagraph{List Dot Forms}\mlabel{a21}%
The tool has direct support for indexed list syntax (`dot forms'), for example as
used in the \S\mref{a23} type schemes:

{\small\testEightgrammartabular{\testEighttypscheme\testEightafterlastrule}}

and in the \texttt{test7.ott} record types, record terms, and record
patterns (below we give fragments of the \texttt{test7.ott} grammars,
omitting some productions):

{\small\testSevengrammartabular{
\testSevenrulehead{\testSevennt{T}  ,\ \testSevennt{S}  ,\ \testSevennt{U}}{::=}{\testSevencom{type}}\\ 
\testSevenprodline{|}{\ldots}{}{}{}{}\\
\testSevenprodline{|}{\testSevenkw{\{} \, \mathit{l_{{\mathrm{1}}}} \, : \, \testSevennt{T_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l_{\testSevenmv{n}}} \, : \, \testSevennt{T_{\testSevenmv{n}}} \, \testSevenkw{\}}}{}{}{}{\testSevencom{record}}\testSeveninterrule
%
\testSevenrulehead{\testSevennt{t}}{::=}{\testSevencom{term}}\\ 
\testSevenprodline{|}{\ldots}{}{}{}{}\\
\testSevenprodline{|}{\testSevenkw{\{} \, \mathit{l_{{\mathrm{1}}}} \, \!\! = \!\! \, \testSevennt{t_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l_{\testSevenmv{n}}} \, \!\! = \!\! \, \testSevennt{t_{\testSevenmv{n}}} \, \testSevenkw{\}}}{}{}{}{\testSevencom{record}}\\ 
\testSevenprodline{|}{\testSevenkw{let} \, \testSevennt{p} \, \!\! = \!\! \, \testSevennt{t} \, \testSevenkw{in} \, \testSevennt{t'}}{}{\textsf{bind}\; \textrm{b}(\testSevennt{p})\; \textsf{in}\; \testSevennt{t'}}{}{\testSevencom{pattern binding}}\testSeveninterrule
%
\testSevenrulehead{\testSevennt{p}}{::=}{\testSevencom{pattern}}\\ 
\testSevenprodline{|}{\mathit{x} \, : \, \testSevennt{T}}{}{\textrm{b}=\mathit{x}}{}{\testSevencom{variable pattern}}\\ 
\testSevenprodline{|}{\testSevenkw{\{} \, \mathit{l_{{\mathrm{1}}}} \, \!\! = \!\! \, \testSevennt{p_{{\mathrm{1}}}} \, , \, .. \, , \, \mathit{l_{\testSevenmv{n}}} \, \!\! = \!\! \, \testSevennt{p_{\testSevenmv{n}}} \, \testSevenkw{\}}}{}{\textrm{b}=\textrm{b}(\testSevennt{p_{{\mathrm{1}}}}..\testSevennt{p_{\testSevenmv{n}}})}{}{\testSevencom{record pattern}}\testSevenafterlastrule
} 
}

Dot forms can be used in symbolic terms in semantic rules:
{\small \[\testSevendruleTyXXRcd{}\]}

Individually indexed projections from dot forms can be mentioned, eg
the $l_j$ below:
{\small \[\testSevendruleTyXXProj{}\]}

Symbolic terms can also include concatenations of two dot forms with a
singleton in between:
{\small \[\testSevendrulereduceXXCtxXXrecord{}\]}

Multiple dot forms within the same semantic rule can share bounds (e.g.~$1..m$):
{\small \[\testSevendruleMXXRcd{}\]}


In more detail, productions can have dot tokens interspersed between the elements. 
These consist of two, three or four consecutive dots (\verb+..+,
\verb+...+, or \verb+....+), indicating lists with minimum lengths $0$,
$1$, and $2$ respectively\footnote{These length minimums are respected
  when parsing concrete lists, but are not present in Isabelle/Coq/HOL
  output.}. 
The tool identifies the maximal sequence of elements on either side of
the dots that are identical modulo anti-unification of some
index. Optionally, there may also be a single terminal on either side
of the dot token, separating instances of the repeated unit.
For example, in the \texttt{test7.ott} production
\begin{verbatim}
  | { l1 = t1 ,  .. , ln = tn }  :: :: Rec                     
\end{verbatim}
there is such a terminal (the `\verb+,+').  The tool identifies
that \verb+l1 = t1+ and \verb+ln = tn+ can be anti-unified as 
(roughly) \verb+l_ = t_+,  taking \verb+_+ to be the bounds \verb+1+ and \verb+n+.
A single production may contain multiple dot forms, but they must not overlap;
nested dot forms (including those with multiple changing indices) are not currently
supported. 

Binding specifications are generalised to match:  an \verb+mse+ can
involve a dot form of metavariables (e.g.~the
$\mathit{typvar_{\mathrm{1}}}..\mathit{typvar_{\mathit{n}}}$ above);
a dot form of nonterminals; or an auxiliary function applied to a dot
form
of nonterminals (e.g.~the
$\textrm{b}(\mathit{p_{\mathrm{1}}}..\mathit{p_{\mathit{n}}})$ above).
Dot forms on the right of a \textsf{bind} are not currently supported.








The generated Isabelle/Coq/HOL/OCaml types for dot forms involve lists of
tuples, e.g.~for the production above:
\par\noindent{\small
\begin{verbatim}
  datatype 
  t = 
   ...
   | t_Rec "(label*t) list"
   ...
\end{verbatim}
}
The generated code for a rule involving dot forms involves, for each
bound that occurs, a generated variable for the list of tuples of
nonterminals and metavariables that occurred with that bound.  For
example, the generated Isabelle for the first three rules above is
shown below (lightly hand-edited for format).  The first involves an
Isabelle variable \verb+l_t_T_list+, and list maps and projections
thereof.
\par\noindent{\small
\begin{verbatim}
Ty_RcdI: "
  [|(formula_formuladots ((List.map (%(l_,t_,T_).( ( G , t_ , T_ ) : Ty)) l_t_T_list)))|] 
  ==> 
  ( G , 
    (t_Rec ((List.map (%(l_,t_,T_).(l_,t_)) l_t_T_list))) , 
    (T_Rec ((List.map (%(l_,t_,T_).(l_,T_)) l_t_T_list))) 
  ) : Ty"

Ty_ProjI: "
  [| ( G , t , (T_Rec (l_T_list)) ) : Ty|] ==> 
  ( G , 
    (t_Proj t (%(l_,T_).l_) (List.nth l_T_list (j - 1))) , 
    (%(l_,T_).T_) (List.nth l_T_list (j - 1)) 
  ) : Ty"

E_Ctx_recordI: "
[| List.list_all (%(l_,v_).is_v v_) l_v_list ;
   ( t , t' ) : E|] 
==> 
 ( (t_Rec (l_v_list @ [(l,t)] @ l_'t_'list)) , 
   (t_Rec (l_v_list @ [(l,t')] @ l_'t_'list)) 
 ) : E"
\end{verbatim}
}
The generated code for substitutions and free variables takes account of such list structure.

Note that at present the generated Isabelle code for these functions
does not always build without change, in particular if tuples of size
3 or more are required in patterns.


Proof assistant homomorphisms in productions can refer to dot-form
metavariables and nonterminals.  For example, the second production
below (taken from \verb+test17.9+) mentions \verb+[[x1 t1 ... xn tn]]+ in the \verb+isa+
homomorphism.  This must exactly match the dot form in the production
except that all terminals must be omitted --- the metavariables and
nonterminals must occur in the same order as in the production, and
the bounds must be the same.  
\par\noindent{\small
\begin{verbatim}
E :: 'E_' ::= {{ isa ( ident * t ) list  }}
  | < >                         :: :: 1  {{ isa  [] }}
  | < x1 : t1 , ... , xn : tn > :: :: 2  {{ isa  List.rev [[x1 t1 ... xn tn]] }}

formula :: formula_ ::=          
  | judgement              :: :: judgement
  | formula1 .. formulan   :: :: realdots {{ isa (List.list_all (\<lambda> b . b) ( [[ formula1 .. formulan ]] ) ) }}

\end{verbatim}
}
The generated Isabelle code for symbolic
terms mentioning this production will involve a list of pairs.  For
example, the rules
\par\noindent{\small
\begin{verbatim}
defn
|- E  :: :: Eok :: Eok_ by

---------------------------- :: 2
|- <x1:t1,...,xn:tn> 

|- t1:K1  ... |- tn:Kn
---------------------------- :: 3
|- <x1:t1,...,xn:tn> 
 \end{verbatim}
}
generate
\par\noindent{\small
\begin{verbatim}
consts
  Eok :: "E set"
inductive Eok tK
intros

(* defn Eok *)

Eok_2I: " ( List.rev  (x_t_list) ) : Eok"

Eok_3I: "[|
(List.list_all (\<lambda> b . b) (  ((List.map (%(x_,t_,K_). ( t_ , K_ ) : tK) x_t_K_list))  ) )|]
 ==> 
 ( List.rev  ((List.map (%(x_,t_,K_).(x_,t_)) x_t_K_list)) ) : Eok"
 \end{verbatim}
}
Note that in the second the list of pairs is projected out from the
\verb+x_t_K_list+ list of triples that is quantified over in the rule.


LaTeX homomorphisms should not refer to dot forms, as either an error
or bad output will be generated. (For LaTeX, one should really specify
a homomorphism for the repeated expression, and also data on how any
list separators should be typeset.  This would require more
special-case treatment, and may not often be required.)



\myparagraph{List Comprehension Forms}\mlabel{a44}%
%
Lists can also be expressed as explicit list comprehensions, primarily
to support more concise typesetting of rules. 
Three different styles are supported.  For example, in a symbolic
term, instead of the dot form 
\par\noindent{\small
\begin{verbatim}
  G |- t1:T1  ..  G |- tn:Tn
 \end{verbatim}
}
one can write any of the following
\par\noindent{\small
\begin{verbatim}
   </ G |- ti:Ti // i           />
   </ G |- ti:Ti // i IN n      />
   </ G |- ti:Ti // i IN 1 .. n />
 \end{verbatim}
}
Similar comprehensions can be used in productions, for example lines
2--4 below.  In addition, comprehensions in productions can specify a
terminal to be used as a separator in concrete lists, as in lines 5--7 below.
(These examples are taken from \verb+test17.10.ott+.)
\par\noindent{\small
\begin{verbatim}
  | { l1 = t1 ,  .. , ln = tn }           :: :: Rec             {{ com record  --- dots }}
  | { </ li = ti // i           /> }      :: :: Rec_comp_none   {{ com record --- comp }}
  | { </ li = ti // i IN n      /> }      :: :: Rec_comp_u_none {{ com record --- compu }} 
  | { </ li = ti // i IN 1 .. n /> }      :: :: Rec_comp_lu_none{{ com record --- complu }}  
  | { </ li = ti // , // i           /> } :: :: Rec_comp_some   {{ com record --- comp with terminal }}
  | { </ li = ti // , // i IN n      /> } :: :: Rec_comp_u_some {{ com record --- compu with terminal }} 
  | { </ li = ti // , // i IN 1 .. n /> } :: :: Rec_comp_lu_some {{ com record --- complu with terminal }}  
 \end{verbatim}
}
In Coq, HOL or Isabelle output, list dot forms and the
various list comprehension forms are treated almost identically.
In LaTeX output, comprension forms are typeset with overbars. 
For example, the rules below
\par\noindent{\small
\begin{verbatim}
 G|- t:{l1:T1,..,ln:Tn}
 ----------------------- :: Proj_dotform
 G|- t.lj : Tj

 G|- t: { </ li:Ti // i/> }
 ---------------------------------- :: Proj_comp
 G|- t.lj : Tj

 G|- t: { </ li:Ti // i IN n/> }
 ---------------------------------- :: Proj_comp_u
 G|- t.lj : Tj

 G|- t: { </ li:Ti // i IN 1..n/> }
 ---------------------------------- :: Proj_comp_lu
 G|- t.lj : Tj
 \end{verbatim}
}
are typeset as follows.
%\testSeventeenTendruleTyXXRcdXXdotform
\[\testSeventeenTendruleTyXXProjXXdotform{}\]
%\testSeventeenTendruleTyXXRcdXXcomp
\[\testSeventeenTendruleTyXXProjXXcomp{}\]
%\testSeventeenTendruleTyXXRcdXXcompXXu
\[\testSeventeenTendruleTyXXProjXXcompXXu{}\]
%\testSeventeenTendruleTyXXRcdXXcompXXlu
\[\testSeventeenTendruleTyXXProjXXcompXXlu{}\]
%
Upper bounds of the form $n-1$ are also permitted, e.g. with
\par\noindent{\small
\begin{verbatim}
 G|- t:{l0:T0,..,ln-1:Tn-1}
 ----------------------- :: Proj_dotform_minus
 G|- t.lj : Tj

 G|- t: { </ li:Ti // i IN 0..n-1/> }
 ---------------------------------- :: Proj_comp_lu_minus
 G|- t.lj : Tj
 \end{verbatim}
}
typeset as below.
\[\testSeventeenTendruleTyXXProjXXdotformXXminus{}\]
\[\testSeventeenTendruleTyXXProjXXcompXXluXXminus{}\]

List comprehension forms can also be used in bindspecs and in
homomorphisms. 

A list form used in a symbolic term does not have to be in the same
style as that in the corresponding production. 
However, if a metavariable or nonterminal occurs in multiple different
list forms in the same inference rule, they must all be in the same style and
with the same bounds.  Moreover, in a production, a list form in a bindspec or
homomorphism must be in the same style and with the same bounds as the
corresponding list form in the elements of the production. 

The comprehension form with a single index, e.g.
\verb+</ G |- ti:Ti // i />+, 
typeset as $\overline{\Gamma \, \vdash \, \mathit{t_{\mathit{i}}} \, :
  \, \mathit{T_{\mathit{i}}}}^{\,\mathit{i}}$, is not common notation.
We introduce it here as it is rather concise but remains unambiguous. 
It does not specify the length of the list, but that is rather rarely required.
%
In some cases one could make the typeset notation even less noisy, by
either omitting the superscript $i$ or omitting both the superscript $i$ and
the subscript $i$'s on $t$ and $T$.  The first is unambiguous if there
is at most one index on each element in the comprehension; the second
if all the elements are indexed by the same thing  (not the case for
this example, but common for comprehensions of single elements,
e.g. \verb+<< Ti // i>>+ for $\overline{T}$). It is conceivable that that
should be automated, though it would bring the typeset and ASCII
versions out of step.

%The tokens used for list dot forms and comprehension forms (\verb+</+,
%\verb+//+, \verb+IN+, \verb+/>+, \verb+..+, \verb+...+, and
%\verb+....+) cannot at present be used in the object language. 




\myparagraph{Code Embedding} It is possible to embed arbitrary code in
the Ott source using the \verb+embed+ section, which can contain
\verb+coq+/\verb+isa+/\verb+hol+/\verb+tex+/\verb+ocaml+  homomorphisms, the
bodies of which will appear in the respective output. 
Depending on the position of the \verb+embed+ section, this will
either be at the beginning of the output file, between the generated
functions (substitution etc.) and the definitions of inductive
relations, or at the end of the file (for any after the first
definition, the \verb+embed+ keyword must be on a line by itself). For example, \verb+test8.ott+ contains the following to
define Coq and HOL \verb+remove_duplicates+ functions.
\par\noindent{\small
\begin{verbatim}
embed
  {{ coq
Fixpoint remove_duplicates (l:list typvar_t) : list typvar_t :=
  match l with
  | nil => nil
  | cons h t => if (list_mem eq_typvar_t h t)  then remove_duplicates t
                else cons h (remove_duplicates t)
end. }}

  {{ hol
val _ = Define `
  (remove_duplicates [] = []) /\
  (remove_duplicates (x::xs) = if (MEM x xs) then remove_duplicates xs 
                               else x::(remove_duplicates xs))
`; }}
\end{verbatim}
}
Within the body of an \verb+embed+ homomorphism, any text between 
\verb+[[+ and \verb+]]+ will be parsed as a symbolic term (of the
\verb+user_syntax+ grammar) and pretty printed, so one can use user
syntax within tex or proof assistant code. An Isabelle example is below.
\par\noindent{\small
\begin{verbatim}
{{ isa

consts
order :: "type => nat"
primrec
"order [[unit]] = 0"
"order [[t*t']] = max (order [[t]]) (order [[t']])"
"order [[t->t']] = max (1+order [[t]]) (order [[t']])"

}}
\end{verbatim}
}
Similar processing can be carried out on separate files, using the
command-line options \verb+tex_filter+, \verb+isa_filter+, etc., as
shown in \S\mref{a20}.


\myparagraph{Hom Sections}
Bindspecs and homomorphisms for productions, and any homomorphisms
for definitions, can be included in an Ott source file either 
attached to the production or definition, as we have shown earlier, or
in separate \texttt{hom} sections. 

The \verb+test10_homs.ott+ example, in Fig.~\mref{a38}, shows this. It
produces the same output as the \verb+test10.ott+ example of
Fig.~\mref{a4}, but the homs have been moved into \texttt{hom}
sections, for example as below.
\par\noindent{\small
\begin{verbatim}
homs 't_'
  :: Var                        {{ com variable }}         
  :: Lam  (+ bind x in t +)     {{ com abstraction }}      
  :: App                        {{ com application }}      

homs ''
  :: reduce  {{ com [[t1]] reduces to [[t2]] }} 
\end{verbatim}
}
Each of these begins with a prefix and then has a sequence of
production name or definition name kernels, each followed by a
sequence of bindspecs and then a sequence of homomorphisms.
\begin{figure}
\fbox{\begin{minipage}{\textwidth}
\verbatiminput{../tests/test10_homs.ott}
\end{minipage}}
\caption{Hom Sections: \texttt{test10\_homs.ott}\protect\label{a38}}
\end{figure}




\myparagraph{\myLaTeX{} Output and Filtering}\mlabel{a20}%
The generated \myLaTeX{} output is factored into individual \myLaTeX{}
commands: for the metavariable declarations, each rule of the syntax
definition, the collected syntax, each rule of the inductive relation
definitions, the collected rules for each relation, the collected
rules for each \verb+defns+ block, and the whole. 
%
This makes it possible to quote individual parts of the definition,
possibly out-of-order, in a paper or technical report. 

Additionally, there is (slightly rudimentary) support for using Ott as a munger. 
There are command-line arguments \verb+-tex_wrap+ and \verb+-tex_filter+.  
Typical usage would be something like this (from the \verb+Makefile+
in the \verb+doc+ directory):
\begin{verbatim}
test7.tex: ../src/ott ../tests/test7.ott ../tests/test7tt.mng
         cd ../src; make tmp_test7_clean.ott
         ../src/ott                                        \
                -tex test7.tex                             \
                -tex_show_meta false                       \
                -tex_wrap false                            \
                -tex_name_prefix testSeven                 \
                -tex_filter ../tests/test7tt.mng test7tt.tex \
                ../src/tmp_test7_clean.ott
\end{verbatim}
The \verb+-tex_wrap false+ turns off output of the default \myLaTeX{}
document preamble, so the generated file \verb+test7.tex+ just contains
\myLaTeX{} definitions.  
The \verb+-tex_name_prefix testSeven+ sets a prefix for the generated
\myLaTeX{} commands
(so the \myLaTeX{} definitions from multiple Ott source files can be
included in a single \myLaTeX{} document).
The \verb+-tex_filter+ argument takes two
filenames, a source and a destination.  It filters the source file,
(roughly) replacing any string found within \verb+[[+ \verb+]]+ by
the tex pretty-print of its parse.  This parsing is done w.r.t.~a generated nonterminal 
\verb+user_syntax+ which is a union of all the user's grammar.  

At present %(unlike our previous munger) 
munged strings are not automatically
put within \verb+$+ \verb+$+, and there is no analogue of \verb+<[+
\verb+]>+.  We probably want to tune up the tex to make it more
convenient, e.g.~to put single rules in math displays (or not);
definitions in centering environments (or not), etc.

The lexing 
%pays some attention to \myLaTeX{} comments, but may not get it
%right.  It 
turns any sequence of \verb+[+ (resp. of \verb+]+) of
length $n+1$ for $n>2$ into a literal sequence of length $n$.

The following two subsections show a source file (\verb+test7tt.mng+) that uses terms of the F$_{<:}$
definition of \texttt{test7.ott}, and the result of filtering it.

Similar filtering can be performed on Coq, Isabelle, HOL, and OCaml files.

\subsubsection{F$_{<:}$ Extracts --- Source File}\mlabel{a29}%
\verbatiminput{../tests/test7tt.mng}

\subsubsection{F$_{<:}$ Extracts --- The Filtered Output}\mlabel{a30}%
\input{test7tt.tex} 


\myparagraph{Isabelle Syntax Support}\mlabel{a39}%
The tool has (limited) facilities to allow the Isabelle mixfix syntax support
and xsymbol to be used.  The example \verb+test10_isasyn.ott+ shows
this in use.  
%

Non-meta productions can be annotated with
\verb+isasyn+ and/or \verb+isaprec+ homomorphisms. 
For example, \verb+test10_isasyn.ott+ contains the production
\par\noindent{\small
\begin{verbatim}
    | t t'      :: :: App   {{ isasyn [[t]]\<bullet>[[t']] }}  {{ isaprec 50 }}
\end{verbatim}
}
The two homs are used to output the Isabelle syntax annotation in the
\verb+t_App+ clause of the datatype definition below.
\par\noindent{\small
\begin{verbatim}
t = 
   t_Var "termvar"  
 | t_Lam "termvar" "t"  ("\<lambda> _ . _" 60)
 | t_App "t" "t"  ("_\<bullet>_" 50)
\end{verbatim}
}
Definitions can be annotated with
\verb+isasyn+ and/or \verb+isaprec+ homomorphisms similarly, e.g.~as below.
\par\noindent{\small
\begin{verbatim}
    defn
    t1 --> t2 ::  :: reduce :: '' {{ isasyn [[t1]] ---> [[t2]] }}  by 
\end{verbatim}
} 
This generates \verb+syntax+ and \verb+translations+ blocks as below.
\par\noindent{\small
\begin{verbatim}
consts
  reduce :: "(t*t) set"

syntax
  "_reduce" :: "t => t =>  bool" ("_ ---> _" 50)

translations
  "(t17 ---> t27)" \<rightleftharpoons> " ( t17 , t27 ) : reduce"
\end{verbatim}
}
Symbolic terms in definitions are printed using any production or
definition syntax.  This (especially with xsymbol turned on) makes the
current goal state during Isabelle proof development much more
readable. 



Further, there is a command line option \verb+-isa_syntax true+.  If
this is set then the tool generates Isabelle syntax annotations from
the source syntax.  For example, the source file production for the
\verb+t_Lam+ clause is
\par\noindent{\small
\begin{verbatim}
    | \ x . t   ::   :: Lam   {{ isaprec 60 }}
\end{verbatim}
}
and the \verb+terminals+ grammar contains a mapping from \verb+\+ to \verb+\<lambda>+:
\par\noindent{\small
\begin{verbatim}
  terminals :: 'terminals_' ::=
    | \                   ::   :: lambda  {{ tex \lambda }}  {{ isa \<lambda> }}
\end{verbatim}
}
This is used (just as for \myLaTeX{} homs) to generate the \verb+("\<lambda> _ . _" 60)+
in the datatype definition above.


Limitations:
(1) the full range of Isabelle precedence and associativity
specifications are not supported;
(2) the automatically generated syntax annotations are somewhat crude,
especially w.r.t.~spacing and parenthesisation;
(3) syntax annotation on meta productions is not propertly supported;
and
(4) it would be desirable to have more fine-grain control of whether
to automatically generate annotations: per-production, per-rule, and
per-file.


%For example, the \verb+out.thy+ produced by
%\verb+make test10_isasyn+ (which includes this option), the annotation
%on 


\myparagraph{Code Generation}\mlabel{a22}%
The Isabelle/Coq/HOL code generation facilities can be used to
generate (variously) OCaml and SML code from the definitions, if
they are suitably constrained. 

For example, the \verb+test10st_codegen.thy+ file uses Isabelle
code generation to produce SML code to calculate the possible
reductions of terms in the \verb+test10st.ott+ simply typed lambda
calculus.

\verbatiminput{../tests/test10st_codegen.thy}



\section{Example: Simply Typed Lambda Calculus}\mlabel{a34}%
This example extends the untyped lambda calculus of \S\mref{a2} with a
simple type system.  We include Isabelle proof script for the
standard metatheoretic results, type preservation and progress.
%
In order to define a reasonable system using the concrete representation of
variables and binding, 
the type system here is slightly unusual.  It allows repeated 
types for identifiers in type environments, with the rightmost being
the significant one.  Type preservation is proved only for closed
terms, though it should be straightforward to weaken that to allow
terms that do not shadow identifiers in a standard library --- a
sufficient condition to prevent capture, as all substitutions
performed during reduction are of values that  are closed except for standard
library identifiers.

The definition presented here represents type environments using proof-assistant list types.  For Isabelle we have also tried a variant using
a functional representation, that gave rise to somewhat simpler proof scripts.

\subsection{Ott source}\mlabel{a35}%
{\small
\verbatiminput{../tests/test10st.ott}
}
\subsection{Generated \myLaTeX}\mlabel{a36}%
\testTenstall

\subsection{Isabelle Proof Script for Metatheory (with Tom Ridge)}\mlabel{a37}%
{\small
\verbatiminput{../tests/test10st_metatheory_autoed.thy}
}



\section{Example: ML Polymorphism (in an OCaml syntax fragment)}\mlabel{a23}%
\subsection{Ott Source}\mlabel{a24}%
{\small
\verbatiminput{../tests/test8.ott}
}
\subsection{Generated \myLaTeX}\mlabel{a25}%
\testEightall

\section{Example: POPLmark F$_{<:}$ with Records}\mlabel{a26}%
Note that, as mentioned in \S\mref{a2}, the concrete variable
representation is not satisfactory for this example. 
\subsection{Ott Source}\mlabel{a27}%
{\small
\verbatiminput{../src/tmp_test7_clean.ott}
}
\subsection{Generated \myLaTeX }\mlabel{a28}%
\testSevenall

\end{document}



\end{document}


%%% Local Variables:
%%% LaTeX-label-sequence-N: 69
%%% LaTeX-label-prefix: "a"
%%% End:
