**************************************************************************
** title:      Bootstrap
** author:     Brian Frank
** created:    05 Jan 08
** copyright:  Copyright (c) 2008, Brian Frank and Andy Frank
** license:    Licensed under the Academic Free License version 3.0
**************************************************************************

Overview [#overview]
********************
The Fantom compiler is written in Fantom itself - which presents a chicken
and the egg problem.  How do you compile the compiler without having
a compiler?  The problem is further compounded by the fact that the build
scripts themselves are written in Fantom.  To solve this problem, the
bootstrap process requires two Fantom installations:

  1.  *rel*: known good Fantom installation (typically the last build)
  2.  *dev*: development environment to build (probably pulled from hg)

By convention we structure our development directory tree like this:

  dev/
    rel/
      bin/
      lib/
      ...
    fan/
      bin/
      lib/
      src/
      ...

The "rel" directory always contains the last released build.  The "fan"
directory contains our main development code branch.  We call our top
level directory "dev", but for the purposes of this discussion "fan"
is the development directory.

Bootstrap Script [#script]
**************************
The easiest way to perform a bootstrap build from hg tip is to use the
"bootstrap.fan" script:
  1. Download latest released build from `http://fantom.org/`
  2. Unzip that build to '{relHome}'
  3. Ensure 'hg' is installed in your path (see [Mercurial]`http://mercurial.selenic.com/`)
  4. Verify 'java_home' env var points to your JDK installation
  5. Execute '{relHome}/bin/fan {relHome}/adm/bootstrap.fan'
  6. Go grab a cup of coffee

NOTE: you need JDK 1.6 to recompile Fantom.  However only 1.5 is required
to run Fantom.

By default devHome is assumed to be a peer to relHome called "fan". If
you'd like to use another directory then specify the "-devHome" option.

This script will perform a number of steps:
  1. Verifies your environment
  2. Clones hg tip or if hg repo exists performs a pull/update
  3. Configures your etc files
  4. Rebuilds your devHome environment from scratch

NOTE: the bootstrap script does *not* update your [substitutes]`#substitutes`.
You must still do that by hand if you plan on rebuilding individual bootstrap
pods yourself.

The rest of this chapter covers all the internal details required to
fully understand bootstrap building.

Dev Home [#devHome]
*******************
By default the build assumes *devHome* to be the home directory
of Fantom installation.  For example if you rebuild jfan, then the output
goes into "devHome/lib/java/sys.jar".  In the case of the *rel*
installation we don't want this default because we will overwrite
ourselves (which leads to some nasty problems).  So you need to set
the *devHome* prop in "etc/build/config.props" of your *rel* installation
to reference the *dev* directory using a URI (not OS path):

  // Must be configured boot build in substitute/release installation
  devHome=file:/C:/dev/fan/

If you forget to do this, then you will get a build error which
looks something like this:

  C:\dev\fan\src>sys\java\build
  ERR: Must update 'devHome' for bootstrap build
  BUILD FAILED [12ms]!

Substitutes [#substitutes]
**************************
On a clean machine with only source code, we don't have any pods compiled
such as 'sys', 'build', or 'compiler'.  In order to run the build scripts
to compile these pods, we need to use our *rel* installation.

Windows Substitutes [#subsWindows]
==================================
To make this all work seamlessly on Windows, the Fantom launcher will
look in "etc/sys/config.props" to see if a substitute runtime should be used.
For example in our environment we map the following scripts to
use the *rel* installation.

  // etc/sys/config.props of your *dev* directory
  runtime.substitutes=                                 \
    /C:/dev/fan/src/buildall.fan       = C:\\dev\\rel  \
    /C:/dev/fan/src/buildboot.fan      = C:\\dev\\rel  \
    /C:/dev/fan/src/jfan/build.fan     = C:\\dev\\rel  \
    /C:/dev/fan/src/sys/build.fan      = C:\\dev\\rel  \
    /C:/dev/fan/src/compiler/build.fan = C:\\dev\\rel  \
    /C:/dev/fan/src/build/build.fan    = C:\\dev\\rel

The launcher will check if any script being run matches one of those
files.  If a match is made, then it will route to the alternate
runtime specified.  Turn on [launcher debugging]`Setup#debugging`
to see exactly what is happening under the covers.

Note that the extension used to run the build scripts needs to
match the substitutes defined in the "config.props".  If you set
your 'pathext' to include ".fan", then Windows will pass the extension
into the launcher.  If you are running an alternate way you might
need to explicitly specify the extension on your command line or
change the extensions defined in "config.props".

Unix Substitutes [#subsUnix]
============================
Unix substitution is implemented by the bootstrap build scripts
using the following shebang:

  #! /usr/bin/env fansubstitute

The fansubstitute script explictly sets FAN_HOME to the value
of the FAN_SUBSTITUTE variable before launching.  So you will
need to export FAN_SUBSTITUTE to reference your *rel* installation.
And of course you have to run your scripts as executables so
that the shebang takes effect.

Also note that building on Unix will skip any .NET targets.

JDK and .NET Tools [#otherTools]
********************************
You need JDK 1.6 or greater to compile from source (only 1.5 is required
for the Java runtime).

Version 2.0 or greater of .NET is required.

In order to compile from source you will need to setup some config
properties to reference your JDK and .NET home directories:

  jdkHome=/C:/dev/tools/java/
  dotnetHome=/C:/WINDOWS/Microsoft.NET/Framework/v2.0.50727/

These paths should be formatted as file system Uris (not OS paths).
These properties should be configured in "etc/build/config.props"
of **both** your *rel* and *dev* environments.

The .NET targets are automatically skipped if not running Windows.
So you can ignore the .NET configuration on Unix platforms.

Buildall [#buildall]
********************
The "buildall.fan" script is the top level build script for compiling
the Fantom distribution.  We commonly run this command to rebuild everything
and run tests on every pod:

  buildall full test

The "buildall" script is executed by the *rel* substitute runtime and
in turn launches two sub-scripts.  The "buildboot.fan" script manages
rebuilding the core runtime modules:

  sys/build.fan
  sys/java/build.fan
  sys/dotnet/build.fan
  sys/js/build.fan
  compiler/build.fan
  compilerJava/build.fan
  build/build.fan

Once the bootstrap modules are compiled, the development environment is
self hosting and can be used to compile the remainder of itself.  This
is done via the "buildpods.fan" script.

Dependencies [#dependencies]
****************************
The bootstrap issue can cause some confusing dependency issues which
are summarized here:

  - The *rel* compiler will be generating the 'sys', 'compiler', and 'build'
    pod files.  This means that the *rel* compiler must be able to
    generate fcode that the *dev* runtime can read.  It also means that
    the *rel* compiler must be able to read any new syntax used by *dev*
    versions of 'sys', 'compiler', and 'build'.

  - The *rel* compiler will actually use the *dev* versions of the pods
    to resolve dependencies.  For example *dev* compiler can reference
    new sys APIs defined in *dev* but not *rel*.  Under the covers this
    works because the 'compiler' and 'build' pod's build scripts specify
    a non-default [dependsDir]`build::BuildPod.dependsDir`.

Because of these restrictions, adding new language features and fcode
changes require some careful planning.

Debugging [#debugging]
**********************
You can use the "dumpEnv" build target to dump key aspects of your
build script environment.  You can run target on "buildall.fan", although
a more concise report is to dumpEnv on "buildboot.fan" and one of the
non bootstrap pods like "testSys/build.fan".

Key things to note about your environment:
  1. all scripts should be using *dev* for devHomeDir
  2. bootbuild scripts should be using *rel* for fanHome
  3. non-bootbuild scripts like testSys should be using *dev* for fanHome
  4. verify javaHome for sys/javafan/build.fan
  5. verify dotnetHome for sys/dotnet/build.fan (if running on Windows)

Summary [#summary]
******************
In summary, you want to make sure of a couple key things:

  1. setup your *rel* installation and never touch it (consider it readonly)
  2. ensure jdkHome and dotnetHome are configured in both *rel*
     and *dev* etc/build/config.props
  3. ensure *rel* etc/build/config.props devHome points to your *dev* installation
  4. make sure your substitutes are configured correctly:
     a. on Unix make sure your FAN_SUBSTITUTE env points to the *rel* installation
     b. on Windows make sure your *dev* etc/sys/config.props substitutes are configured
  5. only put *dev* bin your path and always run your scripts from
     the *dev* installation
  6. never use a working repo for bootstrap (use only the boot repo)