\input texinfo	@c -*-texinfo-*-
@setfilename ../info/mule
@settitle Internal structure of Mule

This file documents the internal structure of Mule (MULtilingual
Enhancement to GNU Emacs).

@titlepage
@sp 6
@center @titlefont{Internal structure of Mule}
@sp 4
@center Version 2.0
@sp 5
@center Ken'ichi HANDA
@center handa@@etl.go.jp
@page

@end titlepage

@node Top, Overview,, (dir)
@chapter Mule specific information

@noindent
This file documents Mule specific information.

We are very sorry for the poor structure of this file.
Currently, this is just a collection of several documents
which are previously located under `doc' directory.

@menu
* Overview::            Overview of the internal structure of Mule
* Character::           Internal representation of multilingual text
* Coding-system::       Encoding of text while reading/writing
* Syntax::              Extended syntax and character category
* Font::                Font handling while displaying text on X
* CCL: (CCL).           Code Conversion Language
* Terminology: (terminology).
* Languages: (languages).  Language specific tips.
* X's font: (XFONT).    X's FONT usage for novice users
* R2L: (R2L).           Right-to-left writing
* EGG: (egg).           Japanese/Chinese inputting methods using Wnn/cWnn
* Quail: (quail).       Imputting methods of multilingual text
* Keyboard Translation: (kbd-trans).
* ISO2022: (ISO2022).   ISO2022 encoding mechanism
* m2ps: (m2ps).         Convert multilingual text to PostScript
@end menu

@node Overview, Character, Top, Top
@section Overview

@noindent
To handle multilingual text, Mule extended GNU Emacs in many aspects.
Mule uses special internal representation of multilingual text,
converts text from/to outernal representation on reading/writing,
special font selection mechanism to display multilingual text on X
window.

@node Character, Coding-system, Overview, Top
@section Character

@menu
* Character type::
* Buffer and string::
* Character object::
* GLYPH::
* Functions::
* Character set::
@end menu

@node Character type, Buffer and string, , Character
@section Character type

There are 6 types of character.  @samp{Type N-M} means that original
N-byte code is represented by M-byte within Mule.

@table @strong
@item Type 1-1
ASCII characters
@item Type 1-2
Characters in one-byte official character-set (e.g. ISO8859-1, Latin-1)
@item Type 1-3
Characters in one-byte private character-set
@item Type 2-3
Characters in two-byte official character-set (e.g. JISX0208, Japanese)
@item Type 2-4
Characters in two-byte private character-sets
@item Type N
Composite characters of variable length
@end table

@xref{Character set} for predefined character sets.

@node Buffer and string, Character object, Character type, Character
@section Buffer and string

@table @strong
@item Type 1-1
1-byte 'C' [C <= 0x7F] (same as the original representation) 
@item Type 1-2
2-byte sequence 'LC1 C1', where
	LC1 = leading character for the character-set,
		0x81..0x8F -- 15 sets
        C1 = 0x80 | (original byte for the character)]
		0xA0 <= C1 <= 0xFF
@item Type 1-3
3-byte sequence 'LCPRV1 LC12 C1', where
	LCPRV1 = 0x9A (for one column) or 0x9B (for two column)
	LC12 = extended leading character,
		0xA0 <= LC12 <=0xDF (if LCPRV1 = 0x9A) -- 64 sets
		0xE0 <= LC12 <=0xEF (if LCPRV1 = 0x9B) -- 16 sets
	C1 = same as above
@item Type 2-3
3-byte sequence 'LC2 C21 C22', where
	LC2 = leading character for the character-set,
		0x90 <= LC2 <= 0x99 -- 10 sets
	C21 = 0x80 | (original first byte for the character),
	C22 = 0x80 | (original second byte for the character),
		0xA0 <= C21,C22 <= 0xFF
@item Type 2-4
4-byte sequence 'LCPRV2 LC22 C21 C22'
	LCPRV2 = 0x9D (for one column) or 0x9E (for two column)
	LC22 = extended leading character,
		0xF0 <= LC22 <=0xF4 (if LCPRV2 = 0x9C) -- 5 sets
		0xF5 <= LC22 <=0xFE (if LCPRV2 = 0x9D) -- 10 sets
	C21, C22 = same as above
@item Type N
n-byte sequence 'LCCMP LCN1 C11 ... LCN2 C21 ... LCNn Cn1 ...'
	all characters 'LCN1 C11 ... LCN2 C21 ... LCNN CN1 ...'
	are displayed on the same column.
	LCCMP = 0x80
	LCN1 .. LCNN = leading character + 0x20, but, for ASCII, 0xA0
@end table

Here's an example of a text with mixture of these types
(at the place of 0x?? comes real binary code) .

"Here comes Latin-1 character of n with ~ '0x81 0xF1' and
here comes Japanese Hiragana '0x92 0xA4 0xA2'."

@node Character object, GLYPH, Buffer and string, Character
@section Character object

@noindent
Emacslisp treats a character object as an integer of value
less than 256 (8-bit).  Mule extends a character object to
19-bit.

The bit fields are divided into 3 parts:
@quotation
f1(5bits):f2(7bits):f3(7bits)
@end quotation

@table @strong
@item Type 1-1: C [C <= 7F] (same as character code itself)
	0:00:00-7F
@item Type 1-2: ((LC1 & 0x7F + 0x10) << 7) | (C1 & 0x7F)
	0:11-1F:20-7F (f1=0,f2=LC1&0x7F + 0x10,f3=C1&0x7F)
@item Type 1-3: ((LC21 & 0x7F + 0x10) << 7) | (C1 & 0x7F)
	0:30-FF:20-7F (f1=0,f2=LC21&0x7F + 0x10,f3=C1&0x7F)
@item Type 2-3: ((LC2 - 0x8F) << 14) | ((C21 & 0x7F) << 7) | (C22 & 0x7F)
	01-0A:20-7F:20-7F (f1=LC2&0x7F,f2=C21&0x7F,f3=C22&0x7F)
@item Type 2-4: ((LC22 - 0xE0) << 14) | ((C21 & 0x7F) << 7) | (C22 & 0x7F)
	10-1E:20-7F:20-7F (f1=LC2&0x7F-0x20,f2=C21&0x7F,f3=C22&0x7F)
@item Type N:
	1F:00-7F:00-7F
@end table

For instance, if '?' is followed by Type 1-2 character '0x81
0xF1', 241 [= 0xF1 = ((0x81 & 0x7F) << 7) | (0xF1 & 0x7F)]
is returned.

In the above table, several blocks are not defined.  Those
are used internally to represent incomplete characters.

  0:01-12:00	leading-char only or invalid char
  0:20-5F:00	LCPRV11/LCPRV12 + LC21 of Type 1-3
  1-8:20-7F:00	LC2 + C21 of Type 2-3
  9-E:00:00	LCPRV21/LCPRV22 + LC22 of Type 2-4
  9-E:20-7F:00	LCPRV21/LCPRV22 + LC22 + C21 of Type 2-4

@node GLYPH, Functions, Character object, Character
@section GLYPH

The original definition of GLYPH is (FACE-ID << 8 | CHAR).
Since Mule, however, requires 19 bits for CHAR, the
definition is changed to (FACE-ID << 19 | CHAR).  So, we can
use only 2024 (= 2^11) different faces.

@node Functions, Character set, GLYPH, Character
@section Functions

To handle multilingual characters, we extended or added the
following functions:

In editfns.c ...

char-to-string: 
Convert arg CHAR to a string containing that character.
If CHAR < 0, it is considered as a multilingual character, and
returned a correct string.

Example:
	(char-to-string ?A) => "A"
	(char-to-string ?あ) => "あ"
	(char-to-string 53794) => "あ"

string-to-char:
Convert arg STRING to a character, the first character of that string.

Example:
	(string-to-char "ABあい") => 65 (== ?A)
	(string-to-char "あい") => 53794 (== ?あ)

sref:
DEFUN ("sref", Fsref, Ssref, 2, 2, 0,
Return the character in STRING at index INDEX.
INDEX starts at 0.
If INDEX does not points to character boundary, -1 is returned.

Example:
	(sref "ABあい" 1) => 66 (== ?b)
	(sref "ABあい" 2) => 53794 (== ?あ)
	(sref "ABあい" 3) => -1 (non character boundary)
	(sref "ABあい" 5) => 53796 (== ?い)

sset:
Store into STRING at index INDEX the character CHAR.
INDEX should point to a character of same bytes as CHAR.
If not, returns nil, else returns CHAR.

Example:
	(setq s "ABあい")
	(sset s 1 ?C) => ?C (s == "ACあい")
	(sset s 2 ?う) => ?う (s == "ACうい")
	(sset s 2 ?A) => ?A (s == "ACA\244\246い")
	(sset s 8 ?A) => nil (out of range)

following-char:
Return the character following point, as a number.
If mc-flag of the current buffer is not nil, the returned character
 may be a multi-byte character.

Example:  If cursor is at 'あ' of buffer "..Aあ..",
	(following-char) => 53794 (== ?あ)
	(let ((mc-flag nil))
	 (following-char t)) => 146 (== leading char of ?あ)

preceding-char:
Return the character preceding point, as a number.
If mc-flag of the current buffer is not nil, the returned character
 may be a multi-byte character.

Example:  If cursor is at 'A' of buffer "..あA..",
	(preceding-char) => 53794 (== ?あ)
	(let ((mc-flag nil))
	  (preceding-char t)) => 162 (== last byte of ?あ)

char-after
First arg, POS, a number.  Return the character in the current buffer
at position POS.
If POS is out of range, the value is NIL.
If mc-flag of the current buffer is not nil, the returned character
 may be a multi-byte character.

Function 'insert' and 'insert-char' also work correctly with
multilingual characters.

	(insert ?あ) -- inserts "あ" at point.

buffer-substring:
Return the contents of part of the current buffer as a string.
The two arguments specify the start and end, as character numbers.
If mc-flag of the current buffer is non-nil, region may be widen
 to meet character boundary.

Example: If a buffer starts with the contents like "あいう..."
	(buffer-substring 1 2) => "あ"
	(buffer-substring 1 3) => "あ"
	(buffer-substring 2 4) => "あい"

Other functions which deal with 'region' also widen range automatically.

subst-char-in-region:
From START to END, replace FROMCHAR with TOCHAR each time it occurs.
If optional arg NOUNDO is non-nil, don't record this change for undo
and don't mark the buffer as really changed.
It also works well with multilingual characters only if the substitution
doesn't alter the length of buffer.

Example:
	(subst-char-in-region 1 10 ?a ?b) => possible
	(subst-char-in-region 1 10 ?あ ?い) => possible
	(subst-char-in-region 1 10 ?a ?あ) => impossible

In functions 'message' and 'format', %c works well with
multilingual characters.

	(message "%c" ?あ) -- shows "あ" in echo area.

In mule.c ...

make-character:
Make multi-byte character from LEADING-CHAR
 and optional args ARG1 and ARG2.

Example:
	(make-character lc-jp ?\244 ?\242) => 53794 (== ?あ)

char-component:
Return a components of multi-byte character CHAR.
Second arg IDX indicate which component should be returned as follows.
 0: leading character or extended leading character,
 1: first byte of the character code,
 2: second byte of the character code.
If the character does not have the componets, 0 is returned.

Example:
	(char-component ?あ 0) => 146 (== lc-jp)
	(char-component ?あ 1) => 164
	(char-component ?あ 2) => 162
	(char-component ?A 1)  => 0

char-leading-char:
Return leading character of CHAR.
If CHAR is not a multi-byte code, 0 is returned.

Example:
	(char-leading-char ?あ) => 146 (== lc-jp)
	(char-leading-char ?A) => 0

char-bytes:
Return number of bytes CHAR will occupy in a buffer.
You can specify a character set to be concerned
 by providing a leading character as CHAR.

Example:
	(char-bytes ?あ) => 3
	(char-bytes ?A) => 1
	(char-bytes lc-jp) => 3

char-width:
Return number of columns CHAR will occupy when displayed.
You can specify a character set to be concerned
 by providing a leading character as CHAR.

Example:
	(char-width ?あ) => 2
	(char-width ?A) => 1
	(char-width lc-jp) => 2

chars-in-string:
Return number of characters in STRING.
Each multilingual character is also counted as one.

Example:
	(chars-in-string "ABあい") => 4

char-boundary-p:
Return non nil value if POS is at character boundary.
The value is:
 0: if POS is at an ASCII character or end of range,
 1: if POS is at a leading char of 2-byte character.
 2: if POS is at a leading char of 3-byte character.
If POS is out of range or not at character boundary, nil is returned.

@node Character set, , Functions, Character

A character set is a set of ordered characters such as
ASCII, right half of ISO8859-1, JIS X0208, and etc.  Mule
identifies a character set by a leading-char assigned to
each set uniquely.

Each character-set is characterized by the following attributes:
@enumerate
@item
Bytes length of code: 1-byte or 2-byte
	ISO8859-1, Right half of JISX0201 (Japanese Katakana) -- 1-byte
	GB2312-1980 (Chinese), JISX0208 (Japanese) -- 2-byte
@item
Columns occupied on a screen: 1-column or 2-column,
	ISO8859-1, Right half of JISX0201 (Japanese Katakana) -- 1-column
	GB2312-1980 (Chinese), JISX0208 (Japanese) -- 2-column
@item
Type: 94-char-set, 96-char-set, 94x94-char-set, or 96x96 char-set,
@item
Graphic set: GL or GR,
@item
Final character: one of '0' thru '~',
@item
Displaying direction: Left-to-right or Right-to-left
@item
Leading character: the system assigns one by one.
@end enumerate

3 thru 5 are notations of ISO2022.

Character-sets are defined by 'new-character-set' function call.

@example
--- mule.c ---------------------------------------------------------
DEFUN ("new-character-set", Fnew_character_set, Snew_character_set, 8, MANY, 0,
  "Define new character set of LEADING-CHAR (1st arg).\n\
Rest of args are:\n\
 BYTE: 1, 2, or 3\n\
 COLUMNS: 1 or 2\n\
 TYPE: 0 (94 chars), 1 (96 chars), 2 (94x94 chars), or 3 (96x96 chars)\n\
 GRAPHIC: 0 (use g0 on output) or 1 (use g1 on output)\n\
 FINAL: final character of ISO escape sequence\n\
 DIRECTION: 0 (left-to-right) or 1 (right-to-left)\n\
 DOC: short description string.\n\
If LEADING-CHAR >= 0xA0, it is regarded as extended leading-char\n\
and BYTE and COLUMNS args are ignored.")
------------------------------------------------------------
@end example

The system pre-defines the following character-sets.

@example
--- mule.el ---------------------------------------------------------
(defconst *predefined-character-set*
  (list
   ;; (cons lc '(bytes width type graphic final direction doc))
   ;; (cons lc-ascii '(0 1 0 0 ?B 0 "ASCII" "ISO8859-1")) ;; predefined in C
   (cons lc-ltn1 '(1 1 1 1 ?A 0 "Latin-1" "ISO8859-1"))
   (cons lc-ltn2 '(1 1 1 1 ?B 0 "Latin-2" "ISO8859-2"))
   (cons lc-ltn3 '(1 1 1 1 ?C 0 "Latin-3" "ISO8859-3"))
   (cons lc-ltn4 '(1 1 1 1 ?D 0 "Latin-4" "ISO8859-4"))
   (cons lc-thai '(1 1 1 1 ?T 0 "Thai" "TIS620"))
   (cons lc-grk '(1 1 1 1 ?F 0 "Greek" "ISO8859-7"))
   (cons lc-arb '(1 1 1 1 ?G 1 "Arabic" "ISO8859-6"))
   (cons lc-hbw '(1 1 1 1 ?H 1 "Hebrew" "ISO8859-8"))
   (cons lc-kana '(1 1 0 1 ?I 0 "Japanese Katakana" "JISX0201.1976"))
   (cons lc-roman '(1 1 0 0 ?J 0 "Japanese Roman" "JISX0201.1976"))
   (cons lc-crl '(1 1 1 1 ?L 0 "Cyrillic" "ISO8859-5"))
   (cons lc-ltn5 '(1 1 1 1 ?M 0 "Latin-5" "ISO8859-9"))
   (cons lc-jpold '(2 2 2 0 ?@ 0 "Japanese Old" "JISX0208.1978"))
   (cons lc-cn '(2 2 2 0 ?A 0 "Chinese" "GB2312"))
   (cons lc-jp '(2 2 2 0 ?B 0 "Japanese" "JISX0208.\\(1983\\|1990\\)"))
   (cons lc-kr '(2 2 2 0 ?C 0 "Korean" "KSC5601"))
   (cons lc-jp2 '(2 2 2 0 ?D 0 "Japanese Supplement" "JISX0212"))
   (cons lc-cns1 '(2 2 2 0 ?G 0 "CNS Plane1" "CNS11643.1"))
   (cons lc-cns2 '(2 2 2 0 ?H 0 "CNS Plane2" "CNS11643.2"))
   (cons lc-big5-1 '(2 2 2 0 ?0 0 "Big5 Level 1" "Big5"))
   (cons lc-big5-2 '(2 2 2 0 ?1 0 "Big5 Level 2" "Big5"))))

(let ((c *predefined-character-set*)
      lc data)
  (while c
    (setq lc (car (car c))
	  data (cdr (car c)))
    (apply 'new-character-set lc data)
    (setq c (cdr c))))
@end example

In addition, the following private character sets are predifined.

@example
--- mule-config.el -----------------------------------------
;; REGISTRATION OF PRIVATE CHARACTER SETS

;; PinYin-ZhuYin
(setq lc-sisheng
      (new-private-character-set 1 1 0 0 ?0 0 "PinYin-ZhuYin" "sisheng_cwnn"))

(setq lc-ascr2l
      (new-private-character-set 1 1 0 0 ?B 1 "Right-to-Left ASCII" "ISO8859-1")) 

;; Vietnamese VISCII with two tables.
(setq lc-vn-1
      (new-private-character-set 1 1 1 1 ?1 0 "VISCII lower" "VISCII1.1"))
(setq lc-vn-2
      (new-private-character-set 1 1 1 1 ?2 0 "VISCII upper" "VISCII1.1"))

;; Three character sets for Arabic
(setq lc-arb0
      (new-private-character-set
       1 1 0 0 ?2 0 "Arabic digit" "MuleArabic-0"))
(setq lc-arb1
      (new-private-character-set
       1 1 0 0 ?3 1 "1-column Arabic" "MuleArabic-1"))
(setq lc-arb2
      (new-private-character-set
       1 2 0 0 ?4 1 "2-column Arabic" "MuleArabic-2"))

;; for Mule IPA
(setq lc-ipa0
      (new-private-character-set 1 1 1 1 ?0 0 "IPA for Mule" "MuleIPA"))
------------------------------------------------------------
@end example

@node Coding-system, Syntax, Character, Top
@section Coding-system

@noindent
`coding-system' is a method for encoding several
character-sets and represented by a symbol which has
properties of 'coding-system and 'eol-type.

You can specify different coding-system on file I/O, process
I/O, output to terminal (if not running on X), input from
keyboard (if not running on X).


@menu
* Structure::   Structure of coding-system
	  o Property 'coding-system
	  o Property 'eol-type
	  o Property 'post-read-conversion
	  o Property 'pre-write-conversion
* Creation::   How to create coding-system?
* Predefined coding-system::
* Automatic conversion::
	  o Category of coding-system
	  o How automatic conversion works?
	  o Priority of category
* Mode-line::   How coding-system is shown in mode-line?::
* ISO2022 restriction::
* Big5::        Special treatment of Big5
@end menu

@node Structure, Creation, Coding-system, Coding-system
@section Structure of coding-system

@subsection Property 'coding-system

The value of the property 'coding-system is a vector:
@quotation
  [ TYPE MNEMONIC DOCUMENT DUMMY FLAGS ]
@end quotation
or the other coding-system.  Contents of the vector are:
@example
  TYPE:	nil: no conversion, t: automatic conversion,
	0:Internal, 1:Shift-JIS, 2:ISO2022, 3:Big5, 4:CCL.
  MNEMONIC: a character shown at mode-line to indicate the coding-system.
  DOCUMENT: a describing documents for the coding-system.
  DUMMY: always nil (for backward compatibility)
  FLAGS (option): more precise information about the coding-system,
    If TYPE is 2 (ISO2022), FLAGS should be a list of:
      LC-G0, LC-G1, LC-G2, LC-G3:
	Leading character of charset initially designated to G? graphic set,
	nil means G? is not designated initially,
	lc-invalid means G? can never be designated to,
	if (- leading-char) is specified, it is designated on output,
      SHORT: non-nil - allow such as \"ESC $ B\", nil - always \"ESC $ \( B\",
      ASCII-EOL: non-nil - designate ASCII to g0 at end of line on output,
      ASCII-CNTL: non-nil - designate ASCII to g0 at control codes on output
      SEVEN: non-nil - use 7-bit environment on output,
      LOCK-SHIFT: non-nil - use locking-shift (SO/SI) instead of single-shift
	or designation by escape sequence,
      USE-ROMAN: non-nil - designate JIS0201-1976-Roman instead of ASCII,
      USE-OLDJIS: non-nil - designate JIS0208-1976 instead of JIS0208-1983,
      NO-ISO6429: non-nil - don't use ISO6429's direction specification,
  If TYPE is 3 (Big5), FLAGS `t' means Big5-ETen, `nil' means Big5-HKU,
  If TYPE is 4 (private), FLAGS should be a cons of CCL programs
    for encoding and decoding.  See documentation of CCL for more detail.
@end example

@subsection Property 'eol-type

The value of the property 'eol-type is:
  nil: no conversion for end-of-line type
  1:   LF
  2:   CRLF
  3:   CR
  vector of length 3: automatic detection of end-of-line type.
	1st element: coding-system of eol-type LF
	2nd element: coding-system of eol-type CRLF
	3rd element: coding-system of eol-type CR

@subsection Property 'post-read-conversion

The value of the property 'post-read-conversion is a
function to convert some text just read into a buffer.  When
the function is called, the text has already been converted
according to 'coding-system and 'eol-type of the
coding-system.  The argument of the function is the region
(START and END) of inserted text.

@subsection Property 'pre-write-conversion

The value of the property 'pre-write-conversion is a
function to convert some text just before writing it out.
After the function is called, the text is converted accoding
to 'coding-system and 'eol-type of the coding-system.  The
argument of the function is the region (START and END) of
the text.

@node Creation, Predefined coding-system, Strucure, Coding-system
@section How to create coding-system?

Mule provides a function `make-coding-system' to create a
coding-system.

FUNCTION make-coding-system: NAME TYPE MNEMONIC DOC &optional EOL-TYPE FLAGS

Register symbol NAME as a coding-system whose 'coding-system
property is a vector [ TYPE MNEMONIC DOC nil FLAGS ] and
'eol-type property is EOL-TYPE.  If `t' is specified as
EOL-TYPE, the value of 'eol-type property is a vector of
generated coding-systems whose 'eol-type properties are 1
(LF), 2 (CRLF), and 3 (CR).  The names of generated
coding-systems are NAMEunix, NAMEdos, and NAMEmac respectively.

Just to make an alias of some coding-system, call a fucntion
`copy-coding-system'.

FUNCTION copy-coding-system: ORIGINAL ALIAS

Make the same coding-system as ORIGINAL and name it ALIAS.
If 'eol-type property of ORIGINAL is a vector, coding-systems
ALIASunix, ALIASdos, and ALIASmac are generated, and
'eol-type property of ALIAS becomes a vector of them.

@node Predefined coding-system, Automatic conversion, Creation, Coding-system
@section Predefined coding-system

In the file lisp/mule.el, the following coding-systems are
predefined.

@example
----- lisp/mule.el -----------------------------------------
(make-coding-system
 '*noconv* nil
 ?= "No conversion.")

(make-coding-system
 '*autoconv* t
 ?+ "Automatic conversion." t)

(make-coding-system
 '*internal* 0
 ?= "Internal coding-system used in a buffer.")

(make-coding-system
 '*sjis* 1
 ?S "Coding-system of Shift-JIS used in Japan." t)

(make-coding-system
 '*iso-2022-jp* 2
 ?J "Coding-system used for communication with mail and news in Japan."
 t
 (list lc-ascii lc-invalid lc-invalid lc-invalid
       'short 'ascii-eol 'ascii-cntl 'seven))
(copy-coding-system '*iso-2022-jp* '*junet*)

(make-coding-system
 '*oldjis* 2
 ?J "Coding-system used for old jis terminal."
 t
 (list lc-ascii lc-invalid lc-invalid lc-invalid
       'short 'ascii-eol 'ascii-cntl 'seven nil 'use-roman 'use-oldjis))

(make-coding-system
 '*ctext* 2
 ?X "Coding-system used in X as Compound Text Encoding."
 t
 (list lc-ascii lc-ltn1 lc-invalid lc-invalid
       nil 'ascii-eol))

(make-coding-system
 '*euc-japan* 2
 ?E "Coding-system of Japanese EUC (Extended Unix Code)."
 t
 (list lc-ascii lc-jp lc-kana lc-jp2
       'short 'ascii-eol 'ascii-cntl))

(make-coding-system
 '*euc-korea* 2
 ?K "Coding-system of Korean EUC (Extended Unix Code)."
 t
 (list lc-ascii lc-kr lc-invalid lc-invalid
       nil 'ascii-eol 'ascii-cntl))
;; 93.12.16 by K.Handa
(copy-coding-system '*euc-korea* '*euc-kr*)

(make-coding-system
 '*iso-2022-kr* 2
 ?k "Coding-System used for communication with mail in Korea."
 nil
 (list lc-ascii (- lc-kr) lc-invalid lc-invalid
       nil 'ascii-eol 'ascii-cntl 'seven 'lock-shift))
(copy-coding-system '*iso-2022-kr* '*korean-mail*)

(make-coding-system
 '*euc-china* 2
 ?C "Coding-system of Chinese EUC (Extended Unix Code)."
 t
 (list lc-ascii lc-cn lc-invalid lc-invalid
       nil 'ascii-eol 'ascii-cntl))

(make-coding-system
 '*iso-2022-ss2-8* 2
 ?I "ISO-2022 coding system using SS2 for 96-charset in 8-bit code."
 t
 (list lc-ascii lc-invalid nil lc-invalid
       nil 'ascii-eol 'ascii-cntl))

(make-coding-system
 '*iso-2022-ss2-7* 2
 ?I "ISO-2022 coding system using SS2 for 96-charset in 7-bit code."
 t
 (list lc-ascii lc-invalid nil lc-invalid
       'short 'ascii-eol 'ascii-cntl 'seven))

(make-coding-system
 '*iso-2022-lock* 2
 ?i "ISO-2022 coding system using Locking-Shift for 96-charset."
 t
 (list lc-ascii nil lc-invalid lc-invalid
       nil 'ascii-eol 'ascii-cntl 'seven
       'lock-shift))			;93.12.1 by H.Minamino

(make-coding-system
 '*big5-eten* 3
 ?B "Coding-system of BIG5-ETen."
 t t)

(make-coding-system
 '*big5-hku* 3
 ?B "Coding-system of BIG5-HKU."
 t nil)
------------------------------------------------------------
@end example

@node Automatic conversion, Mode-line, Predefined coding-system, Coding-system
@section Automatic conversion

@subsection Category of coding-system

Mule has a facility to detect coding-system of text
automatically, however, what mule actually detect is not a
coding-system itself but a category of coding-system.  A
category is also represented by a symbol and a value should
be an actual coding-system.

There are eight categories:
@table @asis
@item *coding-category-internal*:
	coding-system used in a buffer
@item *coding-category-sjis*
	Shift-JIS
@item *coding-category-iso-7*
	ISO2022 variation with the following feature:
	  o no locking shift, single shift
	  o only G0 is used
@item *coding-category-iso-8-1*
	ISO2022 variation with the following feature:
	  o no locking shift
	  o designation sequence is allowed only for G0 and G1
	  o G1 is used only for 1-byte character set
@item *coding-category-iso-8-2*
	ISO2022 variation with the following feature:
	  o no locking shift
	  o designation sequence is allowed only for G0 and G1
	  o G1 is used only for 2-byte character set
@item *coding-category-iso-else*
	ISO2022 variation which doesn't satisfy any of above.
@item *coding-category-big5*
	Big5 (ETen or HKU)
@item *coding-category-bin*
	Any other coding-system which uses MSB.
@end table

The values of these symbols are pre-defined as follows:

@example
----- lisp/mule.el -----------------------------------------
(defvar *coding-category-internal* '*internal*)
(defvar *coding-category-sjis* '*sjis*)
(defvar *coding-category-iso-7* '*junet*)
(defvar *coding-category-iso-8-1* '*ctext*)
(defvar *coding-category-iso-8-2* '*euc-japan*)
(defvar *coding-category-iso-else* '*iso-2022-ss2-7*)
(defvar *coding-category-big5* '*big5-eten*)
(defvar *coding-category-bin* '*noconv*)
------------------------------------------------------------
@end example

but, some of them are overridden in such language specific
files as japanese.el, chinese.el, etc.

@subsection How automatic conversion works?

When coding-system `*autoconv*' is specified on reading text
(this is the default), mule tries to detect a category of
coding-system by which text are encoded.  If an appropriate
category is found, it converts text according to a
coding-system bound to the cateogry.  If the 'eol-type
property of the coding-system is a vector of coding-systems
and Mule detects a type of end-of-line (LF, CRLF, or CR) of
the text, one of those coding-system is used.

Automatic conversion occurs both on reading from files and
inputing from process.  In the latter case, if some
coding-system is found, output-coding-system of the process
is also set to the found coding-system.

@subsection Priority of cateogry

In the case that more than two categories are found, the
category of the highest priority is selected.

A priority of category is pre-defined as follows:

@example
----- lisp/mule.el -----------------------------------------
(set-coding-priority
 '(*coding-category-iso-8-2*
   *coding-category-sjis*
   *coding-category-iso-8-1*
   *coding-category-big5*
   *coding-category-iso-7*
   *coding-category-iso-else*
   *coding-category-bin*
   *coding-category-internal*))
------------------------------------------------------------
@end example

The function `set-coding-priority' put a property 'priority
to each element of the argument from 0 to 7 (smaller number
has higher priority).  Some language specific files may
override this priority.

@node Mode-line, ISO2022 restriction, Automatic conversion, Coding-system
@section How coding-system is shown in mode-line?

Each coding-system has unique mnemonic (one character).
By default, mnemonic of `file-coding-system' of a buffer is
shown at the left of mode-line of the buffer.  In addition,
the mnemonic is followed by an another mnemonic to show
eol-type of the coding-system.  This mnemonic is defined as
follows:
	".": LF
	":": CRLF
	"'": CR
	"_": not yet desided
	"-": nil (for coding-system of nil, *noconv*, or *internal*)
So, usual appearance of mode-line for a buffer which is
visiting a file (*junet* encoding on Unix system) is:

@example
	    +-- mnemonic of file-coding-system
	    |+-- mnemonic of eol-type
	    VV
	[--]J.:----Mule: filename
@end example

The left most bracket is the indicator for inputing method.

When a buffer is attaced to some process, coding-system
for input and output of the process are also shown as
follows:

@example
	    +-- mnemonic of file-coding-system
	    |+-- mnemonic of eol-type of file-coding-system
	    ||+-- mnemonic of input-coding-system of a process
	    |||+-- mnemonic of eol-type of input-coding-system
	    ||||+-- mnemonic of output-coding-system of a process
	    |||||+-- mnemonic of eol-type of output-coding-system
	    VVVVVV
	[--]+_+.--:--**-Mule: *shell*
@end example

This means that Mule is now communicating with shell with
coding-systems *autoconv*unix ("+.") for input and nil
("--") for output.

@node ISO2022 restriction, Big5, Mode-line, Coding-system
@section ISO2022 restriction

For decoding to Type 2 (ISO2022), we have the following
restrictions:

@table @asis
@item Locking-Shift:
Use SI and SO only when decoding with a coding-system
whose LOCK-SHIFT and SEVEN is t.

@item Single-Shift:
Use SS2 and SS3 (if SEVEN is nil) or ESC N and ESC O (if
SEVEN is t).

@item Invocation:
G0 is always invoked to GL, G1 to GR (but only if SEVEN is
nil).  G2 and G3 are invoked to GL by Shingle-Shift of SS2
and SS3.

@item Unofficial use of ESC sequence for designation:
If SEVEN is t, LOCK-SHIFT is nil, and designation to G2
and G3 are prohibited, we should designate all character
sets to G0 (and hence invoke to GL).  To designate 96
char-set to G0, we use "ESC , <F>".  For instance, to
designate ISO8859-1 to G0, we use "ESC , A".

@item Unofficial use of ESC sequence for composit character:
To indicate the start and end of composit character, we
use ESC 0 (start) and ESC 1 (end).

@item Text direction specifier of ISO6429
We use ISO6429's ESC sequence "ESC [ 2 ]" to change text
direction to right-to-left, and "ESC [ 0 ]" to revert it
to left-to-right.
@end table

@node Big5, , ISO2022 restriction, Coding-system
@section Special treatment of Big5

As far as I know, there's several different codes called
Big5.  The most famous ones are Big5-ETen and
Big5-HKU-form2.  Since both of them use a code range 0xa140
- 0xfefe (in each row, columns (second byte) 0x7f - 0xa0 is
skipped) and number of characters is more than 13000, it's
impossible to treat each of them as a single character-set
in the current Mule system.  So, Mule treat them in a quite
irregular manner as described below:

@enumerate
@item
Mule does not treats them as a different character set,
but as the same character set called Big5.
	Caution!! Big5 is a different character set from GB.

@item
Mule divides Big5 into two sub-character-sets:
	0xa140 - 0xc67e (Level 1)
	0xc6a1 - 0xfefe (Level 2)
and allocates two leading-chars lc-big5-1 and lc-big5-2 to
them.  (See character.txt)

@item
Usually, each leading-char (or character-set) has unique
character category.  But lc-big5-1 and lc-big5-2 has the
same character category of mnemonic 't'.  So, regular
expression "\\ct" matches any Big5 (Level 1 and Level 2)
characters.  (See syntax.txt)

@item
If you specify ISO2022 type coding-system on output,
Mule converts Big5 code using unofficial final-characters
'0' (for Level 1) and '1' (for Level 2).

@item
You can use either fonts of ETen or HKU for displaying
Big5 code.  Mule judges which font is used by examining
existence of character whose code point is 0xC6A1.  If it
exists, the font is HKU, else the fonts is ETen.
@end enumerate

@node Syntax, Font, Coding-system, Top
@section Syntax and Category of character

@subsection Syntax

Mule can define syntax of all multi-byte characters by
@code{modify-syntax-entry}.

The first argument of @code{modify-syntax-entry should} be one of below:
@enumerate
@item
ASCII character
@item
multi-byte character
@item
leading character of multi-byte character
@item
partially defined characters returned by:

@quotation
@code{(make-character leading-char arg)}
@end quotation
@end enumerate

There's a restriction of specifying matching character within 
second argument.  If the first argument specifies multi-byte
character or leading char of multi-byte character, the
matching character should have the same leading character.  If
the character is 2-byte code, the first-byte of it should
also be the same with the first-byte of first argument.

@subsection Category

Like syntax, category also defines characteristics of
characters.  The differences are:
@enumerate
@item
Each Character can have more than one category.
@item
User can define new type of category as he wishes.
	Example: See japanese.el
@item
@code{char-category} returns all mnemonics of the character by string.
@item
For regular expression search, you can use the \cm or \Cm (any mnemonics
comes at the place of 'm') instead of \sm and \Sm.
@end enumerate

@node Font, CCL, Syntax, Top
@section Font

FONTSET is a set of fonts which have the same height and style.  A
fontset should hopefully contain enough fonts to display a character of
various character sets.

Mule uses fontset instead of font.  You can specify fontset at any place
where you can specify font.  You can still specify font, in which case,
a fontset which include the font is searched and used.

Like font, fontset is also a string specifying the name.

@menu
* Initial fontsets::	Fontsets which Mule have at startup time.
* Specify fontset::     How to specify a fontset?
* Manage fontset::      How to create or modify a fontset?
@end menu

@node Initial fontsets, Specify fontset, , Font

@subsection "default-fontset"

Mule automatically creates a fontset named "default-fontset" at startup
time.  Each font in this fontset is specifed by a very generic name such
as "-*-fixed-medium-r-*--16-*-iso8859-1" for ASCII and
"-*-fixed-medium-r-*--*-jisx0208.1983-*" for JISX0208 (Kanji).
These values are defined in @file{lisp/term/x-win.el}.

If there's no other fontsets specifed by X's resource, "default-fontset"
is used for the first frame of Mule.

In most cases, this is enough.  You probably don't have to have any
other fontsets.

@subsection  X's resourse

Mule also creates fontsets specified in X's resource "fontSetList (class
FontSetList)".  The value is a comma separated list of fontset names.

@example
*FontSetList: 16,24
@end example

The actual contents of each fontset is specified by "fontSet-xxx (class
FontSet-xxx)" where "xxx" is a name of the corresponding fontset.  The
value of this resource is a comma separated list of font names.

@example
*FontSet-16: -etl-fixed-medium-r-*--24-*-iso8859-1
@end example

Each font name should not contain wild card `*' or `?' in
CHARSET_REGSTRY field because a character set for this font is
recognized by this field.  This means that you don't have to care about
the order of font names.

For instance,

@example
*FontSet-16:\
        -etl-fixed-medium-r-*--16-*-iso8859-1\
	-ming-fixed-medium-r-*--*-*-jisx0208.1983-*
@end example

is enough to tell Mule that the fontset "16" contains ASCII font and
JISX0208 font.  Please note that the second name has only wild card in
PIXEL_SIZE field.  Since Mule try to open a font of the same PIXEL_SIZE
as ASCII font of the same fontset, you'ld better not specify actual
value in PIXEL_SIZE field except for ASCII font.

As for fonts not listed in the specification of fontset, corresponding
font names in "default fontset" is used.

The first fontset in FontSetList is used for the first frame of Mule.
If you want to use "default-fontset" while specifying other fontsets in
the resource, please put "default-fontset" at the first of the value.

@example
*FontSetList: default-fontset,16,24
@end example

In this case, you don't have to have the resource
"FontSet-default-fontset".

@node Specify fontset, Manage fontset, Initial fontsets, Font
@section How to specify a fontset?

You can specify fontset at any place where you can sepcify font.

To change the fontset used for the first frame of Mule:

@enumerate
@item
command line arguments "-fn xxx" or "-font xxx"

If this argument exits, fontset is searched in the following order:
@enumerate
@item
A fontset whose name is "xxx".
@item
A fontset which contains ASCII font "xxx".
@item
Create a new fontset "xxx" which contains ASCII font "xxx".
@end enumerate

@item
In your ~/.emacs,

@example
(setcdr (assoc 'font default-frame-alist) "xxx")
@end example

@end enumerate

To change a fontset after Mule started:

@enumerate
@item
By the command

@example
M-x set-default-fontset<CR>xxx<CR>
@end example

@item
By @key{Ctl-Mouse-3}

@end enumerate

@node Manage fontset, , Specify fontset, Font
@section How to create or modify a fontset?

You can create a new fontset by `new-fontset' and modify an
existing fontset by `set-fontset-font'.

You can get a list of fontset currently created by
`fonset-list'.

You can check if a fontset is already created or not by
`fonsetp'.



@contents
@bye
