######################################################################
## 
##    Copyright (C) 2001,2000, 2003
##    Department of Computer Science, University of Tromsø, Norway
## 
## Filename:      README
## Author:        Frode Vatvedt Fjeld <[email protected]>
## Created at:    Wed Dec  8 15:35:53 1999
## Distribution:  See the accompanying file COPYING.
##                
## $Id: README,v 1.1.1.1 2004/01/13 11:13:13 ffjeld Exp $
##                
######################################################################

Binary-types is a Common Lisp package for reading and writing binary
files. Binary-types provides macros that are used to declare the
mapping between lisp objects and some binary (i.e. octet-based)
representation.

Supported kinds of binary types include:

 * Signed and unsigned integers of any octet-size, big-endian or
   little-endian. Maps to lisp integers.

 * Enumerated types based on any integer type. Maps to lisp symbols.

 * Complex bit-field types based on any integer type. Sub-fields can
   be numeric, enumerated, or bit-flags. Maps to lisp lists of symbols
   and integers.

 * Fixed-length and null-terminated strings. Maps to lisp strings.

 * Compound records of other binary types. Maps to lisp DEFCLASS
   classes or, when you prefer, DEFSTRUCT structs.

Typically, a complete binary record format/type can be specified in a
single (nested) declaration statement. Such compound records may then
be read and written with READ-BINARY and WRITE-BINARY.

Binary-types is *not* helpful in reading files with variable
bit-length code-words, such as most compressed file formats. It will
basically only work with file-formats based on 8-bit bytes
(octets). Also, at this time no floating-point types are supported out
of the box.

Binary types may now be declared with the DEFINE-BINARY-CLASS macro,
which has the same syntax (and semantics) as DEFCLASS, only there is
an additional slot-option (named :BINARY-TYPE) that declares that
slot's binary type. Note that the binary aspects of slots are *not*
inherited (the semantics of inheriting binary slots is unclear to me).

Another slot-option added by binary-types is :MAP-BINARY-WRITE, which
names a function (of two arguments) that is applied to the slot's
value and the name of the slot's binary-type in order to obtain the
value that is actually passed to WRITE-BINARY. Similarly,
:MAP-BINARY-READ takes a function that is to be applied to the binary
data and type-name when a record of that type is being read.  A
slightly modified version of :map-binary-read is
:MAP-BINARY-READ-DELAYED, which will do essentially the same thing as
:map-binary-read, only the mapping will be "on-demand": A slot-unbound
method will be created for this purpose.

A variation of the :BINARY-TYPE slot-option is :BINARY-LISP-TYPE,
which does everything :BINARY-TYPE does, but also passes on a :TYPE
slot-option to DEFCLASS (or DEFSTRUCT).  The type-spec is inferred
from the binary-type declaration. When using this mechanism, you
should be careful to always provide a legal value in the slot (as you
must always do when declaring slots' types). If you find this
confusing, just use :BINARY-TYPE.

Performance has not really been a concern for me while designing this
package. There's no obvious performance bottlenecks that I know of,
but keep in mind that all "binary" reads and writes are reduced to
individual 8-bit READ-BYTEs and WRITE-BYTEs. If you do identify
particular performance bottlenecks, let me know.

The included file "example.lisp" demonstrates how to use this
package. To give you a taste of what it looks like, the following
declarations are enough to read the header of an ELF executable file
with the form

   (let ((*endian* :big-endian))
     (read-binary 'elf-header stream)


;;; ELF basic type declarations
(define-unsigned word 4)
(define-signed sword  4)
(define-unsigned addr 4)
(define-unsigned off  4)
(define-unsigned half 2)

;;; ELF file header structure
(define-binary-class elf-header ()
  ((e-ident
    :binary-type (define-binary-struct e-ident ()
		   (ei-magic nil :binary-type
			     (define-binary-struct ei-magic ()
			       (ei-mag0 0 :binary-type u8)
			       (ei-mag1 #\null :binary-type char8)
			       (ei-mag2 #\null :binary-type char8)
			       (ei-mag3 #\null :binary-type char8)))
		   (ei-class nil :binary-type
			     (define-enum ei-class (u8)
			       elf-class-none 0
			       elf-class-32   1
			       elf-class-64   2))
		   (ei-data nil :binary-type
			    (define-enum ei-data (u8)
			      elf-data-none 0
			      elf-data-2lsb 1
			      elf-data-2msb 2))
		   (ei-version 0 :binary-type u8)
		   (padding nil :binary-type 1)
		   (ei-name "" :binary-type
			    (define-null-terminated-string ei-name 8))))
   (e-type
    :binary-type (define-enum e-type (half)
		   et-none 0
		   et-rel  1
		   et-exec 2
		   et-dyn  3
		   et-core 4
		   et-loproc #xff00
		   et-hiproc #xffff))
   (e-machine
    :binary-type (define-enum e-machine (half)
		   em-none  0
		   em-m32   1
		   em-sparc 2
		   em-386   3
		   em-68k   4
		   em-88k   5
		   em-860   7
		   em-mips  8))
   (e-version   :binary-type word)
   (e-entry     :binary-type addr)
   (e-phoff     :binary-type off)
   (e-shoff     :binary-type off)
   (e-flags     :binary-type word)
   (e-ehsize    :binary-type half)
   (e-phentsize :binary-type half)
   (e-phnum     :binary-type half)
   (e-shentsize :binary-type half)
   (e-shnum     :binary-type half)
   (e-shstrndx  :binary-type half)))


For a second example, here's an approach to supporting floats:

  (define-bitfield ieee754-single-float (u32)
    (((:enum :byte (1 31))
       positive 0
       negative 1)
      ((:numeric exponent 8 23))
      ((:numeric significand 23 0))))




The postscript file "type-hierarchy.ps" shows the binary types
hierarchy.  It is generated using psgraph from the CMU lisp
repository:

  (with-open-file (*standard-output* "type-hierarchy.ps"
                   :direction :output
                   :if-exists :supersede)
    (psgraph:psgraph 'binary-type
	             #'(lambda (p)
	                 (mapcar #'class-name
				 (aclmop:class-direct-subclasses
                                   (find-class p))))
                     #'(lambda (s) (list (symbol-name s)))
	             t))