Previous  Up  Next

Module Kwmarshal

module Kwmarshal: sig .. end

Marshaling with Tags, Versioning, and Magic Numbers

This module is a clone of Marshal that adds a tag and an optional version number to the marshaled payload. These allow an application to distinguish its own marshaled data from that of another application, and generate useful error messages instead of core dumps.

It also adds a file magic number (suitable for file(1)) to the data, to be provide minimal information for non-OCaml programs.

Note that these functions actually use Marshal -- just in a newly structured way -- so all the issues with Marshal apply here (e.g. lack of type safety, issues with extensible variant types, exceptions, and closures).

New versions of all the marshaling functions from Marshal are provided with modified APIs:

The mandatory ~tag is a string that tags or labels the type of marshaled data. It might be an application name, for example. This tag is stored with the marshaled data to identify it.

When unmarshaling, you need to provide the same ~tag value; if the given tag doesn't exactly match the tag stored with the marshaled data, a Kwmarshal.WrongTag exception is raised.

If used, version numbers must "match" as well. The default is that the versions must match at the major version level.

The tag and version are stored in a Kwmarshal.header alongside your payload. The unmarshaling functions return a pair of header * 'a, where 'a is the type of your marshaled payload. Here's an example: 

let s = to_string ~tag:"myapp" ("foo",12) [] in
   (from_string ~tag:"myapp" s 0 : header * (string * int))

Author(s): Keith Waclena
See also Home page

Constants and Types


type extern_flags =
| No_sharing (*
Don't preserve sharing
*)
| Closures (*
Send function closures
*)
| Compat_32 (*
Ensure 32-bit compatibility
*)
The flags to the Kwmarshal.to_* functions below.
val versionno : string
The version number of format of the marshaled data we write.
val magicprefix : string
The file magic number prefix.
type header = {
   tag : string;
   version : string option;
}
The type of the data header.

Exceptions


exception Bad
Exception raised when attempting to unmarshal data without a Kwmarshal magic number.
exception WrongTag of string * string

Exception raised when attempting to unmarshal data with an incorrect tag.

The exception's parameter is the pair of expected tag, actual tag.

exception WrongVersion of string option * string option

Exception raised when attempting to unmarshal data with an incorrect version.

The exception's parameter is the pair of expected version, actual version.

Version Numbers

Versions numbers are string options. Not providing a ~version parameter when unmarshaling (i.e., your given version is None) means you don't want to check the version in the header.

A given ~version parameter of (Some g) never matches a header version of None, and successfully matches a header version of (Some h) iff vequals g h.

No syntax of the actual version "number" string is presumed except by the default Kwmarshal.vequals function; if you override it, you can use any syntax you like.

Functions


val fixflags : extern_flags list -> Marshal.extern_flags list
fixflags flags: map our Kwmarshal.extern_flags to Marshal's.
val vequals : string -> string -> bool
vequals given header: is the given version number equal to the version number in the header?

This function assumes a version number syntax that matches the regexp ^[0-9]+[.]?; the initial number is the major version number.

We compute equality as (m=n), where m is the given major version number and n is the header's major version number.

given : the version number provided via the ?version parameter
header : the version number in the header of the marshaled data
val to_channel : tag:string ->
       ?version:string ->
       Pervasives.out_channel -> 'a -> extern_flags list -> unit
to_channel ~tag ?version chan payload flags: writes the representation of payload on channel chan.

The marshaled data includes a magic number and a Kwmarshal.header, so only Kwmarshal.from_channel can unmarshal the data (Marshal.from_channel cannot).

tag : the tag that identifies this payload
version : the optional version number of the payload
chan : the output channel
payload : the payload
flags : see Marshal.to_channel
val from_channel : ?vequals:'a ->
       tag:string ->
       ?version:string -> Pervasives.in_channel -> header * 'b
from_channel ?vequals ~tag ?version chan: reads from channel chan the byte representation of a structured value, as produced by one of the Kwmarshal.to_* functions, and reconstructs and returns the corresponding value.

The ~tag and ?version are observed as described above.

vequals : the equality function for version numbers
tag : the tag that identifies this payload
version : the optional version number of the payload
chan : the output channel
val to_bytes : tag:string -> ?version:string -> 'a -> extern_flags list -> bytes
to_bytes ~tag ?version payload flags: returns a byte sequence containing the representation of payload.

The marshaled data includes a magic number and a Kwmarshal.header, so only Kwmarshal.from_bytes can unmarshal the data (Marshal.from_bytes cannot).

tag : the tag that identifies this payload
version : the optional version number of the payload
payload : the payload
flags : see Marshal.to_bytes
val from_bytes : ?vequals:'a -> tag:string -> ?version:string -> bytes -> int -> 'b * 'c
from_bytes ?vequals ~tag ?version bytes ofs: unmarshals a structured value like Kwarshal.from_channel does, except that the byte representation is not read from a channel, but taken from the byte sequence buff, starting at position ofs.

The byte sequence is not mutated.

The ~tag and ?version are observed as described above.

vequals : the equality function for version numbers
tag : the tag that identifies this payload
version : the optional version number of the payload
bytes : the byte sequence
val to_string : tag:string -> ?version:string -> 'a -> extern_flags list -> string
to_string ~tag ?version payload flags: Same as Kwmarshal.to_bytes but return the result as a string instead of a byte sequence.

The marshaled data includes a magic number and a Kwmarshal.header, so only Kwmarshal.from_string can unmarshal the data (Marshal.from_string cannot).

tag : the tag that identifies this payload
version : the optional version number of the payload
payload : the payload
flags : see Marshal.to_string
val from_string : ?vequals:'a -> tag:string -> ?version:string -> string -> int -> 'b * 'c
from_string ?vequals ~tag ?version str ofs: Same as Kwmarshal.from_bytes but takes a string as argument instead of a byte sequence.

The byte sequence is not mutated.

The ~tag and ?version are observed as described above.

vequals : the equality function for version numbers
tag : the tag that identifies this payload
version : the optional version number of the payload
str : the string
val to_buffer : tag:string ->
       ?version:string ->
       bytes -> int -> int -> 'a -> extern_flags list -> int
to_buffer ~tag ?version buff ofs len payload flags: marshals the value payload, storing its byte representation in the sequence buff, starting at index ofs, and writing at most len bytes.

It returns the number of bytes actually written to the sequence. If the byte representation of payload does not fit in len characters, the exception Failure is raised.

The marshaled data includes a magic number and a Kwmarshal.header, so only Kwmarshal.from_bytes can unmarshal the data (Marshal.from_bytes cannot).
Raises Failure if payload won't fit in buff

tag : the tag that identifies this payload
version : the optional version number of the payload
buff : the buffer
ofs : the offset into the buffer
len : the size of the available buffer space (after ofs)
payload : the payload
flags : see Marshal.to_string