Module Prelude.Refer

Parse and generate Extended Refer databases.

Refer is an old low-noise, low-overhead, pleasant to edit, flat-file data format for non-recursive key-value data with repeating and optional fields.

Extended Refer is simply the classic Refer format but allowing key names of arbitrary length (classic refer key names had to be exactly 1 character long).

Terminology: a Refer "database" is a collection of Refer records, typically in one or more files (but possibly in a string).

Parsed Refer records are represented by this module as association lists (alists; see Lists.Assoc). Keys are represented without their attached percent-signs. The order of the fields in a record is preserved by the parser.

Types and Exceptions

type t = (string * string) list

t is the type of parsed refer records.

type src = string Gen.t

src is the type of input sources for the Refer parser fold.

Each string of a src is a line of text.

type 'a folder = int -> (t, string) Stdlib.result -> 'a -> 'a

The type of functions called by Refer.fold for each Refer record in a src.

An ('a folder) takes three parameters:

  • the line-number of the first line of the alist representing the Refer record
  • a ((t,string) result) which is (Ok t) if there has been no syntax error, or (Error line) is there has been a syntax error; in this case line is the erroneous line.
  • the accumulator
type 'a errhandler = int -> string -> 'a -> 'a

The type of functions suitable as the first parameter of Refer.witherr.

An ('a errhandler) takes three parameters:

  • the line-number of the error
  • the line of text which is in error
  • the accumulator
exception Syntax of int * string

Syntax (ln,x) is the type of syntax error exceptions raised by Refer.syntax.

Generating Refer Databases From Alists

module Alist : sig ... end

A Refer record can be generated from an association list (alist), and a refer database can be generated from a list of alists.

val assemble : ?fix:bool -> (string * string) list -> string

assemble is Alist.assemble.

Random Refer Records

module Random : sig ... end

Functions to generate random Refer records.

val random : ?keys:(unit -> string list) -> ?values:(?len:int -> ?lines:int -> unit -> string) -> unit -> string

random is Random.record.

Lazy Sequences of Refer Records

module Seq : sig ... end

Functions to generate lazy sequences of refer records.

Input Sources of Refer Databases

val of_channel : ?readline:(Stdlib.in_channel -> string) -> Stdlib.in_channel -> string Gen.t

(of_channel chan) creates a src from the in_channel chan.

val of_string : string -> string Gen.t

(of_string str) creates a src from the refer database represented by the string str.

Parsing Refer Databases

val fold : 'a folder -> 'a -> string Gen.t -> 'a

(fold f acc g) folds over the Refer database on input source g, calling f for each record.

Example: count the number of records in a Refer database (ignoring syntax errors):

  • (fold (witherr ignore (cons (fun _ -> succ))) 0))

Example: if r is "%A 1\n%B 2\n%B 3\n%C 4" then:

(of_string r |> fold (witherr ignore (fun _ -> List.cons)) [])
= [[("A", "1"); ("B", "2"); ("B", "3"); ("C", "4")]]
val iter : (int -> (t, string) Stdlib.result -> unit) -> string Gen.t -> unit

(iter f g) iterates f across the Refer records on input source g.

Example: print the Refer database on g to stdout:

  • (iter (fun _ln -> Result.get_ok >> assemble >> print_endline) g)

Fold Helpers

val cons : ((t, string) Stdlib.result -> 'a -> 'a) -> 'a folder

(cons f) makes an 'a folder out of the List.cons-like function f.

This makes it easy to ignore the line-number parameter.

Example: this is the list of results from the src g:

  • (fold (cons List.cons) [] g)
val snoc : ('a -> (t, string) Stdlib.result -> 'a) -> 'a folder

(snoc f) makes an 'a folder out of the Lists.snoc-like function f.

val witherr : 'a errhandler -> (int -> t -> 'a -> 'a) -> 'a folder

(witherr e f) makes an 'a folder out of e (a errhandler) and f.

N.B. f is not a ('a folder) because it takes a t rather than a ((t, string) Stdlib.result); this is the whole point of witherr, but it's easy to get confused.

The resulting (folder ln r acc) applies f to each t when (r = (Ok t)) and applies e to each x when (r = (Error x)).

Example: this is the (t list) from the src g, ignoring any Error results:

  • (fold (witherr ignore (fun _ -> List.cons)) [] g)

Example: this is the ((int * string) list) of all errors from the src g, ignoring any Ok results:

  • (fold (witherr (curry List.cons) ignore) [] g)

Predefined Error Handlers

For witherr.

val syntax : int -> string -> 'a -> 'b

(syntax) is an error-handler for witherr that raises Syntax.

val ignore : 'a -> 'b -> 'c -> 'c

(ignore) is an error-handler for witherr that silently ignores any syntax errors.

val print : ?msg:string -> string -> int -> string -> 'a -> 'a

(print ?msg fn) is an error-handler for witherr that prints an error message to stderr for each syntax error.

  • fn is the filename
  • ~msg is the error message printed in front of the erroneous line (default: "syntax error")

The error message is in the standard format understood by Unix compiler utilities and editors like Emacs, Vi, Vim, and the like.

Example:

let fn = "/etc/passwd" in
within Refer.(fold (witherr (print fn) ignore) [] << of_channel) fn

might print: /etc/passwd:1:syntax error: root:x:0:0:root:/root:/bin/bash