Expect

Expect is a Tcl extension designed for scripting applications. It's similar to the scripting languages used in telecommunications packages, but much more general. By default, Expect is built on plain (not Extended) Tcl.

In a sense, Expect shouldn't be necessary. It should be possible to script, to connect together under program control, with shell scripts or ordinary Tcl programs. Why was Expect written?

Good Unix programs are written as tools, and are meant to be programmed with. The primary mechanism for hooking tools together is the pipe, combined with the standard input / standard output filter model (though pipes can also work with more file descriptors). However, there is a class of Unix programs that can't be hooked together this way: programs which talk directly to a terminal device (i.e., interactive programs).

The fact that interactive programs can't be programmed via pipes is an accident of history. When Unix was first introduced, the concept of pipes was new and people didn't think you could program interactive applications with them (or they didn't realize the potential benefits). Modern research Unix systems have attempted, with success, to bring the notion of an interactive session back into the fold with a generalization of both pipes and terminals called streams. But those of us using commercial Unix systems still have a problem.

Berkeley Unix added programmable terminal devices, called pseudo terminals (PTYs) years ago. With PTYs, you can convince an interactive program that its talking to a terminal when in fact another program has control of both ends. However, PTYs are extremely complex and hard to program (by Unix standards). What we need is a level abstraction, a tool that makes programming PTYs easier. Expect is one such tool.

A classic example of the problem is programming FTP. On most Unix systems, the FTP program can't be scripted via pipes, mainly because it wants to be able to turn off echoing when it prompts you for your password. If you try to write a simple Tcl program that execs an FTP process and speaks to it via pipes, you'll find it doesn't work.

Example Uses Of Expect

The Expect Interpreter

expect ?-d? -f scriptfile
The Expect interpreter is called expect. It's a plain Tcl interpreter with the Expect language extensions added.

A Typical Expect Script

A typical Expect script controls another process via a dialogue. Expect alternates sending some text to the process and then reacting to the text produced by the process in response. Since the responses will vary, Expect uses pattern matching to distinguish between possible responses. Commonly, after an initial dialogue under Expect's control, the rest of the interaction is turned over to the user. So the typical Expect script looks something like:
  1. Spawn a process
  2. Repeatedly:
    1. Send a message to the process
    2. Expect a set of possible responses, and react
  3. Turn the interaction over to the user

The spawn Command

spawn program ?args?
The spawn command is similar to the Tcl exec command, in that it fires up another process under Tcl's control. However, spawn's process is running asynchronously, so spawn returns control to Tcl immediately. Also, special Expect commands are used to communicate with the spawned process (rather than file descriptors as with exec). spawn returns the process ID of the spawned process.

The expect Command

expect ?-opts? pat1 body1 ... ?-opts? patn ?bodyn?
The expect command is the heart of the Expect language. expect is a control structure that selects a body-i to execute based on matching the patterns pat-i against the output of the spawned process. For example:
expect {*Account:*}
The above command has a pattern but no body; it will simply wait for the spawned process to output the string Account:. Note that (by default) globbing is used for the pattern matching, so the leading * causes any text coming before Account: to be ignored. The trailing * likewise causes any characters after Account: (like blanks) to be ignored. In writing Expect patterns its generally important not to be too specific, or your patterns will fail to match.

A more complex example:

expect "Wait for response" {} "SERVICE UNAVAILABLE" {error}
In the above, we are expecting the typical response to be the string "Wait for response", but we want to generate an error in the case of the "SERVICE UNAVAILABLE" message.

Here's a part of an Expect script that handles typical initial responses from telnet:

expect {
    {Connected*Escape character is*.} {}
    "unknown*" {
	error $expect_out(buffer)
    }
    "unreachable*" {
	error $expect_out(buffer)
    }
    eof {
        error "Connection closed."
    }
    timeout {
        error "Connection timed out."
    }
}
Note that the pattern/body pairs can be grouped together in braces to allow them to span multiple lines (much the way the Tcl switch command works). The first pattern is the one we're hoping for; if we see it, we execute the null body and proceed to the next command in the script. The next two patterns match two common telnet error messages, "unknown host" and "host unreachable"; the bodies turn them into Tcl errors. The timeout pattern is actually a special expect pattern that matches when the command times out (the global variable timeout specifies the timeout in seconds). The eof pattern is similar: it matches iff Expect gets end of file from the spawned process.

This segment of code illustrates another feature: the array variable expect_out has several fields that contain information about the matching process. expect_out(buffer) contains the matching text (and any previously unmatched text).

Each pattern can be preceded by a number of options; the -re option specifies that regular expression pattern matching be used for that pattern instead of globbing.

The timeout pattern can be used in other ways. This short, complete Expect script runs an arbitrary program with a timeout:

#!/local/bin/expect -f
# run a program for a given amount of time
# i.e. time 20 long_running_program
set timeout [lindex $argv 0]
eval spawn [lrange $argv 1 end]
expect

The log_user Command

log_user 0
log_user 1
Normally, Expect echoes the input and output of the spawned process so the user can see it. You can turn this echoing on and off with the log_user command, which takes a Boolean argument. With 0, it turns echoing off.

A Complete Expect Script

This pointless Expect script uses telnet to connect to the daytime port of the host named on the command line and prints just the time in the form 99:99:99.
#!/local/bin/expect -f
log_user 0
if {[llength $argv] != 1} {
    puts stderr "Usage: daytime host"
    exit 1
}
spawn telnet [lindex $argv 0] daytime
expect "scape character*\en"
expect "*\en"
regexp {([0-9][0-9]:[0-9][0-9]:[0-9][0-9])} $expect_out(buffer) _ time
puts $time

The send Command

send string
The send command sends string to the spawned process. The process receives it as if the user had typed it on the terminal. send has fancy options to send the string slowly (to avoid overflowing buffers on a slow remote machine) and to simulate a human typing.

The interact Command

interact string1 body1 ... stringn ?bodyn? 
The interact command returns control of the spawned process to the user. When interact is executed, input to the process comes from the user at the terminal and output from the process goes to the user at the terminal (regardless of the setting of log_user). However, Expect watches the user's keystrokes during the interaction and can arrange to execute Tcl commands when the user types certain strings.

Each string of interest is paired with a body. A null body gives the user access to the Expect interpreter. Here is an example:

set CTRLZ \e032
interact {
	 -reset $CTRLZ {exec kill -STOP 0}
	 \e001    {puts "you typed a control-A\en";
		  send "\e001"
		 }
	 $       {puts "The date is [exec date]."}
	 \e003    exit
	 foo     {puts "bar"}
	 ~~
}
The -reset option preceding the control-Z pattern tells expect to reset the terminal modes; this allows the kill command to temporarily suspend the Expect interpreter. When the user resumes Expect, the terminal modes will be reset again.

Note that the \e001 pattern not only prints a message to the user but also passes the control-A on to the spawned process.

The final ~~ pattern, has no body, so if the user types ~~ they will be given access to the Expect interpreter's command loop.


Keith Waclena
The University of Chicago Library
This page last updated: Fri Sep 2 01:12:29 CDT 1994
This page was generated from Extended HTML by xhtml.