Marc Mertens
LISP DEBUG
Chapter 1
Introduction
LISP DEBUG is a source level debugger, stepper and profiler for LISP
programs. The current implementation works with GCL 2.2 or higher, CMUCL
Lisp (version 24-04-99 or higher), ACL5 and CLISP (version 22-07-99 or
higher) and is tested on LINUX 2.0 (the current GCL version of LISP DEBUGGER
doesn't work on SUN Solaris). Although LISP DEBUG makes use of the
TCL/TK system for the GUI, no TCL/TK extensions are needed for the LISP
system (except for GCL version). LISP DEBUG has the following features
:
-
Source level debugger (highlight in a source window the code being executed).
-
Lisp form oriented not line oriented (function call is highlighted not
the line).
-
Multisource debugging is possible (the system will switch to another source
if needed).
-
Controls :
-
Step (execute the current form and stop before the next form ).
-
Step over (execute the current form and stop before the next form in the
current list).
-
Next (execute the current form and stop at the highlighted form).
-
Continue (execute the current form and don't stop anymore)
-
Breakpoints and conditional breakpoints are placed on lisp forms (not on
lines).
-
Placement of watch points on variables and lisp forms is possible.
-
Evaluation of lisp expressions is possible during debugging.
-
Time traveling (the debugger keeps track of the lisp forms executed + their
environment , this makes it possible to step backwards and forwards through
time).
-
Profiling (count how many times a lisp form is executed).
-
Halting is possible before and after executing of a form.
-
If halted after execution of a form then:
-
The result of the execution of the form is displayed
-
It is possible to change the value of the execution.
-
Extensible (you can extend the lisp control structures the debugger recognizes
by use of syntax definitions).
-
If programs loaded in the debugger encounter, during execution, a continuable
error, the lisp debugger is entered, the call causing the error is highlighted
and you can give the correct result of the call and continue executing
the program (this feature doesn't work in gcl because gcl lacks handler-case
, needed for this feature).
LISP DEBUG works by instrumentation. When a lisp source is loaded in
the debugger the following is happening :
-
A temporary file is created, this file contains the lisp source with
debug code added to it (essentially code to call the debugger, to save
position info and to save the environment of the function call).
-
The temporary file is loaded in the LISP system (you can ask to compile
it before the load).
-
The source code is loaded in a source window to be displayed during the
debug process.
-
You can now use the source window to set breakpoints , watch points ...
. When you run now code in the LISP system the execution will stop at the
breakpoints and the lisp form which should be executed next is highlighted
in the source window. From then on, you can fully control the execution
of the code.
-
If during execution of the debugged code a continuable lisp error happens,
the debugger is entered and the call causing the error is highlighted.
If needed you can give the result of the call to the faulty code and continue
running your program.
-
When you close a source, the source will be removed from the source window
and the original code (without instrumentation code) is loaded in the LISP
system.
The original source code is never touched.
1.1 Warnings
-
Adding instrumentation code has some risks associated with it.
-
Bugs in the debugger can cause malfunction of your code (I have done my
best to avoid bugs but nobody is perfect :().
-
Bugs in the debugger can make that not every part of your program is available
for debugging.
-
Bugs in the compiler/optimizer of lisp can make that there is a different
behavior of debugged programs and non debugged programs.
-
Debugged code is much slower then non debugged code.
-
If you work with recursive data which is impossible to print in LISP, the
debugger will also go in a infinite loop to try to print this data. Turning
off recursive printing in LISP will also solve the problem for the debugger
(as he uses the print functions of LISP) as long as the results are not
displayed in a separate window.
-
Macros are not easy to debug , given a macro call the system will try to
expand it and then try to add debugging code to the expanded code. System
macro's which represents a special control structure (like cond,if ...)
are handled in a special way. If you define your own control structures
you can extend the debugger to handle them in the correct way.
-
Breakpoints can be placed on every lisp form you can evaluate (with the
exception of atomic expressions and some macros (example (defgeneric ...),
(function ...)).
-
If you can't place a breakpoint on a form it is usually possible to place
a breakpoint on certain sub forms.
-
If you make changes to the source , you have to reload the source in the
debugger (you can use copy/paste to load sources in the debugger if you
use emacs or xemacs).
-
If there are bugs in your source code which causes a halt during loading
, the debugger will halt when he loads the modified source code and you
will enter the native debugger. If you exit the native debugger the rest
of your code will be loaded. Always make sure that you can load code before
you try to debug the code.
-
The debugger sits in package &pi0;&pi0;DEBUGGER'' , don't use this
package.
-
In the &pi0;&pi0;USER'' package the function name &pi0;&pi0;debug''
(&pi0;&pi0;deb'' in ACL) is exported , so you may not use this
name for a function.
-
Allegro uses the name &pi0;&pi0;debug'' so I used &pi0;&pi0;deb''
in the case of Allegro.
-
If you choose to compile the debugged code in CMUCL it can take a long
time and lots of memory because the addition of the extra debugging code.
-
If you choose to compile the debugged code in ACL you can get errors because
the not correct (in my opinion) handling of ACL of macros used in optional
parameters of functions.
Chapter 2
Overview of the debugger.
2.1 Quick procedure to get you running.
-
Make sure that X Windows is running,
-
Start LISP (standalone or as a process in emacs (xemacs).
-
Type (debug) at the lisp prompt ((deb) for Allegro). A separate debug window
should open (see Figure: Starting Debugger) . You control debugging using
this window.
Figure 2.1: Start Debugger

-
Load a lisp source in the debugger using < File > < Open > .
Figure 2.2: File Selection

-
After loading the source, you can set a breakpoint by first selecting a
lisp form and then clicking on < Breakpoint > . If a breakpoint is possible
the break pointed code will have another foreground color (default red).
Using the middle mouse button you select a whole function , double clicking
select a list.
Figure 2.3: Source Window

-
Return then to the lisp prompt in the LISP system and type an expression.
If the execution of the expression causes the break pointed code to run
, the currently executed code will be highlighted in the source window
and the debugger waits for your commands. You can now step through the
code , set watch points , evaluate expressions ... .
Figure 2.4: Executing Code

-
For more detailed information of using the debugger go to the user manual.
-
To stop debugging , use < File > < Exit > , the original source (without
debugging code) will be loaded back in the LISP system.
2.2 Using the debugger
2.2.1 Starting the debugger.
To start the debugger , first make sure that X Windows is running , then
start lisp (in a shell , in an emacs (xemacs) process or whatever editor
you want). At the lisp prompt type (debug) ((deb) if you are using Allegro).
A debug window will then start allowing you to control the debugging process.
For CLISP a special prompt is displayed (DEBUGGER::), this is because the
debugger uses in CLISP its own top loop.
2.2.2 Debugging programs.
The debug window consists of the following parts :
-
A menu line ('File','Source','Edit','Options','Tools') to Open/Close sources,
to switch between loaded sources, to paste code to the debugger, to configure
the debugger and to do profiling.
-
A button panel containing the most used functions of the debugger (Stepping
, Break pointing , Watching , Evaluating and time traveling).
-
A source pane which shows the source code, during debugging the debug points
have a red foreground color, the code to be executed is highlighted in
light blue and if you do profiling, code called the specified amount of
times is highlighted in yellow.
-
A watch pane, contains the watched variables and expressions in addition
to their values (this info can also be displayed in a separate window).
The watch pane can also display the result of an expression evaluation
or the result of a call before stepping to the next call.
-
Command pane, used to type in an expression that can be used as a watch
expression , that can be evaluated ,that can be returned as the value of
call or that can be used in a conditional breakpoint.
To debug a function you follow in general the following steps :
-
Load the source code of the function in the debugger , this can be done
in two ways:
-
The code is defined in a source file on the hard disc, you use then <
File > < Open > to load the whole source file in the debugger. This
is the preferred way of working because you can then debug all related
functions in the source file.
-
The code is defined in an editor (like emacs , xemacs ...), select then
the code, use the < Copy > function of the editor to put the code in
the copy buffer. and use then < Edit > < Paste > on the debug window
to load the code in the source window. (There is a change that this is
not working because Copy/Paste is not well defined in X Windows).
-
Once code is loaded in the debugger the debugger will be activated in the
following cases:
-
By a continuable lisp error during executing of the debugged code. The
call causing the error is then displayed and you can try to finish the
execution of the code by giving a result for the faulty call (this is not
working in gcl).
-
By executing a piece of code with a breakpoint on it (for conditional breakpoints
the condition must be full filled as well).
-
As the result of < Step > , < Step Over > and < Step Next > .
-
Because you didn't use < Continue > to stop debugging the previous lisp
expression.
-
Set a breakpoint , this can be done in two ways :
-
Select a form in the source pane and press then 'Breakpoint' to set a breakpoint.
The color of the text in the form changes then to red to indicate a breakpoint
set on this form. The color will only change if you can really set a breakpoint
. Example : in (defgeneric fac (n) (:method ((n integer)) (if (zerop n)
1 (* n (fac (1- n))))) (:method ((n string)) (concatenate n "!''))) you
can set a breakpoint on (:method ((n integer)) (if (zerop n) 1 (* n (fac
(1- n))))), (if (zerop n) 1 (* n (fac (1- n)))) , (zerop n) , (* n (fac
(1- n))) , (fac (1- n)) , (1- n) , (: method ((n string)) (concatenate
n "!'')) , (concatenate n "!'') but not on the (defgeneric ....) form.
It should be clear that a breakpoint is not set on a line but on a form
like a function call ((function arg1 .... argn) , ...) , a special function
call ((cond (( ....) ...) ...) , (defun ...) , (let ...) , (do ....) ....)
or a macro call. Breakpoints just like forms can be nested. The 'Breakpoint'
act as toggle if you select a form which already has a breakpoint selected
with it you remove the breakpoint.
-
Type in an expression in the command pane , select a form in the source
pane and press 'Break If' , this sets a conditional breakpoint , execution
is halted only if the evaluation of the condition gives true as a result.
An error during the evaluation is considered false (a dialog box explaining
the error is displayed in this case). The 'Break If' button act as a toggle,
just like the 'Breakpoint' button.
-
Set a watch point , this can be done in two ways :
-
Select a variable in a function and then press 'Watch' . In the watch pane
you will see now the variable followed by an arrow to its value. If the
variable is not defined the value will be 'Undefined'.
-
Select a form in the source pane, type a expression in the command pane
and press 'Watch exp'. This will set a watch point on the expression in
the command pane, which is evaluated when forms in the selected code are
executed (otherwise 'Undefined' is used as value). A common mistake is
to forgot to make a specific selection.
-
To unset a watch point. Press the results of the watch point in the watch
pane, a dialog box pops up to ask if you want to display the results in
a separate window or delete the watch point. If you work with separate
results windows, then deleting these windows causes the display of the
results in the watch pane.
-
You can go now to the LISP system and type in some lisp expressions. If
a breakpoint is encountered then the execution of the function will be
paused and the code to be executed will be highlighted in the source pane
of the debug window (if needed the corrected source will be loaded or a
repositioning will take place so that the executing code is always visible).
You can react now in the following ways :
-
Press the 'Step' button and execute the highlighted form, pausing at the
next form to be executed (this could be a sub form of the highlighted form).
-
Press the 'Step Over' button, execute the highlighted form, pausing at
the next form within the parent form (so we will not step through code
called during execution of the highlighted form (unless breakpoints are
placed of course)).
-
Select the code where the debugger should pause next and press then the
'Next' button. The debugger will then start executing the code until a
breakpoint is encountered or the selected code is reached.
-
Press the 'Continue' button , the debugger will then start executing the
code until a breakpoint is encountered or nothing has to be executed.
-
Press the 'Back' or 'Forward' button to do time traveling (look at the
previous/next executed code and variable context at that point).
-
If you have used Step or Step Over and all code is executed, then the system
stays in a stepped state, if you execute another debugged function from
the lisp prompted then the system pauses execution waiting on you to press
'Step' , 'Step Over' , 'Continue' or 'Eval'.
-
If you have enabled the option 'Display result last call', then the debugger
stops also just after execution of a call. If stopped after execution of
a call, the result of this call is displayed and the button 'Change Result'
is enabled. If you type in a expression in the command pane and press 'Change
Result' this expression is evaluated and returned as result of the call
instead of the original result.
-
Select a source via < Source > .
-
You stop debugging of a source by selecting the source and then < File
> < Close > . The original source code is then loaded in the LISP system.
-
Another way of stopping of debugging is the < Tools > < Interrupt
> menu, this puts the executed code in the native debugger and stops debugging
of the code with the lisp debugger. The original code is not reloaded in
the LISP system. Use this option with care (it can cause problems in certain
lisp systems).
-
You stop the debugger via < File > < Exit > . Before exiting , the
debugger will load the original sources of all debugged code back in the
LISP system.
Chapter 3
Reference Manual
3.1 Menus
3.1.1 Overview
Following menu items exist in the debugger :
-
< File > < Open >
-
< File > < Close >
-
< File > < Exit >
-
< Source >
-
< Edit > < Paste >
-
< Edit > < Find >
-
< Options > < Color Break >
-
< Options > < Color After Break >
-
< Options > < Color Breakpoint >
-
< Options > < Color Breakpoint If >
-
< Options > < Color Profiling >
-
< Options > < Font >
-
< Options > [Compile debugged code]
-
< Options > [Display result last call]
-
< Options > < Save Options >
-
< Options > [Save on exit]
-
< Tools > < Start profiling >
-
< Tools > < Stop profiling >
-
< Tools > < Interrupt Program >
3.1.2 Detailed description of menu items
< File > < Open >
Opens a window which allows you to select a LISP source. When pressing
Ok , the selected source is processed to add debugging code and to save
the modified code in a temporary file. If the option < Options > [Compiled
debugged code] is true then the temporary file will be compiled and the
compiled code will be loaded in the lisp system, in the other case the
temporary file will be loaded in the lisp system.
Figure 3.1: < File > < Open >

< File > < Close >
The original code of the file in the source window is loaded in the lisp
system. The source window is then emptied and filled with the first loaded
source (if it exist).
< File > < Exit >
Exit the debugger. Before this happens the original version of all debugged
sources is loaded in the lisp system.
< Source >
Allows you to select which of the sources loaded in the debugger is shown
in the source window. This menu is detachable.
< Edit > < Paste >
If in emacs (xemacs) (or maybe your editor of choice) a selected area is
copied to the copy buffer, then this selected area is used to create a
temporary source which is loaded in the debugger. This allows you to debug
functions which are in a editor but not yet in a source file. (Every copy/paste
operation supported by tk_textPaste should work).
< Edit > < Find >
Activate the search function, to search the source currently displayed
in the source window. Three types of search are possible :
-
Case , case sensitive search.
-
No Case , case insensitive search.
-
Regular expression , allows you to use regular expressions in the search.
Figure 3.2: Find

< Options > < Color Break > , < Options > < Color After Break
> , < Options > < Color Breakpoint > , < Options > < Color
Breakpoint If > , < Options > < Color Profiling >
Allows you to change the color of the current executed code, the code to
be executed next, the breakpoints , the conditional breakpoints and the
profiled code.
Figure 3.3: Color

< Options > < Font >
Use this menu to change the font used in the source, watch, command pane
and separate result windows.You better avoid a proportional font, indention
is shown much better if you use a fixed font (the default font is 'fixed').
Figure 3.4: Font

< Options > [Compile debugged code]
When this switch is enabled, the source with debug code added is first
compiled before it is loaded in the lisp system. This could be usefully
on lisp systems which do a pre compile of a function before executing it
for the first time (because of the extra debugging code , this pre compile
could take a long time).
< Options > [Display result last call]
If enabled the debugger will not only stop just before a call is executed
but also just after a call is executed. Different colors are used to highlight
the executed (to be executed) call. If stopped just after execution of
a call, it is possible to change the result of this call with the 'Change
Result' button.
< Options > [Enter debugger in case of error]
If enabled, then during execution of debugged code when there is a LISP
error, the error is displayed in a dialog box and the debugger is entered
(even if there where no breakpoints). The lisp expression causing the error
is highlighted and you can supply a result for this lisp expression.
NOTE: This feature doesn't work for GCL , even if enabled (this is because
handler-case is missing in GCL).
< Options > < Save Options >
Save the options in < Options > in the file $HOME/.lispdebug.lisp. During
startup of the debugger, this file is always executed. This file is a standard
text file and contains the following settings :
-
(DEBUGGER::setting "COMPILE_CODE" 0)
-
(DEBUGGER::setting "SAVE_ON_EXIT" 0)
-
(DEBUGGER::setting "DISPLAY_RESULT" 0)
-
(DEBUGGER::setting "DEBUGPOINT_COLOR" "#ff0000")
-
(DEBUGGER::setting "DEBUGPOINTIF_COLOR" "#808000")
-
(DEBUGGER::setting "CURRENT_COLOR" "#00ffff")
-
(DEBUGGER::setting "AFTER_COLOR" "#008080")
-
(DEBUGGER::setting "PROFILE_COLOR" "#ffff00")
-
(DEBUGGER::setting "FONT" "-*-helvetica-medium-r-normal-*-120-*-*-*-*-*-*")
The explanation of these codes is as follows :
-
DEBUGGER::setting is a LISP function which when called change the configuration
of the debugger in a way defined by the first string (=parameter), the
last arg is the value of the parameter.
-
COMPILE_CODE if equal to 1, enable < Options > [Compile debugged code],
otherwise disable this option.
-
SAVE_ON_EXIT if equal to 1, enable < Options > [Save on Exit], otherwise
disable this option.
-
DISPLAY_RESULT if equal to 1, enable < Options > [Display result last
call], otherwise disable this option.
-
DEBUGPOINT_COLOR is in rgb values the foreground color of a breakpoint.
-
DEBUGPOINTIF_COLOR is in rgb values the background color of a breakpoint.
-
CURRENT_COLOR is the color used to highlight a call just before it gets
executed.
-
AFTER_COLOR is the color used to highlight a call just after it gets executed.
-
PROFILE_COLOR is the color used to highlight code fullfilling the profile
count.
-
FONT is the font used in the debugger.
If you modify or create this file manual (for example because the default
color or font is not supported on your system) be sure to not forget DEBUGGER
(in case of Allegro it must be in uppercase to).
< Options > [Save on Exit]
When enabled, the current options are saved in $HOME/.lispdebug.lisp when
the debuggers stops.
< Tools > < Start Profiling >
Enables the profiling part of the debugger. The profile counters (how many
time a piece of code is executed) are reset to zero and an extra slider
is made visible in the left part of the debug window. When code is executed
the system keep track of the number of times a lisp form is executed. By
using the slider you highlight the code which is at least as many times
executed as the number shown by the slider.
Figure 3.5: Debugger when profiling

< Tools > < Stop Profiling >
Disables the profiling part of the debugger.
< Tools > < Interrupt Program >
Executes an (break ..) command in the lisp system and halts execution of
the code (by going in the native debugger). Be careful with CMUCL I noticed
that to much breaks can break their native debugger, ACL also gave some
problems during interruption.
3.2 Source Pane
3.2.1 Content.
The source pane contains the source of the code which is debugged. The
source which is displayed depends on the following conditions:
-
If you open a new source , then this is the source displayed.
-
You can change the source displayed via the < Source > menu.
-
If you close a source , the first opened source is displayed.
-
If the system encounters a breakpoint , the source containing the breakpoint
is displayed, the window on the source is also repositioned so that the
executing code is visible.
3.2.2 Selection
Selection is the base for setting breakpoints, setting conditional breakpoints
and setting watch expressions. Selection can be done in the following ways.
-
Dragging. Click with the mouse in the source pane (use the left button),
drag the mouse to an other point and release the button.
-
Click somewhere in a function definition with the middle mouse button.
This selects the whole function.
-
Double click with the left button , this selects a whole list.
3.2.3 Highlighting.
Pieces of the sources are highlighted as follows (we use the default colors
of the debugger).
-
Red foreground , this is a breakpoint.
-
Dark green foreground , this is a conditional breakpoint.
-
Light blue background, this code will be executed next.
-
Dark blue background, this is just executed code.
-
Yellow background , this code fulfills the profile count (is at least as
many times executed).
3.3 Watch Pane
3.3.1 Content.
The result pane contains the value of watch variables, the results of evaluating
watch expressions, the result of a evaluation, or the result of previous
executed code. The format used is as follows 'exp - > value' . Here 'exp'
could be a variable , a watched expression, a evaluated expression or &pi0;&pi0;RESULT'
(the result of previous executed code). 'value' could be 'Undefined' (the
variable is not defined or the expression couldn't be evaluated) , (values
.....) the result was caused by a (values ...) statement or just the value(result)
of the variable (expression). If you press on a 'exp - > value' line, a
dialog box pop ups offering you the choice of stopping the watch, displaying
the result in a separate window or just returning.
3.4 Command Pane
3.4.1 Content.
The command pane is the only pane which can be edited. You use it to type
in expressions needed by the debugger. This will be the case in the following
situations:
-
Conditional breakpoints (NIL or non NIL of the evaluated expression determines
if the system stops or not).
-
Watch Expression (the expression to watch).
-
Eval (the expression to evaluate).
-
You want to change the result of a call (with the 'Change Result' Button).
3.5 Buttons
3.5.1 Overview.
The following buttons are available:
-
< Step >
-
< Step Over >
-
< Next >
-
< Continue >
-
< Breakpoint >
-
< Break If >
-
< Watch >
-
< Watch Exp >
-
< Change Result >
-
< Eval >
-
< Back >
-
< Forward >
3.5.2 Detailed description of the buttons.
< Step >
When a program is halted (before or after execution of the highlighted
call), < Step > allows you to proceed to the next call (or do the highlighted
call) and wait then on user input (before the next call or after the current
call).
< Step Over >
When the program has halted, press < Step Over > to execute the highlighted
code and advance execution. The debugger will halt the program in the following
cases (whichever first occurs).
-
A breakpoint is encountered.
-
A lisp form in the same enclosing list as the highlighted form must be
executed.
< Next >
When the program has halted, use < Next > to advance execution. To use
< Next > first select another lisp form and press then < Next > the
execution with then proceed until one of the following two cases are encountered
(whichever first occurs).
-
A breakpoint is encountered.
-
The highlighted code must be executed.
< Continue >
Pressing < Continue > will advance a halted program , the program will
either fully execute or will halt at the next breakpoint.
< Breakpoint >
Use this to set a breakpoint. To set a breakpoint just highlight a lisp
form and then press < Breakpoint > . If during executing of code the
highlighted code must be executed then the debugger halts execution just
before this code gets executed. Use < Step > , < Step Over > , <
Next > or < Continue > to continue execution. < Breakpoint > act
as a switch, so if you try to set a breakpoint on a existing breakpoint
you are actually clearing the breakpoint.
< Break If >
Use this to set a conditional breakpoint. To set a conditional breakpoint,
highlight a lisp form, type a condition in the command window and press
< Break If > . If during execution of code the highlighted code must
be executed then the debugger evaluates the condition and if the result
is non nil execution will halt. < Break If > act as a switch, so if
you try to set a conditional breakpoint on a existing conditional breakpoint
you are actually clearing the breakpoint.
Errors during evaluating the condition counts as a NIL and a message
detailing the error is displayed. Also the condition is only evaluated
in the highlighted code.
< Watch >
Sets a watch point on a variable. To do this select the variable and then
press < Watch > , the variable and its value is then visible in the
result pane.
< Watch Exp >
Sets a watch point on a expression. First select a area where the watch
point is valid, then type in the expression in the command pane and press
< Watch Exp > . When execution happens in the selected area the exp
is evaluated and the result is displayed in the result pane. If the evaluation
of the expression causes an error this is displayed as an error message
and the result is undefined.
< Change Result >
This button is only active if the debugger is entered just after evaluation
of a call (which is highlighted), this can happen because of a lisp continuable
error in the debugged code or because the option [Display result last call]
is enabled. If active you can type in the command pane a expression and
press < Change Result > , the expression will then be evaluated and
the result is used as the result of the displayed call. Hint: use (values
....) if you want to return more then one value.
< Eval >
Evaluates a expression in the debugging context. If during debugging (when
execution has halted and you have control) you feel the need to evaluate
a expression in the context of the highlighted code do the following. Type
in an expression in the command pane and press < Eval > , the result
will then be visible in the result pane. If the evaluation of the expression
causes an error this is displayed as an error message and the result is
undefined. Warning: evaluated in the context doesn't mean that the expression
can change the context. (for example you can not change lexical variables
but you can use their values). Be also careful during time traveling, global
values are not restored during time traveling.
< Back >
Activates time traveling , the debugger keeps track of the 100 lasts forms
executed together with their context. By pressing < Back > you go to
the previous executed form and change the context to the context at that
time. Has the same functionality as the frame concepts in the system debuggers.
< Forward >
Activates time traveling , the debugger keeps track of the 100 lasts forms
executed together with their context. By pressing < Forward > you go
to the next executed form and change the context to the context at that
time.
Chapter 4
Extending the debugger
4.1 Introduction.
LISP is a very extensible language in the way that you can use macros to
define your own control structures. This debugger tries to behave logical
for the control structures as defined in ANSI LISP but doesn't know how
to handle control structures you define yourself. To solve this, you can
modify the way how this debugger adds instrumentation code to the source
(so that your extensions are covered correctly). The way I have made this
possible is to provide you with a special language which allows you to
express the syntax of lisp forms with a little bit of semantics as well.
In fact the core of this debugger is generated automatically by a source
of syntax diagrams defining the syntax of most special lisp forms in ANSI
LISP.
If you want to modify the debugger keep in mind the following points
:
-
Function calls are handled correctly by the debugger, you don't have to
do anything at all for this.
-
Macro's are the problem . The standard lisp macros are covered in the debugger
and he tries to make the best of the ones you define (first he expands
the macro call and then tries to add debugging code to the expanded code)
but the result is not always what you expect. It is here that you can modify
the behavior of the debugger.
-
Modifying the debugger can stops it working, so be careful. Nothing can
go wrong if you make sure that you have made backup copies of the following
files:
-
lispsyntax (this is the source file defining the working of the current
debugger).
-
debugcode.o,debugcode.x86f , debugcode.fas or debugcode.fasl (this is the
core of the debugger for GCL,CMUCL,CLISP or ACL5, loaded when you start
(debug) or (deb)).
4.2 Modifying the debugger
The steps to modify the debugger are as follows:
-
Create a source file defining how the lisp debugger should transform sources.
You can take the file delivered with the debugger as a base and extend
it. This file is '/usr/local/lib/lispdebug/lispsyntax' (if you have done
a default installation of the debugger). Remember, this file will redefine
the way how the debugger transform sources to debugged sources, if you
make errors or forget things the debugger can act strangely. The syntax
and semantics of the language to use is described in the next chapter.
-
Use the lisp function DEBUGGER::process-definition-file to compile this
file to lisp code containing the parse and convertion code of the debugger.
The syntax to use is either :
-
(DEBUGGER::process-definition-file < Source-file > ) , the source file
is compiled to a lispfile (debugcode.lisp) which is then compiled by the
lisp system and then loaded in the lisp system modifying the behavior of
the debugger.
-
(DEBUGGER::process-definition-file < Source-file > < Out > ) , the
source file is compiled to a lispfile (with filename < out > .lisp)
which is then compiled by the lispsytem and then loaded in the lisp system
modifying the debugger.
-
Once 2 is finished you can test the debugger to see if he handles the new
definitions well (to test the normal lisp constructs look at the testgcl.lisp,testcmucl.lisp,testclisp.lisp
and testacl5.lisp files). If you are convinced that the debugger is working
correctly you can make your modifications permanent by copying the object
file generated in (2) to /usr/local/lib/lispdebug or to the value of the
LISPDEBUG environment variable. (If you use 2.a. the object files are either
debugcode.o,debugcode.x86f,debugcode.fas or debugcode.fasl for gcl,cmucl,clisp
or acl).
4.3
Extension Language Of the Debugger.
The extension language of the debugger is strongly based on the syntax
descriptions used in ANSI LISP, so that writing extensions is as simple
as writing a syntax diagram. Be careful however, although the language
looks simple their are some know cavecats which should be dealed with.
4.3.1 The default file used by the debugger.
To give you an idea of how the extension language looks like, look at '/usr/local/lib/lispdebugger/lispsyntax'
the source used to generate the parser/transformer of the debugger delivered
with this package. Although the language used is not yet defined it should
look familiar to you.
4.3.2 Syntax of the language.
The language is composed of the following elements :
[comments]
Everything on a line after a ';' is considered a comment and is neglected.
[white spaces]
Used as separators , the following is a white space :
-
blank
-
new line
-
tab
-
return
[definitions]
These are of the form :
[expressions]
These are of the form :
-
symbol
-
_symbol
-
~symbol
-
#symbol
-
@
-
[ expression ... expression ]
-
[ expression ... expression ]*
-
{expression ... expression}
-
{expression ...|....|...|
expression... }*
-
( expression expresion ...)
-
"text''
[symbols]
Any string of characters with the exception of white spaces , @ , _ ,~,#,(,),[,]
A source in our language is a text file containing definitions and
expressions.
4.3.3 Semantics of the language.
The syntax tells us what the wellformed expressions and definitions are
in our language but it says nothing about their meaning, for this we need
a little bit of semantics. The best way to understand the semantics of
the language is the consumer/producent metaphor. When a expression in our
language is applied on a lisp expression two things can happens :
-
The expression recognizes the lisp expression consuming part of it and
producing another lisp expression.
-
The expression does not recognize the expression an it generates a throw,
no lisp expression is produced.
Lets now put these ideas in practice on the different type of expressions
of our language. Let P be the list produced, E a expression in our language
and L the lisp expression on which we applies expressions in our language.
The parser/generator in the debugger will apply each expression on a given
lisp expression until it gets not a throw and the lisp expression is fully
consumed, the produced list is then the lisp expression with debug code
added. If this sounds inefficient you are right this is just a semantic
explanation , our language is actually compiled to become the parser/generator
of the debugger which has the same effect as our semantic explanation,
but he does it in a more efficient way.
Symbol.
If we apply a 'symbol' on a lisp expression L=(e1 e2... en) or L=() we
have a throw if e1 is not equal to our 'symbol' or if L is the empty list.
If e1 is equal to our 'symbol' then we append P with the symbol and L becomes
(e2 ... en).
Let P=() , L=(defun f (n) (princ n)) and E=defun then applying E on
L gives L=(f (n) (princ n)) and P becomes (defun).
Let P=(), L=(defun f(n) (princ n)) and E=let then applying E on L gives
a throw.
"text''
If we apply "...'' on a lisp expression L=(e1 e2 ... en) or L=() we have
a throw if L is empty or if e1 is not a string. In all other cases L becomes
(e2 ... en) and the first element e1 is added to P. You can think of "text''
as standing for any string.
_symbol
If we apply '_symbol' on a lisp expression L=(e1 e2.... en) or L=() we
have a throw if L is empty. In all other cases L becomes (e2 ... en) and
the first element e1 is added to P. You can think of _symbol as standing
for any list element which must not be changed.
Let P=() , L=(a b c d) and E=_sym then applying E on L gives L=(b c
d) , P=(a).
Let P=(),L=((a b) c d) and E=_sym then applying E on L gives L=(c d)
, P=((a b))
Let P=(),L=() and E=_sym then applying E on L gives a throw.
~symbol
If we apply '~symbol' on a lisp expression L=(e1 e2 .. en) or L=() we have
a throw if L is empty or e1 is a list. In all other cases L becomes (e2
... en) and the first element e1 is added to P. Also e1 is added to the
lexical environment of the debugger. During executing of the debugged code
the system test if the variable is defined in the lexical environment at
the execution point and if it is so the binding of the variable is saved.
This means that you can refer to the binding of this variable during debugging.
You don't have to worry when this variable can be referred (the system
keeps track of this) only indicate that this is defined to become a variable.
Let P=() , L=(a b c d) and E=^sym then applying E on L gives L=(b c
d) , P=(a).
Let P=(),L=((a b) c d) and E=^sym then applying E on L gives a throw.
Let P=(),L=() and E=^sym then applying E on L gives a throw.
#symbol
If we apply '#symbol' on a lisp expression L=(e1 e2 ... en) or L=() we
have a throw if L is empty. In all other cases L becomes (e2 .... en) and
the system tries to add debugging code to e1 before it is added to P. The
debugging code added makes that you can place a breakpoint on e1, and see
where e1 is located in the source. In some cases no debugging code is added
(if e1 is not a list or if e1 represents a macro not recognized by the
debugger). You can use # to indicate that you should be able to set breakpoints.
Let P=() , L=(a b c d) and E=^sym then applying E on L gives L=(b c
d) , P=((add-debug-code a)).
Let P=(),L=() and E=^sym then applying E on L gives a throw.
@symbol
We can always apply @ to L . It consumes nothing and it produces nothing
, its solely purpose is for its side effect, it allows you to set a breakpoint
on the whole expression where it is part of. Use this in (defun ...) (let
...) just before the body of these functions.
[expression1 ... expressionn]
If we apply [expression1 ... expressionn] on a lisp expression L . It will
first do nothing , the expressions after [...] are first applied on L if
this gives a throw the system will first trying to apply expression1 then
expression2 ... on L followed by the expressions after [...]. Consider
this as a kind of optional syntax.
E=[a b] c , L=(a b c d) and P=() then applying E on L gives L=(d) ,
P=(a b c)
E=[a b] c , L=(c d) and P=() then applying E on L gives L=(d) , P=(c)
E=[a b] d , L= (c d) and P=() then applying E on L gives a throw
{e11...e1n|...| em1....emk}
If we apply {e11...e1n|...|
em1....emk} on a lisp expression L . It will first try to apply e11...e1n
on L followed by all expressions after { ... } if this causes a throw,
it will try e21...e2l followed by the rest ... . You can consider { ...
} as a kind of or where you want to try different alternatives.
Example 1 E={a b} c , L=(a c d) and P=() then applying E on
L gives L=(d),P=(a c)
E={a b} c, L=(b c d) and P=() then applying E on L gives L=(d),P=(a
b)
E={a b} c, L=(c d) and P=() then applying E on L gives a throw
[expression1 | ... |
expressionn]* , {expression1 ... expressionn}*
If we apply {...}* or [...]* on a lisp expression. We first try to apply
the expressions after { ...}* or [...]* , if this causes a throw we try
first {...} or [...] followed by the rest, if this fails then we try {...}{...}
or [...][...] and so on. To avoid infinite looping we stop if one {..}
or [...] gives a throw. Use this if you have more then occurrence of the
same elements.
E=[a b]* c, L=(a b a b c) and P=() then applying E on L gives L=(c)
, P=(a b a b)
(expression1 ... expressionn)
If we apply (expression1 ... expressionn) on L=(e1 ... en) or L=() we get
a throw if L is empty or if e1 is not a list . The result in the other
cases is the result of applying expression1 .... expressionn on e1 where
we start with a empty result list, this result list is then added to the
original list.If e1 is not consumed fully we have also a throw.
Example 2 E=(a b c) , L=((a b c) d) , P=() then applying E
on L gives L=(d) , P=((a b c))
E=(a b c),L=((a b c d) d), P=() then applying E on L gives a throw.
definitions (name = expression)
A definition gives a name to a expression , applying this name has the
same effect as applying the expression. You can use definitions to to define
a expression ones and then use it many times. Definitions can be used recursively
, one can refer in a definition to itself.
4.3.4 Cavecats.
-
When using recursive definitions make sure that you do not introduce infinite
looping.
-
Be careful when using #,_,or ~and [],{}* and []* , the effect is not always
what you want. Look at the following expression
-
["string'' | (declare [_declaration])}] {#exp}*
-
This will when applied to ("description'' (declare (string a)) (princ x))
apply with success {#exp}* to "Description'' and (declare ..) and add debug
code to "Description'' and (declare ...) which gives in the case of (declare
..) a syntax error in lisp. The way out of this problem is using some extra
{}, our code becomes then something like :
-
{"string'' {#exp}* | "string'' (declare [_declaration])
{#exp}* | (declare [_declaration]) {#exp}* |
(declare [_declaration]) "string'' {#exp}* |
{#exp}*}
Chapter 5
Porting of the debugger
The debugger is coded in such a way that it should be easy to port it to
another LISP implementation or even another Unix. This chapter describes
how such a port can be done.
5.0.5 Conditions.
The port will be the easiest if certain conditions are fullfilled:
-
The lisp adheres to ANSI LISP.
-
TCL/TK version 8.x (previous versions could work also but I have not tested
them) is installed in the OS together with its libraries. In contrast to
the previous version of the debugger , LISP must not support TCL/TK (except
for GCL).
-
The OS and LISP supports sockets.
If (1) is not full filled then the best strategy to follow is to write
the missing ANSI functions used in the code of the debugger. In case of
failing of TCL/TK the whole interface must be rewritten using another graphical
library. If (3) is not fullfilled you should either extend LISP with some
C coding or choose another connection method to the interface program.
To help porting I will explain how the lisp system talks with the interface.
5.0.6 Overview of the debugger.
To easy porting, the debugger is splitted in two components.
-
The user interface , this is the GUI of the debugger and it is written
in C using the TCL/TK system as a graphical library.
-
A extension of lisp containing the code to parse and change sources, generate
a new parser/generator and code called by the debugged functions.
The interaction between the interface and the lisp extensions is done using
sockets, because this is in my opinion the most supported IPC in LISP and
the OS.
The GUI interface.
The C code is using the following C source files :
-
interface.c , this contain the main body of the interface.
-
hash.c , hash.h , this contains some extensions for hashing and list manipulation
used in interface.c.
-
tclinvoke.c , code to call TCL/TK in a more efficient way.
These three file must be compiled and linked together with the libraries
of TCL/TK to produce the executable 'interface'. This program can be started
in three ways, modifying the way how it talks to the lisp system. For socket
communication, the interface is always the server and it is listening for
connections of the lisp system to it. The program 'interface' is started
via the lisp system, who decides how to start the interface. There are
current three ways of doing this.
-
'interface', the communication takes place via a Unix socket with address
'/tmp/lispdebugger' (this is the method used by CMUCL).
-
'interface pid' , pid is the process-id of the parent of this progam (the
lisp system). This pid is then used by 'interface' to send a signal SIGUSR1(=10)
to the lisp system each time it has output for the lisp system and thus
let the lisp system act on the input. The communication still takes place
via Unix sockets with address '/tmp/lispdebugger' (this is the method used
for GCL).
-
'interface -port' , port is the port number to use , communication takes
place via stream oriented tcp/INET sockets on the port indicated (this
is used ACL and CLISP).
The communication from 'interface' to lisp is done by sending a lisp command
as a string followed by a return to the lisp system (in case of 2 after
the send a signal is send to wakeup the lisp, in all other cases the lisp
system wakes up if there is input on their socket connection). After sending
a command, the 'interface' will not wait for a return. The functions called
from the 'interface' to the lisp system are:
-
(debug-open-file < filename > < compile-t-nil > ) to add debug code
to the program (if needed compile it) and load the changed code in the
lisp system.
-
(debug-delete-watchpoint < tag > ) delete the watch point identified
by < tag >
-
(debug-separate-watchpoint < tag > ) display the watch point identified
by < tag > in a separate window
-
(debug-set-watchpoint < tag > < begin > < end > < source >
< exp > ) sets a watch point identified by < tag > , < begin >
, < end > , < source > indentifies the place of the watch point in
the source and < exp > is the exp to watch.
-
(display-result-exp < exp > ) evaluates < exp > and displays the
result
-
(display-watchpoints) display all current watch points
-
(if-breakpoint < exp > ) checks if we must stop in a watch point because
of a conditional breakpoint
-
(end-debug-eventloop) to end the eventloop at a debugpoint
-
(return-result-exp < exp > ) returns < exp > as the result of a evaluating
a lisp form
-
(step-back-in-time) causes the debugger to step back in time
-
(step-forward-in-time) causes the debugger to step forwards in time
-
(stop-interface) stops and cleans up the lisp system socket part of the
interface.
Communication from the lisp system to the 'interface' takes place in the
following way :
-
First a number followed by a blank indicating the function to call in the
'interface' is send, this number implicit says also how to handle the arguments.
The following functions must be implemented in the interface (function
together with function-code and description). Positions are expressed in
number of bytes from the beginning of the source.
-
0=hightlight-source , arguments begin(integer),end(integer),type(string),color(string)
, should highlight the source text in the indicated color from position=begin
to position=end. type is used to make a distinction between selection.
-
1=display-message, arguments message(string) , displays the message string.
-
2=highlight-error, arguments begin(integer) highlights the line containing
the position begin.
-
3=set-possible-breakpoints , arguments source(string),begin(integer),end(integer)
gives the debugger information about the begin and endpoints of breakpoints
in the source.
-
4=give-control-to-interface , arguments source(string),begin(integer),end(integer)
called from the debugged code to allow the debugger to intervine. In the
'interface' give-control-interface blocks until the user steps or press
continue ... .
-
5-display-result, arguments nr-args(integer) = no of args to follow, the
args next comes always in groups of five args: type(integer)(0=variable,1=exp,10=variable
in separate window,11 exp in separate window),tag(string),exp(string),value(string).
-
6=display-exp-in-interface, arguments exp(string),result(string) used to
display the result of evaluating a expression.
-
7=if-breakpoint, arguments none , check for conditional breakpoints.
-
8=display-time-env, arguments begin(integer),end(integer),source(string)
equivalent as give-control-to-interface but used in time traveling.
-
9=setting, arguments var(string),value(string) used to set the options
settings in the debugger.
-
10=give-control-to-interface-after, arguments source(string),begin(integer),end(integer)
called from the debugged code to allow the debugger to intervine. In the
'interface' give-control-interface blocks until the user steps or press
continue ... .
-
11=lisp-goes-to-debug called if evalling debugged code in lisp gives continuable
error, the interface will then display this error and highlight the problem
code.
-
The arguments of the functions are send as follows :
-
integers, in printed format followed by a blank character so they can be
read with scanf.
-
strings, first the length of the string is send, then a blank character
followed by the string followed by a blank character.
The lisp code.
The lisp code is splitted up in two parts :
-
The main body of the debugger (debugger.lisp).
-
The interface to the 'GUI interface' (cmucl.lisp,gcl.lisp,acl5.lisp,clisp.lisp).
If you want to port the debugger to another lisp, you can leave debugger.lisp
and write a new < lisp name > .lisp file to interface with the 'GUI
interface' If you use the current 'GUI interface' the following functions
should be implemented in this file:
-
(start-interface) , to start 'interface' and make a socket connection.
-
(stop-interface) , to clean up everything after 'interface' is stopped.
-
(send-command command &arg-list) , to send the command function code
followed by the arguments to the interface.
-
(process-incoming) see if there is input waiting on the socket read this
in and evaluate it.
-
A way of detecting input on the socket and then call process-incoming.
For examples I refer to cmucl.lisp,gcl.lisp,acl5.lisp and clisp.lisp.
File translated from TEX by TTH,
version 2.53.
On 23 Oct 1999, 13:04.