Document Home

Previous: Related Stuff You may Need

Debugging Perl


A couple of weeks ago, as I was tussling with some element of the execution of recs_insert.pl and having some difficulty tracking down the error compounded with a balky debugger (more on that later), I thought to myself, "I should write a little something on debugging". Actually, writing something about debugging was not at all what I wanted to be thinking about then, and what I actually said was probably something more like "@!##*&@@!", which is the way comic books used to say "expletive deleted". But in the back of my head I knew that I should probably do something like that, so here we are.


Before you can start debugging any perl program, you have to make sure that it is a valid perl program. In some environments, chunks of code with syntax problems can be read into whatever tool is being used to debug, and the problems identified from there. That is not the case with perl, at least not with the tools I'll be discussing here. The first step you should take with any perl program is to run it through a syntax check. If this is the first time you've tried to execute that specific bit I can almost guarantee to you that there will be something that will bomb out the syntax check, ranging from forgetting to "use" a module from which you are using methods, to mis-spelling scalar desclarations, or to leaving the semi-colon off the end of a line. Or putting one on a line where it shouldn't be. This is particularly appropriate for cgi scripts, because while trying to execute a script with syntax problems from the command line will bomb out with at least some kind of error message, a cgi script called from a browser will likely return nothing more than a cryptic message that will tell you little about the nature of the problem. (While there are some tools that will return fatal error messages to the browser, you should generally avoid their use if you have an alternative. Far too often, the code that enables those messages get left in code when it goes live on the internet, and when there is a glitch and the nature of the problem is printed to the browser a substantial amount of information is conveyed to those who might be interested in compromising the security of the site. I'll get into trapping error messages appropriately soon.)


To check the syntax of your scripts, type "perl -c scriptname.pl" . It is also worthwhile to add the -w switch ("perl -cw ..."), to turn on warning mode. If you were wise enough to "use strict" in the body of the code, the syntax check will also specify inappropriate scalar scoping, which goes a little beyond the normal syntactical problems that are identified. Once your script will pass through the syntax check with a "syntax ok" message, you will have a legal perl script from the standpoint that all of the statements are legal perl statements. That doesn't mean that the manner in which any given statement is expressed will return the result that you are looking for, that the functions that are used in the code are the right ones for the job, or that the overall structure of the script does anything like what you think it supposed to do. It just means that all of the perl i's are dotted and t's are crossed. (On the other hand, this is not such a minor thing, either.)


The debugger, on the other hand, is the tool that you use to track the execution of the script itself. From the time I started working with perl in the linux environment, I have used the ptkdb (read perl tk debugger) debugger, largely because it was very similar to the debugger included with ActiveState Corporation's Perl Development Kit for the Win32 world. (In the last couple of years ActiveState has released new versions of the Perl Development Kit and also rolled it into its Komodo integrated development environment, which can be run on both Win32 and linux machines. These are commercial products, and the cost varies dependent on the specific package you purchase. However, if you are interested in using perl in the Windows world, and there are many reasons for doing so, I can attest that the PDK is a worthwhile investment if you can afford the $100-$200 USD price. If the Komodo environment exhibits similar quality, and I have no reason to assume that it would not, I would expect it to also be a worthwhile investment. This is not to say that there are not other products similarly worthwhile, only that I've found ActiveState's products good and found them responsive as a corporation. Assign to that opinion whatever weight you wish.)


In any event, you can install the ptkdb package easily, either from within the CPAN module or manually. To install the tool with the CPAN module, type "perl -MCPAN -e shell", and you'll be dropped into the interactive module management shell. I've already discussed this, so at this point I'm going to assume that if you are interested in this mode of installation you have already configured the module. If, when you did, you specified that identified prerequisites should be "followed", and you have to do is type "install Devel::ptkdb" and the full set of packages for the environment will be installed. (Note that package names must be specified in a case-sensitive manner, so you should verify the manner in which any package is spelled.) If you configured the CPAN module to "ask", you will be asked if you wish to install each of the requisite packages, which can prove tedious but gets the job done. If you typed "ignore", that was a mistake, you'll have to install the supporting packages one by one. (There is a rationale for this option, but it is not likely to be one that you would be worried about, being largely concerned with repairing broken modules without going to the trouble of re-installing those still functioning.) In this particular instance, that is not an onerous exercise, as I recall there is really only one module on which ptkdb is dependent, the Tk module, so if you are are installing in this fashion simply type "install Tk". If the web server / development machine is a machine approximately of the horsepower of Ralphzilla, you might as well do something else for a few hours. The Tk module is a massive one, and the required C libraries for its operation take a considerable length of time to install.


If you are installing modules manually, you should download the gzipped Tk and Devel-ptkdb packages from CPAN and extract them. Entering first the directory housing the Tk module, type "perl Makefile.PL" (case-sensitive), and a script will be executed that examines the configuration of the machine and writes a file that specifies how the compilation process should proceed on your machine. If the target machine is missing some required library, this script should let you know about it. The configuration script for some modules ask questions as they are run, but I don't believe that to be the case for either of these modules. Next type "make" and the actual compilation process will begin. (It is this process that can take several hours on a machine of modest horsepower.) After this finishes, the command "make test" will perform whatever tests exist to verify the module's functionality. If you are running the tests in an environment like mine and displaying the output from the linux box on a windows machine you will have to have some sort of Xwindows capability active on your desktop when the tests are run or they will fail. But then you would have to have that capability for the ptkdb debugger to work in the first place, so if you have not installed some kind of remote Xwindows capability and you want to use this tool you had best review the section on remote Xwindows display and get your rear in gear.<grin>


When you run the ptkdb debugger, as long as the subject script can compile, you should see a screen something like that below.



Note that if your script will not pass the syntax check, the debugger wil not even start, and the error messages you see are not likely to be excessively helpful. Make sure that you take the time to syntax check your scripts whenever you have made any changes. That small investment of time will save you substantially larger chunks of time.


Regardless, as you can see in the graphic you can type expressions in the box headed "Enter Expr:" in the right-hand side pane and the expression will be included in the window immediately below that entry box. As you can see on the left, the graphic reflects a situation in which the script has just loaded, and execution is paused at the first statement. From here you can [Step In] through the script, successively executing each line; [Step Over] to execute the line without displaying whatever actions are attendant to the line; [Return] to a calling program if that is relevant; [Run] to simply execute the script; [Run To] to run to a designated line; or [Break] to suspend the script's execution.


Stepping through a script with the [Step In] button can quickly become very tedious. Stepping into the line "my $forms=new CGI;", for example, will lead down through the code used by the CGI module for the instantiation of a new CGI object. For the most part, that's probably not what you want to do, although as your perl skills grow and you author your own modules or customize those created by others this may be precisely what you want to do. If you double-click on a line further down in the script, however, it will turn red, which signifies that it has been set as a breakpoint. You can then click [Run To] and the script will execute to that point and pause. You can then click [Run To] and the script will execute to that point and pause. Whatever expressions you have entered into the right-hand window will then display the values they hold at that point in the execution of the script. Note also, in the code window, that you can hover the mouse cursor over an expression and the expression will be evaluated in light of the state of the execution of the script at the statement that is highlighted in green. The breakpoint will not be cleared until the red highlighting is cleared by clicking on the statement. (Sometimes, clicking off the red highlighting will not toggle breakpoint status. Just be aware of that ... it's not a big deal.) Setting a couple of breakpoints within a loop and highlighting expressions at each pause can be a very effective manner of acquiring a very visceral sense of exactly what is going on in that loop.


The ptkdb debugger is a very handy way to debug cgi scripts. All you have to do to implement it is to set the display environment variable in a begin block in the script, as is illustrated in virtually every script I've included as a sample, and some sort of remote Xwindows display capability on the machine to which output is being directed. To actually initialize the debugger, modify the shebang line to read as follows:

"#!/usr/bin/perl -wTd:ptkdb"

Now when the script is called from the browser, in just the manner that it has been called all along, the debugger will be started as well. In fact, it will be started with each invocation of the script, a very good demonstration of what the web server is doing at the other end. Let's say that I'm trying to debug an aspect of the sel_form subroutine that generates the page that allows the user to select the number of forms to print to an individual page. The first time the script is called from within the browser the debugger would initialize, but no changes would be made to what is displayed in the browser. That's because nothing has actually happened yet. I might specify a few scalars in the watch list on the right, and decide that I want to look at the environment at the end of the get_session subroutine before continuing. I would therefore set a breakpoint at the end of that routine and click [Run To]. Once execution paused at that line I could view whatever changes had been made to the scalars to that point, and mouse over other scalars to see what values they held at that point. More likely than not, I'd then set a breakpoint within the sel_form routine, click [Run To], and conduct further investigations.


The best way to figure out what is going on with the debugger is just to mess around with it. If you've been following along with things to this point you've got a working web server environment with several working scripts. Install the ptkdb debugger, and make the change to the shebang line that causes the script to initialize. You know how these scripts work ... call the one you've changed and when the debugger initializes follow execution of the script through different incarnations, tracking the values held in key scalars. That's the best kind of practice you could get.


Unfortunately, execution of the ptkdb debugger can be somewhat problematic. I've run into contexts in which it chokes on issues other than syntax errors, issues it should of course be able to handle. From the errors displayed, it appears that there is at least one situation in which the ptkdb module is not passing arguments to a function in the Tk module appropriately, or perhaps the Tk module is not trapping errors as it should. You may not find this to be a problem on your system, it is currently a problem on Ralphzilla. Rather than debugging this debugger <grin>, I've found myself using the internal perl debugger when the need arises.


The perl debugger is an interactive perl environment that is initialized when perl is called with the -d switch, followed by the name of the script you are debugging. As with the ptkdb debugger, the script must be syntactically correct and able to compile before it can be analyzed with the debugger. The environment will simply bomb out if the script won't compile.


If you've been using the CPAN module to manage your installed modules, the basic perl debugger will remind you of that environment. Once started, you are dropped into a text mode environment with the compiled program loaded in memory. Communication with the debugger is rather cryptic, consisting of one letter commands, sometimes followed by arguments. If you start using this debugger you'll probably want to have a ready listing of those commands at hand. All in all, as with any minimalist tool, the perl debugger is very fast and flexible. The letter b followed by a number sets a breakpoint at that line, for example, and the letter x followed by an expression evaluates that expression. Therefore, if I wanted to look at a generated value for $session_id at line 226 of the full_2.cgi script, I'd

1) start the debugger: "perl -d full_2.cgi"

2) set a breakpoint at line 226: "b 226"

3) run to that point: "r"

4) evaluate $session_id: "x $session_id"

All of this is pretty straightforward. Unfortunately, debugging cgi scripts with the perl debugger is a little less so, although not really all that bad once you see the pattern of how you use it. With this debugger, you must feed the debugger and script the values that would normally come from interaction with the browser client. The standard way to do this is to modify the script slightly to save the cgi environment at a point just before the piece of code you wish to examine and then read that in when initializing the debugger. For example, you could save the cgi environment like this:

WARNING:The method I orginally described for transferring state data into an invocation of the perl debugger does not work, or at least has not been for me. I took that from one of the canonical references for CGI programming under perl ... I guess it just goes to show that you can't trust everything you read <grin>. In the last couple of days I've had cause to try it, as well as some alternative routes, and had difficulty getting them to work as advertised. I have a notion for a subroutine to do something similar, and will make it part of this page when it is finished. If you are finding that ptkdb balks on a CGI script, for now I'd suggest following the rather clunky route of just printing the values you want to test to the browser, which is a method that works rather well even if it is a bit roundabout, or saving the state, modifying the script to remove conflicting parameter references, and simply typing scalar values into the debugger by hand, which is also not a very elegant solution, but does have the advantage of forcing you to get your hands down in the machinery. Just don't turn on the power while they're in there. I won't be held responsible for missing virtual fingers. Note that this does not pertain to non-CGI scripts.


That may not be as much fun as using ptkdb, but I rather expect that when one becomes experienced with it, the perl debugger can be a very quick and efficient tool. My own thought is that the optimal setup would have both available: the basic perl debugger for tracing what is going on with a specific target that you regard as suspect, and the ptkdb debugger for examining the wider environment as a script executes.


Obviously, this has not been an exhaustive treatment of script debugging, but to a point all I can do is get you started ... much of what would be done in any given context would be dependent on the nature of that context. Recognize that the debugging process is the other half of the programming process. Anyone who says that they never have to do it is simply lying. Most material you see published in print or on the web, especially for a widely-used environment like perl, assumes a certain level of familiarity wth a given implementation, and debugging a script that you have adapted from an example from such sources (like this one!) is a good way to pick up a deeper level of understanding about what is going on beneath the surface. Treat the process like that and you'll not only get more from it, you'll have fun doing it.



The following links provide more information about debugging in perl, including documentation about the perl debugger.

Perl Debugging Tutorial from the documentation included with perl 5.6.1
Very good article on debugging perl from the IBM DeveloperWorks Library
Page of links about debugging perl at about.com



Next - Cron