When I launched into this version of the insert script, my primary intent beyond adding half_inning to the storage function was to lean out the code a bit, clean up some pieces that I put in as I was attempting to track the source of a couple of bugs, and generally render it a little more legible and ultimately more amenable to modularization. For example, as I was initially developing the previous version of the script I was trying to track down a series of "unitialized value" errors in the comparisons that determine whether a viable file handle can be established on a given target. I changed the destination for the split functions preceeding those comparisons from arrays to a series of temporary variables to isolate potential sources of that problem. Not considered good form, and not pretty, but certainly functional. (In fact, if you are developing a script under time pressure and a rough version delivers the required functionality with acceptable performance, and support is not an issue over the relevant time frame, it is perfectly acceptable to leave that script running indefinitely. However, if support by someone else is or becomes an issue it is appropriate to revisit the code and clean it up, because legibility is an important component of maintainability.)

As I was thinking about this, however, I realized that I really did not like the manner in which the previous script handled the association of host names with mosix numbers, and since one of the stated reasons for this clean-up phase is to prepare for modularization, I decided that this is the appropriate place to do something about that. I was very pleasantly surprised to discover that nothing exploded in my face as I made the revision, although I will not discount the possibility that there is something ticking back there, waiting to go off. There almost always is <grin>.

You may recall from the previous incarnation that I mentioned that if I were to adopt a more automated approach to this issue I would probably traverse the structure under the mfs mount point, and use that as the basis for performing the actions that establish the environment required to support what follows. In a fashion similar to the previous version, the script tests for the presence of key files and the subroutine that creates them is run if one or the other is not found. (As you will see, in this case that information is stored in two files rather than in one as in the previous version.) Again as in the previous version, the number_config subroutine is executed if that condition is not met. Before I get into the subroutine itself, however, note that I have added the socket module to the operating environment - I'll get to the reason for that in just a bit.

In a sense, the implementation of log and repository file destination management incorporated into this version is an intermediate step on the evolution of this functionality from the explicit designation of machine name and mosix number used in the previous incarnation to an upcoming version in which the names of the destination machines are read in from external files that are written as part of a configuration management component. (I briefly considered storing directory locations similarly, but decided that at that point the additional functionality provided might be counter-productive. Writing these files is a system function, not a user function, and while the flexibility to designate different machines as alternative locations is a real enhancement to the stability of the system, I do not want them to get scattered all over the cluster file system to the extent that it would become difficult to track down the appropriate file to use in database recovery.) In this version I still use the $controller_num and $server_num scalars to establish destination directories, but after that point the information used to perform system operations is pulled entirely from the cluster environment.

Just as before, the number_config subroutine writes the address information required for execution of the remote shell commands that I use to determine the space available on the target machine. Since, however, I have removed explicit specification of machine names from the script, I have to create a third file that holds the names of the machines, and ultimately have to be able to associate those hostnames with the mosix number. I will get to the manner in which I do that in just a bit ... for now I have to get the source information written.

As I said above, the strategy employed by this script involves reading the directory under the mfs mount point and using that as the source of the cluster mosix numbers. As long as the mfs filesystem is mounted, all of those numbers will be there ... that's just the way mosix works.

 
			my ($address,$host,@host);
			my $mfs_mount='/mfs';
			opendir(MFS,$mfs_mount);
			my @machines=readdir(MFS);
			my $out_host_addr="/home/www/host_addr";
			my $out_numbers="/home/www/numbers";
			my $out_hosts="/home/www/hosts";
			open(OUT,">>$out_numbers");
			open(OUT_HOSTS,">>$out_hosts");
			my $machine;
			close MFS;

Now that I have an array with the contents of the mfs mount point directory as elements, I iterate over that array and for each element that had a digit in the first position use a system command to the mosctl utility with the whois option and the $machine scalar.

			foreach $machine(@machines)	{
	
						if ($machine =~ /^\d/) {
				
							system("mosctl whois $machine >> $out_host_addr");
							print OUT $machine."\n";
						}
				}

			open(ADDR,"<$out_host_addr");
			while (<ADDR>)	{
				
				$address=inet_aton($_);
				@host=gethostbyaddr($address,AF_INET);
				print OUT_HOSTS $host[0]."\n"; 
			}
			close OUT_HOSTS;
			close OUT;

As with all things perlish, there are a multitude of ways to do this, from writing off a cookie with this information to writing to a small text-based database. I will likely play around with something like this in a later version, because I am not totally comfortable with relying on the order of the lines in the files to establish the association between related items, but since this is done by the program I can be reasonably sure that it will work the same each time <ahem>, and is therefore perfectly adequate. The importance of the information lies in the way I'm going to access it within the script, and that means that the important part now lies in how I pull that data into an accessible structure. That's where the fun in this revision appears<grin>

Think a little about what is going to have to happen to set up the environment that will allow the remote shell commands to be executed. The mosix number of the target machine will be present, and from that the script will have to be able to get the host name. The best way to do that is with a hash, a special form of array, formerly called an associative array, in which elements are paired and one of the elements is used as an index to access the value of the other. You can find find quite a lot of material on hashes in the various perl tutorials that are on the internet, here, for example, but the basic concept is really quite simple. In this context I'm going to construct a hash in which the mosix number is the key value used to access the host name for the machine with that mosix number, and that is what happens in the hash_assign subroutine.

While this subroutine is generally simple, it does introduce two new perl tools. First, as I mentioned in the paragraph above, in this subroutine I associate the mosix numbers and the host names in a has, which will result in a data structure that will look something like this:

					{ "1" => "ralphzilla-raider",
					  "2"=>"ralphzilla-faxa",
									.
									.
					  "5"=>"ralphzilla-b-free"}

			my %hostnum;
			open(NUMBERS,"/home/www/numbers");
			open(HOSTS,"/home/www/hosts");
			my @mos_num=<NUMBERS>
			my @hosts=<HOSTS>

			my $mos_num;
			my $i=0;

			foreach $mos_num(@mos_num)	{
				chomp $mos_num;
				$hostnum{$mos_num}=$hosts[$i];
				$i++;
		
			}

		$hostnum{$mos_num}=$hosts[$i];

The second new thing in this subroutine lies in the way %hostnum is going to be accessed. Remember my brief discussion of references in the discussion of the previous version of this script? Well, as I was putting this together it occurred to me that this is a relatively straightforward context in which to get started on them. You may have noted in the main body of the script that I defined the scalar $hostref. I did that at that point to get the $hostref scalar established in the scope of the main body of the script, and as a result, the scalar should be visible throughout the script. In the line immediately following the loop that populates the has, $hostref is assigned a reference to the %hostnum hash.

		$hostref=\%hostnum;

This has been a somewhat long-winded digression, but the context I've created is precisely the kind of context in which many bugs occur, and it illustrates the virtues of doing the kind of over-building, as it were, that I'm talking about. If you were absolutely confident with the remainder of the structures that surrounded the code that included the if statement, you would sooner or later recognize that the second = sign was missing. Building an application is inherently incremental. One creates a foundation and then starts adding functionality to that structures. The fewer elements of uncertainty you add as you implement a bit of functionality, the easier it will be to track down malfunctions. As as a side note, you can find out a lot of interesting things as you take out the little pieces that you think you do not really need.)

Anyway, back to the script. I'm now at the point at which I am going to start combining the two loops used to check for filehandle viability and space availability for the log file and the repository file. The essential strategy that I employ is to embed the logic that was incorporated in the two loops previously into a single loop that is itself embedded into a larger loop that executes only twice ... once for the log file and once for the repository.

I begin by initializing the various structures that I use to control the loop and to hold information as the loop executes.

my @prefix=('l','r');
my ($loop,$prefix,$fh,$fh_name,$dir,@destinations,$free);

After initializing the Filesys::DiskFree object as I have done in previous versions, the script enters the loop that will execute once for each element in the array.

my $df=Filesys::DiskFree->new();
$df->df();

foreach $prefix(@prefix)	{

		$loop=1;
		
		if ($prefix eq 'l')	{
		
				@destinations = ("/mfs/$server_num/data/bb_system/",
									"/home/www/");
				$min_space=100;
				$name="bb.log";
				}
				
		elsif ($prefix  eq 'r')	{
		
				@destinations = ("/mfs/$server_num/data/bb_system/",
								  "/home/www/",
								  "/mfs/$controller_num/home/www");
				$min_space=500;
				$name="repository";
				}

		while ($loop)	{
		
				$dir = shift @destinations;
				
				if (! $dir)		{exit;}
				
				free_space($dir);
				
				$fh = new FileHandle($dir.$name,'>>');
				
				if (($free > $min_space) and ($fh)) 	{
				
					$loop=0;
					if ($prefix eq "l") {$log_fh=new FileHandle($dir.$name,'>>');}
					elsif ($prefix eq "r") {$repo_fh=new FileHandle($dir.$name,'>>');}
				}
				$fh->close;
			}

Beyond that, the loop functions pretty much as did each of the loops in the previous version ... the only way out of the loop is to either successfully pass the two tests, in which case the script continues, or the end of the @destinations array is reached, leading to exit from the script. As before, the script terminate if none of the specified locations for either the repository file or the log file are viable. While the previous version, in which it would have been relatively easy to execute different action upon failure if desired, in this version I would implement such by calling a subroutine within the block defined within the (! $dir) if statement that is now comprised simply of exit;.

The changes I made in how host names can be determined are actually put to use in the free_space() subroutine. But first, if you take a look at the early part of that subroutine you will see that the temporary variables that previously held the results of the directory split have been replaced by the array @full_path, and individual elements now are referenced by their position in the array.

	my $dir=shift @_;
	my @full_path=split('/',$dir,4);
	my @result;	

	if ($full_path[1] eq 'mfs')	{

		my $path='/'.$full_path[3];

		$rhost=$hostref->{$full_path[2]};

This is, I think, a good example of using data structures to retrieve information that I previously specified explicitly, and how they can be used to implement code that is both cleaner and more maintainable. What I went through earlier with the new version of the number_config and hash_assign subroutines may seem like a great deal of trouble compared with simply hard-coding the relationship between mosix numbers and host names, but that approach would require that the code be changed if the cluster were to be modified, and that is the kind of thing that might not be caught by inexperienced system maintenance personnel. The number_config subroutine may take a few seconds to run, but it only has to run when the source files are not present. (Actually, in the current state of the system the administrator would have to know to delete those files when a change is made in the cluster in order to force the script to re-create them. This is a substrantial improvement over having to edit the script, but it is obviously not bullet-proof. While it is not unusual for real-world systems to have similarly quirky actions that are required to be performed periodically, the best solution is to have the system somehow sense that requirement and take the action on its own. I am putting that on my to-do list, right now <grin>.) As I get into building modules, you will see that a large part of the rationale for doing things like this lies not so much in their immediate context, but in extending the accessibility of that information beyond the scope of that context, thus enabling the same functionality in the wider framework of the application and, as you will see, beyond.

In the next section, I will start creating modules that will hold many of the subroutines I've used to this point.