Your leaking thatched hut during the restoration of a pre-Enlightenment state.

 

Hello, my name is Judas Gutenberg and this is my blaag (pronounced as you would the vomit noise "hyroop-bleuach").



links

decay & ruin
Biosphere II
Chernobyl
dead malls
Detroit
Irving housing

got that wrong
Paleofuture.com

appropriate tech
Arduino μcontrollers
Backwoods Home
Fractal antenna

fun social media stuff


Like asecular.com
(nobody does!)

Like my brownhouse:
   BetterExplode
Wednesday, November 28 2007
I tend to be a sloppy computer user, opening up windows at the slightest provocation and then getting distracted before finishing with them, leaving them to accumulate in deep stacks on my virtual desktop. I have four LCD screens and lots of desktop, but it's rare that you can see more than fragmentary holes down to the desktop background (currently a scene from the Central Catskills that I photographed back in October).
This steady accumulation of windows is analogous to memory leaks, a common problem encountered by computer programmers. As software runs, it can allocate more memory than it frees up, gradually filling all of memory and leaving nothing for anything else, at which point either the software or the computer itself crashes.
A bad combination is my "window leak" habit while using a program that leaks memory. I love Firefox, but it's got some bad memory leaks, and though I have two gigabytes of RAM in my main machine, with forty Firefox windows open my computer gradually slows down to a crawl. Eventually I'm motivated to free up space (both on the desktop and, in the process, in memory), but things have to get bad first.
While down in Silver Spring I learned of the newly-released Firefox 3.0 beta, and that special attention had been paid in this release to fixing memory leaks. I downloaded a copy then and there.
I immediately used it to tweak a database I've been working on and quickly discovered (in my DHTML-heavy web-based SQL table editor) a Javascript incompatibility with earlier Firefoxes and Internet Explorer. Still, it seemed to run better in the relatively small memory environment of my five year old Evo N410c laptop.
I've been running Firefox 3.0 beta on my home computer since returning from Silver Spring and it's been performing well. If I leave twelve windows open for a couple days I get memory footprint of 180 Megabytes, which is probably a third of the footprint Firefox 2.0 would have left.

I spent most of the day building a tool for importing comma-delimited data files (such as exported Excel spreadsheets) into my custom web-based MySQL tool. Importing such files is actually a rather complex process, because (unlike with a SQL-to-spreadsheet export), one usually needs more information than a spreadsheet provides to turn the data into a table. Microsoft provides export wizards to allow a user to decide which import fields map to which database columns. Such wizards always have more steps than they need, but they provide a usable model for how such a tool can work. In my tool I ended up with a two-step process. First you upload the import file, then the tool parses it to display samples of the data in each field, and you as a user can use dropdowns to decide which fields end up in which database column. Alternatively, the tool provides a way for a new MySQL table to be built from scratch using the data in the import, and it can even name columns using information in a column-labeling row, if one is present.
A large fraction of the work I did for this tool was the building of a parser that can successfully find delimiters while avoiding them should they appear inside blocks of quoted text. The parser crawls through the text file character-by-character, going into different modes depending on whether it runs into quotes or delimiters. One can specify a long list of acceptable quote characters which it will use for going into and out of "quote mode," always keeping track of which character threw it into that mode so it will know to look for it again so it can come back out when it appears. This function ended up being a feature-rich version of PHP's built-in explode function, which I use heavily in my programming. I called my new function BetterExplode. Here it is with all the support functions it needs to do its business.


function BetterExplode($delimiter, $data,  $quotechar="\" '", $bwlLeaveQuotesInPlace=false)
{
//Judas Gutenberg November 28 2007
//moves through data character by character breaking on 
//delimiters so quoted characters are not mistaken as delimiters
//$quotechar can be a space-delimited series of acceptable quotes, 
//though they cannot be mixed when 
//quoting a particular string
//think of it as a more complicated version of explode

	//handle quotes being passed in as ascii values:
	$arrQuotes=explode(" ", $quotechar);
	$quotechar="";
	foreach($arrQuotes as $thisquote)
	{
		if(is_numeric($thisquote))
		{
			$quotechar.=chr($thisquote) . " ";
		}
		else
		{
			$quotechar.=$thisquote. " ";
		}
	}
	$quotechar=RemoveEndCharactersIfMatch($quotechar, " ");
	//echo "-" . $quotechar . "-";
	if (contains($delimiter, "."))
	{
		$delimiter=multichrhandle($delimiter);
		//hacky and imprecise i know - but for multiple 
		//character delimeters this is what i do!
		//and it hardly matters in most cases, where the 
		//second character is chr(10)
		for($i=0; $i<strlen($data); $i++)
		{
			$chr=substr($delimiter, $i, 1);
			if ($i==0)
			{
				$delimiter=$chr;
			}
			else
			{
				//throw away all instances of other 
				//characters in delimiters in $data - 
				//could break things!
				$data=str_replace($chr, "", $data);
			}
		}
	}
	else if (is_numeric($delimiter))
	{
		$delimiter=chr($delimiter);
	}
	$arrOut=Array();
	if($quotechar=="")
	{
		$bwlQuoteOn=true;
	}
	else
	{
		$bwlQuoteOn=false;
	}
	$intArrCursor=0;
	$chrKnownQuote="";
	$chrQuoteToBeEscaping="";
	for($i=0; $i<strlen($data); $i++)
	{
		$chr=substr($data, $i, 1);
		if($i+1<strlen($data))
		{
			$chrNext=substr($data, $i+1, 1);
		}
		else
		{
			$chrNext="";
		}
		if( $chr==$delimiter    
			&&  (!$bwlQuoteOn  || $quotechar=="")  )
		{
			$arrOut[$intArrCursor]=$thisvar;
			$intArrCursor++;
			$thisvar="";
		}
		else if ((($chrKnownQuote=="" 
			&& inList($quotechar, $chr) 
			|| $chrKnownQuote==$chr)) && !$bwlQuoteOn)
		{
			$bwlQuoteOn=true;
			$chrKnownQuote=$chr;
			if($bwlLeaveQuotesInPlace)
			{
				$thisvar.=$chr;
			}
		}
		else if ($bwlQuoteOn && 
			(($chrKnownQuote=="" 
			&& inList($quotechar, $chr) 
		|| $chrKnownQuote==$chr)))
		{
			//handle internally-escaped quotes, that is, a known 
			//quote escaping one immediately following in data
			if($chrQuoteToBeEscaping==$chr)
			{
				$chrQuoteToBeEscaping="";
				$thisvar.=$chr;
			}
			else if($chrNext==$chrKnownQuote)
			{
				$chrQuoteToBeEscaping=$chr;
			}
			else
			{
				$bwlQuoteOn=false;
				$chrKnownQuote="";
				if($bwlLeaveQuotesInPlace)
				{
					$thisvar.=$chr;
				}
			}
		}
		else if ($bwlQuoteOn)
		{
			$thisvar.=$chr;
		}
		else
		{
			$thisvar.=$chr;
		}

	}
	$arrOut[$intArrCursor]=$thisvar;
	return $arrOut;
}

function multichrhandle($strIn)
{
	$out="";
	$arrIn=explode(".", $strIn);
	foreach($arrIn as $chr)
	{
		$out.=chr($chr);
	}
	return $out;
}

			
function inList($strList, $item, $MatchOnlyType="") 
//look to see if item is in 
//the space-delimited strList (similar to my ASP version)
	{
		$arrThis=explode(' ', $strList);
		for ($t=0; $t<count($arrThis); $t++)
		{
			$thislen=strlen($arrThis[$t]);
			if($MatchOnlyType=="last")
			{
				
				if ($arrThis[$t]==substr($item, strlen($item)-$thislen))
				{
					return true;
				}	
			}
			else if ($MatchOnlyType=="first")
			{
				if ($arrThis[$t]==substr($item, 0,$thislen))
				{
					return true;
				}	
			}
			else
			{
				if ($arrThis[$t]==$item)
				{
					return true;
				}
			}
		}
		return false;
	}

function RemoveLastCharacterIfMatch($strIn, $chrIn)
	{
		$out=$strIn;
		if (substr($strIn,  strlen($strIn)-1, 1) ==$chrIn)
		{
			$out= substr($strIn, 0, strlen($strIn)-1);
		}
		return $out;
	}
	
function RemoveFirstCharacterIfMatch($strIn, $chrIn)
	{
		$out=$strIn;
		//echo substr($strIn,   0, 1) . "<br>";
		if (substr($strIn,   0, 1) ==$chrIn)
		{
			$out= substr($strIn, 1);
		}
		return $out;
	}
	
function RemoveEndCharactersIfMatch($strIn, $chrIn)
	{
		$out= RemoveFirstCharacterIfMatch($strIn, $chrIn);
	 	$out= RemoveLastCharacterIfMatch($out, $chrIn);
		return $out;
	}         


For linking purposes this article's URL is:
http://asecular.com/blog.php?071128

feedback
previous | next