A simple partitioning implementation, for use when crawling a few hosts. It hashes on the hostname andn port, plus the directory path, but not filename of the URL.
NB: This can mean that large dynamic applications where the page is changed using the querystring, all fall in the same partition as their hash value is the same.
Returns the path of $url, minus the filename should there be one.
$Revision: 1.1 $