| Terrier |
http://terrier.org/ |
|
Terrier is a software for the rapid development of Web, intranet and desktop search engines. More generally, it is a modular platform for the rapid development of large-scale Information Retrieval applications, providing indexing and retrieval functionalities. The latest release is Terrier 3.0.
|
| |
| Labrador |
http://www.dcs.gla.ac.uk/~craigm/labrador/ |
|
Labrador is a web crawler (spider, bot or robot) written in Perl. Soon to be released as open source.
Labrador is capable of generating TREC formatted crawls of websites, is highly extensible and configurable. |
| |
| Genmon |
http://www.macdee.net/perl/genmon/ |
|
Genmon is a small modular script platform for monitoring devices, services hosts I wrote several years ago. I use it to monitor our search services here at the Department of Computing Science.
|
| |
| Test Collections (WT2G, WT10G, DOGTOV, DOTGOV2) |
http://ir.dcs.gla.ac.uk/test_collections/ |
|
While technically this is data, and not software, the University of Glasgow are the NIST-appointed distributors of the test collections of web crawls for the TREC Web, Terabyte and Blog tracks. If you're experimenting with Information Retrieval systems in a Web context, then these crawls are really a necessity. As queries and relevance assessments are available from NIST for these collections, you can use these to tune your system or approach. More details can be found on the Test Collections website.
|
