diff options
Diffstat (limited to 'debian/htdig/htdig-3.2.0b6/contrib/multidig/README')
-rw-r--r-- | debian/htdig/htdig-3.2.0b6/contrib/multidig/README | 133 |
1 files changed, 133 insertions, 0 deletions
diff --git a/debian/htdig/htdig-3.2.0b6/contrib/multidig/README b/debian/htdig/htdig-3.2.0b6/contrib/multidig/README new file mode 100644 index 00000000..f394e5e5 --- /dev/null +++ b/debian/htdig/htdig-3.2.0b6/contrib/multidig/README @@ -0,0 +1,133 @@ +README for multidig 1.1 + by Geoff Hutchison <ghutchis@wso.williams.edu> + + Copyright (c) 1998-1999 The ht://Dig Group <http://www.htdig.org/> + Distributed under the terms of the GNU General Public License (GPL) + version 2 or later. +-------------------------------- + +This document is part of the "multidig script system" a system of +shell scripts and some modified conf files that makes dealing with +multiple databases easier for ht://Dig. It assumes that you know what +ht://Dig is. If you don't know, see the website at +<http://www.htdig.org/> + +This README is a bit rough around the edges. I don't know what people +really want or need to know about the scripts. I expect a lot of +questions. Hey, maybe I'm wrong. I'm always open to suggestions, +criticisms, corrections, etc. E-mail me at <ghutchis@wso.williams.edu> + +-------------------------------- + +INTRODUCTION: + +* Why write multidig? + + There are many reasons I started the multidig system. The biggest +were the complaints that ht://Dig didn't have much of an +administration interface. If you're looking for one, multidig isn't +it. Yet. The next biggest is that people wanted me to make dealing +with multiple databases easier. If you're looking for this, you're in +the right place. + +* Why should I bother with multidig? + + If you already have a multiple-database setup and it's working +smoothly, you probably don't want to bother. It was written the way +*I* would organize a multiple-database setup. Not suprisingly, it +might be more pain to convert to multidig than it's worth. + If you're planning a multiple-database setup or you have one and +it's not working well, this will help. It hides most of the pain and +suffering behind some shell scripts and generally automates life. :-) + +-------------------------------- + +SETTING UP: + +* How do I install it? + + It's pretty easy to install. It requires bash, or at least a +Bourne-shell that supports the "source" builtin. Obviously, it also +requires ht://Dig. :-) + Change any paths in the Makefile. D a "make install" to install the +scripts in the right place and the config files in the right +place. The Makefile edits the scripts for you so the paths are consistent. + +* Now that it's in, how does it work? + + The multidig script will replace the rundig script that comes with +ht://Dig. Use it through a cron job or some other means of automating +updates. It will run through all the db that multidig knows about, run +htdig, htmerge, move the databases around, etc. As written it tries to +index with the least disk space in the least time. Thus it keeps only +the minimum files and does "update" digs. + After indexing all the db, it merges all the collections, trying to +do the same thing, fastest speed, smallest disk and RAM +requirements. It spits out a short status to STDOUT and a more +complete report to the file referened with the $REPORT option in +multidig.conf. Adding a "-v" to the command-line makes everything more +verbose. + +* Can I convert my previous multiple-db setup? + + Yes. I'm assuming you have a config file for each database you've +set up. In that case, put the databases into a directory with the same +name as the .conf file and tack the name onto the db.list file in your +config directory. This is multidig's list of all databases, so adding +a line here will ensure it's indexed using multidig. + +* How do I add new URLs to databases or add new databases? + + 1) New URLs: Run 'add-urls <db>' and either paste in URLs or + redirect a file or program. + 2) New DB: Run 'new-db <db>' to set up everything for that database. + +-------------------------------- + +COLLECTIONS: + +* What's a collection? + + Version 3.1.0 of ht://Dig added support for merging multiple +databases together. Technically, you merge one database into +another. Multidig makes this a bit easier. You set up a "collection" +of other databases and the multidig script will merge them all +together. + +* Fantastic! How do I define a collection? + +./new-collect <name> +./add-collect <name> +<insert dbs here> + + The add-collect script will go through the list of dbs and make sure +the multidig system actually knows about them. If not, it complains. + +* Can I just generate the collections from my databases? + + Yup, run gen-collect. This is what the main multidig script runs. + +-------------------------------- + +DIRECTORY LAYOUT: + +Here are the locations of files used by multidig: + + $BASEDIR/bin + add-collect script for adding db to a collection + add-urls script for adding URLs to a db + gen-collect script for generating all collections + from their db (called by multidig) + multidig script for generating all db and collections + new-collect script for making a new collection + new-db script for making a new db + $BASEDIR/conf + db.conf template database config + used by new-collect and new-db + foo.conf database config for db foo + multidig.conf config for multidig paths and options + db.list list of all db, one per line + collect.list list of all collections, one per line + $BASEDIR/db + foo/foo.urls URLs used by foo db + foo/db.* actual foo databases |