From: Daniel Veillard (Daniel.Veillard@w3.org)
Date: Tue Sep 08 1998 - 17:42:31 EDT
An interesting question and since I plan to elaborate on this I will
Cc to the mailing-list for archival and suggestions:
> How do you generate your rpm2html.config file ?
Well it's both by hand and generated ... The manual part it to list
all the distributions, assigning subdirs, colors, etc ...
> I can see that sunsite.auc.dk now is listed as mirror site for some rpm directories
> (bot not for others....). Do you check and add these yourself manually ?
> I can see that sunsite.auc.dk is not listed as mirror for the redhat contrib dirs.
> We have these under ftp://sunsite.auc.dk/pub/os/linux/redhat/redhat-contrib/
> and there is also other directories that we mirror which you do not list:
> rawhide, suse and crypto @ relay.
The mirrors lists are actually generated from a mirror lists and a
tool, the design is not yet complete but here is the current scheme
rpm2html.config
\
-> mirrorfind -> rpm2html.config
/
mirror lists
Basically mirrorfind is a program which reads the rpm2html.config,
a mirror list, and then from there make FTP control connections to
the mirrors sites to build up an accurate list for all distrib
mirrors. At the end it dumps the updated rpm2html.config with the new
mirror list
Here is what the mirror list input looks like:
----------- ~/.mirror ---------
;
; Mirror distribution
;
[ftp://ftp.redhat.com/pub/redhat]
;
;North America
;
mirror=ftp://rufus.w3.org/linux/redhat
mirror=ftp://redhat.eznet.net/pub/redhat
mirror=ftp://ftp.lame.org/mirrors/redhat
mirror=ftp://linuxwww.db.erau.edu/pub/linux/distrib/redhat
mirror=ftp://ftp.cc.gatech.edu/pub/linux/distributions/redhat
mirror=ftp://linux.ucs.indiana.edu/pub/linux/redhat
...
; Africa
;
mirror=ftp://ftp.sun.ac.za/pub/linux/redhat
mirror=ftp://ftp.leg.uct.ac.za/pub/linux/distributions/redhat
mirror=ftp://ftp.is.co.za/linux/distributions/redhat
[ftp://ftp.suse.com/pub/SuSE-Linux]
mirror=ftp://rufus.w3.org/linux/SuSE-Linux
mirror=ftp://ftp.gwdg.de/pub/linux/suse
mirror=ftp://ftp.suse.de/pub/SuSE-Linux
[ftp://ftp.caldera.com/pub/OpenLinux]
mirror=ftp://rufus.w3.org/linux/openlinux
...
-------
Rpmfind browse all the distribs listed in the rpm2html.config and
for all distribution whose origin site starts with
"ftp://ftp.redhat.com/pub/redhat" strings, it will check whether there
is a directory carrying the mirror on each site listed.
> We could use a good way of keeping the mirror list as complete as possible.
> If you provided an always updated rpm2html.config list with 'complete' information
> about which sites mirrors the different directories we (the mirrors) could make
> a tool for automatically convert your rpm2html.config file to one suitable for
> our site by just changing the order of the mirrors.
That's exactly the point of mirrorfind.
I don't like much the actual mechanism for mirror discovery, I would rather
prefer a list of Linux FTP sites and for each distribution a list of
subdirectory where the mirror is probably located (a large set of heuristics
will probably more accurate than a fixed list).
Something like:
--------- Better ~/.mirror -----------
mirror=rufus.w3.org
mirror=redhat.eznet.net
mirror=linuxwww.db.erau.edu
...
pattern=/%s
pattern=/pub/%s
pattern=/linux/%s
pattern=/pub/linux/%s
pattern=/pub/linux/distributions/%s
pattern=/pub/Linux/distributions/%s
[ftp://ftp.redhat.com/pub/redhat]
name=redhat
path=RedHat
path=Redhat
------------------
You get the idea, basically adding a mirror would be extremely simple
I could also chhange the syntax to an RDF one, and put these informations
in the RDF/resources/distribs/ directory.
All the problems are not solved yet, but I guess that the whole principle
of linux metadata mirroring should be able to work seamlessly soon without
much human processing needed.
The mirrorfind code is in W3C CVS database
http://dev/cgi-bin/cvsweb/rpm2html/mirrorfind/
I will package it if other people are interested,
Daniel
-- Daniel.Veillard@w3.org | W3C MIT/LCS NE43-344 | Today's Bookmarks : Tel : +1 617 253 5884 | 545 Technology Square | Linux, WWW, rpm2html, Fax : +1 617 258 5999 | Cambridge, MA 02139 USA | badminton, Kaffe, http://www.w3.org/People/W3Cpeople.html#Veillard | HTTP-NG and Amaya.
This archive was generated by hypermail 2b29 : Thu May 10 2001 - 18:40:09 EDT