Hey everybody!!!! very few of you posted your opinion about archives and our lists. Does this mean you don't have an opinion???? Hard to believe :) I will not open it up unless I hear some responses OK (below is the latest scoop) Thanks Kay Subject: STATUS: Mailing list archives Resent-Date: Mon, 24 Nov 1997 20:16:17 -0800 (PST) Resent-From: [email protected] Date: Mon, 24 Nov 97 19:44:04 PST From: Karen Isaacson <[email protected]> Reply-To: [email protected] To: [email protected] There've been lots of questions. I haven't answered most of them. Because I'm a lazy under-achiever? Nah, that's not it. The thing is, things are still a bit in flux. News, in no particular order. (But some of it is important, so skip the boring bits and read the interesting ones.) - Brian caught my flu, so hasn't had a chance yet to revise the isearch/ifetch code to allow a more sensible directory stucture, something we desperately need with over 2000 mailing lists. So no new lists have been indexed yet at http://searches.rootsweb.com/examples.html although by now I do have the old messages from all the lists (even the oft-neglected ones still on the old servers) prepared for indexing. - We observed late on Friday that when things are really rocking, the archiver software (the stuff that builds the nifty threads) interacts in an unfortunate way with the operating system. It's not a serious problem, but we may end up moving it onto a different server. If that happens, there will probably be a new URL. There probably will be a new URL anyhow, because... - Something odd happened last week. On Tuesday, I gave someone a new e-mail address, and they added it to their web page. (It was a "mailto" link, if anyone cases.) The address hadn't existed before then. On Thursday, less than 48 hours later, guess what address was spammed? We knew things were bad, yes, but that bad? OK, that bad. Given things are that bad, we've revised the approach. Brian and I had hoped to have a demo up of what we'll be doing, but the flu kept him from finishing the script, so you get me waving my hands instead. - So, what do we intend so that the web-crawlers don't scoop up all your addresses? A front door, of sorts. Anyone accessing the archives, either the threaded ones or the search engine or, ultimately, the search engine to the threaded messages, will have to come in via a particular page. The page will have a box in it. In the box, you'll type the name of the mailing list. (We'll make it robust, so that JONES, JONES-L, JONES-D, jones, Jones, etc., all work.) Then you click on "submit" and are deposited on a page (build on the fly by a cgi-bin script) that lets you search the Jones archives, or follow a link to the threaded Jones messages. Brian says he can set this up so that, once you're in the "archive area", you can easily go from page to page, but that if you try to jump into the middle, you'll instead be diverted to the front door where you have to type a list name. So no robo-crawler will be able to wander through our message bases collecting addresses, but your listmembers won't have to remember a "password" any more elaborate than the name of the list. - Recent digests (probably a month's worth) will remain available for ordering via e-mail in the usual manner. - I'm aware of two lists that index or cross-reference their messages by message number or digest number. Are there more? If there are only two, I can run a script on your existing archives to add the message and/or digest number to each message, so that you could use the search engine to call the message up that way. I don't want to promise this for your =future= messages, though. I can do this now, or I can do it later, no hurry in letting me know. - Until we can switch over to searching the threaded messages, we'll continue to update the unthreaded message base in parallel, so that all but the most recent messages will be searchable. If I can get things organized right, I'll leave the e-mail search up, but only the most recent messages will be available for searching. That should limit the amount of computer resources consumed. There will be no instruction or training or hand holding or other support for the use of the e-mail search, but if you know how to use it already and want to find a message you saw come through day before yesterday, you'll be set. I'm sorry, there just aren't enough hours in the day for me to provide "better training". - If you want your messages included in the threaded message base =now= be sure to subscribe [email protected] to your mailing list. - If you decide you don't want your messages included in the threaded and unthreaded messages bases, be sure to add the address [email protected] to the reject list for your mailing list. (Most of you can access this via the "edit selected files" button at the bottom of the utility page for your list.) If you don't do this, then on the cutover date (target: December 1st), the address [email protected] will be subscribed to your list and a threaded message base will start being built. - If you want to participate, but want to start with a clean slate, write to [email protected] and ask that your old archived messages be omitted. We can package them up and arrange for you to FTP them, if you'd like. - None of this is final. Well, if you ask me to throw away your archives, and I do, I probably won't be able to recover them. But if decide to participate, and don't like it after awhile, let us know and we can remove the search engine, threaded message base for your list, etc. - Someone commented on the mail-to's being to the poster rather than to the list, so that interesting messages might be lost. I don't know what the answer is here. On ROOTS-L, where we have 9 years and 11 months worth of archived messages, I think people would be confused if someone found a query in 1993 that they wanted to respond to, and posted their response to the list. "Those sound like my ASCHNEWITZes! Shall we compare notes?" Plus many of our lists are closed at this point, so only subscribers can post (as an aside, this isn't true of ROOTS-L, where the messages are screened), so someone responding to an archived message by writing to the list might only end up in the error bucket. Let's revisit this once we're in production, and have a better sense of the size of the problem. One thing we could do is also include an option to "post to the list" on the page. But would that be a good thing, or would that invite noise? If you have comments, I've set the Reply-to on this message to [email protected], as that seems like the best forum to explore these concerns. - - - - - - - - - --- - - - - - - - Subject: Archives Delayed ... Resent-Date: Sun, 30 Nov 1997 19:14:02 -0800 (PST) Resent-From: [email protected] Date: Sun, 30 Nov 1997 18:54:01 -0800 From: Brian Leverich <[email protected]> Reply-To: [email protected] To: [email protected] CC: "Dr. Brian Leverich" <[email protected]> Hi all - Modest bad news: Marc and Karen are ready, but I've just now finished some essential hacks down in the C++ bowels of the Isearch search engine. That's thrown the schedule off, we're going to have to delay the start of the full archive implementation. There's not a lot of work left to do, but it may be as much as two weeks before we turn the full implementation on because of the interplay of consulting assignments (Karen and I have to eat ... ), the December RSL (which comes out next weekend), and other things. That two week delay is a worst-case estimate, though -- I think we'll have most or all of the archives software online much sooner than that. Tim Pierce, if you want to get a X-NO-ARCHIVES blocker built into SmartList's procmail filters, hack fast. You have a little time ... Everybody, sorry for the delay. -B -- Dr. Brian Leverich Co-moderator, soc.genealogy.methods/GENMTD-L RootsWeb Genealogical Data Cooperative http://www.rootsweb.com/ P.O. Box 6798, Frazier Park, CA 93222-6798 [email protected]