Okeley dokeley. I should have some time tomorrow. I may even have time to work on that admin "search-for-user-email" upgrade. :)

ita, you are completely amazing.

Daniel, when I hit that link it gave me your WX identity. You might want to go back in and take out the text between the @@ signs so it won't do that anymore.

ROFL. I had forgotton about that. Thanks.

Copied from WX. I think much of this is more technical in nature and so doesn't need to wait for the Bureaucracy-decision-making-mechanisms to be put in place:

Oh, and about archiving threads: it would be great if the Nillys would still work even when threads have been archived. Might links have to be redone?

Maybe. Let's look at the archiving options.

1) Threadsuck old threads, compress them into zipped files and remove them from the database This is what I thought had been decided a while back in BBaBB. Advantage: would take up the least amount of space. Disadvantage: Nillys to the thread would no longer work. ita came up with the idea of having any links to the original thread take you to the spot on the archive page where the link to zipped thread is. This could be done automatically via the code. Users would have to first download the thread. At that point, they could use their browser's "find in this page" option to find what they're looking for. Not ideal, but it works.

2) Threadsuck old threads, remove them from the database, but DON'T compress them into zipped files. Advantage: With a small tweak to the threadsuck function (adding a name tag to the front of each post) and some other coding changes, Nillys could still work. Disadvantage: Would take up a lot more space. Question: Do we put entire threads in one file, or split them up into (say) 1000 post chunks?

3) Move old threads into a new Archive Database. Advantages: It's a database so it could have the same functionality (searching, etc.) as the original thread database. Disadvantage: As Archive Database grows, we could run into the same database-too-big problems we're having now. Questions: How much coding and HD space would this require relative to option 2)?

I'm most pro option 1. Not least of all because I'm most used to it. Options 2 and 3 don't significantly reduce the total amount of space we're taking up.

I'm pro-1 as well, but is space really an issue? Or are you thinking ahead to "Natter 57 Varieties of Saying Nothing"?

I have to say that being able to search the entire archive can be very useful at times, especially if you're trying to find an old discussion, and aren't quite sure when it occurred. I've done this a lot over at the WX Attic. I'm for the second database. Taking the older threads out of the active database will cut the number of hits for casual searches on a user name (which may have been a contributing cause of the crash) down a lot. The kind of search I mean is of the sort, "Has DX posted today? I'll search on his User Name and check." Or searches to see if someone has asked about you. These are really only looking for recent posts, but right now they search the archived threads as well, so you wind up with every post the user has ever made. For some posters, this is almost equivalent to a full threadsuck.

Putting the archives into a different database would mitigate that somewhat, because usually searches of the archives are more specific, so fewer posts returned per search, and probably less common.

Options 2 and 3 don't significantly reduce the total amount of space we're taking up.

No, but is that the main problem? I thought we were more worried about server load. Also, did we figure out what is taking up all the extra space? When I added of the sizes of the database, filk, and archives, I didn't get the same number that they mentioned to you.

These are really only looking for recent posts, but right now they search the archived threads as well, so you wind up with every post the user has ever made.

I almost always use the "seached subscribed threads" option but I realize that many folks might not think to do that. t edit Nor might many folks think to unsubscribe from closed threads.

Also, did we figure out what is taking up all the extra space?

Right, I forgot about that. From WX:

Your database is 324MB in size, and your home directory is 262MB in size. Your plan includes 350MB of storage. You are over by 236MB.

I don't get this ita. What's taking up 262MB of space if not the database?

[and DX replied]

At least some of it (40 MB or so) is the archived TT and WX threads, and the filk site.

is that the main problem?

It isn't yet, but it will be. I'd hate to have to react to that in a panic, the same way as we're having to react to the server load issue.

As for load, keeping them in the database in a different table offsets them from being part of the day-to-day processing, but I have a suspicion that since the database will still have the same number of records, with the overhead of having more tables that it will still drag it down. Much less, but it will still be a factor.

I don't get this ita. What's taking up 262MB of space if not the database?

I will check with support.