We got 7 new servers this week. We're still setting a few of them up but in summary we got: 4 new web servers, a new site master db to replace the older over-powered pre-cluster one (which will get a new job), a new cluster1 master and cluster1 slave, since those machines only had 2 GB of memory before (their max).New architecture!
We've moved from FastCGI to mod_perl. I've been meaning to do this for ages, but while I was in school I always had more important things to do. It turned out to be incredibly simple, probably because I'd been preparing for it for years.No disks!
Now we don't have to do guess-work balancing FastCGI process counts or trust FastCGI's dynamic mode. Along with lingerd, things be totally rockin'. But... some nodes are still in the process of being converted. See next section.
All our web nodes (all 14+) will shortly be booting and operating entirely from the network, without harddrives, because they suck. So far only a few are doing this, but adding more is trivial: just add to a file IP address, ethernet MAC address, and server role. (But I'm tired after a week of work in the NOC, so I'll finish later)So what's this all mean? Tons more speed, and it's a ton easier for us to manage.
By next week sometime everything new should be fully setup, then we can get back to fun things. (well, things that are fun for you)
Update: I forgot the sad part. We're getting rid of our South Park server names since 1) we pretty much ran out of names, and 2) they were a bitch to remember and type. New names are like gm (global master), gs1 (global slave 1), c01m (cluster 1 master), c02s2 (cluster 2 slave 2), w14 (web 14), and m2 (misc 2). Betyacareeh? :)