Jesse Proulx (![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small} jproulx) wrote in ![[info]](http://l-stat.livejournal.com/img/newsinfo.gif){kind=small} news,@ 2004-12-31 14:11:00 |
|
State of the Goat 2005
Another year is coming to a close, and it has been a good year for all of us here at LiveJournal. I hope 2005 will treat all of you just as well as 2004 treated us!
2004 In Review
2004 was a busy year for LiveJournal. Starting the year off, a few of us employees moved out to Oregon to come work in the LiveJournal office in Portland. Shortly thereafter our entire employee development team was located in the office. With a centralized design and development team we were able to make many improvements and tackle many projects.
Some highlights for 2004 include:
Last January we passed the milestone of the 2,000,000th account, now we host over 5,500,000 accounts. As you can imagine LiveJournal has grown up a lot this year! If this continues we can expect to see 7 or 8,000,000 accounts hosted by the time LiveJournal turns 6!
Obviously this growth quickly brings to light any weak spot in our architecture, but we've been able to tackle traffic problems and roughly anticipate future growth. Starting off the year we converted most of our user clusters to Master-Master pairs, and later on we added a few new user clusters and another global master database server. Later on we started migrating users from MyISAM to InnoDB database tables, and we purchased a few pairs of 64 bit database servers to lighten the load.
We had some vendor supply problems ordering 64 bit servers and the servers had some problems passing burn-in inspection, but now we have two pairs in production doing 65% of our database traffic without blinking an eye. All of the migration to 64 bit servers allows us to free up older hardware, making system administration easier.
During the year we also bought a couple of storage servers that could hold 16 250GB disks each, giving us 8 terabytes of disk space. We started using that to host pictures and static files, and freed up our NetApp to host other things.
For 2005 we're going to continue making optimizations and working towards high availability and reliability.
ScrapBook
Our major focus goal for 2004 (as evidenced by its manynews mentions!) was our new photo hosting service, which we've recently dubbed "ScrapBook". ScrapBook has matured quite a bit since our State of the Goat 2004, and has been made much more robust in the process. Because of ScrapBook development we can now offer Moblogging support, direct posting of galleries to LiveJournal accounts, copying of galleries and pictures between accounts, and many other features. We also wrote our own filesystem to store ScrapBook images on our storage servers, and now LiveJournal uses it to store all sorts of static files.
Employees
Last year we touched on employees who had recently joined the fold, but this year was a little more subdued. In January we hiredmarksmith to come code in-house for random projects, and now he's a development powerhouse, blazing through tasks and to-do lists left and right. Our newest systems administrator Matthew (matthew) is kicking ass (that is, when he isn't damaging vital appendages!), and is providing welcome job relief for lisa. whitaker made it out for another short internship over the summer, but both he and daveman692 left us to pursue their educations. They're working remotely for the time being, and they'll both be back next year. Last but not least we hired bleything to help out our support and abuse department and to act as one of the liasons between our volunteers and office developers.
Plans for 2005
Looking ahead to the future, we plan to focus on a few key projects, while continuing to make improvements in all areas:
Another year is coming to a close, and it has been a good year for all of us here at LiveJournal. I hope 2005 will treat all of you just as well as 2004 treated us!
2004 In Review
2004 was a busy year for LiveJournal. Starting the year off, a few of us employees moved out to Oregon to come work in the LiveJournal office in Portland. Shortly thereafter our entire employee development team was located in the office. With a centralized design and development team we were able to make many improvements and tackle many projects.
Some highlights for 2004 include:
- We won the People's Voice Award for Community website in the Webbys!
- Brad was invited to the MySQL, OSCON, and LISA conferences to discuss LiveJournal's backend
- 64 bit database servers
- High capacity (8TB) storage servers
- Massive database migrations (all but a few users are on InnoDB tables, most are on 64 bit servers)
- New projects Perlbal (Perl load balancer) and MogileFS (Perl distributed filesystem), which helped us to:
- keep disk space cheap
- make systems administration easier
- manage all traffic on the site easily and effectively
- track statistics painlessly
- make the site faster!
- Export journal comments in XML format
- Improved entry updating and editing pages
- Cracking down on comment spam
- Teaming up with EFF
- Working towards better abuse prevention practices
- 100MB disk quota for all paid and permanent accounts
- Get comments you posted e-mailed to you
- Image placeholders on friends pages
- New styles Haven, Opal, Unearthed, and Variable Flow
- FOAF and Atom syndication publishing
- Custom mood theme editor
- Recent Comments
- Quick Quotes in comments
- Comment "Quick Reply"
- Comment thread freezing
- Easier community transfers
- Revamped community membership invitations
- User Picture Icon comments
- The paid user "Express Lane" (thanks to Perlbal!)
- Entry previews when updating or editing
Last January we passed the milestone of the 2,000,000th account, now we host over 5,500,000 accounts. As you can imagine LiveJournal has grown up a lot this year! If this continues we can expect to see 7 or 8,000,000 accounts hosted by the time LiveJournal turns 6!
Obviously this growth quickly brings to light any weak spot in our architecture, but we've been able to tackle traffic problems and roughly anticipate future growth. Starting off the year we converted most of our user clusters to Master-Master pairs, and later on we added a few new user clusters and another global master database server. Later on we started migrating users from MyISAM to InnoDB database tables, and we purchased a few pairs of 64 bit database servers to lighten the load.
We had some vendor supply problems ordering 64 bit servers and the servers had some problems passing burn-in inspection, but now we have two pairs in production doing 65% of our database traffic without blinking an eye. All of the migration to 64 bit servers allows us to free up older hardware, making system administration easier.
During the year we also bought a couple of storage servers that could hold 16 250GB disks each, giving us 8 terabytes of disk space. We started using that to host pictures and static files, and freed up our NetApp to host other things.
For 2005 we're going to continue making optimizations and working towards high availability and reliability.
ScrapBook
Our major focus goal for 2004 (as evidenced by its many
news mentions!) was our new photo hosting service, which we've recently dubbed "ScrapBook". ScrapBook has matured quite a bit since our State of the Goat 2004, and has been made much more robust in the process. Because of ScrapBook development we can now offer Moblogging support, direct posting of galleries to LiveJournal accounts, copying of galleries and pictures between accounts, and many other features. We also wrote our own filesystem to store ScrapBook images on our storage servers, and now LiveJournal uses it to store all sorts of static files.Employees
Last year we touched on employees who had recently joined the fold, but this year was a little more subdued. In January we hired
marksmith to come code in-house for random projects, and now he's a development powerhouse, blazing through tasks and to-do lists left and right. Our newest systems administrator Matthew (matthew) is kicking ass (that is, when he isn't damaging vital appendages!), and is providing welcome job relief for lisa. whitaker made it out for another short internship over the summer, but both he and daveman692 left us to pursue their educations. They're working remotely for the time being, and they'll both be back next year. Last but not least we hired bleything to help out our support and abuse department and to act as one of the liasons between our volunteers and office developers.Plans for 2005
Looking ahead to the future, we plan to focus on a few key projects, while continuing to make improvements in all areas:
- Opening up ScrapBook — we're going to continue developing features, but our immediate goals are to finish the necessary documentation and offer accounts the ability to upgrade disk space.
- Journal usage statistics — our own web report statistics have become a lot easier to maintain, so we'll be investigating the best way to offer individual journal statistics.
- Event Subscription Notifications — a much talked about project on LiveJournal, we now have better means to bring it to fruition.
- Journal post categorization — we're in the planning stages of offering user-managed post categories, so that users can organize their journals better and readers can filter the content they want to see.
- Improved taskflow and usability — we've made a lot of improvements in our usability design practices while developing ScrapBook that we'll be using on LiveJournal projects.
