Archive of Our Own ^beta

Thanks to the wonderful generosity of fandom, we're upgrading the servers for the AO3! Our original two servers are being joined by four more, bringing more storage space, more power and more speed, all paid for through fannish donations to the OTW. We think this calls for a celebration!

The Archive will have some downtime while we swap over to the new servers. We know that this will leave some of you twiddling your thumbs - we'd love it if you could harness some of your unused fannish energy to help us name the servers! We're looking for fitting fannish names which will celebrate the wonderful fannish community and creativity the servers will host.

Names, names, names

We need names for our six servers and their trusty switch: check out Meet the Servers to find out more about them!

We want names which suit the machines and which reflect the fannish community they belong to: powerful, international, and brilliant! We'd like it if their names reflected the percentage of awesome women in fandom (although we think they have a whole variety of gender identities, just like fans).

How it will work

We'll have three periods of downtime while we install the servers - we'll be taking nominations for names during the first two, and we'll open voting during the third.

Nominations stage: 6 February from 18.00 UTC - 8 February 21.00 UTC

We invite all fans to submit the suggested names during this period! You can suggest just one name, or you can submit a full set of seven names (maybe with a theme!). Optionally, you can give a short explanation of why the name[s] would fit / where they come from. This would be particularly appreciated if the name is drawn from a less well-known source, particularly if it is a source not in English (we would LOVE some names from sources not in English and our nominations panel will be international, but sources in languages other than English are harder to research).

There are seven machines to be named- please let us know which machine you would like the name to go to. You can see their full details in Meet the Machines - in your entry please use the short names below:

• DB
• Rails1
• Rails2
• Original1
• Original2
• Storage
• Switch

You can submit names in the following ways:

• In the comments to our nominations posts on transformativeworks.org, LJ, DW, and IJ (coming soon).


• Via Twitter @ao3org


• Via our nominations webform (coming soon)

Nomination format

Nominations should be in the following format:

Name - Machine - Other optional comments / source

For example:

Elsie - DB - She was the best sister in 'What Katy Did', and the cleverest!

Shortlisting stage

A shortlisting panel made up of committee members from International Outreach, Accessibility, Design & Technology, Communications & DevMem will review the nominations and make up a shortlist. The shortlist will be based on the most-nominated names, but we will also give consideration to names which reflect under-represented areas of fandom.

Voting stage: 12 February from 9.00 UTC

During our second period of downtime, we'll post the list of shortlisted names on the OTW blog at transformativeworks.org and ask people to vote for their favourites. The seven names with the most votes will be our winners!

Announcements

The winning names will be announced by 14 February.

Prizes

The servers belong to fandom as a whole, and we hope that lots of people will join us in finding the right names for these fantastic machines. They are fandom's servers, and we think it's only right that fandom should name them.

The servers are the big prize, but we like giving out presents! We can't give goodies to everyone (alas), but every entry will be entered into a prize draw for lovely OTW goodies! Winners will be contacted through the same venue they submitted their entries, and will need to provide a valid postal name and address in order to claim their goodies.

Go forth and name!

We can't wait to see what names fandom comes up with! And once the servers have names of their own, it will be that much easier to create fanworks about them...

As we may have mentioned a time or two, we’re lucky enough to have been able to purchase some shiny new servers for the Archive! You can read all about them at new server announcement post! This is great news for all our users, as it means more power and more possibilities in future.

In order to move over to the new servers and perform associated maintenance and code changes, we will have 3 periods of downtime:

• Sunday 6 February, c. 18.00 UTC, for approximately 3 hours (What time is this where I am?)


• Monday 7 February, c. 17.00 UTC, for approximately 30 minutes (What time is this where I am?)


• Thursday 10 February, c. 18.00 UTC, for approximately 6 hours (What time is this where I am?)

We will send out an update on our Archive status Twitter account AO3_status before we take the site down, and again when we go back up again.

We know that many of you will miss us while we’re gone! To fill the time, we hope you’ll join in our Server Naming Festival! We’ll be posting more details of this on the OTW news blog at http://transformativeworks.org/ very soon.

Thanks for your patience while we perform these exciting upgrades to our system!

Love,

AD&T and everyone at the OTW!

We held our first AD&T meting of the new term on 22nd of January! The meeting was chaired by Elz, who is co-chairing this term along with Amelia.

We welcomed a whole bunch of new members along with several returning staffers. We have people who've been on AD&T for four years, and others who only joined the OTW a couple of months ago. One of the things we want to try to do this year is give people slightly more defined roles within AD&T, so that we can make the best possible use of everybody's talents. We're excited to be bringing some new people and skillsets on board.

2010 reviewed

We looked at back at our achievements from 2010, during which we rolled out icons, metatags, a new tag wrangling interface, all new archive CSS, username editing, challenge signups, assignments and matching, archive skins, work skins, advanced search, downloads, video embeds, sharing code, and kudos, among other things, and upgraded to Ruby 1.9, Rails 3, Unicorn and Nginx. We're proud of what we achieved and we're looking forward to seeing what we can get done in 2011.

News from the hiatus

We discussed everything that happened over the break, including the way we managed the work associated with hosting the Yuletide challenge, what we did well and what we can learn for next year. Yuletide a lot smoother than 2009, but there are still some improvements to be made.

New servers!

We have new servers arriving soon! You can read more about them in our New servers announcement and Meet the Machines. We'll be moving over to the new servers next week, and we'd love you all to join in helping us name them! Watch this space for more news.

Changes to our coding process

We're planning to move our code versioning system from subversion to git. This will make it easier for new coders to join, easier for coders to keep going without being hindered by code freezes, and easier for testers to have a regular schedule. We're hoping to deploy our current release in the next week or so, and then we'll move over.

Other business

We also decided on a few design queries, to clear up some old issues that were no longer relevant and to improve the layout of kudos.

News from our sister and sub-committees

• Coders are getting to grips with git in preparation for our move to the new system.
• Testers are getting starting with the first round of testing for the year, and getting stuck into some new training.
• Support have welcomed a bunch of new members and they are trying to catch up with answering ALL the tickets (there is a backlog from the hiatus - apologies if you have been waiting a little longer than usual for us to get back to you.
• Tag wranglers have been reviewing the wrangling team and making sure that all the fandoms on the Archive have some wrangling love.

If there are things you'd like to do or say, please share them in comments, via the AO3 support and feedback form, by volunteering, or in whatever medium you feel comfortable with. Everyone is welcome to this party!

This meeting round-up by new AD&T staffer Cesy!

We're super-excited to announce that we've just taken delivery of the brand-new servers for the Archive of Our Own! \0/ Our new, vastly expanded server setup gives us more power, more memory, and more room for expansion, and we are THRILLED to be able to have them. They have been purchased with the generous support of fandom - thanks to all our awesome members and donors!

What we're buying

Check out Meet the Machines! for full details on our server family. Our original server setup had just two servers: a main database server and a backup. We're adding three more servers to this, plus a storage server and a switch, so our new setup will look like this:

Money matters

We bought our servers from an awesome company iXSystems, who specialise in open source applications and were really responsive to our needs. They gave us a great deal: our DB server was US$5,554, and the Rails twins were US$4,193 each. The storage server was US$3,014, and the switch that lets them all talk to each other was US$280. In total we spent US$17,234 on new hardware, which is a great price for the kind of machines we've invested in.

Buying new servers also means paying more hosting costs - we own the machines, but we hire space for them with a colocation company who hook them up to the internet. When we had just two servers, we were paying US$228 per month for our 2U of rack space and bandwidth. With the new servers, we'll be paying US$800 per month for 8U of rack space plus extra bandwidth. Because our new servers are such powerful beasts, the company are running a brand new power cable just for us!

All of this investment was paid for by donations from fans! The AO3 is funded by donations to our parent org, the Organization for Transformative Works; the success of our last membership drive, which raised US$18,308.46, gave us the financial confidence to make this investment. Thank you to all our supporters!

If you'd like to support the ongoing development of the AO3 and the OTW, you can donate at any time. A donation of just US$10 makes you a member of the OTW (if you so desire) and confers voting rights in Board elections. Donations over US$50 also qualify for a variety of nommy premiums.

What this means for Archive users

More speed! More features! More fanworks! Once we're up and running with this setup, we'll be much better able to handle high loads, so you'll be seeing much less of our sad 502 page! We'll also be able to start thinking seriously about hosting multimedia fanworks - we still have work to do before we can start hosting fanart, but it's definitely in sight now! We'll also be able to start moving towards implementing Subscriptions on the Archive - one of our most-requested features.

Server installation: Archive downtime

In order to install our new servers and move to the new setup, we will have extended Archive downtime. We're still coordinating with our colocation company, but we're expecting to have two periods of downtime in the week beginning 30th January. All Archive user accounts will be emailed as soon as we know the exact timing (now is a good time to check that your registered email is still correct) and we'll be posting on our news blogs and to our Twitter accounts @ao3_status and @ao3org as more news arrives.

Server naming festival

These servers belong to fandom, so it's only right that fandom should get to name them! During our downtime, we'll be running a server naming festival and soliciting suggestions for names for our servers (and their trusty switch). Meet the Machines to find out what they are like and start thinking about what they should be called! More information on the competition coming soon!

SERVERS YAY!

We are EXCITED! Thank you, fandom!

Thanks to the wonderful generosity of fandom, we've been able to buy a beautiful new set of servers! Meet the machines below!

The DB Server - aka "The Beast"

Our new database server is a serious badass. Built for us by open-source server specialists iXSystems, the DB server is an iX1204 powered by 2 latest-generation Intel 6-Core X5650 processors, and 48GB of RAM. This server will also have 2 Intel X25-M 80GB SSDs (solid-state drives, which have no moving parts) to make reads off the database crazy fast.

The Rails Twins

A set of identical twins, our speedy and strong new Rails servers will split the load of running the Archive app. These two are from the same family as the DB server: iX1204 models, each with 2x Intel Quad Core E5620 processors, 24GB of RAM, and 4x146GB 15k RPM hard drives.

The Originals

Our trusty veterans! The original servers are another set of identical twins: Hewlett Packard ProLiant DL360 G5 models each powered by 2 Quad-Core Intel Xeon E5420 processors. They each also have 16GB RAM, and 4×72GB 15k RPM hard drives. For the last year, these two have had to carry the entire load all by themselves, but now they will both be moving into new roles:

1. One of these machines will now serve as our friendly memcache server and it runs background and other odd jobs.


2. The other one will become the DB's slave. *koff* (Of course the more vanilla-minded among us can think of it as being the DB's loyal sidekick!) It will also run our search engine.

The Storage Server

The QNAP TS-809U-RP is a 2U rackmount Network Attached Storage (NAS) appliance, intended for what the manufacturer describes as "massive data sharing" -- mmmhmm! We're loading this baby to capacity, with 16TB (8x2TB Hitachi Deskstar 7200RPM HDDs) in hot-swap bays to give us greatly increased data storage/backup capability.

The Switch

All of our fabulous servers will communicate with one another through an HP Procurve V1910 Gigabit Layer 2+ smart-managed network switch. The switch has 16 auto-negotiating 10/100/1000 ports and 4 true Gigabit SFP ports.

Welcome to Revision 3520, up from 3483. This is actually a few different releases bundled together, because we had a little flurry of activity over the holiday period which we didn't post release notes for.

Most Archive users will have noticed that the Archive was experiencing very heavy loads during the holiday period, as Yuletide and other holiday challenges brought high traffic to the site. We know that our sad little 502 page was showing hir face more than any of us would have liked! However, our coding and Systems teams deserve a MASSIVE shoutout for their tireless work keeping the site up and dealing with the performance issues. They worked right through the holiday to keep pushing code improvements and tweaking server setups. We'd especially like to thank sysadmin Sidra for the incredible amount of effort she put into coaxing our poor beleaguered servers into working: her work helped get the number of 502s down from over 9000 (on 25 December) to just 9 a few days later. Go Sidra!

If you're wondering whether the Archive will be this slow NEXT year, then we hope the answer is no! The stress test this year taught us some more about how to configure things for the best possible performance. Even better, we're about to buy some new servers which our hosting service described as 'beasts'. More news on that coming soon...

Highlights

Static pages for collections

As part of our performance enhancements, the wonderful Elz worked till the eleventh hour to put in static pages for collections (you may have seen us mention these a time or three on @AO3_Status). These are pages where the whole page is created once and then saved, so it is served up quickly for the next user. They can't have any dynamic information (comment counts, hit counts), but they are MUCH faster at peak times.

All collections have static pages - the url is http://archiveofourown.org/static/collections/COLLECTIONNAME. They generate when people access the urls, so they are only noticeably faster for heavily visited collections. Check out a few of the following:

• DSSS_2010
• yuletide2010
• yuletidemadness2010

Share via Twitter (or not!)

We've noticed lots of our users sharing recs for AO3 works via Twitter. We thought you might appreciate a quicker way of doing this, so we've added a Twitter button which fills in the title and url for you! Just click the 'Share' button to see the button.

We know not all our users will be comfortable with a feature that makes it really easy to share links on external services, so we've included an opt-out - just check out your preferences to select this. If you opt out, it will disable the Twitter option on your works now, and if we introduce sharing options for other external services in the future it will also disable those. (However, opting out won't actually prevent people from sharing links to your work anywhere, it will just make them do more of the work themselves. If you want a higher level of control over who sees your work, you can restrict your works to Archive users only.)

Improvements to kudos

Since we introduced the 'kudos' option a couple of weeks ago, over 51,000 kudos have been left and we've had a ton of great feedback about it. We've fixed a few things that were not quite right with the feature: it's now 'kudos' everywhere (no weird singular usage) and you can opt out of kudos notifications without turning off your comment notifications.

Known Issues

See our Known Issues page.

Release details

Features

• Added server-relieving caching options to collections and challenges
• Added static pages for collections (fandom pages, work pages, etc).
• Added ability to leave kudos from static work pages
• Ensured that static pages would persist across deploys.
• Added 'Share on Twitter' button
• Added the ability to opt-out of allowing sharing of your works on external services (e.g. Twitter).
• Changed kudos-related wording in notification emails and on work pages to take care of the singular/plural issue - no more strange singular 'kudo'!
• Removed revision number from page titles
• More automated tests, continuing our mission to test ALL THE THINGS.

Bug fixes

• Removed author names from "share" box for anonymous works (whups)
• Fixed byline bug causing names of anonymous authors to appear in static files and downloads
• Made it possible to edit a work in a closed challenge without being kicked out of the collection
• Fixed collection links in anonymous work blurbs showing up as unparsed HTML code
• Fixed a bug where you couldn't edit tags on a work if they were related to a meta tag
• tweaked collections pages for display in Internet Explorer
• Cleaned up story notes section, fixing issues which were causing links to endnotes to be encased in invalid HTML and thus look weird.

Systems changes

• Added option to turn off kudos notifications
• Change thinking sphinx delta delayed jobs into an hourly cronjob (relieved load on the server)
• Changing reindex to every two hours (relieved load on the server)
• Gave delayed job its own restart
• Made it possible to run deploy quick on otw2 server (saves some work during repeated deploys)
• Updating static sweeper

Do you love the Archive? Do you have skills in database analysis? Would you like to make these great two tastes taste great together?

We're looking for a MySQL database developer who can help us optimise our database performance on the Archive of Our Own. The Archive is run on MySQL 5.0 and Rails 3, but you don't need to know Rails to help -- we would really like someone who can look over our database query logs and help us spot inefficiencies!

You'd be joining a team of dedicated and passionate volunteers with a love of fandom and tech-geekery. We're looking for someone with the time and skills to help us evaluate and optimise our current setup — this can be a short-term commitment, although if you are able to join us for the long haul your skills will be very welcome.

If you're interested, please contact our Volunteers and Recruitment Committee using 'MySQL database developer' as your subject line, and let us know your skills and experience.

Your Archive needs YOU!

2010 was a wonderful year for the AO3! It was our first full year in Open Beta! We ran two big challenges and lots of smaller ones! We filled up our servers, introduced a bunch of new features, made 1451 code commits and got shout-outs all over the web.

Our team had an eventful year! People moved countries, travelled across continents, changed jobs, got PhDs and had babies. Oh, and we built this great Archive for fandom to enjoy! We feel a great sense of achievement!

We are incredibly grateful to everyone who worked on the Archive during 2010: coders, testers, sysadmins, tag wranglers, support team, designers, and many more!

We're also hugely grateful to all Archive users and members of the OTW for your great feedback, your generous financial contributions (which will pay for beautiful new servers this year), and most importantly, your fannish energy and creations, which are the true heart of the Archive. At the close of 2010 we had:

• 7,927 fandoms
• 11,504 users
• 128,806 works
• 4,949 series
• 591 collections
• 164,450 tags (of which 100,046 are 'canonical' and can be filtered on, 79,294 are character tags, and 45,569 relationship tags)
• 158,639 comments
• 69,153 bookmarks
• 40,299 kudos

We think this is an AMAZING output for just over a year of real usage! GO GO FANDOM!

We're looking forward to seeing what awesome new fannish output 2011 brings! Thanks everyone for your support! Happy New Year!