AO3 News

OTW Board response to concerns about the meta decision

Published: 2013-02-25 13:17:12 -0500

There has been a very active and thoughtful response to our recent announcement in favor of allowing meta on the AO3. We'd like to take this opportunity to express our gratitude to everyone for raising their concerns, showing their support, and otherwise engaging with us as we work to define our policies, refine our processes, and improve our communication. In addition, we'd like to respond to a number of the issues raised and clarify how this decision was reached and what the process will be from this point forward.

For the purposes of this and the previous post, the term "meta" refers to nonfictional fanworks in all media. While text-based nonfiction fanworks have been a frequent focus, this decision and the surrounding commentary is meant to encompass fanworks in all media; this is one reason why multimedia hosting, posting, and filtering will be referenced frequently in conjunction with the decision to support meta.

There is still a long way to go before meta can be fully supported on the AO3, and we will address a number of the concerns about implementation and timing below. Determining how meta should be supported — for example, the details of how multimedia hosting on the AO3 will ultimately look — is a matter for our committees and users to decide through committee collaboration and user input. However, determining whether supporting meta on the Archive is consonant with the OTW’s mission falls squarely within the Board’s purview and duty.

History of the discussion

When the initial question of meta was posed to Board, it was framed as a request for clarification on whether meta fell under transformative works as we defined them for the AO3, and how to proceed with reports of meta as a violation of our Terms of Service (ToS). The Board voted last August to send the meta issue back to the committees for more discussion, in the hope that the committees could work out among themselves issues that the Board had found insoluble. The decision called for balancing the competing concerns of several committees, and the Board had been unable to reach a satisfactory agreement. However, the execution of that plan dragged on for months as we dealt with Board member hiatuses, resignations, and appointments on top of other day-to-day business, and the vote was never put into action.

When the Board reconvened in 2013, we initially had intended to continue with the plan set out by the 2012 Board, but we quickly realized that — partly as a result of the Board’s dramatically changed composition and partly because of a new focus on clarifying the Board's purview — we no longer felt it to be the best course of action. We looked at the conversations that had been happening within and outside the organization for the previous six months and came to the conclusion that it was in the best interests of both our users and our personnel that a basic decision be made as soon as possible, rather than occupy staff and volunteer time in further stretching out a question that we felt it was our responsibility as Board to settle: the question of the scope of the OTW and AO3's missions with respect to meta.

We had many users who had been waiting all that time to find out if their meta could stay on the Archive, and several committees who needed a determination in order to perform their duties. We took a fresh vote, which was unanimously in favor of interpreting the OTW and AO3's missions as inclusive of owing meta the same protections and support as other fanworks. Once that vote had been taken, sending the issue back to committees for a discussion that would not have changed the Board’s stance would have been disingenuous. We felt it was preferable to state a firm decision and engage the committees in determining how best to carry it out.

We are aware that the Board's decision seemed very abrupt to people both inside and outside the OTW, and we acknowledge that more transparency would have been preferable. The Board’s overall workload and the emotional burnout many of us have experienced as a result of the length and intensity of the meta discussion were obstacles that prevented us from communicating effectively. We regret our shortcomings in this area and will strive to do better in the future; we are working to reduce workload and burnout and clarify policies and purview in an effort to prevent this from recurring.

We are committed to fully engaging committees and users in determining how the decision will be implemented, and a revised Archive TOS and FAQ are currently being drafted under the leadership of the Content Policy Workgroup. As with other TOS and FAQ revisions, they will be posted for public comment before they are formally adopted.

Replies to some questions and concerns

We recognize that this decision will not be popular with all users, members, or even OTW personnel. Conversely, the choice to allow meta — and turning over the ability to define and craft specific policy to our committees — is a decision many support. The concerns raised by those leaving comments are ones the Board spent a great deal of time discussing, and we are happy to share our reasoning and to continue answering questions to the best of our ability. Here are some responses to common concerns and questions:

  • Meta does not require new code to be hosted in its bare form — unlike image or video hosting no new code is required for a basic level of service. For example, a nonfiction essay can be uploaded just like a fictional story, or a meta comic can be linked just like a fictional one is now, or a vid focused on commenting on the canon can be embedded like vids that build fictional narratives currently are. While there are ways the AO3 could be better organized to deliver meta, a basic level of hosting is already available.
  • The AO3 is intended to eventually have filtering based on work type/medium, allowing meta to be found and filtered. The intention is to expand the AO3 functionality to better host non-textual fanworks (e.g. vids, podfics, art, etc.), and the most-requested behaviors with respect to meta (filtering, tagging, etc.) all intersect with what will be in place for multimedia hosting and posting.
  • Refusing to host meta and waiting until we have sufficient code for works types would unduly punish users who have already posted meta works in good faith. In addition to posting meta based on good-faith interpretation of the TOS, users have been posting many types of works the AO3 is not strictly prepared to deal with on a technical and usability level, which includes meta of all media and most non-textual fanworks. Allowing and encouraging users to post all types of fanworks has been a cornerstone of the AO3's philosophy as an archive, and it would be disingenuous and unfair to punish one type of fanwork or creator but not others on this basis.
  • While text-based meta faces much less legal challenge than some other fanwork types, it still faces other challenges such as loss of hosting due to failing archives or discontinued blogging platforms. Non-text-based meta, such as meta art and vids, shares many of the same legal challenges as other non-text-based fanworks.
  • Fans should be able to archive all their fanworks together. Besides this general principle, there are specific instances of at-risk archives that include meta fanworks. Grandfathering in previously posted meta or disallowing meta except for that taken in through Open Doors leads to an inconsistent policy likely to cause confusion, conflict, and difficulty in enforcement.

We hope this answers some of your comments and concerns. We welcome further input and look forward to working with our personnel and our users in continuing to welcome a broad range of fannish endeavors under the OTW umbrella.

Comment

February Support Live Chat

Published: 2013-02-18 16:03:18 -0500

Hi! Support here, again! In fact, Support is always here--when you submit a ticket through the Support and Feedback form we'll respond as soon as possible to register your feature suggestion, pass your bug report on to our coders, or do our best to help you out with a problem. However, when it comes to explaining how to do things or why something doesn't seem to be working right, the formal back-and-forth emails of a Support request aren't always ideal.

After receiving positive reviews of our last chat in November, we're going to regularly have Open Chat sessions with the Support Staff in our public chat room (the link will be made available on the day of the chat). The first of these will be this coming Saturday, February 23, 2013 at 16:00:00 UTC lasting through this Sunday, February 24, 2013 at 04:00:00 UTC. Members of Support will be available to interact with you one-on-one in live chat. See what time that is where you live. We are going to try to have future sessions at different times to make sure we eventually cover all time zones. If you can't make it to this one, keep an eye out for the next!

EDIT: We're closed, for this month. A hearty thanks to everyone who came! We will be doing this again!

If you're having a problem using the Archive, want help trying something new, or would like an explanation of one of our features, please drop in and talk to us in person!

Some guidelines, just to keep things running smoothly

We don't have a fancy presentation or material prepared--there are plenty of FAQs, tutorials, and admin posts for that. The point of live chat is to talk with you, not at you. We're happy for you to drop in and say "hi", but it's even better if you drop in and say, "Hi, what's up with my work that won't show as complete even though it is?!"

As Support, our function is to help users with bugs and issues, and pass reports on to our Coders and Systems team, who actually keep the place running. This means that policy questions are way over our pay grade. (Just kidding--none of us get paid!) So, if you have questions or comments about AO3 or OTW policies, good or bad, Support Chat isn't the right place for them. If you do want to talk to someone about policy issues (meta on the Archive, philosophical issues with the tagging system, category change, etc.) we can direct you to the appropriate admin post or contact address so you can leave feedback directly for the people dealing with the area of your concern.

Additionally, if a question looks like it might violate a user's privacy to answer (if it needs an email address or other personal information, for example) we may not be willing to work with it in chat. In those cases, we'll redirect a user to the Support Form so we can communicate via email.

So, now that that's out of the way, what kind of things are we going to talk about?

Live chat is best for questions of a "How do I...?" or "Why does it...?" nature.

For example, you might have been wondering:

  • How do I use the new search and browse system to find a certain type of work?
  • I'd like to run a challenge, but I'm not sure how to do what I want.
  • For that matter, where did my work submitted to an anonymous challenge go?!
  • I want to post using formatting the Rich Text Editor won't give me. How do I do it using a work skin?

We'd be happy to help you with any of these questions, and anything else you're having trouble doing or would like to try doing with the Archive.

Comment

Archive of Our Own Newsletter - January/February 2013

Published: 2013-02-17 14:39:34 -0500

Welcome to the January-February Newsletter! We hope you had a good holiday season and are having a happy new year! Despite the term break in December, January was busy busy busy with upgrades and releases for all. February is also turning out to be a big month with releases, spotlights, tag wrangling fixes, and header posts galore! Here's what we've been up to:

All the Archive news that's fit to print!

2012 was full of Archive milestones. In November we passed the 500,000 work mark in 10,000 fandoms. In December, the Archive passed 100,000 users. Check out this post for further milestones that we passed in 2012.

Tag Wrangling shared their process for wrangling Additional Tags and why Additional Tags are not as un-wranglable as one might think.

Mini-Release 0.9.4 went live with a small number of bug fixes. We were also very happy to bring back invitation requests in December. Release 0.9.5 and its redux went off without a hitch.

Fandom Tags are now alphabetized regardless of articles. Wranglers now have the ability to assign a sort name different from a display name, making it easier for us to wrangle and browse fandoms!

We recently posted a Spotlight on Systems Committee. If you've ever wondered what goes on behind the scenes to support the technical systems of the Archive, read all about it here.

AD&T are hard at work on a redesign of the site header. Check out our little preview and tell us what you think.

And finally, the Board approved meta hosting on the Archive. Please see this post for details on how this decision affects you, the Archive, and the Archive staffers.

Adventures with Support

Things are keeping busy in the Support world. We've got a new co-chair, and the training is letting us solidify all of our training. We're looking forward to a solid year with proactive communication with both our fellow committees and our users!

Open Doors Update

Open Doors is still working toward an automated import for the 852 Prospect Archive and recently held two open house chats (read more here). In the meantime, we've opened up manual importing by inviting all 852 Prospect authors to the Archive. Check out Open Doors' post for further instructions on manually importing works from that particular Archive.

AD&T Committee business of note

We're excited about the upcoming year and are looking forward to everything we'll be sharing with you. On a more serious note, we recently reviewed our emergency plan in the event that our site is compromised and requires an emergency shutdown.

Support Committee business of note

Support will be hosting a Live Chat February 23rd-24th, from 4pm to 4am UTC (What time is that for you?) As Support, our function is to help users with bugs and issues, and pass reports on to our Coders and Systems team, who actually keep the place running. So, if you have questions or comments about AO3 or OTW policies, good or bad, Support Chat isn't the right place for them. If you do want to talk to someone about policy issues (meta on the Archive, philosophical issues with the tagging system, category change, etc.) we can direct you to the appropriate admin post or contact address so you can leave feedback directly for the people dealing with the area of your concern.

Tag Wrangling Committee business of note

We've clarified some major weirdness in our guidelines regarding AU tags and inconsistencies regarding the canonicals for Original Characters in Relationship tags.

Apologies

We've had some scheduled downtime during the past couple of weeks. Each time was for two upgrades and some site maintenance to build a better Archive. Apologies to anyone who was inconvenienced by this! We also received reports from some users that Avast was sending out malware warnings when users tried to access the Archive. The cause of this malware warning was external and no cause for alarm.

Questions? Comments?

We welcome feedback from users! If you have questions or comments, feel free to leave them in the comments of the latest news post, or send in a Support request (if you're reporting a bug, please send that to Support, as they're super efficient - comments on our news posts sometimes get overlooked).

Comment

OTW Board Approves Meta Hosting on the AO3

Published: 2013-02-15 14:05:33 -0500

After a long period of discussion, the OTW Board has voted to allow the posting of meta on the Archive of Our Own. We considered a range of issues while making this decision, including how this move would fit into the overall mission of the OTW, the technical and financial resources required, and demand from users of the Archive and members of the OTW. We determined that there is already a demand for meta on the Archive, and that this use of our resources is consonant with our purview and mission.

We're aware that this decision has taken some time, and we sincerely apologize for the delay. We had hoped to reach a decision sooner, but the complexity of the discussion meant we needed to think carefully about the issues. As the term of some OTW Board members ended while the discussion was ongoing, we also needed time for the new Board members to get up-to-speed with all the issues involved.

What will happen next?

Agreeing to include meta on the Archive is just the first step in this journey. The Board will now work with all related committees to define exactly how meta will be handled. Our committees, including AD&T (which will be doing the work on the technical side), Abuse, Support, and others, will be working with our Content Policy workgroup to design a workable policy.

One of the main tasks ahead of us is to agree on some definitions and policies. We need to agree on definitions that are usable and enforceable. While any category is inevitably fuzzy, we want to preserve the Archive as a site for fanworks (so for example, we don't want it to become a general blogging site). Once we've agreed on these definitions, our committees will have a whole range of tasks ahead of them, including:

  • Drafting revisions to our Terms of Service and FAQ. Revisions to the Archive TOS will be subject to a public review period (as detailed under Section IB of the TOS) before becoming final.
  • Determining technical plans for making meta more accessible. We are already planning changes to posting and browsing on the Archive to allow for multimedia hosting. We do not expect meta to require any additional coding to implement beyond what will be required for these changes, and allowing meta won't change the existing prioritization of these features, but we will need to factor it into our design.
  • Determining tagging policies to allow for multimedia and meta browsing.

What will be allowed?

Our Content Policy workgroup will be posting guidelines on what will fall under the 'meta' category and the policies which will apply to it in the next two weeks.

What does this mean for me?

Going forward, we hope that this will mean you can find and enjoy fannish meta more easily (and screen it out if you're not interested).

If you currently have meta posted on the Archive, or you plan to post some in the near future, you should be aware that our policies are still being finalized. As action on existing meta posts was suspended while Board deliberated on this issue, in the coming months some users may be contacted in connection to how their posts fit the new policies. We recommend that users wait until these policies are made public before putting a lot of effort into new meta posts. However, we hope that, long term, meta writers will feel their contributions to the archive are welcome and can join other fanworks in finding an audience at the AO3.

Thoughts?

If you have thoughts and feedback you'd like us to consider, we ask that you comment here on the AO3 version of this post, to make it easier for the various committees involved to answer you and collate your replies.

Comment

Category Change Says: "We're working!"

Published: 2013-02-13 16:54:39 -0500

This is a short update from the Category Change workgroup. You can read about what our work entails and how we were formed in our introductory post. As we want to keep users in the loop, we wanted to provide some information on what we’ve been doing since then, and what we’re planning to do in the near future.

We began by compiling user feedback we received either in comments made to our last post or directly through the Category Change Contact Form. We then grouped together similar ideas.

After analyzing the feedback through discussion on our mailing list and in several chat meetings, we have identified the issues users have with the current Media categories, as well as their expectations when browsing and filtering. This information has informed our discussions and has been vital to understanding what we want from the new categorization system.

We had a brief end-of-term hiatus from December 17th to January 4th.

Since the beginning of the 2013 term, we have been discussing the first draft of the new categorization system. Once this task is done, we will consult with the OTW committees who would be affected by the changes and solicit feedback from them.

After we have reassessed our initial proposal in light of this feedback, we will make the revised proposal public and ask for users' feedback. We know that this could impact user experience in a big way, so we want to make sure that we have heard the users' concerns and ideas before moving forward with a final proposal.

You’re welcome to comment on this post (at any of its locations) with ideas, feedback or opinions, or you can send them to us through the Category Change Contact Form.

Comment

Sneak Peek: Our Shiny New Header!

Published: 2013-02-12 13:13:52 -0500

As we mentioned in our August newsletter, super coder Sarken and the rest of AD&T have been working hard on a redesign of the AO3 header. This new header will include drop-down menus to make site navigation easier. We're excited about our new header and we'd like to give you a little sneak peek before it goes live!

Presenting: the new header!

This is what the new header looks like without any expanded menus:

Note that the new header does not in any way affect the organization of the dashboard. The header will not affect the layout of any other pages either.

To make it easier to navigate the site, the header has four main drop downs: Fandoms, Browse, Search, and About. For example, clicking on "Fandoms" will show you this:

The "Fandoms" menu is organized by media type. Clicking on a media type will lead to the media type's main listing of all fandoms that belong to that media type. "Browse" and "Search" offer ways to look for content by works, people, bookmarks, tags, and collections - options that are available on our current header but which will now be condensed into two menus. The "About" menu will have links to the blog, the FAQ, and the volunteer page. These four main menus are available to all users.

To sign in, look to the top right corner of the page:

Clicking on "Log In" in the upper right corner will show the log in menu - now a drop down menu, instead of a menu that goes across the top of the page.

If a user is accessing the site from the homepage or from other pages (such as a parent fandom page), there will be an option to access dashboard features:

Clicking on "Hi, [user]!" shows a menu for users to get to specific areas of their dashboard without having to go to their main dashboard page. This menu will not include everything on the dashboard page, just the options shown here: My Dashboard, My Subscriptions (#), My Works (#), My Bookmarks (#), My Collections (#), My Sign-ups (#), My Assignments (#), My History, and Preferences.

Links with (#) will only show up if a user has something saved or uploaded under that link. For example, if a user has fourteen bookmarks uploaded, the link will read "My Bookmarks (14)". If a user has no bookmarks, "My Bookmarks" will not show up. The same goes for subscriptions, works, collections, sign-ups, and assignments. "My History" will only show up if you have history enabled under your preferences.

The header will also be visible on the mobile layout:

As we can see here, the "Fandoms" menu has the exact same options for mobile users. The mobile header reflects the new header; mobile users will have access to the same menus and options as desktop users.

How you can help us

We've been making sure that this header works without Javascript. However, to make this new header as accessible as possible, we'd like feedback from users who use screen readers and other assistive technology. If you're willing to help us out, please contact Support.

When do we get it?

We're hoping to roll out the new header by the end of February. Stay tuned for further details when it goes live!

Comment

Spotlight on Systems!

Published: 2013-02-10 09:11:30 -0500

The OTW's Systems teams work behind the scenes to support, manage, and maintain all the technical systems needed to run the OTW and its projects, such as the Archive of Our Own and Fanlore.

Systems' work mostly happens behind the scenes, but they are BUSY, fielding requests from all parts of the organization and working hard to keep all our sites up and responsive. Systems team members have to be 'on call' in order to deal with emergencies at any time of the day or night: if the Archive of Our Own goes down, it's Systems who fly to the rescue (while over 130 thousand users wait impatiently!).

2012 was a particularly demanding year for Systems because of the speed with which the OTW and its projects grew. Over 2,970,103 people now access the Archive of Our Own in the course of a month, up from 808,000 a year ago. Meanwhile, Fanlore has also grown, passing 400,000 edits in 2012, and other projects have continued to develop. Managing these projects and their volunteers also requires technical resources, and Systems have helped the OTW to transition to some more effective tools over the past year.

Systems highlights

Over the course of 2012, Systems:

  • Handled 557 requests from around the organization \0/
  • Transitioned the OTW website and some related tools and projects to a new host with a third party Drupal vendor, who will provide much-needed technical support for these tools.
  • Dealt with the performance problems on the Archive of Our Own, stepping in to implement major performance enhancements and keep the site up.
  • Researched, bought and installed 3 new servers to host our projects and cope with the ever-growing demands on the Archive of Our Own.
  • Researched hosting options and installed two additional servers after a kindly benefactor donated them to the OTW.
  • Set up new hosting and tools for our volunteers to use, including new hosted environments for our coders, so that coders don't have to install the Archive code on their own machines.
  • Kept everything up and running, with amazing patience and good humour in the most stressful situations.

Find out more!

James from Systems has written up an amazing and detailed account of the main work Systems did in the course of 2012. To get some in-depth insight into the amazing work Systems do, check out: A year with the Systems team

If you're technically minded, or curious about how much hardware is needed to run the Archive of our Own, you'll also enjoy James' posts on our changing server setups over the past year, and our technical plans going forward:

January 2012 server setup
January 2013 server setup
Going forward: our hardware setup and technical plans

Thank you!

Systems do an amazing job of juggling their many responsibilities. We really appreciate their work - thanks Systems!

Comment

Systems spotlight: A year with the Systems team

Published: 2013-02-10 09:08:37 -0500

The Systems team is responsible for all the ‘behind the scenes’ parts of the OTW’s technical work; in particular, we maintain the servers which run all the OTW’s projects, including the Archive of Our Own and Fanlore. As the OTW has grown, so has our job, and we’ve been very busy over the past twelve months!

This update gives an overview of some of the key changes we’ve made this year. While it doesn’t cover every detail, we hope it will give our users a sense of the work we’ve done and (importantly) where we’ve spent money. We’ve included quite a few technical details for those users who are curious, but hope that non-technical users will be able to get the gist.

January 2012

At the start of January 2012, we were maintaining 12 servers: 6 physical machines and 6 virtual ones. You can see more details in January 2012 - our server setup.

February

The Archive of Our Own was suffering performance problems as more users joined the site. We spent time working to make things more reliable and balancing unicorns. We had to disable our online web tracking system (piwik), as it caused slow responses with the Archive. Although our work helped performance, server OTW2 (running Archive-related services) started collapsing under the load.

March

We implemented a system which killed off runaway processes that were created when users were downloading works from the Archive of Our Own.

April

A bug caused Linux systems to have performance issues when its uptime reached 200 days. As our servers all run Linux, we were affected. A new kernel and a reboot on our Linux-based servers fixed the problem very quickly \0/.

June - a month of many happenings!

Our long-serving staffer Sidra stepped down as Technical lead and joint Chair of the Systems group. We have missed her and hope to see her rejoin us in the future.

In response to the rising numbers of visitors to the AO3, we upgraded our colocation bandwidth (the amount of network traffic) to an unmetered 100Megabits/second, which cost an additional $100 per month.

Demands on our servers were also increasing behind the scenes, as the number of coders and the complexity of the Archive meant that the webdevs (used by our coders to develop new code) and the Test Archive, where we test out new code before releasing it onto the live site, were unusable. We upgraded the servers these were hosted on, which increased our virtual server bill by an additional $200 per month.

We decided that we had reached a size where it would be worth buying our own servers rather than using virtual servers for the webdevs. We investigated the costs of buying new servers, but happily later in the month, two servers were donated to OTW. We then started the long task of finding a suitable hosting provider, as the servers were located a long way from our main colocation host and shipping costs were high.

Performance issues on the Archive of Our Own were at their height during June, and we spent lots of time working to address these issues. Some parts of the site were unable to cope with the number of users who were now accessing the site: in particular, we had significant problems with server OTW5 and the demands created by the tag filters, which required a lot of space for temporary files.

In order to reduce the demands on the servers, we implemented Squid caching on the Archive, which alleviated some of the problems. On the 13th of June we decided to disable the tag filters and the Archive became significantly more stable. This reduced the amount of hour by hour hand holding the servers needed, giving our teams more time to work on longer-term solutions, including the code for the new version of the filters.

July

The first of July brought a leap second which caused servers around the globe to slow down. We fixed the issue by patching the servers as needed and then rebooting - with just half an hour turnaround!

We consulted with Mark from Dreamwidth about the systems architecture of the Archive. We got a couple of very useful pointers (thanks, Mark!) as well as concrete advice, such as increasing the amount of memory available for our memcache caching.

A disk died in server OTW2 and a replacement disk was donated by a member of the Systems group.

We started to use a large cloudhosted server space to develop the new system that would replace the old tagging system. This machine was not turned on at all times, only when the developers were coding, or when the new system was being tested. Hiring this server space allowed us to develop the code on a full copy of the Archive’s data and do more effective testing, which more closely replicated the conditions of the real site. Since the filters are such an important part of the AO3, and have such big performance implications, this was very important.

We upgraded the RAM on servers OTW3, OTW4 and OTW5. We replaced all of the RAM in OTW5 and put its old RAM in OTW3 and OTW4. This cost approximately $2,200 and gave us some noticeable performance improvements on the Archive.

We also upgraded the main webserver and database software stack on the Archive.

And lastly, it was SysAdmin Day. There was cake. \0/

August

We started using a managed firewall at our main colocation facility. This provides both a much simpler configuration of the main network connection to the servers, and allows secure remote access for systems administrators and senior coders. It costs an additional $50 per month.

A typo in our DNS while switching over to this allowed a spammer to redirect some of our traffic to their site. Happily we were able to fix this as soon as the problem was reported, although the fix took a while to show for all users. The firewall changes also caused a few lingering issues for users connecting via certain methods; these took a little while to fix.

September

We purchased battery backup devices for the RAID controllers on OTW1 and OTW2, meaning their disk systems are much more performant and reliable. The batteries and installation cost a total of $250.

A hardware based firewall (Mikrotik RB1100AHx2) was purchased and configured for the new colocation facility, costing around $600.

Systems supported the coders in getting the new embedded media player to work on the Archive.

We also migrated transformativeworks.org, Elections and Open Doors to a third party Drupal supplier.

October

The donated, dedicated hardware for Dev and Stage (our webdev and test servers) were installed in their new colocation site, after long and hard hours spent investigating options for hosting companies and insurance. After installation the initial configuration required to run the Archive code was completed. These machines support a larger number of coders than was previously possible, giving them access to a hosted development environment to run the Archive. The hosting cost is approximately $400 per month.

We were able to decommission the virtual machine that was the Dev server (for webdevs) immediately, saving $319 per month - so the new hosted servers are only costing us about $80 more than the old setup. Considerable work was done to get Elastic Search working in our dev, test and production environments (production is the live Archive).

November

We were running out of disk space on OTW5, which is critical to the operation of the Archive. We purchased a pair of 200GB intel 710’s and adapters which were installed in OTW5, for a total cost of $1,700. These disks are expensive, however they are fast and are enterprise grade (meant for heavy production use) rather than home grade, which is significant on a site such as ours. Solid state drives (SSDs) are dependent on the amount of use they endure and the 710’s are rated at an endurance of 1.5PB with 20 percent over provisioning (meaning they will last us far longer than a home grade SSD).

At roughly the same time, the tag filters were returned to the Archive using Elastic Search. There was much rejoicing.

December

We were waiting until the tag filters were back in place before deciding what servers we would need to buy to provide the Archive with enough performance to continue to grow in the following year. After discussing budgets with Finance and Board, we put a proposal through for three servers for a total price of $28,200. We arrived at this price after checking with a number of vendors; we went for the cheapest vendor we were happy with. The difference in price between the cheapest and most expensive vendor was $2,600. The servers will be described in January 2013 - server setup.

Having bought the servers, we needed to host them. We had to decide whether to rent a whole 19-inch rack to ourselves or to try and and squeeze the servers into existing space in our shared facility. In the long term we will likely require a 19-inch rack to ourselves, but as this will cost about $2,100 per month we worked hard to find a way of splitting our servers into two sections so that we could fit them into existing space.

We did this by moving all the Archive-related functions from OTW1 and OTW2, then moving the machines and the QNAP to another location in the facility. At this point we discovered that the QNAP did not reboot cleanly and we had to have a KVM installed before we could get it working. We are renting a KVM (at $25 per month) until we can reduce the reliance on the QNAP to a minimum.

January and February 2013

So far in 2013, we’ve been working to set up the new servers. You can see the details of our new servers and their setup in January 2013 - server setup, and find out more about our plans in Going Forward: our server setup and plans.

In closing

These are only the major items: there are many pieces of work which are done on a regular basis by all the members of the team. The Systems team averages between 30 and 50 hours a week on the organization’s business. The majority of the team are professional systems administrators/IT professionals and have over 90 years of experience between us.

Systems are proud to support the OTW and its projects. We are all volunteers, but as you can see from the details here, providing the service is not free. Servers and hosting costs are expensive! We will never place advertising on the Archive or any of our other sites, so please do consider donating to the Organization for Transformative Works. Donating at least $10 will gain you membership to the OTW and allow you to vote in our elections. (Plus you will get warm fuzzies in your tummy and know you are doing good things for all of fandom-kind!)

Comment


Pages Navigation