AO3 News

This week* on Survey Sunday, we're taking a look at some stats regarding news outlets and awareness of the OTW.

* We planned to post all these posts to the AO3 as well as to the main OTW news blogs, since so many AO3 users responded to the survey, but your friendly neighbourhood crossposter overlooked this post and the next! Sorry, folks! Hopefully those of you on the edge of your seats for news of the OTW survey saw the master post!

These questions read as follows:

77. How did you first hear about the OTW?: [Closed question]
78. I first heard of the OTW by way of this project: [Closed question]
79. The OTW hosts the following projects under its umbrella: [Closed question]
80. Which OTW projects are most important to you? [Closed question]
81. The OTW is represented on a variety of communication platforms. Which of these OTW online outlets have you read or heard of? [Closed question + write-in field]
82. We post the following types of content to our OTW online outlets. Which ones are you interested in, generally speaking? [Closed question]
83. What type of content would you like to see (more of) on the OTW online outlets? [Text box]

About the answers

All the graphs and discussion here represent final numbers for these questions. Readers may notice that this set of questions has a very high rate of people skipping the questions compared to other questions in the survey. Survey fatigue likely accounts for some of the skips appearing in today's set of questions, although as the final questions about the OTW website actually had fewer people skipping them and 36% of all survey takers took the time to write in answers for the final survey question this is probably not the primary reason. One of the responses to that final question specifically pointed out that the survey should have included more "not sure/don't remember" options so, as will be seen in discussion of Question #83 below, a good many of the skips likely also indicated that the respondent simply wasn't sure of their answer. In addition we'll discuss how the survey structure led some people to never see some of these questions.

OTW Awareness

Questions #77 and 78 were quite similar in asking for memories of initial contact. The 82% of all respondents (4,920) who skipped Question #77 did so primarily because they had been redirected to Question #79 once they answered in Question #75 that they had been members of the OTW. People who skipped #75 also ended up being directed straight to Question #79.

Question #77 actually did have a "Don't remember" option, but it was only chosen by 84 people (7.9%). Nearly equal number of respondents chose a fandom community (453 or 42.5%) as their first site of OTW awareness or word-of-mouth (444 or 41.7%). Very small numbers of people chose "An OTW Project" (48 or 4.5%), "Panel discussion at a fannish convention" (24 or 2.3%) or "Panel discussion at an academic conference" (4 or 0.4%). Particularly negligible were mentions of the OTW in the general media (9 or 0.8%).

The survey questions to come suggest that another major reason for the skips to this question is that many survey respondents had never heard of the OTW before taking the survey, and the option "Only heard of it today" was not available.

Question #78 had an enormous number of skips because only the 48 people who answered "Through an OTW project" in the previous question were offered this question. Not surprisingly, the AO3 accounted for almost all the responses, 45 or 93.8%. The remaining 3 respondents selected either Fanlore, Transformative Works & Cultures (TWC), or Legal Advocacy.

OTW Project Interest and Awareness

Question #79, however, offered all the respondents a chance to express their familiarity with OTW projects. A significant number of people still skipped this question (2,025 or 34%). Given the familiarity shown with the AO3 in most of the survey it is perhaps more surprising that there were any respondents who had never heard of it (59 or 1.5%) with the vast majority having used it (3,792 or 96%). While a significant number of people had not heard of Fanlore (1,008 or 25.9%), many more had (1,277 or 32.8%) or were users (1,608 or 41.3%). While there were fewer users of the TWC (362 or 9.4%) there were still a good many survey takers who had heard of it (1,688 or 43.7%), with 1,815 (47%) not having heard of it.

There was a huge drop off when it came to use of the OTW's other projects, however this was largely due to the restricted circumstances under which someone would do so. For example, while 52.1% of respondents knew of the OTW's Legal Advocacy efforts, only 23 (0.6%) claimed to have directly benefited from them. Similarly, 31.5% (1,211) of respondents knew of Open Doors, though only 41 (1.1%) had used it. Given that the OTW had yet to do its first import, from the Smallville Slash Archive, this is not surprising.

Lastly was the Fan Video projects which prompted the least recognition with 69.8% of respondents (2,677) unaware of it and only 65 (1.7%) having used it. However, given that fans were most likely to know of the project (1,095 or 28.5%) if they were vidders or heard of it through fans involved in making fan videos, this still seems like a fairly high number of people.

Question #80 took a different tack by asking not about people's knowledge or use but about their level of interest in OTW projects. Slightly more people skipped this question (2,144 or 36%) and most chose the AO3 with 3,575 or 93.1% of respondents. Legal advocacy barely edged out Fanlore as the second priority project with 896 (28.1%) choosing it and 880 (27.6%) choosing Fanlore. Legal support for fans was also the top third choice, with 479 (16.3%) choosing it and 117 (3%) making it their first choice.

Given the general lack of knowledge about Fan Video projects in Question #79 it's not surprising that this choice came in last as one of the top three options (6.5%) with about double the remaining people choosing Open Doors for one of those slots (532 or 14%).

Also of significance are the 35 people (0.9%) who didn't find any of the OTW projects of importance to them, and the 22% (842) people who had no second or third choices. An additional 1005 people (26%) found they couldn't decide on a second or third choice.

News About the OTW

Questions #81 and 82, which dealt with OTW news and information, had high rates of people skipping the question. Question #81 was skipped by 32% of survey takers.

Question #81 asks about which of 9 distribution/communication platforms respondents either used or were aware of. Of the 4,052, or 67% who did answer, the highest response was for AO3 administrative posts, with 77.6% or 3,146 claiming to be aware of it. This outlet was listed separately from the OTW's various Twitter accounts, which includes AO3_Status and ao3org, and had the third highest response, 1,699 or 41.9%. Chances are that many more people are now aware of the Twitter accounts and the admin posts due to the archive slowdowns. Over one weekend over 1000 additional people began following AO3_Status, which is by far the most followed of any of the OTW news outlets with 6,463 readers.

The second highest response was to the OTW website with 1,862 or 46%. This number was largely consistent with those responding to Question #84 where 1,772 or 35.7% claimed to have read the website before (a difference of only 90 people). Remaining responses were the otw_news mirrors at fannish sites such as LiveJournal or personal RSS feeds for 1,139 people or 28.1%, the Fanlore community on Dreamwidth at 973 or 24%, and 547 people (13.5%) who listed OTW gift merchandise as a point of organizational awareness.

There were also 458 (11.3%) respondents who knew of the OTW's Facebook page, 435 (10.7%) who knew of Fanlore's administrative posts and 309 (7.6%) who had met an OTW representative at an event. There were 123 write-in responses (3%) and they broke down in the following general categories:

1) None of the above (39 people)
2) One of the options already listed (21 people)
3) Were not aware of the OTW before doing the survey (7 people)
4) Random mention online (6 people)
5) Don't know (4 people)

The highest response was "Word of mouth" (46 people) where the respondent "overheard" or saw discussion in their fannish circles. This was most often being done by people who work for the OTW as staffers or volunteers.

Question #82 also had a high skip rate, with 2,042 or 34% of people not answering what sort of news they were interested in. Of the 3,944 people who did answer, 3,396 or 86.1% claimed to be interested in news about the AO3 through its updates. This number is slightly higher than the people who answered in Question #81 that they were aware of AO3 administrative posts. This suggests that Question #81 either split people who were following news on the AO3 site vs. its Twitter accounts, or else people previously unaware that such news existed were now interested in receiving it.

The second highest news of interest was about specific topics or alerts on matters such as on copyright legislation (2,360 or 59.8%). The third and fourth highest response rates were very close, with 1,722 people (43.7%) interested in the fandom Links Roundup posts, and 1,684 (42.7%) interested in project activity announcements other than those for the AO3.

At the bottom of the news interest scale was information about the inner workings of the OTW. The OTW Newsletter had 1,316 or 33.4% following it. There was then a steep drop for volunteer role posts, of interest to 750 or 19%, and OTW Committee Spotlights with 505 (12.8%).

Overall, questions #81 and 82 suggest that at least a third of respondents are either uninterested or completely unaware of news related to the OTW and its projects, and that the greatest interest is for information of changes and updates to the AO3. Of more significance for the overall survey results, and OTW Communications strategy, is that despite the varied communication outlets the OTW maintains, the footprint of people it reaches is relatively small, at least if the news is related to things other than the archive.

Getting to Specifics

The final question, #83, asked survey takers "What type of content would you like to see (more of) on the OTW online outlets?". Only 614 people answered this, resulting in a skip rate of 90%. In fact, the number would likely have been higher because it appears some people believed that all questions required a response for them to continue with the survey. The largest group of answers was "Don't know" or even random characters to create a response (212 or 34%). Although at least one of the "Don't knows" was such a dedicated survey taker that they commented about how they would have felt guilty in leaving the question blank!

An additional group of respondents, 48 (8%) mentioned that prior to taking the survey they had been either unaware of the OTW or of its news outlets so they didn't feel qualified to offer any suggestions. Given some of the comments made in response to this and other questions in the survey it seems that one of the survey's successes was in alerting users previously familiar only with the AO3 as to what the OTW was and what sort of things it did. Some people were quite interested in finding out more once they realized this, with one respondent even suggesting that the OTW run another survey in a year's time as they would be interested in taking it again once they could sufficiently answer more of its questions. Others, though, declared themselves uninterested in OTW news or any of the org's other aspects as they were interested solely in utilizing the AO3 (32 or 5.2%).

Of the people who did offer specific suggestions, the responses were quite varied and indicated that people took this question to mean a variety of things. Despite its following a question listing examples of what news content the OTW produces, some fans seemed to take "its outlets" to mean "things posted on the archive." For example, 20 people (3.2%) asked for "how to" types of information, most of it focusing on features of the AO3, although some of it was in regards to information about volunteering. Another 47 people (7.6%) suggested that the OTW get deeply involved in producing fanworks, highlighting specific fanworks, fans, or fan groups, hosting fanwork contests, posting fanfic recommendations, etc. Some suggested that it should revive metafandom, a non-OTW fannish newsletter which linked to fandom meta posts hosted on LiveJournal (and later for a time, Dreamwidth) or, indeed, produce its own meta posts.

This last tied in with the group of people who made specific suggestions for content the OTW already produces (94 or 15%). A few of these people were expressing support for particular types of content that they liked. In other cases they seemed to be unaware that things they wanted were already available. This could be expected given how many users made it clear that they were either somewhat or very unfamiliar with the OTW and were previously unaware of any of its news outlets. An exception were a few requests for an outlet at Tumblr. At the time of the survey, while the OTW had had a blog registered there since 2011, the Tumblr outlet had only just been launched.

In other cases the unfamiliarity was likely due to the variety of projects and outlets the OTW has which people simply have not come across. In regards to the request for meta posts, the OTW's academic journal Transformative Works & Cultures (TWC) hosts a Symposium blog and is continually seeking new contributors to it, whether they make recurring or one-time posts. Similarly, some people asked for "academic articles" or "research about fans" which is the purview of the TWC, links to news about fans, which occur several times a week in the Links Roundup, OTW newsletters, which go out once a month, or information about the organization which can be found at the OTW website, such as where people can find news.

This group of respondents, however, was considered separately from people who either asked for more of the content already being created, or were looking for specific types of content which they might or might not already be aware of (49 or 8.4%). The largest group of these answers focused on legal matters, requesting more information about cases affecting fans, or activities of the OTW's Legal Committee. Open Doors was also a project cited several times, with an interest about fannish preservation. Other people asked for more news about specific volunteers (which have occasionally been made as "A Day in the Life" posts), about specific committees, for more frequent and detailed information about technical developments at the AO3, etc.

Another set of answers, which we grouped under the term "transparency", targeted more news about either OTW finances or its governance (41 or 6.6%). Another set of requests asked for greater representation of fans outside of western media fandoms, or of fans in non-English speaking countries (21 or 3.4%).

And lastly, 49 people (8.4%) declared themselves satisfied with what the OTW was producing in the way of news and information.



We have received questions since our last post that involved the AO3 section of the survey and we are still compiling some of the questions requested as well as awaiting some input from the related committees. We expect that those will be posted later in July.

In the meantime if you have an interest in some of the survey questions check out our list of all of them and tell us what you're curious about for the upcoming Survey Sundays!

Mirrored from an original post on the OTW blog. Find related news by viewing our tag cloud.

Comment

AO3 accounts and invitations

Published: 2012-07-15 05:27:22 -0400

The demand for AO3 accounts has recently exploded! Unfortunately, the rapid increase in users has also created some site performance issues (you can read more about these and what we're doing about them in our post on Performance and Growth).

We use an invitation system in order to help manage the expansion of the site, and to help guard against spam accounts. Until recently, demand for invitations was low enough that the system didn't result in people waiting for a long time. However, because so many people signed up at the same time, the queue is now really long, and the waiting times are months rather than days or weeks. We know this really sucks for the people waiting for accounts (especially those who are concerned that their work may be deleted from other sites).

We really really want to cut down on waiting times, but we also need to ensure the site remains stable. It's a bit difficult to tell exactly how much difference more registered users will make, because we know that many of the people waiting for an account are already using the site as logged-out users, so to some extent we're already dealing with their extra load, However, every time someone new creates an account, it is a little bit more load on the servers, both because account holders have access to more features (personalisation, subscriptions, history, etc) and because if they're posting works, they also tend to attract more new people to the site who want to access their fanworks! Logged-in users also have more personalised pages, which makes it harder for us to serve them pages from the cache (which puts less load on the site.) At the moment of writing, there are 56,203 registered users on the site and 28,863 requests in the queue: this means we're looking at adding more than half as many users again. That's a pretty massive potential increase, so much as we'd love to, we can't issue invitations to everyone who wants one right away.

What's the plan for issuing more invitations?

Once we got our big performance issues under control, we cautiously increased the number of new invitations being issued every day from 100 to 150. We didn't make an announcement about this right away in case we needed to decrease it again (although lots of people noticed their wait time had decreased!). However, we've been keeping on eye on the site and it seems to be coping happily with the increase.

We need to install some more RAM (see our post on performance) which should be happening very shortly. Once we've done that we'll increase the numbers being issued to the queue again - by another 50 per day initially, and possibly more after that if we don't see any warning signs that it's causing problems.

Even if we up the number of invitations to 300 per day, it's still going to mean that some people will have to wait up to three months. Unfortunately, because we're looking at so much demand, there's a limit to how far we can tackle this. :(

What about invitations for friends?

We used to allow users to request invitations to give out to their friends (just to clear up one source of confusion - user accounts never came with a 'friend invitation' by default, although we did give out some unsolicited ones on our first birthday). However, as the demand increased, we were getting more and more requests of this nature, and it became difficult to keep track of it in a fair way. We decided that in order to make sure we knew exactly how many invitations were being issued, and in order to make it as fair as possible, we'd restrict invitations to the queue only for now. This means that it's 'first come, first served', whether or not you know anyone on the AO3. We know people would really like to build their communities on the site, and we will reenable the option in the future, but only when we're sure the performance situation will allow it.

Can I pay to get an account quicker?

No, not at this time. The AO3 is funded by donations to our parent, the Organization for Transformative Works, but donating doesn't give you an account on the Archive.

When we started the AO3, we decided not to have paid accounts for a few reasons. First of all, we wanted to make it something that you could use whether or not you had any financial resources: we know many fans can't afford paid services or don't have a way of paying for them. Secondly, we wanted to add a layer of protection for fans' real life identities: if you pay for your fan name account with a real name credit card, there is an increased chance that the two identities could be linked (either by accident or via a legal demand for our records; we are committed to fighting for your privacy but can't guarantee that we'd win such a battle). Finally, adding paid features to the Archive itself would have increased the complexity of what our coders had to code, especially if we had some features available only to paid accounts. For all these reasons, we decided to fund the AO3 indirectly via donations to the OTW, which couldn't be linked to your account on the Archive, allowed us to provide the site to everyone whether or not they could pay, and made it easier for us to use existing payment systems which we didn't have to code ourselves.

It's possible that in the future we may consider some form of paid service, if we developed other ways of dealing with the above concerns. However, it's not something we're considering right now - if it does become a more real possibility in the future, we'll post and ask for discussions about it.

I'm worried my works on another site are at risk! What can I do?

We've recently had a lot of direct requests for invitations from users who are worried about their works being deleted from Fanfiction.net. This creates a dilemma for us, because protecting at-risk fanworks is a fundamental part of what we do. However, right now the volume of those requests is so high that there's simply no fair way to prioritise them, which is why we're only issuing invitations via the queue. We're very sorry about this. If you're worried your work is at risk, we recommend you back it up to your own computer (Fanfiction.net has a 'download as HTML' option, or you may wish to use a a tool such as Fanfiction Downloader or Flagfic), so that you can upload it to the AO3 or any other site at a later date. You may also wish to manually back up your reviews and comments - sadly these can't be transferred to the AO3 even if you have an account, but you may want to keep a record of this important part of your fan life for yourself.

We're pleased that so many people want to be here, but very sorry we can't accommodate everyone right away. Thank you for your patience and support.

Comment

AO3 performance and growth: some details

Published: 2012-07-15 05:23:30 -0400

Everyone at the Archive of Our Own has been working hard dealing with the recent site expansion and performance problems. Now that we've been able to deal with the immediate issues, we wanted to give everyone a bit more detail on what's happening and what we're working on.

The basics

Our recent performance problems hit when a big increase in users happened, putting pressure on all the bits of the site not optimised for lots of users. We were able to make some emergency fixes which targeted the most problematic points and thus fixed the performance problems for now. However, we know we need to do quite a bit more work to make sure the site is scalable. The good news is there are lots of things we know we can work on, and we have resources to help us do it.

Some users have been concerned that the recent performance problems mean that the site is in serious trouble. However, we've got lots of plans in place to tackle the growth of the site, and we're also currently comfortable about our financial prospects (we'll be posting about this separately). As long as we are careful and don't rush to increase the number of users too fast, the site should remain stable.

The tl;dr details

What level of growth are we experiencing?

The easiest aspect of site growth for us to measure is the number of user accounts. This has definitely grown significantly: since May 1 almost 12,000 new user accounts have been created, which means a 25% increase in user numbers in the past two months. However, the number of new accounts created is only a small proportion of the overall increase in traffic.

We know that lots more people are using the site without an account. There are currently almost 30,000 people waiting for an invitation, but even that is a very, very partial picture of how many people are actually visiting the site. In fact, we now have approximately one and a half million unique visitors per month. That's a lot of users (even if we assume that some of those visitors represent the same users accessing the site from different locations)!

A bit about scalability

The recent problems we've been experiencing were related to the increase in the number of people accessing the site. This is a problem of scalability: the requirements of a site serving a small number of users can be quite different to those of a site with a large userbase. When more users are accessing a site, any weak points in the code will also become more of a problem: something which is just a little bit slow when you have 20,000 users may grind to a halt entirely by the time you hit 60,000.

The slightly counterintuitive thing about scalability is that the difference between a happy site and an overwhelmed one can be one user. Problems tend to arise when the site hits a particular break point - for example, a database table getting one more record than it can handle - and so performance problems can appear suddenly and dramatically.

When coding and designing a site, you try to ensure it is scalable: that is, you set up the hardware so that it's easy to add more capacity, you design the code so it will work for more users than you have right now, etc. However, this is always a balancing act: you want to ensure the site can grow, but you also need to ensure there's not too much redundancy and you're not paying for more things than you need. Some solutions simply don't make any sense when you have a smaller number of users, even if you think you'll need them one day in the future. In addition, there are lots of factors which can result in code which isn't very scalable: sometimes it makes sense to implement code which works now and revise it when you see how people are using the site, sometimes things progress in unexpected ways (and testing for scalability can be tricky), sometimes you simply don't know enough to detect problem areas in the code. All of these factors have been at work for the AO3 at one time or another (as for most other sites).

Emergency fixes for scalability

When lots and lots of new users arrived at the Archive at once, all the bits of the site which were not very scalable began to creak. This happened more suddenly than we were anticipating, largely because changes at the biggest multifandom archive, Fanfiction.net, meant that lots of users from there were coming over to us en masse. So, we had to make some emergency fixes to make the site more able to cope with lots more users.

In our case, we already knew we had one bit of code that was extremely UNscalable - the tag filters used to browse lists of works. These were fine and dandy when we had a very small number of works on the Archive, but they had a big flaw - they were built on demand from the list of works returned when a user accessed a particular page. This made them up-to-the-minute and detailed, but was a big problem once the list of works returned for a given fandom were numbering in the thousands - a problem we were working around while we designed a new system by limiting the number of returned works to 1000. It was also a problem because building the filters on demand meant that our servers had to redo the work every time someone hit a page with filters on it. When thousands of people were hitting the site every minute, that put the servers under a lot of strain. Fortunately, the filters happen to be a bit of code that's relatively easy to disable without hitting anything else, so we were able to remove them as an emergency measure to deal with the performance problems. Because they were such a big part of the problem, doing this had a dramatic effect on the many 502s and slowdowns.

We also did some other work to help the site cope with more users: largely this involved implementing a lot more caching and tuning our servers so they manage their workload slightly differently. All these changes were enough to deal with the short-term issues, but we need to do some more, and more sustained work to ensure that the site can grow and meet the demands of its users.

Scalability work we're doing right now

We've got a bunch of plans for things which will help scalability and thus ensure good site performance. In the short term (approximate timescales included below) we are:

  • Installing more RAM - within the next week. This will allow us to run more server processes at once so we can serve more users at the same time. This is a priority right now because our servers are running out of memory: they're regularly going over 95% of usage, which is not ideal! We have purchased new RAM and it will be installed as soon as we can book a maintenance slot with our server hosts.
  • Changing our version of MySQL to Percona - within the next week. This will give us more information about what our server is doing, helping us identify problem spots in the site which we need to work on. It should also work a bit faster. We've currently installed Percona on our Test Archive and have been checking to see it doesn't cause any unexpected problems - we'll be putting it on the main site in the next week or so. Percona is an open source version of MySQL which has additional facilities which will help us look at our problems. In addition we hope to draw on the support of the company who produce it (also called Percona).
  • Completing the work on our new tag filters - within the next month. These will (we hope!) be much, much more scalable than the old ones. They'll use a system called Elasticsearch, which is built on Solr/Lucene. These are solutions which don't use the MySQL database, so they cut down on a lot of database calls.

Scalability stuff we're doing going forward

We want to continue working on scalability going forward. We've reached a point where the site is only going to get bigger, so we need to be ready to accommodate that. This involves some complex work, so there are a bunch of conversations ongoing. However, this will involve some of the following:

  • Analysis of our systems and code to identify problem spots. We've installed a system called New Relic which can be used to analyse what's going on in the site, how scalable it is, and where problems are occurring. Percona also provides more tools to help us analyse the site. In addition, Mark from Dreamwidth has kindly offered to work with us to take a look at our Systems setup - Mark runs the Systems side of things at Dreamwidth and has lots of experience in scalability issues, so having his fresh eyes on the performance site will help us figure out the work we need to do.
  • Caching, caching and more caching. We've been working on implementing more caching for some time, and we added a lot more caching as part of our emergency fixes. However, there is still a LOT more caching we can do. Caching essentially saves a copy of a page and delivers it up to the next person who wants to see the page, instead of creating it fresh each time. Obviously, this is really helpful if you have a lot of page views: we now have over 16 million page views per week, so caching is essential. We'll be looking to implement three types:
    • Whole page caching. This is the type we implemented as an emergency fix during the recent performance issues. It uses something called Squid, and it's the best performance saver because it can just grab the whole page with no extra processing. Unfortunately, this can also cause some problems, since we have a lot of personalised pages on the site - for example, when we first implemented it, some people were getting cached pages with skins on they hadn't chosen to use. There are ways around this, however, which allow you to serve a cached page and then personalise it, so we'll be working on implementing those.
    • Partial page caching. This is something we already do a lot of - if there are bits that repeat a lot, you can cache them so that everything isn't generated fresh each time. For example, the 'work blurbs' (the information about individual works in a list of search results) are all cached. This uses a system called memcached. We'll be looking to do more, and better, partial caching.
    • Database caching. This would mean we use a secondary server to do complex queries and then put the results on the primary server, so all the primary server is doing is grabbing them.
  • Adding more servers. We’re definitely going to need more database servers to manage site growth, and we’re currently finalising some decisions on that. At the moment, it looks like the way we’re going to go is to add a new machine which would be dedicated to read requests (which is most of our traffic – people looking at works rather than posting them) while one of our older machines will be dedicated to write requests (posting, commenting, etc). Once we've confirmed the finer details (hopefully this week), we expect it to take about two months for the new server to be purchased and installed.

Resources: finances

We'll be posting separately about the financial setup for the AO3, but the key thing to say is that we're currently in a healthy financial state. :D However, as the site gets bigger its financial needs will also get bigger, and we always welcome donations - if you want to donate and you can afford to do so, then donating to the OTW will help us stay on good financial footing. We really appreciate the immense generosity of the fannish community for the support already you've shown us. <3

Resources: people

A lot of supporting the site and dealing with scalability is down to the people. As we grow, we need to ensure we have the people and expertise to keep things running. We are a volunteer-run site and as such our staff have varying levels of time, expertise, and so on. One important part of expanding slowly is ensuring that we don't get into crisis situations which not only suck for our users (like when the 502s were making the site inaccessible) but also cause massive stress for the people working to fix the problems. So, we're proceeding cautiously to try to avoid those situations.

We've been working hard over the last year or so to make it easier for people to get involved with coding and working on the site. We're happy to say this is definitely paying off: we've had eight new coders come on board during the last few months who have already started contributing code. Our code is public on github, and we welcome 'drive by' code contributions: one thing we'd like to do is make that a bit easier by providing more extensive setup instructions so people who want to try running the code on their own machines can do so.

If you'd like to get more involved in our coding teams, then you can volunteer via our technical recruitment form. Please note that at the moment, we're only taking on fairly experienced people - normally we very much welcome absolute beginners as well, but we're taking a brief break while our established team get some of the performance problems under control so that we don't wind up taking on more people than we can support. We love helping people to acquire brand-new skills, but we want to be sure we can mentor and train them when they join us.

Lots of people have asked whether we'd consider having paid employees. It's unlikely that we'll have permanent employees in the foreseeable future, for a number of reasons (taxes, insurance, etc), but we are considering areas where we would benefit from paid expertise for particular tasks. Ideally, this would enable us to offer more training to our volunteers while targeting particularly sticky sections of code. Paying for help has a lot of implications (most obviously, it would add to our financial burden) and we want to think carefully about what makes sense for us. However, the OTW Board are discussing those options.

We're incredibly grateful to the hard-working volunteers who give their time and energy to all aspects of running the AO3. They are our most precious resource and we would like to take the opportunity to say thanks to all our volunteers, past, present and future. <3

Comment

Farewell Sidra

Published: 2012-07-05 07:27:07 -0400

The OTW Board announces with regret the resignation of Sidra, the co-chair and technical lead of our Systems committee, who has been one of our senior technical staff members from our early days. She has supported virtually all our projects with her vast technical skills and immense generosity with her time, and has been one of the foremost contributors to the Archive of Our Own.

Sidra is staying on as a consultant with Systems, so she is not vanishing completely, but we're hugely grateful to her for all her work and want to thank her publicly for her years of incredible service to the org, and we hope all our members and the users of our varied projects will join us in those thanks!

Mirrored from an original post on the OTW blog. Find related news by viewing our tag cloud.

Comment

Release Notes for Release 0.8.20 and 0.8.21

Published: 2012-07-04 15:58:31 -0400

These release notes bring together details for two small updates, one we deployed shortly after Release 0.8.19 to address a couple of issues which arose, and the one we deployed today.

Some of you may have noticed a temporary site glitch on June 30/July 1. This was caused by a leap second in the UTC time standard which caused problems for a number of sites across the web. Our servers got all confuzzled and had to be rebooted by our ever-alert Systems team. Unfortunately, a small number of kudos and subscriptions emails were lost during this time: many apologies for this!

Today's deploy comprises a handful of small tweaks to the Archive, which make up the last of our performance-related "emergency" updates. We will now focus on the upcoming 0.9.0 release, which we're hoping to have ready to go later this month. This release will come with a major rewrite of the browse & search navigation (replacing the currently disabled filtering system) and a host of bug fixes. Expect significant improvements to rich text editing, collections and challenges, and the help pop-ups all around the site, among many other things.

Over the past few months, we've had a total of eight new coder volunteers coming in, many of whom dived right into our code and already submitted bug fixes or are currently working on feature enhancements. In addition, the Testers group welcomed several new volunteers. We are grateful to anyone donating their time, skills and passion to the Archive, be it as coders, testers, tag wranglers or Support staff. Many thanks also to the members of the Volunteers & Recruiting committee, who have been tireless in getting new people sorted and settled in. \o/

Mini-release: 0.8.20 (deployed 17 June 2012)

  • We made some tweaks to the caching system introduced earlier (more details in our Update on AO3 performance issues), because it was generating error messages for some tag pages.

Current release: 0.8.21

  • We added a help pop-up to the Work Search form, including some tips for searching by tag, which lets you combine ratings, warnings, fandoms, characters etc. into a host of search options.
  • Since it was taking a very long time to generate the search index, making it impossible to search for new works, we've disabled searching by the number of kudos (which were the major bottleneck) until the new system is up and running.
  • Several users contacted Support in alarm when it looked like they were suddenly logged into someone else's account. At no point was security actually compromised, this was merely the result of faulty cookie handling in some browsers confusing the caching and serving up "logged in" pages to guests. Since guests cannot access areas restricted to account holders to begin with, this only affected public pages. We believe the problem is now fixed.
  • Assigning numbered IDs to each invite request created a potential security issue which we fixed in Release 0.8.19. Due to this change, the previous, bookmarkable page to check one's position in the queue wasn't working anymore. We fixed the message and page text accordingly.
  • When checking up on an invite request on the requests page, the search button would just hang without a message if your email wasn't found in the queue. This has been fixed.

Known issues

See our Known Issues page. For the latest site status information, check our Twitter AO3_Status.

Comment

From FF.net to AO3 - some frequently asked questions

Published: 2012-06-21 10:13:09 -0400

The last month or so has seen lots of Fanfiction.net users joining the Archive of Our Own - you are all very welcome! As such, we've had a number of Support questions about the ways in which the AO3 differs from FF.net, so we've put together a quick primer to let you know about a few key details. (Although they're focused around the questions we've received from FF.net users, this post will also be relevant to any new user of the site.)

Getting an account

How do I create an account?

You need an invitation to create an account. This is to help us manage site growth (as you may have noticed, the recent expansion has caused a few performance issues). You can request an invitation by adding your name to the invitations queue.

The invitations queue is really long! Can I get an invitation any quicker?

Because so many people have recently moved to the AO3, the invitations queue is very long and wait times currently reach until next year. At the moment, we're still working on dealing with some performance issues, so we're not able to issue invitations any faster. As soon as we're confident we have those under control we'll review the number of invitations we're issuing and take measures to try to reduce the queue. We're really sorry we can't issue invitations to everyone who wants one right away, but we need to ensure the site can cope with the demand first. In the meantime, if you are concerned that your work might be deleted without warning, we recommend saving a copy to your hard drive so that you have a backup you can repost here or elsewhere. (And don't forget to save your reviews, which can't be reproduced elsewhere!)

I have an account - can I get an invitation to give to a friend?

We normally allow existing users to request invitation codes to give to friends, but because of the very high demand we've had to stop issuing invitation requests for now. We'll reenable this option when we have the performance issues under control - we're sorry to have to disappoint you right now.

Posting

Can I import my stories directly from FF.net?

Unfortunately, no. Fanfiction.net is blocking requests from our server. You can read more about this issue in our news post, Problems with imports from FF.net.

Can I import stories from other sites?

Yes, although there may be issues, depending on where you're importing from. To import a story, choose "post new" from the header menu and then click on the "Import From An Existing URL Instead?" button above the text entry fields. You can also go directly to the Import New Work page. For more information, see the Importing and Mass Editing FAQ and our Known Issues relating to imports.

Can I upload my story from a file?

This is not currently possible. You can copy and paste your text from a Word file or the FF.net document editor, but you can't upload or manage files.

How do I keep my formatting when pasting into the work text box?

By default, the text entry box comes up in the mode for directly entering HTML code. To paste formatted text in, click the "Rich Text" button above the box to switch to the Rich Text editor, then paste your text.

Since the Rich Text option relies on a third-party tool, which comes with its own set of bugs, there are currently several issues when pasting in formatted text. In particular, it does not work properly with Internet Explorer 9. Some of these issues should be fixed in an update fairly soon, but in the meantime you can read about them in our Known Issues. If you do have trouble formatting your work, please contact Support and we'll be happy to help you try to work around the difficulties.

How does tagging work?

There are 7 categories of tags you can enter: Rating, Archive Warnings, Category, Fandom, Character, Relationship, Additional Tags.

The Rating, Archive Warnings, and Category tags for your work are set by the choices you tick in the form while posting. You can read more about our policies on ratings and warnings in our Terms of Service FAQ. The important thing to note is that we usually consider ratings to be in the eye of the beholder, so you should use the rating that seems right to you. "Category" is the Archive term for describing a work based on its main relationship (or lack thereof), i.e. M/M, F/M, Gen and so on.

The Fandom, Character, Relationship, and Additional Tags categories are created by typing into the appropriate boxes on the posting form. Tags should be separated by commas. The autocomplete will show suggestions for existing tags, but you can also create completely new tags. The AO3's tag wranglers will link new tags to the "canonical tag" with the same meaning where possible (for example, all versions of a given character's name will be linked). To make it easier for readers to find your work, it's best to choose clear, non-ambiguous tags. You can read more about how tagging works on the Archive in the Tags FAQ.

How do I post a crossover?

Whereas on FF.net there's a special section for crossovers, a work on the AO3 can be marked as a crossover by adding all applicable fandoms as tags and, optionally, using keywords such as "Crossover" or "Fusion" under Additional Tags.

If your work is based on more than one fandom, all you need to do is to enter all the fandom names in the "Fandom" field when you post, separated by commas. So, instead of having to choose a separate "crossover" category, you would simply enter Bleach, Homestuck, Hawaii Five-O (1968) into the Fandom field. Your work will then be listed under all three fandoms. If you include the "Crossover" tag in the Additional Tags, users will be able to more easily find (or avoid) your work in the appropriate tag searches.

Can I post explicit works? Can I post explicit works featuring underage characters? Can I post Real Person Fiction?

Yes, yes, and yes. If your work is fannish in nature and abides by our Terms of Service, you can post it here. We do ask that you label your work appropriately, which can include using the "Choose Not To Use Archive Warnings" and "Not Rated" options if you prefer not to warn or pick a rating. You may also make use of Additional Tags to add content notes that aren't covered by the Archive Warnings.

Does AO3 have "communities"?

The closest thing to FF.net's "communities" on AO3 are Collections. The main difference is that the collection moderator cannot post stories to the collection personally; that has to be done by the authors of the stories. The moderator can add bookmarks to a collection, which will point to the works instead of gathering them up directly.

You can read more about this feature in the Collections and Challenges FAQ (although this section is currently in need of updating). Collections can also be used to set up gift exchanges and prompt memes.

Search

The Archive offers a search form to find exactly the works you're looking for. However, documentation of all its features is in need of an update. In the meantime, you can find lots of tips & tricks, including several example searches, in this post posts: Disabling filters: information and search tips.

One particularly useful thing to keep in mind is that because Ratings, Warnings, and Categories are all tags, you can search for (or exclude) them. So, for example, if you're looking for explicit slash but don't care for violence, you could enter "Explicit" "M/M" -"Graphic Violence" in the Tag search box along with whatever other terms (fandoms, pairings, etc.) you're looking for.

At the time of posting, browsing filters are turned off for performance reasons (you'll see the grey box where they would usually be on work pages). They'll be replaced in a few weeks with all-new filters which will give you more ways to find things on the site.

Reading

Can I make the text bigger or smaller?

Yes! We don't have these options on the individual work pages, like on FF.net. If you're a logged-in user, then you can change the text size (and most other display features) with a site skin: see the Skins FAQ for more information. If you're logged out, or if you just want a quick and easy way to change the text size, then most modern browsers will let you do this by hitting Control + or Control -, or Cmd +/- on a Mac. (This will work on any site!)

Can I make the text light on dark, change the margins, or choose a sans-serif font?

Yes, if you're a logged-in user. These options aren't on individual work pages like on FF.net, but you can change most aspects of the way the site looks with a site skin: see the Skins FAQ for more information. For performance reasons, we've had to disable skins for logged-out users. We're working on ways of bringing them back. In the meantime, if you need a modified display for accessibility reasons and you don't have an account, please contact Support and they'll help you out.

Communication and Feedback

How do I leave a review? Do I need an account?

To leave a review, just type your feedback into the comment field at the bottom of the work you enjoyed (surrounded by a grey box). You don't need to have an account - if you're not a logged-in user, you'll need to leave a name and email address (your email won't be displayed, but the name you give will).

Can I send a private message to another user?

Currently, no. This is a very frequently requested feature that has been approved for future implementation, but at the moment the only option for private communication is email. Some users opt to display an email address publicly; you can check for one by going to the user's home page and choosing "Profile" from their dashboard. Some also link to their journal accounts (such as Livejournal or Dreamwidth) or blogs in their Bio section.

Can I block a user from leaving me feedback?

No. You can, however, delete user comments on your works, including comments left by users who are logged in. If you feel a user's comments constitute harassment (see our TOS) please submit an abuse report. Note that you should not delete the comments in this case, as once deleted they cannot be recovered for review.

Can I turn off comments/kudos from users who are anonymous/not logged in?

No. For several reasons, including avoiding the exclusion of users who have not yet been able to get an invite, this is not a preference we offer to authors. Comments can be deleted, and spam comments should be marked using the "Spam" button to notify our automated spam-tracking software.

Does the Archive have author/story alerts?

Yes, called "subscriptions"--when an update is posted, subscribed users are sent a notification. You can currently subscribe to authors, stories, and series. To manage your subscriptions, click on the "Subscriptions" link in your Dashboard.

Can I see who is subscribed to me/my works?

No. We know that many users want to know how many people are following their works, so we made the numbers available in our Stats feature. (You can read more about it in our admin post about the Statistics Page.) However, to protect user privacy, you cannot see specific information about subscribers.

I want to favorite an author/work/series. How do I do that?

You can bookmark works and series. In this case, unless a bookmark is set to private, an author can see who has bookmarked their works and what notes they've added. An option to bookmark authors is planned, but hasn't been implemented yet.

More Questions and Troubleshooting

What should I read to learn more about how to use the site?

After reviewing our Terms of Service, we recommend that new users should look through our FAQ pages. We also regularly update our news posts with Tutorials, so check there often!

Where can I read updates on changes to the site or known problems?

All updates and changes will be publicised in our News Posts, many of which are mirrored in several locations, including the OTW's Dreamwidth, Livejournal, and Tumblr pages. All code updates (deploys) come with a set of detailed Release Notes, listing all bug fixes and describing major changes.

Is there a way to know if the site is down/having issues?

You can always check our Twitter feed at @AO3_Status.

Where do I go for help?

Whether it's a question, a bug report, or a feature request, you can submit it all through our Support form. We promise to take your question, suggestion, or problem seriously. You can submit anonymous feedback if you desire, but if you leave an email address we will get back to you with an answer as soon as we are able! Your IP address will be registered for spam protection issues, but that information is never available to our support staff.

Comment

Release Notes 0.8.19

Published: 2012-06-13 07:46:29 -0400

Yet another update from your tireless archive volunteers! James from our Systems Committee has been making adjustments behind the scenes to stabilize the servers and get the most out of our caching, and we've seen some good improvements there. At the same time, we've been working on improving or scaling back the areas of our code where changes will give us the biggest gains.

Filtering

In order to improve performance further, tag filtering on work listing pages is disabled for the time being, until we roll out our new system. You can read more about this change in our post on disabling filters. We know this is an inconvenience for many users, but the filters are really the 800-pound gorilla sitting on top of our database - the works pages are both the most popular and the slowest on the site, which is a bad combination. We've had plans to fix them for a while, and that's underway. However, we need a few more weeks to finish and deploy the upgrade, since it also affects our search engine and quite a lot of our code. Our top priority is to make sure works remain accessible to users, and that new works and feedback can be posted and accessed. Looking carefully at our code and our stats, we concluded that removing filtering was the best way to ensure these goals in the short-term.

You'll still be able to view all the works for a particular tag, view the works for a user or collection in a particular fandom, and use our search feature to refine your results. Our post on disabling filters includes some handy tips to help you find what you're looking for. We hope to have full functionality restored to you soon! As a bonus side effect of this change, we've been able to remove the 1000 work limit on lists of works. This is because without the filters we can rely on the pagination system to limit the amount that we retrieve from the database at one time. So, while you can't filter your results any more, you CAN go through and read every work posted in your fandom! We hope this will compensate a little for the inconvenience.

Work Stats Caching

We've also done more caching of work stats (all the counts of comments, bookmarks, hits, etc.), so you may notice that these update more slowly on index pages now. The information is all still being recorded; we're just waiting a little longer to go get the counts for each work to spread out the load.

People Listings

The alphabetical people listings on the People page weren't actually that useful for finding users, and they were another performance drain.

We've replaced the full alphabetical listing with a listing of 10 random users, and added emphasis on the search. Note you can use wildcards in the search, so if you're not sure of someone's name you can enter part of it followed by an asterisk to get similar names. For example, entering Steve* would get Steve_Rogers_lover, SteveMcGarrettsGirl, stevecarrellrocks, etc.

Invitation Requests

We've suspended user requests for additional invitations for now as well. If you need invitations urgently for a challenge or for an archive rescue project, please contact Support. We also fixed an issue that potentially allowed users to snoop for other emails in the waiting queue.

Thank you!

Thanks to everyone who has been working hard on these issues, especially James, who has put in lots of hours tweaking the servers, and Elz, who has been doing the heavy lifting on code changes. Thanks also to all of you for your patience and understanding while we work!

And finally...

The great news is that so far, this emergency measure does seem to be having a noticeable effect. Our server load has diminished dramatically since we deployed this change:

Graph showing server load, with a mark showing the time of the deploy. The load drops dramatically from this time onwards.

Comment

Disabling filters: information and search tips

Published: 2012-06-12 19:58:16 -0400

Key information: As an emergency measure to deal with recent performance issues, we have disabled browsing filters on the site (the grey box of choices which appears on work index pages). This is a temporary measure to ensure that as many people as possible can access the site. You can still use our tags and advanced search feature to find the works you want. As an additional bonus, removing the filters has allowed us to remove the 1000 works cap on lists of works, so you can browse through all the works in your fandom! Read on for more information!

What's happening

As detailed in our recent post on performance, our coders and sys-admins are continuing to work on the performance issues we've been experiencing. We've made some server adjustments which have alleviated some of the worst problems, but we still need to make some substantial changes to fix the issues. We're aware that lots of users are still unable to access the site; as an emergency measure, we've decided to disable tag filters, which put a very heavy load on our servers. This means that the grey box with tags you can check to filter a list of works will no longer appear on the work index pages. We know this will be an inconvenience for many users, but the filters are really the 800-pound gorilla sitting on top of our database. Removing them for now will mean that people can access the site, even if they can't browse quite as easily as usual.

We've been working on significantly redesigning the part of our code that handles filtering for a while - because it's a major performance burden on some of the most popular pages of the site, refactoring this code to make it more efficient has been a priority for some time now. We're almost done with the rewritten version, but it needs more work and extended testing before we roll it out. (We want to be sure it doesn't introduce new bugs.) So, the filters will go away for a few weeks, and will then be replaced by the new, rewritten version.

One major disadvantage of the way the filters were designed was that they needed to retrieve the tags from the list of works found in order to build the filter options. This meant that we had to limit the number of works returned at one time to 1000, because otherwise building the filters would take too long. A side bonus of removing the filters is that we've been able to remove the 1000 works cap! The browsing redesign in progressaims to work around this issue, so we hope to avoid re-introducing this limitation when filtering returns.

How can I find the works I want?

Although the removal of the filters will make it harder to browse the works listings for specific things, there are still lots of ways to find the works you need.

Fandoms page

If you're looking for a specific fandom, you can browse the Fandoms page. Fandoms are organised by media type; the easiest way to find a particular fandom is to use Ctrl + F (or Command + F on a Mac) to search the page in your browser. The fandom pages will give you a list of all the works in your fandom; unfortunately there will be no way to filter that list down further.

Tags

Clicking on any tag will still bring up works with that tag, or with any tag marked as a synonym. So, if you click on Riza Hawkeye you'll get all the works tagged with 'Riza Hawkeye', 'Riza', 'Riza is awesome', etc. Again, while the filters are disabled there'll be no way to filter this list further.

Advanced Search

If you want more refined control over which works you find, you will need to use our Work Search. This feature could use a little bit of prettifying, but the underlying search is quite powerful. Use the following tips to help you find exactly the works you want:

  • A space equals AND. So, entering Fluff Friendship would find you works tagged with both 'fluff' and 'friendship'
  • | equals OR. So, entering Homestuck | My Little Pony will find you works tagged with 'Homestuck' AND/OR 'My Little Pony'
  • - equals NOT. So, entering Supernatural - Castiel/Dean Winchester will find works tagged Supernatural, but will exclude those tagged Castiel/Dean Winchester.
  • Fandom, Character, Relationship, Rating, Category, and Warning are all classed as tags (as well as the 'Additional tags'). So, you can search for works which are Explicit, or exclude works tagged 'Major Character Death'.
  • Using quotes around a phrase will search for that exact phrase. So, "Harry Potter" will get works tagged with 'Harry Potter', whereas Harry Potter will get works tagged with 'Harry' and works tagged with 'Potter'.
  • Entering a term in the tag field will only find works with exactly that tag - so searching for Charles/Erik will bring up only the few works tagged with exactly that tag, not the ones tagged 'Erik Lehnsherr/Charles Xavier' (whereas if you click on the 'Charles/Erik' tag you'll get works with all variations of that pairing).
  • The search has trouble with tags which have dashes in them. If you search for X-Men, for instance, you noticed you'll get lots with X and no X-Men. To get around this, put the tag in quotes: "X-Men".

As well as searching tags, titles, and authors, you can also search for specific word counts, hits, kudos, and dates - including ranges, which is a useful tool for finding fics in a fandom. For example, you can search for all Stargate Atlantis fics published 5-6 years ago.

Some search examples!

  • Find an explicit Fullmetal Alchemist work with the pairing Riza Hawkeye/Roy Mustang, with no Archive Warnings: Enter "Fullmetal Alchemist" "Riza Hawkeye/Roy Mustang" "No Archive Warnings Apply" Explicit.
  • Find works with Rodney McKay but without John Sheppard: Enter "Rodney McKay" -"John Sheppard".
  • Find works tagged with "Alternate Universe" in either the Homestuck or White Collar fandoms: Enter "Alternate Universe" Homestuck | "White Collar".
  • Find all explicit works tagged as angst, but excluding M/M pairings: Enter Angst Explicit -"M/M"

Search bookmarklets

If you find yourself re-using the same search parameters (only T-rated works, only works under 5,000 words, only works with over 10 kudos) for new fandoms or characters you fall in love with, you could give these custom search bookmarklets a try. They are not official AO3 tools, but made by one of our own and utilizing the Advanced Search functionality. Think of them as a saved search that lets you enter a keyword (such as a fandom name or specific kink) and spits out only the kind of work you want to see. For help in putting together your own saved search, don't hesitate to comment on the post or here.

What next?

This is definitely a short term measure, but we think it will have a big effect on site performance. In a few weeks we hope to deploy our all new search and browse features, which will restore more browsing functionality without placing the same load on the servers. We thank you for your patience and understanding while we continue to work on the problem areas.

Post edited 2012-06-13, 12.00 UTC to reflect some minor changes in functionality & bring it up to date.

Comment


Pages Navigation