AO3 News

Performance issues on the AO3

Published: 2012-01-17 17:05:30 -0500

As many users will no doubt have noticed, the AO3 has been experiencing some performance issues since the start of the year. When we posted on 5th January, we were expecting those problems to ease once the holiday rush was over. However, that hasn't turned out to be the case. We're working on ways of dealing with the performance issues, but we wanted to keep you updated with what's going on while we do that.

Why the slowdowns?

In the past month, over 2000 new users have created accounts on the Archive. At the same time, the number of people reading on the Archive - with or without accounts - has been steadily growing. This has been part of a general trend, as you can see if you look at the graph showing number of visits to the Archive since November:

Line graph showing number of visits to the AO3, November to January. The line gradually goes up (with spikes on Sundays) before peaking dramatically on Jan 2.

We're always much busier on Sundays, but the number of visits has been gradually going up each week since November (and the same holds true for the preceding months). However, before December we were hovering around the 135,000 level for visitor numbers at peak times. You can see that the visitor numbers began to climb more dramatically in December, peaking on 2nd January when we had 182,958 visitors. Crucially, after that spike it didn't drop back down to anything like the levels it had been at previously: we're now at more than 150,000 visits on a regular day, and more than 165,000 on Sundays, our busiest day. Wow!

We were expecting a big spike over the holidays, when there are lots of challenges and lots of people with a little spare time for reading and creating. However, we hadn't expected site usage to remain quite so high after the holidays were over! The increases mean that the site is now under a holiday load every day, which is one reason things have been running a little slowly.

The other reason for the slowdowns is that the increase in our number of registered users, and the holiday challenge season, has produced a big increase in the number of works. In fact, 11,516 new works have been posted since the end of December already! More data in our databases means more work for things like sorting, searching, etc - this means that sometimes the database just doesn't serve up the result you need in time, and the unicorn which is waiting to get that result gives up and goes away (yes, really - our servers are assisted by unicorns :D).

We've been expecting this general effect for a while now, and we've been working towards implementing things to deal with it; however, we weren't expecting quite such a big jump in site usage in the past month!

What are you doing about this?

The Accessibility, Design & Technology and Systems Committees had a special meeting on Saturday to discuss ways of dealing with the immediate problem, as well as longer term plans. It can be tricky to test for high load situations before they actually occur, but once they do occur there's lots of data we can gather to help us address the most crucial issues. (We're also working on implementing more tools which will help us test this stuff before it comes up.)

Short term

More caching: We already cache pages (or sections of pages) across the site - this means we store a copy which we can serve up directly, instead of creating the page every time someone wants to use it. If something changes, then the cache is expired and a new, updated copy is created. Hitherto, we've focused on caching chunks of information which are unlikely to change rapidly: for example, on any works index the 'blurbs' which show the information about each work are cached. However, some of the heaviest load is caused by rapidly changing pages like the works index. We're moving towards more caching of whole pages, so that a new copy of the works index (for example) will be created every five minutes rather than generated each time someone asks for it. This means things like works indexes will be a little slower to update - when you add a new work, it won't appear on the list until the cache expires - but that five minute delay will massively reduce the weight on our servers.

More indexes: We have a few places in our databases - for example the tables for the skins - which could use more indexes. Indexes speed things up because the server can just search through those rather than the whole table. So, we're hunting out places where more indexes are needed, and implementing them. :)

Medium term

Bad queries must die: We have a few queries which are very long and complicated, and take a long time to run. We need to rewrite these bits of the code to make them simpler and faster! In many cases this will be quite complicated (or else we would have done it already), but it's a priority to help us speed things up.

New filters for great justice: The filters that are implemented on our index pages are not really optimal considering the size of the site now - the limitations of that code are the reason we have to have a 1000 work cap on the number of works returned. We have been working on this for a long time - we need to completely throw out what we have and implement a system which works better for the site as it is now. Again, this is really complicated, which is why it's taken us a long time to achieve it even though we knew it was important - the good news is that we have now done quite a lot of work on this area and the first round of changes should be out in the next few months.

Long term

Long term, we're going to be moving to a setup which allows us to distribute our site load across more servers. This will involve database sharding - putting different bits of the database on separate servers - so it will take quite a lot of planning and expertise. If you're a user of Livejournal or Dreamwidth, you might be aware that your journal is hosted on a certain 'cluster' - we'd be moving to a similar system. We want to make sure we do this right, but based on the way the site is growing we think this is now high priority, and our Systems team are working to figure out the right ways forward.

Summary

We know it's really frustrating when the site runs slow or is timing out on you: many apologies. We really appreciate users' patience while we deal with the issues. As you'll see from the above, there are some immediate things we can do to ease the problems, and we also have a good sense of where we need to go from here. So, while these changes need to be implemented as a matter of urgency, we feel confident we will be able to tackle the problems. If you have expertise in the areas of performance, scalability and database management, we would very much welcome additional volunteers.

As we move forward on dealing with problem spots on the site, we may implement some changes which are visible to users: the caching on the index pages and the changes to browsing and searching are two of the most obvious. We'll let you know about this as we go along - we think the effect will be beneficial for everyone, but do be prepared for a few changes! You can keep up with status and deploy news on our Twitter @AO3_Status.

While the growth in the site means we're facing some problems a little sooner than we expected, we're really excited about the fact so many people want to read and post to the AO3. Thanks to everyone for your fannish energy - and apologies for the fact we sometimes slow you down a little.

Comment

Rush hour on the AO3!

Published: 2012-01-05 09:59:30 -0500

As many of you will have noticed, we had some site slowdowns and 502 errors over the first couple of days of the year. Apologies for the inconvenience! If you've run into this problem and been wondering what was going on, you might be interested in this:

Line graph of the last visits on the AO3, 4 December to 3 January. The graph peaks sharply on 1st January

Yes, it looks like lots of fans decided to celebrate the New Year with some delicious fanworks. On Monday 2nd January we had 182,958 visits, and over 1,066,216 pageviews! Furthermore, an octopus swam off with our servers - volta_arovet's Texts From Cephalopods has had 46,301 hits at the time of writing! So, our servers had plenty of work to do!

Over 2000 new users have joined the Archive in the last couple of weeks, and we have hosted several great challenges, including Yuletide (2598 works!), Due South Secret Santa (a more modest 34 works), and Homestuck Ladyfest Exchange (124 works). So, while we're sorry to have had some slowdowns, overall we are super pleased with how well our shiny new servers have held up - those of you who were with us during the holiday season in previous years will remember that the high traffic of holiday challenges made our old servers very sad.

Looking forward, we're not too worried about performance in the immediate future - there are some code improvements we know we need to make which will improve matters a lot, so those will be high priority. If the AO3 continues to expand at the same rate as this year, we will be looking at more servers sooner rather than later. But in light of the graph above, we're pleased that while we certainly slowed down, we didn't grind to a halt! Thanks to all the coders and sysadmins who did the work to make this possible, and thank you to all the OTW members whose donations helped us buy those hardworking servers (we are always grateful for volunteers or donations)! And, of course, thanks to everyone who reads and posts on the AO3 - we're excited to welcome so many of you!

Once again, apologies to those of you who have been affected by the slowdowns - but hurray for so much beautiful fannish activity!

Comment

2011 Year in Review!

Published: 2011-12-31 14:07:42 -0500

2011 was an amazing year for the Archive of Our Own, and we wanted to take a moment to look back and to thank everyone involved, including all of our users and volunteers! AO3 started its open beta about two years ago, towards the end of 2009. That year, we were really still putting the pieces together, building out the core functionality. In 2010, we started to pick up more momentum with people posting their works and archiving their older fic and art. We added gift exchange challenge hosting, kudos, downloads and skins. This year, we've done a lot of work on site performance and infrastructure, usability improvements, and new features like subscriptions and prompt meme challenges. We're looking forward to expanding on that next year and continuing to build a great, stable home for all kinds of fanworks!

Traffic and performance

A drawing of our seven machines!

At the beginning of the year, we moved to a new and bigger set of servers, which gave the site some much-needed room to grow. Our systems team made some tweaks along the way, ensuring that we were getting the best performance out of the new setup. We started using Redis, which is super-fast, for email queues, autocompletes and other background tasks, which took some of the load off our main database. And even with all the work we were doing, it was tough to keep up with how fast the site was growing! 2/3rds of our current registered users signed up this year, and we kept giving out more and more invitations through our invite queue, but the numbers kept climbing - there were over 2,000 people on the waiting list for several months. (We've finally gotten that down now, just by sending out even more as the system could handle it.) And many more site visitors aren't registered users - we now get well over half a million unique visitors each month and there have been 24+ million pageviews in December. We now get as much traffic on an average day as we did last year during Yuletide, which at the time was a huge traffic spike. The period around Christmas, with Yuletide and other holiday exchanges going live, still represents a noticeable jump in traffic, but the difference isn't as great which means more stable site performance. (\o/ We were standing by with fingers crossed just in case, but we were thrilled that no last-minute work was required this year!)

Fun with charts!

AO3 is currently home to over 8,100 fandoms, 31,000 users and 275,000 works! Here's a graph of work, chapter, bookmark and comment posting over the last three years:

You can see that work posting is up this year, but what's much more dramatic is the increase in reading, bookmarking and commenting. There have also been more multi-chapter works and works-in-progress posted this year, which is exciting. And one of the neat features the archive has is the ability to go back and see what you've read or viewed, for registered users who have it enabled. Here's how that looks year-to-year:

Lots of people viewing lots of stuff! There have also been almost 1.5 million kudos left since last year, so there's been no shortage of love to go around. <3

What's on deck for 2012

In the short term, we have a new release coming out hopefully early this month, and that will include improvements to our HTML parser (yay!), some exciting new subscription options and a variety of bugfixes. There are also a ton of other features and improvements that we've been developing this year that we hope to have ready for you in 2012, including the ability to view the site in other languages, art hosting, an on-site support area and a wealth of browsing, filtering and email improvements for both works and bookmarks. We also hope to start a series of international fandom spotlights in January and solicit more input from users about upcoming features.

Kudos!

And finally: thank you! Thanks to all of the authors, artists, and vidders who have posted their works, to the mods who organize challenges and collections, to those who have shared skins for customizing the site, to everyone who creates bookmarks and leaves comments and kudos, encouraging authors and artists and making it easier for other fans to find awesome works. Thanks to everyone for bearing with our growing pains earlier in the year and for supporting AO3 financially, enabling it to continue operating and improving. And many thanks, as always, to everyone who volunteers their time wrangling tags, writing code, testing the site, handling support requests, and maintaining our systems, and also to everyone who's left comments and written in to our support team with feedback, suggestions, and bug reports, all of which are incredibly valuable! The archive is very much a community effort, and it couldn't exist without all of us working together and supporting one another.

Kudos!

Comment

The Accessibility, Design and Technology committee oversees technology-related projects within the OTW. Currently we are responsible for designing and building the Archive of Our Own. Our regular meeting updates keep you informed about developments on the AO3!

This was our final meeting of the year: the OTW takes an end-of-year break during which committees dissolve and are reformed, and committee members take a well-earned rest! We've had an action-packed year, so we're all ready for a break (from meetings at least - a lot of our work goes on as usual). We'll resume in January - we don't take on any new volunteers during our hiatus, so if you volunteer between now and then (and we hope you will - as you can see below, there are several areas we're really keen to build up), you'll have to wait a little while to get started.

Meeting highlights!

Fandom landing pad!

AD&T co-chair and senior coder Elz has been working for a while on improvements to browsing on the Archive. One thing she's been working on is a new 'fandom landing pad' so that browsing to a fandom will give you the option to browse to some important areas relating to that fandom. In this meeting we previewed her new design - going to a fandom landing page will give you a list of pairings and relationships in the fandom, and a list of authors and artists who have created work in that fandom, along with some basic information about the canon source. It's not quite ready for primetime yet, but it's looking very nifty!

Issues for Love!

Issues for love are the requests submitted by users via Support. We try to work through a few of these each meeting: we're working on ways of making it easier for people to see what has been suggested and what has been decided about the suggestions, but for now we'll include a round-up of our discussions in our meeting updates. Note that a decision to implement something does not necessarily mean it will be implemented soon - we have many issues to work on and a limited number of coders! If you want to see the full (and lengthy!) list of things logged for coders to work on you can check out our Google Code Issues. If you'd like to adopt an issue, we welcome new coders!

  • Request to add a setting to prompt-meme challenges to disallow anonymous prompts. This seemed like a handy extra feature without too much coding complexity, so we have logged it as an issue for a coder to work on.
  • Request to add an option to hide 'Share' buttons on works to reclaim screen real estate. It's already possible to disallow use of the share button on your own works, but you still see the button. We sympathise with the desire to reclaim the screen real estate, but we decided that added a user preference to hide the buttons would add too much complexity (the more user preferences there are, the more complicated it becomes for people to figure out what they can set in preferences, so we try to limit the options to things where there is a lot of demand for a setting). Instead, we added some extra code to our buttons so that they can be selected with CSS, so that people can build skins which hide the 'Share' button (or indeed any other button).
  • Suggestion for a 'challenge calendar' listing opening and closing dates for challenge sign-ups, and dates for assignments due, works revealed, authors revealed, etc, which can be opted in when a mod creates the challenge. We loved this idea, but it is fairly complex to implement. Our lovely co-chair Amelia has volunteered to put together a design, so this is something we'll introduce in the future - but probably not for a while.
  • Request for a way to mark WIPs as abandoned, and a way to offer abandoned WIPs up for 'adoption' so that someone can finish them. We all agreed it would be really nice to have a quick way to flag that a WIP would never be finished, so we've logged that as an issue for a coder to implement. The idea of offering works up for adoption seems like it might have more limited appeal, so we agreed that for now, it would be better to leave this as something which people can simply indicate in the tags they use, if so desired (you can add 'Adopt this story' or indeed any other tag you wish as an additional tag to your work).
  • Reflecting on Release 0.8.9

    As most of you reading this will know, we had a big release of new code at the beginning of November. This release included a lot of exciting new stuff; unfortunately, it didn't go as smoothly as we had hoped. In this meeting we reflected on problematic areas and ways that we can improve in future:

    • We combined two big new features: the redesign of our front-end code and the new tag sets code for challenges and collections. We had decided to combine the two because the tag sets needed some front-end work anyway, and at the time we made the decision it made sense to roll the two things into one. However, the tag sets code was time sensitive: because it offers a new system of challenge nominations which significantly reduces the pressure on tag wranglers, we wanted to implement it in time for the big holiday challenges such as Yuletide. This meant that when we combined the two features, we had a lot more stuff to get ready within a set amount of time, which made everything more difficult. When we decided to merge the two, it didn't seem as if this was going to be a problem - but one thing we've learnt is that any deploy can bring unexpected hitches, so in the future if there's a time-sensitive feature we'll be trying to keep that as separate from other code as possible.
    • This was a big visual change, which meant that it had an impact on a large number of users: visual bugs tend to be encountered by lots of users, and even if there are no bugs, people still have lots of feedback about visual changes. We were aware of this; however, given the scale of the response to this deploy we realise we weren't prepared enough. We'll be doing more testing of interface changes in future, and exploring ways of beta-testing them with more users.
    • Since one thing about visual changes is that lots of people just prefer the design they are used to, one thing we could have improved on was providing a way of going back to the old default design. We tried to provide for this with the One Point Faux option, but it had quite a few problems. So, in future this is something we'll be paying more attention to: if we introduce a big change, we'll try to provide ways of opting out. The good news is that going forward, this will actually be easier, because the new skins system is much more lightweight and it should be easier to provide some backwards compatibility (one reason this was problematic this time is because the underlying code for the old system was less than ideal, so everything had been completely rewritten).
    • We didn't have as much support documentation and information as we really needed for this deploy. In particular, we needed much fuller documentation on the new skins features so that people could try them out more easily and our Support team could point to useful information when helping people. There were several factors which led to a lack of documentation: crucially, several of the team who would normally take care of this were dealing with RL issues which limited the amount of time they could spend on it. In order to help avoid problems like this in future, we're building a deploy checklist which includes documentation, to make sure that we've considered whether we need additional documentation regardless of who is available to work on any given deploy. We're also aiming to build up a proper documentation team so that this work is less likely to fall through the cracks: if you're interested in being involved in this team, get in touch with our Volunteers and Recruitment Committee and let them know. We'd love to welcome new people to the team!
    • We also needed more documentation on the new features for testers, so that it was clearer what people needed to test and what they should expect it to do. This is an ongoing aim - we're working to improve our documentation across the board. Improving documentation for testers will also help us to address another issue, which was that feedback from testers got a bit scattered - having clear docs to start with would have helped us make it clearer what feedback needed to go where. Again, we're working on building up our testing procedures generally - if you're interested in getting involved with testing, let us know!

    While the problems we had with this deploy did highlight a number of areas where we need to work to improve, it's not all doom and gloom! There were also a number of things that went right with this deploy - we were able to fix critical bugs within 48 hours of the deploy, the Support team did a wonderful job keeping up with the many Support requests, and there was a huge amount of awesome code in the deploy itself. One reason the site is still in beta is that we're still learning the best processes for development (as well as because our code is new and rapidly changing): in the last four years we've gone from being a tiny group working on coding a 'closed' site (i.e. for the first two years we were just writing the code and testing, we didn't have any real users) to being a much larger group catering for a site of over 28800 users! So, we're still figuring things out - objects may still shift in transit! We're pleased that we've been able to keep the site up and running, and everything largely functional, even though we've had the odd bump along the way. Thanks to everyone who has worked hard to make this true!

    Next deploy

    We're hoping to get one last deploy in before the end of 2011! It will include some updates to our HTML parser, some improvements to our static pages for collections and challenges, and Atom feeds for fandom tags! (YEAY!)

    News from our sub-committees

    • Coders have been working on polishing off the issues to go in the next deploy. We're particularly excited about the forthcoming addition of Atom feeds for fandom tags - having tested this out for a good while now on the F/F tag, we think we can implement feeds without too much additional strain on the servers, and since this is a very popular request we're excited about launching it!
    • Testers have been testing the issues for next deploy, and discussing how they'd like to see the subcommittee develop next year. There have been some great discussions on what worked and what didn't this year, how we can build a stronger testing community, and how we can support our testers.

    News from our sister committees

    • Support have continued to work amazingly hard keeping up with a high number of tickets. Looking forward, they're also thinking about our documentation needs and places we need more information for users.
    • Tag wranglers have been discussing needs for next year with AD&T - the two committees will be meeting in the new term to talk over technical needs for tag wrangling. They've also been surveying all tag wrangling volunteers about their experiences this year, with a view to figuring out what works well and what can be improved on.

    If there are things you'd like to do or say, please share them in comments, via the AO3 support and feedback form, by volunteering (we won't be taking on new volunteers until the new term, but you can get in touch now to let us know you're interested), or in whatever medium you feel comfortable with. Everyone is welcome to this party!

    This meeting round-up by Lucy

Comment

Problems with imports from FF.net

Published: 2011-12-21 09:00:03 -0500

As a number of users have already noticed, imports from Fanfiction.net are not working at present. Our coders have investigated, and it seems that FF.net is blocking requests from our servers. We're not sure at this stage whether this is specifically directed at us or whether it's a side effect of some other action; either way, it's not an issue that we can fix on our end. We're currently reaching out to the folks at Fanfiction.net to see whether we can resolve the problem; however, at present we don't have any ETA for a fix.

We apologise for the inconvenience - thanks for your patience while we figure out this issue!

ETA: This problem also affects bookmarking of works hosted on FF.net, since making an external bookmark involves pinging the site to check it's a valid url. We'll keep you updated as we figure out ways forward with this problem.

Comment

Release notes - release 0.8.10.10

Published: 2011-12-01 04:47:32 -0500

Welcome to Release 0.8.9.2. Elz, Enigel, Firewolf, Jenny, Naomi and sarken contributed code to this release. We're especially excited to welcome a first commit from Firewolf, who recently joined us for a coding internship \0/. The release was tested by Jenn_Calaelen, Tai , XParrot and Zebra. This is a small release, mostly fixing a few urgent bugs.

Highlights!

Rich text editor (mostly) fixed

A number of users have experienced problems with the rich text editor failing to load. We have now fixed this issue; however, we're still working on another bug with the rich text editor. So, most users should be able to use it without problems now, but if you're using Opera 11 or Internet Explorer 9, it will load but not let you actually edit. :( We're sorry about this - we're working to fix the bug completely.

Temporary authentication problem

A user alerted us to an authentication bug! We're happy to say that we were able to fix the issue within a few hours of it being reported and as far as we are aware no users were affected. Thanks to the user who let us know about the problem!

Fixes for collections and challenges

This deploy includes a bunch of fixes for problems which were making life difficult for challenge owners and participants. Thanks to those affected for their patience - we know that bugs in challenges are particularly annoying since most challenges are time sensitive, and we do our best to fix them speedily.

Known Issues

See our Known Issues page.

Release Details

Features

  • Added a page letting people know how they can support the Archive! :D
  • Comments are now paginated, 20 to a page, so if you're reading a work with a lot of comments they won't take forever to load.
  • User inboxes are now paginated, 20 items to a page, so they won't take forever to load.

Bug fixes

  • The rich text editor now works again for most browsers (sadly we are still dealing with issues in certain browsers).
  • Fixed a bug with user authentication which would have made it possible for people to edit tags on works they didn't own.
  • Chapter navigation is now styled in the same way whether you're viewing a full work or chapter-by-chapter.
  • Fixed a bug causing lots of excess white space on wrangling pages.
  • Collections and challenges
    • When a challenge allowed category, rating or warning tags in signups, trying to edit your signup gave a 500 error. This is now fixed
    • Going to the index page for a collection with subcollections gave a 500 error. Fixed!
    • The 'pinch hits' and 'open' sections of challenge assigment pages will now load.
    • Pinch hits now show up on challenge assignments lists (so mods can tell whether pinch hitters have posted or not).
    • Pinch hits which have been fulfilled (or formally defaulted) now no longer show up as an option when you post a new work.
  • Inbox, comments, kudos
    • After you comment on a chaptered work, you will be redirected to your new comment and (if you are viewing chapter-by-chapter) to the chapter you were on, instead of being bumped back to the beginning of the work.
    • After you leave kudos, you will be redirected back to the page you were on instead of being bumped back to the beginning of the work.
    • You can now reply to comments on a chaptered work with javascript off (before, clicking reply redirected you back to the first chapter).
    • If you filter your inbox and then reply to a comment, when you're redirected back your filters will be preserved.

Tests

  • The test for kudos has been refactored.

Comment

The Accessibility, Design and Technology committee oversees technology-related projects within the OTW. Currently we are responsible for designing and building the Archive of Our Own. Our regular meeting updates keep you informed about developments on the AO3!

AD&T and our associated committees and subcommittees have been very busy recently working towards our latest deploy and then working on issues arising from that. This one didn't go as smoothly as we had hoped (understatement!); we knew that there would be bugs revealed by practical use that didn't appear in testing, though there were more than we had anticipated and we have been working hard to fix the immediate issues. We're happy to say that we were able to fix the most pressing problems within 48 hours; a week on from the deploy we've been able to address quite a few more, so those fixes will be deployed soon. We're really grateful to everyone who worked hard on this deploy and on addressing the issues subsequent to it. We are planning to do a thorough review of the deploy and think about the lessons learned and ways we can improve. This week, however, we focused on working through some outstanding business and outlining the tasks we need to complete before the end of the year.

Meeting highlights!

Goals for the rest of the year

We're drawing close to the end of the 2011 term, so we started to think about what our priorities are for the rest of this year. On December 16th all the OTW committees officially dissolve and we take a break before reforming in January (although in practice many members of AD&T tend to do quite a bit of work during the hiatus, heh). So, we talked about a few things we'd like to get done before then:

  • Coding! Several people have code in-progress which they'd like to do some serious work on and hopefully finish by the end of the year. Site navigation, bookmarks improvements and a refactoring of our works code (important for tons of other improvements) are high on this list!
  • Revising our roadmap. We have a longterm plan for what features we'd like to implement on the Archive and when. However, we don't always implement things in exactly the order they are in on the roadmap: we have to be flexible and adapt according to a range of things including the pressing needs of the Archive at a given time, the coding expertise available, the level of difficulty involved in a specific project (this is not always as anticipated), and a bunch of other things. We also add new things to out to-do list based on feedback from other fans. So, we have to review our roadmap regularly to ensure it reflects our capabilities at any given time: right now it's out of date, so we'd like to get it updated to help shape plans for the coming year.
  • Completing our archive import code and rescuing some at-risk archives! Coder Naomi is currently putting the final touches to the code which will allow us to rescue archives which are no longer able to exist independently. Our immediate priority is the Smallville Slash Archive, which was hosted by the late, great Minotaur: several fans have worked hard to preserve this bit of fannish history, but it can't hold out much longer, so we want to help them out by the end of the year. We talked about the various things we need to do to make that a reality - stay posted for more news on this soon.
  • Reviewing our testing procedures: One area we'd like to improve and develop more support is in our testing team, where a very small number of people do very dedicated work. Our testing lead Kylie will be hosting a meeting for current testers to think about what works well and what can be improved, to make sure they can continue to work well into the future.
  • Exploring our tag wrangling options. The Tag Wrangling Committee and the big team of wranglers they manage do an amazing job at keeping the many, many user-generated tags on the Archive in order. However, we're growing at a massive rate and their job has become significantly bigger over a very short space of time. So, we want to talk to the TW Committee now to see if there are technical improvements that could make their lives easier, if the current system is still right for us, and whether there is anything else to think about from a tech point-of-view.

Issues for love!

As part of our recent drive to address features requests and feedback via support more quickly, we had a big drive in this meeting to burn through some of the 'issues for love' which are awaiting committee discussion before they can move on to the next step. A few of the things we discussed:

  • Improving bookmarks: We get lots of support requests asking for more sorting and filtering of bookmarks, and our Support team wanted to know how our plans on this were progressing. We're happy to say this is being actively worked on and we hope to have it out by the beginning of next year.
  • Donate to the Archive: We also get quite a few Support requests asking how to donate time or money to the Archive. We've long been meaning to make this much clearer on the site itself, and we're happy to say that the code for a page with this information has now been submitted by our new intern, Firewolf. If you're wondering in the meantime, both these things are handled via our parent Organization for Transformative Works: get in touch with our Volunteers and Recruitment Committee if you're interested in helping out with the Archive, or make a donation to the OTW to help fund the site. Since the committee term for this year is coming to an end and we'll be taking a break, we won't be welcoming any new people to our teams until we reconvene in January, but we still welcome expressions of interest now!
  • Adding more than one related work: A couple of users had contacted Support to say they had works inspired by more than one other work and couldn't figure out how to show this. It is actually possible to add more than one related work - however, due to an oversight you have to add one, post the work, and then edit the second one in. This is clearly not very intuitive, so we're fixing it - and in the meantime we're adding some help text so people can find the workaround while we look at the more complex bit of the code.

Next deploy

The next deploy is scheduled for some time in the next week (depending on the availability of our team, several of whom have holiday celebrations this week). It will include a fix for the rich text editor (currently completely broken for some people), some fixes for oddities in skins, and a fix for index pages on subcollections (currently not showing up!).

News from our sub-committees

  • Coders worked crazy hard to get our last deploy up and running, and then to fix some bugs arising afterwards. They did an awesome job of coming up with quick solutions to some of the bugs that showed up once we were on the real Archive - thanks to everyone for their hard work! More generally, they have been focusing on getting lots more projects out of the door before the end of the year. They are cooking up some exciting stuff, including navigation improvements and importing, so we're looking forward to having these see the light of day.
  • Testers have been super active lately! They tested the latest deploy in all sorts of situations and configurations, and then did more urgent testing to help fix the things that slipped through the cracks. The testing team is small and they do amazing work - thanks lovely people!

News from our sister committees

  • Support have also been working very, very hard dealing with all the tickets arising from our recent deploy. Every new deploy produces an uptick in tickets, because new code inevitably means some new bugs (this is why whenever big companies release a new OS, it's usually followed shortly after by a bunch of updates!). This deploy produced more tickets than usual - a lot more! - but Support have been doing a sterling job keeping up. If you do find they are a little slower than usual, then rest assured they will get to you as soon as possible.
  • Tag wranglers have been awesome helping Support deal with tag-related tickets. The Tag Wrangling Committee gave AD&T some initial feedback on where tag wrangling stands at the moment, pending a meeting when we'll talk in more detail about tech needs for wrangling.

If there are things you'd like to do or say, please share them in comments, via the AO3 support and feedback form, by volunteering, or in whatever medium you feel comfortable with. Everyone is welcome to this party!

This meeting round-up by Lucy.

Comment

Known issues from the latest deploy

Published: 2011-11-14 07:42:49 -0500

As most people have noticed, we have had some problems with our latest deploy. We'll address some of the larger concerns in a later post (right now, most of our staff are busy dealing with the immediate issues), but we just wanted to do a quick update to let people know that we're aware of some specific issues and we're working on them:

  • Mobile display: We had a big problem when we deployed with mobile displays, because the new mobile skins were not kicking in at all. We've fixed the main part of this issue, so the site is now much more usable on a lot of mobile devices; however, we're still experiencing some bugs with mobile displays, causing button overlays, too much white space, and other oddities. We think that the problem is still related to some parts of the mobile stylesheets not being activated properly: we're looking into it.

  • Some problems in Internet Explorer:
    • the commas separating tags are not showing up (some versions of IE) or they are showing up too aggressively, giving a comma at the end of every tag even if it's the last tag in the list.
    • Some people are reporting problems replying to comments in IE.

  • The skins wizard is behaving erratically and inserting code which messes up other things.

If you're experiencing one of the above issues, rest assured we're looking into it. We've already fixed some issues with mobile display, a problem stopping people from bookmarking, the missing OpenID login, and a few other bugs, so if you were seeing something not mentioned on this list, it's worth checking that the problem is still occurring before you get in touch with us.

If you see a problem that's not on this list, do get in touch with Support via our Support and feedback form.

We'd like to say a huge thank you to Support, who have received more tickets in the last 48 hours than they did in the previous 48 days! Despite this massive amount of work they have kept up with replies and right now there are only ten tickets not assigned to someone to be dealt with - GO TEAM! Thanks also to the coders and testers who have worked like crazy to do urgent bugfixes.

Once again, sorry for the problems this deploy threw up: we're working on the immediate issues right now and thinking about future solutions.

ETA: We've received several reports from people who are unable to use the rich text editor. We're having difficulty pinning down this bug since it seems to manifest inconsistently, but we are looking into it. Apologies for the inconvenience.

Comment


Pages Navigation