AO3 News

Post Header

2014-02-20 13:14:30 UTC

Welcome to Release 0.9.13. Ariana, Elz, Emily E, Enigel, james_, Naomi, sarken, Scott, and Stephanie S. contributed code to this release, which was tested by our awesome testing team: Alison Watson, aralias, Ariana, Camilla M, Emily E, Eve Forbes, Lady Oscar, Michelle Dong, Naomi, mumble, runt, Sam Johnsson, sarken, and Scott.

We're starting the year with a much needed performance fix, a major update of our Rich Text Editor, and a few importing fixes to make the lives of our Open Doors team a little easier. We have many big plans for the next several months and are also working continuously, and often invisibly, on upgrading and improving our server setup. For downtime announcements (both planned and unplanned), please watch our @AO3_Status Twitter account.


TinyMCE Upgrade!

We've updated our Rich Text Editor of choice, TinyMCE, from version 3.5.6 to 4.0.11. This takes care of several long-standing bugs in one fell swoop, and enables us to integrate bug fixes more regularly going forward. We took great care to make TinyMCE work properly with our own HTML sanitizer, which formats the underlying code more or less to our liking. However, some combinations of writing software, browsers, and formatting choices still pose a problem that cannot be easily fixed.

In particular, Google Drive/Docs documents will not retain their italic and bold formatting when pasted into the editor, and text from Scrivener is best pasted from the HTML output option. We've included a bunch of notes and usage tips in the new Rich Text help, accessible through the little question mark symbol after "Type or paste formatted text."

We are doing our best to make the Rich Text Editor work for as many writing platforms as possible, but we're limited by the features and bug fixes TinyMCE's own tireless team of developers gives us. For very complicated formatting (or even colors!), please consider writing your own HTML and using a work skin.

Archivist Imports!

Our amazing Open Doors volunteers have been hard at work preserving stories from at-risk archives! With the exception of two very large collections which were imported in bulk (Smallville Slash Archive and 852 Prospect), they've been mostly using the same importing tool that is available to all our users. Regrettably, they've had to work around several bugs for a while now.

This release takes care of a few smaller issues relating to imports and import management, and introduces a fixed and enhanced archivist feature. This allows designated archivists to import works for others, adding the author's handle and email address from the original archive. Archivist imports automatically generate an email notification to the author with links to further information and claim/delete/orphan options. This page was also updated in this release.

Better caching!

When a user accesses a work listing, e.g. all works for a particular tag, we cache that list of works and serve it to other users wanting to browse the same list. This means the information doesn't have to be retrieved from the database every single time, which makes for happier servers. However, somewhere in the code we have to make sure the listing is refreshed when anything changes, e.g. a new work with that tag is posted.

Previously, that check was done whenever a listing was accessed. We have a lot more people reading or looking at works than posting or editing, and those turned out to be a lot of checks: Was a new work added to this collection? Did this user delete a work? Was this tag added to an existing work? and so on. In short, the database server wasn't quite as happy as we wanted it to be.

We now update all data ("expire the cache") as soon as a change happens, not when a work listing is assembled. This required extensive changes to many parts of our code, as a lot of user actions affect work listings. All kudos to Elz for persevering through several rounds of testing and fixing all the things!

Known Issues

See our Known Issues page for current issues.

We've been having some server overload issues, detailed in our post, What's up with the recent downtimes? In order to cut down on excessive reindexing, we're currently not adding works to the reindex queue when their hit counts are updated in the database. Instead, the work information is refreshed when some kind of feedback (e.g. comment, kudos) is left or when the work gets a new chapter, forcing an update overall.

Sorting works by hit count will be slightly inaccurate as a result, and will lag considerably for works getting traffic but no kudos/comments/bookmarks/new chapters.

Release Details

Bug fixes

  • Works
    • The cache expiry for work listings was rejiggered to happen when data is updated, not when a listing is accessed.
    • New hits on a work are recorded in the database, but don't make it into the search index currently. Instead, the index gets updated when other work information (new comment, new chapter, etc.) changes.
    • Work drafts (showing author and title information) were visible on the Related Works page; this has now been fixed.
    • There was an issue where kudos left by a guest weren't properly added to the list (thanks to stuck caching) unless a signed-in user left kudos right after. This has been fixed and all kudos should show up in the list at the bottom of the work immediately.
    • We've had reports of blank kudos notifications for a long while, and were never able to hunt down the cause. This release includes a fix of what we now believe is the cause. We'll be monitoring the situation!
    • Marking a work for later *and* marking a work as read would both bring up the same success message. This was changed to properly indicate that the work was added to or removed from your list. Please note that changes will take a while to show up!
  • Work Importing
    • Trying to import a work as an archivist was throwing an error 500; this is now very much fixed.
    • Updated Claim page for newly imported external authors.
    • Fixed several bugs with importing from efiction archives, e.g. an issue with the same chapter being imported over and over again for certain story links.
    • The code for our Redirect Lookup Tool to find imported works on the Archive was refactored and streamlined.
    • Improved URL matching in the Redirect Lookup Tool. Extra bits such as ?mode=reply or #comments and other slight differences in the link (www/no www) are taken into account when looking for the existing import.
  • Automated Testing
    • We continue to work on updating and expanding our automated tests. To this end, we have begun using a code coverage tool called You can view the current state of our code coverage by visiting our Github page (look for the little badge in the README section).
    • One of our automated testing tools, called Fakeweb, was outputting deprecation warnings every time we ran our test suite. This was inhibiting easy understanding of the test logs, so we have switched to Typhoeus. We also fixed the causes of other deprecation warnings in the process.
    • At one point, an error 500 was generated when attempting to delete your account and remove yourself as an author from co-authored works at the same time. This was fixed in another release, and we have added automated tests to cover this particular account deletion situation.
  • Misc.
    • Signup summary generation was slow and included a query that was causing issues for the database. We made some changes to help it run more efficiently.
    • Fixed a number of issues with the Rich Text Editor, including non-functioning alignment buttons, problems with the browser spellcheck, missing formatting when pasting from MS Word documents, and others!
    • When toggling between HTML input and Rich Text, the help text "Plain text with limited HTML" would be displayed at all times. It now switches to "Type or paste formatted text" when in Rich Text mode. There is also a new help section for the RTE.
    • In the navigation header, put items under "Browse" and "Search" in the same order (Works, Bookmarks, Tags, Collections/People) for consistency