AO3 News

Release Notes for Release 0.9.3 Redux: The Bug-Fixing

Published: 2012-12-10 11:26:17 -0500

While we were happy to get out the last release in time for gift exchange season, there were a few bugs in the code that we didn't detect in our testing environment. This release fixes them.

Systems maintenance - new servers coming soon!

The AO3 and its sister projects keep on growing! Happily, this growth is supported by the generosity of our users - the OTW's last membership drive raised more than US$18,000! Thanks to your support, we're able to expand our server setup further: we'll shortly be adding three more servers to our server family. These will give us more power to run the AO3 and add more space for our databases, while freeing up one of our older AO3 servers for Fanlore (which is also growing, but doesn't need quite as much muscle as the AO3).

Bumps in the road

Our Systems team are currently getting everything set up and ready for the new servers. This involves some maintenance on the existing servers, including some revisions to our deploy script - which is actually a sizable *collection* of scripts, weaving together several bits of backend magic that tell the servers new code has arrived and we need them to update some stuff. Unfortunately, there were some unforseen problems with these changes which caused the deploy script to fail when we tried to update the site last Thursday. Consequently, the code update had to be rolled back, and some caching had to be temporarily disabled. The dates on works suddenly displayed as "2 days ago" instead of "6 Dec 2012" and the servers seemed a bit unhappy for a while there.

After several hours of code prodding and server wrangling, a second deploy attempt later on Thursday was successfully completed. Many thanks to Elz, James_, and Naomi for their tireless work, and to Jenn and Lady Oscar especially for a whole lot of testing! However, the aforementioned bugs now reared their ugly heads, and an emergency deploy (now without any major hiccups!) happened late Friday night in the US, or very early morning for the unfortunate Systems monkey in the UK.

Known Issues

See our Known Issues page for a full list of current issues.

Release Details

Bug fixes

  • The search index for bookmarks was being updated everytime a bookmarked work got a hit or had some kudos added. This was both an incredible strain on the server and completely unnecessary, so now it's not happening anymore.
  • When a logged-out user left a reply comment, the notification email wasn't going out due to a slight code mishap. This was fixed almost immedately after the deploy, this deploy just makes the fix permanent.
  • Trying to reply to the aforementioned comments from the inbox wasn't working either; it does now.
  • A fix in the previous deploy was messing up paginated pages of works, and was also generating pages that weren't different for logged-in users and guests; this has now been fixed.
  • Individually revealing/de-anoning works in an unrevealed/anonymous collection wasn't working; it does now.
  • Works and bookmarks were appearing on a moderated collection's list of works or bookmarks even before being accepted by a moderator; this has been fixed.

Comment

Release Notes for Release 0.9.3

Published: 2012-12-07 06:24:21 -0500

Welcome to Release 0.9.3. Ariana, Elz, Enigel, look_closer, Naomi, Sarken, Scott and Tegan contributed code to this release, which was tested by our awesome testing team: Alison, Emilie, Hele, Jenn Calaelen, Lady Oscar, mumble, Naomi, Sarken, and Scott.

In this code release, we focused on squashing bugs in our collection and challenge code, and we're proud to close over 50 issues as "done!" in our bug tracker this soon after our big tag filter deploy. Unfortunately, we had to iron out some major bumps in the tag wrangling code as well, forcing us to disable wrangling while working on the fix. Wrangling is now back, with apologies to the wranglers for leaving them with the backlog!

Huge thanks to the testing team, to the wranglers for their patience, and to coder Scott for contributing a whopping 25 fixes to the grand total. As always, much more work needs to be done while we're still in beta, and we thank all our users for sticking with us during this journey.

As part of this deploy, we also made some changes to our server setup. We're still ironing out a few glitches which happened in relation to this, and we had a little more downtime than we expected. Big thanks to James_ for his hard work on this Systems maintenance, and thanks to our users for their patience.

Highlights!

Anonymously commenting on your own anonymous work

Just in time for several gift exchange deadlines, we've introduced a small enhancement that will allow you to respond to feedback once works have been posted, but before authors are officially revealed. This applies to any of your works in a collection (or challenge) that has been set to Anonymous by the collection owner: When you reply to a comment, your name in the thread and any outgoing comment notifications will be masked as "Anonymous Creator", and will be automatically switched to your username once authors are revealed. \o/

Fixes to the search and filter functionality

While it was previously possible to put something like "words: 100" into the main search box (finding only drabbles in any fandom), the recent filter deploy did not include that functionality. Yet! The search code has been adjusted and sorting options have been added to make the following possible:

words:1000, words=1000, words>1000, words<1000
- To find works with exactly 1000 words, more than 1000 words, or less than 1000 words. Replace "words" with "hits", "kudos", "comments", or "bookmarks" to search for those instead.

sort:kudos, sort=kudos, sort<kudos, sort>kudos
- To sort search results by the number of kudos a work has. The default order here is descending ( to reverse that order to ascending. You can replace "kudos" with "author", "title", "hits", "comments", "bookmarks", etc. to sort by those.

Furthermore, you don't have to enclose f/f in quote marks anymore when searching for femslash (or when searching for M/M or F/M works) - just add f/f to your other search terms, if any, and you're done.

Fixes and improvements to search results

When the wranglers make changes to tags or their relationships, our search engine needs to be told to reindex (that is, re-check) the works and bookmarks under those tags, so that search and browse results will continue to list the right works. This wasn't happening properly with all subtagged works in the previous deploy, so as a result we had to turn wrangling off until we could fix it. Works and bookmarks will now be properly reindexed in response to tag wrangling changes, so the wrangling can recommence!

In addition, sorting by hit count was going out of order because the search engine was not being told to reindex works when they got new hits, only when they got new kudos/comments/bookmarks. We can't reindex after every hit for performance reasons, but we will now reindex every ten minutes, which should keep the hit counts from going out of date.

Sometimes, when sorting lists of works, you may find that some works seem to be out of order. For example, you might sort by number of kudos and find that a work with 2 kudos shows up above one with 6. Fear not, your sort order is right! It's just that we cache some bits of the views, so sometimes you see an old count displayed on the work list. This is a bit confusing, but it does help site performance.

Known Issues

In order to address a bug which emerged during this deploy, we had to disable part of our caching which relates to the way dates are displayed in works lists. This means that where it used to show the exact date a work was posted, it currently just says a timeframe such as 'one day ago'. This is temporary - we already have a fix for the underlying problem and just need to test and deploy it. Please bear with us in the meantime!

See our Known Issues page for a full list of current issues.

Release Details

Features

  • When replying to comments on your work while it's in an anonymous collection, your name will be masked as "Anonymous Creator", to be revealed once the work is revealed.
  • Collection owners will now be notified via email when a new work is added to their collection. This email is sent to the address entered on the 'Collection Settings' page (which needs to be present), making it possible to use a shared email address among challenge moderators, instead of just the owner's account address.
  • When users are 'Posting to Fulfill' on a Prompt Meme challenge, the prompt which is being filled is now automatically inserted into the 'Notes' section of the posting form. The section is automatically ticked and expanded so users can see the prompt and remove it if they wish.

Bug fixes

  • Collections
    • Users trying to add a work or bookmark to a moderated collection are now notified when posting that their work or bookmark will not show up until it has been approved by a collection maintainer.
    • Owners and moderators of collections now have the ability to post to the collection even if the collection is set to closed.
    • There was an issue with work data not being updated properly if the work was taken out of a collection, e.g. a work that had been removed from an anonymous collection would still remain anonymous; this has been fixed.
    • A collection's work count was including 'locked' works for non-Archive members as well, causing a discrepancy between the number count shown and the actual number of works listed. This has been fixed to only count publicly visible works when a site user is not logged in.
    • A collection's bookmark count was counting private, and thus invisible, bookmarks. The count has now been fixed to exclude these.
    • A user's work count was including works that were in a hidden challenge, causing a discrepancy between counted and actually shown works. Works are now only counted if they are revealed.
    • Remixes and translations of unrevealed works were showing the name of the inspiring (unrevealed) work instead of the inspired work, breaking the unrevealed status of the inspiring work; that logic has been reversed and work titles are now shown as expected.
    • Commas used in a collection's 'Display Title' were causing internal code errors. An error message is now generated when attempting to use a comma in this field.
    • Previously only collections which a user owned showed up on the user's 'My Collections' page. This page now includes collections which the user maintains (e.g. helps moderate) as well.
    • The placement of the "Update" and "Delete Collection" buttons (as well as a useless "Back" button) on a collection's settings page was confusing and inconsistent with the rest of the site; this has been rectified.
    • Work blurbs on the 'Collection Items' page were confusing. They have been reformatted to easily show the most important information about a particular work.
    • The "You can also individually Manage Items" text and button were formatted oddly on the 'Edit Collection' page; this has been fixed.
  • Challenges
    • Challenges which were closed were wrongly appearing on the Open Challenges page, and generally the page wasn't as useful as we intended it to be. Bugs have now been squashed and sub-pages for Gift Exchanges and Prompt Memes (respectively) have been added. Closing sign-ups or changing the sign-up deadline to a past date removes a challenge from the relevant list.
    • There was some inconsistency in navigation buttons displayed in the information blurb for collections and challenges (such as on the main Collections index), leading to empty button code; this has been adjusted to always show Join/Leave and Sign Up/Edit Sign-up/Cancel Sign-up options as appropriate.
    • After a gift exchange owner had generated potential matches for participants, the (possible) assignments were already showing up as options when participants went to post a new work; this has been fixed to only display assignments in the posting form once finalized and sent out.
    • When defaulting on a challenge assignment, the assignment would be updated accordingly in one own's assignments, but still be available for defaulting and fulfilling on the collection's page; this has been adjusted.
    • When fulfilling an assignment and then deleting the work (e.g. in a blind panic five minutes before the challenge deadline), the assignment would vanish as well. It now reappears to be fulfilled again.
    • The 'Post to Fulfill' button was showing up even on challenges that were no longer accepting submissions. This button is now visible only if the challenge is open.
    • Users attempting to use non-canonical tags in a challenge signup were given an unhelpful error message. Users are now directed to contact the challenge maintainers (who can contact Support to fix the issue if necessary).
    • In challenges using tag sets for sign-ups, tags weren't properly displaying in autocomplete fields if they had been marked as canonical before being linked to other tags (e.g. characters to their fandoms) by a wrangler; this has been fixed.
    • The 'Categories' field on a Gift Exchange signup was not clear about asking for types of relationships (M/M, F/M, etc). A help popup has been added to alleviate confusion.
    • When users attempted to view a restricted work from a collection's "static" page (a feature created to reduce server load during busy challenge times), they were receiving a 404 error message; this has been fixed.
    • Gift notification emails (such as sent out in a gift exchange challenge) were linking to a work's general URL instead of the work's URL inside the collection; this has been fixed.
    • After the creation of either a Prompt Meme or a Gift Exchange the user was taken to two different places depending on the type of challenge created (collection dashboard or collection profile). Now the landing page for both challenge types is the collection's profile page.
    • Prompt Meme request forms were asking for tags called "Freeforms", which are called "Additional Tags" elsewhere; we changed the code to make it easier to adjust the name everywhere and now they're called Additional Tags here as well.
    • A bug was allowing the 'Allowed Prompts' on a Prompt Meme to be set to zero, causing some logic problems. It now requires at least one (1) prompt for sign-ups to work.
    • Prompt Meme request submissions were allowing an unlimited number of characters in the 'Prompt Description' field, but cutting off after a certain number without warning. This has been resolved and descriptions are now limited to five thousand (5,000) characters.
    • In the work 'Notes' section, the link to a prompt that was filled led to a page not viewable by non-collection members. The notes now link to the user and the challenge that the prompt fill was for.
    • Challenge maintainers cannot edit prompts submitted to their challenge, yet the button to do so was displayed for them. The button has now been removed to properly reflect options available to Prompt Meme maintainers.
    • The title text for the "Choose Not To Use Archive Warnings" icon in a work's blurb was broken when no warning had been selected in a prompt meme request; this has been fixed to display the "Choose Not To Use Archive Warnings" instead.
    • In the challenges area, our front end code was hard to read for coders and some of the user-facing text was inconsistent. Now it's pretty and consistent, and also better prepared for eventual interface translation.
    • The buttons to manage assignments on the 'My Assignments' page were ordered as "Default" first, then "Fulfill"; this was different from the order of "Post" and "Cancel" buttons elsewhere on the site and was reversed.
    • On the challenge settings page, there was a white margin underneath the footer that has now been eliminated.
    • On the same page, the date picker pop-up helping you enter start and end dates for your challenge was prettied up and misaligned buttons were aligned properly.
    • The challenge signup summary page is now presented as a table for easier readability.
    • In Internet Explorer 9, the 'Update' button on the challenge's settings page was covered and unclickable; this has been fixed.
  • Searching & Browsing
    • Way back when we deployed the filters, filtering was broken on user works pages. We did the fix live, which means it didn't get added to our code base. Now it's in there and will not re-break when we deploy new code.
    • We also fixed a search indexing problem way in the backend that affected some bookmarks.
    • Some bugs in our code prevented work listings to update properly when tags or tag relationships were changed by the wranglers; this has been fixed.
    • Sorting by hit count was giving wonky results due to the way the numbers were updated for each work; the new code should show works in a more proper order.
    • Orphaned and deleted works were lingering in work lists due to faulty caching; this has been fixed.
    • Using the main search bar, or the "Search within results" box in the filter sidebar, you couldn't do any numerical searches (such as works with a certain word count, or a certain number of kudos); this has been fixed.
    • However, the main search field in the header was turning the > and < symbols into their HTML counterparts - &gt; and &lt; - making some numerical searches impossible; this has also been fixed.
    • After the last deploy, the search hints that pop up when hovering over the main search field in the header were partly outdated; this has been fixed. Examples include: katekyou "alternate universe" sort:>words to find AU works in the Katekyou Hitman Reborn! fandom, shortest works first; and arthur merlin words>1000 sort:hits to find works of over 1000 words that have both Merlin and Arthur (not necessarily paired up), works with the most hits on top.
    • A tiny typo in the filter help text has been fixed.
  • Misc.
    • An option was added that allows admins to temporarily turn off the "Request more invites" form for users, e.g. to control growth during times of heavy server load.
    • Our error 502 page (indicating heavy server load) was still displaying information about performance problems from back in May; this has now been removed.
    • When looking for works for a non-existent tag, a worrying number of error 500 bombs would go off in the background. It now produces a simple and much more appropriate error 404 page.
    • On mobile browsers, the bookmark index pages had a large right margin due to a display bug; this has been fixed.

Comment

Release 0.9.2: The Return of the Filters

Published: 2012-11-08 03:01:49 -0500

Welcome to Release 0.9.1, Part 2: The Filtering; also known as Release 0.9.2 to its friends. Some context for this deploy can be found in our post, Release 0.9.1: An Unexpected Journey.

This code update contains a massive amount of changes, improvements, fixes and happiness, almost all provided by one of our most senior coders, Elz, who now gets to nap for a week. Elz has worked tirelessly over the past months to re-write the tag filtering code from scratch and implement a host of related improvements. Leading a revision like this is an enormous task for a fan with a day job, and while we are aware that users were missing the filters dearly and the planned release date had to be moved quite a few times, we hope all the awesome will more than make up for it!

Elz, mumble, Lucy, Sarken, and Scott all contributed code to this release. It was tested by Alison W, Ariana, bingeling, briar_pipe, Emilie, Fishie Mishie, Hele, Hill, Irene U, Jenn Calaelen, Kylie, Lady Oscar, Lucy, mumble, Radka, Sam, Scott, and supergreak. In addition, lots of other OTW volunteers and members of the Yuletide community pitched in to help with the final testing of the filters - not everyone left their name along with their comments, but everyone's contribution was greatly valued. Feedback from this wider stage of testing will also be used to inform further revisions of the filters: we got lots of good suggestions, but decided to limit the number of enhancements at this stage in order to avoid delaying the main functionality.

Thanks to all our users for your patience while we worked on this - we hope you'll join us in a big THANK YOU to Elz! Our Systems lead James also deserves a special shout-out for his dedicated work and advice: the changes we've made required some major systems-related work. Thanks also to the very many other people who worked on this deploy: testing code, providing advice and support on specific coding challenges, implementing the necessary database changes, and doing all the other tasks that make a change like this possible. <333

Highlights!

New tag filters \0/

TAG FILTERS ARE HERE!!!!

Yes, this release sees the return of the much-missed tag filters, in a new, improved form. Our search and browse functions are now built on Elasticsearch, a far more flexible search engine which uses Solr to perform searches without accessing the database every time. This means that the new filters are MUCH more lightweight than the old ones and won't make such heavy performance demands. HURRAY!

We've written a handy guide to searching and browsing under the new system which walks you through all the details of the new filters. The new filters restore all the functionality of the old ones, with a few extra features. You can now:

  • Sort by number of kudos, comments, or number of bookmarks received! We received lots and lots of requests for this option, so we're very pleased to have been able to implement it.
  • Search within your results and exclude certain tags! The new search box which has been added to filters allows you to narrow down the results of your search even further - see our post on searching and browsing for more details on how it works. It isn't quite the "NOT this" filtering option that users have asked for in the past (that was beyond the scope of this deploy), but it provides at least some of the same functionality.

Bookmark filtering! \o/

A great side effect of rewriting the work filters was that we were able to apply the new filters to bookmarks as well: you can filter all bookmarks for any canonical tag (fandom, relationship, widely used additional tag, etc.) just as you would works, with a few differences. The sorting options are Date Bookmarked (fresh bookmarks, regardless of when the work was posted) or Date Updated (bookmarks of fresh works, including WIPs that recently had chapters added), and you can choose to only display bookmarks marked as a rec, or only bookmarks with some notes added by the bookmarker.

Oh, and we added filtering options to personal bookmarks while we were at it. That's right, one of the most requested features made it into this deploy as well. Users have saved over 1.3 million bookmarks so far, and we're happy to make it easier for you to browse, find and organize them. They have all the features of the work filters, and sorting by Date Updated will show you at a glance which of your bookmarked works has new chapters waiting for you!

New search forms for works and bookmarks

Our Advanced Search form (or simply, Work Search) has been thoroughly redesigned to give our users a more intuitive access to the powerful search feature which we've been offering all along.

All fields in the tags chunk were previously smushed into one "Tags" search field, encompassing everything from fandom to warnings. Naturally, this wasn't obvious from a look at the search form and left many users unsatisfied. The field has now been broken down into options for each kind of tag, making it much easier to see how to search for specific things. The whole form has been reorganised, with similar types of information grouped together, to make it much easier to use. We've also added sorting options for search results. See our guide to searching and browsing for some more tips on how to use it!

New works and bookmarks index pages

The main /works and /bookmarks listings will now only display the 25 most recent works or bookmarks. This lets users have a snapshot of the very latest activity on the Archive, without causing performance problems or being impossible to navigate.

If you're searching for things to read, watch, look at, or listen to, we recommend starting on the main fandoms index or a particular fandom page and working your way down the rabbit hole from there. Tag and author pages will still show all works and provide filtering options. (Hey, remember the 1000 works limit of yore? Gone forever!)

In general, we'd like to move away from listing ALL the data and focus instead on giving users tools to find the things they're looking for. For those who like to browse without any particular goal in mind, we are working on ways to access, for example, new or popular works or hidden gems, and explore the Archive in more interesting ways than a main index of unsorted data. We stopped listing every single Archive member on our /people page when we found that it needlessly added to our performance problems, and we'd like to continue in this vein going forward. Please bear with us while we work on browsing improvements and slowly phase out the main index pages in their current form.

Known Issues

See our Known Issues page for current issues.

Release Details

Features

  • New and improved filtering features for works and bookmarks
  • New and improved search forms for works and bookmarks
  • Personal bookmark filtering
  • A dedicated page for a user's works in collections and challenges, including unrevealed and anonymous ones, for at-a-glance management and editing access, to be found at archiveofourown.org/users/YOURUSERNAME/works/collected
  • To legally host a fanmix on the AO3, you can now embed playlists you have created on 8tracks or Spotify (taking care to follow their own TOS when you do so)

Bug fixes

  • Bookmarks & Works
    • When looking at bookmarks for a metatag (such as Star Trek), only bookmarks with that exact tag would be listed (when it should pull up bookmarks for all its subtags as well); this has been fixed
    • Conversely, bookmarks would sometimes show up on fandom tag pages they didn't belong on; this has also been fixed
    • In the Share box for quickly copy-pasting work info, the Twitter button was slightly broken and didn't fill in title and author anymore when creating a new tweet; this has been fixed
    • The text in the Rich Text Editor was teensy-tiny thanks to the default size in TinyMCE's stylesheet; this should be fixed now
    • Links to the Tags on the Archive FAQ have been added to the help text that pops up for the different tag types (Fandom, Relationship, Additional Tags, etc.) on the "Post New" page
  • Filters & Search
    • Admins are now able to temporarily turn filtering off for works and bookmarks index pages, meaning the index pages for users, tags, collections, etc. would remain accessible even if the underlying search engine was down or under heavy load
    • Searching for an author's works would break anonymity by displaying concealed ("mystery") blurbs of their works, or of works that had been orphaned by them; this has been fixed
    • Relatedly, sorting works by author would make anonymous authors guessable by listing their anon works together with the rest; this has been fixed by treating Anonymous as its own author and shuffling all their works to the top
    • Search results would previously include blurbs of members-only works even for logged out visitors; this has been fixed to hide all traces of locked works when searching
    • Searching works by tag wasn't bringing up works with related tags; it has now been modified so that searching by a canonical tag will bring up works with its synonyms and subtags as well, while putting in a non-canonical tag will restrict results to that tag only
    • For performance and caching reasons, the kudos, comment and bookmark counts (as well as hits) that are used for searching and sorting works are now updated periodically instead of instantaneously, which means ever so slightly outdated search results at a given moment, but also happier servers
    • (Tiny frontend issues for 200, Alex!) If a search returned 0 results, some empty list mark-up would linger unnecessarily; this has been cleaned up
  • Collections
    • Sorting collections by size (i.e. the amount of works in them) was taking a long time and put an undue strain on the servers; we have removed this sorting option until a better solution is found
    • Using the "New Subcollection" button would lead to an error 500; this has been fixed
    • On collection dashboards containing both works and bookmarks, the "Recent bookmarks" section was styled differently than the "Recent works" header; this has been fixed
    • When uploading a header image for a collection, it would show in the footer as well, creating problems with the footer navigation; it will now only be used for the header
  • Notifications
    • In subscriptions emails, the link to series pages was broken; this has been fixed
    • Several email providers (Gmail especially) keep eating our account activation emails, preventing users from finishing the sign-up process; we thought we had conquered the issue when we changed the subject line to something less looking like spam; we have now also expanded the email text to hopefully appear more legit to spam filters
    • Relatedly, "We strongly recommend that you add do-not-reply@archiveofourown.org to your address book to prevent the invitation email from getting blocked as spam by your email provider." was added to the little note that appears after you've added yourself to the invite queue
  • Misc.
    • Using the Reply, Edit, or Delete buttons on comments would jump the page to the top of the comments section; this has been fixed
    • Certain wrangling actions, such as trying to change the canonicals of a tag with lots of subtags or associated works, would time out with an error 502; to fix this the database requests associated with that action have been adjusted to run in the background instead of being (unsuccessfully) applied right away
    • When logged in as an admin, the notes section on bookmarks would overflow the bookmark blurb; this has been fixed

Comment

Release 0.9.1: An Unexpected Journey

Published: 2012-09-22 17:38:41 -0400

The Fellowship of Deploys

Release 0.9.1 will be split into two releases, as the final parts of the filter rewrite will need a little bit more testing time and we wanted to get out these fixes as soon as possible. We are now entering the very last phase of poking and prodding the new filter code (and completely revamped advanced search form) until we can be reasonably sure that nothing will break upon deploy to the Archive. We apologize for the delay in this!

Part 1: The Two Bugfixes

(It's actually around a dozen bugfixes and improvements, but once you commit to a theme, you commit.) Tireless coders Ana, Elz, Enigel, Sarken, and Scott contributed to this release. Testers Jenn Calaelen, Lady Oscar, mumble, Radka, and Sam J. helped confirm that all fixes were working as expected. This deploy includes the following fixes:

  • Design/Display
    • Our intrepid frontend lead Sarken completely revamped the bookmark blurbs and tidied up bookmark listings, making both more useful and easier to navigate, offering more information (such as word count, chapters and number of hits, date last updated, and collections the bookmark is in) at a glance, and basically improving all the things
    • There was a hidden header and help text link on the filter box, only accessible to screenreaders or through keyboard navigation; the fieldset legend and little blue question mark are now visible
  • Works/Tags
    • There was a critical bug that prevented updated works from being bumped to the top of the relevant tag pages (e.g. fandom, characters, relationships...) and would only update the tag pages with newly-posted works; this has been fixed and works that you update with a new chapter should now correctly appear
    • We found an issue where changing a tag name on the admin side would not update the autocomplete options for users, leading to problems when a user selected the old, but now unwrangled tag in some contexts; this has been fixed
    • In the same vein, a collection title change would not update the title in the autocomplete options when trying to add a work to a collection, creating the impression the collection didn't exist; this has also been fixed
    • When a work or series you subscribed to was deleted, your subscriptions page would throw up an error 500 and become inaccessible; we thought this fixed in a previous deploy but tracked down a lingering issue when we were still getting reports of this problem and should be in the clear now
    • For works with many comments, paging to the next set of comments would bump you down to the bottom of the page, forcing you to scroll up to read the comments; paging through comments will now keep you at the top of the comments section
    • In preparation for more backend changes, we renamed the database table for counting hits and widened its scope to keep track of more work stats as well; this shouldn't affect you right now
  • Notifications/Emails
    • We've been experiencing problems with activation emails relentlessly getting eaten by spam filters, resulting in a dozen Support tickets every day to take care of account verification; after some analysis we found that our subject line resembled common spam patterns; we have now renamed the emails to say "[AO3] Confirmation" and will track if this improves matters
    • Related to this, we've fixed a loophole making it possible to request a new password for an unverified account, ultimately resulting in an error 404 when trying to use the new password; users are now asked to contact Support if they didn't receive an activation email instead of being sent a new password

Please note: If you want to ensure that all Archive emails, including subscriptions and comment notifications, reach your inbox, please add do-not-reply@archiveofourown.org to your contacts and/or follow these unofficial instructions for whitelisting all emails from the Archive. Gmail especially seems to place emails from our address in spam folders at random.

Part 2: The Return of the Filters

As most of our users know, we've been working hard the last few months to rewrite the code for our tag filters (we touch upon some of the problems with the old filter code in our post AO3 performance and growth: some details). This deploy doesn't include the new filters, but it does include lots of the background work which is needed to support them. The rest of the code will be in our next deploy - the new filters don't look too different from the old ones, but they are much spiffier and more efficient behind the scenes.


A sneak peek of the new filters (frontend); not pictured: massive backend changes for scalability and maintainability

Comment

Release Notes for Release 0.9.0

Published: 2012-09-03 16:04:50 -0400

Welcome to Release 0.9.0! Andreja, Ariana, Chris, Elz, Enigel, Lal, Nathan, sarken, Sam J. and Scott contributed code to this release, which was tested by our awesome testing team: bingeling, Enigel, highlander ii, Jenn, Kylie, Lady Oscar, sarken, Scott, Xparrot and Zebra.

We have a lot of fixes and improvements in the pipeline, including improved search and filtering for works and bookmarks, but because of the backend work and testing involved with that, we're splitting the release up a bit to minimize complications. This is part one: filtering should be out in our next release, and we'll be starting on some of the systems-level work to pave the way for those shortly!

Highlights!

Public tag relationship pages

We've had lots of requests from users who would like to see more information about how tags on the Archive are wrangled and how they relate to one another. In order to work towards this, we've now made tag information pages visible to all users (until now, these were only visible to tag wranglers). If you go to a tag page you can now see whether that tag has been wrangled and how it relates to other tags. For example, if you go to X-Men: First Class, you can see that this is a fandom tag, that it belongs to the metatags X-Men (Movies) and X-Men (All Media types), and that it has a whole bunch of child tags including a character tag for Charles Xavier, and the 'freeform' tag Beach Divorce.

This post by our Tag Wrangling Committee explains a bit more about tag pages and how our tags are connected to one another behind the scenes. This is an alpha version of this feature: we wanted to start by exposing the basic pages and see how people use them before we develop them further. If you have feedback about how they could be more useful, please do let us know via Support.

Rich Text Editor upgrade

We've had some persistent problems with our rich text editor, which was behaving particularly poorly towards users of Internet Explorer. We've now upgraded to a newer version of the RTE which is better behaved - if you've previously experienced an issue where entering text via the RTE resulted in an error complaining there was no content, this update should fix it for you! We're really glad to have finally squashed this bug.

Tags on bookmarks can now be sorted into categories

A new wrangling feature - previously, tags created by users tagging their bookmarks (using the "Your Tags" input field) would all become Additional Tags (Freeforms). Now, any new tag that is created through a bookmark will initially be listed as an Unsorted Tag, without a category. Wranglers will then be able to sort these tags into the appropriate categories (as well as re-sort existing bookmark-only tags); or, if users put the same tags on works or when bookmarking external works, they will automatically be sorted into whatever category the works tags are created in.

Bookmark tags will still be accessible as before; you should not see any changes in how your old or new bookmark tags work. But with this, bookmark tags will now be able to be merged with matching character, relationship, and fandom tags, which will help users browsing and searching bookmarks to find more relevant works.

Known Issues

See our Known Issues page for current issues.

Release Details

Features

  • Ability to recategorize bookmark tags
  • Alpha version of visible-to-everyone tag pages
  • Updated the Rich Text Editor, made it usable in IE9 and Opera 10+ and fixed several bugs in the process
  • Revised the Support form to make it more useful

Bug fixes

  • Works & Stats
    • Gift notification emails weren't sent when editing a existing work; this has been fixed
    • When previewing a chapter before posting, one was presented with the baffling choice of "Post Chapter", "Revert To Saved" and "Delete Chapter"; the buttons have now been adjusted to be more useful
    • The work information in subscription notifications and in the "Share" text on works didn't include series information; it does now
    • Editing an unposted draft allowed you to orphan it, sending the work into orphan_account's draft limbo; this isn't possible anymore
    • Improved parser handling of a, abbr, acronym, and address HTML tags
    • When trying to import a work from fanfiction.net, a URL check would display information about ff.net imports being blocked, which wrongly affected adultfanfiction.net links as well; this has been fixed
    • On the Stats page, the sort buttons weren't working in mobile Safari (iPod, iPhone), making it impossible to view works in any order but the default (hits); this has been fixed
    • Multi-fandom works were counted as one work for each fandom they were in, inflating total stats (such as the word count) and messing up graphs on the Stats page; they now only show up once in Flat View and in the charts and don't inflate the total numbers anymore
    • The number of "Comments" on the Stats page actually only reflects top-level comments on a work; the label was changed to "Comment Threads" to avoid confusion
  • Collections & Challenges
    • When uploading a new user icon, the current icon would be displayed for reference; the same functionality has been added for collection icons
    • There was no way to delete an icon, once uploaded; this has been fixed
    • In collection blurbs for moderated collections, an empty button would show up where the Sign Up link for an unmoderated collection would be; the underlying logic has been fixed to display Sign Up/Join buttons for moderated/unmoderated collections/challenges with open/closed sign-ups accordingly
    • There was an issue with the "moderated" setting for a collection or challenge interacting with the "anonymous"/"unrevealed" settings and breaking anonymity; this has been fixed
    • An explanation of the "closed" option was added to the collection settings page
    • Collections only had a Delete button on the profile page; a "Delete Collection" button has now been added to the collection settings, and the "Delete" button for challenge settings was renamed to make it clearer that it only affects the challenge setup
    • When trying to submit a challenge signup that didn't adhere to signup restrictions (e.g. too many fandoms offered), the resulting error message wasn't sufficiently clear and has now been improved
    • The Kudos button on static challenge pages wasn't working; it does now
  • Admin
    • Added pagination to the page admins see when managing requests for extra invites from users
    • Added a restriction to the number of invitations that can be requested by users at one time
    • When users submitted an Abuse report, no copy of the message was sent to them even if they requested one; this has been fixed
  • Design & Display
    • On user and collection pages, the default Archive icon would shine through for icons with transparent backgrounds; we have changed this to make it possible to use transparent icons
    • HTML scopes, captions and summaries were added to the tables in the admin area, along with some rationalisation of the tables and some CSS tweaks.
    • The History page wasn't adhering to our design principles for listing work blurbs; the underlying structure has now been rewritten, fixing a pagination display bug in Chrome in the process
    • The "Notes" and "Your Tags" fields in the bookmarking form were missing visible labels; they are now properly labeled and not naked boxes anymore
    • The "View All Bookmarks" and "View Recs Only" buttons on the bookmarks index behaved differently from similar buttons elsewhere on the site; they are now both always visible, with the currently selected view clearly marked
    • The link text on static collection pages (such as this book fandom index for Yuletide 2010) was a rather sassy red color; we made the red slightly darker to reduce eye strain
    • When looking at challenge pages without being signed in, an empty navigation section would show up where the sign-up links would be if logged in; this has been fixed to hide the section completely in this case
    • In Opera, disabling images would make some link text hard to read; a background color has now been defined to solve this problem
    • Text in a monospace font (kbd, tt, code, var, pre) was very small and hard to read; we increased the font size a notch
    • Text on iPod and iPhone screens in horizontal view was very big; we changed the font size setting to be consistent in both orientations
    • In work blurbs, the title heading had two separate class definitions, breaking page validation; this has been fixed
    • In the Share box on works, the "ctrl A / cmd A" instructions to highlight the share text for copying weren't enclosed in <kbd> tags to signify keyboard input; they are now
    • When tabbing through a work blurb, arriving at the small icons for rating, category, warning etc. wouldn't make them visibly selected; they now have a visible outline
    • There was an issue in IE7 with lists of checkboxes, such as rating options in tag sets, overflowing their container instead of creating the proper scrollbar; this has been fixed
    • Added a .caution class to the "Adding a synonym to a canonical tag will make it non-canonical" warning on tag wrangling pages to make the note stand out more
    • Long collection names didn't wrap properly in the collections filter box, creating a horizontal scrollbar; this has been fixed
  • Accessibility
    • The pseuds drop-down in the user sidebar was skipped when tabbing through the links, making it inaccessible for keyboard navigation; this has been fixed
    • The "Expand full list" button in a user's fandom list was skipped in keyboard navigation; it's now included in the tabbing order
  • Help & Documentation
    • On the fandom index pages, such as for Video Games, we advised to "search this page by pressing ctrl F"; we have now included instructions for Mac users
    • For security reasons, we changed the way looking up your email address in the invite queue worked; the respective page text has now been changed to reflect this
    • The help text for the "Any" field in our Work Search was amended to be clearer
    • A few small text edits were made to the instructions on tag edit pages
    • The People Search didn't have proper help text; it now does
    • It is currently possible to add more than one related link (such as remix source or fanart inspiration) to one's work, although not very easily; we have added help text to explain the workaround
    • There was an option on the Preference page called "Format your own titles"; it has been renamed to "Browser page title format" to describe more accurately what it does
  • Error Extravaganza
    • Trying to list a co-author who wasn't actually a fellow AO3 user would throw an error 500; this has been fixed to give a meaningful error message
    • Trying to add a work to a closed collection would throw an error 500 under certain circumstances; this has now been fixed to give a meaningful error message while making it possible to edit an already existing work in a closed collection
    • Trying to edit a bookmark for a work that had been deleted would throw an error 500; this has been fixed
    • Logging out while the first welcome banner was showing would cause an error 500; this has been fixed
    • Dismissing the first login banner, then turning it back on in Preferences would result in an 404 error; this has been fixed
    • There were problems with a few tags causing 500 errors; to fix this we added a constraint to the database to prevent duplicate tag counters from being created
  • Miscellaneous
    • The bottom Select All/Select None buttons in the Inbox weren't working; this has been fixed for all such buttons across the site
    • The "External Works" count on a tag wrangling page now link to the bookmarks for that tag
    • Changed the icon alt text character limit from 50 to 250

Comment

Release Notes for Release 0.8.20 and 0.8.21

Published: 2012-07-04 15:58:31 -0400

These release notes bring together details for two small updates, one we deployed shortly after Release 0.8.19 to address a couple of issues which arose, and the one we deployed today.

Some of you may have noticed a temporary site glitch on June 30/July 1. This was caused by a leap second in the UTC time standard which caused problems for a number of sites across the web. Our servers got all confuzzled and had to be rebooted by our ever-alert Systems team. Unfortunately, a small number of kudos and subscriptions emails were lost during this time: many apologies for this!

Today's deploy comprises a handful of small tweaks to the Archive, which make up the last of our performance-related "emergency" updates. We will now focus on the upcoming 0.9.0 release, which we're hoping to have ready to go later this month. This release will come with a major rewrite of the browse & search navigation (replacing the currently disabled filtering system) and a host of bug fixes. Expect significant improvements to rich text editing, collections and challenges, and the help pop-ups all around the site, among many other things.

Over the past few months, we've had a total of eight new coder volunteers coming in, many of whom dived right into our code and already submitted bug fixes or are currently working on feature enhancements. In addition, the Testers group welcomed several new volunteers. We are grateful to anyone donating their time, skills and passion to the Archive, be it as coders, testers, tag wranglers or Support staff. Many thanks also to the members of the Volunteers & Recruiting committee, who have been tireless in getting new people sorted and settled in. \o/

Mini-release: 0.8.20 (deployed 17 June 2012)

  • We made some tweaks to the caching system introduced earlier (more details in our Update on AO3 performance issues), because it was generating error messages for some tag pages.

Current release: 0.8.21

  • We added a help pop-up to the Work Search form, including some tips for searching by tag, which lets you combine ratings, warnings, fandoms, characters etc. into a host of search options.
  • Since it was taking a very long time to generate the search index, making it impossible to search for new works, we've disabled searching by the number of kudos (which were the major bottleneck) until the new system is up and running.
  • Several users contacted Support in alarm when it looked like they were suddenly logged into someone else's account. At no point was security actually compromised, this was merely the result of faulty cookie handling in some browsers confusing the caching and serving up "logged in" pages to guests. Since guests cannot access areas restricted to account holders to begin with, this only affected public pages. We believe the problem is now fixed.
  • Assigning numbered IDs to each invite request created a potential security issue which we fixed in Release 0.8.19. Due to this change, the previous, bookmarkable page to check one's position in the queue wasn't working anymore. We fixed the message and page text accordingly.
  • When checking up on an invite request on the requests page, the search button would just hang without a message if your email wasn't found in the queue. This has been fixed.

Known issues

See our Known Issues page. For the latest site status information, check our Twitter AO3_Status.

Comment

Release Notes 0.8.19

Published: 2012-06-13 07:46:29 -0400

Yet another update from your tireless archive volunteers! James from our Systems Committee has been making adjustments behind the scenes to stabilize the servers and get the most out of our caching, and we've seen some good improvements there. At the same time, we've been working on improving or scaling back the areas of our code where changes will give us the biggest gains.

Filtering

In order to improve performance further, tag filtering on work listing pages is disabled for the time being, until we roll out our new system. You can read more about this change in our post on disabling filters. We know this is an inconvenience for many users, but the filters are really the 800-pound gorilla sitting on top of our database - the works pages are both the most popular and the slowest on the site, which is a bad combination. We've had plans to fix them for a while, and that's underway. However, we need a few more weeks to finish and deploy the upgrade, since it also affects our search engine and quite a lot of our code. Our top priority is to make sure works remain accessible to users, and that new works and feedback can be posted and accessed. Looking carefully at our code and our stats, we concluded that removing filtering was the best way to ensure these goals in the short-term.

You'll still be able to view all the works for a particular tag, view the works for a user or collection in a particular fandom, and use our search feature to refine your results. Our post on disabling filters includes some handy tips to help you find what you're looking for. We hope to have full functionality restored to you soon! As a bonus side effect of this change, we've been able to remove the 1000 work limit on lists of works. This is because without the filters we can rely on the pagination system to limit the amount that we retrieve from the database at one time. So, while you can't filter your results any more, you CAN go through and read every work posted in your fandom! We hope this will compensate a little for the inconvenience.

Work Stats Caching

We've also done more caching of work stats (all the counts of comments, bookmarks, hits, etc.), so you may notice that these update more slowly on index pages now. The information is all still being recorded; we're just waiting a little longer to go get the counts for each work to spread out the load.

People Listings

The alphabetical people listings on the People page weren't actually that useful for finding users, and they were another performance drain.

We've replaced the full alphabetical listing with a listing of 10 random users, and added emphasis on the search. Note you can use wildcards in the search, so if you're not sure of someone's name you can enter part of it followed by an asterisk to get similar names. For example, entering Steve* would get Steve_Rogers_lover, SteveMcGarrettsGirl, stevecarrellrocks, etc.

Invitation Requests

We've suspended user requests for additional invitations for now as well. If you need invitations urgently for a challenge or for an archive rescue project, please contact Support. We also fixed an issue that potentially allowed users to snoop for other emails in the waiting queue.

Thank you!

Thanks to everyone who has been working hard on these issues, especially James, who has put in lots of hours tweaking the servers, and Elz, who has been doing the heavy lifting on code changes. Thanks also to all of you for your patience and understanding while we work!

And finally...

The great news is that so far, this emergency measure does seem to be having a noticeable effect. Our server load has diminished dramatically since we deployed this change:

Graph showing server load, with a mark showing the time of the deploy. The load drops dramatically from this time onwards.

Comment

Release Notes 0.8.17

Published: 2012-06-09 01:42:50 -0400

Welcome to our third Release in this week! Elz, James, and Naomi contributed code to this release, and Ariana, bingeling, Enigel, Jenn, and Kylie from our testing teams worked it over. Our sysadmins and coders have done more work to address the performance issues that have been affecting the archive as well as several other bugfixes.

PLEASE NOTE: in the name of drastically improving performance, this deploy may have a few side effects that appear at first to be errors or confusing! Please do read over these release notes and make sure that they don't cover a problem you are experiencing before you contact support.

Further efforts to battle the 502 errors!

This release includes caching of most pages for guests using Squid! Squid will serve up saved versions of pages without hitting our database or application, which increases speed and decreases server load for everyone.

The tricky part is making this work with all of the dynamic elements of the site: skins, content that gets updated by users, personalized messages, etc. We have decided to turn squid on quickly to keep the Archive running smoothly but we'll be working on finding the right balance between customization and performance as we go forward, so you may see some tweaks to different aspects of the site as we fine-tune this.

Current issues related to the caching:

  • Site skins have been disabled for logged out users for the time being - if you rely on this feature for accessibility needs, please contact support and we will get you an account ASAP so you can use the skins again.
  • Comments and kudos from guests may not show up at once for other guests. When a guest leaves kudos or posts a comment, they will see the comment/kudos added. If another guest then visits that same page (or the same guest reloads the page), however, they will see the most-recently-cached version, which may not yet show their comment/kudos count.
  • Guests may occasionally see a stray error message or notice appearing at the top of a page that does not appear to be related to anything you've done. We are working to track all of these errors down but it is hard to be sure we've gotten them all. The messages should not affect using the archive.
  • Hits that are handled by Squid (most hits from guests) will not appear in the hit count immediately. The hit counts will be updated once a day from the squid logs.
  • Duplicate hits from the log files (for instance on page reloads by the same guest) will no longer be removed because of technical limitations, so hit counts may increase more quickly in some cases.

Squid will be enabled after we update the code, so you may not notice any changes right away.

For those interested in knowing more about Squid, see the detailed explanation below!

Changes to Subscription emails

We've gotten feedback about how people use their subscription emails and in response we have adjust the subject lines and message content to allow people to identify the content more easily. Emails will now contain subs of one type (author, series, or work) and the name/title of the first one in the subject together with the number of other updates.

Details

  • Subscriptions:
    • Email subjects will now say [AO3] instead of [Archive Of Our Own].
    • Subscriptions will be bundled by type with subject lines of the form [authors] posted [first item] and [#] more, where first item will be one of: [Work Title], [Chapter Title] of [Work Title], [Work Title] in [Series Name].
  • Performance:
    • Skin chooser is turned off for logged out users.
    • Nearly all pages will be cached for logged out users.
    • Comment forms and other forms that are getting data for logged out users will have their details remembered in cookies and filled in by Javascript rather than remembered in the page.
  • Bug Fixes:
    • 500 errors were appearing on some work listings because of an interaction between caching and time zone conversion - this should be fixed now.

Details About Squid

Senior coder Ana has written up some helpful information about Squid for those who are curious:

"Squid is a really powerful tool that does a lot of things, but we’re using it primarily as one thing: a reverse-proxy cache. A reverse-proxy cache is a system designed to cache (that is, store copies of) web pages. It sits between users’ requests and the rest of the site and stores the responses to some requests so that instead of making the server build the page from scratch again, Squid can check to see if someone’s looked at that page recently and pass on the cached version. This is really useful when you want to send the same page to lots and lots of users because it means that instead of forcing the servers to generate the pages over and over, we can store a copy and give that copy out to everyone.

Of course, sometimes pages change: an author edits a story, or someone leaves kudos, so you don’t want to let Squid keep those copies around forever. Right now we let Squid keep copies for 20 minutes, and then it throws them away and gets a new one. This feels like the right balance between keeping things up to date, but not overloading the servers.

In addition, logged in users get customization on every page, in the form of the user bar at the top of the page if nothing else, which means that we don’t want Squid to store or give pages to logged in users. If it did, then every user would see the user bar for whoever made the request that Squid saved, and it would only change every twenty minutes.

This same principle holds true for all on-page customization (such as the skin-chooser), and finding the right balance between customization and cacheability (how suitable a page is for storing and giving out to everyone) is going to be an ongoing project as we try to weigh site performance against nifty features and information."

These release notes written and compiled by Ana, Claudia, Elz, Enigel, Jenn, Lucy, and Naomi.

Comment


Pages Navigation