Release 0.9.2: The Return of the Filters

Published: 2012-11-08 03:01:49 -0500

Welcome to Release 0.9.1, Part 2: The Filtering; also known as Release 0.9.2 to its friends. Some context for this deploy can be found in our post, Release 0.9.1: An Unexpected Journey.

This code update contains a massive amount of changes, improvements, fixes and happiness, almost all provided by one of our most senior coders, Elz, who now gets to nap for a week. Elz has worked tirelessly over the past months to re-write the tag filtering code from scratch and implement a host of related improvements. Leading a revision like this is an enormous task for a fan with a day job, and while we are aware that users were missing the filters dearly and the planned release date had to be moved quite a few times, we hope all the awesome will more than make up for it!

Elz, mumble, Lucy, Sarken, and Scott all contributed code to this release. It was tested by Alison W, Ariana, bingeling, briar_pipe, Emilie, Fishie Mishie, Hele, Hill, Irene U, Jenn Calaelen, Kylie, Lady Oscar, Lucy, mumble, Radka, Sam, Scott, and supergreak. In addition, lots of other OTW volunteers and members of the Yuletide community pitched in to help with the final testing of the filters - not everyone left their name along with their comments, but everyone's contribution was greatly valued. Feedback from this wider stage of testing will also be used to inform further revisions of the filters: we got lots of good suggestions, but decided to limit the number of enhancements at this stage in order to avoid delaying the main functionality.

Thanks to all our users for your patience while we worked on this - we hope you'll join us in a big THANK YOU to Elz! Our Systems lead James also deserves a special shout-out for his dedicated work and advice: the changes we've made required some major systems-related work. Thanks also to the very many other people who worked on this deploy: testing code, providing advice and support on specific coding challenges, implementing the necessary database changes, and doing all the other tasks that make a change like this possible. <333

Highlights!

New tag filters \0/

TAG FILTERS ARE HERE!!!!

Yes, this release sees the return of the much-missed tag filters, in a new, improved form. Our search and browse functions are now built on Elasticsearch, a far more flexible search engine which uses Solr to perform searches without accessing the database every time. This means that the new filters are MUCH more lightweight than the old ones and won't make such heavy performance demands. HURRAY!

We've written a handy guide to searching and browsing under the new system which walks you through all the details of the new filters. The new filters restore all the functionality of the old ones, with a few extra features. You can now:

  • Sort by number of kudos, comments, or number of bookmarks received! We received lots and lots of requests for this option, so we're very pleased to have been able to implement it.
  • Search within your results and exclude certain tags! The new search box which has been added to filters allows you to narrow down the results of your search even further - see our post on searching and browsing for more details on how it works. It isn't quite the "NOT this" filtering option that users have asked for in the past (that was beyond the scope of this deploy), but it provides at least some of the same functionality.

Bookmark filtering! \o/

A great side effect of rewriting the work filters was that we were able to apply the new filters to bookmarks as well: you can filter all bookmarks for any canonical tag (fandom, relationship, widely used additional tag, etc.) just as you would works, with a few differences. The sorting options are Date Bookmarked (fresh bookmarks, regardless of when the work was posted) or Date Updated (bookmarks of fresh works, including WIPs that recently had chapters added), and you can choose to only display bookmarks marked as a rec, or only bookmarks with some notes added by the bookmarker.

Oh, and we added filtering options to personal bookmarks while we were at it. That's right, one of the most requested features made it into this deploy as well. Users have saved over 1.3 million bookmarks so far, and we're happy to make it easier for you to browse, find and organize them. They have all the features of the work filters, and sorting by Date Updated will show you at a glance which of your bookmarked works has new chapters waiting for you!

New search forms for works and bookmarks

Our Advanced Search form (or simply, Work Search) has been thoroughly redesigned to give our users a more intuitive access to the powerful search feature which we've been offering all along.

All fields in the tags chunk were previously smushed into one "Tags" search field, encompassing everything from fandom to warnings. Naturally, this wasn't obvious from a look at the search form and left many users unsatisfied. The field has now been broken down into options for each kind of tag, making it much easier to see how to search for specific things. The whole form has been reorganised, with similar types of information grouped together, to make it much easier to use. We've also added sorting options for search results. See our guide to searching and browsing for some more tips on how to use it!

New works and bookmarks index pages

The main /works and /bookmarks listings will now only display the 25 most recent works or bookmarks. This lets users have a snapshot of the very latest activity on the Archive, without causing performance problems or being impossible to navigate.

If you're searching for things to read, watch, look at, or listen to, we recommend starting on the main fandoms index or a particular fandom page and working your way down the rabbit hole from there. Tag and author pages will still show all works and provide filtering options. (Hey, remember the 1000 works limit of yore? Gone forever!)

In general, we'd like to move away from listing ALL the data and focus instead on giving users tools to find the things they're looking for. For those who like to browse without any particular goal in mind, we are working on ways to access, for example, new or popular works or hidden gems, and explore the Archive in more interesting ways than a main index of unsorted data. We stopped listing every single Archive member on our /people page when we found that it needlessly added to our performance problems, and we'd like to continue in this vein going forward. Please bear with us while we work on browsing improvements and slowly phase out the main index pages in their current form.

Known Issues

See our Known Issues page for current issues.

Release Details

Features

  • New and improved filtering features for works and bookmarks
  • New and improved search forms for works and bookmarks
  • Personal bookmark filtering
  • A dedicated page for a user's works in collections and challenges, including unrevealed and anonymous ones, for at-a-glance management and editing access, to be found at archiveofourown.org/users/YOURUSERNAME/works/collected
  • To legally host a fanmix on the AO3, you can now embed playlists you have created on 8tracks or Spotify (taking care to follow their own TOS when you do so)

Bug fixes

  • Bookmarks & Works
    • When looking at bookmarks for a metatag (such as Star Trek), only bookmarks with that exact tag would be listed (when it should pull up bookmarks for all its subtags as well); this has been fixed
    • Conversely, bookmarks would sometimes show up on fandom tag pages they didn't belong on; this has also been fixed
    • In the Share box for quickly copy-pasting work info, the Twitter button was slightly broken and didn't fill in title and author anymore when creating a new tweet; this has been fixed
    • The text in the Rich Text Editor was teensy-tiny thanks to the default size in TinyMCE's stylesheet; this should be fixed now
    • Links to the Tags on the Archive FAQ have been added to the help text that pops up for the different tag types (Fandom, Relationship, Additional Tags, etc.) on the "Post New" page
  • Filters & Search
    • Admins are now able to temporarily turn filtering off for works and bookmarks index pages, meaning the index pages for users, tags, collections, etc. would remain accessible even if the underlying search engine was down or under heavy load
    • Searching for an author's works would break anonymity by displaying concealed ("mystery") blurbs of their works, or of works that had been orphaned by them; this has been fixed
    • Relatedly, sorting works by author would make anonymous authors guessable by listing their anon works together with the rest; this has been fixed by treating Anonymous as its own author and shuffling all their works to the top
    • Search results would previously include blurbs of members-only works even for logged out visitors; this has been fixed to hide all traces of locked works when searching
    • Searching works by tag wasn't bringing up works with related tags; it has now been modified so that searching by a canonical tag will bring up works with its synonyms and subtags as well, while putting in a non-canonical tag will restrict results to that tag only
    • For performance and caching reasons, the kudos, comment and bookmark counts (as well as hits) that are used for searching and sorting works are now updated periodically instead of instantaneously, which means ever so slightly outdated search results at a given moment, but also happier servers
    • (Tiny frontend issues for 200, Alex!) If a search returned 0 results, some empty list mark-up would linger unnecessarily; this has been cleaned up
  • Collections
    • Sorting collections by size (i.e. the amount of works in them) was taking a long time and put an undue strain on the servers; we have removed this sorting option until a better solution is found
    • Using the "New Subcollection" button would lead to an error 500; this has been fixed
    • On collection dashboards containing both works and bookmarks, the "Recent bookmarks" section was styled differently than the "Recent works" header; this has been fixed
    • When uploading a header image for a collection, it would show in the footer as well, creating problems with the footer navigation; it will now only be used for the header
  • Notifications
    • In subscriptions emails, the link to series pages was broken; this has been fixed
    • Several email providers (Gmail especially) keep eating our account activation emails, preventing users from finishing the sign-up process; we thought we had conquered the issue when we changed the subject line to something less looking like spam; we have now also expanded the email text to hopefully appear more legit to spam filters
    • Relatedly, "We strongly recommend that you add do-not-reply@archiveofourown.org to your address book to prevent the invitation email from getting blocked as spam by your email provider." was added to the little note that appears after you've added yourself to the invite queue
  • Misc.
    • Using the Reply, Edit, or Delete buttons on comments would jump the page to the top of the comments section; this has been fixed
    • Certain wrangling actions, such as trying to change the canonicals of a tag with lots of subtags or associated works, would time out with an error 502; to fix this the database requests associated with that action have been adjusted to run in the background instead of being (unsuccessfully) applied right away
    • When logged in as an admin, the notes section on bookmarks would overflow the bookmark blurb; this has been fixed