AO3 News

Post Header

Our previously announced search engine upgrade will not only provide some much-needed stability improvements, but also bring exciting changes to the way you search, filter, and browse for works and bookmarks on the Archive! These changes range from making current functionality more user-friendly to adding new options and overhauling existing features. We hope you enjoy these improvements to the way you find new things to read or look at or listen to!

Work search and filtering

Over the years, you've sent us a lot of suggestions for enhancing work search and filters, and as fellow Archive users, we're excited to finally implement some!

 

Preview of the new work filters.

Detail of the new work filters, focusing on tag exclusion, crossover handling, completion status, and word count. The full screenshot (linked) shows the whole list of tag types to include or exclude, and additional options such as a date range and work language.

 

  • Exclusion filters: Now you can filter things out of work listings the same way you filter them in: just enter their names in the "Other tags to exclude" field or choose them from a list of the most popular tags. This might have been our most requested feature, and we think our solution combines familiarity, flexibility, and accessibility. The new fields are only available in the filters for now, but you can keep using the old tag exclusion method on the Work Search page -- we'll be updating that interface soon!

  • Crossovers: You can now search for crossovers, or choose to exclude them. Crossovers are defined here as works with at least two unrelated fandom tags, as determined by how tags are wrangled.

  • Work Status: Previously, you could opt to show completed works only. Now you can also search specifically for works in progress, if you're into that kind of thing.

  • Date Updated: To limit your results to works posted or updated in a particular time period, the filters now include a date picker to help you choose start and end dates for your query.

  • Word Count: Lastly, we've added a simple way to look for works in a particular word count range. \o/

Everything you select in the filtering sidebar will continue to narrow down your set of results (i.e. AND filtering), as opposed to including more works that might have one or more of your selected tags (i.e. OR filtering).

Bookmark search and filtering

With the old bookmarking code, wrangling changes to large tags could put a strain on the Archive's servers. Thankfully, Elasticsearch 6 provides a much better way to handle our over 74 million (!) bookmarks, so we've given the code an extensive rewrite. However, in addition to giving bookmarks shiny new exclusion filters, we've had to make some changes to searching, browsing, and filtering bookmarks.

 

Preview of new bookmark listings.

A truncated list of bookmarks, all of the same work, listed under the Veronica Mars tag. The full screenshot (linked) shows the five most recent bookmarks of the work, with bookmarker's notes and tags, and a link to access all bookmarks.

 

  • Redesigned bookmark listings for tags and collections: No more scrolling past hundreds of bookmarks for the same popular work or series when you're browsing bookmarks in a collection or tag -- each bookmarked item will now be listed only once, with the details of its five most recent bookmarks beneath it. (If an item has more than five bookmarks, there will be an "All Bookmarks" link so you can check out the rest.) User bookmark pages and Bookmark Search results will continue to list bookmarks the way they always have.

  • Bookmarker's tags: With the new code, using the "Angst" tag on a bookmark will no longer add that item to the bookmark page for "Angst," nor will the item be included in the results when filtering a list of bookmarks for the Additional Tag "Angst." Now if you want to see everything bookmarkers have tagged with "Angst," you'll need to use the "Bookmarker's tags" field on the Bookmark Search page.

    If you use the tags for a personal rating system or reminders to comment on a bookmarked work, for example, you can still filter your bookmarks just as before! There will also be "Bookmarker's tags" checkboxes and autocomplete fields on the bookmark pages for collections and tags.

  • Searching bookmarks: While it's still possible to search bookmarks, the performance improvements we've made required us to divide the search into two fields: one for information on bookmarked items (e.g. work tags, titles, and summaries) and another for information on the bookmarks themselves (e.g. bookmarkers' tags and notes).

 

Preview of new bookmark filters.

Detail of new bookmark filters, showing exclusion options for tags on the work and tags added by the bookmarker. Further options include search fields for work and bookmark information, and checkboxes to only look for recs or bookmarks with notes. The full screenshot (showing all filterable tags) is linked.

People search

You can now search for fellow users who have created works in your favorite fandoms! \o/ The new "Fandom" field in our People Search form suggests canonical tags as you start typing, so you don't have to worry about getting the name of your fandom exactly right. The more fandoms you put in, the more you'll narrow down the set of results.

 

Preview of the new people search.

The new People Search form, focusing on a search by fandom. The pseud testy has 970 works and 66 bookmarks, including 1 work for the fandom in question. Additionally, one can also search by username or profile content.

 

To make your pseud stand out in the search results, you can follow these instructions on editing pseuds to add a little blurb and user picture. (You can also add links and information about yourself, your fannish history, transformative works policy, or social media presence to your account profile, as laid out in our Profile FAQ.)

Limitations

Since even a state-of-the-art search engine like Elasticsearch suggests some gentle limits to the number of search results displayed (by throwing an error after a point), you will only be able to sift through 5,000 pages at a time, which means 100,000 works or bookmarks.

Keep in mind, however, that the Work Search form lets you change how results are sorted. So if you want to access the oldest works in a fandom -- something that wouldn't show within the first 100,000 results in a big fandom -- you can sort by Date Updated in Ascending order to make sure you get the results you’re looking for! (To sort a filtered list of works, use sort:>posted in the "Search within results" field.)

Some things to try

Once the new search is enabled for everyone, here are a few suggestions for getting to know the new options:

  • Rating: For a list of works rated General Audiences or Teen and Up, don't select a rating to include -- instead check "Explicit", "Mature", and "Not Rated" to be excluded, and all your results should be safe for work! (You can reverse this for your bedtime reading.)

  • Word Count: To get all the actual drabbles in your chosen fandom, enter 100 in both the "From" and "To" fields. (#DrabblePurist)

  • Crossovers: Are you a big fan of canon-verse fic? You can eliminate all AUs by excluding the "Alternate Universe" tag, and crossovers or fusions by selecting the "Exclude crossovers" option.

  • Date: Working on your list of favorite podfics of 2017? Go to the Podfic tag and put 2017-01-01 and 2017-12-31 in the "From" and "To" fields under "Date Updated", to refresh your memory!

  • Status: You've consumed all the completed works for your shiny new OTP? You have skipped the works in progress, because you needed a happy ending now? Nothing left to read? Well, gird your loins and check the "Only incomplete works" ticky to take a second look at those!

Comment

Post Header

At long last, we have upgraded our search engine, Elasticsearch, from version 0.90 aaaaaaall the way to 6.2, which had obviously been long overdue. A lot of other emergencies kept cropping up over the years, and there were never enough volunteers around to handle such a massive code rewrite while also managing the day-to-day upkeep of the Archive. With the help of our contractors (thanks to your generous donations!), we are now getting ready to deploy these changes to the Archive.

For a short period of time, the new search will only be available to a few volunteers for some final testing; everyone else will still see the current search and filtering interface. You might experience some issues in the days before we switch everyone over, as we had to tweak our old code a bit to get both versions of Elasticsearch running. Please bear with us, this will be temporary.

Once we are ready, we will move all accounts to the new system in batches, while monitoring how it holds up under increasing pressure. We currently expect this process to take at least a week, and longer if we run into any problems we have to stop and fix.

As always, we will keep you updated on progress here and through our @AO3_Status account on Twitter. (We'll have a bigger post with more details coming, but you can find a tiny preview below.) Many, many thanks to all the coders and testers who helped carry this over the finish line!

Snapshot of the new work filtering bar, showing the possibility to exclude tags, such as particular warnings

Comment

Post Header

As people get ready to celebrate the end of 2017 in one way or another, we'd like to thank our users for their patience and supportive comments as we navigated downtimes, spam problems, and bumpy infrastructure upgrades together. We accomplished a lot of what we had hoped to tackle this year, and added a lot more to the to-do list for next year. Thanks for sticking with us!

As you might know, we've had to disable invite requests for existing users (due to abuse by spammers) and decrease the number of invites we automatically send out from our queue (ditto). As a result, people have had to wait to create an AO3 account for longer than we'd like. So, for the holidays, we're giving 1 shareable invite to every existing account that:

  • currently doesn't have any invites, and
  • is older than half a year, and
  • has left at least 10 comments, or posted at least 1 work

(Sorry, we had to ensure we don't accidentally let the spammers invite all their spammer friends, so some restrictions apply.)

Check out our FAQ (available in a whole lot of languages) to read up on how to send an invite. You can either email the invite code, or copy-paste the code to share it with people through other means. In that case, our FAQ contains some information on how to use an invite code to create an account.

Happy gifting!

Comment

Post Header

For the past several weeks, the Archive of Our Own has been dealing with an unusually high volume of spam works created to advertise live streams of sporting events. We've taken a number of steps to deal with these spam works, and in the next few days, we'll take one more: automatically hiding works that our spam detection service believes are spam.

We've been using this spam detector behind the scenes for a while now, and it has a 99.3% accuracy rate when it comes to identifying spam works and comments on the Archive. This means there is a small chance some non-spam works will be incorrectly marked as spam.

If your work is marked as spam, we'll send you an email to let you know. Our Abuse team will review your work as soon as possible and unhide it if it isn't a spam advertisement. Works will not be reviewed for other Terms of Service violations while in the spam queue.

We apologize to any users whose works are incorrectly marked as spam and to everyone who is currently waiting for us to review and fulfill their invitation request. We realize this situation is frustrating, but these steps are necessary to help us deliver a spam-free experience to all our users.

Thank you for bearing with us, and if you see any spam that has escaped our net, please report it to Abuse!

Update on November 24, 00:40 UTC: Automatic hiding of spam works is now enabled.

Comment

Post Header

Last weekend, we had to disable new invitation requests to address an influx of accounts flooding the Archive with spam works. While our Abuse team has been banning these accounts and deleting thousands of spam works, the problem persists and would most likely get worse if we sent out invitations again.

We have decided to keep the invitation queue closed for the time being while we take steps to prevent spam from being posted in the first place. This means you will not be able to create an account unless you have previously received an invitation from either a friend or our automated queue. (If you requested an invitation before October 22 and have not received it, please check your spam folder and, if you use Gmail, your "Social" tab. If you are still unable to find your invitation, you can contact Support with your specific request.)

We very much regret denying invitations to legitimate users, but as the amount of spam being posted is affecting everyone's user experience, we currently see no other way to address the problem.

We will reopen invitation requests as soon as we can, although we do not have an estimated date at this time. When requests have been reenabled, the "Get Invited!" link will return to the homepage, and the Invitation Requests page will include a form to add yourself to waiting list. (The option to request invite codes for friends has been disabled since the last spam wave, and we have no plans to bring it back in the foreseeable future.)

Any updates will be provided on this post and our @AO3_Status Twitter account. For more information on the Archive's invitation system, refer to our Invitations FAQ.

Comment

Post Header

We're currently experiencing an influx of spammers who have been creating bogus works and collections to link to their fare. They've become highly adept at using Archive features, and they've been flooding our invite queue with throwaway email addresses to create new accounts. This keeps our Abuse team busy around the clock, deleting spam works as they pop up and trying to weed out obvious spam email addresses before invites are sent out every day. It also prolongs the wait time for everyone else who wants to join the Archive. Our wait list is inching ever closer to 20,000, meaning legitimate users have to wait almost three weeks to receive an invitation email.

As a short-term measure, we've decided to turn off the invite queue for a week, so we can relieve some of the burden on our Abuse team, discuss technical solutions to the problem, and implement a quick fix or two to help with the worst attacks.

If you are a current user, you can check your Invitations page to see if you have any old invites waiting to be sent to a friend or fellow fan.

We are sorry for the long wait times, and we're doing our best to come back soon and get invites out quicker to those currently waiting!

Update on October 23, 11:23 UTC: People who are currently waiting for an invitation should still receive an email while the queue is under review. If you think you should have received an invitation, please wait another day or two, check your spam folder or "Social" tab in Gmail, and use our look-up tool to see if you're still in the queue. If you're sure you should have received an invitation and didn't, you can contact our Support team.

Update on October 30, 23:08 UTC: Please refer to our post "Update: Invitation requests remain disabled for the time being" for the latest information regarding invitations.

Comment

Post Header

If you have podfic, fanvids, or other works with embedded media, users who currently opt to browse the Archive over HTTPS (e.g. via a browser extension) may be unable to access your work. Before we move the Archive to HTTPS, we'll be making some changes to existing audio and video embeds to prevent more widespread issues, but there are also steps you can take now to ensure your content loads for everyone.

The problem

Many of the media players in AO3 works use HTTP links to embed Flash files for playing back audio or video. For example, here's the code for an audio player that uses an HTTP URL as its src:

<embed type="application/x-shockwave-flash" flashvars="mp3=MP3_FILE_URL" src="http://archiveofourown.org/system/dewplayer/dewplayer.swf" width="200" height="27" allowscriptaccess="never" allownetworking="internal"> </embed>

If someone uses HTTPS to access a work with code like that, their browser will notice a mismatch between the page they're on (HTTPS! secure!) and the content it's being asked to display (HTTP! not secure!). When this happens, many browsers will err on the side of security: they won't load or display the embedded media.

Most browsers do allow users to override this behavior and display insecure content, but how easy that is varies a lot from browser to browser, and the process can change from one browser version to the next. (A web search for "display mixed content" plus the name and version of your browser should provide the information you need.) Whenever possible, it's easier -- and safer -- to tell the browser to load the file over HTTPS.

What you can do

To help the browser out, you can simply add that little "s" to the relevant bit of your embed, which will create a secure connection to the file in question. The vast majority of our whitelisted multimedia players already offer HTTPS support. If you only have a few works with audio, video, or even image content which link to external media, you can edit your works, look for the src part and turn the http bit into an https. That's it!

This will ensure that everyone can access your podfic, fanvid, art, or other media, even if they're browsing the Archive in HTTP mode. (Browsers are cool with that mismatch.)

What we will do

Since we can't expect all our users to edit all their multimedia works by hand, we will ensure that all embeds use the correct linkage by doing one big find-and-replace on our end. Because we only allow embeds from a few sites, we can easily find the affected works by searching our database for specific HTML. Then we can run a few simple commands to update the embed code's src from http to https.

This will not touch the content of your work in any way, or alter anything about your work that isn't neatly bracketed by a pair of very specific quote marks. When it's all done, your content will be accessible to users browsing the Archive in secure mode, if it wasn't before. \o/

We are planning to do this on October 4th, right before we switch to HTTPS mode on the AO3.

To make sure that works posted from now on won't run into this problem, we've updated our code to ensure embeds use https links where available, and edited our documentation for audio player embeds.

Edit 09:24 UTC on 05 October, 2017: The update to embedded media files has been completed, but attempts to move the Archive to HTTPS were unsuccessful. HTTP will remain the default for a little while longer, and we'll update you via our Twitter account when we're ready to try again.

Edit 19:16 UTC on 12 October 2017: We successfully made the switch to HTTPS for a few days; however, the extra strain from encrypting all traffic proved too much for our servers at peak times. Until we have installed additional servers (coming soon!), HTTP will remain the default protocol. (Of course, you can still elect to use a secure connection, e.g. via a browser extension like HTTPS Everywhere.) Please follow @AO3_Status on Twitter for futher updates.

Edit 22:15 UTC on 14 October 2017: We have implemented the caching needed to reduce server strain and are currently back on the secure protocol by default. We believe we'll be able to remain on HTTPS, but if it proves too much, we will switch back until our new frontend servers arrive.

Edit 18:28 UTC on 19 March 2018: We have successfully installed our new frontend servers and are now permanently enforcing HTTPS on the Archive. \o/ We still provide HTTP access via insecure.archiveofourown.org.

Comment

Post Header

If you've been following our release notes, you know that we have been working towards full HTTPS support on the Archive for a while now. Today, we're happy to announce that beginning on October 4th, all connections to the Archive will use HTTPS by default.

If you use a modern browser, you won't have to do anything -- we'll just flip a (virtual) switch to enforce a secure connection between your browser and our servers. As a result, all AO3 pages will display a lock symbol and/or a friendly little https:// in the address bar of your browser. Old http:// links to the Archive will automatically redirect to the secure version.

For users who might have trouble accessing secure websites, we will continue to provide HTTP access to the Archive -- via insecure.archiveofourown.org -- for as long as necessary. (You might still run into the odd HTTPS link on the site, for example when downloading a work as a PDF, MOBI, or EPUB file.)

We don't expect any downtime during this transition, and you shouldn't notice any changes. Just to be on the safe side, we will monitor our servers and firewalls and might temporarily revert back to HTTP mode should we notice any problems.

Please keep an eye on the @AO3_Status Twitter account for more updates as we get closer to the switch.

Happy (secure) browsing!

Edit 09:24 UTC on 05 October, 2017: The update to embedded media files has been completed, but attempts to move the Archive to HTTPS were unsuccessful. HTTP will remain the default for a little while longer, and we'll update you via our Twitter account when we're ready to try again.

Edit 19:16 UTC on 12 October, 2017: We successfully made the switch to HTTPS for a few days; however, the extra strain from encrypting all traffic proved too much for our servers at peak times. Until we have installed additional servers (coming soon!), HTTP will remain the default protocol. (Of course, you can still elect to use a secure connection, e.g. via a browser extension like HTTPS Everywhere.) Please follow @AO3_Status on Twitter for futher updates.

Edit 22:15 UTC on 14 October 2017: We have implemented the caching needed to reduce server strain and are currently back on the secure protocol by default. We believe we'll be able to remain on HTTPS, but if it proves too much, we will switch back until our new frontend servers arrive.

Edit 18:28 UTC on 19 March 2018: We have successfully installed our new frontend servers and are now permanently enforcing HTTPS on the Archive. \o/ We still provide HTTP access via insecure.archiveofourown.org.

Comment


Pages Navigation