AO3 News

Post Header

Last weekend, we had to disable new invitation requests to address an influx of accounts flooding the Archive with spam works. While our Abuse team has been banning these accounts and deleting thousands of spam works, the problem persists and would most likely get worse if we sent out invitations again.

We have decided to keep the invitation queue closed for the time being while we take steps to prevent spam from being posted in the first place. This means you will not be able to create an account unless you have previously received an invitation from either a friend or our automated queue. (If you requested an invitation before October 22 and have not received it, please check your spam folder and, if you use Gmail, your "Social" tab. If you are still unable to find your invitation, you can contact Support with your specific request.)

We very much regret denying invitations to legitimate users, but as the amount of spam being posted is affecting everyone's user experience, we currently see no other way to address the problem.

We will reopen invitation requests as soon as we can, although we do not have an estimated date at this time. When requests have been reenabled, the "Get Invited!" link will return to the homepage, and the Invitation Requests page will include a form to add yourself to waiting list. (The option to request invite codes for friends has been disabled since the last spam wave, and we have no plans to bring it back in the foreseeable future.)

Any updates will be provided on this post and our @AO3_Status Twitter account. For more information on the Archive's invitation system, refer to our Invitations FAQ.

Comment

Post Header

We're currently experiencing an influx of spammers who have been creating bogus works and collections to link to their fare. They've become highly adept at using Archive features, and they've been flooding our invite queue with throwaway email addresses to create new accounts. This keeps our Abuse team busy around the clock, deleting spam works as they pop up and trying to weed out obvious spam email addresses before invites are sent out every day. It also prolongs the wait time for everyone else who wants to join the Archive. Our wait list is inching ever closer to 20,000, meaning legitimate users have to wait almost three weeks to receive an invitation email.

As a short-term measure, we've decided to turn off the invite queue for a week, so we can relieve some of the burden on our Abuse team, discuss technical solutions to the problem, and implement a quick fix or two to help with the worst attacks.

If you are a current user, you can check your Invitations page to see if you have any old invites waiting to be sent to a friend or fellow fan.

We are sorry for the long wait times, and we're doing our best to come back soon and get invites out quicker to those currently waiting!

Update on October 23, 11:23 UTC: People who are currently waiting for an invitation should still receive an email while the queue is under review. If you think you should have received an invitation, please wait another day or two, check your spam folder or "Social" tab in Gmail, and use our look-up tool to see if you're still in the queue. If you're sure you should have received an invitation and didn't, you can contact our Support team.

Update on October 30, 23:08 UTC: Please refer to our post "Update: Invitation requests remain disabled for the time being" for the latest information regarding invitations.

Comment

Post Header

Published:
2015-02-01 04:45:18 -0500
Tags:

This batch of code includes several new features and enhancements aimed at assisting the Abuse team in their work, as well as some changes under the hood to protect us against attacks from download bots while keeping disruptions for our users to a minimum.

When we announced that we'd be suspending our automated invitation queue at the beginning of the month, we didn't plan for it to be closed quite this long! Writing, testing, and improving the new code, while also working with Abuse and taking care of our servers through planned and unplanned downtimes, took considerably longer than we expected.

While the queue was closed, we continued to provide invitation codes to existing users upon request. According to our records, we handed out over 5000 such invitations this month, which is more than three times our usual number. To everyone who reached out to friends and strangers to share invitations, we thank you!

If you wanted to create an account in January and couldn't - we're very sorry for making you wait! The queue is now back in business: request an invitation today and receive a code in 48 hours or less!

Credits

  • Coders: Elz, Enigel, james_, Sarken, Scott
  • Code reviewers: Elz, Enigel, james_, Sarken, Scott
  • Testers: Lady Oscar, mumble, Sarken

Details

Admin

  • In order to combat spam works (e.g., works consisting solely of advertisements), our Abuse team will now be automatically notified of accounts with a suspicious level of activity or works with suspicious content, allowing them to investigate before the problem gets out of hand.
  • When our Abuse team hides a work that is under investigation, the creator(s) of the work will now receive an automatic email, letting them know the work has been hidden intentionally and not as the result of a bug. The email also contains a link to the work so that they (but no one else) can access it while it is hidden.
  • Users whose accounts have been permanently suspended (banned) are not allowed to create new works, but it was still possible for other users to list them as co-authors. Now they cannot be added as co-authors unless the ban is lifted.
  • When a user's account is temporarily suspended, the error message they receive when attempting to post, comment, or perform other actions will now let them know when they can expect their suspension to be lifted.
  • Attempting to post, comment, or perform other actions using a suspended account previously resulted in an error message that said, "Please contact us for more information." The message has been clarified to say, "Please contact Abuse," and now provides a link to the appropriate contact form.
  • When an admin deleted an invitation request submitted through our automated queue (e.g., because the requester was likely to be a known spammer), they would be redirected to the first page of the list, which was annoying if they needed to delete more than one request on the same page. Now they will be returned to the page they were on.

Downloads

  • We were receiving reports from users whose IP addresses were blocked from accessing the Archive for downloading too many works too quickly, even though they hadn't done anything wrong. This would happen, for example, if their browser tried to download a file many times, despite the user having only clicked the download button once. We have added code and tweaked server settings to make this much less likely. In particular, we are now serving cached copies of downloads to users, which are currently refreshed on the server as soon as a work is updated.
  • During site-wide downtime because of an overwhelmed server, users were receiving an error page that incorrectly stated they were downloading works too quickly. We've corrected the error page we give when the Archive is down and also added a dedicated error page to let users know when they are posting works too quickly.

Known Issues

See our Known Issues page for current issues.

(Please note that while it looks like we skipped several version numbers since our last batch of updates (Release 0.9.41), you haven't missed out on any new code! The jump is due to a few test deploys to get our deploy script into shape after adding another server to our line-up.)

Comment

Post Header

To combat an influx of spam works, we are temporarily suspending the issuing of invitations from our automated queue. This will prevent spammers from getting invitations to create new accounts and give our all-volunteer teams time to clean up existing spam accounts and works. We will keep you updated about further developments on our Twitter account. Please read on for details.

The problem

We have been dealing with two issues affecting the Archive, both in terms of server health and user experience.

  • Spammers who sign up for accounts only to post thousands of fake "works" (various kinds of advertisements) with the help of automated scripts.
  • People who use bots to download works in bulk, to the point where it affects site speed and server uptime for everyone else.

Measures we've taken so far

We have been trying several things to keep both problems in check:

  • The Abuse team has been manually banning accounts that post spam.
  • We are also keeping an eye on the invitation queue for email addresses that follow discernible patterns and removing them from the queue. This is getting trickier as the spammers adjust.
  • We delete the bulk of spam works from the database directly, as individual work deletion would clearly be an overwhelming task for the Abuse team; however, this requires people with the necessary skills and access to be available.
  • Our volunteer sysadmin has been setting up various server scripts and settings aimed at catching spammers and download bots before they can do too much damage. This requires a lot of tweaking to adjust to new bots and prevent real users from being banned.

Much of this has cut into our volunteers' holiday time, and we extend heartfelt thanks to everyone who's been chipping in to keep the Archive going through our busiest days.

What we're doing now

Our Abuse team needs a chance to catch up on all reported spamming accounts and make sure that all spam works are deleted. Currently the spammers are creating new accounts faster than we can ban them. Our sysadmins and coders need some time to come up with a sustainable solution to prevent further bot attacks.

To that end, we're temporarily suspending issuing invites from our automated queue. Existing account holders can still request invite codes and share them with friends. You can use existing invites to sign up for an account; account creation itself will not be affected. (Please note: Requests for invite codes have to be manually approved by a site admin, so there might be a delay of two to three days before you receive them; challenge moderators can contact Support for invites if their project is about to open.)

We are working hard to get these problems under control, so the invite queue should be back in business soon! Thank you for your patience as we work through the issues.

What you can do

There are some things you can do to help:

  • When downloading multiple works, wait a few moments between each download. If you're downloading too many works at once, you will be taken to an error page warning you to slow down or risk being blocked from accessing the Archive for 24 hours.
  • Please don't report spam works. While we appreciate all the reports we've received so far, we now have a system in place that allows us to find spam quickly. Responding to reports of spam takes time away from dealing with it.
  • Keep an eye on our Twitter account, @AO3_Status, for updates!

Known problems with the automated download limit

We have been getting reports of users who run into a message about excessive downloads even if they were downloading only a few works, or none at all. This may happen for several reasons that are unfortunately beyond our control:

  • They pressed the download button once, but their device went on a rampage trying to download the file many times. A possible cause for this might be a download accelerator, so try disabling any relevant browser extensions or software, or try downloading works in another browser or device.
  • They share an IP address with a group of people, one of whom hit the current download limit and got everyone else with the same IP address banned as well. This can be caused by VPNs, Tor software, or an ISP who assigns the same IP address to a group of customers (more likely to happen on phones). Please try using a different device, if you can.

We apologize if you have to deal with any of these and we'll do our best to restore proper access for all users as soon as possible!

Comment