Email changes and USER STATS!

Published: 2012-04-19 17:07:24 -0400

In brief: YEAY USER STATS PAGES! Stats on downloads and subscriptions! Also, some email changes. Read on for details!

The Archive is thriving! We recently passed 300,000 works, with roughly 575 new works posted each day, and we have more than 40,000 registered users (and a lot more people reading on the site who don't have accounts yet). All those people are showing their love for the amazing fanworks on the AO3: we have more than 71,000 user subscriptions and over 10,000 subscriptions to individual works, and on average, our users leave more than 11,238 kudos per day, and 2,175 comments per day.

Changes to the way we deliver email

All of this love involves sending out a ton of emails from the Archive: right now, an email is sent every time someone leaves kudos or a comment, and every time a user who is being subscribed to posts a new work or a new chapter. We also send out emails for invitations, account verification, challenge reveals, and a few other things. In fact, in total, we now send out about 600,000 emails a month! This causes us a few logistical problems: the high volume of emails is a lot for us to cope with, and it also means that our emails are more likely to be flagged as spam. Additionally, our current method of handling emails means that it's possible for the entire queue to be backed up if for some reason we have a higher volume than average, such as when a big challenge is revealed or if a user with a lot of subscribers decides to upload their entire back catalogue in one afternoon.

Our Systems Committee have been working on better ways of handling our email and solving the above problems for a while, and they've concluded that the best way is to hand it over to the professionals, in this case Amazon Simple Email Service. This will mean we're able to take advantage of lots of specialised stuff designed to make sure our emails get where they're supposed to in a timely fashion. However, it also means that we'll have to pay for sending email: US$0.10 per 1000 emails. This means it's important for us to reduce the amount of emails we're sending; this is also important from a spam point-of-view, as sending multiple emails within a short space of time is one thing that can result in ISPs thinking you're spammers.

Batch delivery for kudos and subscriptions

Right now, the majority of the emails we send out are either kudos notifications or subscriptions notifications. Happily, both of these lend themselves to batching up so we can send one 'digest' email instead of lots of individual ones. We're still working out the precise technical details for this, but right now we're expecting to make the following changes:

  • Kudos notifications will be sent out once per day, and will be organised by work, so you'll get something like "Fangirl1, Fanboy3 and 3 Guests left kudos on 'My Happy Story'". This means you'll still be able to see how many individual kudos you got, but your popularity won't overwhelm the email servers!
  • Subscription notifications will be sent out once per hour, and will list everything posted since the last notification. This means you'll still get notifications in a timely manner, but if an author goes on an archiving spree or uploads their latest multichapter masterpiece, you won't be spammed with loads of emails.

We're hoping that these changes will be good news for most of our users - we've had lots of support requests for grouping notifications like this into fewer emails. However, we know that they will make some people sad, as lots of people like to see the kudos notifications rolling in and to count them up. To make sure you can still get information about how many kudos you get (and much, much more) we're launching another much-requested feature: user stats pages!

New user stats page!

We're launching user stats as an 'alpha' feature, so we can get the basic functionality out there and then make changes based on user feedback. We wanted to give you all a sneak peek of what's planned so you can ask any questions before it goes live. So, without further ado:

User stats page: alpha preview

Screenshot of page headed 'Statistics', with a summary of total subscribers, hits, comments, kudos, and bookmarks to the user's works. A bar graph shows the top five works by hits, and below that is a list of all works with stats for each one.

The alpha version of the user statistics page will show you the total number of subscribers, hits, comments, kudos, and bookmarks to your works. It will also include a nifty little graph showing the top five works by hits, kudos, comments or bookmarks. The graph will be created using Google's chartmaking package, which enables us to create a shiny interactive chart (it turns to a static image if you've disabled scripts). Google doesn't get any of your personal data - only titles of works and the number of hits/comments/kudos/bookmarks. The chart is generated when you load the page on the AO3, so the data is only passed to Google then, and it is only kept on their servers for three weeks.

The statistics page will also include individual stats for all works, which you can group by fandom or view 'flat' so that the works are sorted by whatever you've selected in the list options at the top (Date posted, Number of kudos, etc) but not grouped further. (If you can think of a good name for this, then do let us know!) For each work, you'll be able to see the number of hits, kudos, comments and bookmarks. You'll also be able to see 'referrers' - that is, if someone finds your work through a link somewhere else, then you'll be able to see what link they came from. (This won't store any other information about the users - for example, you won't be able to see who got there via that link.) And, in a future version of the page, you'll be able to see how many times your work was downloaded. (People ask us for this a LOT!)

The new stats features will allow you to see how many subscribers you have, and how many subscribers each work has. It won't tell you who has subscribed to you - this information will remain private to the subscribers (so if you're sekritly subscribing to your arch-enemy's WIP because it's just too good to miss, your secret is safe with us!)

Help us design stats 2.0

We know that lots of users will be THRILLED with the new stats option and you will want to ask us for lots more features! This is an alpha feature and we're keen to hear your ideas - if you'd like to get involved in discussing the design, you can see the documentation on our public facing wiki otwcode at Github - anyone can join and edit there, you're very welcome to come on in and join the party. You should also feel free to leave your comments and suggestions here! We can't promise that all your requests will make it into the design, and the first version will probably be similar to the above, but we'll use them to help us develop the next version.

We're excited about the way we're growing and changing, and really pleased to be adding the much-requested stats page. Let us know your thoughts!