Survey Sunday #6: OTW Website

Published: 2012-07-29 06:46:50 -0400

A short update on Survey Sundays

We've been working on a rough schedule for our Survey Sunday reports to give the committees involved time to review their data and make sure we systematically cover all questions. Unfortunately, this means that we may not be able to answer your Survey Sunday inquiries in a timely fashion, as the related post might be scheduled for weeks or months off. We've done our best to push posts which people showed particular interest in to the top of the pile, but this hasn't always been possible. Although we warned about possible delays in our announcement post, we wanted to give you a specific heads up about this.

In short: we're still very happy to take questions, but we will integrate them into our regular releases, which might mean long delays. Thanks for your patience!

About the answers

Today we’ll be answering questions #84 to #88. These relate to the OTW website (transformativeworks.org) usage, design and content. We've prioritized preparing this data so that our webmasters can factor the results into their planned OTW website redesign. All questions presented in this post are complete and include final numbers, including the last three open (i.e., free-text response only) questions.

It’s important to note that everyone who replied to Question #84 saying that they hadn’t read the site before or that they didn't remember having done so would have skipped all the other site-related questions and automatically been redirected to the last question in the survey, #89, though this fact will also be mentioned when relevant in the corresponding sections.

Additional notes on Freeform Questions #86-#88

In these questions users were given a text box in which to enter their opinions. This type of question obviously requires more time and labor to answer than one with limited choices. This fact, along with the skipping logic, the fact that these questions were at the end of a long survey, and possibly a lack of awareness of the OTW website likely contributed to these questions having a small number of answers compared to the rest of the survey. (Only 4-5% of total survey respondents answered these questions.) Many categories thus represented the opinions of only a very small portion of the respondent pool, so we have chosen not to represent these results as percentages, preferring to focus on the actual counts.

We categorized the answers to these questions by assigning one or more categories to each (similar to tagging) rather than simply dividing responses into distinct bins. Some comments were quite lengthy and included several different ideas, suggestions, and concerns. For this reason, the number of answers in each category will often add up to a number greater than the total number of respondents who answered a question.

84. Have you read the OTW website (transformativeworks.org) before?

Answer Options Response count Response percentage
Yes 1,772 35.7%
I don't remember 1,145 23.1%
No 2,046 41.2%
Answered question 4,963
Skipped question 1,023

82.9% (4,963) of the people who took the survey answered this question. People who answered either "I don't remember" or "No" were immediately redirected to the last question in the survey (question #89), while people who skipped this question, or answered "Yes", were directed to the following questions related to the transformativeworks.org site.

Only 35.7% of the people who answered this question chose "Yes", indicating that they had read the OTW website. 41.2% said "No", and 23.1% said they didn't remember.

graph for question 84, description in the text above.

85. Based on your previous visits to the OTW website (transformativeworks.org), how strongly do you agree or disagree with the following statements?

Answer Options Strongly agree Agree No particular opinion Disagree Strongly disagree Response Count
The OTW website made it easy to find the information I was looking for. 167 748 640 127 24 1,706
The OTW website is easy to navigate. 178 776 600 126 22 1,702
The OTW website met my expectations as a user. 185 766 648 79 18 1,696
Answered question 1,707
Skipped question 4,279

Only 28.5% (1,707) of survey respondents gave an opinion on at least one of the statements. Of the 4,279 people who skipped this question, 3,191 did so automatically by answering "No" or "I don't remember" to question #84. This means that 18.2% of the people who took the survey skipped this question purposefully. Taking into account that only 2,795 people would have seen this question (1,772 who answered "Yes" on question #84 and 1,023 who skipped question #84), about 38.9% of the people who saw this question skipped it.

In general, response count for each statement was nearly identical, and a high percentage of people skipped the question purposefully compared to other sections and questions. The percentage of people expressing a particular opinion on each statement had few variations. Given that the statements are related (particularly "easy to navigate" and "made it easy to find the information I was looking for" – the third option is more ambiguous), this is not overly surprising.

Below we'll analyse each statement separately.

The OTW website made it easy to find the information I was looking for.

Answer Options Response Count Response percentage
Strongly agree 167 9.8%
Agree 748 43.8%
No particular opinion 640 37.5%
Disagree 127 7.4%
Strongly disagree 24 1.4%
Answered question 1,706
Skipped question 4,280

28.5% (1,706) of people who took the survey gave an opinion about this statement. This is almost 100% of the people who answered question #85 in any form (1,707).

53.6% (915) agreed in some manner (whether strongly or not), 37.5% (640) expressed neutrality by choosing "no particular opinion", and 8.8% (151) disagreed in some manner (whether strongly or not).

graph for question 85 first answer, description in the text above.

The OTW website is easy to navigate.

Answer Options Response Count Response percentage
Strongly agree 178 10.5%
Agree 776 45.6%
No particular opinion 600 35.3%
Disagree 126 7.4%
Strongly disagree 22 1.3%
Answered question 1,702
Skipped question 4,284

28.5% (1,702) of people who took the Survey gave their opinion about this statement (about 99.7% of the people who answered this question in some manner).

53.1% (954) agreed in some manner (whether strongly or not), 35.3% (600) expressed neutrality by choosing the "no particular opinion" option, and 8.7% (148) disagreed in some manner (whether strongly or not).

graph for question 85 second answer, description in the text above.

The OTW website met my expectations as a user.

Answer Options Response Count Response percentage
Strongly agree 185 10.9%
Agree 766 45.2%
No particular opinion 648 38.2%
Disagree 79 4.7%
Strongly disagree 18 1.1%
Answered question 1696
Skipped question 4290

28.3% (1,696) of people who took the Survey gave their opinion about this statement; this is about 99.4% of the people who answered this question.

56.1% (951) agreed in some manner (whether strongly or not), 38.2% (648) expressed neutrality by choosing the "no particular opinion" option, and 5.8% (97) disagreed in some manner (whether strongly or not).

graph for question 85 third answer, description in the text above.

#86 - What should we improve about the website, in your opinion?

Answer Category Response count Response percentage
No Opinion 114 30.4%
Mistaken Site 14 3.7%
No Improvements Necessary 24 6.4%
Suggested Improvements/Stated Issues 223 59.5%
Answered question 375
Skipped question 5,611

Question #86 was first of the 3 free-form answer questions in the OTW website section, and used the same skipping logic as 85. Thus, only 2,795 people saw the question. Out of this number only 375 (13.42% of those who saw it) answered it in any form. (See the note at the beginning of this post for a description of how answers to this question were categorized.)

Out of 375 respondents, 114 answered to clarify that they had no opinion, 14 gave feedback for a different site (often AO3), 24 stated that no improvements were necessary and that the site was fine the way it was, and 223 pointed out issues with the site or offered suggestions to make it better.

graph for question 86, description in the text above.

Suggested Improvements

Answer Category Response count
Organization/Navigation 100
Searching and tagging improvements 40
Missing Content 33
Front page content and general page layout related issues 33
Visual concerns 29
Content Presentation 26
Other 15
Total Number of Responses with Suggested Improvements/Stated Issues 223

223 survey takers left comments suggesting site improvements or describing site issues.

The largest number of comments involved the overall organization and navigation of the site (100 answers) and the navigation/searchability of the blog (40 answers). Users expressed frustration with not being able to find the content they were looking for, even though that content is present on the site. Some users were only able to find the information after a Google search of the site, others said they reached it in ways they did not expect or that they wouldn't have found the content if they didn't know it was there in first place. There were several suggestions that information on volunteering should be made easier to access and find, and that the tagging of posts should be improved.

Another common theme was the organization of the front page and the general layout (33 answers). Here one of the biggest points was users not seeing the search box. Some users noted it was hard to find, others asked for a search box, having overlooked it. Other frequent topics included adding some static information in addition to the blog and making the path to access project links and information shorter.

A third common type of feedback was requests for improvements to visual aspects of the site (29 answers). These fell into two main themes. The most frequent request was a review of the color scheme of the site both for accessibility and aesthetic reasons. Others requested that the OTW website layout be more lively, with images.

A fourth common topic included missing content and content presentation. 33 comments involved content being missing or hard to find, including information about the OTW, its projects and committees, and its finances, as well as information about helping the Org by volunteering, joining, or donating. Note that some users identified multiple types of missing content, giving a total of 67 answers.

26 comments were related to content presentation. Within the content presentation feedback two frequent issues mentioned were language clarity on the textual content, and a request for more charts, videos, pictures and general visual cues to make the content more appealing and easier to read

The remaining 15 comments covered a variety of topics not mentioned above, such as improvements to the calendar feature, performance issues, search engine optimization and accessibility.

graph for question 86 suggested improvements breakdown, description in the text above.

Breakdown of missing/hard to find content mentioned

Answer Category Response count
Projects/committees 21
About the Org/about the site/financial information 19
Volunteering/membership/donation 16
Elections 11
Total Number of Responses with Suggested Improvements/Stated Issues 223

While the nature of missing content was the focus of question #87, several respondents specified types of content they had tried to access but could not find, had difficulty finding, or felt could be clarified further in answer to this question. The most commonly mentioned content was information on projects and committees (21 answers), information about the OTW as a whole, such as general structure and finances (19 answers), information on volunteering, donating and membership (16 answers) and elections (11 answers).

graph for question 86 missing/hard to find content breakdown, description in the text above.

87. What information would you expect to find on the OTW website that's not currently there?

Answer Category Response count Response percentage
N/A - No Opinion 98 38.7%
Mistaken Site 6 2.4%
Nothing particular/all set 43 17.0%
Suggested content/improvements to content 106 41.9%
Answered question 253
Skipped question 5,733

Question #87 used the same skipping logic as #85 and #86; thus, only 2,795 people saw the question. 253 (less than 10%) of them decided to answer it.

Of the 253, 98 of them answered to clarify they had no opinion on the topic, 6 mistakenly gave feedback on specific OTW projects (often AO3), and 43 answered that they didn't think anything in particular was missing. 106 answers contained suggestions to add new or improve existing content.

graph for question 87, description in the text above.

Nature of content improvements requested

Answer Category Response count
Has missing content/can use more information 87
There but hard to find 4
Can use more updates 21
Total number of answers with content suggestions 106

Although the question was specifically asking about missing content, the responses covered a variety of concerns. 87 answers mentioned missing content or asked for more detailed content. 4 answers asked for more frequent updates to particular content. 21 respondents stated that the information they were looking for was there but hard to find.

graph for question 87 nature of content improvements requested, description in the text above.

Breakdown by type of expected information not found

Answer Category Response count
Other topics 46
Legal 18
Volunteering/donating/membership 17
Finances 15
Project 13
About the Org/its structure 12
Committes/Board 11
Total number of answers with content suggestions 106

The topic with the greatest number of requests for more information was the legality of fandom and previous cases (18 answers). It is important to note that half of these (9 answers) were specifically requesting information on legal issues with an international scope. Second most frequent request on information was volunteering and donating. Users were looking for more information on volunteering and making donations (17 answers). Specific requests included more current and detailed information on what types of volunteers are needed, what new volunteers should expect, and what is the difference between becoming a member, volunteering, and donating.

Other common requests were more detailed information on OTW finances and how donations are put to use (15 answers), more detailed and up to date information on projects (13 answers), more information on how the OTW is structured, and information on committees and the Board (11 answers).

The remaining answers contained a variety of content requests such as more information on staffers and volunteers as people, FAQs, translations, resources for existing volunteers, or better access/merging of information originating from various OTW communications outlets.

graph for question 87 breakdown of information expected but not found on the OTW site, description in the text above.

88. Do you have any other feedback about the OTW website?

Answer Category Response count Response percentage
N/A - No opinion/no further comments 127 54.3%
Mistaken site/off topic 13 5.6%
Further suggestions/comments 94 40.2%
Answered question 234
Skipped question 5,752

Question #88 asked for additional comments on the OTW website on topics that weren't covered by the previous two questions. Since the question was under the same skipping logic as #85, #86 and #87, only 2,795 people saw it. Out of these, 234 (less than 10%) chose to answer this question.

Of these answers, more than half (127 answers) were to state that the respondent had no further comment or that they had no opinion. Answers that only referred to previous questions by number (e.g., "what I said in question 86") with no further comments were also categorized here.

13 of the comments were feedback on other OTW projects, general commentary on the OTW, or other off topic content such as frustration with the length of the survey (this was the question-before-last of 89 questions).

94 answers included further suggestions or general comments on the website.

graph for question 88, description in the text above.

Nature of feedback on the website

Answer Category Response count
Specific criticism or suggestions 53
No other issues/overall happy 30
Thank you and encouragement 14
General dissatisfaction 5
Positive comments on specific features or aspects 4
Total number of answers with further comment 94

The most common type of feedback given in response to this question was to suggest improvements to the site (53 answers). The most common topics mentioned for improvement were organization and issues about the look of the website.

30 users used this space to state that regardless of their previous suggestions or comments they were overall happy with the site. 14 users thanked the website team or sent them encouragements. 5 answers contained non-specific dissatisfaction with the site as a whole, and 4 answers included comments on particular aspects of the site that worked well for the respondent.

graph for question 88 breakdown by nature of feedback, description in the text above.