Usability Home
Project Description
Season of Usability Description
See Gallery description in Season of Usability 2009 Projects
User research on Gallery usage (from project description email)
A big part of designing Gallery 3 was to limit the scope of the application in a useful way. This was very hard to do without having a very good ideas and not data at all on who exactly our users are. Therefore we would like to accompany the release of G3 with an extensive survey
with the goal of answering the following questions:
- Who are our users?
- What do they use Gallery for?
- How does Gallery 3 compare to its earlier version(s) and other products in the view of our users?
- And of course, what are the most important features that Gallery 3 is still missing?
The survey should be set up in a way that it could be refined and rerun on a regular basis, e.g. 1-2 times a year in connection with major releases. So that we have a good data basis for making a better software for our unique users in the future.
Overall aim: To develop and administer a survey and analyze results to better understand Gallery’s users, with a focus on collecting data that can be translated into actionable steps to improve user experience in Gallery 3 and future versions.
Goals of Project
The goal of this project is to design, implement, and administer a survey and analyze the resulting data with the following in mind:
- To find as true a representative sample of user population as possible, including, but not limited to: administrators, end-users, potential Gallery users.
- To focus on uncovering the following about Gallery users:
- User demographics
- User motivations
- Tasks Gallery is used for- ie. what real-world problems is this tool used to address?
- Common user use cases
- Practical information for design work: eg. How many users does the average gallery have? What kind of photo-management software do user know or use?
- Other user information that will be beneficial to the development of Gallery (add suggestions on this page)
- To design the survey in such a way that it can easily be modified and rerun via a sustainable platform so that it can be adapted for future Gallery purpose
Project Plan
The following is the project plan for Julia Haines, the Gallery Season of Usability student for 2009.
1: Analyzing the Problem Space
- Begin by understanding how Gallery has evolved, especially changes made in G3
- Review any past user research activities and resulting data; analyze what was useful or not
- With help from Gallery community, delineate key areas to investigate that will be relevant to the research goals
- Determine qualitative and quantitative goals for research data. ie. What will make this a successful research project?
Deliverable: Enhanced research plan (wiki page) with clearly-defined focus, goals, and timeline
2: Developing Questionnaire
- Devise the survey on paper (non-interactive) first, focusing on:
- Question types (eg. ratio of closed vs. open)
- Clear wording (eg. positive- vs. negative- directed questions)
- Answer types ( eg. single vs. multiple select; rating scales like Likert or semantic differential scales)
Deliverable: Paper (soft-copy) version of survey
3: Designing the Survey
- Produce interactive electronic (beta) version of the paper survey, focusing on:
- Input options and format based on the purpose and type of question (eg. radio buttons vs. check boxes, etc)
- Embedding feedback and help features
- Accessibility (making sure survey is accessible across browsers, etc)
Deliverable: Online beta version of survey
4: Testing the Survey
- After internal evaluation of the beta version, test with typical participants using a think-aloud protocol, make necessary changes
- Then test with a small sample and make final changes to ensure we’re getting the kind of data we want
- Make sure responses and info identifying respondents can be captured and stored
Deliverable: Final online version of survey
5: Administering the Survey
- Develop and implement strategies for reaching as true a representative sample of Gallery users (and potential users) as possible and getting a good survey response rate (to a certain extent, this will be done throughout the project to this point)
- Develop and implement strategies to try to counteract self-selecting sampling issue
Deliverable 1 (pre-survey): Implementation plan
Deliverable 2 (post-survey): Raw survey data
6: Analyzing and Presenting Data
- Clean up data and filter into different sets;
- Analyze quantitative data using SPSS, etc.
- Analyze qualitative data through the use of affinity diagrams, etc.
- Present data in final research report, possibly including (but not limited to) development of personas, scenarios, common use-cases, etc.
Deliverable: Research Report/Refined Data"
7: Exploring Issues Further
- After looking at survey results, determine the need to utilize other research methods to clarify or deepen understanding such as structured interviews, telephone interviews, contextual inquiries, etc.
Timeline/Log
Week 1 (06/08 - 06/14)
Goal: Begin to research Gallery and background, and focus on becoming acquainted with Gallery community; develop goals for project (and specifically determine the sorts of data that would be most helpful).
Log
Week 2 (06/15 - 06/21)
Goal: Continue step 1, with focus on gathering ideas and determining key data areas/goals. Begin with step 2 in developing questionnaire for these areas. (Step 1, 2)
Log
- Read past survey and interview materials and transcripts
- Further identified key areas to focus on in survey and continued to brainstorm marketing ideas
- Developed rough draft of survey Survey first draft
Week 3 (06/22 - 06/28)
Goal: Continue with formulating survey. Begin designing. (Step 2, 3)
Log
- At meeting on 6/22, determined a change of focus to an open-ended pre-survey, informing a large-scale user survey. This survey will be focused on user motivations, tasks, and desirable qualities in photo-management tools.
- Developed draft of Pre-Survey using the Icon Test platform.
Week 4 (06/29 - 07/05)
Log
- Edited and finalized draft of Pre-Survey
- Developed publicizing and follow up strategy for Pre-Survey. Respondents (and others) will be able to sign up on Gallery-UX mailing list
- After survey is taken down, respondents will be redirected to this page: Pre-survey Over
Week 5 (07/06 - 07/12)
Log
- Pre-survey goes live
- Publicized for pre-survey
Week 6 (07/13 - 07/19)
Log
- Pre-survey remains live
- Continued publicizing
Week 7 (07/20 - 07/26)
Log
- Closed pre-survey (07/20)
- Began editing and redesign of pre-survey to develop SoU 2009 Survey draft
Week 8 (07/27 - 08/02)
Log
- Edited and revised Survey draft and finalized (final on (7/30))
- Developed messaging for invitations and news blurb
- Released SoU 2009 Survey!
Week 9 (08/03 - 08/09)
Goal: Run survey and continue strategies. (Step 5)
Week 10 (08/10 - 08/16)
Goal: Continue survey and publicity. (Step 5)
Week 11 (08/17 - 08/23)
Goal: Continue survey and publicity. (Step 5)
Week 12 (08/24 - 08/31)
Goal: Close survey. Analyze data and develop supplemental materials. Deliver survey results at GalleryCon 2009 (08/27 - 08/29) and discuss next steps. (Step 6, 7)
