Exploratory research and remote usability testing for Ashdel Studio, a small online party supply shop.
Overview
A small e-commerce party supply shop requested exploratory research to get a better understanding of potential customers and feedback on products.
The goal was to learn how people plan, shop for and use products for baby or bridal showers. The data would provide valuable insights that the shop could use to refine or develop products and market them more effectively. Testing of a novel product was also explored for usability and marketing.
Ashdel Studio
Based in Austin, TX, Ashdel Studio specializes in party goods such as wall decor, banners, stationery, guestbooks, cake toppers, and keepsakes.
OBJECTIVES
Answer these questions:
-
Who is the customer of Ashdel Studio?
-
What is the usability of a novel product (Alphabet Activity Guestbook for baby showers) and is anything unclear?
-
What feedback is there for the shop and the Alphabet Activity Guestbook?
METHODOLOGIES
-
Competitive Analysis
-
Recruitment
-
Survey Development
-
Moderated Remote Interviews
TEAM
Sole researcher with assistance from the shop owner
TIMELINE
4 WEEKS
Problem Statement
Ashdel Studio was a fairly new business with a small selection of party goods and some novel products that were in development. They had three specific questions to answer: who is the customer? What is the usability of the novel guestbook? And what feedback is there for the current shop?
-
Who is the customer of Ashdel Studio?
As a new business, Ashdel wanted to know what kind of customers they should be marketing to and how to attract more customers. There are many avenues for party planning and people plan in different ways, learning more about the mental models people have when they go about their planning and what pain points they might have would be valuable to Ashdel Studio.
-
What is the usability of the most novel product—the Alphabet Activity Guestbook for baby showers?
Ashdel had recently released the Alphabet Activity Guestbook as a fun, interactive alternative to traditional baby shower games. It is a kit that contains a blank hardcover book with hand-made paper cutouts of the letters of the alphabet, corresponding animals, decorative scenes and a glue stick. Each guest would decorate a page and write a message for the mother in the notes section. It serves multiple purposes -- a guestbook, an activity for the shower and a keepsake for the mother.
-
What feedback is there for the shop and the Alphabet Activity Guestbook?
Ashdel needed customer insights on the current shop to be able to make improvements and to gain inspiration for future products. There were many items in the store and since it was on Etsy, she needed to know how people found the store, or if they found it at all.
I want to understand the mindset of party planners in order to ensure the shop meets their expectations"
—Ashley, Owner of Ashdel Studio
Alphabet Activity Guestbook
Ashdel Studio Customers
Participants of the research were people who plan parties, specifically bridal or baby showers where activities are encouraged.
The owner and myself explored our network of friends, co-workers and clubs to find appropriate participants. We sent out a screener questionnaire to recruit people who fit the profile of very hands-on party planners, and who were preparing to organize a party or had recently arranged a party.
Roles & Responsibilities
My role was to work closely with the owner of the shop to conduct qualitative user research on the shop to date, as well as the usability of a new guestbook.
-
A competitive analysis was performed of similar sized shops on Etsy, popular stationery stores, as well as similar products to the novel Alphabet Activity Guestbook.
-
I recruited participants, scheduled and conducted video interviews, which were done over Google Hangouts.
-
I developed Google surveys for a separate group of participants to get their initial impressions of the shop and the Alphabet Activity Guestbook.
-
The shop owner was very involved in the development of questions, recruiting and interviews. She sat in on interviews and took notes and debriefed on insights gleaned.
Scope & Constraints
Research was spread out over four weeks and was limited by a small budget, fortunately we were able to recruit around 10 participants for the interviews and 16 participants for the surveys.
Competitive Analysis
Online research of similar shops, search results from Google, Pinterest and Etsy.
Moderated Video Interviews
-
Four Weeks from prep to end; Interviews took place over 3 weeks with ~3 interviews per week
-
10 participants
-
Budget: $25 Amazon gift card/participant
-
~1 hour interviews with screen sharing to observe them searching and moving through tasks
-
Google Hangout (Video and screen sharing functions)
Shop and Product Impression Survey
-
Google survey on their own time (within the four week time frame)
-
16 participants
-
Feedback on first impressions of Ashdel Studio Etsy shop and the Alphabet Activity Guestbook
4
weeks
$250
budget
1. Competitive Analysis
A competitive analysis was done of similar shops and products to get insights into the landscape, marketing, and pricing models.
A competitive analysis was conducted of other Etsy party shops, stationery stores, and products similar to the Alphabet Activity Guestbook. The goal was to identify where Ashdel Studio fits in the party shop space by comparing prices, themes, and marketing and to identify where the shop was strong and where it could grow.
Pinterest is specifically designed as an inspiration board which links to other websites, which made it a good place to start when doing a competitive analysis.
The closest products to the Alphabet Activity Guestbook were DIY digital downloads of an Alphabet Book Game. It was priced lower, but the quality was inconsistent because it depended on the customer.
2. Recruitment
Participants were recruited via my network and the shop owner’s network, as well as by word-of-mouth.
A competitive analysis of the market found products that focused on narrow aspects of skincare, relied too much on algorithms to track results, and were not flexible enough for the user to customize based on their personal habits.
3. Surveys and Questionnaires
A questionnaire was sent to prospective subjects to screen for demographic matches. A survey was sent to another test group of 16 people to get first impressions of the current shop, the novel guestbook, and to get inspiration for new products. This was done to get as much feedback as possible without going over budget.
16
participants
4. Schedule interviews, write interview script and questions
Ten subjects who passed the screening were contacted to set up dates for remote video interviews. I wrote an interview script, which included a task flow and specific questions to be answered.
10
participants
(female, age 20-50)
$25
Amazon gift card
per participant
Task flow (have participants describe out loud what they are thinking as they move through each task.
-
Have participant search for inspiration as if they were party planning. See how they organically would look for products.
-
Guide them to the Ashdel Etsy page, observe how they would search the page for products. Then guide them to the Alphabet Activity Guestbook.
-
Tasks on Alphabet Activity Guestbook: Click through photos, read description and read reviews. Ask them the questions below.
Questions for the Alphabet Activity Guestbook.
5. Moderated Remote Video Interviews
Moderated remote video interviews were conducted to collect qualitative data on party planning mental models, pain points in the process, as well as reactions to the guestbook.
Moderated remote video interviews were used because of time and geographical limitations. I was able to observe what the participant was searching for and ask them questions in real time. And I was able to include participants from across the country instead of just regionally.
~1 hour
interviews
6. Compile and Analyze results
Feedback from the surveys and the interviews were consolidated to answer the main questions Ashdel had: who are the customers? What feedback is there for the novel guestbook? What feedback is there for the shop as a whole?
1. Who is the customer?
-
Anyone planning a party, specifically a bridal or baby shower, where guests participate in activities.
-
People most drawn to Ashdel Studio products enjoyed doing certain aspects of party planning themselves, but allowed for more complicated tasks to be outsourced.
-
Most of our participants chose a party theme first before searching for decor or activities. They primarily use Google search and Pinterest to visually get ideas for what to buy and how to decorate. Decorations were second priority to food and were scrutinized most in the budget (interest in cheap or DIY on decor).
Google search doesn’t feel overwhelming, like Etsy. And it directs to Etsy products. Don’t even know what to look for on Etsy.
2. Usability of the novel Alphabet Activity Guestbook
Product awareness was an obstacle of the Alphabet Activity Guestbook, as it is a novel product. Finding it in the search for party activities and decor was difficult because it overlapped several genres—game, guestbook, keepsake. It took further reading and looking through images to understand what was included, how it was supposed to be used, and what to do with it afterward.
Some participants loved the idea as a stress free creative activity that everyone could enjoy. They appreciated that the traditional guestbook was expanded to an engaging memento, personal to the mother and baby. Instead of just being a book that sits on a shelf to be revisited every once in a while, the mother and baby could use the book to learn the alphabet and it could act as a daily reminder of the party for the mother.
Could these items be organized by party type (vs collection)?
Everything has good descriptions and there are a lot of different items to choose from.
Some concerns for the guestbook were that the materials selected were not conducive to long-term usage, especially for children. Some ideas that Ashdel could use to iterate on the product are plastic sleeves to keep the paper pieces from falling off the paper pages or making vinyl stickers instead of paper ones.
3. Feedback: how can the shop and the product be improved, and was there any inspiration for new products?
Feedback on the Etsy shop focused on the organization of the different products and the usability for customers to find what they are interested in.
I was a little confused at first by the store description, but the actual items for sale are lovely.
Could these items be organized by party type (vs collection)?
Outcomes and Recommendations
Ashdel is uniquely positioned as a provider of handmade party goods. several changes were made to keywords, descriptions and photos that helped Ashdel increase conversions.
Changes made based on research
-
A short Youtube tutorial of the Alphabet Activity book was added to the product page to clarify usage.
-
Image and Title update
-
Changed the image with additional descriptors of how to use the book.
-
Price increase based on qualitative research feedback.
-
Added keywords to expand search discovery and clarify the product.
Changes made based on research results
-
ROI results following the changes
~1%
Conversion rate increase the next year after research and implementation of the suggested edits to the guestbook
0.8%
Conversion growth over a year for themed bundles of party supplies.
There was a conversion rate increase of ~1% in the following year since research and implementation of the edits.
*The conversion rate before research was 2.6% (Jan-Sept 2017). Conversion rate after research and implementation of edits was 3.7%
The new bundle listing had a conversion rate increase of 0.8% over a year.
*The conversion rate before research was 2.5% (Nov 2017-Dec 2018). Conversion rate after research and implementation of edits was 3.3% (Nov 2017-Dec 2019).
Bundle of Alphabet Activity Guestbook and a corresponding banner.
Research Lessons
Remote video interviews were great to get some cursory information for Ashdel to move forward with her store and her new product. I learned how to conduct remote interviews and guide participants through tasks without asking leading questions.
Ideas for further research that Ashdel can do to gain more insight:
-
In-person observational research and usability testing of the Alphabet Activity Guestbook would have provided valuable insights into improvements or new product features. This would be a good follow-up project.
-
Adding a post-purchase survey could be a simple way to get more insight into how the customer reached the product and why they bought it.