Handshake / Feed Summit
In an effort to increase engagement across the Handshake platform to 10 million monthly
active users, some of the most talented individuals across our organization came together to rapidly create, test, and iterate on Handshake’s first ever Feed in a research summit that took place in Austin, Texas.
I was able to lead this effort on the research front, using RITE (Rapid Iterative Testing and Evaluation) methodology with a blend of participants to rapidly collect insights on usability, perception on look and feel, and opportunities to iterate and re-test areas of the new home feed, first time user experience, student profile, and onboarding experience.
METHODS
RITE testing
TIMELINE
2 weeks
MY ROLE
Lead researcher
Experience design
Script creation
Recruitment
Synthesis
Executive shareout
COLLABORATORS
Product design
Product management
Content design
Engineers
C-Suite
Spoilers
This research summit was extremely successful in helping define our team and leadership define the vision for what our feed could/should look like, and was the first domino for additional summits that followed
A year after the summit, the feed has been rolled out to 100% of all Handshake partnered universities across the United States, leading to an 8.5% increase in mobile sessions, 22% increase in clicks to events, and 62% increase in time on the Handshake app
Our January 2024 feed launch was featured in a Forbes article, highlighting quotes from research
I. BACKGROUND
What is RITE testing?
Rapid Iterative Testing & Evaluation (RITE) is a method of concept testing where design changes are identified and solved as quickly as possible. As issues and opportunities are discovered, the project team quickly reacts to debrief, categorize issues, and iterate on designs for the next session(s).
Benefits of RITE testing includes…
An engaged team to improve the product in a short space of time
Alignment on opportunities and user issues
Key decision makers being deeply involved with work
Research objectives
It was my role to run all moderated interviews with participants using a high fidelity prototype designed by designers to gain deep insights into the perceived value and usage of four main sections of our mobile app experience:
Onboarding: Evaluate if the onboarding flow is clear with purpose, and can be completed without obstacles.
Feed: Understand what students want in a feed, which content types are worth engaging with, and overall comprehension of structure.
First Time User Experience: Assess how/if students use highlighted features presented in the FTUE of saving jobs/following employers.
Profile: Observe how students understand the benefits of having a profile + what they’d want it to represent.
Research timeline
With the rapid nature of RITE testing, my research timeline was compressed in an intense effort to complete:
Research planning (1-2 days)
Participant recruitment (5 days)
Conducting 8 interview sessions, with key stakeholders as live observers (3 days)
Synthesis and report out (2 days)
II. METHODOLOGY
Moderated interviews
A moderated user study is a one-on-one conversation with a participant. This method is used when we want to learn about user habits, behaviors, and emotions.
For this study, sessions were held as video calls on Zoom as participants shared their screen and walked through the prototype, being asked to think-out-loud as they interacted with different segments of it.
Live observation
You’ll see some team members taking detailed notes for each participant during the study sessions (I’m on the TV!). Myself and the team used an AirBnB in Austin, TX as a collaborative workspace. Some of the my observers included members of C-Suite, lead designers and engineers, and product managers as this was a high priority project.
RITE schedule
You can see the intense schedule of the sessions I conducted, setting blocks for setup, interviews, debriefs, breaks, and design iterations based on what we learned in the sessions.
Prototype personalization
III. DESIGN ITERATION
In this study, designers created unique screens for Onboarding, Feed, and Profile. Individual prototypes were then customized for each student we tested with, displaying personalized content during onboarding and on the home feed. From research, I was able to help inform design decisions and the experience we wanted to curate.
Debriefs and change logs
In the afternoons, the team gathered to discuss prototype changes to test for the next day based on what we learned from my interview sessions with students.
One simple example of this was restructuring our navigation iconography as participants didn’t know what our new “saved” icon represented an seemed counterintuitive to our existing messaging screen.
IV. SYNTHESIS
Process
As a part of my synthesis process, I use Reduct and Figma tools to rewatch all interview recordings, tag and extract individual clips that build a behavioral pattern, and cluster based on my findings. By doing this method, I’m able to prioritize which insights will have the highest impact and find supporting quotes to hit the insight home. This allows me to then create video compilations or impactful quotes to share with my stakeholders.
V. FINDINGS
Top findings
Onboarding
The onboarding experience was described by participants as clean, simple, welcoming, and asking for the right things at the right time.
Students understood the importance of questions asked in onboarding and held the expectation that their responses would reflect in the form of more tailored job recommendations.
Job location is a significant piece of the onboarding and holistic app experience which was referenced throughout studies, and is something our team is working on optimizing.
6 of 8 students were willing to opt into push notifs and 4 of 8 students were willing to opt into SMS. There is an understanding of the benefit to not miss deadlines + staying on top of your inbox, yet also a hesitation about receiving too many notifs regardless of disclaimers we show students due to the amount of “spam” they receive.
Feed
Participants expect Handshake to primarily serve them job opportunities before other forms of content, which presented a unique challenge for us to navigate afterwards.
Handshake is a comfortable, “safe space” for students. What Handshake offers in the form of familiarity through university affiliation, our competitors cannot. We need to ensure we maintain that identity while simultaneously expanding learning opportunities in the platform.
With that said, a multifaceted feed is an opportunity for Handshake to begin evolving the impression of it only being a job platform. Through the summit and continued research, we now have a strong understanding and conviction for what could make a feed most successful: network integration and engagement, opportunities to build trust, and leveraging content that only Handshake can produce.
FTUE
‘Liking’ was perceived to be more informal, public appreciation, whereas ‘saving’ was perceived to be a more personal action for content that was worth revisiting.
Emojis and reactions seemed to be associated with applicant competition rather than informational content when included on job postings.
Participants primarily thought about friends when presented with the option to share jobs, with the expectation that they could be sent outside of the Handshake app.
Following companies and employers was understood to be an easy way to keep up to date with them as well as to continue seeing similar posts.
Profile
The purpose of completing a profile on Handshake was clear. Students want to show a digital representation of themselves and their resume for employers to view and connect with them.
There was an understanding that the majority, if not all, information entered would be for public consumption for all Handshake users - with the caveat that social actions such as saved posts would be private.
Participants seemed more focused on a professional representation of themselves rather than an informal one on their profile. There is also hesitation towards writing non-academic goals or “fun facts” that could backfire if not executed properly for sections that encouraged more personality.
The findings in this summit have also contributed to our current efforts of redesigning the profile experience.
VI. IMPACT
Where are we today?
Since this summit in November 2022, our product team has worked endlessly on perfecting it to be the perfect balance of active/passive consumption for content that can inspire students to take that next step in their career, and present the most relevant opportunities to them along the way.
The launch of the Handshake feed in January 2024 has had a vital impact on our goal of achieving 10 million monthly active users. Read more about it in our featured Forbes article, highlighting quotes from research!
Since this summit took place, the feed has now…
Resulted in:
8.5% increase in mobile sessions
22% increase in clicks to events
62% increase in time on the feed
Been launched to all partnering universities that use Handshake
Scaled to allow employers, universities, and students who have access to post text, videos, images, and links
Acted as an incubation hub for new ideas and experimentation we’re exploring
We are now working towards the launch of a web experience in mid 2024.
Quotes from teammates
VII. TESTIMONIALS
“I recommend spending some quality time with this findings report. Sean did an amazing job synthesizing the input and output, teeing up findings and evidence. I don’t want to know the kind of time and effort that went into something like this, but well done, Sean!”
“Incredible. This is truly A+ work.”