My experience designing an AI-based fitness training platform for a startup.
Overview
Project ConText
Product Design Internship
Timeframe: 2 years (October 2020 - September 2022)
Team: Tejus Krishnan, Audrey Kan, Helen Casady, Raju Somu
Supervisors: Gopi Prashanth, Helen Casady
Tools used: Figma, Slack, Notion
Project Deliverables
User interface
Design system
Interactive prototypes
User flows
Wireframes
My Role
UI/UX Design
Design System
Interactions & Visuals
Brand Identity
Project Background & Context
Spyn was my first full-time internship, and an opportunity to apply all the design skills I had learned (either in school or self-taught) to a real product with other stakeholders involved. As one of the first people hired to the company, I was solely responsible for the entire user interface for several months prior to us expanding the design team and implementing more structure.
Spyn is a challenging project to document in a portfolio piece, for two primary reasons: 1) the volume of output and 2) the changes to the product’s vision over the years. The app’s design went through at least four incarnations over the two years I spent at the company, each of which could comprise its own case study given the abundance of design artifacts we created for each. The app has since continued to evolve its mission, core features, and brand in the time since I completed my internship. In that sense, this project doesn’t really have an “end” so much as a cutoff point.
As a startup, our design process was scrappy and iterative rather than rigid and linear. Engineers would implement features as they were designed, yielding feedback I would incorporate into new iterations on a daily basis. Features I designed were retained, discarded, reintroduced, and/or reconfigured constantly as the app evolved; all of them were the product of a great deal of thought and effort.
Since the details and design decisions behind each of Spyn’s features and flows could fill a whole book, this case study is instead a broader look at how the design of Spyn evolved through multiple stages over the two years I part of the team.
What is Spyn?
The answer to that question has changed over the years, but at its core, Spyn is a mobile and web-based app for virtual fitness training. Clients can practice exercise lessons at home based on videos uploaded by trainers to their online studios.
Spyn uses computer vision to closely evaluate users’ physical movements relative to trainers’ lessons and score their performance accordingly. By leveraging AI to provide the detailed, specific feedback that only humans were previously capable of, trainers can provide their expertise to a far larger audience than possible with in-person classes alone, without compromising on the depth and quality of instruction afforded to clients.
Spyn’s CV algorithm lends itself to a variety of exercise types, including strength training, cardio, yoga, pilates, and even dance. The detailed metrics used to score users provides clients with targeted, specific ways to evaluate their progress and improve their skill. The app supports many different types of content, including live and private classes, and affords trainers enormous flexibility to customize their video lessons using AI.
Now let’s take a look at how it got there.
V1 (2020)
The beginnings
A Mobile App for Indian Classical Dance?
When I was hired in October 2020, Spyn’s vision was already formulated: leverage the power of AI and computer vision to revolutionize at-home workouts. This was an especially radical idea during the pandemic, where fitness instructors struggled to provide quality training in a job driven by in-person feedback.
Our initial user base was Indian classical dance instructors, who already faced the challenge of keeping the attention spans of young children, and doubly so virtually. The structure of Indian classical dance lent itself well to how we planned to configure the video lessons, and the long-term goal was to make the app abstractable to other body forms, such as yoga, dance, and strength training.
The user journey broke down as follows:
Instructors would record videos of them performing a dance sequence using the Spyn mobile app. They would be instructed to perform the dance slowly so that each individual pose was clearly discernible.
Spyn’s CV algorithm would identify the individual poses in the choreography, and extract vector images of the poses that would be placed on cue cards corresponding to timestamps in the video.
Once the lesson was published, students would perform the dance based on the instructor’s video, and our algorithm would analyze their bodily movements to score how accurately they were performing, as well as highlight which body parts were posed incorrectly.
User Journey
Spyn’s user journey.
Recording Lessons
Below are some of my early Figma designs for the video-recording portion of the app. Instructors would start by adding a spoken-word audio track that sets the beat and tempo of the dance, then dance to this audio in the video portion. The app then auto-generates cue cards for each pose in the sequence, allowing the instructor to manually reorder and delete the cards if need be. The Dance Central game series for the Xbox Kinect was a major source of inspiration for my design.
The use of cue cards to mark individual poses in the choreography was inspired by the Xbox Kinect game Dance Central.
Performing Spyns
I designed a corresponding client-facing experience for the app, in which they would see their own video juxtaposed with that of the instructor, and receive scores (based on color highlights on the cue cards) of how well they performed the dance. A retrospective analysis would reveal which body parts were in error. We also added a calibration step at the beginning to ensure the camera could see the user properly.
Testing & Feedback
As soon as I had a high-fidelity UI iteration ready, our front-end engineers immediately got to work implementing it on Android, while our CV engineers oversaw testing of the app’s scoring capabilities. All of us on the team would regularly test new patches of the work-in-progress app and report how it was performing.
A Bharatanatyam instructor recording a choreography video for her students.
One of our CV engineers calibrating the app.
We also began taking the app to Indian dance instructors directly, and having their young students perform the choreography they were taught via Spyn to see how well it would score them. This was a project where the end users were subject matter experts themselves: we relied on the input of the instructors to understand the choreography structure of a dance lesson and consequently designed the app to reflect its stages.
In addition to patching performance bugs, these testing rounds also yielded crucial insights for improving the UI: for example, showing a sequence of upcoming cue cards a la Dance Central was too distracting for a small mobile interface, and so I modified the design to only show the next and last cue card.
Web App
In parallel with the mobile recording / performance UI, we were building a web app exclusively for instructors to monitor and manage their classes. The web app was meant to provide a comprehensive overview of students’ performance across all of an instructor’s classes (collectively known as their “academy”), as well as provide tools to schedule private lessons, communicate with students directly, assign lessons to student groups, and manage payments. We felt these sorts of administrative tools were more accessible on web than mobile.
The web app also allowed us to introduce an architecture for how content is organized on Spyn. Each video is known as a “Spyn” (or “Lesson”) and receives a difficulty rating from the instructor. Lessons are assigned in sequence to different Classes of students, reflecting how Indian dance instructors would oversee different “batches” of students over the week. Finally, an Academy was to be the overarching environment for all of an instructor’s content, and was the level at which student memberships and payments would be managed.
Spyn - Content Architecture
The screens below show how far I got with the initial designs before we introduced an overhaul to the UI:
V2 (2020-21)
Introducing Studios, a better design system, more focus on mobile
Rethinking the Design
Why redesign the app?
3 months into the project, I began to realize how much the fast pace of our product development was costing me the ability to keep the UI consistent and well-managed. I was creating components and introducing colors and styles in a very undisciplined fashion, a problem exacerbated by how frequently we’d ideate new features and pump out corresponding UI elements. As a student, there were also several gaps in my knowledge of industry best practices for UI design, which I felt needed fixing if this was an app we were launching to the general public. As the person solely responsible for the UI at this point, I felt it crucial to consult an expert’s opinion.
I had our founder/CEO/manager Gopi connect me with his colleague, a veteran design director with over 15 years of experience at Microsoft, Amazon and Facebook, to evaluate our design system and answer all my questions about how to better approach the design. He identified the lack of a clear visual language and the overcrowded component library as my two key problems. He provided several vital recommendations:
Conduct a thorough visual audit of the UI to identify recurring layouts, visual patterns, color/typography styles, and commit them to a clear style guide that I could use as a rulebook for approaching new pages in the UI.
Be restrained in creating components, utilizing the principles of atomic design to build new UI elements out of common building blocks rather than carelessly making new components out of everything.
Make use of Figma features such as Grids, Variants and Auto Layout to establish better constraints on the UI and maintain visual consistency when designing new features.
Consult Google’s Material Design guidelines for tips on how to approach layouts and styling.
A new Design System
I decided to heed the advice I was given by completely overhauling the design system from scratch, feeling the current UI was becoming visually stale. I started a new file with a clearly defined design library for typography, colors, icons, page layout templates, and atomic components, and remade some of the key screens on the web app entirely using these building blocks.
The revamped web app:
Expanding content: Introducing Studios
My effort to overhaul the design system was well-timed, as it coincided with several changes to the app’s vision and design roadmap. One of the most important new concepts introduced during “v2.0” of the app’s development was studios. Whereas previously we had the vague concept of an “academy” to house all student and lesson info for a single instructor, a studio provided a more robust architecture for exercise content, and empowered instructors to build a distinctive brand and business - thereby monetizing their content at a larger scale.
I began exploring designs for a broader content homepage for exercise studios across different body forms, which would be publicly available to browse like a streaming app. Users would be able to join studios on a subscription-based payment plan to receive access to its owner (the trainer)’s exercise videos. This even raised the potential for partnerships with fitness companies and celebrity trainers who could produce and host their content on Spyn. These ideas are explored in the mockups below:
Time to Go Mobile
At the same time we began exploring studios, we accelerated design and development of the Spyn mobile app, whose design was thus far limited to just the video recording interface.
I was greatly aided by the arrival of Audrey Kan as our second UX designer. Whereas I was more comfortable working in high-fidelity with prototyping and visuals, she was very adept at rapidly producing mockups at a lower fidelity and exploring different layouts and information architecture. This gave us a chance to iterate on design ideas more thoroughly before committing them to high-fidelity mockups, which are harder to modify. Our collaboration helped us delegate tasks that played to our strengths, producing better UI at a faster rate.
Our work on the mobile app yielded several improvements to the app, both in terms of new features and revisions to existing ones:
Studios & Channels
We introduced the concept of channels to group lessons in a studio as a sequence, allowing instructors to chain exercises together. Each studio would have a public channel - available to members and non-members alike as a way to preview the content available - and private channels, which were workout series for members only. These pages had both a client and admin view, the latter allowing for content and membership settings to be edited.
Insights & metrics
I designed numerous screens for progress tracking at a studio and channel level, as well as scoring for various metrics our CV algorithm used to evaluate users’ performance, such as their accuracy, fluidity, transitions, and symmetry. Users could also share their score at the end of each exercise on social media as way of tracking their progress with their friends, similar to a feature like Spotify Wrapped.
Video Editing & tutorials
We introduced a better video editor that allowed for video trimming and easier editing of keyframes. We did away with the “pose extraction” mechanism - which we felt cumbersome and difficult to implement within our timeframe - and marked poses using video timestamps instead. We also added an onboarding layer to the video recording experience, with visual tutorials explaining how to create a lesson.
Branding & Signup
On the side, I had some fun creating some brand aesthetics, which ended up in the web app’s signup flow.
V3 (2021-22)
Aesthetic experiments, mobile exclusivity, pivot to PT
UI Experiments
This being a startup, we were very comfortable making drastic, experimental shifts in our app design for the sake of exploration. There was a consensus among the team that our visual design had become oppressively dark, drab and monochromatic, and was lacking a certain “life”. Concurrently, we noticed how much more time we were spending designing and enriching the mobile experience compared to web, and hence attempted to centralize all app features on the mobile app while doing away with web entirely.
I decided to create a new style guide that made heavy use of gradients, soft lighting, blending modes, and vibrant neon colors, giving the app an aggressive jolt of life and color. This iteration was ultimately short-lived, as many of these aesthetic elements were difficult to implement in code, not to mention garish and overly brand-forward (whereas the intent was to spotlight the instructors’ brand). However, it was a valuable experiment in making a daring overhaul to the app’s brand and style guide, which foretold our next (and in my case, last) update to the design system.
We also began to center the user experience around physical therapy, a powerful and unexplored use case for the app. The nomenclature we used for app began to reflect the PT domain.
Routines
This version of the design also introduced more structural/conceptual changes to the app, the most important of which were Routines, which brought a more “atomic” approach to creating workouts. Individual exercise moves could be chained together and repeated, with the AI automatically editing the raw video footage to duplicate/alter the length of individual moves based on pose information. Routines could then be sequenced together as Programs, creating more complex, multifaceted workouts that are evaluated as a whole by the app.
A card reviewing a user’s performance on an individual exercise in a routine.
Routine Scorecards
Below is an example of how routines were to be scored: each set of an exercise comes with a certain number of reps (the instructor need only record one; the AI takes care of the repetition within the video.) The instructor assigns a certain number of sets per exercise to the routine, after which the routine is scored as a whole. The algorithm collects various metrics based on an analysis of the user’s physical movements, such as “mobility” and “fluidity”, which are evaluated on a set-by-set basis. Performance on the individual exercises (Spyns) is also noted in the scorecard.
V4 (2022)
A cross-platform fitness experience and the birth of the business model
Spyn Becomes a Business
Spyn underwent one final evolution during the time I spent there, and this one was certainly more set in stone than the others. We decided to pivot completely into the fitness training space, essentially fulfilling the mission we had the beginning of the project. Having gone through so many iterations of the design, continually refining the algorithm along the way, we were more confident about what product we wanted to sell to customers.
Spyn was to be a cross-platform, AI-driven solution for fitness trainers to bring their whole client base into the app, essentially running virtual studios with a high degree of flexibility regarding content types.
Expanding the team
With our refocused mission statement came a significant expansion and restructuring of our team. With design tasks growing in both size and scope, we hired Helen Casady, a lead designer with over 20 years of experience to oversee design efforts across platforms, as well as Raju Somu, another UX designer based in India, which allowed me to dive deeper into the specific features I was working on rather than being across everything all at once. I went on to design most of the web-facing UI for the new version of Spyn, while Helen oversaw much of the mobile design.
Equally important was the hiring of several fitness trainers to act not only as consultants, but co-designers on the product. They were very involved in evaluating both our Figma designs - which were focused on building out the trainers’ studio and business tools - as well as the latest builds, which they would test with their clients and provide their feedback to us directly. Our test user base suddenly exploded into the hundreds, making our design process radically more user-centric - as well as more organized and efficient, now that we had more manpower and accountability to real users.
A design system for everyone
The first thing Helen and I did was conduct a thorough audit of the design system and enshrine a style guide that everyone - including developers and other designers - could follow. We also decided to take the visual design in a sleeker, more conservative direction based on a survey of other popular fitness apps like Peloton, as well as our desire to focus the app’s branding on the trainers rather than calling too much attention to ourselves.
We settled on a palette defined by steel-blue surface colors with accents of lime-green, cyan and orange, with light use of gradients. We used two typefaces - one for display and one for text - both of which were Google fonts, allowing our developers to faithfully recreate the design in Flutter (Google’s software development kit).
A snapshot of the revamped component library.
Live Classes: Introducing Traks
One of the most important milestones of this Spyn iteration was the introduction of live video workouts, which we called Traks. Trainers could host any of their workout routines as live Traks, providing live feedback to clients over a video call. I designed both the client and trainer-facing sides of the Trak UI, the latter being significantly more complex and multifaceted.
Traks (trainer UI)
The trainer-facing UI for Traks was designed to provide clear visibility into both clients and the exercises involved in the routine, while allowing for easy access to several administrative tools. Below is a snapshot of the UI:
The bottom of the screen displays a reel of the exercises comprising the routine, with the current one highlighted in the corner. Trainers can pause the session or skip an exercise, as well as alter the recovery time between exercises.
The top of the screen displays Trak progress, the time elapsed on the current exercise, and various video controls.
The middle of the screen is dedicated to clients’ videos. Clients can enable/disable their videos and raise their hand if they have questions.
The interface could fit anywhere from 1 to 14 videos on the page at a time:
Traks - Trainer Tools
I also included various other training tools that instructors can access while a session is going on:
Zoom in
Trainers can expand a single client’s video to focus on their performance.
Manage participants
Trainers can see a full list of participants, as well as live metrics on time elapsed and reps completed. They can also invite participants to a session while it’s happening.
Monitor Client Progress
Trainers can see the individual profiles of their clients and take personalized notes on their performance. They can also see what equipment the clients owns at home, as well as any additional factors the client tells them about, like injuries or health conditions.
Live Chat
Trainers can engage in a live text chat with clients.
Add Music
Trainers can add dynamic background music to their exercise queue, and alter it between exercises.
EDit Exercise Properties
Trainers can edit the properties of individual exercises in the queue while the Trak is happening, such as equipment required, duration, and recovery time.
Edit Exercise Queue
Trainers can drag and reorder exercises, as well as add and subtract exercises from the queue, all while the track is going on.
Traks - Client UI
Using the trainer UI I designed as a reference, I then designed a corresponding client-facing UI, which was more minimalistic and focused on one exercise at a time. Unlike a trainer, who would be sitting at their computer overseeing the session, clients would in fact be performing the exercises, and so I had to ensure all the important information about the workout was visible from a distance. I designed a number of different states for the Trak UI depending on the stage of the workout.
Upon starting a workout, a client is taken to an introductory screen describing the workout and telling them which equipment is required, as well as a summary of all the exercises in the queue.
Once the workout starts, clients will perform each exercise individually in sequence, matching the trainer’s video. The trainer will be offering tips and guidance to the class via video. Exercises are complete either by holding a single pose for a required amount of time, or completing a required number of reps.
Each exercise comes with a recovery time in between, which the trainer can calibrate. Clients receive a preview of the next exercise and the equipment required.
Creating Traks
Moving over to the more managerial side of things, I also designed the interface for clients to create Traks, which essentially function like playlists made from exercising workouts in the trainer’s content library.
Trainer Studios & Payment Plans
While the majority of my work on this iteration of Spyn was for desktop, I also designed an extensive series of mobile flows for trainers to set up their studios and subscribe to a payment plan. This was the first time we actually began thinking about the practicalities of monetizing the app, as well as including a payment mechanism in the UI.
Studios & profiles
As established in prior iterations of the app, a studio acts as the trainer’s place of business on Spyn, through which they can host their exercise content, invite clients as paying subscribers, and build a personal brand for themselves. A trainer’s personal profile would act as their studio homepage, and I designed the flow for setting one up.
An interaction flow I arranged in Figma, detailing the various stages of creating a trainer studio profile.
Empty template for creating a trainer profile
Payments & Subscriptions
As of this iteration, we planned for a four-tier subscription system for trainers to join Spyn, with higher tiers allowing for trainer profiles to be visible to the general public rather than just invite-only. I designed a series of screens for upgrading plans and processing payment information, as well as multiple states to the trainer profile screens reflecting the status of their subscription. When arranged as a user flow, these screens became a visualization of our business model.
Reflection
How did the design evolve, and what did I learn?
Design Evolution
V1 (2020)
V2 (2020-21)
V3 (2021-22)
V4 (2022)
What I Learned
Spyn is a monumental milestone in my design career, for it was what kicked it all off. This job was my first time applying any of the design skills I’d learned through school and personal projects to an actual organization with real-world stakes. As a startup, the company’s breakneck pace helped me quickly forge various skills have that gone onto inform all my subsequent design experiences in a number of profound ways. Here are some of the many takeaways I got from being part of Spyn:
Ownership. I got to work on every facet of this product from the ground up, and led the charge with the design as the product’s goals and features shifted and morphed. As one of the first members hired to a team of no more than 10 people, I was not merely executing on a business vision, but had a hand in shaping that vision as well, as all of us were a journey of discovering what would make this product truly succeed.
Cross-functional collaboration. This was my first time working directly alongside other stakeholders, whether it was our CEO, the front-end and CV engineers, other UX designers, and eventually the fitness trainers themselves, who evolved from consultants to participatory designers. Given the small size of the team, everyone was more or less their own “department”, and I as a designer got to immediately see the downstream consequences of my decisions on the rest of the team. Everyone played an instrumental role in bringing the app to life, and it was thrilling to see all the various parts of an organization so clearly, free of bureaucracy. It felt like a microcosm of the design/implementation cycle at larger companies.
Accountability, work ethic, and fast-paced feedback cycles. I received this job right in the middle of the COVID-19 pandemic, and it was exactly the jolt of exciting, fast-paced work I needed to shake myself out of the dreary monotony of quarantine. More importantly, our team’s daily cycles of iteration and feedback led me to produce a truly prolific amount of output in a very short time - something that my subsequent managers have repeatedly cited as one of my strengths.
Design systems. Prior to Spyn, I was coming up with UI in a fairly improvisational, unstructured manner, and was entirely unfamiliar with concepts like grid systems and component libraries to keep the UI consistent. At Spyn, I created not one but four iterations of a design system from the ground up, which not only made me much more organized with my designs, but intimately familiar with every piece of UI in our app.
Domain expertise and flexibility. Prior to Spyn, I was under the assumption that I’d have to be familiar with the domain of the product I’m designing for. Turns out this wasn’t entirely true: if anything, design itself can be the tool for learning about the product’s domain. I didn’t know the first thing about Indian classical dance (nor the fitness industry) when I joined, but talking to end users and building an app that catered to their needs (as well as the nature of the exercises they taught) made me much more familiar with their work. The more time I spent in the company, the more I also developed a stronger big-picture awareness of where a product like this would fit in the business landscape of the fitness industry.
“Killing your darlings”. A term used by writers to refer to extraneous elements that must be removed for the sake of the work, no matter how hard you worked on them. This is every bit as salient in UX design, which is defined by iteration and critique. Make no mistake - Spyn was a slaughterhouse of darlings. I learned the difficult but ultimately liberating lesson of not getting overly attached to the early drafts of my designs, and embracing the change that comes with feedback and testing. My school projects, by comparison, came to an end as soon as I had a single high-fidelity prototype ready for presenting. I had never gotten to experience the cyclical part of the user-centered design cycle until I came to Spyn, where it defined my day-to-day work. I feel both more disciplined and more creative thanks to the work I got to do here.
Thank you for reading!