JAMIE FONG


Ubuntu: Storytelling through Collaborative Mixed Reality



Joy Media x University of Washington (Master's Capstone)



2024



my role





Design Strategist +
Product Designer



the team





3 Designers
1 Sponsor
1 Technical Consultant



Timeline





6 months



Status





In discussion with sponsor for further development



OVERVIEW





Ubuntu is an innovative digital platform that serves as an immersive repository of African knowledge and experiences for Afro-diasporic individuals and the broader public to discover and connect with African culture and history.


The project aims to bridge the gap between diasporic individuals and their ancestral roots, ultimately contributing to a stronger sense of identity and community. Our research found that many Afro-diasporic individuals are seeking to foster a sense of pride and belonging in the world through counter narratives, especially around pre-colonial histories and contributions of African culture to society.


I was responsible for driving the process in both research and design, pivoting based on sponsor feedback, and designing the core interactions within the platform.


This master's capstone project was sponsored by Joy Media, a non-profit consulting firm specializing in culturally diverse content strategies.



Project Goals



Bridge gaps to resources



Access to positive or authentic information on African culture is limited and resource intensive to attain outside of the continent. Ubuntu aims to create a repository of African stories for discovery and exploration.



Build and support an uplifting narrative



Leveraging counter-narratives was mentioned as a powerful tool during interviews. Highlighting stories of progress, achievement, and contribution helps to foster a sense of pride and belonging in the world.



Ensure easy access



Providing an easily accessible platform where people can learn, irrespective of financial, geographical, or social barriers can help bridge a gap to meet the need for discovering these stories.



Empower the diaspora



The outcome is something that is co-created and resonant with the target population. "Nothing about us without us."



assumptions



The following key factors guided my decision-making and approach as a designer



1.



I was an expert in the design process, not the topic area. It was especially important that subject matter experts were brought into the loop.



2.



Innovation was not about utilizing the latest technologies for the sake of it, but finding the most appropriate way to leverage what was possible.



3.



The participants that we talked to in our generative and evaluative research was representative of our broader target population.



key terms



This project utilizes the following key terms, as defined from our research and speaking to subject-matter experts.



African storytelling



Practices and traditions of sharing history, constructing narratives, and connecting communities belonging to and stemming from African cultures.



Afro-Diasporic



Individuals who self-identify as descending from African ancestry, and have had their connection to Africa either voluntarily or involuntarily severed due to immigration or historical events.



Soul (of indigenous knowledge)



The context and intangibles surrounding the story. How the story changes according to who is the story holder at that time.



Counter-storytelling



Magnifying the stories, experiences, narratives, and truths of underprivileged communities through their perspectives. (Castelli 2021)



research





Research was conducted over a 10 week period to develop a deeper understanding of the problem space and foster a strong sense of empathy with primary stakeholders.



1. Field Observations



We began by immersing ourselves in African and Afro-diasproic spaces. We wanted to understand how stories are shared and experienced today. We looked at both museum and non-museum contexts to understand how they were communicating the “soul” of African stories.



Field Observation



Session length: 90 min

Total sessions: 2



Locations



Northwest African American Museum (Seattle, WA)
​
Spirit of Africa Festival (Seattle, WA)



limitations



We visited each location once and took note of our observations with team disucssion. This was not a formal field study.



2. SME Interviews



We talked to subject-matter experts in order to learn deeply about the following topics: the essence of African storytelling
the role of technology in this space, and the relationship between storyteller and audience.

In preparing for these interviews, I proposed an interview format with two sections, (1) broader questions we ask all SMEs on the topic space, and (2) questions specific to their particular field of expertise. Part of the preparation process involved reading the published work of subject-matter experts and formulating relevant and well-informed questions.

Each team member took turns facilitating and note-taking during interviews. Interviews were conducted both in-person and remotely via zoom.



SME Interviews



Session length: 1 hr

Total interviews: 3



Areas of Expertise



Art History and Media

Human-Computer Interaction
​African Food Studies



3. Semi-structured interviews



The largest portion of the generative research phase included conducting semi-structured interviews to get a deeper understanding how self-identifying afro-diasporic people perceive, understand, and connect with African storytelling in the context of reconnection with their ancestry.

Preparation for the study included sending out a screener survey and writing a discussion guide. My role consisted of writing interview questions, reviewing the discussion guide for best practices, project management, and conducting interviews.

Each team member took turns facilitating and note-taking during interviews. Interviews were conducted both in-person and remotely via zoom.



Interviews



Session length: 1 hr

Total interviews: 10



Participant Criteria



Self-identifies with the African Diaspora

Interested in connecting further with African ancestry



Participant ratio



Participant interviews: 9

SME interviews: 5



limitations



All participants were located within the United states.



Analysis



We utilized a mix of coding and affinitizing to parse and analyze our data. The 10 interviews were split among team members to parse out important quotes and pull out emerging themes.



1.



Collecting and Chunking Raw Data



We compiled raw data from participants and identified areas of interest.


Purple Text = example of chunked content within a longer participant quote



2.



Coding Chunked Content



Next, we developed a codebook and assigned different colors to the chunked content based on the corresponding code.


I led the team discussion on identifying emerging themes, where we came to a consensus in forming a codebook.



3.



Affinitizing Coded Content



We moved the coded data to a figjam board and grouped similar content together to create subgroups.


I suggested we do this to gain further context and clarity under each parent code.



4.



Preliminary Observations and Findings



Based on the themes that appeared in the affinitized categories, the team recorded their initial observations.



Insights



I pulled a preliminary set of 5 insights from these observations and the affinity map. Afterwards, I discussed with the team to refine this preliminary set into our final insights.



Areas of opportunity



From these insights, I identified 5 areas of opportunity for potential design interventions. These drove ideation and provided a design direction for a co-design workshop.



Workshop





2 HOURS
Remote via Zoom



I developed a co-design workshop with the goal for us as a team to better understand the context and nuances of what resonated with participants in regard to areas of opportunity, challenges, and potential solutions.

The workshop served to validate and prioritize our insights, forming the foundations of the design stage. My teammates and I alternated in facilitating workshop activities.

Our participants consisted of 3 Afro-diasporic women based in the U.S who did not participate in any prior study on this porject.

Workshop agenda:
  • Icebreaker
  • Co-creating a design direction
  • Ideation session on aligned design direction

Joy Media was impressed by the workshop setup and results. The sponsor saw immense value and potential for this framework to guide future work for bringing in the voice of affected stakeholders.


Part 1: Creating a design direction



In the first half of the workshop, the goal was to validate and prioritize our design opportunities discussing how each opportunity did or did not resonate with participants and why.

I facilitated the discussion for the group to come to a consensus on a design direction and articulated the combined "How Might We" statement within the workshop.



Part 2: Ideation Session



After coming to a consensus on a resonant design direction, we ran a brainstorming activity. Both the designers and participants contributed their ideas. Afterwards, participants were asked to vote on their top choices. Designers refrained from voting. After discussion on why participants liked certain ideas, we ran a secondary brainstorming session to come up with potential barriers. This was so we as designers could be aware of potential barriers when making design decisions.



Examples of top voted ideas and potential barriers



Design Principles



Looking at primary and secondary research alongside workshop insights, I proposed a draft of 5 design principles. The team worked together to narrow to 4 principles to guide our design decisions.



Authentic



Respecting and preserving the soul of indigenous knowledge.



Uplifting



Establishing a focus on empowering narratives by highlighting progress and contributions, while being mindful of not minimizing difficult histories and conversations.



Accessible



Designing for a low barrier of entry and easy discoverability.



Agency



Building with a co-creation framework, not just a participatory one, building a long-term relationship with our stakeholders. “Nothing about us without us.”



Design





Design was based on insights gathered from the research and workshop. Feedback on concepts and feasibility were provided on a regular cadence from the sponsor, technical consultant, and professors. The design by storyboarding method was leveraged to shape and refine ideas.



Initial ConceptING



As our insights pointed towards a desire for accessible travel, early design concepts focused heavily on immersion technology to tell African stories, such as a device that utilizes projection to bring stories to life.


The initial direction was to utilize projection technology, where users can interact with a projected construct of a guide whom they can ask questions and converse with naturally. The guide can introduce them to resources and the system can create visualizations of different environments, events, and artifacts users can interact with in real time.




Storyboard of an early design concept for a projection-based storytelling experience



We also explored ways that AI could be leveraged, from creating a collective consciousness to being a personal storyteller.


Ultimately, the team decided that AI, in its current state, held too many biases to serve as a fundamental building block of the platform. It threatened the authenticity and soul of the knowledge that we were trying to share.


However, AI was leveraged throughout the design process, such as for creating visualizations of concept ideas and generate placeholder content.



Feedback on Initial concept



We received feedback from the sponsor and technical consultant that elements of the initial direction would need to be reconsidered based on feasibility and stakeholder goals.


During this meeting, the sponsor wanted to shift the approach from a B2B approach to a B2C approach.



Feasability



Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed malesuada faucibus ex nec ultricies. Donec mattis egestas nisi non pretium. Suspendisse nec eros ut erat facilisis maximus. In congue et leo in varius. Vestibulum sit amet felis ornare, commodo orci ut, feugiat lorem.



Collaborative Interaction



Keep collaboration as a core part of the experience, however consider how projections are dependent on the position of users viewing them. This adds constraints to simultaneous story exploration.



Accessibility



Designing a novel physical product or device, i.e. projector, decreases accessibility. Creating a physical product is more in line with a B2B approach, supporting museums and institutions



People vs. Institutions



Having a way to discover and connect with stories may be more effective at home, with family and close friends as opposed to in museums. Museums and institutions continue to house African artifacts and often reinforce post-colonial perspectives.



Pivoting the concept



We received feedback that accessibility and feasibility were major areas of concern that needed to be addressed on our initial concept. The team knew we wanted to keep the collaborative nature of the experience and embrace augmented reality, however we felt stuck on how to proceed, even after many hours of discussion.


I began to think about what people would already have on hand, and how we could potentially leverage this to create a novel and immersive experience. During a meeting, I had a lightbulb moment come to me around transforming mobile devices into a looking glass into a new world.


I led a design strategy conversation with the team where we journey mapped our solution, and then focused on a defined scope of intervention for our limited time frame.



Leverage existing technology



Maintain collaborative nature



Keep it low cost and accessible



Enable discovery at home or a museum



Early paper prototype of looking glass concept



With only a month left on our timeline, the scope needed to be kept narrow if our solution was going to have any impact.


I led a design strategy conversation with the team where we journey mapped our solution, and focused on a defined scope of intervention, calling out key functionality we would need to develop within our limited time frame.



Defined scope for the project



User flow within defined scope




Discussion defining the project scope and functionality



Physical Prototyping



Materials: Foam core, paper, acrylic sheets



Since the experience we envisioned merged the physical and digital realms, I advocated for creating a physical prototype to understand context and nuances around the interactions. With this prototype, we aimed to create a rough concept to test a single user's embodied interaction with our product.


Our mixed reality concept relied on sensors within the smartphone such as the accelerometer, gyroscope, and magnetometer to sense its environment. As current augmented reality prototyping tools like Adobe Aero and Bezi rely on camera input, they couldn't accurately represent our desired interaction. Therefore, we went back to basics and experimented with paper prototyping.



A static background was printed and mounted onto foam core.



A hole was cut into foam core overlay to display what someone would see on a mobile screen.



Overlays were created to represent screen interactions.



Overlays could easily be swapped in and out to mimic how the system would react.



Materials with different opacity were leveraged to show different interactions.



Prototype Evaluation and Feedback



With a tight deadline approaching, the team conducted one concept evaluation to gauge any glaring issues with the design concept. We also consulted an SME and technical consultants throughout the process for feedback.



Concept Testing



Session length: 60 min

Total sessions: 1

Limitations: We could not test with our primary stakeholder group due to time constraints



SME Feedback



Session length: 60 min

Total sessions: 2



Technical Consultant Feedback



Session length: 60 min

Total sessions: 2



Final Design





The high fidelity assets were created within 2 weeks' time. I was responsible for creating the mockups and animations, which I created using Figma.


Placeholder images and story is sourced from Culture Bay and was created using AI.



Key Flows



Design Specification



I created labeled diagrams, system maps, and input/output maps to communicate the technical specifications for how the product worked. These specifications were created so that an engineering team could implement the design.


This spec, along with the designs, formed an integral part of the final deliverable to the sponsor.


The template for the input / output maps to communicate multi-modal interaction was created by me, and has proven useful for other teams to leverage in communicating their work.



Diagram showing how the mixed reality model works



Specification for how collaboration works within the experience



Guidelines for content within the Ubuntu platform



Input / output map of the listening portion of the experience



Input / output map of the exploring portion of the experience



Outcome





The project was presented in both academic and industry settings. The sponsor, consultants, and teaching team were thoroughly impressed with the quality, thoughtfulness, and implications of the work.



"I was particularly impressed with the pivot...this is the kind of product—and even program that could be incorporated in museums...classrooms around the world."



Project Sponsor, Joy Media



"I mean this is it. This is the one project I want to show next year to the students in terms of 'this is how you do it'. This is how you go from a broad concept into a very refined outcome. High production quality. This is the project I want to show."



Professor of Design, University of Washington



"[with] this project you've redefined what excellence can mean in a capstone project...[this project] exemplifies how we meaningfully and joyously expand access using equity, diversity, and inclusion as a real driver for meaningful, human-centered innovation."



MHCI+D Director



"the entire project is just exemplary...the way you sought feedback, and then also converted feedback into one plus one equals five."



Technical Consultant, ex-Microsoft



Presentations



SPonsor Presentation



Remote via Zoom
August 2024



The final presentation was a one hour session with Q&A, detailing the design process and resulting design response to the sponsor and teaching team.



DESIGN Use Build Seminar



University of Washington, Seattle
August 2024



DUB (Design Use Build) is an alliance of faculty, students, researchers, and industry partners interested in Human Computer Interaction & Design at the University of Washington.



Capstone Showcase



Town Hall, Seattle
August 2024



The capstone showcase in downtown Seattle presented the culminaing work of the master's program to industry leaders and the broader community.



MHCI+D Immersion Studio



University of Washington, Seattle
September 2024



The team was invited by the program's director to present the project to the incoming cohort during their first week.



Reflection





What went well



Through this project, I felt I was able to significantly strengthen my skills in design strategy and leadership.


I led the team in developing the workshop structure, pivoting the design concept, and defining the system and scope of the design response.


I was also able to leverage my existing design skills and experience to do heavy lifts in order to meet tight deadlines.





What could have been improved



Project management was a challenge the team faced during the project.


We created a Trello board midway through that helped us manage tasks, and I developed a Gantt chart to keep the team on track for the last month of the project.




What I learned



I learned how to guide and lead a team through an entire design project, from broad concept to refined outcome. My teammates were both pivoting into the field, and I had the most design experience.


This project also strengthened my ability to integrate academic research from multiple disciplines into my work.