JAMIE FONG


Enabling
No-Code Data
Exploration and Visualization



Marklogic



2022



my role





UX Designer



Timeline





4 months
(simultaneous with other projects)



the team





1 Designer
1 Product Manager
3 Developers



shipped





Spring 2022



Overview





As the sole designer on a cross-functional team, I took ownership over the design of a critical portion of Data Hub Central, Marklogic's flagship product offering, to enable users to visualize and filter their data through a no-code interface.


In the current state, users had no way to visualize the vast, complex data in their database, and there was no way to query on the data without writing code. Users could only explore their datasets one entity type at a time in a table format — but entity types were related to each other and this type of information was important to surface and visualize.


Users needed a no-code way to run more complex queries and view relationships to get more value from their data. Having a visualization also gives critical feedback from curation steps run on the data.



Project Goals



Democratize data access



Enable non-technical users to explore and gain insights from complex data sets without relying on data analysts or developers, thereby democratizing data access and analysis across their organizations.



Ensure scalability and performance



Design a solution that can handle large, multi-entity data sets efficiently, providing seamless performance and scalability as the data volume and complexity grows.



Accelerate time to insight



Streamline the process of data exploration and visualization, allowing users to quickly identify relationships, and patterns in their data, leading to faster and more informed decision-making.



assumptions



The following key factors guided my decision-making and approach as a designer



1.



Solutions engineers would act as a sufficient proxy for the voice of the user with resource constraints. Solutions engineers talked with users everyday and understood their pain points.



2.



It was assumed that the data sets people wanted to view were already available and accessible through the ingestion and curation processes needed prior to viewing the data.



3.



It was assumed that the target users had a basic understanding of data concepts, such as entities and the relationships between them. While the project aims to minimize technical barriers, some level of data literacy was expected from the users



Understanding the PROBLEM SPACE





Before starting on designs, I had discussions with the product manager to understand why users needed this functionality, and what they were trying to achieve with it. I also made sure I understood which personas would be interacting with the interface, and how they would each use it differently.


The organization had relatively low UX maturity, and our design team was small and resource constrained. Due to this, we were not able to talk to users directly prior to the design work. To understand the user, I set up meetings to talk with solutions engineers who could act as a proxy for users. Solutions engineers worked with customers everyday to create customized solutions and understood their pain points and goals well.



Personas



Through discussions with product management and solution engineers, I identified two primary personas who I would be designing for with the graph exploration



Emilia



Business Analyst



Emilia the business analyst wants to understand the data landscape to support business decisions and identify process improvements.



Stewart



Data Steward



Stewart the data steward wants to verify that his curation steps executed correctly by looking at the data after the steps are run.



FEASibility Analysis:
Selecting a data viz library



The project would be a large engineering lift and resources were limited. A graphical library had to be identified on which the graph and interaction patterns could be built off of.


I took the first step in researching and vetting capabilities for potential data visualization libraries that could support the types of interactions needed and consulted closely with development to discuss feasibility throughout the project.


The data visualization library had to meet the following criteria:

  1. Be open source and free due to budget constraints
  2. Handle very large datasets
  3. Support highly customizable nodes to display icons and text simultaneously
  4. Support interactive nodes to allow drill down into the data

We ultimately landed on Vis.js as it met all the criteria above.



D3



D3 was a very powerful library with many of the capabilities we were looking for, however it didn't support the level of interactivity needed at the node level to drill down into the data.



Vis.js



Vis.js allowed customization on nodes and edges, as well as interactivity to be able to drill down further into the data.



Cytoscape



Cytoscape did not enable enough customization on nodes for our purposes. We needed to be able to put icons and text within the same node.



Gephi



Gephi was not designed to handle the size of datasets that would be represented.



Low Fidelity Design





I started the design process by looking at the designs I had done for a previous project I led on modeling entities as a reference for consistency. I created low fidelity wireframes using LucidChart, moving later to Figma.



I set up feedback sessions with solutions engineers and developers on a regular cadence to make sure that the designs were addressing customer needs and the plan to build them out would be feasible.



Early landing page for the graphical view. Initially, users would enter a query before seeing the graph.



Early ideation for the graphical view.



Feedback and Iterations





I set up feedback sessions with solutions engineers, product managers, and developers on a regular cadence to make sure that the designs were addressing customer needs and the plan to build them out would be feasible.


I solicited feedback from early concepts all the way into the final iteration. I walked my team through storyboards and incorporated their feedback into the final design.


In total I had 13 feedback sessions for the design.


Most critical feedback included:

  • Users wanted to see the graph before querying on it instead of having a zero state prompting a query up front
  • There is too much information on each node, especially when there are a lot of nodes on the graph
  • Users will want to filter at the property level and not just at the entity level


Design CHALLENGE:
FACETING ON MULTIPLE ENTITY TYPES





The Design Problem



The user needed to be able to search and see results for multiple entity types simultaneously. I had also gotten feedback that users wanted to see the data before querying on it.


This posed a challenge both in moving the initial filtering step to a later point in the user flow, and in designing a faceting system that could accommodate hundreds or possibly thousands of different facets. The current faceting system was designed only to accommodate the facets for a single entity type, and would be unusable if there were too many entity types selected.



How I addressed the Challenge



I realized a new faceting system would have to be designed and developed for this to work, and brought a rough design concept to the product manager.


I came up with a solution to have a multi-tiered faceting system that would allow the user to manage filters with multiple entity types. A universal search as well an as entity-specific search allowed the user more granular control over their query. The facets also doubled as a legend for the graph as there could be many different entity types displaying at a given time.


After discussions with the PM and developers, the new faceting system was added to the roadmap for release with the interactive graph.



Faceting functionality



Design CHALLENGE:
Large data sets





The Design Problem



The biggest challenge was deciding how to deal with showing results that could include millions of records. Previous research showed that some have attempted to solve this issue with pagination, but it presents a disorienting experience for the user as each page is displaying essentially a different graph and defeats the purpose of showing connectivity. We also did not want the graph to get so dense that the user could not make any sense of it. Additionally, there were concerns about performance abilities.



How I addressed the Challenge



To reduce clutter in the graph, I developed the concepts of 'base' and 'related' entities. This would allow clustering in groups by entity type on the graph to reduce clutter while still displaying relevant information.


The more records within a group, the larger the node would appear on the graph. These groups could be expanded if the user were interested in investigating further. Because the most likely use case was that a user was looking for a specific record, we did not need to show millions of records at a time. There would be too much information and the user would not be able to make sense of the graph.


Showing a meaningful sample of 1,000 of the most connected records would be sufficient for users who were just wanting to explore the dataset, and users who were looking for something specific would continue narrowing their search with the facets until they found what they were looking for.



Definition of how base entities and related entities behave.



Designing for phased development





In a previous project, we were not able to release the complete design due to the team being reduced to a skeleton crew partway through the project. From this experience, I learned to design 'phases', including an MVP for development to accommodate limited development resources. These phases helped the product manager prioritize what needed to be built first, while also providing developers with a phased approach to building a hefty new technical capability into the product.



Phase 1: Minimum Viable Product



The initial phase set up the foundation for the graph and addressed the most critical user needs for displaying search results, viewing relationships, and viewing further details of results.



Phase 2: Dynamic zoom



The second phase introduced dynamic zoom, where the amount of information on a node changed depending on the zoom level. This would give the user the appropriate information when they would need to see it.



Phase 3: Graph Settings



The third phase introduced the ability to manipulate what was shown on the graph, such as relationship labels or focusing on a specific node cluster to drill deeper.



Phase 4: Group Nodes and Dynamic Sizing



The fourth and final phase grouped nodes of the same type together and had dynamic sizing of the node based on how large the group was. This significantly reduced clutter in the cases of larger result sets.



Design SPECIFICATION +
Implementation Support





I documented a comprehensive technical specification, outlining many logic-dependent interactions with algorithmic behavior. I documented the design using a combination of visual figures accompanied by text explanations in a wiki page to facilitate easy communication with engineering.


I also contributed to writing acceptance criteria in Jira tickets and conducted design reviews. I was active in answering questions developers had regarding the design and collaborated closely with engineering if any modifications needed to be made due to unforeseen technical constraints. I attended daily standup meetings and participated in grooming, sprint planning, and sprint review meetings.



Spec for node interactions



Faceting functionality spec



Accessibility





I led the accessibility efforts for this product, ensuring adherence to WCAG AA standards. I focused on key areas including color contrast ratios and keyboard navigation, making the product usable for a diverse range of users with different abilities. This work was crucial in meeting government contractual obligations and expanding the product's reach.



Design specification for keyboard accessibility on filtering.



Final Iteration





After incorporating the feedback from the solutions engineers and developers, the final iteration was created in Figma. I made clickthrough storyboards to demonstrate the user flows of the most common use cases.


Below are some examples of the 50+ final screens.



Landing state



Example of zoomed out view



Facets applied



Base entity selected



Manage graph settings



Drill down into instance details



Outcome





As the organization was still developing its UX capabilities, quantitative metrics were not yet implemented. However, the design work laid a robust foundation for the team's future efforts. Notable outcomes include:

  • The MVP was shipped to customers in Spring 2022
  • I began the process to file a patent for the facet design
  • I was designated to lead future design work for projects working with complex, interactive data visualizations
  • I advocated for and got a product-wide accessibility epic on the roadmap
  • I formed close working relationships with the engineering team


Reflection





What went well



I worked very effectively with engineering on this project. Involving them early and constantly made implementation smooth.


There were a lot of complex problems to address in this project. Working on this project stretched my ability to work on data viz projects.



What could have been improved



I was not able to talk to real customers. Next time I would push harder for user research in this project and throughout the organization.



What I learned



I learned how to prioritize my designs for MVP and future phases.


I learned how to design for large data sets and a lot about how things worked in the back-end.