Smart Microscope

A smart assistant for viewing digital slides under a microscope
CONTEXT
The DALI Lab
TEAM
3 Designers
3 Developers
3 Data devs & 1 PM
TIMELINE
10 weeks
SKILLS
User Research, UX Design, UI Design, Prototyping
DELIVERABLES
Mobile application prototype
Style guide
THE PROBLEM

Diagnosing & Using AI in Digital Pathology

THE OPPORTUNITY

Smart Microscope

A team at DALI Lab along with Dr. Sriharan at DHMC, developed a smart microscope, a tool to view digital slides under a microscope as if they were real. It uses space tracking on the fake slide and zoom technology to achieve the speed and precision of a real microscope.
Smart Assistant is the tablet interface that controls the microscope.
How might we help pathologists diagnose cases, order additional tests and use AI to aid diagnosis on the SmartScope?
THE SOLUTION

Tablet App: Smart Assistant

Diagnosing Cases

Each tray in traditional pathology contains slides from up to 20 cases. Smart Assistant allows you to work through each tray, highlightling progress on the current case as you work. It displays key information adobe the slides an additional details upon selecting each slide.

AI Assistant

Recommends AI models based on the type of cell in the slide. We display metrics about models in a simple way, explaining terms such as ROC curves and clearly informing doctors of the model’s performance and intended use cases.

Shelf View

Equips doctors with an overview of their trays and cases for the day, three categories of cases to tackle and allows for planning workload. Also emphasizes progress throughout the workday.

The Design Process

From user interviews to market research and our testing process, our ideas changed a lot through the course of this project! Here's how we created our product.
RESEARCH

User Interviews

I interviewed pathologists across DHMC about their work. I focused both on their experiences and challenges in traditional pathology, to see what could be improved in a digital version and on their experiences with digital slides and using AI.

Here are some quotes from my interviews that explain users' frustrations


RESEARCH

Competitive Research

I researched AI assistant technologies in the market and gained insight on which aspects were doing well and which were not through my interviews.

  • Option to switch between slides quickly on the left

  • Limited information and explanation about model, less beginner-friendly

  • Viewing results through a trackpad scroll and zoom is difficult


  • Important case details are highlighted on the left

  • Slide view is small and not intuitively placed in the middle right

  • Can track status of orders through the “orders” tab
DEFINE

Personas & Journey Maps

Based on our interviews we created a user persona of a pathologist new to digital pathology and AI, but excited for a better diagnostic experience.



We then created a journey map to understand the pain points and opportunities for delight in the user’s journey

DEFINE

POVs & Needs

We then defined some needs for our user group based on their journey.

Pathologists using the Smart Microscope need to
IDEATE

Sketching

After our first whiteboard brainstorm and seeing different directions we could go in, we realised that part of what could make travel experiences more authentic was taking people back to their childhood and special, deep memories, contained in scrapbooks.

These could help spotlight some of the deeper connection with places and their culture that tourism loses. We could try to encourage people to share their culture of their hometown and explore other ones through real experiences of people local to the place.

With this idea, we made some crazy 8 sketches to brainstorm features that could be useful and delightful in such an app.


PROTOTYPING

Low Fidelity Grayscales

I wanted to deeply understand the information pathologists look at first while viewing slides. To this end, I created grayscale to user test. We prototyped ways of navigating to the AI assistant, presenting information about models and tracking case diagnosis.



Based on the feedback highlighted above, I iterated and refined to make further prototypes.
PROTOTYPING & TESTING

High Fidelity Prototypes

Your Shelf Display

The first thing each pathologist sees every morning is a list of current cases for the day. We separate the trays into three categories. Each category has different sets of important information.
Based on interviews, I designed for a sense of progress. A doctor gets a visual sense of their work throughout the day and is equipped with the information to plan out their workload.




Diagnosing Cases

A key insight from my interviews was that pathologists have to keep track of many cases with multiple parts and slides. An easy error is signing off a case without looking at all slides and having to backtrack.

Pathologists loved visually seeing how much more is available before making a decision on a case– something which is a pain point with physical trays and slides.



AI Assistant

I focused on making this flow feel accessible and informative to pathologists new to AI.
Categories are automatically recommended based on the cell type on the slide. The catalog focuses on the most important information, an image of the type of result, a description and developer. An information button is available to provide detailed metrics without cluttering the window for repeat users.



Ordering Stains and Levels

A common frustration, especially at Dartmouth Hitchcock Medical Center, is that ordering additional tests on a slide called “stains and levels” is tedious. It is usually 8 clicks and a combination of many keyboard shortcuts on a computer.

I aimed to simplify this via a 3 step ordering process: category → test name → order


TESTING

Feedback & Insights

1. A Training or Onboarding Flow

Talking to first time users at Technigala (our termly tech showcase), I noticed a need for onboarding for first time users. Physical aspects of the scope, such as pressing the hardware button on the microscope to activate the AI mode, require some explanation for new users. Integrating hardware tutorials into the app is a next step.

2. User-Friendly Interface

Users who tried it described it as “the most user-friendly thing out there for pathologists.” They found the switching of slides through a case, the presentation of the AI options and especially being able to order a test with 3 clicks intuitive and easy to use.

ITERATION

Takeaways and Next Steps

While this project was challenging, I enjoyed the process of researching, designing and testing an app in 10 weeks.
These designs are slated to be implemented in the Spring of 2024 and tested further in studies at DHMC to test the effectiveness of the smart microscope.
Here are some of my takeaways from the project:

Understanding foreign terms

I dived deep, learning about medical terms that once seemed unfamiliar. Asking questions about these helped me understand what they meant to doctors and what information they were actually gaining from different words and screens.

Considering all stakeholders

I learnt to consider not only pathologists, but also hospital technicians, researchers & the developers of AI models and what they wanted to showcase. Balancing these needs is a valuable skill I learnt.

Designing for simplicity

Doctors’ chief complaint about current medical interfaces was complexity which slowed them down. With this in mind, I tried to keep any action to within 3 clicks, optimizing for speed. Many doctors have also low familiarity with technology so I supplemented cues with words.
I’d like to thank our partner, Dr. Sriharan, my design mentor, Daniel Lubliner, Professor. Lorie and the entire Smart Microscope team for making this wonderful project come to life.