Deep Learning of Respiratory Sounds Gathered By Wearable Sensors For the Detection of Respiratory Disease
OVERVIEW
24-hour Global Health Hackathon
On March 30, 2019, School of AI launched a 24-hour Global Health Hackathon, sponsored by Accenture, in over 20 cities. The hackathon challenged participants to come up with AI solutions for some of the world’s biggest health care issues. Our team won 3rd place in San Francisco for our proof of concept.
HACKATHON OBJECTIVE
How did we decide on our product concept?
Pick a dataset
Teams were self-organized. We were asked to choose from 1 of 4 datasets*:
NIH chest x-rays
Respiratory rate algorithms
Skin cancer photos
Use AI to help women predict an upcoming period/fertile window (Pslove challenge).
*Teams could also come up with their own health-related AI solution.
After choosing a dataset, teams needed to define their own business problem; create a working demo; and give a 4-minute presentation to pitch the idea.
JUDGING
Judging panel + criteria
Jonathan Lee - Analytics & AI Executive, Accenture
Carson Bentley - School of AI, Dean
Sergii Garkusha - School of AI, Dean
And 3 other Accenture stakeholders
Criteria
Judging criteria was based on scoring in 3 areas:
Innovation
Did the team come up with a new approach to an existing problem?
Did the team make creative use of available resources?
Did the team propose a new challenge in health care and create a solution?
Technical achievement
Did the team complete the hack; does it work; can it demonstrate seamlessly?
While the project doesn’t have to be production-ready, does the idea come to life?
Is the code scalable, maintainable, readable, etc.?
Application to health care
Did the team understand the problem?
How does the product relate to health care?
Is this a product that people want/need?
Is this a product that the medical profession can benefit from?
TEAM & DURATION
Who are we?
Team Ikigai
Engineering: Lei Pan - built the training model for our concept product
Product Design: Natalie Kay - pitch deck, gave 4-min presentation to judging panel & participant audience
Business: John Montoya - Business analysis, academic research
Team Ikigai worked out the details of our business objective and use case together within a 24-hour time frame.
TOOLS & METHODS
Team Slack channel
Market analysis
Defined use case/business objective
Design thinking: hypothesis, problem statement, solution to test
Github
Heroku
BACKGROUND ON RESPIRATORY DISEASES
Impact of respiratory diseases on a global scale
Chronic obstructive pulmonary disease (COPD) is a silent killer in low to middle-income families. According to the World Health Organization, 65 million people (roughly the size of France’s population) suffer from chronic obstructive pulmonary disease (COPD). COPD includes diseases such as asthma, emphysema and chronic bronchitis. In 15 years, it is expected to become the leading cause of death worldwide.
About 334 million people (14% of these being children, globally) suffer from irreversible, chronic asthma. This is roughly the size of the US population.
100 million people suffer from sleep apnea, roughly the size of Vietnam’s population.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5921960/
MARKET ANALYSIS
Similar models have been built with high accuracy
Cardiogram (a health startup) recently partnered with UCSF to demonstrate that the Apple Watch can detect certain serious conditions with high accuracy. For example, they were able to detect arrhythmia with 97% accuracy; hypertension with 82% accuracy; and sleep apnea with 90% accuracy.
HYPOTHESIS
Expand the use of wearable sensors to predict other diseases
If wearables can already detect sleep apnea (a serious respiratory condition) with 90% accuracy, why not expand its use to detect other respiratory diseases with data from breathing sounds?
PROBLEM STATEMENT
Doctor visits are costly and time-consuming
Assumption:
Patients currently have to get evaluated by a doctor at an office visit when experiencing any respiratory symptoms which can be costly, inconvenient and time-consuming.
Pain points:
Going to the doctor’s office costs money and takes a lot of time. Sometimes, it means the diagnosis is a little late because the symptoms have become more severe, especially for noncommunicable diseases (i.e. lung cancer or COPD).
Also, longer term monitoring is usually required to gather significant data on breathing patterns and irregularities. In the case of sleep apnea, the patient needs to be asleep, so monitoring would need to happen at night over several hours.
SOLUTION OVERVIEW
Self-monitoring wearables
It’s easy, affordable and time-saving to use either a mobile app with a connected breathing monitor or wearable device to collect breathing sounds to detect whether there is wheezing, crackling or both which would warrant a doctor’s visit.
PROCESS DESCRIPTION
Move fast and be decisive
Given that this was a 24-hour hackathon, we had to think and move very quickly to complete our project. This did not follow the usual course of design thinking, as I did not get a chance to do any user research or testing. My time was spent working backward from the dataset to develop a purposeful user story and user need that is better served by more accurate data.
Why did we choose a respiratory sound dataset vs. respiratory rate algorithms?
Currently, serious respiratory conditions are diagnosed by well-trained doctors using highly sensitive electronic stethoscopes to detect sounds of wheezes, crackles or both in the lungs. Wearable sensors would be able to conveniently gather respiratory sound data without being intrusive on a patient’s lifestyle. Also, having sound wave data translated into some sort of heat map data visualization could be a much more accurate way to detect early indicators of disease that might otherwise be inaudible.
DESIGN DELIVERABLES + OVERVIEW OF DELIVERED SOLUTION
Training model with proof of high accuracy and a purposeful story
Training model:
We only had enough time to run 1 training iteration because each iteration took 1.5 hours. According to a similar kaggle case study, 10-15 iterations were run with up to 70%-74% accuracy.
Pitch deck:
We gave a 4-minute presentation followed by 2 minutes of Q&A to a panel of Accenture AI executives, School of AI deans and other stakeholders.
RESULTS & REFLECTIONS
Next steps
Given more time, we would have liked to run 10-15 iterations to improve accuracy of results, targeting 70%-74% accuracy*. We would also need more datasets to improve accuracy which can be achieved through the use of wearable sensors.
Each iteration took 1.5 hours to run, so we only had enough time to run 1 iteration for this 24-hour hackathon. Accuracy was only at ~50% but enough to make our proof of concept a compelling one. Respiratory sound datasets are very hard to work with!
Our engineer had trouble deploying the demo on Heroku, so unfortunately it’s hosted on a local server. We probably should have used AWS cloud as suggested by Accenture.
*https://www.kaggle.com/eatmygoose/cnn-detection-of-wheezes-and-crackles/output
MVP Roadmap:
Phase 1: Capture as much data on breathing sounds as possible through the use of wearable sensors.
Phase 2: Start to get more accuracy from sound data → detection of wheezes, crackling or both
Phase 3: With high accuracy comes prediction of possible serious respiratory conditions. At this point, a mobile app synced to the wearable sensor will be able to notify the patient about seeing a doctor.