ЖУРНАЛ КАРДИОРЕСПИРАТОРНЫХ ИССЛЕДОВАНИЙ | JOURNAL OF CARDIORESPIRATORY RESEARCH
№4 | 2021
9
ОБЗОРНЫЕ СТАТЬИ/ ADABIYOTLAR SHARHI/ REVIEW ARTICLES
Дарья Хеммерлинг,
Университет науки и технологий, факультет
электротехники инженерии, автоматики, информатики
и биомедицинской инженерии, Краков, Польша
Бенедетта Синьорелли,
Кафедра наук о здоровье человека, факультет медицины
и хирургии, Университет Флоренции, Италия
Войцех Вояковски,
Отделение кардиологии и структурных болезней сердца,
Медицинский университет Силезии, Катовице, Польша
Михал Тендера,
Отделение кардиологии и структурных болезней сердца,
Медицинский университет Силезии, Катовице, Польша
Томаш Ядчик
Отделение кардиологии и структурных болезней сердца,
Медицинский университет Силезии, Катовице, Польша,
Международный центр клинических исследований,
Университетская клиника Святой Анны Брно, Брно, Чехия
ГОЛОСОВЫЕ ТЕХНОЛОГИИ ПРИ СЕРДЕЧНО-СОСУДИСТЫХ ЗАБОЛЕВАНИЯХ
For citation:
Daria Hemmerling, Benedetta Signorelli, Wojciech Wojakowski, Michał Tendera, Tomasz Jadczyk. Voice technology in
cardiovascular diseases.
Journal of cardiorespiratory research. 2021, vol. 2, issue 4, pp. 9-12
http://dx.doi.org/10.26739/2181-0974-2021-4-1
Ключевые слова:
голосовые технологии, вокальные биомаркеры, искусственный интеллект, голосовые боты, персонализированная
медицина
Daria Hemmerling
,
AGH University of Science and Technology,
Faculty of Electrical Engineering, Automatics,
Computer Science and Biomedical Engineering, Kraków, Poland
Benedetta Signorelli
,
Department of Human Health Science,
Faculty of Medicine and Surgery,
University of Florence, Italy
Wojciech Wojakowski
,
Division of Cardiology and Structural Heart Diseases,
Medical University of Silesia, Katowice, Poland
Michał Tendera,
Division of Cardiology and Structural Heart Diseases,
Medical University of Silesia, Katowice, Poland
Tomasz Jadczyk
Division of Cardiology and Structural Heart Diseases,
Medical University of Silesia, Katowice, Poland,
International Clinical Research Center,
St. Anne's University Hospital Brno, Brno, Czech Republic
ЖУРНАЛ КАРДИОРЕСПИРАТОРНЫХ ИССЛЕДОВАНИЙ | JOURNAL OF CARDIORESPIRATORY RESEARCH
№4 | 2021
10
VOICE TECHNOLOGY IN CARDIOVASCULAR DISEASES
Keywords
: voice technology, vocal biomarkers, artificial intelligence, voice chatbots, personalised medicine
Daria Hemmerling,
Fan va Texnologiya Universiteti,
Elektrotexnika, аvtomatika kompyuter fanlari va
biotibbiyot muhandisligi fakulteti, Krakov, Polsha
Benedetta Signorelli,
Inson salomatligi fanlari bo'limi,
Tibbiyot va jarrohlik fakulteti,
Florensiya universiteti, Italiya
Voytsex Vojakovski,
Kardiologiya va yurakning strukturaviy kasalliklari bo'limi,
Sileziya tibbiyot universiteti, Katovitse, Polsha
Mixal Tendera,
Kardiologiya va yurakning strukturaviy kasalliklari bo'limi,
Sileziya tibbiyot universiteti,Katovitse, Polsha
Tomaş Jadchik
Kardiologiya va yurakning strukturaviy kasalliklari bo'limi,
Sileziya tibbiyot universiteti,Katovitse, Polsha,
Xalqaro klinik tadqiqotlar markazi,
Brno Sent-Anna universiteti kasalxonasi, Brno, Chexiya
YURAK QON TOMIR KASALLIKLARIDA OVOZ TEXNOLOGIYASI
Kalit so'zlar:
ovoz texnologiyasi, vokal biomarkerlar, sun'iy intellekt, ovozli chatbotlar, moslashtirilgan tibbiyot
Introduction
Despite great efforts, cardiovascular diseases (CVD) remain the
leading cause of death worldwide [1]. Thus, novel diagnostic and
treatment solutions are highly demanded to address current challenges
in the field of clinical medicine. Interestingly, recent studies indicate a
potential use of voice technology which covers a wide spectrum of
artificial intelligence (AI) techniques allowing for human language
understanding as well as for predictive analysis of vocal biomarkers.
Physiologically, voice is the sound produced with the usage of the lungs
and the
vocal folds
in the
larynx
. The vibration of vocal folds is
generated when the air is pushed through vocal folds with sufficient
pressure. On the one hand, the spoken language is the easiest and fastest
way of communication. On the other hand, generation of voice requires
using a series of coordinated, complex movements in the head, neck,
chest and abdomen muscles, which impact the signals’ frequency and
amplitude resulting in specific, decodable sounds. By its complex
nature, voice is an unique bio-print characteristic for each person
conveying information about individual's personality, mood and health
status. From a diagnostic point of view, voice is a bio-signal that can be
acquired non-invasively and in an easy, economically-sound manner [
1
,
2
]. Subsequently, a correlation between CVD and alterations in speech
characteristics open new diagnostic opportunities based on deviations
of voice features associated with CVD-mediated systemic inflammatory
process which impacts anatomical structures responsible for voice
generation [2]. Despite the very well-developed digital technology there
is still the challenge to extract specific, important information about the
patient's health condition. Especially due to its complex and dynamic
characteristic, voice can be pronounced in different intonations and with
different emotions. AI-driven digital solutions are still being sought on
how to non-invasively evaluate patient's voice organs and effectively
distinguish between patients with existing disorders and healthy
individuals.
Moreover, the advancements in the field of computer science
leveraged application of human-computer voice interfaces (also called
voice assistants, voice chatbots or conversational agents) allowing
machines to understand spoken language and generate human-like
voice. The aforementioned implementation of voice technology in
clinical field provides interesting tools which useability is currently
being evaluated and tested [
3]
.
This article covers the application of AI-based voice chatbots and
the potential application of vocal biomarkers in the field of
cardiovascular medicine.
Artificial intelligence-driven voice technology in medicine
Definition of voice assistant
Voice assistants (VA) powered with the advanced algorithm of AI
and natural language processing allow for verbal communication
between humans and computers. These conversational agents (i.e.,
Amazon Alexa, Apple’s Siri or Google Assistant) can be installed on
standalone devices called smart speakers or deployed on smartphones.
The emulated human-machine conversations are based on the
application of neural networks which perform voice-to-text analysis and
text-to-voice computation generating natural human voice transforming
day-to-day clinical practice [3, 4].
Application of voice technology in clinical practice
Voice-enabled technologies have the potential to influence everyday
cardiovascular medicine practice by:
(1) Foreign language interpretation and real-time language
translation,
(2) Patient education,
(3) Medication reminders and prescription refills,
(4) Continuity of care,
(5) Automated and paperless collection of medical data,
(6) Remote long-term monitoring,
(7) Diagnostic value of vocal biomarkers.
Foreign language interpretation
There is an incremental need to address language barriers for
patients whose health care workers do not speak their primary language.
Voice technology provides tools that facilitate communication in a safe
and effective manner. Panayiotou et al. reviewed digital language
translation solutions in health care settings. Among 15 iPad-compatible
applications including 8 voice-to-voice and voice-to-text translation
apps, 2 services (Assist and Talk to Me) were found to be clinically
adequate for everyday conversations on subject matters that do not
require a professional interpreter [5].
Patient education
There are numerous potential applications for the use of VA in the
field of patient education and guidelines. Specifically, this Alexa-based
applications can be used to provide information on the cardio-
pulmonary resuscitation (i.e., The Mayo Clinic First Aid) [6] or
information from Mayo Clinic experts on topics related to
cardiovascular diseases providing an access to the verified medical
knowledge [7]. Furthermore, the Answers by Cigna application
available on Amazon Alexa provides health coach programs supporting
treatment plans. Furthermore, users can ask a wide range of health-
ЖУРНАЛ КАРДИОРЕСПИРАТОРНЫХ ИССЛЕДОВАНИЙ | JOURNAL OF CARDIORESPIRATORY RESEARCH
№4 | 2021
11
related questions receiving easy-to-understand responses [8]. A similar
approach is exemplified by the Orbita ENGAGE designed for patients
who can communicate verbally with a VA for medically associated
frequently asked questions, especially based on symptom screening [9].
Medication reminders
Among CVD patients, medication non-adherence is a perceptible
challenge both in the period immediately following an acute
cardiovascular event as well as during long-term follow-up [10]. VA
have been successfully implemented to support pharmacotherapy
management. By saying to Alexa "Manage my medication" or “Refill
my prescription”, registered patients can set reminders to take
medication and request prescriptions with home drug delivery through
the Giant Eagle Pharmacy voice application [11].
Continuity of care
In the broader spectrum, some VA solutions like Orbit Connect are
engineered for long-term follow-up as well as for pre- and post-visit
through digital coaching, assessments, and care team communication.
Furthermore, a holistic approach to patients with CVD should include
mental status evaluation. Importantly, the prevalence of depression in
this group is 3-fold higher in comparison to the general population [12].
Correspondingly, the Talk space voice application for Amazon Alexa
allows users to access depression assessments tools as well as guided
mindfulness techniques [13].
Automated and paperless collection of medical data
Integration of the medical voice AI chatbots with hospital electronic
health systems (EHR) leverages advances in voice technology allowing
for seamless and automatic population of electronic forms [14].
Noteworthy, it is crucial to ensure adequate level of security and privacy
during transmission and computation of patient's protected health
information (PHI). Accordingly, the GDPR (EU) and HIPAA (USA)
regulations must be implemented for each software solution dealing
with PHI.
Practical application of voice chatbot in clinical settings was
exemplified by the CardioCube
®
service deployed on Amazon Echo
smart speaker for automatic collection of patient-reported medical
history at the Cardiology Outpatient Clinic of the Cedars-Sinai Medical
Center (Los Angeles, CA, USA) [14]. Initialization of CardioCube
®
voice assistant was evoked by a verbal command “Computer, open
CardioCube”. Furthermore, patients answered pre-defined clinical
questions which corresponded to the hospital intake form i.e. “Do you
have high blood pressure?”, “Have you ever had a heart attack?”, “Have
you been diagnosed with diabetes?”. The answers provided verbally
were translated into text using cloud-based AI systems and
automatically populated a patient's record in the hospital EHR system.
Healthcare providers could access the complete report through a
standard web-based interface. This interactive approach was shown to
streamline repetitive and time-consuming tasks during patient
registration providing a secure and high accuracy (97.5%) digital tool
automatically generating medical reports.
Remote long-term monitoring
The FCNcare by CardioCube
®
solution was implemented at the
Family Care Network (Bellingham, WA, USA) for remote long-term
follow-up of patients with diabetes and heart failure [15]. Individuals
enrolled in the pilot study received Amazon Echo-deployed
CardioCube
®
software for home use based on reporting actual clinical
status during scheduled conversation sessions between patient and
CardioCube
®
. The voice-based questionnaire consisted of eight
questions: (1) “In the past week, have you missed any dose of your
medication?”, (2) “Are you needing a medication refill?”, (3) Do you
have any medication-related questions that you need your care team to
answer?”, (4) A caring reminder, eating more carbohydrates increases
your blood sugar. All sugary foods contain carbohydrates, as do bread,
rice, pasta, and potatoes. Have you been carefully managing your
carbohydrate intake in the past week?”, (5) “And how about exercises,
how many times in the past week have you exercised?”, (6) “As for this
past week, were you able to check your sugar levels with a
glucometer?”, (7) “And how many times in the past week did you check
your blood sugar level?”, (8) Were the majority of your readings in a
good range?”. Obtained results were analysed and automatically
transferred to the Family Care Network EHR system for review by the
nurse. Importantly, in case of health status deterioration (i.e. patient
reports dyspnea) red-flagging notifications were implemented to
improve useability of the service giving healthcare providers a quick
access to the most crucial reports.
Diagnostic value of vocal biomarkers
In the literature, there are only a few studies that analysed the voice
and speech signals in an acoustic parametrized manner for heart
diseases. The researchers from Mayo Clinic reported a possible
relationship between specific vocal biomarkers and coronary artery
disease (CAD) underscoring the potential use of this simple biomarker
to identify patients at risk [2]. The authors have analysed if patient voice
signal characteristics are associated with the presence of CAD. They
performed detailed acoustic analysis to describe the overall shape of
signal's spectral envelope. With further analyses, authors identified five-
voice features that were associated with CAD. Combining data with the
Atherosclerotic Cardiovascular Disease risk scores, it was possible to
identified two voice features that were independently associated with
CAD (odds ratio OR = 0.37; 95% CI, interquartile range IQR = 0.18-
0.79; and OR = 4.01; 95% CI, IQR = 1.25-12.84; p=.009 and p=.02,
respectively). Both features were more strongly associated with CAD
when patients were asked to describe an emotionally significant
experience. The work was further developed and described by Maor et
al. [16], where the authors have analysed if the vocal biomarker is
associated with hospitalization and mortality among patients with
congestive heart failure (CHF). By extracting a total of 223 acoustic
features for each patient, the main novel finding of this study was that
non-invasive voice signal characteristics are associated with adverse
clinical outcomes among patients with symptomatic CHF [16].
Moreover, Pareek et al. [17] have also evaluated CAD patients. The
results revealed significant variations in spectrograms and specific voice
analyses between active and control group including jitter, shimmer, and
complex parameters such as Relative Average Perturbation being as a
quantitative measure of the voice.
Extraction of acoustic parameters enables an objective assessment
of the voice and speech quality. The registration of the signal might be
done in various manners. New technologies in digital signals processing
enable the recordings without the requirement for access to an anechoic
chamber. The sessions might be done at the doctor’s office, at home,
with a relatively low level of noise. Most smart speakers and VA-
deployed on smartphones have a circular microphone array to provide
voice-only interaction from a distance in standard room conditions. To
perform the analysis with desired goals such as automatic diagnostic or
highlighting health impairments using voice it must be stated what
should be recorded. Voice signals might be recorded in different
manners, depending on what features are desired. The phonation of
sustained vowels with continuous phonation over a certain time are
helpful to find discontinuities in signal's amplitudes and frequencies as
well as changes in loudness levels. The speech recordings bring more
information about the speech speed, pauses length, pitch and loudness
changes. Accordingly, the speech might be acquired from a text read, a
story-tell, a question-answer scenario, repetition of specific syllables
and conglomerate of words. This enables the semantic voice analysis
and extraction of meaningful words, enabling syntax analysis for natural
language processing.
Future directions
The aforementioned use cases confirm the feasibility of using
voice chatbots and vocal biomarker application in the field of
cardiovascular medicine. Noteworthy, VA can be integrated with the
existing
healthcare
ecosystems
leveraging
clinical
adoption
opportunities of voice technology. The further development will enable
constant patient monitoring with an immediate warning in case system
detects health status deterioration including analysis of “invisible” vocal
biomarkers. Such approach might be useful in predicting risk of the
occurrence of health- and life-threatening conditions.
ЖУРНАЛ КАРДИОРЕСПИРАТОРНЫХ ИССЛЕДОВАНИЙ | JOURNAL OF CARDIORESPIRATORY RESEARCH
№4 | 2021
12
Список литературы/ References/ Iqtiboslar
1.
Kones R, Rumana U. Cardiometabolic diseases of civilization: history and maturation of an evolving global threat. An update and call to
action. Annals of Medicine. 2017;49(3):260-74. Epub 2016/12/13.
2.
Maor E, Sara JD, Orbelo DM, Lerman LO, Levanon Y, Lerman A. Voice Signal Characteristics Are Independently Associated With Coronary
Artery Disease. Mayo Clinic Proceedings. 2018;93(7):840-7. Epub 2018/04/17.
3.
Hoy MB. Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants. Medical Reference Services Quarterly. 2018;37(1):81-8.
Epub 2018/01/13.
4.
Suta P, Lan X, Wu B, Mongkolnam P, Chan J. An overview of machine learning in chatbots. International Journal of Mechanical Engineering
and Robotics Research. 2020;9(4):502-10.
5.
Panayiotou A, Gardner A, Williams S, Zucchi E, Mascitti-Meuter M, Goh AM, et al. Language Translation Apps in Health Care Settings:
Expert Opinion. JMIR mHealth and uHealth. 2019;7(4):e11316. Epub 2019/04/10.
6.
Mayo Clinic First Aid. Amazon. [21 NOV 2021]; Available from: https://www.amazon.com/mayo-clinic-first-aid/dp/b0744ljcv2.
7.
Mayo Clinic News Network. Amazon. [21 NOV 2021]; Available from: https://www.amazon.com/mayo-clinic-news-
network/dp/b0711hkbb6
8.
Answers by Cigna. Amazon. [cited 21 NOV 2021]; Available from: https://www.amazon.com/Cigna-Answers-by/dp/B079SMKCZB
9.
Employee Health Manager. Orbita. . [21 NOV 2021]; Available from: https://orbita.ai/orbita-employee-health-manager/
10.
Simon ST, Kini V, Levy AE, Ho PM. Medication adherence in cardiovascular medicine. British Medical Journal. 2021;374:n1493. Epub
2021/08/13.
11.
New ways to manage your medications at home using Alexa. [21 NOV 2021]; Available from:
https://www.aboutamazon.com/news/devices/new-ways-to-manage-your-medications-at-home-using-alexa.
12.
Chaddha A, Robinson EA, Kline-Rogers E, Alexandris-Souphis T, Rubenfire M. Mental Health and Cardiovascular Disease. The American
Journal of Medicine. 2016;129(11):1145-8. Epub 2016/10/25.
13.
Talkspace: Online therapy for all. Amazon. [21 NOV 2021]; Available from: https://www.amazon.com/Talkspace-Online-therapy-for-
all/dp/B085VHRW33.
14.
Jadczyk T, Kiwic O, Khandwalla RM, Grabowski K, Rudawski S, Magaczewski P, et al. Feasibility of a voice-enabled automated platform
for medical data collection: CardioCube. International Journal of Medical Informatics. 2019;129:388-93. Epub 2019/08/25.
15.
FCNcare by CardioCube. [21 NOV 2021]; Available from:
https://jmir.org/api/download?alt_name=jmir_v23i5e22959_app1.mov&filename=62989d3b55a0fbfb33d8e0aac8c1a6bd.mov.
16.
Maor E, Perry D, Mevorach D, Taiblum N, Luz Y, Mazin I, et al. Vocal Biomarker Is Associated With Hospitalization and Mortality Among
Heart Failure Patients. Journal of the American Heart Association. 2020;9(7):e013359. Epub 2020/04/03.
17.
Pareek V, Sharma R, editors. Coronary heart disease detection from voice analysis. 2016 IEEE Students’ Conference on Electrical:
Electronics and Computer Science (SCEECS); 2016.
