false
Catalog
Modeling Mood and Emotional Patterns from Speech i ...
Presentation Q&A
Presentation Q&A
Back to course
[Please upgrade your browser to play this video content]
Video Transcription
So, I'd like to move into some questions now. So, one person wrote in and said, thank you very much for a good presentation, which I would have to agree with that. So, I think one question that I'm noticing is a lot of people are asking, how far are we from using something like this in a usual care clinic? Thank you very much for that question. And that is a question we get from ever so many people. Let me just tell you the challenges that we're faced with and the strategies that we need to go forward with. And so, speech is a very complex thing. Mobile devices are equally complex. There are a series of security challenges at the level of these devices. And so, in order for this to be deployed efficiently in the clinics, it really needs to be crystallized down into features that can be processed on the device itself, is my perspective. So, currently, what we are doing is that we are gathering the data from the mobile device. And it's encrypted very carefully on the device in an asynchronous manner. So, it's a different key that encrypts the data and sends it up to the server where it's decrypted with another key. So, we're in a constant kind of cat and mouse game with the device manufacturers to be able to make this work. So, in the way that we're analyzing our data now, it is really not feasible to make it deployable in a larger scale. So, it has to be on the device. And so, we're actually working on that right now. And it's very difficult to say exactly when it will be available, but I'm hoping that with more resources that we can, and the emerging technology, there's all kinds of technological things that are advancing on mobile devices and how things can be processed in a quick way. So, it would be probably difficult for me to say exactly when, but I'm hoping within a couple of years. Terrific. Another person said, wonderful presentation about fascinating research. To clarify, were the moments identified for intervention found during both moments of mania as well as depression? So, on the slide that I showed, it was really the manic, actually no, it was both mania and depression. Yeah. So, I know the individual that I showed, I know that data well, and that was a depressive state that was identified there, but it worked for both mania and depression. But the important thing to appreciate is it doesn't work for everybody. So, like everything in medicine, if anybody tells you that this intervention, this test or whatever is going to be 100%, and particularly in psychiatric illnesses, then that is almost by definition cannot be true, I think. So, it's probably not going to work for every individual. You see a point in the future where we would provide some of this feedback collected to the patients themselves? Absolutely. And so, this is one of the things that we're very interested in doing is that, and this goes back to one of the phrases that I've heard both in NAMI and PCORI and a number of different advocacy groups, nothing about us without us. So, then, because the question is, if it's processed in real time, how can we give information back to the individual so that they can see how the day has been going, and so they can look back at the week, the month, or the past quarter of time and say, you know, gosh, this is how my emotions and my moods have varied over this past month. And they could then have a setup such that this could be shared with their care providers, and this information could be provided to their care providers. And so, there would be objective evidence that could be used in the assessment and strategy and planning for the care. Another person wrote in and said, of course, there are many other things that our smartphones collect in addition to voice. Would you, do you believe that including those could improve your accuracy in predicting need for intervention? Absolutely. And we are, we're working in that dimension as well. One of the things that characterizes our group is that we are focused on speech. And so, we want to be able to identify the elements and characteristics of speech that are contributing to the bigger picture. And so, our question is really focused on, what are the fundamental features of speech that could help in the prediction? And then, the next stage of this is to then integrate that with all sorts of other things, which would include context as, you know, where the individual is, where the call was made from. Are there calls that the individual, or are there numbers that the individual is calling, you know? And so, it's different when you call for a pizza and they'll have a pepperoni and a Coke versus calling a confidant and saying, you know, I really had a bad day today, you know, can I, well, I'm not sure what I should do. So, there are ever so many complex things, you know, around the data that the phone is collecting. So, in order to really start to make sense of it, I think we need to at least parse out each item before we start to combine them. Wonderful. Someone said, seems data supports this work in identifying mood changes. Has it been tested in actual individuals to intervene before a crisis happens? We've not tested it in actual individuals to intervene. That is our next, you know, clinical trial. We first wanted to determine if we could do this on existing data, because there are so many ethical and, you know, ethical implications, you know, for such a trial. And so, that clinical trial needs to be carefully considered, and we are in the planning stages of that and are excited to move forward with that, but we need to be very careful and deliberate in how this study is designed. Do you envision this kind of, let's say, monitoring being used by patients who consent to have their calls monitored by a computer, which would then notify them and their maybe support network in moments of needed intervention? I think one of the tenets of medical care is the, you know, the engaged consent of the individual to, you know, to receive care. I mean, the fundamentals of providing care is that it's a collaboration between the individual that bears the burden of the illness and then the individual providing the care. And so, the agreement, the consensual therapeutic alliance agreement there, it is a consensual agreement. In psychiatry, we don't do like the surgeons do, we have to sign a consent form before you go into an operation, but there is an implicit consent, you know, when we see patients. And so, with this type of a monitoring system, I do foresee that, at least in the shorter term in the initial stages, it would be very wise for, you know, for a contract, if you will, or some kind of a, you know, a noted agreement for the individual to agree to this. Now, our goal is to have this processed on the device. And so, I'm thinking about an image as to how we would describe this. And the image that I'm thinking about is, you can imagine, you know, information speech going into a device that's affiliated with a fan. And so, the device processes it and drives a needle as to what the severity symptoms or things are, and then the fan just blows out dust. So, my image is that we would not retain data, but rather process it and then effectively delete it, you know, immediately, but just keep the parameters that would indicate the, you know, the metrics that we've been talking about now. So, if we're able to do that, then perhaps then the consent is not so much necessary, but we're still working in a research environment where informed consent is a basic element of the work that we do. Right. I'm just thinking myself about it and thinking about, you know, after hours and weekends and things where it could be alerted, you know, holidays, times are difficult for people where intervention, you know, the speech could indicate that that would be a time of intervention, but nobody would be, quote unquote, monitoring it. Yeah. No, that's something we've thought very much about. And so, one of the things that I think that we emphasize, and this is really just a clinical tool. It's a clinical tool that can augment, you know, the monitoring process. And so, I think through an example that, you know, if an individual is fortunate enough to have a family member, a friend, or somebody who, and again, this goes into another sort of ethical and debate as to, you know, what is the responsibility of someone who shares that kind of an information, but it's similar to having a friend that you call up and say, you know, I'm really feeling lousy and what do I do? And so, the friend or family member will do some, have some questions, and then they will make the judgment as to whether they should go to the emergency room or what they should do. And so, think of it just as a, you know, as an augmenting instrument or tool to help the individual. This is the theme and what drives us. Yeah. I mean, I'm thinking about, you know, many patients of mine will have one or two very close friends or sometimes work colleagues, so they'll sort of say, listen, when I start doing this or this, it means things, means I'm probably not starting to feel well, and I need you to bring that to my attention. And this is sort of the same idea only, you know, using their phone. More objective, you know, that, and so, so we want to, we want to make the subjective more objective, and objective in the sense of measuring and, you know, having specific measures. Right. Right. One last question. In terms of practical, you know, like thinking about it, is it an app that rests on someone's phone? Like what, what is put on their phone? So at the present time, yes, it's an app. It's an app that is installed on a phone. We're just limited to an Android device at the moment, and, and we're just testing it on a few Android models as we, you know, as in going forward. The iPhone system is significantly more, more secure in the sense of having challenges in, in, you know, in getting access to the audio from, from a phone call. So it's really just limited to Android at the moment. Now, if we're able to develop it on an on-phone processing system, then yes, then that would make it much more applicable and easy to adapt to all systems. Well, this has been terrific, and I've, I've enjoyed myself being able to talk with you about this idea and see your data, and I know others have really, obviously by the number of questions, been very interested as well.
Video Summary
The video transcript discusses the challenges of implementing speech analysis technology in usual care clinics. The speaker mentions the complexity of speech and mobile devices, as well as the security challenges associated with data encryption. Currently, data is gathered from mobile devices and encrypted on the device itself before being sent to a server for decryption. The speaker emphasizes the need to process data on the device for efficient deployment in clinics. They mention ongoing work to make this possible and hope it will be available within a couple of years. Additionally, the speaker answers questions about the identification of mood changes and the potential for patient feedback and consent in using the technology. They also discuss the integration of other smartphone data to improve accuracy and the plans for future clinical trials to test the intervention. The speaker concludes by describing the app as being currently limited to Android devices but with potential for adaptation to other systems.
Keywords
speech analysis technology
mobile devices
data encryption
clinic deployment
patient feedback
Funding for SMI Adviser was made possible by Grant No. SM080818 from SAMHSA of the U.S. Department of Health and Human Services (HHS). The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement by, SAMHSA/HHS or the U.S. Government.
×
Please select your language
1
English