Oh, and welcome. I'm Dr. Rob Cotez, Director of the Clinical and Research Program for Psychosis at Grady Health System and an Associate Professor at Emory University School of Medicine. I'm so pleased that you're here joining us for today's SMI Advisor webinar, Self-Report Scales for People with SMI, Introducing the INSPIRE Self-Report Scale. Next slide. That's me. All right, SMI Advisor, also known as the Clinical Support System for Serious Mental Illness, is an APA and SAMHSA initiative devoted to helping clinicians implement evidence-based care for those living with serious mental illness. Working with experts from across the SMI clinician community, our interdisciplinary effort has been designed to help you get the answers you need to care for your patients. Next slide. Today's webinar has been designated for one AMA PRA Category 1 credit for physicians, one Continuing Education credit for psychologists, and one Continuing Education credit for social workers. Credit for participating in today's webinar will be available until March 11, 2024. Next slide. Slides from the presentation today are available to download in the webinar chat. Select the link to view. All right, next slide. Captioning for today's presentation is available. Click Show Captions at the bottom of your screen to enable. Click the arrow and select View Full Transcript to open captions in a side window. Next slide. Please feel free to submit your questions throughout the presentation by typing them into the questionnaire found in the lower portion of your control panel. We'll reserve about 10 to 15 minutes for questions to be answered. We'll reserve about 10 to 15 minutes at the end of the presentation for Q&A. Next slide. And without further ado, I'd like to introduce you to today's faculty for the webinar, Dr. Douglas Nordzy. Doug Nordzy is a Clinical Professor of Psychiatry and Behavioral Sciences at Stanford University School of Medicine, where he is Director of the Lifestyle Psychiatry Clinic and Co-Director of the Stanford Center on Longevity Lifestyle Medicine Program. He co-founded the Stanford Sports Psychiatry Clinic, which serves NCAA, Olympic, and professional athletes in the Bay Area. He developed the Inspire Self-Report Scale for people with SMI to report their symptom severity. And on a personal note, I've known Dr. Nordzy since 2007. He's been a dear friend and mentor over the years, and I'm so excited that he's gonna be giving this talk. So, Dr. Nordzy, thanks so much for leading today's webinar. Thanks, Rob. I appreciate you inviting me to join. And as Rob said, we've known each other for many years, started our careers at Dartmouth and met there. And although it didn't come up in my bio, I've been focused on care of people with schizophrenia and SMI for most of my career, including early days, working in community mental health center settings affiliated with Dartmouth and later in inpatient care. So, great to see you, and thank you for joining the webinar. So, I have no relationships or conflicts of interest to disclose, and the learning objectives are here. Basically, we're going to talk about self-report scales for people with SMI and sort of the characteristics of them and how you might choose amongst them for ones that would best fit your clinical or research setting, and then focus on the ISRS, which is the scale that we developed and that we'll talk about in the presentation today and how to educate a person on how to complete it and how to then use an ISRS to facilitate a discussion of symptoms in the clinical setting. Which is, I'll just say that this latter part is what the psychiatrists that participated in this project with us found so helpful about the scale was using it to sort of prompt an in-depth discussion of what people were experiencing, and we'll talk about that in a moment. So, just to start, most of the clinical rating scales that we hear about for people with psychosis are designed to be rated by a clinician. So, the Brief Psychiatric Rating Scale, or BPRS, the Positive and Negative Syndrome Scale, which is the PANS, those are the ones that you tend to see in clinical trials of medications and such and clinical trials of psychosocial interventions. SANS and NSA16 are focused on negative symptoms, and both types of scales require that a rater be trained, and then they follow a semi-structured interview script that certainly I've done a lot of these interviews with all of these scales, as have most of you, I'm sure. And so, the sort of primary paradigm that we have for sort of, you know, rating the severity of a person's clinical condition is, you know, based in an external person interviewing a patient and sort of documenting a level of symptoms based on that interview, some combination often of frequency and severity of the different experiences like hallucinations or paranoia. And most of these scales have a two- to four-week interval of inquiry. So, over the past two weeks, how often have you felt this or thought this? And then, typically, they're on Likert scales, which means a scale that, where you have either one to five or one to seven sort of items of severity where the higher score is more severe. So, you know, we're used to seeing these ratings. The problem, of course, is that they're time-consuming and cumbersome, and most of us don't use these scales in clinical practice because of the time they require and the, you know, rater training and such. And so, to some degree, while they're helpful in research, really don't get used much in measurement-based care or clinical practice. And, you know, it raises the question, why do we not invite our, you know, people with psychosis to report their symptoms themselves? And, you know, to some degree, there's been skepticism in the literature that people with psychosis would be able to rate their experiences accurately and that, in particular, concerns about lack of insight and maybe people with psychosis would fail to recognize, particularly, you know, positive symptoms of psychosis, delusions, or hallucinations and wouldn't report them adequately. And so that has really led to a, you know, really a much slower development of self-report scales for people with psychosis than we see, for example, for people with depression and anxiety, where the, you know, the PHQ-9 and the GAD-7 are so well-known and routine that many, you know, primary care practices are, you know, our health system at Stanford requires that everybody get a PHQ-9 on an annual basis, you know, to meet, you know, regulations for screening for suicidality. So, you know, those self-report scales are very widely used. And yet, for people with psychosis, we really don't have many. And as you'll see in a moment, I was designing a project with a resident at Dartmouth about 10 years ago, and we wanted to, you know, the project involved needing people to self-report. And we looked around and really didn't see self-report scales. These that are listed here, particularly the top four, which are the ones that are really usable, have developed in recent years. And we'll talk a bit about their characteristics and how they differ from each other. The PETERS and the Community Assessment of Psychic Experiences were older scales developed in psychology for non-clinical general population used to sort of document psychotic-like symptoms in the general population. And the last one there, the SENS, was replaced by the MAP-SR. It's an older self-report for negative symptoms. So, currently, there's a scale called the Colorado Symptom Index, and then a modified Colorado Symptom Index is the MCSI. And the CSI itself was never published and never widely used. And the modified I'll show you in a minute is a little more polished and has had some validation worked on with it and appears to be a valid scale. The 4S, or the Symptom Self-Rating Scale for Schizophrenia, has been developed in Scandinavia and Germany. And it's an interesting scale, very elaborate, though. Has seven domains with up to 15 questions in each. I think it's a total of 75 questions in the scale, and all of which are yes-no questions within the VAS scale at the bottom to rate severity. So, that's sort of the second. Those two are really more comprehensive scales that look at symptoms in multiple domains of schizophrenia or psychosis. The MAPS, Motivation and Pleasure Scale Self-Report, is a sort of a negative symptom rating scale that looks at two domains of negative symptoms that were shown to have reasonable reliability for self-report, and we'll look at that a bit. And then the Personal and Social Performance Scale is actually a widely used scale in clinical trials for rating functioning. And there's just this one study, I think it's in German, where they tested having people self-report their functioning level on the PSP, which is otherwise rated by a professional rater. So, we're going to take a little tour through these different scales that are now in existence and look at their characteristics and how they might fit certain clinical research settings. And then I'll go into more depth describing the scale that we designed, which is called the Inspire Self-Report Scale. So, the Colorado Symptom Index, pretty simple, 14 items. They each have a five-point rating in terms of intensity, and so people are asked to rate the frequency of experiences in the past month. And as you see here, this is from the reference at the bottom that was an unpublished manuscript from 1999 that wasn't widely recognized. And this is actually the original CSI that's a little funky to use. And the next page here is the modified, which you can see is a cleaner presentation of essentially the exact same thing. So, the questions here, there's 14, and I'll show you the second half. And then people rate, you know, so there's this introduction. I'm going to ask you some questions about psychological-emotional difficulties you may have, and people are asked to describe how much they've had the problem in the last month. And then they would say not at all, once, several times during the month, several times a week, or at least every day. And then these others over here are refused, not applicable, or doesn't know. So, you can see that the five points from zero to four, and then people are rating general symptoms like feeling nervous, tense, or afraid, depressed, lonely. And then there's one item here on how often have others told you that you were acting paranoid or suspicious. There's an item about voices. There's an item about sort of ambivalence. And then you can see here number seven is, or difficulty with focusing or cognitive problems. Number eight is odd behaviors. Number nine is feeling out of place, or you didn't fit in, which is an interesting item. I guess that might have something to do with negative symptoms. I'm not quite sure what domain that's pointing at. How often did you forget important things? So, a cognitive item is 10. Problems with thinking too fast, like racing thoughts. And then item 12 is back to suspiciousness and paranoia, again, but asking the person how often do they feel it. And then thoughts of hurting or killing themselves or hurting others. So, those are the 14 items. And you can see this reference is for the modified. And you can see the modifications to the scale. All the 14 items and their prompts are identical. The modifications have to do with the introduction and the formatting of the scale. And so, there's been some research work showing that this is reliable and valid, with the modified version. And again, the rating window is over a month's period of time. So, you know, if you were seeing a person, let's say on a monthly basis for a med check, and you had them rate the scale ahead of time, it would give you a picture of how often they were experiencing these symptom domains in the prior month. But the person is also averaging in their mind, kind of like you would as a rater doing a PANS, where you're asking a person about a two-week window as well. You're trying to sort of capture severity over the course of a month. And in this case, it's really just frequency, not severity that's being rated. So, if, for example, you were asking about hallucinations, you know, did they happen once or multiple times or every day during the month. So, that's the CSI. The 4S has seven domains. And as I said, each domain starts with these eight to 15 yes, no questions. And then at the end of it, there's a VAS scale, a visual analog scale where a person puts an X on a line saying how distressing that domain is. And so, there's a domain about hallucinations and a series of questions about have you heard this or seen something or felt something where the person would say yes, no to all of those 10 items on the list and then say how distressing it was for them. And they're instructed to rate the last one week. So, a shorter rating interval. So, the information would tell you more about how the person's been feeling in recent time rather than a full month. And presumably, people will have an easier time rating something that's more proximal than trying to remember how they've been feeling over an entire month. There was one study in 61 inpatients with some of them rated twice and 91 ratings in which they found that the total score correlated with a short version of the PANS and that items around thought disorder and hallucination had low to moderate correlation with PANS items. And that the hallucination subscale was sensitive to change over different ratings, over two different rating points. They also found that it was inversely correlated with the World Health Organization and well-being scale. So, in other words, greater symptoms in the 4S was associated with less well-being and also inverse relationship to the SDS functioning scale. So, you know, validation that it was giving information that was relevant to people's well-being and functioning. I didn't put in pictures of the items because it's fairly complex, but an interesting scale. And certainly, they found that people were able to use it and rate it well. The maps, as I said, so this is derived from the clinical assessment for interview for negative symptoms, which is designed as a professional rater rating people's symptoms. And then they took items from that to develop this 15-item, five-point intensity rating of negative symptoms that focuses on motivation and pleasure domains and not the third domain of the canes. And they validated it in three different groups of inpatients and outpatients and found that it significantly correlated with both the canes rated by a professional rater as well as the PANS negative subscale. But it did not correlate with the PANS positive in general subscales, which is desirable because you want to be rating negative symptoms and not have it colored by other domains. But they did find in, I guess, the two earlier studies found it didn't correlate with depression, but in this German version that was reported by Engel and Lincoln, they did find a moderate correlation with depression, which, again, is undesirable because then you're capturing a different domain of symptoms. So, interesting scale that could help people to communicate their experience of negative symptoms. And then, finally, the PSP, as I mentioned, which is a functioning scale, 16 items for each in each of four domains of socially useful activities, relationships, self-care, and disruptive behavior. And there's a six-point scale where a person is saying how much they agree, the degree to which they agree or disagree with a statement. And, again, this was only done in German, but they did find a significant correlation between professional rater and a self-report in the activities domain. But they didn't find a significant correlation in the other three domains, so fairly negative outcome from that study. And so, as I mentioned, Danielle Daly, who was a resident with us at Dartmouth, actually received an award from the APA Resident Research Award, the APIRE Award, in 2013, and we conducted a study in 2014. And so, at that time, I had been doing lifestyle work with people with psychosis. As Rob mentioned, one of my big interests over time has been in looking at physical exercise as a lifestyle intervention that can help people to manage symptoms and maintain their mental and physical health better. And so, I had been advising and coaching my patients with schizophrenia to increase their level of physical activity for decades, and I had a group of patients in our psychosis clinic at Dartmouth who had taken that on and were exercising regularly on their own. And so, we wanted to look at that and try to document what was motivating people with psychosis to exercise regularly, and of course, they were telling me about improvements in how they felt in various domains, particularly cognition and energy. And so we were trying to document that. And so we designed this study where we asked people to rate how they felt immediately before and immediately after the next three times they exercised. So each person rated themselves three times. And this was exercise they were doing in the community. And these were all people who were exercising at least 30 minutes, three times a week on their own. And so we really needed a rating that would help us to show sort of a momentary assessment of their symptom experience in order to see how that might change around exercise sessions. And we needed something that would be responsive to change within half hour to an hour, right? Because most people were exercising for about that amount of time. And none of the scales that existed were like that. And even, not that we were gonna send a professional rater out to, we didn't have that much of a budget for this project, but we weren't gonna send a professional rater to the gym with them. But even if we did, a PANS that rated how they felt over the last two weeks wasn't gonna show change before and after an exercise session. So we consulted with our colleagues at the Dartmouth Psychiatric Research Center and how should we design this? And we got two very nice pieces of advice. One was to think about developing a visual analog scale, that visual analog scales, which means essentially you create a, and I'll show you this in the next slide, but you create a 10 centimeter line with end points on it, good or bad, or whatever you wanna put on those end points. And then you tell people to put an X to describe how they're feeling. And that visual analog scales themselves are a well-validated way for people to describe their experience. And then this one particular scale called the Lehman, so that's Tony Lehman, who was a well-known an icon in the field, a giant in the field for many years at Maryland, University of Maryland, who developed what he called the delighted terrible scale or the sort of single points quality of life scale and said, use that because it's a well-validated single item self-report scale. And so we ended up deciding to try to get some measure of how people were feeling in five domains. And so we developed two questions in each of those five domains. And we started it off with the question of how do you feel right now? Because we really wanted people to tell us about their momentary experience. How are you feeling presently? So then they could capture how they were feeling in this short period of time before and after an exercise session. Of course, that would be useful in a variety of other types of research where you might want to, let's say, show change in how people felt before and after a meditation session or a psychotherapy session or doing maybe a work day or going to social skills training experience. And so we had five domains. The first one was a global wellbeing domain that included the delighted terrible scale and a scale rating energy. And again, I'm gonna show you all of these in just a moment. So you'll get a picture of it. And then we had two mood items, depression and anxiety. We had two psychosis items, hallucinations and delusions. We had two items to reflect negative symptoms and we chose social interest and motivation for those two. And then two on cognition. So clarity of thought and concentration. And so that's what the VAS looked like. And this line isn't 10 centimeters depending on the size of your screen, but we created it to be 10 centimeters long and it was actually paper and pencil so that then we could measure it. And so you can then basically come up with a zero to 100 scale by measuring. So on this particular X in this line might be at about 80 if you were to say it was eight centimeters away from the left side. And so then that would become the score just by measuring where the X was placed. And then the way we structured it is that we had the name of the domain. So in this case, depression, and we had just a couple of words to concisely describe the concept, sadness, hopelessness, worthlessness. And then in a separate set of instructions on a page at the beginning, they would get a more detailed definition of each of these. So you can see depression here at the bottom, state of unhappiness, sadness, hopelessness, often with feelings of guilt and worthlessness, et cetera. And so for each of these domains, we gave a detailed description. And this is the description for all of the 10 items that you can see here. So concentration, ability to focus on a given task without being distracted. And so there was a front page with instructions that showed them how to do the, how to put an X on the line and gave them this set of definitions. And then let me go back. And then this, for each item, you get a very brief description and the line. And so we did this, and then this was the outcome of that study. We had, I think, 27 patients who rated themselves three times. So we had a total of, and a few people didn't turn everything in, but somewhere close to 70 ratings. And so on this graphic, a zero would mean that the average rating after exercising was the same as before. And a one, two, three, or four would be rating higher after exercise compared to before, whereas a negative one, two, three, four would be rating lower, rating the domain lower after exercise compared to before. Excuse me. And so what you see here is, on the global item of delighted, terrible, how do you feel about your life right now? People felt more delighted about their life than terrible. You know, by about one point, and again, this is a 10 centimeter line, so it's about a 10% improvement in how they felt overall about their life afterwards compared to before. Energy went up by about 15%. Motivation went up a little over 10%. Social interest, so social interest and motivation being the two negative symptom items, both up about 10%. And clarity of thought and concentration, which are the cognition items, similar improvement. So when we saw anxiety and depression, both went down a little bit. Anxiety went down about a little more than 5%, and depression, a little more than 10%. Hallucinations and delusions, very little change moment to moment on before and after exercise, even though there's some evidence that longer term studies that look at people who start sedentary and then do exercise intervention over six to 12 weeks may show some improvements in psychosis that are more cumulative, they're actually typically smaller than improvements in these other domains. And so what this led us to believe is that, you know, what motivates people with schizophrenia to exercise regularly is improvements in the non-psychotic domains and that, you know, in particular, cognition and negative symptoms and sort of global wellbeing and particularly energy, right? Cognitive energy, sort of overcoming fatigue seem to be the most notable improvements for them with some minor improvements in mood. And that really follows well what you see in the rest of the literature on physical exercise, that improvements in cognition and depression are some of the most consistent findings across many populations, including, you know, otherwise healthy aging adults and people with mood disorders and such. So that was exciting to see and we replicated this in a second sample at Stanford and had similar findings. This is another scale that we really liked called the Subjective Exercise Experiences Scale, which is again a self-report, but not at all designed for psychosis. And in this, and this is a zero to, it's a seven point Likert scale so that on this scale going up by 0.7 would be a 10% improvement. And you can see that these positive wellbeing items around feeling great, positive and terrific and strong are going up to some degree. Exhausted is also going up, but feeling discouraged, crummy and awful are going down. So again, lines up with the, although we don't see the improvement in energy here and the descriptors for exhausted and fatigued here, tired are more about physical energy, whereas our item was more about cognitive energy or, you know, mental energy. And you, but you can see certainly that sort of negative affect feelings going down and some positive wellbeing items going up on that scale as well. So after using it in two studies with the Stanford study was a larger, so about 75 patients, we heard from our participants that the scale was intuitive and easy to use. It captured relevant experiences for them. We found that it was sensitive to change, but also that the VAS, the Visual Analog Scale was a bit awkward to use, particularly for us afterwards to have to measure all those lines, but also that people sometimes got confused about how, you know, where to put the mark and you'd sometimes get a mark that was not clearly in one point on the line. So we thought, well, this is a pretty decent scale. Maybe we should, you know, try to develop this a little bit more into something that could be used, you know, not only in a research setting like this, but also in clinical practice. And by this time I was at Stanford and I was working with another resident named David Spelber who developed this for his senior scholarly project in his residency at Stanford. So we decided to measure the accuracy of the scale. And at the time we had given it a different name because we didn't know what to call it. And at this point we transitioned the name to the INSPIRE Self-Report Scale. In the beginning we called it the Norty Valley Scale because we didn't have a better name for it, but that felt a little too narcissistic. So we changed INSPIRE as the name of our early psychosis clinic at Stanford. So we decided to name it for that because they wouldn't let us call it the Stanford Scale. And so we decided that we wanted to look at the accuracy of self-report relative to a gold standard and look at its relevance for clinical assessment. And we found that when we were thinking about a gold standard, that most of the well-validated scales like the PANS and the BPRS didn't have the same time window. So it was going to be hard if we did a BPRS and then had a patient complete an ISRS at the same time, they might not line up because one scale is rating a four week or two week period of time. And whereas the person is rating how they're feeling today and also the five domains didn't line up well with most other scales because they often don't include the same items and we needed an efficient comparator. So what we ended up deciding to do was to rate the ISRS at the same time by a physician who knew the patient well, as well as the patient themselves as they were coming in for a clinical visit. And so we'd be able to compare how the person rated themselves shortly before a clinical visit with how their clinician rated them at the end of that rating blind to the person's rating. So, as I said, we changed the name to the Inspire Self-Report Scale and we employed a person with lived experience on our research team who helped us to redesign the scale. We decided we wanted to develop a Likert version for ease of use and interpretation and see how well that compared to the VAS version. And so then Akash helped us with sort of designing the language for that to make sure that it was, you know, that it would be well understood and interpretable and we wanted to also develop an ease of use rating so people could tell us how they liked using the rating scale. And then we developed this physician version that was identical to the patient version of the ISRS except that we asked them to collect some basic demographics and also be able to do a discrepancy rating. So, and I'll show you the methods in just a moment, but when the two ratings, once they were compared, disagreed to have a chance for the physician to indicate whose response was more accurate, which came out of a really interesting discussion afterwards so I'll show you how that worked. And just to show you, you'll see that I copyrighted the scale just so that, you know, it wouldn't be, you know, co-opted by some pharmaceutical company and used in other ways. So we allow people to freely use the scale and you can just get in touch with us and we'll give you permission to use it with no charge, but we wanted to copyright it. So this is how the Likert version of the scale looks. And so this is that global item, how do you feel about your life right now? And then there's a neutral in the middle, extremely delighted over to the left and extremely terrible over to the right. And you can see the seven points. So extremely, somewhat, slightly, and slightly, somewhat, and extremely terrible. And so the study design was that we approached clients in the waiting area outside of our Inspire Clinic and we asked them if they wanted to participate in the study. We showed them a consent form and if they decided they wanted to participate, then they were given a copy of the ISRS with brief instructions on how to complete it. And we had a consecutive series of 50 people use the Visual Analog Scale version and then the next 50 use the Likert version of the scale. And then the client would complete their rating, that would be put away. They would go into their visit with their psychiatrist. Psychiatrists would get their copy of the scale and they would, at the end of a 25 minute or so interview, complete their rating, blind to the client's rating, and then the two would be put together and the client and the psychiatrist would look at it together and compare how they rated, which was actually a fun process, right? Oh, gee, you rated this, you rated hallucinations higher than I did. What was that? And then they would talk about it and try to understand, you know, and obviously identify places where they had gotten it right, where they both had agreed, and then talk about the areas in which there was a discrepancy. And then the psychiatrist was tasked with deciding, so any time that there was a discrepancy, was the patient more accurate in their initial rating and they just hadn't asked enough about that domain or didn't get the detail after having talked it over more carefully with the patient, or was the patient, you know, missing insight about their illness and not rating their symptoms as accurately as the psychiatrist had assessed them. And so that became part of the outcomes of the study. So again, we wanted to compare how accurate the Likert would be relative to the VAS scale, and we did a statistical correlation of participant and physician ratings. And then whenever we defined concordant ratings as being within a point of each other on the Likert scale, which was equivalent to 17 millimeters on the VAS scale because this is a seven-point scale, and so that was the equivalent dividing of a 100-millimeter line and then discordant being just greater than a point difference between them. And then, you know, the resolution was that either the participant or the physician had been most accurate in their initial rating of severity, and then we defined accuracy as the combination of when the two ratings were concordant plus the times in which the discordant ratings were resolved in favor of the participant. In other words, on, let's say, the hallucination item, the total accuracy of patient ratings on hallucinations was the sum of, you know, how often the two ratings agreed and how often the psychiatrist decided, really, the participant had had it right in the first place. Excuse me. I'm recovering from a cold and still have a little bit of a tickly, tickly throat here. So here's just showing you the rest of the items on the scale. Here's anxiety, worry, nervousness, or fear. Here's depression. You saw that before. And most of these items follow this sequence of non-mild, mild to moderate, moderate, moderate to severe, severe and extreme for the symptom ratings. Here's energy with degrees of feeling tired versus energized and neutral in the middle. Hallucinations you can see defined as voices, direct communications or visions, and then that sort of mild to extreme sequence. Delusions, where we focus primarily on paranoia and suspiciousness, which are the most common form of delusions, motivation, which was also called drive and ambition and then clarity of thinking. And here it went from sort of clouded to hazy to totally clear. And then concentration, poor to excellent, and social interest, preferring to be alone versus preferring to be with others. And there you see that ease of use scale at the bottom. That's what people were rating. And so we, Anika Chawla, who's one of my colleagues at Stanford who worked with me in the Inspire Clinic, helped us to, you know, was the lead author on the publication on the ISRS, and actually the full scale is in the publication as well, in the Journal of Psychiatric Research last year. And you can see Dr. Spellberg and Jake Ballin and Agnes Klonowski, who also worked with us. So Jake and Agnes and myself and David were the four psychiatrists who participated in the study, and we were the raters. And so here are our findings. You can see that the average age was around 30. We were early psychosis-focused clinics, our average age is younger than many schizophrenia clinics. We were about third female participants, and you can see diagnoses with preponderance of schizophrenia and schizoaffective being about three-quarters of patients. But then we have a fairly larger proportion of sort of other psychotic disorders, given that, you know, being in early psychosis clinics, some people may not have met full criteria for schizophrenia or schizoaffective yet. But you can see that almost all the patients were taking an anti-psychotic medication, 98% total. And this is the correlation. So this is a dense table. So you can see the 10 domains on the left, and then the correlation coefficient for the Likert version, and then a significance value, and then for the VAS version comes next to the right of that, and then the statistical correlation. So higher number means greater correlation. And so you can see, and we put them in order of the — in numerical order for the Likert version. So the global rating was correlated at 0.8, all the way down to social interest at 0.45. So, you know, high to moderate levels of correlation across the items. And you can see the ones that did the best, you know, global well-being, depression, anxiety, clarity, delusions. And then in concentration, hallucinations still above 0.65, and then the motivation, energy, social interest dropped off a bit. And all of those are highly significantly correlated. And then on the VAS, the correlations were a bit lower on most items, better for the energy domain and the social interest domain, but otherwise generally ran a little bit lower because it's, you know, it's a more infinite scale and a little bit harder to get perfectly correlated. And you can see the statistical correlations being a little less tight on those, but still highly significant correlation on all of those. And this is showing it graphically. And this will give you a picture of what I was talking about before of total accuracy. So, for example, on the far left here, the global well-being item, that delighted, terrible scale about how you're feeling about your life right now. The light blue bar tells us that basically 80% of the time that the psychiatrist and the participant rated the same within a point of each other. And then the dark blue bar is when after discussion, the physician concluded that the participant was really more accurate than they had been. And that takes it up to about 95%. And then there's maybe a percent in the green line, which is uncertain, and then maybe 2% or so where the physician rating was thought to be more accurate. And so you can see that trend across all of these, that the concordance was high in the 75% range for almost every item. And then when there was discordance, the original rating by the participant was always judged, had much higher rates of being judged to be more accurate than the physician's original rating. Delusions is really the only item where physician ratings came close to being accurate as often as participant ratings when there was disagreement. And on hallucinations, there's a little more as well. But the vast majority of the time, the physicians ended up deciding that their original rating was inaccurate. And having been one of those physicians, it was usually because I didn't ask enough detail. And so I had to do a rating at the end of my interview because I was running out of time. And so there may have been items like energy or that sort of thing where I just didn't ask enough information to be able to rate it as accurately as the person. And once we talked about it, I understood that they had really gotten it right. And of course, when you think about this in clinical practice, how nice it would be in the opposite scenario where the person comes into their visit with me, they've completed the ratings and they share them with me. And then we use that as a springboard for discussion to talk about how they're doing based on their report of how they're feeling. And so then that would give us the opportunity to talk through all those items and ask the right questions and learn more about the person's experience that day, driven by their description of their experience rather than just by the questions I would ask on my own. This table is comparing, and let's see, I'm getting to the point where I should be wrapping up. This table is comparing the accuracy of the visual analog scale versus the Likert. And you can see the Likert being the red bars had higher concordance most of the time, except on the energy rating and the motivation and delusions. So we generally found that Likert was easier to use and the concordance is pretty strong with it. So we concluded from this that ratings on the Inspire Self-Report scale by people with lived experience of psychosis are highly accurate relative to trained research clinicians who also knew them well. So we were not rating patients that we were just interviewing that day in a blind way. Most of the people that we were rating we'd known for months and years before having done the ratings with them. Accuracy was not as strong in positive and negative symptom domains as it was in the other domains. The ease of use ratings were high across all the scales. The Likert version performed well and was easier to use and to score. And as I said, the clinicians in debriefing afterwards found that the IRS ratings were helpful in directing our attention to areas where the client might be suffering but we might not have asked the right questions. And so just to wrap up then, self-report scales for people with severe mental illness as you're thinking about them in various domains, whether that be clinical for measurement-based care or for research, consider the range of symptoms that are being measured and whether they fit what you need to know about, consider the rating window and whether the scale is designed to look at a relatively short period of time or the current moment or a week or several weeks or a month, how easy is it to use and how sensitive is it to change. If you're trying to show change in response to an intervention, you need a scale that's going to be responsive and not sort of stay stuck. The 4S scale and the MAP-SR have demonstrated validity and the ISRS has demonstrated feasibility in the sense that we didn't do proper validity against a gold standard rating scale. And these scales would have potential value in measurement-based care that could help a participant and their team track personal ratings in relation to treatment steps over time and have potential value in research, particularly the ISRS, helpful in momentary assessment. And of course, all these scales potentially in quantitative self-report of symptoms. So the ISRS is quick, broad and intuitive, easy to use, facilitates communication of current experience as both versions available, the VAS and the Likert. And as I said, it's available for use without cost. Just email me at the address here and I'm glad to share that permission. And there's some references from our paper that I showed you and then these are the other rating scales that we talked about. So I think that's it. I'm handing it back to Rob. Doug, thank you so much. That was really, really great. Really appreciated the overview of all the other rating scales and the other self-reported rating scales and then how that kind of compares to the ISRS. And before we go into the Q&A, can you just hit the next slide for me? Okay, so SMI Advisor is available from your mobile device. Use the SMI Advisor app to access resources, education and upcoming events, complete mental health rating scales and maybe the ISRS could potentially go in there. We'll have to talk about that offline sometime. And even submit questions directly to our team of SMI experts. Download the app now at smiadvisor.org slash app. Okay, all right, next slide and we can go into the Q&A. All right, so we got a lot of really great questions from the audience. One of the questions that came up was, was there a perspective of people with lived experience in designing the questions for the ISRS? Yeah, so it was a two-stage process. And so when we originally designed the VAS version for the two exercise studies, that was Daniel Dolly and myself getting together and just sort of with our experiences caring for people with psychosis, we wrote that version. But then when we decided to do this, sort of feasibility study, the validation study and we were designing the Likert version, that's when we had a person with lived experience in our clinic who was eager to get involved. And so he helped us with creating those more elaborate, so what he really, both tuning up the language all through, but also particularly with the seven point ratings and sort of what to put on each of those items, which was a great help to us to think about how do you ask people to quantify and give them sort of anchors that will be meaningful relative to their experience. Yeah. Okay, thank you. Another question is specifically kind of around the delusion question in the ISRS. I know that one of them kind of focuses on suspiciousness and kind of paranoia. Can you talk a little bit about kind of the challenges potentially in evaluating delusions in the self-report scale? Yeah, and our goal with this was to minimize burden on the person who was completing it and to make it quick and easy to rate. So we focused on suspiciousness and paranoia as being kind of the dominant definition of that. And of course that may miss the mark on some people's experiences, right? If somebody is having a delusion that doesn't sort of fit into a more paranoid type. So that's going to create a... We just didn't want to do what, for example, the 4S does, which is this much more comprehensive list of all the different varieties of delusion or hallucination that a person might have and ask them to rate each individually. So I think in settings where that would be more important, a scale like the 4S might actually be more valuable, right? Where you could then get some, prompt the person to rate sort of their experience of a variety of different delusion types. Whereas the ISRS is going to be sort of more of a global capture, right? Of what's happening and certainly has a bias towards a paranoid delusion as opposed to other subtypes of delusions. Okay, thanks. That makes a lot of sense. What do you think the next study that needs to be done with the ISRS is? Clearly the next study to properly validate it would be to have people receive a sort of more gold standard rating like a PANS in a usual format at the same time as they rated themselves on the ISRS. And again, the challenge there is that, a person is gonna be rated, gonna be instructed in the ISRS to rate themselves in the present moment. Whereas the rater who's doing a PANS is gonna be rating the last two week experience. And so there certainly could be some noise in the concordance there based on the different rating window. But still you would probably expect that that's assuming you were careful to recruit people who weren't having, who were relatively stable, let's say, and not just going in and out of the hospital or something you wouldn't expect necessarily for their ratings to be markedly different between the present moment and what they had had over the more recent window. So a PANS gives you both a positive and negative subscale that could be compared to the pairs of questions on psychosis and negative symptoms in ISRS. And then the global section, the other 16 items on the PANS includes an anxiety item, a depression item. There isn't really cognition there, but, or domains like energy or wellbeing. So those might have to be looked at separately or just the total scores, of course, could be correlated as well. So I think it's a little tricky to interpret, but the sort of usual paradigm for a validation study would be to compare to an established rating scale. And probably the PANS is you could do multiple different scales, you could compare the mood items to a PHQ-9 or a CAT-7, but probably the PANS would be the best comparator. Okay, that sounds good. One more question I have for you is, let's say you were starting a new clinic for people with SMI, starting this new clinic from scratch. And this is something that you've done before. I mean, I'm wondering a little bit about, what self-reported scales would you include sort of in the battery of things that you would want maybe to people, maybe for people to complete before they came into the visit? Yeah, and these days with electronic medical records and the ability to push out scales over my health types of portals, there's a potential to really have that information really come into the visit and the documentation quite efficiently. Yeah, I mean, if I were designing a clinic from scratch, I would like to see an ISRS. And would you also do a GAD-7 and a PHQ-9? They would give more depth of information. So I think that that combination could be really nice, actually. The only thing that the PHQ-9 would give you a measure of suicidal ideation, which you don't get in the ISRS, as well as more depth in other mood and neurovegetative symptoms, which could be useful. And the GAD-7 would give sort of more information about sort of impacts of anxiety that might be related to delusions as well. I think, I don't know that there's another good way to measure delusions in more depth on a self-report scale other than using the 4S, but as I said, that's a pretty long scale that you might not wanna ask people to do every time, but maybe that would be something that could be rated on a less frequent basis in order to get some more depth of symptoms there. Well, thank you so much. And actually one of the folks in the chat had mentioned something about a measure of recovery from psychosis as something maybe also to consider. But unfortunately, we're out of time. There's been a lot of really positive feedback in the chat about just how useful this has been. So Dr. Norzey, again, I really wanna give you a big thank you from SMI Advisor, and maybe now we'll go into the outro. All right. All right, so if you could go to the next slide for me. There it is. Okay, so if there's any topics covered in this webinar that you would like to discuss with colleagues in the mental health field, post a question or comment on SMI Advisor's discussion board. This is an easy way to network and share ideas with other clinicians who participated in this webinar. If you have questions about the webinar or any other topic related to evidence-based care for SMI, you can get a question answered within one business day from one of our SMI Advisor experts. This service is available to all mental health clinicians, peer support specialists, administrators, and anyone else in the field who works with folks with SMI. It's completely free and confidential. Next slide. Okay, SMI Advisor offers more evidence-based guidance through the app. This app includes access to several clinical rating scales that you can use to get insights from right away. Download the app by clicking on the link in the chat or by downloading the slides. Next slide. So to claim credit for participating in today's webinar, you need to have met the requisite attendance threshold for your profession. After the webinar ends, please click continue to complete the program evaluation. The system then verifies your attendance for credit claim. This may take up to an hour and can vary based on local, regional, national web traffic and usage of the Zoom platform. Okay, next slide. And finally, please join us in two weeks on January 26th as Dr. Jason Kolditz presents considerations for the use of online peer recovery forums for individuals with serious mental illness and substance use disorders. Again, this free webinar will be January 26th from noon to 1 p.m. Eastern on Friday. And with that, thank you to our presenter, Dr. Nordze. Thanks to all of you for joining us. And until next time, take care. Thank you so much. It was a pleasure being with you.

self-report scales

serious mental illness

SMI Advisor initiative

INSPIRE Self-Report Scale

well-being

psychosis

negative symptoms

cognition

scale development

validation studies