Research on Development of Korean Adolescent Stuttering Anxiety Test
Abstract
The objective of this study is to develop primary questions in a test of anxiety for adolescents who stutter. This test sets out to evaluate generalized anxiety frequently found in adolescents, social anxiety related stuttering as in previous literature, anxiety associated with communication, and anxiety related to school life.
To evaluate the content validity of the developed Korean Adolescent Stuttering Anxiety Test (KASAT), 12 experts in the field of speech-language pathology participated in the study. The subject of the facial validity was 15 middle school students and 15 high school students. As convergent validity, 13 middle school students and 13 high school students participated in the study.
First, most of the questions in the four areas of KASAT received a CVI score of .70 or higher in a content validity assessment by experts, which indicates that they reflect characteristics of anxiety in adolescents who stutter. Second, the mean of facial validity score in all areas of KASAT was 4.26. This indicates that all of the modified questions are adequate in terms of vocabulary, sentence structure, and content difficulty to be used for middle and high school students between ages 12 and 18. Third, The results of the convergent validity test showed that the internal consistency between the items and the total scores and the internal consistency between the items by the category were all valid as the items of the anxiety test.
The development of KASAT items and the content validity, facial validity, and convergent validity of the KASAT are considered to be the more appropriate items to test the anxiety of adolescents who stuttering.
초록
본 연구는 말더듬 청소년에게 자주 나타나는 범불안, 사회불안, 의사소통과 관련된 불안, 학교 불안 등을 평가할 수 있는 청소년 말더듬 불안 검사를 개발하는 것이다.
내용타당도 측정을 위해 언어치료 전문가 12명, 안면타당도 측정을 위해 중고등학생 각 15명씩 30명, 수렴 타당도 측정을 위해 중고등학생 각 13명씩 26명을 대상으로 하였다. 내용타당도와 안면타당도는 개발된 문항을 제시하고 타당한 정도를 5점 척도로 응답하게 하였고, 수렴타당도는 불안 검사의 각 문항에 대상자가 실제로 응답하도록 하였다. 내용타당도를 실시한 결과를 바탕으로 청소년 말더듬 불안 검사를 수정하여 중고등학생에게 안면타당도를 실시하였다. 안면타당도의 결과를 반영한 검사로 수렴타당도를 실시하였다. 내용타당도는 내용타당도지수를 산출하였고, 안면타당도는 중고등학생 각 집단과 성별의 평균을 구하였고, 수렴타당도는 검사도구의 문항과 총점 간에 상관을 구하였다.
첫째, 전문가 내용타당도 결과 네 범주에 속한 문항 중 대부분의 문항은 CVI 점수의 .70 이상으로써 말더듬 청소년 불안의 특성을 나타내는 문항임을 알 수 있었다. 둘째, 중고등학생들이 문항에 대한 안면 타당도를 평정한 결과 평균 4.26점으로, 수정된 말더듬 청소년 불안 검사의 문항이 모두 12-18세 사이의 청소년들에게 실시하기에 어휘, 문장, 내용의 난이도가 적절한 것으로 나타났다. 셋째, 수렴타당도를 실시한 결과 문항-총점 간 내적일치도와 범주별로 문항 간 내적 일치도를 살펴본 결과 모든 범주에서 청소년 불안 검사 문항이 타당한 것으로 나타났다.
한국 말더듬 불안 검사의 문항 개발과 내용타당도 및 안면타당도, 수렴타당도 과정을 통해 국내 말더듬 청소년들의 불안을 검사할 수 있는 타당한 검사도구 개발이 일차적으로 이루어졌으며 후속 연구를 통하여 청소년 말더듬 불안 검사가 표준화되어 현장에서 사용할 수 있기를 기대한다.
Keywords:
Adolescent stuttering anxiety test, content validity, facial validity키워드:
청소년 말더듬 불안 검사, 내용타당도, 안면타당도Ⅰ. Introduction
Stuttering, a communication disorder, is mainly associated with symptoms such as speech disfluency and resulting psychological attitude problems. Its external characteristics include speech disfluency and associated physical concomitance, while internal characteristics are composed of cognitive aspect and emotional aspect. Self-awareness about stuttering and communication attitudes compose cognitive traits of the disorder, and stuttering causes a host of negative emotions such as punishment, frustration, guilt, and hostility (Kwon et al., 2012). Children who stutter do not exhibit serious psychological attitude problems compared to adult patients with the same disorder, but for stuttering in patients who are in their adolescence or older, internal characteristics are crucial factors in evaluating and treating the condition.
Tests about communication attitude, self-awareness about stuttering, efficacy of fluency, and control scale regarding problems are being used to assess internal characteristics of people who stutter (Kim & Shin, 2007; Kim et al., 2010; Kim et al., 2012; Shin et al., 2013; Sim et al., 2010). Most of these tests are devised to evaluate the cognitive aspects of people who stutter, but there is a dearth of methods for assessing emotional aspects of people who suffer from this disorder. Nevertheless, there is research about the emotional side of stuttering, specifically regarding anxiety in stuttering adults (Kim et al., 2008) and school-age children who stutter (Kim, 2014).
Many negative experiences that result from stuttering are thought to amplify such anxiety (Blood & Blood, 2007), but there are inconsistencies and uncertainties in findings from research on the relationship between stuttering and anxiety (Menzies et al., 1999). Nonetheless, recent studies suggest more persuasive evidence on the relationship between stuttering and anxiety. Many authors express confidence that people who stutter exhibit high levels of anxiety, with more evidence that such an emotion seems to be restricted to social or performance-related situations. Such findings are spurring follow-up studies on social anxiety, fear of negative feedback, and anticipation about social wounds suffered by people who stutter (Craig & Tran, 2014; Iverach et al., 2011; Menzies et al., 1999).
Many cases of high-level anxiety disorders are reported in non-stuttering adolescents as well. In particular, generalized anxiety disorders that are associated with chronic, excessive, and uncontrollable feeling of concern and apprehension, as well as social anxiety disorders linked with significant and sustained fear of social or performance situations observed by strangers or other people, are found prevalently in this population. McAllister et al. (2015) conducted SCARED test on 35 stuttering children and 33 stuttering adolescents, and found that adolescents between ages 12 and 18 exhibited high levels of generalized anxiety, panic, social phobia, and school avoidance. As such, there is a need to survey whether levels problems associated with anxiety displayed by stuttering adolescents are similar to those experienced by non-stuttering adolescents, and confirm if stuttering is associated with other features of anxiety. Results from such a study will enable the development of methods for mediating psychological anxiety found in stuttering children. To achieve this end, a tool for testing anxiety traits of stuttering adolescents needs to be developed.
The objective of the current study is to develop primary questions in a test of anxiety in stuttering adolescents. This test sets out to evaluate generalized anxiety frequently found in adolescents, social anxiety that is suggested as being related to stuttering in previous literature, anxiety associated with communication, and anxiety related to going to school or school life. The current study will also test content validity to confirm whether the developed questions can evaluate anxiety characteristic in stuttering children, in addition to conducting face validity test to see whether the questions can be understood by adolescents. Finally, convergent validity of the questions will be assessed to understand whether the configured questionnaire shows a relationship between the questions and the total score.
Ⅱ. Method
1. Procedures
In the present study, we configured the questionnaires to develop a test of anxiety in Korean adolescents who stutter, referencing previous research (Birmaher et al., 1997; Birmaher et al., 1999; Kim, 2010; Yang et al., 2008). After conducting the content validity test through experts, the adequacy and validity of the questions were tested to understand whether the anxiety areas and questions employed in the survey can provide insights into the level of anxiety suffered by stuttering adolescents among 13-18 years olds. The procedures of the this study are shown in Figure 1.
Schematic showing the development process of anxiety for stuttering adolescent
2. Subjects
To evaluate the content validity of the developed Korean Adolescent Stuttering Anxiety Test (KASAT), 12 experts in the field of speech-language pathology participated in the study. The experts involved in the study were those who had more than 10 years of experience in the field of speech-language pathology or teaching in university.
The subject of the facial validity was 15 middle school students and 15 high school students. As convergent validity, 13 middle school students and 13 high school students participated in the study.
The adolescents who participated in the present study are attending middle and high schools in Busan, Gyeongnam, and Daegu in Korea, do not suffer from communication disorders, and have not received speech language therapy services.
3. Test Tool Configuration and Development
We configured test of anxiety in adolescents who stutter, referencing the Scale of Social Anxiety for Adolescents (SAS-A, LaGreca, 1998, as cited in Oh & Moon, 2002) and a study that adapted it for Korean situations to validate social anxiety scales for children and adolescents adapted for Korea (Oh & Moon, 2002). Furthermore, the authors also used Korean Scale of Social Anxiety for Adolescents (K-SAS-A, Yang et al., 2008), the Screen for Child Anxiety Related Disorders (SCARED, Birmaher et al., 1997), as well as anxiety areas and questions from a validation study adapting the former scales for Korean circumstances (Kim, 2010). The anxiety test includes 12 questions about generalized anxiety related to excessive apprehension, 8 questions about social anxiety related to shyness, 6 questions about communicationrelated anxiety, and 6 questions about school-related anxiety. In total there are 32 questions. Since the test was directed towards adolescents, no questions about separation anxiety and school refusal were included, and considering that the subjects of the test suffer from stuttering, questions about communication-related anxiety were added. The primary test tool was produced after assessing the validity of the questions through non-stuttering children, and the secondary test tool was developed by revising the areas and questions of the anxiety test after having experts on youth analyze their validity.
4. Data Analysis and Statistic Processing
The following data analysis and statistical techniques were used to modify and remove questions that lower reliability and validity of the developed test tool.
First, non-stuttering children and experts on youths were asked to assess the validity of the areas and questions of the youth anxiety test in a 5-point scale. For questions indicated as being invalid, the evaluators were directed to provide comments. The results of content validity assessment provided by the non-stuttering children and experts were used to calculate the content validity index (CVI). Points were assigned to the content validity values (1 point = 0, 2 points = .25, 3 points = .50, 4 points = .75, 5 points = 1.00) and their average was calculated. Questions with a CVI of .80 or higher were deemed to have “key characteristics,” while those with a CVI of over .50 and .79 or under were considered as having “less important characteristics” and modified. Questions that had a CVI of .50 or lower where deemed as having bad validity were removed from the questionnaire (Fehring, 1987, as cited in Kim et al., 2008). In the current study, questions with a CVI under .80 were excluded, and those with a CVI under .90 were modified if there were comments from an expert.
Second, an analysis of internal consistency was conduced to assess whether the modified test of anxiety in Korean adolescents is reliable. More specifically, the areas and questions in the anxiety test were tested for internal consistency, during which questions with low question-area and question-total score relationship were assessed.
Ⅲ. Result
1. Content Validity
Experts found the following content validity values for the youth anxiety scale:
The content validity of questions about generalized anxiety in the anxiety scale for adolescents who stutter was between .65 and .90. Questions 1, 3, 4, 5, 6, and 8 were found to have “less important characteristics” according to their CVI values. Other questions were found to reflect key characteristics of generalized anxiety in adolescents. Experts also proposed that Question 1, “I sweat a lot when I am tense,” pertains to physiological reactions. In responses, the researchers added more aspects of anxiety in the question and modified it to, “My hair becomes white or I sweat a lot when I am tense.” Questions 4 and 8 were modified using more valid terms, in response to expert opinions.
CVI score of content validity about items in generalized anxiety
The content validity of questions about social anxiety in the anxiety scale for adolescents who stutter was between .53 and .92. Questions 14, 15, and 19 were found to have “less important characteristics” according to their CVI values. Other questions were found to reflect key characteristics of social anxiety in adolescents.
The researchers decided to delete Questions 14 and 19, determined to have “less important characteristics,” considering that they overlap with other questions. Question 15, however, had a low CVI because only one expert gave a low score. As such, the researchers decided to leave the question in consideration of other experts.
CVI score of content validity about items in social anxiety
The content validity of questions about communication anxiety in the anxiety scale for adolescents who stutter was between .67 and .82. Questions 22, 24, and 26 were found to reflect “key characteristics” of communication anxiety in adolescents. Questions 21, 23, and 25 were found to have “less important characteristics” according to their CVI values.
CVI score of content validity about items in communication anxiety
The content validity of questions about school anxiety in the anxiety scale for adolescents who stutter was between .67 and .82. Questions 27 and 28 were found to reflect “key characteristics” of communication anxiety in adolescents. No. Questions 29, 30, 31 and 32 were found to have “less important characteristics” according to their CVI values.
CVI score of content validity about items in school anxiety
2. Face Validity
After conducting the content validity assessment, the researchers worked with middle and high school students to conduct face validity assessment of the anxiety scale for stuttering adolescents, which yielded the following results: The score for generalized anxiety area was 4.22, that for social anxiety area was 4.32, that for community anxiety area was 4.04, and that for school anxiety was 4.4. Communication anxiety area was thought to be the most valid in measuring anxiety in adolescents.
The students provided other opinions about unfamiliar words in the questions. These words included “everyday affairs” in Question 2 and “faced with” in Question 10. The researchers changed the words “everyday affairs” to “daily life,” but did not for “faced with” due to the lack of another word that can replace it.
Score of face validity
3. Convergent Validity
After conducting the second content validity assessment, the researchers worked with middle and high school students to conduct convergent validity assessment to understand how much of their own anxiety characteristics are reflected in the questions, which yielded the following results: The question-total score internal consistency assessment resulted in a Cronbach’s Alpha value of .910, which indicates high correlation. Internal consistency among questions in each area is as follows: For generalized anxiety area, the Cronbach’s Alpha value is .841, for social anxiety area it was .708, for communication anxiety area it was .793, and for school anxiety area it was .710. In all four areas, the questions were found valid to test anxiety in adolescents.
Deleting the following questions increased the Cronbach’s Alpha value for their respective areas: Question 11 in generalized anxiety area, Question 18 in social anxiety area, and Question 27 for school anxiety area.
Ⅳ. Discussion
To develop an anxiety test for adolescents who stutter, the researchers of the current study developed questions, which were assessed for content validity by experts in the area. Using the results of the content validity assessment, the test was modified, after which face validity and convergent validity assessments were conducted. These results were in turn used to modify and supplement questions in the anxiety test for Korean adolescents who stutter. As such, the objective of the current study is to verify the validity of questions for evaluating stuttering anxiety in adolescents.
First, questions in the test were categorized into four areas: generalized anxiety, social anxiety, communication anxiety, and school anxiety. In general, tools for testing anxiety in adolescents look for issues such as generalized anxiety, social anxiety, separation anxiety, panic disorder and physical symptoms, and school anxiety (Kim, 2010, Brimaher et al., 1999), but stuttering may amplify anxiety due to various negative attributes related to difficulties in communication (Blood & Blood, 2007). To devise an anxiety test for adolescents who stutter, the researchers added an area to assess communication anxiety in the new test. In addition, areas for separation anxiety, panic disorder, and physical symptoms were excluded from the test in the current study, as they are not closely related to anxieties associated with stuttering and communication.
Most of the questions in the four areas received a CVI score of .70 or higher in a content validity assessment by experts, which indicates that they reflect characteristics of anxiety in adolescents who stutter. Question 14, “I am embarrassed when I am with people I do not know well” and Question 19, “I do not enjoy being with people I do not know well” received low CVI scores and were assessed by the experts as having “less important characteristics” in the social anxiety area. They were deleted for these reasons, as well as because they overlap with other questions.
Second, the results of the first content validity assessment was used to modify the questions in the anxiety scale for adolescents who stutter, and then face validity assessment was conducted on middle and high school students. The students’ average score of anxiety scale questions in all areas was 4.26. This indicates that all of the modified questions are adequate in terms of vocabulary, sentence structure, and content difficulty to be used for middle and high school students between ages 12 and 18.
The respondents indicated that questions in the school anxiety area are the most valid, while those in the communication anxiety area are the least valid. Furthermore, the students answered that Question 30, “I am worried about my school life” as being the most valid, while Question 19, “I am afraid of doing something with other people,” and Question 24, “I am afraid of saying certain words” as the least valid. The communication area was not included in previous youth anxiety testing tools. This is a category added by the current researchers in consideration of characteristics of people who stutter. The lack of this area in general anxiety tests also indicates that communication problems do not directly impact anxiety in non-stuttering youths. Other comments from the students were related to the vocabulary in the test. For instance, the respondents said that unfamiliar words such as “everyday affairs” and “faced with” should be changed, and in response the researchers made modifications to these terms.
Third, results from the content validity and face validity assessments were used to verify and modify the questions. The completely modified anxiety scale for adolescents who stutter was then implemented on middle and high school students. The internal consistency between the questions and the total score was .910, which indicates high correlation. The internal consistency of questions in each area indicates that questions in all category are valid to be used to test anxiety in adolescents. There were, however, questions that increased internal consistency of their respective areas when they were deleted. They were: Question 11 in generalized anxiety area, “It is hard for me to make a decision, "Question 18 in social anxiety area, “I am tense when other people watch me when I am doing something,” and Question 27 for school anxiety area, “When the teacher asks a question during class, I know the answer but cannot answer well.” That is, while the anxiety scale for adolescents has a high level of internal consistency as a whole, the internal consistency of each of the four areas categorized by the researchers was relatively low. When configuring the test initially, the researchers used concepts such as generalized anxiety, social anxiety, communication anxiety, and school anxiety. In the results of test conducted on actual adolescents, such types of anxiety were not clearly separated, with high correlation among anxiety categories. As such, it would be more prudent not to categorize the anxiety scale for adolescents into areas, and rather provide a total score of anxiety.
The authors of the current study are the first to develop an anxiety test for adolescents who stutter. Test tools for internalization problems in adolescents who stutter were restricted to assessing their communication attitudes, stuttering awareness, and self-efficacy. In order to develop a testing tool that can better address the internal characteristics of stuttering, the researchers conducted content validity assessment of test questions through experts and analyzed the validity of those results by asking middle and high school students. In the end, middle and high school students used the test to understand their level of anxiety. Such processes were implemented to determine validity of the questions from the perspective of experts and the tested youths, as well as from the viewpoint of the developers. Also, the process helped implement various ideas. This allowed the researchers to develop an anxiety test for adolescents who stutter that include more valid questions. The results of the current study should be utilized in the future to develop methods for processing results of this anxiety test for adolescents who stutter in a more systematic and refined manner.
Acknowledgments
This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2018S1A5A2A01034844).
이 논문은 년 대한민국 교육부와 한국 2018연구 재단의 지원을 받아 수행된 연구임(NRF-2018S1A5A2A01034844).
References
-
Birmaher, B., Brent, D. A., Chiappetta, L., Bridge, J., Monga, S., & Baugher, M., (1999), Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders(SCARED): A replication study, Journal of the American Academy Child and Adolescent Psychiatry, 38(10), p1230-1236.
[https://doi.org/10.1097/00004583-199910000-00011]
-
Birmaher, B., Khetarpal, S., Brent, D., Cully, M., Balach, L., Kaufman, J., & Neer, S. M., (1997), The Screen for Child Anxiety Related Emotional Disorders (SCARED): Scale construction and psychometric characteristics, Journal of the American Academy of Child and Adolescent Psychiatry, 36(4), p545-553.
[https://doi.org/10.1097/00004583-199704000-00018]
-
Blood, G. W., & Blood, I. M., (2007), Preliminary study of self-reported experience of physical aggression & bullying of boys who stutter: Relation to increased anxiety, Perceptual & Motor Skills, 104(3_suppl), p1060-1066.
[https://doi.org/10.2466/pms.104.3.1060-1066]
-
Craig, A., & Tran, Y., (2014), Trait and social anxiety in adults with chronic stuttering: Conclusions following meta-analysis, Journal of Fluency Disorders, 40, p35-43.
[https://doi.org/10.1016/j.jfludis.2014.01.001]
-
Iverach, L., Menzies, R. G., O’Brain, S., Packman, A., & Onslow, M., (2011), Anxiety and stuttering: Continuing to explore a complex relationship, American Journal of Speech-Language Pathology, 20(3), p231-232.
[https://doi.org/10.1044/1058-0360(2011/10-0091)]
-
Kim, H. J., Chang, H. J., Jeon, H. S., & Shin, M. S., (2012), Research on the development of a speaking ability self-efficacy test (SSE): A pilot study, Journal of Speech-language and Hearing Disorders, 21(2), p191-207.
김효정, 장현진, 전희숙, 신명선, (2012), 말하기 효능감 검사 개발을 위한 예비연구, 언어치료연구, 21(2), p191-207. -
Kim, H. J., Chang, H. J., Shin, M. S., & Jeon, H. S., (2010), Preliminary study for development of Korean version of Perception of Stuttering Inventory, Journal of Speech-Language and Hearing Disorders, 19(1), p129-142.
김효정, 장현진, 신명선, 전희숙, (2010), 말더듬 지각검사(PSI)의 국내적용을 위한 예비연구, 언어치료연구, 19(1), p129-142. -
Kim, J. O., & Shin, M. J., (2007), Preliminary study on the development Korean-subjective screening of stuttering-revised, Communication and Science Disorders, 12(7), p83-94.
김재옥, 신문자, (2007), 주관적 말더듬 선별검사의 국내적용 타당성 확보를 위한 기초연구, 언어청각장애연구, 12(7), p83-94. -
Kim, J. H., (2014), A case study on the Lidcombe program for a school-age child who stutters, Master's thesis, Catholic University of Daegu, Gyeongbuk.
김주희, (2014), 학령기 말더듬아동을 대상으로 하는 리드콤 프로그램 적용사례, 대구가톨릭대학교 보건의료과학대학원 석사학위논문. -
Kim, J. Y., Lee, E. J., & Hwang, M. A., (2008), The relationships between stuttering severity, perception of stuttering severity, and social anxiety of adults who stutter, Communication Science & Disorders, 13(4), p677-690.
김주연, 이은주, 황민아, (2008), 말더듬 성인의 말더듬 중증도와 말더듬 지각 정도 및 사회불안 간의 상관연구, 언어청각장애연구, 13(4), p677-690. -
Kim., M. J., (2010), A validation study of the SCARED: For the elementary school upper grades and middle school students, Master’s thesis, Sookmyung Women's University, Seoul.
김, 민정, (2010), 아동·청소년 불안장애 선별척도(SCARED)의 타당화 연구: 초등학교 고학년, 중학생을 대상으로, 숙명여자대학교 대학원 석사학위논문. -
Kwon, D. H., Kim, S. Y., Kim, H. J., Park, J. W., Shin, M. S., Ahn, J. B., Chang, H. J., Jeon, H. S., & Jeong, H., (2012), Fluency disorders, Seoul, Hakjisa.
권도하, 김시영, 김효정, 박진원, 신명선, 안종복, 장현진, 전희숙, 정훈, (2012), 유창성장애, 서울, 학지사. -
Oh, K. J., & Moon, H. S., (2002), A validation study of the Korean Social Anxiety Scale for Children and Adolescents, Korean Journal of Clinical Psychology, 21(2), p429-443.
오경자, 문혜신, (2002), 한국판 아동청소년용 사회 불안 척도의 타당화 연구, 한국심리학회, 21(2), p429-443. -
Sim, H. S., Shin, M. J., & Lee, E. J., (2010), Paradise-Fluency Assessment, 2nd ed., Seoul, Paradise Welfare Foundation.
심현섭, 신문자, 이은주, (2010), 파라다이스-유창성검사, 제2판, 서울, 파라다이스복지재단. -
Shin, M. S., Jeon, H. S., Kim, H. J., & Chang, H. J., (2013), Developing a Speaking Ability Self-Efficacy Scale(SSES), Journal of Speech-Language and Hearing Disorders, 22(4), p121-140.
신명선, 전희숙, 김효정, 장현진, (2012), 말하기 효능감 검사 개발을 위한 예비연구, 언어치료연구, 22(4), p121-140. -
Menzies, R. G., Onslow, M., & Packman, A., (1999), Anxiety and stuttering: Exploring a complex relationship, American Journal of Speech-Language Pathology, 8(1), p3-10.
[https://doi.org/10.1044/1058-0360.0801.03]
-
McAllister, J., Kelman, E., & Millard, S., (2015), Anxiety and cognitive bias in children and young people who stutter, Procedia - Social and Behavioral Sciences, 193, p183-191.
[https://doi.org/10.1016/j.sbspro.2015.03.258]
-
Yang, J. W., Yang, Y. R., Oh, K. J., (2008), Psychometric properties of the Social Anxiety Scale for Adolescents (K-SAS-A) in Korea, Korean Journal of Clinical Psychology, 27(4), p861-878.
양재원, 양윤란, 오경자, (2008), 아동, 청소년기 사회불안의 측정: 한국판 청소년 사회불안척도(K-SAS-A)의 타당화, 한국심리학회지: 임상, 27(4), p861-878.