Exploratory Factor Analysis: Applications in School Improvement Research 9781536124903, 1536124907

Table of contents :
""Contents""
""Preface""
""Chapter 1""
""Exploratory Factor Analysis: An Overview""
""Abstract""
""Introduction""
""Exploratory or Confirmatory Factor Analysis?""
""Common Factors or Principal Components?""
""The Common Factor Model""
""Are Data Appropriate for Factor Analysis?""
""Considerations Related to the Observed Variables""
""Considerations Related to the Sample""
""Steps and Decisions in the Implementation of Exploratory Factor Analysis""
""Selecting an Estimation Procedure""
""Determining the Number of Common Factors""
""Selecting a Rotation Method"" ""Optimizing Factor Solutions""""Computing Factor Scores""
""Conclusion""
""References""
""Chapter 2""
""Teacher Motivation and Recruitment in High School Special Education""
""Abstract""
""Introduction""
""Review of the Literature""
""Family History/Community Experience""
""Student Achievement""
""Personal Factors""
""Salary""
""Commitment""
""Marketing""
""Method""
""Sample""
""Instrumentation""
""Data Analysis""
""The Final Factor Solution""
""Estimating Coefficients""
""Interpreting the Factors""
""Results""
""Pilot Study""
""Survey Response""
""Factor Analysis"" ""Discussion""""Limitations""
""Recommendations for Future Research""
""Conclusion""
""References""
""Chapter 3""
""Clusters of Teachers Based on Their Perceptions of Learner-Centered Instruction""
""Abstract""
""Introduction""
""Literature Review""
""Learner-Centered Instruction (LCI)""
""Teacher Attitudes and Perceptions of LCI""
""Individual Factors Affecting Teachersâ#x80
#x99
Teaching Styles""
""Methods""
""Participants""
""Instrument""
""Data Analysis""
""Results""
""Descriptive Analysis""
""Exploratory Factor Analysis""
""Cluster Analysis""
""Chi Square Tests"" ""Discussion""""Appendix A""
""Appendix B""
""References""
""Chapter 4""
""Student Voice Opportunities Quantified by Exploratory Factor Analysis""
""Abstract""
""Introduction""
""Literature Review""
""Method""
""Sampling""
""Procedure""
""Setting""
""Instrumentation""
""Validity""
""Reliability""
""Data Screening""
""Data Analysis""
""Results""
""Descriptive Analysis""
""Reliability""
""Research Question 1""
""Research Question 2""
""Research Question 3""
""Research Question 4""
""Research Question 5""
""Discussion""
""References""
""Chapter 5"" ""Spiritual Health in Christian Schools: Implications for Academic Achievement and School Improvement""""Abstract""
""Introduction""
""Review of Literature""
""Method""
""Participants and Instrumentation""
""Data Analysis""
""Results""
""Research Question 1""
""Research Question 2""
""Research Question 3""
""Research Question 4""
""Discussion""
""Limitations""
""Recommendations for Future Research""
""Conclusion""
""Appendix A: Spiritual Health in Christian Schools: Implications for Academic Achievement and School Improvement""

Citation preview

EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD

EXPLORATORY FACTOR ANALYSIS APPLICATIONS IN SCHOOL IMPROVEMENT RESEARCH

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.

EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD Additional books in this series can be found on Nova’s website under the Series tab.

Additional e-books in this series can be found on Nova’s website under the eBooks tab.

EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD

EXPLORATORY FACTOR ANALYSIS APPLICATIONS IN SCHOOL IMPROVEMENT RESEARCH

DIANA MINDRILA EDITOR

Copyright © 2017 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. We have partnered with Copyright Clearance Center to make it easy for you to obtain permissions to reuse content from this publication. Simply navigate to this publication’s page on Nova’s website and locate the “Get Permission” button below the title description. This button is linked directly to the title’s permission page on copyright.com. Alternatively, you can visit copyright.com and search by title, ISBN, or ISSN. For further questions about using the service on copyright.com, please contact: Copyright Clearance Center Phone: +1-(978) 750-8400 Fax: +1-(978) 750-4470 E-mail: [email protected].

NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. Additional color graphics may be available in the e-book version of this book.

Library of Congress Cataloging-in-Publication Data ISBN: H%RRN

Published by Nova Science Publishers, Inc. † New York

CONTENTS Preface

vii

Chapter 1

Exploratory Factor Analysis: An Overview Diana Mindrila

Chapter 2

Teacher Motivation and Recruitment in High School Special Education Matthew D. Lawrence

27

Clusters of Teachers Based on Their Perceptions of Learner-Centered Instruction Yun-Jo An and Diana Mindrila

63

Student Voice Opportunities Quantified by Exploratory Factor Analysis Jennifer E. Elemen

87

Chapter 3

Chapter 4

Chapter 5

Spiritual Health in Christian Schools: Implications for Academic Achievement and School Improvement Chad A. McBane

1

121

vi Chapter 6

Contents Educator Technological Pedagogical Content Knowledge and Student Achievement Wofford L. Green

165

Editor’s Contact Information

199

Index

201

PREFACE In education, researchers often work with large, complex data sets that include a multitude of variables. One question that often arises in such contexts is whether these large sets of observed scores can be accurately summarized into their main components. Another question that may arise is whether the structure of associations that underlies the data represents distinct constructs, or features of a single construct. Researchers may address such questions by examining the correlations among observed variables, but patterns of association are difficult to detect through simple visual inspection, especially when data include large numbers of observed variables. Further, a simple visual examination cannot objectively asses the magnitude of the multivariate associations and does not provide sufficient evidence to support a perceived data structure. Exploratory factor analysis is a multivariate correlational procedure that helps researchers overcome such challenges. It helps reduce large data sets into main components or identify distinct constructs that account for the pattern of correlations among observed variables. These unobservable constructs are refered to as common factors, latent variables, or internal attributes, and they exert linear influences on more than one observed variable. Although exploratory factor analysis is widely used, many applied educational researchers and practitioners are not yet familiar with this

viii

Diana Mindrila

procedure and are intimidated by the technical terminology. This book provides a conceptual description of this method and includes a collection of applied research studies that illustrates the application of exploratory factor analysis in school improvement research. In addition to reporting the findings of their research, authors explained the factor analytic procedure and showed the applicability of the factor analytic results in practice or to inform further research. The first chapter provides a theoretical overview of exploratory factor analysis. It explains the purposes for which this procedure can be used, the related terminology, the distinction between key concepts, the steps that must be taken, and the criteria for making the decisions. This information can serve as a starting point for the applied researcher who needs a brief, conceptual introduction on this topic. The following chapters present a series of research studies in which exploratory factor analysis was employed either by itself or in conjunction with other statistical procedures. The study presented in the second chapter aimed to identify the factors that compel high school special educators to choose careers in high school special education. The current literature shows that schools encounter great challenges in attracting and retaining highly qualified and motivated high school special educators. Survey data collected from 155 high school special educators in a southeastern state yielded four distinct factors that summarize the reasons for becoming high school special educators: a) “Students/clientele,” b) “societal impact,” c) “professional security,” and d) “personal compatibility.” The results of factor analysis indicate that important personal characteristics exist among highly-motivated high school special education teachers. University schools of education, as well as secondary school systems, should undertake to identify these characteristics in prospective educators. If such efforts are pursued early in the process of teacher training, the ability of the American public school system to recruit and retain high quality special educators will be greatly enhanced. The study presented in the third chapter aimed to identify the factors that summarize teachers’ perceptions of learner-centered instruction (LCI)

Preface

ix

and to distinguish clusters of teachers based on their attitudes towards LCI. Further, the study examined the demographic characteristics of the identified clusters, as well as the distribution of the clusters across organizational levels. The sample consisted of 134 K-12 teachers in a southeastern state. Data were collected using a survey consisting of 14 items on a 5-point Likert-scale measuring teachers’ beliefs about the effectiveness of LCI, current practices in creating technology-enhanced, learner-centered classrooms, and barriers to creating technology-enhanced, learner-centered classrooms. Exploratory factor analysis yielded two common factors: a) “learner-centered teacher (LCT),” which included items measuring the extent to which teachers adopt and have knowledge of LCI instruction and b) “concerns about LCI (CLCI),” which included items referring to the concerns that teachers may have in regard to the implementation of LCI, such as the lack of time, the reduced amount of content that can be covered, the increased amount of work required, the lack of student responsiveness to this instructional approach, and incompatibility with one’s subject area. Results from exploratory factor analysis were used to develop a typology of teachers based on their perceptions of LCI. This typology differentiated four categories of teachers based on their attitudes towards LCI: a) Average, b) High LCT, c) High CLCI, and d) Low LCT. A chi-square test showed that the proportion of middle-school teachers was significantly greater in the High CLCI cluster (χ2(6)=12.483, p=.05), indicating increased concerns and reduced implementation of LCI at this organizational level. These findings offer helpful information to practitioners who provide professional development on LCI, particularly at the middle school level. The study presented in the fourth chapter analyzed school leadership praxis for its inclusion of students in organizational leadership dialogue and decision-making, and their influences on student achievement and civic participation. Survey questionnaire data were provided by 215 fulltime enrolled undergraduate students from a public southeastern university about their high school and college experiences. Utilizing exploratory factor analysis, two factors of school leadership and culture were identified: a) participative decision-making; and b) organizational

x

Diana Mindrila

leadership dialogue. These factors were found to possibly influence student achievement and civic participation. The fifth chapter illustrates the application of exploratory factor analysis in researching the effectiveness of faith-based schools. This study used a data-driven approach to investigate the concept of spiritual health in middle school students attending two private Christian schools. A sample consisting of approximately 200 middle school students attending private Christian schools was surveyed using the Spiritual Health Survey, a thirty question instrument on a six-point Likert scale. Three factors of spiritual health emerged through an exploratory factor analysis of the survey data: a) the relationship with God; b) relationship with others; and c) confident testimony. The identification of these factors indicates that respondents perceived spiritual health as a multidimensional construct. The study also reports differing levels of spiritual health among students based on family dynamics, church attendance, and schools attended. Findings also show that spiritual health has a significant relationship with academic performance, explaining two percent of the variation in student GPA. These results indicate that schools should be strategic in their formal improvement planning, and target specific aspects of student spiritual growth. Finally, the sixth chapter summarizes a study that aimed to develop a typology of teachers within the Technological Pedagogical Content Knowledge (TPACK) framework and to identify the groups of teachers that perform better in regards to student achievement as measured by state mandated standardized assessment. A sample of 252 educators from a rural southeastern school district completed a 50-question TPACK survey. Exploratory factor analysis was used to identify underlying factors. Results yielded a 3-factor solution: a) instructional practice; b) technology integration; and c) technology knowledge. Factor scores were then used as input for k-means cluster analysis, which yielded three teacher profiles: a) High-TPACK, b) Mid-TPACK, and c) Low-TPACK. Demographic data of educators such as age, level of education, and years of experience were tabulated by cluster. Results showed that the High-TPACK cluster (N=96) were made up of younger, predominantly male teachers. The Mid-TPACK

Preface

xi

(N=143) cluster skewed older with a majority of female educators. The Low-TPACK cluster (N=14) consisted of educators towards the end of their careers. A chi-square test showed that levels of student achievement differed significantly across the three clusters (χ2(8)= 15.851, p=.045) due to more students performing at a higher level on standardized assessment for teachers in the High-TPACK Cluster. Further, a t-test showed that students performed significantly higher for the teachers in the HighTPACK cluster than in the Mid-TPACK cluster (t(97)=-3.045, p=.004). These results indicate that teachers with higher levels of TPACK have students who perform better on standardized assessment. Suggestions were made for teacher preparation programs as well as professional development to target the skills needed to get all teachers to the level of the High-TPACK cluster. In conclusion, this collection of studies can serve as a reference for the applied researcher seeking examples of factor analytic research in the field of school improvement. The studies presented in this book address a variety of research problems and are among the few using a quantitative approach and exploratory factor analysis to investigate their research topic. They specify how the factor analytic procedure was applied, and explain the theoretical contributions and the practical applications of the factor analytic results. In most studies, results from factor analysis were used for subsequent statistical procedures, thus helping researchers address more complex research questions and enriching the results.

In: Exploratory Factor Analysis Editor: Diana Mindrila

ISBN: 978-1-53612-486-6 © 2017 Nova Science Publishers, Inc.

Chapter 1

EXPLORATORY FACTOR ANALYSIS: AN OVERVIEW Diana Mindrila*, PhD University of West Georgia, Carrollton, GA, US

ABSTRACT In education, researchers often work with large, complex data sets that include a multitude of variables. One question that often arises in such contexts is whether these large sets of observed scores can be accurately summarized into their main components. Another question that may arise is whether the structure of associations that underlies the data represents distinct constructs, or features of a single construct. Exploratory factor analysis is a multivariate correlational procedure that helps researchers overcome such challenges. It helps reduce large data sets into main components or identify distinct constructs that account for the pattern of correlations among observed variables. These unobservable constructs are refered to as common factors, latent variables, or internal attributes, and they exert linear influences on more than one observed variable. Although exploratory factor analysis is widely used, many *

Corresponding Author Email: [email protected].

2

Diana Mindrila applied educational researchers and practitioners are not yet familiar with this procedure. This chapter provides a theoretical overview of exploratory factor analysis. It explains the purposes for which this procedure can be used, the related terminology, the distinction between key concepts, the steps that must be taken, and the criteria for making the decisions at every step. This information can serve as a starting point for researchers who need a conceptual description of this method.

Keywords: exploratory factor analysis

INTRODUCTION Although exploratory factor analysis was developed more than one century ago (Spearman, 1904, 1927) and is widely used in social science research, many applied researchers are not yet familiar with this statistical method. Factor analysis is a generic term given to a set of multivariate statistical procedures aiming to reveal the structure of associations underlying a data set (Fabrigar & Wegener, 2011). These procedures analyze the pattern of relationships among many observed variables such as test scores or survey questions. The goal is to identify a set of underlying dimensions that are referred to as common factors and to estimate the amount of variance in the observed variables explained by each factor (Hair, Anderson, Tatham, & Black, 1998). For instance, factor analysis can be used to identify the factors that influence student or teacher responses on a school climate questionnaire. Similarly, factor analysis can be used to determine the constructs assessed by an achievement test. As a multivariate technique, factor analysis has the advantage of accommodating multiple variables in an attempt to understand complex patterns of relationships. This is not possible with univariate and bivariate methods. Some multivariate methods such as multiple regression, multivariate analysis of variance, discriminant analysis, or canonical correlations are considered dependence techniques, because they are focused only on the relationships between the independent and the predicted variables. In contrast, factor analysis is an interdependence

Exploratory Factor Analysis: An Overview

3

technique which examines the relationships between each variable and all other variables. The resulting factors aim to explain the relationships present in the entire data set, not to predict a dependent variable (Hair et al., 1998).

EXPLORATORY OR CONFIRMATORY FACTOR ANALYSIS? Factor analytic procedures can be used either for exploratory of confirmatory purposes. Both exploratory factor analysis and confirmatory factor analysis are based on the common factor model and seek to estimate the structure of correlations among observed variables based on a smaller set of latent variables (Fabrigar, Wegener, MacCallum, & Strahan, 1999). Exploratory factor analysis is searching for structure among a series of observed variables, or can be used as a data reduction technique (Hair et al.,1995). These techniques are employed when researchers have no a priori knowledge about the number of factors that underlie the data and about which variables are included in each factor (Fabrigar et al., 1999). Confirmatory factor analytic procedures can be used when researchers have some prior knowledge about the actual structure of the data either from prior research or extant theory (Finch & West, 1997). In this case, factor analysis is used to test a hypothesis referring to the precise number of factors and to the variables that should be grouped under each factor (Hair et al., 1998). Exploratory and confirmatory factor analysis can also be used in conjunction with one another. The exploratory procedures are employed first to identify the factor structure; the factor model is then tested in a subsequent study through confirmatory factor analysis. When collecting data from a new sample is not feasible, the initial sample, if sufficiently large, can be split in half: one half can be used for exploratory factor analysis whereas the other half can be used to assess the model fit (Fabrigar et al., 1999).

4

Diana Mindrila

COMMON FACTORS OR PRINCIPAL COMPONENTS? One of the primary uses of factor analysis is summarization and data reduction, which can be achieved by calculating overall scores for each underlying dimension and substituting them for the original set of observed variables (Hair et al., 1998). Exploratory factor analysis can also be used to identify a number of latent dimensions that underlie the data. Before employing exploratory factor analysis, researchers must determine whether the goal of the study is data reduction or the identification of latent constructs (common factors). Through data reduction techniques, a set of observed variables is replaced with a composite score that retains as much information as possible from the original variables. Principal components are not designed to model the structure of correlations among observed variables (Fabrigar et al., 1999). The estimation of common factors should be used when the researcher investigates latent constructs that are believed to exert linear influences on sets of observed variables (Fabrigar & Wegener, 2011). Exploratory factor analysis is based on the common factor model, which postulates that each observed variable is a linear function of one or more common factors and one unique factor (Thurstone, 1947). Common factors are also referred to as latent variables, because they cannot be directly observed; they represent a common feature that explains the variance shared by the observed variables, whereas the unique factor accounts for the unique variance of each observed variable. In contrast, data reduction techniques such as principal component analysis do not make the distinction between common and unique variance. They are linear composites of the original variable and thus contain both unique and common variance. Although many researchers believe principal component analysis is a type of exploratory factor analysis, the two procedures are based on different assumptions, use different computational methods, and are designed to achieve different objectives (Fabrigar, et al., 1999).

Exploratory Factor Analysis: An Overview

5

THE COMMON FACTOR MODEL The common factor model was proposed by Thurstone (1935, 1947) as a general mathematical framework that explains the pattern of correlations among observed variables, which are also referred to as measured variables, manifest variables, or surface attributes. These terms refer to any attributes that can be directly measured such as academic skills or attitudes. A set of observed variables is often referred to as battery (Fabrigar & Wegener, 2011). The common factor model is based on the premise that several observed variables may be correlated because they are all influenced by an unobserved construct. Such constructs are referred to as common factors, latent variables, or internal attributes and have linear influences several surface attributes (Fabrigar & Wegener, 2011). The common factor model also specifies the presence of unique factors, which are also unobservable but influence only one observed variable. In other words, each observed variable is influenced by a unique factor, which is assumed to be unrelated to the other unique factors. Further, a unique factor can be partitioned into a specific factor and error of measurement. The specific factor is a systematic source of variance exerted on a particular observed variable, whereas the error of measurement refers to random influences on the observed variable. Based on these premises, the variance of each observed variable can be partitioned into common variance and unique variance; further, the unique variance can be divided into specific variance and error variance. A term that is frequently used in factor analysis is that of communality, which represents the proportion of the observed variance accounted for by the common factors. Communalities are computed by dividing the common variance to the observed variance or by subtracting the unique variance divided by the observed variance from one (Fabrigar & Wegener, 2011). Mathematically, the common factor model is represented using matrix algebra. In its matrix form, the structure of correlations can be represented as: P = ΛΦΛT+DΨ

(1.1)

6

Diana Mindrila

Table 1. Λ matrix for a two-factor model with five observed variables Factor 1 Λ1,1 Λ2,1 Λ3,1 Λ4,1 Λ5,1

OV1 OV2 OV3 OV4 OV5

Factor 2 Λ1,2 Λ2,2 Λ3,2 Λ4,2 Λ5,2

In the equation above, P represents the population correlation matrix of the observed variables; the matrix Λ is referred to as the matrix of factor loadings and indicates the strength and direction of linear influences exerted by the common factors on the observed variables. The matrix of factor loadings for a model with two common factors and five observed variables is represented in Table 1. The notation ΛT is used for the transpose of the Λ matrix. In matrix algebra, the transpose of a matrix is obtained by representing rows as columns (Table 2). Similar to the Pearson’s r, a factor loading is the square root of the variance in the observed variable that can be explained by a common factor. As a general rule, loadings must have absolute values larger than .32 to be considered significant indicators of a common factor (Tabachnick & Fidell, 2001), and factors should have at least three observed indicators (Costello & Osborne, 2005). Table 2. ΛT matrix for a two-factor model with five observed variables

Factor 1 Factor 2

OV1 Λ1,1 Λ1,2

OV2 Λ2,1 Λ2,2

OV3 Λ3,1 Λ3,2

OV4 Λ4,1 Λ4,2

Table 3. Φ matrix for a two-factor solution

Factor 1 Factor 2

Factor 1 1.0 Φ2,1

Factor 2 1.0

OV5 Λ5,1 Λ5,2

Exploratory Factor Analysis: An Overview

7

The Φ matrix in Equation 1.1 is the matrix of correlations among the common factors. Table 3 represents the factor correlation matrix for a model with two common factors. When researchers assume that factors are not correlated (orthogonal), this matrix can be omitted from the estimation of the factor model, as indicated in equation 1.2. P = ΛΛT+DΨ

(1.2)

The matrix D in the equations 1.1 and 1.2 is the matrix of covariance among the unique factors, as represented in Table 4. The diagonal elements of this matrix represent the variances of the unique factors, whereas the off-diagonal elements represent covariances among unique factors. Because the unique factors are considered unrelated, the off-diagonal elements are assumed to be null. In conclusion, equation 1.1 indicates that the population correlation matrix is estimated by multiplying the matrix of factor loadings by the factor correlation matrix and the transpose of the matrix of factor loadings and adding the covariance matrix of unique factors. Therefore, each observed variable is a function of the sum of variance explained by each common factor in the model and the variance explained by its unique factor. Table 4. D matrix for five observed variables

OV1 OV2 OV3 OV4 OV5

OV1 DΨ1,1 0 0 0 0

OV2

OV3

OV4

OV5

DΨ2,2 0 0 0

DΨ3,3 0 0

DΨ4,4 0

DΨ5,5

8

Diana Mindrila

ARE DATA APPROPRIATE FOR FACTOR ANALYSIS? Considerations Related to the Observed Variables When considering the use of factor analytic procedures, the researchers must examine the properties of the observed variables as well as the properties of the sample (Fabrigar & Wegener, 2011). First, the selected observed measures must be adequate indicators of the domain of inquiry. When these measures are not the most relevant or do not cover all important aspects of the topic of inquiry the validity of the factor structure is undermined, and the strength of the factors may be underestimated. Factor analytic procedures are most accurate when factors are overdetermined, which means that multiple observed variables are influenced by each factor (Fabrigar & Wegener, 2011). For this reason, each factor should be represented by at least three to five observed measures (MacCallum, Widaman, Zhang, & Hong, 1999). A high degree of overdetermination is obtained when the model includes few factors and each factor is measured by six or seven observed variables, and many or all communalities are above .50 (MacCallum et al., 1999). In conclusion, when designing a factor analytic study, researchers should try to estimate the maximum number of factors and use at least five observed measures for each factor (Fabrigar & Wegener, 2011). Another important consideration in a factor analytic study is the quality of the measurement instrument. Factor analytic procedures work best when communalities among observed variables are high, which means that observed variables should have reduced levels of random error (Fabrigar & Wegener, 2011). Finally, a very important consideration is the scale of measurement and the distribution of the observed variables. As indicated earlier, the common factor model postulates linear influences of the factors on the observed variables, which assume that variables are continuous or on a quasiinterval scale (Floyd & Widaman, 1995) and have a normal distribution. In addition to univariate indices of skewness and kurtosis, researchers must also examine the extent to which data are multivariate normal. In general,

Exploratory Factor Analysis: An Overview

9

univariate skewness coefficients above 2, univariate kurtosis coefficients above 7, and a value of Mardia’s coefficient of multivariate kurtosis above 3 are considered indicative of non-normality (Finney & DiStefano, 2006). Nevertheless, more recent factor analytic estimation procedures provide accurate results with both ordinal and non-normal data; therefore, when working with non-normal or ordered categorical data researchers must select an appropriate estimation procedure (Finney & DiStefano, 2006). Multicollinearity is another important assumption in factor analysis. Multicollinearity refers to the correlation among a set of observed variables. Some degree of multicollinearity is desirable and indicates that “true” factors may underlie the data. Before conducting exploratory factor analysis, researchers must inspect the correlation matrix of the observed variables making sure there is a substantial number of correlations larger than .30 (Hair et al., 1998). The degree of multicollinearity can be assessed by computing partial correlations among variables (Hair et al., 1998). These coefficients represent the correlation between variables when other variables are taken into account. Statistical software such as SAS or IBM SPSS allow researchers to compute the anti-image correlation matrix, which consists of the negative values of the partial correlations. When the off-diagonal elements of this matrix have high absolute values, no “true” factors underlie the data and, therefore, factor analysis is not appropriate. Further, the Bartlett test of sphericity can be used to examine the entire correlation matrix for significant correlations among at least some of the variables. A significant test statistic indicates that data are sufficiently correlated for factor analysis. A limitation of this test is that it is more likely to be significant with large sample sizes (Hair et al., 1998). Another way to quantify the interrelationships among observed variables is examining the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy. This index ranges from 0 to 1. Values .80 or above can be considered good, whereas values below .50 are unacceptable and require remedial action either by removing offending variables or by including other related variables (Kaiser, 1970; Dziuban & Shirkey, 1974; Cerny & Kaiser, 1977).

10

Diana Mindrila

Although some degree of multicollinearity is desirable, high degrees of multicollinearity may become problematic, because they indicate the presence of redundant information. The lack of distinction among observed variables may induce estimation bias or cause estimation or convergence problems. The extreme case of multicollinearity, where the variables are perfectly correlated, is called singularity. Complete multicollinearity is assumed to be absent in factor models because it will result in singular covariance matrices, which cannot be used to perform certain calculations because division by zero will occur. Very high multicollinearity can result in matrix entries which approach 0 and while division can occur, results will be unstable. Hence, complete or very high multicollinearity prevents a factor solution. Although multicollinearity issues have been investigated in the context of multiple regression, there is little research on the impact of high degrees of multicollinearity on factor models and structural equation models (Grewal, Cote, & Baumgartner, 2004). Signs of high degrees of multicollinearity may be non-significant coefficient estimates when the overall model is significant, unexpected signs of the coefficients, unstable parameter estimates, path coefficients larger than 1, or high correlations among variables. The inspection of the correlation matrix reveals only bivariate multicollinearity, for bivariate correlations larger than .90. To assess multivariate multicollinearity, researchers may use tolerance, the variance inflation factor, or the Haitovsky’s heuristic chi-square test. Tolerance is the difference between one and R2, in a regression model in which one observed variable is regressed on all the other variables; therefore, there will be as many tolerance coefficients as observed variables. As the intercorrelation among variables increases, the value of the tolerance coefficient decreases. Specifically, values below .2 indicate high degrees of multicollinearity (Menard 1995). The variance inflation factor (VIF) is the reciprocal of tolerance and indicates whether one variable has strong relationships with the other variables. High VIF values indicate high levels of multicollinearity. Although there are no general rules about the values of VIF that should be

Exploratory Factor Analysis: An Overview

11

considered problematic, some authors suggested that values greater or equal to 10 should cause concern (Myers, 1990), and values greater than one may induce estimation bias (Bowerman & O’Connell, 1990). Both VIF and tolerance coefficients can be computed using the IBM SPSS software. Another way to assess multicollinearity is the Haitovsky’s heuristic chi-square test. This procedure tests the hypothesis that the determinant of the correlation matrix is zero, which indicates that the matrix is singular. Values closer to 0 indicate the existence of multicollinearity, while values close to 1 indicate that variables are orthogonal (Rockwell, 1975).

Considerations Related to the Sample Textbooks on factor analysis often specify rules of thumb regarding the sample size needed for this procedure. In addition to a minimum sample size, such rules also specify a required individuals-to-variable ratio. Such recommendations varied greatly across researchers. For instance, Gorsuch (1983) recommended a sample size of at least 100, with at least five participants per observed variable. In contrast, other researchers recommended much larger sample sizes such as 300 (Norusis, 2005) or 500 (Comrey & Lee, 1992). Similarly, recommendations regarding the individuals-to-variable ratio varied greatly across studies. Some authors proposed large ratios such as 20:1 (Hair et al., 1995) or 10:1 (Everitt, 1975; Nunnally, 1978; Kunce, Cook, & Miller, 1975; Marascuilo & Levin, 1983), whereas others recommended a ratio of 5:1 (Bryant & Yarnold, 1995; Gorsuch, 1983; Everitt, 1975), 3:1 – 6:1 (Cattell, 1978), or 2:1 (Kline, 1979). Although generally regarded as a statistical procedure that requires a large sample size, more recent studies show that absolute thresholds are inappropriate. Mundfrom, Shaw & Ke (2005) showed that factor analysis can produce reliable outcomes with samples smaller than 50 when communalities are high and the number of factors is small. Further, Gagne and Hancock (2006) found that a sample size of 25 yielded no

12

Diana Mindrila

Heywood cases1 and no instances of nonconvergence with two factors and factor loadings above .8. Zeller (2006) concluded that a sample size between 10 and 50 was sufficient for two factors and 20 observed variables. Similarly, Preacher & MacCallum (2002) showed a sample size of 10 can yield reliable results. Furthermore, recent developments in statistical software allow researchers to employ model fitting procedures that reduce estimation bias and can produce accurate results with small sample sizes. The sampling selection procedure should also be considered when employing exploratory factor analysis. Random selection reduces bias and enhances the generalizability of the findings; however, in may situations selecting a simple random sample is not feasible. Convenience samples can be used as long as the sample bias is not strongly related to the investigated constructs (Fabrigar & Wegener, 2011). For instance, if the investigated construct is academic achievement but the participating students are selected only from advanced placement classes, the findings related to academic achievement are likely to be biased. In contrast, a study investigating psychomotor skills with the same students may yield more generalizable results. Another aspect that researchers must consider is the issue of missing values. Factor analysis assumes that the data set contains a full set of observations for each individual. When this is not the case, individuals are completely removed from the analysis (listwise deletion). When a case has numerous missing values, this solution is most appropriate; however, when many individuals are removed, the size and the representativeness of the sample may be affected. To avoid this problem, researchers may chose, when appropriate, to impute missing values with the mean from the overall sample or the mean from a random individual within the sample (Gorsuch, 1983). Alternatively, more complex imputation procedures such as regression-based methods (Timm, 1970), or maximum likelihood methods

1

The term “Heywood case” refers to negative error variances or squared multiple correlations larger than one. Such estimates may occur when the model is misspecified (van Driel, 1978) or the assumptions of the common factor model are violated (Velicer & Jackson, 1990).

Exploratory Factor Analysis: An Overview

13

(Arbuckle, 1996; Figueredo, McKnight, McKnight, & Sidani, 2000) can be used to impute missing values.

STEPS AND DECISIONS IN THE IMPLEMENTATION OF EXPLORATORY FACTOR ANALYSIS Exploratory factor analysis is a stepwise technique and, at each step, researchers must decide how to proceed. Assuming that the common factor model is most appropriate for the research question and the data are suitable for exploratory factor analysis, the first step is to select the most appropriate estimation procedure. Next, the researchers must determine the number of common factors that underlie the data and select the most appropriate rotation procedure.

Selecting an Estimation Procedure In the initial stage, researchers must select the appropriate estimation procedure, also referred to as the factor extraction procedure or the model fitting procedure. Statistical software allow researchers to choose from a wide range of estimation methods. Some of the most frequently used model fitting procedures are non-iterated principal axis factoring, iterated principal axis factoring, and maximum likelihood (Fabrigar & Wegener, 2011). Although these estimation methods differ in the computational approach, they assume the same common factor model and, therefore, often produce similar results (Fabrigar & Wegener, 2011); however, when the assumptions of the common factor model are not met, some estimation procedures can be more effective than others (Finney & DiStefano, 2006). Maximum likelihood is very frequently used and is the default estimation procedure in most statistical software. The main feature of this method is the likelihood function, which estimates the likelihood of the specified model given the set of observed values. The resulting estimates

14

Diana Mindrila

are the ones maximally likely to have produced the data for the specified factor model (Fabrigar & Wegener, 2011); therefore, this estimation procedure extracts the maximum amount of variance from the observed variables. It is based on the assumptions that observations are randomly selected, continuous, and have a multivariate normal distribution. Nevertheless, this procedure can provide accurate results with ordered categorical data with at least five categories, and when the assumption of multivariate normality is not severely violated (Finney & DiStefano, 2006). In contrast, non-iterated and iterated principal axis factoring do not rely on distributional assumptions and, therefore, provide more accurate results with non-normal data. Although principal axis factoring methods provide a limited range of goodness of fit indices, they are less likely to yield improper solutions, to produce Heywood cases, or to fail to converge (Fabrigar et al., 1999). More recent developments in statistical software allow researchers to employ estimation methods that are more computationally complex and do not require large sample sizes, continuous variables, or multivariate normal distributions (Finney & DiStefano, 2006). For instance the Mplus 7 software allows researchers to use the Bayes estimation. This procedure employs Markov Chain Monte Carlo algorithms which use the conditional distribution of a set of parameters given other sets to make random draws of parameter values and approximate their joint distribution (Muthén & Asparouhov, 2010). Bayesian estimation incorporates prior information to increase estimation accuracy by reducing the number of Heywood solutions (Lee, 1981; Martin & McDonald, 1975; Mayekawa, 1985). Unlike maximum likelihood, Bayesian estimation does not assume a multivariate normal distribution and performs better with small sample sizes (Schmitt, 2011; Heerwegh, 2014; Asparouhov & Muthén, 2010). Further, the mean- and variance-adjusted weighted least squares (WLSMV) estimation method, which was considered the most effective procedure for ordered categorical and severely non-normal data (Finney & DiStefano, 2006) was outperformed by Bayesian estimation with categorical and ordinal data (Asparouhov & Muthén 2010). Another

Exploratory Factor Analysis: An Overview

15

advantage of the Bayesian procedure is that it allows the computation of complex factor structures, covering a wider range of models than maximum likelihood (Schmitt, 2011; Asparouhov & Muthén 2010).

Determining the Number of Common Factors Another important decision in exploratory factor analysis is determining how many common factors underlie the data. On the first extraction, the number of factors is determined by first identifying the combination of variables that extracts the largest amount of variance, and then proceeding to the combinations that account for smaller and smaller proportions of variance (Hair, et al., 1995). The question is how many of these combinations of variables should be included in the final factor solution. An ideal factor solution should be parsimonious and explain the maximum amount of variance with the least number of factors (Fabrigar et al., 1999); therefore, researchers should stop factoring when the amount of variance explained does not increase much by including an additional factor. There are several criteria that help researchers decide on the number of factors to extract, such as the latent root criterion, the examination of a scree plot, parallel analysis, the examination of descriptive indices of model fit, and the interpretability of the factors. The latent root criterion, also referred to as the Kaiser criterion or the number of eigenvalues larger than one is the most commonly used and the default criterion when running exploratory factor analysis with the IBM SPSS software. In factor analysis, eigenvalues reflect the variance in the correlation matrix thus factors with the largest eigenvalues explain the most variance (Gorsuch, 1983). Only eigenvalues larger than one are considered significant (Guttman, 1954; Kaiser, 1960), and factors with small or negative eigenvalues are usually excluded from factor solutions (Tabachnick & Fidell, 1996). Nevertheless, this procedure is often misapplied, can be arbitrary (Fabrigar et al., 1999) , and was shown to often lead to overfactoring or underfactoring (Cattell & Jaspers,1967; Zwick & Velicer, 1982, 1986); therefore, researchers should use the Keiser

16

Diana Mindrila

criterion in conjunction with other criteria to determine the number of common factors. Another criterion that is frequently used is the scree test (Cattel, 1966; Cattell & Jaspers, 1967). The eigenvalues of the correlation matrix are represented on a scree plot in descending order. The plot is then examined to determine the last substantial difference in eigenvalues from one factor solution to the next, or the point where the curve becomes flat. For instance, the scree plot in Figure 1 indicates that a factor solution with four factors could be optimal. Although this procedure has been criticized for it subjectivity (Kaiser, 1970) and ambiguity when no substantial drop in eigenvalues can be detected (Fabrigar et al., 1999), research indicates that it provides reliable results (Cattel & Vogelman, 1977). Parallel analysis can also be used to determine the number of factors. (Horn, 1965; Humphreys & Montanelli, 1975). This procedure compares the eigenvalues from the sample to the eigenvalues that would be obtained from repeated sets of completely random data. Although simulation studies suggest that this procedure is fairly accurate (Humphreys & Montanelli, 1975), it can also be arbitrary and is not included in major statistical software packages (Fabrigar et al., 1999).

Figure 1. Scree plot.

Exploratory Factor Analysis: An Overview

17

Exploratory factor analysis is a special case of structural equation modeling; therefore, the criteria used for the selection of structural models an also be applied to exploratory factor models. Some statistical software such as LISREL, SAS, or Mplus allow the computation of a set of goodnes-of-fit indices, which indicate the extent to which the factor model fits the data. Such indices are the likelihood ratio statistic (Lawley, 1940), the root mean square error of approximation (RMSEA) (Steiger & Lind, 1980), or the expected cross-validation index (ECVI) (Browne & Cudeck, 1989). The likelihood ratio statistic follows a chi-square distribution and non-significant values indicate good fit. However, this index is highly sensitive to model size and sample size (Finney & DiStefano, 2006) and is rarely non-significant. The RMSEA estimates the discrepancy between the specified factor model and the observed data; therefore, smaller values indicate better model fit. Values smaller than .10 indicate marginal fit, values below .80 indicate acceptable fit, whereas values below .50 indicate good fit (Browne & Cudeck, 1992). The ECVI estimates the extent to which the solution obtained from the current sample can be generalized to other samples. Although there are no guidelines for the interpretation of ECVI, when estimating models with different numbers of factors, smaller ECVI values indicate model better fit; a limitation of this index is that it is sensitive to sample size and less complex models are more likely to be retained (Browne & Cudeck, 1992). In addition to statistical criteria for the inclusion or removal of factors, researchers must also take into account the interpretability of the factors included in the final solution. They must examine the observed variables included in each common factor and determine whether they describe a specific construct. Such decisions are informed by theory and help ensure that results from factor analysis are meaningful and supported by both statistical and theoretical evidence.

18

Diana Mindrila

Selecting a Rotation Method The most important criterion used to select the optimal factor solution is that of a simple structure (Thurstone, 1947). Factor solutions have a simple structure when each factor is clearly represented by a subset of observed measures, and each observed variable relates to only one factor (factor loadings above .32 on one factor and below .32 on the other factors). To obtain a simple structure, estimation procedures are followed by factor rotation. The term rotation indicates that the axes of the factors are turned until another position is reached (Hair et al., 1998). This procedure aims to simplify, or clarify the factor structure, but cannot improve basic aspects of the factor solution, such as the amount of variance explained (Costello & Osborne, 2005). Based on the nature of the relationships between the common factors, researchers can choose an orthogonal or an oblique rotation procedure. Orthogonal rotation procedures assume that factors are not correlated and factor correlations are, therefore, fixed to zero. In contrast, oblique rotation procedures allow the estimation of factor correlations, regardless of the assumed relationship between factors (Fabrigar, et al., 1999). The most common orthogonal rotation is varimax, but other methods such as equamax or quartimax are commonly available. Some of the most frequent oblique rotations are direct oblimin, quartimin, promax, or geomin. From a mathematical standpoint, orthogonal solutions are less likely to produce a simple structure when the distance between observed variables in the multidimensional space is less than 90 degrees (Fabrigar, et al., 1999). Further, oblique rotation methods provide more information by indicating which factors are correlated, and the strength and direction of the factor correlations. Orthogonal rotation procedures yield results that are more easily interpretable but in social sciences, due to the complex nature of human behavior, factors are most often correlated; therefore, the use of orthogonal rotation procedures may lead to a loss of information. Due to the disadvantages of orthogonal rotation procedures, there is little justification for using these methods in social science research. Instead, researchers may first employ an oblique rotation method and, if all

Exploratory Factor Analysis: An Overview

19

factor correlations are negligible, an orthogonal rotation procedure can be used to improve the factor structure. Nevertheless, if at least one relationship exists between factors, oblique rotations are more appropriate (Fabrigar, et al., 1999)

Optimizing Factor Solutions When the factor rotation does not result into a simple structure, and the factor solution does not improve by changing the number of factors extracted, researchers may choose to drop problematic variables and rerun the analysis. Problematic variables have loadings below .32, are crossloading (have significant loadings on more than one factor), or are freestanding (Costello & Osborne, 2005). Researchers should remove one item at a time, starting with the “worst offender”, because the removal of each item may change the configuration of the factor structure. Further, researchers must ensure that the removal of items does not change the integrity of the data (Costello & Osborne, 2005). When item deletion does not improve the factor solution, there may be problems with the measurement instrument, the specification of the factor model, the size of the sample, or the extent to which data meet the assumptions of exploratory factor analysis.

Computing Factor Scores When an optimal factor solution is selected, results from factor analysis can be used in subsequent statistical procedures through the computation of factor scores. Factor scores are composite variables which estimate the location of each individual on the identified factors (DiStefano, Zhu, & Mindrila, 2010). There are various methods for the estimation of factor scores. Based on the complexity of the computation procedure, these methods can be grouped into two categories: a) refined, and b) non-refined (DiStefano, et al. 2010). The non-refined category

20

Diana Mindrila

includes more simple methods such as the computation of a) sum scores by factor, b) sum scores above a cut-off value, c) sum scores of standardized variables, and d) weighted sum scores. The refined methods are more computationally complex and include a) regression scores, b) Bartlett scores, and c) Anderson-Rubin scores. A summary of the advantages and disadvantages of different types of factor scores is provided by DiStefano et al. (2010).

CONCLUSION Despite the widespread use of exploratory factor analysis, many applied researchers are not yet familiar with this procedure. This chapter aims to provide an overview of this method, by explaining its purposes, the related terminology, the distinction between key concepts, and the criteria for making the decisions at each step. This information can serve as a starting point for the applied researcher who needs a conceptual description of this procedure. The following chapters describe research studies that employed exploratory factor analysis either by itself or in conjunction with other statistical procedures. This collection of studies shows that exploratory factor analysis is largely applicable in educational research. Further, results from this procedure can bring valuable theoretical contributions and have a wide range of practical applications.

REFERENCES Arbuckle, James L. “Full information estimation in the presence of incomplete data.” Advanced structural equation modeling: Issues and techniques 243 (1996): 277. Asparouhov, Tihomir, and Bengt Muthén. “Bayesian analysis of latent variable models using Mplus.” Retrieved June 17 (2010): 2014.

Exploratory Factor Analysis: An Overview

21

Bowerman, Bruce L., and Richard T. O'connell. Linear statistical models: An applied approach. Brooks/Cole, 1990. Browne, Michael W., and Robert Cudeck. “Single sample cross-validation indices for covariance structures.” Multivariate Behavioral Research 24, no. 4 (1989): 445-455. Bryant, F. B., & Yarnold, P. R. (1995). Principal components analysis and exploratory and confirmatory factor analysis. In L. G. Grimm & R R. Yarnold (Eds.), Reading and understanding multivariale statistics (pp. 99-136). Washington, DC: American Psychological Association. Cattell, Raymond B. “The scree test for the number of factors.” Multivariate behavioral research 1, no. 2 (1966): 245-276. Cattell, Raymond B. “The scientific use of factor analysis.” (1978). Cattell, Raymond B., and Joseph Jaspers. “A general plasmode (No. 3010-5-2) for factor analytic exercises and research.” Multivariate Behavioral Research Monographs (1967). Cattell, Raymond B., and Silke Vogelmann. “A comprehensive trial of the scree and KG criteria for determining the number of factors.” Multivariate Behavioral Research 12, no. 3 (1977): 289-325. Cerny, Barbara A., and Henry F. Kaiser. “A study of a measure of sampling adequacy for factor-analytic correlation matrices.” Multivariate Behavioral Research 12, no. 1 (1977): 43-47. Comrey, A. L., & Lee, H. B. (1992). A first Course in Factor Analysis. Hillsdale, NJ: Erlbaum. Comrey, A. L., and H. B. Lee. “Interpretation and application of factor analytic results.” Comrey AL, Lee HB. A first course in factor analysis 2 (1992). Costello, Anna B., and Jason W. Osborne. “Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis.” Practical assessment, research & evaluation 10, no. 7 (2005): 1-9. DiStefano, Christine, Min Zhu, and Diana Mindrila. “Understanding and using factor scores: Considerations for the applied researcher.” Practical Assessment, Research & Evaluation 14, no. 20 (2009): 1-11.

22

Diana Mindrila

Dziuban, Charles D., and Edwin C. Shirkey. “When is a correlation matrix appropriate for factor analysis? Some decision rules.” Psychological bulletin 81, no. 6 (1974): 358. Everitt, B. S. “Multivariate analysis: The need for data, and other problems.” The British Journal of Psychiatry 126, no. 3 (1975): 237240. Fabrigar, Leandre R., and Duane T. Wegener. Exploratory factor analysis. Oxford University Press, 2011. Fabrigar, Leandre R., Duane T. Wegener, Robert C. MacCallum, and Erin J. Strahan. “Evaluating the use of exploratory factor analysis in psychological research.” Psychological methods 4, no. 3 (1999): 272. Figueredo, Aurelio José, Patrick E. McKnight, Katherine M. McKnight, and Souraya Sidani. “Multivariate modeling of missing data within and across assessment waves.” Addiction 95, no. 11s3 (2000): 361-380. Finch, John F., and Stephen G. West. “The investigation of personality structure: Statistical models.” Journal of Research in Personality 31, no. 4 (1997): 439-485. Finney, Sara J., and Christine DiStefano. “Non-normal and categorical data in structural equation modeling.” Structural equation modeling: A second course 10, no. 6 (2006): 269-314. Floyd, Frank J., and Keith F. Widaman. “Factor analysis in the development and refinement of clinical assessment instruments.” Psychological assessment 7, no. 3 (1995): 286. Gagne, Phill, and Gregory R. Hancock. “Measurement model quality, sample size, and solution propriety in confirmatory factor models.” Multivariate Behavioral Research 41, no. 1 (2006): 65-83. Grewal, Rajdeep, Joseph A. Cote, and Hans Baumgartner. “Multicollinearity and measurement error in structural equation models: Implications for theory testing.” Marketing Science 23, no. 4 (2004): 519-529. Gorsuch, R. L. “Factor analysis.” (2nd ed.) Hillsdale, NJ: Lawrence Erlbaum (1983). Guttman, Louis. “Some necessary conditions for common-factor analysis.” Psychometrika 19, no. 2 (1954): 149-161.

Exploratory Factor Analysis: An Overview

23

Hair Jr, Joseph F., Rolph E. Anderson, Ronald L. Tatham, and C. William. “Black (1995), Multivariate data analysis with readings.” New Jersy: Prentice Hall (1995). Heerwegh, Dirk. “Small sample Bayesian factor analysis.” Phuse. Retrieved from http://www. lexjansen. com/phuse/2014/sp/SP03. pdf (2014). Horn, John L. “A rationale and test for the number of factors in factor analysis.” Psychometrika 30, no. 2 (1965): 179-185. Humphreys, Lloyd G., and Richard G. Montanelli Jr. “An investigation of the parallel analysis criterion for determining the number of common factors.” Multivariate Behavioral Research 10, no. 2 (1975): 193-205. Kaiser, Henry F. “The application of electronic computers to factor analysis.” Educational and psychological measurement 20, no. 1 (1960): 141-151. Kaiser, Henry F. “A second generation little jiffy.” Psychometrika 35, no. 4 (1970): 401-415. Kline, P. Psychometrics and psychology. London: Acaderric Press. (1979). Kunce, J. T., Cook, W. D., & Miller, D. E. (1975). Random variables and correlational overkill. Educational and Psychological Measurement, 35, 529-534. Lawley, Derrick N. “VI.—the estimation of factor loadings by the method of maximum likelihood.” Proceedings of the Royal Society of Edinburgh 60, no. 1 (1940): 64-82. Lee, S.Y. (1981). A Bayesian approach to confirmatory factor analysis. Psychometrika, 46, 153-160. Lee, S.Y., 1981. A Bayesian approach to confirmatory factor analysis. Psychometrika, 46(2), pp.153-160. MacCallum, Robert C., Keith F. Widaman, Shaobo Zhang, and Sehee Hong. “Sample size in factor analysis.” Psychological methods 4, no. 1 (1999): 84. Marascuilo, Leonard A., and Joel R. Levin. Multivariate statistics in the social sciences: A researcher's guide. Wadsworth Publishing Company, 1983.

24

Diana Mindrila

Martin, James K., and Roderick P. McDonald. “Bayesian estimation in unrestricted factor analysis: A treatment for Heywood cases.” Psychometrika 40, no. 4 (1975): 505-517. Mayekawa, Shin-ichi. Three and Four Mode Factor Analysis with Application to ASVAB Data. No. TR-85-7-ONR. Iowa Univ Iowa City, 1985. Menard, Scott. “Applied logistic regression analysis. Quantitative applications in the social sciences, No. 106.” Thousand Oaks, CA & London: Sage (1995). Myers, R. H. “Classical and modern regression with applications. Boston: PWS and Kent Publishing Company.” (1990). Mundfrom, Daniel J., Dale G. Shaw, and Tian Lu Ke. “Minimum sample size recommendations for conducting factor analyses.” International Journal of Testing 5, no. 2 (2005): 159-168. Muthén, Bengt, and Tihomir Asparouhov. “Bayesian structural equation modeling: a more flexible representation of substantive theory.” Psychological methods 17, no. 3 (2012): 313. Norusis, M. J. “SPSS 13.0 Statistical Procedures Companion. Chicago: SPSS.” (2005). Nunnally, J. C. (1978). Psychometric theory (2nd Ed.). New York: McGraw-Hill. Nunnally, Jum. “Psychometric methods.” (1978): 464-465. Preacher, Kristopher J., and Robert C. MacCallum. “Exploratory factor analysis in behavior genetics research: Factor recovery with small sample sizes.” Behavior genetics 32, no. 2 (2002): 153-161. Rockwell, Richard C. “Assessment of multicollinearity: The Haitovsky test of the determinant.” Sociological Methods & Research 3, no. 3 (1975): 308-320. Schmitt, Thomas A. “Current methodological considerations in exploratory and confirmatory factor analysis.” Journal of Psychoeducational Assessment 29, no. 4 (2011): 304-321. Spearman, Charles. (1904). General intelligence, objectively determired and measured. American Journal of Psychology, 15, 201-293. Spearman, Charles. (1927). The abilities of man. New York: MacmilIan.

Exploratory Factor Analysis: An Overview

25

Steiger, James H., and John C. Lind. “Statistically based tests for the number of common factors.” In annual meeting of the Psychometric Society, Iowa City, IA, vol. 758, pp. 424-453. 1980. Tabachnick BGdFidell, L. S. “Using multivariate statistics.” New York. (1996). Tabachnick, Barbara G., Linda S. Fidell, and Steven J. Osterlind. “Using multivariate statistics.” (2001). Thurstone, Louis Leon. “The vectors of mind: Multiple-factor analysis for the isolation of primary traits.” (1935). Thurstone, Louis Leon. “Multiple factor analysis.” (1947). Timm, Neil H. “The estimation of variance-covariance and correlation matrices from incomplete data.” Psychometrika 35, no. 4 (1970): 417437. Van Driel, Otto P. “On various causes of improper solutions in maximum likelihood factor analysis.” Psychometrika 43, no. 2 (1978): 225-243. Velicer, Wayne F., and Douglas N. Jackson. “Component analysis versus common factor analysis: Some issues in selecting an appropriate procedure.” Multivariate behavioral research 25, no. 1 (1990): 1-28. Zeller, R. A. “Statistical tools in applied research.” Retrieved July 10, 2008. (2006). Zwick, William R., and Wayne F. Velicer. “Factors influencing four rules for determining the number of components to retain.” Multivariate behavioral research 17, no. 2 (1982): 253-269. Zwick, William R., and Wayne F. Velicer. “Comparison of five rules for determining the number of components to retain.” Psychological bulletin 99, no. 3 (1986): 432.

In: Exploratory Factor Analysis Editor: Diana Mindrila

ISBN: 978-1-53612-486-6 © 2017 Nova Science Publishers, Inc.

Chapter 2

TEACHER MOTIVATION AND RECRUITMENT IN HIGH SCHOOL SPECIAL EDUCATION Matthew D. Lawrence* University of West Georgia, Carrollton, GA, US

ABSTRACT The purpose of this study was to identify the factors that compel high school special educators to choose careers in high school special education. A review of the literature found that profound difficulties exist in (1) attracting and (2) retaining highly qualified and highly motivated high school special education teachers. A survey was used to gather relevant data; it was first administered to a single school as part of a pilot study, and then administered to the high school special education teaching cadre at a large suburban public high school in the Southeast. Exploratory factor analysis was then used to identify the factors which lead prospective educators to become high school special education teachers. Four distinct factors emerged from the factor analysis: “Students/clientele”; “societal impact”; “professional security”; and “personal compatibility”. The results of factor analysis indicate that *

Corresponding Author Email: [email protected].

28

Matthew D. Lawrence important personal characteristics exist among highly-motivated high school special education teachers. University schools of education, as well as secondary school systems, should undertake to identify these characteristics in prospective educators. If such efforts are pursued early in the process of teacher training, the ability of the American public school system to recruit, and to retain, high quality special educators will be greatly enhanced.

INTRODUCTION A basic cost-benefit analysis of the effectiveness of special education services yields a litany of troubling facts and figures. It is difficult to find clear and accurate information about total special education spending; however, according to the US Department of Education, federal special education spending rose from $252 million in 1978, to $7.5 billion in 2003, and the DOE reported that $12.6 billion was spent by the federal government on special education services in 2015. Yet, according to McCann (2014) in a briefing on special education spending for the New America Education Policy think tank, state governments are responsible for nearly 45% of special education expenditures, and local school systems cover approximately the same percentage. This means that federal expenditures, which approach $13 billion annually, represent less than 10% of all special education expenditures. As funding has increased, however, the academic and socioeconomic outcomes of students who receive these services continue to lag far behind the outcomes of their nondisabled peers. According to the National Center for Education Statistics (NCES, 2013), the US high school graduation rate was 82.3% for all students in 2014, and 63.1% for students with disabilities. In the state of Georgia, the graduation rate for students with special needs is half that of the general population. Additionally, according to the NCES (2013), 38.4% of students with special needs obtained a postsecondary degree, compared to 51.2% of students who received no special education services. As a consequence of former students with special needs having suboptimal levels of human capital, 54% of adults

Teacher Motivation and Recruitment …

29

with SLD (specific learning disability) report either being (1) unemployed or (2) not in the labor force—and they are twice as likely their nondisabled peers to be living in poverty. Finally, per the NCES (2013), only about half of students who once received learning disability accommodations were living independently during the first eight years after having received their high school diplomas. The preceding statistics indicate that our system fails students with special needs both during their participation in the system (in terms of poor academic outcomes and graduation rates, and after exiting the system (in terms of postsecondary achievement, and employment). In other words, special education is in crisis. Of the many issues that plague special education, teacher quality, teacher retention, and teacher certification remain among the most intractable. According to the US Department of Education (2008), 11.2% of all special educators were not highly qualified in 2008, and McLeskey and Billingsly discovered in a 2004 study that between 82% and 99% of secondary special education teachers were not highly qualified in the academic content area in which they taught. According to McLeskey, Tyler and Flippin (2004), who conducted a study of personnel shortages in special education, 98% of national school districts reported special education teacher shortages, and Bradley, et al., reported in 2011 that 90% of high-poverty school districts have difficulty attracting highly-qualified special educators. Exacerbating this problem, the same report noted that special educators are twice as likely to leave the profession as are their colleagues in general education. Martin (2010), who conducted a study on recruiting and retaining special education teachers in South Carolina, reports that most special education teachers remain in the classroom for a period of approximately six years before succumbing to any one of the myriad factors that contribute to rates of attrition. Carlson, et al., in a 2002 study of personnel shortages in special education, made the point that a conversation about teacher quality in special education is very difficult to hold separately from a conversation about teacher quantity in special education. As it grows more difficult to find and to retain highly-qualified special educators,

30

Matthew D. Lawrence

principals are forced to hire less than highly-qualified applicants, which consequently leads to a special education teaching cadre that is staffed by undertrained, poorly qualified educators. The current framework within which we provide services to students with special needs is untenable. Steps must be taken to identify the factors that motivate prospective educators to enter high school special education in order to ensure that those individuals are identified, encouraged, and supported as early as possible in their academic careers. The purpose of this study was to determine how we can encourage high-quality, highlymotivated, professionally compatible educators to enter the field of high school special education; and, in so doing, help to boost both the rates of recruitment and the rates of retention of such educators. In conclusion, the following study addressed this research question: What are the factors that drive prospective educators to become high school special education teachers?

REVIEW OF THE LITERATURE Regarding the problem of teacher shortages (and special educator shortages), expectancy-value theory is closely related to many of the factors which motivate people to choose teaching as a career in the first place. When people evaluate potential career choices, certain expectancies (i.e., perceptions of teaching self-efficacy) have been shown to be powerfully influential in leading individuals to choose teaching as a career (Watt & Richardson, 2007). The importance of promoting self-efficacy begins with preservice teachers who have not yet entered the profession. In a study of the factors that motivate prospective educators to enter special education, Bremer (2012) found “self-perceived teaching abilities” (p. 140) to have had the most profound impact on preservice teachers’ motivation to enter special education. This finding was especially important in light of the fact that, according to Bremer (2012), only 4% of undergraduates who study special education actually complete their degree, and only a fraction of those individuals ever enter the field as teachers of special education

Teacher Motivation and Recruitment …

31

(Bremer, 2012). Efforts to improve preservice teachers’ self-efficacy can therefore lead to a better-motivated cadre of novice special education teachers, which may in turn positively impact rates of attrition, and thus mitigate the severe personnel shortages that afflict the field. Expectancyvalue, therefore, could be deployed to counterbalance ongoing difficulties with successfully recruiting and retaining high quality educators; and consequently serve as a valuable tool for systems to use in (1) attracting teachers, and (2) promoting sufficient levels of career satisfaction in order to reduce the high attrition rates which currently plague the profession.

Family History/Community Experience Horn (2010), in a study of the impact of familial relationships upon prospective educators’ choosing careers in special education, found that slightly more than half of survey respondents with disabled family members entered special education as a direct result of that fact. Bremer (2012) also found that 80% of preservice special education teachers reported having had experience working with individuals with disabilities, and that these experiences influenced their career decisions. Similarly, Horn (2010) found that 93% of survey respondents reported having had volunteer experience with individuals with disabilities—through church, the Special Olympics, community outreach initiatives, etc.; and almost 70% of those same respondents indicated that these experiences made them more likely to undertake efforts to advocate for and to work with people with disabilities (Horn, 2010).

Student Achievement In order to more effectively attract high quality special education teachers, some attention must be paid to promoting perceptions of professional satisfaction among teachers of students with special needs. In a study of motivation in middle school special education teachers, King-

32

Matthew D. Lawrence

Sears and Baker (2014) found that teachers who express greater levels of professional satisfaction are less prone to experiencing burnout, and are more likely to express a greater degree self-efficacy than are teachers who report being dissatisfied with their professional experiences. Professional satisfaction is a factor which represents itself in teachers looking at their job as a source of intellectual and emotional sustenance; and insofar as these feelings result in higher levels of motivation and commitment, they have the effect of reducing attrition rates (King-Sears & Baker, 2014). Research has shown that professional satisfaction among special educators is closely tied to student achievement—and student achievement can present as either a major driver in promoting special education teacher retention, or as an important source of professional dissatisfaction, and, ultimately, attrition (Perrett, 2001).

Personal Factors A 1996 survey by Westling and Whitten, which sought to determine whether participating special educators planned to remain in the profession for at least five years, found that personal considerations can play an important factor in special education teacher retention. Specifically, this study found that special education teachers who were also the primary earners in their family were less likely to leave the profession. Williams (2004) in a study of special education teacher attrition in North Carolina, furthermore noted that changes in marital status, pregnancies, and other family considerations were all important drivers in the decision of new special education teachers to leave the profession. These findings were underscored in a study by Lemons (2011) of the factors that influence special educators’ career decisions in an Indiana school district. When questioned about special education teacher attrition in this district, many administrators described losing teachers as a result of their spouses having been assigned employment transfers. Insofar as families’ primary breadwinners typically dictate any relocation agenda, these findings serve to emphasize the importance of expanding special educator recruitment

Teacher Motivation and Recruitment …

33

efforts to include individuals whose status in the family-hierarchy is more likely to be head-of-household. Such personal variables need not always fall into the realm of the strictly practical. Smith (2000), in a study of the factors that motivated paraprofessionals to pursue full certification in special education, noted that many candidates expressed a desire to work in a profession that maximized their ability to work creatively. Participants in this study also indicated a desire to enter a profession in which they had full confidence of their ability to excel (Smith, 2000). This result was particularly useful for two reasons: first, it is closely connected to the factor of self-efficacy (explored above), and it underscores the symbiotic relationship between self-efficacy and motivation; second, participants in this study were all working paraprofessionals with significant classroom experience—which similarly underscores the importance of training and preparation as these issues relate to teachers’ motivation to remain within the special education profession.

Salary Miller et al. (1999), and Singer (1992) both found that special education teachers who were working in higher paying school districts were less likely to leave the profession than were special educators in lower paying districts. Incentives, especially financial incentives, can be a powerful tool to attract and retain teachers in hard-to-staff special education positions. Van Alstine (2010) in a study of factors that led to special education teacher attrition, underscored low salaries as a primary driver. In this study, 100% of participants indicated that salary considerations contributed to their plans to leave the profession (Van Alstine, 2010). These teachers reported that their low salaries were not commensurate with the demands of the profession, and that low pay led them to feel unappreciated by the district (Van Alstine, 2010). In an attempt to address such grievances, districts regularly offer socalled “signing bonuses” to teachers who sign contracts to work in subject

34

Matthew D. Lawrence

areas that are critically short of highly-qualified staff (such as special education), and financial incentives appear to be especially powerful in attracting the attention of prospective male special educators. Rice and Goessling (2005) conducted a study of effective ways to recruit male special education teachers, and found a number of different reasons for the relatively small number of men who enter the special education profession. Such reasons included (1) the assumption that special education positions hold a lower status than general education, and (2) the low salaries endemic to education in general. In a 1997 study by Rodriguez that surveyed men for ideas about what can be done to increase the number of males who choose education as a career, 45% responded that offering more money would be the most expeditious method. This tactic could presumably be used to quickly expand the number of men who are currently working in the special education teaching cadre.

Commitment According to Green (2011), special education teacher commitment can have a positive impact on rates of retention. Teachers who are professionally committed are less likely to suffer from many of the factors that lead to teacher burnout, and they are more likely to work more productively, and with higher levels of personal satisfaction, than are teachers who are comparatively uncommitted to the profession. Bremer (2012) noted that higher levels of commitment are usually attended by higher-quality instruction, which promotes stronger self-efficacy, which in turn promotes retention. Some of the factors that lead to increased commitment, according to Green (2011), are (1) positive school cultures, (2) strong teacher-student working relationships, and (3) strong teacherparent working relationships. School districts in high-poverty areas are generally deficient in all three of these areas, especially with respect to parent involvement, and this situation presumably is one of the factors that makes it so difficult for those districts to attract highly-qualified special education instructors.

Teacher Motivation and Recruitment …

35

Marketing Because there is such a limited pool of qualified special education instructors, Thornton, Peltier and Medina (2007) recommended that principals and special education directors become far more proactive in terms of working with colleges and universities to identify prospective candidates for this field. According to these authors, principals should develop a relationship with placement officers at nearby universities so that prospective special education teachers might be preemptively cultivated. Bremer (2012) suggested that efforts to market special education as a career should start even earlier—perhaps as early as middle school. According to Mau, Ellsworth, Hawley and Loper (2004), students begin making career decisions in the very early stages of secondary education. Insofar as novice special educators are far more likely to leave the profession, steps might be taken to begin the recruitment and training process far earlier than it currently occurs. This would help mitigate many of the self-efficacy deficits that currently plague the ranks of newly-hired special educators, consequently promoting more satisfactory levels of teacher retention. The central issue in special education is evident in the uncomfortable juxtaposition of necessity and reality. The current framework is one that takes students who specifically and expressly need teachers of the first excellence, and places them in the hands of the poorly experienced, the insufficiently credentialed, the overworked and the overwhelmed. The end result can be seen in the profoundly disappointing academic outcomes that continue to plague special-needs students across the country—and this fact in turn negatively impacts educators’ levels of motivation to enter, and to remain within, special education—which in turn leads to ever more dismal student outcomes. The “perfect storm” analogy is much overused, but it is appropriate in this case.

36

Matthew D. Lawrence

METHOD The research contained herein was based on a correlational design. According to Daenzer (2009), quantitative correlational research is appropriate when one is seeking to investigate the relationship between variables in order to establish whether or not they correlate. Donnell (2015) notes that correlation research seeks to determine both the existence, and the extent, of the relationship between variables; and insofar as there are numerous variables which inform the decisions of prospective educators to choose special education, this particular design is exceptionally well-suited to exploring the research question above.

Sample A convenience sampling procedure was used for the survey upon which this research is based. According to Gravetter and Forzano (2011), this method is predominant in the behavior sciences; and is also the most feasible, least expensive and expeditious of all the various options. Convenience samples are comprised of individuals who are (1) available and (2) willing to participate in the research (Gravetter & Forzano, 2011). A significant drawback of this method is that the researcher cannot effectively deploy a random process of selection, resulting in a sample that might not be representative, and could perhaps be biased (Ellison, Barwick & Farrant, 2009). Compensating for the lack of randomization, however, was the fact that the projected participants possessed such an exceptional degree of diversity (in terms of race, gender, academic background, etc.) that they presented as a convincing cross-sample of the at-large population. The survey was sent electronically to all special educators from 16 high schools in a large suburban school district in the Southeast. Among these schools, 231 of the projected participants work in a conventional setting; 67 are assigned to low-incidence settings. The participating county does not keep an accounting of disaggregated data about the individual demographic characteristics (age, gender, education, etc.) of its teachers;

Teacher Motivation and Recruitment …

37

however, the survey was constructed in such a way as to gather those data when respondents provided their submissions.

Instrumentation A survey design was most appropriate for this research for several reasons. First, the purpose of such a practice is to draw inferences from a sample population so that predominant trends and characteristics within a larger population can be effectively identified (Jekel, Katz & Elmore, 2009). Second, surveys provide for the relatively swift collection of data, especially insofar as the research is cross-sectional, with the data being collected at one point in time (Jekel, Katz & Elmore, 2009). The principal drawback of cross-sectional survey research is that it is not appropriate if one is seeking to observe systemic changes over a longer period of time (Lavrakas, 2008); however, many of the issues explored in this research present as entrenched and persistent predicaments, and so a longitudinal viewpoint was consequently unnecessary. The specific survey used in this research has been adapted from the FIT-Choice® Scale (factors influencing teaching choice), an instrument developed by Monash University (Melbourne, Australia) researchers that is used to examine individuals’ motivations for choosing teaching as a career. This instrument seeks to measure teachers’ levels of self-efficacy; as well as intrinsic values such as interest in teaching as a career; and also subjective attainment values, which refer to motivations that arise separately from career substance (such as vacation time and salary); and, finally, social utility value, which relates to individuals choosing the education profession as a way to make a contribution to society (Watt & Richardson, 2007). The advantage of employing the FIT-Choice® Scale as the foundation of the instrumentation for this research rests with the fact that its construct validity and reliability are well-established, and there is a wide body of existing literature based on its use in a variety of different functions and settings.

38

Matthew D. Lawrence

Upon its initial development, FIT-Choice® was administered to two cohorts of preservice educators numbering 1140 participants, as well as an additional cohort of established educators numbering 294 individuals (Watt & Richardson, 2007). Cronbach’s Alpha measures of internal consistency were satisfactorily high, ranging from .90 to .97; and exploratory factor analysis for convergent and divergent construct validity found that pattern coefficients ranged from .56 to .95, which is also considered to be strong (Watt & Richardson, 2007). Finally, the relative strength and relevance of the chosen variables was underscored by the fact that no evidence of high cross loadings emerged during initial and subsequent periods of survey implementation (Watt & Richardson, 2007). Insofar as this research focused specifically on special education teachers, adaptations were made with regard to changing references of “Teaching” to “Teaching special education”. Additionally, because this research focused on retention and professional satisfaction, a portion of the FIT-Choice® Scale that examines individuals’ beliefs about the teaching profession was altered in such a way as to facilitate an examination of those specific issues. Due to the alterations made to the FIT-Choice® Scale, and in order to establish content validity, a pilot survey was conducted at a local school in order to account for potential problems with reliability, or validity, or with general administrative issues. According to Stopher and Metcalf (1996), a pilot survey is a preliminary rehearsal of the entire process, and they perform four essential functions: they provide (1) a means of analyzing the survey questions and the sampling process; (2) an initial calculation of variability; (3) a framework within which to project the amount of time it will take for participants to respond to and complete the survey; and (4) data regarding the most efficacious sampling size (Stopher & Metcalf, 1996). The participating faculty consisted of 27 special education instructors of every placement (inclusive, coaching model, low-incidence, etc.); and the participating high school was sufficiently diverse with respect to its student population as to provide an adequate snapshot of special education teaching conditions in the participating county.

Teacher Motivation and Recruitment …

39

The specific program used to create the survey was provided by Qualtrics, an online research company. According to DeSantis (2013) in a reflection on the use of Qualtrics by Stanford University for instructor evaluations, this program is both highly effective and widely accommodating for academic research. Among the benefits is that it allows for only one response per participant; also the responses are completely anonymous (only the researcher can see the individual results); and response data are rapidly downloadable, with no reformatting necessary (DeSantis, 2013). Library staff at California State University concur with the efficacy of Qualtrics, and note the following additional benefits: (1) the program is free of cost and (2) there is no need for participants to install any supplementary software (Bosch, 2015). The sole disadvantage noted was that it is not possible to secure a 100% response rate (Desantis, 2013)—however, this disadvantage is an inherent aspect of all survey research and not exclusively limited to Qualtrics. Cronbach’s alpha was the technique used in this process to provide evidence of internal consistency. Cronbach’s alpha coefficient is useful when researchers seek to determine the extent to which the views of disparate participants coalesce into the measurement of a common, discrete factor (Osborne, 2008). “When using Likert-type scales”, report Croasmun and Ostron (2011) “it is essential that the researcher calculates and reports Cronbach’s Alpha coefficient for internal consistency reliability” (p. 20).

Data Analysis Exploratory factor analysis was used to determine the answer to this research question: What are the factors that drive prospective educators to choose to become high school special education teachers? Factor analysis is a data reduction technique wherein a large number of discrete variables (in this case, variables which reflect special educator recruitment factors) are reduced to reflect a much smaller number of the specific factors which cause individuals to choose to teach high school special education. The development of this technique is ascribed to Charles Spearman (Harman,

40

Matthew D. Lawrence

1976), a researcher who sought to reduce the number of variables which promote individual intelligence (i.e., verbal intelligence; spatial reasoning ability; etc.) into a generalized intelligence factor, g (Myers, 2004). The result of these efforts culminated in the development of the first standardized intelligence tests (Plante, 2011). According to Kline (1994), exploratory factor analysis is used extensively in the fields of sociology and psychology, and it is considered a virtual necessity in studies that use questionnaires or surveys. Kline (1994) describes factor analysis as an interlocking mechanism composed of a number of different statistical procedures which are collectively designed to facilitate researchers’ efforts to simplify and clarify complicated data sets. Kline (1994) furthermore observes that factor analysis is typically applied in situations where researchers seek to understand the relationship between disparate variables. The specific input for the exploratory factor analysis in this research was derived from the Likert-response portion of the adapted FIT-Choice® Scale that measures respondents’ considerations for becoming high school special education teachers. Factor analysis is equally appropriate in efforts to understand correlations; and, as noted by Fabrigar and Wegener (2012), this technique can also be helpful to establish a lack of correlation, i.e., a situation wherein each variable functions individually with no relationship to any other discrete variable. In situations where the researcher has an incomplete picture with respect to the fundamental framework of correlations, exploratory factor analysis—or unrestricted factor analysis— is the most obviously appropriate and advantageous technique to use (Fabrigar & Wegener, 2012). Across the spectrum of sciences, it is often as illuminating to establish nonexistence as it is to establish existence, and exploratory factor analysis is specifically designed to assist with such efforts. Royce (1963) has provided the most commonly used definition of a factor: a construct that is operationally defined by its correlations of variables. Kline (1994) builds on this definition by stating that “…a factor is a dimension or construct of the relationships between a set of variables.”

Teacher Motivation and Recruitment …

41

(Kline, 1994, p. 8). Comrey and Lee (1992) describe factor analysis as a method to help researchers comprehend the correlations between inexact, difficult to measure variables, which is an apt description of the variables which drive the recruitment and retention of special educators. Yong and Pierce (2013) furthermore note that the practice is used to reduce large sets of variables to smaller groups that share a mutual, unobservable variance; and that this consequently produces descriptive categories that can provide insights into fundamental concepts. The purpose of this research was to determine the relationship between the very large number of variables that influence the fundamental concepts of (1) the successful recruitment of special educators, and (2) the enduring retention of same, and so it accordingly becomes clear that factor analysis was the most obviously efficacious technique for in-depth statistical analysis. The final output of exploratory factor analysis, according to Liossi (2002) provides for a continuum of factors that are listed by the order in which they contribute to the variance. “Factor loadings”, which are determinative of the extent to which each variable contributes to the factor are also provided (along a range of +1.0 to -1.0), and variables that have low factor loadings (i.e., a negligible contribution to the establishment of the factor) were preemptively eliminated (Liossi, 2002). Typically, items lower than .3 are eliminated. Additionally, pursuant to the aim of achieving a simple structure, items that load on more than one factor were similarly excised from the data (Netermeyer, Bearden & Sharma, 2003).

The Final Factor Solution Factor loadings are the weights and interrelationships between individual variables and the factor; and determining the optimal number of factors to include in the research is an important aspect of the process. According to Boire (2014) an assessment eigenvalues should be used to evaluate how much additional information is accrued when the number of factors are increased; and how much information is lost when the number of factors is reduced (Boire, 2014). In most statistical software, the optimal eigenvalue threshold is >1—because the factor must reflect for, at a very minimum, as much variance as any single discrete variable (Nunnally &

42

Matthew D. Lawrence

Bernstein, 1994). A scree plot is considered the most efficacious method of producing a visual representation of the number of meaningful factors (Fernandez, 2011); therefore, the appropriate statistical applications were deployed to produce this tool. The combination of eigenvalue calculation and scree plot generation provided for a clear understanding of the optimal number of factors to include. After this initial analysis, the EFA process focused on maximizing the clarity of the loading patterns; this process was designed to bring about a simple structure (Kline, 1994). Kline (1994) states that, “I am in agreement with Cattell [1978] and all serious workers in factor analysis that the attainment of simple structure is essential to factor analysis. Where this has not been done, there is little reason to take the results seriously” (p. 66). According to Kline (1994), of the five criteria used to determine simple structure, only two of them are of the utmost importance: (1) each factor should have a few high loadings, and (2) the rest of the loadings should be zero or close to zero (Kline, 1994). Accordingly, variables which loaded on more than one factor were eliminated; and variables which did not load strongly enough on any factor were eliminated as well. A concluding aspect of the process of determining the final factor solution was to conduct an analysis of the content in order to ensure that items within a factor share a common theme. Field (2005) states that, “If the mathematical factor produced by the analysis represents some realworld construct then common themes among highly loading questions can help us identify what the construct might be” (p. 9). Identifying and grouping the individual items that share a thematic commonality is the most important phase of the process of highlighting and accurately labeling discrete factors. According to Hamosh and Goldman (1986), “The success of factor analysis may be assessed by the ease with which [common themes] may be identified” (p. 76).

Estimating Coefficients The linear relationship between individual variables was established by the Pearson r correlation coefficient, which can range from -1.0 (which is a perfect negative correlation) to 1.0 (a perfect positive correlation) (Yocky,

Teacher Motivation and Recruitment …

43

2011). The central revelation provided by this calculation is found in its ability to determine whether or not individual participants’ strong association with one variable correlates with a high score on another discrete variable (Muijs, 2011). Insofar as this research is attempting to establish the relationship between different variables, and how those variables factor into special educator recruitment/retention rates, evaluating Pearson r was clearly a vital aspect of the process.

Interpreting the Factors Upon the extraction of factors, the results of the survey were subjected to further review by data analysis software provided by SPSS. The interpretability of the factors were optimized though the process of rotation, which maximizes the loading of each variable on a single extracted factor, while minimizing the loading on all of the other extracted factors (Field, 2005). Of the various methods of rotation available, an oblique rotation procedure was the most appropriate because there was a strong possibility that the factors would correlate (Field, 2005), and the concluding outcome of factor analysis was to provide groups of items that had a common theme, and would thus indicate the teachers’ considerations for entering special education. Finally, the interpretation also provided for the factor correlations, which showed the extent to which the factors are related to each other.

RESULTS Pilot Study Although the FIT-Choice® Scale is well established as an instrument supported by evidence of reliability and validity, a pilot study was conducted to identify any potential issues that may require minor changes and additions to the original instrumentation. Such changes were made in an effort to effectively address the research question listed above. The survey was sent out to 27 special education teachers who taught within one

44

Matthew D. Lawrence

of the schools within the participating district. After approximately four days, 20 responses were received, upon which staff participation ceased. The teachers were asked to complete a voluntary survey of 70 questions, which had the potential for resulting in a lower-than-expected participating rate. This response rate, however, was close to 75%, which indicates a relatively simple and unobtrusive experience for respondents. Additionally, a basic descriptive statistical analysis was conducted on the results of the pilot study to formulate preliminary data for the purpose of ascertaining the appropriateness of the framework within which the survey was delivered. No notable problems were observed.

Survey Response The survey upon which this research was based was made available to 298 special education instructors who were employed at the time of the study at the 16 high schools within the public school system which agreed to participate in the survey. The survey was made available to participants for approximately four weeks. Several general reminders were sent out, as well as personal appeals to various contacts throughout the county. Of the 298 potential participants, 155 (N=155) responded within the period of time the survey was available. Rubin and Babbie (2009) write that “The overall response rate is one guide to the representativeness of the sample respondents. If a high response rate is achieved, then there is less chance of significant response bias than if a low rate is achieved. A response rate of at least 50 percent is usually considered adequate for analysis and reporting” (p. 117). Although 52% of potential participants responded, some of the participants did not complete the entire survey. Incomplete responses notwithstanding, the response rate nonetheless falls within the parameters of what is considered “adequate” for social science research.

Teacher Motivation and Recruitment …

45

Factor Analysis Exploratory Factor Analysis (EFA) was used to answer the research question, “What are the factors that drive prospective educators to become high school special education teachers?”, and 39 variables were used to conduct this analysis. Of the various possible extraction methods, Principal Axis Factoring was judged to be the most appropriate because the assumption of normality had been violated: four of the variables had skewness values greater than two, which indicated that the distributions of these variables were highly skewed. According to Tanner (2012), when SPSS is used to calculate skewness, values that fall within +/-1.0 are considered normal; and, although some departure from precise normality is usually considered to be acceptable, the threshold beyond which exists an assumption of skewness is +/-2.0 (Tanner, 2012). Consequently, Principal Axis Factoring was selected in lieu of Maximum Likelihood because the latter method assumes normal distribution of variables. According to Osborne (2015), the process of rotation was established shortly after the development of factor analysis to assist researchers in the process of both clarifying and simplifying the sometimes difficult-tointerpret results which derive from un-rotated factor analysis. The specific method undertaken in this research was promax rotation method, an oblique rotation which, per Osborne (2015) is most appropriate in research which assumes correlations between latent factors, and, moreover, is a process that generally yields results that are either identical or superior to an orthogonal rotation, which assumes that no correlation exists between latent factors. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy, which determines the extent to which data are appropriate for factor analyses, yielded a result of .822. A high number is indicative that the proportion of variance in the variables is the result of underlying factors, and thus confirms the suitability of factor analysis (Rasli, 2006). According to Hinton, Brownlow, McMurray and Cozens (2004), “…a general rule of thumb is that KMO value should be greater than 0.5 for satisfactory analysis to proceed. The higher the better.” (p. 349). Tabachnick and Fidel

46

Matthew D. Lawrence

(2001) furthermore report that any KMO score above .6 indicates the appropriateness of factor analysis. A test statistic of .822 (df = 741, p < .001), therefore, provided strong evidence that factor analysis was applicable for data analysis in this research. Additionally underscoring the suitability of factor analysis are the results of Bartlett’s Test of Sphericity, which, according to Hinton, Brownlow, McMurray and Cozens (2004) determines whether or not a relationship exists between the variables: if no relationship exists, then undertaking factor analysis is purposeless. An appropriate p-value, per Hinton, Brownlow, McMurray and Cozens (2004), should be .300 was examined. On the first run, every variable loaded to at least one of the nine factors. There were 47 item loadings that were greater than .300; and, out of the 47 item loadings, 31 (65.96%) loaded onto one of the first four factors. Next, the number of factors were set to four (per the indication of both the scree plot and the pattern matrix), and the total variance explained by the four factors was calculated to be 51.06%. There were several variables that loaded to more than one factor,

48

Matthew D. Lawrence

and these were systematically and progressively eliminated from those with the lowest loading score to those with the largest. At the conclusion of this process, the pattern matrix showed that there were no variables which failed to load to any of the factors, nor any that cross-loaded on more than one factor. Accordingly, the remaining variables represented a final solution for determining the factors that drive prospective educators to become high school special education teachers. The process of exploratory factor analysis began with a total of 39 variables. Of those 39, 11 were eliminated during the course of the analysis because they either loaded poorly on all the factors or cross-loaded to more than one factor. The final solution, therefore, contained 28 variables; and the total variance explained by the four factors was 61.26%. The items included in the final solution along with the factor loadings are reported in Table 1. The next step in the process was to determine the marker items. Marker items were ascertained for four factors by identifying the single variable for each factor that had the highest factor loading score (Fabrigar & Wengener, 2012). The marker item for the first factor was the variable “I like teaching special education....” The factor loading score for this item was .934; there were ten variables that loaded on the first factor; and the total variance explained by this factor was 34.21%. The marker item for the second factor was the variable “Teaching special education will allow me to influence the next generation”. The factor loading score was .824; eight factors loaded to factor two; and the total variance explained by the second factor was 11.40%. For the third factor, the item marker was “Teaching special education will be a secure job”, with a factor loading score of .953. Five variables loaded on factor three, and the total variance explained by the third factor was 8.61%. Finally, the item marker for the fourth factor was “People I’ve worked with think I should become a special education teacher.” The factor loading score was .782; five variables loaded on factor four; and the total variance explained by the fourth factor was 7.04%.

Teacher Motivation and Recruitment …

49

Table 1. Final factor solution

I like teaching special education... For each statement below, please rate how important it was in YOUR decision to become a teacher of special education, from 1 (not at all important in your decision) to 7 (extremely important in your decision). I am interested in teaching special education... I want a job that involves working with children/adolescents with special needs... I have the qualities of a special education teacher... Teaching special education is a career suited to my abilities... I like working with children/adolescents with special needs... I have good special education teaching skills... For each question below, please rate the extent to which it is true for YOU, from 1 (not at all) to 7 (extremely). Please choose the number that best describes your agreement for each. How carefully have you thought about becoming a special education teacher? I chose teaching special education as a lastresort career... I've always wanted to be a special education teacher... Teaching special education will allow me to influence the next generation... Teaching special education will allow me to shape child/adolescent values... Teaching special education will allow me to work against social disadvantage... I like teaching special education...

1 .934 .869

2

Factor 3

.850 .769 .748 .721 .716 .647

-.514 .406 .824 .805 .774 .934

4

50

Matthew D. Lawrence Table 1. (Continued)

1 Teaching special education will allow me to raise the ambitions of underprivileged youth... Teaching special education will allow me to benefit the socially disadvantaged... Teaching special education enables me to 'give back' to society... Special education teachers make a worthwhile social contribution... Teaching special education allows me to provide a service to society... Teaching special education will be a secure job... Teaching special education will provide a reliable income... Teaching special education will offer a steady career path... A special education teaching job will allow me to choose where I wish to live... School holidays fit in with family commitments... People I've worked with think I should become a special education teacher... My family thinks I should become a special education teacher... My friends think I should be a special education teacher... I have had good special education teachers as rolemodels... I have had inspirational special education teachers... Percentage variance explained 34.21 % Extraction Method: Principal Axis Factoring. Rotation Method: Promax with Kaiser Normalization. a. Rotation converged in 6 iterations.

Factor 2 3 .767

4

.764 .718 .688 .527 .953 .772 .633 .630 .616 .782 .707 .685 .481 .396 11.4 8.61 0% %

7.04 %

Teacher Motivation and Recruitment …

51

Thematically speaking, students/clientele emerged for factor number one, underscoring the fact that high school special education teachers enter the profession as a result of their desire to work with children and adolescents with special needs. The theme of factor number two, societal impact (tangentially related to the first factor) indicated a strong desire among high school special education teachers to work with students with special needs so that they may play a role in shaping our future society— facilitating, in a general sense, better outcomes for every American, and specifically to bring about a better, more prosperous, more inclusive future for students with special needs. The theme that emerged for factor number three, professional security, highlighted the role of structural aspects of the teaching profession in influencing peoples’ decisions to choose high school special education. This was perhaps related to extant critical personnel shortages in the field, and to the fact that special educators are, under most circumstances, guaranteed to find employment. Finally, the theme of factor number four—personal compatibility—spoke to the fact that there is a category of individual who enters high school special education as a career. That person is one whose personality is sufficiently temperamentally wellsuited to the demands of the profession that their colleagues have actively encouraged this career path. Table 2. Factor correlation matrix Factor

student/clientele

societal impact

societal impact .593 professional security .232 .272 personal .414 .346 compatibility Extraction Method: Principal Axis Factoring. Rotation Method: Promax with Kaiser Normalization.

professional security

.272

52

Matthew D. Lawrence

The factor correlation matrix showed that the strongest positive correlation was between factors students/clientele and societal impact — the correlation coefficient was .593 (Table 2). The remaining correlations ranged from .232 to .414. The correlations between the factors showed statistical significance, with p-values all less than .05. The suitability of using promax rotation method, which assumes correlation between the factors, is therefore confirmed. According to Andrew, Pedersen and McEvoy (2011), Cronbach’s Alpha provides values which reflect the extent to which a set of individual variables measures a single construct. These values range from 0 to 1, with desirable values measuring greater than 0.7 in the social sciences (Andrew, Pedersen & McEvoy, 2011). Each of the four factors produced Cronbach’s Alpha scores that are well within the range of reflecting appropriate internal consistency reliability. “Personal compatibility” had the lowest score: .756—which is nevertheless an acceptable result. The Cronbach’s Alpha score for the factor “professional security” was .835; “students/ clientele” rated at .849; and “societal impact” had the highest Cronbach’s Alpha score, .910.

DISCUSSION An exploratory factor analysis of the results of the survey that underpins this research saw the emergence of four distinct factors that drive prospective educators to become high school special education teachers. The factors are party buttressed by findings discussed in the review of the literature; but they are also sufficiently unique insofar they also show some promise in potentially assisting school districts whose special education teaching cadre is understaffed or under-motivated. The first factor that emerged from the analysis is referred to as “students/clientele”, and it relates to personal characteristics evident within both the special educator, as well as within the population of students with special needs at it relates to attracting those educators. All of the variables that loaded onto this factor coalesced around a general theme that special

Teacher Motivation and Recruitment …

53

educators possess unique personal characteristics that make them distinctively suitable for the profession. Some of those variables include, “I want a job that involves working with children/adolescents with special needs”, and “I like working with children/adolescents with special needs”. Also, “I have good special education teacher skills”, and “Teaching special education is a career suited to my abilities”. The variable “I chose teaching special education as a last resort career” had a negative loading, which indicates that special education teachers generally enter the field as a result of long-term career planning and a deep commitment to the profession. The second factor that emerged is referred to as “societal impact”, and it relates to the fact that special education teachers are driven to enter the profession as a result of their desire to positively impact future generations. Some of the variables that loaded onto the second factor include “Teaching special education will allow me to influence the next generation”, and “Teaching special education will allow me to shape child/adolescent values”. Also relatedly emergent were “Teaching special education will allow me to work against social disadvantage”, “Teaching special education will allow me to benefit the socially disadvantaged”, and “Teaching special education will allow me to work against social disadvantage”. The broad theme evident in these results is that special education teachers tend to be driven by ideological considerations in their choice of pursuing this career. There is a clear vein of sociopolitical and philosophical moralism in these responses that, if not perfectly unique, is perhaps largely underrepresented among the motivational considerations of other professionals—both within and outside of education. The third factor that emerged is more representative of personal incentives, and is referred to as “professional security”. Some of the variables that loaded onto this factor include “Teaching special education will be a secure job”, “Teaching special education will provide a reliable income”, and “Teaching special education will offer a steady career path.” A percentage of these responses are probably representative of individuals’ motivations for choosing the teaching profession, irrespective of discipline. They might, however, be especially attractive to prospective educators who

54

Matthew D. Lawrence

wish to enjoy a maximum amount of job security. As mentioned in the literature review, nearly every state in the country is currently suffering from critical shortages of special educators; and these responses relate to the conventional wisdom that special educators are generally essentially guaranteed to remain employed—more so than teachers of other academic disciplines that are not experiencing such severe personnel shortages. The final factor is referred to as “personal compatibility”, and it closely relates to themes that emerged in the first factor, but are more reflective of extrinsic, rather than intrinsic, motivations. These include “People I've worked with think I should become a special education teacher”, “My family think I should become a special education teacher”, “My friends think I should be a special education teacher”, and, finally, “I have had good special education teachers as role-models.” These outcomes indicate that special educators have an affinity for the profession that is reflected in their being motivated and inspired by fellow professionals; and that is furthermore evident in the fact that prospective educators’ choice of discipline is reinforced by their interpersonal relationships.

Limitations The survey that underpinned the research was cross-sectional, which resulted in a limitation. This type of research is generally not appropriate when one is seeking to observe systematic changes over a long period of time; and insofar as the purpose of this research is to bring about systematic changes within the framework by which we educate students with special needs, perhaps a more fully developed longitudinal viewpoint would have been preferable. Another limitation is related to the construction of the questions upon which this research is based. While the questions allowed for deep reflection among the participants about their current professional attitudes, and about past professional decision-making, it did not allow for the participants to provide their own forward-thinking suggestions about how

Teacher Motivation and Recruitment …

55

to bring about positive changes to the profession. Therefore, future research could more effectively solicit constructive collaborative input from special education teachers themselves.

Recommendations for Future Research Conducting a nationwide study of the questions examined in this research would be tremendously helpful. Although special education departments across the country face many of the same challenges, some areas of the country presumably bring about better academic outcomes for students with special needs than do others. A nationwide examination into what works, and what doesn’t, would help facilitate the process of reform. Similarly, a study into the attitudes and experiences of students with special needs, conducted alongside an examination of special education teachers’ views of the profession, could provide a better picture of potential improvements. There is very little extant research into the current and lived experiences of students with special needs, and no large-scale effort has been made to gauge the level of satisfaction those students have with the services they are rendered. This is a large hole in the research, and this presents as a promising future topic. An attempt to solicit the level of satisfaction among parents with children with special needs should also be conducted on a regional basis. Many of these parents presumably have experience with working with special educators from kindergarten through 12th grade; and will consequently be equipped to provide insightful, detailed information about the benefits and drawbacks, the successes and failures, of special education at every level. Districts that boast the highest levels of parental satisfaction with special needs services are those with the most efficacious levels of professional customer service—and, accordingly, efforts could be made by underperforming districts to emulate the procedures and the culture of more successful districts.

56

Matthew D. Lawrence

CONCLUSION Exploratory factor analysis yielded some striking results which hold great promise for future research. The number one factor which drove participants to choose careers in special education was identified as “students/clientele”. This result points to the fact that certain people are uniquely emotionally and intellectually qualified to do the difficult job of educating students with special needs. It also indicates that despite the myriad challenges inherent in the profession, that single consideration has the effect of outweighing a litany of negative elements. How many potential special educators, however, are routinely ignored, marginalized, distracted, or otherwise shunted off into careers for which they are less well suited? How many potential special educators never realize their potential in the first place? It seems that if we are to permanently remedy the issue of special educator teacher attrition, those efforts need to begin with finding, in the first place, the right people for the job. The review of literature indicated that some sporadic efforts are currently undertaken to market special education. Those efforts should be made to be far more robust, and wide-ranging; and further steps should and must be taken to properly and efficiently identify those educators for whom special education is a true calling. And we must do a better job of calling them.

REFERENCES 10 Facts About K-12 Education Funding. Home. June 2005. Accessed May 03, 2017. http://www2.ed.gov/about/overview/fed/10facts/ index.html. Acton, Ciaran, Robert L. Miller, Deirdre Fullerton, John Maltby, and Ciaran Acton. SPSS statistics for social scientists. Basingstoke: Palgrave Macmillan, 2009. Andrew, Damon P. S., Paul M. Pedersen, and Chad D. McEvoy. Research methods and design in sport management. Champaign, IL: Human Kinetics, 2011.

Teacher Motivation and Recruitment …

57

Billingsley, B. S., and J. Mcleskey. “Critical Issues in Special Education Teacher Supply and Demand: Overview.” The Journal of Special Education 38, no. 1 (2004): 2-4. doi:10.1177/00224669040380010101. Boire, Richard. Data mining for managers: how to use data (big and small) to solve business challenges. Basingstoke: Palgrave Macmillan, 2014. Bosch, E. “Research Guides: Qualtrics: What is Qualtrics?” What is Qualtrics? - Qualtrics - Research Guides at California State University, Long Beach. Accessed May 01, 2017. http://csulb.libguides.com/ qualtrics. Bradley, M.C., Daley, T., Levin, M., O'Reilly, F., Parsad, A., & Robertson, A. “IDEA National Assessment Implementation Study. Final Report. NCEE 2011-4027.” National Center for Education Evaluation and Regional Assistance. June 30, 2011. Accessed May 01, 2017. https:// eric.ed.gov/?id=ED522067. Bremer, Cheryl L. “Why special education?: Exploring the influence of motivation on choosing special education as a career.” PhD diss., 2012. Carlson, E., M. Klein, K. Schroll, and S. Westat. “Key Findings – The Study of Personnel Needs in Special Education.” The Study of Personnel Needs in Special Education. 2002. Accessed May 01, 2017. https://education.ufl.edu/spense/key-findings/. Comrey, Andrew L., and Howard B. Lee. A first course in factor analysis. Hillsdale, N.J.: Erlbaum, 1992. Croasmun, James T., and Lee Ostrom. “Using Likert-type scales in the social sciences.” Journal of Adult Education 40, no. 1 (2011): 19. http://files.eric.ed.gov/fulltext/EJ961998.pdf. Daenzer, Brian E. Quantitative correlation of leadership styles and job stress in a midwest United States auto company. PhD diss., University of Phoenix, 2009. http://gradworks.umi.com/33/50/3350853.html. DeSantis, Jennifer. “Reflections on using Qualtrics for Course Evaluations EFS” August 28, 2013. Accessed May 3, 2017. http://web.stanford. edu/dept/lc/ats/UsingQualtricsforCourseEVals_2013Reflections.pdf.

58

Matthew D. Lawrence

Donnell, Wendy M. “A Correlational Study of a Reading Comprehension Program and Attrition Rates of ESL Nursing Students in Texas.” Nursing Education Perspectives 36, no. 1 (2015): 16-21. doi: 10.5480/13-1212. Ellison, Stephen L.R., Vicki J. Barwick, and Trevor J. Farrant. Practical statistics for the analytical scientist: a bench guide. Cambridge: The Royal Society of Chemistry, 2009. Fabrigar, Leandre R., and Duane Theodore. Wegener. Exploratory factor analysis. New York: Oxford University Press, 2012. Field, Andy. Discovering statistics using ibm spss statistics SPSS version 22.0. London: Sage Publications, 2014. Field, Andy. Discovering statistics using SPSS. London: SAGE, 2005. Gravetter, Frederick J. Research methods for the behavioral sciences. Belmont, CA: Wadsworth, 2011. Green, Joseph D. Factors relating to special education teachers' job commitment: A study of one large metropolitan school in Southern California. PhD diss., Pepperdine University, 2011. Hamosh, Margit, and Armond S. Goldman. Human Lactation 2: Maternal and Environmental Factors. Boston, MA: Springer US, 1986. Harman, Harry H. Modern factor analysis. Chicago: University of Chicago Press, 1976. Hinton, Perry R., Brownlow, C., McMurray L. & Cozens, B. Spss Explained. London: Taylor & Francis Group, 2004. Horn, Channon Kaye. “Special educators and their relatives with disabilities: what are the implications?” PhD diss., University of Kentucky, 2010. Jekel, James F., David L. Katz, Joann G. Elmore, and Dorthea M.G. Wild. Epidemiology, biostatistics, and preventative medicine. Philadelphia, PA: Saunders Elsevier, 2007. King-Sears, Margaret E., and Pamela H. Baker. “Comparison of Teacher Motivation for Mathematics and Special Educators in Middle Schools That Have and Have Not Achieved AYP.” ISRN Education 2014 (2014): 1-12. doi:10.1155/2014/790179. Kline, Paul. An easy guide to factor analysis. London: Routledge, 1994.

Teacher Motivation and Recruitment …

59

Lavrakas, Paul J. Encyclopedia of survey research methods. Thousand Oaks, CA: SAGE Publications, 2008. Lemons, Theresa. Factors that influence special education teachers' career decisions in a rural school district in southern Indiana. PhD diss., Walden University, 2011. Liossi, Christina, and Richard Hain. Procedure-related cancer pain in children. Abingdon, Oxon, U.K.: Radcliffe Medical Press, 2002. Martin, Deborah S. No child left behind and its effect on recruiting and retaining special education teachers in rural South Carolina school districts. PhD diss., Capella University, 2010. Mau, Wei‐Cheng J., Randy Ellsworth, and Donna Hawley. “Job satisfaction and career persistence of beginning teachers.” International Journal of Educational Management 22, no. 1 (2008): 48-61. doi:10.1108/09513540810844558. McCann, Clare. “Federal Funding for Students with Disabilities: The Evolution of Federal Special Education Finance in the United States. New America Education Policy Brief.” New America. November 30, 2013. Accessed May 03, 2017. http://files.eric.ed.gov/fulltext/ ED556326.pdf Mcleskey, J., N. C. Tyler, and S. Saunders Flippin. “The Supply of and Demand for Special Education Teachers: A Review of Research Regarding the Chronic Shortage of Special Education Teachers.” The Journal of Special Education 38, no. 1 (2004): 5-21. doi:10.1177/00224669040380010201. Miller, David M., Mary T. Brownell, and Stephen W. Smith. “Factors that predict teachers staying in, leaving, or transferring from the special education classroom.” Exceptional Children 65, no. 2 (1999): 201-18. doi:10.1177/001440299906500206. Muijs, Daniel. Doing quantitative research in education with SPSS. Los Angeles: Sage Publications, 2011. Myers, David G. Psychology. New York, NY: Worth, 2004. Netemeyer, Richard G., William O. Bearden, and S. C. Sharma. Scaling procedures for self-report measures in the social sciences: issues and applications. London: SAGE, 2003.

60

Matthew D. Lawrence

Nunnally, Jum C., and Ira H. Bernstein. Psychometric theory. New Delhi: Tata McGraw-Hill Ed., 1994. Osborne, Jason W. Best practices in quantitative methods. Thousand Oaks, CA: Sage Publications, 2008. Osborne, Jason W. “Under the Hood With Exploratory Factor Analysis: What Is Rotation Really Rotating?” PsycEXTRA Dataset, 2015. doi:10.1037/e558952014-001. Perrett, Debra McClure. Factors influencing the motivation to teach in deaf education, preschool through fifth grade, as identified by teachers in selected oral schools. PhD diss., 2001. Plante, Thomas G. Contemporary clinical psychology. Hoboken, NJ: John Wiley & Sons, 2011. Rasli, Amran. Data analysis and interpretation: a handbook for postgraduate social scientists. Skudai, Johor: Penerbit Universiti Teknologi Malaysia, 2006. Rice, C. J., and D. P. Goessling. “Recruiting and Retaining Male Special Education Teachers.” Remedial and Special Education 26, no. 6 (2005): 347-56. doi:10.1177/07419325050260060501. Robins, Richard W., R. Chris. Fraley, and Robert F. Krueger. Handbook of research methods in personality psychology. New York: Guilford, 2010. Rodriguez, Edwin. What Does Gender Have to Do with It? (Male Teachers in Early Childhood Education). Proceedings. Accessed May 3, 2017. http://files.eric.ed.gov/fulltext/ED415979.pdf. Royce, Joseph R. “Factors as theoretical constructs.” American Psychologist 18, no. 8 (1963): 522-28. doi:10.1037/h0044493. Rubin, Allen, and Earl R. Babbie. Essential research methods for social work. Boston, MA: Cengage Learning, 2009. Singer, Judith D. “Once is Not Enough: Former Special Educators Who Return to Teaching.” Exceptional Children 60, no. 1 (1993): 58-72. doi:10.1177/001440299306000106. Smith, Sherrye Denise. From paraprofessional to credentialed teacher: motivational factors that influence career pathways in special education. PhD diss., 2000.

Teacher Motivation and Recruitment …

61

Stopher, Peter R., and Helen M. A. Metcalf. Methods for household travel surveys. Proceedings. Washington, D.C.: National Academy Press, 1996. Tabachnick, Barbara G., and Linda S. Fidell. Using multivariate statistics. Boston: Allyn and Bacon, 2001. Thornton, Bill, Gary Peltier, and Ricky Medina. “Reducing the Special Education Teacher Shortage.” The Clearing House: A Journal of Educational Strategies, Issues and Ideas 80, no. 5 (2007): 233-38. doi:10.3200/tchs.80.5.233-238. Van Alstine, Rose. Factors contributing toward special education teacher attrition: A case study of Orange County public schools. PhD diss., University of Phoenix, 2010. https://eric.ed.gov/?id=ED519233. Watt, Helen M. G., and Paul W. Richardson. “Motivational Factors Influencing Teaching as a Career Choice: Development and Validation of the FIT-Choice® Scale.” The Journal of Experimental Education 75, no. 3 (2007): 167-202. doi:10.3200/jexe.75.3.167-202. Westling, David L., and Thomas M. Whitten. “Rural Special Education Teachers' Plans to Continue or Leave Their Teaching Positions.” Exceptional Children 62, no. 4 (1996): 319-35. doi:10.1177/ 001440299606200403. Williams, Jennifer Bautel. Special education teacher attrition in North Carolina: the contributions of internal and external factors. PhD diss., 2004. Yockey, Ronald D. SPSS demystified: a step-by-step guide to successful data analysis. Boston, MA: Prentice Hall, 2011. Yong, An Gie, and Sean Pearce. “A Beginner’s Guide to Factor Analysis: Focusing on Exploratory Factor Analysis.” Tutorials in Quantitative Methods for Psychology 9, no. 2 (2013): 79-94. doi:10.20982/ tqmp.09.2.p079.

In: Exploratory Factor Analysis Editor: Diana Mindrila

ISBN: 978-1-53612-486-6 © 2017 Nova Science Publishers, Inc.

Chapter 3

CLUSTERS OF TEACHERS BASED ON THEIR PERCEPTIONS OF LEARNER-CENTERED INSTRUCTION Yun-Jo An and Diana Mindrila College of Education, University of West Georgia, Carrollton, Georgia

ABSTRACT This study aimed to identify the factors that summarize teachers’ perceptions of learner-centered instruction (LCI) and to distinguish clusters of teachers based on their perceptions of LCI. Further, the study examined the demographic characteristics of the identified clusters, as well as the distribution of the clusters across grade levels. Data were collected using an online survey, and participants included 134 K-12 teachers. Two factors were identified through exploratory factor analysis: learner-centered teachers (LCT) and concerns about LCI (CLCI). Cluster analysis revealed four types of teachers based on their scores on the two factors: Average (N=65), High LCT (N=26), High CLCI (N=25), and Low LCT (N=18). The majority of the participants were assigned to the Average and High LCT clusters, which had positive perceptions of LCI. Only a third of the participants were included in the High CLCI and Low LCT clusters. The proportion of middle school teachers was significantly

64

Yun-Jo An and Diana Mindrila higher in the High CLCI cluster (χ2(6)=12.483, p=.05). The High CLCI cluster also had the largest proportion of females and the largest proportion of teachers with 11 or more years of experience.

Keywords: learner-centered instruction (LCI), learner centeredapproaches, learner-centered teachers (LCT), concerns about learnercentered instruction (CLCI), teacher perceptions, attitudes, concerns, exploratory factor analysis, cluster analysis, k-means

INTRODUCTION Our information society requires individuals to effectively manage and use ever-increasing amounts of information to solve complex problems and make decisions in the face of uncertainty. Nevertheless, the prevalent factory model of education falls short in preparing students for the increased challenges of the information age (Reigeluth, 1999). The factory model does not take into account students’ diverse needs and requires all students to learn the same thing at the same time. Consequently, students often perceive school learning as irrelevant to real-life challenges and to their interests. On the other hand, the learner-centered model addresses the personal domain and focuses on the development of real-world skills such as critical thinking, problem-solving, and collaboration. In learner-centered classrooms, students receive support and encouragement, take ownership over their learning, are more engaged, and more willing to learn (Bransford, Brown, & Cocking, 2000; Cornelius-White & Harbaugh, 2009; McCombs & Whisler, 1997; Reigeluth, 1994). Prior research suggests that teachers with more education and professional development training may be more willing to use learnercentered approaches. Research also shows that seniority may be a factor affecting teachers’ teaching approaches (Aydogdu & Selanik-Ay, 2016; Genc & Ogan-Bekiroglu, 2004). More research is needed to identify the factors that summarize teacher perceptions of learner-centered instruction (LCI) and to examine the individual characteristics that may affect

Clusters of Teachers Based on Their Perceptions …

65

teachers’ pedagogical beliefs and teaching practices. This study aimed to identify the factors that summarize teachers’ perceptions of LCI and to distinguish clusters of teachers based on their perceptions of LCI. Further, the study examined the demographic characteristics of the identified clusters, as well as the distribution of the clusters across grade levels.

LITERATURE REVIEW Learner-Centered Instruction (LCI) Unlike the traditional factory-model of education, the learner-centered perspective addresses the personal domain or individual differences, by focusing equally on the learner and learning (APA Board of Educational Affairs, 1997; McCombs & Whisler, 1997). Further, the learner-centered model focuses on developing real-world skills, such as higher-order thinking, problem solving, decision-making, and collaboration skills in addition to content knowledge (Bransford, Brown, & Cocking, 2000; McCombs & Whisler, 1997; Reigeluth, 1994). The learner-centered psychological principles prepared by the LearnerCentered Principles Work Group of the American Psychological Association (APA)’s Board of Educational Affairs (1997) also pertain to the learner and the learning process. The 14 principles are divided into four psychological factors, including cognitive and metacognitive, motivational and affective, developmental and social, and individual differences factors. Learner-centered approaches include but are not limited to problem-based learning, project-based learning, and inquiry-based learning. While learnercentered instruction (LCI) does not take only one form, learner-centered classrooms tend to have the following five characteristics in common: personalized and customized learning, social and emotional support, selfregulation, collaborative and authentic learning experiences, and assessment for learning (An, 2012; An & Reigeluth, 2011). Numerous research studies provide evidence on the effectiveness of learner-centered approaches. For example, Cheang (2009) found that the

66

Yun-Jo An and Diana Mindrila

learner-centered approach was effective in promoting several domains of motivation and learning strategies in a third-year pharmacotherapy course. By comparing and contrasting the findings of the meta-analytical research on the effectiveness of problem-based learning (PBL), Strobel and van Barneveld (2009) found that PBL is significantly more effective than traditional instruction when it comes to long-term knowledge retention, performance improvement, and satisfaction of students and teachers, whereas traditional approaches are more effective for short-term retention. Abdi (2014) investigated the effects of inquiry-based learning on 5th graders’ academic achievement in science classes. The results of the quasiexperimental study indicated that students who were instructed through inquiry-based learning achieved higher scores than those who were instructed through the traditional method. These are just a few examples.

Teacher Attitudes and Perceptions of LCI Only a small number of studies have explored teachers’ attitudes and perceptions of LCI. For example, Yilmaz (2008) conducted a qualitative research study to explore social studies teachers’ views of LCI and learning theories. The participants included three social studies teachers (2 male, 1 female) with considerable teaching experience (5-13 years) and advanced degrees (M.Ed. or Ph.D.) in social studies education. The results showed that all three teachers had positive attitudes toward LCI and were in favor of constructivist learning theory. They believed that LCI has the potential to make students’ learning engaging, enjoyable, challenging, and relevant. In terms of impediments to LCI, most of the challenges mentioned by the participants were related to the organizational structure of their classrooms and schools (e.g., large class size, lack of resources, time constraints, etc.). An and Reigeluth (2011) conducted an online survey to explore K–12 teachers’ beliefs, perceptions, barriers, and support needs in the context of creating technology-enhanced, learner-centered classrooms. The survey results showed that the participants had positive perceptions of LCI. The

Clusters of Teachers Based on Their Perceptions …

67

majority of the participants agreed that LCI is challenging but rewarding. About 70% of the participants believed that they were learner-centered teachers. Only a few participants thought that learner-centered approaches are time-consuming, diminish the amount of content they can teach, are incompatible with their subject areas, or require too much work. Lack of technology, lack of time, and assessment were identified as the major barriers to creating technology-enhanced, learner-centered classrooms. More recently, Tawalbeh and AlAsmari (2015) examined university instructors’ perceptions of LCI in the EFL (English as a Foreign Language) classroom using an adapted version of the questionnaire developed by An and Reigeluth (2011). The results revealed that participants had a positive attitude toward LCI. The majority of the participants agreed that LCI is challenging but rewarding. They also believed that they were learnercentered teachers. On the other hand, Ha (2014) reported three teachers’ critiques of learner-centered education in English language and humanities classrooms at the university level. The participants reported that learner-centered education has been abused by so many “lazy professors who just don’t prepare for their class and let students discuss things in whichever ways they want to” (p. 400). Researchers also noted that teachers may feel students’ independence as a threat to their identity and may not be comfortable with their new role as a facilitator (Robinson, Molenda, & Rezabek, 2007; Sockman, 2015).

Individual Factors Affecting Teachers’ Teaching Styles Research indicates that teachers’ demographic characteristics or individual variables may influence their teaching styles or pedagogical beliefs. For instance, Shulman (1990) noted that new teachers preferred to use teaching styles similar to those used by their own teachers. Genc and Ogan-Bekiroglu (2004) examined the relationships between science teachers’ teaching styles and their demographic characteristics in Tallahassee, Florida. The results showed that teachers with a higher degree

68

Yun-Jo An and Diana Mindrila

in the field of education hold more student-centered teaching styles. The data from the study also suggested that teachers who frequently participated in professional development activities (e.g., workshops, training) reported utilizing significantly more student-centered teaching practices than those who did not participated in professional activities frequently. Further, the results showed that teachers with experience from 1 through 9 years were more likely to utilize a student-centered approach comparing to more experienced teachers. The researchers noted that teachers who recently graduated from college could be more aware of new innovative teaching and learning techniques than senior teachers. More recently, Aydogdu and Selanik-Ay (2016) explored the characteristics of teachers who were more willing to use the constructivist learning approach using the Constructivist Learning Environment Questionnaire (CLEQ) and a modified version of Draw a Classroom Teacher Test Checklist. The participants included 115 primary school teachers in Turkey. The results of the study showed that less experienced teachers were more willing to use constructivist approaches than senior teachers. With regard to the education level, teachers who had graduate degrees were more open to using constructivist approaches than those without advanced degrees. The gender of the participants was not a factor that affected their teaching approach.

METHODS Participants Participants in this study were 134 teachers in Georgia. Email invitations, including the link to an online survey, were sent to K-12 teachers in all school districts in Georgia. Teachers’ email addresses were gathered from publicly available links on school websites. Also, school principals and assistant principals were encouraged to forward the email invitation to their teachers. Participants were predominantly female (79.1%). They identified themselves as Caucasian (80.6%), African

Clusters of Teachers Based on Their Perceptions …

69

American (11.2%), Hispanic American (3.7%), Multiple Ethnicity (2.2%), Asian (1.5%), or Native American or American Indian (0.7%).

Instrument Data were collected using an online survey. The survey included demographic questions and 14 Likert-scale items on a 5-point scale (1= “Strongly Disagree”, 5 = “Strongly Agree”), which was an adapted version of the survey developed by An and Reigeluth (2011). While An and Reigeluth (2011) had 11 items in the Perceptions of Learner-Centered Instruction section of their survey, this study used 14 items to measure teachers’ perceptions of LCI.

Data Analysis Descriptive Analysis. Before conducting statistical analyses, the distribution of the survey variables was examined by computing descriptive statistics and indices of skewness and kurtosis. No missing values were recorded; therefore, imputation procedures were not necessary. Exploratory Factor Analysis. Exploratory factor analysis (EFA) is often used to search the data for quantitative and qualitative distinctions (Gorsuch, 1974) and consists of a set of statistical procedures aiming to determine a number of distinct latent dimensions accounting for a pattern of correlations among a set of variables (Fabrigar & Wegener, 2012). This analysis reveals the structure of associations among variables when the researcher makes no a priori assumptions about the underlying factors (Fabrigar & Wegener, 2012). The latent dimensions, also referred to as “common factors”, are unobservable constructs that exert linear influences on a set of observed variables (Fabrigar & Wegener, 2012). To assess the extent to which the survey data met the assumption of multivariate normality, the Mardia’s coefficients of multivariate skewness and kurtosis (Mardia, 1970) were computed using the R 3.3.2 statistical

70

Yun-Jo An and Diana Mindrila

software. Univariate skewness coefficients above 2, univariate kurtosis coefficients above 7, and multivariate kurtosis coefficients above 3 for were considered indicative of non-normality (Finney & DiStefano, 2006). With a sample size of 134 teachers and an initial set of 14 observed variables, the survey data met the general recommendation of 5:1 casesper-variable ratio and a minimum of 100 cases (Gorsuch, 1983). EFA was conducted with the IBM SPSS Statistics 22 software using the Maximum Likelihood (ML) estimation procedure and Promax rotation. ML extracts the maximum amount of variance when the assumption of multivariate normality is not severely violated and ordered categorical data have at least five categories (Finney & DiStefano, 2006). Promax is one of the oblique rotation procedures, which assume that the common factors are correlated. This rotation procedure was found to provide accurate results in many contexts and thus became widely used (Hendrickson & White, 1964). It begins by conducting the Varimax orthogonal rotation (which assumes that common factors are not correlated) and then transforms the orthogonal solution by raising factor loadings to a power higher than two, which represents the kappa parameter (k) of the rotation (Fabrigar & Wegener, 2012). For the current study, the value of k = 4 was used. The number of common factors extracted was determined based on the examination of the scree plot (Cattel, 1966; Cattell & Jaspers, 1967), the number of eigenvalues larger than one (Kaiser, 1960), as well as the interpretability of the factors (Fabrigar & Wegener, 2012). The strength and the direction of the relationship between common factors and observed variables is indicated by factor loadings. Similar to the Pearson correlation, a factor loadings is the square root of the variance in the observed variable that can be explained by the common factor. As a general rule, loadings must have absolute values larger than .320 to be considered significant indicators of a common factor (Tabachnick & Fidell, 2001), and factors should have at least three observed indicators (Costello & Osborne, 2005). Items with loadings lower than .320 and cross-loading items were sequentially removed from the analysis. When an optimal factor solution was reached, the amount of variance explained by each factor, the correlation between factors, and the internal

Clusters of Teachers Based on Their Perceptions …

71

consistency of each factor were estimated. Further, Bartlett factor scores were computed to estimate the location of each individual on the identified common factors (DiStefano, Zhu, & Mindrila, 2010). Cluster Analysis. Cluster analysis (CA) is a multivariate statistical procedure that begins with a data set containing information about an ungrouped sample of individuals and attempts to organize these individuals into homogeneous groups (Aldenderfer & Blashfield, 1984). Thus, individuals have similar characteristics with those in the same cluster, while distinct from members of other clusters. Kaufman and Rousseeuw (1990) defined cluster analysis as the classification of objects into groups, where the number of groups, as well as their forms, is unknown. The “form” of a group refers to parameters such as cluster means, variances, and covariances, which also have a geometrical interpretation. A similar definition of cluster analysis is provided by Everitt (1993), who described cluster analysis as a useful division of a sample into a number of groups, where both the number of clusters and their properties are to be determined. This statistical procedure is used (a) to develop typologies or classifications, (b) to investigate conceptual schemes for grouping entities, (c) for hypothesis generation through data exploration, and (d) for hypothesis testing to determine whether the types defined through other procedures are in fact present in the data set. Of these goals, the creation of typologies accounts for the most frequent use of clustering methods (Aldenderfer & Blashfield, 1984). Based on the particular perspective on the creation of groups, there are seven major families of clustering methods: (a) hierarchical agglomerative, (b) hierarchical divisive, (c) iterative partitioning, (d) density search, (e) factor analytic, (f) clumping, and (g) graph theoretic (Aldenderfer & Blashfield, 1984). The most popular families of clustering methods are the hierarchical agglomerative, iterative partitioning, and factor analytic (DiStefano & Mindrila, 2012). Each family comprises several clustering methods, and the results obtained when different methods are applied to the same data set can be very different.

72

Yun-Jo An and Diana Mindrila

The k-means clustering method belongs to the iterative partitioning family and allows for cases to switch from their initial cluster assignment to a different cluster when they become more closely represented as typical members of a new cluster (MacQueen, 1967). The k-means passes are also referred to as the “nearest centroid sorting pass” or the “reassignment pass” (Aldenderfer & Blashfield, 1984). The process continues making “passes” through the data set until cases do not change their cluster assignment. Clustering procedures were conducted with the IBM SPSS Statistics 22 software. Model selection. Two through five cluster solutions were examined. After evaluating each cluster solution, the final solution was chosen based on the interpretability of the cluster centroid, match of the solution to previous research and theoretical knowledge of learner centered instruction, and cluster characteristics such as cluster size, and demographic characteristics. Demographic variables such as gender, age range, and years of experience were tabulated by cluster to examine their distribution across groups. Chi-Square Tests. The chi-square test was used to determine whether the prevalence of the identified clusters varied significantly by demographic categories and by school organizational levels. The chisquare test of independence is a nonparametric procedure that helps determine if there is a significant association between two nominal or categorical variables (Moore, Notz, & Flinger, 2015). The two nominal variables are cross-tabulated and, for each cell of the contingency table, the ratio of the squared difference between the observed count and the expected count and the expected count is computed. The chi-square statistic is the sum of these ratios. The chi-square test is, therefore, an omnibus test and a statistically significant test statistic does not indicate which cells of the contingency table recorded significant differences between observed and expected counts; therefore, the a significant chi square test is often followed by the computation of standardized residuals, which indicate the magnitude of the difference between observed and expected counts. Standardized or Pearson residuals represent the ratio of the difference between observed and expected counts and the square root

Clusters of Teachers Based on Their Perceptions …

73

of the expected count (Agresti, 2013). Standardized residuals with values exceeding +/-2 are considered significant, where positive values indicate that observed counts are larger than expected, whereas negative values indicate that observed counts are smaller than expected (Sharpe, 2015).

RESULTS Descriptive Analysis The distribution of the survey variables was examined by computing the mean, standard deviation, minimum and maximum values, and indices of skewness and kurtosis (Appendix A). The survey items with the highest mean ratings were “Learner-centered instruction is challenging but rewarding (M = 3.99, SD = 0.81), “I am familiar with learner-centered approaches” (M = 3.87, SD = 0.89), and “I use learner-centered approaches in my classroom” (M = 3.86, SD = 0.88). The items with the lowest mean ratings were “Learner-centered approaches are incompatible with my subject area” (M = 1.69, SD = 0.82), and “Learner-centered approaches are incompatible with my teaching philosophy” (M = 1.82, SD = 0.99). The items with the highest and lowest average ratings had the smallest standard deviations, indicating reduced response variability. All survey variables recorded non-significant univariate indices of skewness and kurtosis, which indicate a relatively normal univariate distribution. The Mardia test of multivariate normality provided a coefficient of multivariate kurtosis of 4.32 (p = .78). These values indicate that the survey data had a multivariate normal distribution. Further, the Bartlett test of sphericity yielded a significant test statistic (χ2(66) = 693.916, p < .001) and the Keiser-MeyerOlkin measure of sampling adequacy was .834 indicating that data were sufficiently correlated for factor analysis (Hair, Anderson, Tatham, & Black, 1998).

74

Yun-Jo An and Diana Mindrila

Exploratory Factor Analysis The fourteen survey items were used as input for EFA. The Pearson correlation matrix of the fourteen survey variables is provided in Appendix B. The first EFA iteration indicated three eigenvalues larger than one, and the scree-plot showed that the optimal solution should include two factors (Figure 1). Although the three factor solution explained a large percentage of the variance (65.01%), after removing three cross-loading items, one factor only included two items (“I have enough knowledge about learnercentered instruction” and “I am familiar with learner-centered approaches”). The two-factor solution reached a simple structure by sequentially removing two items with loadings lower than .3 (“Learner-centered approaches are incompatible with my teaching philosophy” and “I want to learn more about learner-centered instruction”). Items in each factor had a clear common theme and the two factors accounted for more than half of the variance (54.9%); therefore, the two-factor solution was selected as optimal. The items included in the final EFA solution along with the corresponding factor loadings are reported in Table 1.

Figure 1. Scree plot.

Clusters of Teachers Based on Their Perceptions …

75

The first factor accounted for 42.4% of the variance and included seven items. Loadings for this factor ranged between .341 and .928. The marker item for this factor was “I am a learner-centered teacher.” An examination of the items included in this factor indicated the factor measured the extent to which teachers support and use learner-center approaches. Therefore, this factor was labeled “Learner-Centered Teachers” (LCT). The second factor included five items and accounted for 12.5% of the variance. Item loadings ranged between .371 and .901. The item with the strongest loadings in this factor were “Learner-centered approaches are two time consuming” (loading = .901) and “Learnercentered approaches diminish the amount of content I can teach” (loading = .848). Overall, the items included in this factor referred to teachers’ concerns about learner-centered instruction; therefore, this factor was labeled “Concerns about Learner-Centered Instruction (CLCI).” The LCT and CLCI factors recorded a moderate negative correlation of -.574. The CLCI factor had a high degree of internal consistency (Cronbach’s alpha = 0.816), whereas the LCT factor had a moderate degree of internal consistency (Cronbach’s alpha = 0.635). The estimation of the factors was followed by the computation of Bartlett factor scores, which estimated the individuals’ location on the LCT and CLCI factors. These coefficients were then used for intra-personal comparisons using cluster analysis.

Cluster Analysis The k-means algorithm was employed to estimate two- to five-cluster solutions. The final solution was selected based on the interpretability of the cluster centroids and included four clusters (Figure 2). The most numerous cluster (N = 65) had close to average factor scores on both factors and was, therefore, labeled “Average”. The second largest group (N = 26) had an average LCT score more than one standard deviation above the mean, and an average CLCI score more than one standard deviation below the mean; therefore, this group was labeled High LCT. The third group (N = 25) had close to average scores on LCT and an average CLCI

76

Yun-Jo An and Diana Mindrila

scores more than one standard deviation above the mean; this group was labeled High CLCI. The smallest group (N = 18) was labeled Low LCT and had the average LCT score close to two standard deviations below the mean, and an average CLCI score close to one standard deviation above the mean. Table 1. Final EFA solution Item I am a learner-centered teacher. I believe that all teachers should use learner-centered approaches in their teaching. I use learner-centered approaches in my classroom. Learner-centered instruction is challenging but rewarding. I am familiar with learner-centered approaches. I have enough knowledge about learner-centered instruction. * I am not very comfortable with learner-centered approaches. I am more comfortable with the teacher-centered approaches (e.g., lectures). Learner-centered approaches are too time-consuming. Learner-centered approaches diminish the amount of content I can teach. Learner-centered approaches require too much work for me. My students are passive and not always responsible. They are not ready for learner-centered instruction. Learner-centered approaches are incompatible with my subject area.

Loadings LCT CLCI .928 .697 .677 .640 .590 .429 .341

.901 .848 .631 .529 .371

Note: * Recoded item

As indicated in Table 2, the High LCT cluster had the lowest proportion of females (73%) and consisted mostly of high school teachers (46%). This cluster had a relatively even distribution across age groups and levels of experience. Similarly, the Average cluster consisted mostly of high school teachers (49%) and included individuals with from various age groups and levels of experience. In contrast, the High CLCI cluster had the largest proportion of females (88%) and a large count of middle school teachers (48%). The majority of the teachers (80%) in the High CLCI cluster had more than 11 or more years of experience, and approximately half of the teachers in this cluster were 50-59 years old (44%). Similarly, the Low LCT cluster

Clusters of Teachers Based on Their Perceptions …

77

consisted mostly of middle school teachers (50%), but had a relatively uniform distribution across age groups and levels of experience.

2.00 1.50

Mean Factor Score

1.00 0.50 0.00 -0.50 -1.00 -1.50 -2.00

LCT

CLCI

High CLCI, N=25

-0.02

1.42

High LCT, N=26

1.32

-1.28

Low LCT, N=18

-1.87

0.89

Average, N=65

0.00

-0.28

Figure 2. Cluster centroids.

Chi Square Tests The chi-square test did not yield significant test statistics for the association between cluster assignment and teacher gender (χ2(3) = 1.804, p = .614), age group (χ2(15) = 19.102, p = 209), and years of experience (χ2(18) = 13.940, p = .733). Nevertheless, teacher profiles were not proportionally distributed across grade levels (χ2(6) = 12.483, p = .05). Specifically, the number of teachers in the High CLCI group was significantly higher than expected at the 5-8 grade level (standardized residual = 2).

78

Yun-Jo An and Diana Mindrila Table 2. Demographic information by cluster

Organizational Level

Average N = 65 N

9-12 6-8 K-5 Gender Female Male Age group 20-20 years 30-39 years 40-49 years 50-59 years 60-69 years 70 or older Years of Experience First year 1-2 3-5 6-10 11-15 16-20 More than 20

Cluster High LCT High CLCI N = N = 26 25 N % N % 12 46% 9 36% 5 19% 12 48% 9 35% 4 16%

Low LCT N = 18 N % 5 28% 9 50% 4 22%

32 13 20

% 49% 20% 31%

51 14

78% 22%

19 7

73% 27%

22 3

88% 12%

14 4

78% 22%

5 20 16 23 1 0

8% 31% 25% 35% 2% 0%

3 6 12 3 1 1

12% 23% 46% 12% 4% 4%

1 5 6 11 2 0

4% 20% 24% 44% 8% 0%

3 4 4 7 0 1

17% 22% 22% 39%

3 1 8 9 15 8 21

5% 2% 12% 14% 23% 12% 32%

0 1 3 5 5 6 6

0% 4% 12% 19% 19% 23% 23%

1 0 2 2 5 8 7

4% 0% 8% 8% 20% 32% 28%

0 1 4 4 2 4 3

0% 6% 22% 22% 11% 22% 17%

6%

DISCUSSION The descriptive analysis of the survey items showed that most participants believed that LCI is challenging but rewarding. The results also showed that the teachers were familiar with learner-centered approaches, were using them in their classrooms, and would like to learn more about learner-centered instruction. The items measuring teachers’ concerns about LCI recorded the lowest ratings. These results are consistent with previous findings on teachers’ perceptions of LCI (An & Reigeluth, 2011; Tawalbeh & AlAsmari, 2015; Yilmaz, 2008). The current study is one of the few quantitative research studies on LCI. It contributes to the literature through the identification of

Clusters of Teachers Based on Their Perceptions …

79

multivariate factors relating to teacher perceptions of LCI. Results showed that items measuring teacher perceptions of LCI tend to group into two categories: a) items that measure the extent to which teachers adopt and support learner-centered approaches; and b) items measuring teachers’ concerns about LCI. The two identified factors, LCT and CLCI, represent the perceived dimensions of the LCI model and must be addressed by practitioners aiming to increase the implementation of LCI. Another contribution of the study is the development of a typology of teachers based on their scores on the LCT and CLCI factors. This typology differentiated four types of teachers in regard to their perceptions of LCI: a) Average, b) High LCT, c) High CLCI, and d) Low LCT. Nevertheless, further research with a larger sample from the same population is needed to collect evidence of external validity for the current cluster model. The four cluster centroids described distinct profiles, with large differences in LCT and CLCI factor scores and different demographic characteristics. The majority of the teachers (67.9%) were assigned to the Average and High LCT clusters, which had overall positive perceptions of LCI and lower scores on the CLCI factor. Only a third (32.1%) of the participants were included in the High CLCI and Low LCT clusters. Both of these groups had above-average scores on the CLCI factor; however, this coefficient was approximately half a standard deviation higher for the High CLCI group. Further, the High CLCI cluster had scores closer to average on the LCT factor, whereas the average LCT score was almost two standard deviations below the mean for the Low LCT cluster. One notable finding is that the two clusters with lower LCT scores and higher CLCI scores (High CLCI and Low LCT) consisted mostly of middle school teachers. Although gender and levels of experience did not vary significantly across clusters, it is interesting that the High CLCI cluster had the largest proportion of females and the largest proportion of teachers with 11 or more years of experience (80%). Further research with a larger and more representative sample is needed to examine why middle school teachers have more concerns about LCI compared to other groups and to determine whether females and more experienced teachers are more likely to be assigned to the High CLCI cluster.

80

Yun-Jo An and Diana Mindrila

Clustering teachers based on their perceptions of LCI informs practitioners and decision makers on the most prevalent attitudes toward LCI and the demographic characteristics of each group. Particularly, the findings regarding the characteristics of each cluster provide useful insights into professional development for LCI. For instance, results suggest that professional development efforts at the middle school level should focus more on addressing teachers’ concerns about LCI and on helping them overcome barriers in designing and implementing LCI.

APPENDIX A Descriptive Statistics by Item (N = 134) Mean SD V1. I have enough knowledge about learner3.56 1.04 centered instruction. V2. I use learner-centered approaches in my 3.86 0.88 classroom. V3. Learner-centered approaches require too 2.54 0.99 much work for me. V4. Learner-centered approaches diminish 2.40 1.06 the amount of content I can teach. V5. I am not very comfortable with learner2.42 1.14 centered approaches. I am more comfortable with the teacher-centered approach (e.g., lectures). V6. I want to learn more about learner3.84 0.87 centered instruction. V7. Learner-centered approaches are 1.82 0.99 incompatible with my teaching philosophy. V8. Learner-centered approaches are 1.69 0.82 incompatible with my subject area. V9. I am familiar with learner-centered 3.87 0.89 approaches. V10. Learner-centered approaches are too 2.54 0.98 time-consuming. V11. My students are passive and not always 2.80 1.13 responsible. They are not ready for learnercentered instruction. V12. I am a learner-centered teacher. 3.49 1.02 V13. Learner-centered instruction is 3.99 0.81 challenging but rewarding. V14. I believe that all teachers should use 3.61 0.94 learner-centered approaches in their teaching. Valid N (listwise)

Skewness -0.44

Kurtosis -0.60

-1.21

1.82

0.54

-0.11

0.51

-0.72

0.52

-0.71

-1.00

1.84

1.56

2.47

1.21

1.62

-0.93

1.26

0.53

-0.06

0.02

-0.78

-0.40 -0.91

-0.24 1.68

-0.47

0.27

APPENDIX B

V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 ** *

V1 .465** – .269** – .250** – .280** – 0.155** – 0.033 – .177* .556** – .264** – .235** .401** .222** 0.169

V2

V3

– .319** – .321** – .249** .176* – .273** – .240** .537** – .461** – .287** .651** .492** .434**

.485** – .485** – 0.018 .240** .368** – .359** .595** .376** – .341** – .234** – .216*

V4

– .399** – .183* .285** .369** – .283** .708** .541** – .327** – .317** – .276**

V5

Pearson Correlation Matrix V6 V7

0.062 .294** .333** – .311** .327** .306** – .475** – .237** – .367**

Correlation is significant at the 0.01 level (2-tailed). Correlation is significant at the 0.05 level (2-tailed).

– .201* – 0.114 – 0.008 – .266** – 0.088 .219* .337** .335**

.538** – .180* –. 289** .287** – .287** – .425** – .278**

V8

– .231** .390** .341** – .327** – .381** – .273**

V9

– .349** – .191* .485** – .393** .355**

V10

V11

V12

V13

.515** – .351** – .368** – .300**

– .432** – .314** – .279**

.582** .615**

.582**

82

Yun-Jo An and Diana Mindrila

REFERENCES Abdi, Ali. (2014). “The Effect of Inquiry-based Learning Method on Students’ Academic Achievement in Science Course.” Universal Journal of Educational Research, 2, no. 1, 37-41. Agresti, Alan. (2013). Categorical Data Analysis (3rd ed.). Hoboken NJ: Wiley. Aldenderfer, Mark S. & Roger, K. Blashfield. (1984). Cluster Analysis. Beverly Hills, CA: Sage Publications. An, Yun-Jo. (2012). “Learner-centered technology integration.” In Encyclopedia of E-Leadership, Counseling and Training, edited by Victor C. X. Wang, 797-807. Hersey, PA: IGI Global. An, Yun-Jo. & Charles, M. Reigeluth. (2011). “Creating TechnologyEnhanced, Learner-Centered Classrooms: K-12 Teachers’ Beliefs, Perceptions, Barriers, and Support Needs.” Journal of Digital Learning in Teacher Education, 28, 54-62. APA Board of Educational Affairs. (1997). Learner-Centered Psychological Principles: A Framework for School Reform and Redesign. Washington, DC: American Psychological Association. Aydogdu, Bulent. & Tugba, Selanik-Ay. (2016). “Determination of Teacher Characteristics That Support Constructivist Learning Environments.” Eurasian Journal of Educational Research, 63, 293310. Bransford, John D., Ann, Brown. & Rodney, Cocking. (2000). How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press. Cheang, Kai I. (2009). “Effects of Learner-Centered Teaching on Motivation and Learning Strategies in a Third-Year Pharmacotherapy Course.” American Journal of Pharmaceutical Education, 73, Article 42. Cornelius-White, Jeffrey. & Adam, Harbaugh. (2009). Learner-Centered Instruction: Building Relationships for Student Success. Thousand Oaks, CA: Sage.

Clusters of Teachers Based on Their Perceptions …

83

Costello, Anna B. & Jason, W. Osborne. (2005). “Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most from Your Analysis.” Practical Assessment, Research & Evaluation, 10, 1-9. DiStefano, Christine. & Diana, Mindrila. (2014). “Cluster Analysis.” In Handbook of Quantitative Methods for Educational Research, edited by Timothy Teo, 103-122. Sense Publishers. DiStefano, Christine., Min, Zhu. & Diana, Mindrila. (2009). “Understanding and Using Factor Scores: Considerations for the Applied Researcher.” Practical Assessment, Research & Evaluation, 14, no. 20, 1-11. Everitt, Brian S., Sabine, Landau., Morben, Leese. & Daniel, Stahl. (2011). “Cluster Analysis.” Wiley. Fabrigar, Leandre R. & Duane, T. Wegener. (2011). Exploratory Factor Analysis. Oxford University Press. Finney, Sara J. & Christine, DiStefano. (2006). “Non-Normal and Categorical Data in Structural Equation Modeling.” Structural Equation Modeling: A Second Course, 10, no. 6, 269-314. Genc, Evrim. & Feral, Ogan-Bekiroglu. (2004). “Patterns in teaching styles of science teachers in Florida and factors influencing their preferences.” Accessed June 29, 2017. http://files.eric.ed.gov/fulltext/ ED490781.pdf. Gorsuch, Richard L. (1983). Factor Analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Ha, Phan L. (2014). “The Politics of Naming: Critiquing “LearnerCentered” and “Teacher as Facilitator” in English Language and Humanities Classroom.” Asia-Pacific Journal of Teacher Education, 42, no. 4, 392-405. Hair, Jr. Joseph F., Rolph, E. Anderson., Ronald, L. Tatham. & William, C. (1995). “Black Multivariate data analysis with readings.” New Jersy: Prentice Hall, (1995). Hendrickson, Alan E. & Paul, Owen White. (1964). “Promax: A Quick Method for Rotation to Oblique Simple Structure.” British Journal of Mathematical and Statistical Psychology, 17, no. 1, 65-70.

84

Yun-Jo An and Diana Mindrila

Kaiser, Henry F. (1960). “The Application of Electronic Computers to Factor Analysis.” Educational and Psychological Measurement, 20(1), 141-151. Kaufman, Leonard. & Peter, J. Rousseeuw. (1990). “Partitioning Around Medoids (Program PAM).” Finding Groups in Data: An Introduction to Cluster Analysis, 68-125. MacQueen, James. (1967). “Some Methods for Classification and Analysis of Multivariate Observations.” In Proceedings of the fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, no. 14, 281-297. Mardia, Kanti V. (1970). “Measures of Multivariate Skewness and Kurtosis with Applications.” Biometrika, 57, no. 3, 519-530. McCombs, Barbara L. & Jo Sue, Whisler. (1997). The Learner-Centered Classroom and School: Strategies for Increasing Student Motivation and Achievement. San Francisco, CA: Jossey-Bass. Moore, David. S., William I. Notz. & Michael, A. Flinger. (2013). The Basic Practice of Statistics (6th ed.). New York: W. H. Freeman. Reigeluth, Charles M. (1994). “Envisioning a New System of Education.” In Systemic Change in Education, edited by Charles M. Reigeluth, and Robert J. Garfinkle, 59–70. Englewood Cliffs, NJ: Educational Technology Publications. Reigeluth, Charles M. (1999). “What Is Instructional-Design Theory and How Is It Changing?” In Instructional-Design Theories and Models: A New Paradigm of Instructional Theory, (Volume II), edited by Charles M. Reigeluth, 5-29. Mahwah, NJ: Lawrence Erlbaum Associates. Robinson, Rhonda., Michael, Molenda. & Landra, Rezabek. (2007). Facilitating learning. In Educational Technology: A Definition with Commentary (2nd ed.), edited by Al Januszewski, and Michael Molenda, 15-48. NY: Lawrence Erlbaum. 
 Sharpe, Donald. (2015). “Your Chi-Square Test is Statistically Significant: Now what?.” Practical Assessment, Research & Evaluation, 20. Shulman, Lee. 1990. Aristotle Had It Right: On Knowledge and Pedagogy. East Lansing, Michigan: The Holmes Group.

Clusters of Teachers Based on Their Perceptions …

85

Sockman, Beth R. (2015). “Innovative Teacher’s Perceptions of Their Development When Creating Learner-Centered Classrooms with Ubiquitous Computing.” International Education Research, 3, no. 3, 26-48. Strobel, Johannes. & Angela, van Barneveld. (2009). “When is PBL More Effective? A Meta-Analysis of Meta-Analyses Comparing PBL to Conventional Classrooms.” The Interdisciplinary Journal of Problembased Learning, 3, no. 1, 44-58. Tabachnick, Barbara G. & Linda, S. Fidell. (2001). Using Multivariate Statistics., 2001. Allyn and Bacon. Tawalbeh, Tha’er I. & AbdulRahman, Awad AlAsmari. (2015). “Instructors’ Perceptions and Barriers of Learner-Centered Instruction in English at the University Level.” Higher Education Studies, 5(2), 38-51. Yilmaz, Kaya. (2008). “Social Studies Teachers’ Views of LearnerCentered Instruction.” European Journal of Teacher Education, 31, no. 1, 35-53.

In: Exploratory Factor Analysis Editor: Diana Mindrila

ISBN: 978-1-53612-486-6 © 2017 Nova Science Publishers, Inc.

Chapter 4

STUDENT VOICE OPPORTUNITIES QUANTIFIED BY EXPLORATORY FACTOR ANALYSIS Jennifer E. Elemen*, Ed.D Department of Leadership, Research, and School Improvement University of West Georgia, Carrollton, Georgia, US

ABSTRACT The purpose of this study was to analyze school leadership praxis for its inclusion of students in organizational leadership dialogue and decision-making and their influences on student achievement and civic participation. Survey questionnaire data were provided by 215 full-time enrolled undergraduate students from a public Southeastern university about their high school and college experiences. Utilizing exploratory factor analysis, aspects of school leadership and culture were found to possibly influence student achievement and civic participation: providing students with opportunities to consider, discuss, and engage in dialogue *

Corresponding Author: Email: [email protected].

88

Jennifer E. Elemen concerning different stakeholders’ diverse views and values, to have involvement in decision-making for the school, the school having an effective committee structure, school leaders facilitating effective communication, and stakeholders having an appropriate level of autonomy in decision-making.

Keywords: school culture, distributed leadership, social design, student voice, civic education

INTRODUCTION Educational research on school leadership contributes to improved practice and student achievement outcomes. Twenty-five percent of student learning can be attributed to school leadership variables, compared to 33% attributed to classroom factors (Leithwood, Louis, Anderson, and Wahlstrom 2004, 21). Distributed leadership researchers Spillane and Healey argued that “more descriptive work is necessary to improve our understanding of how leadership is distributed in schools so that we can then explore how these arrangements influence school outcomes” (2010, 254). The dimensions of distributed leadership in different educational settings require further exploration to inform practice (Robinson, 2008). Researchers are beginning to explore the role of students in high school distributed leadership structures (Fusarelli, Kowalski, and Petersen 2011; Pedersen, Yager, and Yager 2012) and the “operations and measures when taking a distributed perspective in school leadership and management research” (Spillane and Healey 2010, 253). Student voice in educational research and practice, particularly in the domain of organizational leadership dialogue and participative decision-making for high schools, has been underutilized. The theoretical foundation of this study (Elemen 2015a; Elemen 2015b) was comprised of social design theory (Jun 1986; Jun 2006; Jun and Storm 1990) and distributed leadership theory (Harris 2008; Hulpia, Devos, and Keer 2009; Leithwood and Jantzi 1999; Leithwood and Mascall 2008; Robinson 2008; Spillane 2006). Social design theory was

Student Voice Opportunities Quantified by Exploratory Factor … 89 developed by Jong S. Jun (1986) and is used to study public administration through the lens of critical theory and constructivism paradigms. This theory relies on the assertion that solutions to public problems can be effectively developed through engaging stakeholders in a democratic process (Jun 1986; Jun 2006; Jun and Storm 1990). Social design theory had yet to be applied to the field of educational leadership. As applied to this study (Elemen 2015a; Elemen 2015b) and informing the hypotheses, social design theory holds that one would expect the independent variables of students’ perceived opportunities to participate in high school organizational leadership dialogue and decision-making to influence the dependent variables of academic achievement and civic participation. Distributed leadership approaches include a diverse array of stakeholders in the decision-making process of organization leadership, and additionally allocate power and tasks to a variety of individuals in the organization (Harris 2008; Spillane 2006). The characteristics of a distributed leadership model include: multiple player leadership philosophy, utilization of hidden leaders, expanded leadership roles, enhancement of human capacity, use of tools in decision-making, two-way communication, and an environment of trust (Engel-Silva 2009, 288). Engel-Silva (2009) recommended future exploration of distributed leadership focusing on stakeholders, such as parents and students, and to research its effects on student achievement. Robinson (2008) called for more research examining leadership from the perspectives of those being led to determine the effects of leadership practices. This is the influential component of leadership, as distinguished from the distributed leadership tasks, in what Robinson refers to as “distributed leadership as distributed influence processes” (2008, 246). In her call for research investigating this topic, she advised a focus on “followership rather than leadership and seek information about sources of influence” (Robinson 2008, 248). Therefore, by reviewing students’ perceptions of their high schools’ leadership practices, it was intended that this study (Elemen 2015a; Elemen 2015b) would fill a research gap by ascertaining whether students who perceived opportunities in high school

90

Jennifer E. Elemen

to participate in organizational leadership dialogue and decision-making would benefit. The purpose of this quantitative study was to examine university students’ perceptions about their high schools’ leadership practices by identifying their opportunities to have participated in organizational leadership dialogue and decision-making. This study also aimed to ascertain the factors of organizational leadership dialogue and decisionmaking in order to understand the aspects of school leadership practices that may establish school cultures promoting student participation. Finally, this study sought to assess if any significant relationships exist between students’ perceived opportunities to have participated in high school organizational leadership dialogue and decision-making and their academic achievement and civic participation in college (Elemen 2015; Elemen 2015b). The research questions were: 1) Do undergraduate students perceive they had opportunities to participate in high school organizational leadership dialogue and decision-making? 2) What are the factors that define organizational leadership dialogue and participative decision-making? 3) Are there significant relationships between individuals’ factor scores and student achievement? 4) Are there significant relationships between individuals’ factor scores and civic participation? 5) Are there significant differences in academic achievement and civic participation between students who perceive they had opportunities to participate in high school organizational leadership dialogue and decision-making compared to students who perceive they did not?

Student Voice Opportunities Quantified by Exploratory Factor … 91

LITERATURE REVIEW Student voice inclusion has the potential to become a powerful way to transform schools for diversity, equity, and social justice (Beaudoin 2005; Robinson and Taylor 2012; Thomson and Gunter 2006), as Fielding’s (2001) framing of students as change agents suggests. Robinson and Taylor (2007) identify four core values for initiating student voice work: communication as dialogue; participation and democratic inclusivity; the recognition that power relations are unequal and problematic; and the possibility for change and transformation. Beginning student voice work may be viewed as a process of making high schools more democratic in their functioning (Apple 2011). Campbell (2009, 104) provides “a framework for school leaders seeking to promote, authorize and validate student voice,” explaining research-based behaviors that school leaders can exhibit in order to promote student voice in their schools, “having a student-centered orientation, promoting student-led events, modeling a vision for student voice that is reflected in school improvement plans, and staff development facilitating shared decision-making” (Campbell 2009, 104). Cheng (2012) adds that student voice work includes embracing a willingness to construct a new partnership with students and ensuring consistent whole institution commitment. These are important steps in changing school cultures and power relations toward Apple’s (2011) ideal of a democratic school.

METHOD This study employed a quantitative survey research design (Elemen 2015a; Elemen 2015b). Survey data were used to examine university students’ perceptions about their opportunities in high school to have participated in organizational leadership dialogue and decision-making and their college academic achievement and civic participation. This was also a correlational study aiming to investigate the relationships between students’ perceptions of their high schools’ leadership approaches’

92

Jennifer E. Elemen

inclusiveness of students’ participation in organizational leadership dialogue and decision-making and university students’ academic achievement and civic participation. This fills a gap in the research by providing greater student voice in the body of research and provides a “youth-centered perspective” (Taft and Gordon 2013, 88) on what school leadership practices influenced young adults’ academic achievement and civic participation. Data collection took place through quantitative survey data collection. Cronbach’s alpha test of reliability was performed to provide reliability to the survey instrument. The study also provided descriptive and correlation data, which were analyzed using descriptive statistics, exploratory factor analysis (EFA), simple linear regression analysis, and a t-test.

Sampling This study utilized a questionnaire intended to survey college students about their perceptions of their high school experiences and opportunities to have participated in organizational dialogue and decision-making (Elemen 2015a; Elemen 2015b). Because the study was both correlational and descriptive, the desired sample size was over 100 participants (Fraenkel and Wallen 2009) from the same university, but from different high schools. Concurrently, Colton and Covert (2007, 265) indicated an ideal sample size of over 100 and possibly over 300 for such a study. To focus the study on students who demonstrated academic achievement in attending a four-year university after high school, a purposive sample was selected of undergraduate participants at one public university in the United States. Granted access was gained through the institutional review board (IRB) process. Participants were sought from a random list of 5,000 undergraduate students, ages 18 to 21, and enrolled as full-time students. E-mail addresses were provided by the university after IRB approval. Of the 5,000 randomly selected undergraduate students enrolled fulltime at one medium-sized public Southeastern university invited to participate in the survey via their university e-mail addresses, 215

Student Voice Opportunities Quantified by Exploratory Factor … 93 completed the first two questions (4% response rate). The response rate declined thereafter (at a substantial rate after the 13th, 20th, and 27th questions). For the purposes of statistical analyses in which participants answered the vast majority of the survey questions, the sample size was 149 participants (3% complete response rate). Table 1. Participants’ ethnicity Ethnicity White Black or African American Other Ethnicity Hispanic or Latino American Indian or Alaska Native Asian Pacific Islander No Answer Totals (n = 148) Note: percentages are approximate.

% 55 33 4 3 1 1 1 2 100

n 81 49 6 5 1 2 1 3 148

Out of 149 responses, 113 participants identified as female (76%) and 33 as male (22%). The age mean was 19.56 years old. Twenty-eight participants (19%) were 18, 49 (33%) were 19, 40 (27%) were 20, and 28 (19%) were 21. Although participants within the 18 to 21 age range were sought, one 25-year-old and three 22-year-olds completed the survey. Their participation accumulated to 3% of the sample and their responses were included in the data analysis. The ethnic composition of the sample is provided in Table 1. A reported 42% of participants were eligible for free or reduced price lunch while they were in high school, indicating lowincome socioeconomic status. Participants reported English being their native language at a rate of 93% with 5% not English and 92% reported the U.S. being their native country while 5% reported being from another country. State representation included 130 (88%) of participants having attended high school in Georgia, 6 (4%) in Alabama, 2 (1%) in Florida, 1 (1%) in Idaho, 1 (1%) in Louisiana, 1 (1%) in New Jersey, 1 (1%) in New

94

Jennifer E. Elemen

York, 1 (1%) in Pennsylvania, 1 (1%) in Texas, 1 (1%) in Wisconsin, and 1 (1%) outside of the U.S.

Procedure Participants were contacted by e-mail and offered the opportunity to participate in the survey. The informed consent provided participants with the purpose and scope of the survey. After agreeing to the informed consent, students were able to voluntarily participate in the online survey format where their personal information was not collected. All data was aggregated for reporting purposes and neither individuals nor institutions were reported.

Setting A medium-sized public Southeastern university provided the setting for this study (Elemen 2015a; Elemen 2015b). Students at this university may have attended different high schools, both public and private. The undergraduate population was approximately 10,000. The student body was over 60% female and less than 40% male. The ethnic composition of the student body was approximately 57% Caucasian/White, 26% African American/Black, 10% other or undeclared, 3% Hispanic, and less than 2% of each of the following: Asian, American Indian/Alaska Native, multiracial, and Native Hawaiian/Pacific Islander. The university offered over 100 programs of study leading to certificates, bachelor’s, master’s, education specialist’s, and doctorate degrees.

Instrumentation The survey instrument contained 63 questions, including several multiple-choice/single response items on a Likert-type scale as well as

Student Voice Opportunities Quantified by Exploratory Factor … 95 quantitative response questions for continuous ratio-scale variables, such as GPA and age. Colton and Covert’s (2007) survey development advice was utilized. LimeSurvey was utilized as the survey administration platform and IBM SPSS Statistics Version 22 was used for statistical analysis. The survey questions were developed and aligned to the literature as follows. The dependent and independent variables described refer to the response or criteria and explanatory or predictor variables, respectively, input as dependent and independent variables to run the necessary statistical procedures in SPSS.

Validity Construct-related evidence of validity was provided with an extensive literature review of the theoretical constructs present in the hypotheses (Fraenkel and Wallen 2009) and was checked by experts in the field. Content-related evidence of validity was provided with at least one expert in the field to review the survey (Fraenkel and Wallen 2009). Two professors of education reviewed the survey instrument and provided their expert feedback. Additionally, one undergraduate university student and an education doctoral student reviewed the survey for readability and comprehension of the survey questions. The feedback from the four reviewers was incorporated. Additionally, a pre-test was conducted with the survey instrument in a pilot study for this dissertation with 37 participants. Participants provided suggestions as to how to improve the survey. The survey instrument was revised accordingly.

Reliability An internal consistency method was applied by computing an index of internal consistency. Cronbach’s alpha demonstrates the extent to which items that are proposed to measure the same feature provide similar results. Such a measure calculates the reliability of items that are not scored correct

96

Jennifer E. Elemen

or incorrect (Fraenkel and Wallen 2009) and was used to demonstrate reliability of the survey instrument used in this study (Elemen 2015a; Elemen 2015b).

Data Screening Prior to analysis, data were collected through LimeSurvey and exported to an IBM SPSS file. Independent and dependent variable items were entered as numeric scale variables. Data were screened using the checklist and criteria advised by Froh (2008) and Tabachnick and Fidell (2001): check for data errors and need for reentry, inspect univariate descriptive statistics for accuracy of input, evaluate the amount and distribution of missing data and resolve the problem(s), create composite variables, check pairwise plots for nonlinearity and heteroscedasticity, identify univariate outliers and nonnormal variables, identify multivariate outliers, and evaluate variables for multicollinearity and singularity. This procedure helped to ensure the accuracy of the data.

Data Analysis Descriptive analysis. Descriptive statistical analysis was conducted using SPSS. Descriptive statistical analysis was conducted to describe the sample’s demographics, including gender, race, ethnicity, age, and socioeconomic status. Descriptive statistical data were also analyzed for participants’ responses to the variables tested. Mean, standard deviation, frequencies, charts, and graphs were used to describe and illustrate the data sample properties, which informed external validity and generalizability. Exploratory factor analysis (EFA). EFA was conducted for data reduction to demonstrate how items grouped. EFA was conducted using SPSS to identify the factors contained in the survey scales intended to measure the independent variables of perceived opportunities to have participated in high school organizational leadership dialogue and

Student Voice Opportunities Quantified by Exploratory Factor … 97 decision-making. This procedure allowed for the detection of structure in the data. Statistical criteria and the interpretability of item groupings were applied to discover the dimensions of the survey data, as the number of factors should be statistically then interpretably defensible (Pohlmann 2004, 17). Items were classified into factors based on the number of eigenvalues greater than one. The Kaiser-Meyer-Olkin measure of sampling adequacy was conducted to test if it was above the recommended value of .6 and if the result from Bartlett’s test of sphericity was p < .001, then it would demonstrate statistically significant findings. EFA was then conducted again with the rotation method Promax with Kaiser Normalization. A scree plot, a pattern matrix table, and the total variance explained by each solution were also used to determine the number of factors. Further, the content of item groupings was analyzed to select the optimal number of factors. Item loadings, which quantify the relationship between individual items and the corresponding factor (Beavers et al. 2013), were used to identify the defining characteristics of each factor and for item selection. Thus, scale items with weak factor loadings were eliminated from the factor solutions (Beavers et al. 2013). Further, items that cross-loaded on different factors were sequentially removed until a simple structure was reached and the optimal solution was identified. Factor scores were then used to relate the theoretical foundations of the study to the independent variable constructs of opportunities to have participated in high school organizational leadership dialogue and decision-making. Factor scores are “values of the underlying constructs for use in other analyses” (DeCoster 1998, 2). Regression factor scores were computed to predict the location of each individual on each factor, “weighted by regression coefficients, which are obtained by multiplying the inverse of the observed variable correlation matrix by the matrix of factor loadings and, in the case of oblique factors, the factor correlation matrix” (DiStefano, Zhu, and Mîndrilă 2009, 4). Further, these factor scores were used to conduct regression analysis in relating the independent variables of opportunities to have participated in high school

98

Jennifer E. Elemen

organizational leadership dialogue and decision-making to the dependent variables of university academic achievement and civic participation. Simple linear regression analysis. Simple linear regression analysis was performed using IBM SPSS on the factor scores of the independent variables measuring levels of students’ perceived opportunities for participation in high school organizational leadership dialogue and decision-making to the dependent variables of students’ academic achievement and levels of civic participation. The two dependent variables were computed as follows. Academic achievement was quantified using the sum of participants’ self-reported university GPA as a continuous variable and college and future career confidence questions. Civic participation was quantified using the sum composite scores of the answers to the civic participation questions. The civic participation score summarized the responses to the civic participation questions portion of the survey. Moore, Notz, and Fligner (2010) explained that “regression describes a relationship between an explanatory variable and a response variable” (p. 125). This allows for the quantification of variables and testing of the hypotheses. Regression analysis is based on the assumption that dependent variables can be measurably influenced by independent variables. Ary, Jacobs, Razavieh, and Sorensen (2005) stated that the Pearson r as a measure of relationships independent of sample size is a form of effect size wherein r = .10 has a small [weak] effect, r = .30 has a medium [moderate] effect, and r = .50 has a large [strong] effect. These values were used to determine if there were statistically significant relationships between the variables. If r = .40 to .60, this may indicate theoretical and/or practical value (Fraenkel and Wallen 2009, 337) and may provide a stronger case for distributed leadership and social design practices in schools. A scatterplot demonstrated the relationships between students’ perceived opportunities to have participated in high school organizational leadership dialogue and decision-making (shown on the x-axis) and their corresponding academic achievement and civic participation (shown on the y-axis). The regression line, the line of best-fit, demonstrated the strength of the relationships between students’ perceived opportunities to have

Student Voice Opportunities Quantified by Exploratory Factor … 99 participated in high school organizational leadership dialogue and decision-making and their corresponding academic achievements and civic participation. The testing of the hypotheses allowed for the accepting or rejecting of the hypotheses. The accepted practice of using the significance value of Pearson’s p < .05 was used to guide the interpretation of the data. If the correlation coefficient, Pearson’s r ≤ .10, then the null hypothesis could be accepted and the alternative hypothesis could be rejected. If the correlation coefficient, Pearson’s r > .10, then the alternative hypothesis could be accepted and the null hypothesis could be rejected. If the alternative hypotheses were accepted, that would suggest that greater values of students’ perceptions of opportunities to have participated in high school organizational leadership dialogue and decision-making (the independent variables) were associated with greater values of students’ academic achievement and civic participation (the dependent variables). The coefficient of determination, R2 > .01, shows the proportion of variation in the response variable that could be explained/ predicted by the explanatory variable. For instance, if R2 is greater than .01, it could be inferred that students’ perceived opportunities to have participated in high school organizational leadership dialogue and decision-making could be attributed to greater than 1% of the variation in students’ academic achievement and/or civic participation. This conclusion would be strengthened by a higher coefficient of determination value, R2 > .16 (greater than 16% of the variation in students’ academic achievement and civic participation) for theoretical and/or practical value (Fraenkel and Wallen 2009, 337). T-test. A t-test was conducted to answer the final research question: are there significant differences in academic achievement and civic participation between students who perceive they had opportunities to participate in high school organizational leadership dialogue and decisionmaking compared to students who perceive they did not? “The t-test is a parametric statistical test used to see whether a difference between the means of two samples is significant” (Fraenkel and Wallen 2009, 229). The mean student achievement and civic participation scores were compared between students with low versus high factor scores,

100

Jennifer E. Elemen

distinguished by the mean, representing their opportunities to have participated in high school organizational leadership dialogue and decision-making. The null hypotheses were that the student achievement and civic participation means of participants with higher factor scores for perceptions of opportunities to have participated in high school organizational leadership dialogue and decision-making would be equal to those with lower level factor scores. The alternative hypotheses were that the student achievement mean and the civic participation mean of participants with higher factor scores for perceptions of opportunities to have participated in high school organizational leadership dialogue and decision-making would be significantly greater than those with lower level factor scores.

RESULTS Descriptive Analysis Participants (n = 149) reported their parents/guardians being high school graduates at a rate of 96% for mothers and 88% for fathers. Parents/guardians holding a bachelor’s degree or higher was reported as 41% of mothers and 38% of fathers. There was a 5% non-response rate to the question regarding fathers’ education and 2% for mothers’ education. Sixty-six percent reported that their parents/guardians had discussed civic issues, such as the government, community, or politics with them growing up, while 30% did not, and 4% did not respond to this question. One hundred twenty-eight (86%) participants reported to having attended public high schools, 10 (7%) attended private high schools, four (3%) attended charter schools, and three (2%) were homeschooled. A majority, 75% of participants, described their high school setting as suburban, 27% rural, and 5% urban. Three percent of participants described their high schools as having a majority of students in poverty, 29% described their high schools as having a majority of students who were working class, 46% described their high schools as having a majority

Student Voice Opportunities Quantified by Exploratory Factor … 101 of students who were middle class, 18% described their high schools as having a majority of students who were upper-middle class, and 2% described their high schools as having a majority of students who were affluent. Participants reported involvement in the following activities while they were in high school: 68 (46%) participated in sports, 118 (79%) participated in clubs, 79 (53%) participated in community service, 22 (15%) participated in student government, 64 (43%) participated in the arts, 48 (32%) participated in religious activities, and 39 (26%) participated in other. The high school unweighted GPA (n = 148) mean was 3.5, median = 3.6, mode = 3.8, and standard deviation = .397. The composite SAT score (n = 103) mean was 1550 and standard deviation = 333.968. The composite ACT score (n = 88) mean was 23 and standard deviation = 4.852.

Reliability The internal consistency of the operationalized variables was measured using the Cronbach’s alpha coefficient. A Cronbach’s alpha value of .70 or higher indicates internal consistency (Colton and Covert 2007, 265). Reliability of the independent variable, perceived opportunities to have participated in organizational leadership dialogue and decision-making, was established with a Cronbach’s alpha value of .938 (n = 135). This calculation included all fifteen items from the social design scale and distributed leadership participative decision-making scale. For the social design scale, comprised of eight items, Cronbach’s alpha was .902 (n = 143). The participative decision-making scale from Hulpia et al.’s (2009) “Distributed Leadership Inventory” based on Leithwood and Jantzi’s (1999) constructs of participative decision-making, previously modified and tested on a sample of 1,902 adult participants of school teachers and administrators (1522 teachers, 248 teacher leaders, 85 assistant principals, 47 principals); Cronbach’s alpha was .81. In the current study it was .885 (n = 139). For the civic education scale that

102

Jennifer E. Elemen

included 12 items, which may influence or be related to variables in the study, Cronbach’s alpha was .851 (n = 133). Reliability for the dependent variable, academic achievement, operationalized with university GPA alone, could not be established with Cronbach’s alpha, as it was only one item. However, GPA offers a commonly accepted numerical and comparable measure of academic achievement (Volwerk and Tindal 2012). Reliability was demonstrated statistically for the college and future career confidence scale comprised of three items with a Cronbach’s alpha value of .808 (n = 145). Combining the sum of university GPA and college and future career confidence into a four-item scale established a composite student achievement variable with a Cronbach’s alpha value of .744 1 (n = 44). The other dependent variable, civic participation, was demonstrated to be reliable with a Cronbach’s alpha value of .849 (n = 131) for the 11 item scale operationalizing civic habits of mind and intentions for civic participation. The findings of this study provide evidence of validity for the survey instrument developed to measure the influences of students’ perceptions of high school leadership opportunities for participation in high school organizational leadership dialogue and decision-making on university student achievement and civic participation (Elemen 2015a; Elemen 2015b).

Research Question 1 Do undergraduate students perceive they had opportunities to participate in high school organizational leadership dialogue and decisionmaking? Descriptive statistical analysis was conducted to discover the extent to which undergraduate students perceived they had opportunities to participate in high school organizational leadership dialogue and decisionmaking. When university student participants were asked whether or not they perceived they had opportunities in high school to participate in school organizational leadership and decision-making (n = 149), the participants who perceived that they had opportunities to participate in high school organizational leadership dialogue ranged from 53% to 74%,

Student Voice Opportunities Quantified by Exploratory Factor … 103 depending on the scale item. The lowest reported item for this scale was 53% of participants perceiving that the voices of minority groups were integrated into community problem solving for their high schools. The highest reported item for the social design scale was 74% of participants perceiving that people worked together effectively at their high schools. Undergraduate students who perceived that they had opportunities for high school participative decision-making ranged from 52% to 84%. Students felt they were included in the decision-making at their high schools with only 52% agreeing with this scale item. However, 84% did feel that leadership at their high schools was delegated to staff for activities important to achieving school goals. Table 2 presents the complete reporting of the social design scale item responses and distributed leadership participative decision-making scale item responses. Table 2. Descriptive data for high school social design and distributed leadership Social Design Scale Item

n

176

% Disagree or Strongly Disagree 36

% Agree or Strongly Agree 63

Overall, I felt there to be consideration of people’s diverse values at my high school. Overall, I felt there to be consideration of people’s diverse values who would be affected by the outcome of deliberation or problem solving of my high school. The ideas of different people were discussed at my high school. Participation went beyond interest-group politics, meaning that people advocated not just for their group (teachers, students, etc.) but in the general interest of the entire high school’s improvement. People worked together effectively at my high school. In making decisions for the high school, alternatives were critically examined in terms of their feasibility.

175

39

61

176

31

69

176

43

57

176

26

74

177

32

68

104

Jennifer E. Elemen Table 2. (Continued)

Social Design Scale Item

n

176

% Disagree or Strongly Disagree 40

% Agree or Strongly Agree 60

Policy design or decisions for the future of the school included input from different people in my high school’s community. The voices of minority groups were integrated into community problem solving for my high school. Distributed Leadership Participative DecisionMaking Scale Item

176

47

53

n

160

% Disagree or Strongly Disagree 16

% Agree or Strongly Agree 84

Leadership at my high school was delegated to staff for activities important to achieving school goals. Leadership at my high school was delegated to students for activities important to achieving school goals. Leadership at my high school was delegated to several different staff members and students. Students had involvement in decision-making at my high school.

161

37

63

159

25

75

160

48

52

There was an effective committee structure for decision-making at my high school. Effective communication was facilitated at my high school. Overall, there was an appropriate level of autonomy in decision-making at my high school, in other words, people had opportunities to make the appropriate choices for themselves on issues that affected them.

158

46

54

157

40

60

164

45

55

Containing possible intervening variables, civic education was operationalized with twelve items contained in the “six proven practices” outlined in the Campaign for the Civic Mission of Schools (2011, 6-7): classroom instruction in civics, government, history, economics, geography, law, and democracy, discussion of current events and

Student Voice Opportunities Quantified by Exploratory Factor … 105 controversial issues, service learning, extracurricular activities, school governance, simulations of democratic processes, and opportunities for students to participate in deliberation. This scale had the largest response rate, as it was placed at the beginning of the survey, the results of which are found in Table 3. In regards to student participation in organizational leadership dialogue and decision-making, 51% agreed that their high schools encouraged students’ meaningful participation in school governance, while 49% did not.

Research Question 2 What are the factors that define organizational leadership dialogue and participative decision-making? Exploratory factor analysis (EFA) revealed the factors of the survey data, which comprise the fifteen total items from the social design and distributed leadership participative decision-making scales (n = 139). EFA was conducted in order to reduce the number of items into a smaller number of representative factors and to demonstrate a lack of multicollinearity (Beavers et al. 2013). A two factor solution emerged containing items with factor loadings > .7 in the pattern matrix, as presented in the scree plot in Figure 1. Factor loadings > .7 indicate strong items in the pattern matrix, which can lead to stable solutions in sample sizes as low as 100 when a factor is comprised of three to four strong items (Beavers et al. 2013). The two-dimension factor solution that emerged cumulatively explained 78.857% of the total variance in the variable as demonstrated with a scree plot in Figure 1. The Kaiser-Meyer-Olkin measure of sampling adequacy was .869, greater than the recommended value of .6, and the result from Bartlett’s test of sphericity was statistically significant, p < .001, further supporting the assumption of variables not being highly correlated. This suggests that the factors of organizational leadership dialogue and participative decision-making may be related, as dialogue often leads to decision-making, but they are distinct enough to have their own separate influences on the dependent variables of academic achievement and civic participation.

106

Jennifer E. Elemen Table 3. Descriptive data for civic education variables

Civic Education Scale Item

n

215

% Disagree or Strongly Disagree 60

% Agree or Strongly Agree 40

I received instruction in civics from my high school. I received instruction in government from my high school. I received instruction in history from my high school. I received instruction in economics from my high school. I received instruction in geography from my high school. I received instruction in law from my high school. I received instruction in democracy from my high school. My high school incorporated discussion of current events into the classroom (for example, local, national, and international issues). My high school had programs that provided students with the opportunity to apply what they learned through performing community service that was linked to the formal curriculum and classroom instruction. My high school offered opportunities for students to get involved in the school or community outside of the classroom. My high school encouraged students’ meaningful participation in school governance. My high school encouraged students to participate in simulations of democratic processes.

215

5

95

214

2

98

211

7

93

211

29

71

213

55

45

213

28

72

211

20

80

212

55

45

212

25

75

211

49

51

213

60

40

Student Voice Opportunities Quantified by Exploratory Factor … 107

Figure 1. Scree plot.

Statistical criteria and interpretability were used to determine the number of factors that were statistically then interpretably defensible (Pohlmann 2004, 17). The items were classified into the groups based on the Kaiser criterion of eigenvalues > 1 (Beavers et al. 2013). Item loadings are the correlations between the survey items and the factors identified, with the strong cross-loadings > .7 reported in Table 4. Items with salient loadings on the first factor corresponded to students having involvement in decision-making for the school, the school having an effective committee structure, school leaders facilitating effective communication, and stakeholders having an appropriate level of autonomy in decision-making. The other items with salient loadings on the second factor corresponded to people’s diverse values being considered, including those who would be affected by the outcome of deliberation or problem solving, and discussed. Therefore, the two factors were named participative decision-making and organizational leadership dialogue. EFA was re-run with the elimination of the social design scale items and distributed leadership participative decision-making scale items with factor loadings < .7 as they indicate only moderate to weak relationships with the factors.

108

Jennifer E. Elemen Table 4. Factor Loadings

Item

Overall, I felt there to be consideration of people’s diverse values at my high school. Overall, I felt there to be consideration of people’s diverse values who would be affected by the outcome of deliberation or problem solving of my high school. The ideas of different people were discussed at my high school. Students had involvement in decision-making at my high school. There was an effective committee structure for decision-making at my high school. Effective communication was facilitated at my high school. Overall, there was an appropriate level of autonomy in decision-making at my high school, in other words, people had opportunities to make the appropriate choices for themselves on issues that affected them.

Loading Participative DecisionMaking

Organizational Leadership Dialogue 1.027 .772

.655 .863 .856 .812 .709

Note: Factor loadings rounding to > .7 are in boldface. Extraction Method: Maximum Likelihood. Rotation Method: Promax with Kaiser Normalization. Rotation converged in 3 iterations. Factor 1 refers to participative decision-making. Factor 2 refers to organizational leadership dialogue.

The marker items, the items with the highest factor loading values, were: “Students had involvement in decision-making at my high school” (.863) for participative decision-making and “Overall, I felt there to be consideration of people’s diverse values at my high school” (1.027) for organizational leadership dialogue. One of the loadings was greater than one because Promax is an oblique rotation method and in that case the loadings in the pattern matrix are standardized regression coefficients, which can, occasionally, become greater than one or less than one. A high

Student Voice Opportunities Quantified by Exploratory Factor … 109 degree of multicollinearity, which may account for this result, was ruled out based on the result of Bartlett’s test of sphericity, which indicated acceptable levels of multicollinearity in the data. Because this item was not strongly related to the other factor, the alternative explanation is that this item is extremely reliable, and, for these data, based on interpretability, this is the optimal solution. The dimensions underlying high school organizational leadership dialogue were comprised of three factor items with Cronbach’s alpha = .870 (n = 145). The dimensions underlying high school participative decision-making were comprised of four factor items with Cronbach’s alpha = .890 (n = 142). These findings suggest that the essential aspects of participative decision-making are students having involvement in decision-making for the school, the school having an effective committee structure, school leaders facilitating effective communication, and stakeholders having an appropriate level of autonomy in decision-making and that the essential aspects of organizational leadership dialogue are people’s diverse values being considered, including those who would be affected by the outcome of deliberation or problem solving, and discussed. Regression factor scores were then computed for each participant on organizational leadership dialogue and participative decision-making. The organizational leadership dialogue factor score mean was 0 with a range of 4.076, a minimum of -2.359, and a maximum of 1.716. The participative decision-making factor score mean was 0 with a range of 4.123, a minimum of -2.196, and a maximum of 1.928. The Pearson correlation coefficient between the two factors (n = 139) was .682 with a significance level of < .001. The calculation of factor scores enabled further exploration into the relationships between the factors measuring the independent variables to the dependent variables of student achievement and civic participation with simple linear regression analysis and a t-test.

110

Jennifer E. Elemen

Research Question 3 Are there significant relationships between individuals’ factor scores and student achievement? Dependent variables. Students’ academic achievement and civic participation comprise the dependent variables, described as follows. Academic achievement. Academic achievement was operationalized with students’ self-reported university GPA. GPA is an indicator of past and potential future academic achievements, such as admission to college and graduate programs, scholarships, jobs, and career opportunities. GPA offers a commonly accepted numerical and comparable measure of academic achievement (Volwerk and Tindal 2012). Participants were also asked three questions about their confidence in their college and future career success. These questions, when combined with GPA, are referred to as ‘student achievement.’ Civic participation. Civic participation was operationalized with a Likert-type scale aligned to Mathews’ (1996) definition of citizen and items were drawn from the commitment to civic participation scale provided by Kahne and Sporte (2008), previously developed by and based on Westheimer and Kahne’s (2004) three conceptions of citizenship: personally responsible, participatory, and justice oriented. The scale used by Kahne and Sporte (2008) was previously tested on a sample of 4,057 12th grade students from 52 high schools in the Chicago area; reliability was calculated to be .73. Voting behavior was also measured for the extent to which participants were eligible to participate in registration and voting. The scale used in this study to operationalize civic participation combined Mathews’ (1996), Kahne and Sporte’s (2008), and Westheimer and Kahne’s (2004) concepts of citizenship and civic participation and were pre-tested in a pilot study with a sample size of 37 student participants enrolled full-time at a medium-sized public Southeastern university (Elemen 2015a; Elemen 2015b). In the pre-test, the participants ranged from ages 18 to 21 years, were 81% female and 19% male, 2% identified as American Indian or Alaska Native, 5% Asian, 27% Black or African American, 5% Hispanic or Latino, 59% White, and 2% other ethnicity,

Student Voice Opportunities Quantified by Exploratory Factor … 111 46% had been eligible for free or reduced price lunch while they were in high school, indicating low-income socioeconomic status, 5% were English language learners, parent educational attainment was at approximately 38% with a bachelor’s degree or higher, 76% reported a suburban high school setting and 24% a rural high school setting, a majority of the high schools’ student populations were reportedly middle class at 62% , working class at 27%, and upper-middle class at 11%. Cronbach’s alpha was .778 for the civic participation scale (with 33 complete responses). An alpha value of .70 or greater indicates internal consistency (Colton and Covert 2007, 265). Independent variables. Students’ perceptions of their high schools’ leadership practices comprise the independent variables, described as follows. Students’ perceptions of their high school’s leadership practices. Students’ perceptions of their high schools’ leadership practices and their opportunities to have participated in high school organizational leadership dialogue and decision-making were informed by social design theory (Jun 1986; Jun 2006; Jun and Storm 1990) and distributed leadership theory (Harris 2008; Leithwood and Jantzi 1999; Leithwood and Mascall 2008; Robinson 2008; Spillane 2006), developed with the participative decisionmaking scale items from Hulpia et al.’s (2009) “Distributed Leadership Inventory.” The social design scale was pre-tested in a pilot study for this dissertation and Cronbach’s alpha was .818 (with 36 complete responses). The participative decision-making scale items from Hulpia et al.’s (2009) “Distributed Leadership Inventory” were based on Leithwood and Jantzi’s (1999) constructs of participative decision-making. The “Distributed Leadership Inventory” was previously modified and tested on a sample of 1,902 adult participants of school teachers and administrators (1522 teachers, 248 teacher leaders, 85 assistant principals, 47 principals); Cronbach’s alpha was.81 (Hulpia et al. 2009). This scale was pre-tested in a pilot study for this study (Elemen 2015a; Elemen 2015b) and Cronbach’s alpha was .867 (with 34 complete responses to these questions).

112

Jennifer E. Elemen

Figure 2. Relationship between perceived opportunities for high school organizational leadership dialogue and student achievement in college.

Simple linear regression analysis revealed that greater values of perceptions of opportunities to have participated in high school organizational leadership dialogue (r = .25) and decision-making (r = .106) had positive relationships with student achievement (n = 134). For organizational leadership dialogue (presented in Figure 2), this was statistically significant (p = .004), while for participative decision-making, it was not (p = .223). The regression equation that predicted college student achievement based on perceived opportunities to have participated in high school organizational leadership dialogue was ŷ = 13.41 + .49*x. Pearson’s r = .25, R2= .063, adjusted R2 = .055, standard error of the estimate = 1.85, F < .05, and p = .004. The coefficient of determination, R2 > .01, R2 = .063, indicating that the proportion of variation in college student achievement could be explained or predicted by perceived opportunities to have participated in high school organizational leadership dialogue was .063, and it could be inferred that 6.3% of the variation in college student achievement could be attributed to students’ perceived

Student Voice Opportunities Quantified by Exploratory Factor … 113 opportunities to have participated in high school organizational leadership dialogue. When high school organizational leadership dialogue and decision-making were compared to university GPA alone, however, only very weak and statistically non-significant relationships emerged. The null hypothesis pertaining to the relationship between organizational leadership dialogue and student achievement was rejected and the alternative hypothesis was accepted. However, the null hypothesis pertaining to the relationship between participative decision-making and student achievement was accepted and the alternative hypothesis was rejected. Therefore, students who perceived that their high schools provided opportunities for students to participate in organizational leadership dialogue tended to have higher levels of college student achievement, especially confidence in their college and future career achievement.

Research Question 4 Are there significant relationships between individuals’ factor scores and civic participation? Simple linear regression revealed that greater values of perceptions of opportunities to have participated in high school organizational leadership dialogue and decision-making were positively correlated at statistically significant levels with higher levels of participants’ civic participation (see Figure 3). Students who thought their high schools provided more student participation opportunities tended to participate in civics more in college. The coefficient of determination, R2 = .145, indicates that 14.5% of the variation in students’ civic participation (n = 123) could be attributed to students’ perceived opportunities to have participated in high school organizational leadership dialogue and decision-making. Pearson’s r = .361 for organizational leadership dialogue (p < .001), r = .033 for participative decision-making (p < .001), and r = .380 and R2 = .145 for the combined the factors (p < .001). The null hypothesis was rejected and the alternative hypothesis was accepted. Therefore, students who perceived that their high schools provided opportunities for students to participate in organizational

114

Jennifer E. Elemen

leadership dialogue and decision-making participate in civics more in college.

Figure 3. Relationship between perceived opportunities for high school organizational leadership dialogue and participative decision-making and civic participation in college. The regression equation that predicts the dependent variable based on the independent variable is ŷ = 32.22 + .99*x. Pearson’s r= .380, R2= .145, adjusted R2= .138, standard error of the estimate = 4.14, F < .05, and p < .001.

Table 5. T-test: Organizational leadership dialogue and participative decision-making and achievement

Student Achievement Civic Participation Note: F < .05.

Organizational Leadership Dialogue & Decision-Making low high low high

n

Mean

Standard Deviation

64 70 55 68

13.1308 13.6706 31.0364 33.5735

1.93321 1.85439 4.02291 4.49317

Standard Error Mean .24165 .22164 .54245 .54488

Student Voice Opportunities Quantified by Exploratory Factor … 115

Research Question 5 Are there significant differences in academic achievement and civic participation between students who perceive they had opportunities to participate in high school organizational leadership dialogue and decisionmaking compared to students who perceive they did not? An independent samples t-test was conducted to compare the means of students who had relatively low levels of opportunities to participate in high school organizational leadership dialogue and decision-making (combined factor scores for organizational leadership dialogue and decision-making < 0) to those who had relatively high levels (combined factor scores for organizational leadership dialogue and decision-making > 0) compared to the mean = 0. As presented in Table 5, this study found that university students who had experienced greater opportunities to participate in high school organizational leadership dialogue and decision-making had an 8% greater mean civic participation score in college (n = 123) with effect size Cohen’s d = .595, which was statistically significant (p = .001), and a 4% greater student achievement mean (n = 134), though not statistically significant (p = .101) (Elemen 2015a; Elemen 2015b).

DISCUSSION This study expanded upon previous research, contributing to the body of research on students’ perceptions of their opportunities to have participated in high school organizational leadership dialogue and decision-making and their effects on academic achievement and civic participation (Elemen 2015a; Elemen 2015b). Student opportunities to participate in high school organizational leadership dialogue and decisionmaking were defined and measured based on social design theory (Jun 1986; Jun 2006; Jun and Storm 1990) and distributed leadership theory (Harris 2008; Hulpia et al. 2009; Leithwood and Jantzi 1999; Leithwood and Mascall 2008; Robinson 2008; Spillane 2006). A valid and reliable survey instrument was developed to assess young adults’ perceptions of

116

Jennifer E. Elemen

their opportunities to have participated in high school organizational leadership dialogue and decision-making and the relationships between these variables and their academic achievement and civic participation in college. Previously, a survey instrument designed for these purposes was not made available. The survey instrument helped to reveal aspects of high school leadership practice and culture conducive to increasing student achievement and civic participation. The results revealed the factors of organizational leadership dialogue and participative decision-making, which increased the understanding of aspects of school culture and leadership that may support student achievement and civic participation. Conducting PCA enabled a statistical then interpretability analysis to develop and support the claims of the following definitions. High school organizational leadership dialogue can be defined as people’s diverse values being considered, including those who would be affected by the outcome of deliberation or problem solving, and discussed. High school participative decision-making can be defined as students having involvement in decision-making for the school, the school having an effective committee structure, school leaders facilitating effective communication, and stakeholders having an appropriate level of autonomy in decision-making. Narrowing the definitions from the broad lists originally informed by features of social design theory (Jun 1986; Jun 2006; Jun and Storm 1990) and distributed leadership theory (Harris 2008; Hulpia et al. 2009; Leithwood and Jantzi 1999; Leithwood and Mascall 2008; Robinson 2008; Spillan, 2006) allowed for a more focused study quantifying the influence of these variables on student achievement and civic participation. The findings of this study (Elemen 2015a; Elemen 2015b) are consistent with the findings of Fusarelli et al. (2011) and Pedersen et al. (2012) that students may not be provided with structures, incentives, nor encouragement to participate in high school distributed leadership and that the exclusion of students from participative decision-making for their high schools may contribute to their further lack of engagement in school and community activities. Forty seven percent of students did not perceive that the voices of minority groups were integrated into community problem

Student Voice Opportunities Quantified by Exploratory Factor … 117 solving for their high schools. The lack of minority outreach and empowerment within school leadership practices may exclude certain groups of students from school, community, and civic participation and this could contribute to gaps in academic achievement and civic opportunities. This has implications for improved school leadership practices leading to improved student learning outcomes.

REFERENCES Apple, Michael W. 2011. “Democratic Education in Neoliberal and Neoconservative times.” International Studies in Sociology of Education 21, no. 1: 21-31. Beaudoin, Nelson. 2005. Elevating Student Voice: How to Enhance Participation, Citizenship, and Leadership. New York: Eye on Education, Inc. Beavers, Amy S., John W. Lounsbury, Jennifer K. Richards, Schuyler W. Huck, Gary J. Skolits, and Shelley L. Esquivel. 2013. “Practical Considerations for Using Exploratory Factor Analysis in Educational Research.” Practical Assessment, Research & Evaluation 18, no. 6: 1-13. http://pareonline.net/pdf/v18n6.pdf. Campaign for the Civic Mission of Schools. 2011. Guardian of Democracy: The Civic Mission of Schools. https://civicmission. s3.amazonaws.com/118/f0/5/171/1/Guardian-of Democracyreport.pdf. Campbell, Tammy L. 2009. “Leadership and Student Voice at One High School: An Action Research Study.” EdD diss. Washington State University. Cheng, Annie Y. N. 2012. “Student Voice in a Chinese Context: Investigating the Key Elements of Leadership That Enhance Student Voice.” International Journal of Leadership in Education: Theory and Practice 15 no. 3: 351-366. doi: 10.1080/13603124.2011.635809. Colton, David and Robert W. Covert. 2007. Designing and Constructing Instruments for Social Research and Evaluation. San Francisco, CA: Jossey-Bass Inc.

118

Jennifer E. Elemen

DeCoster, Jamie. “Overview of Factor Analysis.” 1998. http://www. stathelp.com/notes.html. DiStefano, Christine, Min Zhu, and Diana Mîndrilă. 2009. “Understanding and Using Factor Scores: Considerations for the Applied Researcher.” Practical Assessment, Research & Evaluation 14, no. 20: 1-11. http://pareonline.net/pdf/v14n20.pdf. Elemen, Jennifer E. 2015a. “Opportunities for Student Participation in High School Organizational Leadership Dialogue and DecisionMaking and Their Effects on Academic Achievement and Civic Participation.” EdD diss. University of West Georgia. Proquest(3707421). Elemen, Jennifer E. 2015b. “Students' High School Organizational Leadership Opportunities and Their Influences on Academic Achievement and Civic Participation.” Journal of Research in Education 25, no. 2: 4-18. http://files.eric.ed.gov/fulltext/EJ1098005.pdf Engel-Silva, Michelle R. 2009. “The Role of Distributed Leadership in Quality Educational Organizations.” EdD diss. California State University, Fresno and University of California, Davis. Proquest(3375537). Fielding, Michael. 2001. “Students as Radical Agents of Change.” Journal of Educational Change 2 no. 2: 123–141. Fraenkel, Jack R. and Norman E. Wallen. 2009. How to Design and Evaluate Research in Education (7th ed.). New York: McGraw-Hill. Fusarelli, Lance D., Theodore J. Kowalski, and George J. Petersen. 2011. “Distributive Leadership, Civic Engagement, and Deliberative Democracy as Vehicles for School Improvement.” Leadership and Policy in Schools 10, no. 1: 43-62. doi: 10.1080/15700760903342392 Harris, Alma. 2008. “Distributed Leadership: According to the Evidence.” Journal of Educational Administration 46, no. 2: 172-188. doi: 10.1108/09578230810863253. Hulpia, Hester, Geert Devos, and Hilde Van Keer. 2009. “The Influence of Distributed Leadership on Teachers' Organizational Commitment: A Multilevel Approach.” Journal of Educational Research 103, no. 1: 40-52. doi: 10.1080/00220670903231201.

Student Voice Opportunities Quantified by Exploratory Factor … 119 Jun, Jong S. 1986. Public Administration: Design and Problem Solving. New York: MacMillan. Jun, Jong S. 2006. The Social Construction of Public Administration: Interpretive and critical perspectives. Albany, New York: State University of New York Press. Jun, Jong. S. and William B. Storm 1990. “Social Design in Public Problem Solving.” Public Administration Quarterly Spring: 19-30. doi: 10.2307/40861463. Kahne, Joseph E. and Susan E. Sporte. 2008. “Developing Citizens: The Impact of Civic Learning Opportunities on Students' Commitment to Civic Participation.” American Educational Research Journal 45 no. 3: 738-766. http://www.civicsurvey.org/sites/default/files/publications/ Developing%20Citizens_Web_Version.pdf. Leithwood, Kenneth and Doris Jantzi. 1999. “Transformational School Leadership Effects: A Replication.” School Effectiveness and School Improvement 10 (no. 4): 451-479. doi: 10.1076/1044513495. Leithwood, Kenneth, Karen Seashore Louis, Stephen Anderson, and Kyla Wahlstrom. 2004. “Review of Research: How Leadership Influences Student Learning.” Learning for Leadership Project. Center for Applied Research and Educational Improvement, University of Minnesota, Ontario Institute for Studies in Education and The University of Toronto, and The Wallace Foundation. http://www. wallacefoundation.org/knowledge-center/school-leadership/keyresearch/Documents/How-Leadership-Influences-StudentLearning.pdf. Leithwood, Kenneth and Blair Mascall. 2008. “Collective Leadership Effects on Student Achievement.” Educational Administration Quarterly 44 no. 4: 529561. doi: 10.1177/0013161X08321221. Mathews, David. 1996. “Reviewing and Previewing Civics.” In Educating the Democratic Mind, edited by Walter C. Parker, 265–286. Albany, NY: SUNY Press. Pedersen, Jeff, Stuart Yager, and Robert Yager. 2012. “Student Leadership Distribution: Effects of a Student-Led Leadership Program on School Climate and Community.” International Journal of Educational

120

Jennifer E. Elemen

Leadership Preparation 7 no. 2: 1-9. http://files.eric.ed.gov/ fulltext/EJ973800.pdf. Pohlmann, John T. 2004. “Use and Interpretation of Factor Analysis in the Journal of Educational Research: 1992-2002.” Journal of Educational Research 98 no. 1: 14-22. http://www.tandfonline.com/doi/abs/ 10.3200/JOER.98.1.14-23. Robinson, Carol and Carol Taylor. 2007. “Theorizing Student Voice: Values and Perspectives.” Improving Schools 10 no. 1: 5–17. Robinson, Carol and Carol Taylor. 2012. “Student Voice as a Contested Practice: Power and Participation in Two Student Voice Projects.” Improving Schools 16 no. 1: 32-46. doi: 10.1177/1365480212469713. Robinson, Viviane M. J. 2008. “Forging the Links between Distributed Leadership and Educational Outcomes.” Journal of Educational Administration 46 no. 2: 241-256. doi: 10.1108/09578230810863299. Spillane, James P. 2006. Distributed leadership. San Francisco: JosseyBass. Spillane, James P. and Kaleen Healey. 2010. “Conceptualizing School Leadership and Management from a Distributed Perspective: An Exploration of Some Study Operations and Measures.” The Elementary School Journal 111 no. 2: 253-281. doi: 10.1086/656300. Taft, Jessica K. and Hava R. Gordon. 2013. “Youth Activists, Youth Councils, and Constrained Democracy.” Education, Citizenship, and Social Justice 8 no. 1: 87-100. doi: 10.1177/1746197913475765. Thomson, Pat and Helen Gunter. 2006. “From ‘Consulting Pupils’ to ‘Pupils as Researchers’: A Situated Case Narrative.” British Educational Research Journal 32 no. 6: 839–856. Volwerk, Johannes J. and Gerald Tindal. 2012. “Documenting Student Performance: An Alternative to the Traditional Calculation of Grade Point Averages.” Journal of College Admission Summer: 16-23. http://files.eric.ed.gov/fulltext/EJ992990.pdf. Westheimer, Joel and Joseph Kahne. 2004. “What Kind of Citizen? The Politics of Educating for Democracy.” American Educational Research Journal 41 no. 2: 237-269. http://www.democraticdialogue. com/DDpdfs/WhatKindOfCitizenAERJF.pdf.

In: Exploratory Factor Analysis Editor: Diana Mindrila

ISBN: 978-1-53612-486-6 © 2017 Nova Science Publishers, Inc.

Chapter 5

SPIRITUAL HEALTH IN CHRISTIAN SCHOOLS: IMPLICATIONS FOR ACADEMIC ACHIEVEMENT AND SCHOOL IMPROVEMENT Chad A. McBane*, EdD University of West Georgia, Carrollton, GA, US

ABSTRACT This study assessed the Christian spiritual health of middle school students attending two private, faith-based schools. The goal of the study was to investigate the concept of spiritual health using a data driven approach. A sample consisting of approximately two hundred middle school students attending private Christian schools was surveyed using the Spiritual Health Survey, a thirty question instrument on a six point Likert scale. Evidence of validity and reliability for this instrument was provided through a pilot study using approximately one hundred and fifty *

Corresponding Author Email: [email protected].

122

Chad A. McBane middle school students attending private Christian schools. Three factors of spiritual health emerged through an exploratory factor analysis of the survey data: a) the relationship with God, b) relationship with others, and c) confident testimony. The study also reports differing levels of spiritual health among students based on family dynamics, church attendance, and schools attended. The research discovered that spiritual health does have a significant relationship with academic performance, explaining two percent of variance in student grade point average. Results indicate that faith-based schools should be strategic in their formal improvement planning and target specific aspects of student spiritual growth.

Keywords: spiritual health, faith-based education, Christian education, middle schools, exploratory factor analysis

INTRODUCTION The call for accountability in our school system increases annually. Every year an estimated five million students will attend private schools within the United States, with approximately fifteen percent studying in Christian schools (Council for American Private School Education, 2014). This equates to almost one million students receiving a Christian education each year. Therefore, no longer can Christian schools be content with solely the spiritual growth of their students. Increased rigor throughout our nation’s schools has put pressure on Christian educators to ensure spiritual, academic and social growth within their educational communities. Christians look to the Bible for guidance on this issue; “The fear of the LORD is the beginning of knowledge, but fools despise wisdom and instruct (Proverbs 1:1-7 NIV).” Through a focus on increased academic achievement and spiritual growth of students, Christian schools are now consistently outperforming their counterparts in public schools academically (Coleman, Hoffer, & Kilgore, 1982; Lee & Bryk, 1993; Jeynes, 2010). Research is becoming more widespread on topics of faith-based education due to this change. Interest is growing in the benefits of faith-based education and understanding why these students are consistently high achieving. Achievement gaps between white and minority students are significantly less in faith-based schools, along with the

Spiritual Health in Christian Schools

123

gap between low socioeconomic students and their counterparts (Jeynes, 2005; Jeynes, 2010; Snyder & Dillow, 2015). Researchers in the field of faith based education have published studies demonstrating students at faith based schools academically outperform their public school counterparts. However, researchers have failed to come to a consensus on why this is occurring. Differences in school culture and climate have been concluded to influence achievement in Christian schools (Gaziel, 1997; Morris, 1994). These differences in school culture and climate are evident in the levels of satisfaction that parents have with their child’s school. Parents of students attending faith based schools report an overall 80% satisfaction rate, while parents of public school students report a 56% satisfaction rate (Noel, Stark, & Redford, 2013). Students demonstrated a similar confidence in their faith based school. In a survey of over 1,500 students completed by the Horatio Alger Association of Distinguished Americans (2012) 60% of private school students would give their school an “A.” In contrast only 33% of public school students reported their school as an “A” (Wolniak, Neishi, Rude, & Gebhardt, 2012). This study seeks to further explain the relationship between the academic performance of students in Christian middle schools and their spiritual health. In this study, spiritual health is defined from the perspective of Christian schools as a continuum that begins at the day of physical birth and does not end until the second coming of Jesus (Pazmino, 1997; Habermas, 2001; Wilhoit & Dettoni, 1995). Accepting Jesus Christ as your Savior is the process of justification (2 Corinthians 5:17). The continuum of spiritual health measures where the Christian school student is in the process of sanctification, or becoming more like Christ (2 Timothy 3:16). The current definition of spiritual health is based on theological theory (dogma). The purpose of this research was to identify the perceived factors of spiritual health using a data driven (bottom-up) approach. Further, the study aimed to determine the levels of spiritual health among middle school students attending Christian schools. After quantifying students’ reports regarding their spiritual health, a determination was made as to whether spiritual health is significantly related to academic performance, as measured by student grade point average (GPA). The research questions

124

Chad A. McBane

for this study were designed to investigate the Christian spiritual health and academic achievement of middle school students enrolled in private Christian schools within the state of Florida. To achieve this purpose, a quantitative research design was employed to answer the following research questions: 1) What are the levels of spiritual health reported by middle school students attending Christian schools? 2) What are the factors of spiritual health among middle school students within Christian schools as measured by the Spiritual Health Survey? 3) Are scores on the spiritual health factors statistically significant predictors of student achievement, as measured by GPA? 4) Does the relationship between students’ GPA and spiritual health scores vary across Christian schools?

REVIEW OF LITERATURE In the decades preceding the 1960’s in the United States the Bible was at the center of moral education in public schools. However, with Supreme Court decisions throughout the 1960’s, the Bible was removed from public education. Examined within this review are the differences in school culture, student behavior, parental involvement, and curriculum. The culture within a school can greatly impact the achievement of the students within it. Disparities in the academic achievement between white and minority students are apparent in every area of academic measurement within the public school sector, including standardized tests, GPAs, dropout rates, and retention rates (Jeynes, 2010). Present within the public school system are gaps between students of low and high socioeconomic status and factors shown to decrease these gaps include attendance at religious schools and developing a student’s personal spiritual faith. Faith based schools have distinguished themselves in several areas in regards to school climate. Through a common bond of faith and Christian

Spiritual Health in Christian Schools

125

brotherhood, faith based schools often exhibit a higher level of racial harmony (Irvine & Foster, 1996). This high level of racial harmony has led to research showing the lack of an achievement gap in faith based schools (Jeynes, 2010). Data from the National Educational Longitudinal Study (NELS) of 1988 examined the achievement of students across over one thousand schools. Approximately eight hundred public schools and two hundred Catholic schools were studied by Sebring and Camburn (1992) and Reading scores from NELS tests demonstrated that achievement was more equitably distributed across the entire student body in Catholic schools. The same results of a more equitable distribution of achievement were also seen for students from differing socioeconomic status and minority students when attending Catholic schools. (Sebring & Camburn, 1992). Student spiritual health and attendance at religious schools has also shown to have an impact on student behavior. Students attending faith based institutions are less likely to incur threats of school violence (Irvine & Foster, 1996). The National Center for Education Statistics published a report that detailed the reported crimes at both public and private schools (Snyder & Dillow, 2013). Statistics from this report find that students and teachers at private schools were less likely to feel victimized than their public school counterparts. Specifically, nineteen percent of public school students ages 12-18 reported gangs were present at their school compared with two percent for private school students. In public schools forty-one percent of teachers during the 2011-2012 school year reported that student behavior interferes with their teaching. This number is almost double that of private school teachers. Twenty-two percent of private school teachers reported student misbehavior interferes with their teaching. Tardiness and cutting class was also polled by the teachers. Nineteen percent of private school teachers, versus thirty-eight percent of public school teachers, reported this as a significant problem. Research would indicate that major distractions due to inappropriate student behavior are reduced at religious schools, possibly due to religious schools teaching moral education with an emphasis on love and discipline (Irvine & Foster, 1996). Biblical teaching on love and holiness requires an

126

Chad A. McBane

increased level of self-discipline among students. Researchers have noted this as one reason that students at faith based schools outperform their counterparts in the public school academically (Jeynes, 2005; Jeynes, 2010; Gaziel, 1997). Baumrind (1971) conducted research pertinent to the field of parenting and indicated that a combination of high levels of love and discipline are of great importance when raising children. These same truths hold true in education. Due to the known impact that behavior can have on a child’s education, parents will often send their children to private, faith based schools for the perceived behavioral benefit to their child (Jeynes, 2005). Once families are a part of these communities, parents experience a greater level of parental involvement at faith based schools. Scripture confirms the importance of parental involvement, “Children, obey your parents in the Lord, for this is right” (Ephesians 6:1, ESV). Colemen (1988) developed a theory of social capital that stated the families’ role in the education of children can be a statistically significant predictor of student achievement. Parents can support the school system through volunteer opportunities, but also helping students develop the social capital to function and learn in school. Comer (2015) has been developing social capital reform in schools since the 1960s. His research centers on the concept that students must develop the capacity to self-regulate their bodies and minds in order to take responsibility for their own personal development. Student’s ability to develop is partially determined by the environment. This includes parent and school relationships that can prevent trust and norm difference problems when the two environments are increasingly different. Comer (2015) found that when supporting environments are provided in both the home and school meaningful social development and learning increases. In an atmosphere with increased love, discipline, and parental involvement it would make sense that more work can be completed. Rigorous curriculum within faith based schools includes a greater level of homework (Mentzer, 1988). Within faith based schools, students of low socioeconomic status and minority students have an increased participation in AP and Honors classes, compared to their public school counterparts

Spiritual Health in Christian Schools

127

(Jeynes & Beuttler, 2012). A study published in 2009 by Jeynes noted the relationship between Biblical literacy and student achievement, as defined by GPA. In this study it was noted that the Bible provides students with a challenging, sophisticated piece of literature to study and students that demonstrated a greater knowledge of the Bible, as presented on a Bible quiz, also had higher GPAs (Jeynes, 2009). In his review of the literature, he noted that the study of the Bible encourages disciplined behavior and enhances a student’s ability to understand complex literature. It is important to note that there are recent studies that attempt to account for the difference in test scores between public and private school students. The National Assessment of Educational Progress (NAEP) reports achievement data on public and private school students, along with certain individual, family, and school characteristics. Some research suggests that the difference in achievement between public and private school students can be explained through these different characteristics. A report published in 2007 noted that the academic advantage for private school students greatly diminished when adjustments were made for family background (Wenglinsky, 2007). These characteristics include the family’s socioeconomic status and various indicators of parental involvement. Conclusions were drawn that private schools have an academic advantage due to families that contribute more to the learning process than their public school counterparts (Wenglinsky, 2007; Peterson and Llaudet, 2006). Using data from the National Longitudinal Survey of Youth (NLSY), Duncan and Sandy (2007) also made statistical adjustments to reported test scores in an attempt to explain the difference in achievement between private and public school students. This study found that family background played a larger role in student achievement than whether the school was public or private. Adjustments to statistics demonstrated only a slight advantage for private school students, concluding that private school students have more of the attributes associated with increased achievement (Duncan & Sandy, 2007). Even with conclusions drawn through numerous research studies, educators have examined the impact of religious schools more than the

128

Chad A. McBane

level of individual religious commitment (Jeynes, 2009). This reinforces the need for this study to answer the question of whether an individual’s spiritual health is a significant predictor of academic performance. A recent study on the high achievement levels of students in Adventist schools asked, “Is it the spiritual center that brings heightened meaning to all of curriculum elements and, in turn, creates a more effective learning “whole?” Asking the question is easy; researching it will be difficult, but worth the effort” (Thayer & Kido, 2012, p.114).

METHOD Participants and Instrumentation The participants in both the pilot study and the research study consisted of middle school students enrolled in two private Christian schools within the state of Florida. Participating schools were located in different counties within the state and all received accreditation through Christian Schools of Florida, an organization that serves private Christian schools throughout the state. No research survey instrument was found that measured the spiritual health of students attending Christian schools. Due to the lack of an existing survey instrument the Spiritual Health Survey (Appendix A) used in this research was developed through a combination of two separate measurement tools. Parts of the survey instrument were developed by a team of administrators and Bible teachers and provided thirteen of the survey items for the Spiritual Health Survey. The seventeen remaining survey items were drawn from the LifeWay Spiritual Growth Assessment (LifeWay, 2006). Responses for the 30 items were collected on a 6-point Likert scale (strongly disagree = 1, disagree = 2, somewhat disagree = 3, somewhat agree = 4, agree = 5, strongly agree = 6).

Spiritual Health in Christian Schools

129

Data Analysis Data were collected using Google Forms and exported to an IBM SPSS file. Three individuals had responses removed due to missing values. The process of removing these cases followed the screening checklist and criteria as outlined by Tabachnick and Fidell (2001). After removing these three cases, the sample consisted of 196 respondents. The survey items were used as input for Exploratory Factor Analysis (EFA) with the Maximum Likelihood (ML) estimation method. This method allowed for the estimation of factors by forming groups of items with similar content. Due to potential correlations among factors, the Promax oblique rotation method was used to obtain a final factor solution. Several criteria were used to examine and determine the optimal number of factors, such as the number of eigenvalues larger than one and the examination of the scree plot. Each item was assigned to the factor with the highest loading or correlation (Worthington & Whittaker, 2006). After pairing each item with the factor it most correlates, the items grouped under the same factors were examined for common themes (Worthington & Whittaker, 2006). All factors needed a minimum of three items to be included. A factor correlation matrix was estimated to determine correlation between the identified factors. To investigate the third research question, students’ factor scores on each spiritual health component were used as independent variables in stepwise multiple regression analysis to predict academic achievement. CSF middle school students’ academic achievement was measured through a self-reported GPA of each student taking the survey. School average GPA’s and average scores on the spiritual health factors were calculated in an effort to investigate variations in student achievement and student spiritual health across groups of schools, in addition to helping explain the relationship between spiritual health and academic achievement. Specifically, mean factor scores and average survey scores were aggregated and graphically illustrated by school. This descriptive analysis between schools addressed the fourth research question.

130

Chad A. McBane

RESULTS Research Question 1 Descriptive statistics were calculated and examined to determine reported levels of Christian spiritual health among private Christian middle school students. A sample of middle school students (N=199) were surveyed from two different schools. School A provided 170 respondents, while School B provided 29 students to respond to the survey. Descriptive statistics were computed to address the first research question. Responses to the 30 Likert-style questions were coded for statistical analysis with six representing Strongly Agree, five representing Agree, four representing Somewhat Agree, three representing Somewhat Disagree, two representing Disagree, and one representing Strongly Disagree. Mean responses for each of the 30 variables were examined (see Appendix Table B1). Mean responses ranged from a minimum of 2.83, (SD=1.42) for the variable “I regularly read and study my Bible outside of school” to a maximum of 5.56 (SD=.83) for “I believe the Bible is God’s Word and provides his instructions for life.” Descriptive statistics were then calculated and examined by student reported gender. Responses by all students on the thirty question survey instrument included a range of 3.94. The range was calculated from the lowest mean of 1.93 and the highest mean score of 5.87. These scores included all 30 variables. Male students had an aggregated mean score of 4.32 (SD=.74), whereas female students had an aggregated mean score of 4.48 (SD=.73) on the survey. Additional descriptive statistics were calculated and examined by marital status of parents. Students with divorced parents reported a mean score of 4.30 (SD=.73). Students who have parents that are married reported a mean score of 4.43 (SD=.74). Mean scores of students who reported their parents as being church members was 4.48 (SD=.72). Students who reported their parents were not members of a church reported a mean of 4.08 (SD=.72). Students reporting a GPA of 3.50 or higher had a mean score of 4.47 on the survey instrument. Students reporting a GPA between 3.00 and 3.49 had a mean score of 4.37 on the survey. Students reporting a GPA less than 3.00 had

Spiritual Health in Christian Schools

131

a mean score of 4.18 on the survey. Students with a mean score in the top fifty percent on the Spiritual Health Survey had a mean GPA of 4.49. The lower fifty percent of students on the survey had a mean GPA of 4.34.

Research Question 2 To answer the second research question, exploratory factor analysis (EFA) was conducted to identify the factors of spiritual health among middle school students as measured by the spiritual health survey. The skewness and kurtosis coefficients for the items included in the EFA are reported in Appendix Table B2. These values show that the data met the assumption of normal distribution after removal of the variable, “I believe the Bible is God’s Word and provides His instructions for life” for high Kurtosis. On the first run of EFA, six factors had an eigenvalue greater than one. Examination of the scree plot revealed an elbow at the third or fourth factor. A strong first factor emerged explaining 35.81% of the variance in the data. Subsequent factors explained less of the variance, ranging from 6.98% to 3.51% on the sixth factor. In total, the six factors with an eigenvalue greater than one explained 60.98% of the variance. Next, the pattern matrix for the six factors with an eigenvalue greater than one was explored. All items with loadings greater than 0.30 were examined. On the first extraction, one variable had a negative loading to one of the factors. This variable, “I believe the Bible is God’s Word and provides His instructions for life,” was removed and EFA was run again. Once the data were reduced to a three factor model, variables that did not load or that crossloaded continued to be removed one at a time. The thirteenth extraction yielded a pattern matrix of variables that did not cross-load or fail to load to the identified three factors. This solution was accepted as the final solution and included three latent factors containing 18 of the initial 30 variables. This final solution explained 54.75% of the variance in the data (Table 1).

132

Chad A. McBane Table 1. Factor analysis final solution

I remain confident of God's love and provision during difficult times. I replace impure or inappropriate thoughts with God's truth. I trust God to answer when I pray and wait patiently on His timing. I use the Bible to guide the way I think and act. My relationship with Christ is motivated more by love than duty or fear. My prayers include thanksgiving, praise, confession, and requests. I pray because I am aware of my complete dependence on God for everything in my life. When the Bible exposes an area of my life needing change, I respond to make things right. Peace, contentment, and joy characterize my life rather than worry and anxiety. I engage in daily prayer time outside of school. I go out of my way to show love to people I meet. I forgive others when their actions harm me. I act as if other's needs are as important as my own.

Loadings Relationship with God .73

Relationship with others

.69 .67 .65 .63

.60 .56

.52

.51

.41 .75 .72 .61

Confident testimony

Spiritual Health in Christian Schools Loadings Relationship with God

Relationship with others .61

133

Confident testimony

I show patience in my relationships with family and friends. I serve others and expect nothing .55 in return. I am prepared to share my .94 testimony at any time. When asked about my faith, I .67 remain firm in my testimony. I have relationships with non.62 believers and share my testimony with them. Percentage variance explained 38.01 9.25 7.49 Note. Extraction method: Maximum likelihood. Rotation method: Promax with Kaiser normalization. Rotation converged in 6 iterationsHaving a Confident Testimony emerged as a central theme of the third factor. Variables in this factor highlight the student’s ability to convey their testimony with others they meet in their lives. This factor explained 7.49% of the variance in the final solution of the data. As with the first two factors, Alpha was 0.76, a value considered within the acceptable range.

Factor one included 10 variables with loadings of 0.30 or higher. The marker item was “I remain confident of God’s love and provision during difficult times.” The factor loading for this item was 0.73. The marker item of the second factor was “I go out of my way to show love to people I meet” with a factor loading of 0.75. Five variables loaded under factor two. Factor three included three variables. The marker item for factor three was “I am prepared to share my testimony at any time.” The loading for this item was 0.94. As marker items were identified, the variables that loaded under each factor were examined to determine a common theme. In examining those variables, Relationship with God emerged as the central theme of the first factor. Some variables that loaded under factor one include “My relationship with Christ is motivated more by love than duty or fear” and “I trust God to answer when I pray and wait patiently on His timing.” Variables that loaded under this factor focused on prayer, God’s truth, and God’s word. This factor

134

Chad A. McBane

explained 38.01% of the variance in the data for the final solution. The Cronbach’s Alpha coefficient of internal consistency was 0.87 for this factor, well within the acceptable range. The theme of Relationship with others emerged for the second factor. Each variable loading to this factor focused on a specific action towards others. Variables were focused on showing love, forgiveness, patience, and serving others. This factor explained 9.25% of the variance in the final solution of the data. As with factor one, Cronbach’s Alpha was calculated in SPSS and returned a value of 0.79, which is within the acceptable range. The factor correlation matrix was examined and strong positive correlations were noted between the factors. Factors one and three had the strongest correlation of 0.68. This was followed by factors one and two, with a correlation of 0.59. Factors two and three had a positive correlation of 0.53. These strong correlations among factors validate the use of the Promax oblique rotational method that assumes correlation among factors (Table 2).

Table 2. Correlations between factors and academic achievement

Relationship with God

Relationship with others

Confident testimony

R Sig. (2-tailed) N R Sig. (2-tailed) N R

Relationship with God —

Relationship with others .49** .00

Confident testimony .58** .00

GPA .14 .07

188 —

188 .43** .00

185 -.01 .88

188 —

185 .03

Sig. (2-tailed) N **Correlation is significant at the .01 level (2-tailed).

.72 185

Spiritual Health in Christian Schools

135

Bartlett factor scores were calculated to determine the individuals’ location on the identified factors. In this calculation, students were omitted if they failed to answer greater than three of the survey questions. Factor scores for Relationship with God ranged from a minimum of -4.24 to a maximum of 1.84. Relationship with others had factor scores with a minimum of -3.15 to a maximum of 2.09. The third factor of Confident testimony had a minimum factor score of -3.77 and a maximum factor score of 1.81. Similarities and differences between the factor solution obtained from the pilot study and the results of the current study were examined to provide evidence of reliability and validity for the instrument and strengthen the EFA findings. Two factors, Relationship with God and Relationship with Others, were replicated from the pilot study (Appendix B3).

Research Question 3 Several statistical methods were used to examine the relationship between spiritual health survey scores and GPA. The Pearson correlations between factor scores and GPA were not statistically significant (Table 2). Factor scores were also examined by gender. Results showed higher scores for female students on all three factors. Factor scores were also higher on all three factors for students who reported their parents were members of a church. In an effort to determine the distribution of data between factor scores and GPA, a category was created that grouped students by GPA scores. Category one (n=22) was defined as a GPA less than 2.50. Category two (n=41) was defined as a GPA equal to or greater than 2.50 but less than 3.50. Category three (n=133) was defined as a GPA equal to or greater than 3.50. These groupings were chosen to due to the lack of variance in the distribution of GPA scores. Boxplots (Figures 1 & 2) show the distribution of mean scores on each factor within the GPA categories. Students in category one for GPA had a mean score on Relationship with God factor of -0.42, while students in category three for GPA had a mean score of 0.09. Students in category one for

136

Chad A. McBane

GPA had a mean score of -0.09 on Confident testimony, while students in category three had a mean score of 0.02.

Figure 1. Distribution of “relationship with God” factor scores by GPA category.

Figure 2. Distribution of “relationship with others” factor scores by GPA category.

Spiritual Health in Christian Schools

137

Based on the differences in mean scores on the two factors of Relationship with God and Confident testimony an independent samples ttest was run to examine if the difference in scores between groups one and three are statistically significant. This procedure was employed to determine whether factor scores differ significantly among groups of students with larger differences in GPA. Differences between groups were considered statistically significant if the p value was less than or equal to 0.05. Probability values for Relationship with God showed a statistical significance at the p=0.05 level. Students reporting a GPA of greater than 3.50 had a significantly higher mean score on Relationship with God than students reporting a GPA of less than 2.50 (t=-2.02, p=.05). Descriptive statistics were also used to examine the relationship between student GPAs and student mean scores on the thirty question survey to help draw conclusions on research question three. Students (N=196) reported a mean GPA of 3.59. The minimum GPA reported was 2.17, while the maximum reported was 4.05. Mean scores on the initial set of thirty questions included in the spiritual health survey had a range from 1.93 to 5.87. The survey mean was 4.41. The bar graph in Figure 3 shows the mean survey scores for the three categories of reported student GPA. In order to examine the relationship between survey mean and GPA a scatterplot was generated (Figure 4). Pearson correlation was computed using the survey mean variable and the reported GPA variable. The correlation was significant (r= 0.14, p =0.05). The data from the Pearson correlation led to the completion of a simple regression. After running the linear regression the multiple correlation coefficient was equal to the Pearson correlation at R=0.14. The R squared value that explains the amount of variance in the dependent variable (GPA) that is accounted for by the independent variable (survey mean) was R2= 0.02. Therefore, mean score on the spiritual survey explains 2% of the variance in GPA. Results showed that the survey mean significantly predicted GPA, F(1,194) = 3.91, p =0.05, R2=0.02, η2 =.019.

138

Chad A. McBane

Figure 3. Survey means by GPA category.

Figure 4. Scatterplot for survey mean and GPA.

Spiritual Health in Christian Schools

139

Research Question 4 Descriptive analysis of the mean scores between School A and School B were used to answer research question number four. It is important to note that number of reporting students from each school was disproportionate. School A had 170 students reporting on the survey instrument, while School B had 29. Mean GPA scores were similar with School A at 3.61, while School B reported 3.52. Mean scores on the thirty question survey instrument were also higher at School A with a mean score of 4.45 and a mean score of 4.19 for School B. Mean factor scores after completing EFA for both schools, resulted in lower scores from School B than from School A. Factor three, Confident Testimony, was the area of the survey in which the greatest differences in factor mean score were reported. Three questions from the survey loaded to factor three. Two of these questions demonstrated a difference in score between School A and School B. These questions were, “I am prepared to share my testimony at any time,” and “I have relationships with non-believers and share my testimony with them” (Table 3).

Table 3. Factor three items with difference mean scores between school A and B

I am prepared to share my testimony at any time. I have relationships with non-believers and share my testimony with them.

School B A B A

M 3.69 4.57 3.41 3.79

SE .24 .11 .24 .12

Consistencies between the two schools were evident. Negative factor scores were associated with students whose parents were not church members at both schools, while positive factor scores were associated with students that reported parents as church members. Of particular interest was the degree of negativity associated with Confident Testimony by students that reported no church membership. This suggests an extremely low confidence in students’ ability to share their testimony when church

140

Chad A. McBane

membership is not present. Further analysis of the variable “I attend church services weekly” indicates that students reporting weekly church membership have an increased confidence in sharing their testimony. Both schools reported the same variables holding both the highest and lowest mean scores. The highest mean scores for both schools were the variables of “I believe the Bible is God’s Word and provides His instructions for life” and “I believe that Christ provides the only way for a relationship with God.” High scores on the Spiritual Health Survey variables indicate students have an understanding of the basic tenants of the Christian faith (Pazmino, 1997; Habermas, 2001; Wilhoit & Dettoni, 1995). Students reported low mean scores on the same two variables. Variables with the lowest mean scores were “I regularly read and study my Bible outside of school” and “I regularly contribute time to a ministry at my church.” Habermas (2001) noted that adolescence is a time of concern for self because a lack of maturity exists. Low scores on these particular variables would support Habermas (2001) in that these variables would indicate a lack of using time for Bible study or church service and a focus of time for self.

DISCUSSION One of the objectives of the current study was to provide a data driven definition of spiritual health by identifying the dimensions of this construct. An important contribution of the current research is, therefore, the identification of the three perceived dimensions of spiritual health: a) the relationship with God, b) the relationship with others, and c) confident testimony. The first two factors replicated findings from a pilot study. Items included in the first factor, Relationship with God refer to being in God’s word through knowledge and guidance from the Bible. The Bible can expose areas of life that need change and using it to guide the way one thinks and acts can grow Christians in their process of sanctification. The first factor also revealed that prayer plays an important role in a

Spiritual Health in Christian Schools

141

relationship with Christ. Variables that were related to prayer loaded to factor one. The second identified factor of Relationship with Others demonstrates the importance of outwardly displaying Christian character to others. Forgiveness, patience, service, and love for others were highlighted in the survey questions that loaded for factor two. Learning to show love in everyday life and taking on appropriate Christian service were noted by Pazmino (1997) and supported in this research as a component of adolescence spiritual health. Confident Testimony emerged as the third factor. An ability to share and remain firm in testimony was identified as a component of spiritual health. Accepting Christ as one’s savior has been termed justification (Pazmino, 1997). Once this rebirth occurs there is an empowerment to be more Christ-like and make the transition from being discipled to becoming a disciple (Habermas, 2001). Emerging from this analysis is a three pronged approach to spiritual health with the family, church, and school at the center of a student’s growth. Many factors such as climate, culture, homework, and high expectations have been shown to increase academic achievement (Jeynes, 2010). However, these factors are generally school based. It is also important to look at the personal spiritual health of the student and to examine its impact on academics. Lacking in current research is the impact of the level of Christian spiritual health of the individual on student achievement (Jeynes, 2009). Using GPA groupings, the current study revealed patterns in the data that would indicate that as a student’s Christian Spiritual Health increases, academic performance increases. For example, mean scores on the Relationship with God factor increased as GPA increased. This was also the case for factor three, Confident Testimony. Due to the evidence of a possible relationship between spiritual health and student achievement, further data analysis was completed using a mean score on the survey instrument and student reported GPA. Charting this data indicated that as student GPA increased as students’ mean scores on the survey instrument increased. Regression analysis showed that only 2 percent of the variance on GPA could be explained by the survey mean.

142

Chad A. McBane

However, when discussing GPA, two percent can be the difference between a 4.00 and a 3.92. An examination of scores on the identified factors of spiritual health showed differences in scores. School A had positive scores on all three factors, while School B students reported negative scores on each component of spiritual health. Although the sample sizes are disproportionate, some general hypotheses and further research can be discussed. Looking at the statistical differences in mean scores across all three factors would infer differing degrees of spiritual health. This could be attributed to teachings in Bible and other classes, frequency of Chapel services, and an examination of school improvement plans in the area of student spiritual health could be discussed. Areas of need that are identified by the survey could become pieces of the school improvement planning process. Just as schools use academic assessment data to drive instructional practices, spiritual survey data can be used to address school needs in spiritual education.

Limitations The use of a self-reporting survey comes with some limitations. Within the Christian school a set of values and norms are expected. Students may act one way at school, while behavior changes outside of the school walls. Honesty on a survey associated with deep issues surrounding faith is expected, but the maturity level of students must be understood. Respondents may make inferences about what is being assessed based on the questions and formulate answers that would appear pleasing to the researcher or even the school. Schools administering the survey may have students that are absent on the day of administration, therefore limiting the number of participants. Average attendance rates at the schools surveyed is approximately 98%; therefore, this was not an issue. Students self-reported their GPAs for the survey. Having students remember their cumulative GPA is an unreasonable expectation. Therefore, teachers assisted students by giving them a copy of their cumulative GPA

Spiritual Health in Christian Schools

143

before administering the survey. This limited unintentional error or student misrepresentation of their academic ability and performance.

Recommendations for Future Research The spiritual health survey used in this research to quantify the spiritual health of middle school students in private Christian schools is not specific to only these students. Further research could be done using this instrument and replicating this study in a variety of academic settings. A more comprehensive study replicating the methods and procedures followed in this study would be possible, using students with a greater degree of variance in GPA. This research can be applied to both public and private school students based on their examination of faith. Religion is not something that is only found in students within the private school sector. Students within public schools could also take the Spiritual Health Survey and determine if there is a significant relationship between spiritual health and achievement. Researchers could develop a more comprehensive study by including different types of schools, including public, private, and charter schools. The use of a variety of student populations would help minimize bias towards the survey instrument and provide a more uniform distribution of responses. Of perhaps greatest interest to the researcher is the response that could be garnered by surveying the spiritual health of teachers. Small modifications within the survey would allow for a quantifiable measurement of teacher spiritual health. Researchers could analyze data from both students and teachers to understand the impact of teacher spiritual health on both student academic and spiritual outcomes. In other words, does having a greater spiritual health amongst staff equate to a greater degree of spiritual health among students? Implications for school improvement could also prove beneficial to study. Using this instrument to analyze differences amongst schools, spiritual components of school improvement plans can be studied to understand best practices at Christian schools. How do shepherds best tend to their flock? Using the spiritual health survey can give schools

144

Chad A. McBane

quantifiable data to help better understand the needs of students, addressing specific deficiencies in the spiritual health of students and schools.

CONCLUSION The spiritual health of a student can be compared to the academic health of students in that it is a continuous growth model. Even as one becomes mature in their academic career there is constant learning, whether as a professional student or in the workforce. Spiritually, students are also in a state of continuous growth, from a place of justification to glorification on the day of the second coming of Christ. This research allows educators a multi-dimensional approach to defining the spiritual health of students. Areas of need and growth can be identified with the spiritual health survey and become part of a school’s specific improvement planning. The identification of Relationship with God, Relationship with Others, and Confident Testimony as the components of spiritual health contributes to the definition of spiritual health and the field of Christian education with a data driven approach to spiritual growth. As a tool that can be administered to students at several schools, the spiritual health survey can measure not only the spiritual health of students in a particular school, but within many schools. Accrediting organizations can gain a pulse of how schools are fulfilling the Great Commission through an analysis of data collected at school sites for organizational leaders. As noted in the research, differences in schools can be analyzed. Accrediting organizations and schools can use a data driven approach to analyze the spiritual components of SIPs. Differences in SIPs between schools can be analyzed to formulate best practices when it comes to spiritual growth. These can be shared between schools. When this research is discussed in combination with other research on faith based education, conclusions can be drawn that faith based education can be beneficial for the student. Increased achievement, safer school climate, and parental involvement were noted as characteristics of faith

Spiritual Health in Christian Schools

145

based education. School choice programs that seek to improve academic outcomes for students from disadvantaged backgrounds should consider this research, along with referenced studies when considering the use of faith based schools.

APPENDIX A: SPIRITUAL HEALTH IN CHRISTIAN SCHOOLS: IMPLICATIONS FOR ACADEMIC ACHIEVEMENT AND SCHOOL IMPROVEMENT

146

Chad A. McBane

Spiritual Health in Christian Schools

147

148

Chad A. McBane

Spiritual Health in Christian Schools

149

150

Chad A. McBane

Spiritual Health in Christian Schools

151

APPENDIX B: SUPPLEMENTAL TABLES FOR CHRISTIAN SPIRITUAL SURVEY RESULTS Table B1. Mean ratings by survey question

Survey item I believe the Bible is God's Word and provides His instructions for life. I believe Christ provides the only way for a relationship with God. My prayers include thanksgiving, praise, confession, and requests. I pray because I am aware of my complete dependence on God for everything in my life. When asked about my faith, I remain firm in my testimony. I trust God to answer when I pray and wait patiently on His timing. I understand my spiritual gifts and use those gifts to serve others. My relationship with Christ is motivated more by love than duty or fear. When God makes me aware of His specific will in an area of my life, I follow His lead. I remain confident of God's love and provision during difficult times. I can answer questions about life and faith from a biblical perspective. I go out of my way to show love to people I meet. I engage in daily prayer time outside of school. I show patience in my relationships with family and friends. I am prepared to share my testimony at any time. When the Bible exposes an area of my life needing change, I respond to make things right.

Valid N 199

M 5.57

SD .83

199

5.40

1.04

197

5.31

1.03

198

4.94

1.22

199

4.94

1.11

199

4.86

1.17

199

4.75

1.12

199

4.72

1.14

199

4.70

1.10

199

4.61

1.20

198

4.59

1.21

197 199 199

4.57 4.49 4.47

1.13 1.47 1.04

198 198

4.44 4.43

1.39 1.13

152

Chad A. McBane

Survey item My actions demonstrate a belief in the Great Commission, "Therefore go and make disciples of all nations..." (Matthew 28:19-20) I act as if other's needs are as important as my own. I serve others and expect nothing in return. I forgive others when their actions harm me. Generally, I act the same way in school as I do out of school. I replace impure or inappropriate thoughts with God's truth. I evaluate cultural ideas and lifestyles by biblical standards. Peace, contentment, and joy characterize my life rather than worry and anxiety. I use the Bible to guide the way I think and act. I attend church services weekly. I practice a regular quiet time and look forward to that time with Christ. I have relationships with non-believers and share my testimony with them. I regularly contribute time to a ministry at my church. I regularly read and study my Bible outside of school.

Valid N 199

M 4.36

SD 1.20

197 198 198 199

4.36 4.35 4.29 4.24

1.23 1.25 1.24 1.42

199

4.13

1.40

197

4.05

1.30

199

4.02

1.36

198 199 199

4.01 3.94 3.84

1.28 1.97 1.31

199

3.74

1.53

197 199

3.54 2.84

1.70 1.42

Table B2. Skewness and kurtosis coefficients for survey items

Survey item Valid N (listwise) I regularly read and study my Bible outside of school. I regularly contribute time to a ministry at my church. I have relationships with nonbelievers and share my testimony with them.

Skewness Statistic SE

Kurtosis Statistic

SE

.46

.17

-.46

.34

197

-.02

.17

-1.22

.35

199

-.21

.17

-.90

.34

N 185 199

Spiritual Health in Christian Schools

153

Table B2. (Continued)

Survey item I practice a regular quiet time and look forward to that time with Christ. I attend church services weekly. I use the Bible to guide the way I think and act. Peace, contentment, and joy characterize my life rather than worry and anxiety. I evaluate cultural ideas and lifestyles by biblical standards. I replace impure or inappropriate thoughts with God's truth. Generally, I act the same way in school as I do out of school. I forgive others when their actions harm me. I serve others and expect nothing in return. I act as if other's needs are as important as my own. My actions demonstrate a belief in the Great Commission, "Therefore go and make disciples of all nations..." (Matthew 28:19-20) When the Bible exposes an area of my life needing change, I respond to make things right. I am prepared to share my testimony at any time. I show patience in my relationships with family and friends.

N 199

Skewness Statistic SE -.24 .17

Kurtosis Statistic -.46

SE .34

199 198

-.29 -.34

.17 .17

-1.55 -.37

.34 .34

199

-.38

.17

-.56

.34

197

-.39

.17

-.21

.35

199

-.65

.17

-.21

.34

199

-.57

.17

-.53

.34

198

-.56

.17

-.13

.34

198

-.68

.17

.41

.34

197

-.69

.17

.02

.35

199

-0.49

.12

-0.04

.34

198

-0.57

.17

0.06

.34

198

-0.63

.17

-0.40

.34

199

-0.50

.17

-0.03

.34

154

Chad A. McBane

Survey item I engage in daily prayer time outside of school. I go out of my way to show love to people I meet. I can answer questions about life and faith from a biblical perspective. I remain confident of God's love and provision during difficult times. When God makes me aware of His specific will in an area of my life, I follow His lead. My relationship with Christ is motivated more by love than duty or fear. I understand my spiritual gifts and use those gifts to serve others. I trust God to answer when I pray and wait patiently on His timing. I pray because I am aware of my complete dependence on God for everything in my life. When asked about my faith, I remain firm in my testimony. My prayers include thanksgiving, praise, confession, and requests. I believe Christ provides the only way for a relationship with God. I believe the Bible is God's Word and provides His instructions for life.

N 199

Skewness Statistic SE -0.81 .17

Kurtosis Statistic -0.15

SE .34

197

-0.53

.17

-0.14

.35

198

-0.72

.17

0.03

.34

199

-0.79

.17

0.26

.34

199

-0.73

.17

0.54

.34

199

-0.87

.17

0.43

.34

199

-1.04

.17

1.04

.34

199

-0.89

.17

0.25

.34

198

-1.30

.17

1.42

.34

199

-1.18

.17

1.49

.34

197

-1.63

.17

2.31

.35

199

-2.23

.17

5.55

.34

199

-2.43

.17

7.02

.34

Spiritual Health in Christian Schools

155

Table B3. Pilot study final solution

I believe the Bible is God's Word and provides His instructions for life. My actions demonstrate a belief in the Great Commission, "Therefore go and make disciples of all nations..." (Matthew 28:19-20) When asked about my faith, I remain firm in my testimony. I am prepared to share my testimony at any time. I believe Christ provides the only way for a relationship with God. I understand my spiritual gifts and use those gifts to serve others. I pray because I am aware of my complete dependence on God for everything in my life. I can answer questions about life and faith from a biblical perspective. I evaluate cultural ideas and lifestyles by biblical standards. I practice a regular quiet time and look forward to that time with Christ. I use the Bible to guide the way I think and act. When the Bible exposes an area of my life needing change, I respond to make things right. I remain confident of God's love and provision during difficult times. When God makes me aware of His specific will in an area of my life, I follow His lead. My prayers include thanksgiving, praise, confession, and requests.

Factor Relationship with God .87 .75

.71 .67 .66 .59 .59 .58 .56 .48 .47 .47 .44 .40 .36

Relationship with others

156

Chad A. McBane Factor Relationship with God

Relationship with others .82

I go out of my way to show love to people I meet. I show patience in my relationships with family .71 and friends. I act as if other's needs are as important as my .70 own. I serve others and expect nothing in return. .68 I forgive others when their actions harm me. .54 I engage in daily prayer time outside of school. .48 I have relationships with non-believers and .44 share my testimony with them. Generally, I act the same way in school as I do .43 out of school. Note. Extraction Method: Maximum Likelihood. Rotation Method: Promax with Kaiser Normalization. Rotation converged in 3 iterations.

REFERENCES Act, N. C. L. B. (2002). United States Department of Education. Section, 625, 104-208. AdvancED. (2014). We are Advanc Ed. Retrieved from http://www.advanc-ed.org/about-us. Armstrong, J. S. (1982). The value of formal planning for strategic decisions. Strategic Management Journal, 3(3), 197–211. http://dx.doi.org/10.1002/smj.4250030303. Beechick, R. (1980). Teaching primaries. David C Cook. Boerema, A. (2011). A research agenda for Christian schools. Journal of Research on Christian Education, 20, 28–45. Chicquette, L. J. (2010). The design and implementation of an effective continuous school improvement process. Cardinal Stritch University.

Spiritual Health in Christian Schools

157

Christian Schools of Florida. (2014). School effectiveness and school improvement program accreditation standards. Retrieved from http://csfla.org/accreditation/strands. Coleman, J., Hoffer, T., & Kilgore, S. (1982). High school achievement: Public, Catholic, and Private schools compared. New York, NY: Basic Books. Coleman, J. S. (1988). Social capital in the creation of human capital. American journal of sociology, S95-S120. Comer, J. P. (2015). Developing social capital in schools. Society, 52(3), 225-231. Cram, H. (2013). This is not your father’s accreditation! The Standard, Middle States Association Commissions on Elementary and Secondary Schools. Spring Edition. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. Darling-Hammond, L. (2001). The challenge of staffing our schools. Educational Leadership, 58(8), 12-17. Darling-Hammond, L., LaPointe, M., Meyerson, D., & Orr, M. T. (2007). Preparing school leaders for a changing world: Lessons from exemplary leadership development programs. School Leadership Study (Executive Summary). Retrieved from Wallace Foundation website: http://www.wallacefoundation.org/knowledge-center/school -leadership/key-research/Documents/Preparing-School-LeadersExecutive-Summary.pdf. Datnow, A., & Castellano, M. E. (2001). Managing and guiding school reform: Leadership in success for all schools. Educational Administration Quarterly, 37(2), 219–247. DiMaggio, P. J., & Powell, W. W. (2012). The Iron Cage Revisited: Institutional Isomorphism and Collective Rationality in Organizational Fields [1983]. Contemporary sociological theory, 175. DiStefano, C., Zhu, M., & Mindrila, D. (2009). Understanding and using factor scores: Considerations for the applied researcher. Practical Assessment, Research & Evaluation, 14(20), 1–11.

158

Chad A. McBane

Duncan, K. C., & Sandy, J. (2007). Explaining the performance gap between public and private school students. Eastern Economic Journal, 33(2), 177-191. Eaton, J. S. (2009). Accreditation in the United States. New Directions for Higher Education, 2009(145), 79-86. Egeler, D. (2013). Effective Christian Schooling. Christian School Education, 17(3). Fernandez, K. E. (2011). Evaluating school improvement plans and their effect on academic performance. Educational Policy, 25(2), 338–367. http://dx.doi.org/10.1177/0895904809351693. Figlio, D. (2011). Does competition improve public schools? Education Next, 11(1), 74-80. Finney, S. J., & DiStefano, C. (2006). Nonnormal and categorical data in structural equation models. In G. R. Hancock & R. O. Miller (Eds.), A second course in structural equation modeling, 269–314. Greenwich, CT: Information Age. Flipp, C. [ChrisFlipp]. (2014, March 17). Reliability and validity [Video file]. Retrieved from https://www.youtube.com/watch?v=9ltvDNAsOI. Florida Department of Education. (2014). 2015-2016 Legislative Budget Request Tallahassee, FL. Fla. Stat. § 1002.395. Florida Tax Credit Scholarship Program. (2012). Foltz, B. [BrandonFoltz]. (2014, December 1). Multiple Regression [Video file]. Retrieved from https://www.youtube.com/watch?v=dQNpSabq4M. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (1993). How to design and evaluate research in education (Vol. 7). New York, NY: McGrawHill. Frumkin, P., & Galaskiewicz, J. (2004). Institutional isomorphism and public sector organizations. Journal of public administration research and theory, 14(3), 283–307. Gaziel, H. H. (1997). Impact of school culture on effectiveness of secondary schools with disadvantaged students. Journal of Educational Research, 90(5), 310–318.

Spiritual Health in Christian Schools

159

Gorsuch, R. L. (1988). Exploratory factor analysis. In J. R. Nesselroade & R. B. Cattell (Eds.), Handbook of multivariate experimental psychology (231–258). New York, NY: Springer. Habermas, R. (2001). Teaching for reconciliation (Rev. ed.). Eugene, OR: Wipf and Stock. Irvine, J. J., & Foster, M. (1996). Growing up African American in Catholic schools. New York, NY: Teachers College Press. Jeynes, W. H. (2002). Why religious schools positively impact the academic achievement of children. International Journal of Education and Religion, 3(1), 16–32. Jeynes, W. H. (2005). The impact of religious schools on the academic achievement of low-SES students. Journal of Empirical Theology, 18(1), 22–40. Jeynes, W. H. (2009). The relationship between Biblical literacy, academic achievement, and school behavior among Christian-and public-school students. Journal of Research on Christian Education, 18(1), 36-55. Jeynes, W. H. (2010). Religiosity, religious schools, and their relationship with the achievement gap: A research synthesis and meta-analysis. Journal of Negro Education, 79(3), 263–279. Jeynes, W. H., & Beuttler, F. (2012). What private and public schools can learn from each other. Peabody Journal of Education, 87(3), 285–304. Jones, D. N., & Gill, C. A. (1998). Comparing measures of sample skewness and kurtosis. Journal of the Royal Statistical Society, 47(1), 183–189. Jonassen, D. H. (2012). Designing for decision making. Educational Technology Research and Development, 60(2), 341–359. http://dx.doi.org/10.1007/s11423-011-9230-5. Knight, G. R. (1980). Philosophy and education: An introduction in Christian perspective. Berrien Spring, MI: Andrews University Press. Kreinovich, V., Nguyen, H. T., & Wu, B. (2013). Towards a localized version of Pearson's correlation coefficient. International Journal of Intelligent Technologies and Applied Statistics, 6(3), 215–224. Lee, V. & Bryk, A. (1993). Catholic schools and the common good. Cambridge, MA: Harvard University Press.

160

Chad A. McBane

Levine, D. U., & Leibert, R. E. (1987). Improving school improvement plans. Elementary School Journal, 87(4), 397–412. http://dx.doi.org/10.1086/461504. Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research, 28(4), 612–625. LifeWay Christian Resources. (2006). Spiritual Growth Assessment Process [Measure instrument]. Retrieved from http://www.lifeway. com/lwc/files/lwcF_PDF_DSC_Spiritual_Growth_Assessment.pdf. Marra, Andrew. (2015, July 18). More poor students using vouchers to attend PBC private schools. Palm Beach Post. Retrieved from http://www.palmbeachpost.com/news/news/local-education/morepoor-students-using-vouchers-to-attend-pbc-pr/nm2nZ/. Marzano, R. (2000). Implementing standards-based education. Teacher Librarian, 28(2), 30–32. Mentzer, M. S. (1988). Religion and achievement motivation in the United States: A structural analysis. Sociological Focus, 21, 307–316. Moore, D. S. Notz, W. I., & Fligner, M. I. (2013). The basic practice of statistics (6th ed.). New York, NY: Freeman. Morris, A. (1994). The academic performance of Catholic schools. School Organization, 14(1), 81–89. New England Association of Schools and Colleges. (2006). The impact of accreditation on the quality of education: Results of the regional accreditation and quality of education survey, NEASC 2005. Bedford, MA: Nadia Alam. Noel, A., Stark, P., & Redford, J. (2013). Parent and family involvement in education, from the National Household Education Surveys Program of 2012, first look (NCES 2013-028). Washington, DC: National Center for Education Statistics. Pazmino, R. W. (1997). Foundational issues in Christian education: An introduction in evangelical perspective (2nd ed.). Grand Rapids, MI: Baker Academic.

Spiritual Health in Christian Schools

161

Peterson, P. E., & Llaudet, E. (2006). On the public-private school achievement debate. John F. Kennedy School of Government – Harvard University. Pillow, T. (2015, May 18). Judge dismisses lawsuit challenging Florida tax credit scholarships. RedefinED. Retrieved from https://www.redefined online.org/2015/05/judge-plaintiffs-lack-standing-to-challenge-floridatax-credit-scholarships/. Pohlmann, J. T. (2004). Use and interpretation of factor analysis in the Journal of Educational Research: 1992–2002. Journal of Educational Research, 98(1), 14–22. Powers, B. P. (2003). Growing faith. Eugene, OR: Wipf and Stock. Quantitative Specialists. (2014, February 3). Pearson r correlation in SPSS [Video file]. Retrieved from https://www.youtube.com/watch? v=IIlyI7bsvIQ. Reeves, D. B. (2010). Finding your leadership focus: What matters most for student results. Teachers College Press. Rice, W. R. (1989). Analyzing tables of statistical tests. Evolution, 43(1), 223–225. http://doi.org/10.2307/240917. Rima, S. D. (2000). Leading from the inside out: The art of self-leadership. Grand Rapids, MI: Baker Books. Sebring, P. A., & Camburn, E. M. (1992). A profile of eighth graders in Catholic schools. Based on the National Educational Longitudinal Study of 1988. Siebersma, M., Wheeler-Clouse, S., & Backus, D. (2011). School improvement, step by step. Educational Leadership, 69(4). Smylie, M. A. (2010). Continuous school improvement. Thousand Oaks, CA: Corwin Press. Snyder, T.D., and Dillow, S.A. (2013). Digest of education statistics 2012 (NCES 2014-015). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Snyder, T. D., & Dillow, S. A. (2015). Digest of education statistics 2013 (NCES 2015-011). Washington, DC: National Center for Education

162

Chad A. McBane

Statistics, Institute of Education Sciences, U.S. Department of Education. Step Up for Students. (2015). A few facts about the Florida Tax Credit Scholarship. Retrieved from https://www.stepupforstudents.org/ newsroom/basic-program-facts. Tabachnick, B. G., Fidell, L. S., & Osterlind, S. J. (2001). Using multivariate statistics (4th ed.). Boston, MA: Allyn & Bacon. Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International journal of medical education, 2, 53–55. Thayer, J., & Kido, E. (2012). CognitiveGenesis (CG): Assessing Academic Achievement and Cognitive Ability in Adventist Schools. Journal of Research on Christian Education, 21(2), 99-115. Truman, H. S. (1947, December 15). Statement by the President making public a report of the commission on higher education. Retrieved from http://www.presidency.ucsb.edu/ws/?pid=12802. U.S. Department of Commerce. (2015). QuickFacts: Boca Raton city, Florida. Retrieved from http://www.census.gov/quickfacts/table/ POP060210/1207300. Vlăsceanu, L., Grünberg, L., & Pârlea, D. (2004). Quality assurance and accreditation: A glossary of basic terms and definitions. Bucharest: Unesco-Cepes. Wenglinsky, H. (2007). Are private high schools better academically than public high schools? Center on Education Policy, 2007. Westerhoff, J. H. (1976). Will our children have faith? Los Angeles, CA: Dove Communications. Wilhoit, J., & Dettoni, J. (1995). Nurture that is Christian: Developmental perspectives on Christian education. Grand Rapids, MI: Baker Academic. Williams, R. B. (2006). Leadership for school reform: Do principal decision-making styles reflect a collaborative approach? Canadian Journal of Educational Administration and Policy, 53, 1-22. Wilson, E. (2014). Students celebrate student choice. Florida Catholic. Retrieved from https://www.stepupforstudents.org/wp-content/ uploads/2015/11/14-9-26-students-celebrate-choice.pdf.

Spiritual Health in Christian Schools

163

Wirt, J., Choy, S., Rooney, P., Hussar, W., Provasnik, S., & HampdenThompson, G. (2005). The Condition of Education, 2005. NCES 2005094. National Center for Education Statistics. Wolniak, G.C., Neishi, K.M., Rude, J.D., & Gebhardt, Z. (2012, August). The State of Our Nation’s Youth: 2012-2013. Alexandria, VA: Horatio Alger Association of Distinguished Americans, Inc. Worthington, R.L., & Whittaker, T.A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806-838.

In: Exploratory Factor Analysis Editor: Diana Mindrila

ISBN: 978-1-53612-486-6 © 2017 Nova Science Publishers, Inc.

Chapter 6

EDUCATOR TECHNOLOGICAL PEDAGOGICAL CONTENT KNOWLEDGE AND STUDENT ACHIEVEMENT Wofford L. Green Floyd County Schools, Rome, GA, US

ABSTRACT The purpose of this study was to develop a typology of teachers within the Technological Pedagogical Content Knowledge (TPACK) framework and to identify groups of teachers that perform better in regards to student achievement as measured by state mandated standardized assessment. A sample of 252 educators from a rural Georgia school district completed a 50-question TPACK survey. The survey data were used to identify underlying factors. Results yielded a 3-factor solution: a) Instructional Practice, b), Technology Integration, and c) Technology Knowledge. Factor scores were then used as input for kmeans cluster analysis, which yielded three teacher profiles: a) HighTPACK, b) Mid-TPACK, and c) Low-TPACK. Demographic data of educators such as age, level of education, and years of experience were tabulated by cluster. Results showed that the High-TPACK cluster (N = 96) consisted mostly of younger teachers, and was predominantly male.

166

Wofford L. Green The Mid-TPACK (N = 143) cluster skewed older with a majority of female educators. The Low-TPACK cluster (N = 14) included mostly educators towards the end of their careers. A chi-square test showed that levels of student achievement differed significantly across the three clusters (χ2(8) = 15.851, p = .045) due to more students performing at a higher level on standardized assessment for teachers in the High-TPACK Cluster. Further, a t-test showed that the average percentage of students meeting and exceeding standards was significantly higher for the teachers in the High-TPACK cluster than in the Mid-TPACK cluster (t(97) = 3.045, p = .004). These results indicate that teachers with higher levels of TPACK have students who perform better on standardized assessment. These clusters consisted of teachers who skewed younger and were more willing to take risks to explore technology integration. Suggestions were made for teacher preparation programs as well as professional development to target the skills needed to get all teachers to the level of High-TPACK.

Keywords: Technological Pedagogical Content Knowledge (TPACK), student achievement, educators, technology integration, professional development, exploratory factor analysis, cluster analysis

INTRODUCTION What skillset and demographics determine what type of teacher is most effective in terms of student achievement? This study examined teacher Technological Pedagogical Content Knowledge (TPACK) to identify profiles of teachers and to determine which teacher profiles had the highest level of student achievement. Participants in this study took the Technological Pedagogical Content Knowledge, or TPACK, survey (Koehler & Mishra, 2009). Teachers also provided demographic information such as years of experience, level of education, age, and other items relating to experience. Profiles of teachers were derived from the survey and demographic data. These profiles were compared to standardized testing data of students taught. The goal of the study was to determine whether some teacher profiles were more effective than others in terms of students’ performance on standardized assessment. Was the younger teacher with more technology knowledge more effective? Was the

Educator Technological Pedagogical Content Knowledge …

167

more experienced teacher with more pedagogical content knowledge more effective? This study identified a set of teacher profiles and the instructional effectiveness of each profile. In a first phase, the study was focused on the identification of the factors that underlie the TPACK data. Teachers’ scores on the identified TPACK factors were then used as input for cluster analysis. Through cluster analysis, a classification of teachers emerged, and categories were compared based on student achievement. This procedure indicated the clusters with the highest level of student achievement. Specifically, this study addresses the following research questions: What are the latent dimensions that underlie the TPACK survey data? What profiles of teachers emerge based on the TPACK dimensions (technological knowledge, pedagogical knowledge, content knowledge) and what are the demographic characteristics of these groups? Are any of the teacher profiles arising from the TPACK survey/demographic data associated with higher academic achievement as measured by student standardized test scores?

LITERATURE REVIEW Background of TPACK When future teachers complete pre-service teacher education programs extended time is spent on what to teach and how to teach, but little time is dedicated learning to incorporate technology (Jordan, 2011). Jordan’s (2011) study of first year teachers showed a lack of confidence integrating technology into the curriculum. To use the technology tools provided, there needs to be teaching of how to use the tools (Koehler & Mishra, 2009; Jordan, 2011). Koehler and Mishra (2009) developed the TPACK framework for understanding the best way to incorporate technological knowledge into existing practice. The TPACK framework suggests that technology knowledge is the same as knowing what to teach and how to teach.

168

Wofford L. Green

Koehler and Mishra (2009) suggest technological knowledge is key to preparing students for the modern life. Koehler and Mishra (2009) also admit teaching with technology is a difficult thing to do well because it requires an ability to continue to evaluate curriculum and the ways technology can work within the curriculum. TPACK is the framework where the knowledge domains of technology, pedagogy, and content intersect. Mishra and Koehler (2006) coined the term TPACK in order to explain that technological knowledge is equally important to a teacher’s knowledge base as content pedagogical and content knowledge. Mishra and Koehler (2006) describe TPACK as: “The basis of good teaching with technology and requires an understanding of the representation of concepts using technologies; pedagogical techniques that use technologies in constructive ways to teach content; knowledge of what makes concepts difficult or easy to learn and how technology can help redress some of the problems that students face; knowledge of students’ prior knowledge and theories of epistemology; and knowledge of how technologies can be used to build on existing knowledge and to develop new epistemologies or strengthen old ones” (p. 1029).

Teachers are tasked with educating responsible, literate citizens. That includes modeling appropriate behavior with various technologies. In order to prepare students for this century’s workforce, teachers need to use this century’s tools (Koehler & Mishra, 2009). TPACK enables teachers to determine what works best between curriculums, pedagogical strategies, and if technology is appropriate in a given lesson (Hofer, 2012). When a teacher is going to teach a lesson, that teacher needs to figure out the content of the lesson and the appropriate delivery method. The delivery method may include appropriate use of digital media, or it may not. Without TPACK, a teacher may not be able to use the tools best available to students (Hofer, 2012). Developing the knowledge to use technology seamlessly takes time. Grunwald and Associates (2010) performed a study for Walden University surveying 1,000 K-12 teachers on the uses of technology in the classroom.

Educator Technological Pedagogical Content Knowledge …

169

The study found that with increased use of technology, teachers begin to recognize and value the positive effect it has on the classroom (Grunwald and Associates, 2010, p. 4). Utilizing a survey on school culture, teaching-efficacy, computerefficacy, playfulness, and anxiety, Howard (2011) examined the risk perceptions of teachers when it comes to integrating technology into the classroom. Howard clustered teachers into two groups based on the likelihood of taking the risk of integrating technology. Both groups saw the integration of technology into their curriculum as something that takes time, but each group negotiated that risk differently (Howard, 2011, p. 268). Teachers who were comfortable with technology and saw a benefit to student achievement were more likely to incorporate technology. Teachers uncomfortable with technology had difficulty seeing the benefit of investing time learning how to use it. Time to practice any new teaching strategy or tool, especially those related to technology, is important when it comes to integrating a new tool into a classroom (BECTA, 2004; Howard, 2011). Fear, anxiety, or discomfort with a new strategy is inhibiting. A 2004 study by the British Educational Communications and Technology Agency (BECTA) found teachers with no confidence in dealing with technology work hard to avoid using it (BECTA, 2004, p. 3). While avoidance of technology may help a teacher feel better in the classroom, it doesn’t develop competency with using technology. In order to develop competencies with technology, an investment of time is critical (BECTA, 2004). Akengin (2008) found similar issues with teacher avoidance and competencies related to the use of technology. Akengin (2008) found a disadvantage of using technology is that it would take too much time to prepare the lesson. Educators want students to learn. Means (2010) found teachers who spent time training in use of educational software showed higher levels of student achievement. Erdogan and Sahin (2010) found that teacher preparation programs with an emphasis on technology training produced better qualified teachers. These better qualified teachers are able to enhance learning.

170

Wofford L. Green

For some, the use of technology is a risk-taking behavior (Howard, 2011). Gaining knowledge about technology mitigates that risk. With that knowledge increased, teachers are better equipped to reach today’s learners. Van Vooren and Bess (2013) conducted a study of educators using Twitter to reach out to students and found teachers who were willing to learn something new were able to produce students with higher standardized test scores. Prensky (2001) stated, “The single biggest problem facing education today is that our digital immigrant instructors, who speak an outdated language, are struggling to teach a population that speaks an entirely new language” (p. 2). Teachers in the Van Vooren and Bess study were able to reach out and learn a new language. The risk to learn something new paid off for these students. The TPACK model is critical to bring teachers fluidity of learning styles. Teachers do not need Twitter or knowledge of how to use social media in order to reach their students, but knowledge of technology increases the number of tools at a teacher’s disposal. Those tools are useful when reaching out to students. The current study examined teachers’ TPACK in relation to their demographic characteristics. What does the profile of an effective, high quality, teacher look like? The Center for High Impact Philanthropy (2010) offered a definition of a quality teacher: “A quality teacher is one who has a positive effect on student learning and development through a combination of content mastery, command of a broad set of pedagogic skills, and communications/ interpersonal skills. Quality teachers are lifelong learners in their subject areas, teach with commitment, and are reflective upon their teaching practice. They transfer knowledge of their subject matter and the learning process through good communication, diagnostic skills, understanding of different learning styles and cultural influences, knowledge about child development, and the ability to marshal a broad array of techniques to meet student needs. They set high expectations and support students in achieving them. They establish an environment conducive to learning, and leverage available resources outside as well as inside the classroom” (p. 7).

Educator Technological Pedagogical Content Knowledge …

171

The combination of content mastery, command of pedagogy and a commitment to lifelong learning has a direct relationship to the knowledge needed within the TPACK domains. The skills listed above are the recipe for a quality teacher who can best enhance student achievement.

TPACK and Demographics This study utilized teacher demographic information such as age, experience, gender, as well as certification to assist in building profiles of educators. Literature stated that younger teachers are less likely to be able to devote more time to the profession, but years of experience allows for an expert to be in the classroom (Day, 2012; Hargreaves, 2005; Aziz, 2012). There was no clear definition in the literature on gender superiority in education (Dee, 2006). Clotfelter, Ladd, and Vigdor (2010) did find teachers who were certified performed better than teachers who came in through alternative preparation programs.

TPACK and Factor Analysis in Literature Sahin (2011) and Schmidt (2009) each used exploratory factor analysis in their studies of TPACK. Working within the seven subscales of TPACK, Sahin (2011) started with an item pool of 60 questions. These items were evaluated by ten cross-curricular faculty members (Sahin, 2011). Evaluation criteria consisted of items “totally measuring,” “somewhat measuring,” and “not measuring” (Sahin, 2011). To be included in the survey, each item needed to receive a score of “totally measuring” by a minimum of seven faculty members (Sahin, 2011). Upon completion, Sahin was left with 47 items for the survey. Sahin then tested the survey reliability and validity by utilizing factor analysis and identified seven TPACK factors. As seen in Figure 1 from Sahin (2011), items had factor loadings between 0.60 and 0.90. Sahin determined all items qualify for admission into the survey instrument.

172 Item

Wofford L. Green TK

PK

CK

TPK

TCK

PCK

TPACK

1

0.733

0.800

0.599

0.831

0.823

0.824

0.872

2

0.751

0.833

0.752

0.903

0.892

0.811

0.891

3

0.816

0.872

0.855

0.895

0.877

0.848

0.884

4

0.706

0.849

0.792

0.819

0.865

0.847

0.832

5

0.774

0.856

0.826

0.859

0.882

6

0.650

0.773

0.772

0.858

7

0.633

8

0.732

9

0.779

10

0.736

11

0.614

12

0.798

13

0.653

14

0.639

15

0.708

0.764

KMO

0.940

0.896

0.817

0.796

0.789

0.903

0.878

BTS

3186.27

1316.39

998.56

817.54

830.98

1737.51

1279.14

p