The effect of the structure of test items on their factor composition

Citation preview

THE EFFECT OF THE STRUCTURE OF TEST ITEMS ON THEIR FACTOR COMPOSITION

A Dissertation Presented to the Faculty of the Department of Psychology The University of Southern California

In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy

hy Constance Dora Lovell June 1942

UMI Number: DP30362

All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion.

Dissertation Publishing

UMI DP30362 Published by ProQuest LLC (2014). Copyright in the Dissertation held by the Author. Microform Edition © ProQuest LLC. All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code

ProQuest LLC. 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, Ml 48106- 1346

T h is d issertation, w r i t t e n by

-G.01'JS!f,ANCE...D.QRA._JL0.VELL.................. u n d e r the g u id a n c e o f h&T?. F a c u l t y C o m m itte e on S tud ie s, a n d a p p ro v e d by a l l its mem bers, has been pre sen ted to a n d accep ted by the C o u n c il on G ra d u a te S tu d y a n d R esearch, in p a r t i a l f u l ­ f i l l m e n t o f re q u ire m e n ts f o r the degree o f D O C T O R O F P H IL O S O P H Y

Secretary D a te

jl.una.,....19.42.

Committee on Studies

..

ACKNOWLEDGMENTS The author wishes to express her appreciation to Doctor J. P. Guilford for his suggestion of the study and for his help with it.

Acknowledgments are also made to

Doctor John W. Todd and Doctor Neil D. Warren for their assistance in securing subjects and to Doctor R. R. G. Watt and Mr. Arthur Tait for their aid in the use of the Hollerith machines.

TABLE OF CONTENTS CHAPTER

PAGE

I . THE PROBLEM AND DEFINITION OF TERMS . . . . Introduction ...........

1

.. .. . . . .

1

The p r o b l e m .....................

4

Statement of the problem........

4

Importance of the s t u d y ......... .

8

Definition of t e r m s ............

.

12

Structure.....................

12

Factor composition.............. .. .

14

Organization of remainder of the disser­ tation ............................. II.

REVIEW OF THE LITERATURE............

15 17

Literature on the factor composition of tests of human ability..... ....

17

Literature on the technique of factor analysis studies

......... . .

27

Factor analysis studies of test items, .

27

Selection of subjects in factor analysis studies

.................

28

Selection of items in factor analysis studies.....................

32

CHAPTER

PAGE Literature on difficulties of factor analysis methods . . . . . . . . . . .

III.

THE SOURCES OF DATA

35

.........

41

Construction of test items and adminis­ tration of tests

..................

41

Construction of test items.......

41

Administration of the t e s t s .....

53

Selection of s u b j e c t s ......... . . . . Selection of items

,

Correlation of the data IY.

............

54 55

.............

FACTOR ANALYSIS OF THE D A T A ........ The extraction of the centroid factors

56 63

..

63

Centroid factors in the number series completion data

. . . . . . . . . . .

66

Centroid factors in the figure analogies d a t a ......................... Rotation of the axes

71

...........

Interpretation of the r e s u l t s .....

81 89

Results from the number series completion t e s t ..................

94

Results from the analysis of the figure analogies items

.....

.........

108

The relation of the difficulty of the items to the factors present...

123

CHAPTER

PAGE General results of the

V.

study

....

125

.

131

SUMMARY AND CONCLUSIONS

Summary................................

131

Conclusions .

.........................

137

SELECTED BIBLIOGRAPHY...........................

141

APPENDIX A. SAMPLE OF TEST BOOKLET AND PROCEDURE USED IN ADMINISTRATION OF T E S T S .......... APPENDIX B.

146

GRAPHS OF FACTOR LOADINGS AFTER

FINAL ROT AT IO NS ......................... APPENDIX C.

155

CHARTS OF FIGURE ANALOGIES ITEMS WITH

POSITIVE, VANISHING, ANDNEGATIVE ON EACH FACTOR

LOADINGS .

202

LIST OF TABLES TABLE I.

PAGE Proportion of Passes, Sex Difference of Proportion of Passes, and WTW Ratios for Number Series Completion Items

II.

.......

57

Proportion of Passes, Sex Difference of Proportion of Passes, and "T" Ratios for Figure Analogies Items

III.

.......

58

Inter-correlations of Number Series Completion I t e m s ..............

IV.

61

Inter-correlations of Figure Analogies Items...........................

V.

62

Centroid Factor Loadings for Number Series Completion Items (First Set of Extractions)

VI.

.

72

Centroid Factor Loadings for Number Series Completion Items (Second Set of Extractions)...................... . .

VII.

73

Centroid Factor Loadings for Number Series Completion Items (Third Set of Extractions)

VIII.

..................

74

Communalities for Number Series Completion Items (First Set of Extractions)

....

75

vi kGE Communalities for Humber Series Completion Items (Second Set of Extractions) .......

76

Communalities for Humber Series Completion Items (Third Set of Extractions)

.......

77

Centroid Factor Loadings for Figure Analogies Items............... . Communalities for Figure Analogies Items

82 . .

83

Rotated Factor Loadings for Humber Series Items........................ .

90

Rotated Factor Loadings for Figure Analogies 91

Items........................ . Communalities for Humber Series Items after

92

R o t a t i o n ...................... . Communalities for Figure Analogies Items after R o t a t i o n ........... * ......... .

93

Humber Series Completion Items Arranged in Order of Loadings on Factor I, the Rules for the Items, and the Proportion of Passes

. .

96

for the Items, and the Proportion of Passes„

97

Humber Series Completion Items Arranged in Order of Loadings on Factor II, the Rules

Humber Series Completion Items Arranged in Order of Loadings on Factor III, the Rules

viii TABLE

PAGE for the Items, and the Proportion of P a s s e s .......... . ...................

XX.

98

Number Series Completion Items Arranged in Order of Loadings on Factor IT, the Rules for the Items, and the Proportion of P a s s e s ..............

XXI.

99

Number Series Completion Items Arranged in Order of Loadings on Factor T, the Rules for the Items, and the Proportion of Passes

XXII*

..............

113

Loadings of Figure Analogies Items on Factor V

XXVII.

112

Loadings of Figure Analogies Items on Factor -IT...........................

XXVI.

Ill

Loadings of Figure Analogies Items on Factor I I I ..........................

XXV.

110

Loadings of Figure Analogies Items on Factor I I ........................

XXIV*

100

Loadings of Figure Analogies Items on Factor I

XXIII.

.........

........................

114

Loadings of Figure Analogies Items on Factor V I .........

115

TABLE XXVIII.

PAGE Loadings of Figure Analogies Items on Factor V I I .........

XXIX.

Loadings of Figure Analogies Items on Factor V I I I .............

XXX.

117

Loadings of Figure Analogies Items on Factor I X ..............................

XXXI.

116

118

Bange of Proportion of Passes for Items with High and Low Loadings on Each Factor . . .

124

CHAPTER

I

THE PROBLEM AND DEFINITION OF Tim© I . INTRODUCTION Most tests of human ability are positively correlated. However, the varying sizes of the coefficients show that high performance on any one test is not equally associated with high scores on all the others.

In a table of test

inter-correlations, rough groupings can be made so th&t the association between the tests in any group is' relatively large as compared to their associations with other tests in the battery.

In such a group, the high correlations between

the tests indicate that there is some factor common to per­ formance of the tasks involved.

Any other group in the bat­

tery whose constituent tests are highly correlated involves a similar common factor.

However, the two factors are not

the same, because the tests of one group have relatively low correlations with those of the other.

Any positive relation­

ship existing between groups is an indication of overlapping factors. The presence of underlying associations between tests is shown only crudely by such grouping according to the size of the inter-correlation coefficients.

More refined

techniques are necessary for accurate study of the factor pattern of a correlation table.

The various systems of fac­

tor analysis were developed for this purpose.

They provide

the means of finding (l)'the smallest number of factors by which a table of inter-correlations can be described and (2) the weight of each test in these factors. The categories located in this manner may be defined generally as functional unities contributing to performance in the tasks containing them.

More specific identification

of the operation each involves is made by studying the nature of the tasks which are weighted with them.

Thus, the

factor which has heavy loadings in tests of addition, multi­ plication, and division--and only slight loadings in verbal and performance tests— has been called the number factor. This and other factors identified as verbal, spatial, memory, and mental speed have been located in many different tests, 1 and by several methods of factor analysis. ^Karl 1. Holzinger and Frances Swineford, ”A Study in Factor Analysis: The Stability of a Bi-Factor Solution,” Supplementary Education Monographs , XI/VTII (March, 1939) , pp. 8-9. Karl J. Holzinger and Harry H. Harmon, ■tTComparison of Two Factorial Analyses,” Fsychometrika, III (March, 1938), p. 45. Willard Harrell, ”A Factor Analysis of Mechanical Ability Tests,” Fsychometrika, V (March, 1940), p. 17. Benjamin Balinsky, ”An Analysis of the Mental Factors of Various Age Groups from Nine to Sixty,” Genetic Psychology Monographs. XXIII (February-, 1941), p. 231. Robert Blakey, ”A Re-analysis of a Test of the Theory

Tests which require the subject to do many different things— such as general intelligence tests— are unsuitable for factor analysis work.

They are loaded with so many

factors that a clear-cut description of particular cate­ gories is impossible.

The nature of factors is revealed 2 most clearly in very simple tasks. They are characterized not only by relatively uncomplicated factor patterns but also by high saturations of the categories present in them.

3

However, the fact that a test is simple does not guarantee that it will be a measure of only one factor.

For

example, the number series test— with similar material and of Two Factors,” Fsychometrika, ¥ (June, 1940), pp. 121-36. Robert Blakey, "Factor Analysis of a Non-¥erbal Reasoning Test," Educational and Psychological Measurement, I (April, 1941), p. 197. Clyde H. Coombs,"A Factorial Study of Number Ability”" Fsychometrika, YI (June, 1941), pp. 161-189. L. L. Thurstone, "Experimental Study of Simple Structure," Fsychometrika, ¥ (June, 1940), pp. 153-68. L. L. Thurstone, "Primary Mental Abilities," Psycho­ metric Monographs, I (1938), pp. 79-89. Herbert Woodrow, "The Common Factors in Fifty-two Mental Tests," Fsychometrika, I¥ (June, 1939), pp. 99-108. Ruth Wright, "A Factor Analysis of the Original Stanford-Binet Scale," Fsychometrika, I¥ (September, 1939), pp. 209-20. 2L. L. Thurstone, "Current Misuse of the Factorial Methods,” Fsychometrika, II (June, 1937), p. 76. 3 L. L. Thurstone, "Current Issues in Factor Analysis, Psychological Bulletin, XXX¥II (April, 1940), p. 200.

form in all items— has been found to have loadings in six different factors.

4

Most tests are loaded with several.

This complexity of loading prevents an accurate identification of the type of operation associated with each category.

The best that can be done is to study (1) the

similarities in tests found to be heavily loaded with a factor and (2) the differences between these tests and others in the battery not so loaded.

Use of this procedure has

resulted in tentative identification of a number of the factors and has made it possible to set up hypotheses con­ cerning the operations they involve. II. THE PROBLEM Statement of the problem. It was the purpose of this study to test certain hypotheses concerning the nature of factors found in two tests of human ability: (1) figure analogies and (2) number series completion. In previous factor studies of figure analogies tests, loadings in spatial ability, verbal ability, perceptual 5 speed, deduction, and induction have been obtained. Eor ^L.L. Thurstone, "Primary Mental Abilities,” Psycho­ metric Monographs, I (1938), p. 116. P. Guilford, "Human Abilities,” Psychological Review. XLYII (September, 1940), p. 390.

5 tests of this kind, which have the same type of material and the same form of question throughout, three possible reasons for such complex factor patterns exist.

The individual

items may each be loaded with the several factors, different subjects may use different functions in responding to the same item, and/or

the factor structure may vary from item

to item,resulting

in a complex pattern for the test as a

whole.

With the assumption that the last situation exists,

1. P. Guilford set up hypotheses as to the types of item which might be expected to be loaded with the various fac­ tors found in figure analogies tests.

The following state­

ments concerning these are quoted from his article: . . . Perceptual speed will probably be most in evidence in a timed test when: (1) The figures are small or detailed. . . . (2) The figures are relatively complex. .. . (5) The change from A to B is slight. . .. (4) When in a multiple-choice response the cor­ rect figure is hard to discriminate from among its distractors, the rule or principle having been easy to apprehend. . . . The factor of spatial thinking will be most im­ portant when: (1) The principle of change depends upon a rotation. . .; (2) The principle depends upon inversions. . .; or (3) Upon the rearrangement of parts. The verbal factor might be expected to enter to some extent when the rule is complex or is easily and naturally verbalized,. . . . This will also depend upon the individual, however, and his habits of depending or not depending upon verbalized methods. Here is a good example of vicarious function­ ing of unitary abilities.

6 Deduction may be expected to enter most when: (1) There are fine distinctions to be made among several alternative rules all of which come close to fitting. . . . The relation in Item 9 could be con­ ceived as one of size and change of position or as a whole-part relationship. The responses do not pro­ vide for the first rule but do for the second, there­ fore the second rule is correct. (2) There are fine distinctions to be made among alternative answers. . . . Finally, the inductive process, for which the test was primarily constructed, will have its great­ est opportunity to reveal itself when: (1) All other factors are at a minimum, that is to say, when the figures are large and lacking in detail, no rotation or rearrangement of parts is required, and discriminations are simple. (2) The minimum essential for a correct response requires, a correct apprehension of the rule, . . . . (3) The same kind of principle is not repeated; a frequent change in kind of rule, as in the numberseries test, demands new inductions and not simply the reproduction of already learned ones. . . . The moderate number of kinds of rule and their various combinations give- us much latitude for this. But when the principle is changed too often and too radically, there is much chance of some new ability being brought into importance. . . . The ability to shift mental sets quickly may some time appear as a factor, but has not done so as yet, unless it is to be identified with fluency or some other ability.6 Guilford has also suggested ways of constructing number series completion items to influence their factor loadings.

In tests of this kind, factors have been found

which have been identified as number, induction, deduction, and perceptual speed.

Guilford writes concerning these:

6 Ibid.. pp. 390-92.

7 The number factor is more important when the grasping of the principle and the selection of the answer depend much upon accurate computation. The inductive factor is more prominent when the compu­ tations required are very simple and computational slips are rare. The fact that most of the principles in number series are essentially numerical principles probably precludes an elimination of the number factor entirely. If a test is very much confined to a few kinds of principles, for example, if all of them are simple increments and decrements ■, the number factor should again come into greater prominence and per­ haps the perceptual speed factor. On the other hand, if the principle changes very frequently and varies radically in kind from item to item, the induction factor should gain in importance. From this it would follow that a number-series test that is opti­ mal for measuring inductive ability should change the kind of principle, at least slightly with every new item. Testing the hypotheses formulated by Guilford would involve 1. Setting up a figure analogies test and a number series completion test, each containing several items con­ structed according to the various methods listed above. 2. Administering the two tests to an adequate sample. 3. Correlating, for each test, the responses to each item with those to every other item. 4. Making a factor analysis for each of the corre­ lation matrices thus obtained. If the hypotheses were substantiated by these methods, the results on the figure analogies test would show five 7

» P- 389.

factors.

Items constructed according to the first group of

four suggestions would be heavily loaded with one factor, less heavily loaded with all others.

Items of the next three

types would be heavily loaded with a second factor, but not so heavily loaded with Factor I, and so forth.

Similarly,

factor analysis of the number series test would reveal four factors— with all the items designed to measure induction having heaviest loadings in the same factor; all the items designed to emphasize number ability having heaviest loadings in another factor; and so forth. Stated specifically, the purpose of this study was to test the hypotheses outlined by Guilford according to the method given on page 7. Importance of the study. Factor analysis studies up to the present time have been handicapped greatly by the fact that most tests have very complex factor patterns.

8

Because of this difficulty, identification of factors has had to remain very tentative in nature.

In addition, no

pure measures of the factors have been possible. Thurstone has commented regarding this problem: Now that several of the factors are somewhat better understood, it should also be expected that tests

8 L. L. Thurstone, "The Perceptual Factor," Psychometrika, III (March, 1938), p. 8.

will "be improved by increasing the saturation of the factor that each test is expected to measure and by decreasing the saturations of other factors that are measured by other tests. What we have called the "complexity” of each test should be reduced.® If

Guilford's hypotheses were upheld by the present

study, several results of value would be obtained: 1. The findings would confirm present identifications of the factors studied. 2. They would be usable in setting up tests to measure the presence of the factors more effectively.

For example,

suppose that a measure of the induction factor were desired. If it were known that questions of certain kinds were highly loaded with the induction factor, they could be used, and questions heavily loaded with other factors could be avoided. 3. Considerable controversy has existed as to the importance of factors in mental life.

Some writers regard

them as primary mental abilities; others, as nothing but mathematical artifacts.

10

The present study does not deal

9 L. L. Thurstone, "Primary Mental Abilities," Psychometric Monographs, I (1938), p. 92. Anne Anastasi, "The Influence of Specific Experi­ ence upon Mental Organization," Genetic Psychology Monographs, XVIII (August, 1936), pp. 333-35": Karl I. Holzinger and Frances Swineford, "A Study in Factor Analysis: The Stability of a Bi-Factor Solution," Supplementary Education Monographs, XLVIII (March, 1939), p . 18. Dael Wolfle, "Factor Analysis to 1940," Psychometric

10 with this matter.

However, any decision regarding it would

be facilitated by better measures of the factors. 4.

If some of the controversial factors are later

agreed upon as primary abilities, it is possible that tests of them would be useful in guidance work.

Provided that the

factors essential to performance in various vocations and avocations were known, relative success in a large number of them might be predicted from scores on a small number of primary factors.

Use of measures of underlying unities in

this way v/ould be much more economical than the present use of many tests chosen on the basis of their direct relation to success in individual fields.

The possible application

of factor studies to guidance was, however, not of special concern in the present study, nor, in view of the present status of opinion regarding the subject,could any of the results of this study be of immediate importance in this field. Even if the findings of the present study did not confirm Guilford’s hypotheses, they still would be of value.

They might indicate three things: 1. That the hypotheses were not justified.

Monographs,111 (1940), p. 1 . L. L. Thurstone and Thelma Gwinn Thurstone, "Factorial Studies of Intelligence," Psychometric Monographs, II (1941), p. 9. L. L. Thurstone, "Current Issues in Factor Analysis," Psychological Bulletin, XXXVII (April, 1940), p. 189.

11 2. That the assumption that the factor pattern of the tests was determined by differences in factor loadings from item to item was not justifiable. 3. That the items used were not adequate expressions of the hypotheses.

The possibility of this conclusion might

seem to render doubtful the technique of the study.

However,

the items were constructed to fit the hypotheses as best the experimenter could.

As a check on this work, they were

examined by Guilford, who had formulated the method of construction. items.

He regarded them as apparently satisfactory

If they were not adequate expressions of the

hypotheses, it could be assumed that the construction of satisfactory items was dependent on distinctions not clearly evident in the hypotheses.

For practical purposes of test

construction it would be of importance to know this diffi­ culty. The present study did not provide means of differ­ entiating among the three possibilities enumerated"above. *#k.

If the hypotheses were not confirmed, the next step would be to set up tests of each of the three subsidiary possi­ bilities.

For example, (1) new hypotheses might be con­

structed; (2) subjects might be differentiated according to their heaviest factor score, and factor analyses for dif­ ferent groups of subjects (each stressing a different factor)

might be made to determine how each group would react to the items in these tests; or, (3) verbal reports might be ob­ tained from subjects concerning how they responded to each item, to indicate whether the operations apparently involved were actually present. Whether or not the findings confirmed the hypotheses, they might help clarify the situation regarding test com­ plexity.

This would be of value. III.

Structure.

DEFINITION OF TEEMS

In this study differentiation has been

made between the material, the form, and the structure of a test item.

The term material has been used in reference to

the type of element of which the item was constructed.

Two

materials were used in this study: geometric* .figures and numerals. The term form has been applied to the mode of presentation of the materials.

One test in this study was

in the form of analogies; the other was in the form of the ft

completion of a series.

The material and form of the number series completion items are illustrated by the following sample: __

1

4

.m * .

7

10

13

16

19

,

_a_

_b_

_c_

_d_

_e_

20

21

22

23

24

Answer t

&

. f* -

In this item each element in the first group of numbers is three larger than the one before it.

The task is to

■

13 determine the next number in the series (22) and to select the answer containing it (c) from the second group of numbers. An illustrative figure analogies item is given below: A

B

C

1

(2

3

4

5-

Ans.

(

1 Items of this type contain two figures which stand in a certain relationship to each other (A and B), a third figure (C), and figures (1, 2, 3, 4, 5) from which the answer is to be chosen.

The task is to find the relationship between

A and B and then to select from the last group of figures the one which stands in the same relationship to C as B does to A.

In the illustration above, two dots (Answer Number 2)

are related to one dot (C) as two lines (B) are related to one line (A).

Or A is to B as C is to 2.

In contrast, the term structure has been applied to that aspect of the item dealing with what it makes the testee do.

For example, some items may be so constructed that,

from item to item, the subject must change the rule he uses for answering. among responses.

Other items may demand fine distinctions The structure of an item relates, then, to

the special operation necessary for correct response. Definitions of material and form have been given in this section primarily to make the meaning of structure

14 clear.

In the study, they were held constant throughout

each test.

Attempt was made only to vary structure from

item to item. Factor composition. In this investigation, factor composition has been used to refer to the loadings for each test item obtained from factor analysis of inter-item corre­ lations by the Thurstone technique.

This method involves

the extraction of a small number of centroid factors and ro­ tation of axes into psychologically meaningful positions. With this type of analysis it was possible for each item to have loadings in one or more common factors, the size of the loadings varying from

1 through 0 to -1.

There are numerous methods of factor analysis.

Several

of the preferred ones result in factor matrices which, from a statistical point of view, fit correlation tables equally well.^

No generally acceptable psychological criterion has

been found for final choice among them.

As Guilford has

pointed out, ”At this stage of things factorial, any choice of method or theory is largely a matter of prejudice,in 12

view of the absence of any final proof which compels assent.” v y ... Karl J. Holzinger, ”A Synthetic Approach to Fac­ tor Analysis,” Fsychometrika, V (December, 1940), p. 243. 12

J. P. Guilford, op. cit., p. 368.

15 The Thurstone technique was chosen for this study because: 1. The hypotheses which the study was designed to investigate were formulated largely in connection with the previous work of Thurstone. 3.

The method is designed to yield results which are

psychologically meaningful--a necessary condition if attempt is to be made to identify the operations involved in the factors. 3. The method does not demand any particular kind of 13 factor pattern, as do some of the other types. 4. When the technique is followed, tests of human ability fall into definite clusters, indicating that the factors are differentiable variables. Throughout this report, then, the term factor compo­ sition is a short way of saying Mfactor composition as determined by the Thurstone method of analysis."

It is not

interpreted as having more general meaning. IV.

ORGANIZATION OF REMAINDER OF THE DISSERTATION The rest of the dissertation is organized as follows:

Chapter II contains a review of literature pertinent to the study.

The construction of the test items used in the

13 Kqrl I. Holzinger and Harry H. Harmon,nComparison of Two Factorial Analyses," Psychometrika, III (March, 1938), 59.

16 research and the selection of subjects are discussed in Chapter III.

Following that is a chapter on the factor

analysis of the data and the results of that analysis.

The

final chapter contains a summary of the study and a section on the conclusions from it.

CHAPTER II REVIEW OE THE LITERATURE This chapter, divided into two sections, deals with the literature pertinent to this study.

In the first part,

five investigations of the factor content of tests of human ability are reviewed. cited.

Many other studies could have been

These were chosen because they used the Thurstone

method and because three of them included the type of test studied in the present investigation.

The second section

deals with the material on the technique of factor analysis studies. I.

LITERATURE ON THE FACTOR COMPOSITION OF TESTS OF HUMAN ABILITY

In the first major application of Thurstonefs method of factor analysis, fifty-six psychological tests were given to 240 male college students.

Included in the battery were

a number series test and a figure analogies test.

Inter­

correlations were computed for all of the fifty-six tests, and the coefficients thus obtained were factored by the centroid method.

After rotation of axes, there were nine

factors which could be given meaningful psychological inter­ pretation.

They are listed on the following page, together

18 with, the loadings of the two tests being used in the present study.

Loadings of .20 and above were considered non­

vanishing, and loadings of .40 and above were considered in naming factors.^ Factor Spatial (facility in spatial and visual imagery) Perceptual speed (facility in per­ ceiving detail imbedded in irrelevant material) Number (facility in numerical calculation) Verbal (facility in verbal relations) Word fluency (fluency in dealing with words) Memory Induction (facility in finding rules for items) Deduction (facility in finding and applying rule) R (facility in completing tasks involving some form of restriction in the solution)

Number Series

Figure Analogies

.059

.197

.087 .348 .296

.435 -.029 .179

.003 .258

.182 .007

.503

.392

.287

.254

.091

.34-1

Because the perceptual speed factor was not readily describable from the above study, further investigation was undertaken to determine its nature.

A battery of tests with

heavy loadings in the previously found factors, plus nine new tests designed to emphasize perceptual speed, was given to 215 high school seniors.

Except in one case, all the new

tests were heavily loaded with a common factor, which was identified as perceptual.

However, tests which had hitherto

1 L. L. Thurstone, "Primary;Mental Abilities," Psychometric Monographs, I (1938), pp. 1-121.

19 shown significant loadings in the perceptual factor shifted, in this study, toward other factors.

Thus, identification

of the factor in the new tests with the one previously found was not clear.

The other factors identified were similar

to those of the first investigation. Thurstonefs third study was made with 286 high school seniors as subjects.

Thirty-six tests were used.

Sight of

them were new ones designed to clarify the nature of the in­ duction factor.

The rest included both new and previously

used tests for the other factors. was in the battery.

The number series test

Its loadings were as follows:

Number Perceptual speed Induction Deduction

.27 .11 .26 .47

It had zero loadings on the verbal, word fluency, space, and memory factors.

Although all the new tests designed to

measure induction had loadings in the factor so designated, none of the saturations were very high. To give further evidence as to the nature of the word fluency factor and to measure previously found factors, 710 ’eighth grade children were given a battery of sixty

^ L. L. Thurstone. ”The Perceptual Factor,” Psycho­ metric, III (March, 1938), pp. 1-17. rz

L. L. Thurstone, "Experimental Study of Simple Structure,” Psychometrika, V (June, 1940), pp. 153-68.

20 tests.

Factors located were essentially the same as those

found in Thurstone*s previous studies: number ability, word fluency, spatial ability, verbal ability, induction, memory, and perceptual speed.

The authors felt certain of the

existence of a perceptual speed factor but were not satisfied. with attempts to interpret its nature.

Three additional

factors were found for which no interpretation was thought 4 possible. Blakey conducted a study designed to measure in a non-verbal manner "the higher intellective processes of comprehension, mental alertness, deductive reasoning, in­ ductive reasoning, and spatial relations or analysis."

The

figure analogies test was included in his group of ten tasks. Subjects were 286 high school pupils.

The factors found and

loadings for the figure analogies test on each were as follows: A. Space or perceptual speed (ability involving quick change of response from item to item with only the simplest discrimination necessary) B. Perceptual discrimination (emphasis on analytic perception in which a fine discrimination must be made rather than a speedy response to a simple stimulus) C. Inductive reasoning D. General or deductive reasoning (All tests had loadings on this factor;

.076

.162 .415

^L. L. Thurstone and Thelma Gwinn Thurstone, "Factor­ ial Studies of Intelligence,” Psychometric Monographs, 11 (1941), pp. 1-94.

21 relative amount of projection seemed to increase with the complexity of the mental function involved.) E. Might he deduction

.507 .244

The figure analogies test, having significant loadings in three factors, was regarded as the best general test of all 5 the reasoning processes. These investigations have shown the appearance of similar factors from study to study despite introduction of new tests and elimination of others.

The nature of these

factors may best be judged by studying the tests containing them.

However, some idea of what they involve may be gained

from the descriptions published recently by Thurstone and Thurstone.

They are given below.

Of particular importance

are the descriptions of the number, perceptual speed, in­ duction, deduction, and spatial factors, since they were the ones with which the present study was concerned.

The

descriptions *of the verbal, word fluency, and memory factors are given as well, in order that the other factors may be contrasted with them. The verbal factor V is one of the clearest of the primary mental abilities. It can be expected in any of the tests involving verbal comprehension— for example, tests of vocabulary, opposites and synonyms, the completion tests, and the various reading-comprehension tests. It is also involved in such verbal5

Robert I. Blakey, nA Factor Analysis of a Non-Verbal Reasoning Test,” Educational and Psychological Measurement, I (April, 1941), pp. 187-198.

22

comprehension tests as proverbs, absurdities, and, to some extent, in syllogistic tests and in state­ ment problems in arithmetic where verbal comprehension is significantly involved.s The word-fluency factor W is also one of the most clearly defined primary mental abilities. It is involved whenever the subject is asked to think of isolated words at a rapid rate. It is for this reason that we have called it a "word-fluency factor?’ It can be expected in such tests as anagrams; rhyming; producing words with a given initial letter, prefix, or suffix; or writing words in a given cate­ gory, as boys1 names or things to eat and drink. Any task in which the verbally fluent person has an ad­ vantage should involve this factor, which is clearly distinct from the verbal-comprehension factor. These two verbal factors are, however, correlated. Whether the correlation is in some way intrinsic is a question that cannot yet be answered. . . . Tests of the sort that we have found for this factor have also been used by some investigators as tests of temperamental qualities. It is not unlikely that the word-fluency factor is indicative of some temperamental traits in addition to its cognitive implications.7 The space factor S is another of the clearly de­ fined primary mental abilities. It seems to be involved in any task in which the subject manipulates an object imaginally in two or in three dimensions. . . . The best tests for this factor are those we have called "Cards,” "Figures," and "Flags," which all involve the manipulation of a simple object in two or three dimensions.8 Another primary factor that is clearly defined is the number factor N. It is involved in simple arithmetical tasks. This factor can be expected in any test in which the subject actually does simple 6 L. L. Thurstone and Thelma Gwinn Thurstone, o£. cit., p. 2 . 7

, p. 3.

8 Ibid., p. 4.

23 arithmetical work, but it is not. found in a test simply because it contains numbers. A simple can­ cellation test with numbers probably will not involve the number factor; but if the subject is asked to check every number that is larger than the adjacent numbers, this factor can be expected. Arithmetical reasoning tests with statement problems have been found to involve the number factor to some degree, as well as other factors, such as the verbal and the inductive. The best tests for the number factor are the simple numerical tasks. Two of the number tests have as high validity as the tests for the two verbal factors and the space factor, and the simple number tasks have been consistent in reveal­ ing the number factor in all studies in which such tests have been included. . . . Elsewhere Thurstone has commented that what has been called the number factor may involve a more general category, that it may appear primarily in number tasks at the present time because we have no good tests for it which use other material."1'0

It may be that the differentiating character­

istic of "number” ability is facility in manipulating a wellpracticed symbolic system according to a specific set of rules and that loadings in it are not dependent on the type of material used.

Preliminary experimentation has given support to this hypothesis. 11 The memorizing factor M is one of the clearly defined factors, although the tests for it do not 9

, p. 5.

10 L. L. Thurstone, "Primary Mental Abilities," Psychometric Monographs, I (1938), p. 83. Clyde H. Coombs, ”A Factorial Study of Number Ability," Psychometrika, Y1 (June, 1941), pp. 188-89.

24 have validities so high as the tests for the verbal and the space factors. The memorizing factor M is to be expected in any test in which the subject profits by ability to memorize anything quickly. It is involved in rote memory for words, numbers, paired associates, and the memorizing of names. The factor transcends the immediate nature of the con­ tent; the same memory factor has been found in tests with verbal, numerical, and spatial content. This factor seems to be quite distinct from the other primary mental abilities in that the correlations between the memory factor and the other primaries have been found to be rather uniformly low.12 The inductive factor I has been found in several factorial studies, but the tests for this factor do not have validities so high as we should desire. The factor is involved in tasks that require the subject to discover a rule or principle that covers the material of the test. It has been found in the well-known number series tests and appears in simi­ lar tests constructed with letter series. The de­ ciphering of code also involves the inductive factor. The inductive factor has appeared in tests of varied content, including verbal, spatial, and numerical tasks, so that the factor seems to transcend the immediate nature of the content. Although we have not succeeded, so far, in finding tests with high validities for this factor, the existence of the factor seems to be fairly clear. The inductive factor can be appraised by using a combination of several tests, each of which has appreciable satu­ ration on the factor, until single tests are found with higher validities. . . .13 One difficulty of working with this factor is that any inductive task requires the use of some form of test material,which.may give it loading in another factor. Thus, tasks involving induction may necessarily have complex lp L. L. Thurstone and Thelma Gwmn Thurstone, op. cit., p. 5. ^

Ibid., p . 6.

25 factor loadings The deductive factor D has been indicated in several studies, but it has not always appeared where it might have been expected. This factor should, therefore, be regarded as tentative and subject to reinterpretation if it can be found in clearer form in repeated studies. . . . Further study of the tests in which it has been indicated may give some new interpretation for the primary factors involved, which should be tested with specially designed tests. It seems clear now that our first interpretation of this factor was erroneous. The perceptual-speed factor P has been one of the most troublesome of the primaries. Its exist­ ence has been clearly indicated, and it has appeared in all of the test batteries that have been; analyzed so far. The difficulty with this factor is that we have not been able to locate clearly its bounding hyperplane. To do this, we must find tests which have practically zero saturation on the factor and others in which the saturation is appreciable. Another study of this factor is now being made with individual laboratory tests in an effort to identify it more clearly in the configuration of the test battery. The difficulty with the perceptual-speed factor may be due to our testing methods. The group tests with time-limit procedures may introduce the perceptual-speed factor in so many of the tests that we have no base from which to measure it, with few tests in which this factor is entirely absent. We feel reasonably sure that a primary factor exists that involves perception and speed, but our inter­ pretations cannot be checked with assurance so long as the bounding plane for this primary factor is unstable. The experimental work now in progress may throw light on this factor.16 L. L. Thurstone, Experimental Study of Simple Structure,” Fsychometrika, Y (June, 1940), p. 159. ^ L. L. Thurstone and Thelma Gwinn Thurstone, op. cit., pp. 6-7. ^

Ibid.* t

P*

7 •

26 This factor apparently is not dependent on the test material used.

Thurstone, in a special study of the cate­

gory, found that simple tests had heavier loadings than abstract ones.

His results indicated that the factor may

have involved fluency of association with perceptual material, but that it probably did not depend on visual acuity. 17 The present investigation involved the study of four factors in relation to each of two tests.

A summary of the

various factors found for these tests in the studies cited is given below. Number Series Test Factors intended in present study Perceptual speed Number Induction Deduction Factors found in previous studies Study by Thurstone on college students (p. 17) Number Induction Deduction Verbal Memory Perceptual

.348 .503 .287 .296 .258 Vanishing loading

*^L. L. Thurstone, ”The Perceptual Factor,” Psychometrika, III (March, 1938), pp. 1-17.

27 Study by Thurstone on high school students (p. 19) Perceptual Number Induction Deduction Memory and verbal study.

.11 .27 .26 .47 factors not found in

Figure Analogies Test Factors intended in present study Induction Spatial Perceptual speed Deduction Factors found in previous studies Study by Thurstone on college students (p. 17) Induction Spatial Perceptual Deduction Restricted solution

.392 .197 .435 .254 .341

Study by Blakey on high school students (p. 20) Induction Perceptual or spatial Deduction Spatial II.

.415 None .517 and .244 (two factors found) Not found instudy

LITERATURE ON THE TECHNIQUE OF FACTOR ANALYSIS STUDIES

Factor analysis studies of test items. For tests of

human ability, the approach used in this investigation is a new one.

Studies have been reported of factor analyses

using the Thurstone method in which total tests have been 18 set up to try out hypotheses concerning factors. None have been located in which responses on individual items so constructed were studied. tests this has been done.

In the field of personality

Guilford and Guilford prepared a

set of eighty-nine items with the intention of bringing:out more clearly the nature of two factors they had located in a previous study.

Factor analysis was made of thirty of

these items, and the technique was regarded as a promising one. 19 Other factor analysis studies have been made of items in personality tests.

They are not summarized here because

they did not involve the construction of items to test hy­ potheses . Selection of subjects in factor analysis studies. Investigations have provided evidence that factor patterns 18 For example, see the studies by Thurstone in the previous section which dealt with perceptual speed, word fluency, and induction. 19 J. P. Guilford and Ruth B. Guilford, "Personality Factors D, R, T, and A," Journal of Abnormal and Social Psychology. XXXEY (January, 1939), pp. 21-36. Charles Mosier, "A Factor Analysis of Certain Neurotic Symptoms," Psychometrika, II (December, 1937), pp. 263-86; H. A. Reyburn and J. G. Taylor, "Factors in Intro­ version and Extraversion," British Journal of Psychology, XXXI (April, 1941), pp. 335-40.

29 may vary with differences in the age, sex, race, and specific experience of the subjects used. Factor analysis of the sub-tests of the WechslerBellevue Scale for age groups from nine to fifty-nine has been made with the Thurstone method.

Comparison of the

separate analyses made for the different age groups showed that the same factors did not appear at each level.

Of all

those found, verbal ability and performance ability were most consistent from age to age.

A general factor appeared

in the analysis for nine-year-olds and again in the analysis for the group aged fifty to fifty-nine, but not in between. The author concluded: As a result of the above findings, it could be stated that the mental traits change and undergo reorganization over a span of years. Therefore, when interpreting tests of intelligence, it is of impor­ tance to take into consideration the age of the indi­ vidual. The same test, given to a person of a certain age, may not be measuring the same abilities in him that it would measure when given to an older or younger person. Even though the whole intelligence scale may yield the same factors for a wide span of years, the separate tests that compose the scale may not necessarily be described in terms of the same factors from age to age.21 Other studies have indicated closer correlation between the factors found for children than between the 21

Benjamin Balinsky, "An Analysis of the Mental Factors of Various Age Groups from Nine to Sixty," Genetic Psychology Monographs, XXIII (February, 1941), p. 231.

30 factors found for young adults.

22

They are in agreement with

Balinsky*s finding of a general factor for his youngest' group, hut not for any other group except the one including individuals from fifty to fifty-nine years. Sex differences in factor loadings have also been found.

Woodrow made an analysis by Thurstone*s method of

fifty-two tests (social intelligence, attention, and musical ability). In this research one set of inter-correlations was figured, using the scores of both sexes.

Differences

between the two were then studied by comparing the mean scores of each group for tests found heavily loaded with the various factors.

The two sexes differed very little in the

verbal factor, but there was a significant difference in favor of the men on the spatial factor.

For the other

factors— numerical ability, attention, musical ability, and memory— differences from test to test were not consistently 23 in favor of either sex. 22 T. W. Richards, ftGenetie Emergence of Factor Specificity,” Psychometrika, VI (February, 1941}, p. 37. L. L. Thurstone and Thelma Gwinn Thurstone, op. pit., p. 26. Robert I. Blakey, op. pit., p. 198. I. P. Guilford, ”Human Abilities,” Psychological Review, XLVII (September, 1940), p. 383. Henry 1. Garrett, ”Differentiable Mental Traits,” Psychological Record, II (June, 1938), p. 289. 2^ Herbert Woodrow, ”The Common Factors in FiftyTwo Mental Tests,” Psychometrika, IV (June, 1939), pp. 99108.

31 Another approach to this problem consists of making a separate analysis for each sex.

It was used by Blackwell,

who administered to 100 boys and 100 girls (from 13-| to 15 years) tests of arithmetic reasoning, missing numbers, algebraic computation and reasoning, spatial operations, geometry, and verbal functions thought necessary in mathe­ matics.

Factor analysis by the centroid method indicated 24 different factor patterns for boys and girls. The effect of specific experience on factors has been demonstrated.

Anastasi found that factor patterns for the

same group could be changed from one test period to the next by interpolating experience which might be applied to 25 help performance in the tasks. Woodrow found marked changes in factor loadings of tests accompanying practice 26 on the tasks themselves. Results such as those cited in this section have 2^A. M. Blackwell, ”A Comparative Investigation into the Factors Involved in Mathematical Ability of Boys and Girls,” British Iournal of Educational Psychology, X (June, 1940), pp. 146-53 a n d X (November, 1940}, p . 222. 25 Anne Anastasi, ”The.Influence of Specific Experi­ ence upon Mental Organization,” Genetic Psychology Mono­ graphs , XVTII (August, 1936), pp. 336-38. 26 Herbert Woodrow, f,The Relation- between Abilities and Improvement with Practice.” Journal of Educational Psychology, XXIX (March, 1938), p. 226. Herbert Woodrow, **The Application of Factor-Analysis to Problems of Practice,” Journal of General Psychology, XXI (October, 1939), p. 459.

32 led factorists to recognize that the groups used in their 27 studies must he relatively homogeneous ones. Selection of items in factor analysis studies. No research has been done to show the influence of the material and form of individual items on their factor patterns.

Nor

total tests,investigation has been made to determine the relative importance of these two aspects in the production of factors.

One hundred eighty-six college students were

given fourteen tests.

These could be arranged into three

groups according to the material they involved (four numeri­ cal, four spatial, and five verbal) or into three groups according to the form of the items (four analogies, four generalizations, and five "construction” tests).

Thus, the

numerical group included tests of each different form, and so forth.

Analysis according to both the directed mean

tetrad method and the method of principal components indi­ cated group factors for tests similar in material and other group factors for those similar in form.

Material similarity

appeared more influential than form similarity in the production of factors. 28 27 1. P. Guilford, op. cit., p. 383. S8 George Milton Smith, "Group Factors in Mental Tests Similar in Material or in Structure,” Archives of Psychology, GLVI (1933), pp. 54-55. :

Evidence lias been gathered to show that difficulty of tests and of test items may influence their factor pat­ tern.

In a study which involved breaking up seven tests

into their easy and difficult halves and inter-correlating the scores for the halves, Hertzman discovered that the relations between the parts were different from the relations of the total scores.

For instance, easy numerical tests were

more highly correlated with easy spatial tests than they were with each other.

When both easy and difficult items were

included in a test score, the numerical-spatial correlations were slightly lower than the numerical ones. Apparently the factors which lay behind these tests involving different 29 levels of difficulty were not homogeneous ones. Schaeffer has suggested that the perceptual component of a test may be a function of its relative difficulty.

If

this were true, it would be possible to construct tests of the same material and formal nature which would be loaded in the perceptual factor in easy items and in, say, number 30 or space factors at levels of greater difficulty. .

Max Hertzman, "The Effects of the Relative Diffi­ culty of Mental Tests on Patterns of Mental Organization," Archives of Psychology. CXCYII (1936), pp. 32-33. 30Willis C. Schaeffer, "The Relation of Test Diffi­ culty and Factorial Composition Determined from Individual and Croup Forms of Primary Mental Abilities Tests," Psy­ chological Bulletin. XXXYII (July, 1940), p. 457. (Abstract.)

34 Ferguson's study of fictitious factor matrices led him to the conclusion that each degree of difficulty in a test battery or in a group of test items may appear as an additional factor.

If this were generally the case, factors

resulting from differences in difficulty would be confused with factors resulting from differences in the nature of 31 operations involved. Earlier, Guilford had made an inves­ tigation dealing with this problem.

He studied the inter­

correlations of the ten parts of the Seashore pitch dis­ crimination test for 300 university students. extracted and rotated by Thurstone1s method.

Factors were Factor I had

loadings inversely proportional to the difficulty of the item.

Factor II had significant loadings on the most diffi­

cult items, and Factor III had maximum loadings in the middle range of difficulty.

The conclusion might be drawn

that the factors had a right to distinction only on the basis of the difference in difficulty of the various parts--as suggested by Guilford and implied by Ferguson.

Guilford

has pointed out another possibility: It is more likely, however, that we need merely to recognize the universal rule that changing the difficulty of a test for a population alters its o

'3l'

George A. Ferguson, r,The Factorial Interpretation of Test Difficulty/1 Psychometrika, VI (October, 1941), pp. 323-29.

35 functional content— the manner in which testees work upon it.32 Literature on difficulties of factor analysis methods. Some criticism has been made of factor analysis on the grounds that the factors may merely reproduce the classifi­ cations of mental operations with which the experimenter starts.

Evidence against such criticism has been cited by

Thurstone.

When his first study was planned, a number of

tentative factors were listed in order to insure a wide variety of tests.

One verbal factor was postulated, but two

appeared in the analysis.

The number factor found was much

more restricted than the one expected.

Different reasoning

factors had been assumed for different types of material (verbal, numerical, and spatial).

Only two reasoning fac­

tors were found (induction and deduction), neither of which seemed dependent on the material nature of the tests.

One

visual space factor waslocated instead of separate ones for visualizing in flat space and solid space, as had been postu­ lated.

In general, theprimary factors found were not iden­

tical with the ones expected, though there was some relation between them. 33 32 1. P. Guilford, "The Difficulty of a Test and Its Factor Composition,” Psychometrika, 71 (April, 1941), p. 75.

rZK

L. L. Thurstone, ”Primary Mental Abilities,” Psy­ chometric Monographs , I (1938), p. v.

36 No rational method of finding standard errors for fac­ tor loadings exists, and for that reason some psychologists have hesitated to accept factors.

Harsh has pointed out

that a factor loading is determined from correlation co­ efficients and that, consequently, the reliability of a fac­ tor loading is related to the size of the original coeffi­ cients and the number of subjects used.

However, it is

possible that some of the errors in the original coefficients are cancelled out in the factoring process.

If this is true,

says Harsh, the reliability of factors depends also on the number of variables used.

Another point to be considered

in this connection is that the effect of errors in estimating communalities is cumulative, so that the reliability of the factor loadings depends on the number of extractions.

In

his study of this problem, Harsh administered a test to three different populations of college students.

Centroid

loadings for the first factor were the most consistent.

The

loadings for the successive factors became less consistent as the factors decreased in size.

After rotation, the

factors were more consistent— though this phenomenon may have been merely the result of their increase in size. From this and other such research, Harsh concluded that fairly similar factor loading patterns are revealed when similar

37 groups are tested with the same battery of tests. 34^ Guilford has given further evidence that rotated fac­ tors are not heavily contaminated with errors of sampling. He set up a fictitious factor matrix, and from it the corre­ lation matrix was computed.

Then, assuming a population of

200, sampling errors were introduced.

This correlation

matrix was factored by four inexperienced students.

For

the best solution, the error in the rotated loadings was no larger than the error in the correlation coefficients from 35 which they came. Another investigation of the effect of chance error has been reported by Mosier.

It involved, as well, a study

of the errors introduced by estimating communalities.

From

a fictitious factor matrix, the correlation matrix was derived,

Each coefficient was loaded with a chance error

component corresponding in size and distribution with that to be expected in actual experience with the tests.

Inde­

pendent analyses were made for the following conditions: (1) correlations with chance error, communalities known, (2) correlations without error, communalities estimated, and (3) correlations with error, communalities estimated (carried 34 Charles M. Harsh, ffConstancy and Variation in Pat­ terns of Factor Loadings,” Iournal of Educational Psychology, XXXE (May,-. 1940) , pp. 335-47. 35 I. P. Guilford, ”A Note on the Discovery of a G Factor by Means of Thurstone*s Centroid Method of Analysis,” Psychometrika, VI (June, 1941), pp. 206-07.

to three, four, and six centroid factors in separate analyses.

Inaccuracies in the determination of simple struc­

ture from estimating the communalities were very small. Larger (but still small) was the inaccuracy in the factor pattern introduced by the presence of error in the corre­ lation coefficients.

When both inaccurate coefficients and

estimated communalities were used, the standard deviation of the discrepancies of the obtained factor loadings was .064, in contrast to the mean standard error of .084 for the co­ efficients.

This accuracy in determination of factor load­

ings was, however, obtained only when there were as many (or more) factors in the centroid matrix as in the original fac­ tor matrix.3^ Not all studies have indicated small sampling errors. Smart reported that factors in tests of human ability showed considerable change when the same tests were given to dif37 ferent groups. However, he did not use rotated axes. McNemar also has reported studies indicating large sampling errors.

The first contained eight variables and was factored

^ Charles Hosier, "Influence of Chance Error on Simple Structure: An Empirical Investigation of the Effect of Chance Error and Estimated Communalities on Simple Structure in Factorial Analysis," Psychometrika, IV (March 1939), pp. 3o—44. 37 Russell C. Smart, "The Variation in Pattern of Factor Loadings," Iournal of Educational Psychology. XXVIII (January, 1937), pp. 55-64.

for subsamples of 700 cases.

The second study, based on

fictitious data for 2,500 cases, showed results for twentyfive samples, in each of the following situations:

five

variables, one factor; five variables, two factors; and six variables, three factors.

The other study wo.s based on

data for nine variables from 7,000 cases.

It involved sepa­

rate factor studies for thirty-five samples of 200 cases each.

From these investigations, McNemar drew the general

conclusion that the sampling behavior of the first centroid factor was much like that of correlation coefficients but that sampling fluctuations for further loadings were ”dis38 turbingly large.” In this connection, three points should be made:

(1) With such a small number of variables (5-9),

errors might be expected to be larger because of the greater influence of estimated communalities.

(2) McNemar did not

make successive approximations of communalities until sta­ bility for these figures was reached.

(3) The fluctuations

might have been smaller if rotated, rather than centroid, loadings had been considered. No final conclusion is possible as to the amount of sampling error which may be expected in any factor analysis. However, some standards have been set up for determining which factor weights shall be considered usable. go

In

Quinn McNemar, T,0n the Sampling Errors of Factor Loadings,” Psychometrika, VI (June, 1941), p. 141.

Thurstone*s first study, all the loadings were regarded as nearly zero if within the range- .20.

No factor was con­

sidered significant in naming factors unless it was as large as .40, and loadings of .50 or .60 were thought neces39 sary for naming factors with confidence. In Balinsky's investigation, tests were treated as significant in a fac­ tor if their loadings exceeded three times the standard error of an original correlation of zero. 40 Many articles have appeared dealing with specific points in the actual procedure of factor analysis developed by Thurstone.

For example, a number of criteria for knowing

when to stop factoring have been suggested.

These are

cited in Chapter 17, in connection with parts of the actual analysis to which they refer. L. L.

Thurstone, o£. cit., pp. 78-79.

4:0 Benjamin Balinsky, o£. pit., p. 221.

CHAPTER

III

THE SOURCES OF DATA This chapter contains a section on the construction of the test items and the administration of the tests, a description of the group of subjects used for the factor analysis, a section on the final selection of test items, and a discussion of the correlations of the data. I.

CONSTRUCTION OF TEST ITEMS

AND ADMINISTRATION OF TESTS Construction of test items. The test items were of two kinds: number series completion and figure analogies. All were in the form of multiple-choice items.

Thus, all of

the items of each test had the same material and the same form.* This eliminated the possibility of group factors associated with variation in these aspects.'*' According to Guilford’s suggestions for constructing items to emphasize each of the factors,

£

series completion items were formulated.

forty number They were divided

into eight groups of five items, each of which aimed to See page 32. £ See page 7.

42 stress a different factor.

This plan is shown in the list

"below, which contains (l) the method used in trying to em­ phasize each factor and (2) the rule and answer for each item. First group of five items a. Factors emphasized: number and perceptual speed 1. By use of single principle 2. By avoiding complicated calculation b. Rule and answer 1. Add five; c 2. Add three; c 3. Add four; a 4. Add nine; e 5. Add seven; e 12

17

22

27

32

37

42

37

45

47

52

57

16

19

22

25

28 : 31

34

35

36

37

39

.40

10

14

18

22

26

30

34

38

40

42

44

46

8

17

26

35

:s44'

53

62

1?

65

69

70

71

3

10

17

24

31

38

45

60

56

55

54

52

Second group of five items a. Factor emphasized: perceptual speed 1. By using single, simple principle 2. By using complicated numbers b. Rule and answer 6. Add two; a 7. Add three; c 8. Add five;, c 9. Add ten; a 10. Add 1001; e 375 7l|

377 74%

379 77§

52 102 152 139

149

159

381 80|

383

385

83| 86§

387 89|

202 252 302 352 169

179

189

199

1071 2072 3073 4074 5075 6076 7077

389 90|

938 90%

383 92%

333 92#

388 92|

425 452 402 408 209

219

229 '292 200

7078 8088 8708 9079 8078

43 Third group of five items a. Factor emphasized: induction 1. By use of different principle for each item 2. By use of simple calculations b. Hule and answer 11. Repeat number, add three; d 12. Multiply by two; e 13. Subtract eleven; b 14. Divide by three; d 15. Add t w o , add t h r e e , add f o u r , e t c . ; c _____

12

15

14

13

12

11

16

32

44

58

60

62

64

54

43

32

.23

21

19

11

7

■1

1 3

1 9

1 12

18

1

1 21

27

„

»■* —

_„ „

_

3

3

6

6* ~ 9~ ” 9

1 2

1

2

4

8

98

87

76

65

81

27

9

3

1

1

30

Fourth group of items a. Factor emphasized: number 1. By use of single principle 2. By using principle involving more complicated calculation b. Rule and answer 16. Add three, add four, add five, etc.; b 17. Add nine, add ten, add eleven, etc.; d 18. Add six, add seven, add eight, etc.; e 19. Add eleven, add twelve, add thirteen, etc.; d 20. Add ten, add eleven, add twelve, etc.; a 8

11

15

20

26

33

41

49

50

82

85

90

10

19

29

40

52

65

79

85

89

90

94

100

•

17

23

30

38

47

57

68

75

79

80

82

86

3

14

26

39

53

68

84 >

•96

98

100

101

112

4

14

25

37

50

64

79

95

97

99

108

115

Fifth group of five items a. Factor emphasized: induction 1. By varying principle 2. By using easy calculations

b. Rule and answer 21. Subtract five, add six; d 22. Add four, subtract three; d 23. Divide by two, add four; b 24. Reverse numbers, subtract one; b 25. Subtract eight, subtract seven, subtract six, etc.; e _ _ 13

11

• 10

9

8

7

10-

7

13

9

12

11

10

26

13

17

21

8.5

6 4.5

4

82

27

72

26

52.

19

14

10

7

10

5

11

6

12

7

4

8

5

9

6

80

40

44

22.

29

92

28

40

32

25

1

62

26

25

17

2

3

4

5

Sixth group of five items a. Factor emphasized: number 1. By using only two principles 2. By dependence of answer on accurate compu­ tation b. Rule and -answer 26. Multiply by two, subtract two; b 27. Multiply by three, subtract three; a 28. Multiply by three, add four; b 29. Multiply by two, add five; a 30. Multiply by five, add one; b -------

6

4

8

•*—.- —

r -

•"fr1if,

_____

6

12

10

"22

20

■

18

.

14

8

9

6

18

15

45

42

126

127. 128

129

130

3

7

21

25

75

79

238

237

225

221

156

4

9

18

23

46

51

102

101

75

56

53

30 31

155

156

780

781

782

3905

395

78

781

Seventh group of five items a. Factor emphasized: induction 1. By change of principle 2. By use of relatively simple calculations

45 b. Rule and answer 31. Subtract two, add two, add two; e 32. Divide by two, add three, add one; e 33. Add two, subtract three, add four, subtract five, etc.; d 34. Multiply by two, add two; a 35. Subtract nine, subtract eleven, add nine, subtract eleven; c 8

10 56

10

12

10

12

14

18

16

15

14

12

28 31

32

16

19

20

6

16

12

14

10

19

23

18

24

17

22

23

24

25

26

20 ’22 1

4

10

22

46

94

190

382

360

350

255

198

98

89

78

87

76

67

56

45

54

65

74

76

Eighth group of five items a. Factor emphasized: deduction 1. By using principles which might involve two' answers with answer given for only one 2. By using simple numbers b. Rule and answer 36. Add two, subtract three, multiply by two (not add two), add four; b 37. Add two, add two, subtract five (not divide by two),- subtract two; e 38. Add five, add five, divide by two (not subtract six), subtract two; a 39. Add one, add one, multiply by four (not add twelve), subtract twelve; c 40. Subtract one, subtract one, multiply by two (not a d d three), a d d three ; a 16 ■■18

20

3

2

n

13

15

20

24

26

27

12

13

14

15

3

5

2

4

8

10

7

•12

14

6

8

10

-5

3

.5

7

6

5

2

7

12

6

4

9

14

7

9

2

3

4

16

4

5

6

16

5

4

3

6

9

8

7

11

4

46 Similarly, forty figure analogies items were con­ structed.

These were in four groups of ten, each designed

to emphasize a different factor.

The plan for their con­

struction is given in the list below.

It contains the

methods suggested by Guilford for emphasis of each factor and the rule and answer for each item. First group of ten items a. Factor emphasized: induction 1. By minimizing emphasis on other factors (a) Through use of large figures lacking in detail (b) Through avoiding requirement of rotation or rearrangement of parts (c) Through use of simple.discriminations 2 . By frequent change of principles 3. By making the minimum essential for correct answer the apprehension of the rule b. Rule and answer 1. Subtract part and change size; 3

H

□

A

a

v

a

A □

£. Change curved to straight linesj 5 O

3. Find ratio of numbers; 2

4. Take center’part of figure and change to opposite color; 1 _______ __ _____

A A

11

□ ■ □ A ■

5. Add lines: 3

6 . Change from single to double lines and vice

versa* 5

11

1 7. Chap^p• • • • »

8 . Leave out right hand short line and extend left hand one to opposite end of figure; 1 !

M

N

W

l / 7

V

U

N

l

*

i

9. Connect dots; 1

:: □

A A V Z A

_____,1

10. Change proportions of figure by exerting pressure on upper and lower sides; 4

>

D

□

D

[=□ C S

Second group of ten items a. Factor emphasized: spatial 1. Through use of rotation 2 . Through use of inversions 3. Through rearrangement of parts b. Rule and answer 11. Rotate 180°; 5

48 12. Invert; 5

□ '□ □ a

S'2

13. Change position of parts; 2 A

14. Put front line inside figure behind back one; 1 ^_________________ _________ ©

©

^

A

^

15. Reverse direction of arrows; 2

16. Turn over sideways on axis of longest ____ ____ direction; 5

...

...

i

r

'

17. Turn over sideways; 2

A

A

p

I

..

©

o

n

’