General and Specific Mental Abilities [1 ed.] 1527533107, 9781527533103

Table of contents :
Dedication
Contents
Preface
List of Contributors
1 A Brief History of Theory and Testing of General and Specific Mental Abilities • Dennis J McFarland
2 General and Specific Intelligence Attributes in the Two-Factor Theory: A Historical Review • Alexander Beaujean
3 Cognitive Ability: Psychometric Perspectives on the Importance of General Mental Ability • Kevin R. Murphy
4 Psychometric Issues Pertaining to the Measurement of Specific Cognitive Abilities • Kara M. Styck
5 The Network Approach to General Intelligence • Han L. I. van der Maas, Alexander O. Savi, Abe Hofman, Kees-Jan Kan, & Maarten Marsman
6 Process Overlap Theory: How the Interplay between Specific and General Mental Abilities Accounts for the Positive Manifold in Intelligence • Kristof Kovacs
7 PASS Theory of Intelligence: A Frozen Dinner or a Moving Feast? • George K. Georgiou & J. P. Das
8 An Overlap between Mental Abilities and Temperament Traits • Irina Trofimova
9 Theoretical Challenges for Differentiating General and Specific Abilities • Harrison Kell
10 Within-Individual Variability of Ability and Learning Trajectories in Complex Problems • Damian P. Birney, Jens F. Beckmann & Nadin Beckmann
11 Modeling General and Specific Mental Abilities • Dennis J McFarland
12 Intelligence and Executive Function: Can we Reunite these Disparate Worlds? • Jose Maria Ruiz Sanchez de Leon, M. Angeles Quiroga, and Roberto Colom
13 Speannan's Law of Diminishing Returns and its Implications for Theories of General Intelligence and Intelligence Testing • Moritz Breit, Martin Brunner and Franzis Preckel
14 Diminished 'g': Fluid and Crystallized Intelligence and Cognitive Abilities linked to Sensory Modalities • Lazar Stankov
15 Auditory Processing Abilities and Disorders • Dennis J McFarland
16 Applications of Psychometric Methods to Neuropsychological Models of Speech and Language • Grant M. Walker
17 The Use of Specific Cognitive Abilities in tbe Workplace • Vivian Chou, Rachel Omansky, Charles Scherbaum, Kenneth Yusko, and Harold Goldstein

Citation preview

Edited by Dennis J. McFarland

General and Specific Mental Abilities

General and Specific Mental Abilities Edited by

Dennis J. McFarland

Cambridge Scholars Publishing

General and Specific Mental Abilities Edited by Dennis J. McFarland This book first published 2019 Cambridge Scholars Publishing Lady Stephenson Library, Newcastle upon Tyne, NE6 2PA, UK British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Copyright© 2019 by Dennis J. McFarland and contributors All rights for this book reserved. No part ofthis book may be reproduced,

stored in a retrieval system, or transmitted, in any fonn or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior pennission ofthe copyright owner.

ISBN (10): 1-5275-3310-7 ISBN (13): 978-1-5275-3310-3

To my wife, Loretta and my three sons, John, Christopher, and Michael.

CONTENTS

Preface

.........................................................................................................

x

List of Contributors ................................................................................... xii Chapter One A Brief History of Theory and Testing of General and Specific Mental Abilities Dennis J McFarland

.................................................................................................

1

Chapter Two 25 General and Specific Intelligence Attributes in the Two-Factor Theory: A Historical Review Alexander Beaujean ..............................................................................................

Chapter Three Cognitive Ability: Psychometric Perspectives on the Importance of General Mental Ability Kevin R. Murphy

59

Chapter Four Psychometric Issues Pertaining to the Measurement of Specific Cognitive Abilities Kara M. Styck

80

............................................................................................

..............................................................................................

Chapter Five 108 The Network Approach to General Intelligence Han L. I. van der Maas, Alexander O. Savi, Abe Hofman, Kees-Jan Kan, & Maarten Marsman ............................................................................................

Chapter Six 132 Process Overlap Theory: How the Interplay between Specific and General Mental Abilities Accounts for the Positive Manifold in Intelligence Kristof Kovacs ..............................................................................................

viii

Contents

Chapter Seven.......................................................................................... 153 PASS Theory of Intelligence: A Frozen Dinner or a Moving Feast? George K. Georgiou & J. P. Das Chapter Eight. .......................................................................................... 176 An Overlap between Mental Abilities and Temperament Traits Irina Trofimova Chapter Nine ............................................................................................ 226 Theoretical Challenges for Differentiating General and Specific Abilities Harrison Kell Chapter Ten ............................................................................................. 253 Within-Individual Variability of Ability and Learning Trajectories in Complex Problems Damian P. Birney, Jens F. Beckmann & Nadin Beckmann Chapter Eleven ........................................................................................ 284 Modeling General and Specific Mental Abilities Dennis J McFarland Chapter Twelve ....................................................................................... 3 1 1 Intelligence and Executive Function: Can we Reunite these Disparate Worlds? Jose Maria Ruiz Sanchez de Leon, M. Angeles Quiroga, and Roberto Colom Chapter Thirteen ...................................................................................... 340 Speannan's Law of Diminishing Returns and its Implications for Theories of General Intelligence and Intelligence Testing Moritz Breit, Martin Brunner and Franzis Preckel Chapter Fourteen ..................................................................................... 359 Diminished 'g': Fluid and Crystallized Intelligence and Cognitive Abilities linked to Sensory Modalities Lazar Stankov Chapter Fifteen ........................................................................................ 387 Auditory Processing Abilities and Disorders Dennis J McFarland

General and Specific Mental Abilities

ix

Chapter Sixteen 412 Applications of Psychometric Methods to Neuropsychological Models of Speech and Language Grant M. Walker .......................................................................................

Chapter Seventeen The Use of Specific Cognitive Abilities in tbe Workplace Vivian Chou, Rachel Omansky, Charles Scherbaum, Kenneth Yusko, and Harold Goldstein

...................................................................................

436

PREFACE

The history of testing mental abilities has seen two contrasting approaches, that of psychometrics and that of neuropsychology. These two traditions have different theories and methodologies but overlap considerably in the tests they use. Historically, psychometrics has emphasized the primacy of a general factor while neuropsychology has emphasized specific abilities that are dissociable. Other disciplines have recently become interested in this issue. This book includes the opinion of experts from several fields including psychometrics, neuropsychology, speech language and hearing, and applied psychology. These experts have diverse opinions on the relative importance of general and specific abilities. There is not a consensus about the nature of human mental abilities despite over one hundred years of study. Yet this issue is of importance for many practical concerns. Questions such as gender, ethnic, and age-related differences in mental abilities are relatively easy to address if these are due to a single dominate trait. Presumably such a trait can be measured with any collection of complex cognitive tests. If there are many specific mental abilities, these would be much harder to measure and associated social issues are more difficult to resolve. The relative importance of general and specific abilities also has implications for educational practices. For example, are there specific learning disabilities amendable to remediation and do certain instructional approaches benefit some students more than others? In deciding on a title for this book I at first considered using "general and specific cognitive abilities". However, I chose mental abilities instead. This is due to my feeling that the term "cognitive" is applied much too broadly. For example, perceptual and motor functions are described as cognitive. I once asked an eminent expert in cognitive psychology where cognition began and he replied "at the retina". To me, this implies that all mental events are cognitive! But if a telTIl refers to every aspect of mental life it loses meaning. Perceptual phenomena should be those processes that are sensory-modality specific. Motor processes should likewise be those that involve specific output modalities. The telTIl cognitive should be reserved for describing "thought" rather than "automatic" processes.

General and Specific Mental Abilities

xi

The authors in this text have a diversity of views on issues concerning the relative importance of general and specific mental abilities as well as other issues. For example, are abilities discovered or created? That is, do our constructs represent real entities or are they useful constructions? My 0\Vll thinking about such matters continues to evolve over time.

LIST OF CONTRIBUTORS

Alexander Beaujean Baylor University Jens F. Beckmann Durham University Nadin Beckmann School of Education Durham University Damian Birney School of Psychology Uinversity of Sydney Moritz Breit Department of Psychology University of Trier, Gemmny Martin Brunner Faculty of Human Sciences University of Potsdam, Germany Vivian Chou Baruch College and the Graduate Center City University of New York

Jose Maria Ruiz Sanchez de Leon Universidad Complutense, Spain George K. Georgiou University of Alberta, Edmonton, Canada Harold Goldstein Baruch College and the Graduate Center City University of New York Abe Hofinan University of Amsterdam Kees-Jan Kan University of Amsterdam Harrison Kell Educational Testing Service Princeton, NJ Kristof Kovacs Maarten Marsman University of Amsterdam

Roberto Colom Universidad Autonoma de Madrid, Spain

Dennis McFarland National Center for Adaptive Neurotechnologies Albany,NY

J. P. Das Uinversity of Alberta, Edmonton, Canada

Kevin R. Murphy University of Limerick Ireland

General and Specific Mental Abilities

xiii

Rachel Omansky Baruch College and the Graduate Center City University of New York

Lazar Stankov School of Psychology The University of Sydney, Australia

Franzis Preckel Department of Psychology University of Trier, Gemmny

Kara M. Styck Department of Psychology Northern Illinois University

M. Angeles Quiroga Universidad Autonoma de Madrid, Spain

Irina Trofimova McMaster University Hamilton, ON, Canada

Alexander O. Savi Uinversity of Amsterdam

Han L. J. van der Maas University of Amsterdam

Charles Scherbaum Baruch College and the Graduate Center City University of New York

Grant M. Walker University of California, Irvine Kenneth Yusko University of Maryland

CHAPTER ONE A BRIEF HISTORY OF THEORY AND TESTING OF GENERAL AND SPECIFIC MENTAL ABILITIES DENNIS McFARLAND

The present review summarizes the early development of theories and testing of mental abilities with a particular emphasis on issues related to general and specific abilities. Any such review necessarily omits a vast amount of material. Contributions of certain individuals are covered but it should be kept in mind that their work was in the context of that of their contemporaries. The reader may want to consult some of the original sources to get a flavor for the language and thinking of each period. This review is largely chronological rather than by topic so as to maintain the order of the unfolding of events. Speculation about the nature of human mental abilities has a long history. For example, Aristotle described analytical, practical, and creative varieties of intelligence (Tigner & Tigner, 2000). Testing of mental abilities was practiced by the ancient Chinese (Bowman, 1989) and Greeks (Doyle, 1974). The Chinese exams were part of an elaborate system for evaluating potential civil servants that evolved over a considerable period of time. Bowman (1989) notes that controversy over testing practices in ancient China foreshowed many current issues in mental abilities testing. These included the relative importance of memory and expert knowledge, effects of social class on test performance, and the use of geographical quotas.

The psychometric approach Most accounts of testing of human mental abilities begin at the turn of the twentieth century. This effort was part of the new discipline of experimental psychology that had recently begun with the work of Wundt

2

Chapter One

(Mackintosh, 2011). J. McKeen Cattell, a student of Wundt, devised a test of association in which individuals wrote dO\vn as many words as they could in 20 seconds in response to a single spoken word (Cattell & Bryant, 1 889). This procedure is reminiscent of modern tests of word fluency. Cattell and Galton (1 890) proposed assessing mental abilities with a series of tests measuring simple perceptual abilities, reaction time, and a letter span task. They suggested that collection of data with these tasks on a large number of individuals would provide insight about the constancy of mental processes and their interdependence. Cattell and Farrand (1 896) subsequently reported observations on such tests from students at Columbia University. Wissler (1901) examined Cattell and Farrand's tests using Pearson's recently developed method of correlation. Wissler (1901, page 1) asserted that "If a test is general, then its results should correlate with many other special tests, and, in tum, if there is an integral relation between general and specific ability, the results of the latter should correlate with the fOlmer." He reported that the correlation between various scores was not "significant", but the criterion he used for what he called ttsignificantt! was not based on probability but rather on the colloquial usage of this term. My own calculation of the probability for his findings of a correlation of 0.21 between speed of letter cancellation and speed of naming colors with an n of 159 yields a p < 0.01. Likewise, Wissler described a correlation between auditory and visual digit span of 0.39 as significant but small. Correlations between academic standing and mental tests were generally small, the largest being between logical memory at r� 0.19. In contrast, correlations between relative standing in different classes were much higher, ranging between 0.60 and 0.75. Wissler concluded that the mental tests used by Cattell & Farrand (1896) had little interdependence and that they were not of practical value. This investigation was influential in suggesting that tests of perception and speeded responding were not useful measures of intellect. However Wissler's evaluation of the magnitude of correlations was much different than what is practiced today. Spearman (1904) advocated that a "correlational psychology" be applied to mental tests. Spearman's correlation differed from that of Pearson in that it involved rank ordering the data. He applied this method to "selected laboratory psychics" in young students that included tests of sensory discrimination in three senses and three estimates of intelligence by their teachers. Spearman (1903) found that all nine correlations between these measures were positive and concluded that they all reflected a common element. He offered a theory of "intellective unity" and suggested a hierarchy of the specific intelligences. This hierarchy was

History of Theory and Testing Mental Abilities

3

ordered according to the "saturation" of each item with general intelligence. Thus, each measure was viewed as composed of general intelligence and a specific factor which was uncorrelated with other specific factors. Spearman (1903, page 284) concluded: all branches of intellectual activity have in common one fundamental function (or group of fUnctions), whereas the remaining or specific elements of the activity seem in every case to be wholly different from that in all others.

Jensen (2000) states that Spearman's interests were predominately in theory and the nature of cognition. Spearman (1914) later offered an improved method for verifying his two factor theory that was applied to a larger series of mental tests that were collected by Thorndike. Spearman's method involved the computation of tetrad differences between the ratios of pairs of correlations among four mental tests. If correlations are due to a single common factor the difference between these ratios should be within sampling error. Speammn (1914, page 105) stated: the two factors in success are quite distinct; firstly, there is the state of the particular group of neurons, their development and organization; and secondly, there is the whole cortex. The former may be called the 'specific' factor, as it is specific to that particular performance. The latter constitutes the 'general' factor, since it is required for all performances.

Thus, Speammn offered a primitive hypothesis of how his two factors depended on the physiology of the brain. Spearman also devised several approaches for extracting the loadings of test scores on a single factor (Vincent, 1953). Thus, Spearman could be considered the founder of single-factor analysis. Thompson (1916) countered Spearman's argument with a dice­ throwing experiment that showed that the hierarchy of intelligences could be produced without a general factor (g). Subsequently, Thompson (1919) conducted additional simulations using playing cards. These simulations were done by assigning values dra\Vll from dice or cards to artificial variables that were used to generate simulated test scores. Thompson's experiments are probably the first Monte Carlo simulations of mental test performance. Thompson concluded that Speammn's evidence in favor of a general factor was by no means crucial and could be accounted for by multiple independent factors. This work may be seen as the first demonstration that Spearman's g may be a statistical artifact.

4

Chapter One

Thompson and Speannan continued to debate the issue of a general factor, and it is interesting that their positions evolved. In a series of meetings described by Deary et. al. (200S), Spearman appears to have accepted that there might be group level factors (i.e., factors common to a subset of tests) in addition to general and specific factors. Likewise, Thompson is described as leaning toward Speannan's g and is quoted as saying that "Surely the real defense of g is simply that it has proved useful" (Deary et. aI., page 129). Although historical accounts may present the positions of researchers as static, the views of these two individuals, like those of many other scientists, changed over time. Binet & Simon (1916) sought to develop a means of determining whether a young child was suitable for llOlmal instruction or needed to be sent to a ttspecial class!!. Their intent was to describe the current condition of the child and not to speculate about etiology or prognosis. Their approach differed from the work of earlier researchers such as Cattell & Galton ( l S90) in that of Binet and Simon used more complex tasks involving judgment and reasoning rather than simple laboratory tasks. Binet & Simon (1916, page 41) stated that "The scale that we shall describe is not a theoretical work; it is the result of long investigations . . . ", and "all the tests which we proposed have been repeatedly tried, and have been retained from among many, which after trial have been discarded." Thus, Binet & Simon's scale was the result of an empirical investigation. Boake (2002) states that validity of their scale was based on the increase of scores with age and by the scale's ability to identify mentally impaired children. Binet & Simon (1916) did have thoughts about the nature of the characteristics that distinguished "mental defectives" from other children. They thought that judgment was primary, along with comprehending and reasoning well. On the other hand, they thought that memory was not important. As an example of the relative unimportance of memory, they described the case of a "backward" girl with an exceptional memory, what today might be called a savant. According to Boake (2002), Binet emphasized that a particular test in isolation is of little value and that the important infOlmation was in the subject's average perfOlmance over various tests. On the other hand, in describing what their tests measured, Binet & Simon (1916, page 52) stated that they "find it difficult to define which mental functions are being exercised because they are very numerous". The Binet & Simon scale was subsequently translated and revised by Terman (1916) who added new items and standardized the test on a large group of children. This scale carne to be known as the Stanford-Binet. Tennan included items that discriminated between children of different

History of Theory and Testing Mental Abilities

5

ages and which correlated with the scale as a whole. While the practice of including items that correlated with the total score would seem to imply that the Stanford-Binet scale measured a single ability, Terman stated that a single test alone was not accurate since intelligence has many aspects. Terman (1916) suggested a number of applications for intelligence testing beyond identifying mental defectives. Terman (1918, page 165) asserted that his method ttprobes beneath the veneer of education and gives an index of raw brain powertt. Terman (1924) also spent considerable time emphasizing that the use of mental tests was a methodology that is equally appropriate for experimental psychology as is the use of controlled experiments. He noted that many of his contemporary experimental psychologists were in agreement with this position. Yerkes (1917) criticized both the Binet scale and the Stamford-Binet scale based on differences from his "point scale". Yerkes suggested that tests should be selected to evaluate basic psychological functions rather than being selected to discriminate between age groups. Yerkes's tests of individual psychological functions were designed to be applied in isolation or as a group of tests. In addition, Yerkes asserted that a given test item should be applicable to all age groups and produce a continuous score. However Yerkes point scale ultimately did not enjoy the success of Binet and TelTIlan's scales. Terman (1918) was also involved with the construction of tests used by the US army during the First World War. These scales were designed by a group of prominent experimental psychologists of that period led by Yerkes (Kevles, 1968). The tests developed by this group included the Alpha, which made use of verbal material and was designed for men who could read and write English, and the Beta, which was designed for those who could not. Both the army Alpha and Beta were group tests that involved multiple choice questions. According to Spring (1972), the criterion used to validate these tests was the ability to be a good soldier. This ability was assessed by correlating test scores with officer's ratings of their men in telTIlS of practical soldier value. As such, Spring suggested that this form of validation paralleled Binet's attempt to determining the suitability of children for normal school instruction. Spring suggested that both the army testing project and that of Binet and Simon evaluated the ability of individuals to function in a highly organized institutional structure. Kevles (1968) states that intelligence testing was not accorded much respect prior to the war but gained a considerable following as a result of this project. Thorndike (1918) thought that psychologists tend to reduce the infinitude of tendencies to think and feel and act in certain ways in

6

Chapter One

response to varied situations to a few tendencies called traits, abilities, and interests. Thorndike (1918, page 149) stated that "if the scale by which individuals are measured is very coarsely divided, their differences may be hidden". He also stated (Thorndike, 1918, page 158) that there might be "one single type or as many types as there are individuals" depending on whether one wanted to emphasize commonalities or differences between individuals. Thorndike (1921) viewed intellect in general as the power of good responses. He thought that it would not be wise to spend too much time trying to separate intelligence from emotional and vocational abilities. Thorndike also thought that ability varied according to the particular task. He viewed the value of test scores not in terms of their capacity to identify some general power residing in the individual but in terms of their ability to predict future performance. Thorndike believed that this prophesy was less accurate the more the test content differed from the skills that are to be predicted. Thorndike noted that tests current to his time "favored words, numbers, space-fOllis, and pictures, neglecting three-dimensional objects and situations involving other human beings." He suggested that this might be due in part to convenience. Thorndike developed a number of tests of specific abilities and achievement for domains such as reading (Thorndike,1914a) and mathematics (Thorndike, 1914b). Thurstone (1934) noted that Spearman's method of tetrad differences often failed to show that there was only a single common factor that accounted for the correlations between tests in a battery. According to Thurstone (1934), the proponent of g would consider the tests as inadequate and discard them while the opponent of g would consider Spearman's theory as inadequate. Thurstone believed that neither conclusion was correct and that more than one general factor was necessary to account for observed correlations. Thurstone (1934) stated that "The multi-dimensionality of mind must be recognized before we can make progress toward the isolation and description of separate abilities." Thurstone devised the centroid method to extract these multiple factors (Vincent, 1953). Thurstone (1935) elaborated on methods for extracting multiple factors, noting that this problem has at least two parts. The first of these concerns the minimum number of factors that will account for the observe intercorrelations of test scores. The second concerns the minimum number of factors for each trait that will account for the correlations between test scores (i.e., each trait can be described by of the smallest possible number of factors). This second issue involves the rotation of factors. Thurstone (1935) believed that the solution to the problem of an infinite number of possible factor rotations was to be found in simple

History of Theory and Testing Mental Abilities

7

structure. Simple structure results when there are many factor loadings that are vanishingly small (i.e., a sparse matrix of factor loadings). Thurstone also developed methods for higher-order factor analysis. At this time multiple factor analysis was very labor intensive and Thurstone & Thurstone (1941) acknowledged the efforts of several assistants in perfOlming the manual calculations. Using these methods, Thurstone & Thurstone (1941) characterized six primary mental abilities they describe as clearly indicated: verbal comprehension; word fluency; space; number; memorizing; and inductive reasoning. In addition, they indicated that two more mental abilities were not as clearly defined: deductive reasoning; and perceptual speed. However Thurstone & Thurstone (1941, page 8) stated that: No one knows how many primary mental abilities there may be. We know about one memory factor now, but several new memory factors may be fOlUld.

In addition, they stated that "It should not be assumed that the primary mental abilities are elemental or indivisible." Thurstone (1940) thought that some of his primary mental abilities might be associated with physiology while others might result from experience and education. Wechsler (1939) developed a battery of tests, the revised versions of which are currently the most popular scales of human mental abilities (Rabin et aI., 2016). The Army group examinations were a major source of sub-tests and items used in the Wechsler-Bellevue scale. According to Tulsky e1. aI. (2003a), Wechsler's scale was unusual for the time in that he combined both verbal and non-verbal (perfOlmance) sub-scales. However, Frank (1983) describes Wechsler's initial scale as almost identical to the scales of several previous authors. Frank's table 1 shows five other scales with both verbal and performance tests. Wechsler (1958) recommended considering possible discrepancies between these verbal and perfOlmance tests. Wechsler (1958) describes the final selection of these tests as primarily based on three considerations: (1) that previous studies showed that the tests correlated with composite measures of intelligence; (2) that the tests as a group encompassed a sufficiently diverse content and (3) that the nature of the subjects' failures on the tests had diagnostic implications. Wechsler (1958, page 7) described his views of intelligence as follows: Intelligence, operationally defined, is the aggregate or global capacity of the individual to act purposefully, to think rationally and to deal effectively with his environment. It is aggregate or global because it is composed of elements or abilities which, though not entirely independent, are qualitatively

8

Chapter One differentiable. By measurement of these abilities, we ultimately evaluate intelligence.

Raymond Cattell (1943) described the literature of his time as containing a host of divergent definitions of intelligence. He proposed a scheme where adult mental capacity could be viewed as consisting of fluid and crystalized abilities. Cattell's fluid uitelligence was a general ability to discriminate and perceive relations. Crystalized intelligence consisted of the habits formed previously through the use of fluid intelligence. However, once fOlmed, crystalized intelligence no longer required the use of its fluid counterpart. This theory was subsequently refined to uiclude nine factors accounting for both mental abilities and general personality dimensions (Hom & Cattell, 1966). Of these nine factors, those related to intellectual perfOlmance were fluid intelligence, crystalized intelligence, visualization, speediness, use of concept labels, and carefulness The nine factors in Hom & Cattell's solution were correlated, which these authors interpreted as being due to interactions during the individual's development. Cronbach & Meehl (1955) summarized the conclusions of a committee of the American Psychological Association on the topic of test validation. As discussed previously, test developers had used a variety of procedures to select test items and rationalize their use. Various criteria included: predictive validity, the ability of a test or battery to predict (i.e., correlate with) some future performance of interest; concurrent validity, the correlation of a test with similar measures taken at the same time; content validity, the extent to which test items are a sample of the trait or abilitiy the investigator is interested in, as determined deductively; and construct validity, which includes many fonns of the evidence that supports interpretation of a test, including theory. An important feature of construct validity was an emphasis on theory in the validation process. Cronbach and Meehl (1955) described construct validation as a continuing process, where tests are not validated but rather supported by evidence of their validity. Cronbach (1957) also suggested that the process of construct validation should include experimental research. He noted that investigators using experimental methods had become quite separate from those using correlational methods. This situation is quite distinct from the time when mental tests were initially being devised by experimentalists such as Terman. Cronbach (1957) felt that the two disciplines could mutually benefit from interacting. However, it is not apparent that this interaction occurred at that time. Vincent (1953) asserted that factor analysis was not a statistical technique, owing to guesswork involved in its implementation. This was a common belief among statisticians at the time. Factor analysis developed

History of Theory and Testing Mental Abilities

9

within the field of psychology, quite apart from statistics. The guesswork involved in factor analysis included the necessity of estimating commonalities and the fact that Thurstone's centroid method did not produce unique solutions. These problems were overcome by Lawley & Maxwell (1962) who showed that the factor problem could be solved by use of maximum likelihood. This approach involved finding the matrix of factor loadings that minimized the difference between the common factor model and the observed correlations. Lawley & Maxwell's method also provided a statistical method of determining the number of factors to retain based on the difference between the predicted and observed correlation matrices. The structural equation modeling approach of Joreskog & van Thillo (1972) can be regarded as an extension of Lawley & Maxwell's method that allows for evaluation of many alternative models that are specified by the analysis!. These methods became practical for widespread use with the advent of modem computing hardware and software. One of the more prolific collections of mental abilities was proposed by Guilford (1956) who suggested that there were at least 40. Guilford's approach involved first postulating the existence of some unitary ability and then selecting or designing tests to measure that ability. Guilford believed that these separate abilities could be grouped into a number of classes. Major divisions were between cognitive factors (i.e., discovery), production factors (i.e., convergent and divergent thinking), evaluation factors, and memory. Within each of these factors Guilford conceptualized matrices fOlmed by the intersection of content and processes. Guilford noted that empty cells in his matrices suggested the existence of additional mental abilities. Guilford (1956, page 285) stated that "parsimony has led us in the past to the extreme of one intellectual dimension, which everyone should now regard as going too far in that direction". Kirk & McCarthy (1961) devised the Illinois Test of Psycholinginstic Abilities (ITPA) to provide a means of assessing children with specific disabilities in areas such as language, perception, or behavior. Children with learning disabilities were conceptualized as having wide discrepancies among abilities that result in a failure in academic subjects. Kirk (1968) felt that diagnostic tools such as the Stanford-Binet were inadequate for this purpose. The ITPA consisted of 9 subtests evaluating three processes (decoding, association, and encoding) in each of two channels (auditory­ vocal and visual-motor) at two levels of organization (symbolic and non­ symbolic). As such the ITPA represented an initial attempt to assess specific abilities in children with specific learning problems.

10

Chapter One

Elwood (1969) discussed the possibility of automating psychological testing. He noted that in addition to saving time, automation could potentially standardized presentation of test items, improve accuracy in recording and scoring responses, allow measurement of new dimensions of the response process, and increase the reliability and validity of test results. He described an automated system that could administer most of the WAIS sub-scales. This automated system produced results similar to manual testing. Elwood's apparatus was based on technology that was primitive by today's standards. Yet despite the immense progress in technology, automated testing of mental abilities has yet to become the norm. Daneman and Carpenter (1980) used individual differences in reading comprehension to validate measures of working memory. This work led to a renewal of interest in individual differences and correlational approaches within the field of experimental psychology (e.g., Engle e1. aI., 1999). Much of this work was concerned with the relationship between fluid intelligence and working memory (e.g., Fry & Hale, 1996), a construct originating from cognitive psychology. This trend represents a return to the study of individual differences by experimentalists as practiced by Terman and suggested by Cronbach. Carroll (1993) re-factored correlation matrices from a very large series of prior studies. Carroll's method involved oblique rotation of initial correlation matrices followed by higher-order factoring of the first-order factor correlation matrices, and on occasion, a third step in this process. From these results Carroll (1993) developed a three-stratum theory consisting of narrow, broad, and general factors. Carroll interpreted the factors at each level, a practice he regarded as theory construction. The highest, or third stratum level, was interpreted as general intelligence (i.e., g). Second stratum abilities were interpreted as fluid intelligence, crystalized intelligence, memory, visual perception, auditory perception, retrieval ability and cognitive speediness. Carroll also noted that additional second­ order factors might be identified. According to Carroll (1993, page 68): It is clear that all the leading figures in psychometrics Binet, Spearman, TIun-stone, and Guilford (to name but a few) have had an abiding concern for the nature of intelligence; all of them have realized, too, that to construct a theory of intelligence is to construct a theory of cognition.

Carroll's also described these early theorists as having common sense explanation oftraits rather than well-developed cognitive theories. Carroll's views on mental abilities are illuminated by several exchanges he had with his contemporaries. One of these was with Kranzer

History of Theory and Testing Mental Abilities

11

& Jensen (1991) over whether g was unitary. Kranzer & Jensen (1991)

showed that four principal components derived from a battery of "elementary cognitive tasks" independently contributed to the prediction of an estimate of g. They reasoned that since these components were orthogonal, g must consist of at least four distinct components. Kranzer & Jensen concluded that their results were consistent with the hypothesis of Detterman (1982) that intelligence is the result of a set of orthogonal variables. Carroll (1991) argued that Kranzer & Jensen (1991) only had an estimate of g. He produced hypothetical example factor matrices tbat showed how tbeir results could be produced given a unitary g and argued that it was more parsimonious to assume this. Carroll (1994) also had an exchange with Humphries (1994) who presented what he called a behaviorist's view of intelligence, reminiscent of that of Thorndike. Humphries asserted tbat intelligence is tbe acquired repertoire of all intellectual skills and knowledge available to an individual at any particular point in time. Humphries also stated that intelligence was not necessarily anything more tban a matbematical dimension. Carroll (1994) countered that cognitive abilities such as intelligence are real entities in the individual rather than an acquired repertoire of skills.

The Neuropsychological Approach Finger (1994) suggests that tbe field of neuropsychology can trace its roots to Democritus (ca. 460-370 B.C.), who held that rational fimctions were controlled by the head, and Galen (A.D. 130-200), who associated intellect with the brain. An ancient Egyptian papyrus described the association of head injury with a loss of speech (Sondhaus & Finger, 1988). However, Broca's description of a patient with loss of speech due to a lesion in the left inferior frontal cortex was a particularly critical event in thinking about localization of function that foreshadowed the beginnings of modern neuropsychology (Finger, 1994). Halstead (1947) distinguished between several forms of intelligence, including psychometric intelligence and biological intelligence. Halstead's distinction between these different fonns of intelligence was predominantly in tenns of how they were validated. According to Halstead, psychometric intelligence, as measured by test batteries such as the Stanford-Binet, is validated in terms of sociological criteria such as educational attainment. In contrast, Halstead's biological intelligence is validated in tenns of sensitivity to brain patbology. Halstead stated tbat the relationship between these two forms of intelligence was an empirical matter. Halstead (1947) developed a battery from 27 tests that were given to an assortment of

12

Chapter One

individuals both with and without brain pathology. Halstead notes that Thurstone conducted a factor analysis of these data that produced a four­ factor solution. Halstead also derived an impairment index that was a composite of these individual factors. Halstead concluded that psychometric intelligence was umelated to frontal lobe functioning based on his review of studies examining the effects of frontal lobotomies on the Stanford-Binet. In contrast, he reported large effects of frontal lobectomy on his impailTIlent index. From results such as these, Halsted concluded that the frontal lobes were particularly important for biological intelligence. He also concluded that psychometric intelligence, as measured by the Stanford-Binet, was umelated to biological intelligence. He stated that "one wonders in what sense the term "intelligence" can properly be applied to this test" (Halstead, 1947, page 141). Halstead's student, Ralph Reitan, continued this work (Reitan, 1956). In addition, Reitan (1955) added new tests to Halstead's battery. In contrast to Halstead's approach, Reitan (1964) did not rely on factor analysis and used profiles of individual test scores to characterize the effects of different forms of brain damage. The goal of this type of analysis was to provide a basis for diagnosis of the characteristics and location of lesions in patients with brain damage. This work provided a basis for the development of the Halstead-Reitan battery that has been used extensively for neuropsychological assessment. However, the ability of these tests to diagnose the nature and location of brain damage proved elusive. For example, in reviewing the effects of brain damage on several of these tests, Reitan and Wolfsen (1994) concluded that the evidence for the popular belief that they were specifically sensitive to frontal lobe damage was tenuous. In contrast, neuroimaging techniques were developed that proved very successful in localizing and characterizing brain damage (Doyle et. aI., 1981). With these considerations in mind, Leonberger (1989) suggested that the goals of neuropsychological assessment should change to characterizing the nature of a patient's cognitive deficit rather than the nature of the patient's brain pathology. Teuber (1955) described double dissociation as a method to establish localization of function. This method involves showing that two brain regions are functionally dissociated by two behavioral tests, each test being affected by a lesion in one area and not the other. This method was subsequently used in a large number of neuropsychological studies to characterize the nature of deficits in individuals with brain damage. Reviewing this literature, Shallice (1988) suggests that double dissociations indicate that there are functional specializations for sub-processes in perception, memory, speech, and output systems. Double dissociation is a

History of Theory and Testing Mental Abilities

13

method that is not without its critics however. For example, Plaut (1995) produced a double dissociation by "lesioning" an artificial connectionist network that lacked a modular structure. Plaut asserted that these findings called into question the theoretical implications of reliance on single-case studies. Scoville and Milner (1957) described cases of profound loss of memory following removal of the hippocampus for tbe treatment of intractable epilepsy. This memory loss was described both in terms of causal descriptions of the patient's behavior and also in terms of marked differences between the Wechsler intelligence and memory scales. Subsequent work by Toal (1957) and others (Erickson & Scott, 1977) questioned the adequacy of the Wechsler memory scale for characterizing organic memory problems. Toal described the Weschler memory scale as being based on an ambiguous "common-sense" definition of memory that did not provide a means of detelTIlining what was being measured. Scoville and Milner's report led to a considerable amount of research aimed at characterizing the nature of amnesic deficits and identifying dissociable memory disorders (Butters e1. al., 1995). This research resulted in development of specialized memory test batteries (e.g., Delis e1. aI., 1991) as well as revisions of the Wechsler memory scale (Kent, 2017). Das e1. aL (1975) advocated a process-based approach to understanding mental abilities. They asserted that the brain was the source of cognitive functioning. They based their conceptual model on the information processing approach of Luria and devised a scale to measure these processes (Das & Naglieri, 1997). Interestingly, Golden e1. aL (1978) developed an alternative scale based on their understanding of the work of Luria. The scale of Golden e1. aL (1978) is an attempt to standardize administration and scoring of Luria's tests that cover a large assortment of functions. The scale developed by Das and Naglieri (1997) is more focused on Luria's theoretical account of simultaneous and successive processing. Neither of these test batteries appears to be widely used by neuropsychologists at present (Rabin et,. aI., 2016). Tupper (1999) describes these and a number of other approaches to test development as "neo-Lurian methods". Chase e1. aL (1984) examined the relationship between WArS scores and cerebral glucose metabolism in patients with Alzheimer's disease and controls. They report that verbal subtest perfolTIlance was associated with activity in left parasylvian areas and perfolTIlance subtest performance was associated with right posterior parietal areas. This study pioneered the use of modern neuroimaging methods for the localization of brain areas associated with individual differences in mental abilities. In addition,

14

Chapter One

relationships between perfOlmance on cognitive tasks and neuroimaging has resulted in the generation of novel hypotheses, such as the concept that superior performance is associated with greater mental efficiency (Deary & Caryl, 1997). Livingstone & Hubel (1987a) presented evidence for separate processing of fOlTIl, color, movement, and depth in the primate visual system. They showed that human psychophysical data also supported the concept of separate channels of processing for these same visual features (Livingstone & Hubel, 1987b). Their research did not focus on individual differences. However, this theory of separate processing streams proved to be extremely influential in orienting subsequent research in neuroscience towards considering how infOlmation is processed in specialized networks. This led to a move away from the strict localizationist thinking that had been the focus of investigators such as Reitan. The concept of "selectively distributed processing" may be seen as a resolution in the debate between localization of function and equipotentiality (Mesulam, 1998). Kaplan (1988) outlined the Boston Process Approach which was based on the premise that different individuals could arrive at a solution to a problem in different ways. From this perspective it follows that examination of the source of errors is of considerable importance in describing test behavior. Kaplan suggested that most standardized tests were multifactorial. She described cases where errors on tests such as the WAIS-R block design and word-finding difficulties on the Boston Naming Test could result from different strategies used by different individuals. Kaplan advocated examining qualitative differences in behavior that are not apparent when only considering outcomes. Libon et al. (2013) state that Kaplan's approach to identifying the processes used by individual's to solve problems was influenced by her background in Gestalt psychology. They also indicate that Kaplan thought that there was no fundamental difference between assessment and experimentation in neuropsychology. Both assessment and experimentation involve hypothesis testing. Test batteries developed in the tradition of the Boston Process Approach (the Delis-Kaplan Executive Function System and the California Verbal Learning Test) are currently commonly used by clinical neuropsychologists (Rabin e1. aI., 2016).

Issues and Trends Interpretations of the history of abilities testing have varied over time. Tyler (1965) noted that early mental testers had widely divergent views as to how to define intelligence. This included notions of judgment, abstract

History of Theory and Testing Mental Abilities

15

thinking, ability to leam, etc. Tyler (1965, page 62) stated tbat "The tbing that saved psychology from bogging dO\vn in a mire of semantic confusion was the predominately practical orientation of mental testers." This view reflected the practical orientation of that time that emphasized prediction of diverse domains of performance. Carroll & Maxwell (1979, page 608) stated: A persistent tension has existed between those who believe that hmnan cognitive capacities can be well smnmarized in a single global concept of intelligence and those who prefer to emphasize the multidimensional character of the concept. The bulk of recent research is predicated on a multifactorial view..

More recently, several reviews of the history of research on human mental abilities have emphasized the progressive nature of theory development in this area (e.g., Flanagan et. aI., 2014; Schneider & Flanagan, 2015). This view reflects the current interest in the use of individual differences for theory development. The Cattell-Horn-Carroll (CHC) model is currently popular. Schneider & Flanagan (2015, page 323) state: CHC theory is not so much a new theory but an elaboration of very robust findings that were first discovered by Speannan, Thurstone, and many other early researchers.

Geisinger (2000) has described psychological testing as continuing to evolve rapidly. Likewise, Riley et. aL (2017, page 38) state: Each new version of the Wechsler Adult Intelligence Scale (WAIS) or Wechsler Intelligence Scale for Children (WISC) involves changes based on findings that have emerged in the basic and applied literatures on the nature of cognitive functions.

In contrast, in his discussion of the history of tbe WAIS, Boake (2002) describes this most common test of intelligence as remaining almost unchanged through various revisions. He notes that many of the WAIS sub-scales had been around in one fOlTIl or another for quite some time before the Wechsler-Bellevue was assembled. According to Boake (2002), all of the Wechsler-Bellevue subtests except for Block Design were derived from tbe Army tests. There have been relatively few changes in content since that time. For example, the only change in content for the 3 rd revision was tbe addition of Symbol Search as a supplemental sub-test (Tulsky et. aL,2003b). Most oftbe changes were related to updating norms and providing new ways to compute index scores. Boake (2002) states:

16

Chapter One From a historical perspective, the Wechsler- Bellevue Intelligence Scale is a battery of intelligence tests developed between the 1 880s and World War 1. In their origins, the Wechsler subtests represent the major pre-World War I approaches to cognitive assessment.

Thus, major changes to the WArS scales have come in interpretation rather than content. Public controversy over the testing of mental abilities has "waxed and waned" ever since the initial development of these tests (Cronbach, 1975). These controversies include issues of racial, class, and immigrant differences in intelligence test scores. Cronbach (197S, page 1 1) attributes these to: journalists mining scholarly reports for controversial copy, distorting the original to make it more exciting, pointing up disagreements, and sometimes reporting only the iconoclastic side.

However, it is also clear that psychologists have made controversial public pronouncements of their 0\Vll accord, such as the book by Hemstein & Murray (1994) that discussed, among other things, racial and class differences in intelligence. According to Walsh e1. al. (2014, page 24S) !!Galton, Jackson, Maudsley, and Spearman each invoked science to justify maintaining the societal status quo.!! According to Jensen (2000), Spearman thought that the measurement of g could be used to determine whether individuals were qualified to vote or have children. \¥bile Thurstone believed in the diversity of abilities, he also stated that: If the facts support the genetic interpretation, then the accusation of being lUldemocratic must not be hurled at the biologists. If anyone is undemocratic on this issue it must be Mother Nature" (Thurstone, 1 946, page 1 1 1).

Neisser e1. al. (1996) point out that uncertainty about the nature, origins, and measurement of intelligence make overall generalizations about these issues inappropriate. Neisser e1. al. (1996, page 97) conclude: In a field where so many issues are lUlresolved and so many questions lUlanswered, the confident tone that has characterized most of the debate on these topics is clearly out of place.

According to Cubelli (200S, page 273), the history of neuropsychology: could be described either as a sequence of completely new approaches that substituted the preceding ones, or as a pendulum like movement, in which

History of Theory and Testing Mental Abilities

17

the neurological and the psychological descriptions of the cognitive processes alternated as the main goal of clinical investigation.

In either case, Cubelli stated that historical descriptions are based on simplified abstractions that often "condense and homogenize" opinions. He also felt that "standard quotations without a direct reading should be avoided". This is good advice since reading the original works often gives a different impression than that conveyed by the historian. Cubelli (2005) described how Geschwind discovered that his contemporaries had mischaracterized their predecessors. Geschwind's careful reading of older articles showed that the contemporary theories were not so novel. Unfortunately there seems to be a decline in interest in the history of psychology (Kent, 2017). Many issues that were discussed in the early history of research on intelligence have yet to be completely resolved. One difference between the psychometric tradition and that of neuropsychology concerns how tests might be validated. The psychometric tradition currently places great emphasis on validation by factor analysis. In contrast, neuropsychology has traditionally relied on sensitivity to brain functioning for validation. Theories based on factor analysis have acknowledged the existence of specific abilities. For example, Carroll (1983) used an elaborate hierarchical system in his factor analytic research. Nonetheless, he was a proponent of a singular general ability. Likewise, Halstead (1947) made use of a general impailTIlent index. However the bulk of research in neuropsychology has emphasized dissociations (e.g., Shallice, 1988). Both traditions are currently concerned with theory development. Psychometric methods are used to develop cognitive theory and neuropsychological methods are used to develop brain theory.

References Binet, A., & Simon, T. (1916). New methods for the diagnosis of tlie intellectual level of subnormals. (L'Annee Psych., 1905, pp. 191-244). In A. Binet, T. Simon & E. S. Kite (Trans.), The development of intelligence in children (The Binet-Simon Scale) (pp. 37-90). Baltimore, MD, US: Williams & Wilkins Co. Boake, C. (2002) From tlie Binet-Simon to tlie Wechsler-Bellevue: Tracing the history of intelligence testing. Journal of Clinical and Experimental Neuropsychology, 24, 383-405. Bowman, M.L. (1989) Testing individual differences in ancient China. American Psychologist, 44: 576-578.

18

Chapter One

Butters, N., Delis, D.C. & Lucas, I.A. (1995) Clinical assessment of memory disorders in anmesia and dementia. Annual Review of Psychology, 46, 493-523. Carroll, lB. (1991) No demonstration that g is not unitary, but there's more to the story: Comment on Kranzler and Jensen. Intelligence, 15, 423-436. Carroll, I.B. (1993) Human cognitive abilities: a survey offactor analytic studies. New York, Cabmbridge University Press. Carroll, I.B. (1994) An alternative, Thurstonian view of intelligence. PsychologicalInquiry, 5, 195-197. Carroll, I.B. & Maxwell, lB. (1979) Individual differences in cognitive abilities. Annual Review ofPsychology, 30, 603-640. Cattell, I.M. & Bryant, S. (1 889) Mental association investigated by experiment. Mind, 14, 230-250. Cattell, lM. & Farrad, L. (1896) Physical and mental measurements of the students of Columbia university. Psychological Review, 3, 61 8-648. Cattell, I. M. & Galton, F. (1 890) Mental tests and measurements. Mind, 15, 373-381. Cattell, R.B. (1943) The measurement of adult intelligence. Psychological Bulletin, 40, 153-193. Chase, T.N., Fedio, P. Foster, N.L. Brooks, R., Di Chiro, G. & Mansi, L. (1984) Wechsler Adult Intelligence Scale performance. Cortical localization by fluorodeoxyglucose F I 8-positron emISSIOn tomography. Archives ofNeurology, 41, 1244-1247. Cronbach, L.I. (1957) The two disciplines of scientific psychology. American Psychologist, 12, 671-684. Cronbach, L.l (1975) Five decades of public controversy over mental testing. American Psychologist, 30, 1-14. Cronbach, L.I. & Meehl, P.E. (1955) Construct validity in psychological tests. Psychological Bulletin, 52, 281-302. Cubelli, R. (2005) The history of neuropsychology according to Norman Geschwind: continuity and discontinuity in the development of science. Cortex, 41, 271-274. Daneman, M. & Carpenter, P.A. (1980) Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, 450-466. Das, lP., Kirby, I. & Iarman, R.F. (1975) Simultaneous and successive synthesis" an alternative model for cognitive abilities. Psychological Bulletin, 52, 87-103. Das, I. P., & Naglieri, I A. (1997). Das-Naglieri Cognitive Assessment System. Itasca, IL: Riverside

History of Theory and Testing Mental Abilities

19

Deary, LJ. & Caryl, P.G. (1997) Neuroscience and human intelligence differences. Trends in Neuroscience, 20, 365-371. Deary, LJ., Lawn, M. & Bartholomew, D.I. (2008) A conversation between Charles Spearman, Godfrey Thompson, and Edward L. Thorndike: the International Examinations Inquiry Meetings 19311038. History ofPsychology, 11, 122-142. Delis, D.C., Massman, P.I., Butters, N., Salmon, D.P., Cemmk, L.S., & Kramer, IH. (1991) Profiles of demented and amnesic patients on the California verbal learning test: implications for the assessment of memory disorders. Psychological Assessment 3, 19-26. Detterman, D.K. (1982) Does "g" exist? Intelligence, 6, 99- 108. Doyle, F.R., Pennock, I.M., Orr, I.S., Gore, I.C., Bydder, G.M., Steiner, R.E., Young, LR., Clow, R., Bailes, D.R., Burl, M., Gilderdale, D.I. & Walters, P.E. (1981) Imaging of the brain by nuclear magnetic resonance. The Lancet 318, 53-57. Doyle, K.O. (1974) Theory and practice of abilities testing in Ancient Greece. Journal of the History of the Behavioral Sciences. 10: 202212. Elwood, D.L. (1969) Automation of psychological testing. American Psychologist, 24, 287-289. Engle, R.W., Tuholski, SW., Laughlin, I.E. & Conway, A.R.A. (1999) Working memory, sjort-term memory, and general fluid intelligence: a latent-variable approach. Journal of Experimental Psychology: General. 128, 309-331. Erickson, R.C. & Scott, M.L. (1977) Clinical memory testing: a review. Psychological Bulletin, 84, 1 130-1 149. Finger, S. (1994) History of Neuropsychology. In Dahlia W. Zaidel (ed) Neuropsychology: a volume in Handbook ofPerception and Cognition, San Diego, Academic Press, pp 1-28. Flanagan, D. P., Ortiz, S. 0., Alfonso, V. C., & Dynda, A. (2014) Cognitive assessment: Progress in psychometric theories, the structure of cognitive tests, and approaches to test interpretation. In D. Saklofske, V. Schwean & C. Reynolds (Eds.), Oxford handbook of psychological assessment of children and adolescents. New York, NY: Oxford University Press. Frarik. G. (1983) The Wechsler Enterprise. Oxford, Pergamon Press. Fry, A.F. & Hale, S. (1996) Processing speed, working memory, and fluid intelligence!! evidence for a developmental cascade. Psychological Science, 7, 237-24 1 .

20

Chapter One

Geisinger, K.F. (2000) Psychological testing at the end of the millennium: a brief historical review. Professional Psychology: Research and Practice. 31, 117-138. Golden, C.J., Hammeke, TA & Purisch, A.D. (1978) Diagnostic validity of a standardized neuropsychological battery derived from Luria's Neuropsychological Tests. Journal of Consulting and Clinical Psychology, 46, 1258-1265. Guilford, J.P. (1956) The structure of intellect. Psychological Bulletin, 53: 267-293. Halstead, W.C. (1947) Brain and Intelligence: A quantitative study of the frontal lobes. The University of Chicago Press, Chicago. Herrnstein, R. J., & Murray, C. (1994). The bell curve: Intelligence and class structure in American life. New York: Free Press. Horn, J.L. & Cattell, R.B. (1966) Refinement and test of the theory of fluid and crystalized general intelligence. Journal of Educational Psychology, 57, 253-270. Humphreys, L.G. (1994) Intelligence from the standpoint of a (pragmatic) behaviorist. Psychological Inquiry, 5, 179-192. Jensen, A.R. (2000) Charles E. Spearman: The Discoverer of g. in G.A. Kimble and M. Wertheimer (Eds) Portraits ofPioneers in Psychology, New York, Psychology Press, pp. 92- 1 1 1 . Joreskog, K.G. & van Thillo, M . (1972) LISREL: A General Computer Program for Estimating a Linear Structural Equation System Involving Multiple Indicators of Unmeasured Variables. Research Bulletin RB71-1, Princeton, NJ, Educational Testing Service. Kaplan, E. (1988) A process approach to neuropsychological assessment. In T. Boll & B.K. Bryant (Eds.), The Master lecture series, VoL 7. Clinical Neuropsychology and brain function: Research, measurement and practice (pp. 127-167). Washington, DC, US: American Psychological Association. Kent, P.L. (2017) Evolution of Wechsler's memory scales: content and structural analysis. Applied Neuropsychology: Adult. 24, 232-25 1 . Kevles, D.J. (1968) Testing the army's intelligence: psychologists and the military in world war 1 . Journal ofAmerican History, 55, 565-5 8 1 . Kirk, S.A. (1968) Illinois Test o f Psycholinguistic Abilities: Its origin and Implications. In Hellmuth, J. (Ed.), Learning Disorders (VoL 3), Seattle, Washington Special Child Publications, pp. 397-427. Kirk, SA & McCarthy, J.J. (1961) The Illinois Test of Psycholinguistic Abilities- an approach to differential diagnosis. American Journal of Mental Deficiency, 66, 399-412.

History of Theory and Testing Mental Abilities

21

Kranzer, J.R. & Jensen, A.R. (1991) Unitary g : Unquestioned postulate or empirical fact? Intelligence, 15, 437-448. Lawley, D.N. & Maxwell, A.E. (1962) Factor analysis as a statistical method. Journal ofthe Royal Statistical Society. Series D. 12, 209-229. Leonberger, F.T. (1989) The question of organicity: is it still functional? Professional Psychology: Research and Practice, 20, 4 1 1 -414. Libon, D.l., Swenson, R., Ashendorf, L., Bauer, R.M. & Bowers, D. (2013) Edith Kaplan and the Boston Process approach. Clinical Neuropsychologist, 27, 1223-1233. Livingstone, M.S. & Hubel, D. (1987a) Segregation of form, color, and stereopsis in primate area 1 8 . Journal a/Neuroscience, 7, 3378-3415. Livingstone, M.S. & Hubel, D. (1987b) Psychophysical evidence for separate charmels fot the perception of fOlTIl, color, movement, and depth. Journal ofNeuroscience, 7, 3416-3468. Mackintosh, N.J. (201 1) IQ and Human Intelligence (2nd ed). Oxford University Press, New York. Mesulam, M.M. (1998) From sensation to cognition. Brain, 121, 10131052. Plaut, D.C. (1995) Double dissociation without modularity: evidence from connectionist neuropsychology. Journal of Clinical and Experimental Neuropsychology, 1 7: 291-32l. Neisser, U., Boodoo, G., Bouchard, T.l., Boykin, A.W.,Brody, N., Ceci, S.J., Halpern, D.F., Loehlin, J.C., Perloff, R, Sternberg, R.J., & Urbina, S. (1996) American Psychologist, 51, 77- 1 0 l . Rabin, L.A., Barr, W.B., & Burton, L.A. (2016) Stability in Test-Usage Practices of Clinical Neuropsychologists in the United States and Canada Over a 10-Year Period: A Follow-Up Survey of INS and NAN Members. Archives of Clinical Neuropsychology, 31, 206-230. Reitan, R.M. (1955) The relation of the trail making test to organic brain damage. Journal of Consulting Psychology, 19: 393-394. Reitan, RM. (1956) Investigation of the relationships between "psychometric" and "biological" intelligence. Journal of Nervous and Mental Diseases, 123: 536-54l . Reitan, RM. (1964) Psychological deicits resulting from cerebral lesions in man. In.J.M. Warren & K. Akert (eds) The Frontal Granular Cortex and Behavior, pp. 295-3 1 l . Reitan, R.M. & Wolfsen, D. (1994) A selective and critical review of neuropsychological deficits and the frontal lobes. Neuropsychology Review, 4, 161-198. Riley, E.N., Combs, H.L., Davis, HA & Smith, G.T. (2017) Theory as evidence: criterion validity in neuropsychological testing. In S.C.

22

Chapter One

Bowden (ed) Neuropsychological Assessment in the Age of Evidence­ Based Practice: Diagnostic and Treatment Evaluations. New York, Oxford University Press, pp. 15-43. Scoville, W.B. & Milner, B. (1957) Loss of recent memory after bilateral hippocampal lesions. Journal of Neurology, Neurosurgery and Psychiatry, 20, 1 1-21. Schneider, W.I. & Flanagan, D.P. (2015) The relationship between theories of intelligence and intelligence tests. In Goldstein, S., Princiotta, D., Naglieri, I. A. (Eds.), Handbook of intelligence: Evolutionary theory, historical perspective, and cUJrent concepts. New York, NY: Springer. 3 1 7-340. Shallice, T. (1988) From neuropsychology to mental structure. New York, NY, US: Cambridge University Press. Sondhaus, E. & Finger, S. (1988) Aphasia and the CNS from Imhotep to Broca. Neuropsychology, 2, 87-110. Spearman, C. (1903) General intelligence objectively determined. The American Journal ofPsychology, 15, 201-292. Spearman, C. (1904) The proof and measurement of association between two things. American Journal ofPsychology, 15, 72-101. Spearman, C. (1914). The theory of two factors. Psychological Review, 21(2), 101-115. Spring, I.R. (1972) Psychologists and the war: the meaning of intelligence in the alpha and beta tests. History ofEducation Quarterly, 12, 3-15. Terman, L.M. (1916) The measurement of intelligence: an explanation of and a complete guide for the use of the Stanford revision and extension of the Binet-Simon Intelligence Scale. Houghton Mifflin Co, Boston. Terman, L.M. (1918) The use of intelligence tests in the army. Psychological Bulletin, 15, 177-187. Terman, L.M. (1924) The mental test as a psychological method. Psychological Review, 31, 93-117. Teuber, H-L. (1954) Physiological Psychology. Annual Review of Psychology, 6, 267-296. Thompson, G.H. (1916) A hierarchy without a general factor. British Journal ofPsychology, 8, 271-28 1 . Thompson, G.R. (1919) On the cause of hierarchical order among the correlation coefficients of a number of variates taken in pairs. Proceedings of the Royal Society of London. Series A. Containing Papers ofa Mathematical and Physical Character, 95, 400-408. Thorndike, E.L. (1914a) The measurement of ability in reading. Teachers College Record, 15, 207-277.

History of Theory and Testing Mental Abilities

23

Thorndike, E.L. (1914b) Measurement of ability to solve arithmetical problems. Pedagogical Seminary, 21, 495-503. Thorndike, E.L. (1918) Individual differences. Psychological Bulletin, 15, 148-159. Thorndike, E.L. (1921) Intelligence and its measurement: a symposium. Journal ofEducational Psychology, 12, 124-127. Thurstone, L.L. (1934) The Vectors of the mind. Psychological Review, 41, 1-32. Thurstone L.L. (1935) The Vectors of the Mind, University of Chicago Press, Chicago. Thurstone, L.L. (1940) Current issues in factor analysis. Psychological Bulletin, 37, 189-236. Thurstone, L.L. (1946) Theories of Intelligence. The Scientific Montbly, 62: 101-1 12. Thurstone, L.L. & Thurstone, T.G. (1941) Factorial studies of intelligence. Psychometric Monographs, 2, 94. Tigner, R.B. & Tigner, S.S. (2000) Triarchic theories of intelligence: Aristotle and Sternberg. History ofPsychology, 3: 168-176. Toal, R. (1957) Reliability (internal consistency) of the Wechsler memory scale and the Wechsler-Bellevue intelligence scale. Journal of Consulting Psychology, 21, 131-135. Tulsky, D.S., Saklokske, D.H. & Ricker, J.H. (2003a) Historical overview of intelligence and memory: Factors influencing the Wechsler Scales. In D. S. Tulsky, D. H. Saklofske, G. J. Chelune, R. K. Heaton, R. J. Ivnik, R. Bomstein, A. Prifitera & M. F. Ledbetter (Eds.), Clinical interpretation ofthe WAfS-III and WMS-III (pp. 7-41). San Diego, CA, US: Academic Press. Tulsky, D. S., Saklofskie, D. H., & Zhu, J. (2003b). Revising a standard: An evaluation oftbe origin and development of the WAIS-III. In D. S. Tulsky, D. H. Saklofske, G. J. Chelune, R. K. Heaton, R. J. Ivnik, R. Bornstein, A. Prifitera & M. F. Ledbetter (Eds.), Clinical interpretation of the WAfS-III and WMS-III (pp. 43-92). San Diego, CA, US: Academic Press. Tupper, D.E. (1999) Introduction: Neuropsychological assessment apres Luria. Neuropsychology Review, 9, 57-61. Tyler, L.E. (1965) The psychology of Human Differences. New York, Appleton-Century-Crofts. Vincent, D.F. (1953) The origin and development of factor analysis. Journal of the Royal Statistical Society. Series C, 2, 107-117.

24

Chapter One

Walsh, R.T.G., Teo, T. & Baydala, A. (2014) A Critical History and Philosophy of Psychology: Diversity of Context, Thought and Practice. New York, Cambridge University Press. Wechsler, D. (1939) The measurement of adult intelligence. Baltimore: Williams & Wilkins, Baltimore. Wechsler, D. (1958) The measurement and appraisal of adult intelligence, 4th ed. Williams & Wilkins, Baltimore. Wissler, C. (1901) The correlation of mental and physical tests. Psychological Review: Monograph Supplements, 3, i-62. Yerkes, R.M. (1917) The Binet versus the point scale method of measuring intelligence. Journal ofApplied Psychology, 1, 1 1 1-122.

CHAPTER Two GENERAL AND SPECIFIC INTELLIGENCE ATTRIBUTES IN THE Two-FACTOR THEORY: A HISTORICAL REVIEW A. ALEXANDER BEAUJEAN

Historically, most intelligence research has either focused on some general attribute or on more specific attributes. 1 In this chapter, I describe how both perspectives can be traced back to the work of Charles Edward Spearman (1864-1945) and his two-factor theory. Moreover, I also describe Spearman's approach to the study of intelligence attributes and argue that modem researchers would be wise to incorporate his approach into their O\Vll. Before commencing, I acknowledge that taking a historical approach to a scientific topic can be a double-edged sword (Samelson, 1974; Baumgardner, 1977). It can be helpful because it can show that developments are not always independent of each other; that is, some events are reactions to, or result of, other events in a field's history. At the same time, a historical approach has the potential to give the impression that a field of study developed purposefully, when that is often not the case. One goal with this chapter is to maximize the fOlmer and minimize the latter. Another goal with this chapter is to present an alternative to the way that much modern intelligence work is carried out. Current intelligence research largely follows the Thurstonian tradition started in ef the 1930s and 1940s. Although the statistics are more complex and some of the instrumentation is more sophisticated, the basic philosophy is the same. If the field is going to progress, however, then those who work in it need to consider an alternative paradigm to the one that is currently in place.

1 I use the term intelligence throughout this chapter without any rigor to refer to the general domain of cognitive ability, not any particular attribute -within that domain.

26

Chapter Two

Charles Spearman and the Two-Factor Theory Background

The study of intelligence dates back to Plato (427-347 B.C.), although much of tbis early work was philosophical (metaphysical) and not scientific (Jensen, 1987). By scientific, I mean approaching the topic systematically and investigating phenomena empirically in order to: (a) describe/classify, (b) predict, (c) and account for their causal antecedents and mechanisms (Cattell, 1988). Sir Francis Galton (1822-1911) is usually credited witb being tbe first person to approach the study of intelligence scientifically, although there were certainly those who rigorously studied intelligence attributes before him (Boring, 1929). Because Galton was an heir to a family fortune, he did not need to earn an income. Consequently, he could pursue his varied scientific interests freely (Galton, 1908). Thus, his contributions to intelligence, while progenitorial, were limited. Instead, it was Charles Spearman who developed the first scientific theory of intelligence. For tbe sake of simplicity and brevity, I pick up the history with tbe work of Spearman. Little is known about Spearman's early life (P. Lovie and A. Lovie, 1996). He was educated at Leamington College for Boys (now North Leamington School) and while tbere developed an interest in philosophy. He graduated in 1882 and, likely due to his family's financial constraints, obtained a commission in the British Army (Royal Munster Fusiliers) as an officer of Engineers ("Sapper") and eventually joined the 2nd Battalion in India. Despite his burgeoning military career, Speannan's interest in philosophy never left. His salary was modest, but it went relatively far in India; thus, he was able to continue his boyhood passion for philosophy by acquiring books tbat he would carry from post to post. It was during this time that he became convinced that nif ever a genuine advance was to be made in philosophy, it would come mainly by way of psychology. To this latter study, then, I gradually transferred my allegiance!! (Speammn, 1930a, p. 300). Consequently, he entered and successfully completed Army Staff College (Camberley). Soon thereafter in 1898, he resigned from tbe military-at the age of 34 and with the rank of Captain-in order to go back to school to train in psychology. At the time Spearman was looking to further his education, a unew psychology" had been emerging--{)ne that used the scientific method and biological theory to answer questions instead of metaphysical speculations (Fallace, 2011). Germany housed some of tbe most advanced universities

General and Specific Intelligence Attributes in the Two-Factor Theory

27

in the world at this time and was an epicenter of this new psychology (Young, 1924). Consequently, this is where Spearman had to go to leam this new field. Spearman chose to pursue his graduate training with Wilhelm Wundt (1 832-1920) at the University of Leipzig. It was a program in experimental psychology, but to Wundt experiments were just a means to an end. Their purpose was to aid in deriving causal explanations of psychological phenomena. Theory was of primary importance because it produced the questions that lead to a search for data. In other words, in Wundt's philosophy of science "an experiment was essentially a way of asking theoretical questions" (Danziger, 1980, p. 1 14). The Leipzig program was not just a single person carrying out particular interest. Instead, Wundt had established the "first research community that was held together by a commitment to the belief that psychological questions could be systematically answered by means of experimental methods" (Danziger, 1980, p. 1 10). The nature of such a community was important because as Wundt's commitments increasingly caused him to spend more time out of his laboratory, he could rely on others to continue the work. Spearman, for example, credits two of Wundt's assistants-Felix Krueger and Wilhelm Wirth-with teaching him "the experimental teclmique" (Spearman, 1930a, p. 303) 2 Something often overlooked in discussions of Speannan is that the GelTIlan experimentalists taught him the importance of looking at general laws, not individual differences. In fact, he likened his correlational psychology-the method he used to develop his intelligence theory-not to individual psychology, which "endeavors to discover those small deviations from general law which constitute 'individuality'" but to a "general psychology" that was "in search of laws and unifolTIlities" (Spearman, 1904a, p. 207). Although Spearman later came to appreciate the study of individual differences, he always held that it was insufficient. Nor can it ever suffice merely to calculate numerous correlational coefficients. Science demands also that the results obtained in this way should be systematically interpreted in relation to one another. The most significant feature about correlations, as a rule, is not so much their absolute as their relative values, together with the theorems deducible from these. . Furthermore, even the most elaborate systematization will be quite inadequate if put aside into a watertight compartment labelled

2 He also studied lUlder WlUldt's former pupil Oswald Killpe at Wiirzburg and

George Milller at Gottingen, as well as had "all too fleeting visits" to Carl Stumpf and Hermann Ebbinghaus.

28

Chapter Two "individual differences." It ought, rather, to be thoroughly incorporated with psychology as a whole. (Spearman, 1928, p. 43) Theories ofIntelligence: Monarchic, Oligarchic, and Anarchic

In order to understand Spearman's intelligence theory, it is important to understand the context in which was developed. At that time, three major types of intelligence theories were prominent. Spearman (1914a, 1927b) somewhat pejoratively christened them ttmonarchictt, ttoligarchictt, and uanarchic. n He thought all three were errant, but errant for different reasons. Monarchic. The monarchic view was the revival of the ancient idea that: (a) there is one unitary intelligence attribute, (b) everyone possessed a distinct level of it, and (c) it entered with equal force into every activity (Norton, 1979; Spearman, 1914a). Thus, superior ability in one area would result in inferior ability and other areas. the quick learner is the poor rernernberer; that the man of great artistic gifts, as in music, painting or literary creativeness, is weak in scientific ability or matter-of-fact wisdom ; that divergence above the mode in power of abstract thought goes with divergence below the mode in thought about concrete things; that the man of superior intellect is likely to be of inferior mental health ; that the rapid worker is inaccmate ; that an agile mind goes with a clumsy body; etc., etc. (Thorndike, 1 9 1 0, p. 183)

Those who held the monarchic view used the word intelligence as if it had a technical definition and represented a unitary attribute. The major problem with this view was that there was never a consensus about the defmition or nature of intelligence, nor was there any evidence that it was a unitary attribute (e.g., ttInstinct and intelligencett, 1910). Intelligence eventually became a ttcommon-or-garden varietytt telTIl (Maraun, 1998) that meant something different to everyone who used it. ttIn truth, 'intelligence' has become a mere vocal sound, a word with so many meanings that finally it has none" (Spearman, 1927b, p. 14). This can be readily seen in the symposium on intelligence published in Education Psychology ("Intelligence and its measurement: A symposium",1921), the similar in-person symposium in England (Carr et aI., 1925), as well as a survey done more than 60 years later (Sternberg and Detterman, 1986). Anarchic. At the opposite extreme from monarchic theories were anarchic theories ¥few that posited intelligence is comprised of elementary processes which derive from multiple separate, independent attributes.

General and Specific Intelligence Attributes in the Two-Factor Theory

29

But any consideration of the nervous basis of mental life or of the patent facts of hlUllan nature suggests that a priori it is more rational to look on the mind as a multitude of particular capacities, particular associations and particular acts, all of which may be highly independent of each other. (Aikens et aI., 1 902, p. 374)

The view came from, and was reinforced by, studies by James, Thorndike and others that showed the lack of training transfer (Vernon, 1950). Some thought the implication from this work was that intelligence attributes were also highly specific. For example, if memorizing English poetry did not improve the ability to learn French poetry, then there could be no such thing as a general memory attribute. The view gained its most prominence concerning intelligence in the work of Godfrey Thompson (188 1-1955) and his idea of sarnplingfbonds (for an overview, see Sharp, 1980). Still, Thompson's notion never gained a lot of traction, likely because it was more of a reaction to Speammn's theory than an independent intelligence theory. A more sophisticated evolution of Thompson's ideas can be found in the ideas of mutualism (van der Maas et aI., 2014). Anarchic theories themselves are not as important as the influence they had on instrument development and scaling. Namely, if intelligence was comprised of numerous independent components, then the only thing that can be measured is an average level or sample of the entire domain. This idea is still prevalent in the interpretation of multidimensional intelligence instruments (Schneider & Flanagan, 2015). Oligarchic. The oligarchic view had been the dominant class of intelligence theories for much of the period when psychology was a branch of metaphysics. It stemmed from 1 8th-century faculty psychology (Spearman, 19l4a, 1930b) and was particularly strong in Germany during Spearman's time there (Fallace, 2011). There is not a single definition or conceptualization of faculty psychology because it was not a singular movement (Commins, 1933). Instead, it was a metaphysical notion of how the mind was organized that went through substantial alterations before it made its way into psychology (Schinitt, 1946). In broad strokes, the version of faculty psychology that was popular in 19th-century psychology held that the mind consisted of separate powers called/acuIties (e.g., intellect, memory, imagination, attention, language), which caused the varied manifestations of mental ability. These faculties were enumerated and described largely from introspection and unsystematic observation.

30

Chapter Two

If these faculties were just used as conveniences for classifying mental operations, likely little would have been made of them. That is not what happened, however; instead, they became reified as functional psychological entities. Each faculty was thought to be completely unitary within itself. Thus, one member of the class of operations for a faculty represented all the others in the same class. Between faculties, however, the unity of function was no longer upheld. Instead, faculties were thought to be independent/autonomous in their activity. For example, all operations of observation were thought to be of an equivalent faculty class, but they were thought to be independent of reasoning, memory, etc. (Thorndike, 1903). The notion of between-faculty independence resulted in psychologists coming to believe that each faculty not only needed its own separate measurement (and interpretation), but that a true understanding of an individual required the construction of elaborate mental profiles. Here, each trait receives its O\Vll measurement; the values obtained are plotted on paper and then joined together by a line the "profile" which thus graphically depicts the person's general mental make-up. (Speannan, 1927b, p. 27).

The oligarchic view became very influential in popular psychology (e.g., Sokal, 2001) and education (e.g., Sleight, 1915; Fallace, 2011), so was subject to many examinations-both empirical and philosophical. Much of the resulting work indicated that there were numerous problems with this viewpoint (e.g., James, 1890; Woodworth and Thorndike, 1901) 3 As a result of this scholarship, it became anathema to invoke psychological faculties-at least in academic circles. That did not mean that the tenants of faculty psychology died. To the contrary, the term "faculty" was replaced with new nomenclature (e.g., tttraitstt; Lehman and Witty, 1934) and those who employed the ideas of faculty psychology in their day-to­ day practices carried on business as usual (Vernon, 1950). 3 The major problems with this theory are threefold. First, the faculties do not

constitute lUlitary fimctions. For example, a mere skim of a book on memory (e.g., Baddeley et aI., 2014) would find that memory is not a lUlitary attribute (e.g., working memory is separate from long-term retrieval, procedural memory, implicit memory, etc). Thus, accmately representing the "memory faculty" with a single score from a single instnunent would likely prove impossible. Second, the postulate that faculties are independent of each other is something that can be, and has been, empirically examined. The results are almost lUlanimous in indicating that they are not independent (e.g., Carroll, 1993). Third, faculty psychology involves circular reasoning. I discuss this more in the Latent Variables section.

General and Specific Intelligence Attributes in the Two-Factor Theory

31

Two-Factor Theory 1899 saw the start of the Second Boer War. In 1900, the Royal Army asked Speannan to return to military service, so he took a leave of absence from his graduate studies to serve as Deputy Assistant Adjutant General to Guernsey.4 It was during this time he acquired Inquiries into Human Faculty and its Development (Galton, 1 8 83). Inspired by Galton's ideas and armed with his experimental psychology training, he started conducting experiments with where he was stationed. His aim was to test the monarchic, oligarchic, anarchic theories empirically (Flugel and West, 1963). Having collected data to test the theories, he needed a method to quantify their relations while also accounting for artifacts in the data (i.e., measurement error, restriction of range). Thus, he developed a method of correcting correlations for measurement error (Spearman, 1904b) that he could then use to test the implications of the various intelligence theories (Spearman, 1904a) 5 What Spearman found was that the correlations tended to be positive, but were neither very high nor very low-in essence nullifying all three doctrines. Yet, there appeared to be some sort of principle governing the size of the correlations, so perhaps Galton was correct: the attributes that comprised the ttintellectual facultytt were all related to each other, just not perfectly so. This initial empirical work, as well as the results from some subsequent studies he did thereafter (e.g., Hart and Spearman, 1912; Krueger and Spearman, 1906), lead him to develop the Two-Factor Theory (TFT) 6 TFT had three major components (Spearman, 1933). 1. All instruments that measure attributes within the ttgeneral sphere of ability" (Spearman, 1933, p. 597) are independently obtained estimates of ttthe one great common Intellective Functiontt 4

He later returned to Germany to complete his dissertation (Spearman, 1 905a). If the monarchic doctrine was true, this should result in very high correlations among all the "intellectual" attributes since it held the same general attribute would be involved in performance across all the instrmnents. The anarchic doctrine, on the other hand, predicted that the correlations would be close to zero since each attribute was thought to be independent. If the oligarchic doctrine was true, then this would produce results in between -there should be very strong correlations among some of the scores (i.e., those from instnunents assessing the same faculty), and very low correlations in other cases (i.e., when assessing different faculties). 6 Sante De Sanctis originated this label, not Spearman (Spearman, 1 9 1 4b). 5

32

Chapter Two

(Spearman, 1904a, p. 272). He initially called this function "general intelligence,!! but later just symbolized it as g. 2. Each instrument assesses something independent of g, which Spearman called a ttspecific factor!! and symbolized as S. There is one S for each instrument. 3. If the attributes the instruments assess are sufficiently diverse, the Ss from each instrument will be independent of each other. Otherwise, they may !tbe brought together as a more or less unitary power" (Krueger and Spearman, 1906, p. 103) 7 Spearman did not provide a particular name/symbol for these concepts, but Thomson (Thomson, 1916) later called them group factors. I represent them using the symbol y. What Spearman (Spearman, 1904a) called a "factor" (i.e., g, Ss, ys) is now typically called a latent variable (LV) 8 Thus, before discussnig the components of TFT further, I provide a brief digression into LVs.

Latent Variables A LV is nothnig more than a variable that was not assessed directly (Loehlin and Beaujean, 2016). LVs are used frequently in science. For example, no one has ever directly measured a quark or an element's periodicity. Yet, these variables are paramount to many scientific theories. Shipley (2016) differentiated between two types of LVs: (a) variables representing phenomena that could be directly observed, and (b) variables representing phenomena that could not be directly observed. An example of the first type is measuring air temperature using a mercury thermometer. In principle, air temperature (i.e., average kinetic energy of the air molecules) can be observed and measured. We do not directly measure temperature using a themlOmeter, however; we measure the 7

The original quote is erne ziemlich grofse Gruppe von Leistungen nahe genug verwandt sind, lUll als eine mehr oder weniger einheitliche Leistungsfahigkeit lUlter [a rather large group of activities might be sufficiently akin to be brought together as a more or less unitary power] (Krueger and Spearman, 1 906, p. 103) 8 What Speannan meant by "factor" is more consistent with the APA Dictionary of Psychology's definition of "anything that contributes to a result or has a causal relationship to a phenomenon, event, or action" than a latent variable. The terms factor analysis and latent variable analysis have subsequently been used interchangeably, which has led to some conceptual confusion (MaralUl & Gabriel, 2013).

General and Specific Intelligence Attributes in the Two-Factor Theory

33

height of a colunm of mercury in a vacuum enclosed in a hollow glass tube. Thus, in this example temperature is a LV. The second type of LV is the most pervasive in psychology. With these, the very existence of the attribute the LV is supposedly representing is a hypothesis in and of itself. Thus, it becomes important to distinguish attributes from LVs, since not all LVs will represent an attribute. In intelligence research, LVs are usually used for an economical representation of what is in common among multiple directly measured variables. Statistically finding a LV and then describing it is usually not terribly difficult. There are hundreds-possibly thousands--{)f published works showing the myriad of LVs that can be extracted from a set of scores from instruments thought to measure behaviors resulting from intelligence attributes (e.g., Carroll, 1993). If the use of the LVs stopped at being statistical summaries, there would probably be little criticism of their use. The problem comes when individuals prematurely act as if the LVs represent some attribute that physically exists. I emphasize the word ttprematurelytt because LV reification (i.e., when a LV is sho\Vll to represent some ttthingtt that exists outside of the model) can be very useful when appropriately done (e.g., Tanner, 1947). To do so, however, requires a systematic line of logical/conceptual and experimental research on the supposed attribute the LV represents (e.g., Boag, 201 1 ; Borsboom et aI., 2003). Statistically indicating the presence of a LV-even if replicated across hundreds of studies-is insufficient to make the claim that there is some unitary attribute causing the measured variables to be related. Unfortunately, the systematic research needed to move from the position that a LV is a statistical summary to the LV represent some real psychological or physical attribute is often bypassed (Humphreys and Stark, 2002). Wbat happens all too commonly instead is that a statistical model is developed that postulates one or more LVs. Data is collected that is consistent with the model-usually consisting solely of between­ individual relations-then the model is tested to see how well it fits the data. If the model ttfitstt using some set of idiosyncratic criteria, then the LVs are given names based on the content of the variables that are most strongly related to the LV. As a descriptive device, naming LVs based on what is in common among the variables comprising them is fine. Problems begin to arise when these names come from common-or-garden-variety words that have popular, non-technical meanings (e.g., working memory, processing speed). Wbile this can serve to tie the work to everyday life, it also conflates technical and non-technical meanings of words (Wbite, 2000).

34

Chapter Two

The LVs' names are then taken as evidence that there are equivalent attributes that exist within individuals and those eponymous attributes are then invoked to explain why individuals differ on the variables that comprise the LVs. This is circular reasoning, and it is a major problem because circular explanations are not testable (Gerber, 2011). The situation gets worse, however. Because of the non-technical names and definitions inherent in many LVs within the intelligence sphere, psychologists assume that if the names used to describe the LVs are the same, then the attributes that they ostensibly represent must also be the same. !fRaYing fashioned for ourselves such a name as, say, judgment, then-because the name remains the same-we fall into the belief that we are treating of an entity always the same" (Spearman, 1927b, p. 39). Example: Processing Speed in the Wechsler Intelligence Scales

An example may clarify things. The technical manual for the fifth edition of the Wechsler Intelligence Scale for Children (WISC-V; Wechsler, 2014) indicates that the 10 primary subtests fit a measurement model with five first-order LVs. (There is also a second-order LV in the model, but it is not of particular concern here.) One of those LVs is named processing speed (PS). PS represents what is in common between scores from two instruments (i.e., subtests): Coding (CD) and Symbol Search (SS). Functionally, the two subtests are very relatively homogenous: they both have strict time limits, require decisions about simple visual stimuli, and responses are provided using motor skills (Gregoire, 2013). The WISC-V authors never described PS, much less defined or explained it. They did describe the aggregated score derived from the PS subtests, the Processing Speed Index (PSI). PSI values are supposed to represent the PS variable: "The PSI measures the [respondent's] speed and accuracy of visual identification, decision making, and decision implementation!! (Wechsler, 2014, p. 159). Further, they wrote that difference between individuals who have high and low PSI scores is their "ability to rapidly identify visual information, to make quick and accurate decisions, and to rapidly implement those decisions" (p. 159). To summarize things thus far, PS is a LV comprised of what is in common from two subtests that both require quick decisions about simple visual stimuli and motor responses. The score used to represent PS (PSI) is described as measuring speed and accuracy of decisions made from visual information. Moreover, the creators of the PSI attributed differences in PSI values to differences in individuals' speed of visual identification,

General and Specific Intelligence Attributes in the Two-Factor Theory

35

making decisions, and implementing the decisions. This is circular reasonmg. To compound the problem, psychologists have a long history of studying a concept called processing speed (O'Brien and Tulsky, 2008), which may or may not be the same as PS. It is difficult to know because processing speed is a common-or-garden-variety telTIl that does not have a consensus-driven technical definition, so is used differently across psychologists (Cepeda et aI., 2013). This problem is not discussed in most (perhaps any) WISC-V interpretational guides. Instead, clinicians are told that PS is a unitary attribute within individuals (Flanagan et aI., 2018, p. 358), it is the same as the processing speed attribute assessed by other instruments (e.g., Mascolo and Flanagan, 2016, p. 3 1 1), and intra­ individual differences in PS (as measured by the PSI) can have potential causal influence on school perfolTIlance (Flanagan and Alfonso, 2017, p. 215-216). In light of this situation, one is reminded of John Stuart Mill's apt warning more than a century ago. The tendency has always been strong to believe that whatever received a name must be an entity or thing, having an independent existence of its own; and if no real entity answering to the name could be found, men did not for that reason suppose that none existed, but imagined that it was something peculiarly abstruse and mysterious, too high to be an object of sense. The meaning of all general, and especially of all abstract terms, became in this way enveloped in a mystical haze (Mill and Mill, 1 869, p. 5)

Two-Factor Theory, Continued Having digressed to discuss LVs, I can now return to the TFT. Much of Spearman's academic writing focused on the first part of TFT: g. Scholars have already written volumes about g and Speannan's work (e.g., Jensen, 1998), so there is no need to give much of space to rehashing what others have written. Still, there are some important things to note about the way Spearman approached the study of g. First, Spearman did not prematurely reify g or try to explain it circularly (Spearman, 1 927b, pp.75-76). To the contrary, he warned against premature reification. far from starting with a clear definition of General Intelligence, I had to content myself with using the term as denoting an almost unknown X "implying nothing more than a bare unequivocal indication of the factual conditions of the experiment." To solve this X I proposed, and have since

36

Chapter Two been gradually carrying out, a long investigation upon strictly experimental lines (Spearman, 1 905b, p. 2 3 1 )

That is why later he eventually came to prefer nnon-committal letter of the alphabet g" instead of "general intelligence" (Spearman, 1930b, p. 343). Second, Spearman technically (mathematically) defined g as (1) where max denotes the xth individual's value (score) obtained from an instrument assessing intelligence attribute a; gx is the individual's amount of g; rag is the correlation between the score representing a and g; and ras is the correlation between the score representing a and the S specific to a (Spearman, 1927b, pp. xiv). Of note, Eq. (1) should not be interpreted as saying any general factor is g. Instead, it was only when "branches of [measured] intellectual activity are at all dissimilar, then their correlations with one another appear wholly due to their being all variously saturated with [g] " (Spearman, 1904a, p. 273). For example, if instruments assessing arithmetic fluency, general mathematics calculations, and quantitative reasoning were factor analyzed, there will probably be a general factor among them. The general factor from these instruments, however, would not be g. Third, Spearman provided a way to falsify TFT, at least the part concerning g. [intially the falsification method involved the arrangement of the correlations, but later this evolved into examining tetrad differences (Thomson, 1927). It turns out that the tetrad differences criterion was insufficient to determine g completely and uniquely (i.e., there is indetenninacy in the solution; P. Lovie and A. Lovie, 1995; Steiger and Scli6nemarm, 1978) 9 Nonetheless, TFT was falsifiable in principle-an important aspect of any scientific theory (Hom and McArdle, 2007). Fourth, Speannan was not content with just describing or measuring g; instead, he spent the majority of his career trying to explain the LV (Smith, 1950; Spearman, 1930b). This is important because LV models will never be sufficient evidence to prove that some attribute exists or to understand its nature. The LV could still turn out to be chimera and not have any physical representation (Hom, 2008). To move from a statistically-defined construct to psychological attribute, it also needs to be explained (Boag, 2010).

9

There were additional difficulties besides indeterminacy, but these largely concerned Speannan's particular methods (Dodd, 1928).

General and Specific Intelligence Attributes in the Two-Factor Theory

37

Spearman's explanation was in telTIlS of a combination of noegenic laws (e.g., Ballard, 1929) and abstractness (Spearman,1946).10 As witb his method of falsification, his explanation was incomplete (A. Lovie, 1983; Mereditb, 1948). What is important to note is tbat his explanation not only involved inter-individual differences, but also intra-individual differences. It is insufficient to explain differences between individuals; for a LV to ever have some physical representation it must also be able to explain differences witbin individuals (Molenaar and Campbell, 2009). Attributes other than g. Altbough Spearman spent much of his work on g, that does not mean that he ignored other possible attributes in the intelligence sphere. To the contrary, SpealTIlan devoted considerable space to LVs other tban g in his books (e.g., Spearman and Wynn Jones, 1950; Spearman, 1927b,c). Moreover, much of the work that came out of his University of London laboratory focused on possible attributes other than g (e.g., Cox, 1928; Davey, 1926; Gopalaswami, 1924; Sleight, 1 9 1 1 ; Webb, 1915). The issue was not that Spearman was unaware that other attributes existed-even his critics acknowledged this (Thomson, 1916). The issue was that tbe bar he set for moving from a LV to an attribute was high. He had seen tbe problem witb faculty psychology/oligarchic theories of positing new attributes based on anecdote and personal experience, so was wary about indicating an attribute existed before it went through rigorous study. To understand Speannan's perspective about non-g attributes in the intelligence sphere, it is important to understand what he meant by S (Spearman, 1933). Given set of sufficiently diverse variables from tbe intelligence sphere, their variance can be decomposed into the part that is in common with all the other variables (i.e., g) and the part that is common to a single variable in tbat set (i.e., S). In other words, S was anything other tban g (See Fig. 2·1a) 11

10 The term noegenesis (sometimes noegenesis) was a portmanteau of the Greek nous (vo\)�) and genesis (yivf:(Ju;) to designate the creation of knowledge. The qualitative noegenic laws were: (a) Law of Apprehension of Experience (i.e., perceive the flUldaments [fundamental features] of the problem); (b) Law of Eduction of Relations (i.e., find relations among fundaments of problem); and (c) Law of Eduction of Correlates (i.e., extrapolate, interpolate, or generalize in order to infer a not immediately educed relation from the evidence of the extant relations). 1 1 Ss and ys represented anoegenetic processes, which were a product of noegenesis (Stephenson, 1977).

38

Chapter Two

(a) Ss do not overlap.

(b) Ss do overlap.

Figme 2 - 1 . Speannan's conception of non-g latent variables.

Depending on the original set of variables, S can be comprised of: (a) error (random); (b) unique aspects of a particular instrument (systematic, but not an attribute); or (c) aspects of a particular variable that are only unique for particular sets of variables (systematic, and potentially an attribute). The latter represents the situation where if certain other variables had been included in study, then two (or more) Ss would have something in common with each other beyond g. In such cases, the Ss' overlap may represent an additional LV (see Fig. 2-1b). Spearman (1933) did not disagree that ys could represent attributes, but was hesitant to give them meaning prematurely. First, the classification of LVs as general, group, or specific was arbitrary and only applied to a specific set of instruments. Any factor may upon occasion be general in the sense of being common to all the abilities in some set; any (except g) may be confined to one ability only in a set; and so too any (except g) may be shared by less than all but more than one. The classification becomes futile (lUlless, indeed, some limiting condition be introduced, and this does not appear to have been done). (Spearman, 1933, p. 598)

Second, ys tended to be unstable across studies-both in number and in interpretation. One cause of the instability was the way that LVs were modeled/extracted during Spemman's time (i.e., unrotated from centroid/simple summation or principal components analyses; Hom and McArdle, 2007). Another cause of the instability was between-study differences in the variables included, meaning that ys could "come and go at our will" depending on the redundancy of the instruments (Speammn, 1933, p. 600).

General and Specific Intelligence Attributes in the Two-Factor Theory

39

Third, even if a y was stable it did not necessarily mean it represented a psychological attribute. As with any other LV, ys could represent some psychological attribute or could be statistical chimera. To be a candidate for representing an attribute, a y had to tthave scientific significancett and confer ttsome functional unity on a range [of behaviors] broad enough to be important" (Spearman, 1930b, p. 357). Otherwise, Spearman feared it could lead to a reincarnation of faculty psychology. Despite his caution, Speannan did acknowledge that attributes existed in the intelligence sphere other than g. In fact, he was the first to publish about what he called broad attributes in relation to g. Working with Felix Krueger-one of his Leipzig mentors-they conducted an experiment that combined both the Leipzig experimental paradigm with Spearman's correlational psychology (Krueger and Spearman, 1906). Specifically, they examined within- and between-individual differences in perfonnance on instruments designed to assess some basic skills in the intelligence sphere: pitch discrimination, meaningful memory, tactile thresholds, simple addition, and rote memorization. In addition, they re-analyzed the work of Oehm (1 889), who studied the speed of conducting some simple tasks (e.g., counting single letters, adding pairs of numbers, reading single syllables, learning nonsense syllables). They found that g had little relation to rote memorization (i.e., ttpure retentivitytt) tasks, but did come into play as soon as individuals were required to use the infonnation. For example, rote memorization of numbers had little relation to g; when individuals were required to add those numbers, however, g was involved. Thus, rote memorization could be an attribute in the intelligence sphere independent of g. Later, he acknowledged other ys that were of "sufficient breadth and degree to possess serious practical consequences, educational, industrial, and vocational" (Spearman, 1927b, p. 242) that they likely represented additional attributes in the intelligence sphere (e.g., mechanical, aritlnnetical computation, verbal; Speannan, 1938; Speannan and Wynn Jones, 1950).

Thurstone: An Alternative View of Intelligence and Latent Variables Some have argued that Sir Cyril Burt (1883-1971) provided the first criticism of TFT because he acknowledged the existence of attributes in the intelligence sphere other than g. This is not correct. While Burt's (1909, 1 9 1 1) initial work did focus on ys, he never argued against the centrality of g. In fact, it is likely that Spearman was a ghost co-author for

40

Chapter Two

Burt's 1909 original article (A. Lovie and P. Lovie, 1993). Moreover, since TFT does not specify that g was the only attribute within the intelligence sphere, a model containing both g and ys is fully in line with TFT. The first real alternative to TFT that posed a threat to g came from Louis L. Thurstone (1887-1955).12 Like Spearman, Thurstone came to the study of intelligence from a non-traditional background (Thurstone, 1952; Wood, 1962). He graduated from Cornell University with a degree in engineering, and then went to work with Thomas Edison before earning a Ph.D. in psychology from the University of Chicago. In 1924, Thurstone was offered a position in the psychology department at the University of Chicago. There, he focused largely on statistics and a relatively new area in the discipline: mental test theory (e.g., Thurstone, 1931b). This led him to in-depth studies of psychological measurement problems. Much of his work at this time focused on issues related to scaling instruments for a variety of psychological attributes, ranging from psychophysics to attitudes. In the late 1920s/early 1930s, he turned his attention towards LV methods, especially as they were used in intelligence research (Thurstone, 1931a). Whereas Spearman thought that ys were often too unstable to represent any unitary attribute, Thurstone thought g was too unstable across studies. Thus, the general LV from one study was likely not the same general LV from another study. To Thurstone, g was !fa hodge-podge of unknO\vn abilities combined at unknown weights" (Wood, 1962, p. 12). Consequently, he focused on non­ general LVs (analogous to ys in TFT), unlike Spearman, who used experiments to test theory-derived hypotheses, Thurstone's initial work was not based on any a priori substantive theory. Rather he wanted to know !thow many general and independent factors it is necessary to postulate in order to account for a whole table of intercorrelations!t (Thurstone, 1931a, p. 409). Thurstone's initial method of analyzing intelligence variables required administering redundant instruments (i.e., over-determination), extracting multiple LVs, and then orthogonally rotating them to simple structure to aid with their interpretation (Thurstone, 1936b). While this provided some stability for the LVs, it also precluded a general LV (Thurstone, 1931a). In other words, it prevented the statistical modelling of g in a given dataset. 12 There were critics of TFT before Thurstone, but most of them were of the fashion: is g sufficient? (Thomson, 1920; Kelley, 1928). Some provided mathematical criticisms (e.g., Wilson, 1928), but these largely concerned the issue of indeterminacy, which did not discOlUlt the existence of g as much as indicated that statistical evidence was insufficient proof.

General and Specific Intelligence Attributes in the Two-Factor Theory

41

Thurstone's new method served as the basis for his new "conception of intelligence. " While the complex known as intelligence is very useful in differentiating those who are generally bright and those who are less endowed, it is of great practical and scientific importance to isolate those elements of intelligence which are in some fundamental sense primary (Thmstone, 1 936a, p. 443 444).

This new conception required exploring (a) how variables in the intelligence sphere were affected by a fundamental set of LVs tbat he called tbe primary mental abilities (PMAs); and (b) how each PMA was distinguished from other PMAs, specific LVs, and error. This set off a somewhat long and acrimonious debate between the "British factor analysis" and "the Thurstonians" about whether or not general or broad/group LVs should be included in tbeir statistical models (Mulaik, 1986). While it tums out tbat, statistically, Spearman's and Thurstone's models were not all tbat different (Hedman, 1938), it is probably incorrect to say that their disagreements were unnecessary (e.g., Bartbolomew, 1995; Carroll, 1993). It is doubtful they could have had much rapprochement since their underlying philosophies were so divergent (e.g., Haig, 2013). Spearman largely adbered to a version of realism (Hood, 2013) and studied intelligence from a confirmatory/deductive approach (but see Norton, 1979). Thurstone, on the other hand, adbered to a fictionalist­ instrumentalist philosophy (Block, 1974) and approached the study of intelligence using exploratorylinductive methods. A scientific law is not to be thought of as having an independent existence which some scientist is fortunate to stumble upon. A scientific law is not a part of nature. It is only a way of comprehending nature . . . . The criterion by which a new ideal construct in science is accepted or rejected is the degree to which it facilitates the comprehension of a class of phenomena which can be thought of as examples of a single construct rather than as individualized events. It is in this sense that the chief object of science is to minimize mental effort . . . (Thurstone, 1935, pp. 44 45)

Whether Thurstone's philosophy was born out of his methodological focus or was the cause of it, it allowed him to avoid being constrained by any elaborate and overly structured substantive theory (S. Lovie, 1983). It enabled him, at least in part, to construct a general-purpose statistical system that applied to questions from subdomains ranging from psychophysics to attitudes to intelligence. This is in stark contrast to

42

Chapter Two

Spearman who used methods only as a means to test and refine more substantive theory. Their differences in philosophies can also be seen in how they approached the study of LVs. Speannan's approach was reminiscent of Wundt, in that LV models were tools necessary to aid in testing theory­ derived hypotheses (e.g., Hart and Spearman, 1912) and, subsequently, understanding the tttruet! nature of reality. In contrast, Thurstone's approach was more in line with faculty psychology. It is true that the object of factor analysis is to discover the mental faculties. The severe restrictions that are imposed by the logic of factor analysis make it an arduous task to isolate each new mental faculty, because it is necessary to prove that it is called for by the experimental observations. (l1nrrstone, 1935, p. 35; see also Anastasi, 1938)

The statistical rigor of Thurstone's methods was appealing to psychologists. Consequently, others soon began to adopt his methods-as well as his philosophy-with more diverse sets of variables. While Thurstone was conservative in his exploration of the intelligence sphere (i.e., he thought there were approximately nine PMAs; Thurstone, 1949), the same carmot be said for others using his methods. Thurstone later seemed to realize the consequences of liberating method from theory and tried to correct things (e.g., Thurstone, 1940, p. 235), but the train had already left the station. The number of ys that psychologists were able to find quickly increased throughout the 1940s and 1950s. By the 1960s it was "widely accepted that factors can be fractionated and proliferated ahnost without end" (Ferguson, 1965, p. 47). The only real limits to the number of ys was psychologists' creativity in developing instruments (or batteries of instruments) and finding names for what a set of scores had in common (Humphreys et aI., 1969). Currently, there are 100s of LVs that people have posited exist within the intelligence sphere (e.g., Carroll, 1993; Schneider and McGrew, 201S)-although whether they represent actual attributes or are chimera remains to be determined (Mulaik, 1987).

Jingling Latent Variables More than a century ago, Thorndike noted a major problem with the terminology used in psychology-what he called the jingle fallacy. Psychologists use the same (or similar) words to describe "things" that are functionally distinct. This causes confusion because "we tend to accept all the different things to which they may refer as of identical amount"

General and Specific Intelligence Attributes in the Two-Factor Theory

43

(Thorndike, 1904, pp. 14). The jingle fallacy is ubiqintous in psychological research, and is problematic because it wastes scientific time (Block, 1995). One example of tbis is the terminology used to describe the LVs in Spearman's and Thurstone's models. Spearman and Thurstone not only had different philosophies regarding the nature of LVs, but also had very different conceptualizations about the LVs within each of their models. In TFT, stable ys represent phenomena that were sufficiently broad and meaningful beyond g. Thus, any broad attributes that ys may represent had to be functionally independent of both g and each other (Spearman, 1925). Contrast Spearman's conception of ys to that of Thurstone's conception of PMAs. The interpretation that seems plausible at this time is that the primary factors represent different kinds of mental facilities such as several kinds of memory, several kinds of perceptual closure, several visualizing factors, and several verbal factors. These primary abilities may be regarded as media for the expression of intellect and people differ markedly in the effectiveness with which they can express themselves in these different media. The first-order primary factors may be regarded as separate organs, in a general sense, . . . (Thmstone, 1948, pp. 403 404).

In other words, ys PMAs were comprised of two components: (a) the "media for the expression of intellect" (i.e., specific LV) and (b) a common influence on multiple media (i.e., common LV).

Figme 2-2. Higher-order latent variable model.

44

Chapter Two

Although Thurstone originally represented the PMAs as being statistically independent, he later allowed them to be correlated with each other (Thurstone, 1940,1944,1947). This allowed for the possibility of higher­ order LVs (HOLVs). Originally, he found a single HOLV that was in common with all the PMAs-a type of general LV. An example is shown in Fig. 2-2. Superficially, the general HOLV could represent the same attribute as g, and it has not been uncommon to treat them ttinterchangeably in practice" (Ree and Earles, 1991, p. 276). Neither Spearman nor Thurstone believed them equivalent, however (Thurstone, 1948; Spearman, 1927a), which can be seen by comparing their respective descriptions of the concepts they represent. 13 Spearman statistically defined the g in TFT (see Eq. 1). It represents some attribute that is shared by all other dissimilar entities within the intelligence sphere (Spearman, 1904a). This is very different from Thurstone's description of a general HOLV (cf. Fig. 2-2). In a HO model, ys take precedence; so, a general HOLV represents whatever is in common to the ys. This general factor is what we have called a "second-order general factor." It makes its appearance, not as a separate factor, but as a factor inherent in the primaries and their correlations. (l1nrrstone and llnrrstone, 1941, p. 26)

If a general HOLV is not g, then what is it? Interpretation is a major difficulty with HOLVs-akin to "interpreting shadows of the shadows of mountains rather than the mountains themselves" (McClain, 1996, p. 233). Some have argued that it can be made easier by examining how the HOLV relates to the original measured variables (e.g., Gorsuch, 1983). There is nothing wrong with calculating these relations, but such an interpretational approach neglects an important aspect of the HO model: group LVs fully mediate the HOLV-measured variable relations. That is, whatever phenomenon the HOLV represents, it has to be something that only transmits causal influence on the group LVs, which in turn transmit their causal influence on the original measured variables (Pearl, 2014). If a HOLV actually turned out to represent a psychological attribute, it could not be the same attribute defined by Spearman as g (Spearman, 1927a). Calling them both g is a disservice to both Thurstone and 13 This does not discount the fact that statistical models representing TFT and some alternative theory could fit a given dataset equivalently (e.g., YlUlg et aI., 1999). Equivalent fit does not mean the models are equivalent in terms of what they represent about a substantive theory, however.

General and Specific Intelligence Attributes in the Two-Factor Theory

45

Spearman. Likewise, if a group LVs in a Thurstoinan HO or multiple-LV model was found to relate to the same measured variables as a y in a TFT model, it does not mean that they represent the same phenomenon. y in TFT represents something independent of g, while the group LVs in a HO or multiple-LV model do not. Eqinvocating the LVs (as well as any possible attributes that may represent) is just an instance of the jingle fallacy.

Moving Forward In this chapter, I discussed some of the history and meaning of the two­ factor theory (TFT). While Spearman spent much of his own work focusing on one aspect of it (i.e. g), it does not mean that he ignored the other aspects (S and y). In fact, TFT is the first intelligence theory that allowed for both a general attribute as well as other broad attributes within the same system. More than just being historically president, Spearman's approach to developing and testing TFT can stand as a paradigm for modem investigations of intelligence attributes. To be sure, TFT was flawed and incomplete-but the same can be said for most scientific theories. Still, his approach to the study of intelligence attributes is commendable, and one could do far worse than following in some of Spearman's major footsteps. I conclude this chapter with two concrete suggestions based on Speammn's example. Terminology

It would behoove intelligence researchers to refine their telTIlinology (Spearman, 1931; Maraun, 1998). It was common in the 19th and 20th­ century for psychologists to connect their work to everyday life by trying to put a scientific spin on common-or-garden-variety telTIlS (\¥hite, 2000). This practice is problematic because it contaminates and confuses thinking about scientific phenomena. Moreover, it leads to conceptual homonyms: the same word or phrase having both a common everyday meaning as well as some scientific connotation with a fuzzy boundary between the two. Even in the more scientific connotations of psychological telTIlS there is confusion. There are few consensus-derived technical definitions of psychological attributes (Krause, 2012), which leads to substantial variability in how attributes of the same name are operationalized. To see this, compare g with processing speed.

46

Chapter Two

Neither g nor processing speed can be directly measured. Thus, their definitions are very important. Spearman defnied g technically and mathematically (see Eq. 1). This explicitness not only helps others to know what g is, but is also guards against conflating g with what it is not (e.g., "intelligence", IQ score; Spearman, 1931, 1937). Although there has been some appropriation of the telTIl over the last 100 years (e.g., Horn, 1985), any two individuals who read Spearman's work should have little confusion about what it is he meant by g (e.g., Spearman, 1941). In contrast to g, processing speed has only been described verbally. I provided one description from the WISC-V authors earlier in this chapter. A more refnied version comes from Schneider and McGrew (2018) who described it as ILthe ability to control attention to automatically, quickly, and fluently perform relatively simple repetitive cognitive tasks" (p. 108). There is nothing inherently nothing wrong with this definition, or the general use of verbal approaches to describing phenomena. They should only be thought of as first approximations to defining the phenomena, however, because they are seldom susceptible to falsification and are open to a variety of operationalizations (Stenner et aI., 1983). For example, how would one falsify the claim that values from the PSI on the WISC-V represent individuals' processnig speed? The typical psychometric approach is to gather a group of individuals and give them the PSI subtests as well as a hodgepodge of other nistruments and then calculate the correlations among the scores. The results from these types of studies are largely meaningless when making the argument that the PSI measures processmg speed (Beaujean and Benson, m press). Demonstrating that the PSI has minimal correlations with instruments of tfdissimilar attributestf provides no infOlmation about what the PSI values mean. Likewise, demonstrating that the PSI has non-minimal correlations with instruments of tfsimilar attributestf only provides infOlmation that the scores from the PSI and other instruments are related (Borsboom et al., 2004). An additional problem with verbal descriptions of attributes is that two psychologists could easily select instruments to measure the attribute that are quite dissimilar, yet still match the definition. This is the very situation with processnig speed (Cepeda et aI., 2013). Multiple individuals have created instruments and have used the telTIl tfprocessing speedtf to describe the phenomenon that the scores ostensibly represent. Many of these instruments more-or-Iess conform to Schneider and McGrew's (2018) defmition, yet these instruments require different skill sets. So, their scores are not exchangeable or traceable (Wallard, 2011). In fact, it is probably more the rule than the exception that developers of psychological

General and Specific Intelligence Attributes in the Two-Factor Theory

47

instruments who look to assess non-technically defined attributes construct tasks whose scores are not exchangeable (Floyd et aI., 2005, 2008). In other words, more instances of the jingle fallacy. End Premature Reif lCation

If one's goal is only to predict some outcome (e.g., diagnosis, job perfOlmance), then there is no major problem with adopting Thurstone's instrumentalist philosophy. Here, the LVs merely represent some statistical summary of one or more phenomena and there is no need for any theory about what the LVs represent. One can posit as many LVs or concepts as one thinks is necessary to predict the criteria adequately and develop instruments accordingly. If the scores can be sho\Vll to meet some minimum level of reliability and adequately predict criteria of interest, then that is sufficient evidence for the instrument's usage. The sequelae of a Thurstonian approach is that it precludes any conclusion that the instruments' scores represent attributes that exist outside of its measurement. If one hypothesizes that an attribute in the intelligence sphere exists independently of its measurement and wishes to develop valid procedures for potentially measuring it, then Thurstone's approach is entirely insufficient. Instead, one needs to adopt an approach similar to that of Spearman. Early in his career, Spearman was content that g (as well as Ss and ys) could be represented under some specific circumstances (Norton, 1979; Spearman, 1930b). Consequently, he spent the much of his time investigating whether g represented some statistical chimera or was an actual attribute that could be scientifically explained. His conclusion was that g was likely a "real" attribute and his explanation got so far as his noegenic laws (Spearman, 1930b). What would it look like for individuals currently studying the intelligence fear to take a Spearman-like approach? The starting point for making the case that an attribute in the intelligence sphere exists is the development of an explicit theory of the attribute (Boag, 2015; Maraun, 1998). This involves conceptual analysis of the attribute's theory (e.g., Petocz and Newbery, 2010; Machado and Silva, 2007) In addition, it involves conceptual and empirical investigations of how the attribute is embedded within a greater theoretical system (Hibberd, 2014), which includes being explicit in how the particular attribute relates to other attributes in (and perhaps outside of) the intelligence sphere. This will often involve LVs-either explicitly or implicitly-since intelligence attributes will not typically be directly measurable. As part of this process,

48

Chapter Two

evidence should be gathered about the structure of the attribute (e.g., categorical, ordinal, quantitative) to detelTIline how it should be symbolically represented (e.g., Mencattini and Mari, 2015). A result from this type of approach should be a technical definition of the attribute that is univocally understandable by psychologists who wish to study the attribute. It will likely not be afinal defmition, but it should be one that produces a consistent understanding and measurement of the attribute across investigators, as well as one that allows for further study of the attribute's properties. Unfortunately, too much work in intelligence tries to use a Thurstone­ like philosophy and approach, yet make conclusions as if they had taken the Spearman-like approach. As example is provided by Flanagan, Ortiz, and Alfonso (Flanagan et aI., 2013, pp. 35-38). They presented a set of guidelines for constructing an instrument battery to measure a plethora of broad intelligence attributes. It requires (a) selecting the attributes of interest; (b) administering 2--4 instruments that the authors' indicated measure each of the attributes (they derived this information from a bevy from correlational and LV studies); (c) calculating norm-referenced scores from each of the instruments; and (d) inputting the scores into their proprietary software to obtain values for each of the attributes of interest. The problem with this approach is in their interpretation of the resulting scores as measures of !treal n attributes in the intelligence sphere. They provide no evidence that the LVs from the studies they reviewed are actual attributes with any particular structure, they only provide verbal descriptions of the ostensible attributes, and they explain individual differences on an attribute in telTIlS of the instruments used to measure the attribute. In other words, premature reification.

Conclusion The study of intelligence has a long and interesting history. In this chapter, I described one particular piece of history: Charles Spearman's approach to developing and refining his two-factor theory. While his theory is incomplete, his approach to studying attributes within the intelligence sphere is one that modem researchers would do well to study. If the field is to move forward-focusing either on general or more broad/specific attributes-some common practices need to be re-examined and, likely, discontinued. Understanding SpealTIlan's approach can aid in this process.

General and Specific Intelligence Attributes in the Two-Factor Theory

49

References Aikens, H. A., E. L. Thorndike, and E. Hubbell (1902). Correlations among perceptive and associative processes. Psychological Review 9, 374-382. Anastasi, A. (1938). Faculties versus factors: A reply to Professor Thurstone. Psychological Bulletin 3S, 391-39S. Baddeley, A., M. W. Eysenck, and M. C. Anderson (2014). Memory (2nd ed.). Psychology Press. Ballard, P. B. (1929). Noegenesis. The Journal of General Psychology 2, 433-444. Bartholomew, D. J. (199S). Spearman and the origin and development of factor analysis. British Journal of Mathematical and Statistical Psychology 48, 21 1-220. Baumgardner, S. R. (1977). Critical studies in the history of social psychology. Personality and Social Psychology Bulletin 3, 68 1-687. Beaujean, A. A. and N. F. Benson (in press). Theoretically-consistent cognitive ability test development and score interpretation. Contemporary School Psychology. Block, J. (199S). A contrarian view of the five-factor approach to personality description. Psychological Bulletin 1 17, 187-21S. Block, N. J. (1974). Fictionalism, fimctionalism and factor analysis. In R. S. Cohen, C. A. Hooker & A. C. Michalos (Eds.), Boston studies in the philosophy of science (Vol. 32, pp. 127--141). Dordrecht, the Netherlands: Reidel. Boag, S. (2010). Description and explanation within personality psychology research. In R. Hicks (Ed.), Personality and individual differences: Current directions (pp. 21-29). Queensland, Australia: Australian Academic Press. Boag, S. (2011). Explanation in personality psychology: "Verbal magic" and the five-factor model. Philosophical Psychology 24, 223-243. Boag, S. (201S). Personality assessment, 'construct validity', and the significance of theory. Personality and Individual Differences 84, 3644. Boring, E. G. (1929). A history of experimental psychology. Oxford, England: Appleton-Century. Borsboom, D., G. J. Mellenbergh, and J. van Heerden (2003). The theoretical status of latent variables. Psychological Review 1 10, 203219. Borsboom, D., G. J. Mellenbergh, and J. van Heerden (2004). The concept of validity. Psychological Review 1 1 1, 1061-1071.

50

Chapter Two

Burt, C. L. (1909). Experimental tests of general intelligence. British Journal ofPsychology 3, 94-177. Burt, C. L. ( 1 91 1). Experimental tests of higher mental processes and their relation to general intelligence. Journal of Experimental Pedagogy 1, 93-112. Carr, H. W., Wolf, A., & Spearman, C. E. (1925). Symposium: The nature of intelligence. Proceedings of the Aristotelian Society, Supplementary Volumes, 5, 1-27. Carroll, J. B. (1993). Human cognitive abilities: A survey of factor­ analytic studies. New York, NY: Cambridge University Press. Cattell, R. B. (1988). Psychological theory and scientific method. In J. R. Nesselroade & R. B. Cattell (Eds.), Handbook of multivariate experimental psychology (2nd ed., pp. 3--20). Boston, MA: Springer. Cepeda, Nicholas, J., A. Blackwell, Katharine, and Y. Munakata (2013). Speed isn't everything: Complex processing speed measures mask individual differences and developmental changes in executive control. Developmental Science 16, 269-286. Commins, W. D. (1933). What is "faculty psychology"? Thought: A Journal of Philosophy 8, 48-57. Cox, J. W. (1928). Mechanical aptitude: Its existence, nature and measurement. London, England: Methuen. Danziger, K. (1980). Wundt's psychological experiment in the light of his philosophy of science. Psychological Research 42, 109-122. Davey, C. M. (1926). A comparison of group verbal and pictorial tests of intelligence. British Journal of Psychology. General Section 17, 27-48. Dodd, S. C. (1928). The theory of factors, I. Psychological Review 35, 21 1-234. Factor. (n.d.). ln APA dictionary ofpsychology online. https:lldictionary.apa.orglfactor Fallace, T. D. (201 1). From the German schoolmaster's psychology to the psychology of the child: Evolving rationales for the teaching of history in U.S. Schools in the 1 890s. The Journal of the Gilded Age and Progressive Era 10, 161-186. Fayers, P. M. and D. J. Hand (2002). Causal variables, indicator variables and measurement scales: An example from quality of life. Journal of the Royal Statistical Society. Series A (Statistics in Society) 165, 233261. Ferguson, G. A. (1965). Human abilities. Annual Review of Psychology 16, 39-62. Flanagan, D. P. and V. C. Alfonso (2017). Essentials of WISC-V assessment (2nd ed.). Hoboken, NJ: Wiley.

General and Specific Intelligence Attributes in the Two-Factor Theory

51

Flanagan, D . P., Alfonso, V. c., Sy, M . c., Mascolo, J. T., McDonough, E. M., & Ortiz, S. O. (2018). Dual discrepancy/consistency operational definition of SLD: Integrating multiple data sources and multiple data­ gathering methods. In V. C. Alfonso & D. P. Flanagan (Eds.), Essentials of specific learning disability identification (2nd ed., pp. 329--430). Hoboken, NJ: Wiley. Flanagan, D. P., S. O. Ortiz, and V. C. Alfonso (2013). Essentials of cross­ battery assessment (3rd ed.). Hoboken, NJ: Wiley. Floyd, R. G., R. Bergeron, A. C. McCormack, J. L. Anderson, and G. L. Hargrove-Owens (2005). Are Cattell-Horn-Carroll (CHC) broad ability composite scores exchangeable across batteries? School Psychology Review 34, 329-357. Floyd, R. G., M. H. Clark, and W. R. Shadish (2008). The exchangeability of IQs: Implications for professional psychology. Professional Psychology: Research and Practice 39, 414-423. Flugel, J. C. and D. J. West (1963). A hundred years of psychology (3rd ed.). London, England: Methuen. Galton, F. (1883). Inquiries into human faculty and its development. London, England: MacMillan. Galton, F. (1908). Memories ofmy life. London, England: Methuen. Gerber, J. P. (201 1). Six solutions to the circular nature of trait psychology. In S. Boag & N. Tiliopoulos (Eds.), Personality and individual differences: Theory, assessment, and application (pp. 297-306). New York, NY: Nova. Gopalaswami, M. V. (1924). 'Intelligence' in motor learning. British Journal ofPsychology 14, 274-290. Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum. Gregoire, J. (2013). Measuring components of intelligence: Mission impossible? Journal ofPsychoeducational Assessment 31, 138-147. Haig, B. D. (2013). The philosophy of quantitative methods. In T. D. Little (Ed.), Tlie Oxford handbook of quantitative methods in psychology, Vol. 1 : Foundations (pp. 7--31). New York, NY: Oxford University Press. Hart, B. and C. Spearman (1912). General ability, its existence and nature. British Journal ofPsychology 5, 51-84. Hedman, H. B. (1938). A critical comparison between the solutions of the factor problem offered by Spearman and Thurstone. Journal of Educational Psychology 29, 671-685. Hibberd, F. J. (2014). The metaphysical basis of a process psychology. Journal ofTlieoretical and Philosophical Psychology 34, 161-186.

52

Chapter Two

Hood, S. B. (2013). Psychological measurement and methodological realism. Erkenntnis 78, 739-76l . Horn, J. L. (1985). Remodeling old models of intelligence. In B. B. Wolman (Ed.), Handbook of intelligence (pp. 267--300). New York, NY: Wiley. Hom, J. L. (2008). Spearman, g, expertise, and the nature of human cognitive capability. In P. C. Kyllonan, R. D. Roberts & L. Stankov (Eds.), Extending intelligence: Enhancement and new constructs (pp. 159--194). New York, NY: Erlbaum. Horn, J. L., & McArdle, J. J. (2007). Understanding human intelligence since Spearman. In R. Cudeck & R. C. MacCallum (Eds.), Factor analysis at 100: Historical developments and future directions (pp. 205--247). Mahwah, NJ: Erlbaum. Humphreys, L. G., D. Ilgen, D. McGrath, and R. Montanelli (1969). Capitalization on chance in rotation of factors. Educational and Psychological Measurement 29, 259-27l . Humphreys, L. G., & Stark, S. (2002). General intelligence: Measurement, correlates, and interpretations of the cultural-genetic construct. In R. J. Sternberg & E. L. Grigorenko (Eds.), The general factor of intelligence: How general is it? (pp. 99--128). Mahwah, NJ: Erlbaum. Instinct and intelligence [Special Issue] (1910). British Journal of Psychology 3, 209-270. Intelligence and its measurement: A symposium. (1921). Journal of Educational Psychology 12, 123-147. James, W. (1 890). The principles ofpsychology (Vol. 1). New York, NY: Holt. Jensen, A. R. (1987). Individual differences in mental ability. In J. A. Glover & R. R. Ronning (Eds.), Historicalfoundations of educational psychology (pp. 61--88). New York, NY: Plenum Pres Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger Publishers/Greenwood. Kelley, T. L. (1928). Crossroads in the mind of man: A study of differentiable mental abilities. Stanford, CA: Stanford University Press. Krause, M. S. (2012). Measurement validity is fundamentally a matter of definition, not correlation. Review of General Psychology 16, 391-400. Krueger, F. and C. E. Spearman (1906). Die Korrelation zwischen verschiedenen geistigen Leistungsfahigkeiten. Zeitschrift fur PsychologielJournal ofPsychology 44, 50-114. Lehman, H. C. and P. A. Witty (1934). Faculty psychology and personality traits. The American Journal ofPsychology 46, 486-500.

General and Specific Intelligence Attributes in the Two-Factor Theory

53

Loehlin, J. C. and A. A. Beaujean (2016). Latent variable models: An introduction to factor, path, and structural equation analysis (5th ed.). New York, NY: Routledge. Lovie, A. D. (1983). Aspects of neogenesis: Spearman's system of cognition and applied psychology. In S. Bern, H. Rappard & W. van Hoorm (Eds.), Studies in the history of psychology and the social sciences: Proceedings of the First European Meeting of Cheiron, International Society for the History of the Behavioral and Social Sciences (pp. 120--136). Leiden, The Netherlands: Psychologisch Instituut van de Rijksuniversiteit Leiden. Lovie, A. D. and P. Lovie (1993). Charles Spearman, Cyril Burt, and the origins of factor analysis. Journal of the History of the Behavioral Sciences 29, 308-321. Lovie, P. and A. D. Lovie (1995). The cold equations: Spearman and Wilson on factor indeterminacy. British Journal of Mathematical and Statistical Psychology 48, 237-253. Lovie, P. and A. D. Lovie (1996). Charles Edward Spearman, F.R.S. (1 863-1945). Notes and Records of the Royal Society of London 50, 75-88. Lovie, S. (1983). Images of man in early factor analysis-psychological and philosophical aspects. In S. M. Bern, H. Van Rappard & W. Van Hoom (Eds.), Studies in the history of psychology and the social sciences (pp. 265--247). Leiden, The Netherlands: Leiden University Machado, A. and F. J. Silva (2007). Toward a richer view of the scientific method: The role of conceptual analysis. American Psychologist 62, 671-68 1 . Maraun, M. D., & Gabriel, S . M . (2013). Illegitimate concept equating in the partial fusion of construct validation theory and latent variable modeling. New Ideas in Psychology, 31, 32-42. Maraun, M. D. (1998). Measurement as a normative practice: Implications of Wittgenstein's philosophy for measurement in psychology. Theory & Psychology 8, 435-461. Mascolo, J. T., & Flanagan, D. P. (2016). Case 3--Luke, age 9: A CHC­ based cross-battery assessment case report. In A. S. Kaufman, S. E. Raiford & D. L. Coalson (Ed&), Intelligent testing with the WISC-V (pp. 304--329). Hoboken, NJ: Wiley. McClain, A. L. (1996). Hierarchical analytic methods that yield different perspectives on dynamics: Aids to interpretation. In B. Thompson (Ed.), Advances in social science methodology (Vol. 4, pp. 229--240). Bingley, England: Emerald Group Publishing.

54

Chapter Two

Mencattini, A. and L. Mari (2015). A conceptual framework for concept definition in measurement: The case of 'sensitivity'. Measurement 72, 77-87. Meredith, G. P. (1948). A revision of Spearman's Noegenetic principles. Proceedings ofthe Aristotelian Society 49, 91-114. Mill, J. and J. S. Mill (1 869). Note to analysis of the phenomena of the human mind. London, England: Longmans Green Reader and Dyer. Molenaar, P. C. M. and C. G. Campbell (2009). The new person-specific paradigm in psychology. Current Directions in Psychological Science 18, 1 12-117. Mulaik, S. A. (1986). Factor analysis and Psychometrika: Major developments. Psychometrika 5 1 , 23-33. Mulaik, S. A. (1987). A brief history of the philosophical foundations of exploratory factor analysis. Multivariate BehavioralResearch 22, 267305. Norton, B. (1979). Charles Spearman and the general factor in intelligence: Genesis and interpretation in the light of sociopersonal considerations. Journal of the History of the Behavioral Sciences 15, 142-154. O'Brien, A. R., & Tulsky, D. S. (2008). The history of processing speed and its relationship to intelligence. In J. DeLuca & J. H. Kalmar (Eds.), Information processing speed in clinical populations (pp. 1--28). New York, NY: Taylor & Francis. Oehrn, A. (1 889). Experimentelle Studien zur Individualpsychologie. Dorpat (Tartu), Estonia: H. Laakmann's Buch- und Steindruckerei. Pearl, J. (2014). Interpretation and identification of causal mediation. Psychological Methods 19, 459-48 l . Petocz, A. and G. Newbery (2010). On conceptual analysis as the primary qualitative approach to statistics education research in psychology. Statistics Education Research Journal 9, 123-145. Ree, M. J. and J. A. Earles (1991). The stability of g across different methods of estimation. Intelligence 15, 271-278. Samelson, F. (1974). History, origin myth and ideology: 'Discovery' of social psychology. Journalfor the Theory of Social Behaviour 4, 217232. Schmitt, D. E. (1946). Modern criticism of the Thomistic concept of faculty. M.A. thesis, Loyola University Chicago. Schneider, W. J., & Flanagan, D. P. (2015). The relationship between theories of intelligence and intelligence tests. In S. Goldstein, D. Princiotta & J. A. Naglieri (Eds.), Handbook of intelligence:

General and Specific Intelligence Attributes in the Two-Factor Theory

55

Evolutionary theory, historical perspective, and current concepts (pp. 3 17-340). New York, NY: Springer. Schneider, W. J., & McGrew, K. S. (2018). The Cattell-Horn-Carroll theory of cognitive abilities. In D. P. Flanagan & E. M. McDonough (Eds.), Contemporary intellectual assessment (4tli ed., pp. 73--163). New York, NY: Guilford. Sharp, S. A. (1980). Godfrey Thomson and the concept of intelligence. In J. V. Smith & D. Hamilton (Eds.), The meritocratic intellect: Studies in the history of educational research (pp. 67--78). Aberdeen, Scotland: Aberdeen University Press. Shipley, B. (2016). Cause and correlation in biology: A user's guide to path analysis, structural equations and causal inference with R (2nd ed.). New York, NY: Cambridge University Press. Sleight, W. G. (1911). Memory and formal training. British Journal of Psychology 4, 386-457. Sleight, W. G. (1915). Educational values & methods based on the principles of the training process. Oxford, England: Clarendon Press. Smith, B. B. (1950). Part II: An evaluation of factor analysis from the point of view of a psychologist Journal of the Royal Statistical Society. Series B (Methodological) 12, 73-94. Sokal, M. M. (2001). Practical phrenology as psychological counseling in the 19t1i-century United States. In C. D. Green, M. Shore & T. Teo (Eds.), The transformation of psychology: Influences of 19th-century philosophy, technology, and natural science. Washington, DC: Ameerican Psychological Association. Spearman, C. (1914a). The heredity of abilities. The Eugenics Review 6, 219-237. Spearman, C. E. (1904a). 'General intelligence,' objectively determined and measured. The American Journal ofPsychology 15, 201-293. Spearman, C. E. (1904b). The proof and measurement of association between two things. The American Journal ofPsychology 15, 72-101. Spearman, C. E. (1905a). Die Normaltiiuschungen in der Lagewahrnehmung. Leipzog, Germany: Engelmann. Spearman, C. E. (1905b). Proof and disproof of correlation. The American Journal ofPsychology 16, 228-23 1 . Spearman, C. E. (1914b). The theory of two factors. Psychological Review 21, 101-115. Spearman, C. E. (1925). Some issues in the theory of "g" (including the Law of Diminishing Returns). Nature 1 16, 436-439. Spearman, C. E. (1927a). Material versus abstract factors in correlation. British Journal ofPsychology 17, 322-326.

Chapter Two

56

Spearman, C. E. (1927b). The abilities of man: Their nature and measurement. New York, NY: Blackburn Press. Spearman, C. E. (1927c). The nature of "intelligence" and principles of cognition (2nd ed.). London, England: Macmillan. Spearman, C. E. (1928). A new method for investigating the springs of action. In M. L. Reymert (Ed.), Feelings and emotions: The Wittenberg symposium (pp. 39--48). Worcester, MA: Clark University Press. Spearman, C. E. (1930). Charles Spearman. In C. Murchison (Ed.), A history of psychology in autobiography (Vol. 1, pp. 229--333). Worcester, MA: Clark University Press. Spearman, C. E. (1930) . 'G" and after--a school to end schools. In C. Murchison (Ed.), Psychologies of 1930 (pp. 339--366). Worcester, MA: Clark University Press. Spearman, C. E. (1931). Our need of some science in place of the word 'intelligence'. Journal ofEducational Psychology 22, 401-410. Spearman, C. E. (1933). The factor theory and its troubles. III. Misrepresentation of the theory. Journal of Educational Psychology 24, 591-601 . Spearman, C . E. (1937). Measuring intelligence: A critical notice. The Human Factor 1 1 , 368-370. Spearman, C. E. (1938). Measurement of intelligence. Scientia, Milano 64, 75-82. Spearman, C. E. (1941). Professor Thurstone, a correction. Psychological Bulletin 38, 818-818. Spearman, C. E. (1946). Theory of general factor. British Journal of Psychology 36, 117-131. Spearman, C. E. and L. Wynn Jones (1950). Human ability: A continuation of ''The abilities ofman ". London, England: Macmillan. Steiger, J. H., & SchOnemann, P. H. (1978). A history of factor indeterminacy. In S. Shye (Ed.), Theory construction and data analysis in the behavioral sciences (pp. 136--178). San Francisco, CA: Jossey­ Bass. Stenner, A. J., M. Smith, and D. S. Burdick (1983). Toward a theory of construct definition Journal of Educational Measurement 20, 305316. Stephenson, W. (1977). Factors as operant subjectivity. Operant Subjectivity, 1, 3-16. Sternberg, R. J. and D. K. Detterman (Eds.) (1986). What is intelligence ? Contemporary viewpoint on its nature and definition. Hillsdale, NJ: Ablex. .

General and Specific Intelligence Attributes in the Two-Factor Theory

57

Tanner, J. M. (1947). The morphological level of personality. Proceedings of the Royal Society ofMedicine 40, 301-308. Thomson, G. H. (1916). A hierarchy without a general factor. British Journal ofPsychology 8, 271-281. Thomson, G. H. (1920). General versus group factors in mental activities. Psychological Review 27, 173-190. Thomson, G. H. (1927). The tetrad-difference criterion. British Journal of Psychology 17, 235-255. Thorndike, E. L. (1903). Educational psychology: An introduction to the theory of mental and social measurements. New York, NY: Science Press. Thorndike, E. L. (1904). Introduction to the theory of mental and social measurements. New York, NY. The Science Press. Thorndike, E. L. (1910). Educational psychology: An introduction to the theory of mental and social measurements (2nd ed.). New York, NY: Science Press. Thurstone, L. L. (1931a). Multiple factor analysis. Psychological Review 38, 406-427. Thurstone, L. L. (193 1b). The reliability and validity of tests: Derivation and interpretation offondamental formulae concerned with reliability and validity of tests and illustrative problems. Ann Arbor, MI: Edwards Brotbers. Thurstone, L. L. (1935). The vectors of mind: Multiple-factor analysis for the isolation of primary traits. Chicago, IL: University of Chicago Press. Thurstone, L. L. (1936a). A new conception of intelligence. Tlie Educational Record 17, 441-450. Thurstone, L. L. (1936b). The factorial isolation of primary abilities. Psychometrika 1, 175-182. Thurstone, L. L. (1940). Current issues in factor analysis. Psychological Bulletin 37, 189-236. Thurstone, L. L. (1944). Second-order factors. Psychometrika 9, 71-100. Thurstone, L. L. (1947). Multiple factor analysis: A development and expansion of Tlie Vectors ofMind. Chicago, IL: University of Chicago Press. Thurstone, L. L. (1948). Psychological implications of factor analysis. American Psychologist 3, 402-408. Thurstone, L. L. (1949). Primary abilities. Occupations: The Vocational Guidance Journal 27, 527-529. Thurstone, L. L. (1952). L. L. Thurstone. In E. G. Boring, H. S. Langfeld, H. Werner & R. M. Yerkes (Eds.), A history of psychology in

58

Chapter Two

autobiography (Vol. N, pp. 295--321). Worcester, MA: Clark University Press. Thurstone, L. L. and T. G. Thurstone (1941). Factorial studies of intelligence. Psychometric Monographs 2, 94. van der Maas, H., Kan, K.-I., & Borsboom, D. (2014). Intelligence is what the intelligence test measures. Seriously. Journal ofIntelligence 2, 1215. Vernon, P. E. (1950). The structure of human abilities (2nd ed.). London: Methuen. Wallard, A. (2011). Metrology principles and organization. In H. Czichos, T. Saito & L. Smith (Eds.), Springer handbook of metrology and testing (pp. 23--37). Berlin, Heidelberg: Springer. Webb, E. (1915). Character and intelligence. British Journal of Psychology Monograph Supplement No. 3 1, 1-99. Wechsler, D. (2014). Wechsler Intelligence Scale for Children-Fifth Edition technical and interpretive manual. Bloomington, MN: NCS Pearson. White, S. H. (2000). Conceptual foundations of IQ testing. Psychology, Public Policy, and Law 6, 33-43. Wilson, E. B. (1928). [Review of book The Abilities ofMan, Their Nature and Measurement by C. E. Spearman]. Science 67, 244-248. Wood, D. A. (1962). Louis Leon Tliurstone: Creative thinker, dedicated teacher, eminent psychologist. Princeton, NJ: Educational Testing Service. Woodworth, R. S. and E. L. Thorndike (1901). The influence of improvement in one mental function upon the efficiency of other functions (I). Psychological Review 8, 247-261. Young, K. (1924). The history of mental testing. Pedagogical Seminary 31 , 1-48. Yung, Y.-F., D. Thissen, and L. D. McLeod (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika 64, 1 13-128.

CHAPTER THREE COGNITIVE ABILITY: PSYCHOMETRIC PERSPECTIVES ON THE IMPORTANCE OF GENERAL MENTAL ABILITY KEVIN R. MURPHY

Beginning in the early 1900s, with tests developed by Binet and Simon, Galton, Jastrow and others (Boake, 2002), the measurement of cognitive ability has long been an important area of scientific research and application. Tests that measure a range of cognitive abilities are widely used in making high-stakes decisions, including personnel selection and placement and academic admissions decisions that affect millions of individuals (Frey and Dettennan, 2004; Koenig, Frey and Dettennan, 2008; Ree and Carretta, 1994, 2002; Schmidt and Hunter, 1998; See however Roberts, Goff, Anjoul, Kyllonen, Pallier and Stankov, 2000). Individually-administered tests of cognitive ability are used to help diagnose individual strengths and weaknesses in perfonning tasks that involve active information processing (Murphy and Davidshofer, 2005). Tests of this sort have provided a foundation for theories and models of cognitive ability that continue to be important for both science and practice. The psychometric tradition uses test score patterns and relationships between scores and measures on various tests to develop theories and models of human cognitive ability. These theories and models, as well as the tests arising from this tradition, are driven by a number of considerations, including the types of tasks and items that define these tests, the analytic methods that are used to make sense of test scores and the uses of these tests. As a result of a combination of these three factors, psychometric theories and models of human cognitive ability have developed in a variety of ways that lead to very different measures and

60

Chapter TIrree

models than those that have emerged from other scientific traditions, especially neuropsychological perspectives. In particular, while different tests provide measures of a number of specific cognitive abilities and skills, virtually all of the tests and the theories arising out of this tradition place a strong emphasis on general cognitive ability. In 1905, Binet and Simon developed a highly influential test that they used to assess the intelligence of Paris school children. This test included 30 cognitive tasks, arranged in order of difficulty. These tasks varied widely, some being simple perceptual tasks, but many being verbal comprehension tasks (e.g., defming objects and abstract nouns) and memory tasks (e.g., repeating a list of digits). In many ways, this test set a pattern that was widely followed, creating cognitive ability tests that asked respondents to respond to a number of different types of questions and to perfOlTIl a wide range of mental operations. The Wechsler Intelligence Test for Children (WISC-V; Wechsler, 2014) is a lineal descendant of the original Binet-Simon tests, and its structure reflects features seen in many other cognitive tests, features that have a decisive influence on the types of test theories and theories of cognitive ability that have developed as part of the psychometric tradition. The scales and subtests of the WISC-V are shown in Table 1 . These scales and subtests exemplify a typical test of cognitive ability in that they: (l) sample from a wide array of cognitive, perceptual and memory tasks, (2) include both complex cognitive tasks (e.g., comprehension) and more elementary cognitive operations (e.g., coding), and (3) they include both knowledge items and abstract reasoning items. There are some cognitive ability tests that do not follow this strategy; the Raven's Progressive Matrices (Raven, 2000) test is a widely-respected nonverbal measure of fluid intelligence that is restricted to a single item type that requires respondents to identify patterns in groups of abstract figures. However, the dominant strategy in developing measures of cognitive ability in the psychometric tradition has involved sampling from many types of items, and this sampling strategy, in combination with the analytic methods used to make sense of these tests, has had a substantial impact on the types of theories and models that have arisen out of the psychometric tradition.

How Positive Manifold Shaped Theories of Cognitive Ability By the 1940's, it was widely recognized that scores on cognitive tests that appeared to differ substantially in content were nevertheless almost always positively intercorrelated (Thurstone, 1947). This consistent

Cognitive Ability: Psychometric Perspectives

61

pattern of positive correlations among tests (i.e., positive manifold) has been confirmed in a number of papers and reviews (Ackemmn and Humphreys, 1990; Allinger, 1988; Carroll, 1993; Guttman and Levy,1991; Humphreys, 1979; Jensen, 1980; Ree and Carretta, 2002). In general, virtually any reliable tests that involve active infonnation processing will yield scores that are positively correlated with other infonnation­ processing tests, even if the tests appear to deal with very different content and processes (e.g., paragraph comprehension vs. digit span; general knowledge tests vs. tests of arithmetic). A number of theories of cognitive ability have been developed to explain this pattern of findings. Spearman (1923, 1927) proposed a "two-factor" theory, proposing that all tests that involve active infonnation processing measure two distinct factors: (1) g a general cognitive ability factor, and (2) a test-specific factor, which represented systematic variance in test scores due to the specific content, layout, items types, etc. that characterize that test. Spearman proposed the principle of the indifference of the indicator, according to which the precise content of intelligence tests is unimportant for the purposes of identifying K, because K enters into perfonnance on all kinds of tests. While Spearman's two-factor theory was eventually found to be insufficient to explain the intercorrelations among tests, this theory has had a lasting impact. Many of the theories of cogintive ability emerging out of the cognitive tradition have includes a K factor representing general cognitive ability, and many of the applications of cognitive ability testing have focused substantially (and sometimes entirely) on K. Thurstone (1938) was the first to raise important questions about whether g is a meaningful concept or merely an artifact of the analytic methods used in studying perfonnance on cognitive ability tests. He developed a theory of primary mental abilities (word fluency, verbal comprehension, spatial visualization, number facility, associative memory, reasoning, and perceptual speed), which were originally defined as independent abilities in a model that included no general abilities. This model proved insufficient for explaining the pervasive pattern of positive manifold in cognitive tests, and his later work (Thurstone, 1947; Thurstone and Thurstone, 1941) incorporated both primary mental abilities and a K factor. Modem theories of human cognitive ability are typically described in tenns of a hierarchical structure, with K at the top of the hierarchy and then with more specific abilities described at one or more layers below (Carroll, 1993; Horn, 1985; McGrew, 2005, 2009; Vernon, 2000). Consensus has emerged that the Carroll-Hom-Cattell (CHC) model (Carroll, 1993; -

62

Chapter TIrree

McGrew, 2005; 2009) provides the best representation of the structure of human cognitive ability. This model arrays abilities in three strata, with K at the top, sixteen broad abilities defining the second stratum (these abilities are listed in Table 2) and more than 80 more specific abilities at the third stratum. Table 2 gives a good indication of the breadth of the list of abilities that are subsumed under K. However, it is not necessary to measure all of these abilities to obtain a useful estimate of g It has been argued that a K factor extracted from one test battery will always be the same, within the limits of measurement error, as that extracted from another battery, provided that the batteries are reasonably large and diverse (Mackintosh, 2011).

How Changes in the Landscape of Cognitive Testing Led to Increasing Emphasis on g Cognitive ability tests were originally developed as a diagnostic tool, administered on an individual basis by a trained examiner. These tests were not only used to assess overall levels of proficiency, but also to identify individual strengths and weaknesses. These individual tests are both expensive and time-consuming to administer, but for the first decade or so of cognitive ability measurement, they were the only game in town. 1 In 1917, the US entry into World War I decisively changed the types of tests that were available and the way these tests were used. The United States had a very small army at the point it entered World War I, and it needed to quickly bring recruits into the Army and determine where to place them. The practical need to screen and classify a large number of individuals in a short time led to the development of tests that could be adininistered to groups (including both verbal and nonverbal forms - Army Alpha and Army Beta). The apparent success of these tests spurred the development of comparable tests that could be used in settings ranging from academic assessments to personnel selection. The shift from individual, diagnostic testing to group tests led to a greater emphasis on differences between people rather than differences within people. This evolution of the nature and purpose of testing led to an increasing emphasis on general factors rather than specific ones. The use of this type of measure of general cognitive ability to make high-stakes decisions about individuals soon made clear just how useful such measures could be. 1 Individually administered tests of cognitive ability continue to be used by psychologists, psychiatrists and other trained professionals as diagnostic tools.

Cognitive Ability: Psychometric Perspectives

63

The practical importance of g. There are several reasons why g has turned out to be so important in psychometric tests of cognitive ability. First, it is very difficult to account for the frequently-replicated finding tliat scores on virtually all reliable tests of cognitive abilities are positively intercorrelated (i.e., they show positive manifold) with models that do not include a general factor; in a later section I describe two models that attempt to explain positive manifold without positing a K factor. Second, general ability factors have proved to be consistently useful in a number of contexts in which tests are used to make high-stakes decisions about individuals. Measures of general cognitive ability have been shows to be highly useful in predicting key criteria such as perfonnance in virtually all jobs (Schmidt and Hunter,1985) and academic performance (Jensen, 1988). Arguments for the practical importance of K have been most forcefully made in research on personnel selection and training, where K has emerged as one of the strongest and most consistent predictors of job perfonnance. The effects of general cognitive ability on job performance and job success are so strong and consistent that some researchers (e.g., Ree, Caretta and Teachout, 2015; Ree, Earles, and Teachout, 1994) have argued that specific abilities offer very little incremental infonnation as predictors of these criteria (see also; Ree, Carretta and Earles, 1998; Ree and Earles, 1991, 1992, 1994; Ree, Earles and Teachout, 1994). Many different types of cognitive tests have been used in personnel selection, and there is often considerable attention given to selecting the particular sets of abilities tliat are most clearly related to tlie content of the job. It has long been argued that matching tlie content of selection tests to the content of tlie job was a critical step in establishing tlie validity and relevance of these tests. However, it is not clear that this sort of content matching is eitlier necessary or useful (Murphy, 2009; Murphy, Dzieweczynski and Zhang, 2009). There is a substantial literature showing that the specific content of ability tests is not as important for determining validity as tlie relationship of each test with general mental ability, or g (Carroll, 1993; Hunter and Hirsh, 1987; Jensen, 1980; Ree and Carretta, 2002; Ree and Earles, 1991, 1992; Ree et aI., 1994; Schmidt and Hunter, 1998). Once general cognitive factors have been taken into account, it is rare to find that adding measures of job-specific abilities leads to a substantial change in criterion- related validity (Ree and Carretta, 2002; Ree and Earles, 1991, 1992; Ree et aI., 1994; Schmidt and Hunter, 1998). In addition, there is considerable evidence that the criterion-related validity of cognitive tests, including those that vary widely in content,

64

Chapter TIrree

does not vary greatly across jobs (Hunter and Hirsh, 1987; Ree and Carretta, 2002; Ree and Earles, 1991, 1992; Ree et aI., 1994; Schmidt and Hunter, 1998; Schmidt, Hunter, and Pearlman, 1981). On the whole, this literature has led many personnel selection specialists to question the value of tests of specific cognitive ability, given their limited incremental contribution once general cognitive ability is taken into account. Tests that measure g are particularly useful in situations where it is necessary to rank-order examinees. For example, if 100 individuals apply for 50 jobs, a cognitive ability tests that provides at least an approximate measure of g will be highly useful for identifying the best candidates for the job. Regardless of the specific content of the job itself, tests than measure g will almost always make statistically valid predictions about future job performance (Ree and Earles, 1992; Salgado et aI., 2016; Schmidt and Hunter, 1985). Tests that measure g without also providing useful measures of more specific cognitive abilities are less useful for diagnostic purposes (e.g., trying to understand why a person exhibits low levels of g, identifying individual strengths and weaknesses), but their utility for making important decisions is so well established that this lack of diagnostic information is not seen as a fatal weakness. The Wonderlic Contemporary Cognitive Ability Test (formerly known as Wonderlic Personnel Test) provides a clear example of the factors that lead to so much emphasis on g in personnel testing. It is a short-folTIl cognitive ability test administered with a strict time limit. This test includes a number of questions (practice questions include items such as "If a piece of rope costs 20 cents per two feet, how much rope can you but for 30 dollars?" and "Which of the numbers in this group represents the smallest amount? (a) 0.3, (b) 0.08, (c) 0.33") arranged in order of difficulty) that require active information processing. The test provides a single score (number right) that is widely accepted as a reasonable estimate of g and as a well-validated predictor of perfOlmance in a wide range of jobs (Leverett, Matthews, Lassiter and Bell, 2001; Salgado, Anderson, Moscoso, Bertua, de Fruyt and Rolland, 2016). This brief test is not only used in personnel selection; it is an important component of screening potential players for the National Football League (Zimmerman, 1984). This test: (1) is simple to administer and score, (2) provides useful predictive infOlmation in a matter of minutes, and (3) can be used across an extremely wide range of jobs and settings and is an exemplar of what can be accomplished with a g-centric test. Gottfredson (1997; 2004) sunnnarizes evidence that g not only predicts both job perfOlmance and academic success, but also predicts a number of other measures of life success (e.g., physical well-being, socioeconomic

Cognitive Ability: Psychometric Perspectives

65

success, managing the demands of daily life, avoiding accidents). She argues (Gottfredson, 1986) that differences in general cognitive ability profoundly limit individuals' opportunity and likelihood of achieving higher levels of occupational status, as well as their ability to perform well in complex and demanding jobs. She argues further tliat racial and ethnic differences in occupational attainment are largely a function of differences in average levels of g and are therefore very difficult to change.

The Role of Factor Analysis in Developing Theories and Models of Cognitive Ability Factor analysis has been critically important to the development of psychometric theories of cognitive ability. Pioneered by Speannan, the use of factor analysis as a statistical tool for developing theories of human cognitive ability has had a decisive effect on the structure of those theories and even on the structure of tests that are used to measure cognitive ability. This method was critical in tlie development of theories tliat posited multiple abilities (e.g., Thurstone's Primary Mental Ability theory) and it continues to serve a highly important role in psychometric theories of cognitive ability. For example, Carro ll's (1993) monumental work on the structure of cognitive ability is based on a series of factor analyses, and factor analysis forms the basis of the Carroll-Horn-Cattell (CHC) model, the most widely accepted model of the structure of cognitive ability (McGrew, 2009). Factor analysis is a blanket term that applies to a fairly wide range of analytic techniques, all of which attempt to explain why scores on tests or assessments show particular patterns of intercorrelation. It is a statistical tool that can be useful in making inferences about what particular groups of tests or assessments have in common, and it has been widely used in the psychometric tradition to draw conclusions about common factors (e.g., primary mental abilities) that can explain why tests are intercorrelated and why some tests are more highly intercorrelated than others. The 1930s - 1960s represent the heyday of traditional factor analysis as a tool for understanding cognitive ability. Virtually all of the important theories during this period (e.g., Two-Factor theory, Primary Mental Ability Theory, Cattell's (1971) explication of fluid vs. crystallized intelligence, Vernon's hierarchical theory) were based primarily on factor analyses of ability data. Factor analysis, as conducted during this period, can usually be characterized as a variation on principal components analysis (Gorsuch, 2014). Principal components analysis is a method of orthogonal transfOlmation that creates linear combinations of variables

66

Chapter TIrree

such that a group of k variables is transfOlmed into k linear composites (principal components) that are: (1) orthogonal, and (2) efficient, in the sense that the first principal component accounts for the maximum amount of variance possible, the second accounts for the maximum amount of variance, subject to the limitation that it is orthogonal to the first, the third accounts for the maximum amount of variance, subject to the limitation that it is orthogonal to the first two, and so on. Principal components analysis is a simple transfonnation in which all of the information in the original set of k variables is retained in the k principal components. Factor analysis differs from principal components analysis in that it deals only with common variance. In factor-analytic models, it is assumed that the variance in any particular variable can be separated into two orthogonal components: (1) common variance - variance that is shared with other variables in the analysis, and (2) unique variance - a combination of random measurement error and variability in scores that is due to unique features of that particular test or measure. In its simplest sense, factor analysis (as practiced in the 1930s -1960s) can be thought of as an application of principal components analysis to the common parts of each of the k variables included in the analysis. These factors, then are used to explain the patterns of intercorrelation among the variables included in a factor analysis.

In factor analysis, the number of factors needed to explain all of the common variance in a set of k variables is usually less than k, and sometimes substantially less. One of the challenges faced by factor analytic researchers is to detelTIline whether they have extracted the correct number of common factors. In matrix telTIls, it is common to define factor analysis in telTIlS of: �

=

FPF' + U

[1]

Where: :E =

variance-covariance matrix for k variables Factor pattern matrix, a k x m matrix containing the regression weights linking each of the k variables to each of the m common factors P = matrix of correlations among the factors U = diagonal matrix containing the unique variance of each variable on its diagonal

F =

Cognitive Ability: Psychometric Perspectives

67

Equation 1 is very useful in explaining the principal flaws in the type of factor analysis used in the development and testing of psychometric theories of cognitive ability. Factor-analytic methods have evolved significantly over time, particularly with the development and popularization of confimmtory factor analysis (Joreskog, 1969), which allowed researchers to test the adequacy of a priori factor models rather than letting the computer search for the optimal model in the sample at hand (exploratory factor analysis). However, during the heyday of factor-analytic approaches to developing theories of cognitive ability, the methods that were most widely used to draw inferences about the latent constructs that underlie observed measures of cognitive ability shared two serious, if not fatal flaws, notably: (1) indeterminacy and (2) subjectivity.

Indeterminacy The indetelTIlinacy of factor analysis has long been well knO\vn (McDonald, 1977; Steiger, 1994; Steiger and SchOnemann, 1978). There are many facets of this problem, but two stand out as especially worrisome. First, for any observed variance-covariance matrix, there are an infinite number of variations of F that satisfy Equation 1. The most common methods of factor analysis, in particular those based on principal components tend to produce (much like principal components analysis) a first factor that it as highly correlated with as many variables as possible, a second factor that is as highly correlated with as many variables as possible given that it is orthogonal to the first, and so on. However, it is well known that the set of factors defined in this way can be transfolTIled, via the process of factor rotation 2, to create new F matrix that explains exactly the same variance in the observed variables and exactly as much about the relationships among these variables as the original factor pattern matrices derived through applying principal components analysis to the common parts in a set of k variables. There is effectively an infinite number of F matrices that will satisfy Equation 1, and the choice among these is often complex and subjective.

2 Early methods of factor rotation did very much what the name implies, by

plotting factors on graph paper then physically rotating the axes of these plots to pass through apparent clusters of variables. Later methods used a range of analytic methods and data transformations to search for the best, most interpretable variation on the F matrix

68

Chapter TIrree

Second, there is no a priori way of coming to a unique defmition of U. Factor analysis depends on separating common from unique variance, but there is no good way of detelTIlining what proportion of the variance in a particular variable is due to common factors and what variance is unique. In theory, factor analysis is not even fOlmally possible, since the only good way of making a separation of common and unique variance is to do a factor analysis (if F is known, U � V FPF', where V is a diagonal matrix with the variances of the k variables on the diagonal). The practical solution to this problem is an iterative approximation, in which a preliminary factor solution is used to approximate U and in which that approximation is then used in a second iteration, a process that continues until the fit between the factor model and the observed variance­ covariance matrix among the tests being analyzed stops improving. There is no guarantee, however, that this iterative procedure will arrive at the correct solution for U or that the results of this procedure will replicate. The net effect of the indeterminacy of both F and U is that there is no guarantee that factor analysis will arrive at the same solution or at a correct solution to identifying the common factors that underlie the relationships between the k tests that are factor analyzed. Factor analysis can yield useful insights, but the procedure, especially as operationalized in the 1930s - 1960s is a mathematical mess. -

Subjectivity There are two key decisions in factor analysis, determining the number of common factors needed to account for the relationships among the k variables and making sense of the meaning of the factors. During the heyday of factor analysis, a period during which many of the key theories and models of cognitive ability were developed, both of these decisions were highly subjective. For example, the methods most widely used to detelTIline the number of common factors during this period ranged from rules of thumb that were perhaps appropriate for principal components analysis (eigenvalue less than one rule) but completely unjustified for factor analysis to subjective examinations of scree plots (plots of the decline in eigenvalues as more factors are extracted). Hom's (1965) parallel analysis represented an improvement over these methods, but even with this method, it can be difficult to be certain that a correct decision has been made regarding the number of factors needed. Second, factor pattern matrices and factor structure matrices (matrices of correlations between variables and factors) are often inspected to try and draw conclusions about the meaning and interpretation of common factors. For example, if a

Cognitive Ability: Psychometric Perspectives

69

number of variables that involve reading and vocabulary are al strongly related to a particular factor, that might lead to the inference that this factor represents verbal ability. Various rules of thumb have been used to detelTIline which factors particular variables load on, with very little justification of any of these. Earlier, I noted that factor rotation was developed and used to try and make sense of the meaning of factors. 'While rotation can be helpful, it can often be difficult to make a clear choice among the many different rotations that are available, all of which fully satisfy Equation 1 . As a result, different investigators might choose completely different rotations as providing the best factor solution. The problem is compounded by the fact that the common factors that are obtained by applying principal components analysis to the common part of each variable produce factors that are completely orthogonal. While orthogonal factors are often convenient in a statistical sense, it if unlikely that the latent variables that defme human cognition are all mutually independent. During the period when most factor-analytic theories and models of cognitive ability were being developed, techniques such as oblique rotation (which allows factors to be intercorrelated) were often used, and there was often considerable subjectivity involved in decisions about how highly factors should correlate.

From exploratory to confirmatory factor analysis The shift from the type of exploratory factor analysis described in previous sections to confilTIlatory factor analysis started in earnest in the 1970s, and it is increasingly likely that factor analyses encountered in contemporary research will be confnmatory rather than exploratory in nature. Confnmatory factor analysis solves many of the problems inherent in the types of factor analysis used in the 1930s -1960s. First, because confilTIlatory factor analysis requires researchers to specify a priori the number of factors, which variables relate to which factors, and the correlations among the factors, there is no need for scree tests, parallel analyses, factor rotation and the like. However, confilTIlatory factor analysis does not fully solve all of the problems noted in my discussion of exploratory factor analysis. Confirmatory factor analysis provides an indication of whether or not some a priori hypothesis or theory about the relationships between variables and factors and the interrelationships among factors fits the data. At best, this type of analysis can tell you whether the proposed factor model is plausible given the data. It cannot tell you whether the proposed

70

Chapter TIrree

factor model is the correct one. Even if the model proposed does fit the data, there is no guarantee that any number of alternative models will not fit as well or even better. That is, a good fitting model in confirmatory factor analysis does not tell you that you are correct in your thinking about the underlying factors, merely that your thinking is not demonstrably incorrect. Thus, confimmtory factor analysis has not fully solved the problems of indetelTIlinacy. Confirmatory factor analysis is highly useful for pitting competing models against one another, particularly when these models are nested (i.e., a more complex model contains all of the linkages and parameters specified for a simpler version plus some additional specification). However, even here, there are limits to what can be accomplished with a confimmtory analysis. These analyses may demonstrate, within the sample being analyzed, whether complex models of simpler version of these are preferable, but they still do not address the question of whether the proposed model is the right one. These methods only produce a definitive answer when there is a poor fit between a proposed model and the data. This outcome can give you a pretty good indication that your model is wrong. Finding a good fit between your model and the data is a very good first step, but it does not rule out the possibility that there are a number of other qualitatively different alternated that are just as good or better.

Is g an Artifact of the Factor Analytic Method? 'When a set of variables shows positive manifold, virtually any method of factor analysis is likely to yield a general factor. Controversy has raged over whether g is the cause of positive manifold or merely a symptom. For example, Van der Maas, Kan and Boorsboom (2014) claim that g is simply an index variable (i.e., the sum of scores on several items) and that it plays no causal role whatsoever. They argue that positive manifold can be explained by a mutualism model (Van der Maas, Dolan, Grassman, Wicherts, Huizenga and Raijmakers, 2006), in which the interactions of separate abilities and developmental processes create the appearance of a general factor. Kovacs and Conway (2016) propose a quite different theory that explains the general factor through the common executive functions (e.g., working memory) that underlie the perfOlmance of various cognitive tasks. This theory suggests that rather than being a single cause that underlies the correlation among tests, g is the result of a limited number of executive processes that span a wide range of tests. They cite extensive evidence, ranging from neuroimaging studies to assessments of the types of tests (e.g., complex information-processing tasks) that are

Cognitive Ability: Psychometric Perspectives

71

most likely to show high g loadings. They note many similarities between their process overlap theory and the mutualism theory articulated by Van der Maas et al. (2006). Both tbe process overlap theory and tbe mutualism theory suggest that g only appears to be important, but that it is in fact an artifact of the analytic methods that dominated tbe psychometric landscape under which g-based theories of human cognitive ability emerged. Theories tbat suggest that g is an important and meaningful construct have argued forcefully against tbe claim that g is notbing more than a statistical artifact. Gottfredson (1986, 1997, 1998, 2004) makes a persuasive case the general cognitive ability is a strong and consistent predictor of many facets of life success, ranging from academic and occupational achievement to physical and mental health, and that this pervasive pattern of validity would be difficulty to explain if g was simply a statistical artifact. For example, she points to the substantial real-world implications of a different levels of intelligence, ranging from the likelihood of bearing illegitimate children to dropping out of high school to ending a marriage in divorce. If g is an artifact, it is a persistent one. Jensen (1986, 1998) has devoted a substantial portion of his career to exploring the meaning and implications of g. He, like many before his (e.g., Spearman) consider one of the main tasks of an adequate theory of cognitive ability to explain positive manifold. In particular, he has been concerned with explaining the persistent pattern of correlation among tests that seem to share little content in common and the do not even seem to draw heavily fOlTIl the same sets of executive functions (e.g., digit span backwards and complex reading comprehension tests). He notes that there are several factors that argue for the legitimacy of g as a causal construct, including the invariance of g (i.e., almost any set of tasks that require active information processing will lead to the same g), the extensive network of external correlates of g (summarized earlier by Gottfredson), the high level of heritability of g and the relationship between g and speed of mental processing.

Does it matter if g is real or artifactual? Both proponents of the reality and meaningfuhiess of g and proponents of the theory that g is a simple artifact of the factor analytic method present impressive arrays of evidence in favor of their preferred interpretation. In many ways, however, this argument is almost entirely academic. There are very good reasons to act as if g is meaningful and important; it holds a unique place among all psychological constructs in terms of its ability to predict a wide range of important outcomes. If it turned out that g was not

72

Chapter TIrree

a meaningful construct and was simply an index variable that predicted a number of important outcomes, it is unclear that we would do anything differently. Consider, for example, Van der Maas et al.'s (2014) claim that g is simply an index variable. Suppose this is true, and g is nothing more than an index that emerges as a weighted sum of scores on several different cognitive tasks. This view regarding the nature of g has surprisingly few implications with regard to the ways test scores are used to make decisions about individuals. We would still find it highly useful to compute this index and to use it to predict a wide array of important life outcomes. For example, g is regarded virtually unique among the many predictors of job perfOlmance, in the sense that it predicts perfOlmance in all jobs and shows levels of validity that are comparable to the best available predictor in any particular job (Sclimidt and Hunter, 1988). Even if we came to agree that g was simply an index variable, it is hard to see any plausible replacement for g as a predictor that cuts across all settings. Van der Maas et al. (2014) suggest that if g is simply an index variable, then the search for biological correlates of g is pointless, but this is true only in a very limited sense; Jensen (1986,1988) summarized evidence that g has distinct biological correlates. Regardless of what one believes about the status of g as a causal variable, theories of cognitive ability that include a general factor are likely to retain their importance because of the unique value of g as a variable for predicting human behavior in such a wide range of settings. Rather than arguing over whether or not g is a meaningful construct, it might be useful to examine the situations in which factors other than g are necessary or useful in explaining how individuals respond to tasks that require active infOlmation processing. Personnel psychologists have probably been among the most enthusiastic proponents of g-centric approaches, but even these researchers have come to accept that exclusively g-centric approaches have the potential to blind researchers and practitioners to the multidimensional nature of cognitive ability (Reeve, Scherbaum and Goldstein, 2015) and to limit our understanding of how people translate their abilities into effective performance on the job. Several recent papers (e.g., Judge and Kammeyer, 2012; Lievens and Reeve, 2012; Reeve et al., 2015; Schneider and Newman, 2015) have argued that there are advantages to considering measures of specific constructs as predictors and as explanatory mechanism, even in contexts where more general constructs account for a substantial amount of variance in important criteria, or where the correlations between different abilities (e.g., GMA vs. Gflwd) are high.

Cognitive Ability: Psychometric Perspectives

73

Theories or testing models without g are not likely to gain a great deal of traction among psychologists dealing with problems of personnel selection and assessment, but a sole focus on g also seems to be losing its luster in this field. The same conclusion is likely to hold in virtually any field where cognitive measures are used to make high-stakes decisions about large numbers of individuals (e.g., academic admissions, personnel selection and classification). The distinctive features of the psychometric approach to developing theories of human cognitive ability, particularly theories that give precedence or that devote substantial attention to g are probably a function of three features of this tradition. First, cognitive tests, from the start have been created by sampling a wide range of infOlmation processing tasks, and this sampling leads to g factors that are consistent from tests to test. Second, factor analyses of this type of test are virtually certain to produce general factors, even if tests are designed to measure separate cognitive abilities. Third, a test that yields a single overall score for each examinee is exactly what is needed in settings like personnel selection and academic admissions, where there are often many more candidates than there are positions to be filled, and a valid means of rank-ordering candidates is needed. This does not mean that measures of specific abilities are not useful or important, but it does mean that any discussion of cognitive ability in settings of this sort is likely to require careful consideration of measures of general mental ability.

References Ackerman, P. L., and L.G. Humphreys. 1990. Individual differences theory in industrial and organizational psychology. In Handbook of industrial and organizational psychology (Vol. 1, 2nd ed.), edited by M. Durmette and L. Hough, 223-282. Palo Alto, CA: Consulting Psychologists Press. Allinger, G. M. 1988. Do zero correlations really exist among measures of different cognitive abilities? Educational and Psychological Measurement 48: 275-280. Boake, C. 2002. From the Binet-Simon to the Wechsler-Bellevue: Tracing the history of intelligence testing. Journal of Clinical and Experimental Neuropsychology, 24: 383- 405. Carroll, J. B. 1993. Human cognitive abilities: A survey offactor-analytic studies. Cambridge, UK: Cambridge University Press. Cattell, R. B. 1963. Theory of fluid and crystallized intelligence: A critical experiment. Journal ofEducational Psychology, 54: 1-22.

74

Chapter TIrree

Frey, M.C. and D.K. Detterman. 2004. Scholastic assessment or g? The relationship between the scholastic assessment test and general cognitive ability. Psychological Science, 15: 373-378. Gorsuch, RL. 2014. Factor Analysis: Classic Edition. New York: Routledge. Gottfredson, L.S. 1986. Societal consequences of the g factor in employment. Journal of Vocational Behavior, 29: 379-410. Gottfredson, L. S. 1997. Why g matters: The complexity of everyday life. Intellir;ence, 24: 79-132. Gottfredson, L.S. 1998. The general intelligence factor. Scientific American, 9: 24-30. Gottfredson, L.S. 2004. Life, death, and intelligence. Journal of Cognitive Education and Psychology, 4: 23-46 Guttman, L., and S. Levy. 1991. Two structural laws for intelligence tests. Intelligence, 15: 79 -103. Horn, J.L. 1965. "A rationale and test for the number of factors in factor analysis" Psychometrika, 30: 179-185. Horn, J. L. 1985. Remodeling old models of intelligence. In Handbook of intelligence: Theories, measurements and Applications, edited by B. Wolman New York: Wiley. Hunter, J. E., and H.R Hirsh. 1987. Applications of meta-analysis. International review of Industrial and Organizational Psychology, edited by C. L. Cooper and I. T. Robertson, 321-357. Chichester, United Kingdom:Wiley. Koenig, K.A., M.C.,Frey and D.K. Detterman. 2008. ACT and general cognitive ability. Intelligence, 36: 153-160. Kovacs, K. and RA. Conway. 2016. Process overlap tbeory: A unified account of the general factor of intelligence. Psychological Inquiry, 27: 151-177. Jensen, A. R. 1986. g: Artifact or reality? Journal of Vocational Behavior, 29: 301-331. Jensen, A.R 1998. The g factor: The science of mental ability. Westport, CT: Praeger Joreskog, K. G. 1969. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34: 1 83-202. Judge, T.A., and J.D. Kammeyer-Mueller. 2012. General and specific measures in organizational behavior research: Considerations, examples, and recommendations for researchers. Joumal of Organizational Behavior, 33: 161-174 Lievens, F., and C.L. Reeve. 2012. Where 1-0 psychology should really (re)star! its investigation of intelligence constructs and their

Cognitive Ability: Psychometric Perspectives

75

Measurement. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5: 153-158. Leverett, J. P., TD. Matthews, K.S. Lassiter and N.L. Bell, N. L. 2001. Validity comparison of the General Ability Measure for Adults with the Wonderlic Personnel Test. North American Journal 0/ Psychology, 3: 173-182, McDonald, R. P. 1977. The indeterminacy of components and the definition of common factors. British Journal o/Mathematical and Statistical Psychology, 30: 165-176. Mackintosh, N.J. 201 1 . IQ and Human Intelligence. Oxford, UK: Oxford University Press. McGrew, K. S. 2005. The Cattell-Hom-Carroll Theory of cognitive abilities: Past, present, and future. In Contemporary intellectual assessment: Theories, test and issues (2nd ed.), edited by D. P. Flanagan and P. L. Harrison, 136- 1 8 1 . New York, NY: Guilford Press. McGrew, K. S. 2009. CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37: 1-10. Murphy, K.R. 1996. Individual differences and behavior in organizations. San Francisco: Jossey-Bass. Murphy, K.R. 2009. Content validation is useful for many things, but validity isn't one of them. Industrial and Organizational Psychology: Perspectives on Science and Practice, 2: 453-464. Murphy, K.R, and C. Davidshofer. 2005. Psychological testing: Principles and applications (6th Ed). Upper Sadddle River, NJ: Prentice Hall. Murphy, K.R., J.L., Dzieweczynski and Y. Zhang. 2009. Positive manifold limits the relevance of content-matching strategies for validating selection test batteries. Journal 0/Applied Psychology, 94: 1018-1031. Raven, J. 2000. Psychometrics, cognitive ability, and occupational performance. Review o/Psychology, 7: 51-74. Ree, M. J. and T.R. Carretta 1994. Factor analysis of ASVAB: Confirming a Vernon-like structure. Educational and Psychological Measurement, 54: 457-461 . Ree, M. J., and T.R. Carretta. 2002. G2k. Human Performance, 15: 3-23. Ree, M. J., T.R. Carretta and J.A. Earles. 1998. In top-down decisions, weighting variables does not matter: A consequence of Wilks' theorem. Organizational Research Methods, 1 : 407- 420. Ree, M. J., and J.A. Earles. 1992. Intelligence is the best predictor of job perfOlmance. Current Directions in Psychological Science, 1 : 86 - 89.

76

Chapter TIrree

Ree, M. J., and J.A. Earles. 1994. The ubiquitous predictiveness of g. In Personnel selection and classification, edited by M. G. Rumsey, C. B., Walker, V. and I. H. Harris,127-136. Hillsdale, NJ: Erlbaum. Ree, M. I., I.A. Earles and M.S. Teachout. 1994. "Predicting job performance: Not much more than g" Journal ofApplied Psychology, 79: 5 1 8 -524. Reeve, C.L. C. Scherbaum and H. Goldstein. 2015.Manifestations of intelligence: Expanding the measurement space to reconsider specific cognitive abilities. Human Resource ManagementReview, 25: 28-37 Roberts, R.D., G.N. Goff, F. Anjoul, P.C. Kyllonen, G Pallier and L. Stankov. 2000. The Armed Services Vocational Aptitude Battery (ASVAB): Little more than acculturated leamnig (Gc) !? Learning and Individual Differences, 12: 81-103. Salgado, J. F., N. Anderson, S. Moscoso, C. Bertua, F. de Fruyt and J.P. Rolland. 2016. A meta-analytic study of general mental ability validity for different occupations in the European community. In Work and organisational psychology: Research methodology; Assessment and selection; Organisational change and development; Human resource and peiformance management; Emerging trends: Innovation/ globalisationltechnology, edited by J. Boyle, J. G. O'Gorman and G. J. Fogarty, 5 1 -79. Thousand Oaks, CA: Sage Schmidt, F. L., and I.E. Hunter .1 998. The validity and utility of selection methods III personnel psychology: Practical and theoretical implications of 85 years of research findnigs. Psychological Bulletin, 124: 262-274. Schmidt, F. L., I.E. Hunter and K. Pearlman. 1981. Task differences as moderators of aptitude test validity ni selection: A red herring. Journal ofAppliedPsychology, 66: 166 -185. Schneider, W. J., and D.A. Newman. 2015. Intelligence is multidimensional: Theoretical review and implications of specific cognitive abilities. Human Resource Management Review, 25: 12-27. Spearman, C. 1923. The nature of "intelligence " and principles of cognition. London: Macmillan. Spearman, C. 1927. The abilities of man. New York: Macmillan. Steiger, I. H. 1994. Factor analysis in the 1980s and 1990s. Some old debates and some new developments. In Trends and Perspectives in Empirical Social Research, edited by I. Borg and P.P. Mohler, 201224. Berlin: Walter de Grulyter Steiger, I. H., and P.H. SchOnemarm. 1978. A history of factor nidetermniacy. In Theory construction and data analysis, edited by S. Shye, 136-178. CIncago: University of Chicago Press.

Cognitive Ability: Psychometric Perspectives

77

Thurstone, L. L. 1938. Primary mental abilities. Chicago: University of Chicago Press. Thurstone, L. L. 1947. Multiple-Factor Analysis. Chicago: University of Chicago Press. Thurstone, L. L., and T.G. Thurstone .1941. Factorial studies of intelligence. Chicago: University of Chicago Press. Van der Maas, H.L.I, C.V. Dolan, R.P.P.P Grassman, I.M., Wicherts, H.M Huizenga and M.E.I Raijmakers. 2006. A dynamical model of general intelligence: The positive manifold of intelligence by mutualism. Psychological Review, 113: 842-861. Van der Maas, H.L.I, K. Kan and D. Boarsboom. 2014. Intelligence is what the nitelligence test measures. Seriously. Journal of Intelligence, 2: 12-15. Vernon, P. E. 1950. The structure ofhuman abilities. London: Metliuen. Wechsler, D. 2014. Wechsler intelligence scale for children-fifth edition. Bloomington, MN: Pearson. Zimmerman, P. 1984. The New Thinking Man's Guide to Pro Football. New Yark: Simon and Schuster.

78

Chapter TIrree

Table 1 WISC-V Scales and Scores

In addition to an overall score (Full Scale IQ), the WISC-V provides scores for

Verbal Comprehension Index Similarities- asks how two words or ideas are similar Vocabulary Information - general knowledge questions Comprehension Visual Spatial Index Block design - colored blocks are arranged to replicate a picture Visual puzzles FluidReasoning Index Matrix reasoning - children are shO\vn an array of pictures with one missing square, and are asked to identify the missing item from a list Figure weights - children are shown picture of a scale with one empty side and must detelTIline how to balance the scale Picture concepts - children are shO\vn a series of pictures and are asked which go together Arithmetic - orally administered math problems Working Memory Index Digit span - repeat strings of numbers Picture span - view sets of pictures and recall their order Letter-number sequencing - children are asked to repeat letters and numbers in sequence from memory Perceptual Speed Index Coding Symbol search Cancellation - children scan sets of pictures and mark targeted pictures in a limited amount of time

Cognitive Ability: Psychometric Perspectives Table 2 Second Stratum Abilities in the Carroll-Horn-Cattell Model

Gf Gc Gsm Gv Ga Gir Gs Gt Grw Gq Gkn Gh Gk Go Gp Gps

Fluid Reasoning Comprehension/Knowledge Short-Term Memory Visual Processing Auditory Processing Long-Term Storage and Retrieval Cognitive Processing Speed Decision and Reaction Speed Reading and Writing Quantitative Reasoning General (domain-specific) Knowledge Tactile Abilities Kinesthetic Abilities Olfactory Abilities Psychomotor Abilities Psychomotor Speed

79

CHAPTER FOUR PSYCHOMETRIC ISSUES PERTAINING TO THE MEASUREMENT OF SPECIFIC BROAD AND NARROW INTELLECTUAL ABILITIES KARA M. STYCK

For over a century since Spearman (1904) developed a way to estimate intelligence, psychologists have transitioned away from asking the question, "How intelligent are you?" to asking the question, "In what ways are you intelligent?" Contemporary theories of intelligence have replaced Speannan's g with arrangements of specific broad and narrow cognitive abilities containing little (if any) acknowledgment of a global trait (Das, Naglieri, and Kirby 1994; Gardner 2006; Guilford 1972; Horn 1968; Horn and Noll 1997; Kovacs and Conway 2016; Sclineider and McGrew 2012, 2018; Sternberg 2018). However, a series of assumptions are made when attempting to answer the latter question. First, it is assumed that intelligence is multidimensional-a commonly accepted hypothesis amongst researchers and practicing psychologists. Second, it is assumed that specific broad and narrow cognitive abilities can be measured with precIsIOn. The degree to which intelligence is multidimensional is not disputed within this chapter. However, the tenability of the second assumption has become increasingly weak as methodological advancements have exposed psychometric issues pertaining to the measurement of a multidimensional trait (Brunner, Nagy, and Wilhelm 2012; Bulut, Davison, and Rodriguez 2017; Bonifay et al. 2015; Rodriguez, Reise, and Haviland 2016a, 2016b; Reise, Bonifay, and Haviland 2013; Reise et al. 2013). Moreover, if a trait is truly multidimensional (and not unitary), then it cannot be isolated and measured (Michell 1997, 2003). The purpose of this chapter is to summarize what is currently known about how specific broad and narrow intellectual abilities are estimated from factor analytic studies in order to expose issues pertatining to their measurement. It is argued herein that

Psychometric Issues with Specific Abilities

81

although psychologists wish to answer the question, "In what ways are you intelligent?" modern intelligence instruments are only equipped to answer the question, "How intelligent are you (in comparison to your peers)?" In this sense, specific broad and narrow abilities are not so much measured, per se, as they are assessed as heterogeneous orders (Michell 2012). A few things should be clarified prior to making this argument. First, intelligence is not a scientific construct. The definition of intelligence varies widely depending upon who is being asked to define it and the context in which it is being defined (Sternberg et al. 1981). Along these lines, the telTIl "g" is often mistakenly used interchangeably to reference both the superordinate factor in higher-order models and the breadth factor in bifactor models described in detail later within this chapter. Spearman (1933) clarified that the two factors, g and s, within Two-Factor Theory (Spearman 1927) reference a global intellectual ability (g) and everything else that explains perfolTIlance on "all examinations in the different sensory school, or other specific intellectual faculties" (Spearman 1904, p. 272) that is uncorrelated with g (s). This includes residual task specificity and measurement error in the simple case as well as group factors derived from the overlap of specific ability factors hypothesized to represent "broad" and "narrow" intellectual abilities (Carroll 2012) in the complex case. References made to g within this chapter refer to the global intellectual ability estimated by Spearman (1927) that is uncorrelated with s. References made to s (or specific abilities) within this chapter refer to the complex case wherein specific ability factors overlap and fOlTIl broad and narrow intellectual ability group factors. Another issue in need of clarification regards the inherent limitations of using factor analysis to study intelligence. Factor analysis is applied to cross-sectional data in the construction of most commercially available intelligence test batteries in accordance with classical test theory and results of factor analytic studies are used to infolTIl inferences about the intelligence of individuals. However, people are not ergodic. Two conditions are required for ergodicity (using the average behavior of a group to make predictions about the average behavior of an individual): (a) all individuals in the group must be identical and (b) all individuals in the group must remain the same over time. Consequently, factor analysis may not be the best means for studying intelligence and interpretation of factor analytic studies (or intelligence test scores constructed on the basis of classical test theory) as evidence of intelligence for individuals should be appropriately limited in scope.

82

Chapter Four

Theoretical Models of Intelligence Numerous texts have been devoted to chronicling the development of intelligence theories (e.g., Carroll 1993; McGrew 2005; Thorndike and Lohman 1990; Thorndike 1990, 1997) and a detailed overview of the history of intelligence testing will not be repeated in this chapter. Nevertheless, theoretical models of intelligence are inextricably linked to the estimation of intellectual abilities and some discussion of theory is necessary prior to discussing psychometric issues pertaining to the measurement of specific abilities. Carroll (2003) classified theories of intelligence derived from factor analysis as primarily comprising one of ! three perspectives : (a) a standard multifactorial view, (b) a second­ stratum multiplicity view, and (c) a limited structural analysis view. The standard multifactorial view includes theories of intelligence that acknowledge "the existence of a general factor and of a series of nOll­ general 'broad' factors that together contribute variance to a wide variety of mental performances" (Carroll 2003, p. 7). Theories of intelligence that share this perspective include, most notably, the Two-Factor theory of intelligence (Spearman 1927) and the Three-Stratum Theory of Intelligence (Carroll 2012) as examples. The second-stratum multiplicity view includes theories of intelligence that contend that "there is 'no such thing' (as Hom likes to phrase it) as a general factor, but that non-zero inter-correlations among lower-stratum factors can be explained by accepting the existence of two or more second­ stratum factors, mainly Gf and Gc" (Carroll 2003, p. 7). Models of intelligence that share this perspective include the CHC model of intelligence (Sclineider and McGrew 2018), which adopted both the specific abilities outlined in Gf-Gc theory (Hom 1968; Hom and Noll 1997) and the structure of Three-Stratum Theory (Carroll 2012), the Gf-Gc model of intelligence (Hom 1968; Hom and Noll 1997), the planning attention simultaneous successive (pASS; Das, Naglieri, and Kirby 1994) model of cognitive processes, Process Overlap Theory (Kovacs and Conway 2016), and the Triarchic Theory of Successful Intelligence (Sternberg 2018) as examples. The concept of general intelligence is eschewed within these models and g is often dismissed as a "statistical

1 It should be noted that Carroll's (2003) classification of intelligence theories did not include theories in which nOll-zero correlations between specific cognitive abilities are ignored treated as orthogonal despite evidence to the contrary (e.g., Gardner's [2006] theory of multiple intelligences, early versions of the theory of PMAput forth by Thurstone [1938]).

Psychometric Issues with Specific Abilities

83

artifact" of a positive manifold, reflecting the possibility that exists to extract a general factor from a correlation matrix containing only positive correlations. The third and final perspective that Carroll (2003) identified was what he referred to as the limited structural analysis view. This perspective is shared by those who believe that "a general factor exists but is essentially identical to, or highly correlated with, a second-order fluid intelligence factor Gf, but linearly independent of a second-order crystallized intelligence factor Gc and other possible second-order factors" (Carroll 2003, p. 7). The limited structural analysis view was born from research indicating high factor loadings between Gf and g, often times reaching 1.0 (e.g., Gustafsson's 1984 Hieararchical LISREL-based model [HILI]; Matzke, Dolan, and Molenaar 2010), that led to the conclusion that Gf and g are one and the same. Despite their differences, all three of these perspectives share in common the assumption that intelligence is multidimensional. The differences between each perspective lie within the nature of the hypothesized relationships between observed indicators of intelligence, specific abilities, and g (when it is acknowledged) that are reflected within the specific factor model used to represent intelligence. Contemporary versions of the most commonly administered intelligence test batteries in applied settings provide scores hypothesized to estimate specific abilities as well as at least one score meant to estimate global intelligence (e.g., the Wechsler Intelligence Scales for Children-Fifth Edition [WISC-V]; the Wechsler Adult Intelligence Scale-Fourth Edition [WAIS-N]; the Woodcock-Iohnson Tests of Cognitive Abilities-Fourth Edition [WI­ IV]; the Stanford-Binet Intelligence Scales-Fifth Edition [SB-5]; the Kaufinan Assessment Battery for Children-Second Edition [KABC-II]; Differential Abilities Scale-Second Edition [DAS-2]; and, the Cognitive Assessment System-Second Edition [CAS-2]; Braden 2013). However, intelligence test scores are derived using a "bottom up" approach wherein specific abilities are the intended measurement targets and scores from these measurement targets are aggregated to derive a "full scale" or "total composite" score that correlates with g.

Measuring Intelligence Two primary factor models have been used to study the psychometric properties of intelligence tests: higher-order (or hierarchical) models (Dombrowski, McGill, and Canivez 2017; Keith et al. 2006; Keith et al. 2010) and bifactor models (e.g., Dombrowski, McGill, and Canivez 2017;

84

Chapter Four

Dombrowski, Canivez, and Watkins 20 1 8 ; Styck and Watkins 2016; Styck and Watkins 2017; Watkins and Beaujean 20 1 3). Higher-order models are nested within bifactor models (Yung, Thissen, and McLeod 1 999) and differences across factor models in g factor scores or in the relationships between g and external variables can be negligible (Beauj ean, Parkin, and Parker 2014). However, higher-order models and bifactor models conceptually differ substantially in how g is hypothesized to explain individual differences in intelligence test scores as well as whether or not g is confounded with specific abilities.

Model A : Higher-Order Model

Model B : Bifactor Model

Figure 1 . Generic path models representing a hierarchical model of intelligence (model A) and a bifactor model of a intelligence (model B). Residual terms are omitted for space considerations.

Psychometric Issues with Specific Abilities

85

Higher-Order Models Higher-order models are derived from multiple factor analysis (Thurstone 1947). Factor analysis of the observed variables produce latent variables at the first order of the model, factor analysis of the latent variables at the first order of the model produce latent variables at the second order of the model, and so and so forth. The superordinate factor at the top of the ability hierarch; is a higher-order general latent variable. Many contemponny theories of intelligence that use higher-order models for theory development and testing contain three orders of latent variables (e.g., Schneider and McGrew 2012). However, intelligence test items are typically grouped into subtests that are hypothesized to directly measure specific abilities at the first-order. Consequently, factor analysis of intelligence test scores is nOlmally estimated from subtest score correlations resulting in a string of first-order specific ability group factors and a second-order general latent variable as depicted by model A in Figure 1 . The total variance in observed variables within a higher-order model is composed of three parts: (a) group factor variance, (b) subtest-specific variance, and (c) error variance; whereas, the total variance in the first­ order specific ability group factors is composed of two parts: (a) variance due to the second-order general latent variable and (b) specific ability group factor variance. The specific ability group factor variance is a residual term that captures variance that remains beyond that which is explained by variance in the second-order general latent variable. Notice that the path from the second-order general latent variable to Xl in model A is interrupted by Gj• Theoretical models that impose a higher-order factor structure assume that the second-order general latent variable indirectly influences observed variables. That is, the relationship between the second-order general latent variable and subtest perfOlmance on a given measure of intelligence is fully mediated by specific abilities-the second-order general latent variable directly influences Gl and Gl directly influences Xl. Consequently, the direct relationship between the latent and observed variables is constrained. Beaujean, Parkin, and Parker (2014) demonstrated that the proportion of variance in subtest scores, Xl, X2, and

2 The third order in hierarchical factor models is not confmed to a single factor

the nmnber of factors contained at any given order depends upon the nmnber and underlying dimensionality of the factors (or observed variables) at the prior order. Notwithstanding, most hierarchical factor models of intelligence derived from multiple factor analysis result in a single g factor at the third order (Carroll 1993).

86

Chapter Four

x} that is due to variance in the second-order general latent variable in a higher-order model is equal to the proportion of variance in Xl, X2 and x} that is due to G1 variance. In other words, the variance in subtest scores for a given intelligence test due to variance in specific abilities and variance in the second-order general latent variable is constrained to be equal within first-order factors. This is a defining characteristic of all higher-order factor models that has a number of undesirable consequences. First, the direct influence of either individual differences in the second-order general latent variable or specific abilities (over and above the second-order general latent variable) on subtest perfOlmance for a given intelligence test cannot be ascertained (the exception to this rule is discussed in the section below). Second, the proportionality constraints in higher-order models lead to biased reliability estimates for specific abilities (Reise, Bonifay, and Haviland 2013). Most importantly, proportionality constraints prevent the prediction of external variables (e.g., job perfOlmance, academic achievement) from specific abilities over and above the second-order general latent variable-the primary application of intelligence tests in applied settings.

Bifactor Models Spearman's (1927) specific ability factors were originally hypothesized to be test-specific residual telTIlS as previously explained that were uncorrelated with g, therefore both specific abilities and g were hypothesized to be direct explanations of individual differences in perfolTIlance on tasks thought to require some amount of mental ability within Two-Factor Theory. Carroll (1993) agreed with this notion of g and he applied the Schmid and Leiman (S-L; 1957) orthogonalization procedure to his exploratory factor analytic solutions to decompose the explained variance in observed intellectual task perfolTIlance into its component parts: that which was contributed by g and that which was contributed by specific abilities. Consequently, the g referenced in Two­ Factor Theory (Spearman 1927) and Three-Stratum Theory (Carroll 2012) is a breadth factor and its influence on subtest perfolTIlance is hypothesized to be direct; and, the second-order general latent variable in higher-order models is notg. An alternative factor analytic method to applying the S-L orthogonalization procedure to an exploratory factor analysis solution is to estimate the direct influence of variance in g and specific abilities on intellectual task performance was initially developed by Holzinger (1935). Holzinger's (1935) bifactor model extended "Spearman's Two-factor

Psychometric Issues with Specific Abilities

87

pattern to the case of group factors" (Holzinger and Swineford 1937, p. 41) in which g and specific abilities are all treated as orthogonal first-order factors that directly explain observed variables. The total variance in observed variables within a bifactor model is composed of four parts: (a) g variance, (b) specific ability group factor variance, (c) subtest-specific variance, and (d) error variance. Specific ability group factors in a bifactor model are derived from the common variance that remains in observed variables once the general factor is extracted; that is, they are residual tenns. A generic bifactor model of intelligence is presented in model B within Figure 1 . Notice that g and specific ability group factors independently influence observed variables within the bifactor model. Although Carroll's (2012) Three-Stratum Theory is often depicted as a higher-order model, he believed that Three-Stratum Theory was best conceptualized as a bifactor model (Carroll 1996). Bifactor models are advantageous in that they do not contain proportionality constraints. Consequently, they do not share the same undesirable consequences as higher-order models. The bifactor model, then, is arguably the best factor analytic model for answering the question, "In what ways are you intelligent?" Bifactor models of intelligence separate the total variance in observed variables into their component parts-variance due to individual differences in specific ability group factors, variance due to individual differences in g, and variance unexplained by the model (i.e., the residual combination of subtest-specific variance and measurement error). This can help reveal what is actually captured by the specific ability group factors and their corresponding scores on intelligence tests. Reise (2012) argued that "this partitioning can be invaluable in evaluating and refining an existing instrument and in finther understanding of a trait's structure" (p. 687). It also permits the prediction of external criterion from specific ability group factors over and above g, which is only possible with a bifactor model (Beaujean, Parkin, and Parker 2014).

Measuring Specific Abilities: Evidence of Reliability In the sections that follow, evidence of reliability of specific ability group factors from bifactor models will be reviewed for commonly administered intelligence tests in applied settings. The factor scores associated with specific ability group factors are not the same as the scores you would obtain from an intelligence test administration (Oh et al. 2004; Schneider 2013) and there is a substantial literature on the psychometric properties of observed scores that will not be reviewed, given that the

88

Chapter Four

purpose of this chapter is to summarize what is currently knO\vn about the assessment of specific intellectual abilities from factor analytic studies. Notwithstanding, factor analyses models latent abilities and measurement error separately, and factor scores potentially offer a closer approximation to a person's true underlying abilities than observed scores when factors are well defined (Steiger and SchOnemann 1978). Moreover, recent methodological advancements have permitted the computation of the reliable variance that can be attributed to specific abilities and g, both separately and together, which improves our understanding of what composite scores on intelligence tests actually assess. Corresponding coefficients that estimate how well a set of scores represent a factor and that estimate the bias that might be introduced from forcing a unidimensional model onto a set of items with multiple lUlderlying dimensions (ten Berge and Socan 2004; Bonifay et al. 2015; Reise, Bonifay, and Haviland 2013; Sijtsma 2009) have also been developed which further our understandnig of how precisely specific intellectual abilities can be estimated from commercially available intelligence test batteries.

Model-Based Factor Reliability McDonald (1999) initially developed what Reise, Bonifay, and Haviland (2013) and others have popularized as a family of model-based reliability coefficients, OJ, as a generalization to Cronbach's (1951) a that can be computed from a bifactor model, a S-L orthogonalization of an exploratory factor analysis solution, or from the more recently developed bifactor rotation criteria for exploratory factor analysis (Jennrich and Bentler 2011). The w family of model-based reliability coefficients describe the proportion of total variance in a set of scores that is due to variance in g, specific ability group factors, or some combination of g and specific ability group factor variance. As such, OJ values range between 0 and 1 with higher proportions of explained variance indicating higher factor reliability. There are four OJ variants that have been described in the literature (Brunner, Nagy, and Wilhelm 2012; Rodriguez, Reise, and Haviland 2016a): (a) w, (b) w" (c) Wh, and (d) Wk OJ describes the proportion of variance in a total composite score (i.e., Full-Scale IQ Score on the WISe-V) that is due to all sources of common

Psychometric Issues with Specific Abilities

89

3 variance. It is computed as the proportion of explained variance to total variance in the composite score. High OJ values indicate a highly reliable total composite score. OJs describes the proportion of variance in a subscore for a given specific ability (i.e., Verbal Comprehension Index on the WISCV-V) that is due to all sources of common variance-the blend of specific ability group factors and g. High OJ, indicate a highly reliable composite score accounting for all sources of common variance. The final two coefficients, OJh and OJhs describe the proportion of total variance in the total composite score (i.e., Full-Scale IQ Score on the WISC-V) due to solely variance in g and the proportion of total variance in a subscore for a given specific ability (i.e., Verbal Comprehension Index on the WISC-V) due solely to variance in a given specific ability factor, respectively. Specifically, OJh describes the proportion of total variance in a total composite score (i.e., Full-Scale IQ Score on the WISC-V) that is due solely to g variance; and, OJhs describes the proportion of total variance in a subscore for a given specific ability (i.e., Verbal Comprehension Index on the WISC-V) that is due solely to variance in that specific ability group factor. Relative OJ coefficients can also be computed to determine the proportion of reliable variance in the total score that is due to g (OJhlOJ) and the proportion of reliable variance that is due to any given specific ability group factor (OJhJ OJ,). In addition, the square root of OJhfh' can be computed to estimate the correlation between the factor and the observed unit-weighted composite score. l! has been suggested that OJh/'" values should minimally reach .50 in order for subscores to be interpreted as explaining unique reliable variance in a set of scores (Reise, Bonifay, and Haviland 2013; Gignac and Watkins 2013). OJhfh' < .50 indicate that individual differences in a factor explains less than 50% of the variance in a set of scores, leaving > 50% of the variance in that item set explained by other factors. Reise et a1. (2013) suggested that values closer to .75 may be preferable with the caveat that any guidelines provided for reliability coefficients are somewhat subjective. OJ values are also sensitive to small samples and are most precise when samples sizes are > 300 (Charter 1999). In addition, OJ variants are not independent of one another-high OJh values force low OJhs values. In other words, if an intelligence test yields a highly reliable total composite score, it carmot concurrently yield highly reliable subscores. Thus, only one construct can be reliably measured at a time. 3 Specific formulas for calculating OJ and other indices discussed within this section of the chapter are provided in tutorials from Brunner, Nagy, and Wilhelm (2012) and Rodriguez, Reise, and Haviland (20 l 6a).

90

Chapter Four

A Google Scholar search using the keywords bifactor and intelligence revealed > 3,000 published articles. A comprehensive review of the published literature on bifactor models of intelligence is beyond the scope of this chapter. That being said, generally speaking, when bifactor models are applied to intelligence tests, Wh for specific ability group factors fall well below .50 and Whs for g meet or exceed .75. This remains true regardless of the test administered (Benson, Kranzler, and Floyd 201S; Canivez 2014; Cucina and Howardson 2017; Dombrowski, McGill, et al. 201S; Fenollar-Cortes, Lopez-Pinar, and Watkins 201S; Gignac and Watkins 2013; Lecerf and Canivez 2017; McGill 2016; McGill and Canivez 2017; Nelson, Canivez, and Watkins 2013; Strickland, Watkins, and Caterino 2015; Styck and Watkins 2015; Styck and Watkins 2016; Watkins et al. 2013; Watkins and Beaujean 2013), the participant characteristics (e.g., Canivez, Dombrowski, and Watkins 2018; Collinson et al. 2017; Dombrowski, Canivez, and Watkins 2018; Fenollar-Cortes, Lopez-Pinar, and Watkins 201S; Gomez, Vance, and Watson 2016; Styck and Watkins 2016, 2017; Watkins and Beaujean 2013), the theoretical model of intelligence tested (e.g., Canivez et al. 2017, 201S; Canivez, Dombrowski, and Watkins 2018; Canivez, Watkins, and Dombrowski 2016a, 2017; Canivez, Watkins, and McGill 201S; Canivez and Watkins 2016; Dombrowski, McGill, and Canivez 201Sa, 201Sb; Dombrowski et al. 2015; Dombrowski, Canivez, and Watkins 2018; Dombrowski, McGill, and Canivez 2017; Fenollar-Cortes and Watkins 201S; Lecerf and Canivez 2017; McGill and Canivez 2017; Watkins and Beaujean 2013; Watkins, Dombrowski, and Canivez 2017), or the specific factor analytic techniques applied to analyze tlie dataset (e.g., Canivez and McGill 2016; Canivez, Watkins, and Dombrowski 2016; Dombrowski et al. 2015; Dombrowski, Golay, et al. 201S). These findings strongly suggest that the largest proportion of reliable variance in subtest scores for commonly administered intelligence tests is contributed by variance in g with negligible reliable variance contributed by specific ability group factor variance. Consequently, for most current intelligence tests, composite scores tliat estimate specific abilities (mostly) provide redundant information about an examinee's global intellectual ability. The estimation of specific abilities could be improved, however, if test publishers included redundant sets of subtests to create multiply unidimensional models (Sinharay 2010).

Psychometric Issues with Specific Abilities

91

Factor Determinacy Factor score detelTIlinacy refers to the degree to which factor scores represent true individual differences on a latent variable. Factor scores are indeterminant by the very nature of the common factor model-they do not have a unique solution. Rather, there is an infinite number of equally satisfactory factor scores that can be computed from any given set of factor loadings (Grice 2001). Factor scores can be separated into two portions: indeterminate and determinate. Guttman (1955) demonstrated that factor scores that are highly indeterminate (indeterminate portion of the factor score > .50) can produce two sets of equally satisfactory factor scores that are negatively correlated. This can result in wildly discrepant rankings of individuals along a given factor, which can lead to equally indetelTIlinate relationships with external criteria (Sch6nemann and Steiger 1978; Steiger 1979). Consequently, factor determinacy is an essential component to theory development and psychological test construction and evaluation. It has been recommended that highly determinant factors have factor determinacy values > .90 (Gorsuch 1983). Beauducel's (20 1 1) Factor Determinacy Index (FDI) is a commonly used estimate of factor detelTIlinacy that represents the correlation between the factor scores and the factors. Benson et al. (2018) re-analyzed a subset of 7 studies that were originally analyzed by Carroll (1993). They applied bifactor models to the datasets using confinnatory factor analytic techniques and computed OJ model-based reliability coefficients, the FDI, and other indices to be introduced later on in this chapter. The best fitting models consisted of g and 3 to 5 first-order specific ability group factors (i.e., Stratum II broad abilities originally identified by Carroll 1993). The FDI for g ranged from .90 to .97 and the FDI for the specific ability group factors exceeded .90 in only 3 out of the 7 re-analyzed datasets. In particular, the FDI for Gy­ general memory and learning (.90; Christal 1958), Ga-broad auditory ability (.90; Fogarty 1987), and Gc-crystallized intelligence (.90; Gustafsson 1984) were the only specific ability group factors that displayed acceptable factor determinacy.

Construct RepJicability Model-based estimation of construct replicability, the H index, was developed by Hancock and Mueller (2001) as a means to estimate how well a set of items represents a latent variable. The logic that follows is that a set of items that represents a latent variable well will be replicable

92

Chapter Four

across studies. The H index is the ratio of the proportion of variance in a set of items that is explained by a factor to the proportion of variance in a set of items that is unexplained by the factor. H index values range between 0 and 1, and higher H index values indicate higher construct replicability. H index values > .80 are indicative of factors that are well defined (Rodriguez, Reise, and Haviland 2016a). H indexes for g consistently exceed this suggested threshold for acceptable construct replicability. However, H indexes for specific ability group factors consistently fail to reach .80 with few exceptions (Benson et al. 201S; Cainvez et al. 201S; Dombrowski, Golay, et al. 201S; Dombrowski, McGill, and Canivez 201Sa). This suggests that specific ability group factors may not replicate across studies. Rodriguez, Reise, and Haviland (2016a) pointed out that low H indexes may be especially problematic for researchers who use structural equation modeling to test theoretical relationships between latent variables. If specific ability group factors with low H indexes « .SO) are included in a structural equation model, the relationships between specific ability group factors and other latent variables may not replicate, which ultimately undelTIlines theory development.

Estimating Bias Introduced by Assuming a Unidimensional Model Two additional coefficients have been developed as a means to estimate parameter bias that may result from forcing multidimensional data into a unidimensional model: the explained common variance (ECV; ten Berge and Socan 2004; Sijtsma 2009) and the percent of uncontaminated correlations (PUC; Bonifay et al. 2015; Reise et al. 2013). Referring back to the bifactor model (model B) in Figure 1, each observed variable is explained by variance in two factors: g and a specific ability group factor. For example, correlations between items Xl, X2, and x} are explained by G, and g. Consequently, removing G, from the model may result in spuriously high correlations among items Xl, X2, and X3. If specific ability group factors result in Whs < .50 and H < .80, researchers may be tempted to force a unidimensional model to a dataset despite underlying multidimensionality. According to Rodriguez, Reise, and Haviland (2016a) there are two possible consequences to misspecifying a model in this manner: (a) the resulting general factor may not reflect the common variance among the items and (b) the factor loadings may be biased. The ECV and PUC provide estimates of bias that could be introduced as a result of treating items from an intelligence test as measuring only g.

Psychometric Issues with Specific Abilities

93

The ECV is the percent of total common variance for the general factor and the PUC is the ratio of the number of correlations between items from different group factors to the total number of correlations in the model. High ECV values indicate that the relationship between the observed variables and g (factor loadings) in a bifactor model will be similar to the relationship between the observed variables and g (factor loadings) in a unidimensional model. The PUC defines a "contaminated" correlation as a correlation between items within factors because those relationships are explained by both specific ability group factors and g (e.g., the correlation between the Vocabulary subtest and the Similarities subtests within the Verbal Comprehension Index on the WISC-V); whereas, correlations between items from different specific ability group factors (e.g., the correlation between the Vocabulary subtest within the Verbal Comprehension Index on the WISC-V and the Coding subtest within the Processing Speed Index on the WISC-V) can only be explained by g. Higher PUC values indicate less potential for biased parameter estimates resulting from fitting a unidimensional model to multidimensional data. The ECV and PUC are meant to be interpreted together, as the PUC moderates the relationship between the ECV and parameter bias (Bonifay et al. 2015; Reise et al. 2013). It has been suggested that bias will be minimal when ECV> .70 and PUC > .70 (Rodriguez, Reise, and Haviland 20l6a). The PUC for most theoretical models of intelligence that have been applied to commercially available individually administered standardized intelligence tests is greater than .70. ECV values also consistently exceed .70 in published research across different tests (e.g., Benson et al. 2018; Canivez et al. 2018; Canivez, Watkins, and McGill 2018; Dombrowski, Golay, et al. 2018; Fenollar-Cortes and Watkins 2018; Gomez, Vance, and Watson 2017; Lecerf and Canivez 2017), participants (Canivez et al. 2018; Dombrowski, Canivez, and Watkins 2018; Gomez, Vance, and Watson 2016), theoretical models (Benson et al. 2018; Canivez et al. 2018; Canivez, Watkins, and McGill 2018; Canivez et al. 2017), and factor analytic techniques (Dombrowski, Golay, et al. 2018; Dombrowski, McGill, et al. 2018). These results suggest that minimal bias would occur in parameter estimates if a unidimensional measurement model was fit to intelligence test scores from commercially available intelligence test batteries.

94

Chapter Four

Measuring Specific Abilities: Evidence of Validity Model-based indices of factor reliability, factor determinacy, construct replicability, and potential for parameter bias if a unidimensional model was fit to a dataset with underlying multidimensionality computed from bifactor models of intelligence all indicate that g is estimated well by intelligence tests and that specific intellectual abilities are not. However, evidence of reliability is only one piece of the puzzle. Evidence of validity for intelligence test score interpretations has been gathered primarily from stmctural evidence (i.e., factor analytic studies) and evidence of relationships with external variables (i.e., correlation/regression with convergent and divergent traits) in accordance witli Messick's (1995) framework for validity. Structural evidence largely supports the multidimensionality of intelligence (e.g., Carroll 1993), though research reviewed in the prior section of this chapter suggests tliat tlie specific abilities estimated by individually administered standardized intelligence tests are likely uninterpretable. Evidence of external validity for specific ability scores paints a somewhat less grim picture. In keeping with the organizational structure of the preceding section, evidence of external validity accrued specifically from bifactor modeling studies will be reviewed within this section of this chapter. Bifactor models have been used to examine the degree to which specific ability group factors predict external criterion in two studies (Beaujean, Parkin, and Parker 2014; Kranzler, Benson, and Floyd 2015). Beaujean, Parkin, and Parker (2014) used both a bifactor model and a higher-order model to examine the external validity of specific ability group factors on the WISC-N for predicting reading and writing achievement on the Wechsler Individual Achievement Test-Second Edition (WIAT-II; Wechsler 2002) using the WISC-IV-WIAT-II standardization linking sample. To demonstrate the differences between the two factor models, Beaujean, Parkin, and Parker (2014) kept tlie measurement of the specific ability group factors the same (identical number of specific ability group factors and identical pattern of estimated and constrained factor loadings). They reported tliat the factor model selected made little difference on the prediction of reading and writing achievement when g was the sole predictor. However, there was a substantial difference in the prediction of reading and writing achievement from specific ability group factors between the two factor models. The proportion of variance in reading and writing achievement scores that was explained by the specific ability group factors declined by 25-33% for tlie bifactor model and by only 3-6% for tlie higher-order model when g was

Psychometric Issues with Specific Abilities

95

not entered as a predictor. Recall, the specific ability group factors in a bifactor model are composed of the residual common variance in observed variables that remains once common variance due to the general factor is removed. The relationship between specific ability group factors and indicators of intelligence is substantially weaker in a bifactor model with g variance removed when compared to the higher-order model in which specific ability group factor variance includes g variance as a result because g variance is the predominant detelTIlinant of observed variables in factor analytic studies. Nevertheless, the relationship between specific ability group factors and reading and writing achievement was weaker in both models than the relationship between g and reading and writing achievement in models where g was the sole predictor. Kranzler, Benson, and Floyd (2015) used a bifactor model to investigate the incremental validity of specific ability group factors on mathematics, reading, written language, oral language, and total achievement using the WAIS-N-WIAT-II standardization linking sample beyond g. Results indicated that g was a significant predictor of mathematics, reading, written language, oral language, and total achievement; whereas, verbal comprehension was the only specific ability group factor to explain a non­ negligible amount of variance in achievement scores above and beyond g. Specifically, verbal comprehension explained a significant amount of variance for a subset of reading achievement (squared semipartial rs ranging between .1 0 and .22), written language (squared semipartial r of .09), and oral language (squared semipartial rs ranging between .16 and .24) achievement scores as well as a small significant amount of variance in total achievement (squared semipartial r of .08). However, the weak relationship between specific abilities and achievement is not solely due to the use ofbifactor modeling techniques or to the use of latent variable modeling techniques, in general. Zaboski, Kranzler, and Gage (2018) recently conducted a meta-analysis of the relationship between broad CRC theory specific abilities and academic achievement for 25 studies of ability-achievement relations published between 1993 and 2015. They reported that g explained the vast majority of variance in achievement scores across academic areas with mean rs for g ranging between .72 and .76. The amount of variance in achievement scores predicted by g variance across all academic areas was more than twice the amount of variance explained by variance in all of the specific ability scores combined. Gc-crystallized intelligence explained the most variance in achievement scores of the CHC specific abilities included in the meta-analyses (rs ranging between .30 and .45), followed by Gsm­ short-term memory (rs ranging between .1 0 and .28), Gs-processing

96

Chapter Four

speed (rs ranging between .1 2 and .23), GJ-fluid intelligence (rs ranging between .07 and .39), Glr-Iong-term storage and retrieval (rs ranging between .07 and .24), Ga-auditory processing (rs ranging between .03 and .34), and Gv-visual processing (rs ranging between -.01 and .09). Nevertheless, there are a few notable limitations to the study. Zaboski et al. did not include studies of incremental validity or examine the impact of statistical modeling techniques (e.g., multiple regression vs. structural equation modeling, bifactor model vs. higher-order model) on results. In addition, many studies of ability-achievement relations were conducted using test standardization samples. Despite these limitations, the results of Zaboski et al. and the bifactor modeling studies reviewed within this chapter (Beaujean, Parkin, and Parker 2014; Kranzler, Benson, and Floyd 2015) echo conclusions drawn from Jensen (1989, 1998a) decades ago that individual differences in scholastic achievement are primarily explained by individual differences in g.

Summary This chapter began by summarizing the paradigm shift that has occurred in the way psychologists view intelligence. Contemporary theories of intelligence attempt to differentiate individuals on the basis of specific intellectual abilities, rather than on a global trait. The question, "In what ways are you intelligent?" has all but replaced the question, "How intelligent are you?" Moreover, commonly administered comerically available intelligence test batteries provide a litany of scores that psychologists can use to answer both questions-effectively propelling this movement forward. However, it was argued within this chapter that this shift in ideology has occurred in spite of a growing body of evidence that indicates that specific abilities cmmot be measured (or more accurately, assessed) with sufficient reliability or validity to answer the question, "In what ways are you intelligent?" with reasonable precision. Thus, the field has reached an impasse. The predominant source of information used to support these claims come from factor analytic studies that used bifactor modeling techniques. Bifactor models partition the total variance in observed variables into that which is due to variance in g, that which is due to variance in specific ability group factors, and that which is due to unexplained residual factors (i.e., subtest specific variance and measurement error). Schneider (2013) argued that "the independent portion [of higher-order specific ability group factor variance with g variance removed] is not the 'real Gc.' We care about a sprinter's ability to run quickly, not residual sprinting speed

Psychometric Issues with Specific Abilities

97

after accOlUlting for general athleticism. So it is with Gc: g is a part of the mix" (p. 188). However, extant research has indicated that well over 50% of the total variance in observed variables is due to variance in g. This strongly suggests that this argument grossly understates the role of g in explaining intelligence test perfOlmance according to what is currently knO\vn about the psychometric properties of commercially available intelligence tests and the reliability and validity of interpretations of the test scores that they produce. At the same time, the issue of how to interpret residual common variance after removing variance attributed to g is a serious one that should not be dismissed. van Bork et al. (2017) caution against reifying factors that are identified from factor analytic studies. Factors merely represent explained common variance-they are not entities that exist outside of the factor analytic model. This is true whether we are talking about g or specific ability group factors and the problem with reification is compounded by the misuse of cross-sectional data to predict the intelligence of individuals (i.e., ergodicity). Factor models are also causal models. Arrows in factor analysis are unidirectional pointing towards observed variables indicating that performance on any given observable task assumed to require some amount of intelligence is caused by common and residual factors. However, there may not actually exist any single underlying common cause. Network models have been proposed as an alternative to factor analytic models as a means to estimate bidirectional relationships between variables that are not captured by factor analysis (Epskamp, Rhemtulla, and Borsboom 2017). It is possible that application of such statistical techniques to observable indicators of intelligence may lead to different conclusions regarding specific abilities. Finally, Michell (2012) argued that commercially available intelligence test batteries constructed on the basis of classical test theory do not so much measure intelligence in the sense of providing a quantitative amount of intelligence an individual posseses as they assess the qualitative rank order of individuals in tenns of their intelligence (however defined by a given test battery) in comparison to their peers. Conjoint measurement models (i.e., Rasch models; Rasch 1992) may offer the most promise for truly measuring intelligence, since examinee ability and item difficulty are treated as independent sources of systematic variance in observed item scores. However, application of these models would require that traditional test construction and interpretation practices be abandoned in favor of item response modeling techniques. In the absence of additional evidence, however, psychologists are encouraged to recognize the limitations of the information gleaned from

98

Chapter Four

commercially available intelligence test batteries. Factor analytic studies have consistently indicated that specific abilities do not evidence adequate factor reliability, factor detelTIlinacy, construct replicability, or prediction of external criteria. In addition, empirical research suggests that negligible bias would result from treating scores from commonly administered commercially available intelligence tests as only measuring global intelligence. Consequently, intelligence may be multidimensional, but modem intelligence test batteries are only equipped to answer the question, "How intelligent are you (in comparison to your peers)7" Nevertheless, these conclusions are drawn from factor analytic studies and these statistical techniques may not be the best means for studying intelligence. It may be necessary for reserachers to take a different analytical approach to studying intelligence in order to move the field forward beyond the current stalemate.

Acknowledgements I would like to thank A. Alexander Beaujean and Christopher J. Anthony for their insightful comments on earlier drafts of this chapter, especially regarding the complex task of attempting to measure things that may not, in fact, be measurable. Their comments helped me clarify these complicated issues in a way that I sincerely hope is understandable to readers of this chapter.

References Beauducel, A. 201 1 . "Indeterminacy of Factor Score Estimates in Slightly Misspecified Confirmatory Factor Models." Journal of Modern AppliedStatisticalMethods 10 (2): 583-98. Beaujean, A. A., J. Parkin, and S. Parker. 2014. "Comparing Cattell­ Hom-Carroll Factor Models: Differences between Bifactor and Higher Order Factor Models in Predicting Language Achievement." Psychological Assessment 26 (3): 789-805. Benson, N. F., A. A. Beaujean, R. J. McGill, and S. C. Dombrowski. 2018. "Revisiting Carroll's Survey of Factor-Analytic Studies: Implications for the Clinical Assessment of Intelligence." Psychological Assessment 30 (8): 1028-38. Benson, N., J. H. Kranzler, and R. G. Floyd. 2018. "Exploratory and Confirmatory Factor Analysis of the Universal Nonverbal Intelligence Test-Second Edition: Testing Dimensionality and Invariance across

Psychometric Issues with Specific Abilities

99

Age, Gender, Race, and Ethnicity." Assessment. Advance online publication. Berge, J. M. F. ten, and G. Socan. 2004. "The Greatest Lower Bound to the Reliability of a Test and the Hypothesis of Unidimensionality." Psychometrika 69 (4): 613-25. Bonifay, W. E., S. P. Reise, R. Scheines, and R. R. Meijer. 2015. "When Are Multidimensional Data Unidimensional Enough for Structural Equation Mode1nig? An Evaluation of the DETECT Multidimensionality Index." Structural Equation Modeling: A Multidisciplinary Journal 22 (4): 504-16. Bork, R. van, S. Epskamp, M. Rhemtulla, D. Borsboom, and H. L. J. van der Maas. 2017. "What Is the p -Factor of Psychopathology? Some Risks of General Factor Modeling." Theory & Psychology 27 (6): 75973. Braden, J. P. 2013. "Psychological Assessment in School Settnigs." In Handbook of Psychology: Assessment Psychology, 2nd ed., 261-90. Hoboken, NJ: Wiley. Brunner, M., G. Nagy, and O. Wilhelm. 2012. "A Tutorial on Hierarchically Structured Constructs." Journal of Personality 80 (4): 796-846. Bulut, 0., M. L. Davison, and M. C. Rodriguez. 2017. "Estimating Between-Person and within-Person Subscore Reliability with Profile Analysis." Multivariate Behavioral Research 52 (1): 86-104. Canivez, G. L., R. J. McGill, S. C. Dombrowski, M. W. Watknis, A. E. Pritchard, and L. A. Jacobson. 2018. "Construct Validity of the WISC­ V in Clinical Cases: Exploratory and Confirmatory Factor Analyses of the 10 Primary Subtests." Assessment. Advanced onlnie publication. Canivez, G. L., and M. W. Watkins. 2016. "Review of the Wechsler Intelligence Scale for Children--Fifth Edition: Critique, Commentary, and Independent Analyses." In Intelligent Testing with the wIse- v, 683-702. Hoboken, NJ: Wiley. Canivez, G. L., M. W. Watkins, R. Good, K. James, and T. James. 2017. "Construct Validity of the Wechsler Intelligence Scale for Children Fourth UK Edition with a Referred Irish Sample: Wechsler and Cattell-Horn-Carroll Model Comparisons with 15 Subtests." British Journal ofEducational Psychology 87 (3): 383-407. Canivez, G. L. 2014. "Construct Validity of the WISC-N with a Referred Sample: Direct versus Indirect Hierarchical Structures." School Psychology Quarterly 29 (1): 38-5 1 . Canivez, G. L., S . C. Dombrowski, and M . W. Watkins. 2018. "Factor Structure of the WISC-V in Four Standardization Age Groups:

100

Chapter Four

Exploratory and Hierarchical Factor Analyses with the 16 Primary and Secondary Subtests." Psychology in the Schools 55 (7): 741-69. Canivez, G. L., and R. J. McGill. 2016. "Factor Structure of the Differential Ability Scales-Second Edition: Exploratory and Hierarchical Factor Analyses with the Core Subtests." Psychological Assessment 28 (11): 1475-88. Canivez, G. L., M. W. Watkins, and S. C. Dombrowski. 20l 6a. "Factor Structure of the Wechsler Intelligence Scale for Children-Fifth Edition: Exploratory Factor Analyses with the 16 Primary and Secondary Subtests." Psychological Assessment 28 (8): 975-86. Canivez, G. L., M. W. Watkins, and S. C. Dombrowski. 20l 6b. "Factor Structure of the Wechsler Intelligence Scale for Children-Fifth Edition: Exploratory Factor Analyses with the 16 Primary and Secondary Subtests." Psychological Assessment 28 (8): 975-86. Canivez, G. L., M. W. Watkins, and S. C. Dombrowski. 2017. "Structural Validity of the Wechsler Intelligence Scale for Children-Fifth Edition: Confirmatory Factor Analyses with the 16 Primary and Secondary Subtests." Psychological Assessment 29 (4): 458-72. Canivez, G. L., M. W. Watkins, and R. J. McGill. 2018. "Construct Validity of the Wechsler Intelligence Scale For Children - Fifth UK Edition: Exploratory and Confirmatory Factor Analyses of the 16 Primary and Secondary Subtests." British Journal of Educational Psychology. Advanced online publication. Carroll, J. B. 1993. Human Cognitive Abilities: A Survey of Factor­ Analytic Studies. New York, NY: Cambridge University Press. Carroll, J. B. 1996. "A Three-Stratum Theory of Intelligence: Spearman's Contribution." In Human Abilities: Their Nature and Measurement, 117. Mahwah, NJ: Erlbaum. Carroll, J. B. 2003. "The Higher-Stratum Structure of Cognitive Abilities: Current Evidence Supports g and about Ten Broad Factors." In The Scientific Study of General Intelligence: A Tribute to Arthur R. Jensen, 5-21 . New York, NY. Carroll, J. B. 2012. "The Three-Stratum Theory of Cognitive Abilities." In Contemporary Intellectual Assessment, 3rd ed., 883-890. New York, NY: Guilford. Charter, R. A. 1999. "Sample Size Requirements for Precise Estimates of Reliability, Generalizability, and Validity Coefficients." Journal of Clinical & Experimental Neuropsychology 21 (4): 559-66. Christal, R. E. 1958. "Factor Analytic Study of Visual Memory." Psychological Monographs: General and Applied 72 (13): 1-24.

Psychometric Issues with Specific Abilities

101

Collinson, R., S. Evans, M. Wheeler, D. Brechin, I. Moffitt, G. Hill, and S. Muncer. 2017. "Confirmatory Factor Analysis of WAIS-N in a Clinical Sample: Examining a Bi-Factor Model." Journal of Intelligence 5 (1): 1-5. Cronbach, L. I. 195 1 . "Coefficient Alpha and the Internal Structure of Tests" 16 (3): 297-334. Cucina, J. M., and G. N. Howardson. 2017. "Woodcock-Iohnson-III, Kaufman Adolescent and Adult Intelligence Test (KAlT), Kaufman Assessment Battery for Children (KABC), and Differential Ability Scales (DAS) Support Carroll but Not Cattell-Hom." Psychological Assessment 29 (8): 1001-15. Das, I. P., J. A. Naglieri, and I. R. Kirby. 1994. Assessment of Cognitive Processes: The PASS Theory of Intelligence. Needham Heights, MA: Allyn & Bacon. Dombrowski, S. C., G. L. Canivez, and M. W. Watknis. 2018. "Factor Structure of the 10 WISC-V Primary Subtests Across Four Standardization Age Groups." Contemporary School Psychology 22 (1): 90-104. Dombrowski, S. C., G. L. Canivez, M. W. Watknis, and A. Alexander Beaujean. 2015. "Exploratory Bifactor Analysis of the Wechsler Intelligence Scale for Children-Fifth Edition with the 16 Primary and Secondary Subtests." Intelligence 53 (November): 194-201. Dombrowski, S. C., P. Golay, R. I. McGill, and G. L. Canivez. 2018. "Investigating the Theoretical Structure of the DAS-II Core Battery at School Age Usnig Bayesian Structural Equation Modeling." Psychology in the Schools 55 (2): 190-207. Dombrowski, S. C., R. J. McGill, and G. L. Canivez. 2017. "Exploratory and Hierarchical Factor Analysis of the Wl-N Cognitive at School Age." Psychological Assessment 29 (4): 394-407. Dombrowski, S. C., R. I. McGill, and G. L. Canivez. 20l 8a. "An Alternative Conceptualization of the Theoretical Structure of the Woodcock-lohnson N Tests of Cognitive Abilities at School Age: A Confirmatory Factor Analytic Investigation." Archives of Scientific Psychology 6 (1): 1-13. Dombrowski, S. C., R. J. McGill, and G. L. Canivez. 20l 8b. "Hierarchical Exploratory Factor Analyses of the Woodcock-lohnson N Full Test Battery: Implications for CHC Application in School Psychology." School Psychology Quarterly 33 (2): 235-50. Dombrowski, S. C., R. J. McGill, G. L. Canivez, and C. H. Peterson. 2019. "Investigatnig the Theoretical Structure of the Differential Ability

102

Chapter Four

Scales-Second Edition through Hierarchical Exploratory Factor Analysis." Journal ofPsycho educational Assessment, 37 (1): 91-104. Epskamp, S., M. Rhemtulla, and D. Borsboom. 2017. "Generalized Network Psychometrics: Combining Network and Latent Variable Models." Psychometrika 82 (4): 904-27. Fenollar-Cortes, J., C. Lopez-Pinar, and M. W. Watkins. 2018. "Structural Validity of the Spanish Wechsler Intelligence Scale for Children­ Fourth Edition in a Large Sample of Spanish Children with Attention­ Deficit Hyperactivity Disorder." International Journal of School & Educational Psychology 0 (0): 1-13. Fenollar-Cortes, J., and M. W. Watkins. 2018. "Construct Validity of tbe Spanish Version of tbe Wechsler Intelligence Scale for Children Fifth Edition (WISC-VSpain)." International Journal of School & Educational Psychology, March, 1-15. Fogarty, G. 1987. "Timesharing in Relation to Broad Ability Domains." Intelligence 1 1 (3): 207-3 1 . Gardner, H. 2006. Multiple Intelligences: New Horizons. New York, NY: Basic Books. Gignac, G. E., and M. W. Watkins. 2013. "Bifactor Modeling and tbe Estimation of Model-Based Reliability in the WAIS-N." Multivariate Behavioral Research 48 (5): 639-62. Gomez, R, A. Vance, and S. Watson. 2017. "Bifactor Model of WISC-IV: Applicability and Measurement Invariance in Low and Normal IQ Groups." Psychological Assessment 29 (7): 902-12. Gomez, R., A. Vance, and S. D. Watson. 2016. "Structure of the Wechsler Intelligence Scale for Children - Fourth Edition in a Group of Children witb ADHD." Frontiers in Psychology 7: 1-1 1 . Gorsuch, R L . 1983. Factor Analysis. 2nd ed. Hillsdale, NJ: Erlbaum. Grice, J. W. 2001. "Computing and Evaluating Factor Scores." Psychological Methods 6 (4): 430-50. Guilford, J. P. 1972. "Thurstone's Primary Mental Abilities and Structure­ of-Intellect Abilities." Psychological Bulletin 77 (2): 129-43. Gustafsson, J-E. 1984. "A Unifying Model for the Structure of Intellectual Abilities." Intelligence 8 (3): 179-203. Guttman, L. 1955. "The Determinacy of Factor Score Matrices with Implications for Five Other Basic Problems of Common-Factor Theory." British Journal ofStatistical Psychology 8 (2): 65-8 1 . Hancock, G. R , and R. O. Mueller. 2001. "Rethinking Construct Reliability within Latent Variable Systems." In Structural Equation Modeling: Present and Future, 195-216. Lincolnwood, IL: Scientific Software International.

Psychometric Issues with Specific Abilities

103

Holzinger, K. J. 1935. Preliminary Report on Spearman-Holzinger Unitary Trait Study, No. 5: Introduction to Bi-Factor Theory; Solid and Hollow Staircase Patterns for Sets of Data from Mooseheart. Chicago, IL: University of Chicago. Holzinger, Karl J., and Frances Swineford. 1937. "The Bi-Factor Method." Psychometrika 2 (1): 41-54. Horn, J. L. 1968. "Organization of Abilities and the Development of Intelligence." Psychological Review 75 (3): 242-59. Horn, J. L., and J. Noll. 1997. "Human Cognitive Capabilities: Gf-Gc Theory." In Contemporary Intellectual Assessment: Theories, Tests, and Issues, 53-9 1 . New York, NY: Guilford Press. Jennrich, R. I., and P. Bentler. 201 1 . "Exploratory Bi-Factor Analysis." Psychometrika 76 (4): 537-49. Jensen, A. R. 1989. "The Relationship between Learning and Intelligence." Learning and Individual Differences 1 : 1-37. Jensen, A. R. 1998. "The g Factor and the Design of Education." In Intelligence, Instruction, and Assessment: Theory into Practice, 1 1 11 3 1 . Mahwah, NJ: Lawrence Erlbaum Associates. Keith, T. Z., J. G. Fine, G. E. Taub, M. R. Reynolds, and J. H. Kranzler. 2006. "Higher Order, Multisample, Confirmatory Factor Analysis of the Wechsler Intelligence Scale for Children-Fourth Edition: What Does It Measure?" School Psychology Review 35: 108-27. Keith, T. Z., J. A. Low, M. R. Reynolds, P. G. Patel, and K. P. Ridley. 2010. "Higher-Order Factor Structure of the Differential Ability Scales-II: Consistency across Ages 4 to 17." Psychology in the Schools, 47 (7): 676-697. Kovacs, K., and A. R. A. Conway. 2016. "Process Overlap Theory: A Unified Account of the General Factor of Intelligence." Psychological Inquiry 27 (3): 15 1-77. Kranzler, J. H., N. Benson, and R. G. Floyd. 2015. "Using Estimated Factor Scores from a Bifactor Analysis to Examine the Unique Effects of the Latent Variables Measured by the WAIS-N on Academic Achievement." Psychological Assessment 27 (4): 1402-1416. Lecerf, T., and G. L. Canivez. 2017. "Complementary Exploratory and Confirmatory Factor Analyses of the French WISC-V: Analyses Based on the Standardization Sample." Psychological Assessment, 30 (6): 793-808. Matzke, D., C. V. Dolan, and D. Molenaar. 2010. "The Issue of Power in the Identification of 'g' with Lower-Order Factors." Intelligence 38 (3): 336-44.

1 04

Chapter Four

McDonald, R. P. 1999. Test Theory: A Unified Treatment. Mahwah, NJ: Erlbaum. McGill, R. J. 2016. "Investigation of the Factor Structure of the Comprehensive Test of Nonverbal Intelligence-Second Edition (CTONI-2) Using Exploratory Factor Analysis." Journal of Psychoeducational Assessment 34 (4): 339-50. McGill, R. J., and G. L. Canivez. 2018. "Confirmatory Factor Analyses of the WISC-N Spanish Core and Supplemental Subtests: Validation Evidence of the Wechsler and CHC Models." International Journal of School & Educational Psychology, 6 (4): 239-25 1 . McGrew, K. S. 2005. "The Cattell-Horn-Carroll Theory of Cognitive Abilities: Past, Present and Future." In Contemporary Intellectual Assessment: Theories, Test, and Issues, 2nd ed., 136-8 1 . New York, NY: Guilford Press. Messick, S. 1995. "Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning." American Psychologist 50 (9): 741-49. Michell, J. 1997. "Quantitative Science and the Defmition of Measurement in Psychology." British Journal of Psychology 88 (3): 355-83. Michell, J. 2003. "Measurement: A Beginner's Guide." Journal ofApplied Measurement 4 (4): 298-308. Michell, J. 2012. "Alfred Binet and the Concept of Heterogeneous Orders." Frontiers in Psychology 3 (August): 1-8. Nelson, J. M., G. L. Canivez, and M. W. Watkins. 2013. "Structural and Incremental Validity of the Wechsler Adult Intelligence Scale-Fourth Edition with a Clinical Sample." Psychological Assessment 25 (2): 61 8-30. Oh, H-J, J. J. Glutting, M. W. Watkins, E. A. Youngstrom, and P. A. McDermott. 2004. "Correct Interpretation of Latent versus Observed Abilities Implications from Structural Equation Modeling Applied to the WISC-III and WIAT Linking Sample." The Journal of Special Education 38 (3): 159-173. Rasch, G. 1992. Probablistic Modelsfor Some Intelligence andAttainment Tests. Chicago, IL: MESA Press. Reise, S. P. 2012. "The Rediscovery of Bifactor Measurement Models." Multivariate BehavioralResearch 47 (5): 667-96. Reise, S. P., W. E. Bonifay, and M. G. Haviland. 2013. "Scoring and Modeling Psychological Measures III the Presence of Multidimensionality." Journal ofPersonality Assessment 95: 129-40.

Psychometric Issues with Specific Abilities

105

Reise, S. P., R. Scheines, K. F. Widaman, and M. G. Haviland. 2013. "Multidimensionality and Structural Coefficient Bias in Structural Equation Modeling: A Bifactor Perspective." Educational and Psychological Measurement 73 (1): 5-26. Rodriguez, A., S. P. Reise, and M. G. Haviland. 20l6a. "Applying Bifactor Statistical Indices in the Evaluation of Psychological Measures." Journal ofPersonality Assessment 98 (3): 223-37. Rodriguez, A., S. P. Reise, and M. G. Haviland. 20l 6b. "Evaluating Bifactor Models: Calculating and Interpreting Statistical Indices." Psychological Methods 21 (2): 137-50. Schmid, J., and J. M. Leiman. 1957. "The Development of Hierarchical Factor Solutions." Psychometrika 22 (1): 53-61. Schneider, W . J . 2013. "What I f We Took Our Models Seriously? Estimating Latent Scores in Individuals." Journal of Psychoeducational Assessment 31 (2): 186-201 . Schneider, W . 1 , and K . S . McGrew. 2012. "The Cattell-Hom-Carroll Model of Intelligence." In Contemporary Intellectual Assessment: Theories, Tests, andIssues, 99-144. Schneider, W. 1, and K. S. McGrew. 2018. "The Cattell-Hom-Carroll Theory of Cognitive Abilities." In Contemporary Intellectual Assessment, 4th ed. Guilford, 73-163. SchOnemann, P. H., and J. H. Steiger. 1978. "On the Validity of Indeterminate Factor Scores." Bulletin of the Psychonomic Society 12 (4): 287-90. Sijtsma, K. 2009. "On the Use, the Misuse, and the Very Limited Usefulness of Cronbach's Alpha." Psychometrika 74 (1): 107-20. Sinharay, S. 2010. "How Often Do Subscores Have Added Value? Results from Operational and Simulated Data." Journal of Educational Measurement 47 (2): 150-74. Spearman, C. 1904. "'General Intelligence,' Objectively Determined and Measured." The American Journal ofPsychology 15 (2): 201-92. Spearman, C. 1933. "The Factor Theory and Its Troubles. III. Misrepresentation of the Theory." Journal of Educational Psychology 24 (8): 591-601 . Spearman, C. E. 1927. The Abilities of Man, Their Nature and Measurement London, UK: Macmillan. Steiger, J. H. 1979. "The Relationship between External Variables and Common Factors." Psychometrika 44 (1): 93-97. Steiger, 1 H., and P. H. SchOnemann. 1978. "A History of Factor Indeterminacy." In Theory Constrnction and Data Analysis in the Behavioral Sciences, 136-78. San Francisco, CA: Jossey-Bass.

106

Chapter Four

Sternberg, R. J. 2018. "Triarchic Theory of Successful Intelligence." In Contemporary Intellectual Assessment, 4th ed. Guilford. Sternberg, R. l, B. E. Conway, J. L. Ketron, and M. Bernstein. 1981. "People's Conceptions of Intelligence." Journal of Personality and Social Psychology 41 (1): 37-55. Strickland, T., M. W. Watkins, and L. C. Caterino. 2015. "Structure of the Woodcock-Johnson III Cognitive Tests in a Referral Sample of Elementary School Students." Psychological Assessment 27 (2): 68997. Styck, K. M., and M. W. Watkins. 2016. "Structural Validity of the WISC-N for Students with Learinng Disabilities." Journal of Learning Disabilities 49 (2): 216-24. Styck, K. M., and M. W. Watkins. 2017. "Structural Validity of the WISC-N for Students with ADHD ." Journal of Attention Disorders 21 : 921-28. Thorndike, R. L., and D. Lohman. 1990. A Century of Ability Testing. Chicago, IL: Riverside Publishing. Thorndike, R. M. 1990. "Origins of Intelligence and Its Measurement." Journal ofPsychoeducational Assessment 8: 223-30. Thorndike, R. M. 1997. "The Early History of Intelligence Testing." In Contemporary Intellectual Assessment. New York, NY: Guilford Press, 3-16. Thurstone, L. L. 1938. "The Perceptual Factor." Psychometrika 3 (1): 117. Thurstone, L. L. 1947. Multiple Factor Analysis: A Development and Expansion of the Vectors ofMind. Chicago, IL: University of Chicago Press. Watkins, M. W., and A. A. Beaujean. 2013. "Bi-Factor Structure of the WPPSI-N." School Psychology Quarterly 29 (1): 52-63. Watkins, M. W., G. L. Canivez, T. James, K. James, and R. Good. 2013. "Construct Validity of the WISC-N UK With a Large Referred Irish Sample." InternationalJournal ofSchool & Educational Psychology 1 (2): 102-1 l . Watkins, M. W., S. C. Dombrowski, and G. L. Canivez. 2017. "Reliability and Factorial Validity of the Canadian Wechsler Intelligence Scale for Children-Fifth Edition." International Journal of School & Educational Psychology, 6 (4): 252-265. Wechsler, D. 2002. Wechsler Individual Achievement Test. San Antonio, TX: Psychological Corporation.

Psychometric Issues with Specific Abilities

107

Yung, Y-F, D. Thissen, and L. D. McLeod. 1999. "On the Relationship between the Higher-Order Factor Model and the Hierarchical Factor Model." Psychometrika 64 (2): 1 1 3-28. Zaboski, B. A., J. H. Kranzler, and N. A. Gage. 2018. "Meta-Analysis of the Relationship between Academic Achievement and Broad Abilities of the Cattell-Horn-Carroll Theory." Journal of School Psychology 71: 42-56.

CHAPTER FIVE THE NETWORK ApPROACH TO GENERAL INTELLIGENCE HAN L . J. VAN DER MAAS, ALEXANDER O. SAVI, ABE HOFMAN, KEES-JAN KAN, & MAARTEN MARS MAN

Introduction The study of intelligence is a multidisciplinary endeavor. Progress has, for instance, been made in revealing genetic influences on intelligence (Plomin & von Stumm, 2018), in identifying relevant processes in the brain (Jung & Haier, 2007), and in artificial intelligence (AI). The recent successes with deep learning neural networks in AI are especially spectacular (Schmidhuber, 2015). Despite this progress, it is undisputed that human intelligence is still a puzzling concept. There is no consensus on the definition of intelligence, not even within specific fields that study intelligence, such as psychometrics. Within the latter, there are, for instance, major disputes over whether intelligence is one thing (a general ability) or multiple things (Gardner, 1995). In our view, it is important to have a clear conception of what intelligence is. In this chapter we adhere to such a conception and present a formal model of psychometric intelligence. This model is admittedly overly simplistic, yet illuminates mechanisms that can explain some important and well replicated phenomena in the study of intelligence. The model firstly acknowledges that the brain is a complex system (van der Maas, Kan, Hofinan, & Raijmakers, 2013). Complex systems are open systems consisting of many elements that interact non-linearly. Famous examples are ecosystems such as ant nests, weather phenomena such as hurricanes, and networks such as the internet. One remarkable feature is that such systems display self-organizing behavior: global

The Network Approach to General Intelligence

109

regularities, or phenomena, emerge from the local interactions. The interdisciplinary study of complex systems has resulted in many insights, models, and techniques (Strogatz, 2018; Weisbuch, 2018). These results are evidently of importance for the study of brain functions, especially higher order cognitive functions. Somehow our brains, consisting of billions of neurons, display intelligent behavior: an emergent phenomenon that results from intricate local interactions. Although complex systems research often requires a detailed mathematical model of the system under study, it is possible to model and explain certain empirical phenomena in a less exact way. An important example is the positive manifold in intelligence research. In 2006 van der Maas et al. proposed a completely new explanation of the positive correlations between cognitive tests, based on the idea of networks: a key modeling framework within complex system research. In this chapter we will a) explain the origin and setup of this so-called mutualism model, b) discuss criticisms that have been raised in response to the model, c) extend the model, d) present new statistical techniques for this model, and e) discuss new developments in the network modeling of general intelligence. The network approach is developed in response to the dominant factor approach in the study of general intelligence. Some background on this factor approach and the study of general intelligence is important to understand the network approach to intelligence.

The factor approach to general intelligence It is safe to say that the study of intelligence is the hallmark of differential psychology. The invention of intelligence testing, the development of factor analysis, and the theoretical model of mental power or g have had an enOlTIlOUS influence on the study of individual differences in general. That is, when psychologists study individual differences in some psychological trait, they generally follow the same approach. They construct tests consisting of items clustered in sub-tests, collect data in some sample, perfOlTIl factor analysis or employ some other latent trait approach, and arrive at the conclusion that differences in one or more latent traits explain observed individual differences. The justification of this approach, especially in intelligence research, mainly rests on its predictive power. Psychometric intelligence is a useful construct in personal selection and in the prediction of educational success (Deary et al., 2007). But there is more to science than prediction. Outside differential psychology, in cognitive and developmental psychology for instance,

1 10

Chapter Five

researchers tend to ignore the field of intelligence research. On the one hand, this is somewhat understandable, as the g model of intelligence does not provide an explanation of the 'mechanisms' underlying intelligent behavior. Intelligence is rarely incorporated in, or connected to, explanatory models of cognitive processes. This also holds for models of developmental processes (i.e. the basic g model has no real account for the growth of cognitive ability). It is also unclear if g itself develops and, if not, how intelligence develops (Ackerman & Lohman, 2003). In this sense, the g model is an anti-cognitive, anti-developmental model. On the other hand, the simple neglect of the concept of general intelligence, its empirical basis, and its predictive power, is not satisfactory. The pattern of positive correlations between scores on a wide variety cognitive tests, the positive manifold, is a very robust phenomenon. It is one of the most replicated findnigs ni psychology (Jensen, 1998). Intelligence tests are probably the best (and most controversial) tests the field of psychology has developed (Eysenck, 2018). The division between the study of intelligence on the one hand, and the studies of cognition and cognitive development on the other hand, illustrates the timely relevance of Cronbach's famous division of scientific psychology into two disciplnies, that of mechanisms and that of nidividual differences (Cronbach, 1957). Journals such as Intelligence, Structural Equation Modeling, Learning and Individual Differences on the one hand, and journals such as Cognition, Learning, Memory & Cognition, Brain, and the Journal of Mathematical Psychology on the other hand, represent very different approaches to essentially the same problem. Cronbach was somewhat optimistic about the unification of these two disciplines, but 60 years later the division is still prominent. We thnik one explanation of this state of affairs is the g factor model itself. The main problem with the factor approach lies in the theory. It is simply unclear what g is. It is remarkable that in thousands of publications on factor models of general intelligence this essential question is ignored. Some researchers take a practical ponit of view and just highlight the predictive value of g (Gottfredson, 1998). Others take a statistical point of view and argue that they are not obliged to take a position about the ontological status of latent variables (Jonas & Markon, 2016). We do not think these answers are satisfynig. A theoretically relevant application of the latent variable model requires an answer to questions about the origin and causal nature of individual differences, hence of the latent variable, but there is no consensus at all on the cognitive or biological basis of g (Ackerman, Beier, & Boyle 2005).

The Network Approach to General Intelligence

111

This state of affairs inspired us to look for new explanations of the positive manifold of correlations between cognitive tests. As said, the positive manifold is a robust phenomenon. Since the g model, in whatever variant, is often conceived of as the only possible explanation of this phenomenon, the positive manifold is seen as evidence for g. However, at least two completely different kind of formal explanations exist. One explanation, mutualism theory, is relatively recent and the focus of this chapter. The other one, sampling theory, has been out there for a century, largely ignored, but recently re-introduced.

Sampling theory It is important to understand sampling tbeory. Bartholomew, Deary, and Lawn (2009), and more recently Kovacs and Conway (2016) re­ introduced the sampling tbeory of general intelligence, originally advocated by Thorndike (1927) and Thomson (1951). In these models tbe positive correlations between test scores are due to the fact that any two cognitive tests always share some underlying basic processes (or bonds). That is, cognitive tests are insufficiently specific. The overlap in shared processes will necessarily result in positive correlations between tests. Bartbolomew, Deary, and Lawn (2009) generalized Thompson's (1951) model to account for multiple latent factors. Kovacs and Conway (2016) proposed a more elaborate version of sampling theory in order to account for the effects of domain-general executive processes, identified primarily in research on working memory, as well as more domain-specific processes. Although we think tbat both approaches are unclear in some important respects (Kan, van der Maas, & Kievit, 2016), we do believe sampling is part of the explanation of tbe positive manifold. A sampling type of explanation has also been repeatedly proposed for the relation between genes and phenotypes. For instance, Carmon and Keller's (2006) watershed model describes how specific genes influence 'upstream' endophenotypes. Kievit et al. (2016) extended this model to tbe domain of fluid intelligence. Another example is Anderson's (2001) model of the relation between genes and g.

Mutualism Mutualism or interspecific cooperation is a biological concept like predation and parasitism, and is an important mechanism in the evolution of ecosystems. The mutualism model of general intelligence explicitly takes the ecosystem as a metaphor for our cognitive system. We can, for

1 12

Chapter Five

instance, think of our cognitive system as a lake. In explaining the positive manifold of correlations between cognitive tests, we could think about the differences between lakes. Suppose we investigate the quality of several small lakes in say Europe. We set out to determine how well these ecosystems function, for instance regarding biodiversity. We start out to collect data concerning many different aspects of the lakes, such as measures of the quality of water, and the number and diversity of flora and fauna. Suppose we observe a positive manifold in these measures. In a factor analyses of such data we will observe a dominant general factor (Krijnen, 2004; van Bork et aI., 2017). This would imply that 'good' lakes are better, to varying degrees, than 'bad' lakes in most measured aspects. This raises the question whether there is a g factor for lakes. Is there an equivalent of mental power for ecosystems? We carmot exclude this possibility, but it is not how biologists think. They have other explanations of these phenomena in ecosystems. In mathematical biology, ecosystems are often modeled as systems of coupled differential or difference equations, or more generally as networks. Famous is the Lotka-Volterra model for prey-predator population dynamics (May, 1973; Murray, 2002). This model generates a number of fascinating and complex phenomena, as described in any basic text on population dynamics. Current more realistic models can be very complex and detailed, providing accurate descriptions of, and predictions concerning, the dynamics of large ecosystems (e.g., Prakash & de Roos, 2004). In these models, competitive and cooperative interactions are essential, and will give rise to correlations between different aspects or parts of the ecosystem. This type of interactions in multivariate dynamical systems (see van Geert, 1991) forms the basis of our dynamical explanation of the positive manifold of cognitive perfonnance. We argue that the positive manifold may be a by-product of the mutualistic interactions between different cognitive functions that occur during development. In our proposal, all functions or processes of the system are initially undeveloped and their traits are uncorrelated. During the development of the cognitive system, the dynamical interactions give rise to cognitive growth and to correlations among the various cognitive abilities.

The mutualism model of intelligence The model of van der Maas et al. (2006) is the Lotka-Volterra mutualism model (Murray, 2002). It is based on a few assumptions. First it

The Network Approach to General Intelligence

113

is assumed that the cognitive system consists of distinguishable parts or functions. There is clearly little consensus about the specific basic processes underlying intelligent behavior (Deary, 2002, p . lS3), but what these processes or functions exactly are is only of secondary importance. It is possible to adopt one theoretical framework, like the Sternberg (1988) model or the minimal cognitive architecture model of Anderson (1992), and develop the mutualism model in these telTIlS. In general terms, our 'species' are cognitive abilities as they are for instance measured in intelligence tests. The second assumption is that each of these abilities undergoes auto-catalytic growth as the development of cognitive processes is largely an autonomous self-regulating process (Molenaar, Boomsma, & Dolan, 1993). The third assumption is that this growth is constrained by limited resources such as neuronal speed and the size of neural systems associated with each of the cognitive processes (Garlick, 2002; Jensen 1998). For instance, short term memory reaches a maximum in late childhood, presumably because of limitations of the underlying neural system (e.g., Gathercole, 1999). The fourth assumption is that the limited resources are subject to both genetic and environmental influences. Assuming that these influences are additive suffices to explain several genetic effects in intelligence research. The last major assumption is that cognitive abilities interact dynamically and that therefore cognitive abilities have mutual beneficial relations. Reciprocal causal relations are well knO\vn in the psychological literature. For instance, better short term memory stimulates developing better cognitive strategies, and better strategies increase the efficiency of short term memory (Siegler & Alibali, 2002). Examples of positive influences of language on cognition, and visa versa, are syntactic bootstrapping (Fischer, Hall, Rakowitz, & Gleitman, 1994) and semantic bootstrapping (pinker, 1994). Other examples are the relations between cognition and meta-cognition (Sternberg, 1998), action and perception (Gibson, 1986), and performance and motivation (Dweck, 1986). Some recent papers confirm the existence of such mutualistic relations (e.g., Peng, Wang, Wang, & Lin, 2018). These relations might be weak but, as we shall see later, only small and sparse interactions are required to generate the positive manifold. We also expect that some of these relations operate via the social environment. Success in one task domain may lead to selection for certain trainings that benefit the task domain and other related domains. This is the idea behind the social multiplier effect model of Dickens (2007). This idea can be incorporated in the mutualism model without further assumptions (see van der Maas et aI., 2017).

1 14

Chapter Five

It is possible that there are no facilitating interactions between certain processes, or even competitive or debilitating interactions. A simple example of the latter is the time constraint on cognitive expertise. Becoming an expert in say, music, may not allow other specializations. Van der Maas et al. (2006) demonstrated with computer simulations that the mutualism model can include a good degree of zero or competitive interaction, without affecting the fundamental result of the positive manifold of correlations. Given these assumptions we can formulate the model:

for i

=

1. . W

(1) (2)

Variables Xl represent the W cognitive abilities. Parameters G) are growth parameters, detelTIlining the steepness of the logistic growth function associated with each Xl. Parameters KJ represent the limited resources of the logistic growth processes. The logistic growth function is in accordance with the assumption of auto-catalytic growth. The matrix M contains the interactions My, used to specify the, possibly asymmetric, relations between pairs of abilities in development. K itself is a weighted (by cl) sum of genetic and environmental contributions. The parameters Xo (initial values), a, and K differ over subjects, whereas the matrix M contains population parameters, equal for all subjects. Each ability starts its growth process at a low value and follows a logistic growth curve until an asymptote (larger than K,) is reached. At any point in time t we can take a measurement of x. By rerunning the model a number of times (n), we collect data for n subjects on W cognitive abilities measured at time t. This matrix of data can be subjected to correlational and factor analysis. Three scenarios were investigated in van der Maas et al. (2006): 1) all Kl are uncorrelated and all My = 0, 2) all Kl are correlated and all My = 0, and 3) all K, are uncorrelated and all My � .05. In the first scenario, we found no correlations between the Xl, as expected. The second scenario is in concordance with the g-factor case, as the limited resources do share some common source of variance. In this case we found the expected positive manifold. The third scenario represents the mutualism case. In this case, we also found the positive manifold, even though all individual parameters were uncorrelated, and hence the positive interactions produced the positive manifold.

The Network Approach to General Intelligence

115

Other simulation results Van der Maas et al. (2006) investigated whether the mutualism model could explain other phenomena in intelligence research, without adding new assumptions. One interesting example is the form of M. Initially all My were set to the same value (scenario 3), but it is more realistic if the interactions show some variation. If the interactions are sampled from a nonnal distribution with a mean of .05 and a standard deviation of .06, about 20% of the interactions are expected to be negative. Such an M leads to correlation matrices that require more complex factor models. It was shown that the correlations were well described by higher-order factor models. These higher-order factor models are the standard in current factor models of actual intelligence data. Another conclusion of this simulation is that the presence of the positive manifold requires only a few weak interactions in the mutualism model. This is important, since the empirical results on transfer between cognitive abilities often show no or weak effects, especially with regard to far-1ransfer (Schwaighofer, Fischer, & Biihner, 2015). The literature on transfer is somewhat inconsistent, but transfer effects are often disappointing. Additionally, we studied several developmental effects. First, the low predictive validity of test performance during infancy and early childhood (Honzik, 1983; McCall & Carriger, 1993) easily follows from the model. Second, the model also explains the increase in heritability during development (Bartels, Rietveld, van Baal, & Boomsma, 2002; Fulker, DeFries, & Plomin, 1988; Haworth et al. 2010). Initially, the values of the variables x are determined by the growth parameters a. The influence of K is low in this phase. Only later in development, when variables x reach their asymptotic values, K comes into play, and thus the genetic influences onK.

Criticism Van der Maas et al. (2006) mention a number of limitations. First, one can ask whether the mutualism model is really different from the factor model and sampling theory. In our view, although all these models explain the positive manifold equally well, they are conceptually very distinct. In the g-factor model the correlations are produced by a common source of cognitive performance in many domains. The g-factor is understood as a so-called reflective latent variable. In the network model there is no common source; the positive manifold is produced by the network structure. The statistical g-factor on the other hand, is interpreted as a

1 16

Chapter Five

formative variable; an index of the general quality of the cognitive system, akin to economic indexes such as the Dow Jones Industrial Average. Also in sampling theories the statistical g-factor should be interpreted as a fOlmative variable. Yet test sampling theory and the mutualistic network model are very different. In test sampling theory the positive manifold is essentially a measurement problem. If we would be able to construct very specific tests, targeted at the single basic processes, the overlap in measurement would disappear, and so would the correlations between tests. In the mutualism model the correlations are real, created during development, and will not disappear when our IQ tests become more specific. Second, the mutualism model does not constitute a parsimonious explanation of the data, in the sense that M may contain many parameters, we agree, but to our defense, we do not assume the presence of an unnecessary, rather mysterious latent variable. With respect to the number of entities (explanatory variables), the g model is the more expensive model. Moreover, M can be sparse and still give rise to a positive manifold. Many kinds of restrictions may be placed on the elements in the matrix M, which can be based on theoretical considerations, or the results of experimental studies. To explain the positive manifold, it suffices to assumeMij = c for all subjects. Third, the model is extremely simplistic. To name a few aspects, M most likely differs between subjects, not all growth processes will start at the same moment, and the linear model for the genetic and environmental impact disregards interactions. However, an advantage is that this model can easily be investigated using simulations and fOlTIlal proofs. FurthelTIlore, the simple mutualism model does explain key phenomena in intelligence research. Fourth, the mutualism model in its original fOlTIl carmot explain the so­ called Jensen effect without additional assumptions. The Jensen-effect refers to the correlation between the vector of g-loadings of cognitive tests and the heritability coefficients of these tests, using the method of correlated vectors. Jensen (1998) computed, for a number of data sets, such correlation, which averaged about .S. This is taken as evidence for a biological and genetic interpretation of K. However, this interpretation is problematic. Kan, Wicherts, Dolan, & van der Maas (2013) have shown that the heritabilities and g-loadings are higher for crystallized tests (such as vocabulary and arithmetic) than for typical fluid tests. This finding runs against g theory. FurthelTIlore, the extended mutualism model proposed in van der Maas et al. (2017) is able to explain this empirical pattern.

The Network Approach to General Intelligence

117

Some authors added other points of criticism. Nisbett et al. (2012) mention that the mutualism model does not distinguish between genetic and enviromnental effects and integration with Dickens' (2007) model could provide a more compelling description of the development process. This is partly incorrect (see equation 2), but we agree that the multiplier effect is important. In van der Maas et al. (2017) it is explained how such integration could be established by a redefmition of M. That is, one Xl is used to represent the cognitive environment, which effectively incorporates the multiplier effect underlying Dickens' model. Based on several data sets, Gignac (2014) claims that the g-factor is stable from 3 years on, which he sees as an indirect rejection of the mutualism model. This was recently replicated by Shahabi, Abad, and Colom (2018) with data in the age range of 6 to 12. In general, we think the literature on age (de-)differentiation is rather inconclusive due to various methodological problems (van der Maas, et al. 2006). In most studies, the first years of development are ignored. Also, the mutualism model does not specify the age at which the statistical g-factor should become stable. Also note that it is unclear what the g-theory predicts. According to Gignac, g-factor theory may be suggested to predict the strength of the g factor to be largely constant across all ages, because this theory postulates biological and genetic substrates for g. But the biological substrate is all but constant in childhood, and g-theory is unspecific with regard to such developmental issues. At least a fOlTIlal derivation of a prediction on the change in the strength of g is missing. Gignac (2016) also criticizes the mutualism model. He reasons that it would be very unlikely that the pattern of mutually beneficial interactions between cognitive abilities across individuals arise precisely in a marmer that their latent variable inter-associations can be accounted for by a single latent variable. Van der Maas and Kan (2016) provided a rebuttal to these criticisms, showing that Gignac's main premise is incorrect. In our current evaluation of the model we raise a new limitation of the model. From a developmental point of view the model makes some questionable assumptions. All the nodes and connections are available from birth, and only the values of x change. Below we will discuss the possibility to use growing networks as models of cognitive development.

The extended 'unified' mutualism model Van der Maas et al. (2017) summarize the various extensions that have been proposed over the years. Mutualism remains the core of this model. The addition of a node that represents the environment incorporates

1 18

Chapter Five

Dickens multiplier idea. Cattell's investment theory can be represented by asymmetric relations in the interaction matrix processes

may

be

more

central than

M.

others,

Furthennore, some fluid reflected by

stronger

connections to other nodes. Such central processes, for instance working memory, will then correlate strongly with the statistical g-factor All these extensions can be included into a unified network model with the specification of

M.

We also think that sampling is part of the explanation

of the positive manifold. Our current tests of intellectual functions clearly lack specificity (Lum sden, 1 976). Our 'unified' model is shown in Figure 5-1.

Figure

5- 1 . The unified model o f general intelligence includes test sampling,

reciprocal effects (both rnutualistic and multiplier), and central cognitive variables (such as working memory capacity,

xfl).

The

Xf

and � nodes represent fluid and

crystalized cognitive abilities in the intelligence network. The fi and

Ci

represent

test scores for these abilities, the sum of which is IQ. The g-factor can be extracted using factor analysis on f (and c) tests.

The Network Approach to General Intelligence

Fluid abilities (xJ

Crystallized abilities (xJ 02 .02 .02 .02 .02 .02 .02 .02 02 .02 .02 .02 .02 .02 .02 .02 02 .02 .02 .02 .02 .02 .02 .02 02 .02 .02 .02 .02 .02 .02 .02 02 .02 .02 .02 .02 .02 .02 .02 02 .02 .02 .02 .02 .02 .02 .02 02 .02 .02 .02 .02 .02 .02 .02

ili . M . M .M M M M M ili . M . M .M M M M M ili . M . M .M M M M M ili . M . M .M M M M M ili . M . M .M M M M M ili . M . M .M M M M M ili . M . M .M M M M M ili . M . M .M M M M M d d d d d d d d

oo m m m m m m m oo m m m m m m m oo m m m m m m m oo m m m m m m m oo m m m m m m m oo m m m m m m m oo m m m m m m m oo m m m m m m m d d d d d d d d . 0

.

M

Cognitive environment (x,)

00 . 02 . 02 . 02 .02 .02 .02 .02 04 . 00 . 02 . 02 .02 .02 .02 .02 .04 . 02 . 00 . 02 .02 .02 .02 .02 .04 . 02 . 02 . 00 .02 .02 .02 .02 .04 . 02 . 02 . 02 .00 .02 .02 .02 .04 . 02 . 02 . 02 .02 .00 .02 .02 .04 . 02 . 02 . 02 .02 .02 .00 .02

.

119

Xe

Figme 5-2. M matrix for the lUlified model in Figure 5-2.

With this extended model most limitations of the basic mutualism model are resolved. However, the last limitation we mentioned requires a very different type of network model that we will describe in the last section.

Fitting the model to data Much has been said on the relation between factor and network models. One confusuig and equally fascinatuig fact, is that in both the factor approach and the network approach we can perform factor analysis, with similar statistical results. The difference lies purely in the interpretation of the statistical factor. In our view, the difference between the reflective and fOlmative interpretation of the statistical latent variable is of fundamental importance (Edwards, & Bagozzi, 2000). In the reflective model, the latent variable is the common cause of the scores on the observed variables (Figure 5-3). A good example is the thermometer. Suppose we have a set of thelTIlometers of mediocre quality. When we collect sets of measurements in different locations, we can subject the data to factor analysis. The resultuig factor will represent a physical realistic variable that is the source of the differences in measurement values on each of the thermometers across the locations. In

Chapter Five

120

contrast, in the formative model the latent variable is an index of the state of some complex system. Economic indices are excellent examples; they summarize an economic situation, rather than cause economic success. Another example is health. Health measures summarize the overall state of the body (pIsek & Greenhalgh, 200 I), rather than cause better health reflective model

formative model

Figure 5-3. In the reflective model the latent variable (e.g., temperatnre) is the

common cause of the manifest scores (e.g., thermometer values at different locations). In the formative model the latent variable (e.g., economical index) only summarizes the manifest scores (e.g., health index).

Standard structural equation models will thus not be decisive on the issue, but new possibilities have appeared recently. Since 2006, network science has become a productive area of research in psychology, with applications in especially clinical psychology. As in the mutualism model, the

general

hypothesis

in

psychological

network

models

is

that

correlations between observed behaviors, such as cognitive abilities, psychopathological symptoms, or attitudes, are not due to unobserved common causes, but rather due to the network of interacting psychological and/or biological factors. This approach has led to a number of new insights concerning, for instance, the co-morbidity of clinical symptoms (Cramer, Waldorp, van der Maas, & Borsboom, 2010), early warnings for clinical disorders (van de Leemput et aI., 20 1 4) and the organization of attitudes (Dalege et aI., 2016). This work also led to new statistical analysis and visualization techniques (for

an

overview see Epskamp, Borsboom, & Fried, 201 8). The

most popular models that are used for cross-sectional data are the Gaussian Graphical Model (GGM) for continuous data and the Ising model for binary data. Network changes can be estimated using vector-

The Network Approach to General Intelligence

121

autoregression (VAR). A criticism of the validity of network psychometrics can be found in Forbes, Wright, Markon, & Krueger (2017) with a reply in Borsboom et aI. (2017). For the mutualism model, the work of Kievit and colleagues (20l7a, 20l 7b) is particularly relevant. They extended the latent change score model (LCSM) to bivariate LCSM (McArdle, 2001; McArdle et aI., 2002) such that the relationship between the developments of abilities in two separate domains can be investigated.

.il,.pt .ilz.pt

=

=

P,Y,.p(t-,) + YZlYZ.p(t-l) PzYZ.p(t-l) + Y12Y,.p(t-,)

In this model, a change score (.il) is defined for both domains for person p over time t. The autocatalytic parameter, p, models the growth rate, with a positive p indicating accelerating growth and a negative p indicating decelerating growth. The coupling parameters y reflect the effect of the previous score in one domain (i.e., at time t-1) on the change in score in the other domain. By examining the 121 and 112 parameters, we can detelTIline which domains influence the development of other domains. The idea is that the difference between a g-factor and mutualism model is reflected in the 1 parameters. This is because there is a clear relation between the y parameters in the LCSM and the M matrix in the mutualism model. This technique has now been applied to various datasets (Kievit et aI. 20l7a, 20l 7b; Kievit, Hofinan, Nation, 2018, Hofinan et aI. 2018). In all cases evidence for mutualistic coupling is reported. Psychometric network modeling can also be used as an exploratory means to help detelTIline the number of factors in cognitive data (Golino and Demetriou, 2017). In Kan, van der Maas & Levine (2018) we propose a more confilTIlatory approach to test competing network and factor analytic theories of intelligence against each other. The analysis requires the computation of a Gaussian graphical model of the full partial correlation matrix, which results in a saturated network model. Subsequently, by means of a graphical lasso procedure (Friedman, Hastie, & Tibshirani, 2014), with the built-in tuning parameter ('lambda min ratio') procedure and the extended Bayesian information criterion, the network model is adjusted for the presence of false positive edges (Epskamp et aI., 2012). This results in a non-saturated sparser network model (the 'glassoed network'). Once such model is implemented in the SEM framework and compared to competing models, we may consider the procedure as an example of 'the exploratory mode' of confirmatory modeling (Raykov &

122

Chapter Five

Marcoulides, 2006). Ideally, in a true confimmtory analysis, one would specify the network a priori. The glassoed network can be compared with the usual indices (AIC, BIC) with factor models, such as the higher-order factor model and the bi­ factor model. In a simulation study, Kan, van der Maas & Levine (2018) demonstrate that this approach does not lead to false positives in favor of the glassoed network. Next, they investigated several empirical data-sets and found that the mutualism model generally provides a more satisfactory solution than previous established factor models.

Wired cognition As mentioned before, the dynamics of the (extended) mutualism model are limited. All cognitive abilities x and connections M are there from the beginning, and only the values of x change. Although an improvement over g theory, we think this limitation of the mutualism model requires attention. Therefore, we built the framework of a new theory, in which nodes and connections are dynamically added to the cognitive network (Savi, Marsman, van der Maas, & Maris, 2018). This wired cognition model starts with some rudimentary skills or knowledge, represented by just a few nodes. Over time, it grows to a well connected, extended network of cognitive skills. An example would be arithmetic. Toddlers may know some numbers and perhaps can count to ten. In this situation, the nodes are unstable and the connections are weak. Over time they learn more facts and arithmetic operations, and nodes and connections are added to their cognitive networks. The growth of these networks can be modeled by means of, for instance, a preferential attacliment rule (Barabasi & Albert, 1999). In this way a semantic network can develop with clusters for specific sub-abilities like multiplication. One advantage of this model is that it relates intelligence to education. Teachers help students building their networks of expertise. In Savi et al. (2018) we studied a very specific network, the Fortuin­ Kasteleyn (FK) network model (Fortuin, & Kasteleyn, 1972). This model allows a formal derivation of developmental properties. It can be sho\Vll analytically that an FK model gives rise to the positive manifold. Similar to the mutualism model, this does not require a latent factor. And importantly, the genetics, the environment, and the development of a person are explicitly captured in the cognitive network, by means of the initial network structure, the gradual addition of nodes and edges to the network, and a growth mechanism. The model succeeds in describing

The Network Approach to General Intelligence

123

individuals by unique cognitive networks, making it a truly idiographic model. Future work will include the effect of forgetting, as weakly connected or unconnected nodes are probably prone to be forgotten. A toddler may keep the fact that 5 + 5 � l O in memory for some time, but if this fact is not connected to other knowledge, it might be forgotten soon. Additionally, weakly connected networks allow for sub clusters with incorrect knowledge (misconceptions) or deficient skills. This work is part of a grander goal to understand the growth (and decline) dynamics of a developing intelligence.

Discussion Since the introduction of the mutualism model, network psychometrics and network modelling are rapidly gaining ground in psychology. Conceptualizing psychological traits and abilities as networks of interacting components has become a serious alternative for latent variable modelling. Current applications of network modelling range from psychopathology, attitudes (Dalege, Borsboom, van Harreveld, & van der Maas, 2019), personality (Costantini et aI., 2015), interest (Sachistal, 2018), and intelligence. In the field of psychopathology, the range of applications is especially impressive. Recent work concerns depression, anxiety disorders, post-traumatic stress, complex bereavement, autism, psychotic disorders, substance abuse, the general structure of psychiatric symptomatology, diagnostic manuals, health-related quality of life, and personality traits (for an overview and references, see Borsboom, 2017; Borsboom, Robinaugh, Psychosystems Group, Rhemtulla, & Cramer, 2018). In the last 10 years, we developed and extended our unified mutualism model that incorporates several additional explanatory mechanisms. We distinguish between fluid and crystalized abilities, added test sampling, and allow for a multiplier effect via the environment, which is important to explain various gene-environment correlation effects (Kan, Wicherts, Dolan, and van der Maas, 2013). These extensions are easily incorporated in the basic model. We also accept the idea that some abilities are more central than others. For instance, working memory may be more central than other abilities and therefore display a higher g-loading in statistical analyses. In van der Maas et al. (2017), we argued that network models can give rise to both continuous and discontinuous developmental change. The network approach thus also relates categorical and continuous latent variable models of individual differences.

124

Chapter Five

This extended mutualism model is still very different from g-theory. In the network model the statistical main factor is just an index of the state of the network and not a causal factor itself. In our view, the difference between the fOlmative and reflective interpretation of the statistical g­ factor is of utmost importance and not only for theoretical reasons. First, if the mutualism model is correct, the search for a gene or brain area nfor gt! will be of limited value. By assuming that the uncorrelated capacities K's are partly genetically influenced the mutualism model explains both the high heritability for general intelligence found in twin studies, as well as the increase of this heritability over time (see van der Maas et al., 2006). The recent successes in finding genome sequence differences accOlUlting for significant contributions to the heritability of intelligence (for an overview, see Plomin & von Stumm, 2018) does not change the picture. Such results cannot prove that g is a biological causal entity as they are completely in line with the prediction of the mutualism model (and also sampling theory). Also, the biological unitary g factor is perhaps inconsistent with Plomin and Stumm ' s conclusion that the genetic effects on intelligence are extremely pleiotropic and that hundreds of thousands of SNP associations are needed to account for the 50% heritability estimated by twin studies. Secondly, there are implications for measurement (van der Maas, Kan, & Bosboom, 2014). A crucial difference between reflective and formative models concerns the role of the indicator variables (items or subtests). In the g-model, a reflective model, these indicators are exchangeable. New tests, with different factor loadings, could be added to a test battery without changing the essence of the measured g-factor. In fOlmative models, however, indicators are not exchangeable. 'What we put in the index is a choice that rests on pragmatic grounds, for instance in its predictive value. The best IQ test is simply the combination of tests that best predicts educational or job success. 'When a certain cognitive capacity, say computational thinking, is valued to a greater extent in the current society, intelligence tests should be adapted to reflect individual differences therein. The wired cognition model that we introduced in this paper can be seen as a successor of the mutualism model of general intelligence. Both are network models inspired by formal modeling approaches in the natural sciences. However, the wired cognition model is not an extension of the mutualism model; it differs in too many respects. In the mutualism model the architecture of the network is fixed, all nodes and edges (basic processes) are there from the beginning. The wired cognition model on the other hand, is a model of a developing network.

The Network Approach to General Intelligence

125

Both models share two important ideas. The first is that models of individual differences should be based on models of tbe individual. Witb the network model we intend to bridge two separate research traditions, on the one hand the experimental research on cognitive mechanisms and processes, on the other hand the psychometric research on individual differences in intelligence. Cronbach's (19S7) famous division of scientific psychology in two disciplines is still very true for the fields of cognition and nitelligence. In order to bring tbese fields together we developed a mechanistic model in the cognitive tradition with the aim to explain a set of well-established key phenomena in the research of individual differences. The second shared idea is that a scientific tbeory should be formulated as a mathematical or computational model. In cognitive research, the largely verbal Piagetian and neo-Piagetian models have been succeeded by concrete computational infOlmation processing models, both symbolic and connectionist. The traditional factor models of general intelligence are different ni that they are statistical models of individual differences. They do not specify a (formal) model of intelligence in the individual. Samplnig models and mutualism model have been formulated mathematically. The advantages are that models have to be precisely defined, prediction can be derived unambiguously and unexpected and undesirable by-effect of the model can be detected, for instance, in simulations.

References Ackerman, P. L., & Loliman, D. F. (2003). Education and g. In H. Nyborg (Ed.), The scientific study of general intelligence. Amsterdam: Pergamon. Ackerman, P. L., Beier, M. E., & Boyle, M. O. (200S). Working memory and intelligence: The same or different constructs? Psychological Bulletin, 131, 30-60. Anderson, B. (2001). g as a consequence of shared genes. Intelligence, 29, 367-371. Anderson, M. (1992). Intelligence and development: A cognitive tbeory. Oxford: Blackwell. Barabasi, A. L., & Albert, R. (1999). Emergence of scaling in random networks. Science, 286(S439), S09-S 12. Bartels, M., Rietveld, J. H., van Baal, G. C. M., & Boomsma, D. I. (2002). Genetic and environmental influences on the development of intelligence. Behavior Genetics, 32(4), 236-249. Bartbolomew, D. J., Deary, I. J., & Lawn, M. (2009). A new lease of life

126

Chapter Five

for Thomson's bonds model of intelligence. Psychological Review, 1 1 6(3), 567. Borsboom, D. (2017). A network theory of mental disorders. World Psychiatry, 1 6(1), 5-13. Borsboom, D., Fried, E. I., Epskamp, S., Waldorp, L. J., van Borkulo, C. D., van der Maas, H. L., & Cramer, A. O. (2017). False alarm? A comprehensive reanalysis of "Evidence that psychopathology symptom networks have limited replicabilily" by Forbes, Wright, Markon, and Krueger. Journal ofAbnormal Psycholo?,Y, 126(7), 989. Borsboom, D., Robinaugh, D. J., Psychosystems Group, Rhemtulla, M., & Cramer, A. O. (2018). Robustness and replicability ofpsychopatliology networks. World Psychiatry, 1 7(2), 143-144. Cattell, R. B. (1971). Abilities: Their structnre, growtli, and action. Oxford: Houghton Mifflin. Costantini, G., Richetin, J., Preti, E., Casini, E., Epskamp, S., & Perugini, M. (2017). Stability and variability of personality networks. A tutorial on recent developments in network psychometrics. Personality and Individual Differences, 136, 2019, Pages 68-78 Cramer, A. O. J., Waldorp, L. J., Van der Maas, H. L. J., & Borsboom, D. (2010). Comorbidity: A network perspective. Behavioral and Brain Sciences, 33(2-3), 137-193. Cronbach, L. J. (1957). The two disciplines of scientific psychology. American Psycholofiist, 12(11), 671. Dalege, J., Borsboom, D., van Harreveld, F., & van der Maas, H. L. I. (2019). The Attitudinal Entropy (AE) Framework as a General Theory of Individual Attitudes. Psycholofiical Inquiry. Dalege, J., Borsboom, D., van Harreveld, F., van den Berg, H., Conner, M., & van der Maas, H. L. J. (2016). Toward a formalized account of attitudes: The Causal Attitude Network (CAN) model. Psychological Review, 123(1), 2-22. Deary, I. J. (2002). g and cognitive elements of information processing: An agnostic view. In R. J. Sternberg & E. L. Grigorenko (Eds.), The general factor of intelligence (pp. 151-1 82). Mahwah: Lawrence Erlbaum. Deary, I. J., Strand, S., Smith, P., & Fernandes, C. (2007). Intelligence and educational achievement. Intellifience, 35(1), 13-21. Dickens, W. T. (2007). What is g? Washington, DC: Brookings Institution. Dweck, C. S. (1986). Motivational processes affecting learning. American Psychologist, 41(10), 1040-1048.

The Network Approach to General Intelligence

127

Cannon, T. D., & Keller, M. C. (2006). Endophenotypes in the genetic analyses of mental disorders. Annu. Rev. Clin. Psychol., 2, 267-290. Edwards, J., & Bagozzi, R. (2000). On the nature and direction of relationships between constructs and measures. Psychological Methods, 5(2), 155-174. Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods, 50(1), 195-212. Epskamp, S., Cramer, A. 0., Waldorp, L. J., Schmittmann, V. D., & Borsboom, D. (2012). Qgraph: Network visualizations of relationships in psychometric data. Journal ofStatistical Software, 48(4), 1-18. Eysenck, H. (2018). Intelligence: A new look. Routledge. Fisher, C., Hall, D. J., Rakowitz, S., & Gleitman, L. (1994). When it is better to receive than to give: Syntactic and conceptual constraints on vocabulary growth. In L. Gleitman & B. Landua (Eds.), The acquisition of the lexicon (pp. 333-375). Cambridge: MIT Press. Forbes, M. K., Wright, A. G., Markon, K. E., & Krueger, R. F. (2017). Evidence that psychopathology symptom networks have limited replicability. Journal ofAbnormal Psychology, 1 2 6(7), 969. Fortuin, C. M., & Kasteleyn, P. W. (1972). On the random-cluster model. Physico, 57(4), 536-564. Friedman, J., Hastie, T., & Tibshirani, R. (2014). Glasso: Graphical lasso­ estimation of Gaussian graphical models. Retrieved from https:llcran.r­ project.orglweb/packages/glassol (R package version 1.0). Fulker, D. W., DeFries, J. C., & Plomin, R. (1988). Genetic influence on general mental ability increases between infancy and middle childbood. Nature, 336(6201), 767-769. Gardner, H. (1995). Reflections on multiple intelligences: Myths and messages. Phi Delta Kappan, 77(3), 200. Garlick, D. (2002). Understanding the nature of the general factor of intelligence: The role of individual differences in neural plasticity as an explanatory mechanism. Psychological Review, 1 09(1), 1 16-136. Gathercole, S. E. (1999). Cognitive approaches to the development of short-term memory. Trends in Cognitive Sciences, 3(1 1), 410-418. Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Gignac, G. E. (2014). Dynamic mutualism versus g factor theory: An empirical test. Intelligence, 42, 89-97. Gignac, G. E. (2016). Residual group-level factor associations: Possibly negative implications for the mutualism theory of general intelligence. Intelligence, 55, 69-78.

128

Chapter Five

Golino, H. F., & Demetriou, A. (2017). Estimating the dimensionality of intelligence like data using exploratory graph analysis. Intelligence, 62, 54-70. Gottfredson, L. S. (1998). The general intelligence factor. Scientific American Presents, 9, 24-29. Haworth, C. M., Wright, M. J., Luciano, M., Martin, N. G., de Geus, E. J., van Beijsterveldt, C. E. M. ,Bartels, M., Posthuma, D., Boomsma, D. I., Davis, O. S. P., Kocas Y, Corley, R. P. , DeFries, I. C., Hewitt, J. K., Olson, R. K., Rhea, S.-A., Wadsworth, S. J., Iacono, W. G., McGue, M., Thompson, L. A., Hart, S. A, Petrill, S. A., Lubinski, D., & P10min, R. (2010). The heritability of general cognitive ability increases linearly from childhood to young adu1tliood. Molecular psychiatry, 15(11), 1 1 12. Hofman, A.D., Kievit, R., Stevenson, c., Molenaar, D., Visser, I, & van der Maas, H. L. J. (2018). The dynamics of learning maths: A comparison of the g-factor and mutualistic network theory. PsyArxiv preprint. Retrieved from https:lldoi.orgIl0.31219/0sf.io/xa2ft Honzik, M. P. (1983). Measuring mental abilities in infancy. New York: Plenum Press. Jensen, A. R. (1998). The gfactor: The science ofmental ability. Westport CT US: Praeger Publishers I Greenwood Publishing Group Inc. Jonas, K. G., & Markon, K. E. (2016). A descriptivist approach to trait conceptualization and inference. Psycholofiical Review, 123(1), 90. Jung, R. E., & Haier, R. J. (2007). The Parieto-Frontal Integration Theory (P-FIT) of intelligence: convergmg neuronnagmg evidence. Behavioral andBrain Sciences, 30(2), 135-154. Kan, K. J., van der Maas, H. L., & Kievit, R. A. (2016). Process overlap theory: Strengtlis, limitations, and challenges. Psychological Inquiry, 27(3), 220-228 Kan, K. J., Wicherts, J. M., Dolan, C. V., & van der Maas, H. L. (2013). On the nature and nurture of intelligence and specific cognitive abilities: The more heritable, the more culture dependent. Psychological Science, 24(12), 2420-2428. Kan, K. J., van der Maas, H. L. J. & Levine (submitted). Extended psychometric network analyses on cognitive data from a community­ based sample reveal evidence against g in favor of mutualism. Manuscript submittedfor publication. Kievit, R. A., Hofman, A. D., & Nation, K. (submitted). Mutualistic coupling between vocabulary and reasoning in young children: A replication and extension of Kievit et al.

The Network Approach to General Intelligence

129

Kievit, R. A., Davis, S. W., Griffiths, I., Correia, M. M., & Henson, R. N. (2016). A watershed model of individual differences in fluid intelligence. Neuropsycholofiia, 91, 186-198. Kievit, R. A., Lindenberger, U., Goodyer, I. M., lones, P. B., Fonagy, P., Bullmore, E. T., The Neuroscience in Psychiatry Network & Dolan, R. J. (2017a). Mutualistic coupling between vocabulary and reasoning supports cognitive development during late adolescence and early adulthood. Psycholofiical Science, 28(10), 1419-1431. Kievit, R.A., Brandmaier, A., Ziegler, G., van Hatmelen, A. L., de Mooij, S. M. M., Moutoussis, M., Goodyer I., Bullmore E., lones, P. B., Fonagy, P., Lindenberger, U., Dolan, R. I., & NSPN Consortium (2017b). Developmental cognitive neuroscience using Latent Change Score models: A tutorial and applications. Developmental Cognitive Neuroscience. Kovacs, K., & Conway, A. R. (2016). Process overlap theory: A unified account of the general factor of intelligence. Psychological Inquiry, 27(3), 151-177. Krijnen, W. P. (2004). Positive loadings and factor correlations from positive covariance matrices. Psychometrika, 69(4), 655-660. Lumsden, J. (1976). Test theory. Annual review o/psychology, 27(1), 251280. May, R. M. (1973). Stability and complexity in model ecosystems. Princeton: Princeton University Press. McArdle, J. J., (2001). A latent difference score approach to longitudinal dynamic structural analysis. In Structural Equation Modeling: Present and Future. A Festschrift in Honor of Karl Joreskog, ed. R Cudeck, S du Toit, D. Sorbom. pp. 341-80. Lincolnwood, IL: Sci. Softw. Int. McArdle, I. J., Ferrer-Caja, E., Hamagami, F., & Woodcock, R. W. (2002). Comparative longitudinal structural analyses of the growth and decline of multiple intellectual abilities over the life span. Developmental Psychology, 38(1), 1 1 5-142. McCall, R. B., & Carriger, M. (1993). A meta-analysis of infant habituation and recognition memory perfOlmance as predictors of later IQ. Child Development, 64(1), 57-79. Molenaar, P. C. M., Boomsma, D. I., & Dolan, C. V. (1993). A third source of developmental differences. Behavior Genetics, 23(6), 519524. Murray, J. D. (2002). Mathematical biology. I: An introduction (3th ed.). Berlin: Springer Verlag.

130

Chapter Five

Nisbett, R. E., Aronson, J., Blair, C., Dickens, W., Flynn, J., Halpern, D. F., & Turkheimer, E. (2012). Intelligence: New findings and theoretical developments. American psycholoKist, 67(2), 130. Peng, P, Wang, T, Wang, C, & Lin, X (201S). A Meta-analysis on the Relation between Fluid Intelligence and ReadinglMathematics: Effects of Tasks, Age, and Social Economics Status. Psychological Bulletin, in press. Pinker, S. (1994). How could a child use verb syntax to learn verb semantics? In L. Gleitman & B. Landua (Eds.), The Acquisition of the Lexicon (pp. 377-410). Cambridge: MIT Press. Plomin, R., & von Stumm, S. (201S). The new genetics of intelligence. Nature Reviews Genetics, 19(3), 14S. PIsek, P. E., & Greenhalgh, T. (2001). Complexity science: The challenge of complexity in health care. BMf: British Medical Journal, 323(73 13), 62S. Prakash, S., & de Roos, A. M. (2004). Habitat destruction in mutualistic metacommunities. Theoretical Population Biology, 65(2), IS3-163. Raykov, T., & Marcoulides, G. A. (2006). On multilevel model reliability estimation from the perspective of structural equation modeling. Structural Equation Modeling, 13(1), 130-141. Sachisthal S. M., Jansen B. R., Peetsma T. D., Dalege J., van der Maas, H. L. I, & Raijmakers, M. E. I (in press). Introducing a Science Interest Network Model to Reveal Country Differences. Journal of Educational Psychology. Savi, A. 0., Marsman, M., van der Maas, H. L. J., & Maris, G. K. I. (201S). The wiring of intelligence. PsyArxiv preprint. Retrieved from https:lldoi.org/l0.31234/0sf.io/32wrS. Sclimidhuber, J. (201S). Deep learning in neural networks: An overview. Neural networks, 61, SS-117. Schwaighofer, M., Fischer, F., & Biiliner, M. (201S). Does working memory training transfer? A meta-analysis including training conditions as moderators. Educational Psychologist, 50(2), 13S-166. Shahabi, S. R., Abad, F. I, & Colom, R. (201S). g, mutualism, and development: Cross-sectional evidence from Iranian school children. Personality and Individual Differences, 135, 222-22S. Siegler, R. S., & Alibali, M. W. (2004). Children's thinking (4th ed.). Upper Saddle River, NJ: Prentice Hall. Sternberg, R. J. (19SS). A three-facet model of creativity. In R. J. Sternberg (Ed.), The nature of creativity: Contemporary psychological perspectives (pp. 12S-147). New York, NY, US: Cambridge University Press.

The Network Approach to General Intelligence

131

Strogatz, S . H. (2018). Nonlinear dynamics and chaos: With applications to physics, biology, chemistry, and engineering. CRC Press. Thomson, G. (1951). The factorial analysis of human ability (5th ed.). London: University of London Press. Thorndike, R. L. (1927). The measurement of intelligence. New York: Teachers College. van Bork, R., Epskamp, S., Rhemtulla, M., Borsboom, D., & van der Maas, H. L. 1 (2017). What is the p-factor of psychopathology? Some risks of general factor modeling. Theory & Psychology, 27(6), 759773. van de Leemput, I. A., Wichers, M., Cramer, A. 0., Borsboom, D., Tuerlinckx, F., Kuppens, P., . . . & Derom, C. (2014). Critical slowing dO\vn as early warning for the onset and telTIlination of depression. Proceedings ofthe National Academy ofSciences, 1 1 1(1), 87-92. van der Maas, H. L. J, Kan, K. J., Marsman, M., & Stevenson, C. E. (2017). Network models for cognitive development and intelligence. Journal ofIntelligence, 5(2), 16. van der Maas, H. L. J., & Kan, K. J. (2016). Comment on "Residual group-level factor associations: Possibly negative implications for the mutualism theory of general intelligence" by Gilles E. Gignac (2016). Intelligence, 57, 81-83. van der Maas, H. L. J., Dolan, C. V., Grasman, R. P., Wicherts, J. M., Huizenga, H. M., & Raijmakers, M. E. (2006). A dynamical model of general intelligence: The positive manifold of intelligence by mutualism. Psychological Review, 1 13(4), 842. van der Maas, H. L. J., Kan, K. 1, & Borsboom, D. (2014). Intelligence is what the intelligence test measures. Seriously. Journal of Intelligence, 2(1), 12-15. van der Maas, H. L. J., Kan, K., Hofinan, A., & Raijmakers, M. (2014). Dynamics of development: A complex systems approach. In P. C. M. Molenaar, R. M. Lerner, & K. M. Newell (Eds.), Handbook of developmental systems theory and methodology (pp. 270-286). New York, NY: Guilford Press. van Geert, P. (1991). A dynamic systems model of cognitive and language growth. Psycholofiical Review, 98(1), 3-53. Weisbuch, G. (2018). Complex systems dynamics. CRC Press.

CHAPTER SIX PROCESS OVERLAP THEORY: How THE INTERPLAY BETWEEN SPECIFIC AND GENERAL MENTAL ABILITIES ACCOUNTS FOR THE POSITIVE MANIFOLD IN INTELLIGENCE KRISTOF KOVACS

Introduction: Experimental and differential psychology The scientific study of individual differences in cognition and mainstream experimental psychology have been separated since the birth of modern psychology. Francis Galton's famous work on mental ability (Galton, 1 869) was published a decade before Wilhelm Wundt established the first psychological laboratory, yet Galton had little influence on Wundt who was uninterested in individual differences. James McKeen Cattell, a PhD student of Wundt at the time, under the strong influence of Francis Galton and his theory of intelligence as sensory acuity, intended to change the topic of his dissertation to individual differences in mental ability. The idea was opposed by Wundt, hence only after submitting his thesis in 1866 could Cattell start working on his 'mental tests', which were published in 1890 (Fancher, 1985). Titchener, the main advocate of Wundtian psychology, opposed the application of the methods of experimental psychology to the study of individual differences even more strongly (Brody, 2000). The separation became permanent after Clark Wissler put Cattell's mental tests to an empirical test and found no correlation either between the tests themselves or with an external criterion of intelligence, university grades.

Process Overlap Theory

133

"This study was enormously influential. C . . . ) The results of the study were instrumental in ending the attempt to measme intelligence by the techniques initially advocated by Galton." (Brody, 2000, p. 17.).

Wilhelm Stem, the inventor of the IQ formula outlined a unified framework for experimental and differential psychology, but it did not turn out to be influential: in 1957 Lee Cronbach still lamented on the two distinct lines of research in scientific psychology, correlational and experimental (Cronbach, 1957). This distinction has been also referred to as general vs. differential psychology (Brody, 2000; Deary, 2001). In this chapter the current status ofthis separation will be examined from a methodological perspective, with a focus on identifying domain-specificity in correlational and experimental psychology. This will be followed by a recent theoretical endeavour, process overlap theory, that purports to explain individual differences in intelligence on the basis of mechanisms identified by experimental psychology and neuroscience.

Modularity and intelligence 1 : from mental architecture to individual differences? In the past decades the doctrine of domain-specificity has become dominant in cognitive psychology and the localization of specific functions has been a central theme in neuropsychology and neuroscience. The idea of domain-specificity was expanded in Jerry Fodor's seminal book on the architecture of the mind (Fodor, 1983), which championed in articulating the concept of modularity. The central tenet of Fodor's theory of modularity is that the human mind is comprised of a general processing mechanism as well as domain-specific modules: specific processing mechanisms that react to specific kinds of stimuli only. Fodor criticises 19th century "faculty psychology". He introduces a distinction between what he calls the horizontal and vertical systemization of the mind and argues in favor of the latter. By horizontal fractionation he means identifying domain-general systems that are separated according to the processes involved. For example, in this systemization memory and perception are separated, but the same memory processes are activated regardless of the nature of stimuli: we rely on the same cognitive system to remember the colour of the neighbour's dog, the date of Aristotle's birth or what our first day in school was like. According to Fodor, psychometrics, just like old faculty psychology, belongs to the horizontal tradition, and both are fundamentally wrong. Vertical systemization, on the other hand, is domain-specific: it focuses on the nature of the sensory

134

Chapter Six

information that serves as an input for the given system rather than on the processes involved. That is, a separate system is responsible for processing human faces, objects, or music, even if all these processes contribute to perception. Besides domain-specificity, the other most important feature of modules, according to Fodor, is encapsulatedness, which means that modules are processing information independently, never in conjunction with other modules. Once processing is carried out each module provides output infOlmation for the general-purpose mechanism that all the modules feed into. In this chapter emphasis will be put on these two aspects of the concept of modularity, mostly ignoring other characteristics such as rapid and compulsory processing, computational nature, a lack of top-do\Vll control, innateness, etc. Fodor claims that the roots of a vertical (i.e. modular) systemization of the mind can be found in Gall's phrenology. Gall's famous phrenological map, however, was actually constructed on the basis of individual differences data. Gall claimed that psychological differences in certain characteristics are correlated with morphological differences on the surface of the skull, which are in tum caused by differences in the parts of the brain that lie beneath those surfaces. There is another version of the modularity hypothesis which claims that there is no general-purpose mechanism at all, the mind is entirely made up of domain-specific processes. This view, called 'massive modularity' (Sperber, 1994; Tooby & Cosmides, 1992) is shared by many evolutionary psychologists, but is sharply criticized by Fodor himself (Fodor, 2000). \¥hen cognitive psychologists or cognitive scientists claim that something is modular they usually refer to double dissociation and rely on two sorts of evidence. The first comes from neuropsychological studies: an injury to one part of the brain results in the loss of an ability but leaves another intact, while an injury to a different part of the brain impairs the second ability but leaves the first intact. According to such evidence linguistic, spatial, and numerical cognition have been described as modular as all of them can be impaired without a decrease in perfOlmance in other areas. The second comes from experimental studies and is based on interrerence. If participants have to solve two tasks in parallel and perfOlmance on one does not deteriorate with the onset of the other then the two tasks are considered to tap independent processes; if the two tasks interfere with one another, they are considered to tap the same process(es). For instance, such

Process Overlap Theory

135

experiments were crucial for establishing the multi-component model of working memory (Baddeley & Hitch, 1974). The concept of domain-specificity is indeed well supported by evidence from double dissociation. In fact, such evidence is crucial for modular theories about mental architecture. At the same time, the very concept of mental architecture is a universal one and as such is a within­ individual account of cognition that cannot be automatically applied to the structure of between-individual variation. From a methodological perspective: double dissociation and experimental interference are appropriate pieces of evidence to fractionate at the within-individual level. Yet drawing direct inferences from such evidence to the structure of individual differences is, arguably, a fallacy. Take the following section, for instance, about a book on mental architecture: "One of the themes that emerged from part I of this book was the large number and the great diversity of different cognitive talents that are normally knit together to make up the intelligence of a real human. We saw this in the diversity of artificial networks that have been built to imitate one or other small aspects of hlUllan cognition, such as the ability to recognize faces, to read printed text, to see in three dimensions, to generate locomotion, to discriminate SOlUlds, to discriminate emotions, and to discriminate grammatical sentences. We saw it again in the great variety of severe but isolated cognitive deficits that typically result from localized damage to various parts ofthe living brain. This diversity illustrates that intelligence is not a one-dimensional commodity, something that varies only from greater to lesser. Rather, the intelligence of any human has many dimensions, and in a normal human population the scattered variation in cognitive ability within each of these dimensions will be considerable."

(Churchland, 1996, p.

253, bold added).

The argument essentially identifies separate - within-individual cognitive systems and concludes that if they are independent then their variance must also be independent. Not only does this conclusion not necessarily follow from its premises, it actually seems to be directly contradicted by the - arguably - most replicated result in all of psychology: the positive manifold. The positive manifold refers to the pattern of all-positive correlations that is observed when mental tests are administered to a large sample of people. Even when the tests include different domains such as in the case of a vocabulary test and a mental rotation test, the observed correlations are always positive. Overall, 40-50% of the between-individual variation

136

Chapter Six

in mental test scores is domain-general (Deary, Penke, & Johnson, 2010; Jensen, 1998). This empirical finding will not be refuted by the logical analysis of within-individual evidence on modularity. In other words: the second half of the last sentence of the above quote ("in a normal human population the scattered variation in cognitive ability within each of these dimensions will be considerable") does not follow from the first half ("the intelligence of any human has many dimensions"). A dissociation of two processes in this sense is unrelated to the correlation between them (Kovacs, Plaisted, & Mackintosh, 2006). For instance, imagine if one measured different indicators of strength in both alms in a large sample (the strength of grip, the maximum weight one can lift, etc.). Measures of the strength of people's left arm will most probably correlate with those of the right arm, regardless of 1) people being able to do things with their arms in parallel (lack of interference in an experimental condition), or 2) people can lose only one of their alms in an accident with the other ann remaining intact (selective impailTIlent due to injury). There are several possibilities for two different parts of the brain to be responsible for different domain-specific functions while perfOlmance on those tests can still correlate. One option is that there are certain parameters (e.g. neural conduction velocity, speed or accuracy of neural transmission, myelinisation, efficiency of glucose metabolism) that are common to these cognitive fImctions (Jensen, 1998). Another is that there are mutually beneficial interactions during development (van der Maas et aI., 2006). A third possibility will be the focus of Section 4 in this chapter. The quote above also highlights another important issue: the use of the very word intelligence in a within-individual and in an individual differences context. Not distinguishing these two conceptions necessarily leads to confusion as the two meanings are incommensurable. A modular mind can accommodate a general factor; or to be more precise, individual differences that are domain-general to a large extent are compatible with those differences appearing between people who have a modular mind. One carmot draw inferences directly from within-individual evidence on domain-specificity: such evidence is undetelTIlinistic to the structure of variation, which can logically be either domain-general or domain­ specific, but empirically appears to be largely domain-general. In his influential book Howard Gardner claimed that intelligence is not unitary, instead there are seven different and independent kinds of intelligences (Gardner, 1983). Gardner has since extended his list of intelligences (Gardner, 1999), but from the perspective of the current

Process Overlap Theory

137

chapter the exact list or number of multiple intelligences is less interesting than the methods for separating them. Gardner has eight criteria which a specific ability has to satisfy in order to be considered an "intelligence". 1 . "Potential isolation by brain damage. 2. The existence of idiots savants, prodigies, and other exceptional individuals. 3. An identifiable core operation or set of operations. 4. A distinctive developmental history, along with a definable set of expert "end-state" performances. 5. An evolutionary history and evolutionary plausibility. 6. Support from experimental psychological investigations. 7. Support from psychometric findings. 8. Susceptibility to encoding in a symbol system." (Gardner, 1983, p. 62-69.) Apparently, Gardner's methods for separating "intelligences" show substantial overlap with the cognitive scientist's toolbox for studying mental architecture. The first criterion, potential isolation by brain damage, is basically equivalent to double dissociation based on selective impailTIlent. The sixth criterion, support from experimental psychological investigations relates to interference between various tasks. As argued above: such pieces of evidence are sufficient to theorize about mental architecture, but are not directly relevant for differential psychologists who purport to identify the structure of mental abilities based on correlations between various mental tests. The existence of specific disorders falls under the sarne category: in the arm strength analogy this would be equivalent to a developmental disorder, instead of an accident, affecting only one of the arms. Different developmental or evolutionary histories are also independent of the possible covariation between abilities. Human new-borns have a much higher head to body ratio than adults and in the course of development the head-body ratio substantially decreases. That is, the size of different parts of the body change independently: they have distinctive developmental history. But there is no a priori reason to suspect that this change affects the correlation between head and body size at any stage of development. Similarly, in the course of hominid evolution, the ann length to leg length ratio has decreased; the size of arms and legs changed independently, hence they have different evolutionary histories. Yet again, there is no a priori reason to expect that the correlation between the length of legs and arms have changed at any point.

138

Chapter Six

'What needs to be emphasized from the perspective of the current chapter is that Gardner's book is in fact more similar in its ideas and methods to Fodor's book on modularity then to standard works on human intelligence that focus on individual differences. In fact, due to his criteria, Gardner's book is more about mental architecture then about variation in intelligence; apart from his psychometric criterion, of course, but from that perspective three of his intelligences, linguistic, logical-mathematical, and spatial in fact do correlate.

Modularity and intelligence 2: from individual differences to mental architecture? Psychologists studying individual differences in intelligence have applied the method of factor analysis to explore the structure of abilities responsible for perfOlmance on various mental tests. With the help of factor analysis, the large correlation matrices that consist of the inter­ correlations of diverse cognitive tests can be simplified, assuming that the correlation between any two tests is the result of the tests' correlation with a "latent variable" which is not directly measurable. As discussed in the previous section, tests measuring cognitive abilities always show positive correlations and this phenomenon is referred to as the positive manifold; therefore factor analysis yields a strong general factor, g, that accounts for 40-50% of the variance. Yet it is also true that, among this pattern of all-positive correlations there are clusters of correlations that are stronger than others, and these clusters of strong correlations are thought to reflect what are knO\vn as group factors, representing cognitive abilities. For example, a vocabulary test, a reading comprehension test, and a listening comprehension test might reveal relatively strong positive correlations within the positive manifold. This cluster, then, is thought to reflect a group factor that we might refer to as verbal ability. That is, in the literature of human intelligence, which traditionally focuses on individual differences, specific abilities are identified not through double dissociation, but by the covariance structure of tests with different content. The analysis of large data sets demonstrated that a single general factor is insufficient to explain all of the variance since, for instance, verbal, spatial, and numerical tests correlate more strongly with one another than with tests tapping into other domains. Nor can the correlations between tests be accounted for by only specific factors because nearly half of the total variance in human abilities is cross-domain. The widely accepted three-stratum model acknowledges both g and specific factors at different

Process Overlap Theory

139

levels of a hierarchy, where the correlation of group factors results in g (Carroll, 1993). With respect to the "content" of factors, a widely accepted account is the model of fluid and crystallized intelligence (Cattell, 1971; Hom, 1994). In this model there are domain-specific as well as domain-general group factors, the most important of which are Gf (fluid intelligence) and Gc (crystallized intelligence). Gfis the ability to solve novel problems for which one cannot rely on already acquired skills or knowledge and is usually measured with tests of non-verbal, abstract reasoning. Tests that give loading on Gc measure previously acquired knowledge and typically consist of verbal material, such as vocabulary or reading comprehension. Other important factors of the model are Gv (visual-spatial), Gs (speed), Gsm (working memory). All factors are not created equal: Gfhas a central role among cognitive abilities in the fluid-crystallized model and the correlation between Gf and g is perfect or near-perfect at the latent level (e.g. Gustafsson, 1984). Johnson and Bouchard argued that the major flaw of the fluid­ crystallized model is that it does not posit a general factor on the grounds that general factors extracted from different batteries are not the same (Johnson & Bouchard, 2005), yet large-scale analysis shows that general factors are in fact identical across batteries (Johnson, Bouchard, Krueger, McGue, & Gottesman, 2004). A more recent development, the Cattell­ Horn-Carroll (CHC) model of mental abilities unifies the Cattell-Horn description of specific abilities, in particular Gf and Gc with Carroll's three-stratum model. CHC thus has the Cattell-Horn factors on the 200 stratum and the correlation between 2nd stratum factors is explained with a 3,d stratum general factor, g (McGrew, 2009). Besides CHC, however, there are other models of intelligence. For instance, Vernon proposed a model with g and broad second order factors: v:ed for verbal-educational abilities and k:m for kinaesthetic and mechanic abilities (Vernon, 1961). His group factors are, arguably, more domain­ specific than the ones in the CHC model: whereas Gf and Gc are basically described by whether one has to deal with novel or already acquired information, v:ed and k:m are described by the domain they cover. Importantly, all factorial models stem from the phenomenon that all correlations between mental tests are positive and that there are groups of tests, typically with similar content, for which the correlations are higher than the average correlation between all tests. 'While factor analysis is a useful statistic tool to identify patterns in complex correlational data, the interpretation of factors is problematic. Stevan Hamad for example compared the interpretation of factors to helTIleneutics:

140

Chapter Six "There is a huge hermeneutic component to psychometric analysis. The empirical part is the calculation of the correlations in the extraction of factors; the hermeneutic part is in interpreting the factors, figuring out what on earth they may mean." (Bock, 2000, p. 48.)

Not only the interpretation, but also the very status of factors as constructs is a matter of debate. The most extreme view is probably that factors are mere mathematical artefacts with no reality (e.g. Gould, 1996). This argument capitalizes on the claim that there are many different factorial solutions to a given correlation matrix that are mathematically equally tenable and therefore it is not possible to choose between solutions in an objective fashion. Hence any attempt to give psychological meaning to factors qualifies as "reification": factors do not have validity other than statistical and any particular factorial account is just one of the infinite factorial solutions of a given correlation matrix. The ambiguity of factorial solutions is actually well known among psychometricians: "It is ( ) clear that the rotated factors may take up any position in factor space and that accordingly, as has been argued, there is a virtual infinity of solutions. Since, as has been seen, these are mathematically equivalent there is no mathematical reason for choosing one rather than another." (Kline, 1991, p. 61.) ...

Gould's argument, however, is a fallacy, since the fact that there are an infinite number of factorial solutions does not imply that any factorial solution will do and that it is not possible to reject any of them. For instance, the set of natural numbers consists of an infmite amount of numbers. However, neither ' - 1 ' nor '0.5' are parts of the set of natural numbers. Similarly, even if there is an infinite number of mathematically equivalent factorial solutions it is still possible for a given factorial solution to not fit the data. Indeed, several models ui the history of intelligence research have been discarded, including Speannan's original model ofg, Thurstone's model of Primary Mental Abilities, or Guilford's Structure of Intellect model (Guilford, 1956; Spearman, 1904; Thurstone, 1938). An investigation of possible ontological stances one can take regarding latent variables concluded that one must take a realist view in order for assumptions of latent variable modeling not to be violated (Borsboom, Mellenbergh, & van Heerden, 2003). So if they are not mere statistical artefacts in what sense can factors be interpreted as real? Are they equivalent to processes, mechanisms, etc.? Do they have a meaningful within-individual interpretation? In particular: can the general factor of

Process Overlap Theory

141

intelligence (psychometric g) be identified as a within-individual domain­ general construct (psychological g)? If it can, then the following statement is valid: "Anna used his general intelligence to correctly answer items on both the inductive reasoning test and the mental arithmetic test." This, however, is substantially different from saying that "If Anna performs better on the inductive reasoning test than most people it is very likely that she will perform better on the mental arithmetic test as well". The latter statement leaves the possibility open that Anna in fact did not use the same general ability on the two tests and there is some other reason for the results to correlate. The positive manifold only translates to the second statement, not the first; in order to validate the first statement one has to review other kinds of evidence about fractionation at the universal (or individual) level. That is, the first statement is about mental architecture, not individual differences. As it was discussed in the previous section: the actual evidence from cognitive psychology and neuropsychology questions the validity of the first statement. 'Whether g can be interpreted as a unitary construct has been controversial from an individual differences perspective, too: Kranzler and Jensen have had a prolific debate with Carroll on the subject (Carroll, 1991a, 1991b, 1993, Kranzler & Jensen, 1991a, 1991b, 1993). Kranzler and Jensen factor analysed various elementary cognitive tasks (such as various reaction time and inspection time measures) and found different "elementary cognitive factors", many of which correlated with the g factor extracted from psychometric tests but not with each other. From these results they concluded that g is the result of several independent processes. Carroll disagreed and claimed that the procedure used by Kranzler and Jensen could not extract pure factor scores. He therefore argued that the question could not be decided by the methods employed by Kranzler and Jensen. From the perspective of the present chapter it is worth looking at Jensen's evaluation of the debate: "to show that the general factor involves individual differences in two independent processes, A and B, and is therefore not flUlClamentally lUlitary would require that individual differences in A and B be measilled separately and that A and B are each independently correlated -with the general factor of the psychometric tests. The more difficult condition to satisfy C . . . ) is that it must be assumed that the empirical g factor scores derived from the tests are "pille" g uncontaminated by any non-g "impillities". C . . . ) [But] because it is virtually impossible to prove definitively that the g factor scores are "pure" in this sense, the issue retreats from the scientific arena, and it then becomes a purely

142

Chapter Six metaphysical question whether g is or is not unitary."

(Jensen, 1998, p.

261., bold added).

The last sentence is surprising: while it might indeed be impossible to fractionate g on purely psychometric grounds, it is arguably cognitive psychology, cognitive science, and neuroscience, rather than metaphysics that can shed light on whether g, a general account of between-subject variation, is the result of a single, unitary process or a number of independent processes. If, for instance, different tests load on g but performance on the tests can be dissociated as a result of selective impairment then it is unlikely that g is the result of a single, unitary process, psychometric evidence notwithstanding. Let us return to the analogy of the strength of the alms once more: if we fmd that our measures correlate very strongly, or even perfectly between the two alms it would still be incorrect to claim that "annness" is a unitary construct (i.e., humans only have one arm). Just like the cognitive scientist's toolbox underdetermines whether variation in cognitive abilities is domain-general or domain-specific, the psychometrician's toolbox underdetermines whether domain-general variation in cognitive abilities is the results of a single domain-general process. The architecture of cognition does not detelTIline the structure of correlations between perfolTIlance on various tasks and the latent variable structure of between-subject differences does not detelTIline the architecture of cognition. Therefore, a unitary domain-general cognitive mechanism is a sufficient but not necessary explanation of the positive manifold and is therefore not a necessary interpretation of g. In the next section a theory will be presented that actually does explain the positive manifold without postulating a unitary source of variance. Finally, it is worth mentioning that the use of the expression modularity in both a within-individual and an individual differences context can be confusing, just like in the case of intelligence. Models of individual differences that emphasize group factors are sometimes referred to as modular and modularity is sometimes contrasted to g (e.g. DettelTIlan, 1992). But a module in the cognitive scientist's sense (i.e., an encapsulated domain-specific processor) is not the same as a group factor, just like than the concept of general intelligence is not the same as g. Models focusing on domain-specific variance are not modular in the sense that modularity is traditionally used in cognitive science to describe mental architecture. It might be the case that there is an agreement between such within-individual and between-individual constructs. Such cases are referred to as ergodicity, but they are exceptions rather that the

Process Overlap Theory

143

rule (Molenaar & Campbell, 2009). In the case of intelligence, it does not generally seem to be the casel .

Process overlap theory The most important feature of modularity is the "encapsulated" nature of modules: they are completely independent from one another with respect to infOlmation processing. The general assumption behind the concept of modularity is, therefore, that double dissociation is sufficient evidence to conclude that the tasks dissociated measure completely independent cognitive systems or modules. Yet this is not the only reading of double dissociation; in fact double dissociation is arguably insufficient to claim complete functional independence. In the literature of memory research there has been a controversy between the systems vs. processes approach. From a methodological perspective the debate is about whether dissociation should be taken as evidence for postulating separate memory systems or whether such evidence is perfectly compatible with the existence of functionally overlapping processes: "Although on first appearances it may seem as if the difference between multiple processes and multiple systems is only terminological, it is in fact a fundamental difference. ( ) multiple processes are construed as multiple steps in a stream of processing steps, not as comprising independent systems. Multiple systems operate independently of each other (they are similar to Fodorian modules) whereas multiple processes interact and combine to perform cognitive operations. ( ) an alternative framework, the components of proces sing framework, developed by Morris Moscovitch, provides a more adequate framework that can resolve the conflict between the approaches. ( ) different tasks may draw differentially upon different components in a processing system. If two tasks can be dissociated ( ) ...

...

...

...

then there must be at least one component process that figures differently in the two tasks ( ). Within this framework, dissociations are no longer used to tease apart whole systems, but only differences in reliance on components within a larger system. It is here that the distinction between the systems and process approach becomes sharp." (Bechtel, 2001, p. 491 -492, bold added). ...

1 With the probable exception of the cognitive concept 'fluid reasoning' and the psychometric group factor 'fluid intelligence' (Kievit, 2014).

144

Chapter Six

A recent, process-oriented explanation of the positive manifold, called process overlap theory (pOT, Kovacs & Conway, 2016) draws on the "components of processing" framework. It interprets evidence for dissociation (including neuropsychological, experimental, and developmental bases of dissociation) as ones fractionating processes rather than encapsulated and independent systems. That is, dissociated tests tap processes of which at least one is different, but not necessarily sets of completely different processes. Therefore the processes that are required for perfOlmance on different cognitive tests can overlap and can also be dissociated by brain injury etc. at the same time. Evidence for dissociation between domain-specific cognitive tests makes it difficult to interpret g as 'general intelligence', a unitary system that pelTIleates all human cognition. But such evidence is compatible with an account of the general factor that explains the correlations between these domain-specific tests as the result of overlapping component processes. This is exactly what POT attempts. POT is also strongly motivated by the sampling model of Godfrey Thomson. Thomson, a contemporary of Speannan, demonstrated mathematically that the positive manifold could emerge not only without a single underlying general intelligence but even without a single process being common to all of the tests. He proposed that different mental tests tap a large number of independent processes, some of which are common to more than one test. According to Thomson the correlation between different tests is caused by the overlap of the independent processes necessary to solve the tests; the larger the overlap, the larger the correlation. Using random data (he tlnew dice) Thomson was able to show that the positive manifold can be explained both by postulating a single general ability or a large number of independent processes (Thomson, 1916). A more recent analysis confirmed that from a statistical perspective one cannot decide between the sampling model and the g model, both are sufficient to account for the positive manifold (Bartholomew, Deary, & Lawn, 2009). POT also builds upon research on the relationship between working memory and fluid intelligence. Working memory refers to: "the ensemble of components of the mind that hold a limited arnmmt of information temporarily in a heightened state of availability for use in ongoing information processing" (Cowan, 2016).

Measures of working memory capacity, such as complex span tests, require this type of parallel storage and processing. For example, in the symmetry span test participants have to remember spatial locations, the

Process Overlap Theory

145

presentation of which is interrupted by images where symmetry judgments have to be made. Complex span tests are therefore different than so-called simple span tests, such as digit span, in which participants simply have to recall a list of items. In contrast to simple span tests, variance in complex span tests is primarily domain-general (Kane et aI., 2004). Therefore, similar to intelligence tests, a general factor of working memory capacity can be extracted, and this factor correlates strongly with fluid intelligence: two meta-analyses of latent variable studies investigating the relationship between working memory and fluid intelligence estimate that the correlation is somewhere between r = .72 to r = .8 1 (Kane, Hambrick, & Conway, 2005; Oberauer, Schulze, Wilhelm, & Suss, 2005). FurthelTIlore, the processes that working memory tasks measure beyond storage most likely reflect individual differences in the executive attention component of working memory (Engle & Kane, 2004; Engle, Tuholski, Laughlin, & Conway, 1999; Kane, Bleckley, Conway, & Engle, 2001; Kane & Engle, 2002). According to the executive attention theory of individual differences in working memory capacity (Engle & Kane, 2004; Kane et aI., 2001), working memory and fluid intelligence correlate strongly because both constructs rely to a great extent on executive functions, such as updating, inhibition, and task-switching. Indeed, several recent latent variable studies have demonstrated strong correlations between executive attention and fluid intelligence (Engelhardt et aI., 2016; Shipstead, Lindsey, Marshall, & Engle, 2014; Unsworth, Fukuda, Awh, & Vogel, 2014) The main premise of POT is that a battery of intelligence tests requires a number of domain-general processes, such as those involved in working memory and attention, as well as a number of domain-specific processes. Importantly, domain-general processes are required by the majority (but not all) of test items, whereas domain-specific processes are required less frequently, depending on the nature of the test (e.g., verbal vs. spatial). Therefore, domain-general processes associated with working memory and executive attention will constrain perfolTIlance on most items on most intelligence tests, whereas individual differences in specific processes will impact a narrower range of tests. Such a pattern of overlapping processes explains the positive manifold and thus the general factor as well as the domain-specific clusters of intercorrelated tests that result in group factors. Indeed, the first simulations that tested the theory confirmed POT (Kan, van der Maas, & Kievit, 2016; Kovacs, Conway, Snijder, & Hao, 2018): the positive manifold did emerge from the interplay of domain-specific and domain-general processes postulated by POT (see also McFarland,

146

Chapter Six

2017, for a different but raleted simulation). POT is similar to Thomson's sampling model (Thomson, 1916), but is also different in crucial ways (Kovacs & Conway, 2016). The most important and novel aspect of POT and its main divergence from Thomson's ideas is that it proposes that the processes involved in test perfOlmance are non-additive. Since executive attention processes are involved in the majority of test items, individual differences in executive attention pose general limits on total perfOlmance, acting as a bottleneck, and masking individual differences in more domain-specific processes. Therefore the correlation between two tests is not linearly related to the ratio of overlapping processes. Besides providing an account of the positive manifold, POT also explains a number of important phenomena observed in the study of human intelligence. The first such phenomenon is ability differentiation, which refers to the finding that cross-domain correlations are higher in samples with lower average ability and so g explains more variance in such samples. The second is the worst perfOlmance rule, the finding that worst perfOlmance (e.g., slowest reaction times) is a better predictor of g than average or best perfOlmance. The third is that the more complex a task the higher its correlation with g. Finally, through proposing that the positive manifold is caused by the overlapping activation of the executive attention processes that are involved in both working memory and fluid reasoning, the theory accounts for the central role of fluid reasoning in the structure of human abilities and for the finding that the fluid reasoning factor (Gf) seems to be statistically identical or near-identical to g (Gustafsson, 1984). POT is therefore able to explain why g is both population and task­ dependent (i.e., it explains the most variance in 1) populations with lower ability, 2) worst performance, and 3) cognitively demanding tasks). POT focuses on the limitations of executive attention processes in explaining g and proposes an interaction between the executive demands of the task and the executive functioning of the individual. This is expressed in a fonnal mathematical model (i.e., a multidimensional item response model) that specifies the probability of arriving at a correct answer on a given mental test item as the function of the level of domain-specific as well as domain­ general cognitive processes (Kovacs & Conway, 2016).

Process Overlap Theory

147

Conclusions In this chapter it has been argued that domain-specificity bears different mearungs ill differential psychology and ill cogintiveiexperimental psychology. In the former it relates to finding that individual differences in tests with characteristic content (e.g. spatial or verbal) typically correlate more strongly with one another than with tests that have different content. In the latter it means that the mind can be fractionated into processors of specific content through double dissociation. These two ways of identifying specificity in cognition do not necessarily translate to one another. That is, the general factor of intelligence does not necessarily translate to a domain-general problem solving mechanism and specific cognitive abilities in the differential sense are not the sarne as modules. It has been argued that a large part of incommensurability stems from the interpretation of double dissociation as evidence for completely independent cognitive systems responsible for the processing of domain-specific information. Moscovitch's component process model of memory provides a different interpretation, one that is more compatible with domain-general variance in human mental abilities than a strongly modular approach to mental architecture: Given the emphasis on dissociations, it is easy to lose sight of the fact that these components, though isolable in principle, are typically highly interrelated. The components' function is determined not only by their internal organization but also by the network of connections to other components. (Moscovitch, 1992, p. 265.)

The most important consequence of POT is that g is "not a thing" but instead is the consequence of a set of overlapping cognitive processes sampled by a battery of tests. Therefore the general factor is a formative latent variable (Bagozzi, 2007) and as such it can be thought of as an index of mental functioning. Scores on the general factor represent a summary statistic that can be used to predict various phenomena, ranging from everyday cognitive perfOlmance (e.g., academic achievement and job perfOlmanc) to non-cognitive life outcomes (e.g., socioeconomic status or longevity). Thus POT does not deny the existence of g, but contrary to the standard view, interprets it as an emergent rather than a latent property. Should the theory endure further tests it might eventually fulfil its main purpose: to explain variation in cognitive abilities by accounting for actual test performance with the interplay of general and specific cognitive processes that are identified by cognitive psychology and neuroscience.

148

Chapter Six

References Baddeley, A. D., & Hitch, G. (1974). Working Memory. Psychology of Learning and Motivation, 8, 47-89. http://doi.org/10.1016/S00797421(08)60452-1 Bagozzi, R. P. (2007). On the meaning of formative measurement and how it differs from reflective measurement: comment on Howell, Breivik, and Wilcox (2007). Psychological Methods, 12(2), 229-237; discussion 238-245. http://doi.orgIl 0 .1 037/1082-989X.12.2.229 Bartholomew, D. J., Deary, I. J., & Lawn, M. (2009). A new lease of life for Thomson's bonds model of intelligence. Psychological Review, 1 1 6(3), 567-579. http://doi.org/10.1037/a0016262 Bechtel, W. (2001). The compatibility of complex systems and reduction: A case analysis of memory research. Minds and Machines, 1 1 , 483502. Bock, G. (2000). The nature of intelligence. Chichester: Wiley. Borsboom, D., Mellenbergh, G. 1, & van Heerden, J. (2003). The theoretical status of latent variables. Psychological Review, 1 1 0(2), 203-219. Brody, N. (2000). History of theories and measurements of intelligence. In R. J. Sternberg (Ed.), Handbook of intelligence (pp. 16-33). Cambridge: Cambridge University Press. Carroll, J. B. (1991a). No demonstration that g is not unitary, but there's more to the story: Comment on Kranzler and Jensen. Intelligence, 15(4), 423-436. http://doi.org/10.1016/0160-2896(91)90004-W Carroll, J. B. (1991b). Still no demonstration that g is not unitary: Further comment on Kranzler and Jensen. Intelligence, 15(4), 449-453. http://doi.orgIl0.1016/0160-2896(91)90006-Y Carroll, J. B. (1993). Human Cognitive Abilities: A Survey of Factor­ Analytic Studies. Cambridge University Press. Carroll, J. B. (1993). The unitary g problem once more: On Kranzler and Jensen. Intelligence, 1 7(1), 15-16. http://doi.orgIl0.1016/0160-2896(93)90034-3 Cattell, R. B. (1971). Abilities: Their Structure, Growth, and Action. Boston: Houghton Mifflin. Churchland, P. M. (1996). The engine of reason, the seat of the soul: a philosophicaljourney into the brain. Cambridge, Mass: MIT Press. Cowan, N. (2016). The many faces of working memory and short-term storage. Psychonomic Bulletin & Review. http://doi.orgIl0.3758/s13423-016-1191-6 Cronbach, L. J. (1957). The two disciplines of scientific psychology.

Process Overlap Theory

149

American Psychologist, 12(11), 671-684. http://doi.orgIl 0 . l0371h0043943 Deary, I. (2001). Individual differences in cognition: British contributions over a century. British Journal ofPsychology, 92, 217-237. Deary, I. J., Penke, L., & Johnson, W. (2010). The neuroscience of human intelligence differences. Nature Reviews. Neuroscience, 1 1 (3), 201-1 1 . http://doi.orgIlO .l 03 8/nrn2793 Detterman, D. K. (Ed.). (1992). Is mind modular or unitary? Norwood, N.J.: Ablex. Engelhardt, L. E., Mann, F. D., Briley, D. A., Church, I. A., Harden, K. P., & Tucker-Drob, E. M. (2016). Strong genetic overlap between executive functions and intelligence. Journal of Experimental Psychology. General, 145(9), 1 141-59. http://doi.orgIlO .l 03 7/xgeOOOO 195 Engle, R. W., & Kane, M. J. (2004). Executive attention, working memory capacity, and a two-factor theory of cognitive control. The Psychology ofLearning and Motivation, 44, 145-199. Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. A. (1999). Working memory, short-term memory, and general fluid intelligence: a latent-variable approach. Journal of Experimental Psychology. General, 128(3), 309-3 1 . Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/l05 13 3 98 Fancher, R. E. (1985). The intelligence men: Makers of the IQ controversy. New York: W. W. Norton. Fodor, J. A. (1983). The modularity ofmind. Cambridge, MA: MIT Press. Fodor, J. A. (2000). The Mind Doesn 't Work That Way. Boston: MIT Press. Galton, F. (1 869). Hereditary genius: an inquiry into its laws and consequences. London: Macmillan. Gardner, H. (1983). Frames of Mind: The Theory ofMultiple Intelligences. Basic Books. Retrieved from http://books.google.comlbooks?id�wxj 6npSaykgC&pgis� 1 Gardner, H. (1999). Intelligence Reframed: Multiple Intelligences for the 21st Century. Basic Books. Guilford, J. P. (1956). The structure of uitellect. Psychological Bulletin, 53(4), 267-293. http://doi.orgIl 0 . l0371h0040755 Gustafsson, J.-E. (1984). A unifying model for the structure of intellectual abilities. Intelligence, 8(3), 179-203. http://doi.orgIl 0 .l 0 1 6/0160-2896(84)90008-4 Horn, J. L. (1994). Theory of fluid and crystallized intelligence. In R. Sternberg (Ed.), Encyclopedia of Human Intelligence (pp. 443-451).

150

Chapter Six

New York: MacMillan Reference Library. Jensen, A. R. (1998). The g Factor : The Science 0/ Mental Ability. East (Vol. 17). Retrieved from http://www.jstor.org/stable/10.2307/4236447 Johnson, W., & Bouchard, T. J. (2005). The structure of human intelligence: It is verbal, perceptual, and image rotation (VPR), not fluid and crystallized. Intelligence, 33(4), 393-416. http://doi.org/10.1016/j .intell.2004.12.002 Johnson, W., Bouchard, T. J., Krueger, R. F., McGue, M., & Gottesman, I. I. (2004). Just one g: consistent results from three test batteries. Intelligence, 32(1), 95-107. http://doi.org/10.1 0 16/S0 160-2896(03)00062-X Kan, K.-J., van der Maas, H. L. J., & Kievit, R. A. (2016). Process Overlap Theory: Strengths, Limitations, and Challenges. Psychological Inquiry, 27(3), 220-228. http://doi.org/10.1080/1047840X.2016.11 82000 Kane, M. J., Bleckley, M. K., Conway, A. R. A., & Engle, R. W. (2001). A controlled-attention view of working-memory capacity. Joumal 0/ Experimental Psychology: General, 130(2), 169-183. http://doi.org/10.1037/0096-3445.1 30.2.169 Kane, M. J., & Engle, R. W. (2002). The role of prefrontal cortex in working-memory capacity, executive attention, and general fluid intelligence: An individual-differences perspective. Psychonomic Bulletin & Review, 9(4), 637-671 . http://doi.org/l0.3758IBF03 196323 Kane, M. J., Hambrick, D. Z., & Conway, A. R. A. (2005). Working memory capacity and fluid intelligence are strongly related constructs: comment on Ackerman, Beier, and Boyle (2005). Psychological Bulletin, 131, 66-71; author reply 72-75. http://doi.org/10.1037/00332909.1 3 1 . 1 .66 Kane, M. J., Hambrick, D. Z., Tuholski, S. W., Wilhelm, 0., Payne, T. W., & Engle, R. W. (2004). The generality of working memory capacity: a latent-variable approach to verbal and visuospatial memory span and reasoning. Journal o/Experimental Psychology. General, 133(2), 189217. http://doi.org/10.1037/0096-3445.133.2.189 Kievit, R. A. (2014). Turtles all the way down ? Psychometric approaches to the reduction problem. University of Amsterdam. Kline, P. (1991). Intelligence: the psychometric view. London: Routledge. Kovacs, K., & Conway, A. R. A. (2016a). Has g Gone to POT? Psychological Inquiry, 27(3), 241-253. http://doi.org/10.1080/1047840X.2016. 1202744 Kovacs, K., & Conway, A. R. A. (2016). Process Overlap Theory: A Unified Account of the General Factor of Intelligence. Psychological

Process Overlap Theory

151

Inquiry, 27(3), 15 1-177. http://doi.orgI10.108011047840X.2016.1153946 Kovacs, K., Conway, A . R . A., Snijder, J., & Hao, H. (2018). General lntelligence Explained (Away). Poster presented at: The 59th Annual Meeting of the Psychonomic Society; New Orleans, 201 8 . 1 1.15-18. Kovacs, K., Plaisted, K. C., & Mackintosh, N. J. (2006). Difficulties differentiating dissociations. Behavioral and Brain Sciences, 29(02), 138-139. http://doi.orgI10.1017/S0140525X06349035 Kranzler, J. H., & Jensen, A. R. (1991a). The nature of psychometric g: Unitary process or a number of independent processes? Intelligence, 15(4), 397-422. http://doi.orgI10.1016/0160-2896(91)90003-V Kranzler, J. H., & Jensen, A. R. (1991b). Unitary g: Unquestioned postulate or Empirical fact? Intelligence, 15(4), 437-448. http://doi.orgI10.1016/0160-2896(91)90005-X Kranzler, J. H., & Jensen, A. R. (1993). Psychometric g is still not unitary after eliminating supposed "impurities": Further comment on Carroll. Intelligence, 1 7(1), 1 1-14. http://doi.org/10.1016/0160-2896(93)90033-2 McFarland, D. J. (2017). Evaluation of multidimensional models of WAIS-IV subtest performance. Clinical Neuropsychologist, 31(6-7), 1 1 27-1140. http://doi.org/10.1080113854046.2017.1320426 McGrew, K. S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37(1), 1-10. http://doi.orgI10.1016/j .inte11.2008.08.004 Molenaar, P. C. M., & Campbell, C. G. (2009). The New Person-Specific Paradigm in Psychology. Current Directions in Psychological Science, 18(2), 1 12-1 17. http://doi.org/10. 1 1 1 1/j. 1467-8721.2009.01619.x Moscovitch, M. (1992). Memory and Working-with-Memory: A Component Process Model Based on Modules and Central Systems. Journal o/Cognitive Neuroscience, 4(3), 257-267. http://doi.orgI10.1 162/jocn.1992.4.3.257 Oberauer, K., Schulze, R., Wilhelm, 0., & Suss, H.-M. (2005). Working memory and intelligence--their correlation and their relation: comment on Ackerman, Beier, and Boyle (2005). Psychological Bulletin, 131, 61-65; author reply 72-75. http://doi.orgI10.1037/0033-2909. 1 3 1 . 1.61 Shipstead, Z., Lindsey, D. R. B., Marshall, R. L., & Engle, R. W. (2014). The mechanisms of working memory capacity: Primary memory, secondary memory, and attention control. Journal of Memory and Language, 72, 1 16-141. Spearman, C. (1904). "General lntelligence," Objectively Determined and Measured. The American Journal o/Psychology, 15(2), 201-292.

152

Chapter Six

http://doi.orgI10.2307114l2l07 Sperber, D. (1994). The modularity of thought and the epidemiology of representations. In L. A. Gelman & Hirschfeld S. A. (Eds.), Mapping the mind: Domain specificity in cognition and culture (pp. 39-67). New York, NY: Cambridge University Press. Thomson, G. H. (1916). A hierarchy without a general factor. British Journal o/Psychology, 1904-1920, 8(3), 271-281. http://doi.orgI10. l l l l/j .2044-8295. l 9 l 6.tb00133.x Thurstone, L. L. (1938). Primary Mental Abilities. Chicago: University of Chicago Press. Tooby, J., & Cosmides, L. (1992). The Psychological Foundations of Culture. In J. Barkow, L. Cosmides, & J. Tooby (Eds.), The Adapted Mind(pp. 19-136). New York, NY: Oxford University Press. Unsworth, N., Fukuda, K., Awh, E., & Vogel, E. K. (2014). Working memory and fluid intelligence: Capacity, attention control, and secondary memory retrieval. Cognitive Psychology, 71, 1-26. van der Maas, H. L. J., Dolan, C. V, Grasman, R. P. P. P., Wicherts, I. M., Huizenga, H. M., & Raijmakers, M. E. J. (2006). A dynamical model of general intelligence: the positive manifold of intelligence by mutualism. Psychological Review, 1 1 3(4), 842-61 . http://doi.orgI10.l037/0033-295X.113.4.842 Vernon, P. E. (1961). The structure o/human abilities (2nd ed.). London: Wiley.

CHAPTER SEVEN PASS THEORY OF INTELLIGENCE: A FROZEN DINNER OR A MOVING FEAST? GEORGE K. GEORGIOU & J. P. DAS

What's intelligence all about? Does it depend on school education? Is it an lUlchangeable biological index or can it be improved through instruction? Are all acts or behaviors considered to be 'intelligent' strictly conscious, deliberate, and guided by reasoning? These are some of the basic questions people ask about intelligence. The increased public interest in intelligence has naturally generated also interest in the theories associated with intelligence tests. In this chapter, we present such a theory, namely Planning, Attention, Simultaneous and Successive processing (PASS) theory of intelligence, and discuss how it can be operationalized (see Cognitive Assessment System; CAS) and how it has been used to understand the cognitive foundations of learning. According to Naglieri and Otero (2018), the PASS theory along with CAS has created an opportunity to move the field of intelligence and ability testing by emphasizing (a) that a test of intelligence should be based on a theory of intelligence and (b) that the test should measure basic neurocognitive processes defined by the intellectual demands of the test, not the content of the questions. In what follows, we first present a historical account of PASS theory and then we describe the PASS model. Next, we present CAS and we review the literature on PASS processes and academic achievement. Finally, we present our concluding remarks and ideas for future research on PASS theory.

Evolution and Origins of PASS Theory of Intelligence At least two major roots of the theory can be traced - one in biology and one in cognitive psychology. If we were to draw a family tree for PASS, one of its major branches would be based on the work of Sechenov (1 878), Pavlov (1927), and the triad -Vygotsky, Luria, and Leontieve.

154

Chapter Seven

Further branching draws on cognitive psychology and the work of Estes (Intelligence & Learning), Miller, Galanter, and Pribram (Plans and the Structure of Behavior), Posner and Treisman (Attention), Hunt (Artificial Intelligence), Eysenck (synthesis of Pavlov & Clark Hull's principles of behaviour), and Jensen (Mental Chronometry). Sechenov (1829-1905) is considered the spiritual father of PASS theory. Much of our empirical knowledge has its origin in sensation and perception. Sechenov begins with sensation that provides the basis for thinking. However, it is itself organized by the psychological structure of the individual (see Elements of Thought, 1 878). It is innate, but without experience; the sensory experience has to be coded and organized into some form by the individual. The concept that afferent neurons can have either excitatory or inhibitory functions goes back to early researchers in physiology, the best known among whom again is Sechenov (1878). That we need to respond positively as well as withhold a response, including blocking a habitual response when necessary, is an essential ingredient for self regulation. Several key concepts arising out of Sechenov's physiological studies are central to our contemporary theory: Inhibition, Perception, Memory, Language, and even Consciousness and Freewill. The origin of simultaneous and successive processing is also traced to Sechenov's work. "Perception of simultaneity and succession are specific to distinct receptors.. One is reminded by Sechenov that all stimuli have to be decomposed and then synthesized into spatial and temporal forms; they do not arrive in already organized manner" (Das, Kirby & Jannan, 1979, p. 46).

These are the two basic codes of infmmation - integration irrespective of its content. Pavlov (1927) did several experiments to establish different aspects of activation of neurons: the strength of excitation and inhibition; a balance between the two processes; and mobility between excitatory and inhibitory states that allows us to shift from response to no-response and back to responding. These concepts have a contemporary use in executive functions especially cognitive flexibility and inhibition control. Plarming requires both of these abilities (Das & Misra, 2015). However, Pavlov's second signal system (Pavlov, 1941, Conditioned Reflex, Vol. 2), essentially speech signals, distinguishes humans from other animals. Speech and language as a unique form of 'higher nervous activity' had a direct impact on Luria and Vygotsky. Both Vygotsky and Luria discuss the importance of social and cultural environment. Vygotsky and Luria argued that a significant cultural reconstruction has to take place

PASS Theory of Intelligence

155

in order for the child to shift from the stage of primitive perceptions to the stage of competent forms of adaptation to the external world. This cultural reconstruction involves other people prompting, guiding, rewarding, punishing, restraining, imitating, and modeling the child's behavior. Higher mental functions such as attention, planning, and the two main modes of processing information, simultaneous and successive processing, also require social-cultural interactions.

The PASS Model The functional aspects of brain structures as described in the work of Luria (1966, 1970, 1973) formed the basis of PASS theory. Das, Naglieri, and Kirby (1994) used Luria's work as a blueprint for defming the important components of a neurocognitive view of intelligence. Luria (1970) perceived the brain's basic functions to be represented by three separate but interrelated brain systems that provide four basic psychological processes. The three brain systems are referred to as functional units (Das et aI., 1994; see Figure 7-1). The first functional unit, Attention-Arousal, is located in the brain stem and reticular activating system. This mit provides the brain with the appropriate level of arousal. Since it has strong connections with the frontal lobes, attentional control is also possible. The second functional unit is associated with the occipital, parietal, and temporal lobes posterior to the central sulcus of the brain. This unit is responsible for receiving, processing, and retaining information a person obtains from the external world. It analyzes and synthesizes infOlmation, dividing it in two categories, simultaneous processes and successive processes. Simultaneous processing involves integrating stimuli into groups or the recognition that a number of items share a common characteristic. Examples include recognizing figures, such as a triangle within a circle versus a circle within a triangle, or recognizing the difference between "he had a shower before breakfast' and "he had breakfast before a shower." Whereas simultaneous processing involves working with stimuli that are interrelated, successive processing is required for organizing separate items in a sequence, for example, remembering a sequence of words or actions. The third functional unit involves the frontal lobes. Plarming and its associated executive processes such as decision-making, evaluating, programming, and regulating present and future behavior are the essential functions of frontal lobes. The frontal lobes have intimate connections with the motor cortex and with the structures of the second mit. Their structures become mature only during the fourth to fifth year of life. They develop rapidly and become

Chapter Seven

156

significant for the first forms of conscious control of behavior. Despite Luria' s interpretation of the three functional units, each with separate functions, his focus was on integration among the units, The third functional unit is very closely related to the functions of the first, as both units are concerned with overall efficiency of brain functions; part of the role of the second functional unit is also to establish connections with the third unit. Indeed, according to Reitan ( 1 988), "integration of these systems constitutes the real key to understanding how the brain mediates complex behavior" (p, 333),

INPUT Serial

OUTPUT

ConcWl'ent

Serial

Concurrent

~ Jl --- 1t nmtD ru:-lCTlO�AL Ulm

rntST FUNCTlO�AL UlIIT

AROUS. \.I.l '"

..:

III

\.I.l

8\.I.l � 0

ATTEJI.'110N

�Do g

i:'

0

]""

MOPr

Premature integration of actions, behavioral reactivity lliip7>lsivity DOPr� fDA MOPr CREB)

(Over?)-approval of cmrent actions (or of its absence Satisfaction MOPr� (5-HT DA) MOPr>KOPr

An Overlap between Mental Abilities and Temperament Traits

195

Dynamic features of neurophysiological cycles are based, in tum, on seasonal, daily, metabolic, neurotransmitter and other chemical cycles. In this context, more attention to neurochemistry, and not just to neuroanatomy of behavioural regulation is essential, and future studies of interactions between these chemical systems are needed. Functional constructivism might help to re-formulate what the consistency of temperament traits means, and how these traits could be expressed in a lifetime. If all behavioural acts, even repeated ones, are all being constructed anew and so nothing in behaviour is being repeated, then what exactly do we mean by consistency and stability of temperament traits in a lifetime? Stability of temperament is indeed not a trivial issue, discussed by several prominent researchers who conducted longitudinal and psychogenetic studies (Alarcon et al. 1998, Carter et al. 2017, Flom, Cohen, and Saudino 2017, Krapohl et al. 2018, Plomin 2013, Plomin and von Stumm 2018). A detailed analysis of the ontogenetic stages in the maturation of temperament traits is outside of the scope of this paper, but let us point to most known facts suggesting that regulatory systems (and so temperament traits) are not formed simultaneously in childhood or adulthood. Instead, they likely mature in an asynchronous fashion, gradually unfolding based on changes in neurophysiological capacities of children. These capacities are initially limited by the low myelinisation and interconnectivity of neurons in the fIrst couple of years of life, brief periods of intense trimming death of neurons after birth and in adolescence, low physical endurance in the fIrst 14 years of life, hOlTIlonal adjustments during adolescence, and changes in the density of white and grey matter during several life stages (Lebel and Beaulieu 201 1 , Bava et al. 2010, Benasich, Curtiss, and Tallal 1993), etc. Adults assist children with prioritization, selection and integration of behavioural programs, and often with a partial execution of actions. This adults' assistance compensates for the immaturity of children's nervous systems but variations in the type and volume of such assistance, type of interaction of children with adults, a degree of child's independence, etc. all become factors in children's temperament (i.e. formal dynamical features of their behaviour). This highlights the limitations of the use of infant temperament models for longitudinal and psychogenetic studies on adults. It is not a trivial task to recognize individual differences in young children on temperament traits related to integration and endurance of behaviour, as they are entangled with a child's ability to use adult's assistance. In dealing with this task, as noted above, we suggest to map up the structure of adult temperament in line

196

Chapter Eight

with universal dynamical features of behavioural construction and then, in the ttAdults -7Infantsn direction, see what elements of this behavioural regulation could be recognized in children's behaviour. On genetic analysis o/temperament traits 'When talking about the innate nature and neurochemical basis of CBP, therefore, it is natural to think of genes as being the main underlying biochemical factor. Psychogenetic studies of CBPs using massive samples, however, found surprisingly inconsistent results, and their statistical significance was much lower than one would expect from commonly observed heredity of these CBPs in psychological and psychiatric practices. A contribution of genes was fOlmd only in a few traits (in a low­ to-moderate range) (Sallis et a1. 2018, Brett et a1. 2015, Saudino 2005, Plomin and von Stumm 2018, Krapohl et a1. 2018, Arden and Plomin 2006) and in a few psychiatric disorders. Many of the CBPs studied in humans didn't show significant genetic contribution at all. Those CBPs that did show such contribution (neuroticism, depression, psychopathy, schizophrenia, ADHD and anorexia nervosa) (Eysenck 1990, Sallis et a1. 2018, Saudino 2005, Matheny 1983, Goldsmith and Lemery 2000) had a large portion of variance that could not be explained by genes (40-80%). Yet, despite these rather modest results in psychogenetics, there is a good body of evidence for the biochemical nature of temperament traits and many psychiatric disorders. This suggested the following: 1) Biological and neurochemical regulation of behaviour, as well as the innate nature of some CBPs, carmot be reduced to the structure of the human genome. Instead, several prominent researchers in psychogenetics pointed out the principles of contingency and interaction with environmental factors as the way that genes are expressed (plomin and von Stumm 2018, Krapohl et a1. 2018, Plomin 2013, O'Connor et a1. 1998, Plomin 1986, Schmitt, Eyler, et a1. 2007, Schmitt, Wallace, et a1. 2007). This means that the biochemical nature of CBPs cannot be equated with genes. 2) Disappointingly weak statistical significance of the links between genes and CBPs likely reflects flaws in the selection of variables (temperament scales and tests that assess them) used in psychogenetic studies. Practically none of these studies used neurophysiologically­ validated tests and many of them used variables derived from factor analysis of lexical descriptors Clexical approach!!). This approach inevitably causes a human sociability bias of language and negativity bias of emotional perception which diminishes the statistical significance of the

An Overlap between Mental Abilities and Temperament Traits

197

results (Trofimova 2014, Trofimova et al. 2018). When a model blends socio-cultural and biological factors, studies of genetic correlates of the traits from such a model will most likely struggle to identify what genetic or bio-markers are, and what are not biologically-detelTIlined. Searching for the links between phenomena of a very different nature (genes and processes of social selection) might result in only trivial results if we oversimplify the integrative processes mediating their interaction. For example, the lexically-derived factor-analytic model FFM (Five Factor model of personality) used two traits identified previously in temperament research (neuroticism and extraversion) but could not go beyond them in listing biologically-based traits, despite its claims for finding tfhuman universalstf (McCrae and Costa Ir 1997). In our opinion, for a more adequate analysis, studies in behavioural genetics should use the tests and models that partition their variables, based on psycho-physiological experiments and multi-disciplinary conceptual analysis in biological sciences (Trofimova et al. 2018, Trofimova 2016a, 2018, Trofimova and Robbins 2016, Sulis 2018). 3) 'When conducting statistical analysis, behavioural genetics often treats aspects of behavioural regulation (which are highly entangled) as independent variables (i.e. looking at the parts of a genome and at parts/factors of environments, and then endlessly reporting their associations). Thus, inter-correlations between measured CBP are very common (Sallis, Davey Smith, and Munafo 2018, Flom, Cohen, and Saudino 2017, Frazier-Wood and Saudino 2017), leading to assumptions that inter-correlated CBPs represent tfone factortf when using factor­ analytic language. However, the fact that many systems regulate each other and therefore are not independent will always show up as inter­ correlations in statistical processing of associated data. After all, similar to our five senses (vision, hearing, etc.) that work simultaneously in an ensemble in the regulation of our behaviour, functional aspects of behavioural regulation described in the FET also affect behavioural construction in an entangled manner. Yet, similarly to five sensory systems, these functional aspects likely should be studied and analysed as distinct regulatory systems. In this sense, partitioning of functional aspects of behavioural regulation into a list of variables should not rely on linear correlational methods, such as factor analysis (Trofimova et al. 2018). Moreover, psychogenetic studies report the most significant effect of genes on tfcompositionaltf traits and aspects of behaviour, rather than on single traits or symptoms. For example, stronger statistical significance of these links was found for educational attainment (i.e. for integrated behavioural achievements, rather than for specific intellectual capacities)

198

Chapter Eight

(Plomin and von Stumm 2018, Krapohl et a1. 2018, Plomin 2013, O'Connor et a1. 1998, Plomin 1986, Schmitt, Eyler, et a1. 2007, Schmitt, Wallace, et a1. 2007). Similarly, stronger effects were found for the impact of neuroticism and anxiety on comorbid depression and subjective well­ being (i.e. for a composite of several mental illnesses) (Sallis et a1. 2018, Okbay et a1. 2016), for schizophrenia, ADHD and anorexia nervosa - all knO\vn to have a mixed aetiology and multi-component structure. The benefits of a functional constructivism approach to CBP taxonomies and the FET model are that they suggest variables, which are not independent, which respect feedback and contingent relationships between measured processes, but which are still based on distinct neurochemical biomarkers. These variables reflect components of behavioural regulation that work in an ensemble during construction of behaviour and so could handle inter­ correlations between parts of genomes or traits. 4) Functional constructivism might be useful in solving genetic puzzles of CBPs and what behavioural components are encoded in genomes. Specifics of primates' activities served as a selection factor for their genomes (Leont'ev 1981, Goldberg 2009, Schoonover 2010). Humans and other primates had highly variable and complex environments requiring constant behavioural adjustments. In such environments, the encoding of ready behavioural elements into their genomes, common in animals with stable environments, or genetic imprinting strategies in newborns, as was famously described by Lorentz, would not help: it would be just too much to encode for every possible situation, especially since situations often happen only once in life. Instead, a more adaptive genetic strategy is to encode a universal contingent (i.e. ttif_thentt) ttconstructortt of behaviour with capacities to construct an action tton the gott. Such contingent constructors help primates to consider many possible degrees of freedom Corientationtt colunm in the FET model), to assign priorities to them and to suppress most of them when integrating a behavioral act Cspeed of integrationtt column) and to secure internal and external resources for maintaining the cycles that produce the chosen alternatives Cmaintenancett column). In this sense, genetics likely grants us the ability to stay tuned to specific environmental and internal resources in the construction of contextually-adaptive behaviour. It would be interesting to have psychogenetic studies using the FET model that lists processes (components) of behavioural readiness, universal functional aspects of the construction of behaviour, stages of the cycles underlying the response to specific features of situations in adulthood. 5) One of the puzzles of the human genome is that we have an enOlTIlOUS amount of repeated genetic segments in it. Functional

An Overlap between Mental Abilities and Temperament Traits

199

constructivism presents the genes-based construction of proteins as a probabilistic process. After all, behaviour has universal dynamical aspects that should be repeatedly regulated across various situations, such as energetic plasticity, tempo, maintenance, monitoring the physical environment, monitoring intention of peers, emotional dispositions, etc., as summarized in the FET model. We suggest that repeated sequences in the genome might relate to the universality of these !!construction-related!! aspects. These aspects are present in various situations, and so their encoding would be similar as well. Then, the more often segments are repeated, the higher the probability that in copying processes, the production will use the same configuration. It might be that in evolution, an increase in a number of repeated segments of the genome that were related to benefits for survival, increased species' protection against mutation, consistency in the production of beneficial physiological cycles and so improved the survival rates. When parts of the genome should be !!read!! for a building of specific physiology (which, in tum, regulate our behaviour), a similarity between genome's parts regulating universal dynamical aspects likely helped physiology (and then the behaviour) to be more consistent and integrated. This suggests that a serious multi-disciplinary analysis in biological sciences is needed on how to handle the generational, constructive dynamics between genetic and environmental biochemical factors of behavioural regulation in the development of new psych-psych taxonomies.

Neuro-chemical systems of sustained attention and probabilistic processing Sustained attention as a temperament trait Mental abilities are highly valued in all societies and so it is no surprise that they became a subject of biological analysis right from the birth of psychology at the end of 19th century. As it is well-documented, the pioneers in a fOlTIlal analysis of psychological diversity - Gauss, Galton, Stem - were very intellectually gifted individuals. They likely saw their differences from their peers from a very early age and this probably convinced them of a !!nature, not nurture!! origin of these differences. Many paths into biomarkers of mental abilities were explored since then, from phrenology (morphology of a scalp) to brain structures, genetics, neural networks and neurotransmitter systems. Over the century of investigations, it became clear that mental abilities do not have strict

200

Chapter Eight

representation in specific areas of the brain and they operate in an ensemble, supporting each other's emergence. In studies of intelligence, for example, several authors suggested that intelligence can be presented as a multi-facet concept related to the ability of a person for problem­ solving in probabilistic settings (i.e. something that integrates a set of different types of abilities)(Gardner 201 1 , Raven 1984). Moreover, capacities for probabilistic processing and decision making, as features of intelligence were documented not only in individuals but at a group level, in collectives of social insects, with each of individuals having rather limited intellectual capacities (Sulis 2009). Modern testing of intelligence and other mental abilities includes, for example, memorization, deductions of rules, associations, verbal processing, verbal expression, calculations, 3D-rotation, decision making, classification, imagination, knowledge, understanding abstractions, etc. Out of all these facets of mental abilities, there are at least two that appeared to overlap with temperament traits: sustained attention and probabilistic processing. This overlap was identified due to the fact that these two facets met the criteria of temperament traits: 1) they are linked to specific neurotransmitter systems; 2) consistent individual differences in these traits are found in animals and infants (i.e. pre-cultural individuals); 3) these differences are stable throughout the lifetime, when compared between peers; 4) the traits emerge not only as an ability when measured by aptitude tests but also affect the style and the dynamics of behaviour. Sustained attention is named in the FET as Mental Endurance and relates to the endurance group: to the ability of a person to sustain prolonged and/or intense activity of mental type. Together with two other traits in that row of the FET model (plasticity and probabilistic processing), Mental Endurance (or, in former model - Intellectual Endurance, ERI) was linked to performance of the frontal lobes, with the findings being reported for at least 60 years (Goldberg 2009, Lapiz 2006, Winstanley et a1. 2003, Stuss and Knight 2002, Brown 1985, Pribram 1973, Luria 2012). In differential and temperament psychology, this trait was described as sustained attention (i.e. an ability to stay focused on a task). Thomas and Chess (Chess 1996) included "distractibility" and "attention span" in their model of child temperament; Rothbart and colleagues (Rothbart, Ahadi, and Evans 2000) described the similar temperament trait of Effortful Control (defined as the ability to sustain attention and inhibit impulses, to react to distractions), and Rusalov called it "Intellectual Ergonicity" in his STQ model (Rusalov 2018, Rusalov 2007, Rusalov 2004, 1997b).

An Overlap between Mental Abilities and Temperament Traits

201

Mental Endurance meets the criteria of a temperament trait in the following way. 1) Neurochemical nature. Developments in neurochemistry and psychopharmacology revealed the key role in sustained attention of acetylcholine (ACh) systems (Beane and Marrocco 2004, Mesulam and LARRY 2009, Robbins and Roberts 2007, Robbins 1997, Robbins and Everitt 1995, Sarter, Givens, and Bruno 2001). As noted by Sarter, Givens, and Bruno (2001) sustained attention involves not only frontal-cortical ACh systems but also subcortical Ach systems regulating automatic elements of behaviour. Frontal Ach systems, therefore, might provide attention to complex elements of behaviour in fluid probabilistic activities, such as hunting, chess games, mathematical reasoning, accounting, etc., whereas subcortical ACh systems provide the tfbackground supporttf and well-learned tfbuilding blockstf for such activities. Since more than 90% of our behaviour is automatic and not regulated by consciousness, ACh subcortical and basal ganglia systems that monitor automatic, well-learned or simple elements of behaviour, appear to be a crucial player in mental endurance also required for complex intellectual tasks and behavioural challenges. Sustained, tonic attention to a wide range of current relevant stimuli, which is provided by the ACh systems, is complemented by the functions of cortical noradrenaline (NA) systems, which also were linked to attention but mainly attention to novelty (Beane and Marrocco 2004, Aston-Jones, Rajkowski, and Cohen 2000, Robbins 1997, Coull et al. 1997). 2) Being found in pre-cultural individuals. The existence of consistent individual differences in this trait in pre-cultural individuals was noted in different abilities for animals to hunt, to pursue targets and to sustain attention. However, in humans, this criteria for the Effortful Control as a temperament trait became more clear after studies of Rothbart and Posner on infants (posner and Rothbart 2018, Posner, Rothbart, and Voelker 2016, Rueda et al. 2005) and extensive animal studies (Klinkenberg, Sarnbeth, and Blokland 2011). 3) Stability over the lifetime. Longitudinal studies of effortful control showed that individual differences in this trait are consistent across at least the first quarter of life (posner and Rothbart 2018, Posner, Rothbart, and Voelker 2016, Posner et al. 2014). Moreover, psychogenetic studies also suggest that mental endurance and impulse control, important for sustained attention might have genetic links (Rothbart and Posner 2005, Rueda et al. 2005, Matheny 1989). 4) A distinct dynamical pattern in behavioural regulation. Mental endurance or sustained attention was, of course, studied as an ability

202

Chapter Eight

measured by aptitude tests, tests for intelligence and cognitive impainnents. What makes it also a temperament trait is that it relates to a universal dynamic feature of behavioural construction, namely to an energetic capacity of a prolonged perfOlmance of specific type. In this case, it relates to endurance in inhibition of immediate impulses, in ignoring distractions with persistence on monitoring specific targets of actions. Both excessive and low expression of this trait (i.e. obsessive-compulsive tendencies and ADHD) are usually not considered as abilities but spontaneously affect an individual's style of behaviour when they are present. Probabilistic processing as a temperament trait In the 1920s, Swiss psychiatrist Carl Jung for the first time described a temperament trait of introversion, as opposed to extraversion (Jung 2014). He noticed that healthy people differ in their preferences for orientation and directing their behaviour. Introversion was presented as a CBP with an orientation to 0\Vll thinking, internal thoughts and feelings, and extraversion as behavioural regulation by communication, motives and thoughts of other people. Since then, the defiintion of introversion­ extraversion had fluctuated in various temperament and personality models from its original definition, however, the telTIl became the most frequently used concept in differential psychology. In the research of intelligence in the early mid-20th century, independently from the research on temperament, John Raven, a student of SpealTIlan, described a "general cognitive ability" for deduction of commonalities and rules driving changes in presenting stimuli. Raven called this ability tteductivett (from the Latin word tteducerett, which means "to draw out") and offered non­ verbal Progressive Matrices for measuring this ability, which became one of the most popular methods of measuring non-verbal intelligence around the world (Raven 2000). Temperament research and intelligence research, therefore, have converged on describing and measuring this trait, and both lines of research considered the individual differences in this CBP as biologically-based. What made Jung and other researchers think about this ability as a temperament trait is the impact of this trait on preferences, motivation and orientation of behaviour, and not just efficiency in mental tasks. This trait, called in the FET model initially as Sensitivity to Probabilities, and then Probabilistic Processing, emerges in behaviour as a drive to differentiate and to categorize features of objects/events, to gather probabilistic information related to reality (extreme events, possibilities of future events, frequency, commonality, causal relations between observed events,

An Overlap between Mental Abilities and Temperament Traits

203

etc.). This behavioural orientation expresses itself truly as a drive: individuals gravitate to solving puzzles, observations of nature, learning additional knowledge, even when they have other things to do. People with high expression of this trait not only obey rules better, they actually like learning about rules and their justification. They analyze the logic of these rules and often think about various societal setups, even when there is no need for it. Analytic and logical games, as well as philosophical activities and discussions, are fun for them. Most of the people with preferences for probabilistic learning do not have high sensation- or risk-seeking, contrarily to suggestions to unite Openness to Experience and Sensation Seeking traits. Risk- and sensation­ seeking activities often go contrarily to their tendency to !!play smart!!, plus they often appreciate a high probability of adverse consequences from risks, and avoid complicating their lives. Indeed, there is a common observation of a correlation between neuroticism and high intelligence, and both of these CBPs are based on functional NA systems. However, people with high expression of this trait often take more projects on their shoulders than people with a low expression, have high employment rates, and this might put them in a higher risk of having a job- or traffic-related accidents. They are also less prone to be influenced by peer pressure, as they prefer to derive their own model of reality and events. The behaviour of an individual with high expression of this trait is, therefore, regulated and orientated to internally-developed plans and preferences for actions, being efficient in extraction and processing of new knowledge related to implicit, not just concrete features of events. They often are diagnosed with high learning abilities, as they can efficiently map and re-arrange coming infOlmation into their internal storage of probabilistic expectations of events. Probabilistic Processing, or Sensitivity to Probabilities, as described in the FET model, meets all the criteria of temperament traits: 1) Neurochemical nature. Recent developments in neurochemistry and psychopharmacology revealed the key role of frontal monoamine systems in the ability of an individual for probabilistic processing. Probabilistic processing involves gathering the infOlmation about frequency and causes of events, facilitating the prediction of their future occurrence, based on these frequencies and contributing factors. It appeared that cortical monoamine systems indeed regulate probabilistic processing, enabling an individual to do the following under the lead of noradrenaline (NA) systems: a) to collect a wide range of infonnation about stimuli (i.e. to have strong orienting capacities). As noted above, NA systems were linked to

204

Chapter Eight

such capacities, especially in a context to orientation to novelty (i.e. not pre-determined but highly fluid, probabilistic information) (Beane and Marrocco 2004, Aston-Jones, Rajkowski, and Cohen 2000, Robbins 1997, Coull et a1. 1997); b) to assign priorities to specific sources of infOlmation or features of objects, changing these priorities according to individual needs, with a suppression of a big volume of less relevant infOlmation to be processed. Functionality of dopaminergic (DA) systems indeed was linked to assigning priorities to behavioural elements, be it motivational, cognitive or motor (Robbins 2010, Seamans and Robbins 2010, Rey, Lew, and Zanutto 2007, Grace et a1. 2007, Berridge 2007, Devoto and Flore 2007, McClure, Gilzenrat, and Cohen 2006, Floresco and Magyar 2006, Sealfon and Olanow 2000, Trofimova 2016a, Trofimova and Robbins 2016); c) to maintain a store of gathered knowledge, which is constantly updated in terms of probabilities of events and commonality of features of objects. Such constancy in a collection of probabilities of events and properties of events/objects employs the properties of ACh and 5-HT (serotonergic) systems, allowing to have simultaneous activation of several alternatives for classification and assessment of stimuli during probabilistic processing of events. Indeed, experiments inducing a depletion of 5-HT in orbital-frontal cortex showed that affected individuals, both humans and non-humans lose their ability to re-Iearn infonnation (i.e. to update their ttprobabilistic mapstt in time) and impaired their ability to suppress irrelevant knowledge (Clarke et a1. 2007, Walker, Robbins, and Roberts 2009). Forebrain depletion of 5-HT has also been linked to premature initiation of actions (i.e. impulsivity) (Dalley, Everitt, and Robbins 201 1 , Winstanley et a1. 2003, Miyazaki, Miyazaki, and Doya 2012). From the perspective of the Functional Ensemble of Temperament model, probabilistic processing is an intetplay between cortical monoamine systems with the following distribution of labour: the interactions between the NA and 5HT systems likely regulate the transfers between management of novel vs. established aspects of events; interactions between the NA and DA systems likely regulate expansion/search prioritization of infonnation during probabilistic processing; finally, interactions between DA and 5-HT likely relate to the processing of infonnation related to programming vs. execution of potential actions. 2) Being f01.md in pre-cultural individuals. The existence of consistent individual differences in the probabilistic processing was noted in animals, as the differences in the ability to solve puzzles, learning rules and deduction of algorithms within assigned tasks (Le Pelley et a1. 2005,

An Overlap between Mental Abilities and Temperament Traits

205

Robinson et al. 2008). In terms of humans, infant studies showed that the ability to process probabilistic elements of reality and to seek causes and rules governing events and phenomena can differentiate children from very early childhood. The desire to acquire abstract knowledge such as mathematics, physics, philosophy, or other sciences; the desire to predict events analyzing the past, to derive the fundamental properties of objects, and to find new algorithms to solve problems - such an orientation appears in childhood as a preference for investigative, reasoning, and learning activities. Contrary to Piaget's suggestions that preschoolers don't use cause­ consequence analysis, recent studies have sho\Vll that the ability for probabilistic thinking could be found in two-year-old children (Sobel 2006, Gopnik 2001, Hickling 2001, Harris, German, and Mills 1996) and five-year-old children (Oakes 1990, Bullock 1982, Perner 1991, Spelke 1992). Studies on infant contingency learning (Rovee-Collier 1987, Leslie 1987, Watson 1972, Oakes 1990) showed that even several months-old infants differ in learning the causality of their own actions by observing the relations between those actions and the events that follow them. The ability and orientation to acquire probabilistic knowledge about the world has been studied under the umbrella of intelligence for more than a century. 3) Stability over the lifetime. Most extensive longitudinal studies of the capacities for probabilistic processing were conducted in the context of intelligence (Deary, Pattie, and Starr 2013, Salarirad et al. 201 1 , Luciano et al. 2009). These studies used a set of measures of intelligence, so it is hard to separate the developmental trajectory of probabilistic processing capacities and life changes in other aspects of intelligence. Still, probabilistic processing was a main contributing component in perfOlmance on those tests and emerged as consistency of "fluid" intelligence. Moreover, psychogenetic studies show that "there is something out there" in terms of the genetic factors of intelligence (Savage et al. 2018, Plomin and von Stumm 2018). Further investigations with a use of the FET-related test (STQ-77) and tests measuring probabilistic processing might shed a light on this complex matter. 4) Distinct dynamical pattern in behavioural regulation. As noted above, capacities for probabilistic processing affect the dynamics of behaviour, behavioural orientation, motivation and preferences. For this reason, the FET model suggests this trait relates to the group of "orientational" traits, affecting construction of behaviour by its preferred orientation to specific reinforcers. As famous physicist Richard Feynman put it, there is "the pleasure of finding things out." This behavioural

206

Chapter Eight

orientation is commonly observed in talented scientists and engineers, who have expressed their curiosity about laws of nature since their childhood.

Summary In this chapter we examined the overlap between two concepts, mental abilities and temperament. Both concepts are related to biologically-based consistent behavioural patterns in mentally-healthy people, but there are several differences between them. Temperament, by its original defmition, refers to neurochemical systems of behavioral regulation and also primarily to dynamical aspects of this regulation: endurance, speed of integration, orientational and emotional dispositions, and reactivity. Two temperament traits were analyzed that overlapped with mental abilities studied in the context of intelligence: mental endurance (also knO\vn as sustained attention or effortful control) and orientation to probabilistic processing, preferences for learning commonalities and causalities of events (overlapping with the concepts of learning capabilities and intelligence). The first trait is related to the energetic aspects of behaviour (how long an individual can sustain the activity) and the second trait emerges in orientational aspects of activity, consistently affecting an individual's preferences and the directionality of behaviour. We discussed difficulties in identifying biomarkers of both traits stemming from flaws in early research methods. We pointed out that early longitudinal and genetic studies that looked for these biomarkers and measured the lifelong stability of traits used scales that were based on parents' observations of infants. These scales were not validated by links to specific neurophysiological systems, and descriptors that were used for assessing infants were largely incompatible with the expression of temperament in adulthood. We suggested that the use of the neurochemical model, Functional Ensemble of Temperament, that was developed based on a multi-disciplinary analysis of findings in the psycho-biological sciences to assess adult temperament, might yield reliable results in longitudinal and psychogenetic research.

References Adell, Albert, Analia Bortolozzi, Lloreny Diaz-Mataix, Noemi Santana, Pau Celada, and Francesc Artigas. 2010. "Serotonin interaction with other transmitter systems." In: Muller C, Jacobs B. (Eds.) Handbook of Behavioral Neurobiology ofSerotonin. NY: Elsevier Academic Press.

An Overlap between Mental Abilities and Temperament Traits

207

Alarcon, M., R. Plomin, D. W. Fulker, R. Corley, and J. C. DeFries. 1998. !!Multivariate path analysis of specific cognitive abilities data at 12 years of age in the Colorado Adoption Project.!! Behavioural Genetics 28(4):255-64. Alexandrov, Yuri, Krylov AK and Arutyunova KR."Activity during Learning and the Nonlinear Differentiation of Experience!!.Nonlinear Dynamics in Psychology and Live Sciences. 21(4):391-405 Andre, K., O. Kampman, M. Viikki, E. Setala-Soikkeli, A. Illi, N. Mononen, T. Lehtimaki, and E. Leinonen. 2018. "BDNF and NRGI polymorpbisms and temperament in selective serotonin reuptake inhibitor-treated patients with major depression.!! Acta Neuropsychiatr 30 (3):168-174. doi: 10.1017/neu.2017.37. Arden, Rosalind, and Robert Plomin. 2006. !!Sex differences in variance of intelligence across childhood.!! Personality and Individual Differences 41 (1):39-48. Aston-Jones, Gary, Janusz Rajkowski, and Jonathan Cohen. 2000. !!Locus coeruleus and regulation of behavioral flexibility and attention.!! Progress in brain research 126: 165-182. Azmitia, Efrain C. 2010. !!.l-Evolution of Serotonin: Sunlight to Suicide.!! Handbook o/Behavioral Neuroscience 21:3-22. Ball, Samuel A, Howard Tennen, James C Poling, Hemy R Kranzler, and Bruce J Rounsaville. 1997. !!Personality, temperament, and character dimensions and the DSM-rv personality disorders in substance abusers." Journal 0/ abnormal psychology 106 (4):545. Bari, Andrea, and Trevor W Robbins. 2013. !!Inhibition and impulsivity: behavioral and neural basis of response controL!! Progress in neurobiology 108:44-79. Barr, Christina S, Melanie L Schwandt, Stephen G Lindell, J Dee Higley, Dario Maestripieri, David Goldman, Stephen J Suomi, and Markus Heilig. 2008. "Variation at tbe mu-opioid receptor gene (OPRMl ) influences attachment behavior in infant primates.!! Proceedings of the NationalAcademy o/Sciences 105 (13):5277-5281. Barraza, Jorge A, and Paul J Zak. 2009. "Empathy toward strangers triggers oxytocin release and subsequent generosity.!! Annals of the New YorkAcademy o/Sciences 1 167 (1):182-189. Baruch, !lan, David R Hemsley, and Jeffrey A Gray. 1988. "Latent inhibition and "psychotic proneness" in nOlmal subjects.!! Personality and Individual Differences 9 (4):777-783. Bava, S., R. Thayer, J. Jacobus, M. Ward, T. L. Jernigan, and S. F. Taper!. 2010. !!Longitudinal characterization of white matter maturation during

208

Chapter Eight

adolescence." Brain Research 1327:38-46. doi: 10.1016/j.brainres. 2010.02.066. Beane, M, and RT Marrocco. 2004. ttNorepinephrine and acetylcholine mediation of the components of reflexive attention: implications for attention deficit disorders." Progress in neurobiology 74 (3): 167- 1 8 1 . Beere, Don, and Michael Pica. 1995. ttThe predisposition to dissociate: The temperamental traits of flexibilitylrigidity, daily rhythm, emotionality and interactional speed. n Dissociation: Progress in the Dissociative Disorders. Benasich, A. A., S. Curtiss, and P. Tallal. 1993. tfLanguage, learning, and behavioral disturbances in childhood: a longitudinal perspective. n JAm Acad ChildAdolesc Psychiatry 32 (3):585-94. doi: 10.1097/00004583199305000-00015. Bernstein, Nicolai A1eksandrovitch, Mark L Latash, and Michael Turvey. 1996. Dexterity and its development Taylor & Francis. Berridge, Kent C. 2007. !The debate over dopamine's role in reward: the case for incentive salience." Psychopharmacology 191 (3):391-43 1 . Betts, K. S., P. Baker, R. Alati, J. E. McIntosh, J. A. Macdonald, P. Letcher, and C. A. Olsson. 2016. "The natural history of internalizing behaviours from adolescence to emerging adulthood: findings from the Australian Temperament Project." Psychol Med 46 (13):2815-27. doi: 10.1017/S0033291716001495. Bielsky, Isadora F, and Larry J Young. 2004. nOxytocin, vasopressin, and social recognition in mammals." Peptides 25 (9): 1565-1574. Binet, Alfred and T.R. Simon. 1907. Mentally defective children London: Edward Arnold, UK. Bishop, D. V., and G. Baird. 2001. "Parent and teacher report of pragmatic aspects of communication: use of the children's communication checklist in a clinical setting." Dev Med Child Neurol 43 (12):809-18. Blount, Z. D. 2016. !fA case study in evolutionary contingency. n Stud Hist Philos Bioi Biomed Sci 58:82-92. doi: 10.1016/j .shpsc.2015.12.007. Brett, Z. R., M. Sheridan, K. Humphreys, A. Smyke, M. M. Gleason, N. Fox, C. Zeanah, C. Nelson, and S. Drury. 2015. !fA neurogenetics approach to defining differential susceptibility to institutional care. n Int J Behav Dev 39 (2): 150-160. doi: 10.1 177/0165025414538557. Bro'Wll, I. W. 1985. nFrontal lobes and the microgenesis of action. n Journal of Neurolinguistics 1 :31-77. doi: 10.1016/S09116044(85)80004-X Bro'Wll, Timothy A. 2007. ttTemporal course and structural relationships among dimensions of temperament and DSM-rv anxiety and mood disorder constructs." Journal ofAbnormal Psychology 1 1 6 (2):313.

An Overlap between Mental Abilities and Temperament Traits

209

Bruchas, MR, BB Land, and C Chavkin. 2010. "The dynorphinlkappa opioid system as a modulator of stress-induced and pro-addictive behaviors. n Brain research 13 14:44-55. Bullock, M., Gelman, R., & Baillargeon, R. 1982. "The development of causal reasoning. In W. J. Friedman (Ed.), ." The developmental psychology oftime:209-254. Buss, Arnold H, and Robert Plomin. 2014. Temperament (PLE: Emotion): Early Developing Personality Traits. Vol. 3: Psychology Press. Buzzell, G. A., S. V. Troller-Renfree, T. V. Barker, L. C. Bowman, A. Chronis-Tuscano, H. A. Henderson, I. Kagan, D. S. Pine, and N. A. Fox. 2017. "A Neurobehavioral Mechanism Linking Behaviorally Inhibited Temperament and Later Adolescent Social Anxiety." J Am Acad Child Adolesc Psychiatiy 56 (12): 1097-1 105. doi: 10.1016/j .jaac.2017.10.007. Cacabelos R, Niigawa H, Rodriguez-Amao MD, Gomez-Pan A, Nishimura T . . 1998. nInfluence of somatostatin and growth hOlTIlone­ releasing factor on behavior. Clinical and therapeutic implications in neuropsychiatric disorders." Horm Res. 29(2-3):129-132. Carlezon, William A, Cecile Beguin, Allison T Knoll, and Bruce M Cohen. 2009. "Kappa-opioid ligands in tlie study and treatment of mood disorders." Pharmacology & therapeutics 123 (3):334-343. Carter, B., K. Ford, L. Bray, T. Water, I. Amott, C. Tichanow, and A. Dickinson. 2017. nnIt's not who I amn: Children's experiences of growing up with a long-term condition in England, Australia, and New Zealand." J Spec Pediatr Nurs 22 (4). doi: 10. 1 1 1 1/jspn.12197. Chess, S., Thomas, A. 1996. Temperament: Theory and Practice. York:: BrunnerlMazel. Clark, Lee A, David Watson, and Susan Mineka. 1994. nTemperament, personality, and the mood and anxiety disorders. n Journal of abnormal psychology 103 (1):103. Clarke HF, Dalley JW, Crofts HS, Robbins TW, Roberts AC 2004. nCognitive inflexibility after prefrontal serotonin depletion. n Science 304:878-880. Clarke, HF, SC Walker, JW Dalley, TW Robbins, and AC Roberts. 2007. nCognitive inflexibility after prefrontal serotonin depletion IS behaviorally and neurochemically specific. n Cerebral Cortex 17 (1):1 8-27. Coplan ill, Aaronson CI, Pathangi V, Kim Y. . 2015. nTreating comorbid anxiety and depression: psychosocial and phalTIlacological approaches." WorldJ Pschiatiy 5(4):366-378 doi: 105498/wjpv5i4366.

210

Chapter Eight

Coull, IT, CD Frith, RJ Dolan, RSJ Frackowiak, and PM Grasby. 1997. !The neural correlates of the noradrenergic modulation of human attention, arousal and learning. n European Journal of Neuroscience 9 (3):589-598. Coyle, IT, and GT Konopask. 2012. !The neurochemistry of schizophrenia. n Basic Neurochemistry, 8th edition, ST Brady, GJ Siegel, R. Wayne Albers, and DL Price, eds. (Amsterdam: Elsevier): 1 000-1 O I l . Dalley, Jeffrey W, Barry J Everitt, and Trevor W Robbins. 201 1 . ttlmpulsivity, compulsivity, and top-do\Vll cognitive control. n Neuron 69 (4):680-694. Deary, I. J., A. Pattie, and J. M. Starr. 2013. "The stability of intelligence from age 1 1 to age 90 years: tlie Lothian birtli cohort of 1921." Psychol Sci 24 (12):2361-8. doi: 10.1177/0956797613486487. Degli Uberti, EC, F Petraglia, M Bondanelli, AL Guo, A Valentini, S Sa1vadori, M Criscuolo, RE Nappi, and AR Genazzain. 1995. nlnvolvement of I-L-opioid receptors in the modulation of pituitary­ adrenal axis in llOlmal and stressed rats. n Journal of endocrinological investigation 18 (1):1-7. Depue, Richard A, and Jeannine V Morrone-Strupinsky. 2005. "A neurobehavioral model of affiliative bonding: Implications for conceptualizing a human trait of affiliation. n Behavioral and Brain Sciences 28 (3):313-349. Devoto, Paola, and Giovarma Flore. 2007. !!Dopamine and Noradrenaline Coupling in the Cerebral Cortex." In Monoaminergic Modulation of Cortical Excitability, 189-196. Springer. Donaldson, Zoe R, and Larry I Young. 2008. !!Oxytocin, vasopressin, and the neurogenetics of sociality." Science 322 (5903):900-904. Dunbar, Robin 1M, and Susanne Shultz. 2007. "Understanding primate brain evolution." Philosophical Transactions of the Rayal Society of London B: Biological Sciences 362 (1480):649-658. Eysenck, Hans I. 1990. !!Genetic and environmental contributions to individual differences: The three major dimensions of personality.!! Journal ofpersonality 58 (1):245-26 1 . Eysenck, HJ. 1983. "Psychophysiology and personality: Extraversion, neuroticism and psychoticism.!! Physiological cOJTelates of human behavior 3: 13-30. Eysenck, Sybil BG, Paul R Pearson, G Easting, and John F Allsopp. 1985. !!Age nOlTIlS for impulsiveness, venturesomeness and empathy in adults." Personality and individual differences 6 (5):613-619. Faria, A. V., I. Zhang, K. Oishi, X. Li, H. Jiang, K. Akhter, L. HelTIloye, S. K. Lee, A. Hoon, E. Stashinko, M. I. Miller, P. C. van Zij1, and S.

An Overlap between Mental Abilities and Temperament Traits

211

Mori. 2010. "Atlas-based analysis of neurodevelopment from infancy to adulthood using diffusion tensor imaging and applications for automated abnormality detection." Neuroimage 52 (2):415-28. doi: 10.1 016/j .neuroimage.201 0.04.238. Farrow, T. F., Zheng, Y., Wilkinson, I. D., Spence, S. A., Deakin, I. F., Tarrier, N. 2001. "Investigating the functional anatomy of empathy and forgiveness." Neuroreport 12:2433-2438. Filliol, Dominique, Sandy Ghozland, Johanna Chluba, Miguel Martin, Hans \VD Matthes, Frederic Simonin, Katia Befort, Claire Gaveriaux­ Ruff, Andree Dierich, and Mariatme LeMeur. 2000. !!Mice deficient for (i-and I-L-opioid receptors exhibit opposing alterations of emotional responses." Nature genetics 25 (2):1 95-200. Finn, A. S., C. L. Hudson Kam, M. Ettlinger, J. Vytlacil, and M. D'Esposito. 2013. "Learning language with the wrong neural scaffolding: the cost of neural commitment to sounds.!! Front Syst Neurosci 7:85. doi: 10.3389/fusys.2013.00085. Flom, M., M. Cohen, and K. J. Saudino. 2017. "Tipping points? Curvilinear associations between activity level and mental development in toddlers." J Child Psychol Psychiatry 58 (5):564-572. doi: 10. 1 1 1 1/jcpp.12670. Floresco, Stan B, and Orsolya Magyar. 2006. !!Mesocortical dopamine modulation of executive functions: beyond working memory.!! Psychopharmacology 1 8 8 (4):567-585. Follette, W.C. & Houts, A.C . . 1996. "Models of scientific progress and the role of theory in taxonomy development: a case study of the DSM." Journal of Consulting and Clinical Psychology 64(6): 1 120-1 132. Fox, N. A., N. Snidman, S. A. Haas, K. A. Degnan, and J. Kagan. 2015. !The relation between reactivity at 4 months and Behavioral Inhibition in the second year: Replication Across Three Independent Samples.!! Infancy 20 (1):98-114. doi: 10. 1 1 1 1/infa.12063. Franklin, T. B., B. 1 Saab, and I. M. Mansuy. 2012. "Neural mechanisms of stress resilience and vulnerability." Neuron 75 (5):747-61. doi: 10.1 016/j .neuron.2012.08.016. Frazier-Wood, A. C., and K. J. Saudino. 2017. "Activity level in the lab: Overlap with shyness indicates it is more than pure motoric activity.!! Dev Psychol 53 (9):161 1-1619. doi: 10.1037/dev0000348. Freeman, Walter. 2012. Neurodynamics: an exploration in mesoscopic brain dynamics: Springer Science & Business Media. Gabel, S., Stadler, J., Bjorn, 1, Shindledecker, R., & Bowden, C. L. 1994. "Sensation seeking in psychiatrically disturbed youth: Relationship to

212

Chapter Eight

biochemical parameters and behavior problems. n Journal ofAmerican Academy ofChild andAdolescent Psychiatry 33: 123-129. Gardner, Howard. 201 1 . Frames of mind : the theory of multiple intelligences. New York: Basic Books. Gerra, Gilberto, Paola Avanzini, Amir Zaimovic, Roberto Sartori, Carlotta Bocchi, Mariaemanuela Timpano, Ursula Zambelli, Roberto Delsignore, Federica Gardini, and Emica Talarico. 1999. ttNeurotransmitters, neuroendocrine correlates of sensation-seeking temperament in llOlmal humans." Neuropsychobiology 39 (4):207-213. Goldberg, Elkhonon. 2009. The new executive brain : frontal lobes in a complex world. Oxford ; New York: Oxford University Press. Goldsmith, H. H., and K. S. Lemery. 2000. "Linking temperamental fearfulness and anxiety symptoms: a behavior-genetic perspective. n Bioi Psychiatry 48 (12): 1 1 99-209. Gopnik, A., Sobel, D. M., Schulz, L. E., & Glymour, C . . 2001. "Causal learning mechanisms in very young children: Two, three, and four­ year-olds infer causal relations from patterns of variation and covariation." Developmental Psychology 37:620-629. Grace, Anthony A, Stan B Floresco, Yukiori Goto, and Daniel J Lodge. 2007. ttRegulation of fIring of dopaminergic neurons and control of goal-directed behaviors." Trends in neurosciences 30 (5):220-227. Gray, Jeffrey A. 1982. "On mapping anxiety." Behavioral and Brain Sciences 5 (03):506-534. Gray, Jeffrey A. 1991. "The neuropsychology of temperament." In Explorations in temperament, 105-128. Springer. Harris, P. L., T. German, and P. Mills. 1996. !!Children's use of counterfactual thinking in causal reasoning." Cognition 61 (3):233-59. Heilig, M. 2004. !!The NPY system in stress, anxiety and depression.!! Neuropeptides 38:213-224. Hensler, Julie G. 2006. !!Serotonin.!! Basic neurochemistry:227-248. Hickling, A. K; & Wellman, H. M . . 2001 . "The emergence of children's causal explanations and theories: evidence from everyday conversation." Developmental Psychology 37(5):668-683. Humphreys, K. L., D. Miron, K. A. McLaughlin, M. A. Sheridan, C. A. Nelson, N. A. Fox, and C. H. Zeanah. 2018. !!Foster care promotes adaptive functioning in early adolescence among children who experienced severe, early deprivation." J Child Psychol Psychiatry 59 (7):81 1-821. doi: 10. 1 1 1 1/jcpp.12865. Jones, N. A., and A. Sloan. 2018. !!NeurohOlTIlOneS and temperament interact during infant development." Philos Trans R Soc Land B Bioi Sci 373 (1744). doi: 10.1098/rstb.2017.0159.

An Overlap between Mental Abilities and Temperament Traits

213

Jung, Carl Gustav. 2014. Psychological types: Routledge. Kagan, Jerome, and Nancy Snidman. 2009. The long shadow of temperament Harvard University Press. Kampman, 0., o. Poutanen, A. lUi, E. Setala-Soikkeli, M. Viikki, T. Nuolivirta, and E. Leinonen. 2012. !!Temperament profiles, major depression, and response to treatment with SSRIs in psychiatric outpatients." European Psychiatry 27 (4):245-9. doi: 10.1 016/j .eurpsy.2010.07.006. Kampman, 0., M. Viikki, K. Jarventausta, and E. Leinonen. 2014. !!Meta­ analysis of anxiety disorders and temperament.!! Neuropsychobiology 69 (3): 175-86. doi: 10. 1 159/000360738. Kampman, 0., M. Viikki, and E. Leinonen. 2017. !!Anxiety Disorders and Temperament-an Update Review." Current Psychiatry Reports 19 (5):27. doi: 10.1007/s1 1 920-017-0779-5. Kapur, Shitij . 2003. !!Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia.!! Americanjournal ofPsychiatry. Karam, Elie G, Mariana M Salamoun, Joumana S Yeretzian, Zeina N Mneinmeh, Aimee N Karam, John Fayyad, Elie Hantouche, Kareen Akiskal, and Hagop S Akiska!. 2010. "The role of anxious and hyperthymic temperaments ill mental disorders: a national epidemiologic study." World Psychiatry 9 (2):103-110. Kehagia, Angie A, Graham K Murray, and Trevor W Robbins. 2010. !!Learning and cognitive flexibility: frontostriatal flUlction and monoaminergic modulation.!! Current opinion in neurobiology 20 (2): 199-204. Keveme, Eric B, and James P Curley. 2004. !!Vasopressin, oxytocin and social behaviour." Current opinion in neurobiology 14 (6):777-783. Klein, Daniel N, Roman Kotov, and Sara J Bufferd. 201 1 . "Personality and depression: explanatory models and review of the evidence.!! Annual review of clinical psychology 7:269. Klinkenberg, I., A. Sambeth, and A. Blokland. 201 1 . "Acetylcholine and attention." Behavioral Brain Research 221 (2):430-42. doi: 10.1 016/j .bbr.2010. 1 1 .033. Knudsen, E. I. 2004. !!Sensitive periods in the development of the brain and behavior." Journal of Cognitive Neuroscience 16 (8): 1412-25. doi: 10.1 162/0898929042304796. Krapohl, E., H. Patel, S. Newhouse, C. J. Curtis, S. von Stunnn, P. S. Dale, D. Zabaneh, G. Breen, P. F. O'Reilly, and R. Plomin. 2018. !!Multi-polygenic score approach to trait prediction.!! Mol Psychiatry 23 (5):1368-1374. doi: 10.1038/mp.2017.163.

214

Chapter Eight

Kumari, Veena, Brian Toone, and Jeffrey A Gray. 1997. nHabituation and prepulse inhibition of the acoustic startle reflex: effects of smoking status and psychosis-proneness. n Personality and individual differences 23 (2): 183-191. Lange KH, Isaksson F, Juul A, Rasmussen MH, BUlow J, Kjaer M. 2000. tfGrowth hOlTIlone enhances effects of endurance training on oxidative muscle metabolism in elderly women. n Am J Physiol Endocrinol Metab 279(5):E989-96. Lapiz, M.D., & Morilak, D.A. . 2006. "Noradrenergic modulation of cognitive function in rat medial prefrontal cortex as measured by attentional set shifting capability." Neuroscience 137:1039-1049. Larkin, H., J. J. Shields, and R. F. Anda. 2012. "The health and social consequences of adverse childhood experiences (ACE) across the lifespan: an introduction to prevention and intervention in the community." J Prev Interv Community 40 (4):263-70. doi: 10.108011 0852352.2012. 707439. Le Pelley, M. E., S. M. Oakeshott, A. J. Wills, and I. P. McLaren. 2005. !The outcome specificity of learned predictiveness effects: parallels between human causal learning and animal conditioning. n J Exp Psychol Anim Behav Process 3 1 (2):226-36. doi: 10.1037/00977403.31.2.226. Lebel, C., and C. Beaulieu. 201 1 . "Longitudinal development of human brain wiring continues from childhood into adulthood. n J Neurosci 3 1 (30): 10937-47. doi: 10.1523/JNEUROSCI.5302-10.2011. Leont'ev, A. N. 1981. Problems of the development of the mind. Moscow: Progress. Leslie, A. M., & Keeble, S . . 1987. nDo six-month-old infants perceive causality? ." Cognition 25:265-288. Lu X, Chen Z, Cui X, Uji M, Miyazaki W. 2012. ". Effects of temperament and character profiles on state and trait depression and anxiety: a prospective study of a Japanese youth population,. n Depress Res Treat604684. doi: 1 0 1 15512012/604684. Luciano, M., A. J. Gow, S. E. Harris, C. Hayward, M. Allerhand, J. M. Starr, P. M. Visscher, and I. J. Deary. 2009. "Cognitive ability at age 1 1 and 70 years, information processing speed, and APOE variation: the Lotbian Birtb Cohort 1936 study." Psychol Aging 24 (1): 129-38. doi: 10.1037/aOOI4780. Luria, Aleksandr Romanovich. 2012. Higher cortical functions in man: Springer Science & Business Media. Marm, J. J., Currier, D., Quiroz, J. A., & Manji, K. H. (2006. nNeurobiology of severe mood and anxiety disorders. n In Basic

An Overlap between Mental Abilities and Temperament Traits

215

neurochemistry, 7th edition, edited by G . Siegel, Albers, R. W., Brady, S., & Price, D., 887-909. USA: Elsevier Press. Matbeny, A. P., Ir. 1983. "A longitudinal twin study of stability of components from Bayley's Infant Behavior Record." Child Dev 54 (2):356-60. Matbeny, A. P., Ir. 1984. "Twin similarity in the developmental transfOlmations of infant temperament as measured in a multi-method, longitudinal study." Acta GenetMed Gemellol (Roma) 33 (2) : 1 8 1-9. Matheny, A. P., Ir. 1989. nChildren's behavioral inhibition over age and across situations: genetic similarity for a trait during change. n J Pers 57 (2):215-35. McClure, Samuel, Mark S Gilzenrat, and Ionatban D Cohen. 2006. "An exploration-exploitation model based on norepinepherine and dopamine activity. n Advances in neural information processing systems 18:867. McCrae, Robert R, and Paul T Costa Ir. 1997. "Personality trait structure as a human universal." American psychologist 52 (5):509. McLaughlin, K. A., N. A. Fox, C. H. Zeanah, M. A. Sheridan, P. Marshall, and C. A. Nelson. 2010. "Delayed maturation in brain electrical activity partially explains the association between early environmental deprivation and symptoms of attention-deficiUhyperactivity disorder. n Bioi Psychiatry 68 (4):329-36. doi: 10.1016/j .biopsych.20l0.04.005. McLaughlin, K. A., M. A. Sheridan, F. Tibu, N. A. Fox, C. H. Zeanah, and C. A. Nelson, 3rd. 2015. "Causal effects of tbe early caregiving environment on development of stress response systems in children. n Proc Natl Acad Sci USA 1 1 2 (18):5637-42. doi: 10.1073/pnas. 1423363 l l2. McLaughlin, K. A., M. A. Sheridan, W. Winter, N. A. Fox, C. H. Zeanah, and C. A. Nelson. 2014. nWidespread reductions in cortical thickness following severe early-life deprivation: a neurodevelopmental pathway to attention-deficitihyperactivity disorder." Bioi Psychiatry 76 (8):62938. doi: 10.1016/j.biopsych.2013.08.0l6. Mehrabian, Albert. 1995. "Distinguishing depression and trait anxiety in telTIlS of basic dimensions of temperament. n Imagination, Cognition andPersonality 15 (2):133-143. Mesulam, MM, and RS Larry. 2009. nAcety1choline neurotransmission in CNS." Encyclopedia ofNeuroscience: 1-4. Miyazaki, Katsuhiko, Kayoko W Miyazaki, and Kenji Doya. 2012. "The role of serotonin in the regulation of patience and impulsivity. n Molecular neurobiology 45 (2):213-224.

216

Chapter Eight

Naragon-Gainey, K., Gallagher, M.W., & Brown, T.A. 2013. "Stable tttraif! variance of temperament as a predictor of the temporal course of depression and social phobia." Journal of Abnormal Psychology 122(3):61 1-623. doi: 10.1037/a0032997. Neale, M. C., and J. Stevenson. 1989. "Rater bias in the EASI temperament scales: a twin study." J Pers Soc Psychol 56 (3):446-55. Nery, Fabiano G, John P Hatch, Mark A Nicoletti, E Serap Monkul, Pablo Najt, Koji Matsuo, C Robert Cloninger, and Jair C Soares. 2009. ttTemperament and character traits in major depressive disorder: influence of mood state and recurrence of episodes. n Depression and anxiety 26 (4):382-388. Netter, P., Hennig, I. & Roed, I. 1996. nSerotonin and dopamine as mediators of sensation seeking behavior. n Neuropsychobiology 34: 155165. O'Connor, T. G., K. Deater-Deckard, D. Fulker, M. Rutter, and R. Plomin. 1998. tfGenotype-environment correlations in late childhood and early adolescence: antisocial behavioral problems and coercive parenting. n Dev Psychol 34 (5):970-81. Oades, R. D., Zimmermarm, B. & Eggers, C. . 1996. nConditioned blocking in patients with paranoid, nonparanoid psychosis or obsessive compulsive disorder: Association with symptoms, personality and monoamine metabolism." Psychiatric Research 30:369-390. Oakes, L. M. & Cohen, L. B. 1990. ttInfant perception of a causal event. tt Cognitive Development, 5:193-207. Okbay, A., B. M. Baselmans, J. E. De Neve, P. Turley, M. G. Nivard, M. A. et al. 2016. ttGenetic variants associated with subjective well-being, depressive symptoms, and neuroticism identified through genome­ wide analyses." Nat Genet 48 (6):624-33. doi: 10.1038/ng.3552. Ohnstead, M.C., Ouagazzal, A-M., Kieffer, B.L. . 2009. "Mu and delta opioid receptors oppositely regulate motor impulsivity in the signaled nose poke task." PLoS ONE 4(2):4410. doi: 1 0. 13711journal.pone.00044l O. Oswald L., Wong D., Zhou Y., Kumar A., Brasic J., Alexander M., Ye W., Kuwabara H., Hilton J., & Wand G. 2007. "Impulsivity and chronic stress are associated with amphetamine-induced striatal dopamine release." Neuroimage 36:153-166. Oyama, Susan, Paul Griffiths, and Russell D. Gray. 2001. Cycles of contingency : developmental systems and evolution, Life and mind. Cambridge, Mass.: MIT Press. Paavonen, V., K. Luoto, A. Lassila, E. Leinonen, and O. Kampman. 2018. ttTemperament clusters associate with anxiety disorder comorbidity in

An Overlap between Mental Abilities and Temperament Traits

2 17

depression." Journal of Affective Disorders 236:252-258. doi: 10.1 016/j .jad.2018.04.084. Pelissolo, A, and E Corruble. 2001. n[Personality factors in depressive disorders: contribution of tbe psychobiologic model developed by Cloninger] ." L'Encephale 28 (4):363-373. Peretsky, D., K. Wexler, and V. Fromkin. 1997. nAcquiring language.n Science 276 (5316) :1 177; autbor reply 1 1 80-1, 1276. Pemer, J. 1991. Understanding the representational mind. Cambridge: MIT Press. Persky, H. 1975. nAdrenocortical function and anxiety.n Psychoneuro­ endocrimology 1:37-44. Pessoa, Luiz. 2017. nA Network Model of the Emotional Brainn. Trends in Cognitive Science. 21(5): 357-371. doi: 10.1016/j .tics.2017.03.002 Pfau, M. L., and S. J. Russo. 2015. "Peripheral and Central Mechanisms of Stress Resilience." Neurobiology ofStress 1 :66-79. doi: 10.1 016/j .ynstr.2014.09.004. Pfefferbaum, A., D. H. Matbalon, E. V. Sullivan, J. M. Rawles, R. B. Zipursky, and K. O. Lim. 1994. nA quantitative magnetic resonance imaging study of changes in brain morphology from infancy to late adulthood." Arch Neurol 51 (9):874-87. Plomin, R. 1986. nMultivariate analysis and development behavioral genetics: developmental change as well as continuity.n Behav Genet 16 (1):25-43. Plomin, R. 2013. nCommentary: missing heritability, polygenic scores, and gene-environment correlation." J Child Psychol Psychiatry 54 (10):1 147-9. doi: 10. 1 1 1 1/jcpp.12128. Plomin, R., and S. von Stumm. 2018. !The new genetics of intelligence.n Nat Rev Genet 19 (3): 148-159. doi: 10.1038/nrg.2017.104. Posner, M. I., and M. K. Rotbbart. 2018. "Temperament and brain networks of attention." Philos Trans R Soc Land B Bioi Sci 373 (1744). doi: 10.1098/rstb.2017.0254. Posner, M. I., M. K. Rothbart, B. E. Sheese, and P. Voelker. 2014. nDeveloping Attention: Behavioral and Brain Mechanisms. n Adv Neurosci {Hindawi} 2014:405094. doi: 10. 1 155/2014/405094. Posner, M. I., M. K. Rothbart, and P. Voelker. 2016. "Developing brain networks of attention." Current Opinions in Pediatrics 28 (6):720-724. doi: 10.1 0971MOP.0000000000000413. Pribram, K. H. & Luria, A. R. 1973. Psychophysiology of the frontal lobes. New York: Academic Press.

218

Chapter Eight

Raven, I. 2000. 'The Raven's progressive matrices: change and stability over culture and time." Cognitive Psychology 41 (1):1-48. doi: 10. 1006/cogp.1999.0735. Raven, John. 1984. Competence in modern society : its identification, development and release. London: H.K. Lewis. Rey, Heman G, Sergio E Lew, and B Silvano Zanutto. 2007. ttDopamine and norepinephrine modulation of cortical and subcortical dynamics during visuomotor learning. n In Monoaminergic modulation of cortical excitability, 247-260. Springer. Robbins, Trevor W. 1997. tfArousal systems and attentional processes.!! Biological psychology 45 (1):57-71. Robbins, Trevor W, and Barry J Everitt. 1995. "Arousal systems and attention. n Robbins, Trevor W. 2010. ttFrom behavior to cognition: Functions of mesostriatal, mesolimbic, and mesocortical dopamine systems. n Dopamine handbook:203-14. Robbins, Trevor W, and Angela C Roberts. 2007. "Differential regulation of fronto-executive function by the monoamines and acetylcholine. n Cerebral Cortex 17 (supp1 1):i151-i160. Robinson, Emma SJ, Jeffrey W Dalley, David EH Theobald, Jeffrey C Glennon, Marie A Pezze, Emily R Murphy, and Trevor W Robbins. 2008. "Opposing roles for 5-HT2A and 5-HT2C receptors in the nucleus accumbens on inhibitory response control in the 5-choice serial reaction time task." Neuropsychopharmacology 33 (10):23982406. Rosenblitt J.C., Soler, H., Johnson, S.E., Quadagno, D.M. 2001. ttSensation seeking and hOlTIlones in men and women: exploring the link. " Hormones and Behavior 40(3):396-402. Rothbart, M. K., and M. I. Posner. 2005. ttGenes and experience in the development of executive attention and effortful control. tt New Dir ChildAdolesc Dev (109):101-8. Rothbart, Mary K, Stephan A Ahadi, and David E Evans. 2000. ttTemperament and personality: origins and outcomes. tt Journal of personality and social psychology 78 (1): 122. Rothbart, Mary K1evjord. 1988. "Temperament and the development of inhibited approach." Child Development 1241-1250. Rovee-Collier, C. 1987. ttLearning and memory in infancy.tt In Handbook olin/ant development (2nd ed), edited by J. D. Osofsky, 98-148. New York: Wiley. Rueda, M. R., M. K. Rothbart, B. D. McCandliss, L. Saccomanno, and M. I. Posner. 2005. ttTraining, maturation, and genetic influences on the

An Overlap between Mental Abilities and Temperament Traits

219

development o f executive attention." Proc Natl Acad Sci U S A 102 (41): 14931-6. doi: 10.1 073/pnas.0506897l 02. Rusalov, VM 1989. nObject-related and communicative aspects of human temperament: A new questionnaire of the structure of temperament. n Personality and individual differences 10 (8): 8 17-827. Rusalov, VM 1997. Oprosnik formal'no-dynamicheskih svoystv individual'nosti: Rukovodstvo [Questionnaire of formal-dynamical properties of individuality: Manual]. Moscow: Russian Academy of Sciences. IPAN Press. Rusalov, VM 2004. Formal-dynamical properties of individual (Temperament). Short theory and methods of measurement for various age groups. Moscow: Russian Academy of Sciences, IPAN. Rusalov, V. 2018. nFunctional systems theory and the activity-specific approach in psychological taxonomies." Philos Trans R Soc Lond B Bioi Sci 373 (1744). doi: 10.1098/rstb.20 l 7.0l66. Rusalov, VM and SV Kalashnikov. 1988. "On tbe correlation of psychic plasticity witb tbe integral factors of human brain bioelectrical activity." Individual Psychological Differences and Human Brain Bioelectrical Activity: 5-55. Rusalov, VM and ER Naumova. 1999. nOn the relationship between general abilities and "intellectual" scales of temperament. n PsychoiogicalJournal 20 (1):70-77. Rusalov, VM MN Rusalova, and IG Kalashnikova. 1993. "Bioelectrical Activity of tbe Brain in Subjects with Different Types of Temperament." Zh. Vyssh. Nervn. Deyat43 (3):530. Rusalov, VM MN Rusalova, and EV Strel'nikova. 2000. nTemperament of man and peculiarities of choice between tbe probability of goal achievement and its value. n Journal of Higher Nervous Activity 50:388-398. Rusalov, V.M., & Trofimova LN. 2007. Structure o/temperament and its measurement. Toronto: PSP: Psychological Services Press. Salarirad, S., R. T. Staff, H. C. Fox, L J. Deary, L. Whalley, and A. D. Murray. 201 1 . "Childhood intelligence and brain white matter hyperintensities predict fluid intelligence age 78-81 years: a 1921 Aberdeen birtb cohort study." Age Ageing 40 (5):562-7. doi: 10.1093/ageing/afr065. Sallis, H., G. Davey Smith, and M. R. Munafo. 2018. "Genetics of biologically based psychological differences." Philos Trans R Soc Lond B Bioi Sci 373 (1744). doi: 10.1098/rstb.20l7.0l62. .

.

.

,

,

,

,

220

Chapter Eight

Saper, Clifford B, Thomas C Chou, and Thomas E Scammell. 2001. "The sleep switch: hypothalamic control of sleep and wakefulness. n Trends in neurosciences 24 (12):726-73 1 . Sarter, Martin, Ben Givens, and John P Bruno. 2001. ttThe cognitive neuroscience of sustained attention: where top-do\Vll meets bottom­ up." Brain research reviews 35 (2): 146-160. Saudino, K. I. 2005. tfBehavioral genetics and child temperament. n J Dev Behav Pediatr 26 (3):214-23. Saudino, K. J., S. S. Cherny, and R. Plomin. 2000. "Parent ratings of temperament in twins: explaining the 'too low' DZ correlations. n Twin Res 3 (4):224-33. Savage, I. E., P. R. Jansen, S. Stringer, K. Watanabe, I. Bryois, C. A. et aI, 2018. tfGenome-wide association meta-analysis in 269,867 individuals identifies new genetic and functional links to intelligence. n Nat Genet 50 (7):912-919. doi: 10.1038/s41588-01 8-0152-6. Schmitt, J. E., L. T. Eyler, J. N. Giedd, W. S. Kremen, K. S. Kendler, and M. C. Neale. 2007. "Review of twin and family studies on neuroanatomic phenotypes and typical neurodevelopment. n Twin Res Hum Genet 10 (5):683-94. doi: 10.1375/twin. l0.5.683. Schmitt, J. E., G. L. Wallace, M. A. Rosentlial, E. A. Molloy, S. Ordaz, R. Lenroot, L. S. Clasen, J. D. Blumenthal, K. S. Kendler, M. C. Neale, and J. N. Giedd. 2007. !fA multivariate analysis of neuroanatomic relationships III a genetically informative pediatric sample. n Neuroimage 35 (1):70-82. doi: 10.1016/j.neuroimage.2006.04.232. Schoonover, Carl E. 2010. Portraits of the mind : visualizing the brain from antiquity to the 21st century. New York: Abrams. Schwartz, C. E., P. S. Kunwar, D. N. Greve, L. R. Moran, I. C. Viner, I. M. Covino, J. Kagan, S. E. Stewart, N. C. Snidinan, M. G. Vangel, and S. R. Wallace. 2010. "Structural differences in adult orbital and ventromedial prefrontal cortex predicted by infant temperament at 4 months of age." Arch Gen Psychiatry 67 (1):78-84. doi: 10.10011archgenpsychiatry.2009. 1 7 1 . Sealfon, Stuart C , and C Warren Olanow. 2000. ttDopamine receptors: from structure to behavior. n Trends in neurosciences 23: S34-S40. Seamans, Jeremy K, and Trevor W Robbins. 2010. ttDopamine modulation of the prefrontal cortex and cognitive function. n In The dopamine receptors, 373-398. Springer. Shabani, Sara, Mohsen Dehghani, Mehdi Hedayati, and Omid Rezaei. 201 1 . ttRelationship of serum serotonin and salivary cortisol with sensation seeking." International Journal of Psychophysiology 8 1 (3):225-229.

An Overlap between Mental Abilities and Temperament Traits

221

Shelley-Tremblay, John F, and Lee A Rosen. 1996. "Attention deficit hyperactivity disorder: An evolutionary perspective. n The Journal of Genetic Psychology 157 (4):443-453. Sheridan, M. A., N. A. Fox, C. H. Zeanah, K. A. McLaughlin, and C. A. Nelson, 3rd. 2012. nVariation in neural development as a result of exposure to institutionalization early in childhood. n Proc Natl Acad Sci USA 109 (32) :12927-32. doi: 10.1073/pnas.1200041 109. Sheridan, M. A., K. A. McLaughlin, W. Winter, N. Fox, C. Zeanah, and C. A. Nelson. 2018. nEarly deprivation disruption of associative learning is a developmental pathway to depression and social problems. n Nat Commun 9 (1):2216. doi: 10.1038/s41467-018-04381-8. Shirtcliff, E.A., Vitacco, M.J., Graf, A.R., Gostisha, A.I., Merz, J.L., & Zahn-Waxler, C. . 2009. "Neurobiology of empathy and callousness: implications for the development of antisocial behavior. n Behavioral Sciences and Law 27:137-171. Smith Watts, A. K., N. P. Friedman, R. P. Corley, J. K. Hewitt, L. K. Hink, J. L. Robinson, and S. H. Rhee. 2018. "A Longitudinal and Multidimensional Examination of the Associations Between Temperament and Self-Restraint During Toddlerhood." Child Dev. doi: 10. 1 1 1 1/cdev. 13173. Smoller, J. W., L. H. Yamaki, J. A. Fagerness, J. BiedelTIlan, S. Racette, N. M. et al. 2005. "The corticotropin-releasing hormone gene and behavioral inhibition in children at risk for panic disorder. n BioI Psychiatry 57 (12):1485-92. doi: 10.1016/j .biopsych.2005.02.018 . Sobel, D. M., & Kirkham, N.Z. . 2006. "Blickets and babies: The development of causal reasonmg in toddlers and infants.n Developmental Psychology 42: 1 103-1 1 15. Spelke, E. S., Breinlinger, K., Macomber, J., & Jacobson, K. 1992. "Origins of knowledge. " Psychological Review 99:605-632. Stuss, Donald T, and Robert T Knight. 2002. Principles offrontal lobe function: Oxford University Press. Sulis, W. (2009) "Collective intelligence: Observations and models." In S. Guastello, M. Koopmans, D. Pincus (Eds.) Chaos and complexity in psychology. Cambridge University Press: Cambridge. pp. 41-72. Sulis, William. 2018. nAssessing the continuum between temperament and affective illness: psychiatric and mathematical perspectives. n Phi/os Trans R Soc LandB Bioi Sci 373 (1744). doi: 10.1098/rstb.2017.0168. Sulis, William, and Irina Trofimova. 2001. Nonlinear dynamics in the life and social sciences, NATO advanced science institutes series Series A, Life sciences, . Amsterdam: [OS Press : published in cooperation with NATO Scientific Affairs Division.

222

Chapter Eight

Szechtman, Henry, William Sulis, and David Eilam. 1998. "Quinpirole induces compulsive checking behavior in rats: a potential animal model of obsessive-compulsive disorder (OeD). n Behavioral neuroscience 1 1 2 (6): 1475. Takahashi, Masakatsu, et al. 1990. tfFurther evidence for the implication of a. KAPPA.-opioid receptor mechanism in the production of psychological stress-induced analgesia.!! The Japanese Journal of Pharmacology 53 (4):487-494. Tamakawa, Yuichi, Akihiro Karashima, Yoshimasa Koyama, Norihiro Katayama, and Mitsuyuki Nakao. 2006. "A quartet neural system model orchestrating sleep and wakefulness mechanisms. n Journal of neurophysiology 95 (4):2055-2069. Tanaka, Masatoshi, Masami Yoshida, Hiroyuki Emoto, and Hideo Ishii. 2000. "Noradrenaline systems in the hypothalamus, amygdala and locus coeruleus are involved in the provocation of anxiety: basic studies." Europeanjournal ofpharmacology 405 (1):397-406. Thomas, Alexander, and Stella Chess. 1977. Temperament and development: BrunnerlMazel. Tibu, F., M. A. Sheridan, K. A. McLaughlin, C. A. Nelson, N. A. Fox, and C. H. Zeanah. 2016. "Disruptions of working memory and inhibition mediate the association between exposure to institutionalization and symptoms of attention deficit hyperactivity disorder." Psychol Med 46 (3):529-41 . doi: 10.1017/S0033291715002020. Trofimova, Irina. 1996. nlndividual differences from the point of view of evolutional-synergic approach." Voprosy Psykhologii (1):72-84. Trofimova, Irina. 2009. nExploration of the benefits of an activity-specific test of temperament." Psychological Reports 105:643-658. Trofimova, Irina. 2010a. nExploration of the activity-specific model of temperament in four cultures. n International Journal of Psychology and Psychological Therapy 10-1 :79-95. Trofimova, Ira. 2014. nObserver bias: an interaction of temperament traits with biases in the semantic perception of lexical material. n PloS ONE 9 (1):e85677. Trofimova, Irina. 2016a. !The interlocking between functional aspects of activities and a neurochemical model of adult temperament. n In: Arnold, M. (Ed.) Temperaments: Individual differences, social and environmental influences and impact on quality of life. New York, NY: Nova Science Publishers. Trofimova, Irina. 2016b. nPhenomena Of Functional Differentiation And Fractal Functionality." International Journal of Design & Nature and Ecodynamics 1 1 (4):508-521.

An Overlap between Mental Abilities and Temperament Traits

223

Trofimova, Irina. 2017. nFunctional Constructivism: In Search of FOlTIlal Descriptors." Nonlinear Dynamics Psychol Life Sci 21 (4):441-474. TrofImova, Irina. 2018. "Functionality versus dimensionality in psychological taxonomies, and a puzzle of emotional valence. n Philos Trans R Soc Lond B Bioi Sci 373 (1744). doi: 10.1098/rstb.2017.0167. Trofimova, Irina, and Julie Christiansen. 2016. nCoupling of Temperament witb Mental Illness in Four Age Groups." Psychological reports 1 1 8 (2):387-412. Trofimova, Irina, and Trevor W Robbins. 2016. nTemperament and arousal systems: A new synthesis of differential psychology and functional neurochemistry. n Neuroscience & Biobehavioral Reviews 64:382-402. Trofimova, Irina, Trevor W. Robbins, William H. Sulis, and Jana Uher. 2018. "Taxonomies of psychological individual differences: biological perspectives on millennia-long challenges.n Philos Trans R Soc Land B Bioi Sci 373 (1744). doi: 10.1098/rstb.2017.0152. Trofimova, Irina N, and William Sulis. 2010a. nAn investigation of temperament in adults with Comorbid depression and anxiety. n Advances in Bioscience and Biotechnology 1 (3):190-199. doi: 1 04236/abb201 013027. Trofimova, Irina, and William Sulis. 2010b. nQuestioning the "general arousal" models." Open Behavioral Science and Psychology, 4: 1-8. Trofimova, Irina, and William Sulis. 201 1 . nls temperament activity­ specific? Validation of the Structure of Temperament Questionnaire ­ Compact (STQ-77)." International Journal of Psychology and Psychological Therapy 1 1/3:389-400. Trofimova, Irina, and William Sulis. 2016a. nBenefits of Distinguishing between Physical and Social-Verbal Aspects of Behavior: An Example of Generalized Anxiety." Frontiers in psychology 7:338. doi: 10.3389/fpsyg.2016.00338. TrofImova, Irina, and William Sulis. 2016c. "A Study of tbe Coupling of FET Temperament Traits with Major Depression. n Frontiers in Psychology 7 : 1 848. doi: doi.org/10.3389/fpsyg.2016.01848. Trofimova, I., and W. Sulis. 2018. !There is more to mental illness than negative affect: comprehensive temperament profiles in depression and generalized anxiety." BMC Psychiatry 1 8 (1):125. doi: 1 0.11 86/s 12888-018- 1695-x. Troller-Renfree, S., K. A. McLaughlin, M. A. Sheridan, C. A. Nelson, C. H. Zeanah, and N. A. Fox. 2017. "The benefIcial effects of a positive attention bias amongst children with a history of psychosocial deprivation." Biological Psychology 122: 1 10-120. doi:

224

Chapter Eight

10.1 016/j .biopsycho.2016.04.008. Uher, J., I. Trofimova, W. Sulis, P. Netter, L. Pessoa, M. I. Posner, M. K. Rothbart, V. Rusalov, I. T. Peterson, and L. A. Schmidt. 2018. tfDiversity in action: exchange of perspectives and reflections on taxonomies of individual differences. n Phi/os Trans R Soc Land B BioI Sci 373 (1744). doi: 10.1098/rstb.2017.0172. Waldhoer, M. Bartlett, S.B. & 'Whistler, I.E. . 2014. nOpioid receptors.!! AnnualReview o/Biochemistry 73:953-990 doi: 10.1 146/annurev.biochem.73.01 1303.073940. Walker, SC, TW Robbins, and AC Roberts. 2009. "Differential contributions of dopamine and serotonin to orbitofrontal cortex function in the marmoset." Cerebral Cortex 19 (4):889-898. Watson, I. S. & Ramey, C.T . . 1972. ttReactions to response contingent stimulation early in infancy." Merrill-Palmer Quarterly 18:219-227. Way, Baldwin M, Shelley E Taylor, and Naomi I Eisenberger. 2009. nVariation in the I-L- opioid receptor gene (OPRMl) is associated with dispositional and neural sensitivity to social rejection. n Proceedings of the NationalAcademy o/Sciences 106 (35):15079-15084. Weinstock, Lauren M, and Mark A 'Whisman. 2006. nNeuroticism as a common feature of the depressive and anxiety disorders: a test of the revised integrative hierarchical model in a national sample. n Journal of abnormal psychology 1 1 5 (1):68. Whitely, M., M. Raven, S. Timimi, I. Iureidini, I. Phillimore, I. Leo, J. Moncrieff, and P. Landman. 2018. "Attention deficit hyperactivity disorder late birthdate effect common in both high and low prescribing international jurisdictions: systematic review. n J Child Psychol Psychiatry. doi: 10. 1 1 1 1/jcpp.12991. Wilson, David Sloan, and Edward 0 Wilson. 2008. nEvolutionn for the Good of the Groupn: The process knO\vn as group selection was once accepted unthinkingly, then was widely discredited; it's time for a more discriminating assessment." American Scientist 96 (5):380-389. Winstanley, Catharine A, Yogita Chudasarna, Jeffrey W Dalley, David EH Theobald, Jeffrey C Glennon, and Trevor W Robbins. 2003. "Intra­ prefrontal 8-0H-DPAT and M I00907 improve visuospatial attention and decrease impulsivity on the five-choice serial reaction time task in rats." Psychopharmacology 167 (3):304-314. Wittmann, Walter, Eduard Schunk, Iris Rosskothen, Stefano Gaburro, Nicolas Singewald, Herbert Herzog, and Christoph Schwarzer. 2009. nProdynorphin-derived peptides are critical modulators of anxiety and regulate neurochemistry and corticosterone. n Neuropsychopharmacology 34 (3):775-785.

An Overlap between Mental Abilities and Temperament Traits

225

ZuckelTIlan, Marvin. 1994. Behavioral expressions and biosocial bases of sensation seeking: Cambridge university press. Zuckerman, Marvin. 2014. Sensation Seeking (Psychology Revivals): Beyond the Optimal Level ofArousal: Psychology Press.

CHAPTER NINE THEORETICAL CHALLENGES FOR DIFFERENTIATING GENERAL AND SPECIFIC COGNITIVE ABILITIES HARRISON J. KELL

For over 100 years the relationship between general and specific cognitive abilities has been debated (Kell & Lang, 2017). This debate has been vigorous but largely focused on the relative practical significance of the two classes of abilities. Less consideration has been given to conceptual issues surrounding the distinction between general and specific mental skills. In this chapter some of those conceptual matters are explored. I present and discuss evidence and theory suggesting that the covariances investigators working in the factor-analytic tradition put a premium on to make their inferences are consistent with the existence of discrete, underlying general and specific cognitive abilities - but also consistent with other interpretations, some of which make it difficult or impossible to differentiate these two classes of constructs. The perspectives outlined are not uniquely applicable to cognitive abilities and, with slight modifications, can be made to address other constructs in differential psychology that differ in their breadth of influence (e.g., interests, personality traits).

Abilities as Hypothetical Entities This chapter treats cognitive abilities as hypothetical constructs or source traits in that they are (currently) unobserved, unitary entities that are presumed to cause the differences in perfOlmance on the cognitive tests from which their existence is inlerred (cf. English & English, 1958). These abilities are general (Bakan, 1955) or homologous (Hamaker, Dolan, & Molenaar, 2005) entities in that they are presumed to exist within all individuals in the population under study. The term factor is

Theoretical Challenges

227

treated as a synonym for the hypothetical construct in question (e.g., eoan, 1964; Royce, 1963). It is important to be explicit about terminology because the terms abilities, constructs, andfactors can be defined and used in different ways. Some, for example, use the term "factor" to refer to an abstraction from, or inductive summary of, shared variance across assessments' scores, rather than the imputed cause of that shared variance (e.g., Vernon, 1979). Others treat constructs as "convenient fictions" that are useful for achieving investigators' ends but are not assumed to actually exist (Ball, 201 1 ; Maxwell, 1962). The point is not to criticize these (legitimate) alternative usages but to be clear about what these terms mean in the context of this chapter. Some may object that this "strong" characterization of abilities-as­ entities is an inappropriate reification that distorts how they are treated in the psychological literature. However, Spearman's (1904) original treatment of general mental ability (GMA) as "mental energy'" and a "fundamental function" that permeates all branches of intellectual activity is still frequently cited in contemporary treatments of cognitive abilities (e.g., Ree & Carretta, 2002). Others (e.g., Hom, 1998) have explicitly likened cognitive abilities to chemical elements - scientific entities with verified existential status. Similar treatments of hypothetical constructs as unobserved entities that exert causal influence are exemplified by some conceptualizations of personality traits (e.g., Loehlin & Bouchard, 2001) and vocational interests (e.g., Rounds, 1995). The positing of theoretical entities as being responsible for patterns in data is not unique to psychology and is a scientific inference strategy has been employed for centuries (Ebel, 1974). Some entities ceased to be theoretical when they were observed directly (e.g., atom, electron, gene, gelTIl, neutrino). Other hypothetical entities' existential status has been disproven (e.g., caloric, luminiferous aether, miasma, phlogiston, Streptobacillus pellagrae, toothworm) while yet other theoretical constructs await verification or rejection (e.g., [chemical] element 120, dark matter, graviton, scrapie-causing prion, tachyon).

1 Hypothesizing about various types of mental and nervous energy was not an uncommon activity among those studying psychological topics in the 19th and early 20th centuries. Aside from Spearman, other notable researchers and theorists who speculated about the existence of various types of psychological energy dming this time period include Sigmund Freud ("Q"), Francis Galton, C. G. JlUlg, Herbert Spencer, and William McDougall ("nemin") (McKinnon, 2010; Norton, 1979; Oates, 1930).

228

Chapter Nine

Positive Manifolds to Common Causes The almost universally observed positive intercorrelation of scores on cognitive ability assessments has been the cornerstone of psychometric intelligence testing and factor-analytic theories of abilities for over 100 years (Jensen, 1998; van der Maas, Dolan, Grasman, Wicherts, Huizenga, & Raijmakers, 2006). Thurstone (1934) underscored the importance of the phenomenon: "The best evidence for a conspicuous and central intellective factor is that if you make a list of stunts, as varied as you please, which all satisfy the common sense criterion that the subjects must be smart, clever, intelligent, to do the stunts well, and that good performance does not depend primarily upon muscular strength or skill or upon other non-intellectual powers, then the inter-stunt correlations will all be positive. It is quite difficult to find a pair of stunts, both of which call for what would be called intelligence, as judged by common sense, which have a negative correlation. This is really all that is necessary to prove that what is generally called intelligence can be regarded as a factor that is conspicuously common to a very wide variety of activities. (pp. 3-4)."

Spearman (1904) used this positive manifold (Thurstone, 1935) to justify the existence of his postulated highly general cause of complex mental activity and perfOlmance on diverse cognitive tests: GMA, the general factor, or g (cf. Burt, 1949). Subsequently, the hypothesis that a single factor could account for this positive manifold was disproven (Horn & McArdle, 2007), leading to the introduction of factors of intermediate generality between g and test-specific factors (Spearman, 1927; Thurstone, 1938a). The presence of these narrower factors was inferred from higher correlations among scores within groups of tests with similar content than between groups of tests with dissimilar content: Narrower, but nonetheless still positive, manifolds. Inferring the existence of underlying entities from positive manifolds (regardless of their generality) is an application of the prniciple of the common cause (K. Johnson, 2016), first formalized by Reichenbach (1956) and pervasively employed in factor analysis, even if not acknowledged explicitly (Haig & Evers, 2016). The principle has two major components (Sober, 1988): When two (or more) variables are correlated but there is no discernible causal relationship between them: 1) the correlation can be explained by a third variable, and 2) the correlation is conditional on the third variable such that when the influence of this variable is controlled for the correlation between the two variables will be eliminated. Hypothesizing that unobserved factors cause the covariances

Theoretical Challenges

229

among test scores that are treated as indicators of broad and narrow abilities is an application of the principle of the common cause. In the case of cognitive abilities research, this inferential process is an instance of the more general, long-standing scientific tactic of inducing the existence of theoretical entities from recurring regularities in observational data (Rozeboom, 1961). The pervasive presence of positive correlations among cognitive tests of similar and disparate content makes the inference toward common causes in ability research inevitable: When test scores are positively correlated and those correlations are substantial all loadings on a general factor will be positive (Borg, 2018; van Bork, Epskamp, Rbemtulla, Borsboom, & van der Maas, 2017). The induction of the presence of a general factor is a restatement of the positive manifold and the induction of specific factors is a restatement of the positive manifold among scores on subgroups of tests (Hom & McArdle, 2007). Nonetbeless, the relationship among general and specific factors depends on the investigator's methodology and the underlying theory informing it; different factor-analytic models posit different structural relations among abilities of varying breadth (Kell & Lang, 2017; Lang, Kersting, Hiilsheger, & Lang, 2010). In modem structural equation modeling (SEM) approaches a general factor is usually specified a priori as loading on all tests (Gustafsson & Snow, 1997) while specific factors load on subgroups of tests, usually those that exhibit content similarity (Benson et aI., 2018). Older bifactor (or nested-factor) approaches involved first extracting a general factor, followed by the extraction of orthogonal content-aligned factors to account for the remaining covariance among test scores (Gustafsson & Snow, 1997). Higher-order models begin with tbe extraction - or specification ­ of specific abilities. Scores on these latent specific abilities themselves almost inevitably exhibit positive manifold(s), leading to the inference of a more general common cause posited to be responsible for these covariances (Thurstone, 1947). This process of inferring, or specifying, ever more general factors based on the observation of positive intercorrelations among latent variables often continues until a single, highly general construct is ultimately inferred, usually identified as GMA (Hom & McArdle, 2007; Loevinger, 1940; van Bork et aI., 2017). For example, Gustafsson's (1984) unified model and the Cattell-Horn-Carroll model (McGrew, 2009) feature three levels of latent abilities tbat become progressively broader in their range of influence and culminate in a general factor, while the g-VPR model CW. Johnson & Bouchard, 2005) consists of four ability strata.

230

Chapter Nine

The process of inferring ever more general factors is not limited to the cognitive ability domain, as positive intercorrelations are ubiquitous in psychology (Meehl, 1990). For example, highly general factors have been found or specified among measures of psychopathology (Caspi et al. 2014), personality traits (Musek, 2007), job performance (Viswesvaran & Ones, 2000), self-concept (Judge, Erez, Bono, & Thoresen, 2002), physical dimensions (Burt, 1947), commitment to work (Cooper-Hakim & Viswesvaran, 2005), and vocational interests (Tracey, 2012) - although in many of these subdisciplines there is active debate about how to properly interpret those factors. Moreover, many of these non-ability general factors have been found to correlate positively with GMA - leading to the induction of some even more basic common cause, such as a general fitness factor (prokosch, Yeo, & Miller, 2005) 2

The Challenge of Underdetermination Under some circumstances inferring a common cause from a positive correlation is warranted (Gould, 1996) but the general principle's tenability is also the subject of debate and it does not hold across all circumstances (Amtzenius, 2010). For instance, in medicine Saint's Triad denotes three disorders (hiatal hernia, gallbladder disease, diverticulosis) that tend to co-occur but have no known underlying shared cause (Hilliard, Weinberger, Tierney, Midthun, & Saint, 2004). Psychologists working in the factor analytic tradition have cautioned that positive manifolds are not sufficient for inferring a unitary underlying cause (e.g., Hom & Cattell, 1966). The factors that are inferred from general and specific positive manifolds are necessarily hypothetical and the theories they are embedded in are weak - as are nearly all psychological theories (Eysenck, 1997) and largely do not precisely identify them with structures, properties, processes, or other types of observable entities that lie outside the realm of factor analysis (Eysenck, 1997; Jensen, 1987). Consequently the threat of underdetelTIlination - that empirical findings are compatible with multiple theories and hypotheses (Quine, 1975) - is especially acute. Underdetermination is not a distant philosophical concern, as 2 These inferences are not limited to basic research or theory. In applied settings, when positive correlations are observed between predictors (e.g., cognitive tests, interviews) and criteria (e.g., grades, job performance), their presence is sometimes hypothesized to be due to the influence of some shared lUlderlying cause that affects both those predictors and criteria (e.g., Binning & Barrett, 1989; Ryans, 1939).

Theoretical Challenges

23 1

illustrated by two examples from the history of science. In the early 19th century it was knO\vn that predictions made from Newton's universal laws about Uranus' orbit were inconsistent with astronomers' observations. Two major hypotheses were put forward to explain these observations: Newton's laws were not truly universal or Uranus' orbit was perturbed by a (then) unknown planet (Bamford, 1996; Pannekoek, 1953). Astronomers resolved the controversy when they discovered Neptune in 1 846. Influenced by this discovery, scientists took the same approach to explain the irregularities in Mercury's orbit and proposed they were the consequence of the hypothetical planet Vulcan. This postulated planet could never be definitively detected, and the oddities of Mercury's orbit could only be explained by Einstein's theory of relativity - which showed that Newton's laws are not universal after all (Fontemose, 1973; Levenson, 2016). In exploratory factor analysis, underdetelTIlination manifests in common factor indeterminacy: There are an infinite number of solutions for the observed data (K. Johnson, 2016; Mulaik, 1987). A potential remedy for this indetelTIlinacy is more strictly delimiting the content domain that the results of factor analysis are hypothesized to align with (McDonald & Mulaik, 1979). This strategy is problematic when factors are treated as causal entities, however, as the very concept of construct validity was introduced partially as a consequence of the fact that many psychological constructs carmot be defined solely in terms of content (American Psychological Association, 1954; Cronbach & Meehl, 1955). This strategy is particularly problematic when applied to GMA, which has been explicitly claimed not to be definable in telTIlS of content (Jensen, 1998), leading it to be defined simply as the construct that influences perfolTIlance on all cognitive tasks - a restatement of the positive manifold (Borg, 2018; Horn & McArdle, 2007). In SEM, underdetermination manifests in the form of data being consistent with many different statistical models (Cliff, 1983; Tomarken & Waller, 2003). In particular, under some circumstances bifactor models and higher-order models fit cognitive test data equally well and can be transformed into the other (Murray & W. Johnson, 2013) - despite the fact that general and specific abilities are treated very differently in the two models. In bifactor models, specific abilities are independent of GMA and thus wholly conceptually distinct. In higher-order models, however, GMA is treated as exerting causal influence on test perfolTIlance through its impact on specific abilities - muddying the waters and making it theoretically difficult to identify "where GMA ends and specifics begin". Not only are these two classes of models largely equivalent, statistically

232

Chapter Nine

they can also often be reformulated such that GMA is treated as a fOlmative construct (e.g., Major, Johnson, & Deary, 2012; see also Kovacs & Conway, 2016) an abstraction based on test-takers' specific abilities. In this scenario, there is no general ability that exists beyond specific abilities, only a summary of individuals' standings on those abilities. Further, even when g is treated as an epiphenomenon of specific abilities in factor-analytic models those specific factors carmot be assumed to exist as unitary entities by fiat; their presence is inferred based on content­ aligned positive manifolds, inductions from which are vulnerable to concerns about the applicability of the principle of the common cause and the possibility that those manifolds could be generated by many causal entities (i.e., underdetennination). In fact, cognitive test data are compatible with formulations that do not feature any distinct general ability constructs at all. One such fOlTImlation is the dynamic mutualism model (van der Maas et aI., 2006). This model posits that multiple processes (e.g., knowledge retrieval, working memory) underlie perfOlmance on cognitive assessments. Simulations show that even when these cognitive processes are specified as being initially independent, their mutual influence over time can result in positive manifolds and that, under some conditions, the mutualism and common factor models are statistically equivalent. Moreover, several studies have found that the dynamic mutualism account fits the data better than general ability models (Hofman, Kievit, Stevenson, Molenaar, Visser, & van der Maas, 2018; Kievit et al. 2017). The second approach, and the most enduring competition for g-centered models, is Godfrey Thomson's (1916, 1952) bonds or sampling model, which is perhaps even more radical than the dynamic mutualism model in that it does not feature discrete general or specific abilities. Instead, Thomson postulated that the human mind is undifferentiated in terms of flUlctionality and that when individuals engage with a cognitive test item it activates various hypothetical elements or connections ("bonds"): -

"The psychological meaning of all this is that if, when we attack some task, some test, om ability to solve it depends upon a large number of things genes we have inherited, pieces of information we have acquired, skills we have practiced, little habits of thought we have formed, all and SlUldry influences from past and present then the correlation coefficients between performances in tests will show exactly the same relationships with one another as they would have done had our ability depended on our possession of a small number of cornmon "factors" (plus specifics). This does not prove that we have no such 'factors'. But it does show that perhaps we haven't, that perhaps they are fictions possibly very useful fictions, but still fictions. (Thomson, 1952, p. 283)."

Theoretical Challenges

233

Different items sample different connections, and different arrangements among them, with more complex items activating a larger number of connections. In the bonds model, the ubiquitous positive correlation among scores on cognitive tests is due to the activation of overlapping samples of bonds, rather than a unitary mental function. For example, imagine three tests, one commonly taken to measure mathematical ability ("Math Test"), one verbal ability ("Verbal Test"), and visuospatial ability ("Space Test"). Assume, Math Test taps the mental processes Q, b, c, and d, Verbal Test taps the processes Q, e, j, and g, and Space Test taps processes b, e, h, and i. Scores on Math Test can correlate with scores on Verbal Test solely because they share process Q. Likewise, scores on Space Test can correlate with scores on Math Test solely because they share process b and scores on Verbal Test solely because they share process e. Moreover, imagine a fourth test - Short-TelTIl Memory - that elicits processes c, j, and h. Scores on this test will correlate positively with scores on the Verbal, Math, and Space assessments - without sharing a process common to all four measures. (See Hom and McArdle, 2007, p. 220, for the original presentation of this example.) Despite the complete theoretical incompatibility of the bonds and g-factor models, they fit test data equally well (Bartholomew, Allerhand, & Deary, 2013). The upshot of the preceding is that from the perspective of observed data the distinction between general and specific abilities may seem obvious, but that making those differentiations at the level of unobserved constructs is much more challenging. The statistical fit of those observed data vis-a-vis latent structure is of limited use in arbitrating between various models, many of which differ greatly in how the relationships among general and specific abilities are presented, ranging from them being wholly distinct (bifactor) to interrelated (bigher-order) to the distinction being dissolved completely (bonds).

The Challenge of Local Heterogeneity The positive manifolds from which general and specific abilities are inferred summarize the similarities of the orderings of respondents' perfolTIlances on the cognitive tests in question (Borsboom, 2015; Borsboom & Dolan, 2006). Ability constructs are presumed to be causally responsible for the differences among those individuals' orderings - that is, they operate at the between-person level of analysis (Borsboom, Mellenbergh, & van Heerden, 2003). Implicitly, this is often taken to suggest that all the individuals in a given test-taking sample possess the same mental entities and that, as a consequence, if the activity of those

234

Chapter Nine

entities was to vary for each individual over time than each individual's test-taking perfOlmance over time would differ as a function of the variance of those abilities (Borsboom, Kievit, Cervone, & Hood, 2009). Constructs that operate in the same way across between- and within­ person levels of analysis have been called locally homogenous (Borsboom et aI., 2003). Transposing constructs derived from the population-level to the level of individuals occurs in many areas of differential psychology, including the study of cognitive abilities (e.g., Thurstone, 1938b) and personality traits (McCrae & Costa, 2008). Inappropriately transferring an attribute from a whole to its constituents is a fallacy of illicit transference (Hurley, 2015). Generalizing from a population to an individual is often not possible (Adolf, SchuUlTIlan, Borkenau, Borsboom, & Dolan, 2014), however, and is appropriate only when stringent statistical assumptions can be met (Molenaar, 2004) - which they rarely are (Borsboom et aI., 2003). Consequently, the causes of differences between individuals' relative positions on a psychological assessment carmot be assumed to correspond to mental entities (or any other causes) that necessarily exist at the level of all the individuals in the test-taking sample. For example, when Borkenau and Ostendorf (1998) factor-analyzed the personality scores of a sample of respondents at the between-subjects level, the resulting dimensionality was suggestive of the Big Five traits. However, when variability in the personality scores was analyzed at the individual level over an extended period of time the results often did not fit the Big Five model. Moreover, different individuals' response patterns suggested the presence of differing numbers of traits, with some individuals' data being suggestive of fewer than five factors, others' being indicative of more than five factors, and the same items sometimes loading on different factors for different individuals. Although similarly intensive repeated measures studies of cognitive abilities do not appear to have been conducted, Borkenau and Ostendrof's (1998) findings are illustrative of the dangers of automatically generalizing from populations to individuals in the face of local heterogeneity. Indeed, even some strong supporters of factor analytic cognitive abilities research (e.g., Jensen, 1998; Thurstone, 1947) have remarked that between-person analyses can often shed only limited light on the causal influences that bring individuals to occupy their relative orderings on the cognitive tasks of interest. One reason for local heterogeneity is that cognitive test-takers can achieve their perfOlmance results in a wide variety of ways. For example, Johnson and colleagues (e.g., W. Johnson & Bouchard, 2005; W. Johnson & Gottesman, 2006) argued that labeling certain tests measures of

Theoretical Challenges

235

crystallized ability and otber tests measures of fluid ability is inappropriate because people differ in tbe extent to which they have been exposed to tbe types of content that comprise the tests they are taking and will rely differentially on knowledge versus reasoning to solve the problems they are presented with; perfOlmance on the same test may largely reflect crystallized ability for one individual but largely reflect fluid ability for another individual. Indeed, latent class analysis can be used to identify such qualitatively different groups in the same testing sample (Embretson, 2004). Accordingly, it has been noted that what a task measures carmot be determined solely by inspecting its content (Lohman & Lakin, 2011). Beyond knowledge versus reasoning, and consistent with non-factorial theories of abilities (e.g., Thomson, 1952), many investigations have demonstrated that individuals approach the same tasks using different strategies (Hunt, 1983; W. Johnson, 2018; Kyllonen, Lohman, & Snow, 1984; Mackintosh, 2011), making it difficult to unequivocally link perfOlmance on a test battery (from which general factors are inferred), or subgroups of tests (from which specific factors are inferred), to any particular ability (Johnson & Bouchard, 2004; W. Johnson & Gottesman, 2006). Further complicating matters, some (but not all) individuals will adopt different strategies for different tasks in the same test - and even shift their strategies while in the course of attempting to solve a single problem (Snow & Lohman, 1984). Consequently, separate models may have to be developed for not only single test-takers but also for different segments of single test-takers' performance, if strategy shifting is characteristic oftbose test-takers (Snow & Lohman, 1989). Local heterogeneity blurs the lines between specific and general abilities both between-individuals and, when strategy-shifting is the nonn rather than the exception, within-individuals. Nonetheless, the consistencies in the types of processes that individuals rely on in order to solve cognitive problems present an additional problem for strictly differentiating general and specific abilities. A long-studied (e.g., Binet, 1892; Galton, 1 880) dimension on which individuals have been found to differ is the degree to which tbey rely on mental imagery versus verbal­ analytic-logical strategies for encoding and solving tasks; individuals who rely primarily on mental imagery have been labeled "visualizers" and those who rely primarily on verbal content have been labeled "verbalizers" (Kozhevnikov, Kosslyn, & Shephard, 2005). When presented witb a visuospatial task, for example, visualizers will tend to picture the stimulus itself and break it dO\vn into more easily remembered components (figure decomposition) while verbalizers will tend to associate the stimulus with anotber, more easily remembered object (verbal labeling) (Lohman, 1996;

236

Chapter Nine

Snow & Lohman, 1984). These different strategies can be pervasive, with some individuals using mental imagery-based approaches to solve problems with verbal (Hunt, 1983) or mathematical content (Kozhevnikov et aI., 2005). Consistent preferences for using verbal versus visuospatial strategies complicates interpreting factor-analytic results in terms of general or specific abilities. For example, if a sample of test-takers consists of both verbalizers and visualizers, the verbal strategies will constitute the general factor of test perfOlmance for the fOlmer and visuospatial strategies will constitute the general factor of test perfOlmance for the latter. However, individuals can both explicitly be taught strategies for solving problems (Pezaris & Casey, 1991) and learn them in the process of attempting problems (Kyllonen et aI., 1984), meaning that verbalizers can adopt mental imagery-based strategies and visualizers can adopt analytic strategies - which could comprise the specific abilities factors in an undifferentiated group of test-takers. Thus, one could end up with a perverse (albeit oversimplified) situation where, in a single testing population, visuospatial items primarily measure a general verbal ability in one subgroup oftest-takers but a general mental imagery ability in another subgroup of test-takers, with the specific factor(s) consisting of learned visuospatial strategies in the verbalizer group and specific factors (to the extent they appear) consisting of task-specific visuospatial strategies among the visualizers. Additional challenges of interpretation are only added when considering the extent to which the strategies employed are preexisting (crystallized knowledge) or developed on-the-spot (fluid reasoning) and how much test-takers vary in their approaches across, and within, the tasks comprising the test. The fact that cognitive test-takers' relative positions can be multiply realized is another manifestation of underdetermination. Although the challenges it presents may be particularly difficult to surmount due to the fact that the subject matter in question (mental entities) cannot currently be directly observed, this state of affairs is not unique to cognitive abilities research and exists in many fields, both within and beyond psychology. Experimental psychologists have recognized that the exact mechanism within a treatment group that causes people's positions to change (or fail to change) on a dependent variable often cannot be identified and that identical end states among treated subjects can be brought about by the operation of multiple, yet different, mediating mechanisms (Bullock, Green, & Ha, 2010; Miller & Schwarz, 2018). In brain science, neuroplasticity allows (some) brains to reorganize in response to (some varieties of) injury or degeneration, meaning that the same neural

Theoretical Challenges

237

functions can be subserved by different brain regions or processes across different individuals at the same time or within the same individual at different times (Barrett, 2012). In medicine the zebra aphorism is invoked as a warning not to diagnose a patient with an exotic disease when his or her symptoms are also characteristic of one (or more) common diseases (Montgomery, 2006) while Hickam's Dictum serves as a warning against unwarranted parsimony ("Patients can have as many diseases as they damn well please") (Borden & Linklater, 2013). In genetics, the same phenotype can be realized by different genotypes (Fortuna, Zaman, Ofria, & Wagner, 2017; Yu et aI., 2016) and the same genotype associated with different phenotypes (V ogt, Huber, Thiemann, van den Boogaart, Schmitz, & Schubart, 2008). The challenges of underdetermination are embodied more abstractly in systems theory, where a final state that can be reached through different initial conditions and/or different pathways IS characterized as equifinal (Bertalanffy, 1950), and in SEM, where wedge models can be applied to instances of equifinality and fork models applied to instances where the same starting state leads to different end states (von Eye & Brandtstadter, 1998).

The Challenge of Levels of Analysis The multiple realization of individuals' cognitive perfonnance contrasts with the unique realization of many entities studied in other sciences. For instance, chemical elements - which cognitive abilities have been likened to (Guilford, 1985; Horn, 1998) - have a very specific material definition: "all atoms with the same number of protons in the atomic nucleus" (International Union of Pure and Applied Chemistry, 2014, p. 258). For example, oxygen is defined as an atom with eight protons and radon is defined as an atom with 86 protons (Newton, 2010). If an atom does not have eight protons it is not an oxygen atom and if it does not have 86 protons it is not a radon atom. This is not the case for cognitive abilities. If an individual repeatedly solves cognitive problems (inside or outside of an assessment setting) using his knowledge, his knowledge is a cognitive ability. If another individual repeatedly solves cognitive problems using her reasoning, her reasoning is a cognitive ability. Similarly, if one person uses Strategy 1 to solve most items on a test it is a general ability whereas if another person uses Strategy 1 to solve a subset of items on a test it is a specific ability. Thus, whereas chemical elements are defined in tenns of their structure and composition (cf. Meehl, 1978, 1993), cognitive abilities are defined in terms of their junction, the role they play vis-a-vis other mental entities. In defining

238

Chapter Nine

cognitive abilities in terms of "what they do" (cf. Fodor, 1968; Polger, 2012), the functional perspective can be interpreted as the flip side of Boring's (1923) operationist coin: Rather than cognitive abilities being "what intelligence tests test" they are the "mental entities that solve cognitive problems". A liberal view of this functionalist conceptualization suggests that what constitutes a cognitive ability for one person may not constitute a cognitive ability for another and that a cognitive ability constitutes whatever psychological entity allows a person to successfully complete cognitive tasks (Borsboom et al., 2009). Moreover, if the cognitive strategies that the same person employs over time shift, what constitutes a cognitive ability for the same individual may consist of different entities at different periods of time. By the same token, if a person applies a single type of strategy to a wide variety of problems at one point in time but this same strategy to a small number of problems at a second point in time it will be a general ability in the first case but a specific ability in the second. Treating abilities from a functional perspective may seem radical - and if taken to the extreme perhaps it is - but doing so is consistent with conceptualizing constructs in telTIlS of their role(s) within a theoretical network (Cronbach & Meehl, 1955; Meehl, 1978, 1993). That scientific entities can be described at various levels of analysis (e.g., composition, structure) beyond the functional one - how ability constructs are usually treated (knowingly or not) - suggests that cognitive skills can be treated at multiple levels of analysis as well. Many different analytic taxonomies exist in psychology and philosophy of science (e.g., Allport, 1958; Marr, 1982; Meehl, 1993; Wallace, 1967), and featuring widely diverging dimensions for describing different characteristics of scientific phenomena (e.g., cultural & historical roots, evolutionary adaptive value, objective meaning, subjective meaning; Boccignone & Cordeschi, 2015). However, all of these taxonomies have been influenced by Aristotle's (300 BCE1l996) "four causes", each of which constitutes a distinct way of answering the question of "why" an event has occurred (Robinson, 1995). Each cause can be construed as a different level of analysis. Given the historical influence of Aristotle's taxonomy, its simplicity relative to more recent efforts, and its previous application to psychological constructs (e.g., personality; Rychlak, 1981), it is an ideal approach for exploring the implications - and challenges - of attempting to separate general and specific abilities at different levels of analysis, as suggested by the following. The final cause is the end goal or purpose of a thing. In the case of mental abilities, the final cause (or, level of analysis) is the desire or

Theoretical Challenges

239

motivation to successfully perfOlTIl cognitive tasks (cf. Butz & Kutter, 2016). Differentiating general and specific abilities in terms of end goals is difficult, as presumably individuals do not differ in the ultimate purpose (i.e., solving problems) for which they deploy their general and specific abilities (whatever they may consist of). Aristotle's material cause concerns the physical composition of an entity. Given the mind's apparent lack of physical extension (James, 1 890), and investigators' (current) inability to directly observe it, any distinction drawn between general and specific mental abilities in telTIlS of their material substance will be precarious. Nonetheless, if we accept that visuospatial Imagery and analytic-verbal-logic constructions may constitute qualitatively different types of "mental stuff' it suggests at least the potential for discriminating general and specific abilities in telTIlS of their material composition. As described previously, it is well-established that people differ in the extent to which their cognitive strategies consist of imagistic and verbal content. Snow and Lohman (1984) further discussed the possibility of comparing and contrasting strategies in telTIlS of their routes, or the component operations (e.g., encoding, inferring, mapping) they are comprised of. Consequently, abilities could perhaps be grouped in terms of both the components they are assembled out of and the mental composition of those components (i.e., verbal vs. imagistic). If Person A consistently uses a route to solve a wide variety of problems it could be considered a general ability and if Person B consistently uses this same route to solve only specific subdomains of problems it could be considered a specific ability. Although describing abilities at this material level might allow for the conceptual differentiation of general and specific abilities, without infolTIlation that is not routinely provided during most cognitive testing sessions (e.g., retrospective reports) the two classes of abilities cannot be differentiated empirically or for practical purposes. The fOlTIlal cause pertains to the arrangement, shape, configuration, or form ofa thing. In addition to route differences, Snow and Lohman (1984) described sequence differences: The order in which the components of a mental strategy operate. Thus, two cognitive abilities could conceivably feature identical components that operate in identical modalities but that nonetheless are activated and operate in different orders. The same considerations given to comparing abilities in terms of their composition are also pertinent to comparing abilities in terms of their components' configurations. The efficient cause refers to the thing(s) apart from the entity in question that changes it or sets its movement in motion. In the case of mental abilities this might be treated as the goals, motivations, and desires

240

Chapter Nine

that cause individuals to deploy their cognitive skills to solve problems (making this level equivalent to the final level). This level of description could also be treated as corresponding to the neural substrate that subserves and generates those mental abilities. Problems arise, however, due to the fimctional interpretation of abilities: If a drastically different set of mental processes or entities realizes perfOlmance equally well across individuals those mental processes will likely have different neural substrates. For example, verbal abilities and visuospatial abilities have been associated with the activities of different parts of the brain (Hunt, 2010). If, however, one person solves a set of mathematical problems using mental imagery and another solves them using logical analysis, this "same" mathematical ability (construed fimctionally) will likely have different neurophysiological correlates. Moreover, practice and expertise acquisition have been sho\Vll to alter the brain structure and activity underlying the tasks and procedures rehearsed (Hill & Schneider, 2006; Neumann, Lotz, & Eickhoff, 2016; Petersen, van Mier, Fiez, & Raichle, 1998), suggesting that the neurological underpinnings of the same mental activities of the same person may differ over time. Additionally, just as individual-level causal entities carmot be inferred automatically from psychometric test scores, the causal influence of brain region activity for individuals carmot necessarily be inferred from the results of group-level analyses (Ramsey, Hanson, Hanson, Ha1chenko, Poldrack, & Glymour, 2010). For example, men and women appear to achieve equivalent results on cognitive tests using the operations of different brain regions (Haier, Jung, Yeo, Head, & Alkire, 2005).

Summary and Conclusions For over 100 years, researchers and practitioners working primarily (but not exclusively) in the factor-analytic tradition have put a premium on positive manifolds and, sometimes unwittingly, application of the principle of the common cause to establish the existence of distinct general and specific abilities. In this chapter, theory and evidence have been reviewed that are consistent with the existence of these two differentiable classes of abilities - but also theory and evidence that are consistent with the existence of partially or wholly indistinct sources of variance in cognitive test scores. Challenges to the theoretical distinction between and general and specific abilities arising from the latter include: 1) entities that influence perfOlmance on a wide variety of cognitive tasks carmot be inferred from the overall positive manifold and entities that influence perfOlmance on a circumscribed domain of cognitive tasks carmot be

Theoretical Challenges

241

inferred from more limited positive manifolds due to factor induction being a restatement of the positive manifold phenomenon and underdetennination, 2) individual-level entities cannot automatically be inferred from the between-person level of analysis positive manifolds are observed at, 3) different individuals use different strategies to solve different types of items, 4) abilities cannot be easily - if at all differentiated - in tenns of the goals they are in service of or the brain regions that generate them, and 5) if general and specific abilities can be differentiated in tenns of their component structures and component sequences those analyses will have to occur at the level of individuals and the results of those analyses will not necessarily be consistent across all of those individuals. The co-existence of competing explanations for the same phenomenon poses a challenge - and is perhaps even frustrating but it is also common in the history of science and not unique to the study of cognitive abilities. It is important to note that in the applied world, where many cognitive tests are used (e.g., personnel selection, student admissions), some of the points raised about the difficulties of theoretically distinguishing general and specific abilities will not be relevant; these points pertain to explanations of covariance patterns but the mere existence of those covariances is not in doubt. "Empirical J;" (Thorndike, 1994) and the more limited positive manifolds implying content-aligned abilities are some of the most replicated observations in psychology (Murphy, Dzieweczynski, & Zhang, 2009). Many stakeholders may not particularly care how individuals achieve the scores that generate these manifolds (e.g., knowledge vs. reasoning) but will simply be interested in choosing the individuals with the highest scores. Moreover, many of the assessments needed to uncover fme-grained differences in individuals' strategy usage can require extensive instructions and practice in order for participants to familiarize themselves with the tasks, frequent one-on-one interactions with test administrators, and/or post-test analyses of complex log files (e.g., Gonthier & Thomassin, 2015; Kozhevnikov et aI., 2005; Lotz, Scherer, Greiff, & Sparfeldt, 2017), making them too time-consuming and expensive for use in many applied settings (Hunt, 1987). The gulf between explaining cognitive test scores and using cognitive test scores is a manifestation of the long-standing - and ongoing - scientific debate about the merits of prediction versus explanation (Shmueli, 2010; Yarkoni & Westfall, 2017). Indeed, despite his antagonistic relationship with Spearman, Thomson (1952, p. 283) called mental factors "possibly very useful fictions" and remarked that "the real defence of g is simply that it

242

Chapter Nine

has proved useful" (as cited in Deary, Lawn, & Bartholomew, 2008, p. 15). A definitive resolution to the theoretical and practical debates about general and specific abilities is unlikely to occur in the near future. Psychology has a short history (Hunt, 2010) and its accomplishments are understandably slight compared to those of the natural sciences (Meehl, 1978); some have even claimed that the evidence for the now discredited entity phlogiston was stronger than the evidence supporting any current psychological theory (Borsboom, Cramer, Kievit, Scholten, & Sanja Franic, 2009). It took over 100 years to definitively disprove the existence of phlogiston (Gribbni, 2004) and "scientific" psychology is less than 150 years old. Moreover, predicting where scientific investigation will lead is difficult. What was unthnikable can be rendered obvious: The ability of the heart to circulate blood was once considered mysterious, because muscles were thought to only be able to contract through conscious effort (Grene, 1993), and lymphatic vessels were recently discovered ni the brani (Louveau et aI., 2015). Oppositely, long-held truths and assumptions can be overturned: Two years before Einstein was awarded his doctoral degree, future Nobel prize winner Albert Michelson (1903) wrote: "The more important fundamental laws and facts of physical science have all been discovered, and these are now so fitmly established that the possibility of their ever being supplanted in consequence of new discoveries is exceedingly remote" (pp. 23-24). The chemical bond, a fundamental concept instilled in high school coursework, has itself been revealed to be a useful fiction (Ball, 2011). In light of the niherent difficulty of psychology's subject matter and the unpredictability of scientific discovery, the most appropriate way forward is to follow Robert Oppenheimer's (1953, p. 1 14) credo: "The scientist is free, and must be free to ask any question, to doubt any assertion, to seek for any evidence, to correct any errors."

References Adolf, J., SchuUlTIlan, N. K., Borkenau, P., Borsboom, D., & Dolan, C. V. (2014). Measurement nivariance within and between individuals: A distinct problem in testing the equivalence of intra-and inter-individual model structures. Frontiers in Psychology, 5, 883. Allport, G. W. (1958). What units shall we employ? In G. Lindzey (Ed.), Assessment of human motives (pp. 238-260). New York, NY: Rniehart.

Theoretical Challenges

243

American Psychological Association (1954). Technical recommendations for psychological tests and diagnostic techniques. Psychological Bulletin Supplement, 51, 1-38. Aristotle (1996). Physics. (R. Waterfield, Trans.). Oxford, UK: Oxford University Press. (Original work published ca. 300 BCE). Amtzenius, F. (2010). Reichenbach's common cause principle. Stanford Encyclopedia ofPhilosophy. Retrieved from https:llplato.stanford.edulentries/physics-Rpcc/ Bakan, P. (1955). The general and the aggregate: A methodological distinction. Perceptual and Motor Skills, 5, 21 1-212. Ball, P. (201 1). Beyond the bond. Nature, 469, 26-28. Bamford, G. (1996). Popper and his commentators on the discovery of Neptune: A close shave for the law of gravitation? Studies in History and Philosophy ofScience PartA, 27, 207-232. Barrett, D. A. (2012). Multiple realizability, identity tbeory, and tbe gradual reorganization principle. The British Journal for the Philosophy ofScience, 64, 325-346. Bartbolomew, D. J., Allerhand, M., & Deary, I. J. (2013). Measuring mental capacity: Thomson's Bonds model and Spearman's g-model compared. Intellir;ence, 41, 222-233. Benson, N. F., Beaujean, A. A., McGill, R. J., & Dombrowski, S. C. (2018). Revisiting Carroll's survey of factor-analytic studies: Implications for tbe clinical assessment of intelligence. Psychological Assessment, 30, 1028-1038. Binet, A. (1 892). Mental imagery. Fortnightly Review, 52, 95-104. Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of AppliedPsychology, 74, 478-494. Boccignone, G., & Cordeschi, R. (2015). Coping witb levels of explanation in the behavioral sciences. Frontiers in Psychology, 6, 213. Borden, N., & LinkJater, D. (2013). Hickam's dictum. Western Journal of Emergency Medicine:Integrating Emergency Care with Population Health, 14, 164. Borg, I. (2018). A note on tbe positive manifold hypotbesis. Personality and Individual Differences, 134, 13-15. Boring, E. G. (1923). Intelligence as the tests test it. New Republic, 3 6, 35-37 Borkenau, P., & Ostendorf, F. (1998). The Big Five as states: How useful is the five-factor model to describe intraindividual variations over time? Journal ofResearch in Personality, 32, 202-221 .

244

Chapter Nine

Borsboom, D. (2015). What is causal about individual differences? A comment on Weinberger. Theory & PsycholofiJ', 25, 362-368. Borsboom, D., Cramer, A. O. J., Kievit, R. A., Scholter, A. Z., & Franic, S. (2009). The end of construct validity. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions, and applications (pp. 135-172). Charlotte, NC: Information Age. Borsboom, D., & Dolan, C. V. (2006). Why g is not an adaptation: A comment on Kanazawa (2004). Psychological Review, 113, 433-437. Borsboom, D., Kievit, R. A., Cervone, D. P., & Hood, S. B. (2009). The two disciplines of psychology, or: The disunity of psychology as a working hypotliesis. In I Valsiner, P. C. M. Molenaar, M. C. D. P. Lyra, & N. Chaudary (Eds.), Developmental process methodology in the social and developmental sciences (pp. 67-89). New York, NY: Springer. Borsboom, D., Mellenbergh, G. I, & van Heerden, J. (2003). The theoretical status of latent variables. Psychological Review, 1 1 0, 203219 . Bouchard, T. J., & Loehlin, J. C. (2001). Genes, evolution, and personality. Behavior Genetics, 31, 243-273. Bullock, J. G., Green, D. P., & Ha, S. E. (2010). Yes, but what's tlie mechanism? (Don't expect an easy answer). Journal of Personality and Social PsycholofiJ', 98, 550-558. Burt, C. (1947). Factor analysis and physical types. Psychometrika, 12, 171-188. Burt, C. (1949). The two-factor theory. British Journal of Statistical PsychoI0fiJ', 2, 151-179. Butz, M. V., & Kutter, E. F. (2016). How the mind comes into being: Introducing cognitive science from a functional and computational perspective. Oxford, UK: Oxford University Press. Caspi, A., Houts, R. M., Belsky, D. W., Goldman-Mellor, S. J., Harrington, H., Israel, S., ... & Moffitt, T. E. (2014). The p factor: One general psychopatliology factor in the structure of psychiatric disorders? Clinical Psychological Science, 2, 119-137. Cliff, N. (1983). Some cautions concerning tlie application of causal modeling methods. Multivariate Behavioral Research, 18, 1 15-126. Cooper-Hakim, A., & Viswesvaran, C. (2005). The construct of work commitment: Testing an integrative framework. Psychological Bulletin, 131 , 241-259 . Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302 .

Theoretical Challenges

245

Deary, I. l, Lawn, M., & Bartholomew, D. J. (2008). A conversation between Charles Spearman, Godfrey Thomson, and Edward L. Thorndike: The International Examinations Inquiry meetings: 19311938. History ofPsycholo?,Y, 11, 122-142. Ebel, R. L. (1974). And still the dryads linger. American Psychologist, 29, 485-492. Embretson, S. E (2004). The second century of ability testing: Some predictions and speculations. Measurement, 2, 1-32. English, H. & English, A. (1958). A comprehensive dictionary of psychological and psychoanalytical terms. London, UK: Longman. Eysenck, H. J. (1997). Personality and experimental psychology: The unification of psychology and the possibility of a paradigm. Journal of Personality and Social Psychology, 73, 1224-1237. Fodor, J. (1968). Psychological explanation. New York, NY: Random House. Fontenrose, R. (1973). In search of Vulcan. Journal for the History of Astronomy, 4, 145-l58. Fortuna, M. A., Zaman, L., Ofria, C., & Wagner, A. (2017). The genotype­ phenotype map of an evolving digital organism. PLoS Computational Biolo?,Y, 13 . Retrieved from https:lljoumals. plos.org/ploscompbioll artic1e?rev�2&id� 1 0. 137l/journal.pcbi. l 005414 Galton, F. (1 880). Statistics of mental imagery. Mind, 5, 301-318. Gonthier, C., & Thomassin, N. (2015). Strategy use fully mediates tlie relationship between working memory capacity and perfOlmance on Raven's matrices. Journal ofExperimental Psychology: General, 144, 916-924. Gould S. l (1996). The mismeasure ofman (2nd ed.). New York, NY: WW Norton & Company. Grene, M. (1993). The heart and tlie blood: Descartes, Plemp and Harvey. In S. Voss (Ed.), Essays on the philosophy and science of Rene Descartes (pp. 324-336). Oxford, UK: Oxford: Oxford University Press. Gribbin, l (2004). The scientists. New York, NY: Random House. Guilford, l P. (1985). A sixty-year perspective on psychological measurement. Applied PsycholofiicalMeasurement, 9, 341-349. Gustafsson, J. E (1984). A unifying model for the structure of intellectual abilities. Intelligence, 8, 179-203. Gustafsson, J. E, & Snow, R. E (1997). Ability profiles. In R. F. Dillon (Ed.), Handbook on testing (pp. 107-135). Westport, CT: Greenwood Press.

246

Chapter Nine

Haier, R. J., Jung, R. E., Yeo, R. A., Head, K., & Alkire, M. T. (2005). The neuroanatomy of general intelligence: sex matters Neuro!mage, 25, 320-327. Haig, B. D., & Evers, C. W. (2016). Realist inquiry in social science. Los Angeles, CA: SAGE. Hamaker, E. L., Dolan, C. V., & Molenaar, P. C. (2005). Statistical modeling of the individual: Rationale and application of multivariate stationary time series analysis. Multivariate Behavioral Research, 40, 207-233. Hill, N. M. & Schneider, W. (2006) Brain changes in the development of expertise: Neuroanatomical and neurophysiological evidence about skill-based adaptations. In K. A. Ericsson, N. Chamess, P. Feltovich, & R. R. Hoffman (Eds.), Cambridge handbook of expertise and expert performance (pp. 223-242). Cambridge, UK: Cambridge University Press. Hilliard, A. A., Weinberger, S. E., Tierney If, L. M., Midthun, D. E., & Saint, S. (2004). Occam's razor versus Saint's triad. New England Journal ofMedicine, 350, 599-603. Hofman, A., Kievit, R., Stevenson, C., Molenaar, D., Visser, I., & van der Maas, H. (2018). The dynamics of the development of mathematics skills: A comparison of theories of developing intelligence. Retrieved from https:llosf.io/xa2ft Horn, J. (1998). A basis for research on age differences in cognitive abilities. In J. J. McArdle & R. W. Woodcock (Eds.), Human cognitive abilities in theory and practice (pp. 57-92). Mahwah, NJ: Lawrence Erlbaum. Horn, J. L., & Cattell, R. B. (1966). Age differences in primary mental ability factors. Joumal of Gerontology, 21, 21 0-220. Horn, J. L., & McArdle, J. J. (2007). Understanding human intelligence since Spearman. In R. Cudeck & R. C. MacCallum (Eds.), Factor analysis at 100: Historical developments and future directions (pp. 205 fi 247). Mahwah, NJ: Erlbaum. Hunt, E. (1983). On the nature of intelligence. Science, 219, 141-146. Hunt, E. (1987). Science, technology, and intelligence. In R. R. Ronning, J. A Conoley, & J. C. Witt (Eds.), The influence of cognitive psychology on testinK (pp. 1 1 -40). Hillsdale, NJ: Erlbaum. Hunt, E. (2010). Human intelligence. Cambridge, UK: Cambridge University Press. Hurley, P. J. (2015). A concise introduction to logic (12th ed.). Stamford, CT: Cengage Learning.

Theoretical Challenges

247

International Union of Pure and Applied Chemistry (2014). Compendium of chemical terminolofi)', r;old book (version 2.3.3). Retrieved from https:llgoldbook.iupac.org Ipdf/goldbook.pdf James, W. (1 890). The principles o/psychology. Cambridge, MA: Harvard University Press. Jensen, A. R. (1987). The r; beyond factor analysis. In R. R. Ronning, J. A. Glover, J. C. Conoley, & J. C. Witt (Eds.), The influence 0/ cognitive psychology in testing (pp. 87-142). Hillsdale, NJ: Erlbaum. Jensen, A.R. (1998). The g/actor. Westport, CT: Praeger. Johnson, K. (2016). Realism and uncertainty of unobservable common causes in factor analysis. NoUs, 50, 329-355. Johnson, W. (2018). A tempest in a ladle: The debate about the roles of general and specific abilities in predicting important outcomes. Journal o/Intelligence, 6, 24. Johnson, W., & Bouchard Jr, T. I (2005). The structure of human intelligence: It is verbal, perceptual, and image rotation (VPR), not fluid and crystallized. Intelligence, 33, 393-416. Johnson, W., & Gottesman, I. I. (2006). Clarifying process versus structure in human intelligence: Stop talking about fluid and crystallized. Behavioral and Brain Sciences, 29, 136-137. Judge, T. A., Erez, A., Bono, J. E., & Thoresen, C. J. (2002). Are measures of self-esteem, neuroticism, locus of control, and generalized self-efficacy indicators of a common core construct? Journal of Personality and Social Psycholofi)', 83, 693-710. Kell, H. I, & Lang, J. W. (2017). Specific abilities in the workplace: More important than r;? Journal ofIntellir;ence, 5, 13. Kievit, R. A., Lindenberger, U., Goodyer, I. M., Jones, P. B., Fonagy, P., Bullmore, E. T. & Dolan, R. I (2017). Mutualistic coupling between vocabulary and reasoning supports cognitive development during late adolescence and early adulthood. Psychological Science, 28, 14191431. Kovacs, K., & Conway, A. R. (2016). Process overlap theory: A unified account of the general factor of intelligence. Psychological Inquiry, 27, 151-177. Kozhevnikov, M., Kosslyn, S., & Shephard, I (2005). Spatial versus object visualizers: A new characterization of visual cognitive style. Memory & COKnition, 33, 710-726. Kyllonen, P. C., Lohman, D. F., & Snow, R. E. (1984). Effects of aptitudes, strategy training, and task facets on spatial task perfOlmance. Journal o/Educational Psychology, 76, 130-145.

248

Chapter Nine

Lang, J. W., Kersting, M., Hiilsheger, U. R., & Lang, J. (2010). General mental ability, narrower cognitive abilities, and job performance: The perspective of the nested-factors model of cognitive abilities. Personnel PsycholofiJ', 63, 595-640. Levenson, T. (2015). The hunt for Vulcan. New York, NY: Random House. Loevinger, J. (1940). Intelligence as related to socio-economic factors. In G. M. Whipple (Ed.), The thirty-ninth yearbook of the National Society for the Study of Education: Pt 1. Intelligence: Its nature and nurture (pp. 159-202). Bloomington, IL: Public School. Lohman, D. F. (1996). Spatial ability and g. In I. Dennis & P. Tapsfield (Eds.), Human abilities: Their nature and measurement (pp. 97-1 16). Hillsdale, NJ: Erlbaum. Lohman, D. F., & Lakin, J. M. (201 1). Intelligence and reasoning. In R. J. Sternberg & S. B. Kaufinan (Eds.), The Cambridge handbook of intelligence (pp. 419-441). New York, NY: Cambridge University Press. Lotz, C., Scherer, R., Greiff, S., & Sparfeldt, J. R (2017). Intelligence in action-Effective strategic behaviors while solving complex problems. Intelligence, 64, 98-112. Louveau, A., Smimov, I., Keyes, T. J., Eccles, J. D., Rouhani, S. J., Peske, J. D., . . . & Harris, T. H. (2015). Structural and functional features of central nervous system lymphatic vessels. Nature, 523, 337-341 . Mackintosh, N. (201 1). IQ and human intelligence (2nd ed.). Oxford, UK: Oxford University Press. Major, J. T., Johnson, W., & Deary, I. J. (2012). Comparing models of intelligence in Project TALENT: The VPR model fits better tban tbe CHC and extended Gf-Gc models. Intelligence, 40, 543-559. Marr, D. (1982). Vision. San Francisco, CA: W.H. Freeman. Maxwell, G. (1962). The ontological status of theoretical entities. In H. Feigl & G. Maxwell (Eds.), Minnesota studies in the philosophy of science: Vol. 3. Scientific explanation, space, and time (pp. 3-28). Minneapolis: University of Minnesota Press. McCrae, R R, & Costa, P. T. (2008). The Five-Factory Theory of personality. In O. P. John, R.W. Robins, & L. A. Pervin (Eds.), Handbook ofpersonality: Theory and research (3,d ed., pp. 159-181). New York, NY: Guilford Press. McDonald, R P., & Mulaik, S. A. (1979). Determinacy of common factors: A nontechnical review. Psychological Bulletin, 86, 297-306.

Theoretical Challenges

249

McGrew, K.S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intellir;ence, 37, 1-10. McKinnon, A. M. (2010). Energy and society: Herbert Spencer's 'energetic sociology' of social evolution and beyond. Journal of Classical Sociolo?,Y, 10, 439-455. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consultinr; and Clinical Psycholo?,Y, 4 6, 806-834. Meehl, P. E. (1990). Why sunnnaries of research on psychological theories are often uninterpretable. Psychological Reports, 66, 195-244. Meehl, P. E. (1993). Philosophy of science: Help or hindrance? Psychological Reports, 72, 707-733. Michelson, A. (1903). Light waves and their uses. Chicago, IL: The University of Chicago Press. Miller, J., & Schwarz, W. (2018). Implications of individual differences in on-average null effects. Journal ofExperimental Psychology: General, 147, 377-397. Molenaar, P. C. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement, 2, 201-218. Montgomery, K. (2006). How doctors think: Clinical judr;ment and the practice Qfmedicine. Oxford, UK: Oxford University Press. Mulaik, S. A. (1987). A brief history of the philosophical foundations of exploratory factor analysis. Multivariate Behavioral Research, 22, 267-305. Murphy, K. R., Dzieweczynski, J. L., & Zhang, Y. (2009). Positive manifold limits the relevance of content-matching strategies for validating selection test batteries. Journal of Applied Psychology, 94, 1018-103 1 . Murray, A. L., & Johnson, W. (2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence, 41, 407-422. Musek, J. (2007). A general factor of personality: Evidence for the Big One in the five-factor model. Journal of Research in Personality, 41, 1213-1233. Neumann, N., Lotze, M., & Eickhoff, S. B. (2016). Cognitive expertise: An ALE meta-analysis. Human Brain Mapping, 37, 262-272. Newton, D. E. (2010). Chemical elements (2nd ed.). Farmington Hills, MI: Gale.

250

Chapter Nine

Norton, B. (1979). Charles Spearman and the general factor in intelligence: Genesis and interpretation in the light of sociopersonal considerations. Journal a/ the History a/ the Behavioral Sciences, 15, 142-154. Oates, D. W. (1930). The energy concept in modem psychology. The Journal of General Psychology, 4, 401-406. Oppenheimer, J. R. (1955). The open mind. New York, NY: Simon and Schuster. Pannekoek, A. (1953). The discovery of Neptune. Centaurus, 3, 126-137. Petersen, S. E., Van Mier, H., Fiez, J. A., & Raichle, M. E. (1998). The effects of practice on the functional anatomy of task perfOlmance. Proceedings ofthe National Academy ofSciences, 95, 853-860. Pezaris, E., & Casey, M. B. (1991). Girls who use "masculine" problemsolving strategies on a spatial task: Proposed genetic and environmental factors. Brain and Cognition, 1 7, 1-22. Polger, T. W. (2012). Functionalism as a philosophical theory of the cognitive sciences. Wiley Interdisciplinary Reviews: Cognitive Science, 3, 337-348. Prokosch, M. D., Yeo, R. A., & Miller, G. F. (2005). Intelligence tests with higher g-loadings show higher correlations with body symmetry: Evidence for a general fitness factor mediated by developmental stability. IntelliKence, 33, 203-213. Quine, W. V. (1975). On empirically equivalent systems of the world. Erkenntnis, 9, 3 1 3-328. Ramsey, I. D., Hanson, S. J., Hanson, C., Ha1chenko, Y. 0., Poldrack, R. A., & Glymour, C. (2010). Six problems for causal inference from !MR!. Neuroimage, 49, 1545-1558. Ree, M. J., & Carretta, T. R. (2002). g2K. Human Performance, 15, 3-23. Reichenbach, H. (1956). The direction of time. Berkeley, CA: University of California Press. Robinson, D. L. (1995). An intellectual history of psychology (3,d ed.). Madison, WI: The University of Wisconsin Press. Rounds, J. (1995). Vocational interests: Evaluating structural hypotheses. In D. Lubinski & R. V. Dawis (Eds.), Assessing individual differences in human behavior (pp. 177-232). Palo Alto, CA: Davies-Black. Royce, J. R. (1963). Factors as theoretical constructs. American PsycholoKist, 18, 522-528. Rozeboom, W. W. (1961). Ontological induction and the logical typology of scientific variables. Philosophy ofScience, 28, 337-377. Ryans, D. G. (1939). A note on methods of test validation. Journal of Educational Psychology, 30, 315-319.

Theoretical Challenges

251

Rychlak, J. F. (1981). Personality and psychotherapy (2nd ed.). Boston, MA: Houghton Millin. Shmueli, G. (2010). To explain or to predict? Statistical Science, 25, 289310. Snow, R. E., & Lohman, D. F. (1984). Toward a theory of cognitive aptitude for learning from instruction. Journal of Educational Psychology, 76, 347-376. Snow, R. E., & Lohman, D. F. (1989). Implications of cognitive psychology for educational measurement. In R. L. Linn (Ed.), Educational measurement (3,d ed., pp. 263-331). New York, NY: Collier. Sober, E. (1988). The principle of the common cause. In J. Fetzer (Ed.), Probability and causality: Essays in honor of Wesley C. Salmon (pp. 2 1 1-229). Dordrecht, Netherlands: D. Reidel Publishing Company. Spearman. (1904). "General intelligence," objectively determined and measured. American Journal ofPsychology, 15, 201-292. Spearman, C. (1927). The abilities ofman. London, UK: Macmillan. Thomson, G. H. (1916). A hierarchy without a general factor. British Journal ofPsychology, 8, 271-281. Thomson, G. H. (1952). Godfrey Thomson. In Boring, R. G., Langfeld, H. S., Werner, H., & Yerkes, R. M. (Eds), A history of psychology in autobiography (Vol. 4, pp. 279-294). Worcester, MA: Clark University Press. Thorndike, R. L. (1994). K. IntelliKence, 19, 145-155. Thurstone, L. L. (1934). The vectors of mind. Psychological Review, 41, 1-32. Thurstone, L. L. (1935). The vectors of mind. Chicago, IL: University of Chicago Press. Thurstone, L. L. (1938a). Primary mental abilities. Chicago, IL: University of Chicago Press. Thurstone, L. L. (1938b). Shifty and mathematical components: A critique of Anastasi's monograph on the influence of specific experience upon mental organization. Psychological Bulletin, 35, 223-236. Thurstone, L. L. (1947). Multiple factor analysis. Chicago, IL: The University of Chicago Press. Tomarken, A. I, & Waller, N. G. (2003). Potential problems with "well fitting" models. Journal ofAbnormal Psychology, 112, 578-598. Tracey, T. I (2012). Problems with single interest scales: Implications of the general factor. Journal of Vocational Behavior, 81, 378-384.

252

Chapter Nine

van BOIk, R., Epskamp, S., Rhemtulla, M., Borsboom, D., & van der Maas, H. L. (2017). What is the p-factor of psychopathology? Some risks of general factor modeling. Theory & PsycholofiJ', 27, 759-773. van der Maas, H. L., Dolan, C. V., Grasman, R. P., Wicherts, I. M., Huizenga, H. M., & Raijmakers, M. E. (2006). A dynamical model of general intelligence: The positive manifold of intelligence by mutualism. Psychological Review, 113, 842-861. Vernon, P. E. (1979). Intelligence: Heredity and environment. San Francisco, CA: Freeman. Viswesvaran, C., & Ones, D. S. (2000). Perspectives on models ofjob perfOlmance. International Journal of Selection and Assessment, 8, 216-226. Vogt, G., Huber, M., Thiemann, M., van den Boogaart, G., Schmitz, O. J., & Schubart, C. D. (2008). Production of different phenotypes from the same genotype in the same environment by developmental variation. Journal o/Experimental Biology, 211, 510-523. Wallace, J. (1967). What units shall we employ? Allport's question revisited. Journal ofConsultinr; PsycholofiJ', 31, 56-64. Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psycholor;ical Science, 12, 1 100- 1 122. Yu, M. K., Kramer, M., Dutkowski, J., Srivas, R., Licon, K., Kreisberg, J. F., ... & [deker, T. (2016). Translation of genotype to phenotype by a hierarchy of cell subsystems. Cell Systems, 2, 77-88.

CHAPTER TEN WITHIN-INDIVIDUAL VARIABILITY OF ABILITY AND LEARNING TRAJECTORIES IN COMPLEX PROBLEMS DAMIAN P. BIRNEY, JENS F. BECKMANN & NADIN BECKMANN

The historical perspective of intelligence is a decidedly between­ subjects affair. This is reflected in the dominance of factor analysis as both a psychometric tool for validation and as the cornerstone of the theoretical conceptualisation of intelligence as a hierarchically structured human attribute (Thurston 1938, Hom and Cattell 1966, Carroll 1993, Stankov 2000b, Schneider and McGrew 2012, McGrew 2009). In spite of the significant gains made over the last 120 years in our understanding of its structure, it turns out that knowing what intelligence is and is not correlated with-the psychometric approach to mapping the nomological network (Borsboom, Mellenbergh, and van Heerden 2004, Sternberg 1990)-does not actually tell us much about the basis of intelligence. In this chapter we have a simple objective: to reflect on insights gained in our use of linear mixed-effects models and experimental manipulations to investigate how a within-subject process-oriented approach to human intellect might better augment our understanding of its correlates. In this chapter we first briefly remind ourselves of the foundations of the psychometric approach underlying the Cattell-Hom-Carroll (CHC) theory of intellectual abilities and how this framework continues to evolve (Schneider and McGrew 2012, Schneider, Mayer, and Newman 2016). We then aim to substantiate why the psychometric approach will always provide a limited account of intelligence and what might be done to redress this. One of the particularly interesting features of intelligence tests is the role of complexity, and its corollary, that intelligence is needed to meet the challenges of complexity in everyday problems. However, what

254

Chapter Ten

is difficult, is not always complex, so it is important to be clear of the distinction between difficulty and complexity, and we summarise our view on this. Finally, we present three case studies as (1) the basis of an argument for the importance of considering a process-oriented account of the impact complexity manipulations have on perfOlmance, and (2) as an example of how this might be achieved using repeated-measures designs and linear-mixed effects regression. We conclude with a description of the core components of psychometric complexity as a paradigm for ongoing investigation.

A Hierarchical Perspective on Intelligence : The Psychometric Approach The CHC theory provides an extensively validated framework for conceptualizing and measuring human intellectual abilities (Schneider and McGrew 2012, McGrew 2009). Its foundation is Spearman's (1904) recognition of the theoretical importance of positive manifold-that all cognitive tasks tend to be more or less positively correlated with each other. Spearman suggested that this correlation reflected a general mental enerfi)', or 'J; '. Subsequent research (e.g., Hom and Cattell 1966, Stankov 2000b, Thurston 1938) into a diversity of cognitive tasks demonstrated that performances on some types of tasks tended to be more highly correlated with each other, than they were with perfOlmances on other types of tasks. Careful analysis of these statistically 'similar and different' tasks gave insight into potentially common and distinct functions, in addition to (or instead of) 'g' (Carroll 1993). The observed patterns of convergent and divergent correlations were directly interpreted as the manifestation of distinct, fundamental, latent cognitive abilities. Over time, these abilities mapped out the nomological network of intellect into a dynamic, three-stratum taxonomical hierarchy, knO\vn as the Cattell­ Horn-Garroll (CHC) theory of cognitive abilities (Schneider and McGrew 2012). At the third (top level) stratum is 'g'. A small number of 'broad abilities' define the second stratum, and a larger number of 'narrow abilities' occupy the lowest level or first stratum. McGrew (2009) considered the hierarchy "dynamic" not because the nature of the functions change in degree or type, but because new narrow and broad ability factors can be added to the taxonomy conditional on them meeting this validation standard across multiple samples and contexts.

Within-Individual Variability of Ability and Learning Trajectories

255

An Argument for Process-Oriented Accounts Notwithstanding the extraordinary success of CHC theory ill describing between-subject differences, it has long been recognised that the individual-differences approach to the investigation of psychological attributes generally, and intellectual abilities specifically as we have just described, is incomplete without a consideration of process-oriented accounts (van der Maas et al. 2017, Cronbach 1957, Deary 2001). Lohman and Ippel (1993, p 41) citing Cronbach (1957), McNemar (1964), Spearman (1927) and others, concluded that a major reason why the individual differences approach to the study of intelligence " . . . was unable to achieve one of its central goals: the identification of mental processes that lUlderlie intelligent flUlctioning", was because " . . . a research program dominated by factor analysis of test intercorrelations was incapable of producing an explanatory theory of human intelligence".

They argued for a considered cognitive approach where tasks are designed to detect theoretically specified, qualitative differences (see also, Deary 2001). Lohman and Ippel (1993, p 42) were suggesting that the general idea of test theory as applied statistics (i.e., psychometrics) not only hampered the development of structural theories for the measurement of processes, but actually precluded it. This was consistent with their reading of the earlier recommendation Guttman (1971) had proposed in his presidential address to the annual meeting of the Psychometric Society. Here, Guttman contrasted the purpose of observation in the psychometric testing tradition, which was (and generally still is) to compare individuals, with his proposed, amended purpose to assess the structure of relationships amonx observations. In effect, Guttman was arguing that if one wishes to better understand the processes of intelligence, one needs to take a distinctively within-subjects perspective. It is precisely this agenda that we explore in this chapter. There have been many theoretical and technical developments over the last 25 years in particular that have made it easier to address the role of within-subject variability, we will consider some shortly. Yet, the breadth and impact of what psychometric tests of between-subject intellect predict is truly impressive and hard to ignore (Gottfredson, 2018) the psychometric tradition has served us well. This ubiquity of prediction is in no small way responsible for the status of intelligence testing at the very top of the historical successes of the psychological testing movement of the 20th century (Schmidt and Hunter 1998). We are certainly not advocating for a discontinuation of the psychometric tradition. Yet -

256

Chapter Ten

psychometric tests do not sufficiently explain why or how a prediction should hold in the first place. Again, this limitation is well known. Borsboom, Mellenbergh, and van Heerden (2003) provide compelling argumentation that within-subject level processing must be explicitly incorporated in measurement models if we are to substantively link between-subject models of intellect with what is happening at the level of the individual. It is interesting to note that whereas it is generally well­ accepted to take a dynamic, situation-dependent perspective on other individual differences attributes, like personality (Mischel and Shoda 1995, Minbashian, Wood, and Beckmann 2010, Wood et al. 2019), this is generally not the case for intelligence. This is likely due to the belief that intelligence tests assess maximal performance (Neisser et al. 1996), with its ensuing assumption that measures of maximal intelligence and their use imply "the existence of a stable or permanent capability" (Goff and Ackerman 1992, p 538). To elaborate on why this is a limited perspective, we reflect briefly on these aspects of the standard psychometric approach to developing a test, because this stability is ostensibly antithetical to the notion of within-subject variability.

The Stability Assumption of Intelligence So why is intelligence commonly thought to be stable, and why might this be a problem? First, to be clear, we are not concerned here with the fact that normative population-based scaling reflects an appearance of stability over time. Similarly, we are not overly concerned with the arguments of Cattell (1987) and others (e.g., Ackerman 2017, McArdle et al. 2002) who suggest that the apparent stability of intelligence is a necessary outcome of aggregating across multiple abilities that have different developmental trajectories. In tenns of within-subjects variability, it does not matter too much which level of aggregation one chooses, 'g', broad or narrow. 'While aggregation may obscure differences, or at worst preclude their consideration, because these effects are observed at the between-subject level, a within-subject perspective of intelligence is precluded either way (Borsboom, Mellenbergh, and van Heerden 2004, Borsboom 2015). We believe the more important reason why it has been challenging to integrate an inlierent within-individual mutability into the conceptualization of intelligence, is because of the limitations in traditional test development methods and the rigidness of tenets that have evolved to service the principles of best-test design (e.g., Pedhazuer and Schmelkin 1991, Wright and Stone 1979). To demonstrate, consider the notion of learning, which

Within-Individual Variability of Ability and Learning Trajectories

257

has at its core a within-subject conceptualization. It is generally accepted across various domains of education and psychology, that knowledge and expertise is acquired (at least in part) through the motivated (self­ regulated) investment of cognitive resources - that is, as a direct product of learning (e.g., Ackerman 1996, Ericsson 2003, Ackerman and Beier 2005). However, the facilitating cognitive abilities (e.g., GfJ underlying knowledge acquisition have typically been assumed to be largely immune (or resistant) to training/learning - that is, to exhibit stability. This in spite of tasks, like the Raven's Advanced Progressive Matrices (APM), requiring induction of rules (i.e., learning) on earlier items to best support the induction and application of different rules on later items as a central explanatory process underlying solution (Carpenter, Just, and Shell 1990, Bui and Birney 2014). Technically, a distinct capacity to learn, separate from Gf, is not a threat to the stability assumption because this additional capacity would slot in as a new factor in the CHC framework.

Why the Assumption of Stability is Restrictive As we have alluded to, the stability assumption has historical and somewhat pragmatic origins linked to test design principles. Consider an intelligence test made up of, say, 36 items (like Set II of the APM). Imagine now that there are individual differences in within-task learning from item-to-item that exist and operate in ways that change the nature of the ability being assessed across the test. In such cases, a non-random source of variance will be added to the measurement. If one considers the typical test-development process, this variance will be reflected in lower reliability estimates because, rather than the test measuring one construct, it will measure at least two reliable but imperfectly correlated ones: (1) individual differences in the primary intellectual ability of interest, and (2) individual differences in a secondary, within-task learning factor that might modify in some way the primary ability being measured. If the effect of the latter is strong, then the test will appear urneliable and because reliability is typically considered to be the upper-bound of validity (Pedhazuer and Schmelkin 1991), our confidence in the validity of the test as a whole (as measuring what it purports to measure, Borsboom, Mellenbergh, and van Heerden 2004) will be shaken. In response, the common practice is to screen out items that demonstrate "instability"- that is, to exclude items with lower item-total correlations (or factor loadings), and keep or add items with higher item-total correlations (or factor loadings). Over repeated test-development iterations, the end result is a test that captures a narrowly defined and static component of intelligence.

258

Chapter Ten

This is not a new problem. The limitations of traditional psychometrics has long been recognized as overly restrictive in areas where assessment of dynamic processes is of interest, for instance, Dynamic Testing (Guthke and Beckmann 2000, Grigorenko and Sternberg 1998), complex-problem solving (Beckmann, Birney, and Goode 2017, Domer and Funke 2017), and more recently cognitive flexibility (Beckmann, 2014). The point here is that the psychometric principles of best-test design practice are challenged by constructs that are by definition dynamic, fluid and complexly detelTIlined by contextual and intra-personal factors. In other words, rather than having stability and item internal consistency as their assessment goal, the central focus is on within-subject variability, or as Guthke and Beckmann (2000, p 22) put it, on "change and lack of homogeneity". The notion of constructs entailing abilities to manage dynamic changes in complexity requires a consideration of what complexity is, to which we now tum.

Complexity as the "Ingredient" of Intelligence Jensen (1987) has argued that the most undisputed fact about 'g' is that loadings of tasks on this factor are an increasing monotonic function of the tasks' complexity. This has also been observed and reported more broadly by Gottfredson (1997), who noted that the factor analysis of job attributes also produces a corresponding complexity-of-work factor. The basic tenet here is that high g-loadings correspond with perfOlmances in tasks, occupations and work that are more complex - broadly defined, complexity is the "active ingredient" in tests of intellect (Gottfredson 2018, 1997, Jensen 1987). Thus the view is that because 'g' entails a capacity to deal with complexity, an independent indicator of complexity are correlations with (or loadings on) measures of intelligence that increase with task complexity but all else being equal, not with increases in difficulty generated by other task features (Spilsbury, Stankov, and Roberts 1990, Stankov 2000a, Birney and Bowman 2009). However, correlations do not provide a clear conception of precisely what it is that makes a task complex (Schweizer 1998). Without a clear theory of complexity, researchers have often been left little option but to either adopt an eclectic approach to defining the cognitive complexity of a task (cf Stankov 2000a), or resort to post-hoc interpretations (Gottfredson 1997). This is appropriate if one's goal is simply to develop tasks that are good-enough measures of intelligence, however, a greater emphasis on process accounts

Within-Individual Variability of Ability and Learning Trajectories

259

is needed to understand why these tasks "work". Decomposing complexity seems a good place to startl .

Difficulty vs Complexity In the discussion of cognitive abilities and understanding why intelligence tests work, it is useful to make a finer distinction between difficulty and complexity (Beckmann, Birney, and Goode 2017). Difficulty is atheoretical, in that a rank-ordering of test items that are solved by fewer and fewer people tells us little about what make items difficult, just as correlations alone tell us little about complexity. Difficulty is a statistical concept captured by indices such as the proportion of people who answer an intelligence test item correctly. It is closely related to traditional concepts of ability, in that ability is conversely a function of the proportion of intelligence-test items a person answers correctly, and is thus a "quantifiable level of a person's success" (Beckmann, Birney, and Goode 2017, pI). Complexity on the other hand, is "conceptualized as a quality that is determined by the cognitive demands that the characteristics of the task and the situation impose" (p 1). In the next section we consider an extension of this notion, as proposed by Birney and Bowman (2009) and Birney et al. (2017), and consider the concept of psychometric complexity to differentiate empirical difficulty effects from more process­ oriented accounts of task complexity. We present three case studies that entail investigations of different complexity manipulations that are either observed or designed with the objective to broaden our understanding of within-subject accounts of cognitive abilities. Case I tests for complexity (vs difficulty) in four different tasks that have different within-task complexity manipulations. Case II considers item-level responses to investigate evidence of complexity in the correlates of the within-subject perfOlmance trajectories of item-difficulty and item-order on the APM. Finally, Case III considers a complex-problem solving (CPS) scenario requiring dynamic exploration and decision making to progress an outcome toward some more or less specific goal. Again we investigate evidence of complexity in the

1 There are limits to the ubiquity of the complexity account. There are certainly tasks that are neither difficult nor complex yet predictive of fluid intelligence. For instance, performance on the well-knmvn, simple perceptual inspection time tasks (Deary 2001), or the finding squares task (Oberauer et al. 2003), appear to impose minimal storage or processing load, yet are good predictor of Gf (Oberauer et al. 2008, Chuderski 2014).

260

Chapter Ten

correlates of the within-subject trajectories across explicit, theoretically specified task manipulations and learning opportunities.

Case I: A Within-Subjects Approach to Complexity Birney and Bowman (2009) aimed to differentiate process-oriented, theory-linked complexity factors from other factors that make solution difficult but do not necessarily place higher demands on Gf They investigated or processes by experimentally manipulating cognitive demands in four reasoning tasks (see Fig. 10-1). Two tasks came from the work of Stankov's individual differences research on the ingredients of complexity in fluid intelligence by considering working memory place keepers (WMP, Stankov 2000a, Stankov and Crawford 1993)-a) the Letter Swaps task in which complexity was manipulated ni terms of the number of serial, mental permutations required of three letters; and b) the Triplet Numbers task, where complexity manipulations entailed increasing the nature of conjunctive and disjunctive statements in rule validation of number size. The other two tasks were based on an explicit cognitive theory of relational complexity (RC) (Halford, Wilson, and Phillips 1998)-c) the Latin Square task in which relational complexity was manipulated in terms of the RC demand imposed by the requirement to integrate elements of an incomplete 4x4 matrix, and independently, the number of interim solutions to be held in mind (WMP) while doing so (Birney, Halford, and Andrews 2006, Birney et al. 2012), and d) the Sentence Comprehension task in which the degree of centre­ embeddedness (RC) was manipulated (Andrews, Birney, and Halford 2006). Two indicators of cognitive demand were considered. The first was the difficulty effect-task solution was expected to become more difficult as complexity increased. The second indicator was the complexity effect described previously. That is, the expectation was that increases in cognitive load would demand concomitantly increased investment of or resources (Stankov 2000a). This would be evident ni a statistically significant monotonic increase in the strength of the association between Gf and task complexity on perfOlmance. That is, as complexity increased, the performance of low vs high Gf individuals would diverge. This moderation of complexity on the relationship between task perfOlmance and Gfwe refer to as psychometric complexity. This is to make it clear that the foundation of the distinction between difficulty and complexity is that the latter is a testable theoretical statement, whereas the fOlmer is an atheoretical, statistical observation.

Within-Individual Variability of Ability and Learning Trajectories

Replicating

the

methodology

of Stankov

and

Crawford

261

( 1 993),

repeated-measures analysis of covariance were conducted with Gf (as measured by Raven ' s

APM) as the covariate. The difficulty effect was

evaluated by testing the main effects of the complexity level manipulation on performance. The complexity level

x

test of complexity was the linear contrast of the

Gf interaction effect, which, if statistically significant,

was interpreted to be

indicative of a monotonic

(linear) increasing

association with Gf acro ss the ordered levels of complexity.

A summary of

our reported results are presented in Table 1 0 - 1 . A . Latin Square

B. Sentence Comprehension task

1

Rule: Each row and each column must have one

"The artist that the waiter

instance of every element

warned the chef about

mi

type

1

talked" Who warned?

? • ••

options

C. Letter Swaps Task JKL Swap 1 with 3 Swap 2 with 3 Swap 1 with 2

D. Triplet Numbers Task

1

If the second digit is the l a rgest AND the third digit

I S the smal lest, then pres yes, otherwise press no

D

J KL

D

K LJ

D

JLK

D

LJ K

D

KJL

D

LKJ

1 372

Fig. 1 0- 1 . Example items from Birney and Bowman (2009)

Table

10-1. Summary of p artial 1]2

effect

sizes from ANCOVA

reported in Birney and Bowman (2009)

Task

APM!

Latin Square Task - RC Latin Square Task

- WMP

Difficulty2

.30a

Complexity3

.68

.02

.53

.09

Sentence comprehension test

.26

.35

.05

Letter Swaps Test

.22

.35

.12

Triplet Numbers Test

.21

.38

.06

Notes: 1

=

Between-subj ects mam-effect for APM; 2

manipulation; 3

=

=

Mam-effect for task-level

linear contrast of APM x task-level interaction; a

=

in the LST,

RC and WMP (and their interaction) were included in the one analysis along with APM, thus only one effect size is reported; bold: p < .05 .

262

Chapter Ten

As expected, or was a significant covariate of perfOlmance in all four tasks (.21 , 0.5 0 c Q) 0.4 c cr

� 0 c 0 c cr

�

�

� 0 ,

0.3

0 � 0.2

ro

�

ro

�

0.1

0.1

peo B)

pe�

Specific

00

General

>, 0.5 0 c Q) 0.4 c cr

� 0 c 0 c cr

�

�

� 0 ,

0.3

0 � 0.2

ro

�

General

ro

�

0.1

pe,

0.1

peo

Figure 16-5. The relative frequencies of the number of pes retained in A) all simulated analyses of the specific and general structures and B) analyses with brain

dependence set to .8 or higher.

The challenges to PCA posed by measurement noise, general factors, and correlated factors are independent and stack on one another. This additivity can be seen in the distributions of the nmnber of PCs that were retained, depicted in Figure 16-5, for analyses with and without a general

422

Chapter Sixteen

factor and for analyses with and without a strong dependence of abilities on brain lesions. If either of these phenomena are influencing performances on the task battery under investigation, peA is more likely to fail to discriminate the true number of factors, and if both are present while other parameters of the analysis are unknown, PCA has a less than 20% expected chance of identifying the correct number of factors, let alone producing components that uniquely correlate with cross-validation measures. If peA retains too few factors, then using those scores as dependent measures for lesion­ symptom mapping will highlight areas tliat are frequently damaged together despite their having separate functions. If PCA retains too many factors, neglecting to test tlie validity of components can lead to a search for brain regions that are related to a mental ability that was never actually measured. In the cases where peA accurately recovered the structure according to cross-validation testing, the proportion of variance accounted for (R2) between component scores and latent abilities was nearly as strong as between TAMs and latent abilities, though TAMs maintained a reliable advantage. This result confitms that peA is not necessarily producing completely uninterpretable, random vectors; peA can capture a large portion of the meaningful variance under favorable conditions, but the result also highlights the importance of cross-validation for testing tlie interpretation of factors. Nevertheless, even when peA identifies the correct structure, the scores are contaminated with measurement noise and violations of assumptions, making them a less ideal measure than the simpler TAM scores. 'While the presence of the general ability was problematic for extracting accurate measures of specific abilities with both the PCA and TAM analysis metliods (Figure 16-6, middle), tlie tlieoretical knowledge tliat tliis ability is in fact general engenders a new analysis approach that removes the linear, statistical association between the estimated abilities, allowing the residualized, specific TAM scores to recalibmte toward a better measurement oftlie specific abilities (Figure 16-6, right). The simulation results challenge tlie proposal tliat PCA is an advancement in measurement approaches in the context of aphasia assessments for lesion-symptom mapping, with much simpler approaches providing more valid results. Measurement noise is a legitimate concern in aphasia assessments with tasks that sometimes have as few as 10 trials with a 50% chance of guessing correct, or that depend on other unknO\vn factors besides the intended measurement target due to test design and administration procedures. Rather than combining measures in a way that improves reliability, as averaged nonnalized scores do, peA is highly susceptible to the presence of measurement noise. Furthermore, general

Neuropsychological Models of Speech and Language

423

�o. � �

•

s

�

'"

.�

" � 0 ..

� '"

N

�

C