The Relative Effectiveness of Questions and Their Placement in Directing Study

Citation preview

THE RELATIVE EFFECTIVENESS OF * Appleton and Company, 1884, pp. 203-204*

Hinsdale expressed the close relationship be­ tween reading and study by Indicating that both were an “exercise of the mind*5 differing only in “degree11 since both arts “look to obtaining meaning from the printed page.®

To quote him further:

“But study is more than reading; it may be called intensive reading. -»•••».» Study is the use of books for the serious purpose of gaining knowledge; It looks to the mastery of a subject, or of some portion of a subject, by means of what has been written about it. More narrowly, pupils and students commonly associate study with text­ books. but the association is not a necessary on©.®5 Horn defines the reading spplicable to the social studios as including “those processes that are involved in approaching, perfecting, and maintaining meaning through the us© of the printed page,®®

Horn’s definition of read­

ing Is Identical with the aspect of study with whieh the present Investigation is concerned; that is, the processes involved in selecting, organizing and interpreting facts and ideas gained from the printed page and relating those facts and ideas to experience.

This is In agreement with

Earhart’s definition that “studying in its highest sens© is the process of assimilating knowledge, of reorganizing experience,® and she believes that this kind of studying 5H, A. Hinsdale, The Art of Study, American Book Company, 1900, p. 17.

^Ernest Horn, Methods of Instruction in the Social Studies, Part X7, Report of the Commission on th© Social Studies, American Historical Association, Charles Scribner’s Sons, 1937, p. 152.

**results in rearrangement and assimilation of ideas; in short, it involves thinking.*7

The limitation placed upon

study as used in this investigation does not imply that this is the only nor the most effective kind of study, tfsing visual aids, taking field trips, and making observa­ tions also represent effective methods of study* The difficulty in distinguishing where reading leaves off and thinking begins, is expressed by Lyman. n,.*th@ activities of reading, of thinking, and of studying are considered as three aspects of the on© process by which we learn to use the materials in printed form* All are activities of the mind* We read serious books to get ideas; w© think about them to see what these ideas mean; we study ideas and their meanings endeavoring to make them our permanent possessions and to get ready to use them in problems of our own. From the statements quoted above, it is clear that reading and study Involve thinking. But what is thinking?

What is clear thinking?

According to Dewey^ ^Thinking is inquiry, investigation, turning over, probing or delving into, so as to find some­ thing new or to see what is already known in a different light* In short, it Is quest! oning*wy Others agree with Dewey as to the value of questioning in relation to thinking*

Perrin states that ^real study

7Lida larhart, Teaching Children to Study. Houghton Mifflin, 1909, p. S. 8 R* L. Lyman, The Mind at Work. Scott, Poresman and Company, 1924, p* 14. 9John Dewey, How We Think. D. C. Heath and Company, 1933, p. 265.

Involves reflection.*1 He 'believed that one oould learn to reflect by ^learning to ask himself questions,* an under­ taking which he adds Is 11not an easy taefc**^®

Sir John

Fitch In his Jrt of Questioning wrote, *Th© art of putting a question is itself a mental exercise of some value, and Implies some knowledge of the subject in hand* ......... You are halfway to knowledge of a thing when you can put a sensible question upon It.*l* Hinsdale concludes that; *Skillful questions cause the pupil to define his facts; to clarify his Ideas; to put facts and ideas together In new relations; to compare; to judge, and to draw inferences, mental operations which develop our higher knowledge.*^ !3?h©n he adds, ltW@ must, therefore, pay more than passing attention to this art** Questioning as a factor in study and reading appears frequently In educational literature*

Horn stresses

the importance of questioning when he discusses Its followIng significant functions; 13 1 . ®Questioning is essential to the discovery, stimulation, and guidance of interest** •

2* Questioning *teaches pupils to think."

...

3 . *Questioning helps to improve the accuracy, 10

«

diaries Perrin, *A Symposium on Study in th© Ele­ mentary School,11 Education* 30, December 1909, pp. 242-44. 11

Sir John Fitch, Lectures on teaching, E* L* Kellogg and Company, 1886, p* 172* A* Hinsdale, o-p* git** p. 97. ^■^Ernest Horn, op* clt., pp* 342-43.

clearness, and. organisation of meanings and concepts .11 *** 4. *Skillful questioning develops a more active and aggressive attitude toward learning.* ... 5. *Questioning affords a basis by which pupils and teachers may appraise re­ sults. ...

flmsaft U M m Since time Immemorial questions in both oral and printed form have held an Important place in teaching.

Of

primary concern to th© teacher In Instructing and to th® student in studying Is the need for knowing the relative effectiveness of questions as an aid to learning and re­ tention*

A number of investigations have been made to

determine th© value of questions In learning.

That th®

results of these studies are not in full accord will be shown in the succeeding chapter.

i{

The general problem of this Investigation is to provide more precise Information on the value of questions as a guide to study*

This investigation, pertaining to a

limited aspect of questioning, was undertaken In response to the need for more knowledge of th© techniques for im­ proving th© study habits of pupils. j§2U|g|y||£ Purposes of the Stud? More specifically, th© purpose of the study is to

attempt

to provide answers to th© following questions:

When eighth grade pupils are directed to read a selection on© paragraph at a time and to write the questions which the paragraph answers if in their judgment It contains Information Important enough for them to remember, is that procedure, as productive of learning as when they are given focalized questions to guide their reading and study? Is there any significant difference in th© amount learned and retained when; a. The reading is preceded by a list of main questions, and b. The reading is followed by a list of main questions? c* Hie reading is preceded by a list of main questions, and d* Hie reading Is preceded by a list of main and subordinate questions? ©, Th© reading is preceded by a list of main questions, and f, The reading is followed by a list of main and subordinate questions? g* The reading is followed by a list of main questions, and h* The reading Is preceded by a list of main and subordinate questions? I# Th® reading Is followed by a list of main questions, and j* Th© reading Is followed by a list of main and subordinate questions? k* The and 1* Th© and

reading is preceded by a list of main subordinate questions, and reading Is followed by a list of main subordinate questions?

$r@ there any significant differences between the amount of information gained by the use of ques­ tions as indicated in 1 and 2 above and the amount gained as a result of a single purposeful reading?

Chapter II REVIEW OF RELATED RESEARCH A review of investigations in the field of study and more recently In directed study suggests the complexity of th© problem and emphasizes the intricate relationships between reading, studying, and learning*

Th© extent of th©

research pertaining to these three areas indicates that educators are becoming more conscious of the need for dis­ covering effective methods of learning and for making prac­ tical application to classroom procedures* The problem of directed study has been attacked from many angles and by various methods of research, in­ cluding questionnaires to both pupils and teachers, inter­ views with pupils, inspection of pupils* themes, analysis of pupils* schedules, observation of pupils* activities, and the use of tests*

Symonds^ questions the value of the

first two procedures and also indicates that the "inspec­ tion of finished products** is definitely limited in scope* H© believes that testing Is the best method of investiga­ ting study*

Because many of the procedures which have been

used are highly unreliable, and the results of different investigations concerning the effectiveness of the same technique have been conflicting, conclusions relative to th© present status of directed study can b© drawn only with reservation* *P. M. Symon&s, ^Methods of Investigation of Study Habits,** School and Society. 24, July 1926, pp. 145-152.

Present Status

Stud? Techniques Other Than Questioning

Single reading, rereading, summarizing, outlining, notetaklng, us© of study guides, and questioning are the practices in directed study which have been investigated most frequently*

Results of a few representative studies

In the first six categories will he briefly stated in this chapter^*

Research with reference to questioning will be

presented in more detail with a view toward showing th© need for the present study* Effect of On© and Two Presentations Ffcom th© viewpoint of teaching, the question of the effectiveness of a single reading is most pertinent* In general, pupils and adults read material only once*

How

efficient Is a single reading as a means of gaining infor­ mation?

Does Its effectiveness depend upon th© person

reading, the purpose for which th© reading Is being done, and th® material being read?

Is a second reading signifi­

cantly superior to a single reading?

Is th© reading of

related material more efficient than rereading the same materials for the same length of time? 2For further studies see bibliography under "Directing Study" by G* W* FIearning in Encyclopedia of Educational Research, edited by Walter Monroe, The Macmillan Company, 1941, pp. 406-407. Also see bibliography under "Reading" by W* S* Gray in th© same publication, pp. 925-926* A* E* Traxler, Ten Years of Research in Readings Summary and Bibliography. Educational Records Bureau, New York, 1941, Is also a good source.

Yoakasa®, experimenting with pupils In grades four five, and six sought the answers to some of these questions Using types of reading materials common to th© elementary grades, he found that as a result of a single undirected reading the amount of information gained by pupils at that level varies with the individual, but on the average, less than half of the ideas presented in an article and some­ times as little as one third or less were learned. and Jones^ concur in this finding.

Bietss

According to Yoakam* s

research, the relatively small amount learned is rapidly forgotten, "in some cases having apparently no effect after a month’s time."

The effectiveness of a single read­

ing varies with th© type of test used.

"Almost any kind of

result might be obtained by varying th© type of test and th© system of scoring the answers to th© test."

The grade

level Is also a factor, a single reading being more effec­ tive for pupils in the upper than in the lower grades. For scan© pupils the single reading is as effective as "severe study" might be for others. Yoaksm^ also compared the single reading with two consecutive readings and reported that a second reading A. Yoakam, The Effects of a Single Heading. Studies in Education, Vol. II, lo* 7, March 1924, University of Iowa, Iowa City, Iowa. 4 » Alfred Dietz and Georg© Jones, "Factual Memory of Secondary School Pupils for a Short Article Miich They Head a Single Time," Journal of Educational Psychology. 22, November and December 1931, pp. 586-598, 667-676. C G. A. Yoakam, ££. olt.

adds 'but little to the first.

In this respect his findings

differ from those of Good , who concluded that ”two readings at normal rat©11 prove definitely more effective ?fin terms of the information-test scores than one reading at normal rat©.*

Good takes the position, however, that sine© on©

reading resulted in a fair mastery of th© content, reading a new selection relating to th© same topic is better than spending th© same amount of time in rereading.

Good gener­

alized on th© basis of two very.short selections.

Statis­

tical evidence as to the superiority of rereading is not presented In his article* In general agreement with Yoakam1s evaluation of 7 a single reading is McKee* s rbport of a study on th© same problem.

After a preview test had been taken, three short

selections were read by on© hundred fifth, sixth, and eighth grade pupils*

Three tests were given, one immediate­

ly after reading, one a day later, and one again two weeks later.

HcKe© presents data to show that,

ttTh@ single undirected reading secured very inferior comprehension and retention of the mater­ ial read although it enabled the children to learn something. 2} As reading age, intelligence, and grad© level advanced, the value of the single undirected reading Increased** ^Garter V. Good, ”Th© Effect of a Single Heading Versus Two Readings of a Given Body of I*!at©rial,” Journal of EducaMi?m i H&fehQii, 6 , April 1926, pp* 325-529* ^Faul McKee, ”An Investigation of th© Value of the Single Reading as a Tool,” Journal of the Colorado-wyoming Academy of Science. Vol. I, Bo. 6 , June 1934, pp. 34-35.

li

In a subsequent experiment In which one group read without directions and another group read under the guidance of O questions, McKee concluded that *the directed single read­ ing was very significantly superior to the undirected single reading.

The superiority increased as reading age

and Intell igenc© advanced. After reviewing research on the problem, Stroud^ expresses the opinion that, ®effactive studying is an ef­ fortful activity*

One of th© most ineffective methods Is

reading, merely reading.® Summarizing To date there has been blit little research to justify th© claims which have been mad© in behalf of oral and written summaries, which have long been advocated as effect!v© methods of study.

German©^, experimenting with

a corrected summary technique, compared rereading with the writing of a summary in outline form, and then correcting the outline by glancing through the selection a second time 8 ibi a .

®James B. Stroud, Introduction to General Psychology. Prentice Hall, 1938, p. 535* ■^0. B. Germane, The Value of Summarizing in Silent Heading as Compared with Rereading of the Same Article. doctor1s' dissertation, State University of Iowa, Iowa City, Iowa. Also ^Th© Value of the Corrected Summary as Compared with Rereading of th© Same Article, ' Elementary School Journal. 21, February 1921, pp. 461-464.

and making revisions and additions*

The stannary was written

from memory after a single reading*

He found that the re­

reading group excelled In each grade tested, and concluded that^relative value of a corrected summary as a method of study Is seriously questioned*®

The author warned against

generalizing too confidently from his results *slnce this experiment was conducted in only one school and since only one type of reading material was used**

His analysis of

the summaries revealed that many of the pupils resorted to indiscriminate note-taking*

This Implies the desirability

of developing skill in summarizing before trying to deter­ mine th© effectiveness^of that technique In directing study* 11 In a controlled summary experiment, German© presented evidence to show that the mental summary made in response to a list of focalized questions prepared over the material ^relatively Increases the efficiency as compared with rereading**

This conclusion may be challenged Inasmuch

as the questions appearing in the tost were the same as th© study questions which were employed to direct the reading. Newlun^, as a result of his experiment with fifth grad© pupils in five Hew York City schools, concluded that •^Ibid*. Also, *Th@ Value of the Controlled Mentnl Summary as a Method of Studying,® School and Society. 12, December 1920, pp. 591-593* *2C. 0. Bewlun, Teaching Children to Summarlze In Fifth Grade History. Contributions to Education, No. 404, Bureau of Publications, Teachers College, Columbia Univer­ sity, Hew York, 1930*

pupils at that grad© level can be taught to summarize.

The

author 3tates that, "Summarizing used as a method of study In history does not guarantee an increase of achieve­ ment In history over ordinary study procedure, if the skill is not developed to a sufficient degree, even though the children have a significant amount of th© skill»w And, "The us® of summarizing as a method of study in history will not "be likely to affect achieve­ ment In reading In any significant extent*® Taking Hote3 and Underlining Perhaps the difficult task of controlling the many variables explains the scarcity of investigations as to the relative value of note-taking, which Is one of the most commonly practiced study procedures.

Dynes*'-5 briefly

reports a study which showed that "taking notes, outlining the material, summarizing tho data, and reviewing® to be slightly superior for immediate recall and superior for delayed recall as compared with reading and rereading. Mathews*^, in an experiment with 735 students in grades nine through twelve in ten different schools, had one group read and make no notes, a second group read, underline, and make marginal notes, and a third group read *^J. J* Dynes, "Comparison of Two Methods of Studying History,® Journal of Experimental Education. 1, September 1932, pp. 42-45. *^G. 0. Mathews, "Comparison of Methods of Study for Immediate and Delayed Recall," Journal of Educational Psychology. 29, February 1938, pp. 101-106.

and make outline notes.

A multiple-choice and "an outline

scheme® were used to measure the results.

He found no

statistical differences between the methods employed, but the first group, that is, the one which simply read, made consistently higher scores than the other two groups*

After

suggesting that th© "differences are not entirely reliable,® the author reporteds "it appears that when the time was kept con­ stant merely reading the material with intent to understand and remember was more effective than underlining and making marginal notes or outlin­ ing for immediate recall, and more effective than underlining and making marginal notes for delayed recall. Outlining the material studied was raoro effective for the delayed recall at the ninth and tenth-grads level, but merely reading the mater­ ial was more effective at the eleventh and twelfthgrade level. Of course, since time was kept con­ stant those who did not make notes or outline th® material had more time to read or repeat the read­ ing of difficult passages.® Outlining In 1918 V&iipple*® wrote, "Yfitenover you desire to master material that is at all extensive and complex, make an outline of it*

If you wish to retain this material

commit your outline to memory."

In a few studies outlining

has commanded th© center of attention.

Frequently, howevery

it has been treated as one phaso of Investigations comparing

1 5 Guy M. Whipple, How to Study Effectively. Public School Publishing Company, 1927.

th© effectiveness of several methods of study."^*

limit,

With 346 college students as subjects in "time19 "work-limi t" experiments, Crawford compared the

effectiveness of outlining, a single reading and two read­ ings*

He explained that, "The lack of significant differences between the three methods is not due to the lack of learn­ ing by any of them since each method reveals sign­ ificant superiority over the results of the *no study1 group."

To quote him further, "The most striking fact revealed is that there is no significant difference between reading once, reading twice, and outlining as regards results* The methods are so nearly tied that it would ap­ pear to be about as well to^read twice as to out­ line, and almost as well to*read once as to do either*" The investigator had only a measure of the immediate results of his experiment and therefore suggested the possibility that a delayed tea** might have yielded different conclusions. on

Qhifce different results are reported by Simpson**' who compared the value of training in outlining, answering questions, evaluating, and summarizing at the upper ^German®, on. Git* Also, "The Value of the Controlled Summary as a Method of Studying," School and Society. 13, June 1921, pp. 730-732* 17 Dynes, op* eit. ^Mathews, op* cit. *»0. G. Crawford, "Relative Values of Reading and Outlining as Methods of Study," Educational Method. 8 , May 1929, pp. 433-438. 20Robert G* Simpson, "The effect of Specific Training on Ability to Read Historical Materials," Journal of Educa­ tional Research. 20, December 1929, pp. 343-351.

elementary grade level.

Historical materials were used.

Outlining was found to be superior to the other methods employed.

Simpson recommended that,

"...if intermediate grades are taught to organise their materials of study before attempting to answer questions about the context, their ability to read will be improved considerably more than if they pursue their reading and study in reverse order,* A very Inadequate account of the statistical treatment is given in th© report of the investigation.

The time vari­

able alone in this experiment would limit the Implications of the study* Pi Barton conducted three experiments to determine the effectiveness of outlining in history*

He found a

probability of *9999 that the gains made by the groups using outlining were greater than could be attributed to chance, a finding that has questionable validity because th© duration of each experiment was not the same, nor was the time spent by the control and experimental groups held constant.

To draw his conclusions, th© author combined the

gains of the separate experimental groups and of the control groups*

This was an unwarranted procedure in view of th©

design of the experiments and th© variables introduced* According to Salisbury2^, giving students in grades seven and twelve special lessons in outlining resulted 21W. A. Barton, Outlining as a Study Procedure. Contri­ butions to Education, Ho* 411, Bureau of Publications, Teachers College, Columbia University, Hew York, 1930*

22Rachel Salisbury, "Some Effects of Training in Out­ lining," The English Journal (College Edition), 24, February 1935, pp. 111-116.

19

In decided improvement in reading skill, in ability to solve reasoning problems, and increased success in content subjects in which outlining may be used,

These findings

O '*

are in agreement with those of Wright

, who experimented

with pupils in grades four through seven. Stud? Guides Studyguides, which include printed or mimeographed questions, problems, test exercises, and directions for study, have been subjected to investigation,

llamren2^, in

collaboration with Crawford, reports an experiment in which study guides were used with two groups of twelfth grade students.

The experiment lasted for ten weeks*

At th© end

of five weeks the procedure followed by each of the two groups was reversed.

The study guides wore in printed

form, mad® up primarily of discussion and enumeration ques­ tions,

To determine the results, a test of brief-answer,

multiple-respons®, matching, completion, and true-false items was used.

A comprehensive test was given two months

later to measure th© amount retained.

Results, though not

statistically significant, uniformly in favor of study guides were reported*

However, after presenting a compre­

hensive list of arguments for and against the use of printed 2*\ouise B. Wight, "The Value of a Motivated Assign­ ment: An Experiment in Directed Study,” School of Education Journal, Vol. 5, So. 2, December 1929, University of Pittsburg, pp. 64-67* 24C. G. Crawford and L. H. Hamren, "Experiment with the Use of Printed Study Guides,” Educational Method. 9, June 1930, pp. 541-544.

study guides, the authors state that they are undecided as to the true value of this technique of study* In research conducted by Eskew^, four groups of 7A pupils were taught for on© semester by ti*ro alternate methods by two different teachers*

Two groups were provided

with assignment sheets made out by the teachers.

These

sheets contained a preview of the chapter to be studied, directions on how to study the chapter, and guiding ques­ tions over it*

The other two groups were given brief oral

assignments which included the chap tor number, the number of pages, and the directions, nW@ will study this chaptor beginning at the first of this chapter for

minutes*"

Twenty-four chapters In Rugg’s Changing Civilization in a O ft

Modern World ° were covered*

The study of each chapter

was followed by an objective test of true-false and comple­ tion items*

Six-week tests and a final test covering the

twenty-four chapters were administored*

The author reports

"there Is a positive drift or trend as Indicated throughout by the slightly higher scores" in favor of the group using the study guides*

Eskew also related that according to

the teachers, the group using the study guides did better 25Philip 1* Eskew, "jg Experiment ±a Betormina Ihe Relative Values of the Assignment Sheet and the Page or r Chapter Assignment in a Seventh Grade Social Science Class, master’s thesis, State University of Indiana, Bloomington, Indiana, 1933* S^Harold Kugg, Changing Civilizations in the Modern World, Ginn and Company, 1930.

21

in class recitation, made more frequent use of the diction­ ary, and presented fewer disciplinary problems* nr*

Weaver

, who studied the relative value of in­

tensive study and extensive reading in Unitod States history, concluded that seventh grade pupils gain as much understand­ ing when they read extensively ”from carefully selected accounts of a subject” as when they study wa few carefully selected accounts Intensively with prepared exercises focused on these accounts*” Teaching How to Study In the two decades following the publication of McMurry’s How to Stud?^® r manuals designed to present patterns for study became popular.

Claims for the sugges­

tions contained in these manuals were soon seriously ques­ tioned and subjected to investigation. Coll29, who made a limited survey of the current writings on the problem of how to study, ©specially as they have appeared in books, came to the decision that the experimentation, as carried on in a number of colleges and 27R* B. Weaver, "The Relative Value of Intensive Study and Extensive Reading in United States History,” The School Review. 39, March 1931, pp. 217-226.

28FTank M. McMurry, How to Study and Teaching How to Study. Houghton Mifflin Company, 1909. 2% o h » Coll, m Analytical Study of Modern Contributions on the Problem of ^How to Study . master* s thesis, Univer­ sity of Pittsburg, 1935.

universities, has shown that courses in how to study are practical and produce beneficial results. Some of the research which Coll did not include in his summaryis not so optimistic*

In an experiment by

Batterwecte^, the subjects were divided into three equated groups of 19, 20, and 20 members*

On© group received a

^course in best method of study based upon Whipple* a How to Study Bffeetivq Iv ^X. Lyman* s The Mind at Work^S. and others similar books*” They, however, received no practice in how to study* A second group was given Instructions as to the purpose of

study and definitepractice exorcises In

how to study based upon the principles presented In the books mentioned above*

The third group hod neither instruction

nor practio© In how to study*

The experiment ran for one

school year during which time, in addition to regular classwork, the first two groups followed the prescribed study procedures for two forty-five minute periods each week* All groups were studying geometry and science*

In general,

the group which had practice material did the best, but the author found no statistical differences in favor of any on© group*

In his opinion, the work was most valuable

30J . 5 * Biitterweek, The Problem of Teaching: iilrdi School Puoils How GontributIons to Education, No. 257, Bureau of Publications, Teachers College, Columbia University, Hew York, 1926* slWhipple,

op*

Qlt.

32r o 11o Lyman, The Mind at Work in Studying. Think­ ing, and Heading* Scott, Poresman and Company, 1924.

for the pupils of low ability, had no effeGt on the pupils

of average ability, and had a slightly negative effect upon the superior pupils*

Gilliland

, using tenth grade English classes,

attacked the problem In much the same manner.

In his exper­

iment one group was given instruction in what constitutes desirable study habits and information on how to acquire them*

This instruction was given during on© period each

week for twenty-six weeks. taken from Whipple*

fhe principles of study were

and Kornhauser*

manuals.

Accord­

ing to the Investigator, "One is forced to conclude that Instruction In how to study does not function in producing better school work* It may be that some benefits accrue to those receiving the instruction which are not detected by the measures used In this study." The problem of evaluating study helps was attack­ ed from a different angle by Deich and Jones36# They inves­ tigated the study habits of able and successful pupils* Their findings, which agree with those of Webb'^, showed * E* Gilliland, Jgi Experiment to Determine the Efficiency in How to Study, master's thesis, State Univer­ sity of Kansas, Lawerence, 1927* Whipple, op* pit* S5 A* W. Kornhauser, How to Study. University of Chicago Press, 1924. S6C# Doick and E* F* Jones, A Study of Distinguished High School Pupils in Iowa. Bulletin No, 46, United States Bureau of Education, Washington, B. G», 1923* ^ L. W, Webb, "Students' Methods of Studying Psychology," Journal of Educational Psychology. 11, April 1920, pp. 193-206.

that the procedures emphasized in rules on #how to study” do not differentiate between successful and unsuccessful students. Symon&s

■SD

has made a pertinent statement on the

authenticity of “how to study” suggestions. “Writers on *How to StudyT in the past have derived rules on how to study by the method of arm chair meditation or by resorting to problems uppermost in the minds of educational psychologists at the time they wrote.” And in regard to Whipple1s rules in particular, he comments, “I prefer the explanation that these rules were not the ones that matter much in school success. This explanation is seconded by the results of systematic observation in which it was evident that many of these rules did not even have an opportunity to he exhibited in ordinary modes of study required in school.....While on© would not deny the fact that ail these rules are factors in efficient study one may question Lheir relative importance*” That more than thirty-eight Mhow to study” manuals were published between 1626 and 1939 gives some indication of the importance with which the problem of developing study habits and skills is viewed.

It is interesting to

not© that these manuals, like the older ones criticized by Symonds, include many suggestions not based upon research and are not in agreement as to what constitutes effective study habits.^® ^Symonds, cm* oit. ^ S . B« Laycock and D. E. Russell, ”An Analysis of Thirty-eight How-tc-Study Manuals,” School Review. 49, May 1941, pp. 370-379.

Strang4^ has pointed out two difficulties in determining the content of “hew to study” courses: notably, the skepticism of some educators a3 to the possibility of teaching how to study and the lack of knowledge as to what constitutes effective study*

As a result of her Investiga­

tion, she emphasizes the frank discrepancy which exists between “the methods students think are best and the ones they use.” Supervi sed Study A number of Investigations have been made to determine the general effectiveness of supervised study as compared with undirected study*

Discussions of supervised

study as they have appeared in periodicals have dealt for the most part, with the organization and aduiinistrati on of supervised study, the description of particular plans in operation, the technique of supervising study periods, and the evaluation of the results of supervised study*

Since

many of the studies havo introduced such factors as pupil failure, disability and maladjustment in particular sub­ jects, report cards, and school marks, they are net pertin­ ent to the present investigation and are not discussed in this chapter*

For Information on this aspect of study see

40Buth Strang, “Another Attempt to Teach how to Study,” School and Society* 28, October 1928, pp* 461-466.

Brownell4'*', Woodring and Fleming4^, Gates4

Breed44, and

Heckert45. Questioning as & Stufty Technique In one of the earliest and most comprehensive studies of questioning, Yamada4® discusses the value of oral questions.

This ©valuation Is applicable to questions of

the type used in this study.

According to Yamada:

“The effect of a question or theme or recollection is two-foldj it offers a key to open the storehouse of the latent ideas which otherwise may remain continuously submerged, or It may draw the observer* s attention to a gap In his knowledge so he attempts to fill it with an answer of some sort4 And this done quit© often from an irresistible impulse, sine© ©very question, even the most Indifferent, the most cautious and the most unsuggeative, is more than a mer© question. It la a command for a recollection and the production of a certain answer.” 4^W. A. Brownell, A Study of Supervised Study. College of Education Bulletin $0 . 26, Bureau of Educational Research, University of Illinois, Urbane, 1925.

4% . S. Woodring and C. W. Flemming, Directing Study of High School Pupils. Bureau of Publications, Teachers College, Columbia University, Hew York, 1935, 43 Arthur I. Gates, Th© Improvement of Heading. The Macmillan Company, 1935, pp. 636-652. 44F. S. Breed, “Measured Results of Supervised Study,” School Review. 27, March and April 1919, pp. 186-204, 262-284. 4 ®J, W. Heckert, “The Effects of Supervised Study In English Composition, Journal of Educational Research. 5, May 1922, pp. 368-380*

46Soshlehi Yamada, “A Study of Questioning,” The Pedagogical Seminary. 20, June 1913, pp. 129-186.

McMurray and Parkins4? in the preface to their Advanced Geography wrote3 “Good study, or good thinking requires first that live questions be raised for consideration; then that facts he collected, evaluated and organized until forceful and valuable conclusions be reach­ ed. That is what all efficient people in active life do when they think; and one of the chief duties of the school is to accustom children to that procedure.” Considering the preceding quotations, which- are representative of those appearing in discussions of the value of questioning, it is interesting to note that Mathews and Toepfer4® In their analysis of rules appearing in five widely used “how to study” manuals, make no mention of the us© of questions except for the suggestion, “try to antici­ pate examination questions.” haycock and Russell4®, in a similar analysis of thirty-eight manuals, found “ask and answer questions” given in ©igiht of them*

“Us© your own

questions while reading” appeared In five of the manuals* Graduate students In education and psychology have long recognized the importance of questioning and have endeavored to determine its effectiveness as an aid to learning and retention study.

Several of these investiga­

tions will b© reviewed in this section in an effort to point 47 F. M* McMurry and H. W. Parkins, Advanced Geography. preface, The Macmillan Company, 1921. 4®C. 0. Mathews and Nora Toepfer, “Comparison of Prin­ ciples and Practices of Study,11 School Review. 44, March 1936, pp. 184-192. 4®Laycock and Russell, op. cit.

28

out that sufficient and dependable evidence is not at pre­ sent available to justify sweeping recommendations as to the us© of questions in directing study*

These reviews

will reveal the need for the present investigation. One aspect of G e r m a n © * s t u d y which consisted of three experiments, was termed the controlled summary, which he secured by th@ use of focalized questions.

In

the first experiment involving one hundred pupils in grades six, seven, and eight In on© city school system, the experi­ mental group read the selections and then were given ques­ tions to which they were to write answers.

The control

group read and reread the selections for the same length of time as required by the experimental group*

The study

questions used by the experimental group also served as the criterion test*

The author concluded from the results,

expressed in terms of per cent of gain, that writing answers to questions following reading ^increases the efficiency as compared with rereading,w and wthe more difficult the material, the greater ia the gain in efficiency.”

However,

there w@r© no data to show whether on© of the selections was more difficult than another.

It Is also possible that

th© questions constructed by the author varied In difficulty from selection to selection* 4 somewhat similar experiment was conducted in an­

other city school system with sixth, seventh, eighth, and Germane, $33. cit.

29

ninth grad© pupils* tions before reading*

'Hie experimental group was given ques­ The pupils in this group had two ad­

ditional paragraphs relating to the topic of the selection which were not available to the control group*

The purpose

of these paragraphs was to serve as a **warming up* exercise* Because of this extra reading, the experimental group was allowed three additional minutes*

Again presenting the

results in terms of per cents, the author concludes that the use of questions before reading is more efficient than rereading* Part III of this phase of Germane*s study consist­ ed of two experiments to determine the effectiveness of the ^controlled mental summary,* which meant ^mentally finding the answer to the test questions already placed in the hands of the students, !ias compared with the rereading of the article for the same length of time*

In the first

experiment two classes of college students were the subjects, and In the other, fifty-six pupils in grades six, seven, and eight.

The criterion tests consisted of the questions

which were used to guide the reading*

For both experiments,

the author states that the use of specific questions on the assignssent was more efficient than undirected study. That Germane*s criterion test consisted entirely of the identical questions which directed the study of his experimental groups, that ho presented no evidence as to the significance of the differences obtained in favor of

30

the experimental groups, and that M s samples were relative­ ly small place strict limitations upon the implications of this study.

Germane recognised that his conclusions ?/er©

by no means final. Using different types of materials and different 51 study techniques, Distad reached the following conclusions with respect to reading performance: 1 . aIf the purpose is to measure the Immediate recall on the entire content of a selection aft er a single reading, it appears doubtful whether types of reading directing attention to the content can he justified In view of the relatively large amount of time required for reading.

2. ^Reading with a problem or with questions may be used when definite Information is desired. Mien thus used, directed types of reading are Intrinsically worthwhile in that they develop habits of reading effectively for different purposes * These conclusions are based on an experiment in which ten VI A classes, {ns:25G) divided into five equated groups, took part.

Four types of reading material, geog­

raphy, nature study, narrative, and poetry, were selected. For each of the selections, which were highly factual, a test of thirty items, each of which covored ”one specific fact and could be answered with one word** was constructed. The four study procedures and the four reading selections were rotated according to the plan presented below: S^E. W* Pis tad, nA Study of Reading Performance of pupils Under Different Conditions on Different Types of Materials,” Journal of Educational Psychology. 18, April 1927, pp. 247-258.

-

Grouna

Geo/rraohi cal

A B C JO E

U V m p SQ

Reading Materials Marr ative Nature

P oetrv U m P SQ U

IF P

0 m u m p

0 RQ

Reading Procedures were: U - Undirected; BQ - %>©clfic questions; RQ - Raised questions; P - Problem, Group A as the control group used the ^undirected procedure1* throughout the experiment*

ttTh© pupils were

told to read the selection through once, and then to an­ swer the questions*1* Bach of the other groups follo\?e& this procedure with on© of the selections*

Under the

"raised question procedure* the class was asked what they would like to learn ©bout the topic under consideration* ”Bight questions suggested by the pupils were written on the blackboard*”

After reading and finding the answers to

the questions, the pupils took the test*

In the Mproblem

procedure11 th© pupils were directed to read to 71Find the answer to th© problem which I have written on th© black­ board**

The problem was Ma general thought question*71 The

tost followed immediately after reading*

Bight mimeograph­

ed questions were used to guide the reading under !ispecific questions procedure*”

After th© pupils had read to find

the answers to the questions, they took the test*

The pupils

for all procedures recorded the time required to read each selection.

Adequate time was allowed to permit each pupil

to finish the reading at his own rate*

Th© average means and the average time In seconds for Immediate recall for each procedure are presented belows proce­ dures Geosraohv R T UU) u

m p

14.£ 11.8 15*0 14*3 13*0

147.0 136*4 160,2 157.4 218.4

Mature

k 13*0 16.7 14*9 15.7 14.4

T 142.4 147,4 166.6 163*4 174.8

Harrative f R 18.1 19*0 21.8 19.5 21,5

174.0 179.0 188.0 162*0 206.0

R

Poetrv T

14,8 17*4 17.5 18.3 16.7

128.2 14 (;.8 135.4 131.2 163.0

Averse© K f 15.1 16*2 17.3 17,0 16.4

R - Recallj T ~ Time5 U( A) « Undirected procedure used by Group A. This table reveals that in terms of means, the ^undirected procedure** Is the least effective of the methods tinder con­ sideration*

However, the differences between the average

means for th© undirected procedure when used by th© control group (Group A) and when used by the other groups is greater than the differences between average moans for th© other procedures * Th© author presents a table showing th© ratio between th© means and th© standard error of the means for each group with th© different types of material*

Th©

statistically significant differences taken from this table are presented below*

147.9 150.9 162.6 153*5 190.6

fy « ©OP

© P Si P

©p

O & P

© ©

£

H O

P

I

H

Si Si P

O

H H

p

o

p

*d

£

H ©

©

P

p

©

£3

©

c

c

H ©

©

1

H H ©

©

©

H W

H K

Vl o

Vi g O©

d>

p

p ©

o © Cl & & ©

O P

Op

H H >

ra

P ©P P P G

wo

Op pp s © o «

OP

H M M

Vi O

© P P

to

o

p

© by © Q: P

©

G G O T* G

©

by

©

H

© o

©

Vj

1 53

I© H«fi O +5 ©

questions at beginning of stony questions interspersed at beginning of appropriate paragraphs questions interspersed at end of appropriate paragraphs all questions at end of story

43

had had had had f Y Z X © bQ

S j

*3

as

m

©

*grt

5> fiw M © 43 42

d ©

0

'dP b ©

#-4 A O ©

© X2

d 42 43 ©

@ O © 43

2

© 43

43 d

si 43 jd

O AS

O 43 -o

© d ©

d

©9 © 0 d 0 **H

d O rH £

o 43

©4 d O © %h ©©

1 © © q jjo i© 0 d w xj

ts)K

d }>a © O

S SS Q A *H43 tc © rH H 0 S © © *H S>

©df-443

© d f> © ©Pi

d

X? d «s 43 O «r-i d *H

X3 ©

43

d ^ *H Aj

t>

43 g

P O

AO

© P rHd » rH © O d < 4-4 © ** JiU d O O &H rlB 43

c~

© rH J>u d rH © 0 rH > «c5 «S © «H © XI > d «K >i*d © © d

©

H

©d

© «d«

A

©

«ri*rt

d

j>

as

X!

*• O

© «H 43 5au c8 d E3

o g

©

m

s

& O

85

43© ©• © rH © d O OO O CS 43

rH Ae

Xi *P

©ij ©

*H « p* d © d d & «s «H © & rH m © fd_d ©

©

d A

43

>s

43

cg c3 t3 © flrlXJ©

0O43|-4

0*«H

0

tsXI d

j>» © © £s «! 0 © o

43 ~ 43 .H

6Q ©■

©

TJ 43 X5

O vl © rH»H>4 A Xi 0 0 aS © ©

rH *H ffiO >s

0 © *H

43 v4 © rH

43 © O 0 XI « S3 aSts

d

"£

©

M

43

43 43*rl

o

g

d ©

ra0 *

tA

SBOH

o ©O ©

«

H

OJ

d

54

Difficulty in determining how Washburne distin­ guished between a factual and generalisation item was experienced*

Th© author* s frequent statement that items

in the test repeated verbatim or nearly verbatim the ques­ tions in the reading selection raised doubt as to the valid­ ity of these items as measures of recall of facts and recall of generalisations* The author states that ****In th® two experimental elements —

experimental facts and ejqperimental generalisa­

tions* there is a statistically reliable difference between the forms with and the forms without questions*’* She table of probable errors show this to be true only for f (questions at th© beginning of the story) in Element I and T and Y (questions interspersed at the beginning of approximate paragraphs) in Element II.

It is not ture for any of th©

forms in terms of the total test, Element IX* '501© author further states that *In every element In Ts&tloh questions in the test are related in any way to questions in reading (i.e*, in all but Elements I and fll) Forms f and Y are superior to Forms X and X**5 Hois is true but the differences are in no case statistically significant* The small number of test questions representing each element limits the confidence which can be placed in these differ­ ences*

Certainly, th© results on Element IX, which la from

two to over ten times as long as the other elements in the test, should be the most reliable*

A glance at the table

of probable errors will show that on the test as a whole, there are no statistically significant differences either between the experimental groups and the control group, or between the different experimental groups* In an experiment which lasted five weeks, McIntosh^0, using seventh and eighth-grade geography and h5story classes in two schools, investigated the effects of objective guide questions and self-checking answer sheets upon performance in reading and learning*

The classes were divided Into what

the author terms control, semi-experimental, and ejqperimental groups. way*

The control group continued to work in the usual

Th® semi-experimental group were given mimeographed

copies of objective questions, to guide their study in each assignment and ”perfo-score” answer sheets which were, ”...answer sheets separated from the objective questions, on which symbols are provided, corres­ ponding to the alternatives on the objective questions* The student encircles with a pencil mark the symbol which represents his chosen answer*” The experimental group had the same set of objective questions as the s©mi-experImental group, but they had "ohemo-score* self-cheeking answer sheets containing **#.moisture sensitive inks, which turn blue if the student moistens the spot corresponding to the correct alternative of an objective question, or red if he chooses the ineorroct response.11

* M. McIntosh, The Effect of Objective guide Questions and Self-checking Answer Sheet3 upon Performance In Beading and Learning* master1s thesis, Kansas State College, Manhattan, 1939*

Th© semi-experimental and experimental groups studied twenty minutes each day and then spent the remain­ ing part of th© class period using the study guides and, when necessary, rereading to find th© answers*

The results

were appraised each week hy an objective examination of from forty to fifty items*

Th© author neglects to explain

th© procedure followed by th© control group*

She does not

give th© total length of th© class periods, nor does she tell whether they were th© same length for each group* &©lns of ”fram one to four times their standard error” were reported for the experimental group, th© group using ehemo-score answer sheets on the weekly tests,

wIn

five of th© eight tests the experimental group showed stat­ istically valid gains over the control group,”

In on© school

the experimental group showed *statistically valid gains over th© semi-experimental group,”

Ther© was no statistical

difference between th© *semi-experimental” and th© experi­ mental groups on the results of the objective study guides, An investigation by Gray^ is of special Interest because It deals with a type of reading similar to that re­ quired of those pupils, in th© present investigation, who studied under th© guidance of questions*

To 491 pupils in

grades four, five, and six in throe school systems, Gray I, Gray, nA Comparison of Two Types of Silent Reading as Used by Children In Different School Grades,” Journal of Educational Psychology, 20, March 1929, po, 169176,

administered the Monroe Silent Reading Test and th© Thorndike McCall Silent Reading Test to compare th© effect of ttl) reading for the purpose of answering questions when th© questions are given before the reading is done, and 2) reading for th© purpose of answering questions when the questions are given after the reading is done*” The latter test was mimeographed with the questions appear­ ing above th© selection to which they related* Test was administered In its printed form*

Hie Monro©

The author was hot

primarily concerned with the relative effectiveness of th© two methods but rather with the pupils* ability to use th© two types of reading effectively* Th© nature of the reading process necessary to th© two methods was identified by the author*

For Method I

in which th© questions preceded the readings ”*,*11 Is advantageous to decide upon the key word or words in th© question for which the answer is to b© found* Second, It Is advisable to look through or skim th© reading material until the words or sentences are encountered which seem to relate to the key word In the question* Third, It is necessary to read in detail thntpart of th© selection where th® answer to the question Is found.” For Method XI in which the questions follow th© selection, th® reading* f,,**3sust b© accurate and detailed because th© ques­ tions may concern any or all parts of the selection. Second, success in this method of reading depends in some degree upon the anticipation of the ques­ tions to b© encountered later* Third, such read­ ing demands that details of th© selection as read be held in memory for at least a short period of time*” In comparing the two methods the author writes:

'

w**.If th© above analysis is correct* the first method can h© described as an immediate, somewhat gross process which depends upon a considerable degree of fluency, while the second method involves a more remote end and more detail** Gray, to substantiate his analysis of th© types of reading essential to th© two situations, cites studies by Hitter and hofland^, who showed that ^specialized read­ ing is acquired in the upper grades,1* by Judd and Buswell®6, who found, in studying ©y©-movements, that *as the purpose of reading changes, the method of reading changes,* and by &? Good , who found that ^mental set is an important element in *♦♦,♦ reading** The data presented reveal that th© children in the lcw@r grades seem to lack th© experience to react to the type of reading required, for in the lower grades the pupils were not able to distinguish between the different typos of reading*

There was a more satisfactory adjustment

to the reading required from th© lower to th© upper grades* The author emphasizes th© need for *training for children who are called on to deal with th© two types of reading under consideration*** T* Hitter and W* T# Lofland, *The Relation Between Heading Ability as Measured by Certain Standard Tests and the Ability Required in the Interpretation of Printed Matter Involving Reasoning,* Elementary S. ehool Journal, 24, March 1924, pp. 529-546. 66C. H. Judd and G* T* Buswell, Silent Reading: £ Study of the Various Types, Supplementary Educational Monograph, Ho* 25, University of Chicago Press, 1922* Garter V# Good, *The Effect of Mental-Set or Attitude on the Beading Performance of High School Pupils,* Journal of Educational Research, 14, October 1926, pp. 178-186,

53

Sggggagj In learning to study, as in learning to do other things, pupils have not always learned the most effective way*

The research studies presented in this chapter, while

not constituting an exhaustive list of th© investigations available in the field of directed study, indicate that identifying, evaluating, and comparing effective methods of directing study is an educational problem the answer to which has not been determined*

The investigations her©

cited confirm the recent observation of Flemming^®, who after reviewing the progress of research in this field, wrote, 11The attempts to evaluate procedures for directing learning have stimulated interest In the problem and have encouraged Increased attehtion to this phase of teaching, but have not resulted in many dependable evaluations#1* A critical evaluation of th© research studies presented In this chapter makes possible th© following conclusions: 1* The- investigations cited and numerous other studies of similar character point clearly to the urgent need for determining what effective study methods are* 2* There Is also a need for determining the relative

W. Flemming, Directing Study,1* Encyclopedia of Educational Research, Th© Macmillan Company, 1911, p* 406

60

superiority of the various study techniques for producing the desired learning* 3* Any number of Investigations confirm McCallister*s

69

findings that many of the difficulties in reading ia the subject matter fields have resulted from the pupils* methods of attack, that Is, from careless­ ness or th® failure to perform the reading activities properly*

Germane, Gray, and others felt that their

subjects were not using the study skills being Inves­ tigated with sufficient effectiveness* 4* There Is some evidence to show thnt different methods of study are not equally effective for all pupils, either from grade to grade or for pupils within on© grad®*

Nor do the various methods appear to be equal­

ly effective In all subject matter areas*

A good

technique to be efficient may have to be In skilled hands.

A good method poorly used Is not likely to

be effective* 5* A multiplicity of factors is involved in effective study methods.

Seme of these have been subjected

to rigorous experimentation* others, to superficial Investigation*

Limitations of much of this research

preclude highly valid recommendations as to the 69 James M* McCalister, ^Heading Difficulties In Study­ ing Content Subjects,” Elementary School Journal. 31, November 1930, pp.* 191-201*

best methods# a* A number of th® investigations have been too broad, the investigator having tried to prove or disprove too much at one time, b* Some investigations evidence insufficient control of variables*

More vigorous designs

of experimental procedures and more adequate statistical treatment of results would have improved their quality* Examples of these weaknesses ares 1*. varying the amounts of time spent by con­ trol and experimental groups and then treat­ ing these data for each as If th© time had been constant, 2* * grouping th© results of experiments which were not identical in order to draw general conclusions, 3f« failing to control the teacher variable, 4** falling to use appropriate statistical techniques for th© treatment of small samples. Conceivably, th® findings of these studies are in accord with the actual facts,

However, since

*70More adequate techniques are now available for treat­ ing data statistically than was th© case when some of the studies reviewed were conducted.

62

the variables were not controlled, It Is impossible to determine which factors were operating. c* Relatively small differences have been over©mphasi sed and have been treated as If they were statistically significant, d* Methods of equating groups in some investigations were open to question, e. By emphasising the measurement of factual information, many studies have neglected the desirability of pupils* seeing the whole of an assignment and thus have been concerned only with an Inferior typo of reading and study. Emphasis upon verbal statements of factual information as such has prevented the relating of new Information and Ideas to past experiences. These studies indicate a need for greater at­ tention to interpretation and application of what Is read and. studied, f* Th© true-false test has b©en employed in a number of investigations as the criterion test for determining effectiveness of study techni­ ques.

Jersild*^ and others have pointed out

the limit aliens of this instrument for such A. T* Jersild, ttExamInations as an Aid to Learning,” journal of Educational Psychology. 20, Hovember 1929, po. 602-609.

measurement• g. Some investigators have failed to define the methods employed in sufficient detail so that adequate evaluation can be made, h* In lieu of objective data, some Investigators have resorted to subjective appraisal and have reported their conclusions as if they were based on objective data. I* Occasionally, the conclusions and generalisa­ tions reached have not been warranted by the data presented. 6* The literature reveals that few studies have been conducted at th© lower grade levels to determine whether or not various methods of study which have been recommended can be taught in those grades and to determine the relative value of different techni­ ques for younger children* ?* Ther© seems to be general agreement that a single undirected reading results in a relatively small amount of understanding*

The effectiveness of the

procedure, however, varies with the person doing the reading and the material read as well as the purpose for which the reading is done. 8* Although th© experimental data concerning the effectiveness of questions as an aid to study are fragmentary and incomplete, there is evidence of their value*

Chapter III MATERIALS AMD PROCEDURES

The materials in this study consist of the Iowa Every Pupil Test of Basic Skills for 1936, Test A, Silent Reading Comprehension^. an article on tularemia adapted fro© a bulletin by th® same title published by th© Iowa State Department of Health % a list of seven main questions a list of the same seven main questions with subordinate questions* a sixty-item multiple-choice test based on the reading selection; and a ten-page article nBskimo Year*’1 Which was used as extraneous reading by Group X*

Copies

of th© materials discussed in this chapter may be found in Appendix B. One thousand forty-five eighth grade pupils in thirteen schools, representing seven school systems In Iowa participated in the experiment*

Within each school, the

pupils were divided into six groups by means of the table of random numbers.

All groups took the Iowa Every Pupil

^"The 1936 Iowa Every Pupil Test of Basic Skills for Grades 6, 7, and 8, Test jij Silent Reading Comr?rehensIon, prepared under the direction of Ernest Horn, Maude McBroom (Reading, Basic Study Skills), distributed by Bureau of Educational Research and Service, Extension Division, State University of Iowa, Iowa City, Iowa. SIowa Public Health Bulletin, Tularemia, published by the State Department of Health, Des Moines, Iowa, Vol* LIV, No. 3, 1940. 3E. F. Lindquist, Statistical Analysis in Educational Research. Houghton Mifflin Company, 1940, pp. 262-264.

Reading Test, which hereafter will be referred to as th© Initial Test,

After following their directions for reading

and study, all groups took the test over tularemia*

Here­

after this test will be referred to as the Immediate Test* One month after th© first part of the experiment had been conducted, all six groups took the same test over tularemia again*

This test, which was used to measure delayed recall,

will be referred to as the Delayed Test. Table 1 DESIGN FOE GATHERING DATA Grouos XII IV

Schedule

I

XI

Jan. 7

I

I

I

Jan. 8

T*

f

Jan. 8

0*

Jan. 8 Feb. 8

V

VI

I

X

X

T

T

T

T

2*

3*

4*

5*

6*

X

G

C

C

C

G

jp*

D

D

z>

D

D

1-Iowa Bvery-Pupll Test of Basic Skills, Test A, 1936 f~Ariiele on tularemia C-Test over article on tularemia 2-6 Directions and questions over the selection on tularemia X-An article entitled, ^Eskimo Year* used as extraneous reading material for Group I D-The same test as G, only used to measure delayed recall Materials prepared by the investigator In the preparation of all of the materials, an effort was made to require a higher level of reading and

learning than characterizes much classroom work and some experimental studies*

Th© preparation of the reading

selection, th© wording of the questions and tost items, and th© relative number of questions and test items built for each understanding were aimed at putting the insignificant facts and Important ideas in proper relation to one another* It was hoped that the pupils in order to gain an adequate understanding would have to select, organise, and appraise facts and Ideas while they were reading and studying th© selection.

The experiment was designed to measure the

relative effects of different uses of questions for study*

|g||jyyy|jyy^ gf th£ Since th© emphasis in this study was upon the acquisition and retention of understanding rather than of verbatim facts, th© selection of the content for the article to be read was an important consideration*

Th© following

criteria for th© selection were decided upon* 1* Th© subject matter must b© authentic, Important, and timely enough to justify eighth grade pupils spending time on it* 2* The subject matter must b© sufficiently factual to permit significant generalizations and inferences* 3. The subject matter must be Interesting to pupils of this grad© level* 4. The subject matter must b© r elatively new to the pupils 5. In its final form the article must lb© of th© proper reading difficulty for eighth grade pupils.

Several articles were selected and tried out at different grad© levels.

The article on tularemia came nearest to

meeting the criteria IIsted.febove. In preparing the reading selection on tularemia, an attempt was made to Include only such information as seemed essential to give pupils the Important understandings about the disease.

Th© investigator did not rely upon his

own judgment to determine those basic understandings*

Six

competent people, including two medical doctors associated with the Iowa State Department of Public Health, two members of th© staff of the College of Education of the State Univer­ sity of Iowa, and two graduate students In the same college, were consulted*

Each of these people was given a copy of

an adaptation of the bulletin on Tularemia and was asked to list the understandings about the disease which he felt eighth grad© pupils should gain as a result of reading and studying about it.

(See sample list Appendix B). The six

lists submitted were checked against those basic understand­ ings which the Investigator had selected prior to making the adaptation of the bulletin*

The content of th© selection

was then rewritten in order to Incorporate or give greater emphasis to those understandings on which there was agree­ ment*

In its final form, the reading selection was checked

for authenticity and clearness of concepts by Dr. Milford E* Barnes, Head of the Department of Hygiene and Preventive Medicine at the State University of Iowa.

Permission was secured from the Iowa State Department of Health to include on th© first page of the reading selection an exact reproduction of the Foreword written by Dr* Walter Bierring for the state bulletin.4 (See Appendix B) Th© selection was first tried out with thirty pupils in th© sixth grade at th© University Elementary School to check on the reading difficulties involved.

As

a result of this tryout, minor changes in vocabulary and sentence structure were mad© in order to simplify the read­ ing*

However, the essential nature of th© content was

maintained* General directions for all pupils were then pre­ pared and incorporated in th© reading selection.

Th© aims

of these directions were to motivate th© pupils by giving them a purpose for reading the selection, to prepare them for th© experimental procedure, and to acquaint them with th© need for following directions*

The selection with the

directions on the outside front page was then printed in an eight-page, 6 x 9 bulletin*

(See Appendix B)

Construction S£ the Study jsiyyitjoms From the same composite list of understandings which was used In preparing the reading selection, facts and generalizations contained in the selection and plausible 4Iowa Public Health Bulletin, Tularemia* o&* £*£•

Inferences which could he mad© from an understanding of the Information In the selection were listed*

Questions

were constructed to direct attention to these facts, gener­ alizations, and Inferences*

Th© questions were then group­

ed, and for each group on© main question, the answer to which would require the answer to all or several of the detailed and subordinate questions, was then constructed. Seven main questions were thus prepared* The subordinate questions were then subjectively evaluated with reference to th© contribution each made to­ ward answering the corresponding main question.

Only those

subordinate questions were retained, the answer to which seemed, in the opinion of th© author, essential to formulating answers to th© main questions. In the final form each of the seven main questions was followed by three or four sub­ ordinate questions*

These questions embodied all of the

understandings on which there was unanimous agreement by the six people who submitted lists of understandings* Main and Subordinate Questions 1* $hat has made it possible for people to know more about tularemia today than was known thirty or forty years ago? a* What steps have been followed In gaining information about the disease? b* that part did the United States Public Health Service play In gathering Information about th© disease? c* When was the disease first studied? d. How widespread is th© disease today?

70

2. Why was the discovery of the first case of tularemia in a human being so Important? a* For how many years have we known that man could get the disease? b* What had been the previous experience of Dr. Wherry? e* What was the source of the first case in man? dm What warning did the doctors give at that time? 3* What is meant by the first statement in the Foreword, anemia is a clear-cut example of a disease of animals which is transmissible to people?* eu In what animals is the disease most common? b* How is the disease spread from one animal to another? c* In what three ways may people get tularemia? d* Why Is tularemia considered an occupational disease? 4# How can tularemia be distinguished from other common diseases? a* What are the symptoms of the disease? b* fliere does the sore or ulcer usually appear? c* On what part^ of the body are enlarged lymph glands usually found? 5* How widespread Is tularemia In Iowa? a. How many cases had been reported by 1940? b* During what season of the year is the disease most prevalent? e« Which sections of the state have been most seriously affected? d# What I® the chief source of Infection? 6* Why is tularemia considered a serious disease? a* What is the death rate of the disease in Iowa? b* How long does it sometimes take to recover from the disease? e. ?&iy are doctors unable to prevent the disease? d» Why is It a difficult disease for doctors to cure? How can you guard against getting tularemia? a. In what way would you be most likely to get the disease? b. Of what should you be absolutely certain before eating wild rabbit? c. What insects should you be particularly care­ ful to avoid?

C o n s t r ^ ^ ^ og M^ection^ Directions to the

p u p 113

Six sets of directions, one for each group, were prepared*

To control the teacher variable in the adminis­

tration of the materials, an attempt was made to make the directions so specific and simple that they could be under­ stood by all pupils*

Following the preliminary tryouts,

described later In this chapter, several changes were made In these directions.

The teachers administering the mater­

ials in the final experiment reported that the pupils had no difficulty in following the directions* The main question, and the main and subordinate questions, were mimeographed on paper which had been cut slightly larger than the reading bulletin*

This was done

to permit easy handling of the materials by the pupils with­ out their seeing the questions and directions until the appropriate time.

Hi© reading bulletin, questions, and

directions were assembled with the directions for each group face up Inside the front cover of the bulletin. Hie following excerpts from the six sets of directions will indicate the procedure followed by each group* Group I 2* Head the booklet through once. Head it care­ fully* See how much you can learn about tularemia*

3* ’-lien you have finished reading this booklet through once, close it and start taking the test. You may not be able to answer all the test Items correctly. Do the very best you can. 4* When you have finished taking the test, put all of the materials on one corner of your desk* You may then read the article, '*Eskimo Year.” If you finish reading this article before the other groups have completed their work you may read It through, a second time, TBY HOT TO DISTRBB THE OTHER STUDENTS &ELO WILL STILL BE WOBKim AFTER YOU HAVE FIB"ISHED TAKING : PHE TEST* Remember * once you have finished taking the test you are not to turn back to it st any time. qroHP II

3* When you are told to begin, read the first para­ graph and decide whether or not it contains an idea important ©nought for you to remember. If it does, be sure to understand it* Then writ© on the back of this sheet the most important question which this paragraph answers. 3* Head the second paragraph* If this paragraph contains Information Important enough to re­ member, write a question to cover the important points in the second paragraph* 4* Follow the same directions for every paragraph in the booklet* 5* $hen you have read the booklet through, try to answer each of the questions which you have written* 6* Then turn to the booklet and read again to see whether or not you remembered all the Important information. If there is any part which you did not remember, study that part carefully. Group III 2. Seven main questions that can be answered by reading and studying the Information in the booklet are printed on the back of this sheet. BO MOT READ THE QUESTIONS NOW. When your teacher tells you to begin reading, turn this sheet over and read each question carefully.

3* When you have finished reading the questions, read the booklet through once carefully. As you read look for information that will help answer th© seven questions* 4* After you have finished reading th© booklet, turn back to the list of questions. Read the first question* Think of a good answer for it. If you can answer the question put a cheek mark before It* If you cannot answer it, do not put anything before the question* Do this for all seven questions. 5. After you have finished trying to answer all of the questions, turn back to the booklet to find and study the parts that will help you answer the questions which do not have cheek marks in front of them* 6* When you are sure that you have enough Informa­ tion to answer a question, put a plus sign be­ fore the question. This will show that you have now found th© answer* Group H 2* When your teacher tells you to begin reading, open your booklet and read it through once carefully* 3* On a loos© sheet in the back of the booklet are seven main questions which can be answered by reading and studying the Information in the booklet* DO SOT READ THE QUESTIONS NOW* 4* When you have finished reading the booklet turn to th© seven questions* Read the first question* Think of a good answer for it. If you ean answer it, put a check mark before question 1* If you cannot answer it, do not put anything before the question* Do this for all seven questions* 5* After you have finished trying to answer all of the questions, turn back to th© booklet to find and study th© parts which will help you answer th© questions which do not have check marks in front of them. 6, When you are sure that you have enough informa­ tion to answer a question, put a plus sign be­ fore the question. This will show that you have now found the answer. •. ••••

: ?4

9* Remember-—•yon are not to read the questions on the sheet in the hack of the booklet until you have finished reading the booklet through once* Group V 2* Seven main questions which can be answered by reading and studying the information in th© booklet are printed on th© back of this sheet. Under each main question are several subquestions which will help answer the main question* DO MOT m m THE QPESTIOBS IQW* then your teacher tells you to begin reading, turn this sheet over and read each question carefully* 3* When you have finished reading the questions, read the booklet through once carefully* As you read look for information which will help you answer the questions* 4. After you have finished reading th© booklet, turn back to the list of questions. Head the first main question. Then read the subquestions under this main question* Try to think of a good answer for each sub question* Put a cheek mark before each subquestion which you can an­ swer. i>o not put anything before th© questions which you cannot answer* If you can answer all th® sub questions you should be able to answer th© first main question* Put a check mark be­ fore th© first main question if you can answer It* Bo the same for each group of questions. 5. After you have finished trying to answer all the questions, turn back to th© booklet to find and study th© parts which will help you answer the questions which do not have check marks be­ fore them. 6* i?Ai©n you ar© sure that you have enough information to answer a question, put a plus sign before question. That will show that you have now found the answer* Group n 2. Vtien the teaeher tolls you to begin reading, open your booklet and read It through once carefully.

3* On a loose sheet in the back of the booklet are seven main questions which can be answered by reading and 'studying the information in the booklet* Under each main question ere several smaller or subquestions which will help answer the main question. BO BOf READ THE QUESTIONS MOW. 4* Ihen you have finished reading the booklet, turn to the questions* Read the first main question* Then read the subquestions under the main ques­ tion* Try to think of a good answer for each of these subquestions* Put a check mark before each subquestion which you can answer* Do not put anything before the questions which you can­ not answer* If you can answer all the sub­ questions you should be able to answer the first main question* Put a cheek mark before the first main question if you can answer It* Do the same for each group of questions* 5* After you have finished trying to answer all the questions, turn back to the booklet to find and study the parts which will help you answer th© questions which do not have check marks be­ fore them* 6. then you are sure that you have enough informa­ tion to answer a question, put a plus sign before th© question* This will show you have now found the answer,.*** 9* Remember— you are not to read the questions on the sheet In the back of th© booklet until you have finished reading the booklet through once. Construction of Directions for Administering the Materials In constructing the directions for administering th© materials, an effort was made to control, as far as possible, the participation of the teacher in th© experiment. Each person responsible for administering th© materials was urged to make himself ^thoroughly familiar with the dif­ ferent procedures to b© followed by the six groups*”

The de­

tailed directions were mimeographed in a form which made them

easily follcwed*

For a more comprehensive discussion of

these directions see the section on *Admlnistration of Study Period and Immediate Test.” Construction o£ the Test In general th© construction of the test over tularemia paralleled the construction of the reading selec­ tion and the study questions*

Th© sum© list of basic

understandings was used in th© construction of all of th© materials,

Before building th© test Items, certain specifi­

cations for the test were set up.

These specifications

included the following nine underlying assumptions of the accomplishment which could be approached by means of this multiple-choice test* 1* Beading and studying of a highly factual and Interesting article will produce changes in the Individuals doing that reading and study­ ing* 2* On© aspect of those changes - the information and ideas gained from reading and studying can be reliably measured by means of a total score on a multiple-choice test* 3* It is possible to produce changes in individuals by means of different types of study techniques, to appraise those changes for each group in order to determine whether or not certain of these techniques are more effective than others, and by statistical analysis of the data from the test to determine whether or not th© result­ ing differences are significant. 4* In addition to measuring the effectiveness of the six methods employed in study, the Immediate Test (test over Tularemia) will provide the type of learning desired to the extent that the test is constructed to get at that type of learning.

, '

5, The multiple-choice test, as well as th© study exercises, ©an b© used to motivate the learning by giving the pupils an opportunity to select and appraise the information and Ideas which they consider significant, 6* The test items Included can give an approximate measurement of th© type of behavior that schools expoct and try to get from children when they read and study* 7* Th© reactions of the pupils to th© situations or types of behavior included in the test should giv© a Measure of th© characteristic reactions of pupils when they study In th© ways included in this investigation* 8* The total score on th© Immediate Test (test over Tularemia) may be accepted as a measure of th© amount and quality of the facts and Ideas which pupils possess as a result of reading and study­ ing th® seloetlon* 9* Th© tost can show only th© results of thinking* It cannot reveal th® process of thinking* The specifications also included the following schedule to guide th© actual construction of th© test Items and their assembly into tost form,^ 1* An effort will be mad© to Include items of all degrees of difficulty; from those that very few students will miss to thos© that very f©w will get correct* It will b© permissible to have a piling up of easy items on th© Immediate Test (test given Immediately after reading and study) to have a better range on the Belayed Test (same test given a month later), 2, Since the aim will be to require a genuine under­ standing of the concepts involved before the correct response can be selected, an effort will be mad© to avoid th© following weaknesses in th© construction of th© Items: ®H* E* Hawkes, E* F, Lindquist, 0* R* Mann, Th© Construction and Us® of Achievement Examinations. Houghton Mifflin Company, Chap* III*

a. Using the exact language contained in the article h* Making correet responses which are consist­ ently longer than incorrect ones c. Using the same words or phrases in the correct responses as are used In the study questions d* Making responses which are not grammatically consistent with the question 3* An effort will he mad© to make all of each Item function* 4, Th© incorrect responses to th© items will he made so plausible that they will attract the pupils who do not have a thorough understand­ ing of th© point raised* This will tend to eliminate th© results of rote learning and lessen th© effect of chance selection or random guessing* 5* In general, th© positive rather than th© negative approach will he used* However, If a better Item can be built by requiring pupils to elim­ inate the incorrect responses rather than to select the correct response, negative technique will be employed* 6* For each Item an effort will be mad© to antic­ ipate th@ thinking that the pupils will have to do In arriving at the correct response* Bach Item of the test will be examined by th© author and also be submitted to competent peopl© to determine whether or not Its phrasing, structure, mechanical or typographical arrange­ ment, or any other external weakness may assist the pupil In selecting the correct response without carefully reading and rejecting the In­ correct responses* Such weaknesses as may be discovered will be eliminated in order that the apparent responses will correspond to the in­ tended responses* The following questions will serve as a guide to the examination of Items, a* Is there anything in the structure or wording of the question, other than what was intended, which will help the pupil select th© correct response! b* Is there anything in the Item which the child could disregard entirely and still select th© correct response? c* What type of thinking is necessary to answer this item correctly? Gould that thinking result In the selection of the wrong response?

7* A list of th© facts, generalizations, and infer­ ences, either specifically stated or implied by the selection will be made* From this list, items which will test ability to recognize or recall facts, generalizations, and Inferences will be constructed* 8* So far as seems advisable, the Items in the test will follow the sequential order of the informa­ tion In the reading selection* 9* a0fttoh* questions will not b© Included In the test* An attempt was mad® to Identify th© types of behavior Involved In reflective study. decided upon,

Mine abilities were

An effort was then made to build items which

would require th© exercise of these abilities*

In the

construction of each item, the type of thinking required to select th© correct response was arbitrarily determined* In the following list, the numbers under each heading are th© numbers of the Items in the final test which, in the opinion of the author, should be classified under each of the abilities*

It should be noted that

practically all of th© items could be classified under the ability to recognize correct statement of fact*

For that

reason it did not seem necessary to Include that classifi­ cation in the abilities listed* h

1, Ability to recognize a verbatim statement of fact 3, 4, 7, 10, 29, 59, 41, 42, 45, 59 2. Ability to recognize in a new situation th© applica­ tion of factual Information which is explicitly stated In the selection 6, 12, 14, 22, 24, 25, 26, 27, 48, 47, 51 3* Ability to compare and evaluate number facts and quantitative relationships 1, 8, 9, 15, 30, 31, oo, 53, 55

4* Ability to recognize a correct interpretation based upon information implied by the article 5, 31, 40, 56 5» Ability to recognize a cause and effect relation­ ship 21, 35, 38, 6* Ability to recognize a conclusion drawn from information In the article 2, 11, 16, 17, 18, 20, 34, 36, 44, 45, 46, 40, 54, 58, 7. Ability to suspend judgment because of Insuffici­ ent data 52, 57 8* Ability to draw upon past experience and informa­ tion to make a generalization 13, 23, 32, 60 fe* Ability to recognize problems 19, 88, 33 ®MX© th© author does not claim to have been entirely successful in following th© above specifications, it is hoped that the foregoing discussion and a careful examination of the multiple-choice test as well as the other materials will make it clear that an effort was made1 to construct materials which would place a premium upon the * abilities to select pertinent facts and ideas,-to appreciate the significance and relationships of those ideas, and to Identify desirable behavior as a result of a knowledge of those facts and ideas. In the actual construction of the test, four, five, and six alternate responses were built for each item. In the final revision the least effective of these responses were eliminated.

Preliminary Studies Thirty pupils in the sixth grade at the Univer­ sity Elementary School were used in the first preliminary study, th® purpose of which was to get a check on 1) the reading difficulty of the article on tularemia, 2) th© clearness of the directions to be read by th© teacher to th© children, and 3) the ©as© with which th© children could read their own directions*

Revisions in all of th© mater­

ials were mad© in light of this experience. Eighty-two eighth grad© pupils in the junior high school at Washington, Iowa mad© up th© second preliminary group*

The investigator spent a day in the school.

He

administered the materials to two classes and observed a classroom teacher administer them to a third*

Wherever

weaknesses in the materials or directions were revealed, they were noted and corresponding changes made before th© succeeding administeratIon.

As a result of this tryout,

further revisions were made in th© materials, and tentative time limits for the reading and study period and for th© tost period 'were established.

Th© revised materials were

then administered to a total of 151 eighth grad© pupils In th© Jefferson Junior High school in Dubuque, Iowa and th© Abraham Lincoln School in Oskaloosa, lows.

The final

time limits were established as a result of the experience in these two schools.

One hundred seventy-three papers were selected from these preliminary tryouts, and an item analysis was made*

Following this analysis, a number of items were re­

cast j that Is, either the questions were reworded or the responses were changed*

Bonfunctloning, wrong responses

were eliminated entirely or were changed so that they would seem more plausible*

Wrong responses selected by the pupils

who scored high on the test were studied to detect ambig­ uities and nearness to the correct response, and necessary changes were made* In Its final form the test included sixty Items. It was printed in a six-page railroad fold. HJeegJog ©£ Subjects Th© subjects in this Investigation were 1045 eighth grad© pupils from thirteen schools In the follow­ ing Iowa school systemsi

Cedar Rapids, Clinton, Dubuque,

Oskaloosa, Ottumwa, East Waterloo, and West Waterloo. Letters were sent to the superintendents of the systems asking for their cooperation In the study and for th© names of principals of schools enrolling eighth grade pupils* Letters were then sent to the principals of each school selected, requesting enrollment lists for each eighth grade class.

(See Appendix B for copies of letters).

All of the

principals indicated a willingness to assume the responsi­ bility for the experiment in their school*

The number of

pupils In each of th© forty-five Glasses varied from seventeen to forty-six*

Every eighth grade pupil in each

school participated* Assignment to Groups The pupils from the forty-five eighth grade classes were assigned to six groups by means of th© table of random numbers*^

Each pupil*s name, school, and city,

and his group number were written in the appropriate blanks on the front of th© Immediate Test*

All pupils assigned to

a group remained permanent members of that particular group throughout th© experiment* Assemblage of the Materials A sufficient number of Initial Tests were sent for all eighth grade pupils in each school.

On, th© first

day of the experiment th© Initial Tests were to be admin­ istered according to th© printed directions which accompanied the tests.

(See Appendix B.) Th© direction sheets for each group were slipped,

face up, inside the front cover of th© reading bulletin. Questions for Groups IV and ¥X, which were mimeographed on sheets separate from the directions, were turned face down inside the back cover of th© bulletin. ^Lindquist, on* clt.. pp. 262-264

Th© bulletins and

the tests for different groups were then placed in separate piles from which they were assembled into on© pile for each school.

For each member of Group I, a copy of the article

3Eskimo Yearn was Included for extraneous reading* The experimental materials, sample sets for th© persons in charge of the administration of them, and copies of directions for administering were then securely packaged and Shipped to the principals of th© respective buildings. Presentation of The first responses of the principals indicated that in ©very school th© materials could be administered to th© entire eighth grade population assembled in one room, under the direction of either the principal or a competent teacher#

Where the groups were large, adequate provision

for assistance la the administration of the materials was assured*

Th© abov® plan was followed in every school for

all classes with the exception of the classes selected to take the t©st for previous knowledge, to be discussed later, Hi© materials were sent to the schools on January 3, 1942 with directions to start the experiment any time between January 7 and IS.

Hie principals were permitted

to us© their own discretion as to whether the Initial Test (Iowa Every Pupil Test), the reading and study period followed by the Immediate Test (test on Tuleremia) should b© administered on the same day or on succeeding days.

85

In every case the latter plan was chosen* Administration of Initial Test As stated above, the Initial Test was administer­ ed according to printed directions which accompanied the test*

A copy of this test with the accompanying reading

selection and directions may he found In Appendix B* Adminiatration of Study Period and Immediate Teat The directions pertaining to the actual adminis­ tration of the materials ar© given below*

With the excep­

tion of distributing and collecting th© materials, these directions represent the participation of the teachers In the experiment.

Attention Is here called to the statement

making clear to th© pupils th© purpose of th© experiment. Attention is also called to th© care exercised in making certain that each pupil understood the directions for his group. 1. To administer th© materials, begin by saying: ^Your class has been divided Into six groups, and each group has Its own directions* This is an experiment to find out which group can study best by following its directions exact­ ly as they are given. To make th© experiment scientific it is very important that ©veryone 11sten and follow directions carefully. Here is th© first direction As soon as you re­ ceive a set of materials, put them on the corner of your desk. Bo not open any of the booklets until all of th© materials have been distributed and you hav© been told to do so.**... 3* When th© materials have been distributed, hold up a copy of the booklet entitled Tularemia and say,

8G

"This booklet contains a reading selection about tularemia. The directions are on the front cover. Inside the booklet ar© direc­ tions for your group* Do not remove these directions but slid© them to the top of th© booklet* (Demonstrate) Check to see if th© group number on the directions corresponds to the group number on your test booklet* For example, if your test booklet tells you that you are in Croup S, the directions in­ side th© reading selection should read, fDirections for Croup Twe*f If these numbers do not correspond, raise your hand*** If th© materials are passed out in the order in which they were assembled, these numbers should correspond* If they do not correspond, it will b© necessary for th© pupils to ex­ change reading selection booklets so that they will correspond* If your class enroll­ ment has changed since the materials were assembled, give materials Intended for pupils that have dropped to the new enrol lees* "Fill in th© blanks at the top of th© direc­ tion sheet*" 4.* Allow only Ta brief time to fill In the blanks. up th© booklet Tularemia again and say,

Eold

"Everyone read the directions on this front page* Do not open your booklet or remove the direction sheet until 1 toll you to do so." 5* Pause her© to allow time for the reading of the directions and for any reasonable questions about th© procedure and then say, "How remove the direction sheet from the front of your booklet. B© sure that you read these directions carefully. It Is important for ©veryon© to know exactly what he is supposed to do* Ask questions only about directions which you do not understand*" 6* Pause here to allow time for the reading of th© directions and for any reasonable questions about th© procedure and then say, "Is everyone sure h© knows what he is to do? Group 1, you will read the selection one© and then take th© test Immediately. Remember you ar© not to wait for anyone to tell you to start taking th© tost.

"Group 2, you will read the selection para­ graph by paragraph. For each paragraph de­ cide whether or not it contains anything important enough to remember, and then write the most important question which that para­ graph answers* "Groups 3 and 5, you will read a list of study questions before you start reading the selec­ tion, then you will read the selection to find the answers to those questions, and then you will try to answer th© questions to your­ self. "Groups 4 and 6, you will read the selection first, then you will read a list of study questions and try to answer them* "Is it clear that you are not to write th© answers to any questions? However, you are to try to answer th© questions to yourself* Be very sure that you have a good answer to each study question before you put a check mark before it*" (This Is important). "Remember that in trying to answer the study questions you may turn back to the reading selection as often as you need to." "Also remember that except for Group 1, you ar© not to open your test booklet until I tell you to do so*" Ask members of Groups 3 and 5 to raise their hands Then say, "Be sure you people read th© study questions on the back of your direction sheet before you start reading th© selection*" Ask members of Groups 4 and 6 to raise their hands Then say, "Be sure you read the selection before you read and try to answer th© questions." "Everyone follow directions carefully. Remember we are trying to find out which group studies best. Open your booklet and begin reading." Note the time at which you said, "Begin reading.n

83

8* Move about the room to se© that everyone gets start­ ed immediately. Pay particular attention to Oroup 1 to see that they start taking the test as soon as they finished reading the selection once. When they start taking th© test see that they turn from page 2 to page 3 rather than from page 2 to page 5. As soon as they have started talcing th© test you may collect the reading selections about tularemia* After most of the pupils have had time enough to read th© selection say, ^Remember you do not have to writ© any answers, lien you have finished checking all the answers go hack over them again, each time trying to improve your answer* Refer back to your reading selection as often as you need to** 9* At th© ©nd of 25 minutes say, nIf you have not already finished your reading and study, you will have to leave it now and start taking the test. Put your list of ques­ tions in your booklet. Close your booklet and put it on the corner of your desk. Head th© directions on th© front page of th© test booklet.® 10* Pause her© only long enough to allow time for th© reading of the directions and to answer any ques­ tions which might be asked about the sample* This should require only a minute or two of time. Then say, *B©m©mber, you are not to turn back to th© reading selection or to th© study questions after you hav© started taking the test. Be sure to answer the questions in order. Notice how the pages are numbered. {Hold up a test booklet and demonstrate the order. Indicate that there are Items on page 6.) If you have no questions you may open your test booklet and start taking the test. Do not spend too much time on any on© Item. Leave it and come back to it when you have finished with the rest of the test.® Mote th© time at which you said, rtStart taking the test." 11. After the test has begun, walk quietly about the room seeing that the students are:

89

a) indicating answers in the proper way b) not wasting time on items which are too difficult for them c) not being delayed by broken pencils d) attempting all the Items In the test e) following the consecutive order of th© test f) not forgetting that there are items on page 6 g) going back over th© test if they finish before time is called dive no help other than showing the proper procedure. You may collect th© reading selection booklets at this time, 12, Allow 45 minutes to take the test, Then call time and collect the test booklets. If the reading selections have not been collected, collect them next. Then eollect the extraneous reading selec­ tion used by Group 1. Administrati on of the Test for Previous Knowledge Seven classes enrolling 173 pupils were selected by means of th© table of random numbers to take the test on tularemia at the same time that the other classes were work­ ing with the study materials*

In the letter of January 3 to

the principals of the schools affected, th© following direc­ tions for the previous knowledge group were givenj ”fhis group should work In a room apart from th© other groups. They will need no directions except to fill In the blanks on the front of th© test, read the directions, and answer as many items as they can. There is no time limit for this group.” At a later date copies of the test were sent to th© Junior High School in Newton and to the Horae© Mann Junior HIgb School in Burlington to secure another measure of previous knowledge in which an effort was made to partially control th© factor of guessing.

The directions for adminis­

tering th© tests in these two schools were;

wAfter the tests have been distributed and the information blanks have been filled in, the only directions which need to be given are as follows: •This is a test to determine how much information you have about tularemia* Read each Item of the test carefully* If you feel reasonably certain that you can select the correct response to an item, put an X in the box in front of it* If you do not feel reason­ ably certain that you can select the correct response to the Item; that is, if you think you know just enough to make a good guess, put a G in the box In front of th© item* For th© items for which you are very sure that you do not know th© correct response, leave the box empty* Try all of the items* Do th© best you can with them*1 These directions should be repeated so that each child will understand what is expected of him*” Admini strat1on of th© Delaved Test As stated previously, the Delayed Test was th© Immediate Test administered one month after th© reading and studying had been done.

The appropriate number of these

tests were sent to th© participating schools.

Th© directions

for administering this test were as follows: ”You will note that the Information blanks on the front page of th© test have not been filled In. As soon as th© tests have been distributed ask the students to fill In the blanks* If they can remember their original group number, that should be put in the appropriate place. If this group number Is not easily recalled, instruct them to leave tho space blank* After all of the blanks have been filled in, say* •You will recall that one month ago today you read and took a test over an article on tularemia* We are anxious to discover how much you remember of the Information wh ch you gained at that time. Today you are to tak© the same test* You will have the same amount of time. See if you can do as well

on the test today as you did the first time that you took It* Heady,— open your test booklet* Begin taking the test** Make a record of the time at which you said, 1Begin taking th© test*’ At the end of 45 minutes call time* Collect the tests and return them to your principal*” gugyyyi ^ s en ^ during the Experiment Over 1200 pupils took each of the three tests* Of this number only 1045 pupils took all three tests*

Th©

papers of pupils who missed any one of the three parts of th© study were discarded*

Sufficient materials were sent

to each school, however, so that ©very pupil In the grade could fe©l that h© was participating In th© experiment during th© days which he was present*

Chapter XV RESULTS AMD STATISTICAL TREATMENT Jilliirai On© thousand forty-five eighth grad© pupils In thirteen schools In seven school systems participated in this ©xperlment.

Within each school all eighth grade

pupils were randomly assigned to six groups, each group following a different study procedure*

Th© experiment was

designed to permit the use of analysis of co-variance^ in the evaluation of th® obtained data.

These procedures

make possible greater precision than do the techniques which commonly have been employed in studies of this nature and also yield a highly valid estimate of error*

The

statistical treatment of the data, a subjective appraisal of th® way the various groups used their study procedures, and an item analysis of th© Immediate and Delayed Tests will he presented in this chapter. Presentation of Data for Treatment by Covariance Results on Initial Test Scores on th© Initial Test, the Iowa Every Pupil Reading Test, were used to measure the learning ability of all pupils who participated in the experiment.

This test

^For a complete discussion of these procedures see, E. F* Lindquist, Statistical Analysis in Educational Re­ search , Ecughton Mifflin Company, 1940, pp. 180-207.

93

Is designed to measure four important silent reading abilities^ ability to comprehend paragraphs, to grasp and understand signifleant detail, to organise ideas, and to appreciate the total meaning of the reading selection*

Hie

seventy items of the test are basedj'three relatively long reading selections*

Because of the wide sampling of content

(three reading selections) and the relatively long testing time, th© consistency with which this test measures may be accepted with 4 high degree of confidence*

From th© Iowa

Basic Skills Program in which the test was administered to thousands of pupils In grades six, seven, and eight, a reliability coefficient of over *90 was secured* The results of the Initial Test when administered to all of the pupils in the present Investigation are pre­ sented in Table 2*

The mean, standard deviation, range,

and number of pupils for each group and for the total experiment are given* Table 2 RESULTS OF IHXTIJL TEST

Total

I

II

Qroups III IV

V

VI

Mean

39*529

39.232

38.763

39.838

39.123

39.828

40.431

S* T>*

11.216

11.899

11*485

11.838

11.421

10.717

10.808

Range

11-66

11-61

15-66

16-63

11-65

15-61

11-64

Humber

1045

177

177

173

171

180

167

The general mean for all pupils, was 39.529, and th© standard deviation of these scores was 11*216. The total possible score was 70.

The means of the scores

by groups ranged from 38.763 for Group II to 40.431 for Group VI.

Hi® standard deviations varied from 10.717 for

Group f to 11*899 for Group 1.

Mean scores In Table 2

show the differences In Initial ability (as measured by the Initial Test) for which allowances were made statis­ tically, in the treatment of th© results of the Innediat© Test * The thirteen school means for the Initial Test and th© means for the six groups within these schools are presented in Table 3*

From data presented In this table

comparisons of Initial ability can be made from school to school and from group to group.

Table 3 MSAS SCORES FOR EACH GROUP OH TEE INITIAL TEST iB THE THXRTBEB SCHOOLS

School 1 2 3 4 5 6 7 8 9 10 11 12 13 Total

Total

X

II

Groups 17 III

¥X

V

38*840 34,.833 41*537 41,.375 40*348 41*300 41*095 41,.750 39 *040 42,.000 41*000 401.000 35*349 37..556 33.114 30,.688 39*995 41*152 40*504 38,,045 42.400 44,,250 38 *397 36,,000 44*467 48,.091

39.083 35*857 38*250 41*385 43.917 41*318 41.880 41*136 40 .458 43*588 41*385 40.417 38*364 40 .455 40.008 36.000 32*333 34*667 52.750 44*000 30.000 43*250 30*200 44 *000 44.000 44.500 59*000 41*853 37 *833 41*667 28*500 37*500 35.000 37*000 37.000 31*842 32*571 38*105 33.316 37*889 39.250 40.867 40*563 m .688 39.346 41*760 41*087 41.000 40 .958 39.952 38.000 40*375 43*375 43.875 43.571 36.579 40*722 38.188 38*900 39*870 47.111 44.364 45*636 44 .000 37*300

59*529

38*783

39,,252

39.838

39.123

39 .828

40.431

Results on Immediate Teat The Immediate Test was taken by Group I imme­ diately following the reading of the selection once, and by the other groups Immediately following their study period*

^

This test was designed to measure the amount of

information gained from reading and studying*

A compari­

son of the total amount of learning of each of the six groups (as measured by the Immediate Test) constituted the basis for determining the relative effectiveness of the six study techniques*

Analysis of covariance in which

the measures on the Initial Test were held constant was

96

applied*

o The KIchardson-Kuder formula was used for deter­

mining the reliability of the Immediate Test, of the reliability obtained was *83.

The estimate

The correlation

between the Immediate and Initial Tests was found to be *73* The results of this test when administered to all of the pupils are presented in Table 4,

The mean, standard

deviation, range, and number of pupils for each group and for the total essperfment are given*

Some idea?..of the vari­

ability of performance on the test may be gained from examining this table* Table 4 RESULTS OF IMMEDIATE TEST

Mean 3* D* Hang©

Croups III 17

Total

I

II

35*497

34*514

34.254

35,906

7.875

7*529

8*002

7.729

10-56

lumber 1045

13-52 177

11-51 177 5

V

71

36.117

35.817

36*449

7,999

7.754

7*986

10-54

18-55

12-53

12—56

173

171

180

167

^M* H, Richardson and 0. F. Kuder, 11The Calculation of Test Reliability Coefficients Rased on the Method of Rational Equivalence,w Journal of Educational Psychology. 30, December 1939, pp. 681-687*. The formula Is as follows; « J L

n-1

* .O&rjM..,__

^For formula see Lindquist, on* clt., p. 186* The sums of squares for M x S was substituted for the sums of squares within groups in formula 24 to find the correlation between Initial and Immediate Tests*

97

The general mean for this test was 35*497 and the standard deviation was 7*875* 60*

The total possible score was

The means for the groups varied from 34*514 for Group

I to 36*449 for Group 71, and the standard deviations varied from a low of 7*529 for Group I to a high of 8*002 for Group II* The thirteen school means and the means for each group within those schools are presented In Table 5* Table 5 MSAH SCORES FOR EACH GROUP OH THE IMMEDIATE TEST II TEE THIRTEEN SCHOOLS Groups III IV

School

Total

I

II

1 2 3 4 5 6 7 8 9 10 11 12 13

34*107 37.224 35*522 37*952 34*000 37.000 32*093 31*752 35*730 36.407 36*467 35*647 37.517

32*750 34*583 34.900 35*500 34*333 38*400 33*111 31.313 36.242 33*136 35.000 33.850 38.091

33.917 37.136 33.231 33.333 27.000 36.500 28.750 31.842 35.594 35.560 33*333 33*263 34.867

30*214 36.400 36.417 35*333 35.500 33*333 34.500 32.357 37*633 37.478 37,500 35*778 39.091

35.083 36.077 38.091 38.458 34.182 34.727 34.000 45,750 32.200 37.200 36.667 37.500 30.000 33.500 32.105 30,579 35*906 34.094 38.650 37,917 39.000 38.125 36,938 35.550 40.273 34*875

37.083 39,412 39.583 41.000 35.400 37.833 32.500 32.444 34*846 35.810 34.858 38.261 36.800

35.497

34.514

34.254

35.908

36.117

36.449

Total

V

VI

35,817

Results on Belayed Test The Delated Test was given on© month after the selection had been read and studied*

The purpose of this

test was to measure the amount of information retained after one month in order to determine the relative effec­ tiveness of the six techniques as aids to retention.

Analysis

of covariance procedures by which measures on the Initial Test were held constant were used to effect this ©valuation. Analysis of covariance was also applied to determine the relative effectiveness of the different techniques when the amount learned (as measured hy the Immediate Test) was held constant*

She results of the Delayed fast presented In the

following table show the mean* standard deviation, range, and number of pupils for each group and for the total experiment * Table 6 RESULTS OF DELAYED TEST

Total

I

II

Groups III XV

V

VI

Mean

32.472

32.192

32*006

32.439

32.556

32*533

33*144

S* D.

7*573

7*099

7*812

7*468

7*042

7.173

7.603

Rang©

12-53

Humber

1045

13-47

12-49

14-48

14-50

14-53

16-53

177

177

173

171

180

167

The general mean for the Belayed Test was 52.472, and the standard deviation was 7*575* score was 60*

The total possible

The means for the groups ranged from a low

of 32*006 for Group II to a high of 33*143 for Group VI, and the standard deviations varied from 7*099 for Group I to 7*812 for Group II.

99

Th© thirteen means for the schools and the means for each of the six groups within these schools are present­ ed in Table 7* Table 7 MEAN SCORE FOR EACH GROUP OS THE DELAYED TEST IS THE THIRTEEN SCHOOLS

I

Groups III 17

11

¥

School

Total

1 2 3 4 5 6 7 8 9 10 11 12* 13

30*887 33*694 33.275 33*286 33*760 33.688 29*023 30.314 32*629 32.593 33.644 32.879 33.200

29*917 31*456 33.900 30.250 36.000 32.400 28*111 30*500 34*152 30,955 34.750 31,800 35*545

30.500 34*454 32*615 29.667 29,667 37*667 26.000 30,000 32,158 31.880 32.133 32.053 34.111

29*571 30.083 32.692 32.500 33.200 33.636 34,750 35.176 33.000 29,909 32.909 37.167 38.500 30.600 34.800 32,250 36*500 30*600 34*600 35,000 31*667 31.500 33.333 34.333 31.000 27.333 30*833 31.667 31,000 30,684 29.158 30.722 32.800 33*313 31.936 31.808 33.130 35*100 32.625 32.143 32,625 34.125 36,000 31*571 32.167 33.688 31.900 35.348 33.091 34.091 30.875 30.800

Total

52.472

32*192

32.006

32,439

32,556

32.533

VI

33*144

jglllijyys $£ lyyyi jg. Using analysis of covariance

formulas,

the sums

of squares and the sums of products for M (methods), M x S (methods by schools), W G (within groups), M x 5 + W 0, M + M x S, and M + W G were determined for the Initial, Immediate, and Delayed Test scores. in Table 8*

These data are summarized

In Table 8 the letters, X, If, and Z represent

Immediate, Initial, and Delayed Tests respectively.

M tjJ

TESTS

SUMS OF PRODUCTS FOR IHITIAt., IMMEDIATE ARD DELAYED

1C 0

ARD

OF SUMS OF SQUARES

to to m * os © os

©

K3

to 0> •

"

0* rt

K \W

CO H CO * Os to 60

oa \ Wfc lt

os t£J £> • Os 03 iH

tD

HI

ca

SUMMARY

e* H * © to 03

JJH v |l| M tl

© ♦ O S o fc-

0> C Q K Ikf W

© to *

to H

10

H 03 00 * CD to 03 5

£to 60 • to •# H §

CO -si* 03 « £*• to H 03

H CCO * 03 CD CO ©3 to

CD rH £#***1 H » » O 09 H o to 09 to

03 r4 # o 03 03

CD GO C* to 03 O CO

p H © *

03 to to • to t* 09 03

to © H • H to CO

HI

H OS 03 • «0

03 !>

m

to

02

r-4 •

H to to

£~

C3

C O

to 09 * o H

H

IS 03 © IQ to e* * © H to

ID

to

to•

03 03

M 3

© eo os © C O

03 H O• H © CD fcH H

to H • 03 fcH to 03 H

&-

to GO to • t£* H 09

03 OS * CD CO to 03

© © * to

o *cf* to

iH © © • © © © to © to « © © ©

to

iH © igt • to o 03

03 *©* © to © © to

I> © to • to to ■ R ft

© © • ©

60

© © ©

•d»

H

to

9 ©

to to

H

© © • fcH

H

© rH

H

©

+

m H

m

Si

» C O H S3

©

m

©

M

+

+

IB

S

S3

s

j

Relative Effectiveness of Methods from School to School In order to determine whether or not there were any significant differences in the relative effectiveness of the respective methods from school to school, the reduced M x 8 variance was compared with the adjusted within groups variance by means of the F-test.

The results of this com­

parison for the Immediate Test, vshen the initial scores were held constant, are given In Table 9, instance was 1*100.

The value of F In this

Since an F of 1*30^ was required for

significance at the 5 per cent level, it is apparent that when initial measures are held constant, there was no significant interaction between methods and schools. Table 9 TEST FOB INTERACTION BETWEEN METHODS AMD SCHOOLS Oil IMME­ DIATE TEST m m THE INITIAL SCORES ARB HELD CONSTANT

d. f, ____ Reduced M x S Adjusted within groups

Sums of Y ariance Sonar e a ....

60

1871.387

31.190

966

27379.921

28,344

F = 31,190 (60 d. f.) / 28,344 (966 d. f.) = 1,100

Table 10 shows the results of a similar comparison for the Delayed Test, the Initial Test scores being held constant. 4

No significant interaction existed between

From table of F values, Lindquist, op« cit». pp. 62-65.

methods and schools since the adjusted within groups variance was larger than the M x S variance* of Table 11

Examination

will show that the same situation obtained

for the Delayed Test results when the Immediate Test scores were held constant*

That in no case were the interaction

variances significant indicates that the methods do not differ In relative effectiveness from school to school by amounts greater than may be attributed to chance* Table 10 TEST FOR INTERACTION BETWEEN METHODS AND SCHOOLS OH THE DE­ LAYED TEST WHEN THE INITIAL SCORES ARE HELD CONSTANT

d* f* Reduced M x S Adjusted within groups

Sums of Sauares

V arlano©

So

1435*593

23*926

966

29288.199

30*319

F ss less than I

Table 11 TEST FOR INTERACTION BETWEEN METHODS AND SCHOOLS ON THE DE­ LAYED TEST WHEN THE IMMEDIATE SCORES ARE HELD CONSTANT

d* f. Reduced M x S Adjusted within groups F s less than 1

Sums of Sauares

Variance

60

945.394

15.756

966

19213.378

19.890

Relativeness Effect!veness of the Different Study Procedures mm Measured by the jfamiediate Test The pupils in Croup I, the control group (no

H i

study creations), took the Immediate Test as soon as they had finished reading the selection once,

The other five

! ;l j

groups took the test immediately after their study period# The time spent in studying was the same for the five exper­ imental groups*

Holding the Initial Test measures constant,

analysis of covariance was used to evaluate the over-all effectiveness of the six different methods* To determine whether or not there were any sign­ ificant differences in the over-all effectiveness of the respective methods, the reduced methods variance was com­ pared with the adjusted Interaction (M x S) variance^ by means of the F-test,

5

The results are presented in Table 12,

Because it is likely that some interaction exists, even though the interaction variance does not exceed the within group variance by an amount larger than may be attributed to chance, the interaction variance remains the appropriate error term in effecting this comparison. More­ over the us© of th© interaction variance as an error term makes possible generalisations of the statistical findings based on analyses of covariance to the population of schools of which the schools participating in this investigation were a sample*

\

Table 12 ADJUSTED AID REDUCED VALUES FOR IHITXAL AMD IMMEDIATE TESTS

Source of Variation

d. f*

M + U x S

S3

Variance

2276*755

M x 3 M

59

1819*331

30*836

5

457*424

91*485

Test of significance of differences of methods means F a 2*967 (2*37) (3*34)6

The value of F obtained was not significant at til© 1 pen cent level* 5 pep cent level*

It was, however, significant at the

This indicated that some of the differ­

ences between the methods means might have been greater than could with certainty be attributed to chance*

Considering

th© number of means to be compared, and the possibility that any significant difference obtained might be due to chance, the 5 per cent level of confidence was rejected as not being sufficiently high to warrant comparisons of Individual means.

45 The values of (2*37) and (.3*34) are the values of F necessary for significance at the 5 per cent and 1 per cent levels, respectively* This will be the standard pro­ cedure in this investigation for indicating the values of F necessary for significance at these levels. 7Th© educational Implications of these and subsequent findings will be treated In Chapter V*

105

Relative Effectiveness of the Different Study Froeedurea as Measured by the Delayed Test when Initial Test Scores were Held Constant An evaluation of the relative effectiveness of the different study procedures after a month1s time was based on the results of the Delayed Test*

Holding the

Initial Test scores constant, the data were treated by analysis of covariance*

The results of this analysis,

presented In Table 13, shews that the M (methods) variance was less than the W' Q (within groups) variance, indicating 8 that the F was not significant* Since the differences between th© six methods, as measured by the Delayed Test and held against th© results of th© Initial Test, were no greater than could be attributed to chance, there was no justification for continuing th© analysis by covariance. Table 15 ADJUSTED AMD REDUCED VALUES FOR IMITIAL AMD DELAYED TESTS Source of Variation

d* f*

U + W S W Q M

S3

Variance

29324.769 966

29288.199

30.319

5

36.570

7.314

Obviously, the P is not significant since the M variance is less than the within groups variance. Q

Hie adjusted variance for M x S was first determined. Since th© value was smaller than the adjusted variance withing groups, th© within groups variance was used as th© appropriate estimate of interaction.

106

Relative Effect!venegs of the Different Study Procedures MS Measured by the Delayed Test when the Immediate Test Scores were Held Constant The Delayed Test results were used to determine the relative effectiveness of the different study procedures as aids to retention when the amount learned, as measured by the Immediate Test, was held constant*

Again, using

analysis of covariance, and this time holding the Immediate Q Test measures constant, an ©valuation was made* An F of 3*04 is required for significance at the 1 per cent level* Since the value of F obtained from this comparison as shown In Table 14 was only 1*312, it was not significant at either the 1 or 5 per cent levels*

Hence, the differences with

respect to the six methods as measured by th© Delayed Test when the immediate Test scores are held constant are no greater than may be attributed to chance, and there Is no justification for Carrying the analysis of covariance further* Table 14 ADJUSTED AID REDUCED VALUES FOR IMMEDIATE M D DELAYED TESTS Source of Variation

d. f*

M ♦ I 0

Variance

19343.620

w a M F * 1*312

S3

(2*22}

966

19213.378

19.690

5

130*442

26.088

(3.04)

Th© adjusted variance for M x S was first determined* Since th© value was smaller than the adjusted variance within groups, th© within groups variance was used as the appropriate estimate of interaction*

107

of JgysjjJyyl Resul ta on Immedl ate and Delayed Tests SSSSS*SS5SESBS*S t

* ni..wp»«*i*.

Since th© results presented In the preceding section established that none of the methods used in this experiment was significantly superior to any of the other methods, the covariance analyses were not carried further. However, the differences in the mean scores indicated the desirability of determiningwh ether or not a possible trend favored either placement or type of question* To make this comparison, the means on the Imme­ diate and Delayed Tests were adjusted*

The Initial, imme­

diate, and delayed mean scores and the adjusted immediate and delayed mean scores for each group are presented in Tables 15, 16, and 17*

Table 15 shows the initial and

Immediate mean scores and th© adjusted immediate scores for each of the six groups*

The adjusted scores In this and

succeeding tables represent, in general, th© scores which would have been obtained had the groups been equated.

Table 15 THE INITIAL AND IMMEDIATE MEAN SCORES AND THE ADJUSTED IMMEDIATE MEAN SCORES FOR THE SIX GROUPS

Group I a s 177

Initial Test Mean

Immediate Test Mean

Iramediate Test Adjusted Mean

39.232

34.514

34*636

II

n = 177

38,763

34*254

34,569

III

n = 173

39.838

35.908

55.781

IV

n s 171

39.123

36*117

36.284

V

n = 180

39.828

35,817

35.694

VI

n as 187

40,431

36*449

36.078

General Mean ,for Initial Test = 39,529

Table 16 THE INITIAL AND DEL AXED MEAN SCORES AND TEE ADJUSTED DELAYED MEAN SCORES TOR THE SIX GROUPS

Group I n = 177

Initial Test ._-Mean... .

Delayed Test Mean

Delayed u.yst Adjusted ...... Mean.... ..

39.232

32.192

32* 310

II

n = 177

38.763

32.006

32.311

III

n » 173

39*838

32.439

32.316

IV n = 171

39,123

32.556

32.718

V

n =r 180

39*828

32.533

32*414

VI

n « 167

40*431

33.144

32.785

General Mean for Initial Test = 39*529

100

Table 17 TEE IMMEDIATE AND DELAYED ME-AS 3G0RES AND THE ADJUSTED DELAYED MEAN SCORES FOR THE SIX GROUPS Immediate Test Mean

Group

Delayed Test Mean

Delayed Test Adjusted Mean

I n 4 177

34*514

32.192

32*880

II » s 177

34*254

32+006

32.876

n ■» 173

35+906

32.439

32*151

IV n a 171

56*117

32*556

32+122

III

V

n » 180

55+817

32*533

32*309

VI

n » 167

36*449

33*144

32.478

General Mean for Immediate Test ss 35+497

P^om the data presented in the preceding tables, several comparisons can be mad© between the experimental procedures followed by the six groups*

Again, It Is necess­

ary to point out that these comparisons are based upon tables of adjusted means and cannot be considered as stat­ istically significant* The means for Groups III to VI Inclusive, on the Immediate and Delayed Tests, with the Initial Test results held constant, were consistent!^ superior to the means for Groups I and II, Indicating a trend in favor of the groups using questions prepared by the Investigator*

The means

for Groups IV and VI, groups using main and main and sub­ ordinate questions after reading the selection, were

110

consistently superior to the means of the other groups* The performance of Group II on the Immediate Test was slightly inferior to that of Group I (control group, n© study questions)• On© may conclude that study under th© guidance of carefully prepared main and main and subordinate questions while not statist!©ally superior seems to b® more effective than a single, purposeful reading or having the pupils write the important questions answered by each paragraph*

The superiority, though slight, seems to favor

placement of questions after reading*

g^g^m§igi Is

jig

The Immediate Test contained sixty items*

The

per cent of correct responses for all groups on this test was determined by dividing th© unadjusted mean score for each group by the highest possible score, 60* are shown in Table 18*

The results

The performance on the Immediate

Test, expressed in per e©n$ as compared with optimum per­ formance ranged from the low of 57 for Group II to th© high of 61 for Group VI*

Th© per cent of correct responses

for all groups combined was 59* On th© Delayed Test, which was the Immediate Test given one month after initial learning, the per cent of correct responses for all groups combined was 54*

The

percentages for the individual groups varied from 53 for Group II to 55 for Group VI*

In other words, after one

ill

month, the average retention in relation to optimum learn­ ing was within six per cent of initial learning in relation to optimum learning,^ Table 18 PER CENT OF CORRECT RESPONSES* 01 THE IMMEDIATE AND DELAYED TESTS

Group

Per Cent of Correct Responses on Immediate Test Delaved Teat

I

58

54

IX

57

53

III

60

54

IV

60

54

V

GO

54

¥1

61

55

Mean

59

54

rrper cent of correct responses = mean / 60 (the high­ est possible score)# Since the analyses presented earlier in this chapter revealed that there were no significant differences between the performance of the groups, there is no justifi­ cation for determining the significance of the differences in percentages from group to group*

The data are presented

here to show 1) the relatively poor performance on the test in relation to optimum performance, 2 ) the general similar10

The amount of learning and retention as here iised is a function of the test employed and is subject to what­ ever weaknesses may exist In the test as a true measure of learning.

Ity in performance of all groups, and 3 ) th© small differ­ ences between the performance of the control and experi­ mental groups. Comparing th© differences between Group 1 (no study questions) and the experimental groups, it appears that when tlm© was kept constant, merely reading the mater­ ial with purpose was as effective or nearly as effective as using either main or main and subordinate questions. The factor of purposeful residing which operated for all groups cannot be overlooked*

No attempt was mad© In this

experiment to determine the importance of this factor,

The

investigator is not prepared to say if Its influence is as 11 strong as that suggested by Crawford: definite aim prevents two common errors In th© ©valuation of Ideas; namely, the error of consider­ ing everything as of equal importance, and the error of giving haphazard emphasis or exaggerating some things and neglecting others, Large and complex tasks are unified and organized only through the pursuit of definite alms,'1 Performance in

Previous Knowledge

To measure the previous knowledge that eighth grade pupils had of the material used for study, th© test on tularemia was given to 113 pupils In classes selected at random from thirteen schools.

These results were not

^G , C, Crawford, Methods of Study., published by the author, p. 47,

113

entirely e03155arable to the results for the other groups because the pre-test group was not an integral part of the main experimental design*

However, since the pupils were

from the same schools as the pupils in the other groups, and because they took the tost during the same period that th© other groups studied the reading selection, there was reason to believe that th© results would not have been much different had this group been included in the experimental design* Th© mean score for th© previous knowledge group was 34*729 with a standard deviation of 5.892.

Th© value

of 24*729 cannot be accepted unequivocally as a measure of the information eighth grad© pupils had ed by th© Immediate Test.

of the ideas test­

Since th© test contained sixty

items with from four to six responses per item, chance alone would yield a mean score of between 10 and 15 pro­ viding ©very item was attempted*

It Is evident, therefore,

that the mean score of 24*729 represents something other than chance. That th© factor of guessing and weaknesses inherent in the test operated in this previous knowledge score was recognised*

To try to find other possible explanations for

a mean score so large seemed desirable*

Tularemia or

"rabbit fever" Is not an entirely new topic to most people in Iowa*

Eighth grad© pupils who had had health training

with respect to th© more common diseases were probably able to relate that knowledge to whatever vague Ideas they

114

possessed about tularemia*

On a logical basis, it would

seem, then, that pupils would be better able to select the correct responses on items designed to test ability to relate and interpret general understandings, than items which test detailed factual information*

Examination of the function­

ing of items 3, 6 , 12, 22, and 30 (®factual1*) as compared with that of items 17, 19, 28 , 34, and 35 (^interpretative") would bear out that for these items the above explanation seems to hold.

(See section on item analysis.)

On the

first group of five Items, the per cent of correct response on th© pre-test ranged from 9 to 34*

For the group that

called for generalizations the range was from 53 to 85 per cent.

Items calling for such detailed knowledge as when

th© disease was first discovered, when the first case In a human was identified, how th© disease got Its name, what th© symptoms are, and how many eases were reported over a certain period give little opportunity for reasoning on the basis of past experience* or one doesn’t.

One either knows th© answer

On the other hand, items testing informa­

tion such as how does the United Public Health Service help the people of this country, isfoat is the first thing scien­ tists try to find out about a disease, what precautions should be taken in handling wild rabbits, and why should one avoid being bitten by a flea give considerable oppor­ tunity for generalizing on the basis of past experience. The selection of correct responses to such items, while

115

more than a pur© guess, would not necessarily indicate positive knowledge* A second explanation may he found in one of the principles that guided the construction of the test*

It was

hoped that ,the pupils, stimulated by the purpose, would select and reject details on the basis of their contribution to attaining that purpose* J Therefore, to give weight to the basic understandings it was desirable to include more test items on some details than others*

The most important

understanding to be gained from th© reading and study of th© article was that the wild rabbit is the chief source of tularemia.

Likely, pupils with practically no knowledge of

tularemia, had this bit of Information vaguely In mind.

At

least seven of the sixty items dealt specifically with this Information, and for eight others this Information would have been a factor aiding in th© selection of the correct response. There Is evidence, as will b© noted In the dis­ cussion of the performance of the Individual Items (see pages 126-174 }, that wrong responses which were not plau­ sible to those who had not read the article, and therefore, for th© pre-test group contributed to the selection of the correct response, became plausible after reading the article. It would seem to the investigator that the mean score of 24.729 represents considerably more knowledge than was actually possessed by the eighth grade pupils participating In this experiment*

Accepting, subject to the limitations already noted, the mean score of 24**729 as representing the previous know­ ledge, that amount subtracted from the means of the several groups on the Immediate fast yielded scores, ihlefa for the purposes of this investigation, were considered the amount actually learned as a result of the study procedures*

The

previous knowledge mean subtracted from th® mean scores on th® Delayed Test similarly yielded the amount actually retained as a result of the study procedures*

Table 19 shows

for each group th© per cent retained of the amount learned* Table 19 pm

Q E m RBIAIMED OF MGBMT LSARMED BY EACH GROUP WHEN PREVIOUS KNOWLEDGE IS SUBTRACTED

Grouo

Amount Learned Mean - 24,729

Amount Retained Mean - 24*729

Per Cent Retained

1

10,785

7.463

69

II

9,525

7.277

76*

III

11,179

7.710

69

IV

11,388

7.827

69

V

11,088

7.804

70

VI

11.780

8.414

72

Total

10.768

7.743

72

The relatively high percentage for Group II Is In part at least an artifact resulting from this use of per­ centage, Obviously, the group that gained least to lose and so retained more in proportlon*€6anthe other groups, and by the nature of the case shows the high­ er per cent of retention.

Table 20 PER CENT LEARNED* OF WHAT COULD HAVE BEEN hUARMBD Immediate Test*, Mean - 24,729

Gtoub

Per cent Learned

I

10.785

31

XI

9.525

27

XIX

11.179

32

XV

11,368

32

V

11,088

31

VI

11.720

33

10*768

31

Total

* Th© per cent learned the difference between th© Knowledge Tests divided by seori* 60, and the mead of

of what could have been learned = mean© of the limned!at© and Previous the difference between a perfect the Previous’Knowledge Test.

**The mean score of 24,729 is 41 per cent of the total jjossibl© score, When th© percentages In Table 20 are compared with those In Table 18 , the effect of subtracting the previous knowledge score is noted.

In all probability, neither

tkble gives th© correct picture of the situation because neither one shows how lasueh information about tularemia was possessed by th© pupils, especially how much of the crucial information.

Goi^arison of these data point out the danger

in accepting the average score on a test as a measure of the amount learned.

£§^§£^2^1^ of Group Performance on Study Questions Group I

Group I was given the same purpose for reading the selection as was given th© other groups, ”How can we he protected against tularemia?” They were directed to read the selection through one© carefully, to get as many Important Ideas as possible* pagC214 ).

(8e© specific directions

After reading the selection, they took th©

Immediate Test and then spent the remainder of the period reading extraneous material*

Teachers administering the

experiment reported that the pupils in this group seemed to he following their directions*

Actual performance on

the testsjhowever;is the only objective way of measuring how well the pupils In this group studied.

grotto II The pupils in Group IX prepared their own ques­ tions*

Ther© were directed to go through th© selection

reading on© paragraph at a time, to decide whether or not it contained Information Important enough to remember, and if so, to write the most Important question which that paragraph answered* On the assumption that the quality of the ques­ tions written would Influence the performance of Group II on th© test, it seemed advisable to make an appraisal of the pupils* questions*

The results of the appraisal showed

that most of the questions were stated very well, were clear-cut, and to the point*

The wording, however, of some

showed sloppy thinking! for example, ”How did they find tularemia was th© same germ in animals and man?” and *lhat kind of flies are thought to play an important part In the germ?” On the v other hand, the wording suggests that th© pupils were thinking for themselves and not using book language*

From thirty-five papers selected at random, the

question for paragraph six was asked In the following dif­ ferent wayss 1* 2* 5* 4* 5. 6*

What is the disease like? How doesth© disease act on a human? Whatare th© characteristics of the disease? How do you feel when you have the disease? MiatIs the nature of the disease? Whatare some of the things that happen to a person with this disease? 7 , fthat are the symptoms of this disease? The pupils* questions were placed arbitrarily into three classificationss

1 } those which were large enough to

cover the Important Ideas In the paragraphs, 2) those which were on a single detail in the paragraph, and 3) those which indicate that the pupil had a totally wrong concept of the meaning*

About 38 per cent of the questions were what

could be considered as cover questions vhile about 59 per cent were on a single detail*

Only about 3 per cent of the

questions revealed a totally wrong concept.

Paragraph 2,

which was about the recognition of the first human case of tularemia was more conducive to writing wrong questions than were any of the other paragraphs.

About half of th©

questions showing wrong concepts were written for this paragraph*

The exact questions written by three pupils

are reproduced her©*

The only changes made by the investi­

gator were in spelling* A* Good coverage 1* Who conducted investigations on diseased ground squirrels In Tulare County, California? 2 * What was the name given to the disease? 3* How is the disease spread among ground squirrels? 4* ttihen and by whom was the disease traced to wild rabbits? 5* What was the name given to the disease and what does it mean literally? 6 * Where has the disease been found? 7* What kind of people are more likely to contract tularemia? 8 * Why does the Iowa State Department of Public Health recommend rubber gloves when handling wild rabbits? 9* How is man most likely to become infected with the disease? 10* What are some characteristics of tularemia? 11* Has the amount of tularemia cases increased or decreased from 1934-1939? 12* What are some of the carriers of the disease? 13* In what months were the greatest number of cases of tularemia reported? 14* fthat are two reasons for the few or no cases of tularemia In the western sections of Iowa? 15* {True-False) The meat of a rabbit should be thoroughly cooked* 16* Thougjh the death rate is 2 or 3 per every 100 cases why should it be regarded as serious? 17. Give three ways of preventing the disease. B* Detail questions 1 . $ho first discovered the disease in ground squirrels? 2m thy Is the germ called Bacterium Tularense? 3* How did the germs transmit from squirrel to squirrel? ,, . . 4 , $ho discovered the disease could go to humans? 5 * What does tularemia mean? 6 * Is tularemia a disease of most countries? 7 * that animal gives tularemia most frequently?

8, How can people who handle wild rabbits keep from getting the disease? 9.* How many animals get tularemia? 10* Is tularemia a serious disease? 11. How many Iowan eases have there been from 19541940? 12* From 1S34-1939, how many eases out of 249 were caused by rabbits? 13* Miat months are most severe for ^rabbit fever”? 3.4* Miat sections of Iowa have most eases? 15. Gan the disease be gotten from eating under­ cooked meat? 16. Do many cases result in death? 17* Are there any rules for prevention of tularemia? 18* Should you handle rabbits without gloves on for any reason? C» Poor quality X* What did Dr* McCoy find infected in ground squirrels? 2. Did all the animals have the disease that show­ ed a sign of the disease? 3* Can the disease be Infected to men? 4* Did the disease come In the eye? 5. Is deerfly fever the same as rodents? 6. How many people in the United States had tularemia in 1939? 7* How many eases out of a hundred have been given by the handling of wild rabbits? 8* Can the germ pass through the broken part of the skin? 9* The disease has been defined as what and is caused by what? The number of questions varied from paragraph to paragraph which indicated that the pupils exercised some selection in their reading.

Out of on© hundred papers

examined, the least number of questions were written for paragraphs 2, 8, 15, 16, and 17 which had 86, 95, 92, 81, and 73 questions respectively.

The fewer questions on the

last three paragraphs suggests that a number of pupils lack­ ed time to complete their reading and study.

This means

that they did not get to the step directing them to try to

■fl*>r>

answer their questions mentally# and then reread.

Since

this experiment was primarily concerned with the relative effectiveness of several methods where time was not a vari­ able, no allowance in the interpretation of data should be made for those that did not finish. Approximately 17 per cent of the pupils in this group failed to follow directions as evidenced by the 8 per cent who wrote no questions and the 9 per cent who wrote statements or phrases instead of questions.

The number of

questions written by the pupils varied from tvo to twenty. There were seventeen paragraphs in the selection,

1'he

median number of questions was eleven, Group III

VMMMMHMMP

-r-

The pupils in Group III ?,rere given a lisi of main questions,

(See page 212) They were directed to read the

questions carefully and then read the selection to try to find the answers*

After reading the selection, they were

to Indicate with a check (*^) all of the questions they could answer.

They were then to try to find the answers

to the other questions, and when they had succeeded they were to place a plus sign {+} In front of them.

If they

finished before the other groups, they were to go through the whole list of questions again, trying to refine their answers. An analysis was made of how pupils in Group III marked their questions.

Only forty-five per cent appeared

123

to have followed their directions precisely and to have mad© adequate use of their questions*

Eight per cent of

the pupils put so marks on their papers.

Whether they used

th© questions in their study is doubtful*

Binefceen per cent

put check marks before all questions, suggesting that they might have gone through the list and checked without try­ ing to answer the questions*

lliis procedure had been ob­

served In the preliminary tryouts, and study directions were made more specific in an effort to eliminate this tendency* !h© analyses of how pupils in Group III to VI inclusive marked their questions are subject to limitations* It was impossible to tell whether the check marks before every question meant that th© correct answer was known or believed to be known*

Perhaps the pupils who used both

check marks and plus signs used the latter to indicate that they had rechecked their answers*

group -gr Ih© pupils in Group IV followed th© sssae proce­ dures as Group III except that they read the selection befor© reading and using the study questions*

An analysis

of how these pupils marked their questions was made* Following are th© results £ 1) Bin© per cont of the pupils put no marks on the list of questions*

2) Seventeen per

/

cent put a check mark before all of the questions*

3)

Seventeen per cent marked most but not all of the questions,

suggesting that they had not finished studying, and 4) three per cent marked all questions with both a check and a plus sign.

5) Only fifty per eent appeared to have followed

their directions precisely and to have mad© adequate use of their questions. Group ][

The pupils in Group V were given a list of seven main questions each of which was followed by three or four subordinate questions*

they were directed to read the ques­

tions carefully and then to read the selection to try to find the answers*

After reading the selection, they were

to mark with a check all the questions that they were sure that they could answer.

They were then to turn back to the

reading selection to find th© answers to the other questions, and when they were able to answer these questions to mark them with a plus sign.

If they finished before the other

groups, they were to go through the whole list of questions again trying to refine their answers*

Th© reader should

keep in mind that there were seven main and twenty-six sub­ ordinate questions. At least nineteen per cent did not follow th© directions carefully enough to get the benefit of using the study questions.

Included in this group v/ere those pupils

who mad© no marks at all and those who marked only the main questions.

If th© pupils were following directions, It

appeared that as many as thirty-six per cent had not

125

completed their study*

Only twenty-four per cent had all

th© subordinate questions marked with either a check or plus sign* Subjective evidence points up that probably many of the pupils did not try to answer th© questions mentally* Most of the subordinate questions were marked with a cheek, indicating that the pupils thought they knew the answers to most of the questions the first time they went through them,

performance on th© Immediate lest dicl not stow

command of the information in th© article. Group VI Th© pupils in Group VI followed the same procedure as Group V except that they did not have access to the main and subordinate questions until after they had read the selection once.

An analysis of the performance of this

group showed that only twenty-four per cent had all th© subordinate questions marked either with a check or a plus sign.

Eleven per cent of the group put no marks on their

papers, while four per cent had marked only the main ques­ tions*

As many as 75 out of the 171 papers examined had

been marked only with a check, which if the directions were followed meant that the pupils had gone through th© ques­ tions only one© after reading, and had not turned back to the selection to study the parts that would help them answer the other questions.

A relatively small number of

questions were marked with a plus sign*

Only 30 per cent

126

of th© papers had plus signs before an;; of the subordinate questions, and of these the median number of subordinate questions thus marked was 4* 26*

This was out of a possible

It is th© opinion of th© Investigator that many of th©

pupils in this group, and of the other groups, assumed that they knew the answers to most of the questions and did not reread to cheek themselves* In summary, for Groups XI to 71 inclusive, th© per cent of papers not marked at all was nearly the same* In all groups about th® same number appeared not to have followed directions.

For Groups III and IV, the markings

seemed to indicate that about one-fifth of the pupils did not finish, whereas for Grotips V and VI, th© comparable fraction was two fifths* ing was even greater*

It Is likely th© number not finish­

On the other hand there Is no way

of knowing how many who had actually used the questions and even reread the selection had neglected to mark their papers according to directions. Item

of Imm'edi ate and Delayed Teats

It will be recalled that th© Immediate and Delayed Tests in this Investigation were the same measuring instru­ ment administered to all of the pupils at two different times*

By the R1chardson-Ku&er formula the estimate of

reliability obtained was .83*

A detailed Item analysis will

be presented in this section to show how each of the Items in th© test functioned both times that it was used*

This

127

analysis Includes for each item the index of difficulty, the coefficient of reliability, the per cent of correct response by th© group that took the pre-test, and the per cent of correct response for each of th© six study groups. ©i© index of difficulty for each item on th© Immediate Test was based on a random selection of three hundred papers from the total group*

Test papers for the

twenty-seven per cent of pupils making th© lowest scores

and th© twenty-sewn per cent of pupils making the highest scoros were analysed to determine the per cent of correct responses mad© by each group on th© individual items*

Th©

average of these two percentages was taken as the Index of 11 difficulty* Th© same procedure was followed in determin­ ing th© index for each Item on th© Belayed Test* The coefficient of reliability for individual items was determined by means of a table prepared by Flannigan*^ Th© per cant of correct response for each group was determined by tabulating th© total number of correct ^Th@ Index of difficulty (in reality the Index of easiness) was found by adding the per cent correct for th© upper group to the per cent correct for the lower group and dividing by 2* 3©© J. 0. Flannigan, **General Consideration in th© Selection of Test Items and a Short Method of Estimat­ ing th© Produet-Moment Coefficient from Bata at the lails of the Distribution,11 Journal of Educational Psychology. 30, December 1939, pp. 874-680. 12Plannlgan, Ibid.

128

responses for each item for the whole group and dividing that number by the size of the group* The items in the test were classified according to the nine abilities which were set up to guide th© con­ struction of the test*

(See page 79 ) The Investigator

recognized that most of th© items could have been classified under the heading 11ability to recognize a correct statement of faetn so that category was not included in the classifi­ cation*

The decision as to which ability a particular

Item required was reached only after due consideration of the statement of the question, statement of the correct response, and introspection as to what thought process was involved in selecting th© correct response*

Although

several

of the Items could have been classified under two or more categories^ each was listed under one only.

Four graduate

students in Education mad© independent allocations of th© items*

VUher© there was lack of agreement as to the alloca­

tion, th© author arbitrarily made the decision. The sixty Items fell into three classifications with respect to the level of thinking required to select th© correct responses; those items relating to factual information specifically stated in the selection, those items relating to the Interpretation of the information in order to recognize inferences, conclusions, and problems, and those Stems testing ability to suspend judgment and to

draw on past experience to make generalisations*

13

Th©

third and second classifications were separated because for the former the pupils* choice of correct resoonse depended upon past experience,

Items 8, 9, 30, 37, and 55 which were

listed under the "ability to compare number facts and eval­ uate quantitative relationships" were included in the first classification because the correct response is factual information explicitly stated in th© selection*

Hereafter

these three classifications will be referred to as "factual;" which Includes all Items listed under abilities one and two and items 8* 9, 30, 37, and 55 under ability three, a total of twenty-six 11©ms;"interpretative,1 which includes items 1, 15, 50, and 53 listed under ability three and all items under abilities four, five, six, and nine, a total of twentyeight items; and "associative", which includes the six items listed under abilities seven and eight* Each item Is followed by a brief description or interpretation of the data*

For some Items, th© interpreta­

tion is apparent from the data presented*

Often, however,

the description represents information which the investigator gained from working with the materials*

author recognized the danger and weaknesses in setting up such classifications, libat for one pupil may be an Item calling for Interpretation, may be an item of fact for another depending upon the background and mental ability of th© two pupils* Mfcat may seem to be a factual Item may call for more than sheer memory*

130

il m

About how many years ago did doctors and research workers start studying tularemia? Main Questions 1* Vftiat has made it possible for people to know more about tularemia today than was known thirty or forty years ago? Sub Questions

e* When was the disease first studied?

Ability Involved;

3 » collaring and evaluating quan­ titative relationships

Index of Difficulty I D

*50 *42

% of Correct Responses on Pre-test 9

Index of Reliability I D

.43 *30

% of Correct Responses D I Group 1 2 3 4 5 6

36 43 54 ' 54 59 59

24 30 32 29 39 33

The answer to this item requires the manipula­ tion of numbers to arrive at the correct response* That Groups V and VI had a higher percentage of correct responses on the Immediate Test suggests that th® sub­ ordinate question functioned* For this item questions in any form seemed to be superior to undirected read­ ing as measured by both the Immediate and Delayed Tests*

14 The main and subordinate questions listod for each item are the study questions which foeussed attention upon the Idea tested by th© item. The Information given after ability involved is taken from the classification of Items into nin® categories discussed in Chapter III* !_ and D refer to th© Immediate and Delayed Tests respectively.

131

Item Zt

How has our present knowledge about tularemia been gathered?

Main Question; 2. What has made it possible tor people to know more about tularemia today than was known thirty or forty years ago? Sub Questions a* What steps have been followed to gain information about th© disease? Ability Involved?

6 « recognizing a conclusion

Index of Difficulty .81 .73

% of Correct Responses on Pre-test

% of Correct Responses Croup I D 1 2 3 4 5

64 index I D

Reliability •45 .50

6

79 85 83 82 87 89

80 81 80 81 85 86

As measured by the Immediate Test, qtiestions in any for© seemed to result In performance superior to undirected reading* For both the Immediate and Delayed Tests, the subordinate questions seem to be most effec­ tive. Group XI* s performance was superior to that of Groups I, III, and XV* Most of what was gained as a result of reading and study*was retained after a month1s interval* Item 3*

In what year was the first human case of tularemia Identified?

Main Questions 2* Why was the discovery of the first case of tularemia in a human being so important? Sub Question! a, For how many years have we known that man could get th© disease?

1

N

J-?

Ability Involved5 1 - recognizing a verbatim statement of fact

I D

of Difficulty •43 .41

% of Correct Responses on Pre-test 14

Index of Reliability I D

.34 .27

% of Correct Re soor D Group I 1 2 3 4 5 6

27 44 38 32 54 58

28 34 28 35 33 42

A coraparisou of the per cent of correct responses on the Immediate Test with the per cent of correct responses of the previous knowledge group reveals a fair amount of learning* Performance of Groups V and VI on the Immediate Test suggests that subordinate questions were effective for this itera,,Initial learn­ ing* Group VI maintained its relative position on the Delayed Test* Item 4; What helped In the recognition of the first ease l'1*"TT'"' of tularemia in a human being? Main Questions 2* Why was the discovery of the first case of tularemia in a human being so important? Sub Questions b* What had been the previous experi­ ence of Dr* Iherry? Ability Involved}

1 * recognising verbatim statement of fact

Index of Difficulty I D Index I D

% of Correct % of Correct Responses Responses on Group I D Pre-test *76 74 *65 69 1 55 76 2 72 73 of Reliability 75 3 4 66 80 ♦JFI^ 75 5 *42 6 75 76 *45

In general all groups learned approximately twenty per cent as a result of reading and study. Past experience with health Information might account for the relatively high score for previous knowledge. Group IV which ranked in first position on th© Imme­ diate Test was in last place on the Delayed Test.

133

I S a a s»

What governmental agency took the leadership in studying tularemia?

Main Question* 1* IShat has made it possible fop people to know more about tularemia today than was known thirty or forty years ago? Sub Questions b* $hat part did the united States Public Health .Service play in gathering information about th© disease? Ability Involved;

4 - recognising a correct interpret­ ation of Information implied in the article

Index of Difficulty I D Index I B

% of Correct % of Correct Responses Responses on Oroup I D ♦68 Pre-test ♦75 1 68 68 2 73 68 60 of Hellability 69 3 72 4 65 67 *38 78 5 80 6 84 *41 78

Hot© that the performance of the previous know­ ledge group was better than that of Croups I, II, III, and IV on both the Immediate and Delayed foots* For every group except I, th© performance on the Delayed ..Teat was superior to that on the Immediate lest* It Is possible that an Interference factor operated follow­ ing reading and study and persisted for one month* Conceivably, frequent mention of th© Iowa Public Health Department in the selection, was the interference. The relatively high performance by the previous know­ ledge group suggests reasoning on th© basis of informa­ tion about the United States Public Health Service*

134

Item JSs How did th© disease get its name? Main Questions Sub Question:

son© non®

Ability Involved:

2 ~ recognising th© application of factual information which is explicitly stated

index of Difficulty I D Index 1 D

% of Correct f of Correct Responses Bespouses on Croup I D *50 Pre-test *30 1 24 49 10 58 2 23 of Reliability 47 23 3 47 4 24 *64 5 50 28 *46 50 6 28

Th© drop in reliability of this item is understand­ able* Since the pupils forgot about half of vhat they had learned the factor of guessing was more pronounced on the Delayed lest than on the Immediate* Item 7s

Dfcat animal was th© source of the first case of tularemia in a person?

Main Question: 2« Why was the discovery of the first case of tularemia in a human being so Important? Sub Question: e* What was th© source of th© first ease in man? Ability Involved:

1 - recognizing a verbatim statement of fact

Index of Difficulty I D Index I B

% of Correct % of Correct Responses I Responses on Group D Pre-test *47 40 62 1 *53 56 43 2 86 61 37 3 of Reliability 57 36 4 61 51 5 *14 56 50 6 *00

Prom the point of view of reliability, this item was very inferior both on th© Immediate and Delayed Tests* For all rr^ouos the performance on the Immediate Test was markedly inferior to that

135

of th® previous knowledge group, and for all groups the performance on th® Delated Test was superior to that on the Immediate Test* However, the delayed performance did not equal that of the previous know­ ledge group. Th® performance of the latter suggests widespread knowledge as to the chief source of infec­ tion for tularemia, and the fact that all groups scored lower than th® previous knowledge group can b© explained on th© basis of th© number of animals and Insects mentioned in th© historical discussion In th© selection* Performance on this item Illustrates th© confusion which may result when many similar or related factors are contained in a situation* Undoubt­ edly similar situations appear in many of th© current text® and are common In much of our teaching* On th© last preliminary tryout th© question in the Item was, wThe Infection which caused the first case of tularemia recognised In a human being was traced to what animal?" Sixty and forty-seven per cent in th© upper and lower groups respectively responded correctly to the item* In recasting,th© item, changes were made in two foils — *the tick0 and 11the tame rabbit1* were substituted for nth© dog® andnth© muskrat*® On th© Immediate Test, the corres­ ponding percentages were fifty-four and forty, while on the Delayed Test they were fifty-three and fiftythree respectively. Thus an attempt to make this a better Item resulted in an inferior one* Without a doubt, several sore preliminary tryouts would have improved the quality of this item or convinced th© author of th© desirability of emitting it entirely*

±36

Item 8;

In 1939, how many eases of tularemia were reported in th© United States?

Main Questions 1* What has made It possible for r>eopl© to know more about tularemia today than was known thirty or forty years ago? Sub Questions

d* How widespread Is th© disease today?

Ability Involved:

3 - comparing and evaluating quantitative relationships

Index of Difficulty

% of Correct % of Correct Response Responses on Group D I I *88 Pre-test D. .34 22 35 1 38 37 2 46 Index of Reliability 35 3 38 4 36 39 I .36 27 33 5 D .11 6 27 26

Hot© that th© per cent of correct responses for the previous knowledge group was higher than that for any of th© six groups after reading and study. Th© only explanation which the author can give for th© performance on this Item is that the statement In th© selection that death rat© of the disease is low operated as an interference to th© selection of th© correct response. Evidentally th© pupils had a wrong concept of rates. Th© performance of Groups V and 71 gives some weight to this speculation for these two groups had a subordinate question (other than th© on© contributing to this Item) on the death rate in Iowa, which mig#it have prompted them to select the lowest number. Every group had higher percentages of correct responses on th© Delayed Test than on th© Immediate Test.

137

Item 9 s About what per cent of the number of cases of tularemia reported In the United States has resulted from handling wild rabbits? Main Questions 3* fhat is meant by the first statement In the Foreword, ^Tularemia is a clear cut example of a disease of animals which is transmissible to people*t Sub Question!

none

Ability Involved!

3 - comparing and evaluating quantitative relationships

Index of Difficulty

% of Correct Responses on Pre-test

I ♦74 D.. *72

47 Index of Kell ability I D

*55 *57

% of Correct Response Group I D 1 2 3 4 5 6

78 82 74 73 72 79

70 73 68 69 69 69

Comparing the results of all groups with that of previous knowledge reveals a considerable amount of learning of the information covered by this Item* The results on the Delayed Test show a fairly high degree of retention* With the exception of Croup II which held first place on both tests, there Is very little difference between th© performance of the respect!ve groups* Item 101 What people are th© most likely to get tularemia? lain Question! 3. What Is meant by the first statement in the Foreword, ^Tularemis is a clear-cut example of a disease of animals I’ shick is transmissible to people*? Sub Questions d* Shy Is tularemia considered an occupational disease? Ability Involved!

1 - recognizing verbatim statement of fact

Ildex of Difficulty I B

.99 *99

% of Correct Responses on Pre-test 95

Index of Reliability I D

.20 .20

% of Correct Response I D Group 1 2 3 4 5 6

99 98 99 100 97 99

99 99 99 100 98 98

138

This item was too easy on both th© Immediate and Delayed Tests* and its reliability was too low to make a significant contribution to the quality of the test. On the preliminary tryout the percentages for the upper and lower groups were 100 and 93 respectively* Decision to leave this item In th© test was based upon the crucial understanding which It emphasizes* Item lit How does the United States Public Health Service help the people of this country! Main Questions lhat has mad© it possible for people to know more about tularemia today than was known thirty or forty years ago! Sub Questions b* Vthat part did th© United States Public Health Service play in gathering informa­ tion about the disease! Ability Involved?

6 - recognizing a conclusion

Index of Difficulty I D

*86 *82

% of Correct Responses on Pre-test 70

Index of Reliability I D

*60 .66

% of Correct Responses Group I D 1 2 3 4 5 6

89 84 89 89 89 92

83 75 87 82 84 82

This was a relatively easy item with high reli­ ability* Performance on this item reveals that read­ ing and study resulted In a respectable gain in under­ standing of the concept tested* Much of what was gained was also retained after th© elapse of a month* Croup II was Inferior to the other groups on both the Immediate and Delayed Tests*

139

Item 12 1

Miat characteristic of the tularense g©rm makes it easy for* it to pass from animal to animal one) from animals to people?

Main Questions 3* Miat is meant by the first statement in the Foreword, ^Tularemia is a clear-cut example of a disease of animals which is transmissible to people*? Sub Questionss b* How is the disease spread from one animal to another? e* In what three ways may people get tularemia? Ability Involved5 £*- recognizing application of factual Information explicitly stated Index of difficulty

% of Correct % of Correct Responses Responses on Group I J> I *59 Pre-test B *68 1 63 69 34 67 64 2 Index of Reliability 67 3 73 4 67 73 1 *54 5 63 67 D *47 6 61 65 Thls Item reveals considerable gain as result of reading and study# That the performance on the Delayed Test of all groups was slightly superior to the perform­ ance on th© Immediate Test suggests the possibility that th© fact ox* of reminiscence was operating* The changes in ranking of Groups III and IV on the two tests should be noted*

MO

Item 15: How does tularemia differ from such common diseases as measles, scarlet fewer, and whoop­ ing cough? Main Question: Sub Question:

Hone Hone

Ability Involvedi

8 - relating past experience and Information to a generalisation

Index of Difficulty

$ of Correct % of Correct Responses Responses on Group I D I *32 Pro-test I) .34 1 23 25 25 2 28 30 Index of Reliability 34 3 27 4 35 23 I *20 5 30 28 0 *24 6 32 30

On the last preliminary tryout, question 13 read, ttHow does tularemia differ from most diseases which people have?11 A change was made to malt© the Item more specific and to require more reliance upon past exper­ ience* For this tryout the per cent of correct responses was 25 for the lower group and 57 for the upper * On the Immediate Test, the per cents were 23 and 40 respectively* Obviously, if confidence can be placed in the two difficulty indices, this change made the item less reliable* Little gain in understanding resulted from either undirected or directed reading* Th© techniques used by Groups 1 and II seemed the least effective*

141

Item 14: If you contracted tularemia why could you not expect to b© cured within a few days? Main Question: 6* Wiy is tularemia considered a serious disease? Sub Question: h. How long does it sometimes take to recover from th© disease? Ability Involved:

2 - recognizing application of factual information explicitly stated

Index of Difficulty

% of Correct % of Correct Responses Responses on Group I D Pre-test I *86 86 B *80 1 82 85 76 73 2 Index of Reliability 3 85 90 4 91 87 1 *66 5 87 90 6 89 B *48 90 Ibis is a relatively easy Item with a correspond­ ingly hi^, degree of reliability# though the margin of possible g&In, was small, all groups mad© substantial progress toward grasping the concept tested, and with the exception of Group II seemed to retain that Idea very well* Placing either main or main and subordinate questions in the hands of pupils seemed to b© more effective than undirected reading cr reading and writing questions*

14.2

Itam 155

vhat uro ycur chances of get;ting tul ar©ml a from animals other than the wild rabbits?

Main Question:

5* How widespread is tularemia in Iowa?

Sub Question:

d. 33hat is the chief source of infection?

Ability Involved:

3 - comparing and evaluating quantitative relationships

Index of Difficulty: % of Correct % of Correct Responses Responses on Group I P I *32 pre-test B *29 27 24 1 26 2 28 28 Index of Reliability 28 25 3 4 31 26 I *36 5 29 26 © *28 6 30 29 Practically no gain resulted from either directed or undirected reading* Hi© performance of all groups was very similar* In.general the little gain made was lost with the lapse of on© month* Effective teach­ ing at this point would pay dividends because of th© cruciality of th© idea tested* Item 16: Which statement proves that tularemia is a wide­ spread disease? Main Question: 1* What has made it possible to know more about tularemia today than was known thirty or forty years ago? Sub Questions

d* How widespread is the disease today?

Ability Involveds

6 - recognizing a conclusion

% of Correct % of Correct Responses I D Responses on Group Pro-test I ,58 41 28 1 D* .28 27 22. 2 48 27 43 3 Index of Reliability 45 4 26 47 27 5 I .34 33 6 53 B .04 Index of Difficulty

143

The performance on the Immediate Test was con­ siderably better than that reported for previous know­ ledge, However, most o'f the gain made as a result of reading and studying was dissipated with the passage of one month* The sharp drop la reliability corresponding to the increased difficulty of the item should be noted* For all practical purposes, the results on the Belayed Test may be regarded as due to chance selection of th® correct response* Item 17?

•that Is one of the first things which scientists try to find out about a disease?

Sain Questions 1* What has mad© it possible for people to know more about tularemia today than was known thirty or forty years ago? Sub Questions a* Hftiat steps have been followed in gaining Information about th© disease? Ability Involved;

6 - recognizing a conclusion

% of Correct % of Correct Responses Responses on Croup I B I .69 Pre-test B .64 71 1 65 53 2 72 72 Index of Reliability 72 3 64 73 69 4 X .47 5 69 70 6 B .52 73 72 Index of difficulty

The substantial gain made as a result of reading and study was largely maintained after a months time. That about equal gain was made by all groups suggests the possibility that study under guidance of questions, which stress understanding rather than verbatim factual information may be no more effective in preparing pupils to draw conclusions of this nature than un­ directed purposeful reading*

144

It&13 18; In places where tularemia is common among wild rabbits, why is It difficult to prevent the spread of the disease among people who handle these animals? Main Question* 3* What is meant by th© first statement in the Foreword, nTularemia is a clear-cut example of a disease of animals which is transmissible to people*? Sub Questions c. In what three ways may people get tularemia? Ability Involved*

6 - recognizing a conclusion

Index of Difficulty

% of Correct % of Correct Responses Responses oa Group I D I Pre-test B *23 1 26 22 2 1? 23 21 Index of Reliability 29 3 24 4 25 23 I *22 5 31 23 D .19 6 38 24 Since Groups V and VI did somewhat better on this item than the other groups, the subordinate questions sqohi to be effective, their effectiveness being more pronounced when they are placed at the end of the selection. Hofce, however, that on the Delayed Test, there was practically no difference between any of th® groups* Following the last preliminary tryout, th© second foil was changed from ^The medicine preventing the disease makes people so sick that very few will take it,w to w'Ifc is impossible to tell which animals have the disease until they have been cut open** Th© per cents of correct responses for the upper and lower groups on th© preliminary tryout were 51 and 33 respectively * The corresponding per cents on th© Immediate Test were 35 and 18* Hence, this change In one foil made the item more difficult and reduced th© reliability. Another preliminary tryout would have revealed this weakness.

■f*

r*

H e m 19? If you had to clean wild rabbits what precautions should you take? Main Question: 7* How can you guard against getting tularemia? Sub Questions a* In what way would you be most likely to get the disease? Ability Involved?

9 - recognizing a problem

Index of Difficulty I D

*92 .88

% of Correct Responses on Pre-test 75

Index of Reliability I D

.55 .57

% of Correct Responses Croup I D 1 2 3 4 5 6

93 91 .94 94 97 95

88 89 88 91 92 92

This item tests one or the most crucial under­ standings in the selection. All groups showed con­ siderable gain over the previous knowledge group, the gains being about the same. Groups V and VI showed a slight advantage both in amount gained and in the amount retained. Item 20: ihat could a doctor do to guard against your getting tularemia? lain Question: disease?

6. why Is tularemia considered a serious

Sub Question: e* Vihy are doctors unable to prevent the disease? Ability Involved:

6 - recognizing a conclusion

Index of Difficulty

% of Correct % of Correct Responses Responses on Group I D I .79 Pre-test 76 84 1 D .77 71 2 75 61 77 80 3 Index of Reliability 75 80 4 76 80 I .46 5 79 84 6 D .48

Hals was on® of the cruelal items In the test. The pupils learned only about half of what they could

146

hair© learned and lost about half of what they had actually learned. However, note that they knew this critical information quite well even after a month* s time* Item 211 How can you account for the fact that people #10 do certain kinds of work have tularemia more often than other people? Main Question: 3* What is meant by the first statement In the Foreword, ^Tularemia Is a clear-cut example of a disease of animals which Is transmissible to people*? Sub Questions! e* In whet three ways may people get tularemia? d* Why is tularemia considered an occupational disease? Ability Involved?

5 - recognising cause and effect relationship

Index of Difficulty

% of Correct % of Correct Responses I Responses on Group D I .55 Pre-test D .44 55 1 46 45 50 2 51 55 Index of Reliability 3 42 4 53 46 55 I .39 5 47 6 55 D .46 47 Note the performance of Group II which was lowest on the Immediate Test and highest on the Delayed* In fact after a month* s time, this group had gained one percentage point on correct responses while each of the other groups had lost from eight to thirteen points, *£h© slight gain on the Immediate Test over the previous knowledge group was rather consistent for all groups* However, the loss v/as not as consistent. It appears that for gaining informa­ tion covered by this Item, undirected reading was as effective as reading directed by questions* Items 21 and 10 were covered by the same sub­ ordinate question. It will be recalled that Item 10 was one of the easiest Items on the test. It had a low index of reliability. Item 21, on the other hand, was more difficult* The performance on these two items suggests that, in general, pupils knew to which occupational group tularemia was most common, but even

147

after* reading the selection were unable to recognize why * It should be pointed out that the abilities involved in th© two items are not the same, one being the recognition of a statement of fact and the other* calling for an inference* Item 22: What syxjiptoms would you be the most likely to have if you had tularemia? Main Questions 4* How ©an tularemia be distinguished from other common diseases? Sub Question:

a* What are the symptoms of the disease?

Ability Involved:

2 - recognising the application of factual information explicitly stated

Index of Difficulty

$ of Correct % of Correct Responses Responses on Group I D I *81 Pre-test D .75 1 72 60 23 2 63 49 Index of Reliability 79 53 3 4 76 62 I *45 68 5 46 D *50 66 © . 76 Performance on this item shows considerable gain . as a result of reading and study* The groups using the study questions did hut little better than the control group* Croup VJ which had a subordinate question covering the information needed to respond correctly to this item,A£as well as Group I* Responses on the Delayed Test indicate that about half of what was learn­ ed had been lost at the end of one month* Group II*s performance was poorer than th© other groups*

148

Item 23:

fey might you have difficulty in knowing whether or not you had tularemia?

Main Question: 4* How can tularemia be distinguished from other common diseases? Sub Questions

a* feat are th® symptoms of fee disease?

Ability Involved:

8 * relating to past exporlenee to male® a generalization

Index of Difficulty I D

*33 *44

% of Correct Responses on Pre-test 23

Index of Reliability I *28 I) *50

% of Correct Responses I Croup B 1 2 3 4 5 6

39 41 32 34 32 36

46 38 ^4 tJ>C

39 42 44

fee amount learned as a result of directed and undirected reading ranged from nine to nineteen per­ centage points* Group I exceeded all groups except II on th© Immediate Test and exceeded all groups on the Belayed* With the exception of II, all groups showed better performance on th® Delayed test than on the Immediate* It seems possible that reminiscence operated to affect performance on this item* Item 23 was a better exercise on the Belayed than Immediate Test if evaluated on basis of reliability*

149

Item 241 If a doctor found that yon had some of the symptoms but still was not sure you had tularemia, what would he probably do to he more certain? Main Questions 4* How can tularemia be distinguished from other common diseases? Sub Questions b* ®iere does the sore or ulcer usual­ ly appear? Ability Involved:

2 - recognizing application of factual information explicitly stated

Index of Difficulty I D

% of Correct Responses on Pre-test

*54 *60

% of Correct Responses Group I D 1 2 3 4 5 6

35 Index of Reliability I D

*5© *33

52 63 60 68 61 66

4© 60 56 59 61 60

Performance on the Immediate Test was consider­ ably better than that on the pre-test, and th© amount learned was fairly well retained* The relatively low per cent of correct responses by Group 1 on both the Immediate and Belayed tests indicates that the directed study for all groups was more effective than undirected reading* The superiority for Groups IV and VI suggests that questions following reading were more effective than other placement* *

A.

150

Itejm 25i lhat is another thing which the doctor would probably do to be more certain? Main Question: 4* How can tularemia be distinguished from other coupon diseases? Sub Question* e* On what part of the body are enlarged lymph glands usually found? Ability Involved*

2 - recognising the application of factual information explicitly stated

Index of Difficulty

% of Oorrect $ of Gorreet Responses Responses on Group I D 1 *42 Fre-test D .40 36 1 34 5 2 34 36 Index of Reliability 37 3 42 4 32 37 I .41 5 49 52 D *50 6 41 46

Th© performance of Group I on this item indicates that undirected reading was as effective as the use of the main questions* The subordinate question, specifically focussed on this item, operated for both Immediate and Delayed recall* The performance on th© Delcyod ‘ lest for all th© groups ©xeopt I was superior to that on the Immediate Test. This Is of interest because th© previous knowledge group evidenced practical­ ly no understanding of th© idea tested. An analysis of the responses to this item showed that the foil, ”Examine your body for flea and tick bites,” attracted more people on the Immediate than on the Delayed Test. Foil 1, ”Lis ten to your heart beat,” attracted more on th© Delayed than on the Immediate Test.

Item 26 j When were people first warned of the danger of ©lose contact with wild rabbits? Main creation: 2« Why was th© discovery of th© first case of tularemia in a human being so important? Sub Question: d* Miat warning did the doctors give at that time? Ability Involved!

2 - recognizing application of factual information explicitly stated

Index of Difficulty

% of Correct % of Correct Responses Responses on Group I D I *46 Pre-test B *44 1 41 48 33 2 49 46 Index of Reliability 3 51 63 57 4 47 I *50 52 5 63 6 P *25 61 53

For this item study questions regardless of place­ ment were more effective than either undirected reading or pupils reading and preparing their own questions# Shis relative effectiveness did not hold for th® Delay­ ed Test, for Groups III, IV, V, and VI lest more than Groups I and II* Item 27s Why is tularemia considered a serious disease? Main Question: 6. Why is tularemia considered a serious disease? Sub Question:

all

Ability Involved:

2 - recognizing application of factual information explicitly stated

Index of Difficulty

% of Correct % of Correct Responses JD I Responses on Group I *63 pre-test 54 70 1 B *37 56 62 2 16 62 73 3 Index of Reliability 75 60 4 56 69 5 I *56 53 66 6 B *61

This is one of the best items In the test from tl point of view of cruciality, reliability, difficulty and the amount learned as a result of reading and study*

152

Vftiile this item was classified under ability two, the con­ clusion inherent in the question would have also permit©d placing the item under ability slx{recognising con­ clusions}* However, the information necessary to select the correct response is explicitly stated in the selection* Undirected reading -seemed to be about as effective as specific study onestions for immediate performance, Item 28 i What precautions should you take if you had to skin or clean a wild rabbit? Main {^aestiom 7* How can you guard against getting tularemia? Sub Question: a* In what way would you be most likely to get the disease? Ability Involved:

9 « recognizing a problem*

Index of difficulty

% of Correct % of Correct Responses Responses on Croup I D I .97 Pre-test D .94 1 98 98 85 2 99 97 Index of Reliability 99 95 3 4 99 98 I *40 5 95 98 D .50 6 99 98 Yhls item tests the same understanding as I terns 19 and 33* It seemed legitimate to build three items on the one understanding in view of the cruciallty of the idea involved* Because of this erueiality, the author decided to leave the Item In th© test even though the preliminary tryout revealed it to be very easy* Although th© margin of possible gain was small, as shown by high per cent of correct response on th© pre-test, performance on th© Immediate Yost reveals that all group© practically reached th© optimum, wfcltih gain was not dissipated at the end of a month. This is another Item on which th© reliability Increased aa the difficulty increased*

153

Item 29; What kind of insects is believed to spread tularemia among rabbits? Main Questions 3* What is meant by th© first statement in the Foreword, **Tularemia is a clear-out example of a disease of animals which is transmissible to people1*? Sub Questions b* How is the disease spread from one animal to another? Ability Involvedt 1 - recognising verbatim statement of fact Index of Difficulty

% of Correct % of Correct Responses Responses on Group I B X *66 Pre-test J> *53 X 73 58 48 2 62 50 Index of Reliability 3 72 57 78 4 60 1 *48 5 75 56 3 *35 6 73 53

With th© exception of Group II this item Indicates a fair amount of learning* The undirected study group did practically as well as the other groups* Item 301 About how many cases of tularemia wore reported in Iowa during the years 1937, 1938, 1939, and 1940? Main Questions Sub Questions

5* How widespread Is tularemia in Iowa? Hone

Ability Involveds

3 - comparing and evaluating quan­ titative relationships

Index of Difficulty I .52 X) *31

% of Correct Responses on Pre-test 14

Index of Reliability X 0

*33 *32

% of Correct Responses I Group D 1 2 3 4 5 6

45 41 50 49 54 52

33 31 28 29 28 32

k comparison of the performance on the pre-test and the performance on the Immediate Test reveals a fair amount of learning, and performance on the Delayed

154

Tost compared with the performance on the Immediate Test repeals considerable forgetting* Hot© that while they gained less than the other groups^ Groups I and II proportionately lost th© least* Item Sit What might be one explanation of fewer cases of tularemia in Iowa in 1840 than in 1838 or 1959? Main Question} Sab Question:

Hone Hone

Ability Involved*

4 - recognizing an Inference

Index of Difficulty I D

*42 *29

% of Correct Responses oh Pre-test 26

Index of Reliability 1 D

.19 .18

% of Correct Responses Group I D 1 2 3 4 5 6

37 34 40 47 38 37

20 24 24 29 20 24

Ther© was some gain on this item as a result of reading and study* Undirected reading, except for Group 17# seemed to be as effective as other methods of study* Practically nothing of what had been learned was retained after a month Vs time* Item 52? What might be another explanation of fewer cases in 1840 than In either 1938 and 1939? Main Question: Sub Question:

Ion® Non©

Ability Involved:

8 - relating past experience and information to a generalization

Index of Difficulty % of Correct Responses on Pre-test I *17 D *16 14 Index of Reliability I D

.16 .27

% of Correct Responses D I Group 1 2 3 4 5 6

16 16 14 13 17 16

23 24 20 14 19 14

155

It Is possible that this item was affected by the change made is Item 31 following the preliminary tryout* Os the preliminary tryout for Item 31 th© per cents of the correct responses for upper and lower groups were 41 and 17* respectively* "There Is no good explanation* was omitted from this Item, which omrnisslon tended to make It easier and Iten 32 more difficult* Analysis of 300 papers on th© Immediate Test reveals that for Item 32 there was a piling up on th© fifth foil* which was *fh©r© is no good explana­ tion*" In view of the index of difficulty and reli­ ability of the item, it is possible that guessing alone operated* Mote that, with th© exception of Group IV, all of the groups performed better on the Delayed than on the Immediate Test* Item 33: Xf your father likes to hunt rabbits and your mother cleans the rabbits which he brings home, m at would be the best advice which you could give to both of them? Main Question: 7* How can you guard against getting tularemia? Sub Question? a* In what way would you b© most likely to get the disease? Ability Involved?

9 - recognizing problems

% of Correct % of Correct Responses D X Responses on Group I .89 pre-test 91 90 1 D .83 84 89 74 2 91 86 Index of Hellability 3 94 4 96 90 87 5 I *24 88 93 6 D .23 Index of Difficulty

For some reason this item did not function on th© Immediate Test as it did on the preliminary trial. The per cents of correct responses for the upper and lower groups respectively were 97 and 78 on the preliminary test as ©ompared with 94 and 83 on the Iumtediate Test* Obviously, this change In th© functioning of the item affected Its reliability* There was considerable gain over th© previous knowledge score, which gain was maintained after a month* s time*

j|4;

If you really wanted to guard against getting tularemia, which of the following would you be least likely to do?

Main Questions

5 * How widespread is tularemia In Iowa?

Sub Question;

d. What Is the chief source of infection?

Ability Involved;

6 * recognizing a conclusion

Index of Difficulty

% of Correct % of Correct Responses Responses on Group I B I .35 Pre-test D .43 1 39 44 @2 2 34 43 Index of Reliability 3 32 43 4 35 43 I *28 5 37 42 0. *14 6 34 46

Mien comparing the Immediate and Delayed Test results with the results of the pre-test group, it becomes evident that an interference factor was operating. The Item analysis revealed the following; % of Correct Response Upper a# tt0o in the woods on picnics** 4 b* ^Flay withcats and dogs* 4 e* Pick the ticks off your dog1* 45 d. **0o hunting for rabbits” 47

% of Correct Response Lower 14 13 51 22

Evidently fell (e) was so plausible that it attracted a large percentage of both upper and lower groups* personalizing the Item might also have affected its functioning for many of the children may have concluded that since they coul&nH or didn't care to go hunting for rabbits, they would have to solect the next best response. Following the preliminary tryout, one foil, ”To do any one of the above would be very dangerous,” was eliminated because it attracted 50 per cent of the pupils responding to the Item* This change did not improve the q u a l i t y of the Item very much.

157

IfceBl j|&s

Why should you try to avoid feeing bitten fey a flea, a tick, or some blood-sucking fly?

Main Question: 7* How can you guard against getting tularemia? Safe Questions c* What insects shoal'd you fee particularly careful to avoid? Ability Involved*

5 ~ recognizing a cause and effect relationship

index of Difficulty

$ of Correct -% of Correct Responses I Responses on Oroup 0 I *73 Pre-test £ *65 I 70 63 2 72 59 65 Index of Heliability 65 3 66 68 65 4 5 68 68 1 .50 67 6 65 £ *37

The item analysis reveals that approximately 43 per cent of th© lower group selected response (e), 38Because this is th© easiest way to get Tularemia.* This suggests that after reading and studying pupils of low ability were still unable to recognize th© chief source of tularemia* The variation of perform­ ance on both Immediate and Belayed test precludes speculation as to the relative effectiveness of the study procedures*

158 Item 561

Which of the following animals would be the most dangerous for you to have asa pet?

Main Question: Sub Question:

5. How widespread is tularemia

in Iowa?

d* What is the chief source of infection?

Ability Involved:

6 - recognizing aconclusion

Index of Difficulty

% of Correct % of CorrectResponses Responses on Group I D Pre-test I .70 D .69 70 66 1 66 2 71 76 Index of Hellability 72 3 71 4 73 73 5 71 I *19 64 D .15 70 6 72

The results for this item were similar to those for Item 55. However, the reliability was not as high. Performance of all groups was very much alike on the Immediate Test. Item 37: How many of the cases of tularemia In Iowa for which the State Department of Health was able to find th© source of infection were traced to th© wild rabbit? Main Question: Sub Questions

5. How wide spread is tularemia in Iowa? &* What Is the chief source of Infection?

Ability Involved:

3 - comparing and evaluating quan titatlve relationships

% of Correct % of Correct Responses D I Responses on Group I .28 Pre-test 53 29 1 X> *.27 2 20 20 18 24 23 3 Index of Reliability 28 27 4 23 5 25 I .51 18 24 6 D .58 Index of Difficulty

This is on© of the more difficult items. The control group, Group I, was superior on both the Immediate and Delayed Tests. Of the groups using questions Group IV was somewhat superior. Th© amount actually learned by all the groups, though not large, was fairly well main­ tained.

159

Item 38; How do rabbits usually get tularemia? Main Question: 3* What Is meant by the first statement In the Foreword, ’’Tularemia is a clear-cut example of a disease of animals which is transmissible to people**? Sub Questions b* How is the disease spread from one animal to another? Ability Involved:

5 - recognising a cause and effect relationship

Index of Difficulty

% of Correct % of Correct Responses Responses on Group I D I *55 Pre-teat D .50 * 57 1 55 54 28 2 53 Index of Hellability 3 59 51 54 4 53 I *21 5 62 52 B *30 6 56 46

Performance of all groups was quite similar on th© Immediate Test# The marked gain In the amount learned as a result of reading and study was maintained by Groups I, II, and IT* Item 39$ During which three^months have most of th© cases of tularemia been reported in Iowa? Main Question:

S* How widespread Is tularemia in Iowa?

Sub Question: b* During what season of the year Is th© disease most prevalent? Ability Involved:

1 * recognizing verbatim statement of fact

Index of Difficulty I B

*66 *44

% of Correct Responses on Pre-test 31

Index of Reliability I B

*33 *22

% of Correct Responses Group I D X 2 3 4 5 6

69 71 72 72 74 76

55 53 47 46 57 57

This item which was purely factual showed con­ siderable gain In amount learned as a result of read­ ing and study. There was a marked loss, however, as Indicated by performance on th© Delayed Test. All groups exceeded Group I on the Immediate test. The

ICO

i*©suits for Groups 111 and IV, which used main questions only, were considerably below those of the groups on th© Delayed test* Item 40 1

Why have most of th© cases in Iowa been reported during these months?

Main Questions Sub Questions

Bone Hon®

Ability Involved:

4 - recognising an Inference

Index of Difficulty

% of Correct % of Correct Responses Responses on Group D X I *64 Pre-test B *68 65 75 1 58 73 . 2 69 Index of Reliability 66 3 69 4 71 77 1 *37 S 71 67 D .24 6 67 75 With the exception of Group II, which stood highest on the Immediate lest, all groups had better performance on the Delayed than on the Immediate test, which suggests that reminiscence might have been operating* ^teja 41; From 1934 to 1S40 which sections of the state reported most of th© cases of tularemia? Main Questions

5* How widespread Is tularemia In Iowa?

Boh Question; e. Which sectlcns of th© stat© have been most seriously affected? Ability Involved:

1 - recognizing verbatim statement of fact

Index of Difficulty I D

,60 *39

$ of Correct Responses on Pre-test 32

Index of Reliability X D

*34 .25

% of Correct Responses Group I D 1 2 3 4 5 6

56 60 64 59 64 62

36 36 36 41 32 41

161

this is another factual item which showed con­ siderable gain as a result of reading and study and a marked loss with the lapse of a month, the super­ iority of Groups I? and VI on the Delayed tost favored questions following the reading over other placements* Item 42;

Shy have there been fewer cases of tularemia in some sections of the state than In other sections?

Main Question* Sub Questions

Hone Bone

Ability Involved:

1 - recognising verbatim statement of fact

Index of Difficulty I D

*75 *6&

% of Correct Responses on Pro-test 61

Index of Reliability I D

*72 *67

% of Correct Responses I Group B 1 2 3 4 5 6

71 77 87 77 78 77

73 77 77 67 77 73

This is another factual Item* The Idea which it contains Is by no means crucial* From the point of view of difficulty* reliability* and learning produced* there is ample justification for including such an item in a test* However^ from the point of view of measur­ ing the grasp of essential Information* the item is of little value* This item is quite typical of many items found in standard reading tests* Item 43: ithat might be another reason why there are fewer cases of tularemia In some sections of the state than in other sections? Main Questions Sub Question:

Boa® Ion©

Ability Involved:

1 - recognizing verbatim statement of fact

Index of Difficulty I D

*65 .52

% of Correct Responses on Pre-test 50

Index of Reliability

% of Correct Responses i D Group 1 2 3 4

I D

.50

r

6.5

66 69 65 63

54 51 49 53 49

Iher© was little gain over the previous knowledge group* Most of what was gained was lost by th© end of on© month* Reading without the us© of questions seemed to be as ©ffestive as th© techni ques used by the other groups* Jul.* In order to control a disease, what do doctors need to know about itt Main Questions 1* V&iat has mad© it possible for people to know more about tularemia today than was known thirty or forty years ago? Sub Questions a* -What steps have been followed In gain­ ing Information about th© disease? Ability Involved:

6 - recognizing a conclusion

Index of Difficulty I D

.45 .4*7

% of Correct Responses on Pre-test 32

Index of Reliability I D

*31 .37

% of Correct Responses D Group I 1 2 3 4 5 6

40 45 51 47 44 48

42 44 49 50 42 47

fhis item might have been listed either under abilities six or eight* For some pupils It probably required a high level of reading because they related what they read to past experience In order to reach th© correct conclusion. For other pupils, this may have been a factual item, because they had already mad© this generalization In thair study of other health topics. Group I, without the guidance of ques­ tions, turned in the* poorest performance. Of th© groups using questions, th© results for Groups III and I? were the best*

163

Item 451

llhich of the following is a tame statement about tularemia In Iowa?

Main Questions disease? Sub Questlonj in Iowa?

6. $hy is tularemia considered a serious a* Wa&t is the death rat© of th© disease

Ability Involved:

6 - recognizing a conclusion

Index of Difficulty I D

.28 .25

% of Correct Responses on Pre-test 9

Index of Reliability I D

*41 .33

% of Correct Be spouses Group I D 1 2 3 4 5 6

28 21 25 33 23 30

23 18 20 19 20 26

There was a respectable gain as a result of read­ ing and study with a fair degree of retention* Groups I? and VI, using questions following the reading, seemed to have the advantage on this item* Performance on this item which was difficult was still low in relation to optimum learning* Item 46 i What would be a good conclusion to draw about tularemia in Iowa? Main Question: Sub Questions

5* Bow wide spread is tularemia In Iowa? lone

Ability Involved*

6 ** recognizing a conclusion

% of Correct % of Correct Responses I D Responses on Group Pro-test I *58 59 55 1 B .46 51 2 58 45 54 56 3 Index of Hellability 57 56 4 48 54 5 1 .48 50 48 6 D *50 Index of Difficulty

From the point of view of difficulty, reliability, amount of information gained and retained, this seems to be a very good item* Groups II, V, and VI performed better on the Delayed than on the Immediate Test* Perhaps the reason Groups V and VI did not make a better showing on the Immediate Test was because one of their subordinate questions may have acted as an interference factor*

164

Itegi 47; What is on© way to avoid tularemia? Main Qtteatlonr 7. How can you guard against getting tularemia? Bub Questions b# Of what should you be absolutely certain before eating idtld rabbit? Ability Involved;

2 - recognising application of factual information explicitly stated

Index of Difficulty 1 D

.8© #82

% of Correct Responses on Pre-test 60

Index of Reliability I B

#85 #46

% of Correct Responses Group D I 1 2 3 4 5 6

89 81 90 90 88 89

85 76 80 84 SB 84

This item shows a gain as a result of reading and study# Groups III and IV, using main questions, held the advantage on the Immediate Teat, but that gain was dissipated on th© Belayed Test. After a month1s time, all groups still had a good command of tiie Information tested# Item 48 1

Which Is th© best statement about the effects of tularemia?

Main Question; disease?

6. Why is tularemia considered a serious

Bub Question; b« How long does it sosaetimes take to recover from the disease? Ability Involved;

2 ~ recognising application of factual Inforiaation explicitly stated

Index of Difficulty I B

#60 *54

% of Correct Responses on Pre-test 12

Index of Reliability I D

*69 *64

% of Correct Responses Group I D 1 2 3 4 5 6

62 54 73 70 63 56

60 47 61 59 55 52

Shis is superior item from the point of view of quality and functioning* Considerable gain was made by all groups* fb© us© of main questions seemed to be more effective for both Immediate and Belayed tests* Th© results for Croup I* undirected by study questions, were as good as or superior itthose of Croups II and ¥1 on the Immediate Testj and to" Groups II, I¥, V, and V3E on the Deh yed test* Item 49s How does the Iowa State Department of Health serve th© people of Iowa? Main Question: Sub Question:

lone Bone

Ability Involved:

6 - recognising a conclusion

Index of Difficulty I B

-i

*51 *£$1

% of Correct Responses on Pre-test m

Index of Reliability I D

*54 *47

% of Correct Responses Group I D 1 2 3 4 5 6

51 47 55 58 55 61

56 50 58 48 57 53

Ihile there were no main nor subordinate questions covering this Item, the main questions 5, 6, and 7 would direct attention to th© understanding tested* Perhaps this accounts for the performance of Groups III through VI which surpasses that of Groups I and II on the Immediate test* Groups I, II, III, and V did better on the Delayed test than on the Immediate test which suggests remini scene© might have been operating* Not© that Groups IV and VI lost 10 and 9 percentage points respectively*

Item 50s Which of th© following statements best describes tularemia in Iowa? Main Question: Sub Question:

5. Bow widespread is tularemia in Iowa? Bon©

Ability Involved:

3 * comparing and evaluating quan­ titative relationships

Index of Difficulty

% of Correct % of Correct Responses Responses on Croup I D I *44 Pre-test B *45 51 49 1 28 2 50 41 Index of Reliability 3 52 4© 4 47 46 I *42 5 48 46 B *48 6 55 55 In view of the idea tested and th® difficulty of th© Item* there was a marked gain In th© amount learned* Reading without th© guidance of questions seemed to be about as effective as th© other methods* Except for drcup II* forgetting was negligible* Item 51: What should everyon© who eats wild rabbit know about the way It should be cooked? Main Questions 7. Bow can you guard against getting tularemia? Sub Question: b* Of what should you be absolutely certain before eating wild rabbit? Ability Involved:

2 - recognizing application of factual Information explicitly stated

Index of Difficulty

% of Correct $ of Correct Responses I Responses on Group B Pre-test I .91 95 89 1 D .85 2 91 84 93 95 94 3 Index of Reliability 4 94 92 95 5 96 1 *53 95 6 93 B *68

'This is a very easy Item with a relatively high index of reliability* In view of the narrow margin of possible gain* marked learning took place, most of which was retained after a month *s time. Th© study procedures of all groups seemed to function about equally well.

£67

Itam 52 s About how many oases or tularemia may we expect In 1942? Main Questions Sub Question:

Mon© Hon©

Ability Involved:

7 - suspending judgment

Index of Dlfflculty I 3

.79 .70

% of Correct Responses on Pre-test 66

Index of Hellability I D

*27 *36

% of Correct Responses Group I D 1 2 3 4 5 6

©4 75 76 77 71 80

72 72 75 74 ©9 81

A fair amount was learned and retained by all groups* The group undirected by study questions per­ formed better than th© others on the Immediate Test* For ©very on© hundred people in Iowa who have tularemia what would be the best explanation of how they got the disease? "? Main Question:

5* How widespread Is tularemia in Iowa?

Sub Questions

d* What is the chief source of infection?

Ability Involved:

3 - comparing and evaluating quan­ titative relationships

% of Correct % of Correct Responses Responses on Group I D Pre-test I .43 38 31 B *34 1 2 37 25 32 Index of Reliability 34 35 3 37 4 40 5 32 30 I *50 30 6 37 B *34 Index of Bifflculty

The learning of th© idea tested by this Item was not pronounced? except for Group III about half of what was gained had been forgotten by the end of a month*

/"*Q JLtSO

Item 54s

Which of th® following would be the most likely to reduce th© number of eases of tularemia in this country?

Main questions 7* How ean you gpard against getting tularemia? Sub Questions

all

Ability Involved:

6 - recognising a conclusion

Index of Difficulty I ©

*62 *65

% of Correct Responses on Pre-test 49

Index of Reliability I D

*20 *11

% of Correct Heaponses D Group I I 2 5 4 5 6

58 54 64 68 63 63

68 68 67 69 63 67

A fair amount of learning and good retention was registered by all groups on this item* Except for Group V, all groups on the Delayed Test, exceeded their performance on th© Immediate Test* This Is another of the items that shows negative acceleration* Item 55: For every one hundred people In Iowa who have tularemia about how many would you expect to recover? Main Question; Sub Question;

6* Why Is tularemia considered a serious disease? a* What Is the death rat© of the disease in Iowa?

Ability Involved;

3 - comparing and evaluating quan­ titative relationships

% of Correct $ of Correct Responses I © Responses on Group Pre-test I *64 65 51 1 D *48 57 43 2 16 60 71 3 Index of Reliability 50 64 4 53 63 5 2 .59 54 65 6 © *60

Index of Difficulty

An unusual amount of learning accompanied by approximately twenty per cent loss characterizes th© performance of most groups on this item* Except for Group III, the performance of the control group was equal or superior to the resulis of the other roups*

Itoga 56 s Why should doctors today be better able to rocognlgse tularemia than they were five or six years ago? Main v^uestions Sub Question?

Hone Hone

Ability Involved?

4 - recognising an inference

Index of Difficulty

% of Correct % of Correct Responses Responses on Croup X D I *68 Pre-test D .68 1 70 63 48 62 2 61 Index of Reliability 69 3 67 4 72 62 I .18 5 73 64 D *42 6 70 64 A fair amount of learning and retention was shown on this Item. Crroups II and III retained most of what they learned* Undirected reading seemed to b© as effective for selecting the correct response to this item as was the other techniques* Item 57?

From what you know about the history of tularemia* what conclusion could you draw?

Main Questions Sub Questions

Hone lone

Ability Involved:

7 - suspending judgment

Index of Difficulty

% of Correct % of Correct Responses D Responses on Group I I *25 Pre-test D .34 19 23 I 28 28 18 2 20 Index of Reliability 3 21 21 4 23 27 24 X .07 5 25 6 28 D .15

fhis Item was so difficult and the reliability so low that speculation as to th© Interpretation of the performance would be futile. On th© preliminary tryout the per cents of correct responses for the upper and lower groups respectively were 46 and 16. Such performance did not hold for the experimental grou p s.

.

170

Item 58: What Is the best conclusion to draw about tularemia In Iowa? Main Questions

5. How widespread is tularemia In Iowa?

Sub Questions c. Which sections of the state have been most seriously affected? Ability Involved:

6 - recognizing a conclusion

Index of Difficulty I D

.53 .50

% of Correct Responses on Pre-test 36

Index of Hellability

% of Correct Responses Group I D 1 8 3 4

1 D

.31 *58

5 6

54 39 48 45 55 48

54 46 46 50 50 50

With the exception of Croup II considerable learning took place and the amount gained seemed to be fairly well retained. Croups II, IV, and VI performed better on the Delayed than on the Immediate Test* A single purposeful reading seemed, as effective or superior to other techniques on both the Immediate and the Delayed Tests* Item 59j In what way, other than the bite of an animal, can the tularense germ get into the body Main Questions 3. What Is meant by the first statement in the Foreword, "Tularemia is a clear-cut example of a disease of animals whioh Is transmissible to people"? Sub Question; e. In what three ways may people get tularemia? Ability Involved; Index of Difficulty

1 - recognizing verbatim statement of fact

% of Correct % of Correct Responses Responses on Group I D Pre-test I *84 87 84 1 D .72 87 86 64 2 85 81 Index of Kellability 3 89 84 4 87 78 1 *41 5 88 80 6 D *36

Considerable learning took place, such gain being about the same for the group undirected by study questions as for the experimental groups on both imme­ diate and Belayed Tests* Subordinate questions did not seem to be as effective ms on the Belayed Test.* Item 60s How does tularemia differ from such diseases as whooping cough, tuberculosis, and heart disease? Sain Question: Sub Question:

Mono Hone

ability Involved:

8 - relating past experience and Information to a generalisation

Index of Difficulty

% of Correct % of Correct Responses I Responses on Group D I *09 Pre-test D *18 I 12 13 14 2 13 14 Index of Kelt ability 3 16 9 4 18 15 I -*05 5 11 12 6 B *15 16 15 This Item was so difficult and the reliability was so low that nothing meaningful can be said about it* Group I? was superior to the other groups on nine­ teen of sixty items on the Immediate Test*

The number of

Items on which the other groups excelled were; Group V, 13$ Group VI, 11& Group II, 8; Group III, 8; and Group I, 5. On the Delayed Test Group VI was superior to the other groups on twenty-four Item. in rank order were:

The superiority of the other groups Group IV, 14; Group V, 12 j Group II, 11;

Group III, 8; and Group I, 6*

Ho conclusions can be drawn

from these data because the differences In per cent of correct response from group to group for many items was negligible, In some cases the difference being only one percentage point.

The number of items on which there were marked differences in percentage points was too few to justify any meaningful conclusions.

On nine items; namely, 8, 12, 13,

27, 31, 48, 47, 48, and 54, a superiority of three percent­ age points or more was in favor of the use of main questions. Of these nine items, five were 3factual1*; three, ^Interpre­ tative”; and one 14associative.*

On fifteen Items the super­

iority favored the use of main and subordinate questions* These were items I, 2, 3, 5, 6, 7, 16, 18, 19, 25, 30, 32, 39, 80, and 87*

Of these fifteen items, six were classified

as **factual”; seven, ^interpretative”; and two, ”associative*** Considering placement of questions, again the showing superiority

number of items/was too small to permit conclusive statements. On seven Items, superiority of three or more percentage points for one group over another favored the placement of questions before reading*

Of these seven Items, five were

classified as ”factual” and two, as ^Interpretative.**

For

nine items, there was a superiority of several percentage points in favor of placement of questions after reading.

Of

these nine items, two were classified as "factual”; five, as *interpretative®; and two, as ”associative.*’ The tendency for questions before reading to facilitate performance on ^factual” items and questions after reading to facilitate performance on ^interpretative*1 Items raises the question; Does the placement of questions benefit the performance on different kinds of Items?

If the situation which existed

for the groups of seven and nine items referred to above had obtained for all items in the test, and If the differ­ ences in per cent correct for *factual” Items by the group having questions before reading had been practically the same as the per cent correct for 11interpretative” items by the group having questions after reading, the relative effectiveness of these two groups would have been cancelled. On eight items, numbers 12, 23, 25, 32, 34, 40, 49, and 54, in general, the scores on the Delayed Teat were better than on the Immediate Test*

Of these items only 12

and 25 were classified as "factual*” The retention after a period of one month was ex­ cellent on twelve Items; namely, 2, 4, 10,,11, 24, 35, 36, 44 , 46 , 50 , 51, and 52*

It should be noted thati only four

of these twelve items were the so-called "factual” Items, This tendency for a slower rate of forgetting on items not testing detailed factual information is in agreement with Spitzer’s^5 finding In regard to the few generalization questions in M s study* For six Items, 16, 31, 41, 43, 57, and 60, which were distributed equally among the three classifications being considered, retention was low*

15

H* F* Spltzer, A Study of Retention in Beading, doctor’s dissertation, State University of Iowa, Iowa City, 1938.

The amount gained as a result of reading and study in relation to the margin of possible gain may b© considered poor on eleven items* 52, 34, 37, 46, 57,

They are 5, 7, 8, 13, 15,

and 80; five of which had low reliability*

Twelve items, numbers 1, 3, 6, 19, 22, 25, 27, 28, 30, 48, 55, and 59, showed what may be considered excellent gain in relation to the margin of possible gain*

Of these items on

which marked learning resulted from reading and study, nine were of th® so-called "factual® type.

All nine Items had

relatively high reliability coefficients, ranging from 38 to 69.

Of the eleven Items which showed poor results In

th© amount learned, only throe were "factual," while four were "interpretative" and four were "associative,"

These

facts raise the issue as to whether or not items testing factual information may not show greater learning than Items testing ability to interpret and draw conclusions, Unfortunately, the data available in this investigation Is insufficient to supply a clear-cut answer to this question* If the inference is tenable, then the implication, from the polnt-of-view of classroom teaching, would he that the higher the level of learning required, the more essen­ tial it is that something, more than just reading and study­ ing under the guidance of questions, be don© to insure that learning.

175

Chapter Y SUMMARY AID GORCLUSIOHS

*MSM£3s S*Lila£«ISi& The purpose of this investigation was to deter­ mine the relative effectiveness of questions in directing i study* Two aspects of the problems, the kind of questions and placement of questions were investigated.

The former

phase of the experiment "compared the effectiveness of pupils* quest!ojas as against investigator’s questions and main ques­ tions as against main and subordinate questions* An attempt was mad© to approach a desirable teach­ ing situation*

Th© reading selection, study questions, and

test items were checked against a list of understandings which six competent people agreed to b© important.

All groups

were given the same purpose for reading and study, a general problem, flHow can w© b@ protected against tularemia??f Th© subject matter was chosen for its interest, cruciality, and appropriateness to the age level*

In preparing the study

questions, directions to the pupils, and the multiple-choice 1 test for this investigation, th© author kept in mind Dewey’s statement that ”•*.the important thing is to test a) progress in under standing subject matter, b) ability to use what has been learned as an instrumentality cf future study ^Dewey, John, How We Think. I). C. Heath and Co., Boston, 1933.

±76

and learning, ©) improvement in the general habits and attitudes that underly thinking, curiosity, orderliness, power to review, to sum up and to define, openness and honesty of mind, etc* Th© selection used and the purpose given the pupils to guide them in th© acquisition of certain ideas were designed to avoid dead-level reading and study and to male© pupils feel that those ideas were important enough to be remembered. For that reason the test contained a preponderance of items which emphasised thorough learning and critical thought rather than verbal memory. The subjects were one thousand forty-fiv© eighth grade pupils in thirteen schools from seven different school systems,

kll eighth grade pupils within each school wore

randomly assigned to six groups; used no study questions.

Group 1, th© control group,

The pupils in Group v C wrote their

own questions while Groups III, IV, V, and VI were provided with study questions.

The experimental groups were directed

to try to determine the correct answers mentally*

a multiple-

choice test was administered immediately following the study period and was repeated on© month later. The experiment was designed to permit the use of analysis of covariance in treating the data.

A subjective

appraisal was made to determine whether any of th© groups used their respective study methods with greater facility than the others*

A report of the item analysis included

the index of difficulty, th© coefficient of reliability, th© per cent of correct responses on the pre-test, and

177

th© per eent of correct responses by each group on the Irrsnedlat© and Belayed Tests* The reliability of the test on tularemia was determined by means of th© RIehardson-Kuder formula.

A

pre-test was given to a random sample of classes from the schools in which th© experiment was conducted to determine the amount of

revious knowledge of tularemia possessed

toy eighth grad© pupils* Smsaary M

Jg§|g||||£g £g

J2jgg|g££

1* Th© estimate of reliability of the test on tularemia was*83* Since this reliability was based on results obtain­ ed fro® testing pupils in a single grade, it may toe considered fairly high. 2m The correlation between the Initial and Immediate Tests was found to to© .73; that between the Initial and Delayed Tests, *67; and that between the Immediate and Delayed Tests, .80* 3. The difficulty indices of the Items on the Immediate Test ranged from *09 to .99.

For th© Delayed Test, th© rang©

was .16 to .99. A few of th© items w©r© too easy to discriminate* These Items were left in the test following the pre­ liminary tryout because of their cruel allty and to insure enough easy items on the Delayed Test*

Results

showed that th© test would have been improved had thes© Items been mad© more difficult.

Two items were

too difficult*

Both of these items called for relat­

ing information to oast experience* 4* The coefficients of reliability, computed for each item, from a table prepared by ilannigan, ranged from -*05 to *72 on the Immediate Test and from -*04 to *68 for the Delayed Test. All six of the Items classified as low reliability*

associative* had

Three Items in the test were so easy

that It was impossible to find the reliability co­ efficients with a high degree of accuracy*

Additional

preliminary tryouts should have been mad© and th© faulty items either improved or eliminated. 5* T!h© test for Interaction between methods and schools on th© Immediate and Delayed Tests resulted in no signifi­ cant difference* Therefore# on© may retain as tenable the hypothesis that th© methods, as used in this investigation, did not differ from school to school by amounts greater than could be attributed to chance*

The us© of the

Interaction variance as the error term makes possible generalisation of the statistical findings based on analysis of covariance to the population of schools of which the schools participating in this investiga­ tion, were a sample* 6* The value of F as a test of the over-all effectiveness of the respective methods as measured by the Immediate Test was not significant at the 1 per cent level.

Consequently, it may be stated that for inasediate learning, there was no significant difference in favor of any of the methods under investigation.

Although

the value of F was significant at the 5 per cent level, this level of confidence was rejected as not being sufficiently high to warrant comparIsons of individual means because of the large number of means to be com­ pared and the possibility that significant differences which might have been found would be due to chance. ^ien the over-all effectiveness of the methods, as measured by the Delayed Test, was evaluated, the methods variance was less than the within groups variance when the Initial Test scores were held constant* This indicated that the effectiveness of the methods as aids to remembering, In relation to ability to learn, was no greater than could be attributed to chance* A similar comparison for the Delayed Test when the Imme­ diate Test measures were held constant was made*

The

difference found fell short of significance, even at the 5 per cent level* Therefore, there were no significant- differences in the relative effectiveness of the methods with respect to the amount retained in relation to the amount learned. Even though there were no significant differences between the six methods on either the Insnediate or Delayed Tests,

there seemed to he a positive trend in favor of the groups using study questions as shown by the differ­ ences in the adjusted means for the respective groups. The differences between the adjusted means of Group I and Groups III, IV, V, and VI were 1*924, 2.765, 1.793, and 2.379 respectively.

The differences between

the adjusted means for the same groups and that of Group II on the Immediate Test were 2.023, 2.87c, 1.897, and 2*466 respectively.

All the differences favored

the groups using study questions prepared by the inves­ tigator*

Groups IV and VI, using questions after

reading, were somewhat superior to Groups III and V, using questions before reading.

The differences

between the means of Groups III, IV, V, and VI were all small, ranging from *147 between Groups III r.nd V to .836 between Groups III and IV*

All the differ­

ences between the means on the Delayed Test were small, but in every comparison the slight superiority favored the groups using main and main and subordinate questions* The amount learned as measured by the Immediate Test In relation to what could have been learned was relatively small for all groups. Hie per cent of correct responses when the previous knowledge score was not subtracted ranged from 58 for Group I to 61 for Group VI.

After the

previous knowledge score had been subtracted the per cent learned of what could have been learned ranged from 27 for Group II to 33 for Group VI* Hence, as measured by this test, there was relative­ ly low performance in relation to optimum perform­ ance*

This is In agreement with results usually

reported for other methods studies*

The perform­

ance of all groups was very similar with slight differences favoring the groups using main and main and subordinate questions* The amount retained after a month* s time expressed as a per cent of the amount learned was 72 for all groups combined* There was a striking similarity of the per cent retained of the amount learned for all groups* The group which took the test for previous knowledge made a mean score of 24.729* This mean score probably represents more knowledge than that actually possessed by the eighth grade pupils who had not read the selection on tularemia. The factor of guessing alone would account for approx­ imately half this score*

Knowing that the wild rabbit

is the chief source of tularemia probably made the

iSw

-■ !Q*

correct responses to certain items seem plausible even though understanding of the ideas tested might have been hazy*

Per want of a better measure, however, that

mean was token as the measure of previous knowledge possessed by the six groups* 13# The questions written by pupils in Group II indicated that many eighth grade pupils failed to distinguish between main and detailed questions* Only 39 x3®2*

the questions were large enough

to cover the main ideas In th® paragraphs* the statement of th© questions was good*

In general, There was

evidence to show that many of th© pupils did not finish writing their questions, and naturally then, did not try to mentally answer their questions*

The number of

questions written by th© pupils on the seventeen para­ graphs In the selection varied from two to twenty* The median number of questions per pupil was eleven. 14* Examination of the question sheets for Groups III, IV, ?, and VI suggested that th© pupils in these groups did not make the best possible use of their questions. On many of the papers cheek marks were placed before all of th© questions, which If the pupils were follow­ ing directions, meant that they thought they knew all th® answers after the first reading. the test revealed this not to be true#

Performance on From 8 to 12

per cent gave no evidence of using the questions at

±33

all*

As many as one-fifth of th© pupils in Groups III

and IV and two-fifths In Group 7 and VI did not complete their study as indicated by th© absence of plus signs in front of the main and subordinate questions*

Such

evidence, however, does not necessarily mean that they did not have time to eompl©t© their study* 15* The number of Items for which there was marked superiority in favor of any group was so small that generalisations couidJJW made, but th© slight trend pointed to placement of questions tisafactor in determining what pupils looked for ^ e n they read under the guidance of questions* Group IV, using main questions after reading, was slightly superior in performance to the other groups on the Immediate Test while the performance of Group VI using main and subordinate questions after reading was, slightly superior on th© Delayed Test.

Both

Groups IV and VI appeared to do bettor on items requir­ ing interpretation than did Groups III and V* 16, On eight items the groups, in general, scored higher on the Delayed Test than on the Immediate Test* The items In the test were classified as nfactual,® "interpretative,* and "associative."

Of these eight

items only two were "factual." 17. Retention for eleven items was small* That a number of these items had low reliability might have been a factor in the scores*

Only three of

these items were of the "factual" type.

184

18* Excellent gain In relation to th® margin of gain was noted for twelve items* Nine of these items were of the **factual * type*

The

possibility tinat tests of factual information may yield higher scores than tests designed to measure ability to Interpret* understand, and relate that information to past experience is noted*

lAiiyyisyiii The results of this study are subject to th© fol­ lowing limitationss 1. Th© nature of the experiment made it practically impos­ sible to exercise

adequate

control over the personal

variable* Even though an effort was mad® to set up highly re­ strictive specifications to guide in the construction of superior materials, the investigator has no assur­ ance that someone els© following the same specifications and using a hl$a©r degree of ingenuity might not have built materials superior to those used in this experi­ ment*

If such were the case, conceivably, different

results would have been obtained.

The above limit­

ation is not peculiar to this investigation.

On th©

contrary, by the nature of the problem, it is common to most methods experiments* 2* Th© same test was used to measure retention as was used to measure immediate learning*

185

This limitation was not considered serious since the I effects of repeating the same test have been found to be negligible. 3* Although the reliability of the Immediate Test as a whole was considered satisfactory, a number of the in­ dividual items in the test had low reliability co­ efficients. Several of th® items were too easy to require much thinking.

A few items were too difficult to permit

thinking.

This limitation should be kept In mind

in Interpreting the performance on individual items* 4. In Interpreting the performance of the different groups, on® must recognise the extent to which studying by one or several of the methods required by the Investigation was now to the pupils* This raises the question as to what their performance would have been if the groups had had training In using the methods employed.

Frequent observation of

classroom procedure has convinced th® investigator that most eighth grad© pupils have at some time or other, studied by all of these methods.

It is pos­

sible, but not very probable, that most pupils have attained a hi^i degree of efficiency in their use. 5. Hie previous knowledge group was not an Integral part of the experiment.

136

This fact does not affect th© data as treated by eovari&nee, but may place limitations on the retention and Item analysis aspects of th© investigation. That the pre-test score represents more than th© knowledge of tularemia possessed by eighth grad© pupils should be kept in mind while interpreting th© sections following the analysis by covariance section. 6. This study was probably more highly motivated than Is usually th© ease in school work. The care exercised in making th© content on tularemia soem Interesting and crucial to th© pupils, the specific purpose given to guide all groups in their study, and th© specific directions given to each group were directed toward motivation.

On th© other

hand, th© optimum motivation and direction was probably not reached because many of the pupils, undoubtedly, regarded th© questions as more or less externally Imposed, and hence, may not have dealt with them as seriously as they would have dealt with questions which they themselves had formulated In solving this problem. 7. The experiment covered a relatively short learning time, and the results were appraised by means of one test only. Hi© results might have been different had the experi­ ment been extended ov©r a longer period and if the results had been appraised by several instruments.

£onoJu£|ong Th© conclusions of this investigation, subject to the limitations previously listed, may be stated as answers to five questions* 1* Did any of th© methods of study produce results In terms of the amount learned, which were statistically sur-erior to those of th© other methods? Study under the guidance of main and main ana sub­ ordinate questions, whether placed before or after the reading selection, we*

not statistically superior

to either a single, purposeful reading or to the pupils* writing the main questions answered by informa­ tion In the paragraphs*

This was true when th© results

of learning were measured immediately after the initial learning period, and when measured on© month later* These results are not in agreement with the findings as reported by Washburn/, holmes®, and Boeder*^

A critical

%ohn H* Washburn^ 11The Use of Questions in Social Selene© Material,* Journal of Educational Psychology. 20, May 1929, pp. 321-359. ^Eleanor Holmes, Relative Merits of Heading Guided by Questions Versus Careful Reading ayid Re-reading without Questions, master’s th©3is, University of Chicago, 1989* B« Reeder, A Method of Directing Children* s Study of Geography. Contributions to Education, Ho* 193, Bureau of Publications, Teachers College, Columbia University, Hew York, 1925*

188

review of these investigations, presented in Chapter II, revealed weaknesses in the construction of the learning materials, the design, and experimental control of variables which may, in part, account for this lack of agreement* 2* Did any of the methods of study show slight superiority over any of the others? The groups which used questions prepared by the in­ vestigator were consistently superior to the group reading th© selection once under the guidance of pur­ pose and to the group which wrote Its own questions* Though not significant, th© differences on th© Imme­ diate Test seemed to favor th© use of main questions presented after reading, with th© us® of main and sub­ ordinate questions placed after reading a close second* After one month the differences favored the same two groups but In reverse order*

Only small differences

existed between the four groups using main and main and subordinate questions. 3, Did any of th© groups seem to use their assigned proce­ dures with greater facility than the others? In all th© experimental groups about th© same number of pupils appeared to have followed their directions. More pupils in Groups III and Ilf, the groups using main questions, finished their study than in the other groups*

Because of th© nature of these procedures,

this was the situation which one would expect.

Evidence seemed to show that many pupils using the questions prepared by the Investigator failed to think them through critically before marking them.

This

would suggest that under present methods of study, eighth grade pupils have not been made to feel respon­ sibility for developing on their own account answers to specific questions*

Failure to use questions pre­

sented in printed form independently may be due to lack of training in this type of study or may be due to the emphasis the school places upon oral questioning at the expense of independent reaction to printed ques­ tions# Examination of the questions written by pupils in Group II suggests that eighth grade pupils aro un­ able to distinguish between main ideas and details* Did the groups perform differently on the individual items in the test? The number of items on which there were differences of several percentage points in group performance were too few to Justify any conclusions. Did any of the groups perform differently on items classified as T,factual,* "interpretative,* and "associative"? The "associative" items were too few in number to permit any conclusions as to the difference in per­ formance of the respective groups on this kind of item.

y / Ther® seemed to ba a tendency fop pupils in all groups - to show a greater gals in amount learned on ^factual* items than on ^Interpretative*1 items, and the amount forgotten after a month* s time also seemed to he greater for factual items*

This trend, if true,

would suggest that the higher the level of learning ‘ required, the more essential it is that something other than just reading or studying under the guid- ' ane© of questions be done to facilitate that learn** Ing*

If this inference is tenable, then the relative­

ly marked loss in amount learned as shown in retention c fl 17 studies by Spitaer ,u3pene©r , and Tiedemsn , in part may be a function of the preponderance of factual items in th© tests usod by these investigators. A most significant finding of this study was that merely reading th© material with intent to understand and remember was about as effective as reading and studying under the guidance of questions.

It should be noted that the

pupils following the single, purposeful reading procedure

P* Spitzer, A Study of Retention in Reading, doctor1s dissertation,"State University of Iowa, Iowa City, 1938* 6 IS, M* Spencer, The Retention of Orally Presented Mater­ ials. doctors dissertation, State University of Iowa, Iowa Oity, X94G,* 7 K. R, T1©daman, A Study of Retention in Geography, doctor*s dissertation, State University of Iowa, Iowa flity, 1941.

actually spent less time in study than the experimental groups for while the latter were following assigned study procedures, th© former was taking th© test and reading extraneous material Th© amount learned by any of th© six groups, exper­ imental as well as control, was relatively small*

This find­

ing is not peculiar to this investigation, but rather is characteristic of initial learning experimeiits conducted in school situations*

If learning is to be adequate, t e initial

study period has to be used more efficiently than was th© case in this experiment* If this condition of learning can­ not be sat isfied, th|ya additional teaching is all the more essential*

In all probability, both initial learning and

follow up work hood to be improved* Educational ImpItcations Studying by any method encompasses more than just th© method*

A poor method of study directed by a superior

teacher who provides for individual differences, who en­ courages student initiative and responsibility, and who is able to establish rapport between teacher and pupil is like­ ly to produce results superior to a highly effective method of study directed by an Inferior teacher* While th© findings of this study do not disprove th© value of questions in directing learning, they do re­ strict the confidence which can be placed In the extravagant claims which have been made in their behalf*

Questions per

se will not secure effective study*

A common technique

employed by both teachers and authors of textbooks is to provide questions to guide the study of the lesson,

The

assumption is made that merely making questions available to pupils insures better learning than does reading.

The

evidence supplied by this investigation casts doubt cn this assumption*

The relative superiority of the respective

groups over th© group which read under the guidance of purpose was slight.

Xf questions are to be used to advantage,

then something more than just reading and mentally determin­ ing the answer must be done. Bata presented in this Investigation do not necessarily prove that questions of the types used by the experimental groups represent ineffective aids to learning.

They do,

however, shew that questions of the type used, regardless of placement are not as effective in directing learning aa has been commonly assumed by educational theorists.

If questions

are to be used to advantage, then their uso must be much more critical and. vigorous than was demonstrated by this ^experiment.

Possibly training in more efficient use of ques­

tions Is needed before their effectiveness in dirooting learning Can be determined with certainty. Th© investigator, through obsorvaticn, secured some evidence that over a long period of time pupils can be trained to xise effectively questions of the type under in­ vestigation.

In one school in which a preliminary tryout

was made the pupils had been taught by the problem method since entering kindergarten.

The method of attack on the

study questions by these pupils was quite different from that observed in a second school,

In th© former school th©

pupils took more time to go through, the questions*

There

were evidences that they were trying to answer the questions to themselves before placing a check mark before them* They had more plus signs before the questions than did th© pupils In the other school indicating that a question was not checked until th© pupils were relatively sure that they had satisfactory answers*

The second time through the

questions, th© pupils referred to the selection often before placing plus signs before the questions. In the latter school th© pupils placed check marks in front of the questions almost as fast as they could read them, and one© through the complete list showed little inclination to go back to racheck and refine answers. The author is of the opinion that certain common school procedures have developed attitudes in pupils which prevent them from using efficiently questions of th© types investigated* Much of school Instruction is of th© oral question and answer variety in which th© suggestions for answers are 8 embodied In th© questions themselves. Th© prevailing practice

8Soshlchi Yamada, Study of Questioning," The Pedagogical Seminary. XX, June 1913, pp. 129-186.

of teachers seems to b© to accept answers to questions without requiring of the pupils highly critical and discrim­ inative attitudes of evaluating, comparing, and contrasting to determine the validity of those answers*

Such practice

may seriously interfere with pupils developing and exercising ability to react independently to questions in printed form* ** Yamada has aptly stated: ffIY this kind .of interrogation (suggestive ques­ tions by teachers permitting non-critical answers by pupils) continues for month and years, the child will become a puppet and passive in his thinking, instead of active, self-assertive, and independent of external suggestion*”® He further condemns th© ^pernicious practice” of ^helping or directing pupils too much in their recitation by questions or otherwise” and asks, *Should it then be a matter of surprise that when left to themselves to think and to decide or solve a problem they dawdle and get nowhere*” Teachers, supervisors, and administrators need to recognize that tjher© are many times when a single, purposeful reading is a desirable study procedure*

In terms of time re­

quired by both teacher and pupil, the relatively small gain in favor of directed study, and the desired outcomes, it seems Ibubtful whether study directed by teacher-given questions can b© recommended for preparation of all or even moat of the lessons at the eighth grad© level*

This conclusion may be tentatively hold

until pupils can b© taught to us© questions more effectively than was revealed by this experiment* Ibid.. p. 167.

BIBLIOGRAPHY

15)6

BIBLIOGRAPHY 1. Bain, Alexander, Practical Bssa^s, D. Appleton and Company, Hew York, 1884* 2* Barton, W* A*, Outlining as a_ Study Procedure, Con­ tributions to Education, Ho* 4iT, Bureau of Publications, Teachers College, Columbia Univer­ sity, Hew York, 1930* 3* Beauchamp, W # L*, A Preliminary Experimental Study of Technique In'"the feasteryof Subject ??atter in' ^lemen'tary^hysical'"Science , Studies in Second­ ary' Wu'cation,'Vol* I, ITniversity of Chicago Press, 1923, pp. 47-87* 4* Bird, Charles, Effective Study Habits. The Century Company, Hew York, 1931. 5* Breed, P# S., "Measured Results of Supervised Study,” School Review, 27, March and April 1919. pp. 1866. Briggs, D* H., The Influence of Certain Methods of Making the ^ssignmentT~on Study and Learni ng~*Process in the Social Sciences. 'doctorys dissertation, State University of Horth Carolina, Chapel Hill, 1931* 7. Brink, William G., Directing Study Activities in Sec­ ondary Schools, Doubieday D'oran , and Company , Inc., harden U$fcy,1937 « 8* Brownell, ®# A*, A Study of Supervised Study, College of Education Bulletin, Ho. 26, bureau of Educa­ tional Research, University of Illinois, Urbana, 1925* 9* Butterweck, J* S., The Problem of Teaching High-School Pupils How to ^tudy.^on trTbu tjons to Soucation, Ho. $37, BureatTcf Publications, Teachers College, Columbia University, Hew York, 1926* 10* Coll, John, A» Analytical Study of Modern Contributions on theHFroblem of hHqw to ‘StudyV'1 master1s thesis, Unive rsityof “Pittsburg, 1$3Ej• 11* Crawford, G* C., Methods of Study, published by the author, 1926," ,r'T"r 12* Crawford, C, C., "Relative Values of Reading and Out­ lining as Methods of Study,” Sduicational Method, 8, May 1929, pp. 433-438.

107

13. Crawford, G« G., Hamren, L* H*, ^Experiment with the Use of Printed Study Guides,*1 Educational Method, 9, June 1930, pp. 541-544* 14* Deiek, G*, Jones, E.*, A Study of Distinguished HlghSchool Pupll¥ in iowat Suiietin N o . 46, United States Bureau of Education, Washington, D. G*, 1923* 15. Dewey, John, How We Think. D. G. Heath and Go*, Boston, 1933* 16* Bietze, A* G*, Jones, 0* B., ^Factual Memory of Sec­ ondary School Pupils for a Short Article Which They Read a Single Time,11 Journal of Educational Psychology* 22, Hoy ember and~tTeeember 1931, pp.&$6-598, 667-676. 17* Distod, H* W., f,A Study of Reading Performance of Pupils Under Different Conditions on Differ­ ent Types of Material* nJournal of Educational Psychology. 18, April 1927, pp. ^47-258* 18. Dynes, J* J*, Comparison of Two Methods of Studying History,tt Journal of Experimental Education* 1, September 1&3&, pp * 42-45« ' 19* Barhaht. Lida. Teaching Children to Study. Houghton

Mifhrn, BoKonV'Tgcpsp:--------

20. Encyclopedia of Educational Research* edited by I, S* Monroe, The' MacmlllanGdmpany, Hew York, 1941. 21.

Eskaw. Philip An Experiment to Determine the Relative TCluo3''5T,'the ^sl^enF'Shee^ and the ^age or GhapFerAssTgrnientin a^BeventliGrade SocTal^clence""Glaas* masterT s thesls, State University of Indiana, Bloomington, 1933*

22. Fitch, Sir John, Lectures on Teaching. E. L. Kellogg and Company, New~Tork, 1886• 23* Flannagan, J. G«, ffGeneral Considerations in the Selec­ tion of Test Items and a Short Method of Es­ timation the Product-Moment Coefficient from Data at the Tails of the Distribution,’1 Journal of Educational Psychology, 30, Decem­ ber 1939, pp. 674-680.

24* Frederick, Robert W., Ragsdale, Glarene© E*, Salisbury, Rachel, Directing Learning* D* Appleton* Gentury Company, Hew York, 1938* 23* Gates, Arthur 1*, The Improvement of Reading* The Mac­ millan Company, Mew Tor¥? M B K -06* Germane, C* E*, 11Value of the Controlled Mental Summary as a Method of Studying,** School and Society* 12, December 1920, pp* 591-503* 27* Germane, G* £»., ^Value of the Controlled Summary as a Method of Studying,® School and Society* 13, June 1921, pp* 730-732* 28* Germane, C« B*, *V&lue of Corrected Summary as Compared with Re-reading of the Same Article,” Elemen­ tary School Journal* 21, February 1921, pp* 461-464 • 29* Germane, C* E., The Value of Summarizing in Silent Read­ ing as Compared with'Re-reading of the Same Article * doctor rs disserta11on. Htate Univers H y o F Iowa, Iowa City, 1920* 30. Gilliland, J • F., An Experiment to Determine the Effi­ ciency Tn .How t o &tudy. master^s thesis, "State UniversTty'of Kansas, Lawrence, 1927. 31* Good, C* \r*, ®The Effect of Mental-Set or Attitude on the Reading Performance of High-School Pupils,” Journal of Educational Research. 14, October 1626, pp. 1713-186• 32* Good, Carter 7* ®The Effect of a Single Reading Versus Two Readings of a Given Body of P^aterial,” Journal of Educational Method, 5, April 1926, pp.325-3B9. 33* Gray, 0* D., f*A Comparison of Two Types of Silent Read­ ing as Used by Children in Different School Grades*® Journal of Educational Psychology* 20, March T ^ T p p T ----- ----34* Gray, W* S *, Recent Trends in Reading* Supplementary Educational teonographa, TTnlversity of Chicago, 1939* 35* Hall, J* W., Hall, A* 0*, The Question as a Factor In Teaching, Houghton Mifflin Comoanv, Boston,

i§is

*'

p.fl

36* Hawkes, H* &*, Lindquist, E* P., Mann, C* The Con­ struction and Use of Achievement Examinations, Houghton Mifflin Company, Boston, 1936* 37* Heekert, J* W*, aTh@ Effects of Supervised Study in English Composi tlo nJournal of Educational Research* 5, May 1922, pp* 368-380* 38* Hinsdale, H* A*, The Art of Study, American Book Com­ pany* ^lw“T o r k T l ^ 0“ “* 39* Holmes, Eleanor, Relative Merits of Reading Guided by Questions 7ersu~Careful Wading and Re-reading without Questions * master* a tliesis, Universi ty ofChicago, l9£9* 40# Horn,

Ernest, Methods ofInstruction in the Social Studies, Part XV,Report of the Commission on tlh®"'ooeial Studies, -%ericsn Historical Assoelation, Charles Scribner’s Sons, Hew York, 1937*

41# Iowa Every Pupil Test of Basie Skills for Grades 6, 7, and 8, Test A: Silent Reading Comprehension, distributed by Bureau oF Educational Research and Service, Extension Division, State Univer­ sity of Iowa, Iowa City, 1936* 42* Iowa Public Health Bulletin, Tularemia, published by State Department of ^ealth., Ses Moines, Iowa, Vol. LIV, Ho, 3, 1940# 43* Jerslld, Arthur, T#, 11ExamInations as an Aid to Learn­ ing#*1 Journal of Educational Psychology, 20, Bovember 1&$9, pp. ©62-60$* 44* Judd, G* H.*, Education as Cultivation of the Higher Mental Processes, The Macmillan Company, Hew York, 1936* 45* Judd, G* H#, Buswell, G* T*, Silent Reading; a Study of the Various Types* Supplementary Educational fonograph, Ho* 23, University of Chicago Press, 1922* 46* Komhauser, A* W*, How to Study * University of Chicago Press, 1924* 47# Laycock, S* R*, Russell, D* H#, wAn Analysis of Thirtyeight How-to-Study Manuals,11 School Review, 49, May 1941, pp* 370-379*

2(30

48* Lindquist, £• F*, Statistical Analysis in Educational Research. Houghton Mifflin Company,"Boston, 49* Lyman* R* L*. Th© Mind at Work in Studying Thinking ana~3iaIIFV Scott, Foresman and Uompany, Chicago, 1924* 50* MeCallleter* James M*, "Reading Difficulties in Study* ing Content Subjects," Elementary School Journal* 31, Hoyember 1$50, pp. 191*201. 51# McIntosh, V# M*, The Effects of Objectire Guide Ques­ tions S S ^el^*ch®eking Answer Sheets upon ^eiSfoxmSciT "i'n leading and'Seaming.* masterfs thesis, Kansas State College, Manhattan, 1939* 52* McKee, Paul, "An Investigation of the Value of a Sin* gle Reading as a ^ool,n Journal of the Colorado* Wyoming Academy of Science* 1, June’"I$34,' ppV ""r M~Wm 53, MeMurray, Frank V *, How to Study and Teaching How to Study, Houghton WlifTTn Sompany , Boston ,19(59T 54* McMurray, P* M*f Parkins, H* W*, Advanced Geography* The Macmillan Company, Hev? 'Ydrk,^l!9§’ f7 55* Mathews, 0* 0*, "Comparison of Methods of Study for Immediate and Delayed Recall," Journal of Ed­ ucational Psychology* 29, February'19387 pp* 101-106 , r 56* Mathews, C, 0«, Toepfer, H*, "Comparison of Principles and Practices of Study," School Review, 44, March 1936, pp# 184-192* 57# Monroe, Walter -S* * Types of Learning Required of Pu* ils in the Seventh and K1ghth ^radea and n the Hfpii School,1 .College of Education ”SulIeti n , vol* X I X , Ho# 5, University of Illinois, t7rbana, 1921,

f

58# Newlun, C* 0*, Teaching Children to Summarize in Fifth Grade 111story * 6ontribu tlons to Education, Ho* 404, $ureau of Publications, Teachers College, Columbia University, Hew York, 1930# 59* Perrin, Charles, "A Symposium on Study in the Elemen­ tary School, Education, 30, December 1909#

60* Reeder, 33* H., A Method of Pi reetin# Children*a Study of Geography* flontrlbutions to Education^T o T 193, Bureau of Publications, Teachers College, Columbia University, Hew York, 1925* 61* Richardson, M* H*« Kuder, 6* P*, "The Calculation of Test Reliability Coefficients Based on the Method of Rational Equivalence,” Journal of Educational Psychology* 30* December 1&59. no# 62# Ritter, B# T., Lolland, W. T., "The Relation Between Reading Ability as Measured, by Certain Standard Tests and the Ability Required in the Interpre­ tation of Printed Matter Involving Reasoning,” Elementary School Journal* 24* March 1924* pp# OTCTffg;" .............. 63# Salisbury, Rachel, "Some Effects of Training in Out­ lining,” The English Journal* (College Edition) 24, February T§S&, pp * 111-116* 64# Simpson, Robert G*, nThe Effect of Specific Training on Ability to Read Historical Materials,11 Journal of Educational Research . 20* December l92§* "on•

3*3=331:--------

65# Spencer, E. M*, The Retention of Orallv Presented Mate­ rials * doctor *s 'dissertation', 'State University oFTowa, Iowa City, 1940. 66. Spitzer, H. P., A Study of Retention In Reading* doc­ tor^ dissertation, State University of Iowa, Iowa City, 1938* 67* Strang, Ruth, ”Another Attempt to Teach How to Study,” School and Society* 28, October 1928, pp. 461466* 68. Stroud, J. B., "introduction to General Psychology. Brent Ice Hall, I-tew "fork, 1§^8 • 69# Symon&s, P. M., "Methods of Investigation of Study Habits,” School and Society. 24, July 1926, pp. 145-152* 70. Syraonds, P. M., "Study Habits of High School Pupils," Teachers College Record. 27, pp. 713-724, April 1926*

71. Tledetnan, H. R., A Study of Retention in Geography* doctor^ dissertation, State University of Iowa, Iowa City, 1941. 72. Traxler, A* Ton Years of Research in Reading; SumrL,n-m™ -_ x i j - i r c r r StlOZial B©COP&S mary and" *Bibii ography . E&UC 3 Bureau , R©w YorE , 1941 • 73. Washburn, John X., "The Use of Questions in Social Science Material," Journal of Educational Psychology, 20, May I929, pp. 321-359. 74* Weaver, Robert P., "The Relative Yalu© of Intensive Study and Extensive Reading in United States History," The School Review. 59, March 1931, pp* 217-226." 75* Webb, I». W., "Students* Method® of Studying Psychology,” Journal of Educational Psychology. 11, April 1920, pp. 193-T 76. Whipple, Guy M., How to Study Effectively, Public school "Company, Bl'oomington, Illinois, 1927. 77* Woodrlng, M* H», Flemming, G. W . , Directing Study of High School Pupils . Bureau "of Publications , Feachers"Follege, Oolumbia University, New York, 1935. 78. Wright, Louis© B., ”Th© Yalue of a Motivated Assign­ ment! An Experiment in Directed Study,” School of Education Journal, Vol. 5, Ho. 2, December 1B2B, Unl'versity of Pittsburg, op. 64-67. 79* Yamada, Soshichi, ”A Study of Questioning,” The Pedagogical Seminar??. XX, June 1915, pp. 129186* 80. Yoaksmi. G. A.* The Effects of a Single Reading. Studia s

i F ^ d u c a t l o 3 , T o l . ' - f T , T o «" ? V

of Iowa, 1924*

ttk v e r g ity

APPSED22C

6J Page 1

The 1936 Iowa Every-Pupil Tests of Basic Skills READING SELECTIONS FOR TEST A — SILENT READING COMPREHENSION

PRACTICE SELECTION — CRANBERRIES (For use with Practice Esercise) P

ara­

g ra ph

1

PARAGRAPH

2

P

ara ­

gra ph

3

Although the cranberry is raised on a small scale in Nova Scotia, Holland, and Anstralia, the cultivated cranberry is almost an American institution. It was de­ veloped from native plants and has been associated with Thanksgiving and Christmas holidays since pioneer days. Even before the white man came to the New World, the Indians ate dried, powdered cranberries with wild turkey. Massachusetts is the leading cranberry-producing state, growing more than half of the world’s crop. New Jersey and Wisconsin are next in rank. Cranberries have been cultivated on Cape Cod since about 1810, but it was not until the accidental dis­ covery in 1850 that covering the bogs with sand improved the crop, that the industry prospered. The cranberry plant is a tiny, trailing, woody growth, rising less than a foot above the ground. It thrives on an acid or sour soil, which is valueless for most other agri­ culture. The type of soil best suited for cranberry culture is found only in low peat bogs which can be flooded easily to protect the plants from frost, insects and weeds. The word “ bog” may suggest a swampy area, but during most of the growing season, a cranberry bog is as dry as a well-kept lawn. Water, however, must be available for the purposes already mentioned and also for irrigation.

Page 2

SELECTION NUMBER ONE (For use with Exercise 1) P

ara­

g ra ph

1

P

ara­

gra ph

2

P

ara­

gra ph

3

P

ara­

g ra ph

4

P

ara­

gra ph

5

P

ara­

gra ph

6

Most people in this country have not been interested in forestry problems until recently, and then only because of the work of the Civilian Conservation Corps, or the “ C.C.C.” Our Government, however, has been concerned with these problems for a long time. Government forest work had its real beginning nearly sixty years ago. In 1876 the Department of Agriculture appointed a special agent to study forest conditions. Some years later, about thirty years ago, the Forest Service was organ­ ized as a separate part of the Department of Agriculture. When white men first came to America there were about one and a quarter million square miles of forest in the United States. Now this amount has been reduced to about three-fourths of a million square miles. Over one-half of what remains has been well cut over. More than 100,000 square miles of forest (equalling about twice the area of Iowa) was cut so carelessly and wastefully that the land on which it stood is now nearly useless. If the reckless destruction of forests had been allowed to continue, there would finally have been little timber left. To put a stop to this waste, the Government or­ ganized the Forest Service and established national forests. The areas guarded by this service are the tree-covered public lands in the West and a few forests purchased from private owners in the East. There are now national forests in 31 states with a total area of nearly a quarter million square miles. During recent years the Government has emphasized other values of forestry for the whole country. For the farmer, particularly, there are two great problems which are affected hy forestry practice. One is erosion, or the washing away of soil; the other is the distribution of rain water. Trees and some kinds of crops enable the soil to absorb more rain water. If there is no such protection, the water runs off so rapidly that it carries with it great quan­ tities of valuable top soil and washes ditches in the land. This water, which should take weeks to seep down to the valleys, rushes down the stream beds in torrents, caus­ ing destructive floods. Since little water is held in the soil under these conditions, there are also likely to be long periods of drought. Forestry attempts to aid the far­ mer by preventing this erosion and by helping to control the flow of rain water. We have already seen how the national forests protect farm land, prevent floods, and hold moisture in the soil. In addition to these values, the national forests bring money to the Government in several direct ways. Some full-grown trees are cut each year and are sold for lumber. Ranchers rent some parts of the national forests for use as grazing land. The Government also secures some income by leasing power sites in the streams and by furnishing drinking water to large cities. The money made in these ways in some of the national forests has been enough to pay all the expense of keeping up those particular forests.

.

Page 3

SELECTION NUMBER TWO (For use with Exercise 2) PARAr GRAPH

1

P

ara­

gra ph

2

P

ara­

g ra ph

3

P

ara­

gra ph

4

P

ara­

g ra ph

5

P

ara­

gra ph

6

Although explorers have occasionally visited it, there is one country about which very little more is known now than was known in the days of Marco Polo, over 600 years ago. That country is Tibet, a country located north of India and belonging to China. Several expeditions have gone into Tibet, but chiefly in the eastern part along the Chinese border, so that they have learned little about it. It is hard to believe that an area so large (almost twice as large as Texas) should have kept out all outsiders until recent years. But the explanation is that Tibet is surrounded by the world’s best system of natural fortifications: the Himalaya Moun­ tains on the south, China on the east, and desert plateaus and mountains on the north and west. Within this protection, life has gone on almost the same for hundreds of years; the Tibetans have allowed no modern ideas to creep in. Mainly because educated travelers have been kept out of Tibet, almost nothing is known of the origin and ancestry of the people. It seems certain, however, that they are not Chinese; neither do they resemble the Malay people farther south. Some people think that the Tibetans are somewhat similar to our American Indians, but in color and other features there are indications that they must have come from some type of Mongolian race. It is estimated that about one-seventh of the entire population of Tibet lives in monasteries. Nearly every family has at least one son who lives in one of these monasteries as a lama. The religion of the Tibetans is Buddhism, and the lamas are their priests. These lamas live a life of idleness and are supported by the rest of the people. This is made possible by the superstition of the native Tibetans, who believe that ghosts and devils will interfere with their affairs unless stopped by lamas. Few people in Tibet live in towns. The people not living in the few towns or in the monasteries are either wandering nomad herders in the uplands or settled farmers in the valleys. The nomads live throughout the year in their yak-hair tents, tending herds of yaks. The yak is an awkward cowlike beast of burden from which they also obtain milk and meat. Since the valley folk do not wander about, they build thick mud-walled houses with flat roofs. They carry on farming under conditions that an American farmer would consider a heavy handicap; both methods and implements are very crude. In parts of Tibet, especially among the country people, a most unusual marriage custom, that of polyandry, is practiced. This means that a woman may have several husbands. In Tibet these husbands are usually brothers. One husband may take care of the home in the valley, another will be in charge of the yaks or sheep in the uplands, a third may be a trader, and so on.

205

Page 4

SELECTION NUMBER THREE (For use with Exercise 3) P

ara­

g ra ph

1

P

ara­

gra ph

2

P

ara­

gra ph

3

P

ara­

g ra ph

4

P

ara­

gra ph

5

P

ara­

gra ph

6

Most people have thought that the story of the lost continent of Atlantis is pure myth, but modern discoveries have made it seem possible that the story is true. The legend is that much of the space now filled by the Atlantic Ocean was once occupied by bodies of land. On the largest body of land, called the continent of Atlantis, a civilized people is supposed to have lived. The first hint that there might be some truth in the story came in 1898. While workmen were attempting to fish up a broken cable in the Atlantic, their grappling irons brought up soil and pieces of rock which examination proved to have come from volcanoes. Furthermore, there was evidence that the cooling of this lava from the volcanoes had taken place on land and that the lava had been under water only a few thousand years. These facts seem to show that this part of the ocean bed was once dry land. There is also, in certain animals and insects, some evidence which seems to support the theory about Atlantis. For instance, the monk seal, an animal which always re­ mains close to land, is found in our West Indies and also in the Mediterranean Sea. This wide separation on opposite sides of the ocean seems to indicate that the ancestors of these seals may have once followed a coast-line which no longer exists. Certain ants, moths, butterflies, and earthworms found in the Old World are also just like some found in the New World; yet these insects and worms could not possibly cross the Atlantic. The only other possibility seems to be that there was once land where there is now sea. Certain skeletons of our early ancestors which have been found in Europe may furnish additional evidence. From these skeletons, scientists know that a certain race of people (called the Cro-Magnons) came to Western Europe many thousands of years ago. No signs of these people have been found in Central or Eastern Europe; there­ fore it is believed that they came from the west or southwest. Not a single boat or drawing of a.boat has been discovered among the remaining traces of this race. This fact suggests that there may have been land by which they could reach what is now Europe. Further support for the Atlantis legend is found in likenesses between people on opposite sides of the Atlantic. Skulls found in Europe resemble skulls found in South America. Some European languages are similar to the language of American natives. Ancient religious beliefs in Europe and America also show many similarities which indicate that they might have come from the same source. Thus certain Indians of South America had a mythical hero, Brochica, who supposedly carried the world upon his shoulders; this legend seems only another form of the Creek myth about Atlas. It hardly seems probable that people and animals separated by a great ocean which they could not cross, could by pure chance be so much alike. It appears reasonable, from these and the other facts mentioned, to believe that there may once have been large bodies of land in parts of the Atlantic Ocean.

THE 1936 IOWA EVERY-PUPIL T ESTS OF BASIC SKILLS FOR GRADES 6 , 7 AND 8

PREPARED UNDER THE DIRECTION OP Ernest H,

A,

Horn,

M a u d e M c B R O O M (R e a d i n g , B a s i c S t u d y S k i l l s )

GREENE (Language) E.

P.

P.

Lindquist

B,

KNIGHT

(Arithnetic)

(G e n e r a l E d i t o r )

In collaboration with other members of the faoultiee of the College of Education and the Uni­ versity Experimental Schools, State University of Iowa, and with the Bpeoial assistance of: J. Lloyd Rogers (Reading and Basic Study Skills); Ruth Clendenen (Basic Study Skills); R. P. Netzer (Language); and Thelma LewiB (Arithmetic).

TEST A: SILENT READING COMPREHENSION Do not open

t h is booklet, or turn it over, then hait for f u r t h e r i nstructions.

until you are told, to do so.

NAME_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Clast name

first

na»e

middle

name o r

Sex_ _ _ _ _ _ _ _ _ _

initial)

First

fill f h e s e blanks;

Age on last b i r t h d a y _ _ _ _ _ _ _ _ _ _ _ _

(boy or girl)

N um be r of m o n t h s s i n c e last, bir th d ay _ _ _ _ _ _ _ _ _ _ _ _ _ Grade_ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Schoo]_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _— _

Teacher_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

City- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Date_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Pubil s h o u l d n o t w r i t e in s p a c e b e l o w t h i s d o u b l e l i n e .

Scores

•

Par t

II:

(P a r a g r a p h Comorehension) (Organi za t Ion)

Part ttt-

(Understanding of S t a l l s )

III.

Part IV:

NOTE:

(Total

Nunbers

in

Me a n i n g . )

Exercise 3

( S e l e c t i o n 1)

( S e l e c t i o n '2)

( Sel ect i on 3)

1:

(t)

( 18)

( q)

Col,

2:

( 8)

( 36)

3:

( 6)

Col.

3:

( 5)

(1 s )

4:

( 3)

Col.

4:

(s)

( 8)

Total Score on Test A:

( 70)

i:

( 6)

Col.

2:

( q) C o l .

2:

Col.

3:

(7) C o l .

Col .

^^

parentheses indicate possible score.

(Add part scores vertically.)

D i s t r i b u t e d by: B u r e a u of E d u c a t i o n a l R e s e a r c h E x t e n s i o n D i v i s i o n , State U n i v e r s i t y of Iowa City, Iowa C o p y rig h t,

(Add column scores horizontally. )

Col.

Col.

Co 1 . 4'.

P ar t T o t a l s

'—O*.

T-

E x e rc i se 2

O

Pa r t

Exercise 1

1Q36 by S t a t e U n i v e r s i t y o f Io**a

and S ervice Iowa

2

EXERCISE I - QUESTIONS ON SELECTION 1 Column 1 ;

□

paragraph Comprehension

1. What is the main idea of the first para­ graph?

1) "C. C. C." stands for Civilian Conservation Corps. 2) in 1876 the Department of Agriculture appointed a forestry agent. 3) our Government has been interested in forestry problems for many years. 4) The Forest Service is a part of the Department of Agriculture. 5) Very few people have been interested in for­ estry until recently. 2. What is the main idea of the second para­ graph? 1) There is much less forested land in this coun­ try than there used to be. 2) we once had 1,250,000 square miles of forests. 3) over one-half of the original forested -land in this country has been made valueless by careless cutting. 4) Most of the timber which has been cut was wasted.

□

7. According to this article, which three of the following should be placed as important sub-topics under the heading: "Chief Pur­ poses of National Forests"? 1) 2) 3) 4) 5) 6) 7) 8)

1) 2) 3) 4)

provide work for the C. C. C. prevent waste of timber. make money for the Government. protect soil from erosion. furnish timber for Government use. furnish drinking water. help distribute rain water properly. provide national parks.

18. Which one of the following would make the best heading for the fourth and fifth paraIgraphs together? 1) 2) 5) 4) 5)

Causes of floods and drought. How the Forest Service Is conducted. Values of forestry to farmers. The farmerTs great problems. How the Government conserves timber.

4. What is the main idea of the fourth para­ graph?

9. Which two of the following belong with "Leases of Power Sites" and "Sale of Drink­ ing Water" as sub-topics under the same head­ ing in an outline of this article? 1) 2) 3) 4) 5) 6) 7)

1) Forestry helps everyone, but especially farm­ ers. 2) The Government has practiced forestry for many years. 3) Farmers are concerned with the distribution of rain water. 4) Erosion means the washing away of soil.

□

To To To To To To To To

3. 7/hat is the main idea of the third para­ graph?

1) Early lumbering methods were recklessly de­ structive. 2) To guard against waste, a large number of na­ tional forests have been established. 3) Thirty-one states have national forests. 4) Our national forests now cover 250,000 square miles.

□

Column 2; Organization of Ideas

5. The principal purpose of the fifth para­ graph is to explain which of the following?

How How How How

floods are caused. ditches are washed in the land. erosion takes place. forestry helps the farmer.

Prevention of erosion. Purchase of private lands. Payment of expenses. Sale of lumber. Reduction of wasteful cutting. Rental of grazing land. Sources of income.

10. which three of the following describe the most serious effects on farms of the de­ struction of forests, as discussed in this article? 1) 2) 3) 4) 5) 6) 7) 8) 9)

Frequent forest fires. periods of drought. Too much moisture in the soil. Fewer jobs. More frequent floods. Not enough grazing land. More power sites. Loss of valuable soil. Impure drinking water.

6. What is the main idea of the sixth para­ graph? 1) National forests save farm lands and prevent floods and drought. 2) some timber is sold from the national forests. 3) National forests make some money, often enough to pay expenses. 4) Land in national forests is rented for grazing purposes.

(Go r i g h t on to Column 3 w i t h o u t w a i t i n g . )

(Co r i g h t on to Column 2 w i t h o u t w a i t i n g . ) Score on Column 1

Score on Column 2

20^3 Column !5: Understanding of significant Details 11. How long lias the Government been engaged in some kind of forestry work?

□ 1) 2) 5) 4) 5)

since C. C. C. camps were established. Over fifty years. About thirty years. Since white men came to America. since the Department of Agriculture was estab­ lished.

□ 1) 2) 3) 4)

12. Why, principally, did the begin its work in forestry?

To To To To

□

Government

prevent floods and droughts. preserve the timber. prevent erosion. add to its income. 13. According to this article, how is the preservation of forests of most value in preventing floods?

1) Water is stored in the leaves of the trees. 2) Rain water is soaked up and held by the forest soil. 3) Water is stored in forest ponds and lakes. 4) Water in the soil is kept from evaporating. 5) Forests contain many dams and power sites.

□ 1) 2) 3) 4) 5)

14. What is meant by "erosion," according to this article?

Seepage of water from the soil. Carrying away of soil. Prevention of floods and droughts. Irrigation of the soil. Protection of the soil.

□ □ 1) 2) 3) 4) 5)

□

Comprehension of Total Meaning

18. Which one of the following do you think best describes the author*s purpose in writ­ ing this article?

1) To present a history of the Forest Service. 2) To make a plea for better support of the For­ est Service. 3) To warn people of the dangers of reckless cut­ ting of trees. 4) To explain why the national forests are impor­ tant. 5) To show that farmers benefit more than others from the Government's work in forestry. 19. Which one of the following statements appears best supported by this article? 1) The national forests could be justified even though we had no use for lumber at all. 2) The Government is trying to make too much money from the national forests. 3) Only a certain class of people benefits from the national forests. 4) The Government is the only agency that does any forestry work in this country. 5) Private owners of forest land are not inter­ ested in forestry.

(// -you f i n i s h , e a r l y ,

s b e n d t h e r e m a i n i n g t i m e in c h e c k i n g y o u r a n s w e r s t o any o f t h e q u e s t i o n s in this exercise. Do n o t t urn to t h e n e x t b a g e s un­ t i l you a r e t o l d to do s o . )

15. Which paragraph tells how the national forests were obtained? (Write the number of the paragraph.) 16. The national forests now make up what part of the total forested area of the country?

Practically all of it. Two-thirds. slightly more than one-half. one-third. One-eighth.

□ 1) 2) 3) 4)

Column jt:

17. About how large are all national forests combined?

More than four times as large as Iowa. Twice as large as Iowa. As large as thirty-one states. About half as large as the entire country.

(Go r i g h t on to Column 4 w i t h o u t w a i t i n g . )

Score on column 3

Score on Column 4

4 EXERCISE 2 - QUESTIONS ON SELECTION 2 Column 1:

Column Zz

Paragraph Comprehension

Organization of Ideas

1. 7/hat is the main idea of the first para­ graph? 1) Marco Polo was an explorer 600 years ago. 2) Several expeditions have gone part way into Tibet. 3) Very little is yet known about Tibet. 4) The country of Tibet belongs to China. 2. 7/hat is the main idea of the second para­ graph? 1) Tibet is very large, almost twice as large as Texas. 2) The Himalaya Mountains stop visitors from the south. 3) Outsiders have been kept out of Tibet by na­ tural barriers. 4) it is hard to believe that Tibet is so large. 3. 7/hat is the main idea of the third para­ graph?

n

1) Very little is yet known about the beginnings of the people of Tibet. 2) The Tibetans resemble neither Chinese nor Ma­ layans . 3) in the opinion of some, Tibetans resemble American Indians. 4) Tibetans are probably of Mongolian origin.

7. Which three of the following belong under the heading: "Natural Barriers Which Sepa­ rate Tibet from the Outside World"? 1) 2) 3) 4) 5) 6) 7) 8) 9)

Lowlands. Malay. Desert highlands. Rivers and valleys, China. Himalaya Mountains. Military fortifications. The Indian Ocean. The large size of the country.

8. If you were given these two sub-topics for an outline: "A. It is possible that these people are related to our Indians B. These people may have come from a Mongolian tribe," which one of the following would you use as a heading that would include both these ideas?

□

1) The ancestry of the people is not certain. 2) The Tibetan people do not resemble their neighbors. 3) The Tibetans may have had Mongolian ancestors. 4) Some think that the Tibetans resemble American Indians. 5) Tibetans have kept out educated travelers.

4. What is the main idea of the fourth para­ graph? 1) The priests in Tibet are called lamas. 2) Buddhism is the religion of the Tibetans. 3) The people of Tibet support an unusually large number of priests. 4) The lamas protect the people against ghosts and devils. 5. What is the main idea of the' fifth para­ graph? 1) The lowland farmers live in flat-roofed mud houses. 2) Most of the people of Tibet may be divided in­ to two groups: nomads and farmers. 3) The yak is very important in the life of the Tibetans. 4) There are not many towns in Tibet. 6. 7/hat is the main purpose of the last para­ graph? 1) To tell what the husbands do. 2} To give a definition of polyandry. 3) To describe a peculiar marriage custom of Ti­ bet. 4) To explain that polyandry is practiced princi­ pally by country people.

(Go r i g h t on to Column 2 w i t h o u t w a i t i n g . )

Score on Column 1

9. Which three of the following ideas from this article are signs that the people of Tibet are backward? 1) 2) 3) 4) 5) 6) 7) 8)

Visitors are- kept out by natural barriers, The Tibetans have accepted no modern ideas. Their farming is very crude. Few people in Tibet live in towns. The Tibetans do not resemble their neighbors. Very little is known of their history. They have many superstitions. They look something like American Indians.

10. In which two ways do the Tibetans differ most from the people of nearly all other countries of the world? 1) 2) 3) 4) 5) 6J 7)

They are superstitious. They have very few towns. They live by herding and farming. They practice polyandry. a large proportion of them live in monasteries. They are of Mongolian or Indian origin. They believe in ghosts.

(Go r i g h t on t o Column 3 w i t h o u t w a i t i n g J

Score on Column 2

Column 3 ;

□

understanding of Significant Details

11. What is the principal reason that so little is known of the country of Tibet and its people?

1) Tibet is almost twice as large as Texas. 2) Tibet has not accepted modern ideas. 5) outsiders have been kept out by natural barri­ ers. 4) Tibet is north of India and belongs to China. 5) The people of Tibet are very superstitious.

□ 1) 2) 3) 4) 5)

1) 2) 3) 4)

13. From what fact given in the selection would you conclude that yaks are important to Tibet?

They can They are They are They are shelter.

□

be tended in large herds. awkward animals something likecows. the chief export of Tibet. beasts of burden and furnishfoodand

14. Why does it seem that it should be easy for outsiders to get into Tibet?

Comprehension of Total Meaning

17. Which of the following do you think is the best title for this entire article? 1) 2) 3) 4) 5)

How We Know about Tibet A History of Tibet How Tibetan People Make a Living Tibet, Land of Mystery Where Tibetans Came From

18. What two main ideas do you think the writer of IHTs article intended his readers to get from it?

IS. Y/hy have the priests been able to gain such a hold upon the people?

The priests of Buddhism are called lamas. There are many monasteries in Tibet. The people are very superstitious. The priests do not have to do any work. The people practice polyandry.

□

Column 4:

1) 2} 3} 4} 5)

Tibet is north of India and belongs to China. We have been unable to learn much about Tibet. The people of Tibet are not all alike. The religion of Tibet is Buddhism. Life in Tibet is very different from life in America. 6) There is no explanation for the way Tibetans live. 7) Tibet is almost twice as large as Texas.

( I f you f i n i s h e a r l y , - s b e n d t h e r e m a i n i n g t i m e in c h e c k i n g y o u r a n s w e r s to any o f t h e q u e s t i o n s in this exercise, bo n o t go on to t h e n e x t t>age un­ t i l you a r e t o l d t o do so, - and do n o t t ur n back to b a g e s 2 and 3 . )

1] 2] 3] 4]

The people are anxious to secure modern ideas. The country is so large. Tibet is north of India and belongs to China. The country is surrounded by natural fortifi­ cations. 5] The Tibetans are highly religious.

□

15. What has made it possible for Tibetans to keep their old form of civilization un­ changed for so long a time?

1) The size of their country. 2) Their peculiar religious customs. 3) The natural geographical characteristics of their country and of its surroundings. 4) The fact that they do not have the same origin and ancestry as their neighbors.

□

16. what accounts far the difference between the homes in the highlands and those in the lowlands?

1) It is colder in the highlands. 2) The people in the highlands are always moving about. 3) The people in the lowlands have crude imple­ ments for building. 4) Nearly all the people of the lowlands live in town.

(So r i g h t on to Column 4 w i t h o u t w a i t i n g . )

Score on Column 3

Score on Column 4

6

EXERCISE 3 - QUESTIONS ON SELECTION 3 Column 1:

Column

Paragraph comprehension

1. What Is the principal first paragraph?

purpose

of the

1) To explain how Atlantis was discovered. 2) To remind the reader of the story about Atlan­ tis. 3) To warn the reader that the story is only a myth. 4) To explain that Atlantis was a body of land.

□

2. What is the purpose graph?

of the second para­

1) To show that Atlantis is only several thousand years old. 2) To explain how broken ocean cables were first repaired. 3) To tell why people first had good reason to believe in the Atlantis story. 4) To tell that the story of Atlantis had its be­ ginning less than fifty years ago.

7. What are two main reasons why scientists think that TEe Cro-Magnon people may have come to Western Europe by way of Atlantis rather than by some other route? 1) Signs of these people have been found in Amer­ ica. 2) Workmen found skeletons of these people in the ocean bed. 3) There is no evidence that they had boats. 4) Their skeletons have been found in Western Eu­ rope. 5) They lived in Europe thousands of years ago. 6) No signs of these people are found in Eastern Europe. 8. The title, "Kinds of Evidence Supporting the Theory about Atlantis," would be a good title for an outline of this article. There would be four main headings in this outline. One of these main headings would be "Simi­ larities between Natives of Europe and Amer­ ica." Select the other three main headings from the list below.

3. What is the main idea of the third para­ graph? 1) The monk seal, found in the Mediterranean and in the West indies, always remains close to land. 2) There are many animals, worms, and insects that are unable to cross large bodies of water. 3) Certain insects found in the Old World are like those found in the New World. 4} Many present likenesses between living crea­ tures in Europe and America could be ex­ plained if the story of Atlantis were true.

organization of Ideas

1 2

3 4 5 6

7 8 9 10 11

Ancient religious beliefs. The Cro-Magnons. Living creatures in Europe and America. Indian myth about Atlas. Languages in the Old and New Worlds. Discoveries in the ocean bed. The monk seal. Skulls found in Europe and in South America. Remains of earlier peoples. Insects and worms. The broken Atlantic cable.

4. What is the main idea of the fourth para­ graph? 9. In an outline like the one suggested in the preceding question, what three items from the list below could be used as subtopics under the heading, "Similarities between Natives of Europe and America"?

1) The Cro-Magnons were a very unusual race. 2) The story of Atlantis is supported by the re­ mains of earlier peoples. 3) The Cro-Magnons were not a sea-going people. 4) Scientists have proven that the Cro-Magnons came to Europe many thousands of years ago.

□

5. Which statement best summarizes the evi­ dence given in the fifth paragraph?

1) Americans and Europeans have similar religious beliefs. 2) Natives of Europe and the Americas were alike in several respects. 3) Similar skulls are found In various parts of the world. 4) The Greek myth about Atlantis was believed by the Indians.

□

1) 2) 3) 4) 3) 6) 7) 8) 9)

Ancient religious beliefs. The Cro-Magnons. Living creatures in Europe and America. Indian myth about Atlas. Languages m the Old and New Worlds. Discoveries in the ocean bed. The monk seal. Insects and worms. The broken Atlantic cable.

6. What is the principal purpose of the sixth paragraph? (Co r i g h t on to Column 3 w i t h o u t w a i t i n g . )

1) To present the conclusion one could draw from the preceding paragraphs. 2) To present additional evidence for the theory about Atlantis. 3) To prove that the story about Atlantis is true. 4) To emphasize the point of the fifth paragraph. ( Go r i g h t on t o Col umn 2 w i t h o u t w a i t i n g . )

Score on Column 1

Score on Column 2

209 Column 13:

□

understanding of significant Details

10. What fact about the discoveries by the workmen in 1898 most strongly suggested that another continent once existed?

1) The rocks they found had not always been under water. 2) They brought up rock and soil from the ocean bed. 5) The Atlantic cable had been broken. 4) They found skulls in the ocean bed which showed that people had lived there. 5) They found soil just like that found in Europe.

□

11. Why does the present location of the monk seal in the West Indies and in the Med­ iterranean suggest that there might once have been more land between these places?

1) The monk seal always lives in the Atlantic 0cean. 2) It never goes far out at sea. 3) It is found only in warm countries. 4) It is much like other animals found in Europe. 5) It does much traveling.

□ 1) 2) 3) 4) 5)

12. What fact about certain Insects fur­ nishes an argument for the story about At­ lantis?

They may have crossed the ocean. They usually live near the ocean, Their skeletons were found in the ocean bed. Their remains have been found on Atlantis. They are found on both sides of the ocean.

□

13. Why does the Indian myth about Brochica suggest the possibility of truth in the At­ lantis legend?

1) The ancestors of these people must have sailed across the ocean. 2} The civilized people who lived on Atlantis had a myth very similar to this one. 3} Since there are so many myths about Atlantis, they must be true. 4) There must have been some connection between the ancestors of these Indians and those of the Greeks.

Column 4;

Comprehension of Total Meaning

!5. Which of the following do you think best tells the chief purpose which the writer seemed to have in writing this story about Atlantis?

n

1) To interest people in this and other legends. 2) To show how scientists collect evidence to prove their theories. 3) To show that legends are usually true. 4) To prove that the legend about Atlantis is true. 5) To present the reasons for thinking the story about Atlantis may be true. 116. Which of the following is the best state­ ment of the conclusion to be drawn from this larticle? 1) We will never know whether or not the story about Atlantis is true. 2) It is impossible for this story to be anything but a myth. 3) There is good reason to think that another continent once existed. 4) There is positive proof that there was once another continent. 5) There is no reason for thinking that the At­ lantic Ocean has ever been different. 17. Which of the following statements do you think is best supported by this article? 1) Modern discoveries will show that all legends are true. 2) There may be some truth in certain legends. 3) There is some truth in every legend. 4} Legends are always based upon facts. 5) Legends can never be true.

( I f you f i n i s h e a r l y , s b e n d t h e r e m a i n i n g t i me in c h e c k i n g y o u r a n s w e r s to any o f t h e q u e s t i o n s in this exercise. Do n o t t ur n back to t h e e a r l i e r bages.)

14. What fact makes the story about Atlantis seem the best explanation for the likenesses between natives of Europe and America? 1) People who lived on Atlantis were just like the people on the other continents. 2) The likeness was probably accidental. 3) People who are different could not live on the same continent. 4) Some of the ancient people must have crossed the ocean in boats. 5) People could not have crossed a great ocean thousands of years ago.

(Go r i g h t on to Column 4 w i t h o u t w a i t i n g . )

Score on Column 3

Score on Column 4

8

DIRECTIONS: of questions out how well those ideas,

You have been given two booklets for this examination. This test booklet contains a numbe about the reading material in the other booklet. The purpose of these questions is to fin you can determine the important ideas in material which you read, how well you can organiz and how well you understand the material.

This test is made up of three exercises, one for each of the three reading selections Inthe othe booklet. Your teacher will tell you when to begin on each exercise. When told to begin on the first exercise, you are to turn to the other booklet and skim rapidl through the first reading selection. Then you are to turn to the questions in Exercise 1 (in this book let) and answer as many of them as you can. In answering any question, you may go back and reread an of the material as many times as you need to in order to decide upon thecorrect answer. Donot linge too long over difficult questions, but go on to others that are easier. Each question is followed by several possible answers, like the questions at the bottom of thi page. For most questions, only one of these answers is correct. You are to read carefully all of tb answers suggested, decide which one is the best answer, and then write the number of that answer in th square in front of the question. Some of the questions call for two or three answers, like question number 2 below, in these cases you are to choose as many answers as are called for and then write the numbers of these answers in th squares provided, putting only one number in each square. Before beginning work on the test, you will be given a short practice exercise that will make clea to you exactly what you are to do. Your teacher will work through part of this exercise with you. Go no to page one of the booklet of reading selections and skim rapidly through the "Practice Selection which you will find there. Read it only to get a general idea of what it is about; then come back t this page and wait for further instructions.

PRACTICE EXERCISE

1. What is the main idea of the first para­ graph? 1) Cranberries grow in many parts of the world. 2) The cranberry has been grown in America more than elsewhere. 3) We have had cranberries at holiday times since pioneer days. 4) The Indians ate dried cranberries before the white man came.

2. Here is a heading for an outline: "The Best Conditions for Cranberry Growth." Which three of the following statements would you include as important subtopics under this heading? 1) A region must fit certain conditions for cran­ berries to grow well. 2) Cranberries need a low peat bog. 3) Cranberries grow best in a sour soil. 4) Cranberry bogs are dry part of the time. 5) Water must be handy for flooding and for ir­ rigating cranberries. 6) The cranberry plant does not grow very tall.

,___ 13. Here are two subtopics for an outline A. Massachusetts grows more than half of th world’s crop. B. Cranberries have been grown on Cape Co since 1810; successfully since 1850. Which of the following would be the best headin to include theseC two ideas?

N\

1) The industry has prospered since 1850. 2) Massachusetts has long been a leader in thi: industry. 3) Half of the world’s crop is grown in one state 4) Cranberries have been grown on Cape Cod sinc< 1810. 4. Why can cranberries be grown that would otherwise be useless? 1) 2) 3) 4)

in placei

Cranberries need water for irrigation. Cranberries grow best on a sour soil. Cranberries do not grow very tall. Covering cranberry bogs with sand improves th< crop. 5. What important information about cran­ berries does this material present?

1) 2) 3) 4)

Why The The How

cranberries are so widely grown. uses for cranberries. needs of the cranberry plant. anyone can raise cranberries.

IOWA

PUBLIC HEALTH BULLETIN

TULAREMIA

P u b lish ed by th e

STATE DEPARTMENT OF HEALTH Des Moines, Iowa

Volume LIV

1940 July, August, September

No. 3

IOWA STATE DEPARTMENT OF HEALTH

S T A T E BO A R D O F H E A L T H E X O F F IC IO M E M B E R S

A P P O IN T IV E (B y G o v e rn o r)

G e o rg e A. W ilso n , G o v e rn o r E a r l G. M ille r, S e c r e ta r y of S ta te W . G. C. B ag le y , T r e a s u r e r o f S ta te C, B. A k e rs , A u d ito r o f S ta te M a r k G. T h o r n b u rg , S e c r e ta r y of A g r ic u ltu r e W a l te r L. B ie rrin g , M.D., S ta te H e a lth C o m m issio n e r

E d w a r d M. M y ers, M .D., P r e s id e n t, B oone H e r b e r t E. O sceo la

S tro y , M.D., S e c r e ta r y ,

W a lte r A. S te rn b e rg , M.D., M o u n t P le a s a n t E rw in J. G o ttsc h , M.D., S h e n a n d o a h W . J. C onnell, M.D., D u b u q u e

A D M IN IS T R A T IV E S T A F F W a lt e r L. B ie r r in g , M .D ., C o m m is s io n e r

M a rv in F . H a y g o o d , M.D., C .P.H ., D ire c to r o f L o c a l H e a lth S e rv ic e s P r e v e n ta b le D is e a s e s — E p id e m io lo g y C a r l F . J o r d a n , M .D., C .P.H ., D ire c to r V e n e r e a l D is e a s e s R e g n a r M. S o re n se n , M.D., C.P.H., D ire c to r T u b e rc u lo s is C h a r le s K . M c C a rth y , M.D., D ire c to r M a te r n a l a n d C h ild H e a lth J . M. H a y e k , M .D., D ire c to r

S ta te H y g ie n ic L a b o r a to r y U n iv e rs ity of Io w a , I o w a C ity M. E. B a rn e s , M.D., D r.P .H ., D i­ re c to r P u b lic H e a lth E n g i n e e r in g a n d I n d u s tr ia l H y g ie n e A. H. W ie te r s , M.S., D ire c to r L a w E n f o rc e m e n t H e rm a n B. C a rlso n , A tto r n e y D ire c to r

V ita l S ta tis tic s E r ic P. P fe iffe r, M.D. C P H D ire c to r ' ” L ic e n s u re a n d R e g i s tr a tio n H . W . G refe, D ire c to r E x e c u tiv e C le rk , C. E. F o o te

P u b lic H e a lth N u rs in g E d i th S. C o u n try m a n , R.N., D ire c to r

O F F IC E S 1027 D es M o in es S tr e e t (o n e b lo c k n o r th of S ta te H o u se ) D E S M O IN ES

FOREWORD Tularemia is a clear cut example of a disease of animals which is transmissible to human beings. As with all other diseases, modern knowledge of tularemia has been attained step by step. The germ which causes tula­ remia had first to be discovered. The disease in the California ground squirrel, the cottontail rabbit, the jack rabbit and other animals, required thorough study. The part played by bloodsucking animal parasites in spreading the germ, called for carefully conducted experiments on susceptible animals. A first human case had to be traced to its source in an infected rabbit. Finally, the study of human cases due to deer-fly bite, led to the identification of-the causative organism of tularemia as one and the same germ in animal and man. These contributions to our knowledge required the special training, dogged persistence and indefatigable efforts of medi­ cally trained officers of the United States Public Health Serv­ ice and of physician-investigators. Fuller conception of this disease has been acquired through the collaboration of varied professionally trained workers. The veterinarian, the zoologist, the entomologist, the conservation­ ist, the attending physician, the health officer, the research worker in field and laboratory—all have worked together to­ ward completion of the present, comprehensive picture of tu­ laremia.

Commissioner of Health.

IOWA PUBLIC HEALTH BULLETIN PUBLISHED QUARTERLY BY THE IOWA STATE DEPARTMENT OP HEALTH WALTER L. BIERRING, M.D., Commissioner.................................. Editor Edition 17,000 Entered as Second Class Matter, Post Office at Des Moines, Iowa, Act of C o n ­ gress, July 16, 1894. Acceptance for mailing at Special Rate of Postage pro­ vided for in Section 1103, Act of October 3, 1917. Authorized on M a y 29, 1929.

TULAREMIA Carl F. Jordan, M.D., C.P.H., Director, Division of Preventable Diseases, Iowa State Dept, of Health, Des Moines, Iowa.

INTRODUCTION A glance a t the accompanying chart (Figure 1) is enough to show th at in 1938 and 1939 Iowa experienced extraordinary prevalence of a disease called tularemia. What is tularemia? W hat causes tularemia? Who blazed the trail leading to present knowledge of this d is e a s e ? Who discovered a n d named the causa­ tive germ of tula­ remia? Who gave the disease its pec u I i a r name ? H ow an d by whom is the dise a s e acquired ? How many cases occur and how widespread is tu­ laremia in the United States ? Why did so many people suffer from tularemia during the late f a l l and early winter months of the p a s t two years? Is tular­ emia primarily a disease of man or of animals? How may the disease be prevented? The purpose of the f o l l o w i n g pages is to ans w e r questions like the above­ stated and to pre­ sent a brief ac­ count of tu la r ­ Fig. 1—Tularemia in Iowa. Number of cases emia. reported for the 6-year period 1934-1939

2-iO

5

BLAZING THE TRAIL 1. Discovery of the causative germ. Prior to 1908, tularem ia was an obscure and unrecognized disease. From 1908 to 1911, George W. McCoy, M.D., Director of the Hygienic Labora­ tory of the United States Public Health Service a t Washington, conducted investigative work on diseased ground squirrels in Tulare county, Cali­ fornia. Doctor McCoy found th a t some of the ground squirrels w ere.in­ fected. with the plague bacillus, the germ which caused many human cases of bubonic plague following the California earthquake of 1900. But strange to say, not all of the animals showed the plague germ. Some gave evidence of a disease resembling plague but no germ could be recovered by using the regular culture tubes. I t was only when change was made to a special culture medium containing coagulated egg yolk, th a t Doctor McCoy and his associate, Charles C. Chapin, M.D., succeeded in isolating a new germ from ground squirrels infected with the plaguelike disease. This microbe was named Bacterium tularense, after Tulare county in which the research work was done. Although very minute, the organism of tularem ia is plainly visible when seen through the microscope. Doctor McCoy demonstrated animal to animal spread of infection among ground squirrels through the bite of the flea ( Ceratophyllus acutus, Baker) and the louse (Haematopinus columbianus, Osborn). Although Doctors McCoy and Chapin suffered illness while working with the plague-like disease of rodents in California, the infection was not at th a t time regarded as being transmissible to man.

2. First human case recognized. Likewise associated w ith Doctor McCoy in the California studies was William B. W herry, M.D., of the United States Public Health Service, la ter Professor of Bacteriology a t the U niversity of Cincinnati. Experience with the plague-like disease of ground squirrels was prob­ ably the chief factor in leading to recognition by Doctor W herry and his associate, Doctor B. H. Lamb, of the first case of human illness found to be caused by the tularense germ. In this case which occurred in 1912, infection entered through the lining of the eye lids, causing marked in­ flammation of the eye, fever, general discomfort and large, tender “ker­ nels” near the ear (so-called oculo-glandular form of the disease). Very interesting is the fact th a t Doctor W herry traced the source of infection to wild rabbits (an epizootic* was sweeping through cottontail rabbits at the time) and warned of the danger to human beings of close contact with diseased wild rabbits.

3. Tularemia named; human cases studied. D uring the years 1919-1921, Edward Francis, M.D., another United States Public H ealth Service trail blazer, was assigned to investigate hu­ man cases of fever in Utah, caused by bite of the deer-fly (Chrysops discalis). Doctor Francis established identity of deer-fly fever with the plague-like disease of rodents. Using human blood from fever patients, he transm itted the disease to guinea pigs and jack rabbits. His studies interrupted by fever, the doctor hastened back to the Laboratory in Washington, D. C. He took with him the inoculated animals in order th a t the germ m ight be kept alive until his recovery. Doctor F ran ­ cis’ illness was found to be caused by the tularense germ. He afterw ard returned to Utah, transm itted the infection to jack rabbits and guinea pigs through the bite of the deer-fly (Chrysops discalis) and the rabbit louse (Haemodipsus ventricosus, Denny) and gave tularem ia its name which literally means “tulare in the blood.” •N o te : W hile an outbreak of communicable disease in hum an beings is called an epidemic, such an outbreak in anim als is called a n epizootic.

6 4. Ticks found to spread germ. Iu 1924 R. R. Parker, Ph.D., R. R. Spencer, M.D., and Doctor Francis, all representatives of the United States Public Health Service, described the part played by ticks (notably the Rocky Mountain wood tick, Dermacentor andersoni, and the rabbit tick, Haemophysalis leporis palustris) m the spread of tularemia among animals.

5. Further study of tularemia. Walter M. Simpson, M.D., Director of the Diagnostic Laboratories, Dayton, Ohio, made thorough study of human cases of tularemia, espe­ cially among market men in Dayton. Through the agglutination test on blood serum specimens, he discovered tularemia cases which had occurred over a period of years from 1908 to 1928. Doctor Simpson in 1929 published a book which contains an excellent account of the disease. Many other investigators have during past years contributed to the knowledge of tularemia.

OCCURRENCE OF TULAREMIA IN THE UNITED STATES The following table and accompanying map (Figure 4) show the num­ ber of cases of tularemia reported in the United States and Territories during the decade 1930-39; also the average annual rate of reporting, per 100,000 population. The table and map are based on data made available to the Iowa State Department of Health through the courtesy of Health Commissioners of the various States and Territories of the United States. TULAREMIA IN THE UNITED STATES (1930-1939)

State

Annual Rate Per 100,000

Nevada ............. .......8.57 Wyoming......... ..... 5.86 Utah ................ .......4.31 Virginia ........... .......3.03 K entucky......... .......2.49 Illinois ............ .......2.46 M issouri........... .......2.12 Kansas ............ .......2.05 Montana ......... .......2.00 Louisiana......... .......1.88 Maryland ......... .......1.70 Arkansas ......... .......1.62 Georgia ........... .......1.56 Iowa ................ .......1.35 Ohio ................ .......1.33 Minnesota ....... .......1.23 Tennessee ....... .......1.15 Wisconsin ....... .......1.03 New Mexico ..............95 Indiana ........... ..........94 South Carolina ..........88 Oklahoma ....... ..........59 Texas .............. ..........52 Alabama ...................48 North Carolina ..........45 Idaho .............. ..........42 Oregon .... ..........39 California ....... ..........32

Total Cases

60 136 232 795 704 1,927 827 394 106 398 284 322 471 342 900 323 313 300 40 322 162 147 318 136 152 20 39 191

State

Annual Rate Per 100,000

Total Cases

West Virginia .......... 31 56 Michigan ....................26 122 Colorado ...................24 25 F lo rid a........................18 29 Pennsylvania ............ 16 149 Mississippi ................ 09 19 Delaware ................... 08 2 New Je rse y ................08 33 Washington ..........! .06 10 New Hampshire 06 3 Maine ...................... ... 2 New York ................. .05 61 Nebraska ................... 02 3 Vermont ............."" * q 0 South Dakota ............ 06 4 Massachusetts ....... .... 2 Arizona ............... 39 17 North Dakota ........ '.10 7 Rhode Island 0 0 Connecticut 0 0 Washington, D. C.. _ 28 Territory Alaska ................. ... 2 Canal Zone .......... q 0 Hawaii ........ q 0 Puerto Rico .......... q 0 Virgin Islands ............ 0 0 Total cases for decade 10 928

roio;

TTOEN1A TOTAL CASES

1 0 ,3 2 8

IN THE UNITED STATES - 1 9 3 0 - 3 9

DEFINITION OF THE DISEASE Tularemia has been defined by Doctor Francis as “an acute infectious disease caused by Bacterium tularense and occurs under natural condi­ tions in over 20 kinds of wildlife, especially in wild rabbits and bares. Man becomes infected by contact of his bare hands with the raw flesh and blood of these animals or by bites of blood-sucking ticks and flies which have previously fed on animals infected with Bacterium tularense.

THE NATURE OF ILLNESS Symptoms of tularemia include fever, chills, sweating, headache, vomit­ ing, malaise, prostration. In the great majority of cases, a sore or ulcer (see Figure 3) marks the place of entrance of infection. Along with the ulcer which is usually on a finger or hand, tender “kernels” which are really enlarged lymph glands, appear a t the elbow and in the arm pit.

Fig-. 3 The above photograph is taken from a special bulletin of the United States Health Service entitled The Pathology of Tularemia” and prepared by Doctors R D Lillie an A Edward Francis. ’

In a group of 259 Iowa patients 235 or 90% showed the presence of ulcer and tender lymph glands (ulcero glandular form of the disease? Of the remaining 24 patients, 22 had no visible ulcer; two patiente had “ Aa^mation of the eye (ocu o glandular form oft X e m i a l o fte n T lS le d ! (mortality) rate ,s low, sickness and disabfflty

OCCURRENCE OF TULAREMIA IN IOWA The accompanying map (Figure 4) indicates the counties in low* which tularemia was reported during the seven-year period 1933 It will be observed th at many counties, esnecialW • S 1Mo3-1939. half of the state, have been entirely’fr S from X ™ * 1" , tie "°rthern reason for this will be evident later (See pages 12 an*»

During the summer of 1937 and 1938, specimens of the dog tick (Derm , centor vanab ilis) were sent from Iowa to the Rocky Mountain Laboratory

of the U nited States Public Health Service, Ham ilton, Montana. Some of these ticks were found to harbor the tularem ia germ . In recent years a number of Iowa persons are known to have acquired tularem ia as the result of tick bite.

F ig . 1 3 — T h e r a b b it tick , H a em a p h y sa lis . lep o ris p a lu s tr is ; fem a le (a b o v e ) a n d m a le (b e lo w ). (C o u r te sy R. R. P a r k e r )

F ig . 14— T h e d o g tic k . C ten o cep h a lid es c a n is ’ (F r o m W . A . R ile y a n d O. A . J o h a n n se n )

F ig . 15— T h e w ood tic k or com m on d o g tic k , D e r m a c en to r v a r ia b ilis ; fem a le (a b o v e ) a n d m a le (b e lo w ). (F rom R. A . Cooley)

PREVENTION 1. The best m eans of prevention of tularem ia is avoidance o f direct contact w ith infected anim als, particularly during an epizootic. Persons who find it necessary to handle wild rabbits, are urged to wear rubber gloves w hile dressing the anim als. 2. Cooking destroys the germ o f tularem ia; sa fe ty lies in cooking meat thoroughly from surface to center. 3. D u ring the summer season, people should be tick-conscious to avoid the possibility o f tularem ia resultin g from tick bite.

TULAREMIA DIRECTIONS TO THE STUDENT:

Have you ever heard about rabbit fever? The medical name for it is tularemia. It is a much more common disease in Iowa than most people think, and it is more serious than is generally believed. People in Iowa are asking themselves, “ How can we be protected against tularemia?” You will find the answer to this question in this booklet. Much of the information in this booklet was taken from a bulletin published by the Iowa State Department of Health in 1940. The F orew ord which is given on the next page was written by Doctor Walter Bierring and is given here exactly as it was published in the bulletin. Do not spend too much time on it because many of the ideas will be explained more fully in the body of the booklet. Your class has been divided into six groups. Each group will have its own set of directions. Your directions are on the loose sheet which has been slip p ed in the fro n t of you r booklet. The directions tell you exactly what to do and in what order to do it. See how well you can follow these directions. They tell you what to do after you have finished reading. You will not have to wait for your teacher to tell you what to do next. The other booklet which you have been given contains a test over the material you will read in this booklet. DO NOT OPEN YOUR TEST BOOKLET UNTIL YOUR TEACHER TELLS YOU TO DO SO. Remove the direction sheet from the front of this booklet. Be Read your directions care­ fully. If you have any questions raise your hand. sure to g et the sheet m arked directions.

DO N O T T U R N T H IS PA G E U N T IL YOUR T EA CH ER T E L L S YOU TO DO SO.

FOREWORD Tularemia is a clear cut example of a disease of animals which is transmissible to human beings. As with all other diseases, modem knowledge of tularemia has been attained step by step. The germ which causes tularemia had first to be discovered. The disease in the California ground squir­ rel, the cottontail rabbit, the jack rabbit and other animals, required thorough study. The part played by bloodsucking animal parasites in spreading the germ, called for carefully conducted experiments on susceptible animals. A first human case had to be traced to its source in an infected rabbit. Finally, the study of human cases due to deer-fly bite, led to the identification of the causative organism of tularemia as one and the same germ in animal and man. These contributions to our knowledge required the special train­ ing, dogged persistence and indefatigable efforts of medically trained officers of the United States Public Health Service and of physician-investigators. Fuller conception of this disease has been acquired through the collaboration of varied professionally trained workers. The veter­ inarian, the zoologist, the entomologist, the conservationist, the at­ tending physician, the health officer, the research worker in field and laboratory—all have worked together toward completion of the present, comprehensive picture of tularemia. WALTER BIERRING, Commissioner of Health

TULAREMIA D ISCO V ERY O F T H E CAU SATIVE GERM

Prior to 1908, tularemia was an unrecognized disease. From 1908 to 1911, George W. McCoy, M.D., Director of the Hygienic Laboratory of the United States Public Health Service at Wash­ ington, conducted investigations on diseased ground squirrels in Tulare County, California. Doctor McCoy found that some of the ground squirrels were infected with bubonic plague. But strange to say, although the animals showed signs of a disease resembling the plague, not all of them had the plague germ. Only after much effort was the germ of this plague-like disease isolated. It was named B acterium tularense after Tulare County in which the research work was done. Although very minute, the organism of tularemia is plainly visible when seen through the microscope. Doctor McCoy proved that the disease was spread among ground squirrels by the bite of the flea and the louse. Even though Doctor McCoy suffered illness while working with this plague-like disease, the infection was not at that time thought of as being transmissible to man. F IR S T CASE IN A H U M A N B E IN G RECOGN IZED

Doctor William B. Wherry was associated with Doctor McCoy in the California studies. Probably Doctor Wherry’s experience with the plague-like disease which was attacking ground squirrels was the chief factor which led to his recognition of the first case of human illness directly traceable to the tularense germ. His asso­ ciate in this discovery was Doctor B. H. Lamb. In this case, which occurred in 1912, infection entered through the lining of the eye lids, causing marked inflammation of the eye, fever, general dis­ comfort, and large, tender “ kernels” near the ear. Very interest­ ing is the fact that Doctor Wherry traced the source of infection to wild rabbits and warned people of the danger of coming into close contact with diseased wild rabbits. T U L A R E M IA NAM ED

During the years 1919 to 1921, Edward Francis, M.D., another United States Public Health Service trail blazer, was assigned to

investigate human eases of fever in Utah caused by the bite of the deer-fly. Doctor Francis proved that the deer-fly fever was the same as the plague-like disease of rodents. He discovered that the deer-fly could transmit the disease from animal to animal. Doctor Francis gave the disease its name tularemia which literally means “tulare in the blood.” During the past few years, many other in­ vestigators have contributed much to our present knowledge of this disease. PREVALENCE OF THE DISEASE

In 1939 over two thousand cases of tularemia were reported in the United States. The disease has been found in all of our fortyeight states and in the District of Columbia. Since 1935 tularemia has been reported in ten foreign countries. About ninety out of every one hundred eases of tularemia in this country have resulted from handling wild rabbits. Because hunt­ ers, housewives, and people who dress or prepare rabbits for market often handle infected animals they are more likely to contract tularemia than other groups of people. The greatest danger occurs in the cleaning of infected animals. Then the hands come into direct contact with the raw flesh and blood of the rabbit. ONE WAY THE GERM ENTERS THE BODY

The tularense germ can pass through unbroken skin, therefore if a person’s hands come into direct contact with the raw flesh or blood of infected animals, the germ may enter the body. That ex­ plains why the Iowa State Department of Public Health recom­ mends the wearing of rubber gloves by anyone who handles wild rabbits. NATURE OF THE ILLNESS

Tularemia has been defined by Doctor Francis as “ an acute in­ fectious disease caused by Bacterium tularense and occurs under natural conditions in over twenty kinds of wildlife, especially in wild rabbits and hares. Man becomes infected by contact of his bare hands with the raw flesh and blood of these animals or by bites of blood-sucking ticks and flies which have previously fed on ani­ mals infected with Bacterium tularense.” In most cases of tularemia a sore or ulcer develops where the

germ enters the body, usually on the finger or hand. Along with the ulcer, tender “ kernels,” which are really enlarged lymph glands, appear at the elbow and in the arm pit. These are accom­ panied by fever, headache, chills, and the loss of appetite. A person with tularemia is usually sick and weak for several months. Some­ times a year is required for one to fully recover. As yet no vaccine for the prevention or satisfactory serum for the cure of the disease has been developed.

OCCURRENCE IN IOWA

Iowa has not escaped the disease. In the years 1938 and 1939 a very severe outbreak swept through the rabbit population of some sections of the state, killing the animals in large numbers. For the seven year period, 1934-1940, 390 human cases were reported to the Iowa State Department of Health. The number of cases reported for each year was as follows: 1934 — 8 1935 — 2 1936 — 19 1937 — 11

cases cases cases cases

1938 — 107 eases 1939 — 166 eases 1940— 77 cases

SOURCES OF HUMAN INFECTION

The Iowa State Department of Health has tried to trace the source of infection for each case of tularemia reported. For the six-year period, 1934 to 1939, 249 cases were traced to their source, and the wild rabbit was found responsible for 233 of them. You can see that in Iowa the wild rabbit has been the source of more cases than all the other animals together. The dog, cat, skunk, muskrat, squir­ rel, woodchuck, and raccoon have been other sources. It is believed that some of these animals had eaten a diseased rabbit just before coming into contact with the person who became infected. Blood­ sucking insects such as fleas, ticks, and blood-sucking flies are thought to play an important part in carrying the disease from animal to animal. A few people in Iowa have been infected in this way. This is one of the reasons why the Iowa State Department of Health warns people concerning the danger of ticks, particularly during the summer months. The mere handling of a tick may re­ sult in infection. In one case a man pulled a tick from his horse

and then, using the same hand, removed something from his eye. Two days later his eye became infected, and he developed a case of tularemia.

SEASONAL PREVALENCE

By far the greatest number of cases of tularemia in Iowa have been reported during November, December, and January. During these three months more cases were reported than during the other nine months of the year together.

SECTIONAL DISTRIBUTION

Many of the counties in the northern and western sections of Iowa have reported few or no cases of tularemia in human beings. There are two possible explanations for this. First, there are fewer rabbits in those sections of the state, and second, the disease has not been as common among the rabbits in northern and western Iowa as it has in other parts of the state. Most of the cases in people have been reported from the south central and southeastern sec­ tions of the state. Many of the rabbits in those sections of Iowa have had tularemia.

DANGER OF EATING UNDERCOOKED MEAT

The Iowa State Department of Health suggests that all wild meat be thoroughly cooked from surface to center. Eating undercooked meat of an infected animal is very dangerous. The disease which one may get in this way may be more serious than the usual form of the disease.

SERIOUSNESS OF THE DISEASE

The death rate in Iowa due to tularemia is only two or three for every hundred cases reported. The disease should, however be regarded as serious because people are kept from their work for a long time.

PREVENTION

If the people of Iowa would follow the recommendations of the State Department of Health, many cases of tularemia could be avoided. The following are the recommendations of the depart­ ment: 1. The best means of prevention of tularemia is avoidance of djreet contact with infected animals, particularly during a period in which the disease is common among these animals. Persons who find it necessary to handle wild rabbits are urged to wear rubber gloves while dressing the animals. 2. Cooking destroys the germ of tularemia. Safety lies in cook­ ing meat thoroughly from surface to center. 3. During the summer season, people should be tick-conscious to avoid the possibility of tularemia resulting from tick bite.

Main Questions 1# What has made it possible for people now to know more about tularemia than was known thirty or forty years ago?

2, Why was the discovery of the first human case of tularemia so important?

3, What is meant by the first statement in the Foreword, "Tularemia is a clear-cut example of a disease of animals which is transmissible to people?

4* How can tularemia be distinguished from other common diseases ?

5, How widespread is tularemia in Iowa?

6* Why is tularemia considered a serious disease?

7* How can you guard against getting tularemia?

MAIN AND SUBORDINATE QUESTIONS 1* What has mad© it possible for people to know more about tularemia today than was known thirty or forty years _ ago? ~"a# What steps have been followed in gaining information about the disease? b* What part did the United States Public Health Service , play in gathering information about the disease? c# When was the disease first studied? d# How widespread is the disease today? 2* Why was the discovery of the first case of tularemia in a human being so important? a. For how many years have we known that man could get the disease? b. What had been the previous experience of Dr# Wherry? c# What was the source of the first case in man? d# What warning did the doctors give at that time? 3# What is meant by the first statement in the Foreword, "Tularemia is a clear-cut example of a disease of animals which is transmissible to people"? a# In what animals is the disease most common? b# How is the disease spread from one animal to another? c. In what three ways may people get tularemia? d. Why is tularemia considered an occupational disease? ‘4. How can tularemia be distinguished from other common diseases? a. What are the symptoms of the disease? b# Where does the sore or ulcer usually appear? c# On what part of the body are enlarged lymph glands usually found? 5# How widespread is tularemia in Iowa? a. How many cases had been reported by 1940? b. During what season of the year is the disease most prevalent? c. Which sections of the state have been most seriously affected? d# What is the chief souroe of infection? 6. Why is tularemia considered a serious disease? a# What is the death rate of the disease in Iowa? b# How long does it sometimes take to recover from the disease? c. Why are doctors unable to prevent the disease? d. Why is it a difficult disease for doctors to cure? 7, How can you guard against getting tularemia? a. In what way would you be most likely to get the disease? b. Of what should you be absolutely certain before eating wild rabbit? c# What insects should you be particularly careful to avoid?

DIRECTIONS FOR GROUP ONE Name

_______ S c h o o l_________________ C it y

1. Read this page of directions before you start read­ ing the booklet. DO NOT START READING THE BOOKLET UNTIL THE TEACHER TELLS YOU TO DO SO. 2. Read the booklet through once. Read it carefully. See how much you can learn about tularemia. 3. When you have finished reading this booklet through once, close it and start taking the test. You may not be able to answer all the test items correctly. Do the very best you can. 4. When you have finished taking the test, put all of the materials on one corner of your desk. You may then read the article, "An American in the Arctic". If you finish reading this article before the other groups have completed their work you may read it through a second time. TRY NOT TO DISTURB THE OTHER STUDENTS WHO WILL STILL BE WORKING AFTER YOU HAVE FINISHED TAKING THE TEST. Remember, once_ you have finished taking the test you are not to turn back to it at any time» 5. When the teacher tells you to begin reading, open the booklet and read the Foreword and go right on reading the rest of the booklet.

DIRECTIONS FOR GROUP TWO _______ S c h o o l_________________ C i t y

1# Read this page of directions before you start read­ ing the booklet. DO NOT START READING THE BOOKLET UNTIL THE TEACHER TELLS YOU TO DO SO. 2. When you are told to begin, read the first paragraph and decide whether or not it contains an idea im­ portant enough for you to remember. If it does, be sure to understand it. Then write on the back of this sheet the most important question which this par­ agraph answers, 3. Read the second paragraph. If this paragraph contains information important enough to remember, write a ques­ tion to cover the important points in the second para­ graph. 4. Follow the same directions for every paragraph in the booklet. 5. When you have read the booklet through, try to answer each of the questions which you have written, 6. Then turn to the booklet and read again to see whether or not you remembered all the important information. If there is any part which you did not remember, study that part carefully. 7. DO NOT OPEN YOUR TEST BOOKLET UNTIL YOUR TEACHER TELLS YOU TO DO SO. 8. When the teacher tells you to begin reading, open the booklet and read the foreword and go right on reading the rest of the booklet.

DIRECTIONS FOR GROUP THREE Name. .

School

C it y

1* Read this page of directions before you start read­ ing the booklet* DO NOT START READING THE BOOKLET UNTIL THE TEACHER:TELLS YOU TO DO SO. 2» Seven main questions that can be answered by reading sind studying the information in the booklet are printed on the back of this sheet* DO NOT READ THE QUESTIONS . -NOW. When your teacher tells.you to begin reading, turn this :sheet over and .read each question carefully. 3. When you have finished reading the questions, read the booklet through once carefully. As you read look for information that will help answer the seven questions* 4. After you have finished reading the booklet, turn back to the list of questions. Read the first question. Think of a good answer for it. If you can answer the question put a cheok mark (\/) before it. If you can­ not answer it, do not put anything before the ques­ tion. Do this for all seven questions. 5. After you have finished trying to answer all of the question^, turn back to the.booklet to find and study the parts that will help you answer the questions which do not have check marks in front of them. 6. When you are sure that you have enough information to answer a question, put a plus sign (+) before the question. This will show that you have now found the answer. 7. DO NOT OPEN YOUR TEST BOOKLET UNTIL YOUR TEACHER TELLS YOU TO DO SO. 8. When the teacher tells you to begin reading, turn to the questions on the back of this sheet. Read them carefully. Then read the Foreword and go right on read­ ing the rest of the booklet. (over)

DIRECTIONS FOR GROUP FOUR Name

S c h o o l__________________ C it y

1. Read this page of directions before you start reading the booklet. DO NOT START READING THE BOOKLET UNTIL YOUR TEACHER TELLS YOU TO DO SO. 2. When your teacher tells you to begin reading, open your booklet and read it through once carefully. 3. On a loose sheet in the back of the booklet are seven main questions which can be answered by reading and studying the information in the booklet, DO NOT READ THE QUESTIONS NOW, 4. When you have finished reading the booklet turn to the seven questions. Read the first question. Think of a good answer for it. If you oan answer it, put a check mark (/) before question 1. If you cannot answer it, do not put anything before the question. Do this for all seven questions. 5. After you have finished trying to answer all of the questions, turn back to the booklet to find and study the parts which will help you answer the questions which do not have check marks in front of them, 6. When you are sure that you have enough information to answer a question, put a plus sign (+) before the question. This will show that you have now found the answer. 7. DO NOT OPEN YOUR TEST BOOKLET UNTIL YOUR TEACHER TELLS YOU TO DO SO. 8. When your teacher tells you to begin reading, open the booklet and read the Foreword and go right on reading the rest of the booklet, 9. Remember— you are not to read the questions on the sheet in the back of the booklet until you have fin­ ished reading the booklet through once.

218

DIRECTIONS •FOR GROUP FIVE • • Namia

'

'

S chool

:

City__________ >____

1. Read this page of directions before you start redding . the booklet* DO NOT START .-READING THE BOOKLET.UNTIL VTHE TEACHER TELLS' YOU TO DO SO, ' ' . ! f ''" 2. Seven main questions which can be answered by reading and studying the information in the "booklet are ,,-printed on the.back of this sheet, Under.each main ., question are several subquestions which wili help answer .the .main,.question, 'DO NOT REAb THE QUESTIONS NOW, When your teacher tells you to begin reading, ....turn, this sheet, over a n d r e a d each question carefully. 3. When you, have finished*reading the questioner.read the booklet through otice carefully. As you read look'for ,.information, which will help you answer the questions., 4. After you have'finished reading the sbb