Merced, CA--Teaching language is a tough job, but proving student comprehension is even tougher.
A team of researchers from University of California campuses in Merced and Santa Cruz hope to change that when it comes to vocabulary. They believe they have created a new approach to assessing vocabulary that not only shows how students improve in their understanding of words, but also helps teachers assess the finer points of comprehension.
The U.S. Department of Education's Institute of Education Sciences (IES) has awarded $2.3 million to Jack Vevea of UC Merced, and Judith Scott and Susan Flinspach of UC Santa Cruz to develop and examine this new assessment method on a large scale with fourth- and fifth-graders across California.
The traditional method to judge a child's vocabulary knowledge is to test knowledge of definitions using a small set of words. These words are chosen for their ability to discriminate between students and are not necessarily tied to the content taught in schools. Using this method, the child is either determined to know the word or not based on a single answer to a single question. The new assessment tool is based on words from textbooks and novels that students are likely to encounter. In addition, this tool explores levels of knowledge.
"A definition provides one bit of information about a word," said Scott, the lead researcher on the project. "Knowing a word also involves knowing other related words, knowing the part of speech, and knowing how it can be used."
This new tool gives educators a fine-grained understanding of a student's comprehension. Instead of focusing on a single answer to a single question, children will answer a variety of questions about each vocabulary word on the test. Those answers will be scaled to create a score indicating each student's level of comprehension for each word.
Vevea's role in the project is to handle the statistical aspects of the process of developing and testing the tool. A quantitative psychologist at UC Merced since 2007, Vevea develops methods to analyze what does and doesn't work.
"This grant gives us the opportunity to develop this new tool, and to assess its effectiveness for a diverse population of children to find out whether it can accurately measure comprehension despite differences in the students' socioeconomic and ethnic backgrounds," he said.
For his part in assessing the tool's effectiveness among the range of student participants, Vevea's share of the grant award will be about $400,000. His portion of the project is important, Scott said, because school districts need to see concrete data that shows how well this tool measures vocabulary comprehension before they can consider switching testing methods.
"The most important part of this research is the application," she said. "If we're going to spend four years developing and studying this method, we need to be able to show teachers and administrators whether it works consistently better than what they are doing now."
