Research Projects

Integration of Technology in Instruction: the effectiveness of a collaborative creativity software program for elementary school students

Client: 

Tech4Learning

The Challenge: 

Schools are under increasing pressure to develop students’ academic skills. One way to achieve these goals is through the effective integration of technology in instruction. Tech4Learning designed Pixie, a collaborative creativity tool, to address this challenge, but sought independent scientifically proven evidence that Pixie can increase student academic skills.

The Solution: 

SEG designed and conducted a scientific, quasi-experimental effectiveness study ofTech4Learning's Pixie Program. During the 2010-2011 school year, SEG conducted a national study with approximately 1,000 3rd, 4th, and 5th grade students, in 38 classrooms, in California, Georgia, Ohio, South Carolina, and Texas to evaluate the impact of using Pixie on student achievement. Students in both a treatment and control group were measured at the beginning and end of the school year using the Stanford 10 Achievement Test to evaluate academic growth. Analysis of covariance was used to measure growth and adjust for potential differences between the groups. The findings of this study provide substantial support for the effectiveness of Pixie in improving student Reading Comprehension and Mathematics skills.

Product Effectiveness Research: Higher Education Science Textbooks

Client: 

Wiley Publishing

The Challenge: 

Textbooks continue to play a major role in instruction. Institutions of higher education depend on textbooks. Textbook selection is a critical component of the instructional process. Faculty and other members of the higher education community want to be confident that the textbooks they select for their courses are effective. Wiley, a publisher of higher education textbooks, recognized the importance of scientifically-based research in the selection process. As a leading provider of research services, SEG was asked to design and implement a controlled study of the effectiveness of their Visualizing Series textbooks. The Wiley Visualizing Series relies heavily on visual content and is grounded in decades of research that suggests that students learn better with words and pictures than with words alone.

The Solution: 

SEG designed and conducted a scientific effectiveness study of Wiley's Visualizing Series textbooks. A quasi-experimental design was used to investigate effectiveness of three different science and social science titles. Eight institutions of higher education reflecting different institutional characteristics (e.g., size, geography) participated in the study. At each participating institution, a set of classes using the Wiley Visualizing Series (treatment group) and a set of classes not using the Wiley Visualizing Series (control group) were identified. At the beginning of the semester each student in both the treatment and control groups was administered an assessment of the content in the content area represented by the title; all students were again assessed using the instrument at the conclusion of the semester. Analysis of Covariance was used to compare the knowledge levels of the two groups, adjusting for potential differences between the groups present at the beginning of the study. The results demonstrating that students who used the Wiley Visualizing Series learned more was described in a report for use by the Publisher and institutions considering adopting the Wiley Visualizing Series.

Professional Development: The Impact of Integrating Technology in Instruction

The Challenge: 

Schools are under increasing pressure to implement the Common Core Standards and to develop students’ career and college readiness skills. One way to achieve these goals is through the effective integration of technology in instruction which requires extensive teacher training and professional development.

The Solution: 

During the 2010-2011 school year, SEG Measurement conducted a year-long, multi-site study with approximately 1,000 6th, 7th and 8th grade teachers and students in 42 classrooms in Minnesota, Missouri and Texas, to evaluate the impact of teacher technology training using Atomic Learning on student achievement. The study compared the growth in academic skills of whose teachers used Atomic Learning (Treatment Group) to those in classes whose teachers did not use Atomic Learning (Control Group). The study compared student academic growth in the Treatment and Control Groups as well as the growth in Reading Comprehension and Mathematics attained by students in the Treatment Group and Control
Group between the beginning and end of the 2010-2011 school year.

Scientifically-based Research: ELL Product Effectiveness

Client: 

Hampton-Brown Publishers

The Challenge: 

The population of students whose primary language is not English is increasing rapidly and the demand for effective English Language Learner (ELL) instruction has grown. SEG's client, a publisher of supplemental curriculum for use with ELL students, believed their instructional program was effective and wanted to ensure that this claim is supported by scientifically-based research. As a leading provider of educational research services, SEG was asked to design and implement a scientific study of program effectiveness.

The Solution: 

SEG designed and conducted a scientific effectiveness study of the client's ELL instructional program. An experimental design was used to investigate program effectiveness. Several schools participated in the study with half of the classes involved using the Publisher's instructional program and the remaining half relying on traditional instructional practice. Both groups were measured at the beginning of the school year and were assessed again at the conclusion of the school year. The English language skills of both groups were compared statistically to determine if students in the Publisher's program showed greater improvement. The findings were documented in a written report for use by the Publisher and by schools considering the program.

Scientifically-Based Research: High School ELL Program Effectiveness

Client: 

National Geographic School Publishing

The Challenge: 

As the population of students whose primary language is not English increases, there is a growing demand for English Language Learner (ELL) instruction at the high-school level. At the same time, buyers and users of such programs are demanding evidence of effectiveness before adoption. SEG’s client, a publisher of ELL curriculum and instructional programs, recognized the importance of scientifically-based research (SBR) in proving program effectiveness. As a leading provider of scientifically-based research services, SEG was asked to design and implement a controlled study of program effectiveness.

The Solution: 

SEG designed and conducted a scientific effectiveness study of the client’s ELL instructional program. A quasi-experimental design was used to investigate program effectiveness. More than 100 schools participated in the study, with half of the classes using the Publisher’s instructional program, and the remaining half using traditional instructional practice. Analysis of Covariance was used to adjust for potential differences between the groups resulting from non-random assignment. The English language skills of both groups were measured at the beginning of the school year and at the conclusion of the school year. The proficiency of the two groups was compared statistically to determine if users of the program showed greater improvement. The effectiveness of the Program was described in a report for use by the Publisher and schools considering the Program.

Show the relationship between parental involvement and student academic and behavioral outcomes

The Challenge: 

It is widely believed that the extent of parental involvement can affect that academic growth of young students. "I Care" offers a broad parental involvement program for elementary students. While informal feedback and testimonials suggested that the "I Care" program is effective, "I Care" asked SEG to collect and analyze additional data to support this belief.

The Solution: 

SEG analyzed data from several schools using the "I Care" Positive Parenting Character Curriculum. SEG examined academic results, asking the question does greater involvement of parents using the "I Care" curriculum contribute to greater student academic gains? The study involved approximately 300 students from three schools looking at the growth in language arts and mathematics test scores over time. The results showed that higher levels of parental involvement using the "I Care" Positive Parenting Character Curriculum increases the amount of growth in language arts and mathematics over time.

The Effectiveness of an Online Course Management and Instructional Support Tool for Secondary Education

Client: 

Cengage

The Challenge: 

Do students in classes that use Cengage Learning's MindTap show gains in course knowledge and skill?

The Solution: 

The study used a mixed-method, qualitative and quantitative research design to evaluate this question. First, using a pre-post, treatment-group-only design, this study evaluated student growth in knowledge and skills. Second, a qualitative study of instructor and student perceptions of MindTap effectiveness using surveys and interviews was conducted. The findings indicate that students in classes using MindTap substantially increased their course knowledge and skills.