SEGway February 2016
Research and Assessment News from SEG Measurement
|
|
|
31 Pheasant Run
New Hope Pennsylvania 18938
800.254.7670
267.759.0617
|
|
|
|
Greetings!
Welcome to the February 2016 issue of SEGway. We have been fortunate to meet with literally thousands of educators over the past several months, and we have worked hard to reflect what we are hearing out in the field in SEGway. We want to continue to hear from you about your needs, so we can better target the services SEG Measurement provides as well as the information we share in this newsletter.
In this issue, we share the most common questions we are hearing about efficacy research. These questions and the answers provided should help you better plan your efficacy research. Also, we provide an overview of the Every Student Succeeds Act, which takes effect this fall.
We encourage you to learn more about our work in assessment and efficacy research by connecting with us at conferences, and look forward to continuing the dialogue with you through the newsletter, on Twitter (@segmeasure) and on LinkedIn (SEG Measurement).
Sincerely,
Scott Elliot
SEG Measurement
|
|
|
 |
|
Are You Ready For Fall 2016?
ESSA Requires School Programs to be Evidence-Based!
The recently enacted ESSA, or the Every Student Succeeds Act, continues to emphasize the importance of proof of effectiveness for schools adopting products and the vendors that produce them. The act was signed into law in December and replaces the No Child Left Behind Act (NCLB). ESSA's goal is to ensure that all students are prepared for a career or college.
ESSA gives states and districts more authority than the NCLB to distribute federal funding, select assessments, and be involved in other areas of education that were previously federally mandated.
The Every Student Succeeds Act asks for programs and interventions to be "evidence-based." Also, the act mandates that states include nonacademic factors into their accountability systems to broaden the scope of school success, moving beyond traditional measures, with the use of indicators such as of student engagement or access to advanced coursework.
Robert E. Slavin, Director of the Center for Research and Reform in Education at Johns Hopkins University, writes in a recent Huffington Post article, "ESSA defines four categories of evidence based on their strength:
-
'Strong evidence' meaning supported by at least one randomized study;
-
'Moderate evidence' meaning supported by at least one quasi-experimental study;
-
'Promising evidence' meaning at least one correlational study with pretests as covariates; and
-
Programs with a rationale based on high-quality research or a positive evaluation that are likely to improve student or other relevant outcomes and that are undergoing evaluation."
Funding programs would give preference to those education products having evidence of effectiveness from one of the first three categories above.
Additionally, the replacement for School Improvement Grants requires local educational agencies to include "evidence-based interventions" in their support and improvement plans. It states that educators should be seeking proven programs, as defined by rigorous standards.
It appears that proof of effectiveness is an important consideration in the new educational landscape.
SEG Measurement can help you get your program ESSA ready through a summer efficacy study (with results available in August) or a fall randomized study! For samples of our student engagement measures, or a full study report, please write to us atinfo@segmeasurement.com. We look forward to hearing from you!
|
|
|
|
|
|
Recent Questions Asked by Educators and Publishers Considering Conducting an Efficacy Study
|
We have recently received several questions about designing and conducting effectiveness research. We are happy to share these answers for those who may be considering a study. If you have specific questions that are not covered here or if you are interested in exploring any of these questions further, please contact us. We are here to help.
-
What is a typical budget for a study and how do I recover my costs? Where is the return on my investment? Often we hear from educational product providers that they want to conduct a study, but are unsure how much to invest. We view effectiveness research as a critical activity that is necessary to support product development, marketing or sales and should be included within the product development or marketing budget. For most studies, the costs can be covered with one or two district sales resulting from providing evidence of effectiveness. Also, most of our effectiveness studies are multi-faceted and allow for the collection of quantitative results, testimonials, market research intelligence, and a better understanding of best practices/optimal implementation all within a single study. This helps to make the most out of every dollar spent on the research. There are many other benefits and returns on the research investment.
-
What if I want to know whether our product gets students more interested in a subject, more engaged, or less likely to have a behavior problem? What if I am interested in teacher or parent outcomes? There are a wide range of student outcomes in education. Not every study is focused on whether or not students learn more or do better on a test as a result of using your product. Studies can investigate the effectiveness of the product or service for any expected outcome including whether students are more motivated, students are more engaged, students have an increased sense of confidence, teachers have made changes to their teaching, parents are more involved, or communication has improved. Your unique products and services drive the goals and design of the study, and often times one study is designed to investigate multiple outcomes in parallel.
-
I do not want to test the students any more than they already are during the school year. Can we still do a study? Yes! In many cases, a study with a focus on student learning can be conducted without imposing additional testing on students. There are many important decisions to be made regarding the goals, design, and best measures to use to meet the goals of the study. In many cases, a study with a focus on student learning can be conducted without imposing additional assessments on students. The use of assessments already administered by the district or state may be able to serve as pre and/or post assessments.
-
Do the studies have to take the entire school year? The length of the study depends on the implementation being evaluated and the specific goals of the study. In some cases, a focus on just one or two units could be conducted in a few weeks. In other cases, it is critical to track students through the year or across years. There is no perfect fixed length for all studies and it is certainly possible to conduct a study in a short period of time.
-
Can I start with a small study and then move on to something larger? Yes! It is certainly possible to start with a small scale study to gather information and initial findings. These findings often help to inform the final design, minimum usage for implementation, and measures for a larger study.
-
Our product is continually being updated. At what point should a study be conducted? Product development cycles have gotten shorter and shorter over the years and experts increasingly advocate shorter more frequent studies. Some product updates don't affect the results of an effectiveness study and a study could be conducted even though a new version will be released in the near future. In other cases, it is important to wait for a new version to be released. Studies should be designed to account for these types of ongoing changes so that the findings from the study can be understood and applied to newer versions as applicable. This allows for the study to be valuable for the desired amount of time. At a certain point though, the findings from an old study will no longer apply and a new study will be needed. Typically, study results from more than three years ago will start to be questioned.
What questions do you have? Let us know what questions you want answered. Contact info@segmeasurement.com to share your feedback. |
Contact SEG today to find out how we can help you establish the effectiveness of your education product or service:
|
|
SEG At Upcoming Conferences
Let's Meet!
We are looking forward to seeing our colleagues and meeting new friends at the upcoming conferences. We are participating in several sessions & we invite you to join us.
|
|
Look for us at these upcoming conferences:
-
SITE 2016, March 21 - 25, Savannah, GA
-
ASU+GSV Summit, April 18 - 20, San Diego, CA
-
AAP Content in Context, June 6 - 8, Philadelphia, PA
We would love to meet with you and discuss how we can help you build strong assessments and get the proof of effectiveness you need for success.
If you would like to meet with a representative from SEG Measurement to discuss how we might help you with your assessment and research needs, please contact us at info@segmeasurement.com.
|
About SEG Measurement
Building Better Assessments and Evaluating Product Efficacy |
SEG Measurement conducts technically sound product efficacy research for educational publishers, technology providers, government agencies and other educational organizations, and helps organizations build better assessments. We have been meeting the research and assessment needs of organizations since 1979. SEG Measurement is located in New Hope, Pennsylvania and can be accessed on the web at
|
|
|
|
