SEGway April 2014

2013 Logo 3d gradient
SEGway
 
SEG Measurement Newsletter                   April 2014
In This Issue
Featured Project
Common Core
Measurement Moment
Technical Corner
About SEG Measurement
Upcoming Conferences
 
Like us on Facebook
Follow us on Twitter
View our profile on LinkedIn
Find us on Google+
View our videos on YouTube

31 Pheasant Run
New Hope, Pennsylvania 18938
(267) 759-0617
Greetings!

It was a treat talking with many of you at the AERA and NCME conferences earlier this month.  It was a great chance to join the conversation and we are looking forward to more stimulating discussion at the upcoming IMS Learning Conference.  

SEGway offers a chance to extend that conversation and offer useful information about efficacy research and assessment. Without our interactions with our extended educational family, we would not be able to provide this helpful information and the SEG team would not be able to deliver the high quality solutions we do.  Please continue to provide us with your feedback and share your thoughts. 

This issue provides several interesting discussions. With the number of English Language Learners growing, we share with you the innovative work being done by the ELPA21 Consortium. 

This is followed by our thoughts on the Common Core and where it is headed. Einstein is the subject of our "Measurement Moment" section.  In the "Technical Corner," we discuss why you should be interested in statistical power.

We encourage you to learn more about our work in Assessment and Efficacy Research by connecting with us at the IMS Learning Conference in May.  Also, take a look at our homepage at www.segmeasurement.com, as it is continually updated with developments within the education marketplace.  And, if you want even more up-to-date information, follow us on Twitter at @segmeasure. 

As always, feel free to email me atselliot@segmeasurement.com.  I'd love to hear what's on your mind.

 

Sincerely,

 

Scott Signature

 

 

Scott Elliot

 

SEG Measurement

 

Featured Project
ELPA21: Assessing second language learners

The percentage of students for whom English is their second language (ELL) is expected to reach 25% by 2025.  In some states, the percentage of ELLs already exceeds this number. 
 
While assessing ELLs is certainly not a new practice, there is a movement afoot to take ELL assessment to the next level.  Leading the charge is a consortium of 8 states developing new assessments for ELLs, with the name of ELPA21.  Dr. Scott Elliot, from SEG Measurement is proud to be a part of the design and implementation of the ELPA21.  The operational assessment is slated to launch in 2015.
 
The assessments reflect the state of the art in assessment.  They will be fully delivered by computer, over the internet.  The assessments will incorporate performance assessments to measure reading, writing, speaking, and listening.  The item types will include technology-enhanced items that ask students to drag and drop words or objects, or to click on a hotspot rather than simply selecting from four response options as with traditional multiple choice questions.
 
ELPA21 is focusing on two types of assessments for Grades K-12 ELLs.  First, a screener will be used to identify entering students' level of English language proficiency and identify whether or not they should be provided with ELL services.  Students for whom their first language is not English can enter the school system at nearly any grade level and at any point during the school year, making fast accurate assessment critical.  Second, a summative assessment of ELLs is being developed to provide a measure of the proficiency levels of ELLs receiving instruction to determine if they will continue to need services and for use in meeting state and federal government reporting requirements. 
For more information about ELPA21, check out the web site (www.elpa21.org) or contact us at:info@segmeasurement.com.
 

The Future of the Common Core
Long live Common Core; Death to Common Core
teacher-girl-reading.jpg
Depending on your point of view, the Common Core State Standards are either the best thing to happen to education or one step away from the Devil. It certainly is the conversation starter at any gathering of educators.  Here is one take on its future.
As Yogi Berra said, "making predictions is hard, especially when they are about the future."  The prediction offered should not be taken as a judgment of whether or not the Common Core is good or bad.

First, it is important to keep in mind that most education movements come and go, with about a 10 year shelf life.  Take for example the whole language movement, phonics movement, back to basics, minimum competency testing--and of course--No Child Left Behind.  

I would argue that the Common Core Movement hit its peak at the end of 2013. That is not to say that it does not have several years of life left; I only argue that it has reached the top of the curve and is on the down slope.  

Why educational movements have such short lifespans is the subject of speculation. As I see it, there are several factors working against greater longevity.  Let's face it; we are impatient.  We want to see quick solutions. Combine this with the fact that educational movements are typically overhyped at the beginning, with dramatic promises of success. When we don't see dramatic changes within a few years, we lose our collective patience. We move on to the next educational "shiny object." Add to this the political component of educational reform; educational reform movements tend to follow the political tides and election cycles.

So, I think Common Core will run its course over the next several years. I say this not so much to find fault with Common Core, but in recognition of the typical educational reform movement tenure.     

Although the Common Core has been a major feature in the educational landscape for the the past several years, it seems as though it has "jumped the shark."  The Common Core is being implemented in schools throughout the country. But, at the same time, we are seeing growing push back from several groups and state-level resistance on several fronts. As vibrant as Common Core appears today, its shelf life may be expiring.
 

The good news is my prediction will either be right or wrong. :)

Measurement Moment
Einstein Was Wrong 
outer_space2.jpg

Few scientists are as admired or as easily recognized as Albert Einstein.  SEG Measurement is only about 20 miles from Einstein's long time residence in Princeton, New Jersey.  So, we were particularly intrigued to learn of a major error in Einstein's work.

 

While nearly everyone in the scientific community agrees that Albert got it right with the Special Theory of Relativity, we were surprised to learn from a recent NPR broadcast that Einstein did get some things wrong.  And, more of interest for this column, he made calculation errors in nearly 30% of the papers he wrote!

 

One error in particular is quite interesting. In 1931, well before the days of the Big Bang Theory, Einstein believed that the size of the Universe was constant, not expanding.  He presented his argument along with his calculations in an unpublished paper.   

 

His calculations seemed to support his theory of constant size for the universe...except for one problem.  In one of his key calculations he made a "sign error"; he incorrectly treated a number as negative (used the minus sign) instead of treating it as positive (using a plus sign).

 

Well, at least I am glad to see I am in good company. 

 

Matrix
Technical Corner
Why statistical power is important to you

In a recent issue, we explained the concept of statistical power.  In response, several people commented that it was not clear why a company considering undertaking an efficacy study should "be that concerned about the fine points of statistics." 

 

Why?  Because statistical power presents one of the key risks in any efficacy study.  Statistical power will impact how you design your study, especially in terms of sample size.  The number of participants in a study affects many things, not the least of which is cost.

 

Sample size affects your ability to make confident statements about the effectiveness of your product.  This is especially true, if less than dramatic differences are observed between your intervention and the control group, not using your product. To oversimplify, a small sample size will make it difficult to draw conclusions about small to modest differences between the intervention and control groups.  Let me explain further. 

 

You are likely familiar with the concept of statistical significance.  This is often stated as the "probability that the observed differences or relationship would occur by chance"--- often set at 95%.  You may have heard someone summarize research findings saying "the results were significant at the .05 level."  This means that if we conducted the study 100 times, we would likely find that the differences or relationship would likely be seen 95 times. 

 

Equally important to statistical significance, but too often ignored, is the concept of "statistical power."  It is useful (though not precise) to think of this as the "opposite" of statistical significance.  Statistical power refers to the level of confidence you can have that a conclusion of "no significant differences" is supportable.  This is particularly important when small or modest differences are found.  And, this is often the case in educational efficacy studies.

 

In practical terms, consider this. Pretend for a moment that we are conducting a study comparing the learning outcomes for a group using a new educational tool to a group that is not using the new tool.  If we found that the intervention group using the new tool showed learning gains one half of one standard deviation (.50; Effect Size=.50) above the control group, we would only need a total sample size of 128 students to be comfortable in concluding the observed differences are "real."   But, as is the case for many studies of this kind, what if we found differences between the two groups of only two tenths of one standard deviation (.20; Effect Size=.20).  In this case, to be confident in concluding that the differences are "real" we would need a total sample size of 788 students.

  

It is largely for this reason that we advise our clients to consider recruiting 800+ participants.  While this may not be possible for cost and operational considerations, it is wise to keep this in mind when contemplating an effectiveness study of your product or service.   

 

godly-enter-key.jpg
To better understand this issue,  check out SEG's research reports here on our website.
SEG Measurement can help you design your efficacy research study to maximize success.  Call us today to find out how we can help you with your assessment and research needs.


About SEG Measurement
medical-team-portrait.jpg

SEG Measurement conducts technically sound product efficacy research for educational publishers, technology providers, government agencies and other educational organizations, and helps organizations build better assessments. We have been meeting the research and assessment needs of organizations since 1979. SEG Measurement is located in New Hope, Pennsylvania and can be accessed on the web at

 

SEG At Upcoming Conferences
Let's Meet!

We were pleased to see many of our colleagues at the American Educational Research Association Conference (AERA) and the National Council on Measurement in Education Conference (NCME) earlier this month.  Interest in educational products and services at the district and state level are increasing and we are seeing a wave of new innovations. We look forward to seeing you at the upcoming conferences we will be attending.   

  • IMS Global Learning Consortium Learning Impact Leadership Institute Presentation: "Preparing for Multiple Online State Assessments: Questions and Answers Regarding Cross Consortia Technology Collaboration", May 8, 11am, Pontalba Room, New Orleans, LA  
  • Association of American Publishers (AAP) Content in Context Conference, June 2-5, Washington, DC  
  • Head Start's National Research Conference on Early Childhood, July 7-9, Washington, DC  
  • EdNET 2014 Conference, September 28-30, Baltimore, MD -- Bronze Sponsor

If you would like to meet with a representative from SEG Measurement to discuss how we might help you with your assessment and research needs, please contact us at info@segmeasurement.com.

Join Our Mailing List!