WNPR (Morgaen Donaldson comments regarding recent $100M gift to Connecticut schools)
Inequity Persists in Gifted Programs
Inside IES Research (Neag School’s Del Siegle is interviewed about gifted education research findings)
4 Ways Schools Help or Hinder Gifted Students
Education Week (Study by Professor McCoach and postdoc Rashea Hamilton featured)
Gifted Classes May Not Help Talented Students Move Ahead Faster
The Hechinger Report (Neag School’s Betsy McCoach is interviewed about recent research on gifted education)
Op-Ed: The Replication Crisis Is Good for Science
Editor’s Note: The following piece, authored by Eric Loken, associate professor in the Neag School, was originally published in The Conversation and republished in UConn Today.
Science is in the midst of a crisis: A surprising fraction of published studies fail to replicate when the procedures are repeated.
For example, take the study, published in 2007, that claimed that tricky math problems requiring careful thought are easier to solve when presented in a fuzzy font. When researchers found in a small study that using a fuzzy font improved performance accuracy, it supported a claim that encountering perceptual challenges could induce people to reflect more carefully.
However, 16 attempts to replicate the result failed, definitively demonstrating that the original claim was erroneous. Plotted together on a graph, the studies formed a perfect bell curve centered around zero effect. As is frequently the case with failures to replicate, of the 17 total attempts, the original had both the smallest sample size and the most extreme result.
“As a psychologist and a statistician, I believe confronting the replication crisis is good for science as a whole.”
— Eric Loken, Associate Professor
The Reproducibility Project, a collaboration of 270 psychologists, has attempted to replicate 100 psychology studies, while a 2018 report examined studies published in the prestigious scholarly journals Nature and Science between 2010 and 2015. These efforts find that about two-thirds of studies do replicate to some degree, but that the strength of the findings is often weaker than originally claimed.
Is this bad for science? It’s certainly uncomfortable for many scientists whose work gets undercut, and the rate of failures may currently be unacceptably high. But, as a psychologist and a statistician, I believe confronting the replication crisis is good for science as a whole.
Practicing Good Science
First, these replication attempts are examples of good science operating as it should. They are focused applications of the scientific method, careful experimentation and observation in the pursuit of reproducible results.
Many people incorrectly assume that, due to the “p<.05” threshold for statistical significance, only 5 percent of discoveries will prove to be errors. However, 15 years ago, physician John Ioannidis pointed to some fallacies in that assumption, arguing that false discoveries made up the majority of the published literature. Replication efforts are confirming that the false discovery rate is much higher than 5 percent.
Awareness about the replication crisis appears to be promoting better behavior among scientists. Twenty years ago, the cycle for publication was basically complete after a scientist convinced three reviewers and an editor that the work was sound. Yes, the published research would become part of the literature, and therefore open to review – but that was a slow-moving process.
Today, the stakes have been raised for researchers. They know that there’s the possibility that their study might be reviewed by thousands of opinionated commenters on the internet or by a high-profile group like the Reproducibility Project. Some journals now require scientists to make their data and computer code available, which makes it likelier that others will catch errors in their work. What’s more, some scientists can now “preregister” their hypotheses before starting their study – the equivalent of calling your shot before you take it.
Combined with open sharing of materials and data, preregistration improves the transparency and reproducibility of science, hopefully ensuring that a smaller fraction of future studies will fail to replicate.
While there are signs that scientists are indeed reforming their ways, there is still a long way to go. Out of the 1,500 accepted presentations at the annual meeting for the Society for Behavioral Medicine in March, only 1 in 4 of the authors reported using these open science techniques in the work they presented.
Improving Statistical Intuition
Finally, the replication crisis is helping improve scientists’ intuitions about statistical inference.
Researchers now better understand how weak designs with high uncertainty – in combination with choosing to publish only when results are statistically significant – produce exaggerated results. In fact, it is one of the reasons more than 800 scientists recently argued in favor of abandoning statistical significance testing.
We also better appreciate how isolated research findings fit into the broader pattern of results. In another study, Ionnadis and oncologist Jonathan Schoenfeld surveyed the epidemiology literature for studies associating 40 common food ingredients with cancer. There were some broad consistent trends – unsurprisingly, bacon, salt and sugar are never found to be protective against cancer.
But plotting the effects from 264 studies produced a confusing pattern. The magnitudes of the reported effects were highly variable. In other words, one study might say that a given ingredient was very bad for you, while another might conclude that the harms were small. In many cases, the studies even disagreed on whether a given ingredient was harmful or beneficial.
Each of the studies had at some point been reported in isolation in a newspaper or a website as the latest finding in health and nutrition. But taken as a whole, the evidence from all the studies was not nearly as definitive as each single study may have appeared.
Schoenfeld and Ioannidis also graphed the 264 published effect sizes. Unlike the fuzzy font replications, their graph of published effects looked like the tails of a bell curve. It was centered at zero with all the nonsignificant findings carved out. The unmistakable impression from seeing all the published nutrition results presented at once is that many of them might be like the fuzzy font result – impressive in isolation, but anomalous under replication.
The breathtaking possibility that a large fraction of published research findings might just be serendipitous is exactly why people speak of the replication crisis. But it’s not really a scientific crisis, because the awareness is bringing improvements in research practice, new understandings about statistical inference and an appreciation that isolated findings must be interpreted as part of a larger pattern.
Rather than undermining science, I feel that this is reaffirming the best practices of the scientific method.
Originally published in The Conversation.
She Is Not Throwing Away Her Shot
UConn Magazine (Latest issue features Neag School alumna Alexandra Eckhardt ’11)
The Replication Crisis Is Good for Science
The Conversation (Eric Loken writes original piece)
‘UConn Gives 2019’ Breaks Last Year’s Records
UConn Today (Neag School’s Educational Psychology project places second University-wide in annual Giving Day)
UConn Expert Discusses Restraint and Seclusion in Public Schools
UConn Today (Brandi Simonsen is interviewed about the use of restraint and seclusion in public schools)
UConn Expert Discusses Restraint and Seclusion in Public Schools
Editor’s Note: This piece originally appeared on UConn Today.
As federal lawmakers prepare to debate legislation greatly reducing the use of restraints – and eliminating the use of seclusion – in public schools, a recent hearing of the U.S. House of Representatives Subcommittee on Early Childhood, Elementary, and Secondary Education focused on the use of restraint and seclusion and featured testimony from UConn expert George Sugai. We asked Brandi Simonsen, a professor of special education in UConn’s Department of Educational Psychology at the Neag School of Education and co-director of the Center for Behavioral Education and Research, about the use of restraint and seclusion and whether the federal government has a role to play in regulating their use in public schools.
Q: This is an understandably emotional topic for people. How common are restraint and seclusion in American public school classrooms today?
A: Restraint and seclusion are not common in public school classrooms. However, it’s difficult to know exactly how often they occur, because the definitions and reporting requirements vary by state, and sometimes by district. The best estimate of how frequently restraint and seclusion are used comes from the Civil Rights Data Collection (CRDC), which includes reported data from almost every public school in the U.S. As summarized in the CRDC, 0.2% of all students experienced a crisis intervention – meaning restraint or seclusion – during the last data collection period, which was from 2015 to 2016, and approximately 71% of the students restrained and 66% of those secluded were identified with disabilities under the Individuals with Disabilities Education Act, also known as IDEA.
“It’s critical that educators understand that behavior is a form of communication. That is, behavior serves a function. Students use their behavior to communicate that they want to get something – like attention or an activity – or that they want escape or avoid something.”
— Professor Brandi Simonsen
Q: What does research tell us about any potential emotional or psychological impact on students from the use of restraint and seclusion in the classroom?
A: The long-term effects of restraint and seclusion have not been well-studied. We often hear about egregious abuses of these practices in the news, and we assume experiencing restraint and seclusion is likely traumatic for students, educators, and families. So, while we don’t have great data about the long-term effects of restraint and seclusion, we know quite a bit about the long-term effects of trauma on people’s mental, behavioral, and physical health, cognitive functioning, and overall well-being.
Q: Are there situations or circumstances where restraint or seclusion are appropriate?
A: Restraint and seclusion should only ever be used in a crisis situation, when there is a risk of serious and imminent danger and the restraint or seclusion reduces that risk. These are not planned or therapeutic interventions. They should never be a routine part of a behavior support plan, or used as a consequence for behavior. With preventive and positive behavior supports in place, many minor problem behaviors are handled quickly and efficiently, and the likelihood of more serious behavior is reduced. However, even when the best preventive practices are in place, emergency situations arise, so it’s very important that schools have procedures in place to respond to behavior-related emergencies.
Q: What are the alternatives to restraint and seclusion in the classroom?
A: Prevention is the best alternative. Once a crisis occurs, educators have likely run out of really good options. At best, educators’ goals should be to de-escalate the situation and maintain safety for everyone involved. But in some cases, a crisis intervention does become necessary to maintain safety. So the best alternative to restraint and seclusion is to prevent a crisis situation from happening. How do we do that?
To start with, positive, proactive, and preventative behavior intervention strategies are critical to support all students, but are especially important to prevent escalating situations with students who have a history of these behaviors, including students with disabilities. So, things like having expectations for students, explicitly teaching social and emotional skills, and providing specific feedback to students on their behavior – finding opportunities to celebrate their accomplishments – create an environment that functions as a “protective factor” for students with a history of challenging behavior.
Then, it’s critical that educators understand that behavior is a form of communication. That is, behavior serves a function. Students use their behavior to communicate that they want to get something – like attention or an activity – or that they want escape or avoid something. So, it’s critical that we teach them a more appropriate way to meet this same function, and then make other adjustments to the classroom and school environment to set these students up for success. Some of these adjustments are simple. For example, providing reminders, establishing predictable routines, adjusting academic instruction and tasks, and arranging the environment to ensure the replacement skills “work” for the student are all simple strategies that increase the likelihood of a student being successful and reduce the likelihood of a crisis situation.
To effectively put these types of practices in place, educators need training and ongoing support. In fact, when we look at the research on decreasing restraint and seclusion in schools, professional development is often a part of successful efforts. If a crisis does happen, we need well-trained staff to calmly de-escalate the situation and, then if necessary, move to crisis intervention.
But overall, prevention is the focus of how we set up our schools and classrooms, how we develop interventions to support students with a history of challenging behavior, and how we respond after a crisis to ensure that both students and educators are more likely to be successful the next time.
Q: Are states equipped to address issues surrounding restraint and seclusion in the classroom, or is additional federal intervention necessary?
A: States vary in the level of policy, guidance, and preparation provided to districts and schools. Federal intervention, such as the proposed Keeping All Students Safe Act, may be helpful. The challenge, as you pointed out, is that this is an emotionally charged topic, so it’s critical that experts help legislators understand the nuances – which were reflected in Dr. Sugai’s recent testimony to Congress – that are involved in implementing restraint and seclusion procedures. This mean promoting positive and proactive supports for all students and intensifying those supports to meet the needs of students with a history of challenging behaviors. It also means providing broad-based staff training in those positive and proactive supports and de-escalation strategies, and then providing more targeted staff training in safely and ethically using crisis responses in real cases of serious and imminent danger. And the final, important part involves monitoring, evaluating, and communicating any use of crisis responses.
As an example of federal guidance, the U.S. Department of Education developed the 2012 Restraint and Seclusion Resource Document, which is a great example of highlighting the conditions under which crisis responses may become necessary, but also emphasizing the importance of prevention.
Q: Experts at UConn have been at the forefront of this issue. Can you explain the work currently underway to improve approaches to behavioral intervention in schools?
A: Given the importance of prevention, most of our work at UConn focuses on Positive Behavioral Interventions and Supports (PBIS). I am fortunate to co-direct the National Technical Assistance Center on PBIS, along with partners from the University of Oregon, University of South Florida, and University of Missouri. The PBIS Center just began its twenty-first year, and the next five years (2018 through 2023) represent a $32.6 million federal investment to define, develop, implement, and evaluate a multi-tiered approach. This new approach will improve the capacity of state and local education agencies and schools to establish and sustain the PBIS framework in order to improve supports and outcomes for students with or at-risk for disabilities, and to enhance school safety and learning conditions that promote the well-being of all students.
There are preliminary published case studies that indicate that implementing sound behavior support practices within a PBIS framework may reduce the use of restraint and seclusion for students that have challenging behavioral histories. The National Technical Assistance Center on PBIS will be working over the next five years to continue to explore this topic, and develop resources to support educators in reducing the need for restraint and seclusion.
For more information on the National Technical Assistance Center on PBIS, please visit www.pbis.org.
| About the Expert Brandi Simonsen 860-486-2763 brandi.simonsen@uconn.edu Professor of special education with tenure in the Department of Educational Psychology at the Neag School of Education. Co-director of the Center for Behavioral Education and Research at UConn. Co-director of the National Technical Assistance Center on Positive Behavior Interventions and Supports. Senior Advisor to the National Center on Intensive Interventions. Simonsen conducts research, publishes, teaches, and provides training/technical assistance in the areas of school and class-wide Positive Behavior Interventions and Supports (PBIS), positive and proactive professional development supports for teachers, and applications of PBIS in alternative education settings. Education MS in Special Education: Exceptional Learner, University of Oregon, 1999 BA in Elementary Education and Psychology, William and Mary, 1998 |
Related Stories:
