STUDENT ACHIEVEMENT RESULTS
The SimCalc mission is to define an innovative, integrated curriculum
and technology that enables extended and widespread access to a robust,
integrated, multifaceted mathematical understanding of rate and
variation concepts. SimCalc addresses the connection between "how fast?"
(velocity) and "how much?" (position) descriptions of both familiar
phenomena and more formal mathematical systems, using multiple
representational forms, including graphs, linguistic descriptions, and
algebra.
SimCalc's core target is urban, low SES 7-9th grade students, with
access to affordable yet powerful technology. Equitable access to
powerful mathematics, such as the mathematics of change, has been a
defining concern of all SimCalc work (Kaput & Roschelle, 2000). By
clarifying the conditions under which mainstream children can learn
these concepts, we can lay the groundwork for continuing elevation of
national and state standards and key assessments over the long term,
which will lead to systemic conditions in which all students have an
opportunity to master these powerful but challenging ideas.
Over the course of approximately 8 years and two major NSF grants, we
have refined and tested our innovations. This work has involved deep
research on student cognition, technology designs, and alternative
curricular sequences (1993-97), resulting first in a proof of concept
largely detached from systemic factors. Subsequent work tackled systemic
issues of curricular integration, teacher professional development, and
assessment (1997-2000). We published and presented our work widely in
more than 50 publications and hundreds of presentations (significant
SimCalc publications are included in the reference section and are
marked with "*"). Throughout, we have contributed broader impacts via
the involvement of graduate students, support for early-career
investigators, extensive work with teachers, and service on policy
bodies informing standards development. Over time, we have
systematically addressed three barriers to wide-scale adoption: access,
scope, and student achievement results. With respect to access, SimCalc
originally required software and hardware that cost $5,000 per student.
With Texas Instruments (TI) as market partner, we now can deliver
SimCalc on the technology most widely used in mathematics classrooms,
the "TI-83 Plus" graphing calculator, which costs less than $100 per
student (Hegedus & Kaput, 2001). With respect to scope, SimCalc was
initially an addition to an already crowded curriculum that would
attract few teachers. We now have refined it into replacement units
intended to be adopted easily and widely (Kaput, in press).
We have achievement data from extensive field trials. SimCalc has been
tested in interventions ranging from 15 to 45 hours in over 30 different
trials, in over 15 school settings, with over 1,500 students, in
contexts ranging from New Bedford, MA, Newark, NJ, Syracuse, NY, and San
Diego, CA. Students included at-risk middle and high school students,
gifted middle school students, classes of remedial first-year college
students, AP Calculus students, and pre-service and in-service teachers.
The form of the intervention has varied from replacement units to
after-school settings to summer or weekend enrichment settings.
Different trials also varied the content significantly, with a trend to
increasing integration with grade-level-appropriate algebraic content.
All trials collected extensive data, usually including video, field
notes, and student work. In at least 20 of the trials, we collected
pre-post data, albeit with no control, since the content addressed was
not generally included in existing school curricula. Note that these
field tests were designed to produce formative data rather than
summative results. Formative studies are often smaller, given that they
are designed to produce information to refine an innovation (Brown,
1991; Lesh, 2002) and not to answer questions of effectiveness of the
mature intervention.
We highlight two notable findings from this formative work.
1. In fifteen studies, we have worked with 7th, 8th and 9th
graders. Much of the early and ongoing work involves SimCalc staff working
directly with students. However, in nine (9) studies, we have worked
with classroom teachers. In a recent example, a teacher offered a 17-hour after-school SimCalc
class to local 7th, 8th and 9th grade students.
Dartmouth is a small town that includes a wide variety of income levels. The nine 7th
graders and five 8th graders that signed up for the class were
high-performing students taking the class as an honor, while the twenty-four
9th graders were low performing students, taking the class because
they had failed or nearly failed the 8th grade Massachusetts
Comprehensive Assessment System achievement exam. Twenty-four students completed
the course. No attempt was made to retain those who dropped out because
the classroom was sized for 25. No systematic differences appear
between the group that persisted and the group that dropped out on
either pre-test achievement or grade level. Reasons given for dropping
the class included that it was too hard (2), that is was too easy (2)
and conflicting activities (9).
Figure 1: Performance in math of change
of twenty-four 7th, 8th, and 9th grade students before and after SimCalc.
As shown in Figure 1, the class made significant as well as important
gains from the pre-test to the post-test (t(23)=11.68; p<.0001;d=1.7).
The gains within each grade-level were also statistically significant.
The 9th graders, the most disadvantaged group, paralleled the gains of
the more advantaged students, and, by the end, matched their incoming
state.
2. In the summer of 2000, thirteen High School and four Middle School math
teachers participated in a two-week summer institute on SimCalc,
conducted by Jim Kaput and SimCalc staff. It met for 6 hours a day for
4.5 days, with a half-day given over to testing and administrative
matters. They extended their mathematics understanding, analyzed videos
of students and teachers using the same lessons, and planned for
integration of the technology into their own classrooms. Participants
were recruited from a local area, which is one of the worst performing districts
in Massachusetts as measured by the MCAS
examination. Seventy percent of the students that these teachers work with are on
free and reduced lunch. The teachers who participated had a range of
backgrounds, including at the high end, some who had taught AP Calculus.
As illustrated in Figure 2, teachers showed statistically significant
and important improvement in performance on the post-test compared to
the pre-test (t(16)=6.16;p<.0001;d=1.1).
Figure 2: Performance in math of change and variation
of 17 Middle and High school teachers before and after SimCalc.
From interviews, we also know that these teachers were able to make use
of the materials in the following year. For example, one teacher wrote:
I have just begun the chapter on slope and on Thursday introduced the
slope as a rate. I had wonderful results as far as the understanding was
concerned. I have never introduced slope in this way and saw how well
they grasped the concept. This was an average group of Freshmen in
Algebra I. They were able to come up with many examples of slope as a
rate of pay, distance, ratio of students to teachers. They were also
able to connect this idea to graphing linear functions….
Although these results support the hypothesis that SimCalc can be
implemented by middle school teachers, we have not yet measured the
robustness of implementation outcomes with a probability sample of
teachers who vary on critical variables and conditions. We are in
exactly the position the designers of the IERI program anticipated: we
have converging innovation research results from ROLE and similar
funding sources, which are not yet robust, rigorous implementation
research results. Thus, we applied for and received an IERI planning
grant to help us conceptualize a transition (Confrey, Castro-Filho, &
Wilhelm, 2000) from formative design experiments (e.g., Lesh, 2002) to a
more rigorous methodology for implementation research (Cook, in press;
Torgerson, 2001). Within the modest scope of that grant, we made
important headway on building a partnership among innovation and
implementation organizations and on three critical issues:
- Conceptualizing our innovation framework.
- Developing a framework and plan for implementation
- Defining an assessment blueprint based on a review of all previously available data.
|
