SimCalc Classroom Connectivity Project
|
Report on the After-School Algebra Enrichment Experiment
Conducted by the SimCalc Project
March 4th - April 5th 2002
Executive Summary
What Happened and Who Were Involved?
With the enthusiastic cooperation
of staff from high school and middle school
, the SimCalc Project engaged in an extended after-school teaching
experiment on Tuesdays-Thursdays, 2:30-3:45PM from March 5th to April
4th, 2002 in an iMac Lab. Approximately 35 students enrolled in
the course, roughly a third being volunteers from middle school(7th and 8th
graders) and the remaining students being 9th graders who were
low-scoring on the 8th grade MCAS. The course was most ably taught by
a high school mathematics faculty member, and assisted by SimCalc
staff as well as several other mathematics teachers. The curriculum focused on core ideas in algebra: slope as
rate-of-change, linear functions, modeling, and simultaneous conditions.
Classes were videotaped, copious field notes were taken, and
pre-/post-tests were administered, yielding a comprehensive picture of
the event and its consequences.
Methods and Goals of the Intervention
The SimCalc approach to these ideas
exploits tightly linked motion simulations, graphically definable and
editable functions, and algebraic formulas running on both computers and
graphing calculators. While this approach has been widely tested across
the U.S. and in other countries, this particular teaching experiment, for
the first time, examined impacts and opportunities of an added
ingredient, classroom connectivity. In particular, most classes
involved students working in small groups to create functions to satisfy
various conditions (usually modeling some given situation), and then
sending their functions to the teacher who systematically aggregated
their work and displayed it on a common classroom display. In this way,
each student's work becomes quickly available for discussion, but more
importantly, can interact with other students' constructions in the
public display. Due to the shared nature of the students' work, their
personal identity becomes intimately engaged in their mathematical
activity. A central goal of the investigation was to determine how to
tap this engagement for optimally productive mathematics learning and to
determine the pragmatic conditions for successful use of these methods
and technologies.
Results Significant Improvements for the Students
The primary means by which success
was measured was through pre- and post-tests, where approximately two
thirds of the items were chosen from released 10th grade MCAS test
items. We are pleased to report that both sub-populations of students
made significant gains on all items but one (where ceiling effects were
seen). The mean test scores increased from 44% on the pre-test to 66%
on the post-test. Aggregate results, as well as the tests themselves,
can be found in the full report, and individual student scores have been
submitted separately. We regard the data as worthy of note because the
items were either challenging 10th grade items or even more challenging
AP Calculus and SimCalc items, and the students were either 7-8th
graders or low-performing 9th graders. Test scores of the Middle School students
exceeded those of the 9th graders, but the High School students showed
somewhat larger gains than did the 7-8th graders from pre- to post-test.
Furthermore, on some MCAS items, the students' post-test means exceeded
both the High School and statewide means on the respective items. One last positive
result of the intervention was de facto faculty development which will
likely lead to further use of these advanced technological resources by
the math faculty in the coming years. We have promised full
support of these efforts.
Classroom Activities
Classroom activities were designed to exploit the computer connectivity
of the lab network by utilizing a version of MathWorlds that enables
students to send their mathematical constructions to the teacher's
computer from which the teacher could display any or all of the
students' constructions as needed for the activity at hand. This
sharing of constructions led to mathematically intense interactions by
the students and engaged them in public discussion about what they saw
in terms of graphs and the motion of dots controlled by the graphs.
Here we illustrate one such activity and how it offers an alternative
method for understanding (a) the idea of slope as rate-of-change,
velocity in this case, (b) the meaning of y-intercept as the initial
position, and (c) the variation of parameters, the "m" and "b," in the
formula for a linear function, Y=mX+b.
Staggered Start–Staggered Finish:
In this activity, we have six groups of students, with 3 to 5 people per
group, where the groups are numbered from 1 to 6. Each student within a
group must travel at 3 feet per second for 5 seconds but start at a
position equal to their group number. Thus in this example, each
student travels at the same rate for the same amount of time. Once
their functions have been aggregated using the network on the teacher's
computer and displayed, the variation between groups is apparent and
becomes the focal point of a lively class exchange. The variation is
provided by parameterizing the "b" in Y=mX+b, which in this context is
now more than just the "y-intercept"–it is a student's starting position
and group number. Discussion topics include "Which dot am I and how do
I know?", "Which group is mine and how do I know?", and when a student
does not construct the motion according to the given directions, "Who is
the outlier?", "How do we know?", and "What needs to be done to make
him/her fit in?".
|
|
Figure 1: Aggregated data being projected by the teacher for classroom discussion.
|
Figure 1 shows the aggregated motion on the teacher's display, the
result of the teacher collecting the functions that the students have
produced over the network. Note that, as expected, overlap occurs in
the six position graphs as students' functions fall into Groups, but
nonetheless, each student is represented by a dot in the upper half of
the screen. Here the Groups are still visible in the vertical alignment
of the dots. Animating the motion leads to all of the dots moving at
the same speed but off-set in Groups, as in a parade, because all the
persons in a given group travel side-by-side. Individuals in Group 1
are farthest to the left and those in Group 6 are farthest to the right.
Displaying the work publicly via projection of the teacher's computer
screen leads to intense class discussion centered around where an
individual student is in the aggregate. Students are asked to identify
themselves, the "Which dot are you?" question, in the aggregate and
present convincing arguments to the class justifying their choice (which
is fairly simple in this case, but much more demanding in later
activities). We also emphasize the generic formula that describes all
the groups, i.e. Y=3X+b where "b" is the Group number.
Summary Performance Data
This section reports on the overall results of the study, highlighting
certain questions that showed particular significant gain from the pre-
to the post-test. Included are some results from the
attitude/background questionnaire administered at the time of both
tests.
Pre- and Post-Tests
We administered a pre and post-test of 20 items to assess the effect of
the teaching experiment. The majority of these items (11) were taken
from the set of released MCAS items with the remaining items extracted
from AP Calculus exams (2) and SimCalc test items (7). The latter were
selected from a pool of items developed by the SimCalc Project and
refined over several years of use. Sixteen of the assessment items were
multiple choice and the rest were open response. We adopted a scoring
rubric for the MCAS items that scored multiple choice questions as 0 or
1 and open response questions on a 0 to 4 scale. The SimCalc and AP
Calculus items followed the same rubric. Two questions from the SimCalc
items were scored with 2 points as they were multiple response questions
(Items 14 and 19). The maximum score for the test was 31 points.
Additional items were included on the post-test, but are not included in this report.
Importantly, the test used challenging items from the Grade 10 MCAS
whereas the students were of varying ability from grades 7 through 9.
The Middle School students from grades 7 and 8 showed a higher ability
in both the pre- and post-test. The weaker 9th grade students had an
average of 218 on their 8th grade MCAS test. This mixture led to
interesting social interaction in our non-standard classroom setting.
We also administered a questionnaire on both the pre- and post-tests to
investigate differences between the backgrounds of the middle and high
school students, and how their mathematical beliefs and the environment
in which they are taught affect their performance. We are correlating
this data with other sources of mathematical background for our sample
of students. The group of students as a whole increased from pre-to
post-test with statistically significant gains.
Results
The five-week teaching experiment had a positive effect on the
mathematical behavior of both groups of students as presented in Table
1:
|
GROUP
|
PRE-TEST
MEAN
|
PRE-TEST
VARIANCE
|
POST-TEST
MEAN
|
POST-TEST
VARIANCE
|
p1
|
|
ALL
|
13.6 (44%)
|
19.04
|
21.1 (66%)
|
21.07
|
<0.0001
|
|
7th & 8th
|
16.2 (52%)
|
23.96
|
23.8 (77%)
|
14.62
|
<0.0001
|
|
9th
|
11.7 (38%)
|
8.06
|
19.2 (62%)
|
17.72
|
<0.0001
|
|
Table 1: Pre- and Post-Test Means and Variances.
|
|
1This p-value corresponds to a paired t-test that was completed to see if
a statistically significant increase had occurred in the means of the
student before and after the 5-week session.
|
Even though the group of 7th and 8th grade students (n = 10) showed higher
test averages than the 9th grade students (n = 14) the latter group
exhibited a higher gain in their group mean (64%) with less variance.
This illustrates that while the 5-week session had a very positive
effect on both groups it had a more positive effect on the 9th grade
students, a fact that we regard as significant because these students
were chosen on the basis of weak 8th grade MCAS performance and
represent the more challenging students to teach.
In addition, we have shown that it is possible to increase the ability of
students to complete some 10th grade MCAS items in a short period of
time before they reach the 10th grade.
 |
|
Figure 2: Percent correct on each of the twenty test items.
|
Figure 2 highlights the pre-/post-test percentages of correct responses
by item. Modest to excellent gains were achieved in every item of the
test except one (Item 2) and there it was a statistically insignificant
change, probably due to ceiling effects.
We now examine two questions that showed the largest pre/post changes
and compare these scores with those of High School students
and their statewide peers.
The first example we examine in detail is Item 5.
5. Based on the graph, which organization showed the most growth in membership over the 10-year period?
A. The Math Club
B. The Hiking Club
C. The Drama Club
D. The Drama Club and the Hiking Club are tied for the most growth.
|
 |
|
Figure 3: Item 5 from the Pre-/Post-Test.
|
The content of this item was not explicitly taught during the experiment
and served as a face-validity item in our test. For this reason, it was
evident that the students acquired skills that were applicable to a
wider set of mathematical tasks not directly addressed in the 5-week
session. The question tests graphical interpretation and comparison of
starting and ending point differences, skills which were only implicitly
used in our activities, as illustrated in our example activity Staggered
Start–Staggered Finish.
The scores below show that our sample of students scored very similarly
on the pre-test as both their High School and statewide peers in the
previous year (about 46% correct). The post-test showed significant
relative gain (almost 100%) in their performance.
 |
|
Figure 4: Comparison of participant scores on Item 5 with larger populations.
|
Another large-effect example is Item 10. Again, the content of this
question was not explicitly taught and involved interpreting a linear
function with Pi(Π) as a coefficient.
|
10. The circumference, C, of a circle is found by using the formula C =
Πd, where d is the diameter. Which graph best shows the relationship
between the diameter of a circle and its circumference?
|
|
|
|
|
|
Figure 5: Item 10 from the Pre-/Post-Test.
|
The scores show that the performance of the participants was very
similar on the pre-test to their High School and statewide peers in the
previous year (about 33%). Once again the post-test figures show
significant gain in their performance.
 |
|
Figure 6: Comparison of participant scores on Item 10 with larger populations.
|
Summary of Attitude/Background Questionnaire
Our attitude/background questionnaire investigated students' responses
to questions regarding their present math classes, whether they liked
working in small groups, their beliefs regarding their ability in and
attitudes towards both Math and English, and their reasons for taking
the course. We used a 5-point Likert "agreement scale" (1 = Not at All to
5 = Very Much).
The results show no significant impact in students' response to
fundamental affective issues such as "Do you like math." We did not
expect a change in such a small amount of time–although there may also
be a damping effect because some students may not have regarded much of
our non-traditional activities as "math." The frequency graph exhibits
a standard normal distribution of response by the group of students both
before and after completing the course, with an average score of 3.40.
Also evident is that our groups of middle school students have a more
positive attitude towards math (3.60) than our group of high school
students (3.23) – not surprising given the nature of the populations.
One notable result, which is illustrated in the frequency charts below, is the
 |
|
Figure 7: Student responses to the statement "I like math in general."
|
 |
|
Figure 8: Student pre-test responses to the statement "I like to work in small groups."
|
difference between the middle and high school groups of students. It is
clear that the 9th graders liked working in groups after the program
more than they did before. The average response increases from 3.80 to
3.92 from pre- to post-test. Compare this with the group of middle
school students whose average response scores dropped from 3.40 to 3.20.
The program did not have any notable affective results for the middle
school students although they increased their overall performance. This
result speaks to how the two populations of students differ in their
response to instruction and how this correlates with the improvement of
their work.
 |
|
Figure 9: Student post-test responses to the statement "I like to work in small groups."
|
Conclusions
Conclusion, Reflections, and a Look Ahead
The participants showed very positive results on core math topics for
two very different populations of students in an intense after-school
program exploiting new technologies and new curricular approaches.
Importantly, the results include substantial gains on 10th grade MCAS
Items despite the fact that one sub-population was comprised of middle
school students and the other was comprised of the 9th graders who had
the lowest district scores on the 8th grade MCAS.
While the results are very positive, we expect that (a) improvements can
(and will) be made in subsequent iterations of the course and (b) that
the high level of classroom support provided to an extremely competent
teacher helped lead to the positive results. Hence, while (a) suggests
further improvement and broader applicability, (b) suggests that
comparable performance may be more difficult to achieve in less
well-supported circumstances with more typical teachers under typical
classroom circumstances. Overall, however, we are confident that the
approaches used with this class are widely applicable given access to
the technology and the curriculum materials.
The SimCalc Project stands ready to provide support to both middle and
high school teachers who wish to use either the network-based materials
or the previously constructed and commercially available materials (at
no cost). We would also support the mathematics teachers whom provided their services
to the program should they wish to train other teachers in the use of any parts of
our technologies and resources — including the use of motion detectors in combination with
SimCalc software.
We are particularly interested extending our research by offering a
brief but intense follow-up course next September to certain students
who took the Spring course. This course would focus on ideas underlying
calculus and be somewhat more intellectually challenging than the Spring
course, which deliberately held to linear function ideas that occur
mainly in Algebra I.
|
|
|
