 |
| Summary of the FY 2000 Longitudinal Studies for the Illinois Mathematics and Science Academy |
| Introduction |
| The Illinois Mathematics and Science Academy has dedicated its nearly fifteen years in secondary education to creating "decidedly different learners" who are expected to make a positive difference in Illinois and throughout the world beyond. The goals of the Longitudinal Study are to gather evidence documenting IMSA's impact on the lives of our graduates and, in turn, the impact of their lives on the world. A longitudinal study provides the opportunity to measure the effects of the passing of time, filled with college, employment, and a variety of other life experiences that provide opportunities for our graduates to make a difference. Furthermore, the years of separation from IMSA impart a perspective to these graduates on the long-term value of their IMSA experience. Since 1990, IMSA has conducted annual surveys of its graduates at several points in time following their graduation. Currently, those post-graduation points have been set at one year (+1), four years (+4), and seven years (+7), with a +12 study planned for FY 2002. Each study has its particular strengths. The +1 findings combine the perspective of one year of college with an appraisal of a high school experience that ended only one year prior to the interview. The +4 findings offer a cross-validation of the +1 findings and a significantly greater time perspective. The +7 findings, which seek little feedback on the IMSA experience, provide a clearer picture of graduate studies and career pursuits. This summary report will focus on the following two areas:
|
| General Methodology |
| Sampling For the past six years, we have randomly selected thirty pairs of students, matched for racioethnicity and gender, from each year's graduates. Phone surveys were administered to the first set of 30 graduates. For each student who could not be reached, the alternate for that student from the second set was contacted and interviewed as a replacement for the original subject. This protocol effectively has provided a 100% response rate. However, some respondents had not responded to all items. Hence, some item analyses were based upon less than 30 responses. Survey Format The survey instruments for the classes consisted of a variety of demographic items, Likert-type attitude assessment items, and open-ended questions. In addition to gathering basic demographic and academic characteristics, questions in the survey are driven and shaped by several broad questions.
The Comparison Groups Contrasts between IMSA students and one or more comparison groups were conducted in each of the FY 2000 studies. In the +7 study, Illinois Board of Higher Education data for all students who had graduated from an Illinois state university are incorporated into the report. The +4 and +1 studies, while incorporating some national data, focus upon comparison groups of residential and non-residential students from the National Consortium for Specialized Secondary Schools of Mathematics, Science, and Technology (NCSSSMST), of which IMSA is a member. These are comparably selective and discipline-focused institutions from across the southern and eastern regions of the country. The +1 study also incorporates an Illinois comparison group selected from honor students from three non-residential public schools identified as having fully developed honors programs with multiple opportunities for students to access advanced placement courses. All participating schools were instructed to follow a similar interview protocol (although not all did so) and used largely identical survey instruments. Therefore, this study was able to contrast the responses of IMSA students with those from a comparison group of similarly high achieving students. Limitations The responses are observations by students, not objective professional evaluations. Hence, the comparisons across schools and states should be interpreted with the understanding that such geographic and cultural differences may also create a difference in perspectives that will affect rating levels. We cannot assume, for example, that standards for satisfaction in more urban and industrialized states would be the same as those for more rural states. Similarly, there may be differences between schools in terms of the academic standards of both the high school and colleges attended. Student perceptions of the relative qualities of their high schools and colleges will be governed by their inherently limited experiences. The sample sizes for most of these studies is a relatively small 30 students per class per year, with non-responses bringing the total response levels down to as low as 25 for some items. Hence, moderate-sized fluctuations can be expected which are due only to random variance. On the other hand, if there are real differences, we may not be able to conclusively identify them due to the constraints that small sample sizes place on achieving statistical significance. Both issues are considered in the findings. Finally, trends over years may be reflective of changes beyond those in the school; e.g., a negative attitude trend may have been influenced by increased cynicism in the general society. |
| Contrasts Across Multiple Years of the Longitudinal Study |
| Contrasts between the +1 and +4 points averaged across the Classes of 1994, 1995, and 1996 show small increases for almost all item rating levels at the +4 point, with three items demonstrating significant differences (see graph below). The perception that high school was challenging increased statistically significantly for each of the graduating classes as well as for the overall means depicted below (from 3.48 to 3.74). To a more marginal extent, the overall perception of high school teacher expertise also increased significantly. Finally, the generally relatively low-rated item for college attention to students experienced a marginally significant increase. |
Item Means Contrasted between the +1 and +4 Surveys for the Classes of 1994 - 1996 |
|||||||||||
|
| The graphs below show the general stability of rating levels between the three years for each item. The only statistically significant differences for each item for the +1 ratings were between the Class of 1994 and the Class of 1995 for "college intellectual challenge" and "college teaching expertise". The .45 drop in "considering ethical implications in decision making" from 1994 to 1996 was not significant, but was large enough to warrant future monitoring. There were no statistically significant between-year ratings differences for any of the +4 items. |
Item Means Contrasted between +1 Surveys for the Classes of 1994 - 1996 |
|||||||||||
|
Item Means Contrasted between +4 Surveys for the Classes of 1994 - 1996 |
|||||||||||
|
| In terms of the general ratings levels for the more specific "how well did IMSA improveÉ" items, respondents to both surveys consistently give very high ratings to IMSA's influence on their critical thinking skills, moderately lower ratings on creative thinking and research skills, and yet slightly lower ratings on ethical decision making. In terms of items focused upon more general academic perspectives, respondents are very satisfied with their IMSA experience. This is demonstrated by their response to the IMSA part of the "IMSA and college experiences" items on both surveys (with ratings between 3.48 and 3.75). Additionally, a general item on only the +1 survey and not part of the graph, "how satisfied are you with IMSA" received a mean 3.47 rating, and the +4 survey-only item (hence not in the graph) "how well did IMSA prepare you academically" received a 3.58. Not only are the IMSA items from the "IMSA and college experiences" section on both +1 and +4 surveys rated very highly, they all are rated significantly higher than those for college. The gap is greatest for attention to students, which receives the lowest ratings of the college items. Hence students assign very positive global satisfaction ratings to IMSA, very positive ratings of critical thinking skills, and moderately positive ratings to the other specific areas of learning. |
| The FY 2000 Edition of the Longitudinal Study - Class of 1998 +1 Study |
| In general, IMSA students reported a solid level of satisfaction with their high school experiences and a positive college freshman experience. As displayed in the graph below, their current very high level of overall satisfaction with IMSA (the item "satisfaction at one year" [after graduation]) was slightly above that for the other school groups. As found in the cross-year study for the more specific "how well did high school improve" items, respondents consistently give very high ratings to IMSA's influence on their critical thinking skills, lower (but still positive) ratings on creative thinking and research skills, and yet lower ratings on ethical decision making. In terms of statistically significant differences from the NCSSSMST and Illinois comparison groups, IMSA was higher than both for critical thinking, lower than both for research skills, and lower than the Illinois group both for creative thinking and considering ethical implications in decision making (and non-significantly but noticeably lower than the NCSSSMST group for ethical implications). For the specific subject areas, IMSA was significantly higher than the NCSSSMST group for mathematics and lower for science (the Illinois comparison group was not given these items). No comparison groups were asked to rate their social science or English preparation. |
Item Means Contrasted between IMSA and the Comparison Groups |
|||||||||||
|
| When asked to rate their high school experiences alongside their college experiences (in terms of intellectual challenge, research opportunities, teacher expertise, and attention to students), IMSA consistently rated their high school experience somewhat higher than the other schools, and significantly much higher than their college experience (see graph below). Additionally, they rated their college experience lower overall than the comparison schools. Note that though IMSA students had rated research skills significantly lower than the other schools, they rated research opportunities higher. |
Item Means Contrasted between IMSA and the Comparison Groups (continued) |
|||||||||||
|
| Hence, with respect to comparison group counterparts, while IMSA students assign both lower and higher (but mostly positive) ratings to items on more specific aspects of learning, they assign a higher current global satisfaction rating and consistently higher ratings of their high school experience with respect to college ratings. |
Attend School in Illinois? |
Current Academic Level Status? |
Anticipate Completing a Double Major?  |
| There were three additional differences between IMSA and comparison groups found in the study (see graph above). IMSA respondents were significantly more likely to be enrolled in an Illinois school than their Illinois comparison group counterparts (although IMSA's figure was higher than in the previous year and the Illinois comparison group figure was lower), they appear to be accumulating college credits at a significantly faster rate than their NCSSSMST counterparts (subject to more precise pinpointing of survey date cutoffs), and have a significantly greater tendency than the comparison group students to aspire to completing a double major. In addition to the differences graphed above, and as expected, IMSA and NCSSSMST students were more than twice as likely as the Illinois comparison group to have majors in areas related to either math or science (62.0% and 63.0% versus 29.2%, respectively). |
| The FY 2000 Edition of the Longitudinal Study - Class of 1995 +4 Study |
| IMSA students reported a solid level of satisfaction with their high school experiences and success in their subsequent life experiences. The comparison NCSSSMST students demonstrate similar characteristics. IMSA students felt very well prepared academically by their high school, as did the consortium school members. For the more specific "how well did high school improve" items, IMSA again was significantly higher than the NCSSSMST comparison group for critical thinking, but now also was significantly higher for considering ethical implications in decision making. The NCSSSMST schools were slightly higher for research skills, while IMSA was marginally higher for creative thinking. When asked to contrast their high school experiences with their college experiences (in terms of intellectual challenge, research opportunities, teacher expertise, and attention to students), IMSA again rated their high school experiences consistently higher than the other schools (significantly so for "high school intellectual challenge"), and higher than their college experiences, although only the "expertise in teaching" and "attention to students" high school-to-college differences were statistically significant (see graph below). IMSA and NCSSSMST college ratings were very similar. |
Item Means Contrasted between IMSA and the NCSSSMST Comparison Group |
||||||||||||
|
| Hence, when contrasted with other NCSSSMST students, IMSA students assigned similar to higher ratings in almost all categories, with two of those differences achieving statistical significance. |
| Academically Under- represented Groups Level  |
Current Academic Level Status? |
Anticipate Completing a Double Major?  |
| There were additional differences between IMSA and the NCSSSMST comparison group found in the study (see graph above). The higher percentage of academically underrepresented groups for IMSA respondents was statistically significant for the +4 study. Just as the +1 IMSA students were accumulating credit hours earlier, the +4 IMSA students they appear to be graduating significantly sooner than their NCSSSMST counterparts (subject to more precise pinpointing of survey date cutoffs) and have a greater (though not quite statistically significant) tendency to attempt or complete a double major. . In addition to the differences graphed above, and as expected, IMSA and NCSSSMST students were more than twice as likely as the national graduate pool to have majors in areas related to either math or science (55.0% and 61.0% versus 26.0%, respectively). Also, 41% of the IMSA respondents' colleges were in the state of Illinois. |
| The FY 2000 Edition of the Longitudinal Study - Class of 1992 +7 Study |
| In general, IMSA graduates at the +7 point report that they are doing well. Most (90.0%) aspire to earning a post-baccalaureate degree, with at least one-third already having completed it. Most (89.0%) are either satisfied or very satisfied with the choice of their major, and most (94.1%) also are satisfied or very satisfied with their current employment. Contrasts to prior year +7 reports show that the respondents from the Class of 1992 were more likely to be married, more likely to be planning to earn a master's or doctorate degree, and more likely to already have earned one. In fact, only ten percent of these IMSA graduates intend to earn only a bachelor's degree. In contrast to the IBHE sample, IMSA respondents were more satisfied with their college major (3.22 versus 3.44), more satisfied with their current employment (3.26 versus 3.47), and much less likely to be employed in Illinois (24.0%versus 63.0%). This year's upcoming +12 investigation will explore this trend more fully, when a larger percentage of IMSA graduates presumably will be employed. IMSA graduates continue to commit to community service and leadership activities beyond the four-year post-IMSA point. Fifty percent of IMSA graduates from the Class of 1992 indicate continued involvement in teaching and mentoring, social service programs such as Habitat for Humanity, and health services (as volunteers). |
 |
© Copyright 1999, 2001 Illinois Mathematics and Science Academy. All rights reserved. Last Updated: June 13, 2001 Created by:Adam Van Den Boom ('98) Content Design: Dr. Steve Cordogan |