Study Abstract

Systemic Research, Inc., in collaboration with Research and Evaluation Associates, Inc., received a grant (REC-0080724) from NSF to conduct a three-year evaluative study (from June 15, 2000 to June 14, 2003) of NSF's Comprehensive Partnerships for Mathematics and Science Achievement (CPMSA) program.

In the fall of 1999, the National Science Foundation (NSF) launched a new Urban Systemic Program (USP) in Science, Mathematics, and Technology Education by consolidating Urban Systemic Initiatives (USI) and Comprehensive Partnerships for Mathematics and Science Achievement (CPMSA) under the Division of Educational System Reform (ESR). The new USP program, with extended options for K-14 and K-16 technological workforce programs, will continuously impact more than half of all public school students in the United States.

CPMSA was initiated in 1992 to strengthen student literacy in science, mathematics, engineering, and technology (SMET), with specific emphasis on significantly increasing the number of students who will enter SMET careers, especially those who are underrepresented in SMET fields. During the four fiscal years prior to and including FY 1998, CPMSA sites submitted their annual core data using the Tabulated Indicators for Systemic Changes (TISC), which is a data collection instrument compatible to the other SI's core data systems. Systemic Research, Inc., a corporation with its principal offices in Norwood, Massachusetts, has designed, developed, and implemented the TISC system since 1995. TISC is an electronic core data collection system with a built-in progress analysis module tailored to assist both NSF program officers, as well as all CPMSA sites. Systemic Research compiled and published CPMSA Fact Book-I (Overall Progress Report) and Fact Book-II (Individual Site Progress Report) as a result of the TIC data collection, compilation, and analysis effort.

The evaluative study effort consists of four distinct products:

• TISC data collection: Collect upgraded annual TISC data (both Quantitative and Qualitative) for the next three years including missing AY 1998-99 data in the current TISC
• Annual CPMSA Fact Book with a Narrative Progress Summary: Compile and publish CPMSA Fact Book-I and Fact Book-II based on TISC with an additional section for a narrative progress summary
• CPMSA Case Stories: Publish exemplary CPMSA success stories based on two site visits each year
• CPMSA Evaluative Study Report: Publish an overall program evaluative study report each year focusing on causal inferential statistical models

An Advisory Panel of national experts on education reform and evaluation will review, guide, and provide feedback to our study design, as well as outcome reports. Systemic Research will host an Annual CPMSA/TISC Workshop designed for Data Managers & Evaluators to explain TISC, to promote data collection, and to share new developments in the area of evaluation and assessment.

The principal investigator for this study is Jason J. Kim, Ph.D., President of Systemic Research, who initially developed the TISC prototype program including numerous electronic template-based data collection/analysis systems for educational reform programs for K-16. Dr. Kim (PI) is currently leading another three-year evaluative study of USI granted from EHR/REC. Dr. Peggy A. Richmond, the president and founder of Research and Evaluation Associates, Inc., joined as a Co-PI to provide her expertise and experience in educational evaluation.