|
Return to Assessment Homepage
Academic Profile Assessment 2004-5Report Summary
For the first time since the Academic Profiles was administered in 1996, Ferris Graduating Seniors closely approximated the total score of comparable comprehensive institutions and exceeded the sub-scores for seniors in mathematics and natural sciences. Students show consistent improvement in total score and sub-scores for all general education outcomes across all four years. It is important to note that while entering freshmen at Ferris score significantly lower than average score of entering freshmen at comparable institutions, our students still make greater gains than average, a measure of the effectiveness of our general education program.
The Academic Profile Test
The Academic Profile test was developed by the Educational Testing Service for outcomes assessment of general education in colleges and universities. The questions on the test are concerned with issues, themes and ideas from the humanities, social sciences, natural sciences and mathematics. The test also measures “a student’s ability to read carefully, to make judgments about the clarity, correctness, or organization of the materials as pieces of writing, and to think critically about the issues and arguments presented….The Academic Profile focuses on the academic skills developed through general education courses rather than on knowledge acquired about the subject taught in these courses….The academic skills…measured by the test were those identified in the Association of American Colleges 1985 report Integrity in the College Curriculum.” [1]
Ferris State University chose to use the Academic Profile Test as one assessment of our general education curriculum outcomes. Using Academic Profile provides a measure which can be compared with those of comparable colleges and universities nationally.
Ferris Students who take the Academic Profile Test
Ferris State University administered the Academic Profile in 1996, 1998, 2000, 2002, 2005. The tests are administered to students in English 150, a general education course in which most students enroll during the freshman year, and other 100-level courses in the humanities, such as Humanities 100. Tests are also administered to students in 300-level general education courses, such as English composition, Eastern Religions and Literature courses. The Academic Profile offers two versions of its test, long and short. The University General Education Committee selected the short version of the test for general education outcomes assessment. The short Academic Profile Test requires at least 50 minutes to administer and complete.
Test Categories
The Ferris outcomes assessment categories show a high degree of correlation with the categories of the Academic Profile Test, allowing for an accurate interpretation of outcome data. The Ferris State University General Education Assessment Outcomes Committees are Communication, Critical Thinking & Lifelong Learning, Cultural Enrichment, Global Consciousness, Mathematics, Race/Ethnicity and/or Gender, Reading, Scientific Understanding, Social Awareness, and Writing. The Academic Profile Test assesses the following categories: Critical Thinking, Humanities, Mathematics, Natural Sciences, Reading, Social Sciences, and Writing.
How the test was administered
During Fall (04) and the beginning of Winter (05) semester, faculty who teach English 150 or Humanities 100 and 300-level writing courses were requested to volunteer for Academic Profiles testing. Faculty members were very cooperative in including their students in this assessment. A positive faculty attitude and the value placed on assessment by faculty are major factors in ensuring a positive student attitude, according to ETS.[2] This is very important in determining students’ motivation and attitude about taking this challenging test, which surely affects the results. In 2005, Ferris tested 471 lower classmen consisting of 243 entering freshman, 99 freshman, and 129 sophomores as well as 334 upper classmen including 202 juniors and 132 seniors. The 2002 testing group Ferris tested 400 freshmen and 407 upperclassmen. The 2000 Ferris testing group included 580 freshmen and 429 upperclassmen and in 1996 355 freshmen and 236 seniors. Testing was coordinated by Susan Booker Morris, General Education Coordinator, and Robert von der Osten and was administered by Ms. Tina Smith and Ms. Debra Vance.
Academic Profile Overall Results Ferris State University, 2004-5
FIGURE 1: Overall Results
Overall Mean Scores, 2005
Ferris’ Academic Profile comparative group
Academic Profile data are reported for five particular groups of institutions: Research/Doctorate Universities Comprehensive Colleges and Universities Liberal Arts Colleges Associate-of-Arts Colleges Specialized Institutions
Ferris is included in the Comprehensive Colleges and Universities category, even though the university offers a considerable number of associate degree programs and two professional medical degrees. The Academic Profile Comprehensive Colleges and Universities group fits within the Carnegie Classification category “Master’s (Comprehensive) Colleges and Universities….The institutions offer baccalaureate programs and, in many cases, graduate education through the master’s degree. More than half of their baccalaureate degrees are offered in two or more occupational or professional areas, such as engineering or business administration.”[3] The Academic Profile Comparative Data Guide includes test results for 34,563 upperclassmen and 26,511 freshmen at Comprehensive Colleges and Universities.
Figure 2: Mean score in 75th percentile nationally
As in earlier test reports, Ferris students continue to show improvement from freshmen to upperclassmen in general education. A comparison of mean scores for 1996, 2000, 2002, and 2005 shows that Ferris upperclassmen score better than freshmen in general education subjects. In fact, the improvement in the most recent test is greater than the improvement reported for 1996. The improvements in mean scores of upperclassmen over freshmen in 2005, 2002, 2000 and 1996 are as follows: 2005 13.89 2002 11.47 2000 12.3 1996 12.0 The results are detailed in Figure 3. However, 2005 data shifts to measure incoming freshman against seniors rather than averaging the first two and last two year cohorts; the data for 2000, available in an earlier report, has been deleted to make space for the 2005 data. A more detailed analysis of performance across all four years follows this illustration.
Academic Profile Scores 2005, 2002, & 1996
Figure 3
Ferris' 2005 freshmen scored higher than the 1996 freshmen, yet the 2005 seniors still scored higher than the 1996 and even the 2002 upperclassmen. Significantly, not only did the seniors show more improvement over the freshmen in 2005 than they did in 1996, but also the 2005 freshmen started out scoring in sub-group scores higher than the freshmen in 1996. Yet, the 2005 seniors still gained significantly over the 2005 freshmen, and at a higher level of achievement than the 1996 upperclassmen. See Figure 4.
Figure 4: Mean score comparison of Ferris 1996 and Ferris 2005 Freshmen and Seniors.
Comparisons of academic skills and proficiencies subscores show an increase in senior student performance. In addition to overall mean scores, the Academic Profile tests and reports mean scores on specific categories, reported as “skills subscores” (Critical thinking, Reading, Writing and Mathematics) and “context-based subscores” (Humanities, Social Sciences and Natural Sciences). It is important to note that there were significant gains in reading, natural science, social sciences, and math over 2002 data. See the graph in Figure 5.
Figure 5: Comparison of Seniors to Freshmen, by category, 2005
A comparison by student year shows steady improvement over four years at Ferris State University.
Figure 6: Comparison by Year
It is worth noting that all areas show improvement over four years. In some areas such as reading and natural sciences there is a marked improvement between junior and senior years, perhaps in the case of science indicating that science courses have been deferred until later years.
Ferris seniors have a higher overall level of proficiency than Ferris freshmen in Critical Thinking, Writing, Reading and Mathematics. The Academic Profile Score Report includes an analysis of scores that provides information about the proficiency level of the students taking the test. However, what constitutes proficiency is not clear. Students’ scores are measured according to a level of proficiency in the following areas: Critical Thinking, Reading, Writing and Mathematics. In explanation of the analysis, the ETS states:
The skills measured by the Academic Profile are grouped into proficiency levels---three proficiency levels for writing, three for mathematics, and three for the combined set of skills involved in reading and critical thinking. The tables… indicate the estimated percentages of students who are proficient, marginal and not proficient at each proficiency level in reading and critical thinking, in writing and mathematics. A student classified as marginal is one whose test results do not provide enough evidence to classify the student either as proficient or as not proficient.[4]
Results are detailed in the following tables (Figure 7).
Figure 7: Ferris freshmen and senior students compared by Level of Proficiency.
A comparison with 2002 shows an increase in the percentage proficient and in the level of proficiency.
Ferris Students’ Academic Profile scores in 2002-3 fit well within the national sample of Comprehensive Colleges and Universities. Although the national proficiency levels tend to be higher than those at Ferris (especially in Critical Thinking, Reading and Writing), Ferris students begin at a much lower level of proficiency than the national level. The increase in the Ferris students’ level is therefore comparable. As entry requirements slowly increase at Ferris, as planned, this discrepancy should decrease, although it warrants close attention. (see Comparative Data Guide at www.ets.org/hea/acpro and Figure 8, below)
Figure 8: Ferris Freshman and Seniors compared to national sample of comprehensive Colleges and Universities.
This is extremely significant data. For the first time, Ferris graduating seniors have a total score nearly equivalent to comparable institutions, and even out score the national scores in mathematics and natural sciences. This is despite the fact that entering students score lower than the national norm. Ferris students make greater gains than the national norm.
Figure 9: Comparison of Ferris and National Proficiency Levels.
General Education Outcomes Assessment Committees will consider the data from Academic Profile tests, along with other assessment instruments which they have developed, in seeking to improve our efforts in general education.
These data will be analyzed by the ten general education outcomes assessment committees[5] to assure that the general education skills and proficiencies of Ferris students are adequately measured and successes and weaknesses in general education identified. Some assessment committees have also developed other instruments to assist in the task of continually analyzing the desired outcomes in each of the general education areas and attempting to determine whether Ferris is successful in achieving them. Each committee is to report its findings to the University General Education Committee, via the General Education Coordinator, and is requested to report its findings to the faculty who teach in the specific general education areas through whatever means it deems reasonable. [1] The Academic Profile User’s Guide, The College Board and Educational Testing Service, 1998 [2] The Academic Profile Comparative Data Guide, Educational Testing Service, 2000. [3] A Classification of Institutions of Higher Education, The Carnegie foundation for the Advancement of Teaching, 1994 edition [4] The Academic Profile Score Report and Interpretive Guide, 2003 [5] Ferris State University General Education Assessment Committee areas: Communication, Critical Thinking and Lifelong Learning, Cultural Enrichment, Global Consciousness, Mathematics, Race/Ethnicity and/or Gender, Reading, Scientific Understanding, Social Awareness, Writing. The members of the committees include elected and volunteer representatives from throughout the university community. |
