Assessment Basics
Assessment is the systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development." (T. Marchese, 1987)
Assessment is the ongoing process of:
- establishing clear and measurable expected outcomes of student learning
- ensuring that students have sufficient opportunities to achieve those outcomes
- systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations
- using the resulting information to understand and improve student learning
(L. Suskie's 2007 summary of several assessment definitions)
Why conduct Assessment?
There are several fundamental reasons for institutions of higher education to conduct
- Broadly speaking, one reason centers on showing ourselves and others that we are achieving our mission, vision, and strategic goals.
- The primary reason, however, is to use the assessment process to improve the work that we do with students and for our region, commonwealth, nation, and global community.
Academic assessment gives faculty a body of evidence on which to base decisions designed to improve teaching and learning. This body of evidence documents how well students are performing relative to faculty-defined learning outcomes. - Finally, a related - and often overlooked -
purpose of academic assessment is to promote a shared understanding of purpose among faculty, and between faculty and students, by clearly articulating objectives and expectations. Many faculty find that the most valuable part ofassessment is in the initial formulation of learning outcomes, a formulation that, if properly conducted, necessitates searching examination by colleagues of their common enterprise as teachers and scholars in a particular discipline.
A General Framework for an Assessment Program
- Create a Statement of Intended Outcomes: List objectives for student learning that are clearly stated and for which evidence of accomplishment can be gathered. These should be stated in terms of what we want students to be able to do, to know, or to think, not what we, as teachers, do. It may be possible to rephrase existing program goals in terms of intended learning outcomes.
- Review Current Practices: List goals and outcomes that are already stated for the program. Are these stated in terms of what students know, think, or do? List any evidence that is gathered on a regular basis (graduate admissions, exit interviews, comprehensive exams, etc). Is this evidence of effectiveness used to improve the program?
- Create an Assessment Plan: Decide where and how to gather evidence of the effectiveness of the program and teaching methods. Normally, no more than 3 or 4 intended outcomes would be assessed in a given year. If a program is being assessed (such as a major), it is possible to collect evidence from a sample rather than from all students in the program.
Also decide criteria for successful accomplishment of the intended outcomes ("75% of students will demonstrate mastery of X at the highest level"). - Implement the Assessment Plan: Systematically collect evidence of the extent to which objectives are being achieved.
- Use the results: Use evidence to improve programs by addressing intended outcomes that are not being fully achieved. It is also possible to decide that some intended outcomes may be unrealistic or inappropriate.
- Refine the Assessment Plan: Review the usefulness of the assessment program and make relevant modifications. Is the assessment plan providing useful information?
Some Principles of Assessment
- Student learning goals should be important to the program and clearly stated.
- Assessment practices and evidence should be useful.
- Assessment methods and measures should be
cost effective and as accurate as possible. - Program assessment focuses on program effectiveness, not on individual faculty.
- Evidence can be gathered from a sample of students, courses, or student work.
- Assess no more than 3 or 4 goals each year.
- Where possible, use multiple sources of evidence for each outcome but the multiple sources might be spread out over years.
- Gather evidence using methods that are appropriate and relevant to your discipline. These methods may be quantitative, qualitative, or both.
- Not every faculty member in a department needs to be involved in
assessment every year.
The Bottom Line: assessment involves reflecting on our work in order to continually improve our work.
Taking an Outcomes Approach to Education
Process Type Statements focus on institutional procedures or faculty actions (means). These are especially relevant for a program review and answer the question, "Is our program consistent with our goals?"
- We require students to take courses in the following areas...
- We require majors to complete 36 credits in the major.
- Our classes will be structured around collaborative learning.
- We will teach in ways to encourage active learning.
- The program will enjoy a national reputation for excellence.
Outcome Type Statements focus on the impact of our actions, such as student knowledge and skills (results). These are relevant for assessment and enable us to address the question, "Is our program creating the results we intended?"
- Students will demonstrate knowledge of the central theoretical debates in the discipline of...
- Students will identify, analyze, and evaluate arguments as they occur in their own and other's work.
- Graduates will be able to work effectively as team members.
- Students will locate, evaluate, and synthesize information from a variety of sources.
- Students will express themselves clearly according to the conventions of standard English.
- We expect our graduates to become socially responsible citizens.
Learning Outcomes
A learning outcome states a particular kind of knowledge or a particular skill, attitude, or value that students are expected to possess by the time they complete a course, a program, or some other area of the curriculum, such as general education. Although student learning is presumed to be partly a function of a program's effectiveness, learning outcomes are statements about students' knowledge and skills, not about curriculum or faculty. Thus, "Students will demonstrate the depth and breadth of our program through their post-graduate experiences including educational achievements, career paths, and other life experiences" is not a learning outcome, because it states that students will demonstrate the program's "depth and breadth" rather than specific knowledge and skills. To keep the focus on students, learning outcomes typically begin with a phrase such as "Students will demonstrate knowledge of..." or "Students will demonstrate the ability to..."
Rubric
A rubric, or scoring rubric, is a grid used to assess student performance relative to one or more learning outcomes. Typically, each column of the grid names an outcome, while each row identifies a level of performance. At each intersection of row and column, a few sentences describe the qualities that define a given level of performance on a given outcome.
Course-Embedded Assessment
Validity in Assessment
In assessment, validity refers to the correspondence between assessment results and student learning. Suppose that a means of assessment is designed to measure whether a student knows x. The means of assessment is valid if results indicate that the student knows x only when the student really does know x.
Since it is impossible to know whether a student knows x without relying on some means of assessment, the only way to demonstrate that a means of assessment is valid is to compare its results with those from other means of assessment. If several means of assessment agree in indicating that the student knows x, it is likely that the student indeed knows x. In this way, one means of assessment serves to check the validity of another means. Thus, it is important to use multiple means of assessment.
Reliability in Assessment
In assessment, reliability refers to the consistency with which a means of assessment yields a certain result. Thus, a scoring rubric is reliable if multiple assessors, applying it to a particular piece of student work, arrive at the same score for that piece of work.
One way to establish that an assessment method is probably reliable is for the assessors who will be using it to hold a practice session in which they apply the method to examples of student work, compare their scores, and discuss whatever differences they find. This activity is likely in itself to increase the reliability of the assessment method because the comparison and discussion of scores
Another way to increase the reliability of an assessment method is to use teams of readers, rather than individuals, to score each piece of student work. The members of a team score the work independently, discuss their results, and settle together on a single score.
Assessment vs. Grading
Letter grades are useful for evaluating individual student performance but normally do not provide information that is sufficiently specific for program assessment. The table below is adapted from Nichols, The Departmental Guide and Record Book For Student Outcomes
| Intended Learning Outcomes | Student A | Student B | Student C | Student D | Student |
|
| Outcome I | 3 | 4 | 1 | 2 | 3 | 2.6 |
| Outcome II | 2 | 5 | 3 | 2 | 5 | 3.4 |
| Outcome III | 4 | 5 | 2 | 3 | 4 | 3.6 |
| Outcome IV | 4 | 3 | 4 | 5 | 3 | 3.8 |
| TOTAL | 13 | 17 | 10 | 12 | 15 | |
| STUDENT GRADE | C | A | D | C | B |
Typically, grades sum the evaluations of multiple outcomes. Students with the same grade could vary considerably in their ability on a single outcome. Students with different grades could be equal in their ability on a single outcome. If we want to know about student ability related to an outcome, we need to collect information specific to that outcome.
Major Methods of Assessing Student Learning
- Standardized Tests
- Locally-developed Tests
- "Embedded" Test Items
- Capstone Courses
- Capstone Experiences
- Portfolios
- Qualitative Interviews or Observation
- Surveys of Student Attitudes
- Surveys of Alumni
- Presentations
- Performances
- Theses