• EHHP Home Page
• Faculty and Staff Directory
• Departments

Assessment System

The unit assessment system's primary purpose is to monitor initial and advanced candidates in the unit's programs. Data generated in the process are used to improve the unit and its programs. The assessment system contains clearly delineated articulation of a sequential, developmental process through which our candidates acquire the knowledge, skills and dispositions of competent teachers as expressed in our conceptual framework. The assessment system is an integral part of how we operate as a unit; we use it frequently to facilitate the growth of educational professionals across the span of their careers. The purposes of the School of Education's unit assessment system are multiple and interrelated. They are: 1) to inform applicants/candidates of the School of Education's expectations prior to admission and of their progress once admitted; 2) to inform the unit and programs on candidates' progress; and 3) to provide data that are used, in conjunction with data from other sources, to analyze unit operations and to improve the unit and its programs.

Prior to the development of the formal unit assessment system, the unit used a variety of assessments and evaluations to make decisions about candidates, programs, and unit operations. However, data were not being used as systematically and comprehensively as possible. In our new system we use a number of existing evaluation procedures while others have been developed, piloted, implemented and further refined to meet specific unit assessment needs. Our comprehensive system collects data using multiple measures and draws from multiple sources both external and internal to the School of Education. Each item at each assessment point has a rationale for its inclusion (i.e., it relates to the conceptual framework; it reflects best practice; and it is grounded in unit, state and national standards) (see Rationale for Unit Assessment System Requirements UG, M.A.T., M.Ed. ). Acceptable and unacceptable performance is clearly defined for each item. A number of opportunities were built into the system to check for bias, accuracy, consistency, and fairness. Our commitment to pilot testing and refining items in the assessment system was one of those checks.

The development of the assessment system occurred simultaneously with the revision of the conceptual framework, revision of all programs due to new institutional, state and national standards, and changes in certification levels in our state (see Status of Program Curriculum Revisions or Table 1 on p. 3).   This has resulted in improved initial and advanced programs.

The initial design of the skeleton of the assessment system took place over a period of a year from spring 2001 to spring 2002 (see UAS Timeline Graphic ). During this time a team of faculty drawn from several departments in the unit developed the assessment system rationale, the decision points, the relationship to the conceptual framework, and the key items that would provide the necessary data on initial and advanced candidate progress (see UAS Timeline ). A preliminary design, researched and created by two faculty members and a visiting clinical instructor, was presented to the unit for consideration and refinement in fall 2001 (see UAS Conceptual Part I ). The plan was developed using ideas in practice at outstanding sister institutions, an extensive research base, and current College of Charleston and School of Education assessment activities. The design accommodated internal and external assessments (aligned with professional, state, national and institutional standards) and covers the span of our educational programs from initial through advanced. Throughout the process we followed our particular style of operating as a unit, to work often as a committee of the whole, in order to reach consensus. The overall system design and individual components of the system were discussed and debated in several full School of Education faculty meetings throughout the time of its design and early implementation.   We were fortunate during this time to have three clinical faculty members working on staff with us. In addition, we were also fortunate to have received a grant from the State Department of Education's Office of Teacher Quality. Through this grant, a team of unit and Arts and Sciences faculty, as well as members of the P-12 community and the greater Charleston community, provided feedback and design ideas to the unit.

In fall of 2001, the responsibility to direct the development of the assessment system was formalized as a task of the Assessment Committee. Representatives from each of the unit's departments serve on this committee. Several different groups of people, representative of our professional community, both internal and external to the School of Education, had multiple inputs into its development. The system, as it was under development, was shared widely. Groups were not just asked to receive our ideas but to deliberately consider, to question, and to provide input. The Assessment Committee formally shared its work and sought input from others in the professional community such as:

• The School of Education Advisory Board
• The Teacher Education Council
• The College of Charleston Graduate Council
• The School of Education Administrative Council
• Teacher Quality Coalition Team
• Program Advisory Committees
• The Dean’s Student Advisory Council
• The Teacher Leadership Alumni Council

Our assessment system is based on NCATE's notion of multiple decision points. (see UAS Conceptual Part 2 ). All candidates (initial and advanced) enter into the same system but at different places. This reflects and models our belief in the developmental nature of becoming a master teacher and a life-long learner. Decision points, referred to as unit assessment points (UAPs), provide clearly marked progress points for all candidates as they move through programs. The system insures minimum competency, timely feedback to candidates, and fairness for all candidates. The decisions made at each unit assessment point are discussed further below.

The South Carolina ADEPT System (Assessing, Developing and Evaluating Professional Teachers) is a critical component of our assessment system. ADEPT is a statewide system of integrated performance-based standards, guidelines, and strategies designed to identify competence in the teaching profession. ADEPT integrates the state and national standards for teacher performance and aligns with our conceptual framework (see Standards Alignment Matrix). Our confidence in this system has been strengthened by an external review by the Anderson Research Group that concludes “the program [ADEPT] provides a clear and explicit definition of good teaching, contains clear expectations for teacher knowledge and performance, provides a common language for teachers and administrators to talk about good teaching…and focuses on continued growth and development of teachers” (Anderson, 2003, pg. iv). This is the system not only used to evaluate our candidates in field experiences and clinical practice, but also by the state's school districts in teachers' provisional years in the profession. By using the ADEPT system we support the state system of evaluation, prepare our candidates for future evaluations, and capture very specific data that can be aggregated and used for unit decision-making about candidates and programs. The ADEPT Performance Dimensions are used in field experiences and in clinical practice as the state performance standard for our candidates.

As we worked with the new assessment structure we realized that no existing data sources provide sufficient data on the dispositions of our initial and advanced candidates. The Assessment Committee, with the support of the unit faculty, engaged in a six month long process to define dispositions and their measurement over time. This work resulted in Dispositions Across a Professional Life Span, a major work that describes behaviors that we would expect to see in relationship to both the dispositions and the developmental level of a candidate. This work was partially supported by the Standards and Assessment grant from the South Carolina State Department of Education and was presented at the First Annual Conference on Dispositions held at Eastern Kentucky University on November 21, 2002. We received additional input and considerable interest from colleagues around the nation. Dispositions are assessed at each UAP from admissions through completion for all candidates. Faculty members also use them for individual conferences with candidates.

In spring 2002, we began to implement components of the unit assessment system, to gather data in each component, and to modify the system as needed. The implementation of the unit assessment system coincides with major curriculum changes in all of our programs (see Status of Program Curriculum Revisions or Table 1, p.3); these changes constitute both a challenge and an opportunity. On the one hand, we had to postpone implementation of some components of the assessment system because of curricular changes. Some of the curricular changes have been so extensive (e.g., the existing bachelor's degree in elementary has been reconstituted into three degrees – early childhood, elementary, and middle grades leading to a complete overhaul of the program courses) that it did not make sense to implement components of the unit assessment system until the new curriculum was in place. On the other hand, as major curriculum and program revisions were undertaken we had the opportunity to embed the unit assessment system within program assessment systems. This implementation results in a seamless, coherent assessment structure, which candidates, programs, and the unit use for improvement. We are also planning for the future. Our M.Ed. in middle level education (approved to begin fall 2005) was structured considering the conceptual framework and the unit assessment system from its inception. Despite these changes, many of the data items in the system have been fully used to make decisions about initial and advanced candidates, programs and unit operations for over two years. For example, the special education programs (initial and advanced) have pilot tested the full unit assessment system since their revisions were completed a full year ahead of other programs. The faculty in these programs volunteered to implement the whole unit assessment system as a pilot and to build the connection between unit and program assessment (SPED presentation to faculty). We have a full two years of data from these programs and their candidates on both the initial and advanced levels. In addition, ADEPT data are available for three years for all programs, and external data (Praxis tests) are available for all programs for three years.

In spring 2003 our unit made the decision to use electronic portfolios to assist candidates' presentation of performance-based evidence at UAP 2 (admission to clinical practice) and UAP 3 (recommendation for certification). Our research on portfolios and commitment to using information technology led to the decision to use electronic portfolios as a way to collect candidate work in professional courses, field experiences and clinical practice. We recognized that electronic portfolios allow for program and candidate needs while at the same time providing aggregated data to the unit. We adopted the LiveText system and decided to phase-in the requirement to use LiveText based on when candidates took their program required technology course (undergraduate initial: EDFS 326, advanced initial: EDFS 687).   All portfolios share a common format based on the conceptual framework's three Elements of Teaching Competency (ETCs). An initial rubric was designed for unit wide portfolio assessment. After a semester of using the rubric, the data gathered indicated that the rubric was confusing and the data were therefore not reliable. New rubrics were developed by each program for portfolio assessment, and a new rubric was designed for the unit wide portfolio assessment (see Portfolio Unit Rubric and Portfolio Program Rubrics and Requirements) .

The unit assessment system also serves to monitor M.Ed. advanced candidates. In the original design of the system, we did not account adequately for the candidate-specific nature of the advanced programs. A revised series of Unit Assessment Points and the identification of the corresponding data to be gather at each point for advanced candidates is now in place. Data presented in Standard 1 illustrate how advanced programs are assessed at both the unit and program levels.

At the completion of clinical practice, interns have always provided feedback on their programs. During the 2001-2002 the questions asked of interns were changed to reflect the Teaching and Learning Standards. The revised Teacher Education Program Evaluation provides invaluable data on interns' perceptions of program effectiveness.  

The College of Charleston systematically collects data from graduates through an alumni survey sent out annually from the Office of Institutional Research. Given our desire to obtain feedback from graduates and their employers in a form compatible with the conceptual framework, we designed School of Education graduate surveys and employer surveys in September of 2003 (see Feedback from Candidates, Graduates, and Employers).   The surveys consist of a series of questions based on ADEPT professional dimensions that are organized according to the conceptual framework's Elements of Teacher Competency. The first draft of the surveys contained all of the ADEPT professional dimension descriptors. To condense the survey, faculty members were asked to review the first draft and select statements they felt were most vital in assessing candidate knowledge, skills, and dispositions.   In addition, we included one question for each of the eight dispositions.

The final draft of the survey was printed on two different colors of paper, in order to distinguish between the copy to be completed by the graduate and the one to be completed by his/her employer. Using addresses obtained from the Office of Alumni Affairs, we sent surveys to all School of Education Bachelor's and Masters of Arts in Teaching graduates from the past five years (1998-2002). Each graduate was asked to complete one copy of the survey as a self-reflection and give one copy to his/her employer for completion. In future years, we plan to send the survey only to candidates who graduated two and five years ago.   Approximately 500 surveys were sent out in February 2004 and 102 graduate surveys were returned and 25 employer surveys were returned. Over 150 surveys were returned because addresses were no longer current. Attempts to encourage more returns were unsuccessful. A focus group of employers is scheduled this fall to discuss the surveys and how to improve the return rate. A discussion of the survey is on the agenda of the next meeting of the Teacher Leadership Alumni Council .

The assessment system captures the candidate proficiencies outlined in the conceptual framework. Our slogan, “Making the Teaching and Learning Connection” is enacted in three Elements of Teacher Competency (ETCs). These elements, understanding and valuing the learner, knowing what and how to teach and assess and how to create an environment in which learning occurs, and understanding self as professional serve as an umbrella for candidate proficiencies (e.g., the knowledge and skills described in the SOE Teaching and Learning Standards and professional and state standards, and the School of Education dispositions). ETCs are the defining categories in the unit assessment system as applicants and candidates pass through the unit assessment points. At each UAP we have identified measurable candidate proficiencies (knowledge, skills, and dispositions) that are used to ensure that our candidates, in all programs and at all levels (initial and advanced) achieve unit outcomes defined in our conceptual framework. The relationship between the Elements of Teacher Competency, unit, state, and professional standards is described in the Rationale for Unit Assessment System); by making this alignment clear, we are confident that the unit assessment system collects data that are useful for all candidates and programs. Unit data are gathered at three critical UAPs during the program and at two additional assessment points after candidates complete our programs. (see Unit Assessment System-All).

Our conceptual framework guides advanced candidates as they progress through a series of experiences designed to strengthen their teacher leadership qualities. They develop a deepened understanding and application of the Elements of Teacher Competency in their program. The lines that separate the ETCs become less distinct at the advanced level for the goal at this level is to integrate and synthesize the elements in classroom practice. To foster the goal of integration, each program has designed specific coursework, field experiences and culminating experiences.  

Initial undergraduate and M.A.T. candidates and their performances are assessed through a series of five UAPs (see Unit Assessment System Initial Undergraduate and Unit Assessment System Initial Graduate). The first three UAPs provide data for the unit and programs to make admission or continuation decisions. Programs have placed other decision points internally in their individual program systems that support the unit system.   Data from multiple sources are collected at these decision points. The final two UAPs provide longitudinal feedback on candidate performance in the field. The Rationale for Unit Assessment System Requirements describes how multiple assessments are used to monitor candidates' development of knowledge, skills, and dispositions required by the profession, state, and institution. Table 2.1 describes how assessments are used to monitor undergraduate and M.A.T. candidate performance. As illustrated, the admissions criteria at these two levels differ slightly.

Table 2.1
Data Collected to Monitor Initial Candidate Progress

Advanced candidates enter our unit assessment system at UAP 3 – Competent Level (see Unit Assessment System-All). Most are active professionals who hold current licenses and certifications. They currently are teaching and complete the program part-time or take a leave of absence to complete the advanced program. Advanced candidate performance is assessed on the unit level by a series of five unit assessment points. They enter an advanced program at UAP3; their ability to continue is assessed and communicated to them at UAP 3.5; and a degree is award upon successful completion of all requirements at UAP 4. Follow-up of candidates (UAP 5) is conducted by programs. In addition, it is the Graduate School policy to continuously assess candidates at the advanced level semester by semester. This is done in two ways, by monitoring course grades and by monitoring progress towards degree.

Two of our approved advanced programs, the M.Ed. in Science and Math for Teachers and the M.Ed. in Modern Languages, both of which are interdisciplinary programs, admit some candidates who are not currently certified as teachers. These are candidates who are seeking knowledge and preparation for positions outside of school such as in labs, museums, youth programs, and aquariums. Admission of candidates to these degree programs is evaluated by the graduate program directors using specific admissions criteria specific for each program.

Table 2.2 describes how assessments common to all M.Ed. programs are used to monitor M.Ed. candidate performance. To demonstrate that a candidate has acceptable knowledge, skills and dispositions to continue in the program and ultimately be recommended for graduation, the candidate must also meet the requirements of the specific program. These requirements differ from program to program due to the nature of advanced degree work that is very program specific. Table 2.3 presents program-specific assessments used at UAPs 3.5 and 4.

Table 2.2
Data Collected to Monitor All Advanced M.Ed. Candidate Progress

Table 2.3
Data Collected to Monitor Advanced M.Ed. Candidate Progress by Program

Table 2.4 provides a summary description of the types of data used to assess candidate performance.

Table 2.4
Assessment of Candidate Performance

The School of Education has been careful to insure that assessment procedures are fair, accurate, consistent, and free of bias; as a unit we seek quality and credibility of assessments. Each item at each assessment point has a rationale for its inclusion (see Rationale for Unit Assessment System Requirements). Items were carefully chosen and refined along the way to support our conceptual framework and to be credible assessments of the knowledge, skills, and dispositions we expect all candidates to develop. We pilot test assessments and make needed adjustments. For example, we initially required a writing sample at undergraduate admission. We piloted this for two semesters and found that the data were of little consequence and that a passing Praxis I writing score was sufficient.

We ensure fairness, accuracy, consistency, and non-bias in our assessment procedures and any decision-making that affects candidates.   Table 2.5 contains a partial listing of the strategies we employ to ensure accuracy, fairness, and consistency in assessments and policies.

Table 2.5
Assuring Accuracy, Consistency, and Freedom from Bias in Assessments

The preceding discussion describes the assessments used for initial and advanced programs. The assessments are quite different between these programs for several reasons. First, uniform standards inform initial teacher preparation programs. Professional standards such as INTASC define good teaching for all initial candidates. South Carolina’s ADEPT system also applies to all initial candidates. Although advanced programs align with National Board for Professional Teaching Standards (NBPTS), these standards are not the same across teaching areas, and some advanced programs (e.g., our Science and Mathematics for Teachers) respond to several sets of professional standards. Second, external assessments (Praxis II and Praxis III) are available to determine candidates’ knowledge and skills at the end of initial programs. Such standardized assessments do not exist for advanced programs. Third, given common standards and the fact that initial candidates will be seeking certification (and ultimately will be seeking teaching positions), a portfolio requirement across all initial programs makes sense. Such a requirement is not appropriate for advanced programs with very different outcomes.

The School of Education maintains the unit assessment system to provide regular and comprehensive information to manage and improve operations and programs of the unit. It is used to evaluate initial and advanced candidate proficiency, initial and advanced program quality, and unit operations. As Table 2.6 (see p. 53) attests, the sources of data are multiple, internal and external to the School of Education, and are collected in a systematic manner. Data from initial and advanced candidate assessment are a major source of information for program assessment. This is augmented with other data sources to make decisions about unit operations.

back to top

Data Collection, Analysis and Evaluation

The School of Education maintains an assessment system that provides regular and comprehensive information. It is used to evaluate initial and advanced candidate proficiency, initial and advanced program quality, and unit operations. As the following table attests, the sources of data are multiple, internal and external to the School of Education, and are collected in a systematic manner. Data from initial and advanced candidate assessment are a major source of information for program assessment. This is augmented with other data sources to make decisions about unit operations.

Table 2.6
Some Sources of Data Used to Make Decisions about Unit Operations and Programs

Assessment of candidate performance, program effectiveness, and unit operations is coordinated through the dean's office. Aggregating and summarizing data are a shared responsibility of the dean, program directors, department chairs, the director of the Office of Certification and Clinical Practice and the associate deans. For example the dean is responsible for collecting and summarizing the department reports into a single annual report; the chairs are responsible for faculty evaluations; the associate deans are responsible for tabulation and submission of follow-up survey data to the Administrative Council and the Teacher Education Council; and the director of Clinical Practice and Certification is responsible for the compilation of data for the Annual Title II report.

Faculty members systematically and continuously use data to reflect on and improve their own practice. The annual evaluation of individual faculty members relies significantly on the course evaluations completed by candidates. Faculty members work with their department chairs to reflect on their teaching performance, set teaching goals and develop ways to improve their teaching, scholarship and service.

As data are made available to the dean, they are evaluated, interpreted, summarized and reported. The reports are shared with the applicable governing and advising bodies. Recommendations regarding changes in unit operations are made at every level following established policies and procedures.

We are assisted in our efforts to improve candidate performance, program quality and unit operations through the use of a database system (EDA-SYS).   A professional knowledgeable in information technology was specifically hired to work alongside the Assessment Committee to design this database. EDA-SYS mirrors the unit assessment system and allows us to collect individual, aggregated, and summarized data. Several rubrics are stored in this system. Many reports can be generated in real time (i.e., the number of enrolled undergraduate middle level majors, the number of candidates anticipated to be ready for clinical practice in fall 2005, the average GPA of all secondary education minors, and a list of those who applied but have not met UAP 1 (admissions) requirements. This allows for continuous planning based on current data. Part of the data in our system comes directly from the College of Charleston Student Information System (SIS) and is updated daily. The staff in the Office of Clinical Practice and Certification enters other data (i.e., ADEPT scores, dispositions scores, graduate and employer survey data). We have worked with the Office of Institutional Research to ensure that the data in our system are accurate and complete, and we have developed procedures to protect the integrity of the system (i.e., faculty members have private user names and passwords). The security of the system is the responsibility of the dean's administrative assistant. All candidates who are seeking initial certification, undergraduate and graduate, are in this electronic system. Advanced candidates are in the system but program level data are kept by program directors. We are seeking ways to merge the database system of the Graduate School with EDA-SYS so that data entry is done only once. We expect the advanced candidate component of the electronic database to be fully operational by spring 2005.

Data regarding formal candidate complaints are not kept in the electronic system but in the dean and department chair's offices. There are procedures and policies that govern the lodging of formal complaints and the documentation of their resolution (see School of Education Policies and Procedures Handbook ). The complaints are analyzed as part of the unit assessment annually.

back to top