Standard 2

Element 3: Use of Data for Program Improvement

Major Indicators for Program and Unit Decision-Making. The unit assessment system includes data for monitoring unit operations as well as candidate progress. The types of information incorporated into the unit assessment system are related to programs, personnel, resources, candidates, and graduates. An assessment coordinator works with program and unit committees to develop/select assessments for all levels of the system and to use the data for decisions about candidates, programs, and the unit. Data are disaggregated by program area and important candidate demographics as well as aggregated across programs to aid in making decisions.

Since it would be impossible to do an in-depth analysis of each important aspect of the unit each year, a schedule has been developed for in-depth studies. The schedule follows in Table 2.3:

Figure 2.2 shows the data dissemination and use process. All program area assessments and additional unit assessments, as well as personnel and operations data, are entered into the unit data management system (currently more than one database, but moving to a more integrated data management system) by the assessment coordinator and Tk20 coordinator as soon as they are available. At the end of each semester, it is the responsibility of the assessment coordinator to formally analyze and share assessment data with program coordinating committees, TEC, the Graduate and Undergraduate Studies Committees, and the Administrative Council. The assessment coordinator will also prepare an annual report for more universal dissemination.

Figure 2.2: Unit Assessment System Data Dissemination and Use

Ongoing Communication and Progress Monitoring. Candidates and advisers can look at candidate-level progress data at any time through use of the Academic Requirements Tracking System (ARTS). ARTS forms are reports on the progress that a student has made toward completion of his/her plan(s)/degree(s). The ARTS system also produces reports and data files that can be processed by statistical packages and other commercial programs to further quantify or process the candidate data, if staff has a need for further analyses.

Two additional software applications encourage information sharing among faculty, staff, and candidates. BlackBoard is a university-wide, web-based system that functions as a container for course materials and activities. It includes a course management system and other tools and features, such as an online course development tool. In the course development tool, instructors create a list with their overall course goals for the students and then delineate the individual unit tasks. BlackBoard can be used to post assignments, manage course data, record grades, generate course statistics, and generate online surveys. Candidates are encouraged, and sometimes required, to participate in discussion boards.

Tk20 is a vendor-supplied online support application for colleges of teacher education. The application is customized according to a unit’s conceptual framework, standards, artifacts, rubrics, portfolios, surveys, and other forms. With Tk20, instructors can create a course syllabus and assignments, each assignment tied to one or more standards, and send them to students; students can complete assignments and upload them to instructors; rubrics for rating assignments can be created and shared; and students can create artifacts, such as lesson plans, and upload them for feedback.

Examples of Unit Assessment System Data Driven Decisions. Significant revisions were made to the unit overall, beginning in 2002 and becoming fully operational in 2004, as a direct result of unit assessment system data. Prior to this revision, it had been 20 years since any substantive changes had been made. There were six primary motivators for the 2004 changes:

• The new licensure system adopted by the State prompted the need to re-examine programs and align curricula with the new standards;
• Accreditation standards became more outcome based and required changes in the way programs and the unit operated and were evaluated;
• There was a growing concern among candidates—identified through the C&I Surveys, Summative Evaluations, and faculty advising—about redundancy in courses;
• The C&I Surveys, Summative Evaluations, and faculty advising indicated that candidates were concerned that they were not receiving sufficient training in special education, multicultural education, reading, technology, student assessment, and teaching methods;
• The C&I Surveys, Summative Evaluations, and faculty advising showed that many candidates were not able to enroll in the courses they needed in order to graduate on time;
• Analysis of EBI Survey data showed that candidates were not satisfied with the quality of advising services.

The 2004 revisions included changes to course sequencing to make program coursework less redundant and more cohesive. Interdisciplinary courses were blocked so that they could have a common, more meaningful, field experience. Required hours in general education courses were reduced from 60 to 45 hours so that candidates could have more hours in field experience, reading, special education, multicultural education, and technology. A candidate cadre system was adopted for elementary education candidates so that they could get the courses they needed when they needed them, and to encourage more candidate interaction, collaboration, and professional growth. The number of weeks spent student teaching was increased because of comments about the need for more time in the classroom. The things that were working were not revised. For example, the unit retained the five-year model in most initial program areas, which allows fifth year candidates to gain more field experience and at least 15 hours of graduate credit towards their master’s degrees.

The unit continues to use unit assessment system data to make decisions. As soon as the Kansas Performance Assessment (KPA) requirement was adopted by the State and the role of the unit was made clear in preparing teachers for this assessment, the existing inquiry-based unit plan and the action research project that investigated issues related to the candidates' teaching and learning (completed during the internship period) were aligned with KPA.

An important recent finding from assessment data was that candidates are not afforded equal opportunities to view samples of quality teaching practices. Since the unit is not always able to control the quality of teaching that candidates view when they are in the field, efforts have been made to bring good teaching practices to the methods classes. For example, in the foreign language area, a collection of professional development videos were purchased through a Center for Teaching Excellence teaching grant, "Teaching Foreign Languages K–12", produced by WGBH Educational Foundation with the American Council on the Teaching of Foreign Languages in 2003. These videos allow all candidates to observe quality teaching practices.

Although a large percentage of candidates are able to make adaptations to meet individual student needs, data analysis identified this as an area of concern. Since data show that classrooms are becoming more diverse and the array of student needs is growing, our candidates must be knowledgeable in ways of modifying and adapting lessons and activities to meet students’ various needs and promote learning among all students. Ways to meet the needs of ESL students, special needs students, gifted students, and students who are having difficulty grasping one or more concepts are being examined. More courses and field experiences include assignments on adapting/modifying activities to meet student needs. The process began with faculty members determining the activities to be included in a program area and where to include the activities.

Survey data indicate the need for more emphasis on how to use lesson or activity assessments to document and evaluate student learning and modify teaching practices. Therefore, an extended discussion on how to use lessons or activities as assessments of student learning was added to program curricula.

Two years ago, the school and district leadership program areas noted that a number of candidate papers were not as well reasoned and written as expected. As a result, candidates are now asked to turn in a first draft of their papers and provide a detailed critique of the draft, after which the students are required to prepare a final version of their papers. Candidate concerns over burgeoning tuition costs caused the unit to examine ways in which the rate of increase of these expenses could be slowed. This examination led to a reduction in the number of credit hours taken during their professional year field experiences. The length of time required in the schools remained the same; the costs decreased.

Over the last three years, evaluations of candidate work in the C&I courses have revealed that many candidates enter their respective programs with limited understanding of curriculum theory, planning and development, and evaluation skills. As a result, the C&I courses have been substantially revised and greater linkages have been made among them. The revisions should assist students in seeing how curriculum and teaching can be better conceptualized and supervised at both the building and district levels.

At the unit level, EBI Survey data showed that the quality of advising that students were receiving from faculty could be enhanced. As a result, centralized advising for all initial candidates will become a reality in the fall of 2007, with the enhancement of services offered by the Advising Center. The center currently provides advising only for freshmen and sophomore pre-education students. Candidates will also be served after the change. This change will provide consistently accurate information to university students in unit programs. It will also “free up” faculty time so that they can focus even more clearly on their teaching and scholarship roles.

Summary. From the beginning of their academic programs of study, the unit fosters our candidates' preparation and understanding with the most current research and best practices, and ensures content knowledge acquisition appropriate to their professional aspirations, while developing a high regard for professionalism as they progress through our programs. By preparing educators as leaders in their future careers and by continually working to create programs designed to further enhance all levels of education, we believe these future educators will be positioned to bring about long-term, fundamental change. Our candidates learn to anticipate and plan for the future, construct and apply a coherent, integrated understanding of teaching and learning, engage in collaborative problem-solving and critical inquiry, strive to enable all students to reach their potential, and continually assess and improve their practice for the benefit of all students.