STANDARD 2. ASSESSMENT SYSTEM AND UNIT EVALUATION
2.1
The unit has an assessment system: PASS-PORT, a web-based system that provides candidates, faculty, and administrative staff a tool to gather, demonstrate, and evaluate candidate performance (Evidence: Assessment System Diagram & Unit Assessment Plan). The assessment system collects, analyses, and disseminates data on applicant qualifications, based upon the unit’s conceptual framework, university requirements, state standards, and program requirements of specialized professional associations (SPA). Faculty and unit administration collect and analyze data on applicant qualifications, candidate performance, and unit operations both internally and externally. These data are used to improve the performance of candidates, the unit, and its programs. The development of the assessment system began in the fall of 2013.
Between 2011 and 2013, the School of Education and Behavioral Sciences (SoEBS) underwent substantial personnel changes with the Dean; Director of Teacher Education, Certification & Field Experiences; Coordinator of Assessment and Student Advisement; Department Chair and several senior faculty retiring or leaving the unit (Evidence: Historical View). The unit was without an Assessment Coordinator until a Unit Assessment Coordinator was appointed in fall 2013 and renamed the Director of Unit Assessment and Accreditation in fall 2014. The new Director oversaw the initial adoption of PASS-PORT in fall 2013, assumed responsibilities for combining the then existing paper-based assessment system with the new system; led the full implementation of PASS-PORT in fall 2015, and provided faculty and candidate training and support.
The unit assessment system is monitored and assessments are added, deleted, or revised based upon results of data analysis and decision made by the Teacher Education Council (TEC). The unit indicates that the TEC, includes faculty from the School of Arts and Sciences and the School of Agriculture and Applied Sciences, and is representative of the Langston University (LU) professional community. As members of partnership schools are not members of the TEC, the unit established the Teacher Education Advisory Council in spring 2015. The Advisory Council is charged with advising the Dean of the SoEBS, Director of Teacher Education, Certification, & Field Experiences, Director of Unit Assessment & Accreditation, and other appropriate administrators. This Council:
-
Provides a forum for the exchange of information, suggestions and perspectives about professional education practices and programs;
-
Reviews unit programs that lead to educator certification in meeting institutional, state, and national standards;
-
Reviews unit data and makes recommendations for continuous improvement specific to the assessment system and candidate performance. (Evidence: Advisory Council Membership & Advisory Council Constitution)
PASS-PORT allows candidates to collect specific artifacts in professional education courses, upload them to PASS-PORT, and submit them to their course instructor, unless noted otherwise on the Master List (exhibit 2.4.a. PASS-PORT Master List). Assessments submitted to SPA’s as outlined in program reviews, are aligned to the individual content areas and are anticipated to be included in PASS-PORT by fall 2016.
Data are collected at three (3) transition points in the initial program and at four (4) transition points for the advanced program (Evidence: Unit Assessment Plan, Assessment System Diagram, & Navigating Teacher Education F15). The comprehensive unit assessment system details the data collected at each transition point for initial and advanced programs. The updated undergraduate teacher education handbook, Navigating Teacher Education F2015, provides a graphic of the transition points and outlines all requirements associated with each transition. The graduate handbook is currently under revision to ensure consistency between the handbook and the assessment plan. The graduate handbook has a completion date of December 2015 and a distribution date of spring 2016.
The assessment system, PASS-PORT, encompasses a broad range of candidate performance indicators beginning with program admission criteria and continuing throughout the program. Evidence supports data collection for all transition points at the initial level and advanced level (Evidence: Unit Assessment Plan & Assessment System Diagram).
The unit has due process procedures for resolutions and documentation of candidate complaints. The unit provides evidence of a student complaint procedure form (exhibit 2.4.e. Complaint Form). Procedures for complaints are available in the undergraduate teacher education handbook (Evidence: Navigating Teacher Education, pages 19-20) and will be included in the graduate handbook currently in revision process.
Drafting policies and procedures for ensuring that key assessments of candidate performance and evaluation of program quality and unit operations are fair, accurate, consistent, and free from bias is a priority for the Assessment & Curriculum Committee. Although policies and procedures have not been committed to paper, they are planned for drafting in spring 2016 and for implementation in summer 2016. A system of a yearly review of assessment instruments and unit operations was instituted in summer 2015. A unit specific Desire2Learn (D2L) site was created and solicits feedback on current handbooks, instruments, and operations (BOE members will be able to access the site during the on site visit). Additionally, trainings on the use of instruments were implemented in fall 2015. Faculty were trained in the admission to Teacher Education interview and Teacher Education Clinical Practice interview process in September 2015. The Chair of the Assessment & Curriculum Committee and a member of the committee created a training video, provided clarity for the use of the rubric, created practice interview clips, and led faculty through the process to ensure that the process and tools are fair, accurate, consistent, and free from bias (Evidence: Interview Training Video available on site).
That the unit has timelines for data collection and analysis related to candidates and unit operations. Policies, procedures, and practices for ensuring that data are regularly collected, compiled, aggregated, disaggregated, summarized, analyzed, and used for continuous improvement is summarized in the Unit Assessment Plan (Evidence). Evidence of significant changes made to courses, programs, and the unit in response to data gathered from the assessment system can be found in viewing the old plans of study and the agendas and minutes of the TEC and the Professional Learning Community for Reading (available on site).
2.2.b
The unit is reviewing and revising unit processes and procedures specific to data collection and dissemination with PASS-PORT, its electronic online data management system that was selected in fall 2013, piloted in fall 2014 and spring 2015, and fully implemented in fall 2015 for its assessment system. With the use of PASS-PORT, the unit is transitioning from a paper-based assessment system. Currently, PASS-PORT is being used at the unit level with plans to use it at the program level by fall 2016. PASS-PORT has the capacity to provide regular and comprehensive information on applicant qualifications, proficiencies, competences, unit operations and program quality.
Timelines for data collection and analysis related to candidates and unit operations are being refined to ensure that PASS-PORT provides for the systematic use of data to evaluate the efficacy of courses, programs, and field and clinical experiences. To ensure that key assessments of candidate performance and evaluations of program quality and unit operations are fair, accurate, consistent, and free of bias. A system of a yearly review of assessment instruments and unit operations was instituted in summer 2015 with the creation of a unit specific Desire2Learn (D2L) site. The site was designed to collect feedback on current handbooks, instruments, and operations (BOE members will be able to access the site during the on-site visit). Additionally, trainings on the use of instruments was implemented in fall 2015 (Evidence: Interview Training Video available on site).
The unit is in the process of aligning course objectives, assessments, rubrics, and syllabi with revised SPA standards. In addition, the unit is working to ensure consistency across campuses regarding course content, textbooks, and syllabi for courses taught on more than one campus. Syllabi submitted for fall 2015 show the alignment of objectives, content, textbooks, and assessments across campuses (syllabi available on site). An annual program quality internal review (PQIR) process will be implemented in spring 2016 to ensure quality evidence is collected and analyzed in a systematic manner and used to inform and improve unit and program operations (Evidence: PQIR Draft).
2.3.a What AFIs are recommended for removal?
1.The unit’s assessment system was developed with limited involvement of the professional community.
ITP,ADV
2.The assessment system does not provide for the regular analysis of data to improve unit operations.
ITP,ADV
The assessment system does provide for the regular analysis of data to improve unit operations.
2.3.b What AFIs are continued from last visit?
1.The unit’s assessment system was developed with limited involvement of the professional community.
ITP,ADV
The unit does not provide evidence of involving the professional community as it is in the process of developing a procedure.
2.3 Areas of concern related to continuing to meet the standard
The unit assessment system does not provide evidence to exhibit the disaggregation of data and the relationship of the component parts of the assessment system.
Rationale: It is not evident from information provided in the IR or exhibits that the unit disaggregates data and the relationship of the component parts of the assessment system.
Evidence: Unit Assessment System
The teacher education council (TEC), who monitor, add or delete assessments in the assessment system annually, appears to not have documentation of collaboration.
Rationale: Because other substantive documentation of work of this committee was not provided as visible evidence, it is difficult to determine collaboration, composition thereof, or to the extent to which it is functionally engaged in shaping the assessment system as is described in the IR and it appears that there is no participation of the professional partnership community.
Evidence: TEC Membership
TEC Agendas & Minutes (available on site)
Advisory Council Agendas & Minutes (available on site)
The unit provides conflicting evidence regarding the number of transition points for the initial level and advance level programs in ESL, SPED, ED LEADERSHIP, ELA, and other programs, and confusing exhibits for various evidences.
Rationale: It is not clear how many transition points exist at the initial level, and other programs, or which exhibit represents what for exhibit 2.4.a.
Evidence: Unit Assessment System
Navigating Teacher Education F15
The IR indicates that there are exhibits for policies, procedures and practices for candidate performance; for ensuring data are regularly collected; for fairness, accuracy, consistence, free of bias; for managing candidate complaints; for significant changes made to course programs and; for unit response to data gathered from the assessment system are provided.
Rationale: It is not evident from information provided in the IR that there are:
-
policies, procedures, and practices for ensuring that key assessments of candidate performance are fair, accurate, consistent, and free from bias;
-
evaluations of program quality and unit operations are fair, accurate, consistent, and free from bias;
-
policies, procedures and practices for ensuring that data are regularly collected, compiled, aggregated, disaggregated, summarized, analyzed, and used for continuous improvement;
-
policies, procedures and practices for managing candidate complaints;
-
examples of significant changes made to courses, programs, and the unit in response to data gathered from the assessment system.
Evidence: Interview Training Video (available on site)
D2L TEU site (available on site)
Navigating Teacher Education F15
TEC Agendas & Minutes (available on site)
Professional Learning Community for Reading Agendas & Minutes (available on site)