Assessment of student learning and achievement demonstrates that the institution’s students have accomplished educational goals consistent with their program of study, degree level, the institution’s mission, and appropriate expectations for institutions of higher education.
Current Snapshot. City Tech has undertaken a major transformation in how educational effectiveness is measured since the last Self-Study:
- Under the direction of the Office of Assessment and Institutional Research (AIR) and using a continuous improvement framework, the College has institutionalized a comprehensive system of educational effectiveness assessment at the institutional, program, and course levels. Institution-level assessment measures student attainment of General Education outcomes across all schools and programs; program assessment measures attainment of Student Learning Outcomes. Faculty also assess courses identified as critical courses within their degree programs in a systematic manner. This approach is implemented through a cyclical process in which faculty, chairs, and deans work collaboratively to ensure alignment of assessment with course, program, and institutional goals and objectives.
- Faculty leadership of assessment activities has been effectively expanded so that Assessment Liaisons are in place in every department, Critical Courses have been identified, Assessment Liaisons have been appointed in each school, discussion of assessment results occurs in every department, and assessment workshops are designed to involve the broadest possible segment of the various constituencies.
- Frequency of assessment reporting has been increased and has yielded results used to strengthen the institution, providing evidence of commitment to the appropriate assessment of student achievement throughout educational offerings, regardless of certificate or degree level. The college uses assessment data to make investments in resources that promise to improve teaching and learning.
- Moreover, City Tech has emerged as a leader in academic assessment within the CUNY system, winning recognition as best-in-CUNY at the Association for Institutional Research national forum on best practices in five of the past six years (2012-2017).
Criterion 1. Clearly stated educational goals at the institution and degree/program levels, which are interrelated with one another, with relevant educational experiences, and with the institution’s mission;
Institutional Educational Goals. Educational effectiveness at the institutional level is correlated with relevant educational experiences and aligned with the College’s mission. The Education Goals, presented in Standard I Criterion 3, reflect outcomes of students’ learning experiences in General Education curricula and address the knowledge, skills, and habits of mind that all students should acquire as they complete their courses of study. These outcomes lay the foundation for lifelong learning.
In 2013, CUNY implemented the system-wide Pathways initiative to ensure that regardless of a CUNY student’s college of origin, he or she must fulfill a common set of general education requirements called Pathways. In turn, Pathways credits are universally transferrable across CUNY colleges; they must be accepted by the receiving institution. Very much like the College’s independently derived General Education goals, Pathways seeks to produce well-rounded graduates who have a critical appreciation of diverse cultural and intellectual traditions, a sense of history, and the ability to contribute to the improvement of society. Pathways also emphasizes skills through which such knowledge is gained: reading, writing, and quantitative reasoning. Pathways facilitates transfer from associate to baccalaureate degree programs and ensures coherence and rigorous intellectual standards across the University. City Tech’s General Education curriculum is aligned with the system-required CUNY Pathways (General Education Learning Goals). Faculty effort over the course of the 2015-2016 academic year validated the alignment between University Pathways course requirements and the General Education offerings of the College (Alignment of Gen Ed Outcomes to Pathways). Further, faculty have aligned City Tech’s General Education outcomes with institutional career and professional goals.
Program Level Outcomes. All programs have clearly articulated student learning outcomes that are published in the College Catalog (Catalog 2017-2018). Additionally, all departments have documented course alignment with program outcomes, providing evidence of validity for the course requirements within their degree programs (Assessment Documentation Folder). All degree programs will provide evidence of the alignment between the program-level Student Outcomes and the college’s revised Mission Statement. This mapping provides an opportunity to streamline the assessment process, yielding a more manageable, sustainable, faculty-driven assessment process.
Criterion 2. Organized and systematic assessments, conducted by faculty and/or appropriate professionals, evaluating the extent of student achievement of institutional and degree/program goals. Institutions should:
a. define meaningful curricular goals with defensible standards for evaluating whether students are achieving those goals;
Meaningful curricular goals and defensible standards for evaluation, as defined by the responsible academic department, are in place. All assessment tools undergo rigorous validation efforts and analysis before being used to assess Student Learning Outcomes, providing evidence of the institution’s commitment to ensuring that student learning opportunities provide rigor, coherence, and assessment to monitor student progress and make improvements as warranted. The Student Outcomes for each degree program are documented in the college catalog and on the college website, on the “Degree” page for each degree program under “Overview” (Sample: Accounting AAS Overview; Catalog 2017-2018).
The institution systematically evaluates its educational programs on a cyclical basis. Since spring 2016, department chairs and their Assessment Liaisons inform the school dean and upload documentation on the status of their assessment activities on an annual basis, rather than the previous three-year assessment cycle, in order to ensure compliance with assessment plans submitted to the College. These documents are maintained on the college S-drive, a shared location. (Assessment Documentation Folder). The following documents are maintained for each program/department on this secure server:
- Documentation of program-level assessment plans/cycle: a chart of the assessment activities of the program;
- Curriculum Maps aligning courses to program outcomes;
- Program-level assessment reports for all degree programs;
- Assessment cycle and plans for Critical Courses;
- Rationale for Critical Courses: narrative explaining why the course was selected;
- Reports for Critical Courses: documentation regarding the status of the critical course;
- Minutes of department meetings to discuss assessment; and
- The Department Assessment Committee membership roster, if applicable.
b. articulate how they prepare students in a manner consistent with their mission for successful careers, meaningful lives, and, where appropriate, further education. They should collect and provide data on the extent to which they are meeting these goals;
The outcomes defined for each degree program are designed to support the mission of the college by enabling students either to enter the workforce directly or be prepared to continue their education. This alignment has been documented for each degree program. In 2009, under the leadership of the provost, a new General Education Committee began to rethink General Education in significant ways, so that instead of being considered primarily as a set of distribution requirements, General Education is now understood to be a constellation of knowledge, skills, and habits of mind that provide the critical underpinning of all specialized technological and professional training. The resulting redesign of curricula serves rapidly changing professional programs and provides grounding to help students pursue meaningful, well-rounded lives. (General Education Learning Goals). Members adopted 14 of AAC&U General Education competencies as a tool to assess General Education (Gen Ed Assessment Workbook). We ensure the validity of the LEAP VALUE framework through rigorous validation and inter-reader reliability processes.
The CUNY system delegates the measurement of General Education outcomes to its constituent colleges. Thus General Education Student Outcomes are assessed college-wide and coordinated through the Office of the Provost and the AIR Office. In 2015, a committee of faculty drafted an academic assessment strategic plan, which was approved by the assessment committee liaisons from the college (City Tech Assessment Strategic Plan 2015_2018). The college reports the General Education results as the percentage of students that meet or exceed each performance indicator. The results are shared with the president, provost, deans, department chairs, the Student Government Association, Student Affairs offices, and the wider college community. In addition, the results on student success are shared via college social media (Twitter, Facebook, Instagram). Based on the timeline developed by the Assessment Committee, City Tech is currently on schedule with all planned assessment activities (Gen Ed Assessment Timeline 2016-2022).
As of the Fall semester of 2017, the college will complete the full-scale assessment of outcomes in all of the fourteen pillars related to General Education outcomes. As the work is completed, General Education briefs are prepared and disseminated, in easy to understand, non-specialist prose, to present outcomes in each area to the campus community (Gen Ed Assessment Briefs Folder). In areas where careful examination has been completed, enhancements have been introduced, with promising results.
For example, Information Literacy is crucial in any institution, but most especially in a college of technology. During 2013, the College’s full-scale assessment results of Information Literacy indicated that only 60% of the students met or exceeded the faculty target. The Library faculty, supported by the college’s Living Lab Fellows program, were responsible for leading the institution’s effort to address this shortcoming (see Table 1). When the re-assessment of Information Literacy occurred in 2016, the college increased the student attainment of faculty standards by six percent, with 66% of the students meeting or exceeding the faculty criteria.
Table V.1 Faculty and Student Participation in Information Literacy Intervention Efforts
Semesters | Faculty Participation (N) | Student Participation (N) |
---|---|---|
Spring 2013 to Spring 2015 | 990 | 18,700 |
Fall 2015 to Spring 2017 | 763 | 14,380 |
Source. City Tech Library, August 2017.
Similarly, in one of the most diverse colleges in the nation, where more than half of the student population report a first household language other than English, reading comprehension is an issue of paramount importance. The results of the 2013 assessment of Reading indicated that only 65% of the students met or exceeded the criteria (see Table 1). As a result of this assessment, Reading Effectively Across the Disciplines (READ) was developed and institutionalized. The results of the 2016 Reading assessment indicated that 76% of the students met or exceeded the criteria of proficiency, demonstrating that the interventions derived from outcomes assessment made an important difference.
Figure V.2 Reading: Percentage of Students who Met or Exceeded Faculty Criteria
After the second full-scale assessment cycle for writing was complete, a similar improvement was noted. In 2013, only 68% of the student met or exceeded the faculty criteria for Writing per the City Tech AAC&U modified rubric. Improvement strategies were implemented to address the shortcomings through the Writing Across the Curriculum and Living Lab Title V grant fellows. In 2016, the full-scale Writing assessment results indicated that 73% of the student met or exceeded the faculty target.
The General Education competencies are not program-specific; they underpin all of our degree programs. Broad-based participation in the assessment of General Education competencies is essential, and it has been successfully pursued. The table below (Table V.3) demonstrates this broad participation. As illustrated, the results of the General Education/Institutional Outcomes assessment are utilized for institutional planning and ensure the integration of institutional and program goals, student achievement of these goals, and the identification and implementation of improvement strategies to improve outcomes.
Lastly, the University, the AIR office and several academic departments conduct alumni and exit surveys to determine the extent to which graduates report themselves as prepared for their careers, enriched personal lives, and further education. The college monitors reports of preparedness for the workforce. In the most recent census, approximately 95% of City Tech alumni reported they were prepared for the workforce (2016). A minimum of 90% of alumni indicated that City Tech had prepared them to write effectively, communicate effectively, engage in quantitative reasoning, think critically, and work effectively in teams (2016 Alumni Survey Report, Exit Survey Report Spring 2017).
Table V.3 Number of Students Assessed for General Education at City Tech from 2013 to 2016.
Gen Ed Competency (students assessed) | 2013 | 2014 | 2015 | 2016 | Grand Total |
---|---|---|---|---|---|
Civic Engagement | 85 | 85 | |||
Creative Thinking | 75 | 75 | |||
Critical Thinking | 69 | 69 | 425 | 563 | |
Ethical Reasoning | 166 | 166 | |||
Foundations and Skills for Life Long Learning | 78 | 139 | 217 | ||
Information Literacy | 105 | 95 | 356 | 0 | 556 |
Inquiry & Analysis | 220 | 220 | |||
Integrative Learning | 99 | 99 | |||
Intercultural Knowledge | 211 | 211 | |||
Oral Communication | 158 | 749 | 0 | 907 | |
Quantitative Literacy | 34 | 166 | 200 | ||
Reading | 149 | 177 | 201 | 527 | |
Teamwork | 143 | 143 | |||
Writing | 660 | 356 | 317 | 1333 | |
Grand Total | 1175 | 350 | 1530 | 2247 | 5302 |
Note. Assessment data are collected on a 3-year cycle. Some departments collect more frequently for their program level assessment purposes. (Gen Ed Participation Summary 2013 to 2016)
c. support and sustain assessment of student achievement and communicate the results of this assessment to stakeholders;
AIR’s website (City Tech AIR Home Page) hosts support materials for assessment, past assessment data and results, and the roster of Assessment Co-Chairs and Liaisons for each of the schools (Assessment Liaison Roster 2009 to present). Appointed by the provost, the Assessment Co-Chairs are responsible for assisting each department’s Assessment Liaison with department/program assessment activities, and for communicating the results of assessment to the college community. To sustain systematic assessment of student achievement, City Tech convened a faculty task force to evaluate software available to assist with regular assessment responsibilities, leading to the adoption of TK20®. The college-wide Assessment Steering Committee meets annually in August to discuss objectives for the coming year. There is a three-year cycle for program and General Education assessment. School assessment committees meet twice per semester and once per semester for a college-wide meeting (Assessment Dates 2016 to 2018). Department-level discussions occur on a regular basis, led by either the chair or the Assessment Liaison. Finally, assessment activities and results are also reported to the student body through the Student Government Association (SGA) in a joint presentation by AIR and the Office of Student Life and Development (Survey Highlights SGA April 2017, SLD General Brochure). Results are also shared via social media.
Assessment activity results are communicated to internal stakeholders in multiple ways. First, they are conveyed through department-level discussions, often shared with professional advisory boards. These boards are convened to gain valuable feedback for program improvement as well to invite employer perspectives on desired alumni competencies. Data received from external constituencies is highly valued and its solicitation is encouraged by the professional accreditation organizations. Second, this information is reported to the provost and the deans of each school on a yearly basis. Third, General Education assessment activities conducted at the college level are communicated to senior administrators, faculty, and staff during college-wide General Education Committee meetings (Gen Ed Assessment Activities). The committee members are responsible for reporting information back to their departments.
Criterion 3. Consideration and use of assessment results for the improvement of educational effectiveness. Consistent with the institution’s mission, such uses include some combination of the following:
a. assisting students in improving their learning;
Assessment strengthens student learning across the curriculum from developmental education through the majors. For example, developmental courses have benefited from a consistent use of assessment results to identify strengths and weaknesses. Currently, 53% of City Tech students enter the college with one or more areas of developmental need (Reading, Writing, and/or Mathematics). For reading, assessment results were used to identify a software package, purchased using Carl D. Perkins funds, that improved student skills substantially. In addition, READ program was launched as a result of General Education Reading assessment results. Initially funded by CUNY Office of Academic Affairs and subsequently by Perkins allocations, this successful initiative has been institutionalized.
During the 2015-2016 academic year, the AIR office conducted an item analysis on the CUNY Elementary Algebra Final Exam and examined performance and exit from remediation trends. The results of this Research Brief (April 2016) were used to inform City Tech’s effort to revise its remedial mathematics program, which was discussed in more detail in Standard IV. (CUNY Elementary Algebra Research Brief)
Assessment’s contribution to student learning in the majors through its essential role in program review is discussed in Standard III.
b. improving pedagogy and curriculum;
Assessment in Grant-Funded Projects to Support Student Learning. Reviewing the results of assessment of General Education outcomes has required the College to place a strong emphasis on the improvement of writing, reading, and quantitative reasoning. Several major grants have enabled City Tech to address these areas more effectively. Among them, US Department of Education (USED) Title V grant (2010-2015) of $3.1M enabled the college to re-envision General Education as a “Living Lab” and to develop a correlative culture of assessment for learning. A USED Minority Science and Engineering Improvement (MSEIP) grant (2015-2018) “supports development of digital tools and pedagogy strategies for teaching math.” These grants have enabled the college to build capacity in the use of high impact educational practices through faculty development activities coupled with classroom implementation. As each grant requires an assessment of outcomes and an evaluation of project impact, the college has a substantial portfolio of project evaluation reports that document both student outcomes and institutional impact (NSF Research Spending).
Living Lab Learning Library (L4). As a result of a five-year Title V grant awarded in 2011, the college institutionalized one of the major support programs developed, the Living Lab Learning Library (L4). Based upon General Education Learning Outcomes assessment, L4 is a faculty-generated curriculum and pedagogy resource established in Spring 2016 to assist faculty in improving weaknesses identified through Gen Ed assessment activities. These resources help faculty to explore best practices in addressing areas of observed weakness. The results of the Fall 2016 general education assessment results, presented in Table V.4, indicated support was needed with respect to Ethical Reasoning. Thus, L4 has placed an emphasis on the areas where students need support, including: a) ethical self-awareness, b) ethical issue recognition, c) application of ethical perspectives/concepts, and d) evaluation of different ethical perspectives/concepts (see Figure V.5).
Table V.4 2016 Full-Scale General Education Assessment Results
General Education Competency | Percentage of Students Who Met or Exceeded Faculty Criteria | Number of Students Assessed |
---|---|---|
Civic Engagement | 77% | 85 |
Critical Thinking | 70% | 200 |
Ethical Reasoning | 58% | 166 |
Oral Communication | 80% | 749 |
Teamwork | 78% | 186 |
Figure V.5 Ethical Reasoning: Percentage of Students who Met or Exceeded Faculty Criteria (Gen Ed Assessment Briefs Folder)
Perkins Funding. The Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV) provides federal postsecondary funding via the New York State Education Department (NYSED). City Tech receives approximately $1M annually to support career and technical education (CTE) for associate degree students. Funded by Perkins, the Emerging Scholars (ES) program was used as a model to pilot test new research-centered sections of several courses in which hands-on research experiences were featured as a high impact practice. The results from the pilot were encouraging and revealed a similar effect to the ES program itself, based upon the Student Learning Outcomes assessed (Brown, Cumming & Pasley, 2017). In the most recent year, 251 students participated, and scale-up to reach more students is anticipated.
c. reviewing and revising academic programs and support services;
All degree programs are required to complete a self-study, including an external review, within a cycle of no longer than seven years. In some cases, the renewal of professional accreditation by specialized bodies serves as this review. This self-study schedule is maintained by the associate provost to ensure compliance by all degree programs and departments and is reported to the University (for a description of the process, see Standard III). At the program level, a comprehensive listing of strategies for improvement is derived from assessment results (Program Improvement Summary). It is important to note the ways that the assessment of learning outcomes has led to changes in the structure, content, and pedagogy of City Tech’s programs. Below are three abbreviated examples chosen from several found in the document “Examples of Program Level Improvement Strategies.”
- Structure: Assessment data of baccalaureate student outcomes in Communication Design showed a general weakness in problem-solving, reflected in the requirement to “Create a portfolio representing conceptual and visual problem solving abilities.” In response, the department put forward a curriculum proposal to change the format of the course COMD 4801, so that students will spend significant one-on-one time with the instructor.
- Content: In conducting program-level assessment activities, chemistry faculty determined that students struggled with of one of the program outcomes, a sound understanding of the fundamental principles of chemistry, and identified that students struggled with understanding stoichiometric relationships. Faculty determined that the improvement strategy should be implemented in CHEM 1110 where the course coordinator will meet with all faculty to stress uniform coverage of the topic by teaching all sections using two different methods.
- Pedagogy: In the Fall 2015 semester, students fell below the target for showing hands-on skills in mechanical engineering technology systems, a basic skill necessary to successful program completion. Smaller size hands-on group projects were introduced in all feeder classes involving labs and CAD courses. The faculty also proposed to purchase more equipment and to allocate more space for the lab classes.
These examples also highlight the integration of academic assessment processes into institutional planning and improvement. The feedback loop ensures that assessment results are used at a granular level to improve the quality of education at City Tech and to keep academic programs in tune with emerging employment trends.
d. planning, conducting, and supporting a range of professional development activities;
Faculty Commons. The Faculty Commons: A Center for Teaching, Learning, Scholarship and Service was established following a recommendation of the 2008 decennial review as a comprehensive professional resource for faculty and staff. One of the center’s most important functions is to elevate the visibility of good teaching and promote the implementation of high impact pedagogical practices across the institution. Extensive professional development is offered to address faculty needs. The AIR office and the Office of Sponsored Programs are co-located in the Faculty Commons to make their services easier for faculty to access (Faculty Commons Pedagogy and Practice). Since 2009, 413 full-time and part-time faculty (unduplicated count) have participated in the AIR assessment-related workshops, leading to an increased understanding of the centrality of assessment to improving educational effectiveness. (2013 Faculty Survey Results, 2016 Faculty Survey Results)
e. planning and budgeting for the provision of academic programs and services
Tech Fee. Many academic programs’ technology needs are supported through the Tech Fee, and resource allocation decisions are informed by assessment results. Given the technological focus of City Tech programs, constant enhancement of equipment and software is necessary. The college’s Tech Fee (a substantial source of technology funding: $3.4M in the most recent year; $4.1M projected for AY 2017-2018) is used for technology to enhance student learning (Student Technology Fee Plan 2016-2017, Student Technology Fee Plan 2017-2018, CUNY Tech Fee Policy). Deans receive assessment reports from department chairs. Informed by these assessment reports, the deans have substantial responsibility for deciding what will be submitted for Tech Fee funding to improve courses and program outcomes.
CUNY Performance Management Process. This annual PMP report, discussed in detail in Standard I, tracks important indicators of success using various assessment measures. Results indicated that City Tech’s incoming transfer student population was underperforming during their first year in comparison to transfer cohorts at other CUNY institutions. This finding led the college to invest substantially in hiring new transfer advisors. Accurate advisement, fair evaluation of transfer credit, and a clear degree plan are essential underpinnings for student success in technical and professional programs that have limited allowance for elective coursework. (2011-12 PMP_Report, 2012-13 PMP_Report, 2013-14 PMP_Report, 2014-15 PMP_Report, 2015-16_PMP_Report).
f. informing appropriate constituents about the institution and its programs
The AIR office provides different types of assessment results in a format that can be accessed by multiple City Tech constituencies. The AIR website contains information about the City Tech Assessment Plan, Gen Ed assessment tools, designated Assessment Liaisons per school and departments, surveys, student performance data, and the CUNY Performance Management Process. The online dashboard is a comprehensive and customizable tool that allows users to obtain graphical reports about key indicators such as Enrollment Trends, Six-Year Graduation Rates, One-Year Retention Rates, and Graduation Trends. By these means the AIR office disseminates assessment results and related literature. (City Tech AIR Home Page)
g. improving key indicators of student success, such as retention, graduation, transfer, and placement rates
While program outcomes are correlated with improved retention and graduation rates, it is difficult to show a discrete impact of any one program on overall graduation and retention rates. However, the development of significant programs in core areas of academic difficulty that were identified by the assessment of student learning outcomes contributes to the slow but steady increase in associate degree graduation rates (Six-year Graduation Rates Trends). Positive results have been achieved. This comprehensive and integrated approach has proven successful.
The First Year Program. Retention and graduation have long been key concerns for City Tech and the College makes a concerted effort to address the issue. Several special programs are offered as part of the First Year Program, which includes the Coordinated Undergraduate Education (CUE) Initiative, a widely emulated model (CUE Report 2015-2016). These programs support a smooth transition from pre-freshman admission to successful entry into college programs.
The READ Program. Through this program, workshops for faculty were offered on strategies to improve student reading comprehension in gateway courses across the three schools (biology, computer engineering technology, and marketing) in a number of our larger academic programs. These workshops were offered to 70% of the instructors in these three key courses to improve the reading skills competency and ultimately the retention and graduation rate in their corresponding programs. These programs have overall shown marked improvement (Cumming and Miller, 2017). READ impact is discussed above.
ASAP. Based upon extensive research, CUNY designed ASAP to help motivated students earn an associate degree as quickly as possible, with a goal of graduating at least 50% of students within three years or less. Initiated at City Tech in Fall 2015, ASAP began by serving students in STEM disciplines and, given the strength of the model, has been expanded to selected non-STEM areas (ASAP – City Tech). It is discussed in detail in Standard IV. Efforts in this direction are promising. The ASAP program has the following retention rates:
- The Fall 2015 starting cohort had 233 students: 93.1% were retained into Spring 2016 and 83.3% into Fall 2016 (1 year retention), vs. 63% for associate and 75% for baccalaureate students in the general population, clearly a statistically significant difference;
- For the Spring 2016 starting cohort, 169 students were enrolled. The program retained 95.3% into Fall 2016 (1 semester retention). This was the highest retention rate across all CUNY ASAP campuses for the cohort;
- The Fall 2016 cohort enrolled 330 students, a 41.6% increase over the previous Fall cohort; and
- Graduation rates for the first cohort were: 21% in two years, with a projected 2.5-year rate of 47%.
h. implementing other processes and procedures designed to improve educational programs and services
Mathematics. The College has begun to focus intensively on increasing the success rate of students in foundational mathematics courses. A locally developed exam, a direct method of evaluating student learning, was the basis for Peer-Led Team Learning (PLTL) initiatives in the Department of Mathematics, the Department of Construction Management and Civil Engineering Technology (CMCE), and the Department of Chemistry. Item analysis was conducted to identify areas of weakness, and programming was planned for improvements.
It was hypothesized that improving areas of weakness would translate to higher course-level pass rates. In the Department of Mathematics, the pass and withdrawal rates of the mathematics sections with Peer-Led mentoring (MAT 1175, MAT 1275, MAT 1375, MAT 1475, and MAT 1575) have been compared with the institutional rates at the end of each academic year. The overall results of Peer-Led sections range from five to 20 percent higher pass rates and a one to ten percent lower withdrawal rate. These results have led the Department of Mathematics to add more course sections with an additional one-hour Peer-Led component (PLTL Documentation).
Peer-Led pedagogies have also made important contributions to student success in other areas. The Department of Construction Management and Civil Engineering Technology (CMCE) requires all freshmen to complete the Statics course (CMCE 1115) as a prerequisite to their design courses. Approximately 90 to 120 students enroll in Statics each semester. A student’s performance in Statics is predictive of success in the curriculum; without satisfactorily passing the course, it is unlikely the student will be retained in the major. Peer-Led workshops were implemented in Statics in Spring 2012 in an effort to increase student performance and decrease withdrawal rates. Prior to the implementation of Peer-Led workshops, the rate of students passing with a grade of C or better was about 50%. This percentage has increased steadily from 52% in Spring 2012 to 79.5% in Fall 2014 and the withdrawal rate has decreased from 33% in Spring 2012 to 16.5% in Fall 2014 (CT Grade Distribution Report Spring 2012, CT Grade Distribution Report Fall 2014). In other sections utilizing the PLTL method, similar improvements have been observed.
Criterion 4. If applicable, adequate and appropriate institutional review and approval of assessment services designed, delivered, or assessed by third-party providers;
Not applicable.
Criterion 5. Periodic assessment of the effectiveness of assessment processes utilized by the institution for the improvement of educational effectiveness.
Since City Tech’s last Self-Study, assessment efforts have been vastly expanded, with extensive investment of time and energy of faculty and staff. As reported in the last two faculty survey administrations, 2013 and 2016 (2013 Faculty Survey Results, 2016 Faculty Survey Results), faculty understand the importance of using assessment to help them build a better learning experience. City Tech has activities and measures in place for assessing the effectiveness of the assessment processes (City Tech AIR Assessment, Assessing Assessment 01, Assessing Assessment 02, Assessing Assessment 03, City Tech Internal Evaluation of Assessment 2015_16, NILOA Feedback), including instruments for assessing the assessment of students, faculty, staff, administration, and alumni. They also include assessing our communication strategies with these diverse constituencies.
A recent faculty survey indicated that the majority of full-time and part-time faculty understood the importance of the assessment process. Specifically, faculty indicated that their departments have a systematic approach to assessing student learning, and use the results to make improvements (median=4 on 5-point scale) (2016 Faculty Survey Results). Faculty indicated that they have respect for the assessment process, partially due to the psychometric properties that are considered in developing and administering the direct assessment instruments. For example, faculty are required to provide validity evidence for their assessment instruments, as well as applicable reliability evidence. When assessment practices are conducted in accordance with current best practices, there is a higher level of faculty buy-in for the process, particularly when faculty can see the results of their efforts via improvements in student learning (Cumming and Miller, 2017).
We also assess this process by examining several aspects of the assessment activities. Faculty began assessing the assessment process during Spring 2014, through a Critical Course evaluation activity across the three schools to consider how critical course assessment impacted the program level outcomes. (Critical Course Assessment Workshop Agenda-May 2014). During the 2015-2016 academic year, the college undertook an external review of the assessment system coordinated by the National Institute of Learning Outcomes Assessment. The internal evaluation was submitted to NILOA and feedback was received in the fall 2016 semester and was shared with the CTAC (steering committee), president, provost, school deans, and assessment committee members (Internal Evaluation of Assessment, NILOA Feedback). As a result, City Tech has made improvements to its assessment system to ensure:
- Accessible institution-level learning outcomes (General Education) are communicated to students, as well as to external stakeholders;
- Full and transparent alignment between program outcomes and institutional outcomes (General Education to Program Level outcomes);
- Assessment results are shared with the appropriate constituencies;
- The assessment protocol and results are available on either the TK20 dashboard or the shared drive;
- Communications via the AIR website as well as social media outlets are coordinated with the director of communications, to disseminate results to all constituents;
- Chairs in each school are required to report to their respective deans on assessment; and
- Assessment results are used in institutional planning and budgeting decisions.
The overall findings of the external evaluation indicated that the assessment activities resulted in course and program-level improvements. However, the external reviewers recommended that greater transparency be achieved among different college units and that connections between them be strengthened to ensure that students benefit from an integrated assessment system. As a result, the AIR Office has recommended expansion of City Tech Assessment Committee membership to include representation from Offices of the Registrar, Enrollment Management, Student Affairs, Academic Advising and Financial Aid.
During the past two years, City Tech has seen the benefit of integration of non-classroom staff into the assessment process. Student Life and Development Offices are actively participating in assessment activities, and AIR is actively involved in Student Life and Development activities.
These activities include:
- Reviewing assessment data from the AIR website to ensure that learning outcomes for the co-curricular program support the acquisition of pre-professional (career) skills, and all institutional efforts to support student achievement;
- Facilitating methods of disseminating assessment results to the student body;
- Administering a student survey to evaluate the new student orientation programs;
- Facilitating communication among student groups, faculty and administration; and
- Planning student life activities to reinforce general education outcomes.
Much has changed in the college’s approach to assessment since the last Self-Study. A robust structure for faculty involvement has been created. Faculty liaisons are in place in every department. Assessment faculty co-chairs have been appointed in each school. Discussions of assessment results are held in every department. Workshops and conferences are held, to involve the broadest possible segment of the college’s constituents. The college has emerged as a leader within the CUNY system with respect to academic assessment and a strong culture of utilizing assessment data to improve teaching and learning has emerged. However, the assessment efforts outside of Academic Affairs is still in its nascent stage with plans to institutionalize the academic assessment model and expand to other service areas. In sum, the college fully meets each of the criteria in Standard V. Assessment data has influenced the pursuit of initiatives and grant funding to improve educational effectiveness, and the results have produced positive results, reflected in enhanced student performance in key areas. There is much still to do in improving retention and graduation rates, but much on the horizon that is promising, and the assessment data and processes are in place to support the setting of college priorities in pursuing the most promising programs and interventions.
During the past ten years, we have seen an increase in the graduation rates of bachelor’s degree programs grow from 9% and peak at 30%—and we are still striving to increase this number as we transition from an institution that granted primarily two-year degrees to an institution that has granted more bachelor’s degrees than associate degrees since 2013-14 (Six-year Graduation Rates Trends, Graduation Trends at NYCCT). With increased completion rates at both the associate and bachelor’s degree levels, our students have gained the education and skills to obtain gainful employment that has made a significant economic impact on their lives. The most recent studies by PayScale, a research company that studies compensation, showed City Tech in a positive light. Their 2015 study of 597 colleges showed City Tech associate degree graduates with the highest mid-career salaries in the group, while their 2016 study placed us third among their cohort in overall compensation (PayScale Press Release). Recently, a well-publicized study by a group of Stanford University economists led by Raj Chetty, showed City Tech fifth among almost four hundred public colleges and universities in the social mobility of its students, measured by the number who are able to advance at least two income quintiles above their starting point. (America’s Great Working-Class Colleges NYT)
Conclusion and Recommendation
City Tech is committed to improving critical measures of student success in one-year retention rates and six-year graduation rates. Assessment is integral to improving these indicators. Sufficient data and processes are in place to support institutional prioritization of the most promising interventions and programs. Newer programs such as Early College partnerships with City Poly and P-Tech high schools and ASAP, both designed to support students’ timely degree completion, provide models that can be scaled to move the dial on retention and graduation. Further, the impact of assessment is currently being broadened, as part-time faculty and non-instructional staff engage in assessment activities. Looking ahead, bringing assessment within Academic Affairs, Enrollment, and Student Affairs into closer coordination has the potential to provide the institution with a more holistic understanding of students’ educational experiences and the complex array of factors that contributes to student success.
Recommendation 2:
Build on the academic continuous improvement model to strengthen overall institutional assessment and effectiveness
(Standard V)
Expand practices instituted to assess student learning outcomes to engage all units across the college in a continuous assessment process in which data are gathered and analyzed to guide institutional directions and improvement:
- Expand the City Tech Assessment Committee membership to include representation from student support units, business services, and administration;
- Expand assessment of learning outcomes to include student learning beyond the classroom.
- Undertake targeted assessment of the effectiveness of intra-institutional and external communications in support of the communication-related goals defined in Recommendations 1, 3, 4 and 5.
There may be a delay of no more than one business day before the appearance of your comment due to moderation, in compliance with the Open Lab terms of use.
Questions about the report or review process? Contact Kim Cardascia at KCardascia@citytech.cuny.edu
Questions about the OpenLab? Email the OpenLab team at openlab@citytech.cuny.edu.