Romina Plozza, Acting Director – Outcomes Assessment, Shoreline Community College
Ann Garnsey-Harter, Associate Vice President – Planning, Institutional Effectiveness, & Project Management, Shoreline Community College
Jack Kahn, President, Shoreline Community College
Over the last couple of years, Shoreline Community College implemented a systematic and ongoing process of learning outcomes assessment to inform planning for student achievement and resource allocation. This article explores how the College went from a sanction of warning to being back in compliance thanks to a significant change in culture.
Learning Outcomes has been a source of concern for the college for over ten years. The College faced several challenges in the last few years, including four accreditation recommendations; two of which were out of compliance, and a sanction of warning, received in February 2023. The COVID-19 pandemic made it harder to function as faculty, staff, and administrators had to pivot to fully remote operations, and in March 2022, the College experienced a ransomware incident and lost data, reports, and dashboards.
These challenges and changes contributed to the College’s lack of systematic data-informed processes. However, through sustained, targeted efforts, Shoreline took those challenges and turned them into opportunities. After receiving the NWCCU sanction of warning, the College was able to create new structures, improve and leverage the methodology already in place, and focus on systematizing and scaling up the effort.
In only a few years, Shoreline was able to move away from a culture of “why do it” (why do assessment, continuous improvement, etc.) to one of “let’s do it” and create a sustainable and meaningful system of learning outcomes assessment. Below is a list of what was accomplished within the last three years.
- Revised institutional outcomes, now called Shoreline Student Learning Outcomes (SSLOs).
- Developed Transfer Program Outcomes (TPOs) for general transfer degrees.
- Revised many Program and Course Learning Outcomes.
- Implemented department and program assessment plans (every department and every program assess learning outcomes every year).
- Implemented a 3-year cycle assessment plan for institutional learning outcomes (two outcomes assessed every year).
- Integrated learning outcomes assessment into institutional planning. Learning outcomes is now a section of the annual Program Reviews, which all areas of the College fill out; this ensures a direct connection to resource allocation and budget development each year.
- Leveraged learning outcomes assessment data to improve teaching and learning to address equity gaps.
Shoreline’s Mid-Cycle Evaluation and visit was on April 18-19, 2024. The NWCCU peer evaluation team described witnessing a changed institution since the prior Ad Hoc visit in fall of 2022. A culture shift was pointed out several times during the visit as well as in the evaluation report:
“It is evident from numerous meetings and interviews there is a rebuilding of trust throughout the College, the mission of the College is at the forefront of the direction for the College, there appears to be a healthy culture in place, and a leadership team willing to act on the many initiatives outlined in the Mid-Cycle Self-Evaluation Report.”
“The structure, process, and support provided has resulted in a shift in culture around learning outcomes assessment on campus and increased collaboration to improve student learning and the student experience at Shoreline.”
How did Shoreline, in just a few years, go from assessment and data-informed planning happening only in pockets to those activities being woven into the fabric of the College?
Below is a list of tips that we hope will be useful to those of you who are or will soon be embarking on a similar journey:
- Ensure support from upper leadership.
Shortly after arriving to Shoreline in July 2022, President Kahn created a new unit called Planning and Institutional Effectiveness (PIE), reporting directly to him. This unit comprises all areas dedicated to assessment and continuous improvement including research, planning, and learning outcomes leadership. The change in name was intentional with a specific tie to accreditation requirements and the recognition of an ongoing need for the planning element for continuous improvement. A new leader of this unit (Associate VP-Planning, Institutional Effectiveness, and Project Management) was hired with that emphasis in mind and invited to apply to the NWCCU fellowship for further support and training.
Accreditation also became integrated into institutional practices. Accreditation updates are now at every board meeting and every meeting of the Executive Team.
In addition to the formation of PIE, Shoreline established the Accreditation committee to review accreditation standards and recommendations and help the campus successfully prepare for our Mid-Cycle Evaluation. The intention of establishing the committee was to be more inclusive and more accountable across the college to the accreditation work. Previous attempts had placed a disproportionate amount of work on one person or a handful of people which made establishing a new culture more challenging.
The Vice President of Instruction served as sponsor to various learning outcomes assessment initiatives. Additionally, he helped the Learning Outcomes Assessment Team advocate for time to be dedicated to performing assessment and sharing assessment results during Opening Week (the week before Fall quarter begins when all faculty return to campus for professional development) and Day of Learning (a dedicated professional development day in April).
The academic Deans of each division joined the Learning Outcomes Assessment Committee (LOAC). This was instrumental to the successful implementation and scaling of the learning outcomes assessment efforts. The Deans also prioritized working with faculty and made time at division and department meetings for hands-on learning outcomes activities and training. The impact of this practice is best explained in this quote by one of our Art professors:
“Doing assessment work as a group is a big part of changing the culture. The omnipresence of the work across campus and that it is an agenda item in every meeting one way or another, is a huge part of transforming the culture.”
Finally, Shoreline Student Learning Outcomes (SSLOs) data have been incorporated into the College’s new Equity-Centered Strategic Plan as one of the key indicators of student achievement. All programs review SSLO data and make plans to improve student learning and close equity gaps within their annual Program Review.
- Provide tools and training for faculty.
In 2022, Shoreline hired an Acting Director of Outcomes assessment, a position that has recently become permanent. In addition to this role, the College invested in a Learning Outcomes Faculty Liaison (100 hours a quarter for 3 quarters), and a part-time Research Analyst. Shoreline instructors consistently reflected that having a point of contact for learning outcomes assessment was one of the most important factors that contributed to such a significant culture shift:
“It was so helpful to have a person we knew we could go to whose job is just this instead of this being an added responsibility to an already impossible job description. It made all the difference to have that one-on-one support.”
“Having a dedicated person to this effort working across divisions and with higher administration helped ensuring that clear and manageable processes were built so people know what they are doing and why and can actually do it in addition to all their other responsibilities.”
The Learning Outcomes Team and the Learning Outcomes Assessment Committee (LOAC) engaged in thoughtful planning (department and program assessment plans, clear timelines, etc.) and used various communication methods including messaging via Deans, email/Canvas messages, division meeting communications, and a newsletter shared by both the Faculty Senate Council and Deans to ensure faculty were aware of all the different learning outcomes assessment activities.
Training was provided to faculty around learning outcomes assessment, writing measurable learning outcomes, and equity and learning outcomes. Training sessions happened in the form of a quarter-long cross-discipline faculty institute, team meetings with Shoreline Student Learning Outcomes and Transfer Program Outcomes work teams, professional development sessions, and one-on-one meetings.
In addition to training, the LOAC refined and created several resources for faculty including a website with all reports and important information, a monthly newsletter, step-by-step guides to revising and assessing learning outcomes, toolkits for each institutional outcome that explain what the outcomes mean and how to assess them, how to read dashboards resources, etc.
- Collaborate across divisions, departments, and offices.
As Suski, L. (2018) stated: “A key element for building commitment to action is involvement. This type of ownership through involvement reduces the kind of buy-in and support problems that arise from top-down approaches.” The Learning Outcomes Assessment team supported the faculty by providing guidance and structure, but faculty truly stepped up, were engaged, and helped the learning outcomes team shape our processes to help build what we have today.
This collaboration between faculty and administrators put everyone on the same side working toward a common goal: supporting our students. Below are a few examples of this successful collaboration.
- Learning outcomes assessment work was integrated into established meetings throughout the year (Executive Team meetings, dean and director meetings, faculty program coordinators meetings, and department and division meetings).
- Faculty from different divisions and departments worked jointly on curriculum matrices for their professional technical degrees and transfer degrees.
- Teams of faculty and admin across divisions worked on revising the institutional outcomes and creating transfer program outcomes for our general transfer degrees.
- At the College’s Day of Learning, in April, faculty worked on Transfer and Shoreline Student Learning Outcomes assessment and at the beginning of Fall quarter, during Opening Week, assessment results will be presented and discussed with the campus community.
- Faculty Senate Council, Curriculum Committee, and the Learning Outcomes Assessment Committee shared information and provided feedback on assessment plans and procedures.
- Seek and value outside perspectives.
After receiving the warning from the NWCCU, understandably, the College community felt unsure about the progress thus far and the ability to communicate it with peer evaluators. Seeking an outside perspective on the work done and to be done was extremely valuable.
Shoreline hosted a mock accreditation visit conducted with three external mock peer evaluators, and our NWCCU liaison kindly offered to read a draft of our report. Both experiences taught the College a great deal about how to better “tell Shoreline Community College’s story.”
Shortly after being hired, the Acting Director of Outcomes Assessment researched professionals in similar positions at other colleges in Washington state and formed the Learning Outcomes Leadership Team. The group meets quarterly and shares experiences, tips, and resources. This group provided invaluable feedback on our processes and continues to be a wonderful think tank.
The Associate VP-Planning, Institutional Effectiveness, and Project Management and the Acting Director of Outcomes Assessment also joined the 2023-2024 NWCCU Mission Fulfillment Fellowship and made “getting off warning” Shoreline Community College’s project. The insight gathered at site visits and provided by other fellows helped refine Shoreline’s learning outcomes processes and tools.
- Assess, seek feedback, and make improvements.
Feedback was especially important when revising the institutional learning outcomes, a process which lasted almost three years. Feedback provided by faculty, staff, admin, students, and advisory committees allowed groups of subject-matter-experts to develop assessable outcomes and an equitable and transparent assessment process.
Initial Course and Program outcomes assessment reports were modeled on examples from other colleges. Both templates were rewritten based on first year’s submissions, feedback from the LOAC and the WA Learning Outcomes Leadership team. The templates are now much more streamlined and focus on the data collection aspect as the discussion of the data and planning is now integrated into Program Reviews.
One of the biggest concerns for faculty engaged in this work was that their reports would “fall into a black hole.” It was that feedback that first sparked the idea to add the deans to the LOAC. Having the deans read and comment on faculty’s reports and being able to advocate for resources to implement their plans proved that the college was all in on this.
Scaling this type of operation took years of building trust, and trust was built by continuously asking for feedback and making changes. It was built by following through.
- Center equity.
In A New Decade for Assessment, Montenegro, E. & Jankowski, N. (2020) note the importance of involving students in assessment work, disaggregating data and using that data to make changes, making context-specific approaches, as well as embedding equity in all things assessment. Here are examples of how Shoreline enacted those four items:
1) Meaningful student involvement
- Students helped define the college-wide categories and review the language of the outcomes.
- The Learning Outcomes Assessment Committee encouraged faculty to be transparent about learning outcomes and co-construct rubrics with students when possible.
2) Data disaggregation
- Learning outcomes assessment data dashboards will be implemented starting fall quarter 2024 to allow for disaggregation of data at the course, program, and college level. Data dashboards inform plans to support all students while attempting to close equity gaps.
3) Context-specific approaches and responses
- The Learning Outcomes Assessment team worked with professional technical programs that have specialized accrediting/certifying bodies to understand what that process looks like and how it aligns with what we were setting up college wide.
- Both the Acting Director and the Faculty Liaison worked with individual instructors to make assessment a meaningful process for them and their programs.
4) Equity embedded in all things assessment
- Training was provided to faculty to write learning outcomes with an equity lens.
- Subject-matter-experts developed versions of Shoreline Student Learning Outcomes keeping in considerations different academic paths.
- Learning outcomes at all levels were re-written with an equity lens.
At Shoreline Community College, we are committed to student success and continuous improvement using assessment data. We are very proud of the progress made to address NWCCU’s concerns regarding student learning outcomes, and we are excited for the positive impact that a focus on learning outcomes assessment will have on our students.
References
Montenegro, E. & Jankowski, N. (2020, January) A New Decade for Assessment: Embedding Equity into Assessment Praxis. (Occasional Paper No. 42). National Institute for Learning Outcomes Assessment (NILOA).
Suskie, L. (2018). Assessing Student Learning: A Common Sense Guide (3rd ed.) San Francisco, CA: Jossey-Bass
