A Case for Innovative Design Post-Occupancy Evaluation on the Impacts of School and Furniture Design on User Satisfaction and Student Engagement
BY Xuemei Zhu in conjunction with Diego Barrera
BACKGROUND
School facilities are particularly important due to their substantial scale as a whole and their significant impacts on school-aged children. School construction is often the largest sector of total nonresidential construction in any given year (Arsenault, 2015). In 2011, about 90% of the 55.5 million school-aged children attended 100,000 public schools in the U.S.
Physical environment in schools has shown significant impacts on user satisfaction and student learning (Barrett, Davies, Zhang, & Barrett, 2015; Barrett, Zhang, Moffat, & Kobbacy, 2013; Park & Choi, 2014). Recent school designs have shown a trend toward providing more flexible and adaptable environments to accommodate changing demographic and classroom needs, and to facilitate collaborative and engaged learning (Arsenault, 2015). But the actual impacts of such design interventions have rarely been studied. More empirical evidence in this area is needed to help further inform school design and improve learning outcomes.
In recent years, the Coppell Independent School District (ISD) in Texas has taken bold steps to improve the educational environments of its students. In the fall of 2014, the Coppell ISD opened the doors of the Richard J. Lee Elementary School—the first Net Zero elementary school in Texas, and one of the first in the country. This school is innovative in many ways beyond its sustainable features. It is free of traditional hallway spaces, and provides flexible and adaptable common spaces to facilitate collaborative learning and promote student engagement. Richard J. Lee Elementary is a compact, two-story learning environment that is visually open and makes excellent use of space. Instead of traditional classrooms, grades are organized into learning communities placed around a central collaboration space. The design was driven to support a non-traditional curriculum. Varied types of learning zones can accommodate each student’s learning preference, be it focused, active, or collaborative. The open design provides connectivity, transparency, and inclusivity while promoting informal supervision. The library transforms into an extension of the small learning communities. Even exterior spaces normally reserved for play or support become places for active learning.
In the summer of 2015, the Coppell ISD also introduced new and flexible school furniture to most of the other schools in the district, with the hope they would provide more flexible spaces for collaborative learning.
1. STUDY AIM, DESIGN, AND PROGRESS
1.1. Study Aim and Conceptual Framework
This newly opened school and the new furniture in existing schools provided unique and rare “natural experiment” opportunities for studying the impacts of innovative design. The researchers and designers at Stantec Architecture teamed up with the researcher at Texas A&M University to develop this post-occupancy evaluation project. It aims to examine the impacts of collaborative learning spaces and furniture on user satisfaction and student engagement in the learning process.
A conceptual framework (Figure 1) was developed based on the socio-ecological theory (McLeroy, Bibeau, Steckler, & Glanz, 1988) to guide this study. It considers student engagement and learning to be influenced by three domains of factors, including personal factors (e.g., socioeconomic status [SES], age, gender and ethnicity), social factors (e.g., school curriculum, teachers and peer influences), and built environment in school (including both school buildings and furniture). These three domains also interact with each other. Satisfaction with school design and furniture design act as mediating factors between the objective design characteristics and the outcome of student engagement.
1.2. Study Design
Study schools for this study include (1) School A—the Lakeside Elementary School, which had traditional school and furniture design as of May, 2015 and had its furniture replaced with new and flexible ones supportive of collaboration in the summer of 2015; and (2) School B—the Richard J. Lee Elementary school, which is the newly designed school with innovative features for both the school building and furniture, and was opened in the fall of 2014. The enrollment size was 500 for School A and 539 for School B in the 2014-2015 academic year.
Figure 2 illustrates the study design with three components. Study 1 is a cross-sectional study comparing school A (Lakeside Elementary with traditional school design) in Scenario A1 (with traditional furniture) with School B—Richard J. Lee Elementary (with new school design and new furniture). Study 2 is a pre-post interventional study comparing two scenarios in the same school A, before (Scenario A1) and after (Scenario A2) traditional furniture is replaced by new ones in both classrooms and collaborative common spaces. Study 3 is comparing School A in Scenario A2 (traditional school with new furniture) with School B (with new school and furniture design).
This study expects that new school design and furniture will lead to significant improvements in student engagement. The specific hypotheses for 3 study components are listed below.
Study 1 Hypothesis:
Students and teachers in School B will report greater student engagement than their peers in School Scenario A1 (with traditional school design and traditional furniture).
Study 2 Hypothesis:
In School A, students and teachers will report greater student engagement in Scenario A2 (after the replacement of traditional furniture with new and innovative ones) than in Scenario A1 (before furniture changes).
Study 3 Hypothesis:
Students and teachers in School B will report greater student engagement than their peers in School Scenario A2 (with traditional school design and new furniture).
1.3. Data Collection and Current Progress
Data collection was mainly conducted through online surveys using Survey Monkey. As of January, 2016, Study 1 complement of this project was completed. In May of 2015, teachers and students in both study schools were invited to complete an online survey, which asked about their background information, satisfaction with built environments in school (indoor environmental quality, school buildings, and furniture) and students’ level of engagement and disaffection in the learning process. With the exception of questions about background information such as school, grade and gender, all other questions asked respondents about how much they agreed or disagreed with each statement, using a Likert scale ranging from “Strongly Disagree” to “Strongly Agree”. In May of 2016, the same survey will be conducted at both schools again, and at that time, School A would have been using the new furniture for one year.
Satisfaction with school design was captured through questions about indoor environmental quality and questions about the building and specifies areas. Questions for indoor environmental quality were adapted from “the Occupant Indoor Environmental Quality (IEQ) Survey” for school, which was developed by the Center for the Built Environment (CBE) at the University of California Berkeley. This is a survey that captures user satisfaction with four sub-domains, including acoustic performance, lighting, thermal comfort and views in a building.
Questions about satisfaction with the school building and specific areas, as well as school furniture were developed by the researchers at Stantec Architecture. They ask for users’ feedback on the building and furniture solutions in their facilities and how those solutions meet educational goals. The focus is on the role that school buildings and furniture play as a tool in the learning environment, and how they influence the comfort of students, teachers, and administrators.
Questions about the outcome variable—student engagement in learning—were adopted from a validated survey instrument—the “Engagement versus Disaffection Scale” (Skinner, Kindermann, & Furrer, 2009). The Skinner article provided multiple options for the Emotional Disaffection portion of the scale, for these purposes, the team selected a subset of what was offered. They capture information about four sub-domains of student engagement, including behavioral engagement, emotional engagement, behavioral disaffection and emotional disaffection.
2. DATA ANALYSIS AND RESULTS
2.1. Data Analysis
The survey responses were downloaded from Survey Monkey and cleaned and analyzed using the SPSS software. Among those survey questions using Likert Scale to measure environment and engagement, some were positive statements while others were negative statements. Therefore, the researchers reverse coded all ratings for negative statements so that the values from all environment and engagement questions are consistent in a way that higher values always indicate more positive evaluations.
Descriptive statistics was conducted to examine the mean, standard deviation and distribution of all variables. For all sub-domains of satisfaction and engagement measures, except for thermal comfort, one average rating was also calculated to reflect the overall evaluation for each sub-domain. For the sub-domain of thermal comfort, due to the reversed nature of two sub-groups of measures (see Table 2), two average ratings were calculated to reflect conditions of indoor temperature in relation to outdoor temperature. Independent sample t tests were conducted for each built environment and engagement variable to compare School B with School A1 in terms of users’ satisfaction with built environment in school and levels of engagement in learning.
2.2. Results from Student Surveys
A total of 350 responses were collected from two schools, among which 335 provided complete responses for questions about environment and engagement, and were considered valid. There is no significant difference between two schools in grade levels and gender composition of student respondents. But School A had more respondents (Table 1).
For indoor environmental quality, School B received significantly higher ratings for most items (Table 2). For example, in terms of acoustics, school B was rated 0.42 and 0.77 points higher for “Outdoor traffic noise distracts me” and “Noise from kids in outdoor playground/fields disturbs me” (both reverse coded), respectively, on a 5-point scale. Among four sub-domains of indoor environmental quality, the between-school difference was the greatest for the sub-domain of “views”, with specific items rated 0.85, 1.10, 0.76, and 0.79 points higher on a 5-point scale in School B. Figure 3 is a radar chart that illustrates all individual items that were rated significantly higher in School B than in School A1.
School B also received significantly better ratings for student satisfaction with school buildings in general and some specific areas (common work/collaboration spaces, library, and classroom), as well as all items about satisfaction with school furniture (Table 3). Figure 4 presented all individual items that were rated significantly higher in School B.
The outcome variables—levels of behavioral and emotional engagement and disaffection—were also compared between two schools (Table 4). School B outperformed School A1 again in 11 out of 20 individual items, including two items of behavioral engagement, three items of emotional engagement, two items of behavioral disaffection and four items of emotional disaffection. For the other nine items that did not show significant differences, eight items still had higher means in School B. Figure 5 presented all individual engagement and disaffection items that were rated significantly higher in School B.
In addition, for the average ratings of four sub-domains, three sub-domains (emotional engagement, behavioral disaffection and emotional disaffection) also had significantly higher ratings in School B (Table 4). For the other sub-domain of behavioral engagement, the average rating for School B was relatively higher at a marginally significant level (0.5 < p < 0.1).
Overall, the results from student surveys supported the hypothesis about greater satisfaction and better student engagement in the new school with new furniture (School B) than in the traditional school with traditional furniture (School A1). These revealed differences are promising, implying the potential of using innovative school and furniture design to improve student engagement and learning outcomes.
2.3. Results from Teacher Surveys
Survey data from teachers have a very limited sample size (34 for school A1 and 12 for School B), which made meaningful between-school comparisons at this stage (Study 1) impossible. For Study 2 and Study 3 of this project, it is still possible to utilize survey results from teachers for analysis.
3. WHAT IS NEXT
For Study 1, the completed analysis only includes t tests comparing ratings for two schools. For the next step, multivariate linear regressions will be conducted to predict student engagement using personal, social and built environmental factors. Results from these additional analysis will help us better understand the specific impacts of these multi-level factors on the outcome of student engagement. If possible, the researcher would also like to request data for socioeconomic information and/or individual performance from the Coppell ISD for a more comprehensive analysis. For dissemination of study findings, conference presentations and journal papers will be possible outlets, and are strongly encouraged.
For Study 2 and Study 3 of this project, additional data analysis plans will be developed after the data were collected in May, 2016.
REFERENCES
Arsenault, Peter J. (2015). School buildings in 2015: designing for students. Architectural record, 203(1), 141.
Barrett, Peter, Davies, Fay, Zhang, Yufan, & Barrett, Lucinda. (2015). The impact of classroom design on pupils’ learning: Final results of a holistic, multi-level analysis. Building & Environment, 89, 118-133. doi: 10.1016/j.buildenv.2015.02.013
Barrett, Peter, Zhang, Yufan, Moffat, Joanne, & Kobbacy, Khairy. (2013). A holistic, multi-level analysis identifying the impact of classroom design on pupils’ learning. Building and Environment, 59, 678-689. doi: 10.1016/j.buildenv.2012.09.016
McLeroy, K. R., Bibeau, D., Steckler, A., & Glanz, K. (1988). An ecological perspective on health promotion programs. Health Educ Q, 15(4), 351-377.
Park, Elisa, & Choi, Bo. (2014). Transformation of classroom spaces: traditional versus active learning classroom in colleges. Higher Education, 68(5), 749-771. doi: 10.1007/s10734-014-9742-0
Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A Motivational Perspective on Engagement and Disaffection Conceptualization and Assessment of Children’s Behavioral and Emotional Participation in Academic Activities in the Classroom. Educational and Psychological Measurement, 69(3), 493-525. doi: 10.1177/0013164408323233
——————————————–
About Dr. Xuemei Zhu
Dr. Zhu, associate professor, Department of Agriculture, Texas A&M University, is interested in environment-behavior research and its design application, with a special focus on healthy and walkable communities, healthcare facilities and learning environments. Her research is supported by agencies such as the National Institutes of Health (NIH), the Robert Wood Johnson Foundation (RWJF), and the American Institute of Architects (AIA) and has been widely published and presented.
About Diego Barrera
Diego Barrera is a design architect with more than a decade of experience in educational facility design. He specializes in creating innovative solutions using the latest technology, resources and trends affecting educational architecture.