Blog by Ana Valencia, postdoctoral educational design researcher for 4TU.CEE and TU/e innovation Space a.m.valencia.cardona@tue.nl
It is Friday 9:00 am. You meet your final bachelor project team, composed of two Biomedical students, a Psychology and Technology student, and a Mechanical Engineering student. Together, you work to find a solution to a challenge you have deemed relevant: How to remove orthopedic casts in a friendly way for child patients.
TU/e innovation Space building
The challenge was brought forward by a local hospital (i.e., the challenge owner), which believes students can bring in a fresh perspective, and help find a feasible solution to this issue. You and your teammates work closely together with the challenge owner to find a relevant direction to the challenge, and to define a concrete problem definition. In this process, you are supported by TU/e innovation Space and its staff. TU/e innovation Space provides a ‘hub’ and resources where you can work in this interdisciplinary project (e.g., prototyping rooms, working areas, etc.). It also facilitates weekly coaching sessions with innovation Space Coaches on the interdisciplinary aspects of the projects, such as team communication, stakeholder management and integration of ideas. Furthermore, each team member is supported by a supervisor from your respective faculties, who guides and supports the development of the project from the different disciplinary perspectives.
Challenge Based Learning & the Innovation Space Bachelor End Project (ISBEP)
The above scenario illustrates Challenge Based Learning (CBL). CBL is an innovative educational concept with ever-growing relevance in engineering education. In CBL, students collaborate to develop solutions to open-ended challenges of societal relevance. CBL is considered a rich learning environment, where engineering students can develop their knowledge, broaden their (professional) skills by engaging in interdisciplinary team-work and in real-life multi-stakeholder situations, and by designing solutions to complex problems.
CBL is at the center of the vision in education of Eindhoven University of Technology (TU/e) for 2030. TU/e is engaged in several educational experiments to pilot this educational concept and evaluate its value for students’ learning. One such experiment is the Innovation Space Bachelor End Project (ISBEP; described in the above scenario), which is a final project for students completing their bachelor, and available to students from all programmes at the university. ISBEP is an alternative to the (individual) traditional Bachelor End Project (BEP) carried out at the respective departments. ISBEP is defined as inter-programme because, while students work in interdisciplinary challenge-based projects, students are assessed individually, and following the rules and regulations of their own departments.
So far, ISBEP has been successful in providing a valuable alternative educational experience to students. What we have observed is that students are highly motivated to take part in a project of these characteristics. They are triggered by the interdisciplinary aspect of it, their contribution to society, the opportunity to make a difference, and to work closely together with challenge owners.
The Challenge of Inter-programme CBL Assessment
Despite the positive reactions towards ISBEP, there are continued efforts and opportunities to enhance this educational concept and related practices. One important focus of the continued improvement efforts by TU/e innovation Space is the assessment practices within ISBEP. Reports from practice have suggested some challenges in this area.
Currently, assessment in ISBEP consists of assessment for learning (or formative assessment) and assessment of learning (or summative assessment).
Assessment for learning takes place in different ways: Through the interaction with innovation space coaches during the weekly coaching sessions; through self-reflection questions on the achievement of learning goals (which students answer thought a leaning management system); through plenary progress meetings with all stakeholders involved on the progress and direction of the project; through feedback sessions with disciplinary supervisors, and through the interactions with challenge owners.
Assessment of learning, as previously mentioned, takes place individually at each of the departments. This form of assessment is led by the disciplinary supervisor (and depending on the department, sometimes by secondary examiners or examination committees). Assessment is thus conducted following the learning outcomes for the bachelor college as guidelines, but adapted for each programme by the different departments, and based on the learning outcomes for the traditional BEP. Hence, as the assessment follows the different departmental rules and regulations, students within the same ISBEP team may have different assessment criteria, milestones and deliverables.
The above poses different challenges for assessment. In terms of formative assessment, selected reported challenges include:
- Achieving agreement on the learning goals of students, given the large amount of stakeholders, with potentially conflicting interests;
- Clearly defining roles, expectations and responsibilities for different stakeholders; for example, in the differences between innovation Space coaches, supervisors, and challenge owners, and the type of feedback/input students can expect of them;
- Keeping track of the agreements and communication by students with different stakeholders in relation to the projects, as the learning occurs with different environments (i.e., TU/e innovation Space, the different departments, and challenge owners’ grounds).
In terms of summative assessment, some reported challenges include:
- Reaching more transparency on learning outcomes, assessment criteria and assessment procedures. Students join ISBEP with the expectation of different assessment procedures to those of regular BEPs.
- In the same fashion, reaching a unified set of learning goals and assessment criteria across programmes, that is well aligned with CBL;
- Formalizing the input of other stakeholders involved in the learning of students, as part of assessment of learning; and
- Designing tools for assessment that will facilitate the integration of these different views into the assessment.
Overall, we have observed that a better alignment between learning goals, learning activities and assessment practices for ISBEP ought to be reached. The challenge here lies in engaging the different departments (i.e., programmes) in this discussion, and reaching agreements on how ISBEP should be implemented and assessed.
Towards a solution
The above challenge has been addressed by means of a research project[i], supported by 4TU.CEE (Center For Engineering Education), in collaboration with TU/e innovation Space. The project started in May 2019 and follows an educational design research approach (See Figure 2): An iterative approach of problem/context analysis, design/prototype, evaluation, analysis, redesign, and so on.
Figure 1. Iterations of systematic design cycles (Plomp, 2013) and phases in the research project
The project has been divided in three phases. Phase 1, which has just been concluded, has focused on the exploration of the context and analysis of the issues related to CBL assessment practices in the ISBEP programme.
The research questions guiding phase 1 were the following:
How can Inter-Program Challenge-Based Learning (CBL) at TU/e innovation Space (i.e., ISBEP) be characterised? And what aspects/characteristics of ISBEP influence assessment practices?
These questions have been researched using a qualitative research approach (exploratory study of ISBEP) over a period of five months. To this end, observations and interviews with the different stakeholders of ISBEP were conducted. The challenges listed above are partly based on this research activity. A more detailed report of this phase is currently under development, and the results of the study will feed into the redesign of the ISBEP programme.
Phase 2, focuses on the redesign of the (formative and summative) assessment practices of ISBEP. The following research question guide this phase:
What are the CBL associated design principles, in particular those related to assessment practices (formative and summative) that align well with CBL (i.e., learning goals and activities in ISBEP)?
To this end, co-creation sessions with educators, educational experts, students, and other university staff will be conducted. The goal of these sessions is, based in the learnings from Phase 1, to redesign the learning goals, learning activities, and (formative and summative) assessment practices of ISBEP. These sessions are scheduled to take place in March, April and May, 2020. The goal is to increase the alignment between these elements, as well as the university’s vision on CBL. A second goal of Phase 2 is to test improvements/interventions on formative assessment practices. Results for Phase 2 are expected in the second half of 2020.
Finally, the outcomes of Phase 2 will lead to the development of a detailed (new) assessment plan for ISBEP, to be evaluated and implemented in Phase 3. The research questions guiding this last phase are:
How can (new) assessment practices be implemented, and evaluated? How can the insights gained from Phase 2 be used for the further development of assessment in the ISBEP program, also in terms of scaling up?
The end goal of Phase 3 is thus to submit the new assessment for revision by the Joint Program Committee; the university’s entity in charge of reviewing and approving new assessment plans. Furthermore, we aim at fully implementing the new (formative and summative) assessment practices in ISBEP. The results of Phase 3 will lead to the identification of new aspects to be improved, and lead to a new design iteration. Phase 3 will also lead to the identification of insights than can guide our efforts in scaling up ISBEP in the near future, and the design of new CBL modules/programs university-wide. The results for Phase 3 are expected in 2021.
References
Plomp, T. (2013). Educational Design Research: An Introduction. In T. Plomp & N. Nieveen (Eds.), Educational Design Research. Part A: An Introduction (pp. 1–206). Netherlands Institute for Curriculum Development (SLO). https://slo.nl/publish/pages/2904/educational-design-research-part-a.pdf
Ana Valencia
[i] Members of the research team: Dr. ir. Ana Valencia (Postdoctoral Educational Design Researcher TU/e innovation Space, 4TU.CEE); Prof.dr.ir.arch. Isabelle Reymen (Scientific Director TU/e innoSpace); Prof.dr.Birgit Pepin (4TU.CEE, Eindhoven School of Education); Dr.ir. Miguel Bruns (TU/e innovation Space). For questions, please contact the author at a.m.valencia.cardona@tue.nl
