Aarhus University Seal

Competition with feedback 

Short description

The students develop solutions to predict the categorisation of two large data sets using machine learning. The solutions are part of a class competition to find out who can come up with the most precise solution. This mimics real research strategies in machine learning with data analysis. 

Motivation

We wanted to make our assignment resemble a real machine learning competition. The advantage is that it encourages the students to think about how to assess performance in an impartial manner and they test real research strategies in the field. 

Learning objectives

After the course, the students will have the necessary theoretical and practical background to work with modelling and understanding different types of genomic data sets, with focus on predictability and classification. The students will develop detailed knowledge about current statistical learning methods, which are specially adapted to (large) genomic data sets.

Execution

THE PROCESS

Group work

  • The students must improve existing solutions or develop completely new solutions in a machine learning competition, where they have access to two large data sets. One dataset is about the disease ALS, and the students have to predict the expected disease progression (course of disease). The other dataset is a cancer dataset, where the students must classify tissue samples as 'Tumour' or 'Normal' for several different tissues. Both sets of data are split into two parts: 

    • Training set: Here, the students know the answer (disease progression for ALS patients or Tumour/Normal for some cancer samples). 

    • Test set: Here, the students do NOT know the answer, but must try to predict it through machine learning and estimate how good they think their solution is. 

Partial submission with feedback from the teache

  • Along the way, the students submit their answers and are evaluated on about half of their answers. We have a scoreboard so they can see how well they are doing. They get hints about which strategies have been used and have the opportunity to improve their models during the course.

Adapting their model

  • The students continue to work on their models during the course and submit their finished model.

Competition decision

  • Finally, their final submission is evaluated on the entire test set. We choose a winner, award diplomas and discuss the different solutions in plenum.

RESOURCES FOR STUDENTS

Literature: 

  • Gareth James et al. 2022: "An Introduction to Statistical Learning in R with Applications in R". 2nd edition. 

  • Selected research articles and book chapters.  

Reflections

Outcome

  • Our experience is that the opportunity to work in groups while having an element of competition in the exercises is highly motivating for many. 

  • At the same time, the design forces students to use key elements from machine learning. Students get unbiased estimates of test error using e.g. cross validation and learn how to compare models - both theoretically and practically. 

  • At the end of the course, the students also learn how to combine models – and the final assignments are very often combinations of models with different biases and variances. 

  • "Gamification" has a strong motivating effect on many people. Furthermore, students on a machine learning course are highly motivated if they have tried something identical to a real ML competition (kaggle.com). 

  • The advantage is also that we do assignments that we, as teaching staff, find fun/motivating. 

Challenges

  • Not everyone thinks competition is fun. Some students do quite poorly in the competition and never really get fully involved.

  • Finding data that is close to the research and also good/easy enough to use for teaching is often challenging. There are often challenges with 'real' data, making it frustrating for students to work with it. 


Basic information

Educator Palle Villesen, Thomas Bataillon, Asger Hobolth
Faculty and department NAT, Bioinformatics Research Centre (BiRC) 
Degree programme Bioinformatics 
Course/subject Statistical and Machine Learning in Bioinformatics
Level of study Master’s 
Teaching format Lecture, Classroom instruction 
Extent Course
How was the case conducted?

Onsite in-person teaching (including the use of Brightspace) 

Links or references to literature or resources

  • Gareth James et al. 2022: "An Introduction to Statistical Learning in R with Applications in R". 2nd edition. 

  • Selected research articles and book chapters.  

Contact