Avatar

Michiel Straat

Postdoc in Machine Learning

Bielefeld University

Biography

I am a postdoctoral researcher in the Machine Learning group at Bielefeld University, where I lead a junior research group on robust life-long machine learning in the research network SAIL: SustAInable Life-cycle of Intelligent Socio-Technical Systems. In order to address important demands for AI systems such as transparency, human agency, safety and resource-efficiency, the interdisciplinary research network SAIL focuses on the full life-cycle of AI systems. Before this, I performed my PhD research in the Intelligent Systems group at the University of Groningen.

Interests

  • Machine Learning
  • Computational Intelligence
  • Statistics and Statistical Physics
  • Data Science

Education

  • Postdoc in Machine Learning

    Machine Learning group, Bielefeld University

  • PhD in Machine Learning

    Intelligent Systems group, University of Groningen, PhD thesis

  • MSc. in Computing Science

    University of Groningen

Featured Publications

October 2023 Off-line Learning Analysis for Soft Committee Machines with GELU Activation

Layered Neural Networks with GELU Activation, a Statistical Mechanics Analysis

In this work we show theoretically that using GELU activation in neural networks induces continuous phase transitions and we analyse the location of the phase transition. We furthermore introduce and analyse a combination of Erf and GELU activation.

PDF Project Poster DOI
June 2022 An Industry 4.0 example: real-time quality control for steel-based mass production using Machine Learning on non-invasive sensor data

An Industry 4.0 example: real-time quality control for steel-based mass production using Machine Learning on non-invasive sensor data

We present a typical Industry 4.0 case study: the real-time quality estimation of steel in a high-throughput production line using Machine Learning methods.

PDF Project
April 2021 Supervised Learning in the Presence of Concept Drift: A modelling framework

Supervised Learning in the Presence of Concept Drift: A modelling framework

A statistical physics based modelling framework is developed in which we study standard training algorithms (SGD, LVQ1) under concept drift and in addition compare ReLU vs sigmoidal activation.

PDF Project Slides DOI
November 2020 Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation

Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation

Our systematic comparison of networks with ReLU and sigmoidal units in model situations reveals surprising differences in their training and generalization behavior.

PDF Project DOI

Projects

SAIL

A multidisciplinary project focusing on transparent AI, human agency, safety and resource-efficiency.

Statistical Physics of Learning

Typical case analysis of machine learning scenarios

Reinforcement learning based control of dynamical systems

Employing AI techniques to model and control dynamical systems.

Machine Learning for health

Using Machine Learning methods to improve health care

Region of Smart Factories (RoSF), Industry 4.0

Focuses on the development of intelligent, connected and customized production processes

Recent Publications

Quickly discover relevant content by filtering publications.

Layered Neural Networks with GELU Activation, a Statistical Mechanics Analysis

In this work we show theoretically that using GELU activation in neural networks induces continuous phase transitions and we analyse …
PDF Project Poster DOI

Off-line Learning Analysis for Soft Committee Machines with GELU Activation

In this work we show that the GELU activation function in two-layer neural networks causes a continuous phase transition, independent …
PDF Project Poster

An Industry 4.0 example: real-time quality control for steel-based mass production using Machine Learning on non-invasive sensor data

We present a typical Industry 4.0 case study: the real-time quality estimation of steel in a high-throughput production line using …
PDF Project

Supervised Learning in the Presence of Concept Drift: A modelling framework

A statistical physics based modelling framework is developed in which we study standard training algorithms (SGD, LVQ1) under concept …
PDF Project Slides DOI

Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation

Our systematic comparison of networks with ReLU and sigmoidal units in model situations reveals surprising differences in their …
PDF Project DOI

Complex-valued embeddings of generic proximity data

We propose and evaluate an information-preserving complex-valued embedding for general non-psd proximity data.
PDF

On-line learning dynamics of ReLU neural networks using statistical physics techniques

We introduce exact macroscopic on-line learning dynamics of two-layer neural networks with ReLU units in the form of a system of …
PDF Project

Learning Vector Quantization and relevances in complex coefficient space

In this contribution, we consider the classification of time series and similar functional data which can be represented in complex …
PDF Project Project DOI

Statistical Mechanics of On-Line Learning Under Concept Drift

We introduce a modeling framework for the investigation of on-line machine learning processes in non-stationary environments. We …
PDF Project DOI

Prototypes and matrix relevance learning in complex Fourier space

In this contribution, we consider the classification of time-series and similar functional data which can be represented in complex …
PDF Project DOI
See all publications

Recent & Upcoming Talks

Koopman-based Modeling of Rayleigh-Bénard Convection

In this talk I will present our recent work on Koopman-based surrogate modeling of Rayleigh-Bénard Convection.
Jan 22, 2024 12:00 AM Applicaitons of Intelligent Systems (APPIS), Las Palmas de Gran Canaria, January 2024
Project Slides

Statistical Physics of Learning

In this talk I discuss the main principles behind the statistical physics of learning.
Jun 27, 2023 12:00 AM Group seminar Machine Learning, Bielefeld, June 2023
Project Slides

Modelling adversarial training

In this talk I discuss my recent work on analyzing adversarial training procedures that make machine learning models more robust.
Mar 24, 2023 12:00 AM Workshop on Intelligent Systems and Computational Intelligence (WISCI), Groningen, March 2023
Slides

An Industry 4.0 example: real-time quality control for steel-based mass production using Machine Learning on non-invasive sensor data

In this talk a typical Industry 4.0 setting is presented: the use of sensor measurements for real-time quality control and fault …
Jul 20, 2022 5:40 PM World Congress on Computational Intelligence (WCCI), International Joint Conference on Neural Networks (IJCNN) 2022
Slides

Feedback Alignment methods for training neural networks

In order to compute weight updates, backpropagation uses complete knowledge of the downstream weights in the network. In [1] it is …

Mar 3, 2021 4:00 PM Intelligent Systems group seminar, Groningen, March 2021
Slides

Dynamics of on-line learning in two-layer neural networks in the presence of concept drift

In this talk I will discuss our recent work on learning in non-stationary situations. A modelling scenario of neural networks learning …
Aug 12, 2020 7:00 PM Summer Workshop on Statistical Physics and Machine Learning, Les Houches, August 2020
Slides Video

Dynamics of on-line learning in two-layer neural networks in the presence of concept drift

Jul 2, 2020 11:30 AM The 12th Mittweida Workshop on Computational Intelligence (MiWoCI)
Abstract

Towards a statistical physics analysis of multilayer ReLU neural networks

Sep 12, 2019 11:05 AM The 11th Mittweida Workshop on Computational Intelligence (MiWoCI)
Slides Abstract

On-line learning dynamics of ReLU neural networks using statistical physics techniques

Apr 26, 2019 11:40 AM European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning - ESANN 2019
Slides

Online Learning Dynamics of ReLU Neural Networks

The rectifier activation function (Rectified Linear Unit: ReLU) has become popular in deep learning applications, mostly because the …
Mar 29, 2019 2:30 PM Second International Workshop on Intelligent Systems and Computational Intelligence - WISCI 2019
See all talks

Contact