smn¹

Research

In my research, I seek to understand the principles that allow large-scale neural networks (including the human brain) to rapidly adapt and systematically generalize.

In some of my recent work, we identify conditions under which modular neural networks will compositionally generalize, find that simply scaling neural networks can lead to compositional generalization, and discover how transformers' multi-head attention can capture compositional structure in abstract reasoning tasks.

About

I am a postdoctoral researcher in the Department of Computer Science at Princeton University where I work with Brenden Lake at the intersection of machine learning and cognitive science.

I completed my Ph.D. in Computer Science at ETH Zurich in February 2025 where I was supervised by João Sacramento and Angelika Steger and worked on meta-learning and compositional generalization in neural networks. During the final year of my Ph.D. I spent six months in the Foundational Research Team at Google DeepMind researching efficient inference in large-scale mixture of experts architectures.

Prior to my doctoral studies, I completed my Master's thesis at the University of Cambridge with Maté Lengyel and received an M.Sc. in Neural Systems & Computation from ETH Zurich and UZH in 2020, after having simultaneously pursued and obtained B.Sc. degrees in both Electrical Engineering and Psychology from RWTH Aachen in 2017.

Publications

* – equal contributions

1. Scale leads to compositional generalization

   F Redhardt, Y Akram, S Schug
   arXiv 2025
   arxiv

2. Meta-learning & compositional generalization in neural networks

   S Schug
   Doctoral Thesis ETH Zurich
   thesis

3. Attention as a Hypernetwork

   S Schug, S Kobayashi, Y Akram, J Sacramento, R Pascanu
   ICLR 2025 (oral)
   paper ∙ code ∙ arxiv ∙ tweet

4. When can transformers compositionally generalize in-context?

   S Kobayashi*, S Schug*, Y Akram*, F Redhardt, J von Oswald, R Pascanu, G Lajoie, J Sacramento
   NGSM workshop at ICML 2024
   arxiv ∙ workshop

5. Discovering modular solutions that generalize compositionally

   S Schug*, S Kobayashi*, Y Akram, M Wołczyk, A Proca, J von Oswald, R Pascanu, J Sacramento, A Steger
   ICLR 2024
   paper ∙ arxiv ∙ code

6. Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysis

   A Meulemans*, S Schug*, S Kobayashi*, N Daw, G Wayne
   NeurIPS 2023 (spotlight)
   paper ∙ arxiv ∙ code ∙ tweet

7. Online learning of long-range dependencies

   N Zucchet*, R Meier*, S Schug*, A Mujika, J Sacramento
   NeurIPS 2023
   paper ∙ arxiv ∙ code

8. A complementary systems theory of meta-learning

   S Schug, N Zucchet, J von Oswald, J Sacramento
   Cosyne 2023
   poster

9. A contrastive rule for meta-learning

   N Zucchet*, S Schug*, J von Oswald*, D Zhao, J Sacramento
   NeurIPS 2022
   paper ∙ arxiv ∙ code ∙ tweet

10. Random initialisations performing above chance and how to find them

    F Benzing, S Schug, R Meier, J von Oswald, Y Akram, N Zucchet, L Aitchison, A Steger
    OPT2022 workshop at NeurIPS 2022
    paper ∙ arxiv ∙ code ∙ tweet

11. Presynaptic stochasticity improves energy efficiency and helps alleviate the stability-plasticity dilemma

    S Schug*, F Benzing*, A Steger
    eLife 10: e69884, 2021
    paper ∙ biorxiv ∙ code ∙ tweet

12. Learning where to learn: Gradient sparsity in meta and continual learning

    J von Oswald*, D Zhao*, S Kobayashi, S Schug, M Caccia, N Zucchet, J Sacramento
    NeurIPS 2021
    paper ∙ arxiv ∙ code

13. Task-Agnostic Continual Learning via Stochastic Synapses

    S Schug, F Benzing, A Steger
    Workshop on Continual Learning at ICML 2020
    paper ∙ workshop

14. Evolving instinctive behaviour in resource-constrained autonomous agents using grammatical evolution

    A Hallawa, S Schug, G Iacca, G Ascheid
    EvoStar 2020
    paper

Resources

2025

2024

2023

2022

2020

2019

---

¹ Survival of motor neuron 1, also known as component of gems 1, is a gene that encodes the SMN protein in humans.