Piotr Piękos

Deep Learning
PhD Candidate

Piotr Piękos

Piotr Piękos

Deep Learning
PhD Candidate


Scalable Reasoning

About me

I am a 4th year PhD Candidate at KAUST AI Initiative, supervised by Prof. Jürgen Schmidhuber . I am interested in scalable methods that improve reasoning in neural networks. In practice, it means that I mostly work on Mixture-of-Experts, analyzing and improving transformer architecture, LLMs and (goal-conditioned / hierarchical) reinforcement learning algorithms.

I have published in top ML conferences such as NeurIPS, ACL, ICLR or ICRA. My publications have been distincted as oral presentations in ACL and ICLR. The list of all my publications can be found below.

I studied simultaneously Mathematics (BSc+MSc) and Computer Science (BSc) at the University of Warsaw. I wrote my Master's thesis on improving BERT's mathematical skills under the supervision of Mateusz Malinowski and Henryk Michalewski

I have 4 years of industry experience. Apart from my 2024 Amazon research internship, I worked for over 3 years as a Deep Learning engineer before starting my PhD. During that time I worked with some of the biggest Polish companies like Allegro, LPP or AmRest.

Apart from deep learning, I enjoy bicycle trips, playing chess, and playing some instruments: I play guitar, and I have started learning to play the piano.

Publications


Experience

  • May 2024 - October 2024

    Amazon

    Applied Scientist Internship

    Creating hierarchical discrete representations from dense vector representations used for representations of product in the Amazon catalog.Lead to a publication about Hyperbolic Residual Quantization apart from being implemented in the internal Amazon systems.

  • June 2023 - Aug 2023

    IDSIA

    Research Internship

    Working on sample efficient Goal-Conditioned RL and improving the speed of the Transformer. The result of that were two publications, "Efficient Value Propagation with the Compositional Optimality Equation " and "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention".

  • Sept 2022 - Present

    KAUST

    PhD Student (GPA 4.0/4.0)

    Supervised by Prof. Jürgen Schmidhuber. I am working on generalization, sample efficiency in context of language processing and reinforcement learning.

  • Jan 2022 - Jul 2022

    AWARElab, IMPAN (Institute of Mathematics, Polish Academy of Sciences)

    Researcher

    Working on AdaSubS. Implementing experiments on the sokoban environment

  • Oct 2021 - Jan 2022

    Allegro.PL

    Research Engineer

    Researching neural search methods for improving search recommendations on the Allegro.pl platform

  • July 2017 - June 2020

    ITmagination

    ML/DL Engineer

    In ITmagination I created DL/ML solutions to help other companies. Example projects are: Object detection system for evaluating influencers, recommender system or a system to help with estimating staff needs in physical shops.


Awards

EEML 2025 Summer School

Best Poster Award — "Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing"

ICLR 2023

Oral Presentation — "Fast and Precise: Adjusting Planning Horizon with Adaptive Subgoal Search"

R0-FoMo NeurIPS 2023 Workshop

Best Paper Award — "Mindstorms in Natural Language-Based Societies of Mind"

ACL 2021

Oral Presentation — "Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoning"

EEML 2021 Summer School

Best Poster Award — "Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoning"

Contact Me

Email

piotr.piekos@kaust.edu.sa

Social