Piotr Piękos

Deep Learning
PhD Student

01

About me

I am a 3rd year PhD Student at KAUST AI Initiative, supervised by Jürgen Schmidhuber . I am interested in reasoning in neural networks. In practice, it means that I mostly work on analyzing and improving transformer architecture, LLMs and (goal-conditioned / hierarchical) reinforcement learning algorithms.

I have 4 years of industry experience. Apart from my 2024 Amazon research internship, I worked for over 3 years as a Deep Learning engineer before starting my PhD. During that time I worked with some of the biggest Polish companies like Allegro, LPP or AmRest.

I studied simultaneously Mathematics (BSc+MSc) and Computer Science (BSc) at the University of Warsaw. I wrote my Master's thesis on improving BERT's mathematical skills under the supervision of Mateusz Malinowski and Henryk Michalewski

Apart from deep learning I enjoy bicycle trips, playing chess and playing some instruments (I play guitar and I am started learning to play the piano).

Don't hesitate to contact me if you have questions about my work, want to chat about research overall or just want to play a chess game!

02

Publications


02

Experience

  • May 2024 - October 2024

    Amazon

    Applied Scientist Internship

    Creating hierarchical discrete representations from dense vector representations used for representations of product in the Amazon catalog. The work will be published after going through internal Amazon processes.

  • June 2023 - Aug 2023

    IDSIA

    Research Internship

    Working on sample efficient Goal-Conditioned RL and improving the speed of the Transformer. The result of that were two publications, "Efficient Value Propagation with the Compositional Optimality Equation " and "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention".

  • Sept 2022 - Present

    KAUST

    PhD Student (GPA 4.0/4.0)

    Supervised by Prof. Jürgen Schmidhuber. I am working on generalization, sample efficiency in context of language processing and reinforcement learning.

  • Jan 2022 - Jul 2022

    AWARElab, IMPAN (Institue of Mathematics, Polish Academy of Sciences)

    Researcher

    Working on AdaSubS. Implementing experiments on the sokoban environment

  • Oct 2021 - Jan 2022

    Allegro.PL

    Research Engineer

    Researching neural search methods for improving search recommendations on the Allegro.pl ("Polish Amazon") platform

  • May 2020 - Sept 2021

    Working on Master thesis

    After promising initial results I decided to focus fully on my Master's thesis, the effect of that is the Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoning paper.

  • July 2017 - June 2020

    ITmagination

    ML/DL Engineer

    In ITmagination I created DL/ML solutions to help other companies. Example projects are: Object detection system for evaluating influencers, recommender system or a system to help with estimating staff needs in physical shops.


04

Contact Me

Email

piotrpiekos@gmail.com

Social