Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoning
Piotr Piękos, Henryk Michalewski, Mateusz Malinowski.
I am a PhD student at KAUST, advised by Jürgen Schmidhuber, working on systematic generalization and reasoning. During my Masters at University of Warsaw I was advised by Mateusz Malinowski and Henryk Michalewski. I also have experience developing machine learning models for some of the biggest polish companies.
My main goal is to create a general architecture that is able to learn the underlying rules of the environment and generalize to new scenarios. Therefore, my work mostly focuses on transformers, recurrent neural networks, reinforcement learning and graph neural networks.
Implemented BERT MLM layers along with preprocessing necessary for pretraining BERT for trax - Google's open source jax based library.
I had a pleasure to work on a research project related to searching the optimal traffic light control with reinforcement learning with TensorCell group.