Projects

This ongoing work aims to design new online reinforcement learning algorithms that learn approximately optimal strategies efficiently and robustly. In particular, we study the sample complexity of online Q-learning methods when some prior knowledge about the dynamics is available or can be learned efficiently. For systems that evolve according to an additive disturbance model, we propose a new optimistic Q-learning method that learns an optimal policy (under perfect partial knowledge) or sub-optimal policy (under noisy partial knowledge) in a number of samples that is independent of the cardinalities of state and action spaces. Current work focuses on extending our algorithms and results to problems with continuous spaces.

One of the most important requirements for utility companies is to have an accurate forecast of the demand for the next couple of days. In this project, I worked on a complete pipeline for time series prediction. This includes enhancing a time-series cleaning and imputation algorithms utilizing low-rank approximation by singular value decomposition, and developing and tuning a deep learning model capable of predicting three days-ahead of electricity demand. The algorithms are deployed in a real-time forecasting platform.

Detecting anomalies in electricity consumption, especially from low-resolution signals, is an important and non-trivial task. I developed an anomalies detection algorithm suitable for operating on low-resolution consumption signals by employing a highly nonlinear gradient boosting model. To augment this detection model, I designed a novel synthetic control model that estimates anomalies' magnitude. The two developed models jointly outperform other existing methods.

Detecting anomalies in electricity consumption, especially from low-resolution signals, is an important and non-trivial task. I developed an anomalies detection algorithm suitable for operating on low-resolution consumption signals by employing a highly nonlinear gradient boosting model. To augment this detection model, I designed a novel synthetic control model that estimates anomalies' magnitude. The two developed models jointly outperform other existing methods.

Map reasoning is an intuitive skill for humans and a fundamental skill with important applications in many domains. In this project, we aim to evaluate the capabilities of contemporary state-of-the-art Large Vision-Language Models (LVLMs) for reasoning on maps and comparing their capabilities with human participants on the coregistration task. We additionally propose and release a novel dataset to serve as an initial benchmark for map reasoning capabilities. We run an extensive analysis on the performance of open-source LVLMs showing that they struggle to achieve good performance on our dataset. Additionally, we show that coregistration is intuitive to human participants that were able to achieve close to perfect accuracy in a time-constrained manner.

The majority of reinforcement learning (RL) literature focuses on modeling and learning the expected random return incurred by RL agents. Operating solely based on expectations has a solid footing in the decision and expected utility theories [Von Neumann and Morgenstern, 1947, Barbera et al., 1999]; Nonetheless, recent works argue in favor of modeling and learning the distribution of the random returns, which gave rise to a sub-field known as distributional reinforcement learning [Bellemareet al., 2017, Barth-Maron et al., 2018, Mavrin et al., 2019, Urpí et al., 2021]. This project aims to explore the theories and methods behind this field.

In this project, we study some of the numerical methods of the identification of linear time-invariant (LTI) systems. LTI systems play an important role in many engineering disciplines as many systems are linear or can be approximated effectively by linear dynamics. The process of retrieving the structure of dynamical systems from observable data is known as system identification, and a popular algorithm for that is the eigensystem realization algorithm (ERA). We study a variant of ERA that uses randomized matrix decomposition approaches to improve numerical efficiency.

This project investigates the inherent noises in neural networks trained by Stochastic Gradient Descent (SGD) with weight decay regularization. We empirically investigate how SGD noise is affected by training hyperparameters such as batch size and learning rate. Further, we quantify the distribution of this noise when it is measured at the output of neural networks. Experiments are done on multiple network architectures and the MNIST and CIFAR10 datasets.

Models simulating the spread of infectious diseases are indispensable tools that support the decision-making processes in the event of an epidemic. As part of the KACST response to COVID-19, I developed an agent-based disease spread model capable of simulating novel infectious diseases. I built this disease transmission model in a parallel, vectorized, and computationally efficient implementation. Ongoing work aims to utilize the developed model in policy support and recommendations.

The objective of this project is to design, fabricate, and test a laptop-based radar capable of forming Doppler, range, and synthetic aperture radar (SAR) images. The fabricated radar produces frequency-modulated chirps and operates in the ISM band with a frequency of 2450 MHz. An Arduino Zero is utilized for digital processing, and auxiliary MATLAB functions are implemented to filter and denoise received signals.

The objective of this project is to investigate the design process of the rectangular microstrip patch antenna (MSA) through computer simulation using MATLAB and CST Studio simulation platforms. The designed antenna operates at 3.8 GHz frequency, it achieved directivity of 8.1 dB, and its input resistance was in the desired range of 50 Ω at the resonance frequency.