I am a Computer Science PhD student at SLP lab at the Hebrew University of Jerusalem, under the supervision of Dr. Yossi Adi. I have a broad research interest in Language Modelling and learning. The focus of my current research is Speech Language Modelling - from evaluation, to representation and modelling. I'm also a self-proclaimed beer-geek and homebrewer.
TL;DR - We post-train Code LLMs to reason about and simulate code execution given inputs. We then show that models can use this ability to self-verify and self-fix their own generated code. This gives an additional boost over standard reasoning models in competetive coding.
TL;DR - We present a new benchmark on spoken stress detection and understanding. Despite being a key communication method, existing models often over look and perform poorly. Based on an automatic data synthesis approach we are able to acheive strong performance by fine-tuning a speech-aware LM.
TL;DR - We release CWM, a 32B param LLM, to advance research on code generation with world models. To improve code understanding beyond what can be learned from training on static code alone, we mid-train CWM on a large amount of observation-action trajectories from Python interpreter and agentic Docker environments, and perform extensive multi-task reasoning RL.
TL;DR - We perform an extensive empirical evaluation of audio tokenisers across varied use cases, and acoustic domains. We also provide an extensive taxonomy and updating database of tokenisers.
TL;DR - We conduct the first scaling analysis of interleaved speech-text LMs. We find that they scale much more efficeiently than text only LLMs, making them feasible. Furthermore, scaling dynamics are very different showing one should allocate much more compute to parameters in place of training data, relative to existing suggestions.
TL;DR - We introduce slam, a recipe for training high-quality Speech LMs on a single academic GPU in 24 hours. We do so through empirical analysis of model initialisation and architecture, synthetic training data, preference optimisation with synthetic data. We empirically demonstrate the recipe scales up getting results on par with leading SLMs in a fraction of the compute.
TL;DR - We introduce SALMon🍣, a novel evaluation suite to evaluate acoustic modelling abilities of speech LMs. The proposed benchmarks both evaluate the consistency of the inspected element and how much it matches the spoken text. We show leading models perform poorly compared to humans.
TL;DR - We formalise the task and evaluation of speaking style conversion, going beyond converting timbre to also change pitch changes and rhythm. We introduce a simple and efffective method for speaking style conversion based on pre-trained discrete speech tokens.
TL;DR - We introduce a new adversarial setup against text classifiers named universal adversarial policies. Under this setup one learns a single perturbation policy which given a text and a classifier selects the optimal pertubations (which words to replace) in order to reach an adversarial text. The policy must generalise to many unseen texts. We learn such a policy with reinforcement learning which succesfully generalises to unseen texts from as little as 500 texts.