About
Hello! My name is Zachary Novack, and I am currently a 3rd Year Computer Science PhD Student at UC San Diego, where I am advised by Prof. Julian McAuley and Prof. Taylor Berg-Kirkpatrick. Previously, I studied statistics and machine learning at Carnegie Mellon University, and was primarily advised by Prof. Zachary Lipton and Prof. Simon DeDeo.
As someone passionate about building Generative Music and Audio systems, my research primarily focuses on three pillars od bringing the state of the art in music generation to practical usability: Controllability, Efficiency, and Interactivity. Specifically, I am interested in investigating training-free control for generative music models (ICML 2024) and accelerating generative systems to faster than real time speeds (ISMIR 2024). Additionally, a high level goal of mine is to leverage AI systems to help real musicians understand (AES 2024), learn, and perform music more effectively.
In the past, I’ve worked on general multi-modal reasoning tasks (ICML 2023) and empirical deep optimization theory (ICLR 2023).
In my free time, I enjoy making experimental computer music, and teach the front ensemble at 10-time world championship finalist POW Percussion Ensemble!
Updates
| October 2024: | Our work on accelerating text-to-music diffusion models is out on arXiv! |
| October 2024: | Our work on long-form text-audio contrastive learning is out on arXiv! |
| September 2024: | Our work on the largest dataset of public domain of symbolic music scores is out on arXiv! |
| June 2024: | Our work on accelerated training-free editing and control for text-to-music diffusion models is accepted at ISMIR 2024 in San Francisco! |
| May 2024: | Our work on training-free editing and control for text-to-music diffusion models is accepted at ICML 2024 in Vienna as an ORAL, and our work on unsupervised lead sheet generation is accepted at the AES Symposium for AI and the Musician in Boston! |
| January 2024: | Our work on training-free editing and control for text-to-music diffusion models is out on arxiv! |
| October 2023: | Our work on unsupervised lead sheet generation is out on arxiv! |
| June 2023: | Started Research Scientist internship with Nicholas Bryan at the Adobe Research Audio Group! |
| April 2023: | Our work on augmenting CLIP zero-shot inference with hierarchical label sets was accepted to ICML 2023 in Honolulu, Hawaii! |
| March 2023: | Our work on augmenting CLIP zero-shot inference with hierarchical label sets was accepted to the ICLR 2023 1st Workshop on Multimodal Representation Learning! |
| January 2023: | Our work on understanding implicit regularization mechanisms in SGD was accepted to ICLR 2023 in Kigali, Rwanda! |
| December 2022: | Our work on understanding implicit regularization mechanisms in SGD got accepted to the NeurIPS 2022 Workshop on the Benefits of Higher Order Optimization in Machine Learning (HOO-ML), as a Spotlight and won Best Poster! |
| September 2022: | Began CS PhD at UCSD! |
| May 2022: | Submitted senior thesis on modeling social media addiction on Twitter to CMU Kilthub. |
| May 2022: | Graduated from CMU with B.S. in Statistics & Machine Learning, and a minor in Sonic Arts! |