Research

Deep Learning Tools for Audacity

We provide a software framework that lets deep learning practitioners easily integrate their own PyTorch models into the open-source Audacity DAW. This lets ML audio researchers put tools in the hands of sound artists without doing DAW-specific development work.

Invited talk at Bay Innovative Signal Hackers Meetup (2021)

Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition

In this work, we exploit hierarchical relationships between instruments in a few-shot learning setup to enable classification of a wider set of musical instruments, given a few examples at inference. See the supplementary code on github.

update: this work won the Best Paper Award at ISMIR 2021! :)

ISMIR 2021 Poster Video

Software

Audacity with Deep Learning

I am contributing a deep learning framework and a deep model manager that connects to HuggingFace to Audacity. This project was funded by a Google Summer of Code grant. Read the Work Product Summary.

audacitorch

PyTorch wrappers for using your deep model in Audacity, and sharing it with the community!

torchopenl3

A PyTorch port of the openl3 audio embedding (ported from the marl implementation).

Philharmonia Dataset

PyTorch dataset bindings for 14,000 sound samples of the Philharmonia Orchestra, retrieved from their website.  [github]