Repository files navigation

General tools for voice analysis.

Thevoice package is being developed to be an easy-to-use set of tools to deal with audio analysis in R. It provides a free and user-friendly toolkit for audio analysis, enabling researchers to extract, tag, and analyze voice data efficiently. It supports the extraction of audio features, enrichment of structured datasets with audio summaries, and automatic identification of spoken segments—while introducing novel features. It also allows audio analysis based on musical theory, associating frequencies with musical notes arranged in a score viagm package.

The package has been tested extensively since 2019, including:

• Real-world applications: Dozens of uses, e.g. sex prediction from voice features and speaker diarization in audiobooks.
• Validation: Successful tests on open datasets and LibriVox recordings.

If you want to contribute, report bugs or request new features, use the 'Issues' tab on Github.

# Development version from GitHubinstall.packages(c('devtools','tidyverse'))devtools::install_github('filipezabala/voice')# Stable version from CRANinstall.packages('voice')

If you wish to perform a full installation, proceed to Section 4.

If you're compiling R packages from source, you may need to installRTools, a collection of Windows-specific build tools for R.

If you're compiling packages, ensure you haveXcode Command Line Tools installed. You also may needmacOS tools.

# Install Xcode on MacOSxcode-select --install

More details may be found athttps://filipezabala.com/voicegnette/.

# packslibrary(voice)library(tidyverse)# get path to audio filewavDir <- list.files(system.file('extdata', package = 'wrassp'),                     pattern = glob2rx('*.wav'), full.names = TRUE)

# minimal usageM <- voice::extract_features(wavDir)glimpse(M)

# creating Extended synthetic dataE <- dplyr::tibble(subject_id = c(1,1,1,2,2,2,3,3,3), wav_path = wavDir)E# minimal usagevoice::tag(E)# canonical datavoice::tag(E, groupBy = 'subject_id')

url0 <- 'https://github.com/filipezabala/voiceAudios/raw/refs/heads/main/wav/doremi.wav'download.file(url0, paste0(tempdir(), '/doremi.wav'), mode = 'wb')

You may use the commandvoice::embed_audio(url0) if you wish to show a playbutton when compiling an .Rmd file. Seehttps://github.com/mccarthy-m-g/embedrfor more details aboutembed_audio() related functions.

M <- voice::extract_features(tempdir())summary(M)

voice::piano_plot(M, 0) # f0voice::piano_plot(M, 0:1) # f0 + f1

(f0_spn <- voice::assign_notes(M, fmt = 0, min_points = 22, min_percentile = .85)) # f0(f1_spn <- voice::assign_notes(M, fmt = 1, min_points = 22, min_percentile = .85)) # f1

Must haveMuseScore andgm.

library(gm)line_0 <- gm::Line(as.character(f0_spn))m0 <- gm::Music() +  gm::Meter(4, 4) +  line_0gm::show(m0, to = c('score', 'audio'))

line_0 <- gm::Line(as.character(f0_spn))line_1 <- gm::Line(as.character(f1_spn))m1 <- gm::Music() +  gm::Meter(4, 4) +  line_0 + line_1gm::show(m1, to = c('score', 'audio'))

Python-based functionsdiarize andextract_features (when the latter is inferringf0_praat andfmt_praat features) require a configured Python environment.

The following steps are used to fully configurevoice onUbuntu 24.04 LTS (Noble Numbat). Reports of inconsistencies are welcome.

Command line tool and library for transferring data with URLs.

# installing dependenciessudo apt-get updatesudo apt-get install -y libssl-dev autoconf libtool make# installing curlsudo apt install curl# verify installationcurl --version

ffmpeg is a cross-platform solution to record, convert and stream audio and video.

sudo apt-get updatesudo apt-get install ffmpeg

sudo apt-get updatesudo apt-get install portaudio19-dev libasound2-dev libfontconfig1-dev libmagick++-dev libxml2-dev libharfbuzz-dev libfribidi-dev libgdal-dev cmake cmake-doc ninja-build

MuseScore is an open source notation software.

sudo add-apt-repository ppa:mscore-ubuntu/mscore-stablesudo apt-get updatesudo apt-get install musescore

R is a free software environment for statistical computing and graphics. To find out your Ubuntu distribution uselsb_release -a at terminal.

sudo sh -c'echo "deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/" >> /etc/apt/sources.list.d/cran.list'sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9 sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9gpg -a --export E084DAB9| sudo apt-key add -sudo add-apt-repository ppa:c2d4u.team/c2d4u4.0+sudo apt-get update&& sudo apt-get upgradesudo apt-get install r-base r-base-dev

RStudio is an Integrated Development Environment (IDE) for R. Check for updateshere.

sudo apt-get updatesudo apt-get install gdebi-corewget https://download1.rstudio.org/electron/jammy/amd64/rstudio-2025.05.0-496-amd64.debsudo gdebi rstudio-2025.05.0-496-amd64.deb

"Packages are the fundamental units of reproducible R code."Hadley Wickham and Jennifer Bryan. The installation may take several minutes. At terminal run:

sudo R

Running R as super user paste the following, row by row:

packs<- c('audio','reticulate','R.utils','seewave','tidyverse','tuneR','wrassp')install.packages(packs,dep=TRUE)update.packages(ask=FALSE)devtools::install_github('egenn/music')devtools::install_github('flujoo/gm')

To configure thegm package.

usethis::edit_r_environ()

Add the lineMUSESCORE_PATH=/usr/bin/mscore to/root/.Renviron file. To exit use:wq at VI. Save and restart the R/RStudio session.

Miniconda is a free minimal installer forconda, an open source package, dependency and environment management system for any language—Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN and more, that runs on Windows, macOS and Linux.
Follow the instructions athttps://docs.conda.io/en/latest/miniconda.html.

At terminal:

cd~/Downloads/wget -r -np -k https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.shcd repo.anaconda.com/miniconda/bash Miniconda3-latest-Linux-x86_64.sh

Do you accept the license terms? [yes|no]yes.

Miniconda3 will now be installed into this location: /home/user/miniconda3 [ENTER]

You can undo this by runningconda init --reverse $SHELL?yes

Do you wish the installer to initialize Miniconda3 by running conda init?yes.

Close and reopen terminal.

conda update -n base -c defaults conda

The following packages will be INSTALLED/REMOVED/UPDATED/DOWNGRADED:... Proceed ([y]/n)?y

conda create -n pyvoice python=3.12

The following (NEW) packages will be downloaded/INSTALLED:... Proceed ([y]/n)?y

conda activate pyvoicepip install -r https://raw.githubusercontent.com/filipezabala/voice/master/requirements.txt

The following steps are used to fully configurevoice onMacOS Sonoma (Link to MacOS Sequoia). Reports of inconsistencies are welcome.

Install Homebrew, 'The Missing Package Manager for macOS (or Linux)' and remember tobrew doctor eventually. At terminal (command + space 'terminal') run:

/bin/bash -c"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

GNU Wget is a free software package for retrieving files using HTTP, HTTPS, FTP and FTPS, the most widely used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

brew install wget

Python is a programming language that integrate systems. According tothis post, it is recommended to install Python 3.8 and 3.9 and make it consistent.

brew install python@3.12python3 --version pip3 --version

ffmpeg is a cross-platform solution to record, convert and stream audio and video. The installation may take several minutes.

brew install ffmpeg

The XQuartz project is an open-source effort to develop a version of theX.Org X Window System that runs on macOS.

• Download and runhttps://github.com/XQuartz/XQuartz/releases/download/XQuartz-2.8.5/XQuartz-2.8.5.pkg
• Will take around 320 MB of disk space
• Send XQuartz-2.8.5.dmg to Trash

Follow the instructions fromhttps://guide.macports.org/chunked/installing.macports.html.

sudo port selfupdate&& sudo port upgrade tcllibsudo port install tcllib

MuseScore is an open source notation software.

• Download and runhttps://musescore.org/en/download/musescore.dmg
• Drag MuseScore 4 to Applications folder
• Will take around 320 MB of disk space
• Unmount MuseScore-4.5.2 virtual disk and send MuseScore-Studio-4.5.2.251141402.dmg to Trash

R is a free software environment for statistical computing and graphics.

• Download and run the pkg file according to you architecture fromhttps://cloud.r-project.org/bin/macosx/
• Will take around 180 MB of disk space

RStudio is an Integrated Development Environment (IDE) for R.

• Download and runhttps://download1.rstudio.org/electron/macos/RStudio-2025.05.0-496.dmg
• Drag RStudio to Applications folder
• Will take around 770 MB of disk space
• Unmount RStudio virtual disk and send RStudio-2025.05.0-496.dmg to Trash
• Typecommand + space 'rstudio'
• Tools > Global Options... > Appearance > Merbivore (Restart required)

"Packages are the fundamental units of reproducible R code."Hadley Wickham and Jennifer Bryan. Typecommand + space 'terminal'

sudo R

Running R as super user paste the following, one line at a time.

packs<- c('audio','reticulate','R.utils','seewave','tidyverse','tuneR','wrassp')install.packages(packs,dep=TRUE)update.packages(ask=FALSE)devtools::install_github('egenn/music')devtools::install_github('flujoo/gm')

Miniconda is a free minimal installer forconda, an open source package, dependency and environment management system for any language—Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN and more, that runs on Windows, macOS and Linux.

For 64-bit version use

cd~/Downloadswget -r -np -k https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.shcd repo.anaconda.com/miniconda/bash Miniconda3-latest-MacOSX-x86_64.sh

For M1 version use

cd~/Downloadswget -r -np -k https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.shcd repo.anaconda.com/miniconda/bash Miniconda3-latest-MacOSX-arm64.sh

In order to continue the installation process, please review the licenseagreement. Please, press ENTER to continueENTER.

You can undo this by runningconda init --reverse $SHELL?yes

Close and reopen terminal.

export PATH="~/miniconda3/bin:$PATH"conda update -n base -c defaults conda

The following packages will be INSTALLED/REMOVED/UPDATED/DOWNGRADED:... Proceed ([y]/n)?y

conda create -n pyvoice python=3.12

The following (NEW) packages will be downloaded/INSTALLED:... Proceed ([y]/n)?y

Close and reopen terminal.

conda activate baseconda activate pyvoicepip install -r https://raw.githubusercontent.com/filipezabala/voice/master/requirements.txt

# downloadurl0 <- 'https://github.com/filipezabala/voiceAudios/raw/main/wav/sherlock0.wav'wavDir <- normalizePath(tempdir())download.file(url0, paste0(wavDir, '/sherlock0.wav'), mode = 'wb')

Diarization can be performed to detect speaker segments (i.e., 'who spoke when').

# diarizevoice::diarize(fromWav = wavDir, toRttm = wavDir, token = 'YOUR_TOKEN')

Thevoice::diarize() function creates Rich Transcription Time Marked (RTTM)¹ files, space-delimited text files containing one turn per line defined by NIST - National Institute of Standards and Technology. The RTTM files can be read usingvoice::read_rttm().

# read_rttm(rttm <- voice::read_rttm(wavDir))

Finally, the audio waves can be automatically segmented.

# split audio wavevoice::splitw(fromWav = wavDir, fromRttm = wavDir, to = wavDir)dir(wavDir, pattern = '.[Ww][Aa][Vv]$')