Repository files navigation

Music Mashup and Video Creation
with Node.js, FFMPEG, and Contentful

Website ·YouTube Channel ·Buy Me a Coffee ·Instagram ·Report Issue

Auto Mashup createsmusic mashups automatically. It also automatically creates and uploads videos to itsYouTube channel and each video's associated thumbnail to its Instagram page (@automaticmashup).

The Auto Mashup project is split into two repositories.

The first repository (https://github.com/amamenko/auto-mashup)handles the scraping ofBillboard charts for song details and ranking information, as well as individual song data acquisition (e.g. tempo, key, mode, song structure and timestamps, beat positions, instrumental/vocal stems, etc.) using data fromSpotify,Genius,YouTube, and other sources. This data is then stored inContentful's Content Management System.

The second repository (the one you are visiting right now) is responsible for hosting the front-end code of Auto Mashup's official website,auto-mashup.vercel.app, using the data stored in Contentful to find mixable songs, creating those mashups, selecting non-blacklisted mashups and mixing them altogether, creating an associated video slideshow and uploading it to itsYouTube channel, and uploading the associated video thumbnail to the@automaticmashup Instagram page.

The basic functionality of this repository's code logic is:

Front-End Code:

• auto-mashup.vercel.app is a responsive website built withReact.
• The burger menu of the site was created with the help of the libraryreact-burger-menu.
• The most recentAuto Mashup YouTube video URL is updated automatically when a new video is posted in Contentful with Contentful'sContent Management API. Upon app mount, a request is made via Contentful'sContent Delivery API to get the YouTube URL. This URL is then used to embed the YouTube video onto the homepage of the website with the libraryreact-player.

Find Mixable Songs:

• Search the Contentful CMS for songs that have the samemode (either major or minor).
• Narrow down mashup song combinations to those that are within onesemitone up or down, have an up to 5% difference intempo, and in which the instrumental song has at least four continuoussong sections that match the sections of the vocal song.
• Remove mashup entries and associated audio assets if either one or both of the instrumental or vocal song entries have been deleted from Contentful due to not being found on any Billboard charts.

Create Mashups:

• Mashups are created within a customDockerized AWS Lambda function written in TypeScript. The AWS Lambda Docker image is bundled with Node 18 and FFmpeg/FFprobe binary files.
• If both instrumental and vocal entries exist, both audio assets are downloaded from Contentful and mixed usingfluent-ffmpeg. Acomplex filter is used totrim anddelay the matching vocal sections tomatch the beats of the appropriate associated sections of the instrumental track.
• Thepitch andtempo of the vocal audio are manipulated usingFFMPEG'srubberband feature to match the instrumental track using a predetermined key-scale and tempo-scale factor. This factor is also used to adjust thebeat positions of the vocal track identified withessentia.js.
• Audio inputs are normalized with FFMPEG'sloudnorm filter to have similar audio volumes.
• Vocal sections that are below a certain length threshold with respect to their associated instrumental section arelooped.
• Fades are added between each vocal section and between each loop.
• The mixed MP3 mashup audio asset and entry containing data about the instrumental and vocal songs is stored in Contentful's CMS using theContent Management API. Additional information about the mix - such as vocal track mix start, mix end, andtotal duration - is also noted for use when mixing multiple mashups together.

Create Weekly Mix and Video of Selected Mashups:

• Randomly selects either 20 vetted mashups or at least 28 minutes total worth of continuous vetted mashup mixes without any repeats.
• InDecember, at least 5 holiday songs (instrumental, accompaniment, or both), if available, are included throughout the mashup mix.
• Downloads each mashup's audio MP3 asset and each included song's cover photo from Contentful for use in both thumbnail creation and slideshow inclusion.
• Creates a full audio MP3 mix withFFMPEG of all the selected mashups with acrossfade to transition between each mashup mix.
• Creates a thumbnail JPEG image with dimensions 1280x720 usingnode-html-to-image. This thumbnail image has to be a JPEG in order to be able to be uploaded to Instagram viainstagram-web-api. The thumbnail image contains the Auto Mashup logo and the most recent Billboard ranking week date superimposed onto a collage of eight randomly selected cover photos of songs featured in the current week's mashup mix.
• Creates a slideshow image with dimensions 1280x720 (to ultimately create a 720p video) for each mashup withnode-html-to-image. Each slideshow image contains each song's track title, artist, cover photo, latest Billboard chart positions, and key/tempo information.
• Combines all the slideshow images with the full mashup audio mix usingFFMPEG to show the appropriate slide image during the accompanying song's duration. Each slide isoverlayed over andfades into the next slide.
• Adds a green-screen "Like and Subscribe" overlay banner at the beginning of each video and a green-screen "Thanks for Watching" overlay banner at the end of each video by using FFMPEG'scolorkey filter.
• Combines the Auto Mashup intro video (featured in GIF format at the top of this section), hosted on Contentful, with the earlier-created main mix by first re-encoding each video with thex264 library and FFMPEG. The two videos are thenconcatenated with FFMPEG to create a final merged MP4 video.

Upload to YouTube and Instagram:

• Creates a video title containing the most recent Billboard chart week date and three random artists featured in the current week's mashup video.
• Creates a video description containing details about Auto Mashup as well as its social links. The description also contains the timestamps and song details of all of the mashups featured in the current week's mashup video.
• Uploads the video with a custom thumbnail to Auto Mashup'sYouTube channel by means of theyoutube-api, which leveragesYouTube's Data API - particularly thevideos:insert andthumbnails:set endpoints.
• After successful upload to YouTube, records the new YouTube URL to Contentful using theContent Management API.
• Uploads the thumbnail image to Auto Mashup's Instagram account (@automaticmashup) using theinstagram-web-api in a similar manner as featured in my other automated Instagram posting project,Daily Pixel Mike. This post features the most recent Billboard week date and three random artists featured in the current week's mashup video. A comment is automatically added to this post that includes a series of hashtags that also include the names of the three random artists featured in the caption.

Server deployed viaAWS EC2 instance. Client-side website deployed withVercel.

Distributed under the MIT License. SeeLICENSE.txt for more information.

Auto Mashup -automaticmashup@gmail.com

Avraham (Avi) Mamenko -avimamenko@gmail.com

Project Link:https://github.com/amamenko/auto-mashup-mix

• YouTube
• Billboard
• instagram-web-api
• Contentful
• AWS
• node-cron
• FFMPEG
• fluent-ffmpeg
• node-html-to-image
• Best-README-Template