Apple completed its $ 400 million acquisition of the Shazam music recognition app in 2018. Now it brings audio recognition capabilities from Shazam app developers in the form of the new one ShazamKit. The new framework will enable app developers – including those on both Apple platforms and Android – to create apps that can identify music from Shazam’s huge song database or even from their own custom catalog of pre-recorded audio.
Many consumers are already familiar with the Shazam mobile app, which lets you see what song you’re listening to at the touch of a button and then take other actions – such as: View the lyrics, add the song to a playlist, explore music trends, and more. Shazam was first launched in 2008 and was already one of the oldest apps on the App Store when Apple grabbed it.
Now the company is making better use of Shazam than just a music identification utility. With the new ShazamKit, developers can now use Shazam’s audio recognition capabilities to create their own app experiences.
The new framework consists of three parts: Shazam Catalog Recognition, which developers can use to add song recognition to their apps; custom catalog recognition that compares with any audio on the device; and library management.
Shazam Catalog Recognition is what you probably think of when you think of the Shazam experience today. The technology can recognize the song that is playing in the area and then get the song’s metadata such as title and artist. The ShazamKit API will also be able to return other metadata such as genre or album art. And it can tell where in the audio the match occurred.
When matching music, Shazam doesn’t match the audio itself to make this clear. Instead, it creates a lossy representation called a signature and compares it. This method significantly reduces the amount of data that must be sent over the network. Also, signatures cannot be used to reconstruct the original audio, which protects the user’s privacy.
The Shazam catalog contains millions of songs and is hosted in the cloud and maintained by Apple. It is regularly updated with new tracks as they become available.
If a customer is using a developer’s third-party app for music recognition through ShazamKit, they may want to save the song to their Shazam library. This can be found in the Shazam app if the user has it installed, or it can be accessed by long-pressing the Music Recognition Control Center module. The library is also synchronized across devices.
Apple suggests that apps alert their users that recognized songs will be saved in this library as no special permission is required to write to the library.
Credit: Apple
ShazamKit’s custom catalog recognition feature could meanwhile be used to create synced activities or other second-screen experiences in apps by recognizing the developer’s audio, not the one from the Shazam music catalog.
This could enable educational apps where students follow a video lesson where some of the audio from the lesson could start an activity in the student’s companion app. It could also be used to enable mobile shopping experiences that came up while watching a favorite TV show.
ShazamKit is currently in beta on iOS 15.0+, macOS 12.0+, Mac Catalyst 15.0+, tvOS 15.0+ and watchOS 8.0+. On Android, ShazamKit comes in the form of an Android Archive (AAR) file, and it also supports music and custom audio.