Perch and MBARI Clips: Signal and Noise

In a previous post, I was checking to see if Perch 2.0 places similar humpback vocalizations near each other in its embedding space. “Bloop” sounds seemed to show up reliably in the 15 closest neighbors to a strong bloop, so it’s time to see if the embeddings perform similarly for another type of sound.

Here’s a fairly different sound. It has some tonal similarities to the bloop but the timbre is very different. It’s a sort of…tonal growl?

It doesn’t seem to be a terribly common humpback vocalization, which means this could be a particularly good test of whether related vocalizations show up near each other in the embedding space. I’ll match the bloop test and check the closest fifteen:

It’s worth noting that only one of these (7531) is close in time to the original: it’s the previous five seconds, actually, which makes it more than likely it’s the same whale making the same sound. It’s also interesting that (and this is not obvious from the data shown here) all of these clips are in the >90% humpback-presence confidence bucket from the humpback detector model.

Overall, those are pretty similar. Some are tonally similar, some are more similar in timbre, and many of them are similar in both.

Let’s see how well the matches are for a less-definitive clip (at least according to the humpback detector model). Here’s a descending humpback vocalization that’s much weaker compared to the noise. The detector model put this clip in the 70-90% positive bucket:

And the closest fifteen:

Now this is interesting.

The neighbors all sound like similar whale vocalizations, but the neighbors also all share the weak-signal/high-noise profile of the original. Not a clear-sounding vocalization among them. The embeddings seem to be capturing relationships between the entire scenes, not just the calls themselves.

Also, none of the fifteen neighbors is in the >90% detector bucket.¹ Eight of them are in the 70-90% bucket (7012, 7153, 7125, 7346, 7269, 7172, 7351, 6769), four are in the 50-70% bucket (5665, 5846, 5743, 5744), and three of them are in the 30-50% bucket (4479, 4241, 4452). So while the humpback detector scored those last three clips below a 50% probability of containing a humpback vocalization, the Perch embeddings seem to reflect both the presence of a humpback vocalization and the specific type of vocalization.

It seems notable that the neighbors are all matches for the particular type of humpback vocalization and prominent background noise. No high-signal clips in the group.

So Perch may be grouping these clips by vocalization type, but also by overall sound profile.