Smart flow echo match queues same song each time

[TheKingPin]

Issue description:

Hi

Im testing Jellyfin with Audiomuse and noticed that echo match will queue the same next song each time.

I have tested and replicated it in the logs, I get the same result regardless of what song I choose it seems. The echo match will also pair the next song the same.

I cannot say for certain that this is an issue on symfonium side or jellyfin/audiomuse.

Logs:

Upload description: thekingpin

Additional information:

Maybe its related to this:

Smart Queue/Flow to avoid adding songs that are already in the queue

Feature description: I use the Smart Flow and Smart Queue a lot. However it will often add songs that have already been played in the current queue or songs that were already in the queue. Problem solved: Duplicate songs (I’ve had the same song repeated 4 times in a single queue) being added to the queue. Brought benefits: Keeping a clear and clean listening experience with repeats which can cause frustration. Other application solutions:     Additional description and context:     Screen…

Reproduction steps:

Media provider:

Jellyfin

Screenshots:

[Tolriq]

Please provide a lot more details about the songs you play what is queued and the actual result. It’s hard to guess the issue from song ids and large logs.

In all cases for Echo Match the algo is simple Symfonium asks for the most similar song and use that, so not much freedom.

[NateG]

I didn’t mention that I also use Jellyfin and audiomuse. Unfortunately getting logs for this is difficult since my music library is massive and replicating is random.

[TheKingPin]

Sure, I will try and provide all the details, if there is anything missing let me know and I can add it.

In this example I am playing a song from my liked tracks (Porcupine Tree - Collapse the light in earth), then I hit Smart flow → Echo Match and let it continue. The matched song will always be the same, in this case (Coldplay - Everyday life). The same happens if I play the song from other screens also.

Is there maybe a misunderstanding from my end. I thought that it will queue a different song each time? Or is “Echo match“ a once off process, Song A best match is Song B forever until a better match is found in future through library change for example.

@Tolriq

Symfonium asks for the most similar song and use that, so not much freedom.

Because if I go back in future and play “Porcupine Tree - Collapse ….” again it matches with the same coldplay song. Yes the match is good, but, it would be nice discover other matches if available.

Just tested the other smart queue functions, and “Transition Maestro“ and “Steady vibes“ will also queue the same songs Again, maybe I am misunderstanding how the smart queue should function.

[Tolriq]

Yes the tracks are the tracks returned by AudioMuse, I think @neptunehub have plans to had a little randomness there.

[TheKingPin]

Ok. Thanks for clearing that up. I didnt know the developer had an account here. Would be great to hear from them.

[Tolriq]

Yes we need either randomness at source, or normalization of the distances values so that clients can make proper decisions.

[TheKingPin]

But smart queue with Tracks [Sonic Analysis based] seems to work, I load a track and it continues to load suitable tracks. This gives me impression audiomuse is doing the analysis correct. I get it, “echo match” probably looks at more variables, but, audiomuse is exposing all values needed to make a decision or not?

[Tolriq]

Read again what I wrote ? AudioMuse returns the best match so every time the same one, that’s how AudioMuse works …

So it’s normal and behave as expected. AudioMuse use internal distance values that are not normalized so Symfonium have no way to properly decide if other tracks in the list are still close enough to be chosen or not.

So TL;DR; exactly what I said above

[neptunehub]

Hi all,
thanks for the feedback and excuseme if I’m replying only now but I missed this new message on the forum.

The Similar Track API on the plugin, reproduce 1:1 the Playlist From Similar song of AudioMuse-AI integrated front-end. This functionality just do an exact match of the most similar song of the starting one. No more, no less. This is his expected bheaviour.

Also the Find Path API on the plugin, reproduce 1:1 the Song Path of AudioMuse-AI integrated front-end. This functionality also give an exact path between a starting song and an ending one. No randomenss.

There is a new Song Alchemy API on the plugin, that reproduce 1:1 Song Alchemy functionality of AudioMuse-AI integrated front-end.

This is an example of code:
curl POST ‘``http://YOUR-JELLYFIN-URL``:PORT/AudioMuseAI/alchemy’
-H ‘Content-Type: application/json’
-H ‘Authorization: MediaBrowser Client=“MyCLI”, Device=“Ubuntu CLI”, DeviceId=“ubuntu-cli-01”, Version=“1.0.0”, Token=“YOUR-JELLYFIN-API-TOKEN”’
-d ‘{
“items”: [
{“id”:“7190693ae7d0b7740fbfc26e5bddd0b3”,“op”:“SUBTRACT”},
{“id”:“2caeeff701c08929f03261e95cdc022d”,“op”:“ADD”},
{“id”:“574a710aa6fbe82963a9533484e243ff”,“op”:“ADD”},
{“id”:“e614f2119e654493012ea80f7dd5c617”,“op”:“ADD”}
],
“n”: 10,
“subtract_distance”: 0.2,
“temperature”: 1
}’

Basicallty it do a centroid of track to add, and find similar song. Then a centroid of track to subtract and it remove the song to near to them. Also it have randomness (the temperature, 0 no randomness, 1 is alrady a good randomness).
If you put only one song in ADD, the result is Similar Track API with randomness.
This off course if Symfonium developer would like to add this additional functionality.

I don’t like to change the basic API functionality because it have a different goals: play a list of song with the best smooth transition between one and the other, and off course randomness is in contrast to this.
You can off course start with a different song/seed to have a bit different song list, or just reply in a shuffled order.

Hope this could help.

[Tolriq]

The problem as we discussed earlier is that the distance value you return everywhere is not normalized.

Similar Track can return a list of tracks, but since the distance value is not really meaningful the client can’t decide the max distance it accept as similar and take a random item in the list that match that distance.

With a normalized value between 0 and 1, a client can decide than a distance less than 0.10 is acceptable as similar and can do more things based on the distance.

[neptunehub]

Hello everyone,

Let me recap how our distance metrics work to make sure we’re all on the same page.

The AudioMuse-AI Similar Track endpoint primarily uses Cosine distance, which is the default and highly recommended metric. This means the distance value (D) you get from the API will be in the range of 0 to 2, where 0 means the tracks are identical and 2 means they are completely opposite. Off course you will never get 2 in a real use case scenario, instead 0 is pretty common to the duplicated song (like some song, different album, in fact we use this value for duplicate removal).

If you prefer a more intuitive similarity score between 0 and 1, you can easily convert the distance D with this formula: S = (1 - (D / 2))

For example, if you request 200 similar songs, you might see the nearest track has a distance of D = 0.0457, which becomes a similarity score of S = 0.97715. The 200th track in that list might have D = 0.1172, which is a similarity of S = 0.9414. This shows that even the last item in the list is still very similar.

Now, if you want to introduce some variety instead of always getting the same top results, you have two options. You could implement a softmax sampling approach on your end by requesting 200 tracks and then sampling the 100 you need based on their similarity scores.

Alternatively, an easier way is to use our Song Alchemy endpoint. It’s cover this purpose in his different use case. Just provide a single song and set the temperature parameter (e.g., to 1) to get a varied but still relevant list of tracks. Doing some test 1 is already very high (for example on 10 song, you have like only 3 in common between 2 different request)

Personally, I don’t like getting random results when searching for direct similarities, which is why we separated this logic into a different endpoint. Similar Track is for deterministic results, and Song Alchemy is for discovery.

Let me know if this clears things up or if there’s anything else I can do to help with the integration.

[Tolriq]

Thanks for the details on the distance that was missing from your details the last time we spoke about this need.

As a reminder Symfonium is not an AudioMuse client per se. It needs to deal with different things to achieve consistent results between things. And as you know Plex was here a long time before so kinda set the basis on this.

The needs of max distance are dependent on the smart flow used and other cases, and needs to deal with other things like songs marked for skip and everything.

TL;DR; Symfonium is provider agnostic and needs to have data in a common format to handle many things after retrieval.

[Chaphasilor]

Just to add to this, while normalizing the distances would be nice, I’m not sure how feasible/usable this actually is. All libraries are different, and the different “genres” within the libraries might not all have the same density or size. So for one specific genre, a normalized distance of 0.2 might be good radius to include similar tracks, but for another genres 0.4 or 0.05 might be more suitable instead.
It’s probably best to handle this on a best-effort basis by combining the amount of requested tracks with the cosine similarity. That should ensure only the most similar tracks are returned at all, and then the cosine similarity can be used to filter out any tracks that reasonably similar to the seed track (e.g. because there are only a couple of tracks from that genre in the library).
That all is only if you need explicit control.

If not, using the “song alchemy” endpoint with a temperature above 0 should do a good job of adding in some variation.

[neptunehub]

Of course I totally understand I just want to help!

I just released the new AudioMuse-AI plugin that add this new API:

Max Distance

Get in input the id of the song and return as a result the max distanace possible from that song and the id of this max distantsong. In case of cosine similarity (enabled by default in AudioMuse-AI) the distance is between 0 and 2 where 0 is the same song and 2 id the totally opposite song.

curl 'http://YOUR-JELLYFIN-URL:PORT/AudioMuseAI/max_distance?item_id=07a998a337ab3fd4576006ae301d1d94' \
  -H 'Authorization: MediaBrowser Client="MyCLI", Device="Ubuntu CLI", DeviceId="ubuntu-cli-01", Version="1.0.0", Token="YOUR-JELLYFIN-API-TOKEN"' \
  -H 'Accept: application/json'

Output

{"farthest_item_id":"c1ea3fb87ff0cd6e68b881715104fb28","max_distance":1.3198845386505127}

Where:
v0.1.25 => is for Jellyfin 10.11
v0.1.24 => is for Jellyfin 10.10.7

This plugin map with the actual :devel image of AudioMuse-AI and will be released in v0.7.5-beta soon.

Remember min 0 max 2 and doing some test in my library I got something like 1.4 - 1.5 depending from the starting song.

Hope this could help bringing randomness and unpredictability in the world

[TheKingPin]

Thank you @neptunehub for your continued support in this. I am so happy that someone is finally working actively on a feature that I have long waited for, without having to move to the Plexiverse.