If I am not wrong, symfonium creates the instant mixes on the client side, independent of whatever algorithms the server uses for its own version of the feature. A toggle to let users select between server-side or client side implementation of instant mixes will give more flexibility
Problem solved:
In my specific case, I use Audiomuse-AI with its jellyfin plugin to replicate plex’s sonic analysis features on jellyfin. This creates much more accurate instant mixes than either the jellyfin’s native implementation or symfonium’s. Having the option would free the hassle of having to shift to jellyfin’s player every time I want to listen to a good instant mix playlist.
Brought benefits:
At present this feature is only beneficial to a subset of jellyfin users who use audio analysis with Audiomuse-AI, but I expect self-hosted music servers to increasingly implement similar features sooner or later. In such case users would be losing out on better mix algorithms unless symfonium implements its own version of audio analysis based music recommendations which I think is not feasible.
So, though not a critical feature, it can be turned into long-term goal to give users more flexibility on how their instant mixes should be created.
Other application solutions:
Finamp does this for jellyfin by calling the server’s InstantMix api endpoints instead of doing it on the client-side.
Additional description and context:
This will probably need a separate implementation for each media source type to make it complete. But the work needed for each source would only be limited to diverting the function call to create instant mix to the server api instead of whatever the function symfonium is using now and parsing the response similar to how the general playlists for that specific media source is parsed. Well I don’t know the inner workings of symfonium, so just my two cents
Once Jellyfin is back online, the AudioMuse-AI middleware will be loaded automatically. Your applications can now call Jellyfin’s API endpoints directly so no additional proxying through the AudioMuse-AI service is required.
I also manually called the api at the original endpoint and checked audiomuse’s logs, the original endpoint is indeed returning results from audiomuse’s analysis.
Unless I am doing something wrong, it is not calling the jellyfin server at all for the playlist. Just to be safe I checked it for all types of mixes/shuffling listed here and it isn’t interacting with the server for any of them for playlist generation
Radio mixes (Requires version 5.7.0+)
image
Those mixes are only available for providers who expose an similar artist API.
Symfonium will query the provider for that list then use that as a source for the personal mix generator.
The provider needs to be online for this to work.
If your provider does not return any data, or there’s network errors, Symfonium will fallback to Instant mixes.
But symfonium isn’t even firing a request to the server for playlist generation in this case… and jellyfin does expose similar artist api. I feel like the feature was supposed to be implemented but probably forgotten about sometime during the app’s development. Can you check it on your end after your holiday ends?
I rechecked jellyfin logs at elevated logging level and it seems you are right, symfonium is requesting with similar items and getting a valid response. But it is sending the artist id instead of the song id at {itemId} to the correct endpoint for retrieving similar tracks which actually returns similar artists instead of returning tracks when the id corresponds to an artist. So it isn’t working as intended if it is supposed to work like Use similar tracks for radio mix turned on for jellyfin.
PS: well, it seems like even fixing that wouldn’t get me what I want because it looks like jellyfin’s instantmix implementation is different to what it returns for similar items api.
Would also like audiomuse support for the sonic analysis features in symphonium. Have it all set up with the instant mixes but would be cool to get the sonic features in the app too
Just to chime in here, the AudioMuse-AI Jellyfin plugin only overwrites the /Items/{ItemId}/InstantMix endpoint. While there are specific instant mix endpoints for artists/albums/etc, the generic /Items endpoint accepts any of these types and produces a suitable mix for each. It always returns a list of track IDs.
Given that AudioMuse’s instant mixes are static at the moment (meaning they always return the same tracks in the same order, if the initial track is the same), it is clear that Symfonium for some reason currently isn’t using the AudioMuse mix.
A potential reason for this is Symfonium using the /Songs/{itemId}/InstantMix endpoint for track mixes (theoretically makes sense), which is not overwritten by the plugin.
If that is the case we could either try to add support for the specific instant mix endpoints (/Albums, /Songs, etc.) to the AudioMuse plugin, or Symfonium could switch to using the generic /Items/{ItemId}/InstantMix endpoint.
Updating the plugin is probably the better solution, since Symfonium is probably not the only client using the specific endpoints
Only the radios uses the server endpoints to get similar stuff but not the InstantMix part of Jellyfin because Symfonium have additional things like excluded from mixes songs and skipped songs handling and ensures better artists / album splitting than what Jellyfin provides.
What would be needed for AudioMuse is to expose an endpoint to provide the same things as Plex.
A sonically similar track list from a source list and a sonic path between 2 tracks. That would be something fully replacing Plex and allowing users to use SmartFlows in the same way as Plex and users could finally ditch Plex
Ah okay, good to know. AudioMuse (and by extension the plugin) currently only expose an endpoint that works like the instant mix (you give it one seed track, and it returns a list of tracks ordered by similarity).
The InstantMix override actually goes beyond that already, and can handle multiple tracks (e.g. from the same album). Is that the “source list” you are referring to? If so, I think we could move that functionality to the main AudioMuse server, and expose it as a separate endpoint (also through the Jellyfin plugin, no need to configure the address for the AudioMuse server).
We haven’t looked into a “sonic path” yet, but we’ll try to find a good way to do that
Thanks for you feedback!
Yes endpoints directly for the addon but with jellyfin network config would be perfect.
I do not think cheating the InstantMix endpoint to get the similar song ordered by similarity is the correct way, as while it do that for now I suppose it will be improved in the future and so can’t be trusted for the long term.
2 new endpoints and people will leave Plex
Do you by luck know by memory if there’s an endpoint to check loaded plugins and their version so I can detect AudioMuse easily in the future ?
Okay, we’re working on it! We already have a proof-of-concept for the path between two tracks, after that we’ll think about the track list based on a source list (but things like the playlist instant mix already work very similar to that, so it shouldn’t be hard).
Overriding the Instant Mix endpoint of Jellyfin will probably stay as it is, because this way any Jellyfin client (including Jellyfin Web) will get access to the improved mixes right-away if the plugin is installed. And that was part of the goal from the start ^^
And yes, there’s an endpoint for that, simply /Plugins
Forget to answer about that it was actually a typo, there’s no big need for that. One can just call the endpoint multiple times.
And yes overriding is good for normal clients, but a proper endpoint to have access to the similar list in order is also necessary.
As you said currently the override returns always the same data in the proper order, but while it’s nice for apps that will directly use the sonic data, it’s not optimal as an instant mix feature, so I’m sure the plugin will be updated to actually add some randomness and everything for a good mix and then the end point becomes non working for the other usage without an easy way for Symfonium to detect that.
Hence the need for another endpoint that specify that it returns in sonic order and won’t change the definition in the near future.
With the sonic path, then all the radio and dj modes from Plex can be implemented and people will rejoice. So many sticking just because of that.
Okay then we’re on the same page now
Override as default + additional endpoints for “custom” implementations like yours.
Randomness is also something I’m not happy with, we’ll need to think about how to improve that. Any solution will end up making the mix slightly “worse” since it’s not strictly ordered by similarity anymore, but that should be fine. Question is just how/where to add the randomness.
But you mentioned a good point there, we should still expose the “raw” list of tracks ordered by similarity, without any randomness. That makes absolute sense.
Another question though: is there any kind of documentation on the Plex sonic analysis endpoints? I couldn’t even find a proper API documentation for Plex, did you reverse-engineer the endpoints yourself?
Because it would be good to have a reference of how the inputs and outputs look/work on Plex, so that we can do it in mostly the same way.
For example, I assumed that the “track list based on source list” would generate tracks that are in some way similar to the given source list, but it if you say that you could just call the endpoint multiple times, then that means that the resulting list has at least one similar track for each source track?
I’d appreciate any details!
Enable the option in the provider then use the radio mix.
