Can't finish syncing music from Airsonic Advanced server

Issue description:

Hello fellow music enthusiasts, I hope you’re all doing well.

First congrats on the app! I really enojoyed Synfonium as a client for the Airsonic server.

I have a TrueNAS server set up, and it’s running an Airsonic-Advanced server inside a Kubernetes container with 750k songs. I’m having trouble syncing my music collection. Tried more than 10 times, but it never went passed 700k sync mark.

The folder structure is as follows:

DEBIAN LINUX

  • Collection root folder\
    • Style & Rhythm\
      • Artist\
        • Album\
          • Album tracks . xyz*
          • Disc number\¹ Album tracks . xyz**
          • Repeated versions of the Album\³
            • Repeated album tracks . xyz*
            • Repeated Disc number\¹ Album tracks . xyz**
        • Compilation (V.A)\
          • Compilation tracks . xyz*
          • Disc number\¹ Album tracks . xyz**
        • Single Tracks . xyz*
    • Playlists\
      • Playlist folder\
        • Tracks.xyz*
    • RAR Collection²\

Particularities:
¹ Only in cases the album has multiple discs;
² We still have lots of albums in zip and rar folders;
³ Lots of albums have repeated versions not yet validated as a subfolder of the album with the ID (v2, v3, v4, vN);

  • We need to playback the following formats: FLAC, MP3, M4A, APE, WAV, OGG, AIFF, WMA, MPC, WV, DSD, AAC, M4P, OPUS;
    ** We have albums which has ideograms (japanese\chinese\korean characteres) and accents;
    *** The collection is not fully organized. It has lots of variations and eventually extra subfolders.

Side note: We upgraded from Airsonic 10.6.1 a couple weeks ago which I managed to sync after a few atempts, and the app worked great. Although Airsonic Advanced it isn’t a recommended platform, we feel it’s the most complete for our needs right now.

Regards,
ADM

Logs:

3 Logs contained in the file below

FILE WITH LOGS AND SCREENSHOTS
debug-and-screenshots-during-sync-edited.zip (1,1,MB)

Screenshots:

Contained in the above ZIP file:

SERVER SIDE

AIRSONIC ADVANCED MEDIA FOLDERS CONFIG
AIRSONIC ADVANCED MEDIA SERVER SETTINGS
AIRSONIC ADVANCED TRANSCODING SETTINGS
AIRSONIC VERSION

SYNFONIUM SIDE

SYNFONIUM SYNC CONFIGS

Additional information:

Compatibility mode is insanely slow on such large collection, Symfonium won’t be the app that fit your needs.
5 hours sync each time you add a song is not sustainable.

Compatibility mode prevent Symfonium from directly syncing the songs but needs 1 query per album. So is more prone to issue on the server side to the data it expose.

From the logs:
End Insert Song (true): 650350 [17839,45s]

The sync does succeed and there’s “only” 650k song synced without error.

Ping @sentriz as I’m pretty sure Gonic could cover all your needs.

Do you have to use Airsonic-Advanced?

If you’re flexible I can recommend using jellyfin as source. While jellyfin itself offers a poor experience with that many songs (both the website version and desktop players/android apps struggle a lot and are hardly responsive) it works really well in Symfonium since that uses its own local database.

My collection is a bit bigger than yours:
library

Syncs usually take around an hour with jellyfin as source and my server’s hardware isn’t that powerful. I have jellyfin running in a jail on my TrueNAS Core server which has an 8-core xeon from 2017 with a turbo frequency of 2.5ish GHz. So on more modern hardware syncing could be a lot faster.

When I initially tried to use navidrome as source the scan took over a day, which I considered to be unacceptable.

I wish you good luck in any case.

Yes Navidrome would need a small tweak on it’s code to fix the slow sync on large database.

Ping again @deluan if he’s back and available.

Thank you all for your answers.

Before upgrading to Airsonic advanced I did manage to sync the same collection with my Airsonic server. But I could not reproduce the steps to do it, and the amount of files read by Air. Advanced is much larger than the old one.

I`ll try your suggestions and see if it can fit in the purpose of the project.

But just to be sure, did I configure Airsonic and Symfonium correctly at least? I tried different settings in the past days but couldn`t access the files except thru the folders tab.

@655321 Our case is almost exactly like yours… Except that we’ve upgraded to TrueNAS Scale and are now hosting the app in a container.

Thanks.
ADM

From the logs yes you did, but hey are not complete so it’s hard to fully know.

That shouldn’t matter. Jellyfin is available via TrueCharts in SCALE:
Here’s a guide for that.
However you could alternatively install jellyfin via docker/kubernetes.

I’d suggest using that jellyfin instance only for music as it becomes pretty much unusable for anything other than Symfonium, which only utilizes the API.
If you want to use jellyfin for movies/tv shows as well I’d suggest running 2 jellyfin containers, 1 for music and 1 for the rest.

A perk of using jellyfin is the good multi value tag support (multiple artists/album artists/genres) which Navidrome and all the other subsonic variations I’ve tried lack.

Playlist support in jellyfin could be better, I’ve had trouble importing my playlists in a way that only the user who “owns” them can see them in Symfonium, which is a problem on the jellyfin side of things.
In case you want to read about my problems and a clunky workaround that I found.

If you’re using lastfm to scrobble you’ll find that the lastfm plugin for jellyfin is currently not working and not maintained. You can get server-side scrobbling for jellyfin working by selfhosting multi-scrobbler and using a webhook plugin in jellyfin, however that only works well for online plays, not for offline plays.
Here’s a link to multi-scrobbler. Great software if you’re into scrobbling.

If you only care about scrobbling what you play in Symfonium, the way to go is pano scrobbler, which is able to scrobble online and offline plays accurately and comes with further functionality like pattern replaces via regular expressions.
Here’s a link to pano scrobbler.

If you care about synced lyrics, the only way I’m aware of to get them into symfonium currently is if you have embedded them as the LYRICS tag in your files.
External .lrc and .txt files are not yet supported via the API.
They will be supported when the next update of jellyfin arrives tho.
If you want to read up on that, I’ve asked in the jellyfin forum.

If you want to use symfonium with jellyfin outside of your local network you’ll need to have dyndns + a reverse proxy configured if you don’t use a ovpn or wireguard vpn or something like tailscale to “remain” in your local network while you’re somewhere else.
I’m using duckdns as dyndns and nginxproxy as reverse proxy.

Feel free to ask me if you have specific questions about my setup.

@655321 Thank you for the careful and long response. We’ll test both Jellyfin and Gonic.

The main goal of our project is to listen to music not easily available in other mainstream platforms. And it’s music only, except for a collection of instrumental music videos that we do not know if we are going to incorporate or not.

But so far the tags are so messy that it must have a browse by folder option. The scrobbling function you mention is quite interesting, but I am still grasping the concept, and will definitely look into it. Does the music sent to last.fm and other scrobble services reflect back in the Synfonium client so it improves the recommendations? What if the media isn’t on their database? Do they use audio fingerprints or other algorithms to make suggestions?

And the lyrics I think it’s a nice addition! I’ll also look into it. Thanks again.

No music is sent to last.fm. It’s pretty much a simple database that players can send information as text to.
The time that you listened to a song, for how long you listened to it, the title, artist and album in most cases.

It will be added as it is. If your media is tagged incorrectly those incorrect tags will show up in your last.fm listening history.
The upside of that is that (if you never fix/change your tags) another player will be able to correctly pull your play counts for these songs.
The downside is that the recommendation system within lastfm only works properly if your tags are correct. If your album is for example called “In The Court Of The Crimson King (2019 Remaster) 24Bit 96KHz” that will show up as is and last.fm might recommend “In The Court Of The Crimson King” to you as it does not understand that both are the same album. Depending on your preference that might even be a good thing if you want to know how many times you’ve listened to which exact version of a song.

If you decide to fix your wrong tags at a later point and wish to fix last.fm as well to keep both in sync, you can get a Pro account which enables you to edit your lastfm database directly.

I never had the need to as I curate all my tags manually but I just tested it and you can browse Folders via “Library → Files” within Symfonium for a jellyfin source.

Depending on the player 2 things might get synchronized.

  1. The play counts of songs.
  2. The “Loved” status of songs.

Recommendations themselves are available within lastfm.

I think it’s much simpler. They correlate which artists are listened to how often by which people and interconnect them that way. But that’s just me guessing. I don’t know how they actually do it. It works fairly well tho, I’ve discovered many an artist through last.fm recommendations.
Recommendations also get better over time as the database grows. I’ve scrobbled without a major break since 2009 and am at 253k scrobbles now.

Any time.