Library scanning too slow

Issue description:

Hi,

I am trying out the app and I wanted to add a folder from my Google Drive which has around 30k songs.

It seems that tags scanning from the folder is really really slow, around 1 song every 1-2 seconds (probably depending on the size of the mp3 file).

Is there any way to speed this up, it kinda sucks that I have to wait half a day to get my library show up in the app.

Is this going to happen every time I rescan the library to add newly added songs?

If it is relevant, I have turned off the option to fetch metadata from the internet when connecting to Google Drive since all my tags are “perfect” and I’d like to use those, instead of random tags from the internet.

Thanks!

Logs:

Upload description: franjo

Additional information:

 

 

Reproduction steps:

 

 

Media provider:

Google Drive

Screenshots:

     

Next version is a little faster but Google API is currently quite slow.

Fetch additional is only for artist image and biography so not related to the tags.

And no the tags are cached so sync is way faster after for new songs.

Alright, thanks! I will test further!

Hi Tolriq,

Symfonium managed to finish scanning my library, bit it seems that the third of the files is missing from the library. Last night, the scan just ended about 2/3 way into the scan, and I haven’t been able to restart it again. When I start the sync from media provider settings, nothing happens.

At first I thought that I hit some kind of rate limit, but today, the same thing happens, I can’t restart the sync.

I have attached the log named “franjo-missing-files” when I click on the sync option and nothing happens.

Hum in the logs I can see a sync starting.

You do not see anything at all in the interface ?

For the missing files I fear you faced the same issue another use had but without logs of the first scan it’s hard to understand why they are not rescanned on the second attempts :frowning:

Would be nice if you could join the beta on Play Store that have a few changes for GDrive to be able to provide logs from a full scan.

Nope, there’s nothing in the interface when I click on the Sync option. Can I remove and readd the media provider? Will that delete already cached tags?

Is me providing logs for a full scan viable at all? Full scan takes all day for me and I’m all over town most of the days and I already wasted my data package from the previous full scan, in the time it took me to come home from work :smile:

Besides, Play Store says that the beta is full.

You can use beta maniac to join the beta don’t know why the Play Store never shows correctly when there’s room.

If you remove the provider it remove the cache yes :frowning:

Unfortunately without the logs showing why a file fails but is not considered failed by Symfonium it’s hard to understand and fix.

I fixed a few details I saw, but that one no idea, and if the file is not seen as being in error it’s cached and so not rescanned if the time does not change.

I’ll see to add a way to remove specific files from cache if possible.

Ugh! I’m not a fan of providing my google account credentials to random apps!

I’ll wait for the next release and try again. Will uninstall and reinstall reset the trial period?

No it just add one day, you’ll need to contact me with the ID.

But next version is delayed a lot due to tons of Google Compose bugs :frowning:

Alright! In the meantime, I’ll try clearing the storage and try again fresh if I manage to stay at one place :smile: Maybe over the weekend.

Please be sure to have logging enabled if you try so I can see and fix the issue.

Hi,

I managed to repeat the process, and again, the scanning stopped at about two thirds of the library.

I’ve turned on the debugging, but only after 2500 tags have been imported :smile:

It is uploaded under “franjo-missing-files-v2”

90% of the log is about no internet access:

2024-10-26 18:58:07.648 Verbose GoogleDriveLogger  <-- [853] HTTP FAILED: java.net.UnknownHostException: Unable to resolve host "www.googleapis.com": No address associated with hostname
2024-10-26 18:58:07.654 Verbose GoogleDriveLogger  <-- [429] HTTP FAILED: java.net.UnknownHostException: Unable to resolve host "www.googleapis.com": No address associated with hostname

And they are correctly seen as error so not cached.

They should be scanned again in you sync again. Try with logs enabled and tell me a non scanned song name so I can check what it says for it.

There wasn’t any moment where internet was not available, so I’m not sure why those errors show up.

I was at home the entire time where I have 500Mbit fiber optics, so I would’ve definitely noticed if internet was out.

As before, now I can’t run the sync again, nothing shows up in the interface. Do you want me to run the full import again with debugging turned on from the start, and then I can provide an example file name?

Btw, I have now run the scan manually again, and it does generate the log file which says the files are downloaded, even if nothing is showing up in the inteface.

So maybe you have some issues there, with showing up the repeated sync inteface?

Also, is there a way to see how many files are scanned and cached in the database? It would help with verifying that repeated sync is actually working, even if its not showing up.

This is not internet down this is dns not answering for a long time.

Where are you looking at for the showing up? If you mean in the home page this is normal, only the first scan is visible there as you can’t use the app without data so it’s important to have the progress, future syncs are not shown in big, click the filter button to see the status of the running sync.

That’s also where you start the sync so maybe it’s just a missunderstanding about how sync works ?

Yeah, I kinda expected that the sync info will again show up on the homepage as it did the first time.

Now that I have run the sync again, all my media is showing up (30k, which is close to the real number, if now the exact number).

The issue with first sync remains (it breaks after 2/3 of files), but at least I am now aware that repeated sync is working and can import the remainder of the files!

The first sync DNS issues lasted a very long time so even the next version won’t recover from that and you’ll still need to sync again to get the missing ones.

But so most things works as expected.

Yeah, that is good!

A quick question before I consider this matter final.

Is is possible to backup and restore the entire library when I e.g. switch devices or reinstall the app after factory reset? I see there is backup and restore in the settings, but I’m not sure if it includes the library itself.

The point is to minimize the need to repeat the full rescan of the library further down the road.

No the backup won’t store everything the data is in the files and the source of truth.