If I have an album/artist that is stored locally on my phone, but also have the same artist linked to my server, is there a Quick identifier (color coding of icon or something) that’ll help me distinguish between the two?
No there’s not you can easily filter with [Wiki] How to globally filter displayed data (Media providers, libraries, only offline media, ...)
What would be the purpose?
Purpose: If in my car, especially if I don’t have great reception, I’d rather play the locally stored option at a glance instead of creating a filter. It’s just more convenient while driving.
It’s not creating a filter it’s just quick filtering on the provider source. There’s also automatic filters to online only providers, or only offline cached media.
Offline cache from your server inside the app is the normal way to handle your need, not to have duplicated content locally and on the server that can’t be matched.
In the car you are supposed to use Android Auto and in all case would not see a small indicator.
Ahhh, I didn’t notice that offline cache option during my demo. Unfortunately, Mazda rolled android auto out the year after my current model vehicle. I suppose that’s reason enough to buy a new car. Thanks.
Well using offline cache will be cheaper 
But with next version preloading and offline cache it should cover all your needs. If you have different needs feel free to open another more precise issue.