iTunes Library Import from OS X: Uncovering Unicode Normalization Issues in Plex

While importing my iTunes library to salvage ratings, playlists, etc, I had a look at /volume1/Plex/Library/Application Support/Plex Media Server/Logs/Plex Media Server.log, and I saw things like this:

May 05, 2016 09:10:58 [0xdcab3b40] WARN - Could not find path: /volume1/Entertainment/iTunes/iTunes Media/Music/N-Trance/Happy Hour/07 Andaré Con Mé.m4a

Weird. Okay, let's have a look.

sh-4.3# pwd
/volume1/Entertainment/iTunes/iTunes Media/Music/N-Trance/Happy Hour
sh-4.3# ls -lA
total 80816
---------- 1 user users  6236218 Jan  6  2007 01 Da Ya Think I'm Sexy_.m4a
---------- 1 user users  6519540 Jan  6  2007 02 Paradise City.m4a
---------- 1 user users  5151643 Jan  6  2007 03 Tears In The Rain.m4a
---------- 1 user users  6280303 Jan  6  2007 04 Broken Dreams.m4a
---------- 1 user users  6037025 Jan  6  2007 05 Feel Good.m4a
---------- 1 user users  6342689 Jan  6  2007 06 D.I.S.C.O..m4a
---------- 1 user users  6681405 Jan  6  2007 07 Andaré Con Mé.m4a
---------- 1 user users  6084380 Jan  6  2007 08 If.m4a
---------- 1 user users  7047265 Jan  6  2007 09 Superstition.m4a
---------- 1 user users  7470149 Jan  6  2007 10 Amadeus.m4a
---------- 1 user users 12780991 Jan  6  2007 11 The Mind Of The Machine.m4a
---------- 1 user users  6003130 Aug 25  2008 12 Paradise City (Gyr-8 Remix).m4a
drwxrwxrwx 1 root users     2816 May  4 15:18 @eaDir
sh-4.3# stat *
[...other files removed...]
  File: ‘07 Andaré Con Mé.m4a’
  Size: 6681405     Blocks: 13072      IO Block: 4096   regular file
Device: 28h/40d Inode: 229338      Links: 1
Access: (0000/----------)  Uid: ( 1026/    user)   Gid: (  100/   users)
Access: 2016-05-04 22:26:27.210546079 +0200
Modify: 2007-01-06 11:07:30.000000000 +0100
Change: 2016-05-04 15:18:39.461239146 +0200
 Birth: -
[...other files removed...]

The file clearly exists, so I start to think there's some strange encoding issue with some characters — primarily accented ones — in the XML file (created on OS X) and how Plex is reading it (on Linux).

Let's look at another one:

May 05, 2016 09:47:14 [0xdcab3b40] WARN - Could not find path: /volume1/Entertainment/iTunes/iTunes Media/Music/DJ Envy & Red Café/Buck Buck - Single of the Week/01 Buck Buck (feat. Sheek Louch).m4a

I've confirmed it exists, so, let's have a look at the XML file for this track:

<key>Location</key><string>file:///volume1/Entertainment/iTunes/iTunes%20Media/Music/DJ%20Envy%20&#38;%20Red%20Cafe%CC%81/Buck%20Buck%20-%20Single%20of%20the%20Week/01%20Buck%20Buck%20(feat.%20Sheek%20Louch).m4a</string>

Well, fsck. This is what's causing the problem; iTunes has encoded the filename in the XML file as "é" (0x65cc81; e [0x65] + combining acute accent [0xcc81]) and Plex isn't recognizing it as being the same as "é" (0xc3a9, the UTF-8 encoding for Unicode Character 'LATIN SMALL LETTER E WITH ACUTE' [U+00E9].

More info from Wikipedia:

Code point sequences that are defined as canonically equivalent are assumed to have the same appearance and meaning when printed or displayed. For example, the code point U+006E (the Latin lowercase "n") followed by U+0303 (the combining tilde "◌̃") is defined by Unicode to be canonically equivalent to the single code point U+00F1 (the lowercase letter "ñ" of the Spanish alphabet). Therefore, those sequences should be displayed in the same manner, should be treated in the same way by applications such as alphabetizing names or searching, and may be substituted for each other.