Crash when folder contains filenames with Windows encoding

[abu baniaz]

This is how my one looks. I used your zip file.

[twol]

pöllö.mp4 is just empty file. Attached zipped folder. Windows is Finnish locale. I'd guess this is Python3 related issue..

interesting in that when I unzip there are 2 files there pöllö.mp4 and a 2nd with the ö replaced by hex 94. On copying to the moves folder, both appear but the 2nd now shows as pll.mp4 with the hex characters stripped out. Both are accepted without a crash.

[birdman]

pöllö.mp4 is just empty file. Attached zipped folder. Windows is Finnish locale. I'd guess this is Python3 related issue..

Or a Windows one, since filenames are supposed to be utf-16(?).
That filename shows up as pФllФ.mp4 on my Linux systems. (The Ф is a Cyrillic Capital Letter EF - Unicode U+0424.)

But it is a valid utf-8 string (even if it doesn't look as you'd expect). Not sure why the ö (o with diaresis - U+00F6) should show up differently in different places, though.

[birdman]

interesting in that when I unzip there are 2 files there

What unzip are you using?
Mine finds one file and says so:

[parent]: unzip -l ../mymovies.zip  
Archive:  ../mymovies.zip 
  Length      Date    Time    Name 
---------  ---------- -----   ---- 
        0  2022-06-23 16:44   mymovies/pФllФ.mp4 
---------                     ------- 
        0                     1 file

And we disagree about the name which is there too.

However, if I actually peep into the zip file the filename which is there is:
p~ll~.mp4
where both ~s are byte 0x94.
Which in Unicode is a non-printable character. (CANCEL CHARACTER).
Which might be what triggers the bugs.

[birdman]

However, if I actually peep into the zip file the filename which is there is:
p~ll~.mp4
where both ~s are byte 0x94.
Which in Unicode is a non-printable character. (CANCEL CHARACTER).
Which might be what triggers the bugs.

On reflexion 0x94, whilst being perfectly legal in an ext4 filename, is NOT legal utf-8.
That should be the 2-byte sequence 0xc2 0x94.

But ö is U+00f6 in Unicode, which is 0xc3 0xb6 in utf-8.

Anything that looks at filesystem names (or, as in console output, might echo this back) has to be able to cater for the result being non-utf8 when decoded.
So this might be what contributes to triggering the bugs.