Hello Guest, if you are reading this it means you have not registered yet. Please take a second, Click here to register, and in a few simple steps you will be able to enjoy our community and use our OpenViX support section.
Results 1 to 11 of 11

Thread: Terrestrial EPG foreign charater corruption

  1. #1

    Title
    Forum Supporter
    Donated Member
    Join Date
    Mar 2017
    Posts
    48
    Thanks
    12
    Thanked 13 Times in 11 Posts

    Terrestrial EPG foreign character corruption

    Hi All,

    For a long time EPG descriptions and sometimes, programme titles are out of whack on “smart quotes”, foreign accents, etc.

    I'm not sure if it's channel/Mux specific but it's most often seen on BBC HD broadcasts (probably where we hang out the most). I'm wondering if I'm missing a character-encoding/language setting somewhere which would sort it.

    Vu+ Duo²
    OpenViX 5.2.042 (2019-06-02)
    Kernel / Drivers: 3.13.5 / 20190429
    Terrestrial Freeview only [Vuplus DVB-T NIM(TT3L10) (MultiType DVB-C/T2) x4]
    (Crystal Palace)

    OpenWebIf: Settings > Automatic Language shows:
    EPG Language selection 1: 'Original'
    EPG Language selection 2: 'None'

    Any ideas, maybe tuner set-up?

    e2-openvix_character-encoding_meta.pnge2_epg_accent-char_0_19_4440_4084_233A_EEEE0222_0_0_0_20190223183957.jpg

    TIA!
    Last edited by alphacabbage1; 13-06-19 at 16:00. Reason: charater != character

  2. #2

    Title
    Forum Supporter
    Donated Member
    Join Date
    Mar 2017
    Posts
    48
    Thanks
    12
    Thanked 13 Times in 11 Posts
    For anyone landing here, my best guess at an answer/workaround comes from a MediaPortal forum post:
    • Freeview EPG encoding is erratic, with sources sometimes using non utf-8 encoding (ISO 6937, [ISO-8859-*])
    • utf-8 assumption corrupts on storage
    • Scripts could clean up the data but it's better dealt with at source (broadcaster) or on storage (e2/OpenVix)

    I'm not sure how to test or go about any of that but it might point the way for people who know.

  3. #3
    birdman's Avatar
    Title
    Moderator
    Join Date
    Sep 2014
    Location
    Hitchin, UK
    Posts
    7,771
    Thanks
    235
    Thanked 1,656 Times in 1,305 Posts
    Quote Originally Posted by alphacabbage1 View Post
    • Freeview EPG encoding is erratic
    Certainly true. Some of it seems to have passed through an ISO-8859-* (or, I suspect, a Windows-1252) to utf8 encoder twice.
    In practice you can usually work out what was meant.
    MiracleBox Prem Twin HD - 2@DVB-T2 + Xtrend et8000 - 5(incl. 2 different USBs)@DVB-T2[terrestrial - UK Freeview HD, Sandy Heath] - LAN/USB-stick/HDD

  4. #4

    Title
    Forum Supporter
    Donated Member
    Join Date
    Mar 2017
    Posts
    48
    Thanks
    12
    Thanked 13 Times in 11 Posts
    Quote Originally Posted by birdman View Post
    you can usually work out what was meant.
    Yup, it's not just humans though -- occasionally, it'll hit autotimer searches (esp. when the title's affected).

    One day I'll try and script something to work through .meta files (possibly, .eit) so at least the archive is clean.

    I'm surprised the problem exists this day and age -- it's almost like the UK's not part of Europe.

  5. #5
    birdman's Avatar
    Title
    Moderator
    Join Date
    Sep 2014
    Location
    Hitchin, UK
    Posts
    7,771
    Thanks
    235
    Thanked 1,656 Times in 1,305 Posts
    Quote Originally Posted by alphacabbage1 View Post
    Yup, it's not just humans though -- occasionally, it'll hit autotimer searches (esp. when the title's affected).
    Not that many searches on UK Freeview where you'd need a non-ASCII character.

    The only one I can think of in the last few years was "Les Misérables", and that was OK. Well, it was OK for me - I've just looked again at the second image in #1.

    Although the real oddity there is why BBC ONE HD was showing Baptiste, while BBC ONE London was showing "Les Misérables" - which had finished its series on Feb 3rd.
    Last edited by birdman; 22-06-19 at 00:11.
    MiracleBox Prem Twin HD - 2@DVB-T2 + Xtrend et8000 - 5(incl. 2 different USBs)@DVB-T2[terrestrial - UK Freeview HD, Sandy Heath] - LAN/USB-stick/HDD

  6. #6
    abu baniaz's Avatar
    Title
    Moderator
    Join Date
    Sep 2010
    Location
    East London
    Posts
    23,339
    Thanks
    6,422
    Thanked 9,146 Times in 6,224 Posts
    Quote Originally Posted by birdman View Post
    Not that many searches on UK Freeview where you'd need a non-ASCII character.

    The only one I can think of in the last few years was "Les Misérables", and that was OK. Well, it was OK for me - I've just looked again at the second image in #1.

    Although the real oddity there is why BBC ONE HD was showing Baptiste, while BBC ONE London was showing "Les Misérables",
    London does not have a HD BBC channel.

  7. #7
    abu baniaz's Avatar
    Title
    Moderator
    Join Date
    Sep 2010
    Location
    East London
    Posts
    23,339
    Thanks
    6,422
    Thanked 9,146 Times in 6,224 Posts
    Quote Originally Posted by birdman View Post
    The only one I can think of in the last few years was "Les Misérables", and that was OK. Well, it was OK for me.
    You have a bug thread for this.

    Thread below is not publically viewable
    https://www.world-of-satellite.com/s...ebIF-interface

  8. #8
    birdman's Avatar
    Title
    Moderator
    Join Date
    Sep 2014
    Location
    Hitchin, UK
    Posts
    7,771
    Thanks
    235
    Thanked 1,656 Times in 1,305 Posts
    Quote Originally Posted by abu baniaz View Post
    London does not have a HD BBC channel.
    No, but its SD one shows as BBC London and, except when regional news is being broadcast, it broadcasts the same programmes as the HD channel. So it shouldn't have been broadcasting Les Misérables at 9pm on Sun 24 Feb 2019.
    MiracleBox Prem Twin HD - 2@DVB-T2 + Xtrend et8000 - 5(incl. 2 different USBs)@DVB-T2[terrestrial - UK Freeview HD, Sandy Heath] - LAN/USB-stick/HDD

  9. #9
    birdman's Avatar
    Title
    Moderator
    Join Date
    Sep 2014
    Location
    Hitchin, UK
    Posts
    7,771
    Thanks
    235
    Thanked 1,656 Times in 1,305 Posts
    Quote Originally Posted by abu baniaz View Post
    You have a bug thread for this.
    So I do.
    However, that stated that the on-screen EPG was fine and AutoTimers worked OK - it was just the EPG view in OpenWebIF that was wrong.
    FWIW, the name in the recording of the programme as seen via OpenWebIF is OK.
    MiracleBox Prem Twin HD - 2@DVB-T2 + Xtrend et8000 - 5(incl. 2 different USBs)@DVB-T2[terrestrial - UK Freeview HD, Sandy Heath] - LAN/USB-stick/HDD

  10. #10
    EnoSat's Avatar
    Title
    Member
    Join Date
    Mar 2013
    Location
    Slovakia
    Posts
    57
    Thanks
    10
    Thanked 10 Times in 9 Posts
    use two-bit charset in epg , edit your encoding.conf

  11. #11
    birdman's Avatar
    Title
    Moderator
    Join Date
    Sep 2014
    Location
    Hitchin, UK
    Posts
    7,771
    Thanks
    235
    Thanked 1,656 Times in 1,305 Posts
    Quote Originally Posted by EnoSat View Post
    use two-bit charset in epg , edit your encoding.conf
    The problem is with what the broadcaster sends.
    It's difficult to detect an incorrect, but valid, string without knowing what it should be in the first place and, if you knew that, there's be no need to look at the incorrect one at all.
    MiracleBox Prem Twin HD - 2@DVB-T2 + Xtrend et8000 - 5(incl. 2 different USBs)@DVB-T2[terrestrial - UK Freeview HD, Sandy Heath] - LAN/USB-stick/HDD

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
This website uses cookies
We use cookies to store session information to facilitate remembering your login information, to allow you to save website preferences, to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners.