EPG Translator (Discussion thread)

[twol]

Anyone else having an issue with translator Plugin?
It used to translate straight away, now it take forever.
Running openvix 5.3.039 edision os mega
Tried running it on openvix 5.4.003 and it won't even open just crashes the receiver

Crash log??

[malcolmp]

Hi,
Yes I'm having the same problem, on three boxes, running openvix 5.4.003 on two and openatv on the other (although none of them crash)

[Willo3092]

Seems to be a problem with the url and control characters

<734133.0304> 09:57:41.3081 [Screen] Showing screen 'InfoBarSummary'.
<734133.0393> 09:57:41.3170 [Skin] Parsing embedded skin '<embedded-in-translatorMain>'.
<734133.0400> 09:57:41.3177 [Skin] Processing screen '<embedded-in-translatorMain>', position=(460, 80), size=(1000 x 610) for module 'translatorMain'.
<734133.0506> 09:57:41.3283 Traceback (most recent call last):
<734133.0507> 09:57:41.3284 File "/usr/lib/enigma2/python/mytest.py", line 239, in processDelay
<734133.0509> 09:57:41.3286 callback(*retval)
<734133.0510> 09:57:41.3287 File "/usr/lib/enigma2/python/Screens/InfoBarGenerics.py", line 2785, in extensionCallback
<734133.0511> 09:57:41.3288 File "/usr/lib/enigma2/python/Tools/BoundFunction.py", line 9, in __call__
<734133.0512> 09:57:41.3289 File "/usr/lib/enigma2/python/Screens/InfoBarGenerics.py", line 1661, in runPlugin
<734133.0513> 09:57:41.3290 File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1687, in main
<734133.0514> 09:57:41.3291 File "/usr/lib/enigma2/python/mytest.py", line 334, in open
<734133.0516> 09:57:41.3294 dlg = self.current_dialog = self.instantiateDialog(screen, *arguments, **kwargs)
<734133.0517> 09:57:41.3294 File "/usr/lib/enigma2/python/mytest.py", line 271, in instantiateDialog
<734133.0520> 09:57:41.3297 return self.doInstantiateDialog(screen, arguments, kwargs, self.desktop)
<734133.0521> 09:57:41.3298 File "/usr/lib/enigma2/python/mytest.py", line 301, in doInstantiateDialog
<734133.0523> 09:57:41.3300 dlg.applySkin()
<734133.0524> 09:57:41.3301 File "/usr/lib/enigma2/python/Screens/Screen.py", line 239, in applySkin
<734133.0525> 09:57:41.3302 File "/usr/lib/enigma2/python/Screens/Screen.py", line 270, in createGUIScreen
<734133.0526> 09:57:41.3303 File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1324, in onLayoutFinished
<734133.0526> 09:57:41.3304 File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1459, in getEPG
<734133.0527> 09:57:41.3304 File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1480, in showEPG
<734133.0528> 09:57:41.3305 File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1403, in translateEPG
<734133.0529> 09:57:41.3306 File "/usr/lib/python2.7/urllib2.py", line 155, in urlopen
<734133.0530> 09:57:41.3307 File "/usr/lib/python2.7/urllib2.py", line 430, in open
<734133.0531> 09:57:41.3308 File "/usr/lib/python2.7/urllib2.py", line 448, in _open
<734133.0532> 09:57:41.3309 File "/usr/lib/python2.7/urllib2.py", line 408, in _call_chain
<734133.0533> 09:57:41.3310 File "/usr/lib/python2.7/urllib2.py", line 1229, in http_open
<734133.0534> 09:57:41.3311 File "/usr/lib/python2.7/urllib2.py", line 1196, in do_open
<734133.0535> 09:57:41.3312 File "/usr/lib/python2.7/httplib.py", line 1058, in request
<734133.0536> 09:57:41.3313 File "/usr/lib/python2.7/httplib.py", line 1092, in _send_request
<734133.0537> 09:57:41.3314 File "/usr/lib/python2.7/httplib.py", line 945, in putrequest
<734133.0538> 09:57:41.3315 httplib.InvalidURL: URL can't contain control characters. '/m?hl=en&sl=auto&q=Criminal%20Minds:%20Beyond%20Bor ders%207515%20Im%20Namen%20der%20Hoffnung%20(2016) %20[12+]%207515%20Kate%20und%20Peter%20sind%20in%20Casabla nca%20entf\xc3\xbchrt%20worden.%20Die%20Verbrechen %20verlangen%20400.000%20Dollar%20L\xc3\xb6segeld. %20Jack%20Garrett%20und%20sein%20Team%20finden%20h eraus%20dass%20einer%20der%20Reisef\xc3\xbchrer%20 Gesch\xc3\xa4fte%20mit%20einem%20H\xc3\xa4ndler%20 im%20Souk%20machte%20bei%20dem%20die%20Entf\xc3\xb chrten%20zuletzt%20gesehen%20wurden.%20Doch%20dies e%20Spur%20f\xc3\xbchrt%20ins%20Leere.%20Bei%20der %20Recherche%20nach%20fr\xc3\xbcheren%20Entf\xc3\x bchrungen%20von%20Touristen%20st\xc3\xb6sst%20Mont y%20auf%20einen%20Stamm%20der%20Tuareg.%207514%20A lec%20Smight%207514%20Gary%20Sinise%207514%20Alana %20De%20La%20Garza%207514%20Daniel%20Henney%207514 %20Tyler%20James%20Williams%207514%20Annie%20Funke %207514%20Wendy%20Phillips%207514%20Tony%20Pasqual ini%207514%20Michael%20Benyaer%207514%20Ali%20Hill is%207514%20Kelly%20Frye%207514%20Peter%20Banifaz% 207514%20Rachid%20Sabitri%207514%20Maz%20Siam%2075 14%20Fouad%20Hajji%207514%20Wiley%20M.%20Pickett%2 07514%20Zuhair%20Haddad%207514%20Marwan%20Salama%2 07514%20Ezra%20Dewey%207514%20Declan%20Whaley' (found at least '\xc3')
<734133.0539> 09:57:41.3316 [ePyObject] (CallObject(<bound method Session.processDelay of <__main__.Session instance at 0xb1416e68>>,()) failed)


dmesg

[abu baniaz]

opkg install enigma2-plugin-extensions-translator

Same for me

11:34:27.4187 Traceback (most recent call last):
11:34:27.4188   File "/usr/lib/enigma2/python/Components/ActionMap.py", line 57, in action
11:34:27.4191   File "/usr/lib/enigma2/python/Screens/PluginBrowser.py", line 159, in save
11:34:27.4192   File "/usr/lib/enigma2/python/Screens/PluginBrowser.py", line 163, in run
11:34:27.4193   File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1687, in main
11:34:27.4195   File "/usr/lib/enigma2/python/mytest.py", line 334, in open
11:34:27.4197     dlg = self.current_dialog = self.instantiateDialog(screen, *arguments, **kwargs)
11:34:27.4199   File "/usr/lib/enigma2/python/mytest.py", line 271, in instantiateDialog
11:34:27.4201     return self.doInstantiateDialog(screen, arguments, kwargs, self.desktop)
11:34:27.4201   File "/usr/lib/enigma2/python/mytest.py", line 301, in doInstantiateDialog
11:34:27.4203     dlg.applySkin()
11:34:27.4204   File "/usr/lib/enigma2/python/Screens/Screen.py", line 239, in applySkin
11:34:27.4205   File "/usr/lib/enigma2/python/Screens/Screen.py", line 270, in createGUIScreen
11:34:27.4206   File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1324, in onLayoutFinished
11:34:27.4207   File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1459, in getEPG
11:34:27.4208   File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1480, in showEPG
11:34:27.4209   File "/usr/lib/enigma2/python/Plugins/Extensions/EPGTranslator/EPGTranslator.py", line 1403, in translateEPG
11:34:27.4210   File "/usr/lib/python2.7/urllib2.py", line 155, in urlopen
11:34:27.4212   File "/usr/lib/python2.7/urllib2.py", line 430, in open
11:34:27.4213   File "/usr/lib/python2.7/urllib2.py", line 448, in _open
11:34:27.4213   File "/usr/lib/python2.7/urllib2.py", line 408, in _call_chain
11:34:27.4214   File "/usr/lib/python2.7/urllib2.py", line 1229, in http_open
11:34:27.4216   File "/usr/lib/python2.7/urllib2.py", line 1196, in do_open
11:34:27.4217   File "/usr/lib/python2.7/httplib.py", line 1058, in request
11:34:27.4218   File "/usr/lib/python2.7/httplib.py", line 1092, in _send_request
11:34:27.4219   File "/usr/lib/python2.7/httplib.py", line 945, in putrequest
11:34:27.4221 httplib.InvalidURL: URL can't contain control characters. '/m?hl=en&sl=auto&q=Nur%20in%20Sky%20Sportsbars:%20UEFA%20Europa%20League%202020/2021%20:%20Roter%20Stern%20-%20Hoffenheim%207515%20%20%207515%20Action%20vom%20UEFA%20Europa%20League-Spiel%20Roter%20Stern%20gegen%20Hoffenheim.%20N\xc3\xa4here%20Infos%20zu%20DAZN%20Inhalten:%20Jetzt%20auf%20www.dazn.com' (found at least '\xc3')
11:34:27.4221 [ePyObject] (CallObject(<bound method ActionMap.action of <Components.ActionMap.ActionMap instance at 0xaef2da30>>,('WizardActions', 'ok')) failed)

[abu baniaz]

Version 1.4 also crashes.

There is a httplib.py file uploaded that is meant to avoid the crash

https://www.linuxsat-support.com/thread/137709-epg-youtube-trailer-translator-plugin/?postID=572506#post572506

[birdman]

Seems to be a problem with the url and control characters

Except they're not control characters. These are utf-8 characters.

\x3\xbc is ü
\x3\xb6 is ö

So there's a character-set problem somewhere.

[ccs]

There's something similar on OpenPLi, can't find the commit it refers to.....

I know the problem. Unfortunately twisted added a security check whether URLs are valid. For them valid means no special character is included. Means a URL with ä,ü,ö for example is invalid, even if normal browsers can handle it

I thought we had reverted the commit here. But it seems we haven't. See what we can do...

https://forums.openpli.org/topic/79889-openpli-release-candidate-80-available/?view=findpost&p=1281721

[gilvey]

so is there a quick fix for this and if so how? or will a fix be implemented in the next openvix update. would rolling back to an older version of openvix solve anything?

[birdman]

I know the problem. Unfortunately twisted added a security check whether URLs are valid. For them valid means no special character is included. Means a URL with ä,ü,ö for example is invalid, even if normal browsers can handle it

Or encode all bytes in the url > 0x7f as %xx?

[ccs]

Or encode all bytes in the url > 0x7f as %xx?

Maybe this is the commit/revert I was looking for.....

https://github.com/OpenPLi/openpli-oe-core/blob/2c04c797461152d7d3383dcf8c4b3bbf2fdbce3a/meta-openpli/recipes-devtools/python/python-twisted/0001-Revert-Prevent-CRLF-injections-described-in-CVE-2019.patch

[twol]

That fix is on OE-A 4.4 unfortunately

[simonc]

If it's ceased to function on 5.3, then, I'd suspect that google have changed something (assuming it uses Google translate) and the plugin isn't getting a response.

As for 5.4, it seems that Python itself has had a security patch of some sort between 5.3 and 5.4, and that means the plugin should now be URL encoding anything it sends out as a query.

Either way, the plugin author needs to do some work. You might find the patch Abu mentioned on the linuxsat forum helps while you're waiting. If you do patch httplib.py from that link, don't believe the poster thgat says that it's "ATV's stupid changes": This has been triggered by the people that maintain Python itself. The plugin should actually have been doing the URL encoding properly in the first place, but has been getting away with it until now.

[ccs]

That fix is on OE-A 4.4 unfortunately

I thought that's what ViX was now based on?

This, I think, is the original commit in OE-Core 4.4

https://github.com/oe-alliance/oe-alliance-core/commit/54b7305219fdf7e372678637cba3f6f6ad27b42f

[Willo3092]

Either way, the plugin author needs to do some work.

There was mention on another forum that the plugin author is now deceased. Not sure if it's true or not

[ccs]

.... and a "temporary" fix here...

https://github.com/oe-alliance/oe-alliance-core/commit/eb924b4b2d3ef5a0c6e27f8fc1b85b32d638ce5e#diff-0a83f8bbd65fe05f23213473c86a6d79909d86cc0128ea99491b332be550e6de