wget from github works on command line but failes when called as .sh

[Daviemck]

I have a line of code which is downloading a file from my github. When I test this from command line it works as expected, but when I call this from a .sh from inside my plugin I get the following error

wget: error getting response: reset by peer.

I call the script like this:

/bin/busybox ash ./tmp/getfiles.sh

if I run the same script from command line it works. Can anyone shed any light? Is my script being called differently when being called from the plugin from a command line?

Thanks in advance

[Huevos]

If it is a plugin why are you using wget? Why aren't you using python built in methods?

[Daviemck]

If it is a plugin why are you using wget? Why aren't you using python built in methods?

Most likely as I'm learning as I go along, I'm certainly no master with python, but when I've tested python commands equivalent to wget I don't see a progress bar or anything similar (which from a user experience can appear to be that nothing is happening, depending on the download size), I'm running a sub-routine in the plugin which uses a combination of python and shell script; i've found myself use os. and os.system so frequently in python, that it's just easier for me to drop to a script to do it. But if you can suggest a way I could perhaps do this, I'll happily give it a try.

[birdman]

Most likely as I'm learning as I go along, I'm certainly no master with python, but when I've tested python commands equivalent to wget I don't see a progress bar or anything similar (which from a user experience can appear to be that nothing is happening, depending on the download size),

So you're expecting stdout/stderr from a subprocess to get displayed on the screen? Does it??

I call the script like this:

/bin/busybox ash ./tmp/getfiles.sh

Don't.
The first line should be:

#!/bin/sh

and the file should have its execute bits set. Then just run:

/tmp/getfiles.sh

However, you do have to consider where stdin, stdout and stderr are. When run from the command line they are all your tty. When run from enigma2 they are not. stdout and stderr are (IIRC) running to the debug log (if the user has one). stdin will run from /dev/console (as will stdout and stderr with no debug log). This is probably not what you want.

[Huevos]

Should be using urllib.

Also, you can push anything to the display as you go along. E2 has a built in progress bar. You just need to call it from your plugin.

[Daviemck]

So you're expecting stdout/stderr from a subprocess to get displayed on the screen? Does it??

I'm not sure what those are, but when you call the code as I have you see a command line window and every output to screen that you would see had you run it from the command line - Don't ask me how, I've just looked through how others have achieved this and worked with it.

The first line should be:

#!/bin/sh

and the file should have its execute bits set. Then just run:

/tmp/getfiles.sh

I think you are confusing the code of the .sh file (which has that in it) to how I'm calling it from within python - I only quoted one line from the .sh file, not the entire file. The rest of the .sh script runs fine of which there are over 100 lines, it's just this one line that's being problematic

However, you do have to consider where stdin, stdout and stderr are. When run from the command line they are all your tty. When run from enigma2 they are not. stdout and stderr are (IIRC) running to the debug log (if the user has one). stdin will run from /dev/console (as will stdout and stderr with no debug log). This is probably not what you want.

OK, WAY too confusing, if there is not an easy answer to this perhaps I'll find another way. If I had to guess I'd say that when executed from within the plugin, the .sh is being executed with different privileges or different access

[Daviemck]

Should be using urllib.

Also, you can push anything to the display as you go along. E2 has a built in progress bar. You just need to call it from your plugin.

OK Thanks, I can look up reference to urllib, but in the absence of a reference guide to E2, where do I found out more about how to call a progress bar?

[birdman]

I'm not sure what those are, but when you call the code as I have you see a command line window and every output to screen that you would see had you run it from the command line - Don't ask me how, I've just looked through how others have achieved this and worked with it.

Do you have a debug log enabled? If not, enable one and test what happens next.

I think you are confusing the code of the .sh file (which has that in it) to how I'm calling it from within python - I only quoted one line from the .sh file, not the entire file.

OK, then just make it executable and execute it. You've already told the script which handler to use.

...perhaps I'll find another way.

As had been noted, using urllib directly within python would be better.

If I had to guess I'd say that when executed from within the plugin, the .sh is being executed with different privileges or different access

No. In both cases it will be run by root (well, it certainly will if run from enigma2).

Also note that the error message indicates that that the connexion is being cleared at the server (remote) end.
So, have you looked at your github server logs to see why it is chopping the connexion?

Or perhaps you could post the script (as a zip file?

[Huevos]

You're not going to find tutorials on this stuff anywhere. You need to look at code in E2 to figure it out. Have a look in screens/servicescan.py for an example of progress bar.

for an example of urllib look in ABM updateproviders.py or yweather plugin.

Sorry I can't post a link from this device.

[Rob van der Does]

I have a line of code which is downloading a file from my github. When I test this from command line it works as expected, but when I call this from a .sh from inside my plugin I get the following error

wget: error getting response: reset by peer.

I call the script like this:

/bin/busybox ash ./tmp/getfiles.sh

if I run the same script from command line it works. Can anyone shed any light? Is my script being called differently when being called from the plugin from a command line?

Have a look at the attached plugin: it has a build in updater (in the plugin folder), which does something along the lines as you're trying (if I understood you correctly).

[Daviemck]

I'm just going to take a minute to reset this discussion.

During early development I was storing the files to download on an internet based web server - Using, as I mention above, wget as part of a lengthy script that runs and everything works exactly as expect. The ONLY change I've made is moving the files I need to download to github (and obviously updating the location in the script) and now I get the error reset by peer. I did a quick bit of testing with urllib earlier today and that writes a file of 162k - I've explored the file and that is an http error 403 (forbidden) response. This is ONLY happening when executed from within a python environment. So the only thing I can ascertain from this is must be changing something like the user agent as it requests data.

It was mentioned above that I should check the logs on github - I can't see anyway of doing that.

Rob van der Does, I'll check out that plugin, thanks

[abu baniaz]

When Pembo wrote script to download ABM config file, it would not work with GitHub.



When Huevos did so with Python, works fine.



Sent from my XT1032 using Forum Fiend v1.3.3.

[Huevos]

Can you give us the URL to try.

[birdman]

I did a quick bit of testing with urllib earlier today and that writes a file of 162k - I've explored the file and that is an http error 403 (forbidden) response.

162k for a 403 response?
Was there any other useful info in that?

This is ONLY happening when executed from within a python environment. So the only thing I can ascertain from this is must be changing something like the user agent as it requests data.

Well, python's urllib isn't go to send the same UA as wget.
But wget run from within python would, as it's still wget.

It was mentioned above that I should check the logs on github - I can't see anyway of doing that.

Probably not if it's on github - I though you might be running your own server.

[dsayers]

When downloading from github within python use raw i had similar issues.