General advise required on memory use and maybe EPG/CrossEPG flushing

[meggiedude]

Lo.

Been struggling lately with Vu+ Duo running very slow.
Hardly any free memory.
It got to the point where I could not load the latest VX OE3.0 update and had to go back to an old OE3.0 restore (build 560'ish IIRC).

But even that was slow today.

Using Crossepg with OpenTV for the 28.2E UK epg and the rytec feeds for the German and Italian epg.
Don't know if thats the best setup - but its what I've been using. It updates at 4:30am each morning.

Looking on the pendrive (where the picons and crossepg directories are saved to) the crossepg.descriptors.db was ~80MB.
So deleted Crossepg directory and reloaded it.

Seems quicker now - but for how long i don't know.
Have been using swapfile (256Mb) for a while now to see if this will help things.

Is there general guidance on how best I can setup the epg for these sats and I can make best use of the limited memory avail.
BTW I have 4 plugins including bbc-iplayer and itvplayer loaded - but nothing else.
Also is it advisable to flush the epg every now and then to help things (via say a cron??).

Cheers

[Larry-G]

Looking on the pendrive (where the picons and crossepg directories are saved to) the crossepg.descriptors.db was ~80MB.
So deleted Crossepg directory and reloaded it.

Cheers

Without wanting to take this thread off topic, this is the exact reason why we always recommend against using internal flash for EPG as it corrupts easily then as you discovered the it can grow huge in a very little time frame.

any way just thought i should point that out.

[Rob van der Does]

When using Cross on the DUO it's a good idea to use a swap-file.
And maybe the CacheFlush plugin might help as well.

[IAmATeaf]

If crossepg did download to internal then I think stores the files in /etc/enigma2/crossepg on VoX so have a look and check the contents and remove.

[meggiedude]

When using Cross on the DUO it's a good idea to use a swap-file.
And maybe the CacheFlush plugin might help as well.
Thanks.

What does this plugin do - ie which stuff does it remove and from where?

[Larry-G]

if you mean CacheFlush it does pretty much that. it flushes the Cache ( the RAM ) it's a temporary measure to release memory and can help on boxes with low amounts of ram, or at least thats how i understand it.

[meggiedude]

if you mean CacheFlush it does pretty much that. it flushes the Cache ( the RAM ) it's a temporary measure to release memory and can help on boxes with low amounts of ram, or at least thats how i understand it.
OK, so it just does stuff to the RAM.
I can do that, but what I am after if a regular way to flush the pendrive based Crossepg directory, so the Ram doesn't get filled too much in the first place.
So is a cron the best way for this? Say before the 4:30am crossepg download.

[Rob van der Does]

I can do that, but what I am after if a regular way to flush the pendrive based Crossepg directory, so the Ram doesn't get filled too much in the first place.
I wouldn't worry about that, unless you use a 32MB stick or so.......
You said
Been struggling lately with Vu+ Duo running very slow.
Hardly any free memory.so that's a RAM-issue, not flash.

BTW: Not sure how you measure low memory (it's free + cached + buffered).

[meggiedude]

I wouldn't worry about that, unless you use a 32MB stick or so.......
You said so that's a RAM-issue, not flash.

BTW: Not sure how you measure low memory (it's free + cached + buffered).
The point about free memory is fair comment. Just its reacting like there's no Ram, but I take your point.
The flash has 33.5M free so that's not an issue.

You say I wouldnt worry about the pen drive issue, but the point is it keeps going slow and I can't use the latest build. The pen drive is 2gb btw, so not a issue.
If the crossepg files are getting too big or corrupt as has been suggested then wouldn't removing the directory via a cron (or equiv) every few days be the best option wouldn't it?
What am I missing?
:confused:

[Larry-G]

The point about free memory is fair comment. Just its reacting like there's no Ram, but I take your point.
The flash has 33.5M free so that's not an issue.

You say I wouldnt worry about the pen drive issue, but the point is it keeps going slow and I can't use the latest build. The pen drive is 2gb btw, so not a issue.
If the crossepg files are getting too big or corrupt as has been suggested then wouldn't removing the directory via a cron (or equiv) every few days be the best option wouldn't it?
What am I missing?
:confused:

not really as removing the epg files while enigma is running would do nothing as it's held in live memory, it would therefore re-write the corrupt epg files when the box shuts down next.

flushing the cache would be the best thing to try.

[meggiedude]

Hmm, still struggling to grasp this.
Bearing in mind I've reflashed the machine a few times now and redownloaded the crossepg via the menus - I assumed this would rewrite the usb crossepg directory, and everything would be hunky dory. Yet this afternoon I found an 80MB crossepg.descriptors.db file (which I have since removed) But you are suggesting this deletion was a waste of time?
So are you saying that I have still got corrupt epg data in flash after a rebuild?? I am finding that difficult to comprehend.

Second stupid question :D :
Duo is spec'd at 384 RAM and 128 Flash, yet Ram on box shown at total 138380 kb (in my case with most of this used). Is this right?

[Larry-G]

Hmm, still struggling to grasp this.
Bearing in mind I've reflashed the machine a few times now and redownloaded the crossepg via the menus - I assumed this would rewrite the usb crossepg directory, and everything would be hunky dory. Yet this afternoon I found an 80MB crossepg.descriptors.db file (which I have since removed) But you are suggesting this deletion was a waste of time?
So are you saying that I have still got corrupt epg data in flash after a rebuild?? I am finding that difficult to comprehend.

If you have re flashed the system then no traces of the epg would remain in the ram as thats wiped during the flashing process.

What I was saying is that the EPG is held in live memory and only saved to or read from the storage device at shutdown, boot up and during the actual epg download. This is also why you must kill enigma before deleting the epg as it will just rewrite the files from memory, which is why a cache flush may be helpful to you.



Second stupid question :D :
Duo is spec'd at 384 RAM and 128 Flash, yet Ram on box shown at total 138380 kb (in my case with most of this used). Is this right?

Linux works very differently to windows in that it considers unused memory as wasted memory. Linux systems will always use as much memory as possible freeing up portions when requested for other tasks.

Personally id look at your exact EPG setup as it may be that your downloading way too much at once.


Sent from my GT-I9300 using Tapatalk 2

[Rob van der Does]

1- The available memory (RAM) you see, is the amount available in userspace. The RAM that has been reserved for the system doesn't show.
2- The Cross db can be deleted on the spot (i.e. while Enigma is alive). But up to now I've never ever heard of problems in that area.
What can give a problem is a corrupt epg.dat, and that does happen sometimes. And when located on USB or HDD it will survive a systemflash and be re-used. To delete that file, enigma should be stopped first.
3- A Cross EPG import is very memory (i.e. RAM) hungry. If you notice the system slowing down, that might be a sign of arisingmemory problems.
As I said before, activating a swapfile and using CacheFlush (set to 30 mins or so) might be a good idea for the DUO.

[meggiedude]

OK, Gents.

Thanks both of you for your input here.

So, should the need arise, in terms of removal of the EPG off the USB, and for that matter - effectively to stop any "bad" EPG data being rewritten to the USB from memory, the best way is to, say:


init 4???
Remove EPG data off USB ??
Init 3 to restart Enigma2???
Re-download EPG


As I say I have the Cacheflush loaded and will continue to monitor. Swapfile has been there for a while and does seem to get used (a little)

In terms of the best way to get the EPG data I want (with minimum Memory issues) : I am looking to have the UK epg (as Priorty 1) - I currently get this from the Opentv feeds.
and also idealy have the Italian and German EPGs (equal Priory 2) - I currently get these from the Rytec feeds.

So is crossepg with these Opentv/Rytec options the best option for me, or is there better options, bearing in mind the poor little Duos tiny memory lol.
If your requirements were as above, aside from buying a new receiver :D , how would you configure it.

Thanks for all this btw - I am learning much.

[Rob van der Does]

init 4???
Remove EPG data off USB ??
Init 3 to restart Enigma2???
Re-download EPG

Correct.




Swapfile has been there for a while and does seem to get used (a little)
That's correct. If you have a current image and activate the swap via the swapfile manager 'swappiness' will be set to zero. This means that the swap will only be used when needed.




So is crossepg with these Opentv/Rytec options the best option for me, or is there better options, bearing in mind the poor little Duos tiny memory lol.
If your requirements were as above, aside from buying a new receiver :D , how would you configure it.

No: I would (and do) use XMLTV importer for the Rytec sources. This importer works far more efficient then Cross and is less memory hungry.
So best is to use both (Cross for OpenTV and XMLTV for all other), but take care that they are set to different times (doing the import simultaneous would probably fill E2).

[meggiedude]

No: I would (and do) use XMLTV importer for the Rytec sources. This importer works far more efficient then Cross and is less memory hungry.
So best is to use both (Cross for OpenTV and XMLTV for all other), but take care that they are set to different times (doing the import simultaneous would probably fill E2).
Ahh, OK. Maybe this is where I was doing it wrong. Was using crossepg for it all - and all in one go at 4:30am.

So will have a look at splitting this as you suggest when I return home.
Ta.