View Full Version : [ViX_Misc] EPG corrupt - can't work out why it keeps happening
Hi All,
I have this recurring issue with the EPG getting corrupt and then quite often, when I reboot the EPG is empty until I run my CrossEPG and EPGRefresh.
FYI I have CrossEPG and RefreshEPG setup as follows:
EPG General settings - dedicated USB drive formatted by box (ext), EPG set to save to file epgupp.epg on USB stick, Save EPG set to every "1 hour", refresh every set to "3 hours"
PowerTimer - 05:50hrs - biring box out of deep standby to Standby - at end do nothing (i.e. leave box in standby).
EPGRefresh - Refresh automatically at 06:00hrs Fake record one or two T2 channels for the Freeview EPG, duration 180,
CrossEPG - set to run once per day at 06:12hrs - configured provider - Sky UK OpenTV onoy (no XMLTV or scripts)
By and large this is working very well and when I look at the EPG at say 8am it is fully populated and looking good. At some point during the day however things start to go wrong and the EPG seems to be getting corrupted. when I roboot the box the box will sometimes come back up and the EPG is loaded and all is good. Sometimes though (at least once per day so really annoying) it comes back and the EPG is totally empty. I then have to go through the motions of running crossepg and EPGRefresh manually. Once done the EPG comes back nicely. Sometimes I can do a reboot right after and it is all gone again. I have even gone into the EPG settings and done a "save" right after manually running the CrossEPG and EPGRefresh.
Something somewhere and somehow is knackering my EPG data and I just cant get to the bottom of it. Can some kind soul take pity on me and have a look at my logs so I can try to get this fixed? I have tried initially using the 1Tb HDD to store the EPG and recently added the dedicated USB stick to see if that solved the issue. It didn't. I just can't work out why the EPG file is getting so frequently corrupted. I have been careful to use only 1 provider for 28.2 and one provider (the channel zap/record (either manually or by using EPGRefresh) for the T2 channels but even so it is still going bad.
I would prefer to stick to the CrossEPG and the OpenTV data (rather than Ritec) and rather than other IPK's that I know are available as I like CrossEPG and I can run a manual scan and get late changes to the EPG as I want when I want nice and easy. I don't think its crossEPG that is causing the corruption as it works well - it is only later on in the day that the corruption seems to happen.
I do suspect there is a bug somewhere here (either with the Zgemma H7S or Enigma2 or openVIX but don't have the skills to analyse any deeper).
For the purposes of debugging I have just rebooted my box (15:09 on 12/4/2020), the EPG was empty after reboot and the /media/usb/epgupp.dat file was missing from the usb stick. I then ran (at 15:14) a manual CrossEPG. I then went into settings and did a SAVE in the EPG menu and this created the epgupp.dat file. The EPG at this point was all present and correct (as I also zapped to a T2 channel to populate the Freeview EPG entries.
I then did a reboot to see if the EPG survived and ..... It was fine.
In the log prior to all of this (rather large log) there are multiple EPG is corrupt messages but no idea from my side as to why and what would have corrupted it. I have attached that log below.
59903
This has been happening for ages and I have tried all sorts of changes, tweaks etc.
Thanks all,
Paul
Andy_Hazza
12-04-20, 16:41
Why are you using both? As been reported previously, CrossEPG has bugs. You just need to use EPGRefresh and just use a fake recording on channel Iepg which is in the Sky Info section of the Bouquets created by ABM. I use Sky FTA Only setup.
Sent from my iPhone using Tapatalk
I like crossEPG and it does seem to work okay. I didn't realise that crossEPG had bugs.
I did used to use just EPGRefresh and I did do a fake recording of "iEPG data 1" channel and it did not seem as up-to-date as the CrossEPG openTV data. It might well have been placebo effect but when all of the program changes take place on BBC1 for the Coronavirus update and Pointless gets shoved over to BBC2 a CrossEPG manual update seems to pick the changes up by about 2pm the same day but when I tried the EPGrefresh fake recording of "iEPG data 1" it did not seem to reflect the changes. Also I was having great problems with channel 4 HD not getting populated with EPGrefresh on "iEPG data 1" fake recording where CrossEPG always populates it.
I'm assuming that the OpenTV data source is not one and the same thing as "iEPG data 1" but if it is then I admit that is a bit confusing.
Anyway - thats why I went with CrossEPG.
also - I thought the issue was with CrossEPG and EPGImporter (ref Abu Baniz in thread https://www.world-of-satellite.com/showthread.php?56969-EPG-missing-after-reboot)
I did not realize it was a problem if it was used alone (which in essence mine is as the don't use the EPGImport for satellite update.
Why are you using both? As been reported previously, CrossEPG has bugs. You just need to use EPGRefresh and just use a fake recording on channel Iepg which is in the Sky Info section of the Bouquets created by ABM. I use Sky FTA Only setup.
Sent from my iPhone using Tapatalk
:-) - I can also see that you have been around this issue for a long time so please don't think I'n disagreeing with anyting you are saying - I have 5 minutes experience of all of this and you have tons :-)
If you really thing that iepg data is good enough and the best and stable then I will swap to that - I am just a bit miffed that the data does not seem very up-to-date compared to CrossEPG and openTV. That's all I was saying really ha ha .....
Paul
Andy_Hazza
12-04-20, 17:08
Channel 4 populates fine here using EPGRefresh.
You can always quickly update your EPG anytime by just going onto the Iepg 1 channel and leaving for 2mins and it fully populates again.
But my setup works fine, if anything CrossEPG/OpenTV struggles to get Channel 4 epg, but since using EPGRefresh worked flawlessly.
Some peeps just don’t like change, but unfortunately you sometimes have to move on and sadly you need to move on from CrossEPG/OpenTV.
Sent from my iPhone using Tapatalk
Its all a bit confusing
In post #23 in thread https://www.world-of-satellite.com/showthread.php?59312-Channel-4-HD-Missing-EPG/page2 you mentioned on 1/April that you are using "Sky UK FTA setup in ABM (AutoBouquetsMaker) & CrossEPG (OpenTV) and works fine for me on my varying boxes". so can I ask what's changed your mind since 1st April to today - is there some more info. I am missing (threads) that I need to read :-)
Also in thread https://www.world-of-satellite.com/showthread.php?62466-How-do-I-get-epg-to-populate-HD-versions-ala-CH4-HD-etc&highlight=crossepg Abu Baniaz mentions in post #3 that "With regards to obtaining EPG data for it, there are several processes to do it.
The easiest is to use CrossEPG downloader"
Confused DOT com
Like I said - all I really want to do is:
1. Get late BBC changes to be picked up (say at 3pm using a manual scan)
2. Get C4 HD consistently populated
3. Not have a corrupted EPG
I have tried EPGImport and tried EPGRefresh and tried CrossEPG and CrossEPG seemed to be the most up-to-date and always had C4 HD but........ I am getting some corruption somehow.........
If it's crossEPG then I will ditch it. I just want to be sure it is really the culprit in this case (with no double downloading taking place).
It's a bit of a minefield this :-(
Paul
Andy_Hazza
12-04-20, 17:28
Its all a bit confusing
In post #23 in thread https://www.world-of-satellite.com/showthread.php?59312-Channel-4-HD-Missing-EPG/page2 you mentioned on 1/April that you are using "Sky UK FTA setup in ABM (AutoBouquetsMaker) & CrossEPG (OpenTV) and works fine for me on my varying boxes". so can I ask what's changed your mind since 1st April to today - is there some more info. I am missing (threads) that I need to read :-)
That was on an older image just doing updates as & when available.
What I then did to test it fully was to do a complete fresh install with no settings backups, so setup as new and CrossEPG failed to pull in Ch4 data. So I removed CrossEPG from the plugins and then proceeded to use EPGRefresh and that’s worked ever since pulling in Ch4 epg data etc.
Sent from my iPhone using Tapatalk
Andy_Hazza
12-04-20, 17:32
Where are you storing your epg data? How many storage devices have you got mounted etc?
Sent from my iPhone using Tapatalk
I use CrossEPG for $ky without an issue.
I used EPGRefresh after Vix first introduced it in an earlier image and found odd little glitches in EPG which never (and still doesn't) exist in CrossEPG. If I remember, someone else mentioned it also.
Don't know what happened with all that as I moved back to CrossEPG/OpenTV where it just works, at least for me.
Hi,
Andy - Thanks for the update there..... I have 2x devices. 1x Internal HDD formatted by box (ext) and storing recordings etc and 1x USB formatted by box and storing only the EPG. I did originally not have the USB stick but added it in case storing the EPG on the internal HDD was causing an issue (some threads suggest this and other contradict and say not problems). As the issue is still there with the USB I will most likely get rid of it and just stick to the internal HDD.
I did (yesterday) disable CrossEPG and added the iEPG Data 1 to my EPGRefresh and let it run on schedule at 6am. I found that it was having a few gaps in the schedules (particularly BBC1 South SD, Sky news SD so I added these as extra services in EPGrefresh as well as BBC1 HD and BBC 1 SD Freesat channels (to populate them). I hope this works okay and will stick with it for a week or so to see if I get any new corruptions to the EPG with normal use) but I do miss the CrossEPG as it was quick and easy (took 3 minutes to run) whereas the EPGrefresh with 5 services set to 5 mins each takes 25 mins to run (should I want to do a manual scan to get the "late changes" to the schedules).
It will also be interesting to see if the EPGRefresh when manually run at 3pm picks up the late changes. Its easy for me to see the errors as I have a special Bouquet set-up with BBC1 HD, BBC1 HD (T), BBC 2 HD, BBC 2 HD (T) where the (T) entries are the Terrestrial. I can see immediately in the EPG of the Freeview and Satellite disagree on a given channel at a given time (they often do and the Freeview is always right). When I notice this difference (occuring near or at a program I am recording using Autotimer) I then run a refresh to get the satellite channels updated with the changes to the schedule so restoring the balance of the world for me (bit OCD I guess but it grates on my mind when I see BBC1 HD Sat showing a different program to BBC1 HD Freeview at 5:30pm just before my recording of the BBC News at 6pm). I have a suspicion that the EPGRefresh method may not be as up to date with the changes as CrossEPG (this is the reason I chose to go with CrossEPG after trying EPGRefresh with iEPG Data 1 for some weeks a while back).
If so I will have to choose - corrupted EPG from time-to-time - if indeed CrossEPG is causing the corruption (which I don't have any evidence for at this time) or out of sync channel listings using the EPGRefresh method.... I will see over the coming week if I get any corruptions to the EPG and if not then I might assume CrossEPG is the culprit. I will do lots of "edge" things to try and produce a corruption (delete recordings mid-flow etc.). Reason -- In the back of my mind I have a "feeling" that the corruption may have occurred when I was watching a show on BBC1 HD at Lunchtime and that show was also recording. I was also on the live service of the same channel and had paused the live broadcast 10 minutes later (after cooking lunch) I came back and while in timeshift I went into the "File List" and deleted the active recording (as I was watching it live in Timeshift so didn't need it). I selected the option "Stop recording and delete file" and then carried on watching the timeshifted version of the program. I just wonder if that caused an issue. I do these odd "edge" things from time-to-time so if there is a bug there then it may not come to light as it is not the common action to be doing. Probably totally not this but I just wanted to put it out there. Later that afternoon I noticed loads of the "corrupt EPG" messages in the debug logs so I put my thinking hat on and this is the only thing I personally did that was "out of the ordinary" - put 2 and 2 together and got 10 as it is most likely another background thing going on that caused the corruption.
BBBUK - Thanks for that feedback. I don't "feel" it is the CrossEPG causing the issue (Yet) either but can't really see anything in the logs to tell me what it was. Grrrrrrrrr........
Cheers,
Paul
Andy_Hazza
13-04-20, 10:43
Hi Paul,
Personally I would scrap the additional usb device and store your epg data to your HDD.
Regards,
Sent from my iPhone using Tapatalk
I agree Andy,
I have removed the USB stick and renamed the EPG to save to the hard drive and now called epgnew.dat
Before I did this I noticed that the EPG file was missing from the USB stick again and the EPG corrupt errors were back in the debug log. This was just using EPGRefresh this morning (automatic) and using the box to watch a few programs today and do a few pauses on live channels etc. Nothing out of the ordinary.
I still don't really know how to analyse how the EPG cache file is getting corrupt though.
Here is an extract of that log at the point of the first corrupt message (and the preceding few lines and a few lines after). I put a line space before and after the error to make it easier to find. It is at < 11402.712>
If anyone can shed any light I'd be really grateful
So. Some progress (in case anyone reads this).
I have renamed my EPG back to epg.dat and stored it on the internal HDD.
I have reverted back to using CrossEPG as it was not that which was causing the "EPG is Corrupt" messages in the logs (as it happened with onoy EPGRefresh) and also the EPGRefresh data was not as up-to-date as the openTV data from CrossEPG
I have run the script under the thread https://www.world-of-satellite.com/showthread.php?52890-Remove-EPG to delete the crossEPG data and the EPG file
Run a manual CrossEPG (OpenTV data only for 28.8) and a manual EPGRefresh (only does a fake recording of Freeview channel to populate Freeview EPG)
At that point all was well. The EPG data was good and it survivied a full reboot following a Save EPG in the menus
This is as it was and I will start from scratch and wait to see if any more EPG Corrupt messages appear in the logs. If they do I don't know how to analyse the cause so any pointers would be most appreciated.
Cheers,
Paul
I kind of feel I might be just talking to myself here and trying to debug a problem without the skills to do so ..... but anyway I will post my findings still here in case it helps someone else in the future.
I had the settings for Automatic Refresh every 2 hours and Automatic Save every hour under the EPG settings. I have a feeling that the action of Save/reload may be somehow corrupting the epg data in RAM thus leading to the
[eventData] EPG Cache is corrupt (eventData::~eventData), you should restart Enigma!
error after some hours. I read that this might be causing an issue in another thread https://www.world-of-satellite.com/showthread.php?45036-Memory-Leak
I have now disabled both of these options (as I only set them due to a lack of understanding on my part) and I now "think I understand" that they are not really needed in my case and that if I reboot the EPG data is sent to the file on the hard drive (epg.dat) and when it comes back up that data is read automatically anyway.
We will see if that stops the corruption of the EPG data in memory after some hours.
I'm used to talking to myself anyway at the moment :-)
Maybe this link will give you a clue.....
https://www.world-of-satellite.com/showthread.php?62439-Crash-at-end-of-recording-when-end-margin-starts&p=494288&viewfull=1#post494288
You're not talking to yourself ;) I've been following the thread with interest! I too had issue with EPGRefresh when CrossEPG was initially dropped from the image. The intention (as I understand it) was that the OpenTV reader was built in to the standard EPG handling in the DVB code in the same way as Freesat EPG handling (or Virgin EPG on cable or other EPG readers on other sat/terrestrial systems). The Freesat/Freeview EPG reader has worked well over the years and generally populates the EPG without any user intervention. The EIT data which actually contains the programme info is broadcast continuously on the main channels, so once you're tuned to a BBC/ITV/CH4 channel you will get the week's EPG data after about 20 minutes or so without any user intervention needed. The so-called "OpenTV" EPG used by Sky is carried only on a specific transponder, so you have to tune to it to download those specific SKY channels' EPG info. My understanding of CrossEPG was that it tuned to that specific transponder and stored the resultant data in its own database and then populated the EPG which is held in RAM on the receiver. The new OpenTV reader now built into the enigma DVB code reads the very same specific transponder and populates the EPG, so it should get the very same data as CrossEPG. The difference being that CrossEPG builds the data into its own database also as an interim step. Therefore, you should get exactly the same EPG data whether you use CrossEPG or the built-in reader! I think that what may happen sometimes is that CrossEPG's internal database can become corrupt and you may then subsequently get issues in the EPG info as presented on-screen. The only solution in that case may be to delete the actual CrossEPG folder and also the EPG file and start from clean.
When i started using EPGRefresh I had it set to do a "fake record" at a particular time of day and all worked well initially. Then I noticed that I was getting gaps in the schedule occasionally and missed a recording of F1 on CH4HD last Autumn at some stage. Other users have had no issues whatsoever with EPG Refresh on 28.2, so "your mileage may vary" as they say. What I ended up doing is uninstalling EPGRefresh, reinstalling CrossEPG and deleting any EPG and Cross files on my system and starting from clean. No issues with Cross since the start of this year. I have a combined terrestrial and satellite bouquet for my local Saorview (Freeview equivalent) and Freesat viewing. CH4HD is built into that bouquet at position 104. I have also a separate SKY FTA bouquet which would also contain CH4HD and other SKY FTA channels not carried on Freesat. I run CrossEPG at 04.30 and it fills in the CH4HD data which is not picked up by the built-in Freesat reader since the move back to SKY. Cross seems to work fine in combination with the standard EIT readers built into enigma in my case.
Long story short - I have standard EIT/Freesat readers enabled in EPG handling and OpenTV reader disabled. I run CrossEPG to gather SKY-specific EPG data not picked up by the built-in readers. I don't have any automatic load/save of the EPG enabled as it is held in RAM, except at system startup and shutdown where it is loaded or saved to disk.
Ha ha. I wondered if my snidey comment about talking to myself would get a reply.
Thanks for that fat-tony - sounds like "deja vous" for me..... The only difference is that I was using the EPG settings to save and re-read on a cycle. I definitely think it is that corrupting the EPG in memory. I have had it running for several hours now and thus far no corruption. This is looking very good :-)
The odd thing is that using the fake recording of iEPG Data 1 deffo seems to give less up to date "late changes" but if they are the very same source then that makes no sense and I must have been imagining it (quite possible as all of this is making my head hurt). In any case its definitely a lot quicker (CrossEPG) than zapping 3 or 4 channls using EPGRefresh (as I have to zap a few Freeview ones too unless I change the job each time I want to do a manual run. So as long I have no more corruption I am happy sticking with the EPGRefresh for the Terrestrial zapping and CrossEPG for the OpenTV data for Sky.
For the development folk. Would it be worth looking into why the EPG settings for Read and refresh would be causing a corruption of the EPG in RAM so consistently? It looks like it could be a bug.
Cheers,
Paul
I'll report back here if the new setup causes any corruptions
You're not talking to yourself ;) I've been following the thread with interest! I too had issue with EPGRefresh when CrossEPG was initially dropped from the image. The intention (as I understand it) was that the OpenTV reader was built in to the standard EPG handling in the DVB code in the same way as Freesat EPG handling (or Virgin EPG on cable or other EPG readers on other sat/terrestrial systems). The Freesat/Freeview EPG reader has worked well over the years and generally populates the EPG without any user intervention. The EIT data which actually contains the programme info is broadcast continuously on the main channels, so once you're tuned to a BBC/ITV/CH4 channel you will get the week's EPG data after about 20 minutes or so without any user intervention needed. The so-called "OpenTV" EPG used by Sky is carried only on a specific transponder, so you have to tune to it to download those specific SKY channels' EPG info. My understanding of CrossEPG was that it tuned to that specific transponder and stored the resultant data in its own database and then populated the EPG which is held in RAM on the receiver. The new OpenTV reader now built into the enigma DVB code reads the very same specific transponder and populates the EPG, so it should get the very same data as CrossEPG. The difference being that CrossEPG builds the data into its own database also as an interim step. Therefore, you should get exactly the same EPG data whether you use CrossEPG or the built-in reader! I think that what may happen sometimes is that CrossEPG's internal database can become corrupt and you may then subsequently get issues in the EPG info as presented on-screen. The only solution in that case may be to delete the actual CrossEPG folder and also the EPG file and start from clean.
When i started using EPGRefresh I had it set to do a "fake record" at a particular time of day and all worked well initially. Then I noticed that I was getting gaps in the schedule occasionally and missed a recording of F1 on CH4HD last Autumn at some stage. Other users have had no issues whatsoever with EPG Refresh on 28.2, so "your mileage may vary" as they say. What I ended up doing is uninstalling EPGRefresh, reinstalling CrossEPG and deleting any EPG and Cross files on my system and starting from clean. No issues with Cross since the start of this year. I have a combined terrestrial and satellite bouquet for my local Saorview (Freeview equivalent) and Freesat viewing. CH4HD is built into that bouquet at position 104. I have also a separate SKY FTA bouquet which would also contain CH4HD and other SKY FTA channels not carried on Freesat. I run CrossEPG at 04.30 and it fills in the CH4HD data which is not picked up by the built-in Freesat reader since the move back to SKY. Cross seems to work fine in combination with the standard EIT readers built into enigma in my case.
Long story short - I have standard EIT/Freesat readers enabled in EPG handling and OpenTV reader disabled. I run CrossEPG to gather SKY-specific EPG data not picked up by the built-in readers. I don't have any automatic load/save of the EPG enabled as it is held in RAM, except at system startup and shutdown where it is loaded or saved to disk.
The EIT data on BBC 1 HD (Sat) definately is out of date compared to crossEPG. Just now at 5pm it was sheduled for "Pointless" on BBC1 HD (28.8) but the daily "Coronavirus Update" program on Freeview. A few hours ago it was "Pointless" on both versions of BBC1 and the BBC made a change at about 3pm (as they are doing every bloody day) for the Coronavirus update program (shifting Pointless to BBC 2 HD). I don't understand why the hell they can't do a few weeks worth at once rather than on the day at 3pm for a program that's on in an hour (but that's another story).
Anyway - my point (I do have one) is that the EIT data (I was sitting on BBC 1 HD 28.8) is NOT getting updated but as soon as I ran the CrossEPG OpenTV scan it was updated so I stand by by "feeling" that the data is more up-to-date on the CrossEPG as opposed to just sitting on the channel (or indeed running a fake record of the channel (or "possibly even" zapping to iEPG Data 1 - although this test did not test that to see any difference as I would need two boxes to do that).
:-)
Paul
Well.... It's even more confused than me now.
the late change was partially reflected in the CrossEPG data for BBC1 HD 28.8. Pointless was added to BBC2 replacing whatever was set on BBC2 HD at 17:15 BUT it left Pointless at 17:15 on BBC1 HD 28.8. Now that the program has started I can see that the program is the new schedule (Coronavirus update) so that's correct and when I hilight it you may be able to just see that the following "phantom" program is also highlighted in Blue (the one that will not be on BBC 1 HD at 17:15. Most odd indeed.
59945
Now...... If I go to the EIT info on the actual channel (BBC1 HD 28.8) THAT IS CORRECT !?!?!? BUT importantly it is not updating the EPG with its correct stuff:
59946
My setting in the EPG screen is as follows:
59947
Surely if I have "Enable EIT EPG" set to yes then the correct data from the Now/Next (also taken from the EIT should overwrite the phantom entry in the EPG)
Bloody confusing or what?
Paul
Surely if I have "Enable EIT EPG" set to yes then the correct data from the Now/Next (also taken from the EIT should overwrite the phantom entry in the EPG)Now and Next is not part of the EPG. It's just Now and Next (all that Freeview had originally) and the data may be different to that in the EPG data being sent at the same time.
Bloody confusing or what?It can be, yes.
abu baniaz
14-04-20, 22:21
We really need to look at the EPG corruption issue.
I agree Abu, pretty sure it is the EPG settings for save and load that was causing it.
There is another area that needs looking at too. During the process of my tweaking to try and determine the causes I went into my tuners and fiddled a bit, changed some settings ad did a retune etc. Also ram ABM (just to make sure all was well before fuurther debugging). I noticed after doing this that the CrossEPG scheduler was not kicking in at 06:20 as it was meant to. Some searching on the forums led me to this post: https://www.world-of-satellite.com/showthread.php?39738-CrossEPG-scheduled-download-not-working&p=305545&viewfull=1#post305545 Post #21 from your good self :-)
I tweaked the file "settings" under /etc/enigma2 and noticed I had multiple 3601 entries (I have a Zgemma H7S with twin SAT and single T2). I removed them just leaving:
config.Nims.0.diseqcA=282
config.Nims.1.diseqcA=282
config.Nims.2.configModeDVBT=True
config.Nims.2.terrestrial=All regions, United Kingdom, (Europe DVB-T/T2)
All of a sudden ...... CrossEPG kicked back into life on an auto schedule
Something somewhere is making a mess of these settings and it would be good if that cause could be investigated and fixed so you don't get the bad settings in "Settings" or CrossEPG tweaked to ignore the bad settings.
Cheers,
Paul
Now and Next is not part of the EPG. It's just Now and Next (all that Freeview had originally) and the data may be different to that in the EPG data being sent at the same time.
It can be, yes.
Hi birdman, thanks for that. My point was only that Now/Next is set to get data from EIT and so is the EPG when zapped to a channel but the EPG side does not seem to change the EPG to the latest EIT data (that is correct in the Now/Next. I guess I'm misunderstanding the settings in the EPG config (not surprising as I am making it up as I go along):
59952
Just so you're clear on this - the EPG data that you're getting (or not in your case) when tuned to BBC1 is the Freesat EPG which is uploaded by a distribution company (I think it is Redbee - at least it was some time back), so it may not be being updated in real time. The CrossEPG data is sourced from the SKY EPG, which may be more current, given that it's a paid service for most users of that system.
I just saw your screenshot of your EPG settings and that may be part of the issue. You should have "Enable Freesat EPG" set to yes, otherwise tuning to BBC is only going to update your now/next from the EIT. The setting "Enable OpenTV EPG" is really only of use if you manually tune to the IEPG channel (or use EPGRefresh to do so). The OpenTV EPG is only carried on that transponder. I have "Enable OpenTV EPG" set to no and "Enable Freesat EPG" set to yes and "Enable EIT EPG" set to yes. The Freesat EPG contains compressed programme information using a proprietary compression key which came into the public domain some years ago. But to read the EPG data you need to have the Freesat EPG set to yes. Otherwise you would have to use CrossEPG or EPGRefresh as the source of your data.
Ah Okay..
the reason I set the Freesat EPG to "no" is because it doesn't carry the Channel 4 HD data anymore so I decided not to use it. ditching Freesat in favour of Enigma was all driven in the beginning by the loss of Channel 4HD so I had that sticking in my mind. I assumed that it was not needed if I was using CrossEPG for OpenTV and running that once per day (or even every hour as I see that is an option in CrossEPG). If I continue to use CrossEPG for OpenTV Data at least daily (as I am doing now and plan to do going forward) then do I still need the "Enable Freesat EPG" set to "yes"? Will that not then cause a conflict as having 2x sources for the same data?
Sorry to be so thick :-)
What's wrong with using Freeview for Channel 4 HD ?
I only have one freeview tuner so want to be able to watch/record C4 HD on Sat and Freeview as needed. That was kind of the point of getting an H7S :-)
If you stick with your current setup (no Freesat EPG enabled) and run CrossEPG on an hourly basis it would probably sort your issues. The Freesat EPG data definitely doesn't seem to get updated in real time. A couple of weeks ago CH5 stopped showing Neighbours on a daily basis but I had autotimer recordings set for the whole week as the EPG was still showing it as running each day. Even tuning to CH5 didn't resolve the EPG info. I ran Cross and it refreshed the whole EPG correctly and the autotimers sorted themselves out subsequently.
I think in my case the use of Freesat EPG enabled and Cross running a couple of times per week is probably the wrong strategy and I should stick with one source. It works out ok 95% of the time but I occasionally see overlapping entries in the EPG when the time of a programme gets changed or another programme is substituted.
If you stick with your current setup (no Freesat EPG enabled) and run CrossEPG on an hourly basis it would probably sort your issues. The Freesat EPG data definitely doesn't seem to get updated in real time. A couple of weeks ago CH5 stopped showing Neighbours on a daily basis but I had autotimer recordings set for the whole week as the EPG was still showing it as running each day. Even tuning to CH5 didn't resolve the EPG info. I ran Cross and it refreshed the whole EPG correctly and the autotimers sorted themselves out subsequently.
I think in my case the use of Freesat EPG enabled and Cross running a couple of times per week is probably the wrong strategy and I should stick with one source. It works out ok 95% of the time but I occasionally see overlapping entries in the EPG when the time of a programme gets changed or another programme is substituted.
:thumbsup:
Over 24 hours and no corruptions in the logs (where I was getting them every 3 or 4 hours) so that is it for me. The settings in the EPG setup screens definitely corrupted my EPG in memory.
My advice is to leave them turned off unless there is a very good reason to put them on.
I can't comment on FreeSat EPG as don't use it but what contains bugs - CrossEPG or EPGRefresh and for which service?
I'm hearing different thing and can only go by my own personal experience and that is EPGRefresh had occasional (or it appeared) corrupt EPG when it was first replaced with CrossEPG in Vix.
Since then I went back to CrossEPG as I don't and haven't had an issue with it for $ky EPG at least.
Yes, there are occasional last minute changes made that are obviously reflected in EIT (now/next) than what may be in EPG (downloaded 6am each morning)
In my opinion neither have bugs that affected me. Both CrossEPG when used for OpenTV data only and EPGRefresh are both good for me. It is the menu options within the EPG settings itself that was causing the corruption to in in-memory EPG data and then the log entries for Corrupt EPG data in the debug logs):
Here:
59960
and......
If your settings file for enigma2 has the nims settings wrong then that stops crossEPG schedule from running ie.
I agree Abu, pretty sure it is the EPG settings for save and load that was causing it.
There is another area that needs looking at too. During the process of my tweaking to try and determine the causes I went into my tuners and fiddled a bit, changed some settings ad did a retune etc. Also ram ABM (just to make sure all was well before fuurther debugging). I noticed after doing this that the CrossEPG scheduler was not kicking in at 06:20 as it was meant to. Some searching on the forums led me to this post: https://www.world-of-satellite.com/s...l=1#post305545 Post #21 from your good self :-)
I tweaked the file "settings" under /etc/enigma2 and noticed I had multiple 3601 entries (I have a Zgemma H7S with twin SAT and single T2). I removed them just leaving:
config.Nims.0.diseqcA=282
config.Nims.1.diseqcA=282
config.Nims.2.configModeDVBT=True
config.Nims.2.terrestrial=All regions, United Kingdom, (Europe DVB-T/T2)
All of a sudden ...... CrossEPG kicked back into life on an auto schedule
Something somewhere is making a mess of these settings and it would be good if that cause could be investigated and fixed so you don't get the bad settings in "Settings" or CrossEPG tweaked to ignore the bad settings.
In my humble opinion both are bugs elsewhere in the system (or features or quirks) that cause issues further down the line (particularly with the EPG data and with the CrossEPG scheduling.
cheers,
Paul
I have those two options set in EPG but I only have one EPG source. Maybe two EPG sources and timing of the EPG save can cause issue.
I have those two options set in EPG but I only have one EPG source. Maybe two EPG sources and timing of the EPG save can cause issue.
Maybe. Mine were set to save every hour and read every 2 hours (from memory) the timing gaps here may also be a factor.
Well for a start either use OpenTV (and epg refresh) or Freesat. Don't use both. If you pull from both which do you want displayed? How do you control that? LOL.
Maybe. Mine were set to save every hour and read every 2 hours (from memory) the timing gaps here may also be a factor.Read what every 2 hours?
Read what every 2 hours?
Automatic refresh (Read file in from hard drive) in the EPG settings. I "had" it set to that as I mistakenly thought I was doing the right thing. Both settings are now "off" per the image in post #35
Automatic refresh (Read file in from hard drive) in the EPG settings. I "had" it set to that as I mistakenly thought I was doing the right thing. Both settings are now "off" per the image in post #35Those files are for use on client receivers. So you are correct to disable them.
1) Turn on debug logs. This will show if the box crashes or does not behave properly.
2) Now disable CrossEPG and remove the plugin altogether from the receiver.
3) Disable Freesat EPG (you can re-enable this later once it works properly).
4) Set up EPG refresh to download from IEPG data channel (do a manual run).
5) Set up EPGImporter to download any packages not captured in step 4 (do not duplicate data sources. So do not download any UK satellite data using EPGImporter. Do a manual run).
6) Once you have the EPG fully populated restart enigma (this forces a save to epg.dat).
7) Wait and see what happens.
Maybe for that particular one the descriptions can be added to to state that. That would stop accidental enablement for someone else perhaps.
e.g.:
59977
Mine is working just fine now Huevos so I will leave it as-is mate.
What I have noticed for CrossEPG (since setting it to run every hour when in standby) is that my EPG is nicely up-to-date most of the time now :-)
I have found a bug in CrossEPG though. Every time it runs I get:
< 45649.002> [eDVBLocalTimerHandler] no transponder tuned... or no TDT/TOT avail .. try to use RTC :)
< 45649.002> [eDVBLocalTimerHandler] getRTC returned time=0. RTC problem?
< 45752.439> [Task] job Components.Task.Job name=SoftcamCheck #tasks=1 completed with [] in None
< 46069.028> [CrossEPG_Auto] onTimer occured at Thu 16 Apr 2020 11:20:00 BST
< 46069.030> [CrossEPG_Auto] Running CrossEPG Thu 16 Apr 2020 11:20:00 BST
< 46069.031> [CrossEPG_Auto] Time set to Thu 16 Apr 2020 12:20:00 BST (now=Thu 16 Apr 2020 11:20:00 BST)
< 46069.131> [Errno 2] No such file or directory: '/media/hdd//crossepg.log'
< 46069.131> [CrossEPG_Auto] providers selected for download:
< 46069.131> [CrossEPG_Auto] skyuk_astra2_28.2
< 46069.235> [CrossEPG_Wrapper] executing /usr/crossepg/crossepg_downloader -r -d /media/hdd/crossepg
In a part of the crossEPG code it has an extra slash!! Oddly enough though it must be only on one place as the crossEPG.log file does get written to every hour with all of its doings.
i.e. like this:
16/04/2020 11:20:00 SIFTeam CrossEPG Downloader 0.8.6+gitrAUTOINC+80b61723e7 (c) 2009-2014 Sandro Cavazzoni (https://github.com/oe-alliance/e2openplugin-CrossEPG)
16/04/2020 11:20:02 This software is distributed under the terms of the GNU Lesser General Public License v2.1
16/04/2020 11:20:03 Interactive: OPEN cmd received
16/04/2020 11:20:03 EPGDB opened (root=/media/hdd/crossepg)
16/04/2020 11:20:03 Clearing old aliases...
16/04/2020 11:20:03 Loading 'aliases.conf'...
16/04/2020 11:20:03 Loaded 1 aliases
16/04/2020 11:20:03 Adding new aliases...
16/04/2020 11:20:03 Completed
16/04/2020 11:20:03 Interactive: OK action sent
16/04/2020 11:20:03 Interactive: DEMUXER cmd received
16/04/2020 11:20:03 Interactive: OK action sent
16/04/2020 11:20:03 Interactive: FRONTEND cmd received
16/04/2020 11:20:03 Interactive: OK action sent
16/04/2020 11:20:03 Interactive: DOWNLOAD cmd received
16/04/2020 11:20:03 Started OpenTV events download
16/04/2020 11:20:03 Reading dictionary '/usr/crossepg/providers/skyuk_astra2_28.2.dict' ...
16/04/2020 11:20:03 Completed. Read 512 values
16/04/2020 11:20:03 Reading channels...
16/04/2020 11:20:03 Reading pid 0x11...
16/04/2020 11:20:13 Done
16/04/2020 11:20:13 Read 882 channels
16/04/2020 11:20:13 Reading titles...
16/04/2020 11:20:13 Reading pid 0x30...
16/04/2020 11:20:21 Done
16/04/2020 11:20:21 Reading pid 0x31...
16/04/2020 11:20:28 Done
16/04/2020 11:20:28 Reading pid 0x32...
16/04/2020 11:20:32 Done
16/04/2020 11:20:32 Reading pid 0x33...
16/04/2020 11:20:38 Done
16/04/2020 11:20:38 Reading pid 0x34...
16/04/2020 11:20:45 Done
16/04/2020 11:20:45 Reading pid 0x35...
16/04/2020 11:20:52 Done
16/04/2020 11:20:52 Reading pid 0x36...
16/04/2020 11:20:59 Done
16/04/2020 11:20:59 Reading pid 0x37...
16/04/2020 11:21:06 Done
16/04/2020 11:21:06 Read 4.29 MB
16/04/2020 11:21:06 Parsing titles...
16/04/2020 11:21:07 Titles parsed
16/04/2020 11:21:07 Reading summaries...
16/04/2020 11:21:07 Reading pid 0x40...
16/04/2020 11:21:19 Done
16/04/2020 11:21:19 Reading pid 0x41...
16/04/2020 11:21:31 Done
16/04/2020 11:21:31 Reading pid 0x42...
16/04/2020 11:21:38 Done
16/04/2020 11:21:38 Reading pid 0x43...
16/04/2020 11:21:47 Done
16/04/2020 11:21:47 Reading pid 0x44...
16/04/2020 11:21:59 Done
16/04/2020 11:21:59 Reading pid 0x45...
16/04/2020 11:22:10 Done
16/04/2020 11:22:10 Reading pid 0x46...
16/04/2020 11:22:22 Done
16/04/2020 11:22:22 Reading pid 0x47...
16/04/2020 11:22:33 Done
16/04/2020 11:22:33 Read 16.20 MB
16/04/2020 11:22:33 Parsing summaries...
16/04/2020 11:22:37 Summaries parsed
16/04/2020 11:22:37 Ended OpenTV events download
16/04/2020 11:22:37 Interactive: SAVE cmd received
16/04/2020 11:22:37 Interactive: CLOSE cmd received
16/04/2020 11:22:37 EPGDB closed
16/04/2020 11:22:37 Interactive: OK action sent
16/04/2020 11:22:37 Interactive: QUIT cmd received
The extra slash won't make any difference.
The extra slash won't make any difference.
There's something not quite right about it though - CrossEPG files (database and logs) should be in /media/hdd/crossepg/ not in /media/hdd//
I'll have a look through my debug and CrossEPG logs.
@smipx - what @Huevos says makes most sense - don't have Freesat and OpenTV EPG readers enabled at the same time. EPGRefresh should work with the iEPG transponder and gather the data correctly and you could run it several times per day providing you have a free tuner. I think the clock warning you're seeing in the log is because you have the receiver in standby, perhaps? My receiver is always in one of two modes - on and in use or in deep standby. We tend to use the box in the evenings and mostly watch recorded stuff (so we skip the ads!) and then put it in deep standby at bedtime.
EDIT - checked logs
A snip of the debug log for my run of CrossEPG this morning:
< 490.040> [CrossEPG_Auto] onTimer occured at Thu 16 Apr 2020 04:35:00 IST
< 490.045> [CrossEPG_Auto] Running CrossEPG Thu 16 Apr 2020 04:35:00 IST
< 490.050> [CrossEPG_Auto] Time set to Fri 17 Apr 2020 04:35:00 IST (now=Thu 16 Apr 2020 04:35:00 IST)
< 490.150> [Errno 2] No such file or directory: '/media/hdd//crossepg.log'
< 490.150> [CrossEPG_Auto] providers selected for download:
< 490.150> [CrossEPG_Auto] skyuk_astra2_28.2
I suspect that the code may be trying to trim the log, but is looking in the wrong place. The reason is that my cross log (located in /media/hdd/crossepg/crossepg.log) is over 400KB in size and contains every run of Cross since 2018. I only schedule Cross to run twice a week in any case.
I'll raise a bug report to the devs so they can have a look.
There's something not quite right about it though - CrossEPG files (database and logs) should be in /media/hdd/crossepg/ not in /media/hdd//
I'll have a look through my debug and CrossEPG logs.
@smipx - what @Huevos says makes most sense - don't have Freesat and OpenTV EPG readers enabled at the same time. EPGRefresh should work with the iEPG transponder and gather the data correctly and you could run it several times per day providing you have a free tuner. I think the clock warning you're seeing in the log is because you have the receiver in standby, perhaps? My receiver is always in one of two modes - on and in use or in deep standby. We tend to use the box in the evenings and mostly watch recorded stuff (so we skip the ads!) and then put it in deep standby at bedtime.
EDIT - checked logs
A snip of the debug log for my run of CrossEPG this morning:
< 490.040> [CrossEPG_Auto] onTimer occured at Thu 16 Apr 2020 04:35:00 IST
< 490.045> [CrossEPG_Auto] Running CrossEPG Thu 16 Apr 2020 04:35:00 IST
< 490.050> [CrossEPG_Auto] Time set to Fri 17 Apr 2020 04:35:00 IST (now=Thu 16 Apr 2020 04:35:00 IST)
< 490.150> [Errno 2] No such file or directory: '/media/hdd//crossepg.log'
< 490.150> [CrossEPG_Auto] providers selected for download:
< 490.150> [CrossEPG_Auto] skyuk_astra2_28.2
I suspect that the code may be trying to trim the log, but is looking in the wrong place. The reason is that my cross log (located in /media/hdd/crossepg/crossepg.log) is over 400KB in size and contains every run of Cross since 2018. I only schedule Cross to run twice a week in any case.
I'll raise a bug report to the devs so they can have a look.
:) Nice one - this should be put to bed
There are some other bugs/features raised in this thread though (my post #35 lists 2 other candidates for fixing) that the DEV's may want to address in the fullness of time.
Powered by vBulletin® Version 4.2.5 Copyright © 2024 vBulletin Solutions Inc. All rights reserved.