Weird EPG errors. EPG Refresh. Recent versions of OpenViX e.g. 5.4.008.

[BrokenUnusableAccount]

Try this out and see how it fares.

I will, unless I realise it's wrong before tonight.

Everyone: I made the debug log files available at the bottom of message #99.
Since I seem to be incapable of explaining what I think is going on without confusing myself and getting it wrong it might be best to look at them and try and understand that way if you want to understand.

They're logs from starting with an empty EPG and manually zapping to IEPG data 1 for 3+ minutes.

[LraiZer]

Quick initial thoughts on this? Thanks for debugging this Brian, are you saying we may only need to decrement the mjd date if start seconds are sent greater than 1 day and the bugus times will then be correct with no adjustment required to the sent start seconds?

// if start seconds are sent greater than 1 day (86400)
// we need to decrement startTimeBcd by 2 days (172800)

if (startSeconds > 86400)
	startTimeBcd -= 172800;

[BrokenUnusableAccount]

Quick initial thoughts on this? Thanks for debugging this Brian, are you saying we may only need to decrement the mjd date if start seconds are sent greater than 1 day and the bugus times will then be correct with no adjustment required to the sent start seconds?

// if start seconds are sent greater than 1 day (86400)
// we need to decrement startTimeBcd by 2 days (172800)

if (startSeconds > 86400)
	startTimeBcd -= 172800;

No.
Some of what I posted late last night, well I was living up to my handle. I was confused.

Today I'm saying as far as I can see

		if (startSecond >= 86400)
		{
			startSecond -= (0x20000 - 86400);
			--startMjd; // Interpret weird time beyond 86400 seconds as previous day
		}

should always get you back to sanity.

I guess maybe this would do it too:

if (startSeconds >= 86400)
	startTimeBcd -= 0x20000;

To be more cautious you could instead just throw away any record where startSecond >= 86400.

[ccs]

You can maintain old EPG data for as long as 12 hours in epg/settings.

It might at least help seeing where the phantoms are coming from without staying up too late.

[birdman]

It may not be programs spanning midnight being the problem but 11pm - note the +1hr

This, of course, means that in one timezone they are in one day whilst in a different timezone they are in a different day.
So the sent EPG contains a "standard" Mjd, requiring a -ve seconds offset?

[Joe_90]

The start_time in the schedule would/should always be coded as UTC. Any timezone (or DST) offset should be coded in the TOT in order to allow the software calculate and present the data correctly in the local time in the EPG viewed by the user.

[birdman]

The start_time in the schedule would/should always be coded as UTC. Any timezone (or DST) offset should be coded in the TOT in order to allow the software calculate and present the data correctly in the local time in the EPG viewed by the user.

That's what I think too. But I also suspect that someone provides the data and someone else converts this into what gets transmitted, and it's not impossible that somewhere in between there is a mismatch of intent. Particularly as I doubt the original data uses Mjd format.

[LraiZer]

I guess maybe this would do it too:

if (startSeconds >= 86400)
	startTimeBcd -= 0x20000;

This correctly fixed the many bogus events i was seeing on the Gemporia Craft channel. Looks good to me, thanks! Maybe you should submit a patch with your fix?

[BrokenUnusableAccount]

This correctly fixed the many bogus events i was seeing on the Gemporia Craft channel. Looks good to me, thanks! Maybe you should submit a patch with your fix?

Okay. I will.

[abu baniaz]

Thanks. Added here
https://github.com/OpenViX/enigma2/c...539180660df1a6

Pull request submitted to PLi
https://github.com/OpenPLi/enigma2/pull/2889

[Joe_90]

That's what I think too. But I also suspect that someone provides the data and someone else converts this into what gets transmitted, and it's not impossible that somewhere in between there is a mismatch of intent. Particularly as I doubt the original data uses Mjd format.

The DVB Standard requires the MJD format. I extracted that info and posted it earlier in the thread:

start_time: This 40-bit field contains the start time of the event in Universal Time, Co-ordinated (UTC) and Modified
Julian Date (MJD) (see annex C). This field is coded as 16 bits giving the 16 LSBs of MJD followed by 24 bits coded as
6 digits in 4-bit Binary Coded Decimal (BCD). If the start time is undefined (e.g. for an event in a NVOD reference
service) all bits of the field are set to "1".
EXAMPLE 1:
93/10/13 12:45:00 is coded as "0xC079124500".
duration: A 24-bit field containing the duration of the event in hours, minutes, seconds. format: 6 digits,
4-bit BCD = 24 bit.
EXAMPLE 2:
01:45:30 is coded as "0x014530".

[birdman]

The DVB Standard requires the MJD format. I extracted that info and posted it earlier in the thread:

Precisely.
But the original data provider will not be providing the time in Mjd format, so some other entity will be doing a conversion somewhere.
That conversion code may be doing odd things with timezone manipulation. We don't know, but it could explain what is being seen.

[adm]

Precisely.
But the original data provider will not be providing the time in Mjd format, so some other entity will be doing a conversion somewhere.
That conversion code may be doing odd things with timezone manipulation. We don't know, but it could explain what is being seen.

Is this the correct fix? Where does the incorrect start seconds data come from?

As far as I’m aware the ghost/duplicate entries are for programs that have been previously broadcast and on the day they were broadcast must have had the correct EPG data suggesting the correct startMJD and correct startseconds. Where has the incorrect startseconds now come from? Probably not from the broadcaster as others have reported that other means for fetching the IEPG data don’t show these types of problem.

Also, in my experience, when the problem occurs a re-fetch of the IEPG doesn’t correct the problem UNLESS the EPG cache (and epg.dat file) are first deleted, and often by stopping Enigma to accomplish the deletion. If the EPG cache and epg.dat are deleted and the EPG data then populates from the IEPG then the EPG appears to be problem free suggesting that raw IEPG data wasn’t the cause of an incorrect startseconds value.

[ronand]

The raw data is fine. It's just some entries got incorrectly processed by the reader to 36 hours later than they should. By the time you notice the incorrect entries it looks they come from past programs.

[adm]

The raw data is fine. It's just some entries got incorrectly processed by the reader to 36 hours later than they should.

That's my point. If the raw data is fine the bug is before the point where the value of startseconds exceeding the number of seconds in a day is being used to establish the start time.
The bug fix above is at a point where the incorrect (startseconds) data is being used and not where it is being incorrectly assigned.