Weird EPG errors. EPG Refresh. Recent versions of OpenViX e.g. 5.4.008.

[Huevos]

Without the patch being applied first, the fix for updating old events was just removing more correct events that were overlapped by the bogus events, resulting in the event update fix making the first issue 10 times worse.

128 is an old removed free source type so i guess you could temp just change OPENTV=16384 to that as a bypass hack until update patch is fully tested and working?

Could you make a patch for this please?

[bbbuk]

The over the air data received by an Enigma box over air must be correct or else the same problem would be seen on Freesat and Sky branded boxes. It must be some software running on the Enigma box that is wrongly interpreting the broadcast EPG data and creating values of startseconds that are not sensible.

Delete epg.dat and epg cache at this point and request a forced EPG data fetch and the EPG data for day B is now correct. The content of this forced EPG data must be identical to that of the forced EPG request issued before the epg.dat and epg cache deletion but then it didn’t result in the EPG data being corrected.

This is what I was trying to get at earlier by saying that I don't notice these issues with Sky UK when using CrossEPG to grab EPG data! Have previously seen when I switched from CrossEPG to EPGRefresh/OpenTV.

It's also looks like CrossEPG does some work-around maybe with epg cache as it creates a new instance of EPG cache.

Doesn't this, at least for Sky UK, potentially mean that broadcast data is correct and yes there could be multiple issues and EPG cache being other issue?

[BrokenUnusableAccount]

This is what I was trying to get at earlier by saying that I don't notice these issues with Sky UK when using CrossEPG to grab EPG data! Have previously seen when I switched from CrossEPG to EPGRefresh/OpenTV.

It's also looks like CrossEPG does some work-around maybe with epg cache as it creates a new instance of EPG cache.

Doesn't this, at least for Sky UK, potentially mean that broadcast data is correct and yes there could be multiple issues and EPG cache being other issue?

I did look quickly at the OpenTV part of CrossEPG and as far as I could see the relevant part of the code looked very like that in OpenViX's opentv.cpp.
No obvious sign of any means for CrossEPG to already be correct that I noticed.

[BrokenUnusableAccount]

As I tried to explain in #103, #105 (note: in #105 I wrote 249 when I should have written 239)

They are kind of treating the field I've called startSecond as a signed time.
However it doesn't roll from meaning a +ve number to meaning a -ve number in the usual place (when the MSB flips to a 1).
Instead it seems it should be treated as a signed negative number if it's greater than or equal to 86400.

Then just as when you interpret a normal negative signed 17 bit number as unsigned the result is 0x20000 too high.

Let me put that another way just in case some people still can't see it.

I think that startSecond (including the 0 LSB that's been added) is the bottom 17 bits from a larger (say 32 bit) signed integer that is only allowed to take on even values between -44672 and 86398.

[birdman]

They are kind of treating the field I've called startSecond as a signed time.

Which seems and odd thing to do.
Would you ever consider that I'm writing this at -11:00 on Monday?

It sounds more like and artefact of the way they process service suppliers data rather than being intentional. It's noticeable that it only seems to affect programmes which wrap between days at midnight. That's midnight UK, so for many Western European countries these do not span two days - they are completely within the second day.

But since the OpenTV spec isn't published we can only ever speculate, and just have work with whatever is sent out.

[BrokenUnusableAccount]

Which seems and odd thing to do.
Would you ever consider that I'm writing this at -11:00 on Monday?

Perhaps the day is the day the program finishes?
Perhaps the day part is not allowed to be before today?

It sounds more like an artefact of the way they process service suppliers data rather than being intentional. It's noticeable that it only seems to affect programmes which wrap between days at midnight. That's midnight UK, so for many Western European countries these do not span two days - they are completely within the second day.

But since the OpenTV spec isn't published we can only ever speculate, and just have work with whatever is sent out.

But presumably genuine Sky boxes know what to do with it.

[BrokenUnusableAccount]

While we have developer types in here can anyone tell me what's going on in opentv.cpp a few lines underneath where I was working?

		uint8_t *otvt = new uint8_t[titleLength];
		memcpy(otvt, buffer + 9, titleLength);
		crc32 = opentv_crc(otvt, titleLength);
		delete [] otvt;

WT* is it necessary to allocate memory to make a copy before working out the crc and then release it afterwards?
If you look at the definition of opentv_crc further up it can't modify it's input data.
It doesn't make any sense to me.

[LraiZer]

WT* is it necessary to allocate memory to make a copy before working out the crc and then release it afterwards?
If you look at the definition of opentv_crc further up it can't modify it's input data.
It doesn't make any sense to me.

Wasn't this also used and deleted somewhere instead in the early code creation during debug testing and then that code was removed and just a quick temporary delete added back here and then all this was forgot about?

Looks like you've got something else you'll have to fix

[LraiZer]

Could you make a patch for this please?

[PATCH] New ENIGMA_EPG_V8 file structure

I think this patch should open up all the cache management code that was previously being blocked.
Will needs a lot of monitoring and testing over a period of time to see what it actually does or breaks!

[Huevos]

Just for the curious, here is a debug data dump log of the current bogus events being fixed.
Data cycle appears to currently be running at 139 seconds to populate the epg

If you inspect the stream can you see these erroneous values?

[Huevos]

[PATCH] New ENIGMA_EPG_V8 file structure

I think this patch should open up all the cache management code that was previously being blocked.
Will needs a lot of monitoring and testing over a period of time to see what it actually does or breaks!

Thanks. That is now up on our repo for testing.

[Joe_90]

I'm ready to test when software is available

[LraiZer]

If you inspect the stream can you see these erroneous values?

These erroneous values? e3 e0

They are in the stream buffer data that i output in the log file.

e3 << 9 = 116224
e0 << 1 = 448
116224 + 448 = 116672
1615766400 + 116672 = 1615883072
1615883072 - 131072 = 1615752000

BOGUS: Sewing Street / Yarn Lane REPEATS
 mjd=59288(bcd:1615766400) bcd=1615883072 bcdfix=1615752000 sec=116672(E3 E0) duration=43200
b5
22
e3
e0
54
60
6b

[Huevos]

Is it possible to get CRID from this stream? Simon is planning some code for programming series link.

[birdman]

There does seem to be some end-user documentation for an OpenTV API here:

https://opentv.nagra.com/sites/default/files/api/medialive_os_api/5.2.4/CCOM/index.html

Not sure how complete this is, but I can't find any reference to CRID data in it.