AB-Com PULSe 4K SE. Crazy offer only £99! FREE UK DELIVERY!
Superb quality and spec AB-Com PULSe 4K SE. Crazy offer! Only £99! FREE UK DELIVERY! 4K UHD, Enigma 2, Multiboot 4 images & more!...
AB-Com PULSe 4k Rev II Twin tuner only £149! FREE UK DELIVERY!
Superb quality and spec AB-Com PULSe 4K Rev II Twin Satellite tuner only £149! FREE UK DELIVERY! 4K UHD, Enigma 2, SATA HDD facility, Multiboot 4 images & more!...

[Zgemma H7] Weird EPG errors. EPG Refresh. Recent versions of OpenViX e.g. 5.4.008.

  • Thread starter Thread starter BrokenUnusableAccount
  • Start date Start date
I have a modified version of opentv.cpp logging tons of stuff as IEPG data 1 is scanned now.
Definitely beginning to understand how it works, but not yet where it goes wrong.
 
To save me searching can anyone point me to the location of the code that limits the size and number of debug logs?
 
Menu/Setup/System/Logs/Settings
 
Has anyone checked the stream data to see if bogus events are being sent by the provider? I will be busy decorating for next two weeks, but will try to look if i have some free time.

TODO: fix source type issue, update epg.dat from V7 to V8.

I think you may also be seeing an issue in epgcache code that prevents many of the epg readers inc. OpenTV from updating/removing previously cached events, which would result in you seeing overlapping events from events that were not removed/updated as no longer needed. Fixing this issue would also require updating the epg.dat save/load functions and the file structure of the epg.dat file so the epg Version would also need incrementing from V7 to V8. We need to adjust the eventData type(0xFF) size of the epg source from uint8_t to eventData type(0xFFFF) uint16_t. OpenTV source type(0x4000) and other many of the other readers with a greater source type ID than 255 are currently wrongly being set/saved/loaded as PRIVATE(0X00) source type. Some of the epgcache code checks for PRIVATE or 0 source type and these checks are blocking the wrongly set OpenTV source type (0x00) from ever updating or removing events, it is only allowing new events to be added.

When i next get some free time, i can make patch to fix this issue to test.
 
LraiZer said:
Has anyone checked the stream data to see if bogus events are being sent by the provider?
Click to expand...
That's what I'm trying to do first. But it's taking me quite a long time to figure out everything I need.
I'm working in
Code: 
https://github.com/BrianG61UK/enigma2/tree/Dev2
but there's hardly anything there yet.
 
You'll probably have to change them in /etc/enigma2/settings......

Code: 
config.crash.daysloglimit=8
config.crash.debug_path=/media/hdd/logs/
config.crash.debugloglimit=4
config.crash.enabledebug=True
config.crash.logtimeformat=2
config.crash.sizeloglimit=10
 
LraiZer said:
Has anyone checked the stream data to see if bogus events are being sent by the provider?
Click to expand...

If this was happening wouldn't everyone see the same double time slot entry? Even from the observations in this thread some people are seeing the problem on days where i see nothing and visa-versa.
 
adm said:
If this was happening wouldn't everyone see the same double time slot entry? Even from the observations in this thread some people are seeing the problem on days where i see nothing and visa-versa.
Click to expand...
Note that I think I only see it if I scan IEPG data 1 after midnight and before some unknown time approximately between about 04:00 and 08:00, plus it's possibly worse the sooner after midnight, but not at all sure about that last bit.
I also wonder if it could be due to a weird compiler bug which might mean only some boxes have the problem.
 
adm said:
If this was happening wouldn't everyone see the same double time slot entry? Even from the observations in this thread some people are seeing the problem on days where i see nothing and visa-versa.
Click to expand...

I saw lots of doubles this morning on tomorrow's (Friday 12th) EPG slots just before midday, containing entries from Wednesday 10 just before midnight. All have the same 36 hour 24 minute (2184 minutes) offset from the original broadcast time. I know I keep banging on about this value being 0x888 or binary 100010001000 which I think is being caused by a bug in the Julian date handling for events starting before midnight and straddling the midnight hour, but I think the answer lies there.
 
BefuddledBrian said:
To save me searching can anyone point me to the location of the code that limits the size and number of debug logs?
Click to expand...

Use PuTTY and save log on your PC!

Telnet to box with PuTTY

stop enigma2
Code: 
init 4
start enigma2 with debug level 4 logging
Code: 
ENIGMA_DEBUG_LVL=4 enigma2
end debug logging
Code: 
CTRL+C
start enigma2 in normal mode again
Code: 
init 3
 
fat-tony said:
I saw lots of doubles this morning on tomorrow's (Friday 12th) EPG slots just before midday, containing entries from Wednesday 10 just before midnight. All have the same 36 hour 24 minute (2184 minutes) offset from the original broadcast time. I know I keep banging on about this value being 0x888 or binary 100010001000 which I think is being caused by a bug in the Julian date handling for events starting before midnight and straddling the midnight hour, but I think the answer lies there.
Click to expand...

I think all agree that this should be the first place for Brian to look to debug and fix this issue ;)
 
LraiZer said:
...TODO: fix source type issue, update epg.dat from V7 to V8.

I think you may also be seeing an issue in epgcache code that prevents many of the epg readers inc. OpenTV from updating/removing previously cached events, which would result in you seeing overlapping events from events that were not removed/updated as no longer needed. Fixing this issue would also require updating the epg.dat save/load functions and the file structure of the epg.dat file so the epg Version would also need incrementing from V7 to V8. We need to adjust the eventData type(0xFF) size of the epg source from uint8_t to eventData type(0xFFFF) uint16_t. OpenTV source type(0x4000) and other many of the other readers with a greater source type ID than 255 are currently wrongly being set/saved/loaded as PRIVATE(0X00) source type. Some of the epgcache code checks for PRIVATE or 0 source type and these checks are blocking the wrongly set OpenTV source type (0x00) from ever updating or removing events, it is only allowing new events to be added.

When i next get some free time, i can make patch to fix this issue to test.
Click to expand...
Would this explain why these phantom entries arent noticable (that I have observed) when using CrossEPG as i've noticed its code does something with epgcache (creates new instance I think)?
 
bbbuk said:
Would this explain why these phantom entries arent noticable (that I have observed) when using CrossEPG as i've noticed its code does something with epgcache (creates new instance I think)?
Click to expand...

As far as I am aware, CrossEPG reads the Sky EPG transponder data and stores it in its own database and then populates the EPG cache afterwards. I don't know whether it shares or uses the same code used by the OpenTV reader that EPGRefresh or OpenTVzapper calls. So, it may not encounter the same corruption issue.
 
If you install RadiotimesXmltvEmulator from the feed, as it is based on CrossEPG, you should also see these bogus events in the xmltv file that it generated in /tmp
 
I never noticed before... No event_id in that format.
Code: 
 <programme start="20210316110000 +0100" stop="20210316120000 +0100" channel="282_2_6155">
  <title lang="en">Lorraine</title>
  <sub-title lang="en">Entertainment - Magazine</sub-title>
  <desc lang="en">Lorraine Kelly presents a topical mix of entertainment, discussion and showbiz glamour as well as the latest fashion, food and celebrity gossip.</desc>
 </programme>
 
"tvheadend" has some interesting (and very old) comments about event_id's. eg...

Code: 
https://github.com/tvheadend/tvheadend/commit/be1a42d9092550ffa892e39ef91026e1405192d2

Try to detect duplicate EPG entries from the DVB feed and adjust
EPG accordingly. The EPG code will search for events with the same
DVB event ID +- 2 events from the current one. If the event id is
equal, the prvious (old) entry will be removed in favor of the new one.
Reason for not blindingly trusting the event id is that some networks
seem to (incorrectly) reuse IDs.
Click to expand...
 
Sorry - I couldn't work on this last night - I had to get up early this morning for something.
Tonight however I have:

This is the section of opentv.cpp I changed to log stuff, the lines I have added/changed are marked with //BB
Code: 
OpenTvTitle::OpenTvTitle(const uint8_t * const buffer, uint16_t startMjd)
{
	uint8_t descriptor_tag = buffer[0];

	if (descriptor_tag == OPENTV_EVENT_TITLE_DESCRIPTOR)
	{
		uint8_t descriptor_length = buffer[1];
		uint8_t titleLength = descriptor_length > 7 ? descriptor_length-7 : 0;

		uint32_t startSecond = (UINT16(&buffer[2]) << 1); //BB

		startTimeBcd = ((startMjd - 40587) * 86400) + startSecond; //BB
		duration = UINT16(&buffer[4]) << 1;

		//genre content
		//uint8_t flag1 = buffer[6];
		//uint8_t flag2 = buffer[7];
		//uint8_t flag3 = buffer[8];

		char tmp[OPENTV_EVENT_TITLE_LENGTH];
		memset(tmp, '\0', OPENTV_EVENT_TITLE_LENGTH);

		if (!huffman_decode (buffer + 9, titleLength, tmp, OPENTV_EVENT_TITLE_LENGTH * 2, false))
			tmp[0] = '\0';

		title = convertDVBUTF8(tmp, sizeof(tmp), 5);

		eDebug("[OpenTV] +titl:\"%s\" SMJD:%i Ss:%i SBD:%i len:%i", tmp, startMjd, startSecond, startTimeBcd, duration); //BB

		/* storing all the crc unique titles in the title reader phase,
		   would give us a titles reduction of 155,000 down to 13,000!
		   we currently add/delete as we go with reduction size ~5000 */
		uint8_t *otvt = new uint8_t[titleLength];
		memcpy(otvt, buffer + 9, titleLength);
		crc32 = opentv_crc(otvt, titleLength);
		delete [] otvt;
	}
}
Just after midnight I see the "ghost events" showing with startMjd corresponding to today (the day that started just some minutes ago) and startSecond corresponding to start times of over 24:00.
The start times seem to be the old original start time plus 2^17 seconds which is 131072 (or 0x20000) seconds which is 2184 (or 0x888) minutes and 32 seconds.

:confused: but thinking hard.

Code: 
https://www.brian-gregory.me.uk/GDL/Putty%20zgemmah7.lan%2020210312-124826.log
https://www.brian-gregory.me.uk/GDL/Putty%20zgemmah7.lan%2020210312-233623.log
https://www.brian-gregory.me.uk/GDL/Putty%20zgemmah7.lan%2020210313-000145.log
 
Last edited:
BefuddledBrian said:
Just after midnight I see the "ghost events" showing with startMjd corresponding to today (the day that started just some minutes ago) and startSecond corresponding to start times of over 24:00.
The start times seem to be the old original start time plus 2^17 seconds which is 131072 (or 0x20000) seconds which is 2184 (or 0x888) minutes and 32 seconds.

:confused: but thinking hard.
Click to expand...
Given that there are only 86400 seconds in a day (0x15180) that looks as though 0x20000 is a flag (meaning unknown) that needs to be masked off.
Or it's a bit that's been shifted down that shouldn't be there....(signed vs unsigned right-shift)?
 
You must log in or register to reply here.

Similar threads

C
EPG Plugin confusion
Replies
14
Views
9K
ccs
ccs
adm
[ET10x00] EPG not populating from saved data after deep standby
Replies
15
Views
5K
birdman
birdman
smallfrak
[GiGaBlue QUAD+ PLUS] Regular freeze after scheduled EPG Refresh, no CrashLog, debug log confusing
2
Replies
24
Views
12K
smallfrak
smallfrak
bassethound
Discovery Channel abandons Canal +
Replies
0
Views
2K
bassethound
bassethound
bassethound
Sky lodges appeal over Sky News Freeview EPG move
Replies
0
Views
2K
bassethound
bassethound

OpenViX Feeds Status

Back
Top