ABM with use of IPTV Streams

[bbbuk]

Hi,

Thanks to the work that @Huevos has done by adding ability to use IPTV streams, I was then looking for a quicker/easier method of removing "HD" channels.

I know I can use the <deletes> tag but I have to go through all the channels and add all the HD channel numbers.

Therefore I thought about the old "custom" regional provider which was designed to remove HD channels and leave us with just SD channels.

Then I was going to use the new <streams> tag to replace certain channels with IPTV streams.

However, I couldn't seem to get this working. I removed my custommix file and tried just my regional custom area but some channels are missing (ie Sky1).

Now I know this channel wouldn't clear but was hoping it would still at least show so that I can then replace this channel with an IPTV stream.

Ultimately, I was looking for an easier way of removing HD channels leaving me with SD channels and any non working onces I can replace with IPTV streams. For the odd few where this isn't possible (ie +1 channels), I had planned on just using the <deletes> tag.

Am I correct in assuming the "custom" region is no longer working?

[dsayers]

The streams insert will only work with channels that are already there so if Sky one is removed you can't replace it so you need to use HD region.


Abu's latest commit allows to force streams to empty slots so that should work but it forces service reference "1:0:1:0:0:0:0:0:0:0:" https://github.com/oe-alliance/AutoBouquetsMaker/commit/ba2ed436bb53e05c57252cee5112d5310c965576 but then again it may not work if custom region removes the channels.

[abu baniaz]

Ideally, I would like to have a custom option instead of separate regions.

Remember, we exclude the dead transponders. Make a separate provider file and delete the broken TSIDs lines

https://github.com/oe-alliance/AutoBouquetsMaker/blob/master/AutoBouquetsMaker/providers/sat_282_sky_uk.xml#L465:L474

EDIT:
Or delete this section
https://github.com/oe-alliance/AutoBouquetsMaker/blob/master/AutoBouquetsMaker/providers/sat_282_sky_uk.xml#L528:L532

[Huevos]

How does making Custom a separate function help? And what differences does it make here?

All the stuff BBUK is asking can be done in a CustomMix file in hacks section, probably with just a few lines of code.

[bbbuk]

Thanks for replies.

I sorted it out. I had forgotten that "custom" region settings (rather than usual SD or HD) removed those now dead transponders.

Thanks again to @Huevos for adding ability to use IPTV streams in place of channels.

[bbbuk]

@Huevos

I've been using this feature for couple of months without an issue. Thank you.

Would it be easy/possible, to add service type to this feature?

So instead of using default service type 1: , could it be possible to specify gstreamer (4097) or another service type (5001 or 5002)?

[Huevos]

No it is not possible. Why would you want to do that?

[bbbuk]

No it is not possible. Why would you want to do that?Some IPTV streams have had little glitch recently but when I manually changed service ref of them so they begin with "4097:" instead of "1:" then it's okay as gstreamer takes over and buffers a few seconds.

[bbbuk]

I've submitted a pull request for consideration because using service type 1 for IPTV streams isn't always ideal as it requires constant stream with no interruption whereas other service types buffer the streams and therefore reduce buffer underruns/glitches.

There are several possible service types to use for IPTV streams (assuming one has added appropriate plugin) as taken from this thread (http://www.world-of-satellite.com/showthread.php?58875-IPTV-EPG-(5002)-In-channel-selection-screen&p=465725&viewfull=1#post465725):


Recognised playback types
1, processed by the SoC (for when buffering is not required),
4097, processed by gstreamer via servicemp3,
5001, processed by gestreamer via gst-player and
5002, processed via extplayer3.

Pull request here (https://github.com/oe-alliance/AutoBouquetsMaker/pull/65).

This is designed to compliment @Huevos commit here (https://github.com/oe-alliance/AutoBouquetsMaker/commit/c2e51147147f219e11971ab0d94dca80d8e467c2).

My coding skills aren't fantastic so apologies if this can be coded better :)

Thanks

[dsayers]

I've submitted a pull request for consideration because using service type 1 for IPTV streams isn't always ideal as it requires constant stream with no interruption whereas other service types buffer the streams and therefore reduce buffer underruns/glitches.

There are several possible service types to use for IPTV streams (assuming one has added appropriate plugin) as taken from this thread (http://www.world-of-satellite.com/showthread.php?58875-IPTV-EPG-(5002)-In-channel-selection-screen&p=465725&viewfull=1#post465725):


Recognised playback types
1, processed by the SoC (for when buffering is not required),
4097, processed by gstreamer via servicemp3,
5001, processed by gestreamer via gst-player and
5002, processed via extplayer3.

Pull request here (https://github.com/oe-alliance/AutoBouquetsMaker/pull/65).

This is designed to compliment @Huevos commit here (https://github.com/oe-alliance/AutoBouquetsMaker/commit/c2e51147147f219e11971ab0d94dca80d8e467c2).

My coding skills aren't fantastic so apologies if this can be coded better :)

Thanks

I've tested this and it seems to be great.

[carlowen]

I've submitted a pull request for consideration because using service type 1 for IPTV streams isn't always ideal as it requires constant stream with no interruption whereas other service types buffer the streams and therefore reduce buffer underruns/glitches.

There are several possible service types to use for IPTV streams (assuming one has added appropriate plugin) as taken from this thread (http://www.world-of-satellite.com/showthread.php?58875-IPTV-EPG-(5002)-In-channel-selection-screen&p=465725&viewfull=1#post465725):


Recognised playback types
1, processed by the SoC (for when buffering is not required),
4097, processed by gstreamer via servicemp3,
5001, processed by gestreamer via gst-player and
5002, processed via extplayer3.

Pull request here (https://github.com/oe-alliance/AutoBouquetsMaker/pull/65).

This is designed to compliment @Huevos commit here (https://github.com/oe-alliance/AutoBouquetsMaker/commit/c2e51147147f219e11971ab0d94dca80d8e467c2).

My coding skills aren't fantastic so apologies if this can be coded better :)

Thanks


Hi, I have been following this and your other post about streams. It have been really useful. I have managed to replace channels with streams and would like to be able to add streams in to other empty slots but have been able to do that. Also you latest comment about a small buffer is great and it looks like it has worked. Can i please see your custom mix or the hacks section. Many thanks

[bbbuk]

Hi, I have been following this and your other post about streams. It have been really useful. I have managed to replace channels with streams and would like to be able to add streams in to other empty slots but have been able to do that. Also you latest comment about a small buffer is great and it looks like it has worked. Can i please see your custom mix or the hacks section. Many thanksYes using the 4097 stream type instead of default 1 buffers the stream (ie gstreamer does it rather than box). I've not had an issue yet using this method. Just waiting on powers-to-be to consider including it in ABM to compliment the existing work done on streams.

All I choose "custom" area I require, remove the section of code from provider file that normally removes the dark channels so they would then show but albeit dark.

I then just use custommix file, with something like in it:


<custommix>
<streams>
<!-- Swap channels that are no longer working with IPTV streams -->
<stream url="http:websiteaddress.com:port/live/usersemailaddres@com/password/336.ts" target="106" />
</streams>

This effectively just replaces the dark sky one channel with an IPTV stream (and using the code changes above, the IPTV stream is buffered reducing glitches, etc).

[Huevos]

Personally I think if this is to be added it needs to be on a stream by stream basis, not a single type from a config setting. Also I am reluctant to add a config setting at all for IPTV. IPTV is part of Custom and nothing in Custom is driven from config settings.

[bbbuk]

Personally I think if this is to be added it needs to be on a stream by stream basis, not a single type from a config setting. Also I am reluctant to add a config setting at all for IPTV. IPTV is part of Custom and nothing in Custom is driven from config settings.Fair enough and it makes sense. I'll see if I can code this so it reads service type when it reads the IPTV streams you coded into ABM.

[bbbuk]

I've had a look and I believe I've got it to read service types (ie 4097) from custommix file via adding appropriate code here (https://github.com/oe-alliance/AutoBouquetsMaker/blob/8710cd4f7fb45852d544b2ab8ac66d1cc32a2316/AutoBouquetsMaker/src/scanner/tools.py#L173:L182).

However, I'm stuck on how to mention this in def bouquetservicesline (https://github.com/oe-alliance/AutoBouquetsMaker/blob/8710cd4f7fb45852d544b2ab8ac66d1cc32a2316/AutoBouquetsMaker/src/scanner/bouquetswriter.py#L929).

There surely has to be away but I don't know much about ABM I'm afraid.

I'll just keeping hard-coding the 4097 with iptv streams instead.

Thanks anyway.

[dfox]

Is there anyway to enable recording using service reference 5001?

[gmanpanthro]

I've had a look and I believe I've got it to read service types (ie 4097) from custommix file via adding appropriate code here (https://github.com/oe-alliance/AutoBouquetsMaker/blob/8710cd4f7fb45852d544b2ab8ac66d1cc32a2316/AutoBouquetsMaker/src/scanner/tools.py#L173:L182).

However, I'm stuck on how to mention this in def bouquetservicesline (https://github.com/oe-alliance/AutoBouquetsMaker/blob/8710cd4f7fb45852d544b2ab8ac66d1cc32a2316/AutoBouquetsMaker/src/scanner/bouquetswriter.py#L929).

There surely has to be away but I don't know much about ABM I'm afraid.

I'll just keeping hard-coding the 4097 with iptv streams instead.

Thanks anyway.

Hi is there anyway you can update these files to work with abm 3.0? As it keeps crashing at the point of bouquet generation.

[Huevos]

Hi is there anyway you can update these files to work with abm 3.0? As it keeps crashing at the point of bouquet generation.

Please start your own thread. You need to post your debug log with extra debug enabled. And any other files involved including CustomMix file.

[Huevos]

@bbbuk, sorry I forgot about this thread. Abu reminded me about your PR.

Lets try a bit of a different way. Code is attached.

Stream type is set in the xml file. So to add a stream type do it like this...


<stream url="http://stream.source:port/live/username/F36/password/308.ts" target="701" name="our name 1" streamtype="4097" />

So stream type is case by case, not global. Are you happy with this format?

[Huevos]

"Stream type" is a bad description. Should be "playback type".

Anyway, try it and see if the concept works for you.

[bbbuk]

@bbbuk, sorry I forgot about this thread. Abu reminded me about your PR.

Lets try a bit of a different way. Code is attached.

Stream type is set in the xml file. So to add a stream type do it like this...


<stream url="http://stream.source:port/live/username/F36/password/308.ts" target="701" name="our name 1" streamtype="4097" />

So stream type is case by case, not global. Are you happy with this format?I like that :). Provides more flexibility if want certain service refs on different streams.

I'm using my modded version currently so I'll try this likely at weekend from unmodified version before applying your POC.

With regards to description, I think something like ServiceRef or ServiceRefType. For consistency really as the original introduction of recording IPTV in E2 uses ServiceReference (see original git (https://github.com/OpenViX/enigma2/commit/c59bfcc1fa72dc56c4abd7789f8bef0b2bf30fc8)).

[Huevos]

All the other attributes are lower case so we need to stick to that.

Give it a test, then we will decide what to name it.

Also maybe limit allowed types to 1, 4097, 5001 and 5002.

[bbbuk]

Okay tried this with vanilla ABM except for your two modded files and crashed at end of scan.

Relevant part of crash log:-
< 260.929> [ABM-DvbScanner] Reading transponders...
< 264.859> [gRC] main thread is non-idle! display spinner!
< 266.247> [ABM-DvbScanner] Scan complete, netid: None
< 266.257> [ABM-DvbScanner] Added/Updated 74 transponders with network_id = 0x20
< 266.257> [ABM-DvbScanner] Reading services (SKY)...
< 270.370> [ABM-DvbScanner] Read 606 services with bouquet_id = 0x1001
< 270.370> [ABM-DvbScanner] Reading services extra info...
< 285.513> [ABM-DvbScanner] Read extra info for 733 services
< 285.523> [ABM-Manager][read] sat_282_sky_uk, Done
< 285.535> [eDVBFrontend1] set dynamic current limiting
< 285.632> [ABM-Manager][save] Saving...
< 285.643> [ABM-Manager][save] Bouquets to hide: {'sat_282_sky_uk': [1050]}
< 285.643> [ABM-Manager][save] TV bouquets to keep: []
< 285.643> [ABM-Manager][save] Radio bouquets to keep: []
< 285.644> [ABM-Manager][save] Add provider prefix to bouqets: False
< 285.644> [ABM-BouquetsWriter] Writing lamedb...
< 286.979> [ABM-BouquetsWriter] Wrote 62 transponders and 733 services
< 287.033> [ABM-Tools][customLCN] No custom video LCN file for sat_282_sky_uk.
< 287.042> [ABM-Tools][customLCN] No custom radio LCN file for sat_282_sky_uk.
< 287.123> [ABM-Tools][customMix] Reading CustomMix file for sat_282_sky_uk.
< 287.148> Traceback (most recent call last):
< 287.148> File "/usr/lib/enigma2/python/Plugins/SystemPlugins/AutoBouquetsMaker/scanner/main.py", line 523, in doBuildIndex
< 287.148> File "/usr/lib/enigma2/python/Plugins/SystemPlugins/AutoBouquetsMaker/scanner/manager.py", line 151, in save
< 287.149> File "/usr/lib/enigma2/python/Plugins/SystemPlugins/AutoBouquetsMaker/scanner/tools.py", line 192, in customMix
< 287.149> if streamtype and "stream" in customised["video"][target]: # if a stream was added above, a custom streamtype may also be added
< 287.149> KeyError: 170
< 287.149> [ePyObject] (CallObject(<bound method AutoBouquetsMaker.doBuildIndex of <class 'Plugins.SystemPlugins.AutoBouquetsMaker.scanner.m ain.AutoBouquetsMaker'>>,()) failed)

When I removed the custom xml file there is no crash.

I added to end of custom xml file the following tag info:-

target="106" streamtype="4097" />

[abu baniaz]

Can you run your mix file through an XML validator please?

[bbbuk]

Passed xml validation check.

I have just noticed that in @Huevos example xml file, it also has a "name" tag in addition to the "streamtype" tag he mentioned. The "name" tag is new and isn't in my xml file as all I did was add "streamtype" tag. Eg:


<stream url="http://stream.source:port/live/username/F36/password/106" target="106" streamtype="4097" />

[abu baniaz]

This worked for me



<custommix>

<streams>
<stream url="http%3a//xxxxxxxx.ts" target="107" streamtype="4097" />
</streams>
</custommix>

[bbbuk]

Ok, thanks :)

Edited xml so it had just one stream and it was okay. I'll slowly add the other streams to see where it falls over but I suspect I know where it will. I'll post back with results...

[bbbuk]

Found what error was eventually :)

It's what happens when the "target" channel doesn't exist so in my error above channel 170 no longer exists (most likely moved).

Commented out that and one or two others and works fine. I'll have to figure out what these channels were/are and where they've moved to.

Could it be coded so that it gracefully handles situations where target channel doesn't exist like ignoring that line and moves on to next one?

[Huevos]

Found what error was eventually :)

It's what happens when the "target" channel doesn't exist so in my error above channel 170 no longer exists (most likely moved).

Commented out that and one or two others and works fine. I'll have to figure out what these channels were/are and where they've moved to.

Could it be coded so that it gracefully handles situations where target channel doesn't exist like ignoring that line and moves on to next one?Ok, so we need some sort of sanity test...

Remove...

if streamtype and "stream" in customised["video"][target]:

Replace with...

if streamtype and target and target in customised["video"] and "stream" in customised["video"][target]:

Not tested. :confused:

[bbbuk]

Ok, so we need some sort of sanity test...

Remove...

if streamtype and "stream" in customised["video"][target]:

Replace with...

if streamtype and target and target in customised["video"] and "stream" in customised["video"][target]:That works, thanks.

Things tested:-
* Defaults to servicereftype of 1 when tag not present
* Doesn't crash if target stream doesn't exist
* Can have different service ref types for different channels

Thanks again :)

[Huevos]

Should we limit serviceRefTypes to 1, 4097, 5001, 5002? Are there any others?

[bbbuk]

They are the only serviceref types that I know of.

How about defaulting to 1 if serviceref type isn't one of those mentioned?

[Huevos]

Here is the final version (not tested).

Attribute name has been changed to "servicereftype".

Text for adding to "README.txt":

You can also give your stream a "service reference type". This allows you to tell the receiver which playback software
to use for that individual stream.

In this example "servicereftype" has been added.

<custommix>
<streams>
<stream url="http://stream.source:port/live/username/F36/password/308.ts" target="118" servicereftype="4097" />
</streams>
</custommix>

The following is a list of currently valid "service reference types".

1, processed by the SoC (for when buffering is not required),
4097, processed by gstreamer via servicemp3,
5001, processed by gestreamer via gst-player,
5002, processed via extplayer3.

[Huevos]

They are the only serviceref types that I know of.

How about defaulting to 1 if serviceref type isn't one of those mentioned?Code attached above should do that automatically.

[bbbuk]

@Huevos, above code works fine as intended except for small typo in tools.py:-
SERVICEREF_ALLOWED_TYPES = [1, 4097, 5001, 5002]

Thanks again :)

[bbbuk]

Create a PR here (https://github.com/oe-alliance/AutoBouquetsMaker/pull/128) regarding your work on this @Huevos.

Thanks again :)

[Huevos]

https://github.com/oe-alliance/AutoBouquetsMaker/commit/9fe1fea0c9209b6439061041c94612030135731f