PDA

View Full Version : Starting work on a new plugin... AUTO CHANNEL UPDATER



hashman
23-05-11, 16:45
ok

i've finally done it, i've designed what I want to do, now I'm going to sit down and start work within the next few weeks and months on an idea i've had since months, but never given it enough thought as to its implementation.

Here goes the idea in a few words:
Contexte


Many sky users will notice, that if they don't update their bouquets regularily, many channels will start to disappear with the very common "path not found in SID" error
this is because BSkyB change their frequencies (they manage their lot of frequencies as they please... so we should adapt)

This was very frustrating for me, because I have 4 boxes, 2 Enigma1 (dm500) and 2 Enigma2 (1 dm800 and vu+duo)
The problem is not loading the bouquets via dbedit... but its that all 4 boxes are used by different people (sister,brother and myself to be precise) => everyone has their own bouquets, and its a pain in the ass updating all 4 because neither of them is computer savy and I have to do the support!


Idea


1. A simple web interface that will allow to generate a bouquet based on channel names (and not satellite.xml file)
- behind each channel name will be a frequency that will not be seen by user, therefore, he doesn't care if he got the right sky sports 1 or the right bbc (out of the multiple ones available in dbedit)

2. a plugin which updates the userbouquets and custom made satellite.xml file on a daily basis with the webserver that has a php script running which would generate the files on the run

3. whenever a frequency would change, all it would take is to update it once for the change to be replicated to all users that have the given channel in their bouquet


The key to success


1. community run: i'm thinking of a community system where anyone with the correct rights would be able to assign a new frequency/pid to a channel

2. auto updated: further down the line, using sites such as kingofsat the update of a channel could be automated to a certain extend


Advantages


1. I have the bouquet i create once updated automatically
2. I don't have to recreate/customize my bouquets every time the channel changes frequency


The first script will probably be enigma2 because i'm connected to my vu+duo at the moment and playing around on it to call and download content from a web server, worst case scenario i'll just use wget LOL

I'll use a space on a webhost account i got as i don't think i'll be generating too much traffic...

So look out for this space :)

bassethound
23-05-11, 16:52
Sounds like a great plan hashman im sure there a plenty of people on here willing to help out on your project

punisher
23-05-11, 17:12
yeh sounds fantastic mate and can see loads of people wanting to use this plugin :D

digidude
23-05-11, 18:14
youll be surprised at how popular this will become m8, believe me, i know ;)

a tip for you to save you days of fustration though, for it all to 'auto' update, have a VERY simple control file on the server, and when you update the boquets pack, update the control file as well, this will save you having to create numerous complex scripts to do one simple thing

pooface
24-05-11, 13:47
I was speaking to andy about such a feature a while back ... But he doesn't have the time to work on it just yet... Worked in a very similar fashion tho...

One thing to keep in mind ... Sky Sports 1 is a popular name on many packages, e.g. 28.2, 13 and 19 (UK, IT & DE) all have the name. So, will need to be careful to get the correct one in the bouquet updating. Also, can't say doesn't matter which bbc they get, as in bouquets people have regional setups. E.g. Sky 1 have UK & Eire variations with same name. Someone in UK will want the UK variation, and someone uk Eire will want the Eire variation...

Good luck with the project tho, and be great to get it going, and possibly even working with all different sats & packages...

hashman
24-05-11, 14:02
thanks,
at the moment the progress is so far:
1. i've designed the database structure
2. i'm reading up on the lamedb file structure, as it'll be source of entry

next steps:
3. understand the conversion from KOS data to lamedb

Then i'll work on the rest :)

My way of export will be very simple, without making too complicated features, it'll simply be what dreambox edit does: i.e upload a list of files to your /var/engima2

As for managing the different channels with same name on different satelites: i'll just base it on the satellite :) i see that info in lamedb already

too bad andy didn't work on it, would have saved me the hours already spent
cheers

pooface
24-05-11, 14:25
Does it work on the box itself, or as a pc program that connects to your box?

mickbrush
24-05-11, 14:58
This is a top idea just to be clear this would mean I could keep my personalised bouquets and update and without it messing up. That would be much better than the updater I am using now ;)

hashman
24-05-11, 15:16
it'll be online
i'm at the moment working on a php script that decrypts the lamedb somehow... no idea yet, just manage to read file, but i've downloaded dreamboxedit sourcecode, will see if i can figure it out

the idea is :

once you create your bouquets, behind a channel is a reference that is added in the bouquet file and at the end will be generated a lamedb file with all the channels that are in yoru bouquet and a corresponding satellite.xml file
all this will be downloaded via a cron job on a daily basis by the dreambox from the website (based on a username)
so if a channel changes frequency, which is updated on the website, at the next sync, it'll be updated in the dreambox automatically

punisher
24-05-11, 15:17
will it update it in your favorites list too or just the satellites.xml file? as normally it would only update the satellites.xml file and you would manually have to remove the old channel and replace it with the new one in your favorites list

hashman
24-05-11, 15:38
my idea is to create fresh bouquet files and fresh lamedb and satellite files every time its synced
the idea is that i'll create one bouquet, and it'll be spread through the different dreamboxes as they'll download the whole set everytime
This is also important if i add new channels to my bouquet from newly added channels to a satellite, at the next sync (or manual sync), they'll appear on the dreambox :)

the same can be applied to others, if you create a bouquet, and i want it, i can set my dreambox to sync your bouquets, so any change you make, will show up on my box automatically...that's the whole point of community run (atleast the idea i have)

maybe whist implenting, i'll realise the limitations to my current knowledge, but my idea is what i just tried to describe :)

hashman
24-05-11, 17:03
progress going good so far, i've managed to determine:

Channel name / provider / sid/NS/TSID/ONID/STYPE

i've matched for some examples the SID to the same as the one in KOS, ...

rossi2000
24-05-11, 18:49
you can always use the dreamboxuk frontend and database to create your new bouquets, its updated with freq changes etc and is in line with slys epg numbering.
upload them to your hosting.

then you can install the autoupdate plugin and initiate a download of the new bouquets from your hosting on each box. this is what i currently do with family / freinds

i just create a new set of bouquets and upload them to my hosting, then the user can initiate the download of new bouquets from there box and it installs them. you can always set it up as a cron job too.

Alanp
25-05-11, 14:06
Good luck with this will be great when its working

digidude
25-05-11, 15:08
the autoupdater i made for the tm800 (with help with the actual script), a channel list is uploaded to the server, and a very simple text based control file is also edited. This control file is checked once every 24 hours, if the control file has changed, a message onscreen asks if you want to update, if you say yes the boquet list is downloaded, installed and settings reloaded, if you choose no, itll ask again in 24 hours, or you can do it namually from my plugin / ppanel

Larry-G
25-05-11, 17:01
you can always use the dreamboxuk frontend and database to create your new bouquets, its updated with freq changes etc and is in line with slys epg numbering.
upload them to your hosting.

then you can install the autoupdate plugin and initiate a download of the new bouquets from your hosting on each box. this is what i currently do with family / freinds

i just create a new set of bouquets and upload them to my hosting, then the user can initiate the download of new bouquets from there box and it installs them. you can always set it up as a cron job too.

agreed mate. i use B16MCC's DBUK services database / front end software my self on my DM600 for this exact purpose. although i can see the appeal of a fully automated system with no user input needed.

dymebox
03-06-11, 15:03
This sounds awesome! Been after this for some time!

Good Luck!

boxer29
06-06-11, 20:23
A very good idea
good luck

fintannl
06-06-11, 23:22
the bane of my life. so irritating to find channels that used to work no longer work. had planned to log a post asking a simple question, with all the frequency changes how come sky customers have no issue. and in return why is it not possible to emulate.

Larry-G
06-06-11, 23:26
the bane of my life. so irritating to find channels that used to work no longer work. had planned to log a post asking a simple question, with all the frequency changes how come sky customers have no issue. and in return why is it not possible to emulate.

because sky and their receiver manufacturers paid millions of £ for development and still do, something the amateur teams such as VIX / blackhole / VTI etc. can not do. it's written into the sky firmware same as all other official service providers

Sicilian
09-06-11, 13:18
Moved out of downloads only section ;)

toz
12-06-11, 18:43
Any updates on this project?

Cheers,
Tam.

hashman
13-06-11, 21:15
hi

i'm working in parrall on the user interface, but more time is being spent on parser for the kos website
i've broken down the tv and radio, and am trying to parse the changes feed, and thinking on the best way to detect frequency changes to update the satellite.xml and bouquet list

hopefully a first working beta in the days to come, but have not had much time lately, i think i should have more time by end of week so will be able to progress

hashman
17-06-11, 17:21
good progress today:
the web scripts I can now:
- Import the data off KOS
- Generate a lamedb and import data off a lamedb as well (it was a backup method in case i couldn't get the KOS data imported)

Next step:
- Setup a bouquet system to generate bouquets
- Setup business logic to detect the update of channel frequency (and therefore SID)
- Setup interface to define the bouquets and create links that would autogenerate the bouquet/lamedb for download directly to the box
- Make an easy package to install that would setup the cronjob and update bouquets on box automatically (for distribution purposes)

will continue after weekend :)

punisher
17-06-11, 18:17
Great progress mate keep it up :D cant wait to see this

banie01
17-06-11, 18:23
Thats great progress Dude!

Stanman
17-06-11, 18:25
Great progress mate, cant wait for the final product.

Larry-G
11-07-11, 11:06
not meaning ti hijack this thread but a new image has been released with this feature incorporated.
the new DBUK = dreamboxuk image comes with the ability to automatically update it's sky uk bouquet pack with the aid of the B16MCC services database all without the need of a pc directly from the box and image it self.

pooface
11-07-11, 11:14
Only problem I see with that, is that it's for the dbuk b16mm channel proggy.

This plugin will be miles better imo, since it will retrieve all data for all satellites from the kos website. Personally prefer that idea...

Sent with Tapatalk

Larry-G
11-07-11, 11:23
Only problem I see with that, is that it's for the dbuk b16mm channel proggy.

This plugin will be miles better imo, since it will retrieve all data for all satellites from the kos website. Personally prefer that idea...

Sent with Tapatalk

yes i see your point but the b16 front end database is updated very regularly, in most cases before KOS even acknowledges changes have been made. yes at the moment this feature is only working fir sky uk but rossi is looking at incorporating others into it. personally i'm looking forward to both projects to see what they can do.

pooface
11-07-11, 12:21
yes i see your point but the b16 front end database is updated very regularly, in most cases before KOS even acknowledges changes have been made. yes at the moment this feature is only working fir sky uk but rossi is looking at incorporating others into it. personally i'm looking forward to both projects to see what they can do.

If others can be incorporated in to it, then that would be great... Would be nice to see it released as a stand-alone plugin ;)

Larry-G
11-07-11, 12:32
rossi has now incorporated the CST cyrus / vhannibal and MR. mister bouquets into his plugin, although at the moment it's a built in feature of the DBUK image but i'll have a word see if he might consider doing a plugin for VIX.

pooface
11-07-11, 13:23
rossi has now incorporated the CST cyrus / vhannibal and MR. mister bouquets into his plugin, although at the moment it's a built in feature of the DBUK image but i'll have a word see if he might consider doing a plugin for VIX.

But does that just download the latest version of those bouquets? As my problem is that I have certain bouquets that I like to edit. I like getting rid of certain bouquets all together, adding a few new ones. Everytime a new bouquet set is released, it's a nightmare updating it, so I often take a little while getting latest channels etc uploaded...

So, what I need is something like what this plugin sounds like it would do ... take my bouquets as they are, and when a channel changes it's details, it will automatically update the one in my bouquet with the correct settings...

Larry-G
11-07-11, 13:30
the bouquet packs such as vhannibal etc will just be a overwrite with the newest packs, the sky uk sets however will link directly into the b16mcc frontend server and update the bouquet on the box with all new changes etc. iv'e not used it as i dont use the image but from all accounts it's a very nice easy to use feature.

delboy13
18-08-11, 10:46
was there any further news on this plugin as nothing has been written lately about it

mcquaim
18-11-11, 13:03
Hi folks,

I was actually thinking about this problem last night so did a search this morning and came across this thread. Has this ever been developed? It sounded like a brilliant idea..

I kow somebody posted earlier in the thread that sly have spent millons developing this solution but how is it actually achieved when the sly box doesn't even have to be connected to the internet to keep the channels up-to-date??

I suppose, it is the million pound question and the person with the answer would be rich...

Cheers,
mcquaim

Stanman
18-11-11, 18:45
Sly update it over the air, assuming comes down the LNB

mcquaim
18-11-11, 19:09
Yeah, but they must have some way of taking a new xml or equivalent and update on the fly without needing a restart. I assume that this data is in no way accessible to any non-sky receiver?

Larry-G
18-11-11, 19:20
Hi folks,

I was actually thinking about this problem last night so did a search this morning and came across this thread. Has this ever been developed? It sounded like a brilliant idea..

I kow somebody posted earlier in the thread that sly have spent millons developing this solution but how is it actually achieved when the sly box doesn't even have to be connected to the internet to keep the channels up-to-date??

I suppose, it is the million pound question and the person with the answer would be rich...

Cheers,
mcquaim

it's coded directly into the firmware of the sky box, all they need do is send the instructions to the box via the satellite feed and the firmware does the rest.

tandaman
19-11-11, 11:12
Now that really would be handy