Open EPG to DVB Viewer

January 6, 2009

Just a quick note, as discussed in another forum, to ask for extra Freesat EPG functionality in DVBViewer and recording service.

Freesat EPG data is broadcast on transponder 11427 and this is where DVBViewer gets the EPG from. However, for UK users with only one tuner this means that the EPG will only update if they tune to a channel they don't regularly watch or want to watch.

The EPG is, however, broadcast an all Freesat transponders, just with a few differences which I've copied in below. If possible, please could DVBViewer and Recording service be updated so that they can receive EPG data from all Freesat transponders? I will supply a list and details of the ones I've tested so far if you need it but you could leave it to the user to add the list.

Inclusion of this functionality would mean that for UK users the EPG would update while watching regularly watched channels including, but not limited to, all BBC, ITV and Channel 4 channels.

Details as promised. Transponder 11427 is the one that DVBViewer currently uses for Freesat EPG data. Please ask if you want any more information, thanks.

Transponder 11427 transmits at a symbol rate of 27500000 while the other transponders transmit at 22000000

11427 uses PIDs 3002 and 3003 to transmit EPG data - other transponders use PIDs 3841 and 3842

11427 transmits EPG at a faster rate - 30 seconds is enough to generate a full EPG on 11427 but other transponders need 30 minutes to get the full list.

January 6, 2009

Done, there now is a FreeSAT EPG (PID 3842) entry in the MHW options. This should do it :biggrin:

January 6, 2009

Done, there now is a FreeSAT EPG (PID 3842) entry in the MHW options. This should do it

Thanks for such a quick reponse, Lars. I look forward to testing itin the next beta!

February 25, 2009

Well, I'd like very much to see this feature work for italian sky epg too.

From what I know, it would not be too hard to decode the sky italy openepg implementation as it seems to be very similar but simpler than the UK one.

Italian SKY Openepg is broadcasted on Hotbird 11881 V on Alice channel, as you probably already know.

As soon as you have a bit of spare time, Lars, think about it. It should be a nice feature indeed.

February 25, 2009

Italian SKY Openepg is broadcasted on Hotbird 11881 V on Alice channel, as you probably already know.

No I don't Got any information about this (links etc.)?

February 25, 2009

No I don't Got any information about this (links etc.)?

Of course. It's a pleasure to see such a fast response.

Here you can find the first implementation of a plugin (developed for vdr-linux):

http://lukkinosat.altervista.org/

There is also a Huffman dictionary of common words used in epg (UK and ITA) and if you download the plugin youl'll find the C sources.

Here you can find the plugin (written for mediaportal by djblue) that performs the same thing. You'll find some useful infos in that thread too:

http://forum.team-mediaportal.com/developm...esat-epg-43261/

You will find something useful in particular on page 4 of the forementioned thread:

http://forum.team-mediaportal.com/attachme...nepgtoxmltv.zip

This piece of software parses a .TS recording of the Alice 11881 V channel (HotBird SKY Italy) and extracts from there all the EPG data, then it writes them to a XML file.

Let me know if you need anything else, I'll be happy to help in any way.

February 25, 2009

ok, this might take a little longer and I won't promise anything. but I will study the material and see if and how we can incorporate it.

February 25, 2009

Thank you very much Lars.

February 26, 2009

You might both be interested in this - DJBlu is writing something specifically for DVBViewer and the app he's working on already had support for Sky Italia. It's separate from the MediaPortal one and there's also something there about how simply the 'green button' series link feature works on Sky+.

See from post 337 onwards on this page

February 26, 2009

DJBlu is writing something specifically for DVBViewer

Yes I've read it too, but AFAIK the data structure of the freesat EPG (DVBViewer is already able to read it) is not so different from the Sky Italy EPG. DjBlue itself says that the italian epg data structure is simpler under some aspects. Furthermore I'd really prefer that such a feature would be integrated into the main app rather than thinking to implement it into a plugin (integrated into the main app by the developer=more supported).

Anyway, I'll wait for the DjBlue implementation and will see how it works, if he'll be the first to reach the deadline....

BTW, here there is the italian sky epg data structure (in italian language):

http://lukkinosat.altervista.org/howto.pdf

(Copy and paste the link into the browser and don't click it, otherwise it will not work)

Edited February 26, 2009 by Topogigi

February 26, 2009

OK!

Well in that case please could I request that if you have time you look at the UK sky EPG at the same time? I think it has the same format. It has info for many more channels than the Freesat EPG does.

Thank you.

February 26, 2009

The freesat epg is simple EIT epg with a twist. all text is hufman encoded.

The sky epg is far from that easy. It seems to be something in the line of MHW with added BAT processing with a userdefined descriptor for the channelID and hufman decoding for the text.

I personally think it would be much easier and more effective to support the works of DJBlue from our side (if he wishes it) than trying to reprogram the routines by ourselves all over again (of course we can not integrate GPL'd code).

February 26, 2009

That's fine. I only suggested in case it was easy. To be honest, the Freesat EPG covers almost all free to air channels which is all most people care about. It's a lot of hard work with little reward to try and configure hardware and software in order to view NDS Videoguard encrypted channels and they are the only ones missing from the EPG.

February 28, 2009

I will be releasing something later in the week that will listen on the port that DVBViewer streams on.

I will have something that will write an EPG.tmp file. I will send the message to refresh the EPG to DVBViewer.

What I would like to know is how to add events to the PVR if this is possible externally.

March 1, 2009

I will be releasing something later in the week that will listen on the port that DVBViewer streams on.
I will have something that will write an EPG.tmp file. I will send the message to refresh the EPG to DVBViewer.

What I would like to know is how to add events to the PVR if this is possible externally.

Nevermind figured it out.

Accessing it through the COM.

You will be able to use the Series link function when I have finished with this.

March 1, 2009

I think it would be a good idea, if the Plugin would work in the Recording Service to.

\DVBViewer\Plugins\Plugins1 ...Plugins32

Because in the next version (current Beta) DVBViewer and Recording Service can run at the seam time.

And the Recording Service records and collect the EPG in this case.

March 1, 2009

I require some information from a developer.

I am using the COM to connect to DVBViewer and I am able to update the EPG however

I need to know how I can set the Genre as IEPGItem doesn't contain it.

Any help is appreciated.

Thanks.

March 1, 2009

Thanks for the work DJBlu. Sounds really useful. I can't believe how quickly DVB software has developed, from a UK 'Freesat' point of view, over the last year or so.

March 1, 2009

It's the content (ID). We only support the predefined DVB content IDs (=genre ID).

So you would need a mapping of your genres to the appropriate contentID.

You can find a list of the ID=Name values in the file language\english.lng section [Genre].

If further help is needed, just ask.

March 1, 2009

Got it, cheers.

I will have to modify it and add the lang file with the program then.

Thanks for the help.

March 3, 2009

Hi guys,

Another question.

I have got my program to update the EPG via COM. However its very slow. Is there any faster way of importing the EPG?

I am using the buffer. Adding all the items then Commiting the changes. But 119,000 records is taking a good 7-8 minutes.

Any help is welcomed.

March 3, 2009

If you want to use XML you could use this (only 4.0+ and recording service after 1.4.7.10)

  IEPGManager2.ImportEPG(const XMLData: WideString);

format:

<?xml version="1.0" encoding="UTF-8"?>
<epg>
<programme start="20090303031500" stop="20090303040000" channel="281479273054356">
<eventid>903030315</eventid>
<title>Lost(65)(G) (66)</title>
<event>Amerikansk spændingsserie.</event>
<description>(R)(65)(G)</description>
<charset>1</charset>
<content>0</content>
</programme>
</epg>

Where

Start = Startdatetime in format YYYYMMDDHHmmSS (Year 4 digits, Month 2 digits, Day 2 digits, hour 2 digits, minute 2 digits, seconds 2 digits)
stop = Enddatetime in format YYYYMMDDHHmmSS (Year 4 digits, Month 2 digits, Day 2 digits, hour 2 digits, minute 2 digits, seconds 2 digits)
channel = EPGChannelID of type Int64
eventid = the eventID (must be unique for each channelID context) of type Dword
title, event and description should be selfexplaining (description allows crlf, title and event NOT).
charset = keep this with the default value 1 of type byte
content = the DVB genre ID of type byte

add you entries to the in memory xml document and write it into a string, which you send to the DVBViewer (or you could post it to the service, I can explain the details if you're interessted).

Each data packet is processed in a own background thread and does not block the DVBViewer.

Things to consider:

Use UTF-8!!
send in chunks of 5000 - 10000 entries per xml document to keep memory usage at a acceptable level.
use the more accurate new EPG channelID ( (Ord(tunertype)+1) shl 48 + NetworkID shl 32 + TransportstreamID shl 16 + SID )
check if interface is available (-> older versions of the DVBViewer).
Make sure your EventID for each event is always the same. Means: don't generate them random, use a datetime to integer function, this is save because a eventID only needs to be unique in the context of a channelID.

This is what @delphi uses for his xmltv importer (for the viewer and also for the recording service) and I use it to send EPG data from the service to connected DVBViewer clients. :biggrin:

March 4, 2009

use the more accurate new EPG channelID ( (Ord(tunertype)+1) shl 48 + NetworkID shl 32 + TransportstreamID shl 16 + SID )

Reference this.

I am using this calculation as there is no BitShift Function in vb.net

EPGChannelID = ((tunertype+1) * (2 ^ 48)) + (NetworkID * (2 ^ 32)) + (TransportID * (2 ^ 16)) + SID

Which I beleive does give me the correct EPGID however could someone calculate the following

TunerType = 1

Network ID = 2

Transpo ID = 2041

Service ID = 9211

I get 562958677124091

Is this correct as no data is being imported into DVBViewer.

March 4, 2009

Your calculation is correct...

March 4, 2009

Your calculation is correct...

Thanks its now working.

Imports in less than 8 seconds now 119,000 events.

Just going to pretty it up now and will have a release later on.

March 5, 2009

Amazing :-)

March 9, 2009

Ok,

Before everyone gets too excited. This is the stage I am at.

This file when set up will grab the data from DVBViewer and will update your epg.

I am having trouble with the series link and channel updates do to some bugs with DVBViewer.

So rather than make you wait for something that may take weeks I thought I would let you sample what I have.

To set this up. You need .net 2.0 framework installed along with a copy of DVBViewer 4.0 or above.

Extract both the files from this link to somewhere on your computer, no need for the .dict files just these two.

Open EPG to DVBViewer

To set this up.

Before running the file OpenEPGtoDVBViewer you need to set up the stream for the file to use.

When you install DVBViewer you need to install the NetStream plugin.

Set up the netstream plugin to Stream "All transponder" on unicast. Choose your IP and port.

When this is up and running tune to "EPG Audio Channel"

Now start the OpenEPGtoDVBViewer.exe

click on settings.

Change your IP and port and navgate to your DVBViewer installation path.

Set your UK Region. then click apply.

Now click "Run Update Now"

Wait 2 minutes and your EPG will be uptodate.

You can change the update time and it will monitor your epg and will update it with any changes.

Please be aware that this will not touch your Favourites yet as there is a bug in DVBViewer. I will add these features in a week or so.

Feedback is welcomed.

March 9, 2009

I am having trouble with the series link and channel updates do to some bugs with DVBViewer.

That's quite a strong statement without having even contacting one of the DVBViewer programmers.

I wrote it before and can only repeat it: I think your work is important and useful for the DVBViewer and it's users and I will try to support your work as much as possible. But without specific question from you it is hard to do, because I don't know what you excactly need or want to do. I'm just a PM away

March 9, 2009

That's quite a strong statement without having even contacting one of the DVBViewer programmers.

I wrote it before and can only repeat it: I think your work is important and useful for the DVBViewer and it's users and I will try to support your work as much as possible. But without specific question from you it is hard to do, because I don't know what you excactly need or want to do. I'm just a PM away

I have posted in the bugs section,

Write a simple script that tries to delete item 0 from the favourites. It will not work. The answer comes back as False from DVBViewer and whilst this error is there I cannot control the Favourites completely.

Also the groups will not delete.

I have tried everything and they still remain.

If you know how to remove the item 0 and the groups then please help as I can get this finished.

March 9, 2009

OpenEPGtoDVBViewer.rar - 0.10MB

Newer version,

compatible with Vista.

All users please download this version.

March 9, 2009

Do not mess with the favorites. It is not up to a plugin to change channels or favorites. There is normally no sane reason for it. User usually don't like it if something changes their carefully constructed favorites setup. trust me

And please contact me directly via PM with problems while working the plugin.

March 9, 2009

Do not mess with the favorites. It is not up to a plugin to change channels or favorites. There is normally no sane reason for it. User usually don't like it if something changes their carefully constructed favorites setup. trust me

And please contact me directly via PM with problems while working the plugin.

OpenEPGtoDVBViewer.rar - 0.10MB

Thats the new file. You can enter your I.P address in manually.

And the reason for the Favourites is people do(me being one) want their favourites to reflect how Sky UK display the channels. There is the option in the plugin to not update the favourites. If you don't want people to update them then why have it in the COM Interface?

March 9, 2009

Is this version working only for SKY UK? Can I try it also for SKY IT?

March 9, 2009

http://www.borntoshare.co.uk/OpenEPGtoDVBViewer.rar

please use this one.

I haven't fully tested Sky Italy but it is programmed for it so give it a go.

March 9, 2009

DJBLU thank you for your job,

I tested it with SKY IT, log follows, all seems OK but resulting EPG in DVBViewer is empty. (I'm not using favorites, don't know if it could be a problem)

Channel.xml and EPG.xml are created and filled with all the data (files are too big to upload in this board)

I'm available for additional test if you need more info.

3/9/2009 23.39.01 : Welcome to OpenEPG to DVBViewer converter

3/9/2009 23.39.01 : ======================================

3/9/2009 23.39.01 : Written and compiled by DJBlu

3/9/2009 23.39.01 :

3/9/2009 23.39.01 : Loading ... Please Wait

3/9/2009 23.39.01 : Language set to Italiano

3/9/2009 23.39.31 : Woke up, beginning Channel and EPG grab, set to grab 8 days of information

3/9/2009 23.39.31 : Removing unwanted Events

3/9/2009 23.39.31 : Deleted 0 events

3/9/2009 23.39.31 : Connected to Transport Stream

3/9/2009 23.39.31 : Searching Network Information

3/9/2009 23.39.41 : Got Network Infomation 66 transponders found

3/9/2009 23.39.41 : OpenEPG version is Sky Italia, Loading huffman tree

3/9/2009 23.39.41 : Hufman tree loaded

3/9/2009 23.39.41 : Searching for channels .. please wait

3/9/2009 23.40.14 : Channel grab complete, found 425 channels, this count will reduce as not all are used.

3/9/2009 23.40.14 : Starting EPG grab

3/9/2009 23.40.32 : Found 39714 new EPG Entries

3/9/2009 23.40.32 : Data Grab Complete

3/9/2009 23.40.32 : Inspecting Channels .. Please wait

3/9/2009 23.40.32 : Channel's scanned and loaded.

3/9/2009 23.40.32 : Adding channels to list

3/9/2009 23.40.32 : Add to list complete

3/9/2009 23.40.32 : Decoding Huffman encoded summaries .. please wait

3/9/2009 23.40.53 : Huffman Decode complete

3/9/2009 23.40.53 : Saving EPG and Channels ... please wait

3/9/2009 23.40.56 : Channels and EPG Saved

3/9/2009 23.40.56 : Exporting EPG Data to DVBViewer

3/9/2009 23.40.59 : Export to DVBViewer complete

3/9/2009 23.40.59 : All tasks complete, going to sleep

March 9, 2009

DJBLU thank you for your job,

I tested it with SKY IT, log follows, all seems OK but resulting EPG in DVBViewer is empty. (I'm not using favorites, don't know if it could be a problem)

Channel.xml and EPG.xml are created and filled with all the data (files are too big to upload in this board)

I'm available for additional test if you need more info.

3/9/2009 23.39.01 : Welcome to OpenEPG to DVBViewer converter

3/9/2009 23.39.01 : ======================================

3/9/2009 23.39.01 : Written and compiled by DJBlu

3/9/2009 23.39.01 :

3/9/2009 23.39.01 : Loading ... Please Wait

3/9/2009 23.39.01 : Language set to Italiano

3/9/2009 23.39.31 : Woke up, beginning Channel and EPG grab, set to grab 8 days of information

3/9/2009 23.39.31 : Removing unwanted Events

3/9/2009 23.39.31 : Deleted 0 events

3/9/2009 23.39.31 : Connected to Transport Stream

3/9/2009 23.39.31 : Searching Network Information

3/9/2009 23.39.41 : Got Network Infomation 66 transponders found

3/9/2009 23.39.41 : OpenEPG version is Sky Italia, Loading huffman tree

3/9/2009 23.39.41 : Hufman tree loaded

3/9/2009 23.39.41 : Searching for channels .. please wait

3/9/2009 23.40.14 : Channel grab complete, found 425 channels, this count will reduce as not all are used.

3/9/2009 23.40.14 : Starting EPG grab

3/9/2009 23.40.32 : Found 39714 new EPG Entries

3/9/2009 23.40.32 : Data Grab Complete

3/9/2009 23.40.32 : Inspecting Channels .. Please wait

3/9/2009 23.40.32 : Channel's scanned and loaded.

3/9/2009 23.40.32 : Adding channels to list

3/9/2009 23.40.32 : Add to list complete

3/9/2009 23.40.32 : Decoding Huffman encoded summaries .. please wait

3/9/2009 23.40.53 : Huffman Decode complete

3/9/2009 23.40.53 : Saving EPG and Channels ... please wait

3/9/2009 23.40.56 : Channels and EPG Saved

3/9/2009 23.40.56 : Exporting EPG Data to DVBViewer

3/9/2009 23.40.59 : Export to DVBViewer complete

3/9/2009 23.40.59 : All tasks complete, going to sleep

Are you using Version 4.0 or higher?

March 9, 2009

Thank you for reply, now I'll try your new version, result will follows in few minutes.

March 9, 2009

This version seems not to finish correctly, latest message is exporting to DVBViewer but now it's more than 5 minutes that nothing happens (EPG in DVBViewer is empty)

3/9/2009 23.56.36 : Welcome to OpenEPG to DVBViewer converter

3/9/2009 23.56.36 : ======================================

3/9/2009 23.56.36 : Written and compiled by DJBlu

3/9/2009 23.56.36 :

3/9/2009 23.56.36 : Loading ... Please Wait

3/9/2009 23.56.36 : Language set to Italiano

3/9/2009 23.57.06 : Woke up, beginning Channel and EPG grab, set to grab 8 days of information

3/9/2009 23.57.06 : Removing unwanted Events

3/9/2009 23.57.06 : Deleted 0 events

3/9/2009 23.57.06 : Connected to Transport Stream

3/9/2009 23.57.06 : Searching Network Information

3/9/2009 23.57.15 : Got Network Infomation 66 transponders found

3/9/2009 23.57.15 : OpenEPG version is Sky Italia, Loading huffman tree

3/9/2009 23.57.15 : Hufman tree loaded

3/9/2009 23.57.15 : Searching for channels .. please wait

3/9/2009 23.57.48 : Channel grab complete, found 425 channels, this count will reduce as not all are used.

3/9/2009 23.57.48 : Starting EPG grab

3/9/2009 23.58.10 : Found 42524 new EPG Entries

3/9/2009 23.58.10 : Data Grab Complete

3/9/2009 23.58.10 : Inspecting Channels .. Please wait

3/9/2009 23.58.10 : Channel's scanned and loaded.

3/9/2009 23.58.10 : Adding channels to list

3/9/2009 23.58.10 : Add to list complete

3/9/2009 23.58.10 : Decoding Huffman encoded summaries .. please wait

3/9/2009 23.58.33 : Huffman Decode complete

3/9/2009 23.58.33 : Saving EPG and Channels ... please wait

3/9/2009 23.58.36 : Channels and EPG Saved

3/9/2009 23.58.36 : Exporting EPG Data to DVBViewer

March 9, 2009

OK, I understand the problem, restarting the application I receive these messages on log:

3/10/2009 0.06.30 : Welcome to OpenEPG to DVBViewer converter

3/10/2009 0.06.30 : ======================================

3/10/2009 0.06.30 : Written and compiled by DJBlu

3/10/2009 0.06.30 :

3/10/2009 0.06.30 : Loading ... Please Wait

3/10/2009 0.06.30 : Language set to Italiano

3/10/2009 0.06.30 : Found Channel file, Loading

3/10/2009 0.06.30 : Loaded 0 Channels

3/10/2009 0.06.30 : Found EPG File - Loading please wait .. This can take a few seconds

3/10/2009 0.06.35 : EPG File Loaded

Opening the channels.xml I found that it is empty.

EPG.xml is full, this is an extract of the beginning part of the file:

<?xml version="1.0" standalone="yes"?>

<EPG>

<Theme>Altri Programmi</Theme>

<Title>Phoenix</Title>

<Desc>Phoenix</Desc>

<IsNew>False</IsNew>

</EPG>

<EPG>

<Theme>Altri Programmi</Theme>

<Desc>Un canale di attualita' dove i tre filoni principali, sociale, culturale e ecclesiale, si fondono con sapienza ed armonia, per diffondere la cultura cristiana.</Desc>

<IsNew>False</IsNew>

</EPG>

<EPG>

<Theme>Altri Programmi</Theme>

<Title>Telepace</Title>

<Desc>Il ruolo di questa emittente e' di annunciare Cristo e diffondere la voce del Papa all'umanita'. Inoltre e' una sempre presente compagna di viaggio dei cammini istituzionali di Benedetto XVI.</Desc>

<IsNew>False</IsNew>

</EPG>

<EPG>

<Theme>Sport - Calcio</Theme>

<Title>Bochum - Bayern Monaco</Title>

<Desc>Bundesliga - 24a giornata. Le sfide piu' emozionanti del massimo campionato di calcio tedesco. Nella stagione 2007/08 ha vinto il titolo il Bayern Monaco.</Desc>

<IsNew>False</IsNew>

</EPG>

<EPG>

<Theme>Sport - Calcio</Theme>

<Title>Valencia - Recreativo</Title>

<Desc>Liga - 27a giornata. Le sfide piu' emozionanti del massimo campionato di calcio spagnolo. Nella stagione 2007/08 ha vinto il titolo il Real Madrid.</Desc>

<IsNew>False</IsNew>

</EPG>

March 9, 2009

I will be taking a look into 13e tommorow so I'll check it then I'll upload a copy then.

Sign In

Open EPG to DVB Viewer

Recommended Posts

uglyned

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Posted Images

Lars_MQ

Link to comment

uglyned

Link to comment

Topogigi

Link to comment

Lars_MQ

Link to comment

Topogigi

Link to comment

Lars_MQ

Link to comment

Topogigi

Link to comment

uglyned

Link to comment

Topogigi

Link to comment

uglyned

Link to comment

Lars_MQ

Link to comment

uglyned

Link to comment

DJBlu

Link to comment

DJBlu

Link to comment

Tjod

Link to comment

DJBlu

Link to comment

uglyned

Link to comment

Lars_MQ

Link to comment

DJBlu

Link to comment

DJBlu

Link to comment

Lars_MQ

Link to comment

DJBlu

Link to comment

Topogigi

Link to comment

DJBlu

Link to comment

The Gus

Link to comment

DJBlu

Link to comment

Lars_MQ

Link to comment

DJBlu

Link to comment

DJBlu

Link to comment

Lars_MQ

Link to comment

DJBlu

Link to comment

varesino

Link to comment

DJBlu

Link to comment

varesino

Link to comment

DJBlu

Link to comment