Integrating OTA and Pay TV Services Using Tvheadend: Part 5 EPG Configuration
In previous articles, we've discussed why Tvheadend provides a better viewing experience. We've also installed Tvheadend along with the necessary support tools. In the last two articles, we configured Tvheadend for use with an HDHomeRun and an HDMI encoder. In this article, we'll configure Tvheadend to grab and display electronic program guide (EPG) information.
Tvheadend supports three types of EPG grabbers: Over-the-air, Internal, and External.
Over-the-air (OTA) data is broadcast along with the digital video stream using a system called Event Information Table (EIT). EIT information is handy because it's free and available on most OTA channels. Unfortunately, the information supplied is limited in terms of content and time-span. If you're using an HDHomerun and don't want the possible expense or effort associated with setting up xmltv to provide additional information, EIT may be all you need. Tvheadend will,by default, scan OTA channels and load the EIT data into its program guide.
EPG information for IPTV networks, including our set-top-box (STB) IPTV network, has to come from an external source such as Schedules Direct or zap2it.com. Tvheadend provides two methods of downloading and importing this type of guide data: internal grabber modules and external grabber modules. For our purposes, the main difference between the two methods is that internal grabber modules handle the scheduling and downloading of EPG data for a single set of channels (lineups), while external grabber modules can download EPG data for multiple lineups and must be scheduled and executed externally using cron.
In the Tvheadend interface, open Configuration - Channel / EPG - EPG Grabber Modules
In our setup, we're going to turn off the use of EIT data and exclusively use EPG information from xmltv. It should be noted that EIT data and xmltv data can coexist for OTA channels, but if you leave EIT enabled, Tvheadend will periodically scan all OTA channels to download new EPG data. Ultimately, it's personal preference as to whether you enable or disable EIT.
Scroll through the EPG Grabber Name list and select any module that shows a green check (enabled) icon. Un-check Enabled on the parameters pane, then click Save.
Find, select, enable, and save the EPG grabber module that fits your needs. 'Internal XMLTV: North America (Data Direct)' is the simple choice for a setup with a single channel lineup. 'External: XMLTV' requires additional setup but will work in single or multiple lineup configurations.
If you're using the internal grabber, you need to tell Tvheadend where to find the xmltv configuration file you created when you installed and configured xmltv (see part 2 for details).
If you're using the external grabber, all parameters can be left at default. Simply select Enabled and click Save.
Open Configuration - Channel / EPG - EPG Grabber
This page is used to configure whether channel names, numbers, and icons will be determined using information in the xmltv data. Also, you can change the scheduled time and frequency for loading EPG data using internal EPG grabber modules. The defaults are fine in most cases.
Let's have a quick look at Configuration - Channel / EPG - Channel Tags.
You should see a few tags that were created automatically as we created our networks. These will appear as channel groups in Kodi and can be used for filtering and sorting.
Open Configuration - Channel / EPG - Channels
Here, you'll see a list of all of your configured channels. Notice the EPG source field is blank for all channels. Note that if you're using the internal grabber, the grabber may have already run and EPG sources will be shown. It may also take a few hours for the internal grabber to execute and determine the sources,
Open Configuration - Channel / EPG - EPG Grabber Channels.
Nothing there either. Let's fix that.
The procedure for internal grabbers is easy. Wait a few hours and check the logs. If things are configured correctly, eventually the grabber will execute and load guide data. Also (by default) the channels are associated with a specific channel within the guide by matching the Tvheadend channel name with the name used within the guide data.
External grabbers require a bit more work. We need a script to download guide data and feed that data into Tvheadend. We need to perform that procedure periodically, so we'll use a cron job to schedule the time and frequency.
If you worked through part 2, there is a sample script named grabepg in /home/hts/tvh. Here is the contents of that file:
#!/bin/bash
#grab the first SD lineup
tv_grab_na_dd --config-file=/home/hts/tvh/myxmltv.conf --output=/home/hts/tvh/myxmltv.xml --dd-data=/home/hts/tvh/dd.xml
#grab the second SD lineup without re-download
tv_grab_na_dd --config-file=/home/hts/tvh/myxmltv2.conf --output=/home/hts/tvh/myxmltv2.xml --dd-data=/home/hts/tvh/dd.xml --reprocess
#merge the two xml listing files into one
tv_cat /home/hts/tvh/myxmltv.xml /home/hts/tvh/myxmltv2.xml >/home/hts/tvh/myxmltvall.xml
#pump the epg data into Tvheadend
netcat -U /home/hts/.hts/tvheadend/epggrab/xmltv.sock < /home/hts/tvh/myxmltvall.xml
The shell script executes an xmltv grabber for the first lineup, outputs the xml lineup data and stashes all data in a temporary file for re-use. Then, the xmltv grabber is executed again (using the cached data to avoid re-download), and outputs the second xml lineup. The two lineups are concatenated into a single xml file containing both lineups. That all-inclusive file is then pushed into Tvheadend using a Unix socket opened when the external grabber was enabled.
You'll probably need to modify this script a bit for your purposes. You can add, delete, or change lineups as needed. One important note, check 'man netcat' on your system and make sure '-U' is the proper parameter for outputting to a Unix socket. Some systems may use '-u' or other. The '-U' notation is correct for Ubuntu 16.04.
Run the script from a terminal shell watching for any errors. It will likely take a while to complete depending on your lineup and channel count. Once the script completes successfully, run it a second time. It seems that the first time the data is fed to Tvheadend, it is used to create channel to EPG entry associations only. After the initial run, the guide data is loaded for each channel. Watch the Tvheadend log to see that new channels are found at the end of the first run and new broadcasts are found at the end of the second run.
Open Configuration - Channel / EPG - EPG Grabber Channels.
You should now see a list of channels supplied by xmltv and the associated Tvheadend channel name.
If there was a problem doing the name match on any or those channels, you will see a blank entry in the Channels column. You can manually set the association by selecting that list row, clicking Edit, and selecting the corresponding Tvheadend channel name from the Channels drop-down.
Open Electronic Program Guide
You should See the guide data loaded from xmltv. You can click on the information icon in the first column to see EPG information for each entry.
We have guide information, but we need to keep the information updated so that we continue to see what programs will be available in the coming weeks. We're going to do that update using a cron job. There are multiple way to schedule cron jobs, but the simple, safe method is to add an entry to the hts user's personal crontab. The format of a personal crontab entry is:
minute hour day-of-month month day-of-week command
Let's add an entry that will update the EPG data every night at 2:04 am. Enter this in a terminal:
sudo su hts
crontab -e
You may need to choose an editor, nano is simple and straight forward. Scroll to the bottom of the crontab file and add this line:
04 02 * * * /home/hts/tvh/grabepg
Save the file and exit.
cron is powerful and can be configured to automate a multitude of tasks. You can find more information here https://help.ubuntu.com/community/CronHowto.
If you've worked through each of the articles in this series, you now have a Tvheadend installation with multiple content sources and EPG information for all channels. Tvheadend can be used as a stand-alone interface to record and stream live content, but it also works really well with full featured front-ends like Kodi. You are probably already familiar with Kodi and possibly how to set up a back-end PVR within Kodi, but we'll do a quick run-through of that procdedure in the next article.
Tvheadend supports three types of EPG grabbers: Over-the-air, Internal, and External.
Over-the-air (OTA) data is broadcast along with the digital video stream using a system called Event Information Table (EIT). EIT information is handy because it's free and available on most OTA channels. Unfortunately, the information supplied is limited in terms of content and time-span. If you're using an HDHomerun and don't want the possible expense or effort associated with setting up xmltv to provide additional information, EIT may be all you need. Tvheadend will,by default, scan OTA channels and load the EIT data into its program guide.
EPG information for IPTV networks, including our set-top-box (STB) IPTV network, has to come from an external source such as Schedules Direct or zap2it.com. Tvheadend provides two methods of downloading and importing this type of guide data: internal grabber modules and external grabber modules. For our purposes, the main difference between the two methods is that internal grabber modules handle the scheduling and downloading of EPG data for a single set of channels (lineups), while external grabber modules can download EPG data for multiple lineups and must be scheduled and executed externally using cron.
In the Tvheadend interface, open Configuration - Channel / EPG - EPG Grabber Modules
In our setup, we're going to turn off the use of EIT data and exclusively use EPG information from xmltv. It should be noted that EIT data and xmltv data can coexist for OTA channels, but if you leave EIT enabled, Tvheadend will periodically scan all OTA channels to download new EPG data. Ultimately, it's personal preference as to whether you enable or disable EIT.
Scroll through the EPG Grabber Name list and select any module that shows a green check (enabled) icon. Un-check Enabled on the parameters pane, then click Save.
Find, select, enable, and save the EPG grabber module that fits your needs. 'Internal XMLTV: North America (Data Direct)' is the simple choice for a setup with a single channel lineup. 'External: XMLTV' requires additional setup but will work in single or multiple lineup configurations.
If you're using the internal grabber, you need to tell Tvheadend where to find the xmltv configuration file you created when you installed and configured xmltv (see part 2 for details).
If you're using the external grabber, all parameters can be left at default. Simply select Enabled and click Save.
Open Configuration - Channel / EPG - EPG Grabber
This page is used to configure whether channel names, numbers, and icons will be determined using information in the xmltv data. Also, you can change the scheduled time and frequency for loading EPG data using internal EPG grabber modules. The defaults are fine in most cases.
Let's have a quick look at Configuration - Channel / EPG - Channel Tags.
You should see a few tags that were created automatically as we created our networks. These will appear as channel groups in Kodi and can be used for filtering and sorting.
Open Configuration - Channel / EPG - Channels
Here, you'll see a list of all of your configured channels. Notice the EPG source field is blank for all channels. Note that if you're using the internal grabber, the grabber may have already run and EPG sources will be shown. It may also take a few hours for the internal grabber to execute and determine the sources,
Open Configuration - Channel / EPG - EPG Grabber Channels.
Nothing there either. Let's fix that.
The procedure for internal grabbers is easy. Wait a few hours and check the logs. If things are configured correctly, eventually the grabber will execute and load guide data. Also (by default) the channels are associated with a specific channel within the guide by matching the Tvheadend channel name with the name used within the guide data.
External grabbers require a bit more work. We need a script to download guide data and feed that data into Tvheadend. We need to perform that procedure periodically, so we'll use a cron job to schedule the time and frequency.
If you worked through part 2, there is a sample script named grabepg in /home/hts/tvh. Here is the contents of that file:
#!/bin/bash
#grab the first SD lineup
tv_grab_na_dd --config-file=/home/hts/tvh/myxmltv.conf --output=/home/hts/tvh/myxmltv.xml --dd-data=/home/hts/tvh/dd.xml
#grab the second SD lineup without re-download
tv_grab_na_dd --config-file=/home/hts/tvh/myxmltv2.conf --output=/home/hts/tvh/myxmltv2.xml --dd-data=/home/hts/tvh/dd.xml --reprocess
#merge the two xml listing files into one
tv_cat /home/hts/tvh/myxmltv.xml /home/hts/tvh/myxmltv2.xml >/home/hts/tvh/myxmltvall.xml
#pump the epg data into Tvheadend
netcat -U /home/hts/.hts/tvheadend/epggrab/xmltv.sock < /home/hts/tvh/myxmltvall.xml
The shell script executes an xmltv grabber for the first lineup, outputs the xml lineup data and stashes all data in a temporary file for re-use. Then, the xmltv grabber is executed again (using the cached data to avoid re-download), and outputs the second xml lineup. The two lineups are concatenated into a single xml file containing both lineups. That all-inclusive file is then pushed into Tvheadend using a Unix socket opened when the external grabber was enabled.
You'll probably need to modify this script a bit for your purposes. You can add, delete, or change lineups as needed. One important note, check 'man netcat' on your system and make sure '-U' is the proper parameter for outputting to a Unix socket. Some systems may use '-u' or other. The '-U' notation is correct for Ubuntu 16.04.
Run the script from a terminal shell watching for any errors. It will likely take a while to complete depending on your lineup and channel count. Once the script completes successfully, run it a second time. It seems that the first time the data is fed to Tvheadend, it is used to create channel to EPG entry associations only. After the initial run, the guide data is loaded for each channel. Watch the Tvheadend log to see that new channels are found at the end of the first run and new broadcasts are found at the end of the second run.
Open Configuration - Channel / EPG - EPG Grabber Channels.
You should now see a list of channels supplied by xmltv and the associated Tvheadend channel name.
If there was a problem doing the name match on any or those channels, you will see a blank entry in the Channels column. You can manually set the association by selecting that list row, clicking Edit, and selecting the corresponding Tvheadend channel name from the Channels drop-down.
Open Electronic Program Guide
You should See the guide data loaded from xmltv. You can click on the information icon in the first column to see EPG information for each entry.
We have guide information, but we need to keep the information updated so that we continue to see what programs will be available in the coming weeks. We're going to do that update using a cron job. There are multiple way to schedule cron jobs, but the simple, safe method is to add an entry to the hts user's personal crontab. The format of a personal crontab entry is:
minute hour day-of-month month day-of-week command
Let's add an entry that will update the EPG data every night at 2:04 am. Enter this in a terminal:
sudo su hts
crontab -e
You may need to choose an editor, nano is simple and straight forward. Scroll to the bottom of the crontab file and add this line:
04 02 * * * /home/hts/tvh/grabepg
Save the file and exit.
cron is powerful and can be configured to automate a multitude of tasks. You can find more information here https://help.ubuntu.com/community/CronHowto.
If you've worked through each of the articles in this series, you now have a Tvheadend installation with multiple content sources and EPG information for all channels. Tvheadend can be used as a stand-alone interface to record and stream live content, but it also works really well with full featured front-ends like Kodi. You are probably already familiar with Kodi and possibly how to set up a back-end PVR within Kodi, but we'll do a quick run-through of that procdedure in the next article.
Comments
Post a Comment