[[wdtv:wdtv_umsp_crash_course_v2]]
Trace: » wdtv_umsp_crash_course_v2
Index

Index

Table of Contents

WDTV UMSP Crash Course Version 2 ( Beta )

Admit that you have always wanted to know how to develop a UMSP Plugins but never had the time to learn. Here I will walk you through and at the end of the course we will have created a fully functional UMSP Plugin!

Day 1

Our objective

We will create a youtube umsp plugin fully configurable and more flexible. It will possible to stream all youtube video searched by user or by a custom search pattern. Let`s begin with a Hello World course style.

Registration UMSP compliant plugins

The first thing to do is to create a root folder named Hello World. Put the following code in use /tmp/conf directory in a file named umsp.php. If it is not exist, create it. 

<?php
 
global $myMediaItems;
 
$myMediaItems[] = array (   
                        'id'             => 'umsp://plugins/hello_world',
                        'parentID'       => '0',
                        'dc:title'       => 'Hello World',
                        'upnp:class'     => 'object.container',
                        'upnp:album_art' => '',
                        );
 
?>

where:

  1. id: is the url similar a http url. Infact umsp://plugins/hello_world point to hello_world.php file
  2. parentID: is the parent umsp id, 0 means root or the head
  3. dc:title: string to visualize to umsp menu
  4. upnp:class : type of upnp object, object.container means a folder type
  5. upnp:album_art : icon

Configure plugin to point a media resources

The url: 

umsp://plugins/hello_world

point to the following file hello_world.php: 

<?php
    function _pluginMain($prmQuery) {
 
    $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        => 'Hello World - SubItem01',
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://rkpisanu.altervista.org/wdtv/Hello.flv'),
            'protocolInfo'    => '*:*:*:*',
    );
 
 
    return $Items;
    }
?>

Create temporary directory named umsp-plugins in /tmp: 

mkdir -p /tmp/umsp-plugins
cd /tmp/umsp-plugins

and put the file hello_world.php in /tmp/umsp-plugins.

Now go to the wdtv and select UMSP. Yeah, folder Hello World is alive, but give us a error “There is no media in the current folder.”. Something goes wrong.

This is the screenshots: wdtv_umsp_crash_course_sg_001

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_01/umsp_crash_course_v2_Day_01.zip

Find out the Hello World bug's, in other word Debug

Fortunately for http stream we can debug the php code in web browser ( Internet Explorer, Firefox, etc… ).

Go to this url: 

http://wdtvlive-IP/umsp/umsp-test.php

in my case ip is 192.168.2.3 : 

http://192.168.2.3/umsp/umsp-test.php

and the bug came into the open. This is the php error: 

  Parse error: syntax error, unexpected ',' in /tmp/umsp-plugins/hello_world.php on line 9

So in line 9 there is a ”)” unwanted, then remove it. This is the correct php code: 

<?php
    function _pluginMain($prmQuery) {
 
    $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        => 'Hello World - SubItem01',
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://rkpisanu.altervista.org/wdtv/Hello.flv',
            'protocolInfo'    => '*:*:*:*',
    );
 
 
    return $Items;
    }
?>

Return to WDTV and try to watch video.

This time, it works. Infact subfolder Hello World - SubItem01 is visible and after click it, the video streaming started.

This is the screenshots: wdtv_umsp_crash_course_sg_002

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_01/umsp_crash_course_v2_Day_01.zip

Day 2

Remember to backup your work

When we restart or power off our wdtv, we lose submenu because /tmp/umsp-plugins directory is a temporary directory and not data saved definitively. To avoid this issue, create an archive file umsp-plugins.tgz in the /tmp/conf persistent folder on the WDTV : 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php
mv -f umsp-plugins.tgz /tmp/conf

but remember, /tmp/conf is a system configuration directory to store a little persistent data in your wdtv, so for huge file project we use to store ours data in usb stick. After reboot, the firmware uncompress automatically umsp-plugins.tgz file and put all contents in /tmp/umsp-plugins.

Simple Hello World Http Proxy

Create the file proxy_hello_world.php and put it in /tmp/umsp-plugins: 

<?php
//Needed for header
ob_start();
 
$url="http://rkpisanu.altervista.org/wdtv/Hello.flv";
 
header( "Location: $url" );
//Needed for header
ob_flush();
?>

This proxy is very simple: it redirects to the web site without any changes. The hello_world.php file with SubItem01 Item now point to localhost (127.0.0.1) webserver proxy. 

<?php
    function _pluginMain($prmQuery) {
 
    $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        => 'Hello World - SubItem01',
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php',
            'protocolInfo'    => '*:*:*:*',
    );
 
 
    return $Items;
    }
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_02/umsp_crash_course_v2_Day_02.zip

Why http proxy is so useful

The youtube url of our video is http://www.youtube.com/watch?v=SEfhM3T4aV0 , but it is not usable directly in umsp plugin because Youtube use a sort of load balancer and protection to avoid download it directly. The real video url decripted is something like this:

http://o-o.preferred.seabone-mil1.v11.lscache2.c.youtube.com/videoplayback?sparams=id,expire,ip,ipbits,itag,ratebypass&fexp=901700&itag=43&ip=87.0.0.0&signature=204AE4FB98877C262CEE078B92DAB52D4036767F.D176A16AFD8BAE76937F1FA1B5DB690077C4A3CD&sver=3&ratebypass=yes&expire=1311523200&key=yt1&ipbits=8&id=4847e13374f8695d

This url is expired now.

Whe need a script who get youtube url and put the real video url. Don't worry, here it is:

File yt_url.sh : 

#!/usr/bin/env bash 
wget -UF -qO- -c "$1" | grep videoplayback | grep "embed type" | sed "s/;/\\`echo -e '\n\r'`/g" | grep url_encoded_fmt_stream_map | sed -e 's/%7C/|/g' -e 's/%253A/:/g' -e 's/%252C/,/g' -e 's/%2522/\"/g' -e 's/%253B/;/g' -e 's/%252F/\//g' -e 's/%253F/\?/g'  -e 's/%253D/=/g' -e 's/%3D/=/g' -e 's/%2B/+/g' -e 's/%25252C/,/g' -e 's/%2526/\&/g' -e 's/%26/\&/g' -e 's/%2C/|/g' |  sed "s/|/\\`echo -e '\n\r'`/g" |  grep -o "http:.*" | grep mp4 | tail -1 | cut -d";" -f1

Usage: 

./yt_url.sh http://www.youtube.com/watch?v=SEfhM3T4aV0

http://o-o.preferred.seabone-mil1.v20.lscache8.c.youtube.com/videoplayback?sparams=id,expire,ip,ipbits,itag,source,ratebypass,cp&itag=18&ip=87.0.0.0&signature=8D06FA7AAC03403E36AAA64AB1F4AF0DBBBD471C.802CC26AC26FA0BCC11068650FDD4B8FF202292A&sver=3&ratebypass=yes&source=youtube&expire=1319407200&key=yt1&ipbits=8&cp=U0hQTlRLVV9FSkNOMF9MRVlDOlY1ZWFmdUotdVpR&id=4847e13374f8695d&quality=medium&fallback_host=tc.v20.cache8.c.youtube.com&type=video/mp4

Now create the file yt_url.sh and put it in /tmp/umsp-plugins with right permission ( chmod +x yt_url.sh ).

The file /tmp/umsp-plugins/proxy_hello_world.php now run the yt_url.sh shell and redirect to the url received in output: 

<?php
//Needed for header
ob_start();
 
$url = shell_exec("/tmp/umsp-plugins/yt_url.sh http://www.youtube.com/watch?v=SEfhM3T4aV0");
 
header( "Location: $url" );
//Needed for header
ob_flush();
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_02/umsp_crash_course_v2_Day_02.zip

What plugin does behind the scenes?

What plugin does behind the scenes? From here our plugin acts as a black box. We know the input, output, but not what it happens in the various steps. For simple projects it can be fine, but complex projects need to be sure what we want is actually executed. To achieve this, we need a new tool called logging. It will draw step by step everything that our plugin running.

Now the file /tmp/umsp-plugins/proxy_hello_world.php with logging become: 

<?php
//Needed for header
ob_start();
 
include_once('/usr/share/umsp/funcs-log.php');
 
global $logLevel;
global $logIdent;
$logLevel = L_ALL ;
$logIdent = 'HelloWorldPlugIn';
 
_logDebug("Begin");
$url = shell_exec("/tmp/umsp-plugins/yt_url.sh http://www.youtube.com/watch?v=SEfhM3T4aV0");
_logDebug("url = " . $url);
 
header( "Location: $url" );
 
_logDebug("End");
 
//Needed for header
ob_flush();
?>
?>

Now go to view log file /tmp/umsp-log.txt: 

 
root@WDTVLIVE~:# tail -f /tmp/umsp-log.txt
2011.07.24 09:29:51 HelloWorldPlugIn - Begin
2011.07.24 09:29:53 HelloWorldPlugIn - url = http://o-o.preferred.seabone-mil1.v11.lscache2.c.youtube.com/videoplayback?sparams=id,expire,ip,ipbits,itag,algorithm,burst,factor&algorithm=throttle-factor&itag=43&ip=87.0.0.0&burst=40&sver=3&signature=423419BDFDE1F637D9E6D9D8E85122B60846F6E6.0803B689CE39E47608FD95B59D7028473EF8BE1A&expire=1311523200&key=yt1&ipbits=8&factor=1.25&id=4847e13374f8695d

2011.07.24 09:29:53 HelloWorldPlugIn - End

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_02/umsp_crash_course_v2_Day_02.zip

Time passes memories fade

What will happen two months later when we reread the plugin code ? We do not remember most of the reasons why we have developed those programming instructions. Comment the code is a must to avoid to loss work and time. 

<?php
//Needed for header
ob_start();
 
#Load log function and inizialize it
include_once('/usr/share/umsp/funcs-log.php');
 
global $logLevel;
global $logIdent;
$logLevel = L_ALL ;
$logIdent = 'HelloWorldPlugIn';
 
#Get real Youtube Video Url 
_logDebug("Begin");
$url = shell_exec("/tmp/umsp-plugins/yt_url.sh http://www.youtube.com/watch?v=SEfhM3T4aV0");
_logDebug("url = " . $url);
 
#Redirect to real Youtube Video Url
header( "Location: $url" );
 
_logDebug("End");
 
//Needed for header
ob_flush();
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_02/umsp_crash_course_v2_Day_02.zip

Day 3

Pass parameter to proxy

Now it is time to pass a parameter to proxy in the same manner of Youtube Url to make it a bit dynamic:

Original Url: http://www.youtube.com/watch?v=SEfhM3T4aV0”

Proxy Url: http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=SEfhM3T4aV0

The hello_world.php file with this tricks, can contain more SubItemXX to point several Youtube Video: 

<?php
    function _pluginMain($prmQuery) {
 
    $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        => 'Hello World - SubItem01',
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=SEfhM3T4aV0',
            'protocolInfo'    => '*:*:*:*',
    );
 
 
    $Items[] = array(   
            'id'              => 'SubItem02',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        => 'Hello World - SubItem02',
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=N5bEbgBBAyU',
            'protocolInfo'    => '*:*:*:*',
    );
 
    return $Items;
    }
?>

proxy_hello_world.php becames: 

<?php
//Needed for header
ob_start();
 
#Load log function and inizialize it
include_once('/usr/share/umsp/funcs-log.php');
 
global $logLevel;
global $logIdent;
$logLevel = L_ALL ;
$logIdent = 'HelloWorldPlugIn';
 
_logDebug("Begin");
 
//Get Youtube url vars
$v = $_GET['v'];
_logDebug("v = " . $v);
 
 
#Get real Youtube Video Url 
$url = shell_exec("/tmp/umsp-plugins/yt_url.sh http://www.youtube.com/watch?v=" . $v);
 
_logDebug("url = " . $url);
 
#Redirect to real Youtube Video Url
header( "Location: $url" );
 
_logDebug("End");
 
//Needed for header
ob_flush();
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_03/umsp_crash_course_v2_Day_03.zip

Remove Shell Proxy and Use PHP Youtube Proxy

Shell proxy is easy to develop but it is a little slow, because it is fork a lots of process. Now this is a php proxy entirely rewritten in php. It is a good example of data scraping. It use WDLXTV Proxy Routine (Credit to the WDLXTV Authors). 

<?php
//Needed for header
ob_start();
//User Agent
ini_set('user_agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9');
//Make Dinamic Link
$link = 'http://www.youtube.com/watch?v=' . $_GET['v'];
//Get HTML Page  
$content=file_get_contents($link);
 
preg_match('/url_encoded_fmt_stream_map=([^&$]*)/', $content, $map);
foreach(explode(',',urldecode($map[1])) as $url_map) 
  {
  preg_match('/&itag=(\d+)/', urldecode($url_map), $url_quality);
  preg_match('/url=(.*)/', $url_map, $urlentry);
  $decoded_url = urldecode($urlentry[1]);
  if (strpos($decoded_url,';')) 
    {
    $hash_qlty_url[$url_quality[1]] = SubStr($decoded_url,0,strpos($decoded_url,';'));
    } 
  else
    {
    $hash_qlty_url[$url_quality[1]] = SubStr($decoded_url,0,strrpos($decoded_url,"&itag"));
    }
  unset($url_quality);
  unset($url);
  }
$quality = "360P";
if (file_exists('/conf/config')) 
  {
  $config = file_get_contents('/conf/config');
  if(preg_match('/YOUTUBE_QUALITY=\'(.+)\'/', $config, $config_quality)) 
    {
    $quality = $config_quality[1];
    }
  }
$quality_map = array('1080P' => 37, '720P' => 22, '480P' => 35, '360P' => 18, '270P' => 18);
$quality_map_values = array_values($quality_map);
$index = array_search($quality_map[$quality], $quality_map_values);
while (!isset($hash_qlty_url[$quality_map_values[$index]]) && ($index < (count($quality_map_values) - 1))) 
  {
  $index++;
  }
$quality = $quality_map_values[$index];
$url=$hash_qlty_url[$quality];
 
header( "Location: $url" );
//Needed for header
ob_flush();
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_03/umsp_crash_course_v2_Day_03.zip

Refactoring - Improving the design of existing code

Refactoring is a technique for restructuring an existing body of code, altering its internal structure without changing its external behavior. Since each refactoring is small, it's less likely to go wrong. The system is also kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring. Why is it so important ? Because Making our project easier to navigate and find stuff. We should refactor our project structure so the next guy knows how to find/add what is needed.

Some basic rules:

  1. put the code more than 5 lines in functions
  2. the main must be readable, very small and represent the logical flow of high-level program
  3. put functions in a library file for sharing and reusing code
  4. put bit of code in small file for sharing and reusing code

Day 4

Add a Youtube Title and ThumbNail

Parsing Youtube html page, we have discovered Youtube Title and ThumbNail ( Icon ) tag and they have been added to our umsp code. The function GetYTData parses a real Youtube URL as a parameter. Now you can configure a url as an id.

File hello_world.php : 

<?php
//-------------------------------Main Block-------------------------------------------------------
//---Load Library---
#Load log function and inizialize it
include_once('/usr/share/umsp/funcs-log.php');
global $logLevel;
global $logIdent;
$logLevel = L_ALL ;
$logIdent = 'HelloWorldPlugIn';
_logDebug("Begin");
 
#Load scraper_lib function
require('/usr/share/umsp/plugins/scraper_lib.php');
 
function _pluginMain($prmQuery) 
  {
  //Get Youtube Data Array
  $Item=GetYTData('http://www.youtube.com/watch?v=SEfhM3T4aV0&feature=related');
  $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=SEfhM3T4aV0',
            'protocolInfo'    => '*:*:*:*',
  );
 
   //Get Youtube Data Array
  $Item=GetYTData('N5bEbgBBAyU'); 
  $Items[] = array(   
            'id'              => 'SubItem02',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=N5bEbgBBAyU',
            'protocolInfo'    => '*:*:*:*',
  );
 
  return $Items;
  }
 
 
?>

File scraper_lib.php : 

<?php
//------------------------Begin Define function Block-----------------------------------------------
//Extract string between 2 tag string
function getResult($str,$hash)
  {
  $p=array();
  $p[0]=strpos($str,$hash[0]);
  $p[1]=strpos($str,$hash[1],$p[0]);
  return substr($str,$p[0]+strlen($hash[0]),$p[1]-$p[0]-strlen($hash[1])+1);
  }
 
//Get Youtube Data Parsing HTML from ID or Complete URL
function GetYTData($v)
  {
  //YTData
  $YTData = array();
  //Parse Url or ID
  $queryData = array();
  $url = parse_url($v); 
  parse_str($url['query'], $queryData);
  //If Youtube Complete Url [http://www.youtube.com/watch?v=ID], get the ID
  if ($queryData['v'] !='') 
    { 
    $v=$queryData['v'];
    }
  //Get html result from Youtube Url
  $link="http://www.youtube.com/watch?v=" . $v;
  $content=file_get_contents($link);
  //Get Youtube Title
  preg_match('/<meta property="og:title" content="(.*?)"/i',$content,$title);
  //Save Youtube Title in array
  $YTData['title']=html_entity_decode($title[1], ENT_QUOTES);
  //Get Youtube ThumbNail
  preg_match('/<meta property="og:image" content="(.*?)"/i',$content,$image);
  //Save Youtube ThumbNail in array
  $YTData['image']=$image[1];
  return $YTData; 
  }
 
 
//------------------------End Define function Block-----------------------------------------------
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh scraper_lib.php
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_04/umsp_crash_course_v2_Day_04.zip

Read a list of Youtube Url from FileList

Now you can configure a Youtube Url list in a file.

File yt_list_misc.lst : 

http://www.youtube.com/watch?v=SEfhM3T4aV0&feature=related
http://www.youtube.com/watch?v=g5epPS9O0Mo&feature=related
http://www.youtube.com/watch?v=UaqISEs_uj0&feature=related
http://www.youtube.com/watch?v=cW7BvabYnn8
http://www.youtube.com/watch?v=CEQouX5U0fc&feature=channel
http://www.youtube.com/watch?v=5OFThORmR-s&feature=channel

File hello_world.php : 

<?php
//-------------------------------Main Block-------------------------------------------------------
//---Load Library---
#Load log function and inizialize it
include_once('/usr/share/umsp/funcs-log.php');
global $logLevel;
global $logIdent;
$logLevel = L_ALL ;
$logIdent = 'HelloWorldPlugIn';
_logDebug("Begin");
 
#Load scraper_lib function
require('/usr/share/umsp/plugins/scraper_lib.php');
 
function _pluginMain($prmQuery) 
  {
  //Read a list of Youtube Url from file
  $filename = "/tmp/umsp-plugins/yt_list_misc.lst"; // pathname of file
  $Items=GetYTUrlByFileList($filename);
  return $Items;
  }
 
 
?>

File scraper_lib.php : 

<?php
//------------------------Begin Define function Block-----------------------------------------------
//Extract string between 2 tag string
function getResult($str,$hash)
  {
  $p=array();
  $p[0]=strpos($str,$hash[0]);
  $p[1]=strpos($str,$hash[1],$p[0]);
  return substr($str,$p[0]+strlen($hash[0]),$p[1]-$p[0]-strlen($hash[1])+1);
  }
 
//Get Youtube Data Parsing HTML from ID or Complete URL
function GetYTData($v)
  {
  //YTData
  $YTData = array();
  //Parse Url or ID
  $queryData = array();
  $url = parse_url($v); 
  parse_str($url['query'], $queryData);
  //If Youtube Complete Url [http://www.youtube.com/watch?v=ID], get the ID
  if ($queryData['v'] !='') 
    { 
    $v=$queryData['v'];
    }
  //Get html result from Youtube Url
  $link="http://www.youtube.com/watch?v=" . $v;
  $content=file_get_contents($link);
  //Get Youtube Title
  preg_match('/<meta property="og:title" content="(.*?)"/i',$content,$title);
  //Save Youtube Title in array
  $YTData['title']=html_entity_decode($title[1], ENT_QUOTES);
  //Get Youtube ThumbNail
  preg_match('/<meta property="og:image" content="(.*?)"/i',$content,$image);
  //Save Youtube ThumbNail in array
  $YTData['image']=$image[1];
  //Save Youtube ID in array
  $YTData['id']=$v;
  return $YTData; 
  }
 
//Read a list of Youtube Url from file
function GetYTUrlByFileList($filename) 
  {
  //If file exists than parsing else return a empty list
  if (file_exists($filename))
    {
    $lines = file($filename);
    foreach ($lines as $line) 
      {
      //Get Youtube Data Array
      $Item=GetYTData($line);
      $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
      );
      } 
    }
  return $Items;
  }
 
//------------------------End Define function Block-----------------------------------------------
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh scraper_lib.php yt_list_misc.lst
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_04/umsp_crash_course_v2_Day_04.zip

Day 5

Read a list of Youtube Url from PlayList

Now it is possible to configure a Youtube Playlist with GetYTUrlByPlayList function. One problem is necessary to resolve:

  1. time to parse all youtube video

It is not simply to resolve, because it is necessary to split the parsing step with the umsp discover step.

File hello_world.php : 

<?php
//-------------------------------Main Block-------------------------------------------------------
//---Load Library---
#Load log function and inizialize it
include_once('/usr/share/umsp/funcs-log.php');
global $logLevel;
global $logIdent;
$logLevel = L_ALL ;
$logIdent = 'HelloWorldPlugIn';
_logDebug("Begin");
 
#Load scraper_lib function
require('/usr/share/umsp/plugins/scraper_lib.php');
 
function _pluginMain($prmQuery) 
  {
  //Read a list of Youtube Url from PlayList
  $Items=GetYTUrlByPlayList('http://www.youtube.com/watch?v=4ZZGIw7FZgQ&playnext=1&list=PLE4F2ECD7A43466D2');
  //$Items=GetYTUrlByPlayList('PLE4F2ECD7A43466D2');
  return $Items;
  }
 
 
?>

File scraper_lib.php : 

<?php
//------------------------Begin Define function Block-----------------------------------------------
//Extract string between 2 tag string
function getResult($str,$hash)
  {
  $p=array();
  $p[0]=strpos($str,$hash[0]);
  $p[1]=strpos($str,$hash[1],$p[0]);
  return substr($str,$p[0]+strlen($hash[0]),$p[1]-$p[0]-strlen($hash[1])+1);
  }
 
//Get Youtube Data Parsing HTML from ID or Complete URL
function GetYTData($v)
  {
  //YTData
  $YTData = array();
  //Parse Url or ID
  $queryData = array();
  $url = parse_url($v); 
  parse_str($url['query'], $queryData);
  //If Youtube Complete Url [http://www.youtube.com/watch?v=ID], get the ID
  if ($queryData['v'] !='') 
    { 
    $v=$queryData['v'];
    }
  //Get html result from Youtube Url
  $link="http://www.youtube.com/watch?v=" . $v;
  $content=file_get_contents($link);
  //Get Youtube Title
  preg_match('/<meta property="og:title" content="(.*?)"/i',$content,$title);
  //Save Youtube Title in array
  $YTData['title']=html_entity_decode($title[1], ENT_QUOTES);
  //Get Youtube ThumbNail
  preg_match('/<meta property="og:image" content="(.*?)"/i',$content,$image);
  //Save Youtube ThumbNail in array
  $YTData['image']=$image[1];
  //Save Youtube ID in array
  $YTData['id']=$v;
  return $YTData; 
  }
 
//Read a list of Youtube Url from file
function GetYTUrlByFileList($filename) 
  {
  //If file exists than parsing else return a empty list
  if (file_exists($filename))
    {
    $lines = file($filename);
    foreach ($lines as $line) 
      {
      //Get Youtube Data Array
      $Item=GetYTData($line);
      if ($Item['title']=="") continue;
      $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
      );
      } 
    }
  return $Items;
  }
 
//Read a list of Url from Youtube PlayList
function GetYTUrlByPlayList($PlUrl) 
  {
  //Parse Url
  $queryData = array();
  $url = parse_url($PlUrl); 
  parse_str($url['query'], $queryData);
  //var_dump($queryData);
  $PLID=$queryData['list'];
  if ( $PLID =='') $PLID = $PlUrl;
  if (substr($PLID,0,2)=='PL')
    {
    $PLID=substr($PLID,2);
    }
  $PLURLDATA='http://www.youtube.com/view_play_list?p=' . $PLID . '&playnext=1';
  //Get html result from Youtube Playlist Url
  $content=file_get_contents($PLURLDATA);
  //Get Youtube PlayList data-video-ids
  preg_match('/<div id="playlist-bar".*data-video-ids="(.*?)"/i',$content,$YTVars);
  //Explode Ids in array
  $YTIDS=explode(',', $YTVars[1]);
  foreach ($YTIDS as $line) 
    {
    //Get Youtube Data Array
    $Item=GetYTData($line);
    if ($Item['title']=="") continue;
    $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
    );
    } 
  return $Items;
  }
 
//------------------------End Define function Block-----------------------------------------------
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh scraper_lib.php yt_list_misc.lst fly_streams.php
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_05/umsp_crash_course_v2_Day_05.zip

Read a list of Youtube Url from SearchList

Now it is possible to configure a Youtube Search with GetYTUrlBySearchList function.

File hello_world.php : 

<?php
//-------------------------------Main Block-------------------------------------------------------
//---Load Library---
#Load log function and inizialize it
include_once('/usr/share/umsp/funcs-log.php');
global $logLevel;
global $logIdent;
$logLevel = L_ALL ;
$logIdent = 'HelloWorldPlugIn';
_logDebug("Begin");
 
#Load scraper_lib function
require('/usr/share/umsp/plugins/scraper_lib.php');
 
function _pluginMain($prmQuery) 
  {
  //Read a list of Youtube Url from SearchList
  $Items=GetYTUrlBySearchList('http://www.youtube.com/results?search_query=english+kids&suggested_categories=27%2C24%2C22&page=1');
  return $Items;
  }
 
 
?>

File scraper_lib.php : 

<?php
//------------------------Begin Define function Block-----------------------------------------------
//Extract string between 2 tag string
function getResult($str,$hash)
  {
  $p=array();
  $p[0]=strpos($str,$hash[0]);
  $p[1]=strpos($str,$hash[1],$p[0]);
  return substr($str,$p[0]+strlen($hash[0]),$p[1]-$p[0]-strlen($hash[1])+1);
  }
 
//Get Youtube Data Parsing HTML from ID or Complete URL
function GetYTData($v)
  {
  //YTData
  $YTData = array();
  //Parse Url or ID
  $queryData = array();
  $url = parse_url($v); 
  parse_str($url['query'], $queryData);
  //If Youtube Complete Url [http://www.youtube.com/watch?v=ID], get the ID
  if ($queryData['v'] !='') 
    { 
    $v=$queryData['v'];
    }
  //Get html result from Youtube Url
  $link="http://www.youtube.com/watch?v=" . $v;
  $content=file_get_contents($link);
  //Get Youtube Title
  preg_match('/<meta property="og:title" content="(.*?)"/i',$content,$title);
  //Save Youtube Title in array
  $YTData['title']=html_entity_decode($title[1], ENT_QUOTES);
  //Get Youtube ThumbNail
  preg_match('/<meta property="og:image" content="(.*?)"/i',$content,$image);
  //Save Youtube ThumbNail in array
  $YTData['image']=$image[1];
  //Save Youtube ID in array
  $YTData['id']=$v;
  return $YTData; 
  }
 
//Read a list of Youtube Url from file
function GetYTUrlByFileList($filename) 
  {
  //If file exists than parsing else return a empty list
  if (file_exists($filename))
    {
    $lines = file($filename);
    foreach ($lines as $line) 
      {
      //Get Youtube Data Array
      $Item=GetYTData($line);
      if ($Item['title']=="") continue;
      $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
      );
      } 
    }
  return $Items;
  }
 
//Read a list of Url from Youtube PlayList
function GetYTUrlByPlayList($PlUrl) 
  {
  //Parse Url
  $queryData = array();
  $url = parse_url($PlUrl); 
  parse_str($url['query'], $queryData);
  //var_dump($queryData);
  $PLID=$queryData['list'];
  if ( $PLID =='') $PLID = $PlUrl;
  if (substr($PLID,0,2)=='PL')
    {
    $PLID=substr($PLID,2);
    }
  $PLURLDATA='http://www.youtube.com/view_play_list?p=' . $PLID . '&playnext=1';
  //Get html result from Youtube Playlist Url
  $content=file_get_contents($PLURLDATA);
  //Get Youtube PlayList data-video-ids
  preg_match('/<div id="playlist-bar".*data-video-ids="(.*?)"/i',$content,$YTVars);
  //Explode Ids in array
  $YTIDS=explode(',', $YTVars[1]);
  foreach ($YTIDS as $line) 
    {
    //Get Youtube Data Array
    $Item=GetYTData($line);
    if ($Item['title']=="") continue;
    $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
    );
    } 
  return $Items;
  }
 
//Read a list of Url from Youtube SearchList
function GetYTUrlBySearchList($SearchUrl) 
  {
  _logInfo("GetYTUrlBySearchList-->SearchUrl = " . $SearchUrl);
  //Parse Url
  $queryData = array();
  $url = parse_url($SearchUrl); 
  _logInfo("GetYTUrlBySearchList-->url = " . print_r($url,true));  
  parse_str($url['query'], $queryData);
  _logInfo("GetYTUrlBySearchList-->queryData = " . print_r($queryData,true));
  for ($page_index = 1; $page_index <= 2; $page_index++) 
    {
    _logInfo("GetYTUrlBySearchList-->page_index = " . $page_index );
    $search_query = preg_replace("/ /","+",$queryData[search_query]);
    _logInfo("GetYTUrlBySearchList-->search_query = " . $search_query );
    $url_page='http://' . $url[host] .  $url[path] . '?search_query=' . $search_query . '&page=' . $page_index;
    _logInfo("GetYTUrlBySearchList-->url_page = " . $url_page);
    $content=file_get_contents($url_page);
    _logDebug("GetYTUrlBySearchList-->content = " . $content);
    preg_match_all('/\/watch\?v=(.*?)" class="ux-thumb-wrap result-item-thumb"/i', $content, $ids);
    _logInfo("GetYTUrlBySearchList-->ids = " . print_r($ids,true));
    foreach ($ids[1] as $line) 
      {
      //Get Youtube Data Array
      $Item=GetYTData($line);
      if ($Item['title']=="") continue;
      $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
      );
      } 
    }
  _logInfo("GetYTUrlBySearchList-->Items = " . print_r($Items,true));	
  return $Items;	
  }
 
//------------------------End Define function Block-----------------------------------------------
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh scraper_lib.php yt_list_misc.lst
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_05/umsp_crash_course_v2_Day_05.zip

Day 6

Read a list of Youtube Url from UserList

Now it is possible to configure a Youtube Search with GetYTUrlByUserList function.

File hello_world.php : 

<?php
//-------------------------------Main Block-------------------------------------------------------
//---Load Library---
#Load log function and inizialize it
include_once('/usr/share/umsp/funcs-log.php');
global $logLevel;
global $logIdent;
$logLevel = L_ALL ;
$logIdent = 'HelloWorldPlugIn';
_logDebug("Begin");
 
#Load scraper_lib function
require('/usr/share/umsp/plugins/scraper_lib.php');
 
function _pluginMain($prmQuery) 
  {
  //Read a list of Youtube Url from UserList
  $Items=GetYTUrlByUserList('http://www.youtube.com/user/wearebusybeavers');
  return $Items;
  }
 
 
?>

File scraper_lib.php : 

<?php
//------------------------Begin Define function Block-----------------------------------------------
//Extract string between 2 tag string
function getResult($str,$hash)
  {
  $p=array();
  $p[0]=strpos($str,$hash[0]);
  $p[1]=strpos($str,$hash[1],$p[0]);
  return substr($str,$p[0]+strlen($hash[0]),$p[1]-$p[0]-strlen($hash[1])+1);
  }
 
//Get Youtube Data Parsing HTML from ID or Complete URL
function GetYTData($v)
  {
  //YTData
  $YTData = array();
  //Parse Url or ID
  $queryData = array();
  $url = parse_url($v); 
  parse_str($url['query'], $queryData);
  //If Youtube Complete Url [http://www.youtube.com/watch?v=ID], get the ID
  if ($queryData['v'] !='') 
    { 
    $v=$queryData['v'];
    }
  //Get html result from Youtube Url
  $link="http://www.youtube.com/watch?v=" . $v;
  $content=file_get_contents($link);
  //Get Youtube Title
  preg_match('/<meta property="og:title" content="(.*?)"/i',$content,$title);
  //Save Youtube Title in array
  $YTData['title']=html_entity_decode($title[1], ENT_QUOTES);
  //Get Youtube ThumbNail
  preg_match('/<meta property="og:image" content="(.*?)"/i',$content,$image);
  //Save Youtube ThumbNail in array
  $YTData['image']=$image[1];
  //Save Youtube ID in array
  $YTData['id']=$v;
  return $YTData; 
  }
 
//Read a list of Youtube Url from file
function GetYTUrlByFileList($filename) 
  {
  //If file exists than parsing else return a empty list
  if (file_exists($filename))
    {
    $lines = file($filename);
    foreach ($lines as $line) 
      {
      //Get Youtube Data Array
      $Item=GetYTData($line);
      if ($Item['title']=="") continue;
      $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
      );
      } 
    }
  return $Items;
  }
 
//Read a list of Url from Youtube PlayList
function GetYTUrlByPlayList($PlUrl) 
  {
  //Parse Url
  $queryData = array();
  $url = parse_url($PlUrl); 
  parse_str($url['query'], $queryData);
  //var_dump($queryData);
  $PLID=$queryData['list'];
  if ( $PLID =='') $PLID = $PlUrl;
  if (substr($PLID,0,2)=='PL')
    {
    $PLID=substr($PLID,2);
    }
  $PLURLDATA='http://www.youtube.com/view_play_list?p=' . $PLID . '&playnext=1';
  //Get html result from Youtube Playlist Url
  $content=file_get_contents($PLURLDATA);
  //Get Youtube PlayList data-video-ids
  preg_match('/<div id="playlist-bar".*data-video-ids="(.*?)"/i',$content,$YTVars);
  //Explode Ids in array
  $YTIDS=explode(',', $YTVars[1]);
  foreach ($YTIDS as $line) 
    {
    //Get Youtube Data Array
    $Item=GetYTData($line);
    if ($Item['title']=="") continue;
    $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
    );
    } 
  return $Items;
  }
 
//Read a list of Url from Youtube SearchList
function GetYTUrlBySearchList($SearchUrl) 
  {
  _logInfo("GetYTUrlBySearchList-->SearchUrl = " . $SearchUrl);
  //Parse Url
  $queryData = array();
  $url = parse_url($SearchUrl); 
  _logInfo("GetYTUrlBySearchList-->url = " . print_r($url,true));  
  parse_str($url['query'], $queryData);
  _logInfo("GetYTUrlBySearchList-->queryData = " . print_r($queryData,true));
  for ($page_index = 1; $page_index <= 2; $page_index++) 
    {
    _logInfo("GetYTUrlBySearchList-->page_index = " . $page_index );
    $search_query = preg_replace("/ /","+",$queryData[search_query]);
    _logInfo("GetYTUrlBySearchList-->search_query = " . $search_query );
    $url_page='http://' . $url[host] .  $url[path] . '?search_query=' . $search_query . '&page=' . $page_index;
    _logInfo("GetYTUrlBySearchList-->url_page = " . $url_page);
    $content=file_get_contents($url_page);
    _logDebug("GetYTUrlBySearchList-->content = " . $content);
    preg_match_all('/\/watch\?v=(.*?)" class="ux-thumb-wrap result-item-thumb"/i', $content, $ids);
    _logInfo("GetYTUrlBySearchList-->ids = " . print_r($ids,true));
    foreach ($ids[1] as $line) 
      {
      //Get Youtube Data Array
      $Item=GetYTData($line);
      if ($Item['title']=="") continue;
      $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
      );
      } 
    }
  _logInfo("GetYTUrlBySearchList-->Items = " . print_r($Items,true));	
  return $Items;	
  }
 
//Read a list of Url from Youtube UserList Upload Video
function GetYTUrlByUserList($SearchUrl) 
  {
  _logInfo("GetYTUrlByUserList-->SearchUrl = " . $SearchUrl);
  //Parse Url
  $queryData = array();
  $url = parse_url($SearchUrl); 
  _logInfo("GetYTUrlByUserList-->url = " . print_r($url,true));  
  $url_page='http://' . $url[host] .  $url[path] . "#p/u";
  _logInfo("GetYTUrlByUserList-->url_page = " . $url_page);
  $content=file_get_contents($url_page);
  _logDebug("GetYTUrlByUserList-->content = " . $content);
  preg_match_all('/\/watch\?v=(.*?)" class="ux-thumb-wrap contains-addto"/i', $content, $ids);
  _logInfo("GetYTUrlByUserList-->ids = " . print_r($ids,true));
  foreach ($ids[1] as $line) 
    {
    //Get Youtube Data Array
    $Item=GetYTData($line);
    if ($Item['title']=="") continue;
    $Items[] = array(   
            'id'              => 'SubItem01',
            'parentID'        => 'umsp://plugins/hello_world',
            'dc:title'        =>  $Item['title'],
            'upnp:album_art'  =>  $Item['image'],
            'upnp:class'      => 'object.item.videoItem',
            'res'             => 'http://127.0.0.1/umsp/plugins/proxy_hello_world.php?v=' . $Item['id'],
            'protocolInfo'    => '*:*:*:*',
    );
    } 
  _logInfo("GetYTUrlByUserList-->Items = " . print_r($Items,true));	
  return $Items;	
  }
 
 
//------------------------End Define function Block-----------------------------------------------
?>

Save it: 

cd /tmp/umsp-plugins
tar -cvzf umsp-plugins.tgz hello_world.php proxy_hello_world.php yt_url.sh scraper_lib.php yt_list_misc.lst
mv -f umsp-plugins.tgz /tmp/conf

This is the files: http://rkpisanu.altervista.org/umsp_crash_course_v2/Day_06/umsp_crash_course_v2_Day_06.zip

Appendix

Tools

Firebug

What is Firebug?

Firebug integrates with Firefox to put a wealth of web development tools at your fingertips while you browse. You can edit, debug, and monitor CSS, HTML, and JavaScript live in any web page. It is a powerful tool for inspecting web page and its tag.

Download Firebug

LiveHTTPHeaders

What is LiveHTTPHeaders ?

The goal of this project is to adds information about the HTTP headers

LiveHTTPHeaders HomePage

License

GPL

http://www.gnu.org/licenses/gpl.html

wdtv/wdtv_umsp_crash_course_v2.txt · Last modified: 2011/10/23 20:16 by rkpisanu
Back to top
www.chimeric.de Creative Commons License Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0





Mail