|
BBC iPlayer plugin & Limelight CDNs June 04, 2014 03:17AM |
The background:
thread 1
tong2shot wrote:
> at this moment, streams from limelight are not downloadable.
> I tried several times with limelight, even tried rtmpdump, but no luck,
> maybe I missed some parameters, or applied wrong parameters.
> We'll try to investigate this issue.
> If get_iplayer is working with UK proxy applied,
> FRD should also be working with UK proxy applied...
I wrote:
> I have no problem downloading from the Limelight CDN;
> Attachement "Get_iPlayer_info.txt" contains the get_iplayer command used
> and the command prompt window output
> from downloading successfully the program (quality 480p) ...
> notice that:
> flashvhigh1 mode = Akamai CDN -> FAILURE, while
> flashvhigh2 mode = Limelight CDN -> SUCCESS.
thread 2
I wrote:
> Limelight (ll) CDNs used to also be very reliable and the only ones who worked for overseas
> users, since they're not blocked at server level - these are the ones FRD connects to
> when used with a UK proxy. Sadly, especially after the introduction of the
> new version of TV iplayer, there has been a change implemented on those
> (possibly deliberate, to fence off international users, or just a technical glitch, maybe?),
> that makes them exhibit a very erratic behaviour, also discussed by tong2shot here
(-> Limelight sometimes produces strange behaviour with or without UK proxy,
sometimes it gets stuck at "connect" command, but after waiting for awhile (5 minutes),
restart the download process then the stream is downloaded successfully).
> However, with some perseverance, I was always able to download from those...
> FWIW, this Limelight CDN issue is not unique to FRD: Even in the case where I configure Firefox
> to use a UK http proxy and try to play a BBC iplayer TV programme, I have to reload the
> page several times for the show to start playing in the browser (via a ll CDN).
All these past days I have been puzzled by the fact that get_iplayer (a perl script)
could establish immediate - on first attempt - connection with the limelight servers to download from them
(when used with a UK HTTP proxy from a non-UK physical location; it prefers
the Akamai CDNs from within the UK or when using a UK VPN), while FRD (even with the
latest bbc plugin 1.1.12) and Firefox browser exhibit the "problematic" behaviour
quoted above... There must be a "correct" parameter in get_iplayer, not found in the rest...
The tests:
I used this very short (4:26) sample video file:
Original Comedy Shorts - Matt Berry: Lone Wolf
which doesn't expire for another 11 months...
I set up a tested and working UK proxy in Firefox, then used rtmpdumphelper
(with a special "KSV" version of rtmpsrv binary)
to generate and save the proper rtmpdump command for the Limelight CDN.
I have attached it as "LimelightCDN_currentSWF.bat"
The "authentication string" (as=adobe-hmac-sha256&...) has a lifespan of 90 - 120 min.
(after it expires, you can get it renewed by visiting from a UK IP this URL:
mediaselector/5/.../p01zfq9t
then scroll down to the media bitrate="1500" section and copy the value of "authString="
for supplier="limelight"/> )
When you run the batch file within the lifespan of a valid authString,
you either
1. end up with a working download (if lucky), or
2. end up with this error:
(rtmpdump was run with the --verbose switch) in the case of such a failure,
see "Ll CDN error.txt" file.
You have to repeat running the batch file at a later instance,
hoping to result in a successful download - this coincides
with what is currently observed in FRD/Firefox.
Side note: The -W (= SWF URL) parameter found by rtmpsrv:
http://emp.bbci.co.uk/emp/smp-flash/smp.swf
as found by using Firefox's "Web Developer" tools;
but I don't think that rtmpdump can, on its own, handle such redirections...
The solution (?)
get_iplayer constructs an almost identical rtmpdump command to the one
I generated using rtmpsrv, with one vital exception:
It uses a fixed, hard-coded, value for the "-W" parameter.
In fact, the value it uses comes from the previous version of
BBC iPlayerTV and is:
so as to replace the current value of "-W" with the older one,
used by get_iplayer, and so ended up with file
"LimelightCDN_oldSWF.bat" (attached).
NO MATTER HOW MANY TIMES (AND WHEN) I RAN THIS BATCH FILE DURING
THE VALIDITY OF THE authString, I ALWAYS WAS SUCCESSFUL IN STARTING
A DOWNLOAD, NOT EVEN ONCE DID I GET HIT BY THE "RTMP_ReadPacket"
ERROR!
This matches what I experience with get_iplayer: instant & rock-steady connections
to the Limelight CDNs.
My gut feeling is that when the new interface of BBC iPlayerTV was introduced some
three months ago, someone over at the Beeb neglected to reconfigure the Limelight
CDNs, which remained fine-tuned to the old SWF URL and thus do not play well with
the new SWF that came with the revamped GUI - this must be the cause of the intermittent
connections observed ever since.
Of course, this could change in the future, but for the time being I bet the devs that
if a new bbc.frp is compiled using the "old" SWF URL, this would result in solid, reliable
downloads from Limelight CDNs again, doing away with the need for multiple retries...
{with the aid of an extractor and a Java decompiler, I have peeked into the current
bbc.frp 1.1.12 and have found that the value of "-W"
my theory myself, but since I am Java illiterate, I found no easy way to recompile the edited
file "BbcFileRunner.java" back to .class - I'll leave this to the real experts
}
thread 1
tong2shot wrote:
> at this moment, streams from limelight are not downloadable.
> I tried several times with limelight, even tried rtmpdump, but no luck,
> maybe I missed some parameters, or applied wrong parameters.
> We'll try to investigate this issue.
> If get_iplayer is working with UK proxy applied,
> FRD should also be working with UK proxy applied...
I wrote:
> I have no problem downloading from the Limelight CDN;
> Attachement "Get_iPlayer_info.txt" contains the get_iplayer command used
> and the command prompt window output
> from downloading successfully the program (quality 480p) ...
> notice that:
> flashvhigh1 mode = Akamai CDN -> FAILURE, while
> flashvhigh2 mode = Limelight CDN -> SUCCESS.
thread 2
I wrote:
> Limelight (ll) CDNs used to also be very reliable and the only ones who worked for overseas
> users, since they're not blocked at server level - these are the ones FRD connects to
> when used with a UK proxy. Sadly, especially after the introduction of the
> new version of TV iplayer, there has been a change implemented on those
> (possibly deliberate, to fence off international users, or just a technical glitch, maybe?),
> that makes them exhibit a very erratic behaviour, also discussed by tong2shot here
(-> Limelight sometimes produces strange behaviour with or without UK proxy,
sometimes it gets stuck at "connect" command, but after waiting for awhile (5 minutes),
restart the download process then the stream is downloaded successfully).
> However, with some perseverance, I was always able to download from those...
> FWIW, this Limelight CDN issue is not unique to FRD: Even in the case where I configure Firefox
> to use a UK http proxy and try to play a BBC iplayer TV programme, I have to reload the
> page several times for the show to start playing in the browser (via a ll CDN).
All these past days I have been puzzled by the fact that get_iplayer (a perl script)
could establish immediate - on first attempt - connection with the limelight servers to download from them
(when used with a UK HTTP proxy from a non-UK physical location; it prefers
the Akamai CDNs from within the UK or when using a UK VPN), while FRD (even with the
latest bbc plugin 1.1.12) and Firefox browser exhibit the "problematic" behaviour
quoted above... There must be a "correct" parameter in get_iplayer, not found in the rest...
The tests:
I used this very short (4:26) sample video file:
Original Comedy Shorts - Matt Berry: Lone Wolf
which doesn't expire for another 11 months...
I set up a tested and working UK proxy in Firefox, then used rtmpdumphelper
(with a special "KSV" version of rtmpsrv binary)
to generate and save the proper rtmpdump command for the Limelight CDN.
I have attached it as "LimelightCDN_currentSWF.bat"
The "authentication string" (as=adobe-hmac-sha256&...) has a lifespan of 90 - 120 min.
(after it expires, you can get it renewed by visiting from a UK IP this URL:
mediaselector/5/.../p01zfq9t
then scroll down to the media bitrate="1500" section and copy the value of "authString="
for supplier="limelight"/> )
When you run the batch file within the lifespan of a valid authString,
you either
1. end up with a working download (if lucky), or
2. end up with this error:
ERROR: RTMP_ReadPacket, failed to read RTMP packet body. len: 10827and no actual download - I have attached the last lines of an rtmpdump log
(rtmpdump was run with the --verbose switch) in the case of such a failure,
see "Ll CDN error.txt" file.
You have to repeat running the batch file at a later instance,
hoping to result in a successful download - this coincides
with what is currently observed in FRD/Firefox.
Side note: The -W (= SWF URL) parameter found by rtmpsrv:
http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/1.9.23/1.9.23_smp.swfis actually the result of a redirection of this generic SWF URL:
http://emp.bbci.co.uk/emp/smp-flash/smp.swf
as found by using Firefox's "Web Developer" tools;
but I don't think that rtmpdump can, on its own, handle such redirections...
The solution (?)
get_iplayer constructs an almost identical rtmpdump command to the one
I generated using rtmpsrv, with one vital exception:
It uses a fixed, hard-coded, value for the "-W" parameter.
In fact, the value it uses comes from the previous version of
BBC iPlayerTV and is:
http://www.bbc.co.uk/emp/releases/iplayer/revisions/617463_618125_4/617463_618125_4_emp.swfTo test my theory, I edited file "LimelightCDN_currentSWF.bat"
so as to replace the current value of "-W" with the older one,
used by get_iplayer, and so ended up with file
"LimelightCDN_oldSWF.bat" (attached).
NO MATTER HOW MANY TIMES (AND WHEN) I RAN THIS BATCH FILE DURING
THE VALIDITY OF THE authString, I ALWAYS WAS SUCCESSFUL IN STARTING
A DOWNLOAD, NOT EVEN ONCE DID I GET HIT BY THE "RTMP_ReadPacket"
ERROR!
This matches what I experience with get_iplayer: instant & rock-steady connections
to the Limelight CDNs.
My gut feeling is that when the new interface of BBC iPlayerTV was introduced some
three months ago, someone over at the Beeb neglected to reconfigure the Limelight
CDNs, which remained fine-tuned to the old SWF URL and thus do not play well with
the new SWF that came with the revamped GUI - this must be the cause of the intermittent
connections observed ever since.
Of course, this could change in the future, but for the time being I bet the devs that
if a new bbc.frp is compiled using the "old" SWF URL, this would result in solid, reliable
downloads from Limelight CDNs again, doing away with the need for multiple retries...
{with the aid of an extractor and a Java decompiler, I have peeked into the current
bbc.frp 1.1.12 and have found that the value of "-W"
http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/1.9.18/1.9.18_smp.swf?1.9.17is hard-coded inside file "BbcFileRunner.class"; I thought I could somehow edit it to test
my theory myself, but since I am Java illiterate, I found no easy way to recompile the edited
file "BbcFileRunner.java" back to .class - I'll leave this to the real experts
}
|
Re: BBC iPlayer plugin & Limelight CDNs June 04, 2014 04:46PM |
TVjunkie Wrote:
-------------------------------------------------------
> ...
> The solution (?)
>
> get_iplayer constructs an almost identical
> rtmpdump command to the one
> I generated using rtmpsrv, with one vital
> exception:
> It uses a fixed, hard-coded, value for the "-W"
> parameter.
> In fact, the value it uses comes from the previous
> version of
> BBC iPlayerTV and is:
> [www.bbc.co.uk]
> s/617463_618125_4/617463_618125_4_emp.swf
> To test my theory, I edited file
> "LimelightCDN_currentSWF.bat"
> so as to replace the current value of "-W" with
> the older one,
> used by get_iplayer, and so ended up with file
> "LimelightCDN_oldSWF.bat" (attached).
> NO MATTER HOW MANY TIMES (AND WHEN) I RAN THIS
> BATCH FILE DURING
> THE VALIDITY OF THE authString, I ALWAYS WAS
> SUCCESSFUL IN STARTING
> A DOWNLOAD, NOT EVEN ONCE DID I GET HIT BY THE
> "RTMP_ReadPacket"
> ERROR!
> ....
That's another superb analysis, thanks.
I just tested, and you are right.
> {with the aid of an extractor and a Java
> decompiler, I have peeked into the current
> bbc.frp 1.1.12 and have found that the value of
> "-W"
> [emp.bbci.co.uk]
> ions/1.9.18/1.9.18_smp.swf?1.9.17
> is hard-coded inside file "BbcFileRunner.class";
That's the spirit. Although that could have been done easier. FRD is an open source project, you can read the source code here.
And for BBC plugin, you can read the main class here.
Username & password : wordrider
> I thought I could somehow edit it to test
> my theory myself, but since I am Java illiterate,
> I found no easy way to recompile the edited
> file "BbcFileRunner.java" back to .class - I'll
> leave this to the real experts }
That's why we have this topic. Do you want to give it a try?
Come to the dark side, we have cookies
-------------------------------------------------------
> ...
> The solution (?)
>
> get_iplayer constructs an almost identical
> rtmpdump command to the one
> I generated using rtmpsrv, with one vital
> exception:
> It uses a fixed, hard-coded, value for the "-W"
> parameter.
> In fact, the value it uses comes from the previous
> version of
> BBC iPlayerTV and is:
> [www.bbc.co.uk]
> s/617463_618125_4/617463_618125_4_emp.swf
> To test my theory, I edited file
> "LimelightCDN_currentSWF.bat"
> so as to replace the current value of "-W" with
> the older one,
> used by get_iplayer, and so ended up with file
> "LimelightCDN_oldSWF.bat" (attached).
> NO MATTER HOW MANY TIMES (AND WHEN) I RAN THIS
> BATCH FILE DURING
> THE VALIDITY OF THE authString, I ALWAYS WAS
> SUCCESSFUL IN STARTING
> A DOWNLOAD, NOT EVEN ONCE DID I GET HIT BY THE
> "RTMP_ReadPacket"
> ERROR!
> ....
That's another superb analysis, thanks.
I just tested, and you are right.
> {with the aid of an extractor and a Java
> decompiler, I have peeked into the current
> bbc.frp 1.1.12 and have found that the value of
> "-W"
> [emp.bbci.co.uk]
> ions/1.9.18/1.9.18_smp.swf?1.9.17
> is hard-coded inside file "BbcFileRunner.class";
That's the spirit. Although that could have been done easier. FRD is an open source project, you can read the source code here.
And for BBC plugin, you can read the main class here.
Username & password : wordrider
> I thought I could somehow edit it to test
> my theory myself, but since I am Java illiterate,
> I found no easy way to recompile the edited
> file "BbcFileRunner.java" back to .class - I'll
> leave this to the real experts
}
That's why we have this topic. Do you want to give it a try?
Come to the dark side, we have cookies
|
Re: BBC iPlayer plugin & Limelight CDNs June 06, 2014 04:49PM |
vaffangool Wrote:
-------------------------------------------------------
> You can quickly test any swfVfy URL in get_iPlayer by preceding a known working download with the command lines:
> get_iplayer --rtmp-tv-opts="--swfVfy=http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/1.9.23/1.9.23_smp.swf?1.9.23"
> get_iplayer --rtmp-tv-opts="--swfVfy=http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/617463_618125_4/617463_618125_4_emp.swf"
> get_iplayer --rtmp-tv-opts="--swfVfy=http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/1.9.18/1.9.18_smp.swf?1.9.17"
> A recent test shows all three work equally well -- for now.
Thanks for registering and posting
I am well aware of the --rtmp-tv-opts option; yes, all three swfVfy URLs work for now, but NOT EQUALLY WELL
if you are coming from outside the UK and use the options -p="http://proxyhost:proxyport" --partial-proxy
The latest value
Level3 CDNs (accessible both from within & outside the UK), but ONLY IN A RANDOM WAY for Limelight CDNs
(as I have explained in detail in my previous post), which are the ones get_iplayer (and FRD, for that matter)
connects to via UK HTTP Proxy.
Only the older (middle one) swfVfy URL
of GiP to the newer swfVfy URL found in git. If you are actually within the UK (you don't say where you are), then GiP prefers the
Akamai CDNs both for SD & HD content (usually tv modes ending in 1, e.g. flashhigh1, flashhd1), so it would appear
in your test that all 3 different swfVfy URLs work equally well - yes they do, for Akamai that is...
@tong2shot:
Will you please be releasing a new bbc plugin with the old SWF URL, to rectify the Limelight CDN issue?
Many thanks in advance
-------------------------------------------------------
> You can quickly test any swfVfy URL in get_iPlayer by preceding a known working download with the command lines:
> get_iplayer --rtmp-tv-opts="--swfVfy=http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/1.9.23/1.9.23_smp.swf?1.9.23"
> get_iplayer --rtmp-tv-opts="--swfVfy=http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/617463_618125_4/617463_618125_4_emp.swf"
> get_iplayer --rtmp-tv-opts="--swfVfy=http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/1.9.18/1.9.18_smp.swf?1.9.17"
> A recent test shows all three work equally well -- for now.
Thanks for registering and posting
I am well aware of the --rtmp-tv-opts option; yes, all three swfVfy URLs work for now, but NOT EQUALLY WELL
if you are coming from outside the UK and use the options -p="http://proxyhost:proxyport" --partial-proxy
The latest value
http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/1.9.23/1.9.23_smp.swf(which is actually the one BBC iPlayer uses in a browser) always works for Akamai CDNs (if in the UK) &
Level3 CDNs (accessible both from within & outside the UK), but ONLY IN A RANDOM WAY for Limelight CDNs
(as I have explained in detail in my previous post), which are the ones get_iplayer (and FRD, for that matter)
connects to via UK HTTP Proxy.
Only the older (middle one) swfVfy URL
http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/617463_618125_4/617463_618125_4_emp.swfALWAYS WORKS RELIABLY for Limelight CDNs (as well as Level3 CDNs), that's why I haven't updated my local copy
of GiP to the newer swfVfy URL found in git. If you are actually within the UK (you don't say where you are), then GiP prefers the
Akamai CDNs both for SD & HD content (usually tv modes ending in 1, e.g. flashhigh1, flashhd1), so it would appear
in your test that all 3 different swfVfy URLs work equally well - yes they do, for Akamai that is...
@tong2shot:
Will you please be releasing a new bbc plugin with the old SWF URL, to rectify the Limelight CDN issue?
Many thanks in advance
|
Re: BBC iPlayer plugin & Limelight CDNs June 06, 2014 11:36PM |
@vaffangool
Test video file: D-Day 70 - The Heroes Return: 3. Highlights
I have used the following GiP command to try the new SWF URL with a Limelight CDN:
Command Prompt Window output attached as "GiP_p01zf67p_newSWF_failure.txt"
GiP fails to download from ll (flashvhigh2 mode), with the characteristic
@tong2shot:
I have just updated to bbc.frp 1.1.13 - FANTASTIC NEWS!
Not only did the download commence immediately (video file at the beginning of this post),
but I didn't have to use a UK proxy either, since it appears that THE TOR PLUGIN
HAS STARTED TO BE FUNCTIONAL AGAIN for BBC iPlayer !!!!
(see attachment "bbc1113_Tor_success.jpg" )
And this came just in time, because it had become quite difficult to find good working UK proxies for the past 2 days...
So, for those overseas users, update to bbc.co.uk plugin 1.1.13 and inside its options
set "Limelight" as preferred CDN and tick "Enable Tor" (I don't know for how long the tor plugin
will do the job, it was certainly working at the time of posting...)
tong2shot, I owe you bigtime!
Test video file: D-Day 70 - The Heroes Return: 3. Highlights
I have used the following GiP command to try the new SWF URL with a Limelight CDN:
get_iplayer --rtmp-tv-opts="--swfVfy=http://emp.bbci.co.uk/emp/releases/smp-flash/revisions/1.9.23/1.9.23_smp.swf" -p "http://88.150.168.176:8089" --partial-proxy -i --pid=p01zf67p --modes=flashvhigh --force -w --file-prefix="D-Day70_TheHeroesReturn_3.Highlights_201406062000" --raw(NB: the UK proxy used NO LONGER WORKS, hence I've not hidden its details)
Command Prompt Window output attached as "GiP_p01zf67p_newSWF_failure.txt"
GiP fails to download from ll (flashvhigh2 mode), with the characteristic
ERROR: RTMP_ReadPacket, failed to read RTMP packet headerIf the old SWF URL is used, download succeeds...
@tong2shot:
I have just updated to bbc.frp 1.1.13 - FANTASTIC NEWS!
Not only did the download commence immediately (video file at the beginning of this post),
but I didn't have to use a UK proxy either, since it appears that THE TOR PLUGIN
HAS STARTED TO BE FUNCTIONAL AGAIN for BBC iPlayer !!!!
(see attachment "bbc1113_Tor_success.jpg" )
And this came just in time, because it had become quite difficult to find good working UK proxies for the past 2 days...
So, for those overseas users, update to bbc.co.uk plugin 1.1.13 and inside its options
set "Limelight" as preferred CDN and tick "Enable Tor" (I don't know for how long the tor plugin
will do the job, it was certainly working at the time of posting...)
tong2shot, I owe you bigtime!
Sorry, only registered users may post in this forum.