bbc.co.uk downloads truncated....
Posted by ilanad
|
Re: bbc.co.uk downloads truncated.... February 01, 2015 04:07AM |
I wouldn't push the alert button just yet, but there certainly is something going wrong with the HD streams 
But first, the "Tor plugin" issue is a separate thing from the "truncated downloads" problem;
once the Tor plugin effectuates connection with the geo-filtered BBC servers (these are "playlist" servers that FRD
must connect to in order to gather data for the available streams), then it gets out from the play and the actual RTMP
download is realised via a direct connection between the Limelight (for <=480p) or the Level 3 (720p) CDN and your ISP.
Using a UK (HTTP) proxy or the Tor plugin may grant you access to the Akamai CDN servers (for all video resolutions),
but those servers are GEOLOCKED at FMS level, meaning they won't start the requested stream once they
verify your non-UK originating IP address. FRD will produce a "0 bytes" download in those cases...
I was not having any out of the ordinary problems with previous version (1.0.1) of the Tor plugin,
neither should anyone who heeds to my guidelines described here. Updated to 1.0.2 and so far so good...
Now, the "truncated downloads" problem:
Limelight CDN is a set of de-centralised media servers; your copy of FRD connects to the one server which is closer to
your physical location. You can find this info out by loading - DIRECT CONNECTION, NOT USING A PROXY - the
following URL:
Ll CDN identity
You can geo-search the yielding IP address to pinpoint the limelight PoP (point of presence).
I myself, physically located in south-eastern Europe, am not experiencing the interrupted downloads issue;
it may well be that those affected by it are downloading from a CDN node hammered by
extra heavy requests at peak times.
When I download from the Limelight CDN,
I only download one file at a time and - whenever possible - do it after peak times, i.e. past 00:00 GMT.
It is indeed true that in the past downloading from the Ll CDN was done with very high speeds (>=1.2 MBps),
but in the latest months that the (illicit) overseas downloads grew more plentiful, I get no more than half that speed...
@ilanad:
Both of your sample programmes download fine at 468p resolution (with Tor plugin) from the Ll CDN.
It is indeed a shame that the rtmp plugin DOES NOT HAVE RESUME SUPPORT. In the past I had time-outs
at 80-90% of a BBC file; when FRD would "resume", it would delete the partial file and restart from scratch...
If rtmp resume support is ever to be considered, please take a look at this patch:
rtmpdump "BBC iPlayer files" special resume patch
which is special for iPlayer downloads...
Level 3 CDN
@ilanad:
Both of your sample programmes IN FACT LACK A HD (720p) version; the "Watch in HD" option
in the iPlayer pages is MISLEADING.
@tong2shot:
If you load via a UK proxy the Mediaselector/5 API URLs:
vPID=p00pvbj7
vPID=p00pvbj7
you'll see for yourself that no <media bitrate="2800" version has been uploaded to either Ll or Ak CDNs - if FRD
is configured to 720p/level3, then a "0 bytes" download will be produced!
@rancor
Your sample programme downloads fine in SD 468p (with Tor enabled) from Ll CDN.
As far as the 720p version is concerned, I can replicate your issue with an incomplete
download sized 47.17 MB coming from the Level3 CDN.
It may well be that the BBC uploaded a truncated file to that CDN.
You are not right saying
Also, if you download the file with the dedicated tool get_iplayer,
You can use one of the free available UK VPNs
(try Free UK VPN) and configure BBC plugin
to 720p/443 (or 80)/Akamai - NB: These free VPNs have default RTMP port 1935 blocked!
... and do not expect great speeds!
Another Level3 issue was recently reported to the get_iplayer support forum
(they are adamant about not supporting overseas users - don't post there if not in the UK!):
https://squarepenguin.co.uk/forums/topic/trouble-downloading-octonauts-s01e02-in-hd/
It turns out (this is my inspection result) that although the Mediaselector/5 API
produces the data for a Level 3 stream, the same stream is non-existent
on the server itself, which returns
"ERROR: Closing connection: NetStream.Failed".
To recap with HD streams:
1) For many recent TV shows, no HD versions are available at all
(no matter what the iPlayer pages indicate).
e.g. last week's Graham Norton has a L3 HD stream:
The Graham Norton Show - S16E15,
which FRD can fetch just fine, while last Friday's episode does not:
The Graham Norton Show - S16E16,
FRD resulting in a "0 bytes" download.
2) Some Level3 HD uploads have been reported to be "problematic".
Use the Akamai CDN via a UK VPN if you must obtain the 720p files.
And I believe the Beeb personnel are not daft - they must've already noticed all this huge bandwidth
being spent to overseas clients - perhaps this is the beginning of more changes to come.
They have already moved their live TV & live Radio streams over to HTTP streaming (AppleHLS & AdobeHDS),
and they announced that at least audio-on-demand will be migrated in 2015 to HTTP streaming, too!
I bet they have similar plans for video-on-demand; if this is realised, no Tor plugin or free (open) HTTP
proxy could handle all these video streams... A high bandwidth paid-for UK VPN will then be mandatory for us
overseas BBC lovers...
Edited 1 time(s). Last edit at 02/02/2015 12:03AM by TVjunkie.
But first, the "Tor plugin" issue is a separate thing from the "truncated downloads" problem;
once the Tor plugin effectuates connection with the geo-filtered BBC servers (these are "playlist" servers that FRD
must connect to in order to gather data for the available streams), then it gets out from the play and the actual RTMP
download is realised via a direct connection between the Limelight (for <=480p) or the Level 3 (720p) CDN and your ISP.
Using a UK (HTTP) proxy or the Tor plugin may grant you access to the Akamai CDN servers (for all video resolutions),
but those servers are GEOLOCKED at FMS level, meaning they won't start the requested stream once they
verify your non-UK originating IP address. FRD will produce a "0 bytes" download in those cases...
I was not having any out of the ordinary problems with previous version (1.0.1) of the Tor plugin,
neither should anyone who heeds to my guidelines described here. Updated to 1.0.2 and so far so good...
Now, the "truncated downloads" problem:
Limelight CDN is a set of de-centralised media servers; your copy of FRD connects to the one server which is closer to
your physical location. You can find this info out by loading - DIRECT CONNECTION, NOT USING A PROXY - the
following URL:
Ll CDN identity
You can geo-search the yielding IP address to pinpoint the limelight PoP (point of presence).
I myself, physically located in south-eastern Europe, am not experiencing the interrupted downloads issue;
it may well be that those affected by it are downloading from a CDN node hammered by
extra heavy requests at peak times.
When I download from the Limelight CDN,
I only download one file at a time and - whenever possible - do it after peak times, i.e. past 00:00 GMT.
It is indeed true that in the past downloading from the Ll CDN was done with very high speeds (>=1.2 MBps),
but in the latest months that the (illicit) overseas downloads grew more plentiful, I get no more than half that speed...
@ilanad:
Both of your sample programmes download fine at 468p resolution (with Tor plugin) from the Ll CDN.
It is indeed a shame that the rtmp plugin DOES NOT HAVE RESUME SUPPORT. In the past I had time-outs
at 80-90% of a BBC file; when FRD would "resume", it would delete the partial file and restart from scratch...
If rtmp resume support is ever to be considered, please take a look at this patch:
rtmpdump "BBC iPlayer files" special resume patch
which is special for iPlayer downloads...
Level 3 CDN
@ilanad:
Both of your sample programmes IN FACT LACK A HD (720p) version; the "Watch in HD" option
in the iPlayer pages is MISLEADING.
@tong2shot:
If you load via a UK proxy the Mediaselector/5 API URLs:
vPID=p00pvbj7
vPID=p00pvbj7
you'll see for yourself that no <media bitrate="2800" version has been uploaded to either Ll or Ak CDNs - if FRD
is configured to 720p/level3, then a "0 bytes" download will be produced!
@rancor
Your sample programme downloads fine in SD 468p (with Tor enabled) from Ll CDN.
As far as the 720p version is concerned, I can replicate your issue with an incomplete
download sized 47.17 MB coming from the Level3 CDN.
It may well be that the BBC uploaded a truncated file to that CDN.
You are not right saying
If you play the Level3 HD version in Firefox (via UK proxy), then playback stops at 02:53.Quote
All files playing ok in browsers (with UK proxy).
Also, if you download the file with the dedicated tool get_iplayer,
get_iplayer -p "http://[proxyhost]:[proxyport]" --partial-proxy --pid=b0517vrl --raw --modes=flashhd --attempts=1you get:
INFO: skipping flashhd1 mode INFO: Trying flashhd2 mode to record tv: Great British Railway Journeys: Series 6 - 20. Oakham to Cambridge INFO: File name prefix = Great_British_Railway_Journeys_Series_6_-_20._Oakham_to _Cambridge_b0517vrl_default RTMPDump v2.4 GIT-2015-01-15 (Compiled by KSV) (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL Connecting ... INFO: Connected... Starting download at: 0.000 kB INFO: Metadata: INFO: duration 1745.28 INFO: moovPosition 36.00 INFO: width 1280.00 INFO: height 720.00 INFO: videocodecid avc1 INFO: audiocodecid mp4a INFO: avcprofile 100.00 INFO: avclevel 41.00 INFO: aacaot 2.00 INFO: videoframerate 25.00 INFO: audiosamplerate 48000.00 INFO: audiochannels 2.00 INFO: trackinfo: INFO: length 43632000.00 INFO: timescale 25000.00 INFO: language und INFO: sampledescription: INFO: sampletype avc1 INFO: length 83773440.00 INFO: timescale 48000.00 INFO: language und INFO: sampledescription: INFO: sampletype mp4a 48301.104 kB / 173.92 sec (9.9%) Download may be incomplete (downloaded about 9.90%), try resuming INFO: Command exit code 2 (raw code = 512) INFO: Checking existence of audiodescribed version INFO: No specified modes (flashhd) available for this programme with version 'au diodescribed' (try using --modes=flashhigh,rtsphigh) ERROR: Failed to record 'Great British Railway Journeys: Series 6 - 20. Oakham t o Cambridge (b0517vrl)'If you are desperate to acquire the 720p version, then the Akamai CDN is all you are left with...
You can use one of the free available UK VPNs
(try Free UK VPN) and configure BBC plugin
to 720p/443 (or 80)/Akamai - NB: These free VPNs have default RTMP port 1935 blocked!
... and do not expect great speeds!
Another Level3 issue was recently reported to the get_iplayer support forum
(they are adamant about not supporting overseas users - don't post there if not in the UK!):
https://squarepenguin.co.uk/forums/topic/trouble-downloading-octonauts-s01e02-in-hd/
It turns out (this is my inspection result) that although the Mediaselector/5 API
produces the data for a Level 3 stream, the same stream is non-existent
on the server itself, which returns
"ERROR: Closing connection: NetStream.Failed".
To recap with HD streams:
1) For many recent TV shows, no HD versions are available at all
(no matter what the iPlayer pages indicate).
e.g. last week's Graham Norton has a L3 HD stream:
The Graham Norton Show - S16E15,
which FRD can fetch just fine, while last Friday's episode does not:
The Graham Norton Show - S16E16,
FRD resulting in a "0 bytes" download.
2) Some Level3 HD uploads have been reported to be "problematic".
Use the Akamai CDN via a UK VPN if you must obtain the 720p files.
And I believe the Beeb personnel are not daft - they must've already noticed all this huge bandwidth
being spent to overseas clients - perhaps this is the beginning of more changes to come.
They have already moved their live TV & live Radio streams over to HTTP streaming (AppleHLS & AdobeHDS),
and they announced that at least audio-on-demand will be migrated in 2015 to HTTP streaming, too!
I bet they have similar plans for video-on-demand; if this is realised, no Tor plugin or free (open) HTTP
proxy could handle all these video streams...
A high bandwidth paid-for UK VPN will then be mandatory for us
overseas BBC lovers...
Edited 1 time(s). Last edit at 02/02/2015 12:03AM by TVjunkie.
Sorry, only registered users may post in this forum.