bbc.co.uk downloads truncated....

Re: bbc.co.uk downloads truncated....

tong2shot — Wed, 11 Feb 2015 22:20:46 +0000

@ilanad : that's what I did, setup a debian+LXDE VM to tunnel the VPN connection through ssh with a local dynamic port forwarding. Because I don't trust my VPN provider to tunnel my connection system-widely. Then use the sock proxy produced by ssh in FRD, or browser, or other apps.

Re: bbc.co.uk downloads truncated....

ilanad — Fri, 06 Feb 2015 03:48:04 +0000

well I'm having no luck at all. I may set up a VM with just FRD in it so I can give it a dedicated proxy. I was hoping that maybe the plugin could be updated to let it switch in the proxy/vpn info for it when needed. I have a addon for my Kodi media center that does that, switches in a local proxy when it connected to the BBC live streams.... will report back.

Re: bbc.co.uk downloads truncated....

TVjunkie — Tue, 03 Feb 2015 01:02:48 +0000

ilanad wrote:
-------------------------------------------------------
> @tvjunkie
>
> oh frack. well can you give some pointers to the
> proper setup of the VPN?

Hi :)-D
I am not really an expert when it comes to VPN connections...:S
I do know that the PPTP VPN client is native to Windows Oses (XP and up).
I follow to the letter the guides posted on the respective VPN sites and it just works!
I have an old desktop PC with XP, and old-ish laptop with Vista and a newer
laptop with 7 - all three connect with no problem to VPN servers, once the
VPN connection has been setup as per the guides...
Some various guides picked from VPN sites:
vpnuk
vpnsurfing
bestukvpn
(Always follow the "manual" installation guides - do not install
pre-configured clients. All the PPTP VPNs differ between them in
the hostname of the VPN server, the username and password;
all the rest is almost identical.
Some Free VPNs have a PW that changes every 4-6 hours, to
prevent user abuse; first get the new PW, then try to connect.
Some others have a fixed PW, but once you connect you must
open the default browser of your system to acquire an
authentication to connect to the rest of the web...)
Please also take a look at the "Troubleshooting" section of:
hotfreevpn
(I have no affiliation with any of the above sites)

If you have an Internet Security (Antivirus+Firewall) software installed,
maybe you need to go to its documentation to see how it needs to be
configured to allow VPN connections.
It might also be an issue with your Modem/Router - or maybe your ISP
is blocking VPN connections - it's up to you to remedy these problems...

As a last resort, you might try an OpenVPN solution:
freevpnsoftware
However, this requires some user expertise:
1) An OpenVPN client has to be downloaded and installed.
2) OpenVPN UK Server configuration files have to be
downloaded (from site) and correctly manually installed.

When setting up new VPN connections or installing VPN clients,
it may be necessary to "Run as admin" / Be logged in as admin /
temporarily disable UAC (for Vista/7/8/8.1).
I don't have anything more to add really, this is beyond the scope
of the FRD forum :-P
Good luck (and possibly report back with the results;-) )
And remember: Most VPN services have only ports 80 (http) & 443 (https) open!

> can the proxy be enabled ONLY for the BBC plugin
> (didn't see it in the setup) since that's the only off
> shore service I use.

A VPN service is much different to a proxy service;
Once you are connected to the VP Network, ALL YOUR TRAFFIC
IS REROUTED THROUGH THE VPN SERVER - as the name suggests,
it is as if your computer is moved to a different physical location.
If the VPN speed is sufficient, you can usually do your other stuff while
the BBC files are downloading (with a possible exception being
e-mail clients, e.g. Thunderbird etc.)

> and Hulu is far worse, can't get anything from it
> any more and I'm from the US.

Yes, since hulu have implemented SWF type 2 verification some two years ago,
it is impossible to download anything with tools using the rtmpdump code.
Being in the US, you can at least stream the shows in your browser unhindered :P
What I referred to is the fact that Hulu have also implemented very strict and
sensitive US Proxy/US VPN detection mechanisms that make it very difficult
for "off shore" clients to get access to hulu streams.
(You in the US call it "off shore", people in the UK call it "overseas" !).

Regards

Re: bbc.co.uk downloads truncated....

ilanad — Mon, 02 Feb 2015 04:35:30 +0000

@tvjunkie

oh frack. well can you give some pointers to the proper setup of the VPN? I (thought) I followed the instruction and get either get a "Connenction Refused" or "Malformed SOCKS" error. and can the proxy be enabled ONLY for the BBC plugin (didn't see it in the setup) since that's the only off shore service I use.

and Hulu is far worse, can't get anything from it any more and I'm from the US.

Re: bbc.co.uk downloads truncated....

TVjunkie — Mon, 02 Feb 2015 03:19:32 +0000

I wrote:
-------------------------------------------------------
> @ilanad:
> Both of your sample programmes IN FACT LACK A HD
> (720p) version; the "Watch in HD" option
> in the iPlayer pages is MISLEADING.

My most sincere apologies 8-). I made a crucial mistake ::o
Both of your sample programmes DO HAVE A HD version,
but ONLY ON THE AKAMAI CDN! :X
(Download possible only via UK VPN - see tips sent to @rancor)

> @tong2shot:
> If you load via a UK proxy the Mediaselector/5 API URLs:
> vPID=p00pvbj7
> vPID=p00pvbj7
> you'll see for yourself that no media bitrate="2800" version has been uploaded to either Ll or AK CDNs.

I made a mistake; those URLs are for the "BBC One ident" (short logo clip of the TV channel at the beginning...).
The correct Mediaselector/5 API URLs are the following:
vPID=b03phw8v
vPID=b03phkl1
Both of those do produce entries with media bitrate="2800", and for both CDNs ( supplier="level3" & supplier="akamai").
However, the stream parameters for the L3 CDN are NON-FUNCTIONAL, they point to a PLACEBO stream that is never
to be found on the L3 server!
This is indeed sad, as that L3 HD stream was the only one accessible by FRD via Tor plugin/HTTP proxy! :X:X:X
If you try to dump this L3 HD stream with RTMPdump:

rtmpdump -r "rtmp://iplayertokfs.fplive.net:1935/iplayertok" -a "iplayertok?token=c3RhcnRfdGltZT0yMDE1MDIwMTIzMjUxNyZlbmRfdGltZT0yMDE1MDIwMjAzMjUxNyZkaWdlc3Q9NjhmMzJiNTk4MDYzYmU2NjU2MjM1NzdjM2ExOWIyNDc=" -f "WIN 16,0,0,296" -W "http://emp.bbci.co.uk/emp/SMPf/1.10.16/StandardMediaPlayerChromelessFlash.swf" -p "http://www.bbc.co.uk/iplayer/episode/b03pmbnm/" -C O:1 -C O:0 -y "mp4:3200kbps/modav/bUnknown-2938fa83-e42e-4c58-85e5-434035fa5f5b_b03phw8v_1421665716298" -o "b03phw8v_1421665716298.flv"

you get:

ERROR: Closing connection: NetStream.Play.StreamNotFound

With FRD, if you succeed to connect to the L3 server (via Tor/proxy), FRD sits idle at first at the "DOWNLOADING" stage,
then ERRORs out!

> e.g. last week's Graham Norton has a L3 HD stream:
> The Graham Norton Show - S16E15,
> which FRD can fetch just fine, while last Friday's episode does not:
> The Graham Norton Show - S16E16,

While the latter was indeed true at the time of writing, now the Mediaselector/5 API DOES produce
HD stream data for latest episode (S16E16):
vPID=b051c7hc,
but, again, the L3 stream is "virtual"; its data does not initiate a real stream from the server!

So, I am now pushing the alert button because:

IT APPEARS THAT FOR NEW VIDEO CONTENT FROM THE BBC IPLAYER,
THE HD VERSION IS ONLY AVAILABLE FROM THE AKAMAI CDN (geo-blocked at
RTMP server level)!

This may be intentional (the most probable scenario) or the result of something gone awry
with the Level 3 CDN (it is a remote possibility, since the iPlayer API continues to yield
L3 stream parameters...).
In any case, this is indeed a sad day... UK VPNs a solution for now, but expect the free ones to vanish
into thin air as they get abused with time; and even the paid-for ones are not full proof:
As the "hulu" case in the US has shown, it is quite feasible for the BBC to detect and block selected VPN IPs...
A sad day indeed - at least we have the Limelight CDNs (Noone knows for how much longer...:?)

Re: bbc.co.uk downloads truncated....

rancor — Sun, 01 Feb 2015 10:40:23 +0000

@TVjunkie thanks! Download in HD working via VPN (Akamai, port 80). Speed max. 600 kB/s, average 300 kB/s (one file).

Re: bbc.co.uk downloads truncated....

TVjunkie — Sun, 01 Feb 2015 04:07:17 +0000

I wouldn't push the alert button just yet, but there certainly is something going wrong with the HD streams :-(

But first, the "Tor plugin" issue is a separate thing from the "truncated downloads" problem;
once the Tor plugin effectuates connection with the geo-filtered BBC servers (these are "playlist" servers that FRD
must connect to in order to gather data for the available streams), then it gets out from the play and the actual RTMP
download is realised via a direct connection between the Limelight (for <=480p) or the Level 3 (720p) CDN and your ISP.
Using a UK (HTTP) proxy or the Tor plugin may grant you access to the Akamai CDN servers (for all video resolutions),
but those servers are GEOLOCKED at FMS level, meaning they won't start the requested stream once they
verify your non-UK originating IP address. FRD will produce a "0 bytes" download in those cases...

I was not having any out of the ordinary problems with previous version (1.0.1) of the Tor plugin,
neither should anyone who heeds to my guidelines described here. Updated to 1.0.2 and so far so good...

Now, the "truncated downloads" problem:
Limelight CDN is a set of de-centralised media servers; your copy of FRD connects to the one server which is closer to
your physical location. You can find this info out by loading - DIRECT CONNECTION, NOT USING A PROXY - the
following URL:
Ll CDN identity
You can geo-search the yielding IP address to pinpoint the limelight PoP (point of presence).

I myself, physically located in south-eastern Europe, am not experiencing the interrupted downloads issue;
it may well be that those affected by it are downloading from a CDN node hammered by
extra heavy requests at peak times.
When I download from the Limelight CDN,
I only download one file at a time and - whenever possible - do it after peak times, i.e. past 00:00 GMT.
It is indeed true that in the past downloading from the Ll CDN was done with very high speeds (>=1.2 MBps),
but in the latest months that the (illicit) overseas downloads grew more plentiful, I get no more than half that speed...
@ilanad:
Both of your sample programmes download fine at 468p resolution (with Tor plugin) from the Ll CDN.

It is indeed a shame that the rtmp plugin DOES NOT HAVE RESUME SUPPORT. In the past I had time-outs
at 80-90% of a BBC file; when FRD would "resume", it would delete the partial file and restart from scratch...
If rtmp resume support is ever to be considered, please take a look at this patch:
rtmpdump "BBC iPlayer files" special resume patch
which is special for iPlayer downloads...

Level 3 CDN

@ilanad:
Both of your sample programmes IN FACT LACK A HD (720p) version; the "Watch in HD" option
in the iPlayer pages is MISLEADING.
@tong2shot:
If you load via a UK proxy the Mediaselector/5 API URLs:
vPID=p00pvbj7
vPID=p00pvbj7
you'll see for yourself that no is configured to 720p/level3, then a "0 bytes" download will be produced!
@rancor
Your sample programme downloads fine in SD 468p (with Tor enabled) from Ll CDN.
As far as the 720p version is concerned, I can replicate your issue with an incomplete
download sized 47.17 MB coming from the Level3 CDN.
It may well be that the BBC uploaded a truncated file to that CDN.
You are not right saying

Quote
All files playing ok in browsers (with UK proxy).

If you play the Level3 HD version in Firefox (via UK proxy), then playback stops at 02:53.
Also, if you download the file with the dedicated tool get_iplayer,

get_iplayer -p "http://[proxyhost]:[proxyport]" --partial-proxy --pid=b0517vrl --raw --modes=flashhd --attempts=1

you get:

INFO: skipping flashhd1 mode
INFO: Trying flashhd2 mode to record tv: Great British Railway Journeys: Series
6 - 20. Oakham to Cambridge
INFO: File name prefix = Great_British_Railway_Journeys_Series_6_-_20._Oakham_to
_Cambridge_b0517vrl_default
RTMPDump v2.4 GIT-2015-01-15 (Compiled by KSV)
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
INFO: Connected...
Starting download at: 0.000 kB
INFO: Metadata:
INFO:   duration                1745.28
INFO:   moovPosition            36.00
INFO:   width                   1280.00
INFO:   height                  720.00
INFO:   videocodecid            avc1
INFO:   audiocodecid            mp4a
INFO:   avcprofile              100.00
INFO:   avclevel                41.00
INFO:   aacaot                  2.00
INFO:   videoframerate          25.00
INFO:   audiosamplerate         48000.00
INFO:   audiochannels           2.00
INFO: trackinfo:
INFO:   length                  43632000.00
INFO:   timescale               25000.00
INFO:   language                und
INFO: sampledescription:
INFO:   sampletype              avc1
INFO:   length                  83773440.00
INFO:   timescale               48000.00
INFO:   language                und
INFO: sampledescription:
INFO:   sampletype              mp4a
48301.104 kB / 173.92 sec (9.9%)
Download may be incomplete (downloaded about 9.90%), try resuming
INFO: Command exit code 2 (raw code = 512)
INFO: Checking existence of audiodescribed version
INFO: No specified modes (flashhd) available for this programme with version 'au
diodescribed' (try using --modes=flashhigh,rtsphigh)
ERROR: Failed to record 'Great British Railway Journeys: Series 6 - 20. Oakham t
o Cambridge (b0517vrl)'

If you are desperate to acquire the 720p version, then the Akamai CDN is all you are left with...
You can use one of the free available UK VPNs
(try Free UK VPN) and configure BBC plugin
to 720p/443 (or 80)/Akamai - NB: These free VPNs have default RTMP port 1935 blocked!
... and do not expect great speeds!

Another Level3 issue was recently reported to the get_iplayer support forum
(they are adamant about not supporting overseas users - don't post there if not in the UK!):
https://squarepenguin.co.uk/forums/topic/trouble-downloading-octonauts-s01e02-in-hd/
It turns out (this is my inspection result) that although the Mediaselector/5 API
produces the data for a Level 3 stream, the same stream is non-existent
on the server itself, which returns
"ERROR: Closing connection: NetStream.Failed".

To recap with HD streams:
1) For many recent TV shows, no HD versions are available at all
(no matter what the iPlayer pages indicate).
e.g. last week's Graham Norton has a L3 HD stream:
The Graham Norton Show - S16E15,
which FRD can fetch just fine, while last Friday's episode does not:
The Graham Norton Show - S16E16,
FRD resulting in a "0 bytes" download.
2) Some Level3 HD uploads have been reported to be "problematic".
Use the Akamai CDN via a UK VPN if you must obtain the 720p files.

And I believe the Beeb personnel are not daft - they must've already noticed all this huge bandwidth
being spent to overseas clients - perhaps this is the beginning of more changes to come.
They have already moved their live TV & live Radio streams over to HTTP streaming (AppleHLS & AdobeHDS),
and they announced that at least audio-on-demand will be migrated in 2015 to HTTP streaming, too!
I bet they have similar plans for video-on-demand; if this is realised, no Tor plugin or free (open) HTTP
proxy could handle all these video streams...:-( A high bandwidth paid-for UK VPN will then be mandatory for us
overseas BBC lovers...8-)

Re: bbc.co.uk downloads truncated....

rancor — Sat, 31 Jan 2015 21:58:37 +0000

Still problem with HD files - no change (enable "hightest available" or "720p"). Working with other quality level.

Re: bbc.co.uk downloads truncated....

ntoskrnl — Sat, 31 Jan 2015 21:22:10 +0000

Okay, I updated the Tor plugin, please try v1.0.2.

Re: bbc.co.uk downloads truncated....

ad48 — Sat, 31 Jan 2015 19:12:41 +0000

Tor is proxy

Re: bbc.co.uk downloads truncated....

rancor — Sat, 31 Jan 2015 17:49:32 +0000

Google translate:

I have the same problem:

File (HD) [www.bbc.co.uk] download 47.17 MB (level3 and limelight and port 1935)

Other files (HD): 0 bytes size

Download with "lowest available" OK

All files playing ok in browsers (with UK proxy).

Sat Jan 31 18:35:09 CET 2015 WARNING: Unable to verify signatures on consensus document, discarding...
Sat Jan 31 18:35:12 CET 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Sat Jan 31 18:35:16 CET 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Sat Jan 31 18:35:16 CET 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Sat Jan 31 18:35:16 CET 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Sat Jan 31 18:35:16 CET 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Sat Jan 31 18:35:16 CET 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Sat Jan 31 18:35:16 CET 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Sat Jan 31 18:35:18 CET 2015 WARNING: Couldn't create fir C:\Documents and Settings\USER\Dane aplikacji\VitySoft\FRD
Sat Jan 31 18:35:44 CET 2015 WARNING: Failed to download router descriptors: Directory request timed out
Sat Jan 31 18:36:41 CET 2015 WARNING: video duration is null
Sat Jan 31 18:36:51 CET 2015 WARNING: video duration is null
Sat Jan 31 18:37:03 CET 2015 WARNING: video duration is null
Sat Jan 31 18:37:19 CET 2015 WARNING: Couldn't create fir C:\Documents and Settings\USER\Dane aplikacji\VitySoft\FRD
Sat Jan 31 18:38:31 CET 2015 WARNING: Couldn't create fir C:\Documents and Settings\USER\Dane aplikacji\VitySoft\FRD
Sat Jan 31 18:39:35 CET 2015 WARNING: video duration is null
Sat Jan 31 18:39:44 CET 2015 WARNING: video duration is null
Sat Jan 31 18:39:53 CET 2015 WARNING: video duration is null
Sat Jan 31 18:39:57 CET 2015 WARNING: Couldn't create fir C:\Documents and Settings\USER\Dane aplikacji\VitySoft\FRD
Sat Jan 31 18:42:17 CET 2015 WARNING: video duration is null
Sat Jan 31 18:42:43 CET 2015 WARNING: video duration is null
Sat Jan 31 18:43:45 CET 2015 WARNING: Couldn't create fir C:\Documents and Settings\USER\Dane aplikacji\VitySoft\FRD
Sat Jan 31 18:43:52 CET 2015 WARNING: video duration is null
Sat Jan 31 18:44:00 CET 2015 WARNING: Couldn't create fir C:\Documents and Settings\USER\Dane aplikacji\VitySoft\FRD
Sat Jan 31 18:44:45 CET 2015 WARNING: Couldn't create fir C:\Documents and Settings\USER\Dane aplikacji\VitySoft\FRD
Sat Jan 31 18:46:15 CET 2015 WARNING: video duration is null
Sat Jan 31 18:46:19 CET 2015 WARNING: Couldn't create fir C:\Documents and Settings\USER\Dane aplikacji\VitySoft\FRD
Sat Jan 31 18:46:30 CET 2015 WARNING: Couldn't create fir C:\Documents and Settings\USER\Dane aplikacji\VitySoft\FRD

Re: bbc.co.uk downloads truncated....

tong2shot — Sat, 31 Jan 2015 10:47:37 +0000

Try to delete "tor-gb" directory from your FRD config directory (Help->Browse to log file, then up one directory, config directory is log's parent).

I'll try to test it further at night or tomorrow.

Re: bbc.co.uk downloads truncated....

ilanad — Sat, 31 Jan 2015 02:45:59 +0000

changed it to Limelight and now instant finish with 0 bytes, same two links

very strange log file

Fri Jan 30 19:36:46 MST 2015 WARNING: Couldn't create fir X:\bin\FreeRapid-0.9u4\config
Fri Jan 30 19:37:16 MST 2015 WARNING: Couldn't create fir X:\bin\FreeRapid-0.9u4\config
Fri Jan 30 19:38:36 MST 2015 WARNING: Couldn't create fir X:\bin\FreeRapid-0.9u4\config
Fri Jan 30 19:38:46 MST 2015 WARNING: Unable to verify signatures on consensus document, discarding...
Fri Jan 30 19:38:50 MST 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Fri Jan 30 19:38:52 MST 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Fri Jan 30 19:38:52 MST 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Fri Jan 30 19:38:52 MST 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Fri Jan 30 19:38:52 MST 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Fri Jan 30 19:38:52 MST 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Fri Jan 30 19:38:52 MST 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Fri Jan 30 19:39:11 MST 2015 SEVERE: Status code je 500
Fri Jan 30 19:39:39 MST 2015 SEVERE: Status code je 500
Fri Jan 30 19:39:42 MST 2015 SEVERE: Status code je 500
Fri Jan 30 19:39:46 MST 2015 WARNING: Couldn't create fir X:\bin\FreeRapid-0.9u4\config
Fri Jan 30 19:39:50 MST 2015 SEVERE: Status code je 500
Fri Jan 30 19:40:00 MST 2015 WARNING: video duration is null
Fri Jan 30 19:41:46 MST 2015 WARNING: Couldn't create fir X:\bin\FreeRapid-0.9u4\config
Fri Jan 30 19:42:16 MST 2015 WARNING: Couldn't create fir X:\bin\FreeRapid-0.9u4\config
Fri Jan 30 19:42:30 MST 2015 WARNING: Couldn't create fir X:\bin\FreeRapid-0.9u4\config
Fri Jan 30 19:42:46 MST 2015 WARNING: Consensus signed by unrecognized directory authority: 23d15d965bc35114467363c165c4f724b64b4f66
Fri Jan 30 19:42:50 MST 2015 WARNING: video duration is null

Re: bbc.co.uk downloads truncated....

ilanad — Sat, 31 Jan 2015 00:21:52 +0000

quality : highest available
RTMP Port 1935 (default)
CDN: Level 3

Using Tor (no proxy)

just tried again and same results (different sizes)

it has been working GREAT for months, just started doing this wednesday

will give Limelight a shot

Re: bbc.co.uk downloads truncated....

tong2shot — Fri, 30 Jan 2015 23:05:17 +0000

Please post your BBC plugin settings: quality, port, CDN, tor. Are you using proxy?

Are those downloads completed? Are there any error messages?

Currently I'm downloading your example #2, it's now 60MB of 255MB. I have slow connection, so it will take a while.

My plugin settings:
Quality : lowest available.
Port: 1935
CDN: Limelight.
Tor: enabled, but I'm using proxy, so tor will be ignored.

Edit: it's now 98MB of 254MB, and I get "Pipe broken" error. It's probably connection issue.

bbc.co.uk downloads truncated....

ilanad — Fri, 30 Jan 2015 21:52:54 +0000

started a couple days ago, downloads are coming up WAY short.....

these should be around 530mb each, 52mb and 2.9mb ....

[www.bbc.co.uk]

[www.bbc.co.uk]