another weirdness for bbc.co.uk
Posted by ilanad
|
Re: another weirdness for bbc.co.uk May 20, 2015 12:56AM |
tong2shot wrote:
-----------------------
> So it's proxy problem.
> Here is my plugin settings :
> SettingsConfig{videoQuality=480p,
> rtmpPort=1935 (default), cdn=Limelight}.
rancor wrote:
-----------------------
> the problem only occurs when downloading files
> with max. 480p (old series without HD versions),
> HD (720p) downloading OK.
> This is not a problem with the proxy.
rancor is right: Unfortunately, this is NOT
a wrong proxy issue; instead it is a more ominous
change implemented by the BBC sometime
over the last weekend, or most probably since
Mon May 18th 2015!
If this is a permanent change, it would mean that
by using the Tor plugin/http UK proxy/"smart DNS"
service, only the HD (720p) TV files are accessible
from overseas (i.e. non-UK IPs) by FRD or other
BBC iPlayer downloaders...
ANALYSIS (for the devs and others who need
to get to the bottom of this):
Let me again bring you a quick summary from
some of my previous posts:
There are 3 different CDNs that serve BBC TV
content over RTMP protocol:
1. Akamai (Ak), hostname: "cp41752.edgefcs.net"
This serves all available quality variants (resolutions),
224p, 360p (@480kbps), 360p (@800kbps), 468p, 720p.
This has always been geo-blocked at FMS level,
meaning the server would reject connection from
overseas if Tor plugin/http UK proxy/"smart DNS"
is used...Successful connection & download achieved
via UK VPN (PPTP, OpenVPN and others), Socks4 proxy,
SSH tunnel to a UK server and the likes...
2. Limelight (Ll), hostname: "bbcmedia.fcod.llnwd.net"
This serves all SD resolutions, i.e.
224p, 360p (@480kbps), 360p (@800kbps), 468p.
This has been always non-blocked at FMS level,
meaning Tor plugin/http UK proxy/"smart DNS", when
used with FRD, could successfully start a fetch...
3. Level 3 (L3), hostname: "iplayertokfs.fplive.net"
This serves only HD resolution, 720p. Has always been
non-blocked at FMS level, so the same usage
comments I made for Ll apply for this one, too.
NB: From late Jan 2015 to mid-March 2015, this CDN
was partially broken, often for both UK and overseas,
but usually for non-UK IPs - this has been now fixed.
The last time I downloaded something from Ll was
on Sat 16/05/2015, ca. 15:00 UTC (pid=b05vs0x4).
Sometime since then, the BBC have implemented
geo-blocking to Limelight CDN, too!
Further research on my part has shown that the
hostname has changed to: "bbcfmt.fcod.llnwd.net",
which is UK-only!
With both Ak & Ll blocked at server level, no SD
download is possible anymore with just "proxy" usage...
The issue had been reported earlier today (19/05)
in the get_iplayer support forum at the following URL:
about overseas access, the thread was initially
hastily closed, and later completely deleted!
More info can be found at the GetiPlayerAutomator
issue tracker (this is a MacOS frontend for get_iplayer),
where "overseas access" is better tolerated,
since the original author of the app is an American:
get-iplayer-automator/issues
(and the following comment by "mostlyotter").
I have conducted my own tests and all
agree with mostlyotter's findings:
If I use a (verified to work) UK http proxy in Firefox,
sample programme pid=b05wdnst DOES NOT PLAY
in SD from Ll CDN (nor it does, of course, from Ak);
see attached screenshot.
If I connect through a UK VPN, then the same show
DOES STREAM OK from the Ll CDN, see 2nd screenshot...
To establish this further, I used the mediaselectorURL
to gather the necessary data to compile an rtmpdump
download command for Limelight:
pid=b05wdnst => vpid=b05wdnqk
Then, from a UK IP (proxy):
RTMPstreams-data
=> rtmpdump command for 468p (token lasts about 1hr):
So, it is a fait accompli: by changing the Limelight
hostname to a geo-blocked one, they have
effectively cut-out all non-UK traffic from this CDN too
Be sure they will soon patch up that other
loophole, i.e. the L3 CDN....
Later in the year, the beeb may also remove
the RTMP streams altogether, in favour of
AppleHLS (they have done so for live radio),
or AdobeHDS (as they have done for live TV).
The CDNs for those HTTP formats are all
geo-blocked at server level...
PS: I thank the forum moderators for letting me
post this (and previous) analysis here; I genuinely
feel that (through search engines) many people
will find the documentation invaluable!
A sad day really for BBC lovers!
(Gonna mourn privately now... )
-----------------------
> So it's proxy problem.
> Here is my plugin settings :
> SettingsConfig{videoQuality=480p,
> rtmpPort=1935 (default), cdn=Limelight}.
rancor wrote:
-----------------------
> the problem only occurs when downloading files
> with max. 480p (old series without HD versions),
> HD (720p) downloading OK.
> This is not a problem with the proxy.
rancor is right: Unfortunately, this is NOT
a wrong proxy issue; instead it is a more ominous
change implemented by the BBC sometime
over the last weekend, or most probably since
Mon May 18th 2015!
If this is a permanent change, it would mean that
by using the Tor plugin/http UK proxy/"smart DNS"
service, only the HD (720p) TV files are accessible
from overseas (i.e. non-UK IPs) by FRD or other
BBC iPlayer downloaders...
ANALYSIS (for the devs and others who need
to get to the bottom of this):
Let me again bring you a quick summary from
some of my previous posts:
There are 3 different CDNs that serve BBC TV
content over RTMP protocol:
1. Akamai (Ak), hostname: "cp41752.edgefcs.net"
This serves all available quality variants (resolutions),
224p, 360p (@480kbps), 360p (@800kbps), 468p, 720p.
This has always been geo-blocked at FMS level,
meaning the server would reject connection from
overseas if Tor plugin/http UK proxy/"smart DNS"
is used...Successful connection & download achieved
via UK VPN (PPTP, OpenVPN and others), Socks4 proxy,
SSH tunnel to a UK server and the likes...
2. Limelight (Ll), hostname: "bbcmedia.fcod.llnwd.net"
This serves all SD resolutions, i.e.
224p, 360p (@480kbps), 360p (@800kbps), 468p.
This has been always non-blocked at FMS level,
meaning Tor plugin/http UK proxy/"smart DNS", when
used with FRD, could successfully start a fetch...
3. Level 3 (L3), hostname: "iplayertokfs.fplive.net"
This serves only HD resolution, 720p. Has always been
non-blocked at FMS level, so the same usage
comments I made for Ll apply for this one, too.
NB: From late Jan 2015 to mid-March 2015, this CDN
was partially broken, often for both UK and overseas,
but usually for non-UK IPs - this has been now fixed.
The last time I downloaded something from Ll was
on Sat 16/05/2015, ca. 15:00 UTC (pid=b05vs0x4).
Sometime since then, the BBC have implemented
geo-blocking to Limelight CDN, too!
Further research on my part has shown that the
hostname has changed to: "bbcfmt.fcod.llnwd.net",
which is UK-only!
With both Ak & Ll blocked at server level, no SD
download is possible anymore with just "proxy" usage...
The issue had been reported earlier today (19/05)
in the get_iplayer support forum at the following URL:
https://squarepenguin.co.uk/forums/topic/tvmodes-flashvhigh-flashhigh-flashstd-not-working/but since they do not tolerate discussion
about overseas access, the thread was initially
hastily closed, and later completely deleted!
More info can be found at the GetiPlayerAutomator
issue tracker (this is a MacOS frontend for get_iplayer),
where "overseas access" is better tolerated,
since the original author of the app is an American:
get-iplayer-automator/issues
(and the following comment by "mostlyotter").
I have conducted my own tests and all
agree with mostlyotter's findings:
If I use a (verified to work) UK http proxy in Firefox,
sample programme pid=b05wdnst DOES NOT PLAY
in SD from Ll CDN (nor it does, of course, from Ak);
see attached screenshot.
If I connect through a UK VPN, then the same show
DOES STREAM OK from the Ll CDN, see 2nd screenshot...
To establish this further, I used the mediaselectorURL
to gather the necessary data to compile an rtmpdump
download command for Limelight:
pid=b05wdnst => vpid=b05wdnqk
Then, from a UK IP (proxy):
RTMPstreams-data
=> rtmpdump command for 468p (token lasts about 1hr):
rtmpdump -r "rtmp://bbcfmt.fcod.llnwd.net:443/a8984/e3" -a "a8984/e3?as=adobe-hmac-sha256&av=1&te=connect&mp=iplayerstream/secure_auth/1500kbps/modav/p02qwwtc_b05wdnqk_1431888493764.mp4,iplayerstream/secure_auth/480kbps/modav/p02qwwtc_b05wdnqk_1431888495161.mp4,iplayerstream/secure_auth/800kbps/modav/p02qwwtc_b05wdnqk_1431888493280.mp4&et=1432087491&fmta-token=03820e4e2b1cb1084f83ece19547f88a23aa1af6bd35c5ec1c27f47970c7f7df" -f "WIN 17,0,0,188" -W "http://emp.bbci.co.uk/emp/SMPf/1.11.13/StandardMediaPlayerChromelessFlash.swf" -p "http://www.bbc.co.uk" -C O:1 -C O:0 -y "mp4:iplayerstream/secure_auth/1500kbps/modav/p02qwwtc_b05wdnqk_1431888493764.mp4" -o "b05wdnqk_1431888493764.flv"The above command (with valid token) DOES NOT WORK from a non-UK IP:
RTMPDump v2.4 GIT-2014-03-02 (Compiled by KSV) (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL Connecting ... INFO: Connected... ERROR: Closing connection: NetStream.Failedhowever it DOES WORK when a UK VPN is used:
RTMPDump v2.4 GIT-2014-03-02 (Compiled by KSV) (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL Connecting ... INFO: Connected... Starting download at: 0.000 kB INFO: Metadata: INFO: duration 1699.84 INFO: moovPosition 36.00 INFO: width 832.00 INFO: height 468.00 INFO: videocodecid avc1 INFO: audiocodecid mp4a INFO: avcprofile 77.00 INFO: avclevel 31.00 INFO: aacaot 2.00 INFO: videoframerate 25.00 INFO: audiosamplerate 48000.00 INFO: audiochannels 2.00 INFO: trackinfo: INFO: length 42496000.00 INFO: timescale 25000.00 INFO: language und INFO: sampledescription: INFO: sampletype avc1 INFO: length 81592320.00 INFO: timescale 48000.00 INFO: language und INFO: sampledescription: INFO: sampletype mp4a 1408.545 kB / 7.36 sec (0.4%)
So, it is a fait accompli: by changing the Limelight
hostname to a geo-blocked one, they have
effectively cut-out all non-UK traffic from this CDN too
Be sure they will soon patch up that other
loophole, i.e. the L3 CDN....
Later in the year, the beeb may also remove
the RTMP streams altogether, in favour of
AppleHLS (they have done so for live radio),
or AdobeHDS (as they have done for live TV).
The CDNs for those HTTP formats are all
geo-blocked at server level...
PS: I thank the forum moderators for letting me
post this (and previous) analysis here; I genuinely
feel that (through search engines) many people
will find the documentation invaluable!
A sad day really for BBC lovers!
(Gonna mourn privately now...
)
Sorry, only registered users may post in this forum.