Visit the full NBR website
HomeBusiness  

TelstraClear limitless weekend shines harsh spotlight on data caps, contention

LATEST: Some businesses seething, but TelstraClear says need for compo after weekend go-slow

Data caps, that pesky sword hanging over our head, shaping our Internet usage, are back in the spotlight.

Back in August 2010, I had mused in a blog post (Not speed alone) that while fibre will deliver faster last-mile broadband speeds for urban dwellers, many of these people are unable to take advantage of the broadband speed they already have.

People and businesses bring up data caps every time as a major hurdle.

Focusing on increasing speed without addressing data caps is, ultimately, self-defeating.

Since then, most ISPs have increased their data caps. As a percentage change, the increases have been large. For example, Telecom doubled the cap on all its smaller and medium sized plans. TelstraClear was a later comer to the party. Its is now cutting monthly costs which gives customers more data for the same monthly price. For example. $96 a month now gets 90GB instead of 60GB (a 50% increase).

Unmetered weekend
Starting from 6 pm on Friday, 2nd December, TelstraClear turned off usage metering for the weekend. Like a starter’s gun going off, customers unleashed their suppressed demand and hit the pipes. TelstraClear’s network barely managed to stand up to the assault, with cable download speeds plummeting to dial-up levels. Customers vented their frustration and there is a real danger that this experiment will lead to the conclusion that current data caps are critical to allow decent Internet speeds for all.

That is wrong. Why that is the wrong conclusion needs a longer explanation after some preliminary analysis.

Overseas comparisons
But before doing that, let’s take a quick look at the situation overseas.

Remember, with TelstraClear we can now get 90 GB per month for $96. The parent company, Telstra, in Australia provides (converting to New Zealand dollars) 200 GB for $90 and 500 GB for $115. So for our $96 we get about a third to a fourth of the Australians, even after TelstraClear’s current 50% increase.

Looking at the United States and taking Comcast as the most hard-nosed example of ISPs there aggressively applying data caps, our $96 per month gets us a package called Blast! with a download speed of 20 Mbps. Note that most ISPs in USA don’t have formal data caps and price Internet connections by download speed. However, Comcast has a 250GB ‘threshold’ which is in effect a data cap. Cross that and Comcast calls it “excessive use”. Customers who repeatedly cross the threshold apparently get a polite call. A customer who crosses the threshold 3 times in a 6 month period can have their account suspended for a year, i.e. Comcast’s policy is to boot out customers who regularly exceed 250 GB per month. That’s why the 250GB is for all practical purposes a data cap.

So, even in comparison with the most restrictive ISP in the United States and a 50% hike in our data caps, we get about a third for the same price.

I have deliberately compared TelstraClear with Comcast so that we get the lowest difference (one third). There are other comparisons which show that our data caps are up to a tenth of those overseas, particularly for the more typical data cap levels of 10-20 GB.

The hotel analogy
There is one more piece to look at before getting back to the (wrong) conclusion that the current data caps are critical to allow decent Internet speeds for all.

As an analogy, imagine if there was only one hotel in New Zealand. The owner only leases hotel rooms by the floor for 20 years to 4 big operators. Customers of these big operators lease a small number of rooms for a few years but they retail hotel beds by the night. So, the ‘retailers’ are leasing rooms (from the big operators) but selling hotel beds to their customers.

Now the retailers decided that they would have more beds than rooms they lease and put in beds only in line with what they sold. That way the retailers can lease less rooms, keep their costs down, and maximise profits. Customers are also happy because the price of a bed night is lower than it would be if some beds remained unoccupied each night.

Periodically, as the overall demand for beds in New Zealand increases, the hotel owner adds a floor. Imagine that building costs are going down rapidly. The hotel owner will only add a floor just ahead of an increase in demand. In the meantime, the retailers juggle the number of beds and room leases while the 4 big operators continue to take a big cut of the money.

This is what is happening in the Internet access market, especially the international IP transit market. Keep in mind that something like 80% of our Internet content comes from overseas using international IP transit. It’s easy to see why the international IP transit market is therefore critical.

ISPs are the ‘retailers’ in this market. Retailers sell bed nights (data per month) to customers but buy rooms (bandwidth) from the 4 big operators in the monopoly hotel called Southern Cross Cable.

ISPs sell Internet access at sticker rates of “up to” a download speed of say 20 Mbps but are really selling data allowances. They do this confident in the knowledge that customers will use their Internet connection intermittently. So they buy much smaller upstream bandwidth per customer, say 64 kbps.

But this model is increasingly under threat. Customers now want to stream video or backup their data to the cloud. They want high quality video calling and (used to) share files with peers. All of these services use the full pipe and suffer from under-provisioning of backhaul and international IP transit.

Economics 101
If you prefer pictures over analogy, let’s draw some graphs to illustrate basic economics.

Figure 1: fully provisioned demand and supply

The graph above shows what Internet demand from people and businesses (D1) would look like in a free market where the last-mile speed is fully provisioned internationally. An equilibrium will be reached with the supply (S1) to determine the price (P1) and quantity of data (Q1).

The problem is that price P1 is too high for most people and businesses. At over a thousand dollars a month, it would simply put Internet connections out of reach for most. Also, a large part of the time the international bandwidth at an aggregate level would be idle and therefore wasted.

So ISPs purchase a fraction of the international IP transit bandwidth, called ‘contention’. The ratio varies widely across ISPs with a 1 Gbps pipe typically servicing between 1,000 and 5,000 retail connections of 10 Mbps.

Figure 2: shift in supply curve from contention

The impact of contention is to shift the supply curve to the right (S2). For the same data quantity Q1, price falls from P1 to P2. Done well, individual customers hardly notice the difference and both ISPs and customers are better off. Problems arise if ISPs are over aggressive, which shows up as congestion and customers complaining about a slow Internet connection.

However, New Zealand ISPs also intervene on the demand side by imposing data caps that suppresses demand. A contended international connection (supply side measure) does not stop customers from trying to download as much as they can from the limited bandwidth. Just a few customers using their contended Internet connection non-stop or ‘excessively’ would cause congestion for other customers. So ISPs limit the quantity of data an individual customer can download in a month to a very small fraction of what the contended Internet connection could theoretically download if used non-stop or ‘excessively’.

Figure 3: shift in demand curve from data caps

The result is that the demand curve shifts to the left (D2). The price remains at P2 but the demand is lower at Q2.

In summary, that’s the situation in New Zealand today - ISPs have delivered a lower price Internet connection by a combination of contention and suppressing customer demand.

Putting it together
It’s now time to go back and look at the question of the lessons from TelstraClear’s disastrous (for customers at least) unmetered weekend.

The conclusion that current data caps are critical to allow decent Internet speeds for all is, in my opinion, wrong. Given that New Zealand data caps are a fraction of what people and businesses get overseas for the same price we pay every month, continuing to suppress demand undermines our digital future.

Doling out periodic increases in data caps is not acceptable any more. Things need to change. Giving ultra fast speeds in the last mile actually makes the problem of data caps more acute.

So what’s the solution? Look at the graphs and it is evident that shifting the demand curve is only one option. The other one that hasn’t really been addressed by the industry and government is to shift the supply curve to the right. This allows higher data caps without an increase in retail prices.

One obvious answer is to increase supply by building another international cable. However, there are other things that can be done to shift the supply curve now, without waiting for another international cable.

I will address some possibilities in my next post. However, it is clear that one of the immediate steps is to let sunlight expose the murky international IP transit market.

Vikram Kumar is chief executive of InternetNZ.

More by Vikram Kumar

Comments and questions

The article is good because it raises the supply and demand issue however it misses two key points.

1) The consumers speed will only be as fast as the slowest bottleneck. There are multiple bottlenecks between the consumer and the international cable, each one which has contention controlled by the relevant supplier. Most bottlenecks have nothing to do with the international cable.

e.g. Consumer <> local cabinet <> local exchange <> handover <> ISP <> international cache <> international cable.
This is not 100% accurate but you get the idea.

2) Only things that download from point to point directly increase the use of the international cable. i.e. Peer to Peer file sharing and gaming. YouTube, google, Itunes, Microsoft Downloads, Video on demand etc are all locally cached in NZ. i.e. a youtube video is cached (stored) locally on the first download and from then on all requests for that file get satisfied from the NZ cache.

If the internet was slow for all things this weekend on Telstra then it was purely a Telstra problem, either infrastructure or contention.. The feedback on Chris Kealls earlier post verified that everything was slow.

I wouldn't accept any reason until I had seen the switch logs starting at local exchange / cabinet of a complaining consumer and every switch all the way to the Telstra. Would take less than 15 minutes. Telstra will of course hold that close to their chest.

This weekend was a marketing ploy - nothing less.

Well put Samuel.

I'd also raise the point that while looking at international benchmarks is helpful in some instances, i'd challenge Vikram to name another market of 4 million people spread over a wide hilly area that is a very long way from the major content sources....

NZ is different, in my opinion it needs different solutions rather than nation envy...

No mention has been made of TelstraClear's refusal to peer with local ISPs. If they'd bothered to peer directly at the exchanges, then a lot of this nonsense would have been avoided.

Instead, TelstraClear customers are exposed to the vagaries of Telstra's international capacity... and the customers are the ones who pay the price.

The economics provided here is misleading as it does not include the fact that extra data use causes additional demand for other resources in the broadband production chain, such as backhaul. The real marginal cost of 'all you can eat' unmetered internet is the effect of congestion on other users. Metered usage does the same thing as a carbon charge or even petrol tax for road users - making users internalise the effect of their usage on other users.

The use of metered charging in NZ is an artifact of the constrained international bandwidth faced by NZ operators when the internet was first developed (this was only addressed when the Southern Cross cable was commissioned, but the 'strategic imperatives' had already been set in place). Markets that start with metered usage create an ongoing strategic 'problem' for firms offering flat rate tariffs. As internet users have varying demands for capacity, the first firm to offer a 'flat rate' tariff attracts all the high-volume (high-cost) users, leading to greater congestion on that network than others - just as illustrated over the weekend for TelstraClear. The flip side is that the networks with metered charging get a 'cost;less bonus' - as they have the remaining low-volume users as their customers, so their congestion levels actually reduce, providing better service for their customers. The only solution for the flat-rate firm is to install more backhaul resources to reduce congestion, but the cost of this investment must be recvovered from consumers. If the extra resource is to be installed, flat-rate tariffs must rise. This leads to an even bigger gap between between the prices paid for flat-rate and metered internet. However, if flat-rate prices go up, some high-volume but low-value users with low values for their rusage will inevitably complain - but they have been causing congestion for lower-volume higher-valuing ones. The industry 'solution' is to find some other proxy for high-valued use - which has been to introduce high speed options. In NZ, we have offered capped tariffs for 'best efforts' speed on DSL. In other countries (Europe, USA) , a real choice of speed exists. The low-value users have 'all you can eat' on low-cost, low-speed plans, whilst high-valuers pay a premium for speed. in the USA, fewer than 30% of internet users pay for faster speeds than the basic offering.

In NZ, we have had (relatively)fast base speed for all users, but paid for usage - this is reflected inthe good average speeds at peak time illustrated in the Akamai speed surveys. We also do not pay a lot compared to other countries when usage is based on what the average user actually consumes.

The graphs that should have been drawn would show an implicit cross-subsidy from low-volume users to high-volume ones. This is not trivial - in Korea, I have been told that at peak hours, well over 95% of network capacity is consumed by the 10% of heaviest users.

Telstraclear: Over Promise - Under Provide.

Remember T-BOX?

Investors should be concerned.

One thing no one ever seems to mention with regard to speed is the upload rate. Everyone blathers on about the download rate assuming the internet has the same model as TV - where consumers passively suck down content presented to them.

What is missing from the debate (at least I've never seen it) is the *upload* speed. Sure, most people don't produce anything but for those that do (creative types, business, game servers etc) the cost for a decent upload speed is horrifically high.

The pundits and politicians all talk about the Digital Economy vital for New Zealand but not one of them agonizes about the ridiculous cost of *uploading* (exporting) content of value. That tells me they actual have no clue as to what it would really take to enable a true digital economy that New Zealand would profit from - rather than consuming content produced elsewhere around the Globe. If New Zealand wants compete in the digital era and improve its digital ex

One thing no one ever seems to mention with regard to speed is the upload rate. Everyone blathers on about the download rate assuming the internet has the same model as TV - where consumers passively suck down content presented to them.

What is missing from the debate (at least I've never seen it) is the *upload* speed. Sure, most people don't produce anything but for those that do (creative types, business, game servers etc) the cost for a decent upload speed is horrifically high.

The pundits and politicians all talk about the Digital Economy vital for New Zealand but not one of them agonizes about the ridiculous cost of *uploading* (exporting) content of value. That tells me they actual have no clue as to what it would really take to enable a true digital economy that New Zealand would profit from - rather than consuming content produced elsewhere around the Globe. If New Zealand wants compete in the digital era and improve its digital exports it really needs to address *upload* speeds rather than continually framing the debate about (Hollywood- and ISP-friendly) *download* speeds.

the only Lesson TelstraClear has learnt is NOT to have unmetered weekends. This failed PR stunt doesn't really tell us anything useful about the feasibility of unmetered breadband plans, or of affordable 200GB+ plans. Obvioulsy if customers are getting free interwebs over a 48hr period they are going to go crazy with the downloading and have completely atypical internet use. I like the Comcast model: no cap, but the application of a reasonable use clause in contracts, allowing for the odd month of huge downloading, but making sure people keep it reasonable most of the time.

Stunt or not, Capacity or not, it is now proven unambiguously that 'demand at the right price' hugely exceeds current utilisation, which when liberated grossly exceeded infrastructure ability to deliver. Actual utilisation of an uncapped internet service remains unknown as the service became effectively capped by over demand. Meanwhile providers wallow in their obscene prophets.

I was running speed tests just before and after midnight sun-mon.. and surprise surprise my ping to sydney reduced by 4 times and my mbps went up 10x in the space of a min. Telstra were capping our download speeds the whole weekend. So what the hell was the point of the weekend? Im compiling a letter of complaint for capping my speed, i understand speed going down due to use but actively capping speed i pay good isk for is BS

In terms of the data caps themselves, I do think that the telcos could increase these significantly without significant additional cost. The "unlimited" plans attract torrent and other heavy users that may download terrabytes in a 24 hour period. This 1-2% of users is the main issue for data shaping and management. Caps could be significantly increased (say 250-300mb) while still controlling the mega downloaders.

................. does any one Read there contracts ISP's only Promies Broadband speeds over 0.6mps on a res accounts, also residential contracts say that over 3 days of effected service is compensatable. so telstra is in its right to with Held compensation. Lastly the average speed in new zealand is between 2.97Mbit/s to 3.4Mbit/s

« Back to home page