With the media and legal spotlight being shined on this traffic category in recent days, a number of competing sites have either decided to shut themselves down, or dramatically alter the way files are uploaded or downloaded from their servers.

One such service to make such a drastic change was FileSonic, who has halted any new uploads, and is only allowing users to download their personal files.

The graph above compares several of the leading storage and back-up services in North America and how their traffic fared both before and after Megaupload was shut down. 

What is interesting is that before FileSonic stopped allowing downloads at approximately 18:00h (UTC-5) on January 22^nd, it was the clear front-runner application, generating significantly more traffic on this network than any of its competitors (prior to the Megaupload shutdown, the two appeared roughly equal).

We will be keeping a keen eye on further changes to these storage and back-up services, and will be sure to provide an update if anything significant develops.  For more information on Sandvine’s Internet traffic trends, visit our Global Internet Phenomena Report.

In a controversial, incredibly strange, yet kind of catchy video released last year, Megaupload claimed to have 1 billion users, accounting for 4% of Internet traffic.

Our Global Internet Phenomena Report, which has followed the growth of the service in recent years, confirms that Megaupload was the most popular Storage and Back-Up Service in almost every region.

The table below shows the combined usage of Megaupload and MegaVideo on fixed access networks in several regions from our Fall 2011 study:

What will happen now that Megaupload has been taken down? The most likely scenario is that users will switch to other Storage and Back-Up Services and video sites, causing new category leaders to emerge.  Currently the second most popular Storage and Back-up Service varies by region, but some of the top contenders to replace Megaupload are Rapidshare, zShare, Hotfile, and MediaFire.

As always, we will observe and report on the continued evolution of Internet traffic in future Global Internet Phenomena reports.

Kerfye Pierre had recently returned from volunteering in the aftermath of January’s Haiti earthquake when she got the most outrageous mobile phone bill of her life.  The tab? Nearly $35,000.  “I was like, ‘That cannot be possible,’ ” the 27-year-old Washington woman said of the moment she saw that staggering monthly statement.

In another recent story, a woman visiting Egypt got a bill for $37,000 upon her return.  In this case, the provider (Telus) did provide notification of the spike in data use, but the subscriber overrode the cut off and used 1.6GB of data, resulting in the eventual bill shock.

These subscribers are not alone; in fact, an FCC survey indicates that one in six mobile phone users within the United States have experienced an unexpected increase in the billed amount without a corresponding increase in voice usage.  In these instances, the subscriber needs to be notified right away, with current data.  It is no use to warn subscribers 12 hours later, when they may have surpassed their limits.

Stories like this have led Austrian mobile operators to sign a voluntary code of practise, just this week.  In Canada, Rogers provides usage advice on their fixed cable network. Similarly, AT&T has proposed to move towards different billing models in the US, and it is likely they will also provide similar real-time notification.

In United States, the Federal Communication Commission has proposed rules that will protect consumers from such “bill shock”, but there is some doubt whether this is under the jurisdiction of the FCC versus the Federal Trade Commission whose mandate is consumer protection:

• Over-the-Limit Alerts: The FCC’s proposed rules would require customer notification, such as voice or text alerts, when the customer approaches and reaches monthly limits that will result in overage charges.
• Out-of-the-Country Alerts: The FCC’s proposed rules would require mobile providers to notify customers when they are about to incur international or other roaming charges that are not covered by their monthly plans, and if they will be charged at higher-than-normal rates.
• Easy-to-Find Tools: The FCC’s proposed rules would require clear disclosure of any tools offered by mobile providers to set usage limits or review usage balances. The FCC is also asking for comment on whether all carriers should be required to offer the option of capping usage based on limits set by the consumer.

These proposed rules are quite similar to ones that came into force in the European Union in March, 2010.  The EU rules go so far as to cut-off roaming data access once a defined “cut off limit” is met.  Besides the obvious reason why this is good for consumers, the EU’s press release goes on to further note that:

Thanks to the EU’s roaming rules, the price that operators pay each other per megabyte (MB) downloaded has been limited to a safeguard level of 1€ per MB, and it will fall over the next two years. These savings should be passed on to consumers and deliver lower prices for surfing the Internet while abroad.

So not only can customers vacation knowing that they won’t get a nasty surprise in the mail a few weeks later, but they should also experience lower roaming costs in general.

On the other side of the coin, what do these regulations mean for the mobile carriers?  Well, it takes an awful lot of technology to build and maintain a network in the first place, and now carriers must deploy new functionality on top of the existing network.  At a minimum, the network operators must be able to monitor usage and make that data available to web portals and push mechanisms like e-mail and text messaging.  For a network operator, an end-to-end solution that seamlessly integrates with the existing infrastructure is necessary to limit the operational complexity and deployment overhead.  Let’s keep in mind that the ultimate bearer of additional network operating costs is the end consumer, so everyone benefits when a solution can be deployed quickly and to the satisfaction of the regulatory bodies.

Until such consumer-protection measures are in place in the US, the FCC’s Consumer Task Force has put forth a number of strategies for mobile subscribers to help them avoid bill shock.  I’ve reproduced them below, with slight edits to make them less US-specific:

• Understand your calling pattern for making voice calls, and ask your carrier for a plan that would be best for your kind of use
• If you are an infrequent phone user, consider a pre-paid plan
• Understand what your roaming charges are and where you will incur them
• Understand your options for data and text plans
• If you are going to use your mobile phone outside your place of residence for voice, e-mail, and other services, make certain to find out beforehand what charges may apply (and be sure to know the specific charges associated with each type of usage)
• Ask how your carrier what features they provide to help you avoid bill shock, such as phone or text alerts, by letting you monitor your account online, or by giving you other information

Finally, for more information about the FCC’s initiatives relating to bill shock, you can check out the FCC’s bill shock page.

The most credible reports, such as that from The Telegraph explain that the shut down involved the withdrawal of more than 3,500 Border Gateway Protocol (BGP) routes, used by networks to establish routing between one another. When a border router is turned off or removed, its routes are automatically withdrawn, and the networks beyond it become unreachable. Egypt, although a large country, is served by a small number of border routers.

Other reports, such as this Huffington Post story, appear to be misinformed. This article blamed a solution from a U.S.-based company, Narus, whose products incorporate deep packet inspection (DPI) technology. The article then went on to call for Congress to pass rules on the use of DPI. ⁽¹⁾ It also named a variety of companies, including Sandvine, that include DPI in their solution sets.

As Sandvine has described in its submissions to the Privacy Commissioner of Canada, and other global legislative bodies, the proper focus of any rules should not be on any technology, but rather on applications of technologies. For example, the same Single Lens Reflex (SLR) camera technology used to take photos at a family birthday parties is also used in surveillance cameras that may invade and undermine individual privacy rights. Should rules be created for SLR technology? Or should they focus on the protection of privacy?

Sandvine’s network policy control solutions comprise applications to:

• Manage network congestion
• Report on traffic trends for capacity and business planning
• Mitigate malicious traffic
• Measure network performance
• Create innovative new service tiers

All of these solutions are designed to improve the subscriber’s Internet experience. None of these solutions involve censorship.

In fact, DPI is a key technology used in a very wide variety of consumer and network equipment.  The wireless router in one’s home uses DPI to ensure maximum quality of experience for time-sensitive applications like VoIP or gaming, and also allows multiple devices in a home to share a single Internet connection.

Firewalls, many built right into popular PC operating systems, use DPI technology to analyze packets and identify viruses, trojans, and spam. Libraries, schools and government institutions rely on firewalls to protect themselves and their users from attacks.

Load balancers and routers, indispensible hardware that distribute traffic on the Internet and private networks, use DPI to identify where a given packet or URL should be routed and what priority it should be given.

In reality, there is very little difference between the network infrastructure in countries where citizens enjoy the free and open exchange of information and those that unreasonably restrict such rights.  The difference lies not in technology, but in the principles and policies that are implemented.

Footnote:

1.  On a related note, legislation has been proposed in the US (Bill S.3480 – Protecting Cyberspace as a National Asset Act of 2010, a.k.a. the Internet “Kill Switch” bill),  that provides the President with the ability to control systems that constitute critical cyber-infrastructure.

Notably, for Sandvine, the Order appears to preserve an environment in which deployment of Sandvine’s Network Analytics, Fairshare, Usage Management and Quota Management policy applications can continue to thrive.  Other significant aspects of the Order are highlighted in the sections below.  Sandvine continues to provide solutions to service providers globally, in support of their compliance with their respective regulatory and legislative environments.

The Rules in Brief

The basic rules require:

1. Transparency of traffic management practices, performance characteristics and the commercial terms of service. Mobile broadband providers must also disclose third-party device and application limits and any relevant criteria for use of such third-party offerings.
2. No blocking of lawful content, applications, services, or non-harmful devices. A less onerous no blocking rule applies to mobile broadband providers, which may not block access to lawful websites and may not block applications that compete with their own video or voice telephony services.
3. No unreasonable discrimination such as to favour one’s own application offerings over competitive offerings. The FCC acknowledged that some discrimination is beneficial. They also noted that paid prioritization is likely to be unreasonable, but will be subject to case-specific review. This rule does not apply to mobile providers.
4. All of the rules are generally subject to Reasonable Network Management.

Relevance to Sandvine’s Solution Portfolios

Business Intelligence

While the Order does not address Business Intelligence directly, gathering usage data about a broadband Internet access network would seem to be a necessary first step in effecting Network Policy Control and assessing any reasonable network management practice. Without this data there would be no way to determine the reasonableness of any network management technique. It certainly appears that the gathering of network data is not precluded in any way by the Order, and, to the contrary, may be a necessary pre-requisite to assessing compliance with the Order.

Traffic Optimization

Not surprisingly given the history of the debate, the Order deems an application-agnostic approach, such as the one that Sandvine introduced to the market with its Fairshare solution, as ‘likely reasonable’. Paragraph 73 of the Order states:

“Differential treatment of traffic that does not discriminate among specific uses of the network or classes of uses is likely reasonable. For example, during periods of congestion a broadband provider could provide more bandwidth to subscribers that have used the network less over some preceding period of time than to heavier users. “

The Order does not take a position on whether application-specific Traffic Optimization techniques are inherently reasonable or not, presumably leaving the issue open to case-by-case analysis. The Order seems to anticipate the deployment of such techniques, as in the Transparency discussion (paragraph 56) the Order requires service providers to disclose:

“…whether and why the provider blocks or rate-controls specific protocols or protocol ports, modifies protocol fields in ways not prescribed by the protocol standard, or otherwise inhibits or favors certain applications or classes of applications.”

The Order is clear that content, applications, services and non-harmful devices cannot be impaired or degraded to render them effectively unusable, unless doing so can be deemed necessary for reasonable network management.

Service Creation

The Order also seems to allow the introduction of tiered or differentiated services offerings, such as those supported by Sandvine’s Quota Management and Usage Management solutions. For example, the Order states (paragraph 72) that service tiers based on subscriber usage levels are reasonable:

“Some commenters suggest that open Internet protections would prohibit broadband providers from offering their subscribers different tiers of service or from charging their subscribers based on bandwidth consumed. We are, of course, always concerned about anti-consumer or anticompetitive practices, and we remain so here. However, prohibiting tiered or usage-based pricing and requiring all subscribers to pay the same amount for broadband service, regardless of the performance or usage of the service, would force lighter end users of the network to subsidize heavier end users. It would also foreclose practices that may appropriately align incentives to encourage efficient use of networks.  The framework we adopt today does not prevent broadband providers from asking subscribers who use the network less to pay less, and subscribers who use the network more to pay more.”

The Order further states that broadband offerings which give users choice of their own traffic priorities (paragraph 71) are reasonable:

“Thus, enabling end users to choose among different broadband offerings based on such factors as assured data rates and reliability, or to select quality-of-service enhancements on their own connections for traffic of their choosing, would be unlikely to violate the no unreasonable discrimination rule, provided the broadband provider’s offerings were fully disclosed and were not harmful to competition or end users. We recognize that there is not a binary distinction between end-user controlled and broadband-provider controlled practices, but rather a spectrum of practices ranging from more end-user controlled to more broadband provider-controlled. And we do not suggest that practices controlled entirely by broadband providers are by definition unreasonable.”

Operations Management

The Order explicitly recognizes the ability of service providers to take action with respect to harmful network traffic and to utilize solutions such as the Network Protection solution offered by Sandvine (paragraph 88):

“Broadband providers may implement reasonable practices to ensure network security and integrity, including by addressing traffic that is harmful to the network.”

In a footnote that describes traffic that is harmful to the network, the Order identifies spam, botnets, and distributed denial of service attacks.

Additionally, the Order’s Transparency requirements (paragraph 56) state that certain performance characteristics of service offerings be disclosed, such as the “expected and actual access speed and latency, and the suitability of the service for real-time applications.” Certain applications in Sandvine’s Operations Management suite could help service providers satisfy this requirement.

Reasonable Network Management and Transparency

Reasonable Network Management

The Order defines reasonable network management (paragraph 82):

“A network management practice is reasonable if it is appropriate and tailored to achieving a legitimate network management purpose, taking into account the particular network architecture and technology of the broadband Internet access service.”

So, different techniques may be deployed in different networks, thereby recognizing the architectural differences between fixed and mobile networks, for example. In the same paragraph, the Order further states:

Legitimate network management purposes include: ensuring network security and integrity, including by addressing traffic that is harmful to the network; addressing traffic that is unwanted by end users (including by premise operators), such as by providing services or capabilities consistent with an end user’s choices regarding parental controls or security capabilities; and reducing or mitigating the effects of congestion on the network. The term “particular network architecture and technology” refers to the differences across access platforms such as cable, DSL, satellite, and fixed wireless.

In paragraph 92, the Order recognizes that other practices not included in the definition above may also be considered a “reasonable network management practice” and that broadband service providers “should have the flexibility to experiment, innovate and reasonably manage their networks.”

The Order further recognizes that “appropriate and tailored” in the definition of reasonable network management does not rise to the same “narrowly and carefully tailored” standard as previously used by the FCC’s Comcast Network Management Practices Order in 2008. The FCC expressly acknowledged that this previous standard was unnecessarily restrictive (paragraph 85). In the same paragraph the Order states that service providers “need not necessarily employ the most narrowly tailored practice theoretically available to them.”

Transparency

Service Providers must be aware of the detailed Transparency requirements of the Order, most notably for disclosure of Network Policy Control practices in respect of (Paragraph 56):

• Congestion Management: If applicable, descriptions of congestion management practices; types of traffic subject to practices; purposes served by practices; practices’ effects on end users’ experience; criteria used in practices, such as indicators of congestion that trigger a practice, and the typical frequency of congestion; usage limits and the consequences of exceeding them; and references to engineering standards, where appropriate.
• Application-Specific Behavior: (as discussed under Traffic Optimization above)
• Device Attachment Rules: If applicable, any restrictions on the types of devices and any approval procedures for devices to connect to the network.
• Security: If applicable, practices used to ensure end-user security or security of the network, including types of triggering conditions that cause a mechanism to be invoked (but excluding information that could reasonably be used to circumvent network security).

In paragraph 59, the Order notes that “We also expressly exclude from the rule competitively sensitive information, information that would compromise network security, and information that would undermine the efficacy of reasonable network management practices.” In paragraph 56, with respect to disclosure of the commercial terms of a service, the Order states that service providers must disclose in their Privacy Policy “whether network management practices entail inspection of network traffic, and whether traffic information is stored, provided to third parties, or used by the carrier for non-network management purposes.”

While public comments to the NPRM were streaming in, a federal appeals court finally issued a ruling in the Comcast vs. FCC case that launched the Network Neutrality debate in full force in 2008. The court found that the FCC didn’t have authority over Comcast’s network management practices.  So who owns the Network Neutrality debate? The FCC has started another process to try to get the jurisdiction that the courts said that they did not have, which caused countless US Congressmen to caution the FCC that they were stepping on the toes of Congress. All the while, industry was working together to try to come up with their own proposal. 

Last week, Verizon and Google issued a very public stance on the network neutrality topic.  In essence, their joint policy proposal enshrines the notions of reasonable network management (like the draft rules in the NPRM and the FCC’s broadband Policy Statement of 2005 before it) and the importance of transparency of such practices.
At Sandvine, we’ve long advocated that an “unmanaged network is not a neutral network”.  We took the opportunity to educate policy makers on the nature of today’s Internet; outlining how certain applications and users, left unmanaged, take more than a fair share of network resources, to the detriment of the quality of experience of others.  Access to the Internet needs to be equitable.  In consultations with the FCC, and other industry leaders such as the National Cable and Telecommunications Association, I have recommended the following criteria for ‘reasonable network management’:

1. Narrowly-tailored
Management is implemented only where congestion exists and when congestion is causing quality of experience issues for a large number of subscribers.

2. Proportional and reasonable effect
Policy has an effect on subscribers or applications that is proportional to the effect the user or application is having on the network.  Policy applies the smallest reasonable intervention to alleviate congestion and improve quality of experience for the majority of subscribers.

3. Legitimate and demonstrable technical need
Congestion management and quality of experience issues can be demonstrated to exist in the network and management’s technical remedies are effective in achieving its targeted goals.

4. Transparent disclosure
Operator can disclose its traffic management policies in a simple, useful and predictable manner.

5. Auditable
Service provider can demonstrate that the above requirements are met through its auditing and reporting capabilities.

It seems like the principles of reasonable network management and transparency are gaining broad acceptance across the industry.  While Sandvine supports the spirit behind the Verizon Google (V-G) proposal, I believe that one substantial improvement could be made with respect to the appropriate use of prioritization, where the V-G proposal is unclear. In one instance the proposal says: “Prioritization of Internet traffic would be presumed inconsistent with the non-discrimination standard, but the presumption could be rebutted.” And in another:
“Reasonable network management includes any technically sound practice: … to prioritize general classes or types of Internet traffic, based on latency”. 

I believe that the V-G proposal is trying to avoid anti-competitive prioritization (such as between service providers of a given application class, such as Google versus Yahoo!), while supporting prioritization for the purposes of managing congestion and preserving the quality of experience for users and applications (such as prioritizing time-sensitive VoIP traffic over e-mail during times of congestion), which I support. I am confident that any rules that may result from this proposal would clarify this point.

The V-G proposal wisely carves out “managed services”, which allows service providers to offer their own differentiated value-added IP-based services, such as an IP video or voice service, and prioritize that traffic to give subscribers the quality of service they expect. Also, the V-G proposal suggests that wireless networks be excluded from the rules, which I believe is a prudent approach for now. Given the dynamic evolution of wireless networks, a wait-and-see-how-it- works-in-fixed approach seems acceptable to me.  And last but not least, the V-G proposal calls for limited FCC jurisdiction, preventing a blank-slate of authority by the FCC over Internet broadband access, and essentially keeping other aspects of the Internet value chain unregulated. This is the outcome that Internet content providers are looking for, and the outcome that will continue to spur growth in consumer content choice.

Alongside the very Hollywood-esque show is CableNet. This always well-attended “product demo-focused” pavilion of the show, organized by CableLabs, provides insight into timely key themes such as interactive TV, video on demand, and broadband over cable. It gives operators the chance to see a broad range of product demonstrations within a very focused area.

Sandvine took the opportunity to demonstrate its latest product in our Network Business Intelligence Portfolio.  Network Data Analytics is focused on providing an executive level dashboard view of how subscribers are using their broadband service. It aims to answer questions like “How do my DOCSIS 3 subscribers use their service compared to the DOCSIS 2 subscribers?” or “Where will I get the best return on my CAPEX budget?”. It was great to see the many cable operators who  dropped by to explore. They were quite intrigued by the flexibility of the product and how it could potentially help them to enhance their businesses and their customers quality of experience. After all, providing better broadband is the end goal.

In addition to a wide range of operators, we were pleased to host FCC Chairman, Julius Genachowski, when he dropped by to see some of the trends that Sandvine is currently reporting on.

Were you at the show? Please share your better broadband observations and experiences.

This theme was also echoed at the Light/Heavy Reading panel in which our CTO, Don Bowman, participated in titled “Policy, DPI & Net Neutrality: Heaven or Hell”.  The general consensus by the panelists was that the definition of reasonable and transparent network management, although ambiguous, did provide service providers the latitude necessary to manage their networks for optimal subscriber experience and to move ahead with their plans. 

More specifically, closer to home, there is a lot of buzz about how to optimize mobile traffic to improve the subscriber quality of experience.  This is in the form of advanced techniques for delivering video since video is the “killer app” in mobile, just as it is on the fixed side.  Service providers want video on their network but need to ensure good quality – Sandvine solutions can easily interoperate with such specialized applications for traffic optimization.  Implementing network efficiencies, either at the edge with 4G technologies or in the core, is one of the 4 prongs that AT&T Mobility CEO Ralph de la Vega advocated in Tuesday’s CTIA keynote to spur mobile growth, the other 3 being releasing more spectrum, leveraging compatible technologies (Wifi, Femtocells), and optimizing applications for delivery over the air.  It’s exciting to be part of this mobile Internet explosion!

However, over the years discrimination has become synonymous with prejudice, and it is in this light that the network neutrality debate has focused. However, differentiating one thing does not have to come at the expense of another. Differentiation can be a means of reducing waste and increasing efficiency. Consider the case of a nation in south-east Asia. Assume a popular networked video game console is launched, and that all of the gaming servers are located in Redmond, Washington, USA. Assume that the nation has 2 primary means of reaching the west coast of the US: one that goes east to west, and one west to east. Based on distance alone, the east to west underwater cable will have the lowest latency. But, the overland west to east cable will also have many more routing exchanges, which add even more latency (as much as 1-2ms per exchange).

In this environment, if the ISP were to route gaming traffic destined to Redmond via the short cable, and email traffic via the long cable, there would be an advantage given to the gaming traffic, and no disadvantage to the email traffic. Thus differentiation does not need to come at the expense of anything. This network is more efficient, and all users would achieve a better experience using it.

Is it the intent of the new fifth principle to disallow such activity? The explicit wording would appear to do so. But that goes against the spirit of innovation and relentless search for efficiency that has been the core of the Internet culture.

Now, why might a municipality be able to invest, and a telecom company cannot? The municipality might have a different weighting on factors such as return on investment. Faced with the prospect of 0.01% interest, the local government might feel infrastructure has a better payback. The telecoms company might instead be looking for a 24 month return on capital. The municipality may also be able to weight intangible factors like increase in property tax due to higher income people settling there.