4G mobile broadband services and devices. But beyond all the buzzwords and hype, which companies can reliably provide next-generation speed?
We decided to find out by testing each of the four major national
carriers--AT&T, Sprint, T-Mobile, and Verizon--in 260 locations
spread among 13 U.S. cities. We found some clear winners and losers, and
some good news about wireless service in the United States as a whole.
Here are our conclusions.
Wireless data speeds have soared: Since this time last
year, the major wireless carriers, as a group, have increased their
average download speeds for laptop-modem users by more than threefold,
an apparent result of their urgent transition from 3G to
4G network technology.
(We measured the best service we could get--3G or 4G--in each testing
location.) Over laptop modems, the Big Four carriers now have a
collective average download speed of roughly 3.5 megabits per second in
our 13 testing cities, versus a nearly 1-mbps average download speed in
those cities at the beginning of 2010, a remarkable improvement.
In our previous
wireless-network performance studies,
we measured the "reliability" of the data service, expressed as the
percentage of tests in which we could obtain a good connection. But our
test results show that network service has improved to the point where
it's rare to find an unusable signal or no signal at all. So we have
retired our reliability measurement--another testament to the dramatic
improvements of the past year.
Verizon's 4G LTE is for real:
Verizon's 4G LTE service,
which is now in 38 U.S. markets, was widely available in 12 of our 13
testing cities. (We didn't go out of our way to test in areas served by
Verizon's LTE network; we haven't changed our list of testing cities in
the three years we've done these tests.) Our laptop-modem tests on
Verizon clocked speeds that were far faster than those on competing 4G
networks in the same tests (twice as fast as the second-fastest service,
in fact). Verizon's network had an average download speed of roughly
6.5 mbps and an average upload speed of 5.0 mbps.
One important caveat: A relatively small number of Verizon customers
currently use this new network. During our testing period, Verizon
offered only two laptop-modem models that worked on the network, and
none of the company's smartphones could take advantage of the new 4G
speeds. The performance of Verizon's network could degrade as more
people--and devices--connect to it.
And there's a downside to Verizon's 4G success. While the new 4G LTE
network is lightning-fast, our smartphone-based tests suggest that the
3G CDMA network that most Verizon smartphone customers use today may
actually be getting slower. The connection speeds we measured on our
Verizon (3G CDMA) testing smartphone (a
Motorola Droid 2)
stayed the same or decreased in 10 of our testing cities since last
year. And at the moment, those CDMA phones are all that's available to
Verizon Wireless customers.
T-Mobile smartphones are fastest: Verizon may have the fastest network for laptops, but in our tests T-Mobile had the speediest results for smartphones. The
T-Mobile HTC G2
we used for testing produced a 13-city average download speed of almost
2.3 mbps; that's about 52 percent faster than the second-fastest phone,
Sprint's HTC EVO 4G, which had an average download speed of 1.5 mbps.
T-Mobile also impressed in our laptop-modem tests. Although only half as
fast as Verizon's, T-Mobile's download speeds averaged almost 3 mbps in
our tests--more than a threefold increase from the carrier's nearly
0.9-mbps average download speed in our January 2010 survey. With these
laptop- and smartphone-based results, T-Mobile is proving to be a worthy
challenger to its much-larger competitors.
AT&T continues to grow, but perhaps not fast enough: AT&T, the
big winner in our January 2010 survey,
has continued to ramp up throughput speeds at about the same pace,
judging from this year's survey results. Its average download speeds in
our laptop-modem tests grew 76 percent to a roughly 2.5 mbps average
this year. But each of its competitors showed bigger jumps in download
speeds over the past year, resulting in a third-place finish for
AT&T in this year's speed results.
And AT&T's speed gains didn't translate well to our smartphone-based tests: The average download speeds we measured on our
Apple iPhone 4
(1.4 mbps) increased only 15 percent over the speeds we measured on the
same device in early 2010. However, AT&T intends to launch its own
4G LTE network later this year, a move that might tip the balance of the
4G speed race in its favor once again.
Sprint needs more 4G: In the cities where
Sprint offers its 4G WiMax service,
customers saw large speed increases over the past year. Sprint's
average download speeds grew 170 percent to 2.1 mbps in our tests this
year; the result would have been even better had the WiMax service been
more consistently available throughout our test locations. But in cities
such as New Orleans, Phoenix, and San Diego, where Sprint still relies
on its 3G CDMA network for data service, download speeds have fallen,
and remain well below the 1 mbps mark.
Next page: The test results, and our methodology
4G Speed-Test Results: Reading the Charts
In our study we tested both with representative smartphones and with a
laptop employing a USB modem recommended by the carrier. The
laptop-based testing, which uses the Ixia industry-standard testing
software, provides more precise metrics than smartphone testing does.
The laptop results are a good measure of the maximum performance
possible on a network and are a satisfactory predictor of the speeds
that the network will likely deliver to smartphones in a year or so.
We use Ookla, an FCC-approved Web-based speed test, to measure data
rates on smartphones. Those results aren't as precise for a number of
reasons: we must use different smartphones on different networks, and
the results necessarily reflect the limitations of the smartphone's
radio chipset, processor, and battery, and the test itself comes with a
somewhat higher margin of error.
The charts below (click to see enlarged versions) list the cities in the
leftmost column; moving rightward across the chart, you can see the
speed averages and network latency times for each of the four wireless
networks. Speeds are expressed in megabits per second (mbps). Latency
(or the time it takes a single small packet of data to travel to a
network server and back) is represented in milliseconds. We recorded
download and upload speeds and latency times during our laptop-modem
tests, and download and upload speeds in our smartphone tests. (For more
details, see "How We Test.")
Speed-Test Methodology in a Nutshell
Our testing method is designed to approximate the experience of a real
laptop-modem or smartphone user on any given day in their city.
PCWorld's testing partner, Novarum, tested in each of our 13 cities
during the first six weeks of 2011. At each of our 20 testing locations
in each city, we took a "snapshot" of the performance of each wireless
service, testing for upload speed, download speed, and network latency.
We looked for the fastest signal available for each carrier, searching
first for 4G service and then, failing that, defaulting to the carrier's
3G service. In all, we ran 177,000 timed performance measurements from
260 testing locations in both urban and suburban environments. (See "
How We Test" for additional information.)
Because we couldn't test every city in the country, we chose 13 cities
that are broadly representative of midsize and large wireless markets in
terms of size and topography: Baltimore, Boston, Chicago, Denver, New
Orleans, New York, Orlando, Phoenix, Portland, San Diego, San Francisco,
San Jose, and Seattle. Because wireless signal quality depends to a
large extent on variables such as network load, distance from the
nearest cell tower, weather, and time of day, our results can't be used
to predict exact performance in a specific area. Rather, they illustrate
the relative performance of wireless service in a given city on a given
day. Each speed number has a margin of error of plus or minus 5
percent.
Next page: Verizon's new 4G network impresses, but its 3G network stagnates
Verizon's new LTE service
smokes," says Novarum CTO Ken Biba, who helped test the network. The
speeds tell the story: Verizon's 13-city average download speed for
laptop modems is roughly 6.4 mbps, more than double the average download
speed of our study's second-place finisher, T-Mobile.
And that average includes Verizon's result in Portland, the only city in
our study that has no LTE service yet. Excluding Portland and looking
at the performance of the LTE network only, Verizon's average download
speed jumps to almost 7 mbps. Only in Orlando did the network average
less than 5 mbps, coming in at roughly 4 mbps.
Upload speeds were just as impressive. Overall, Verizon's upload speeds
averaged roughly 5 mbps in our 13 testing cities; average upload speeds
reached nearly 9 mbps in San Diego and San Jose. LTE networks differ
from older 3G networks in that they are designed to be symmetric--that
is, the pipe going from the client device up to the network is as wide
as the pipe going down to the client. In many of our 260 testing
locations, the Verizon network delivered upload speeds that were faster
than its download speeds. San Diego's average upload speed was faster
than its average download speed.
Such fast upload speeds can make
bidirectional apps
like videoconferencing, online gaming, and, later, mobile Voice over IP
(VoIP) work far more smoothly and look and sound better. In these apps,
the data you send from your device is just as important as the data you
receive.
Verizon's LTE service
is available, latency times averaged just 114 milliseconds,
significantly shorter than latency times in the HSPA+ and WiMax networks
we tested.
Verizon's LTE network gives us a nice look at the future of wireless
service, but only a minority of the operator's customers are using the
network at the moment. Verizon currently sells only two models of USB
modems that can tap the network, and the company isn't saying how many
modems it has sold. New LTE phones aren't likely to arrive until this
summer. So Verizon's LTE network currently handles nowhere near the
number of devices it will have to support in the future.
"Verizon's new 4G network is a screamer, but that's partly because
there's hardly anyone using it yet," says Craig Moffett, a senior
analyst for Sanford C. Bernstein & Co.
Verizon has been assuring skeptics that its network will remain just as
fast when loaded up with devices. "We're very comfortable with the
speeds we have said all along that our customers should expect: on
average, 2 to 5 mbps on the uplink and 5 to 10 on the down," says
Verizon Wireless spokesperson Thomas Pica. "That's on a fully loaded
network.''
Moffett accepts that claim: "Even as [the network] begins to get loaded
with the first smartphones this summer, it will probably keep the crown;
as usual, theirs is the network to beat."
Still, at present, Verizon's smartphone subscribers rely on the
company's 3G CDMA network. And that network, as demonstrated in our
tests, actually became slower over the past year.
In our
January 2010 survey of 3G service,
we measured average download speeds of around 1 mbps in almost all of
our testing cities (the 13-city average was 1.078 mbps) on our Motorola
Droid smartphone. In those same cities this year, we saw very similar
performance on our Droid 2 smartphone--again, most speed results were
grouped around the 1-mbps mark, but the 13-city average download speed
was 7 percent lower than last year's, at 1.008 mbps.
We found further evidence of a stagnant CDMA network in laptop-modem
tests in Portland, where the Verizon LTE service is not available. We
found an average download speed of 0.8 mbps in Portland last year, and
clocked an average speed of only 0.55 mbps this year. This, of course,
is lousy news for Verizon smartphone users, including those who recently
bought the new Verizon iPhone.
Did Verizon build its impressive LTE network at the expense of further
upgrades to its 3G CDMA network? Are the majority of Verizon subscribers
paying the price for the blazing speeds enjoyed by just a few? Verizon
chose not to comment on these questions.
Next page: T-Mobile's HSPA+ network offers competitive speeds
HSPA+ network service
and phones as "4G" this year. Its ad campaign promoting the
offering--you know, pretty girl, polka dots, poking fun at AT&T--has
been hard to avoid. But our test results show that the carrier has been
spending its money on far more than ad campaigns.
In short,
T-Mobile's network
is fast--far speedier and more reliable than it was just a year
ago--and is indeed pumping out speeds that are competitive with the 4G
services of the other providers. T-Mobile scored the fastest download
speeds in our smartphone tests, and took a respectable second place
behind Verizon Wireless in our laptop-modem tests.
T-Mobile more than tripled its download speeds in our smartphone tests
since last year. In our smartphone tests using the T-Mobile HTC G2, we
measured a 13-city average download speed of 2.3 mbps.
T-Mobile's 13-city average a year ago
(testing on an HTC G1) was 0.72 mbps. In Denver and Seattle, our
T-Mobile phone averaged download speeds of more than 3 mbps. We were
able to achieve a connection speed of more than 2 mbps in 52 percent of
our tests.
Upload speeds also rose dramatically from last year, improving from a
0.134-mbps average last year to almost 1 mbps this year. The T-Mobile
network produced average upload speeds above the 1-mbps mark in five of
our testing cities: Baltimore, Boston, New York, Orlando, and Seattle.
T-Mobile also scored very well, and improved considerably, in our
laptop-modem tests. The network averaged almost 3 mbps for downloads,
with average results nearing the 4-mbps mark in New York, Orlando, and
Seattle. Overall, T-Mobile's download speed in our 13 testing cities
grew 226 percent from last year's (very 3G-like) 0.87-mbps average
speed. Latency times averaged 173 milliseconds, not high enough to
disrupt services like HD streaming video, but enough to degrade VoIP
call quality slightly.
T-Mobile's competitors say that the HSPA+ technology it uses is not
really 4G as T-Mobile claims. That may be technically true, but T-Mobile
has proven that through systematic software enhancements it can deliver
speeds that are competitive with the 4G networks of its rivals. Given
the near-term upgrade path of HSPA+ technology, T-Mobile will likely be
able to continue doing so for the next few years.
Next page: AT&T's HSPA+ network delivers 4G-like results, but the growth of data speeds is slowing
AT&T's HSPA+ service
is definitely delivering 4G-like speeds. In our laptop-modem tests, the
service produced an average download speed of 2.5 mbps in our 13
testing cities.
AT&T tells customers to expect download speeds of "up to
approximately 6 mbps" in "key markets such as Chicago, Houston, and
Charlotte [North Carolina]." Although we didn't see many 6-mbps scores
in our laptop-modem tests, the network did hit download speeds of more
than 2 mbps most of the time (64 percent of the time, to be exact). In
fact, AT&T showed average speeds of roughly 2 mbps or greater in all
of the 13 cities in which we tested. The network produced its fastest
average download speeds in Chicago (3.3 mbps) and San Francisco (3.0
mbps).
AT&T's upload speeds were also strong, and similar to T-Mobile's.
Upload speeds in our laptop-modem tests grouped around the 1-mbps mark,
with Baltimore hitting a high of almost 1.4 mbps. This is a substantial
step up from AT&T's 13-city average upload speed of 0.77 mbps in
last year's tests, if not as dramatic an improvement as we saw in
AT&T's download speeds.
AT&T's HSPA+ network produced latency times that were very similar
to T-Mobile's. We measured an average delay of 169 milliseconds across
13 cities (T-Mobile's average was 173 milliseconds); we saw the highest
average latency scores in San Diego (273 milliseconds) and San Jose (226
milliseconds).
Yet the growth of AT&T's data speeds has slowed. Last year we found that
AT&T's data speeds had increased 72 percent over the
previous eight months.
This round, AT&T's speeds continued to grow over the past year, but
not as rapidly, and certainly not as swiftly as its competition.
Consequently AT&T finished third in both our laptop and smartphone
performance tests. In our laptop-modem results, AT&T trailed
T-Mobile only slightly, but showed well less than half the download
speed of Verizon LTE.
AT&T's slowing growth was even more apparent in our smartphone
tests. In our early-2010 study, we measured a 13-city average download
speed of almost 1.3 mbps on our AT&T iPhone 4, an improvement of 54
percent over the previous year. In this year's tests using the same
phone, that number moved up to 1.5 mbps, an improvement of only 15
percent.
Some cities were better than others for AT&T smartphones: Chicago
saw an average speed of 2.5 mbps while San Diego averaged only 0.8 mbps.
Upload speeds improved dramatically, however, as our AT&T
smartphone averaged 0.2 mbps in our 2010 tests and improved to just
about 1 mbps this year.
AT&T believes that its new 4G smartphones (which weren't available
at the time of our testing) and other devices will better utilize the
speed of its network. "AT&T has introduced two 4G phones--the
Motorola Atrix and the
HTC Inspire--and
has announced plans for about 20 4G devices this year," the company
says in an e-mail. "Regarding network speed, thorough and expansive
testing has concluded time and time again that AT&T operates the
nation's fastest mobile broadband network."
AT&T's speed increases over the past two years can be attributed to
software upgrades and infrastructure improvements. The operator
completed a networkwide
upgrade to HSPA 7.2
technology in late 2009, then announced earlier this year that it had
finished another upgrade to HSPA+ technology, which it says allows for
maximum theoretical download speeds up to 14.4 mbps. AT&T also has
been investing large amounts of capital in fiber-optic lines for the
movement of cellular data to and from the core of its network.
AT&T plans to launch its own 4G LTE network, as well as some 4G LTE smartphones to match, later this year.
Next page: Sprint's WiMax network offers good speeds, but inconsistent availability
Sprint offers its WiMax service
in most of our test cities, actually connecting with the WiMax signal
using our Sprint 3G/4G modem proved a hit-or-miss proposition. For
instance, in San Jose, California, we measured download speeds of below
(sometimes well below) 0.5 mbps in 8 of our 20 testing locations, a sure
sign that no WiMax service was available in those places.
When the
4G service
is unavailable, Sprint devices downshift to the company's 3G CDMA
service, which, our laptop-modem tests suggest, may have slowed somewhat
over the past year.
Average download speeds slowed considerably in New Orleans (-24
percent), Phoenix (-31 percent), and San Diego (-24 percent)--the three
cities in our tests where no WiMax is available.
Sprint says no such slowdown has occurred. "The 3G speed results you
saw do not match what we see, and what the independent third party
testing our network has reported," says Sprint spokesperson Stephanie
Vinge-Walsh. "We haven't seen any significant degradation in 3G from
last year to this year; our 3G speeds remain in the same range and at
the same high dependability levels."
Sprint's 13-city average download speed of roughly 2.1 mbps represents a
mix of CDMA and WiMax--3G and 4G--connection speeds. Overall, we
recorded throughput speeds of more than 2 mbps in about half of our
tests. In the majority of our test cities where WiMax was available, we
noted (anecdotally) a roughly 50-50 chance of connecting to the service.
There were exceptions: In Baltimore, Boston, and Chicago, the
laptop-modem speed results reflected that the 4G network was available
throughout the cities, with a few exceptions.
Of its 4G WiMax service, Sprint says users should expect average
download speeds of between 3 mbps and 6 mbps, with peaks of more than 10
mbps. Our tests left us skeptical of Sprint's claim. We never saw a
speed higher than 7 mbps, and we reached speeds of 6 mbps or more in
only 5 of our 260 testing locations. The WiMax network produced a fair
number of speeds within the 3-to-6-mbps window, but not consistently.
Sprint's upload speeds also tell the tale of a 4G service with spotty
coverage. In many of our testing cities, we saw mainly two kinds of
upload speeds: those of 1 mbps and above, suggesting that we had managed
to hook into the WiMax service, and those that were below (sometimes
well below) 0.4 mbps, suggesting that we had connected to the 3G CDMA
service. Overall, Sprint's average upload speed remains stalled in
3G-land, at just 0.6 mbps.
Sprint's CDMA and WiMax networks, combined, produced the worst average
latency score in our tests, at 214 milliseconds. Such network delay can
begin to degrade the smooth operation of real-time applications like
video chatting and VoIP calling.
The same disparities in Sprint's 3G and 4G networks showed up in our
smartphone tests. In locations where WiMax coverage was spotty or
nonexistent, average download scores were well below the 1-mbps mark. In
cities where we could regularly connect with the WiMax network (Boston,
Chicago, and New York), we saw download-speed averages of 2 mbps or
greater.
Despite its overall speed gains, Sprint's service ranked last in both
download speeds and upload speed in this year's laptop-modem tests. Had
Sprint's WiMax network
been widely available in all of our testing cities, the results would
have been much different. The 4G network isn't slow, it's just not in
enough places.
"Coverage has always been their Achilles' heel in 4G, and financial
problems at [WiMax partner] Clearwire
have slowed down their 4G network expansion nearly to a stop," says
Sanford C. Bernstein's Moffett. "A year ago, they were first to market;
now they're at real risk of falling behind."
The 4G Cometh
An important transition from 3G to 4G is under way and will continue
raising the bar for fast mobile broadband. If speeds continue increasing
at the rate they have been over the past year, 3G data service (and
speeds) will soon become just an unpleasant memory. Our tests show,
conclusively, that the
4G wireless service the carriers now offer--if it's available in your neighborhood--is already significantly faster than 3G service.
What will that mean? The 4G service will very likely speed up your
consumption of Web-based content, and smooth the operation of services
such as streaming video. In fact, 4G speeds are likely to let you do
things with your mobile device that you simply couldn't do with a 3G
connection, applications such as video chatting, online gaming, and VoIP
calling. 4G is the first incarnation of wireless broadband that might
finally
free people from the desktop, allowing us to manage our online lives whenever and wherever we want.
Next page: How we test mobile network speeds
Test Your Smartphone Data Speed."
