Network testing and benchmarking is an important way to improve operator performance and reputation.

Independent mobile network testing was developed during the 1990s to provide objective measurements of network quality. First used by operators in Europe and North America, it is now well established in emerging markets. Independent network testing has become an essential part of strategic reporting in addition to being a key element of customer experience and customer retention programs and marketing to attract new customers.

MNOs must ensure that they can deliver the levels service performance required by their licenses, regulators, shareholders and supervisory boards, in addition to being able to substantiate marketing claims made to existing and new customers.

There are a number of independent testing and benchmarking providers, including Ookla, Opensignal and Umlaut (formerly known as P3). Different methodologies are used by these providers; Ookla and Opensignal both exclusively use crowd sourcing, while Umlaut (P3) uses a mixture of drive tests, walk tests and crowd sourcing. These providers all offer high quality and reliable testing and benchmarking solutions for a range of connectivity and performance optimisation challenges. This article looks in more detail at the latest Umlaut score, released in November 2023.

The Umlaut score, which claims to be the de-facto global industry standard for MNOs, is a measure for management to compare their networks with others around the globe to enhance strategic decision making. Although the examples below relate to Western Europe, the framework used by Umlaut is based on a rule set applied globally, to provide a fair, transparent and independent evaluation of all networks. Umlaut conduct benchmarks in 70 countries with more than 180 mobile networks being evaluated with the Umlaut score.

The independent framework is the basis for measuring the actual customer experience as well as mobile network performance and capability, allowing technical analysis with a high level of detail. The Umlaut testing methodology and scoring models are regularly updated in response to technical developments.

The Umlaut score also takes into account growing customer expectations, expanding data volumes and rising data speeds by raising the requirements and thresholds of the tests each year. Despite this, most operators are able to improve their performance, demonstrating that Umlaut ’s challenging network tests contribute to the improvement of the networks.

The methodology of testing has evolved as networks themselves have changed, from 2G through to 5G. Testing has also responded to changing patterns of network usage, for example by analysing and scoring networks on their ability to handle data downloading via HTTPS, FTP and other protocols. Both data upload and download speeds are tested. Recently the Umlaut test has also included specific tests related to OTT traffic generated by, for example, YouTube and e-Gaming.

DACH 2023 Test Result Overview

The DACH (Germany /Austria /Switzerland) markets of Western Europe are recognised as pioneers in mobile network testing and tests have taken place in these countries on an annual basis for many years. The latest tests, performed in October 2023, cover a range of settings including drive and walk tests in all major cities, a large number of medium and smaller population centres, and in rural areas. Carrying out tests simultaneously in three separate countries means comparisons and benchmarks can be made cross border as well as within each country.

The tests look at both data and voice network performance in the different locations described, and also in specific settings including highway driving and rail travel. From this it is possible to identify the operators which have the best overall network performance in each country as well as in the individual settings. The ranking is a good advertisement for operators and provides regional consumers with useful advice while choosing a service provider.

In Germany the 2023 tests show that Deutsche Telecom won its 13th Championship in a row. Both Deutsche Telecom and Vodafone increased their overall scores significantly compared to the previous year, with the data category accounting for the largest gains. Third place operator Telefonica/O2 saw a more modest increase overall, but was able to improve its score significantly in Munich to become the leading operator in the city, overtaking country leader Deutsche Telecom and well ahead of Vodafone.

In Austria, Magenta extended its winning streak to six years in pole position. This is a remarkable achievement given that A1 and Hutchinson 3, Austria’s other two operators, have not been resting on their laurels; both have also seen improvements in their scores year-on-year in the 2023 tests.

In Switzerland all three operators Swisscom, Sunrise and Salt rank as outstanding networks. Swisscom and Sunrise are ranked in the top 5 operators worldwide, while Salt got its first ‘Outstanding’ result in 2023.

Apart from the reputational improvement to customers, these positive results show that operator efforts are paying off. In Deutsche Telecom’s annual report, the consecutive number 1 ranking of Deutsche Telecom (in Germany) and Magenta Telekom (in Austria) are referred on multiple occasions as one of the great achievements for the group, giving additional confidence to stakeholders.

Crowd Test

The separate crowd test crowdsources user experience data from large numbers of customers, meaning every user may be a part of the test, and delivering network performance feedback from many devices. This test is intended to supplement the measured values of the drive and walk tests, which are aimed at assessing peak performance.

Crowdsourced data is analysed to assess the quality of the everyday mobile phone performance delivered to large numbers of customers. In addition to voice, assessed categories include broadband coverage, active download and upload speeds, latency and stability. The results of crowdsourcing account for 25 per cent of the overall rating. Crowd tests show the network performance received by end users although the devices and tariffs used also have an impact on these results.

Network Reliability

The network reliability rating is not a separate category, but an additional view of the results of the other categories. For this purpose all measured values are divided into basic or everyday requirements, known as Qualifier KPIs, and values related to maximum performance, Differentiator KPIs.

The network reliability measurement only takes into account the Qualifier KPIs from the voice and data categories of the main tests as well as the basic results from the crowdsourcing. This makes it possible to investigate how well the networks fulfil the everyday requirements of end users.

Test Methodology

The DACH market testing program uses four test vehicles in each country, each equipped with nine smartphones. One Samsung Galaxy S23 per network operator is used to carry out the voice measurements, another S23 is used for the data tests and a third to establish the connections for the “Conversational app“ test case. For all measurements, the smartphones are set to “5G preferred” mode wherever this is supported by the network.

Network operators are given access to the testing framework conditions at an early stage, including the smartphones used, parameters taken into account and the testing schedule. Regular dialogue is intended to ensure firmware versions used on test smartphones optimally support technologies such as carrier aggregation and 5G variants such as DSS, NSA or SA.

Network Testing in Emerging Markets

More than 200 mobile networks in over 120 countries have now participated in mobile network testing programs similar to the DACH region one, including over 100 in emerging markets. Recently tested or benchmarked operators include Airtel, Jio, Globe, Movitel, MTN, Ooredoo, STC, Telkomsel, Tigo, TMCel, Vivo, Vodacom, Vodafone, Yettel and Zain in countries including Brazil, India, Indonesia, Kuwait, Mozambique, Nigeria, Qatar, Turkey, Saudi Arabia, Serbia, South Africa and Zambia.

Mobile network testing and benchmarking aims to fairly and transparently assess the global development of networks, to improve network quality and performance, and ultimately to deliver better service for customers, while improving operators’ reputations and increasing their business.

In today’s interconnected digital landscape, cybersecurity threats continue to evolve at an alarming rate. One such threat that keeps security professionals awake at night is the rise of AI-powered DDoS attacks. These attacks combine the destructive power of distributed denial-of-service (DDoS) attacks with the sophistication and agility of artificial intelligence (AI) technology. Understanding the implications of AI-powered DDoS attacks is crucial for organizations of all sizes and industries to protect their networks and data.
… [visit site to read more]

Insight

The Mobile Network Test is the annual testing of networks held by German magazine Connect in partnership with Umlaut (an Accenture company). It is regarded as one of the most important and widely recognised benchmarks in the industry. Beginning in Germany in 1993, the testing eventually expanded into Switzerland in 2011 and Austria in 2012. The goal of the tests, say Connect magazine, is “to investigate the maximum network performance while keeping an eye on everyday aspects.”

Let’s take a look at the results for the DACH region (Germany, Austria and Switzerland).

Germany
This year, both Deutsche Telekom and Vodafone have improved significantly compared to the previous year. Deutsche Telekom emerged as the winner, closely followed by Vodafone and Telefónica in second and third place respectively. Deutsche Telekom also takes the lead in terms of the drive tests in large and small towns and in the walk tests conducted in large cities.

In the voice category, Deutsche Telekom again takes the lead in all scenarios, hitting 99% an over in every category except for railways. Again, Vodafone and Telefónica are close behind, but it’s moving out into the rural areas where the gaps become starker.

Deutsche Telekom has won the Mobile network test in Germany for a thirteenth time, improving on last year’s results by 15 points, rating as outstanding. Vodafone has also made good progress, improving by 11 points on last year, and ranks second this year. Telefónica achieves higher 5G share than Vodafone in major cities, and ranked No 1 in its HQ place, Munich.

Austria
Austria’s top three operators this year were Magenta, A1 and Hutchinson 3. Magenta took the lead in the walk tests carried out in major cities across the country, closely followed by A1 and Hutchinson 3.

In the most difficult scenario, in the train connections used by the test team, the achieved points drop more significantly compared to other categories, and the results show the rankings are more pronounced. Therefore, due to the lower rankings, it is clear here that there is more room for optimization in the trains’ area.

Again, results in the quality reduce the further out of the city you travel, and so do the differences between the operators. This is a similar case with the railways again, with Magenta taking the lead in the voice test. But despite this, the report found Austrian connections on roads “particularly pleasing”. This, Connect emphasise, shows that mobile Internet connections also work reliably on car journeys through Austria.

To summarise, Magenta took the top spot in Austria this year, being rated “outstanding” again, and got its own all-time high score this year, surpassing 970, the first for 6 years in a row, and entering the global TOP5 Club.

A1 also scored “outstanding”, Hutchinson 3 improved on last year also, taking the bronze medal this year as it too made good progress with its 5G rollout, according to the Network Test’s “single review”.

Switzerland
Finally, in Switzerland, there have been significant improvements on last year too. All three providers (Swisscom, Sunrise and Salt) were rated “outstanding”, with Swisscom snatching the top spot. Sunrise is the only operator to have been outstanding for 8 consecutive years. Despite the high level of play here, there were still differences, which included Salt ranking a distant third place in the drive test in major Swiss cities. On the roads and railways, the performance of all three firms hardly differs from that of the cities, which is important to note.

This is the sixth year in a row that Swisscom has won the test in Switzerland, and its scores this year was more than impressive, achieving a huge 981 points out of a possible 1,000. Taking second place this year was Sunrise, with the judges noting that their improvement on last year means “this result deserves the highest recognition”. Both Swisscom and Sunrise are in the global TOP5 club.

Salt achieved the “outstanding” grade for the first time this year, although it was noted there is room for improvement in the availability of voice telephony on the connecting roads.

The method
The sophisticated and advanced methodology of the network test undertaken by Umlaut and Connect considers both the top performance and everyday requirements of the user, taking into account logistics, voice connection, data connection, crowdsourcing, broadband coverage, data rates and latencies, stability and reliability. “Our test is a result of a value-based initiative, and the responsibility we as an industry undertake to create a cleaner and more sustainable future. The fact that all mobile network operators were able to implement energy efficiency features and measures and were able to improve their results – in some cases significantly – speaks for itself. Chapeau to the industry,” said Hakan Ekmen, Global Networks Lead, Comms Industry and simultaneously CEO at umlaut.

This year’s results show impressive improvements from all parties in the DACH region. But, as ever, the goal of these tests remains the same: to investigate the maximum network performance while keeping an eye on everyday aspects.

News 

The UK incumbent BT is rushing to remove Huawei equipment by the end of the year to avoid fines from the government, according to reports from The Telegraph 

With just nine days until the deadline, BT still needs to switch millions of customers over to another supplier. BT has already been granted an 11-month deadline by the government, which was originally set at January 2023. 

In 2020, the UK government introduced sanctions on Huawei, banning the company from critical elements of the UK telecommunications infrastructure, and ordered that all of the country’s mobile operators remove all Huawei equipment from their 5G networks by the end 2027. Earlier this year, BT met a deadline to lower the amount Huawei technology in its radio access network. 

Being a major supplier of equipment for a decade, this is no small task. BT have said that replacing its full-fibre and 5G networks, for which they relied on Huawei heavily, would cost around £500 million. The replacement kit is to be produced by Ericsson. 

Companies that miss the deadline could be fined up to 10% of their revenue, or £100,000 for every day that passes the deadline by the government. As BT generated a revenue of £20.7 billion (in the year up to March 2023), the company’s fine could reach up to £2 billion. 

“We’ve met our initial targets – both our radio access network (RAN) traffic levels and sites were below the levels required by the Government for its July 2023 deadline. Our focus is now on work in the core for the Government’s deadline,” said a spokesperson for BT. 

UK communications regulator Ofcom is set to report on the progress of the deadline early next year. 

Want to keep up to date with all of the latest international telecoms news? Sign up for Total Telecom’s daily newsletter 

Also in the news:
Vivendi takes legal action over TIM sale
Gigaclear secures £1.5bn in fresh financing
Scotland to receive £450m in Gigabit Broadband Upgrade  

News

A group of 16 House Republicans have signed a letter demanding answers following the testimony of Alan Davidson, the Assistant Secretary of Commerce for Communications and Information at the National Telecommunications and Information Administration (NTIA)

This article was originally published by our sister title ‘Broadband Communities’

Press Release

First phase of an investment project in a new backbone to strengthen Italy’s position and Sparkle’s routes in the Mediterranean, ensuring resilience through an innovative path

Rome, 22 December 2022 – Sparkle, the first international service provider in Italy and among the top global operators, launches a project for the development of GreenMed, a new submarine cable system that will cross the Adriatic Sea connecting Italy to Croatia, Montenegro, Albania, to Greece and Turkey, before extending further into the Mediterranean.

GreenMed will create an innovative fibre optic infrastructure corridor to connect, through a diversified and low-latency route, Central Europe, the Balkans, and the Central-Eastern Mediterranean countries. From Italy’s Adriatic coast, the system will connect by land to the rich digital hub of Milan and, from there, to the other major European internet and cloud nodes.

The project also aims to support the strong development of the Balkan digital market – with a growth forecast of 25% (CAGR 22-29) – by offering a secure and diversified route in addition to the already existing terrestrial backbones.

Italy confirms its role as the primary hub of the Greater Mediterranean, connecting with the GreenMed system also the island of Crete in Greece, which in recent years has become a significant digital gateway for the region and the landing point for Sparkle’s BlueMed cable which connects Italy with France, Greece and various countries bordering the Mediterranean until reaching Aqaba in Jordan.

Enrico Bagnasco, CEO of Sparkle, comments: “With GreenMed, Sparkle continues the expansion of its network by creating a route with highly innovative features that crosses the Mediterranean basin to support the demand for intercontinental connectivity and the growing digitalisation of the Balkan area.” 

About Sparkle

Sparkle is TIM Group’s Global Operator, first international service provider in Italy and among the top worldwide, offering a full range of infrastructure and global connectivity services – capacity, IP, SD-WAN, colocation, IoT connectivity, roaming and voice – to national and international Carriers, OTTs, ISPs, Media/Content Providers, and multinational enterprises. A major player in the submarine cable industry, Sparkle owns and manages a network of more than 600,000 km of fiber spanning from Europe to Africa and the Middle East, the Americas and Asia. Its sales force is active worldwide and distributed over 32 countries.

Find out more about Sparkle following its X and LinkedIn profiles or visiting the website tisparkle.com.

Keep up to date with the latest international telecoms news by subscribing to the Total Telecom daily newsletter.

Also in the news:
Vivendi takes legal action over TIM sale
Gigaclear secures £1.5bn in fresh financing
Scotland to receive £450m in Gigabit Broadband Upgrade  

Insight

Written by Paul McHugh, Area Director UK at Cradlepoint

For those who’ve been around wireless for a while, the cellular vs Wi-Fi debate may seem a little tired. However, the debate persists, and the technologies continue to co-exist, as both Wi-Fi and cellular continues to evolve.

Wi-Fi is a type of wireless local area network (WLAN) that is favoured for supplying wireless connectivity to the home, office, campus and other facilities with “best effort” connectivity. While it can work in many environments, it is best suited indoors. Meanwhile, cellular connectivity is the dominant player outdoors, on mobile phones and many other devices.

However, recent events and connectivity ecosystem evolution have triggered significant changes to the dynamic between the two. The roll out of 5G has brought increased capacity, coverage, mobility, speeds, and lower latency. Meanwhile, Wi-Fi 6 is getting closer to cellular with increased capacity, coverage, and higher speeds.

Another important shift is cellular has become an attractive alternative to Wi-Fi for enterprises — specifically for those looking for greater support for business critical applications, wanting complete control of their network. This is evident in the rise of private cellular networks (PCNs), among today’s enterprises. Helping drive this growth in enterprise PCN are changes to spectrum policy, including the allocation licensed spectrum for enterprises. This enables enterprise companies to operate their own PCNs and exercise complete control over the network more easily.

As the enterprise evolves, three major wireless technologies are increasingly co-existing: public cellular, private cellular and Wi-Fi.

However, thanks to PCNs and the enhanced performance they provide in the form of increased coverage, mobility, reliability, security, and predictable network performance — enterprises receive something they’ve never possessed before: choice.

The rise of PCN Deployments

So, why may some enterprises be opting for private 5G or LTE over Wi-Fi? One of Wi-Fis limitations includes reliability. Because Wi-Fi operates on unlicensed spectrum, it may be available but not necessarily useable because of signal interference, traffic congestion or a minimal coverage area. In terms of security and capacity, Wi-Fi also comes up short when compared to a private 5G network. For example, private cellular networks can eliminate

credential-based attacks thanks to SIM-based authentication. Network users must have approved physical SIMs or electronic SIMs to access the network, giving enterprises more control over who enters their network. Also, even if a bad actor gets their hands on a device with an approved SIM, they’d only have access to the portions of the network for which that device is approved.

In terms of mobility, cellular networks are deterministic – meaning the network determines how to assign cellular clients to the cellular network access points (APs), and when to handoff to another cellular AP based on signal strength, QoS (quality of service) standards assigned by the enterprise network administrator, and other identifiers. Since Wi-Fi networks are not deterministic, this vastly improves the network reliability for mobile devices that roam between cellular APs in a PCN.

Size matters

Providing connectivity for large areas isn’t easy. When it comes to coverage, in many situations private cellular networks make a lot more sense as well. Often, private cellular can cover 10x the space outdoors compared to traditional Wi-Fi.

A warehouse or an outdoor storage yard could require hundreds of Wi-Fi access points. That’s a lot of hardware to install and maintain. In contrast, a company could adequately cover the same area with a few dozen 5G private cellular access points. The same dynamics also apply for seaports, manufacturing facilities, campus environments, and mining operations.

Then, there’s the question of reliability. Many warehouses and industrial environments now use sensors and other devices to connect or have more visibility into their operations and machinery requiring the constant sharing of data between machinery and databases. When it comes to automation and robotics, the lower latency that private 5G offers means greater control for enterprise users.

Then there’s the way 5G combines with edge computing, which provides near real-time processing by bringing data processing to the point of data creation, such as a factory or warehouse floor instead of the cloud. The low latency necessary for real-time data transfer, the control of the network across which that data transfers, and the inherent security a PCN provides that data is why PCNs are more reliable.

Simple and Secure

Lastly, what’s become increasingly difficult for the enterprise to ignore is that with the maturation of private cellular solutions, the current value proposition for Wi-Fi to support business and mission critical applications continues to shrink. PCN solutions are not only more comprehensive, but enterprises will find it’s easier than ever to deploy and manage them after deployment.