Mobile


ARM Announces Project Trillium Machine Learning IPs

ARM Announces Project Trillium Machine Learning IPs

Today’s Arm announcement is a bit out of the norm for the company, as it’s the first in a series of staggered releases of information. For this first announcement Arm is publicly unveiling “Project Trillium” – a group of software solutions as well IP for object detection and machine learning.

Machine learning is indeed the hot new topic in the semiconductor business and has particularly seen a large focus in the mobile world over the last couple of months, with announcements from various IP companies as well as consumer solutions from the likes of Huawei. We’ve most recently had a more in-depth look and exploration of the topic of machine learning and neural network processing in a dedicated section of our review of the Kirin 970.

Whilst we had a great amount of noise from many industry players on the topic of machine learning IPs. Arm was conspicuously absent from the news and until now the focus has been on the CPU ISA extensions of Armv8.2, which introduce specialised instructions which simplify and accelerate implementations of neural networks with the help of half-precision floating point and integer dot products.

Alongside the CPU improvements we’ve also seen GPU improvements for machine learning in the G72. While both of these improvements help, they are insufficient in use-cases where maximum performance and efficiency are required. For example, as we’ve seen in the our test of the Kirin 970’s NPU and Qualcomm’s DSP – the efficiency of running inferencing on specialized IPs is above an order of magnitude higher than running it on a CPU.

As Arm explains it, the Armv8.2 and GPU improvements were only the first results towards establishing solutions for machine learning, while in parallel they’ve examined the need for dedicated solutions. Industry pressure from partners made it clear that the performance and efficiency requirements made dedicated solutions inevitable and started work on its machine learning (ML) processors.

Today’s announcement covers the new ML processors as well as object detection processors (OD). The latter IP is a result of Arm’s Apical acquirement in 2016 which saw the company add solutions for the display and camera pipelines to their IP portfolio.

Starting with the ML processor – what we’re talking about here is a dedicated IP for neural network model inferencing acceleration. As we’ve emphasised in our NN related announcements of late, Arm also emphasises that having an architecture which is specifically designed for such workloads can have significant advantages over traditional CPU and GPU architectures. Arm also made a great focus on the need to design an architecture which is able to do optimised memory management of the data that flows through a processor when executing ML workloads. These workloads have high data reusability and minimising the in- and out-bound data through the processor is a key aspect of reaching high performance and high efficiency.

Arm’s ML processor promises to reach theoretical throughput of over 4.6TOPs (8-bit integer) at target power envelopes of around 1.5W, advertising up to 3TOPs/W. The power and efficiency estimates are based on a 7nm implementation of the IP.

In regards to the performance figures, Arm agrees with me that the TOPs figure alone might not be the best figure to represent performance of an IP; however it’s still useful until the industry can work towards some sort of standardisation for benchmarking on popular neural network models. The ML processor can act as a fully dedicated and standalone IP block with its own ACE-Lite interface for incorporation into a SoC, or it can be integrated within DynamiQ cluster, which is a lot more novel in terms of implementation. Arm wasn’t ready to disclose more architectural information of the processor and reserves that for future announcements.

An aspect that seemed confusing is Arm’s naming of the new IP. Indeed Arm doesn’t see that the term “accelerator” is appropriate here as traditionally accelerators for Arm meant things such as packet handling accelerators in the networking space. Instead Arm sees the new ML processor as a more fully-fledged processor and therefore deserving of that naming.

The OD processor is a more traditional vision processor and is optimised for the task of object detection. There is still a need for such IP as while the ML processor could do the same task via neural networks, the OD processor can do it faster and more efficiently. This showcases just how far the industry is going to make dedicated IP for extremely specialised tasks to be able to extract the maximum amount of efficiency.

Arm envisions use-cases where the OD and ML processors are integrated together, where the OD processor would isolate areas of interest within an image and forward them to the ML processor where more fine-grained processing is executed on. Arm had a slew of fun examples as ideas, but frankly we still don’t know for sure how use-cases in the mobile space will evolve. The same can’t be said about camera and surveillance systems where we see the opportunity and need for continuous use of OD and ML processing.

Arm’s first generation of ML processors is targeted at mobile use while variants for other spaces will follow on in the future. The architecture of the IP is said to be scalable both upwards and downwards from the initial mobile release.

As part of Project Trillium, Arm also makes available a large amount of software that will help developers implement their neural network models into different NN frameworks. These are going to be available starting today on Arm’s developer website as well as Github.

The OD processor is targeted for release to partners in Q1 while the ML processor is said to be ready mid 2018. Again this is highly unusual for Arm as usually public announcements happen far after IP availability to customers. Due to the nature of SoC development we should thus not expect silicon based on the new IP until mid to late 2019 at the earliest, making Arm one of the slow-adopters among the semiconductor IP vendors who offer ML IP.

Related Reading

European Union Fines Qualcomm $1.23 Billion for Anti-Competitive Apple Deal

European Union Fines Qualcomm $1.23 Billion for Anti-Competitive Apple Deal

This morning the European Union’s European Commission became the latest regulatory body to fine Qualcomm over anti-competitive actions undertaken by the company. The investigation, which we’ve been expecting the results of for some time now, found Qualcomm guilty of abusing its dominant market position in LTE modems, with Qualcomm paying Apple to exclusively use its modems. As a result the Commission has levied the largest fine to date against Qualcomm, totaling over €997 million ($1.23 billion).

Qualcomm has been under scrutiny by regulators in one form or another for over a decade at this point. The company has previously been fined by China, South Korea, Taiwan, and there is still an ongoing investigation in the United States. While the precise infraction has varied some from fine to fine, in all cases regulators have cited Qualcomm for abusing its position in the cellular modem market in order to freeze out any competition. This has included their position to forcibly bundle unrelated patents and refusing to license out standards essential patents to customers who didn’t buy Qualcomm chips.

The European Commission’s case, by contrast, is perhaps the most interesting of the cases as it’s the most contemporary, dealing with Qualcomm’s actions from 2011 to 2016. The Commission’s case is solely focused on LTE shenanigans – other cases have tended to focus on CDMA or a mix of CDMA and LTE – with the regulatory body finding sufficient evidence of an anti-competitive Apple deal to charge the company under antitrust laws.

The Apple deal, which we first found out about in a US FTC investigation last year, had Qualcomm paying Apple royalty rebates in order to ensure Apple’s exclusive use of Qualcomm’s LTE modems. And while royalty rebates alone are not inherently illegal, the fact that Qualcomm was doing it in order to prevent other competitors from gaining a foothold in the LTE modem market – primarily Intel – is what makes it illegal. And while it’s just one of many handset vendors in the EU, Apple ships a large enough percentage of all handsets that landing an Apple deal can (and did) make or break an LTE modem vendor; so stopping Apple from looking outside Qualcomm would go a long way towards ensuring no other competition for Qualcomm cropped up.

Meanwhile, Apple’s cooperation with investigators has driven a large wedge between the two companies. Apple has been suing Qualcomm for another $1 billion in royalty rebates it says are still owed, and Qualcomm has been suing Apple for what they see as an unfounded global attack against the company. Apple has since begun multi-sourcing modems – starting with Intel’s XMM 7360 for the iPhone 7 in 2016 – so the European Commission’s case is more about punishing Qualcomm for past actions than it is about correcting any present market conditions.

Finally, while the Commission’s findings are not binding in other nations, this ruling sets the stage for what’s likely to be the most interesting of Qualcomm’s ongoing cases: the United States Federal Trade Commission. The US FTC has been investigating Qualcomm since the start of 2017 over the Apple deal and other aspects of Qualcomm’s business, so the fact that the Commission found enough evidence to fine Qualcomm indicates that the FTC could rule similarly on the same evidence. Never mind any other regulatory bodies out there who haven’t already begun investigating Qualcomm over the Apple deal. As a result this is likely not the last time we’ll see Qualcomm fined for their misdeeds with Apple.

Update: Qualcomm has issued a statement saying that they disagree with the Commission’s ruling, and that they will be appealing the fine to the General Court of the European Union.

“We are confident this agreement did not violate EU competition rules or adversely affect market competition or European consumers,” said Don Rosenberg, executive vice president and general counsel of Qualcomm. “We have a strong case for judicial review and we will immediately commence that process.”

Update 01/25: Qualcomm has also sent over a note reiterating that Apple broke the exclusivity agreement with the launch of the iPhone 7 in September of 2016. The agreement was set to expire 3 months later

The Samsung Exynos M3 - 6-wide Decode With 50%+ IPC Increase

The Samsung Exynos M3 – 6-wide Decode With 50%+ IPC Increase

With the public announcement of the Exynos 9810 having finally taken place, Samsung engineers are now free to release information on the new M3 CPU microarchitecture. We have a first look into the new microarchitecture and what kind of improvements w…