If you total up the energy spent debating the merits and demerits of the Apple iPhone X event and various devices announced today, odds are that you could actually power another keynote, one where the basic question is why: why does the iPhone X matter? Why it’s even possible and where it could lead us – and why Apple is best positioned to lead us there. The answer is as obvious as what you would expect Jony Ive to say in one of the promotional videos for a new device: the ability to marry design, user experience, software and hardware. And the silicon. Every single event leaves me more in awe of Apple’s semiconductor group – and what they seem to achieve with each passing year.
Today was no different. Apple’s ability to design custom silicon — its new A11 Bionic processing unit, a new GPU, new image processing chips, S3 watch processor and low power wireless chips (W2), True Depth Camera System — is at the heart of every new product launched today. The $349 phone-making, music streaming Apple Watch (version 3) needed a low power WiFi and Bluetooth connectivity chip. No problem: W2 is reporting for the duty. (Wi-Fi is 85 percent faster, while Bluetooth and Wi-Fi are 50 percent more power efficient.)
Small LTE antennas, the tiny circuitry to make its electronic SIM work, is handiwork of chip mavericks. The watch actually lives up to the hype, thanks to the S3, Apple’s third-generation architecture featuring a faster dual-core processor – faster app launch times, smoother graphics and even Siri works better.
The reason Face ID works is because of some key silicon innovations — yes, there is that TrueDepth camera system made up of a dot projector, infrared camera and flood illuminator and a seven megapixel camera. Face ID projects more than 30,000 invisible IR dots.
The resulting IR image and dot pattern is then used to create a mathematical model of your face and send the data to the secure enclave to confirm a match, while adapting to physical changes in appearance over time. What decodes the data captured by this camera (for lack of a better descriptor) are neural capabilities of its A11 Bionic chip. I saw this first hand and was blown away by the effectiveness of Face ID.
**
The FaceID is a perfect illustration of Apple’s not so secret “secret sauce” — a perfect symbiosis of silicon, physical hardware, software, and designing for delight. Their abilities to turn complex technologies into a magical moment is predicated on this harmonious marriage of needs.
I started following, writing and now investing in technology nearly a quarter century ago — watching giants like Microsoft, Apple and Intel gain heft and clout in the industry. These companies were the key players in the world of computers — much more than demand creators such as IBM, Compaq and Dell.
Microsoft and Intel — WinTel to us old timers were attached at the hip. Microsoft released new versions of Windows. Intel made ever new powerful processors that made using Windows easy as a breeze. And that in turn prompted Windows world to figure out how to use up that extra oomph that Intel was sending their way. This created a flywheel of innovation, demand and ultimately wealth that turned Microsoft into the behemoth it is today. Intel and Microsoft sucked most of the profits from the PC industry, leaving enough scraps for their feeder army – the PC makers.
While Intel and Microsoft were creating this vortex sucking in the profits from the PC ecosystem, Apple having bet on Motorola’s chips to power its clearly superior operating system and better industrial design, was losing ground. Sure there was the internal management turmoil, but let’s not forget that Motorola couldn’t compete with Intel and its money gusher and marketing machine. I don’t know if old time reporters would agree with me, to me, weaker Motorola only exacerbated Apple’s problems and reduced it to a bit player.
The iPhone represented a fresh start for the company — and Steve Jobs, had learned his lessons well. Don’t depend on a third party to be enabler of your key innovations and capabilities. I have written about the critical need for vertical integration for today’s giants in the past.
Just imagine Apple having to depend on Qualcomm to supply its chips — it will be tied into Qualcomm’s ability to come up with new technology — and thus will be working on a timeline defined by the San Diego chip giant. Mind you, this doesn’t factor the harsh reality of paying Qualcomm premiums and having to worry about losing chip supplies to someone with bigger orders and desire to work on razor thin margins.
Qualcomm had announced a system similar to Apple’s True Depth camera in August, except it isn’t available just yet and it isn’t clear when they came to market. Qualcomm executives told the New York Times that bright sunshine, hats and other such artifacts can cause problems with facial recognition, but as Apple has no such plans because it spent a vast amount of resources to develop alogrithms to overcome such challenges. This puts Qualcomm’s customer at a disadvantage over Apple, which can make hay while the sunshines and be ready for more wizardry when the next version of the device rolls out of its factories.
Controlling your own destiny is a smart business strategy and Apple isn’t the only one doing it. Today Google and Facebook are designing and producing their own highly optimized hardware for networking and data centers, mostly because the industry vendors like Cisco Systems made gear that had to fit the needs of many companies. Google is designing its own chips — especially to conduct resource hungry machine learning and artificial intelligence tasks.
At a recent Wired Business conference, veteran Google technologist Urs Holzele pointed out that when they were thinking about introducing voice-control system on its Android OS, they would need twice as much compute infrastructure as the company owned if every Android user used it for three minutes a day. “Even for Google that is not something you can afford, because Android is free, Android speech recognition is free, and you want to keep it free, and you can’t double your infrastructure to do that,” Hölzle told the conference attendees. Instead the company created specialized machine learning chips. Apple is doing the same for it’s FaceID.
**
The phone business is a brutal and competitive business. There was a time when Motorola was king of the phone business — thanks to its Startec and later Razr phone. It got complacent, missed a step or two and ceded its mobile leadership to the Koreans: Samsung and LG. Nokia dismissed touch and iPhone as a joke. In the end, it was Cupertino who had the last laugh.
The phone industry is unforgiving — HTC might have been the first with an Android phone — today, it is looking for a bailout from the mandarins of Mountain View, California. The half-life of a modern handset maker can feel shorter than that of a house fly and perhaps that is why Apple’s smartphone rivals – Huawei in particular is going for vertical integration — and trying to take control of their fortunes, and extend their longevity.
And that brings me back to Apple and FaceID. The reason FaceID works as smoothly as it did in my hands on is because the camera and its wizardry are married to the ability of the A11 chip to use neural networking technology to utilize the facial recognition algorithms and then seamlessly hide them inside the operating system and the device design. One doesn’t need to know any of that— as long as it feels like magic.
It all starts with Silicon. Unlike software which can be written, discarded and rewritten at a rapid clip, the law of reality makes it hard for a chip to be designed, tried and manufactured at scale. So in a sense, Apple’s chip and hardware teams have to peer almost two-to-four years into the future, predict what could be possible, what they can make possible and then make it work.
A11’s neural capabilities might be utilized for FaceID today, but it wouldn’t surprise me to see a version of this technology identifying foods we are eating, tagging them and storing them in a HealthKit database (measuring our caloric intake). Or a future version enabling unique augmented reality experiences to go hand in hand with other technologies.
**
Apple says nearly 500 million people walk through its 400+ retail stores around the world. In a manner of speaking, the Face ID and Watch calling needs to work anytime any one of these million people picks up the iPhone X or Apple Watch for the first time and tries it out or takes it home. Unlike the Mac faithful, the real world is more brutal and unforgiving. Apple needs every bit to make sure that consumers don’t go elsewhere. And it all starts with a chip.