Q&A with IBM CEO Arvind Krishna
Q&A with IBM CEO Arvind Krishna
Roundtable Session at IBM Analyst Summit 2024
IBM’s value proposition can often come across in the consumer space as a big question mark - what are they doing today, and how did they hit $15 billion of revenue last quarter? IBM no longer has a consumer division, and instead focuses on several key areas - consulting, cloud, AI, and research. The consulting business drives what IBM does, and it’s built on the technology developed in the other sides of the business. When a company wants to drive business efficiency or upgrade its technology, it calls IBM.
Personally I’ve focused a lot on the silicon from IBM: the mainframe, the AI accelerators, and the research that goes into new semiconductors. We’ve just seen the launch of Telum II and the Spyre accelerator, but IBM is also working with Japanese company Rapidus on deploying its 2nm technology.
IBM uses these breakthrough technologies to power most of the top companies in the world. A big chunk of the top 100k companies are using IBM in some shape or form - whether it’s hardware, consulting, or software like Red Hat.
On the software side, IBM is driving home its AI capabilities. IBM has a series of language models called Granite, and recently announced Granite 3 - its next generation. Starting with a 30 billion parameter base, Granite 3 has been fine-tuned and optimized at different levels to run across IBM’s watsonx AI infrastructure for use by clients. IBM’s belief is that smaller, more optimized models, are the future of AI for both utility and cost, citing that they are customer zero in almost everything they deploy.
Several times a year IBM connects directly with the Analyst community to give an overview of the developments inside the business, unit by unit. At this event, CEO Arvind Krishna held a one-hour open Q&A session, covering everything from the adapting business model to the vision of watsonx and how IBM is addressing areas such as sustainability, talent, and company perception.
The following is a cleaned up transcript from the Q&A.
Q: Dave Vellante, The Cube Research - You have said that consulting is one of the foundational elements of IBM’s business, and we’ve heard about IBM going through the software-ization of that business. I know you haven’t given margin guidance, but what does that business look like long term? How are you thinking about the economics of that business?
Arvind: I have given guidance! Someone once said “software will eat the world”. Have you seen it happen though in infrastructure? Yes - infrastructure has been eaten by software, such as IaaS, Terraform, Amazon. You can deploy infrastructure through software. The change is the labour complexity that used to be there. On consulting, I think that consulting is good, and we have visibility for at least 10 years or more. I say that because as people do transformation, more digitalization, and are changing every enterprise process, and people want one brand, not one per location - to do all that takes a lot of heavy lifting, and not everyone has the skills in-house. Hence consulting has a long and positive runway.
When you do that, over time it will change and be digitized. You have got to save cost - the cognitive task has 3 buckets. The first bucket is to read stuff and summarise, and technology can do that for you. The second bucket, you go and talk to people and decide what needs to be done. The third bucket is to do the implementation. That might get more efficient, but it will not replace humans. The technology will make the human more productive. But overall that first third will be accelerated.
We have to work with clients and do the heavy lifting - it used to be 30 people and 6 months. Now it can be 6 people over a month, and we use AI to read the documentation. But on the model margin question - I think that consulting should be mid-teens margin. Not 30%. We're 4-5 months from that. We're 10% margin now, and it hasn't budged a whole lot, but if we can get to 13-14%, that's good, and we have line of sight.
Q: Tony Baer, dbinsight - We’ve heard a lot today about hybrid AI and automation - what has been absent is watsonx - it has only been mentioned in passing. Is watsonx still as strategic to IBM as all the other initiatives? What's the trend of investment? Where do you need to innovate?
A: watsonx is extremely strategic to us. It's two things - a specific product first, but it's also a brand family name of what IBM does on AI and specifically on generative AI. It's both. Our branding isn’t ‘Code Assistant for Z’ - it's the ‘watsonx Code Assistant for Z’. It’s the same with [our assistants for] Ansible.
Our mainframe team isn’t here today but soon will come and talk about capabilities on making system administration easier with watsonx. We’re just in the process of rolling out our code assistants for C and Java internally first. But we're already handling 30 million calls a month on watsonx assistants for customer service with clients. That's how strategic it is. As we go forward to roll out our LLM models, our Granite models, they'll go through redhat. We wanted an easier entry point too, so RHEL-AI is also watsonx. Some people might not want the complete solution, they just want to play with the model through RHEL.
Q: Where does it come into play with watsonx.governance, watsonx.data? It is supporting all these services, but what about the other parts of watsonx?
A: Governance to me gets a lot of airtime, and a lot of people want to talk about it. But it's not been much of a buying gavel. [We'll keep plugging], we’ll keep investing there - we’re not going to decrease investment. As more regulated industries are moving up with AI, and regulation will need to talk about what is the data, how do you fine-tune it, testing - all those questions will come up. But might be a year or two from becoming real questions. Maybe it's axiomatic - AI getting deployed at scale will make those questions real. People are kicking the tyres right now, and that’s not a buying gavel, but we’re sticking with it. Getting data ready for AI is absolutely critical - all of those are coming under the watsonx family name. If you give us two weeks [for an analyst event], we can cover all those in depth!
Q: Arun Chandrasekaran, Gartner - How is IBM using AI internally? Which business functions are you deploying AI, what have you learned, and can you share metrics?
A: Over the last 24 months, this is a mixture of automation and AI. You can't tease them apart. The public number is that we’ve had $1.6 billion of efficiencies. That's a $15 billion baseline of expenses, and how much are we spending now. It’s a real number, and it's on that base. Think of that as back office functions and third-party services. We have a line of sight to $3 billion by the end of 2025. That's what we've stated on earning calls. Out of $1.6bn, you can attribute about half directly to AI. For example, if I look at our 1-800-IBM-SERV (IBM’s support hotline), about 75% of all queries can now be answered by Self-Serve, which wouldn't be able to be done without AI. That’s a complete five year journey, not just a one year journey. The AI was there because we took our internal answer engine, reading through the documentation is a nightmare - AI can do it and give a great answer. If I look at our HR chatbot - over 600 people's worth of work taken out. This isn't high-end HR, it's low-end HR.
Here’s an EVL example - someone would ask about employment verification data. They'd call the HR manager to get it. The HR manager would look up the internal HR system, verify the information is correct, and then ask questions to verify. That's lots of human time and touchpoints, 30 minutes or so. The team asked if they're on the intranet, do we need to verify? Why can't I just write a bot to do it? The only question that needs to be asked should be “Where do you want it sent?”. It's not like customer service at scale, but there are 100s of these things that can reduce 600 people's worth of work out of the business. It's tangible. It's in procurement, invoicing, and accounts payable. We can go on and on. It applies to all industries on the planet.
Q: HFS Research - I really enjoyed the asset-based consulting talk yesterday, it aligns well with our vision. The one thing I’d like to understand from you is where do you think the BPO (business process outsourcing) and managed services fit in? It was a bit silent. The big four are betting big on management services right now. What is your reaction to them?
A: BPO and managed services are two different things. An asset-based approach also has a fundamental advantage to classic BPO. BPO has done well on location-based arbitrage, and on a much tougher disciplined process. Now ask the question - why do I need 1200 people doing accounts payable? That's not a made up number - it’s a number from one of our clients. People don't want to take the risk on unstructured invoices - humans can read, and refer to the original contract. Once you can do all that, then you agree to pay.
What does that really need? You need to read across multiple systems - there is no return on investment on bringing the data together in a structured form. If you could do that, you could automate it! If the formats keep changing, it's a continuous task. But if you can read, you can do that matching. So how many can you read at a high confidence level? We can make 50-60% now, maybe up to 90% as you learn more and more. If someone is of high risk, e.g. making fraudulent invoices, it's easy for AI to learn and keep a list. But also people make mistakes - you give those to humans. I think that over 3-5 years. things like that could probably become 80/20 - so 80% automated. That might not be everyone - more complicated ones take more time.
We have an approach to BPO. We always make ourselves client zero before rolling it out. But we're making headway. For managed services, most of what we do is very bespoke. But if you have standard ones, like SFP and Salesforce as managed services, there might be a big take-up there.
Q: Bob Evans, Cloud Wars - Your technological innovation over the last 2/3 years is amazing, but there’s also been an innovation in your go-to-market strategy. Not very long ago, you competed with everybody - now you have $7b partnerships. How did that happen and where is that going to go from here?
A: On the partnership strategy, it's quite simple. To me, you can look at someone else that's growing and highly desired by clients - you can either go compete with them, or you can add value to that partner with clients, and that is a better strategy to follow. If I look back at ourselves, massive successes with IBM has always been through partnering and integration. For example if you look at the early days of mainframe and software, middleware to integrate the bulk. So to me, it's coming back to its roots - you can't have the arrogance to compete on all fronts. Let's get better through partners - that has led to those 4 out of 7 being at $1 billion, the other three could hit a billion. They all go at a different rate of pace. Out of those four, only one of those was at $1bn three years ago, and the other three have been quick.
On the go-to market, I believe that - if you want to go buy something, you research it. Maybe talk to the company, or online reviews, YouTube, TikTok, or you read. With a car, you rarely listen to the dealer. The same is true for our clients. But what they don't know is how easy it is to deploy, so go-to-market has to shift to being more experiential rather than transactional. If you make it experiential, people walk away with a great experience, and it builds trust, and then you get repeat business. That's the single biggest change across consulting and technology. Technology has a large investment in CSMs for clients. There is really great progress in the top 300 clients, then we keep bringing it further down into the market.
Q: Constellation Research - What three products that in 2025 are going to have the biggest percentage share in revenue? Will Quantum be there?
A: Quantum will be, but not in 2-3 years. Quantum as being useful is where GPUs for non-gaming uses were in 2010. In 2020-2021 people woke up to them being AI chips - we knew about that in 2015, but the world took a little while to accept that. If we know about quantum utility, then it takes a couple of years. From there, the percentage growth there is infinite.
If I look right now, Openshift is probably one of the higher percentage growth rates, with almost 50% CAGR since acquiring Red Hat. Openshift was $100-$150m at acquisition, now it is well north of $1bn or $2bn - that’s 10x in 5 years. Ansible is at a bigger growth rate. watsonx has had extremely good traction in revenue growth, and that will move well into the $100 millions. Apptio and Turbonomic have had great growth rates. Inside the mainframe, the watsonx code assistant for Z is off the charts.
Q: Gardner - The stock is performing well. There is steady progress in acquisitions - what is your current weight basis for further growth?
A: If you can wave a magic wand and change the regulatory environment, I would have an infinite appetite. But that's not unique to us, everyone is going through that. A lot of acquisitions can take a lot of time and effort, and time tends to decrease the value of deals. That's a big dampener on the whole M&A area. I don't want to start a new lane - I doubt we'd be allowed to acquire anything in mainframe space, hardware or software. It's in our lane, but we’re parked on the side. On automation, we've gone after Turbonomic, Apptio, and Hashicorp. There are more things like that that fit, but as a platform strategy. They all fit into what I call AI OPs. I want to do all of these things - observability, resource management, fin ops, deployment - and because you can do it ‘closed loop’. To close the loop you have to do it all, autonomous redeployment for resilience, for performance, for cost. We have some missing parts, so we could do stuff there. In the whole data space there are some attractive properties - we've gone after a few small ones but nothing very sizable. It has to fit the strategy and have adequate economic returns for the shareholders. If they're baked into the price, perhaps they’re not great for us to acquire.
Q: Andy Thurai, Constellation Research - All of the platforms you have now, it's still more towards augmenting human intelligence. No messaging to autonomy or semi-autonomy. Is it because you’re not trying that out? What's the concern and plan going forward?
A: We are one of the only ones with sufficient scar tissue from being beaten by the market for trying to do autonomous AI systems decades ago and customers didn't want it. They didn't want black boxes, they wanted it explained and to be able to tinker. New technology is not always quite ready to embrace it at that level. I think that as people get comfortable with it. The fact that 75% of all our current customer queries are answered by AI, that's all low-risk level. Our Healthcare insurance client, the second largest in the US, is now doing 46% of all phone calls and chats with an autonomous AI. That might get to 70% - I doubt it'll go above 70%.
It’s more because the market is not ready for it than we cannot do it. There is media hype over job displacement. But there is an additional point - when people talk about AGI, right now I don't see how we get there from here with the current state of deployed technology. I just don't see it. These systems which we love are all inherently probabilistic at the root. They learn from trillions of tokens, but at the heart, at the end of day, they're a probabilistic system. Very powerful, and very good at predictions. But it doesn't contain hard knowledge. How do we combine that with these systems? I’m not even sure it's sufficient, but it's a necessary first step to get to those kinds of systems and even superintelligence. Take Newton's laws of physics, it can't be probabilistic, it has to be there. When people begin to talk about that, not just scaling laws, I think we need to be realistic. That's one intuition to get to those types of systems.
Q: Kate Halterhoff, Redmonk - On sustainability. I'm interested in how you're pivoting that mission in the AI era. We’ve talked a lot about these cool models, but of course the energy they consume is concerning. What are you doing to mitigate the ecological harms of our AI moment?
A: How much time do you have!
The current generation of GPUs are incredible. But were they designed for AI workloads? Is that why someone invented them? No, but it turns out they can do AI, but they're not necessarily great for AI workloads. I personally believe that in less than 5 years, or maybe less than 3, we will see silicon that can do the same quantity of AI that we do today's GPUs at 1% of the power. Maybe the incumbents will do that, but that's the real correct long-term answer.
Now, why do you need a really large model? Suppose I have no idea what I want to do, like upgrading a consumer chatbot that has wide domain training. If I have no idea what question I’m going to ask it, it has to be able to do everything. So it's 300b, going onto 1 trillion parameters. But if I know I want to summarize business documents in English into something I can read, maybe that's only ten billion parameters. 10b to 300b is a 1000x difference in the amount of compute it needs. So when we say people should use 10-30b parameter models, it's going to save you 90% of the power. But it also plays to a huge business paradigm - the purpose of being sustainable and profitable are linked. Sometimes people forget that.
If you're in a country that doesn’t want to use a big public model, then you need a small model for a $100m datacenter, not a $1b datacenter. It also opens up the aperture of what more you can do. It has to be as good as the big ones on specific tasks, but it doesn't have to do everything. It also makes fine-tuning easier. Can we make something that does AI at any scale but at 1%? Yes. We're not all the way there, we're middle of the way there. Also, can we make models that are as good that don't need as much energy? Absolutely.
We're doing a lot of work, with NASA on a climate change model, to predict things that are going to happen due to climate change. We worked with the UAE - it turns out that 50C is the limit of what humans can live - hotter than sous vide, you're effectively being cooked. We worked with Abu Dhabi government for finding urban heat spots with AI. If it crosses 50, what do you do? Perhaps you switch off the industries that generate a lot of heat. But that's purely us giving back to the community, we’re not charging for it.
Q: Ted Schadler, Forrester Research - Regarding models. I hear about the opportunity to go enterprise-specific proprietary domain models. There could be 100k in a single account, and someone has to build, maintain, operate, and manage them end-to-end. Model operations are becoming an important layer in the stack. Discuss.
A: It's not just model operations. The way it's done today is all vertical - software to model to GPU to network to CPU. That's fragile - we all learned this lesson in this industry a while ago. Three years from now, it'll break due to a backwards compatibility issue.
We have to start at the basics - the stack should isolate peak performance from the bottom-most layer. Write a layer above it to take advantage - there's an example, Linux. Linux doesn’t care what people run it on. Linux isolates it and gives 90-95% even if you went to the raw hardware. We have to do that in AI. Then people will write models all over the place to deploy, and then move through the next layer up. The team will get there.
We're working on these things - 95% of the bottom layer by not being vertical is reasonably accurate. Now I agree on the 100k models deployed - but do you need 100k base models, or are they in the 10-100 range? Then we have to add a skill, or tiering, and how to do that in layers. If I take it and mix it, like a chemical equation with a new molecule, it creates a problem. But if I can separate some layers, that's why we're investing into a technology called “Instruct Lab” to be able to make it into layers. Now with the better base model, you can quickly go to market. The end value is from fine-tuning their data for their industry.
Q: Mirabel Lopez, Lopez Research - There’s an interesting intersection of AI and security - but not a lot to hear about that today in the portfolio. How are you thinking about growth and opportunities?
A: If you look at the amount of data being collected, a single company can generate petabytes of data per day. It didn't used to be that much, not because it wasn’t there, but because they had no idea how to process and use it. Our partners in Palo Alto Networks collect petabytes of data for a single customer and exabytes for a vertical set of clients. For pattern matching or ML, basic AI isn't there. But if I now want to combine data from there with application data, that's only going to be possible with LLMs. We're working on it, but it's not yet out there.
We’re also looking at LLMs as the only way for anomaly detection to happen - bad guys will get cleverer. Pattern matching is great, it's a first filter, a layer of defence, but it's only a layer. Anomalous behaviour detection is a better way of doing things - but how do you do it? Using an AI approach is a better answer for those things, but over time, I would like to include some non-security data to non-anomalous datasets.
Q: Sanjiv - IBM has depth in research and products. But when talking to peers and the market, there's a perception that IBM is a legacy company/brand. How are you changing that dynamic? What marketing or approaches can you take to break that perception?
A: Amongst clients who purchase with us above a certain size, they don't have this impression. For those who don't know us, yes they may have that impression. We know what the issue is - five years ago, we made progress with those who knew us. Some of the issues are always going to be there because we don't have a consumer product.
We tried with The Weather Company (acquired 2016, sold 2024), but couldn't connect the two things. It had 100m users (up to 800m users), as it was the tech inside Apple and Samsung’s default weather apps, but consumers didn't care. We haven’t figured out how to grab this consumer side with it. We tried with PC games and the PC. If we can succeed in consumer, it will give us implicit recognition and earned media.
So how do we get to these others? For IBM, the world has changed. Meta, and Google, all do great deep technology. How much progress have they made on metaverse or goggles? Some of this isn't driven by marketing, it's driven by adoption. So we are investing in Granite, RHEL AI, and focus technologies like Terraform and Ansible - we have to go big into the developer and operator community and help them adopt them, and not worry about money. That will be the way through this. People need to know us and get to know our technical capability. A media campaign I think will not work for us. Others who have succeeded with media campaigns also have consumer products to attach to.
But it's not as dire for IBM as it sounds - we have 30k clients we can name, and 10-20k we can't name, but we've made progress. Out of the top 100k clients in the world, we'd like them to know us and what we’re capable of.
Q: Chrissy Healey, Gartner - How are you rethinking your talent agenda and culture given your business model changes?
A: On culture, these are not unique observations by us, but often when businesses don't do well, like 10 years ago, people get insular, dividing an ever-shrinking pie, and they become risk-averse. If you were risk-averse, and if we take the code assistant for Z, the culture holds to request more time to work on it and underplay the capability in case it doesn't work. If you're risk-embracing, you’ll say you can take 3 months with two dozen people then you realise you need more people you’ll ask for more, promise the moon, and you keep working. The Goldilocks zone is somewhere in the middle. You can take away 50-70% of possible productivity being risk-averse. So you need to embrace with 50-70% confidence, and it gives them a chance to fail but helps drive innovation and confidence. For example, in sales and consulting, being risk-averse is too rigid, and you end up with the highest price, rather than leaning in and partnering with risk to help drive down costs with some unknowns.
On talent, I think we have really smart, really capable people at IBM. We have to watch if they're willing to come along with this journey, embracing risk and growth. I think 80% of the people probably are, and the other 20% perhaps might need to be replaced - people have to be fit for purpose. Last year we hired ~30k people. We are clearly on that path. That's why we insisted that anyone who is a leader of people has to be back in the office three days a week. People learn culture by observing others. Not mandating it for everyone - most employees might think if managers are in, they should come in. I think they're going to make a difference in culture and talent.
Q: Dan Newman, Futurum - IBM has an incredible heritage in silicon, and not always getting the notoriety it deserves. We’ve seen this rapid pace of innovation in the market driven by annual cycles - what does that enable IBM to do? How do you see this pace and rate of change putting pressure on continued development with this model leapfrogging in AI? How does that change IBM research?
A: A lot of the pace of LLM model development is using brute force. Before LLMs came around, there was a lot of labelling and bespoke-ness - we went at a human pace. But LLMs turned it to machine pace. The machine taught itself. Now models are teaching newer models, models teaching models. That's why the paranoia that people have is around more compute and more data centers. That is what is driving that. I think that, like the mid-90s, many of these will survive and be spectacularly successful, but the vast majority will spectacularly fail. But the winners will outweigh the failures.
We do not want to be a silicon foundry, we’re not making a fab. We enable others such as Rapidus on their 2nm fab. That's going to be a core business. We'll enable that, and hopefully that'll help us and work with us just like we do with Samsung. We will design chips, like the mainframe, with a small AI core to do some AI right inside the processor. But for LLMs, more area was needed, so we made the Spyre accelerator and put 192 in a system. We will keep going down that path for what we do. We will ask the research team to work on making things 100x better - but that's a computer architecture breakthrough. If you have a really sparse matrix, and doing computations, if you take a GPU approach you’re not taking advantage of sparsity. You get the advantages of parallelism, but not what needs to move forward. It's all dataflow. The best will be power efficient but also have to showcase performance. That's the art of the possible. But also, combining quantum and LLMs to come up with something in a much more efficient way for the base case of LLMs. Is it possible? Yes. Do we know how to do it? Maybe not quite yet.
Q: When talking with other CEOs, what are a couple things that stand out in those conversations that excite or concern you?
A: What excites me? Nicolas Carr wrote an article called ‘IT Doesn't Matter’ - it was about as wrong as someone could predict!
What excites me is that there isn’t a CEO that I’ve met in the last five years for whom technology is not a top priority in their agenda. Airlines, banks, medical, retailers. Walmart probably has 30-40k employees under the CIO. That excites me - it means we're at the heart. When I go visit countries and talk to governments - I tell them they all obsess over banking. You want to have your own banking system, you want to have your own payment system, you want it all - but technology is 10x more important to the country's success. It’s what gets you jobs - technology. What do people want - technology. What will the companies be competitive on - technology. The attention people focus on finance vs technology is crazy. It tells me there's a long tailwind in this industry.
What scares me the most - the China/US decoupling on technology is the most worrisome. It forces people to make choices. Two parallel systems won't be able to interoperate. It means the scale is smaller, and there will be a group of people wedded to that. It's almost like the Cold War - like an Iron Curtain. In the developing market, you have to work with one and the other gets annoyed with you. That does worry me, the single biggest worry.
Final Comments from Arvind
One of the analysts here said that IBM is not known to be innovative. The conversation I have with at least some of our existing clients, and we do grow them at the rate of a few thousand a year, has become very much about how we’re a part of their future, and how they want us to help them. I was talking to a very large financial client a few minutes ago, and he said he had a very different perspective on IBM compared to five years ago. They want to do more AI with us, they want to do more systems with us, they want to do more migrations with us.
Customers are saying this all on their own, because they can see what we're doing.
Unlike some others, we don't hold people hostage. If they want to decrease their engagement, we don't like it, but we don't then increase the price. I think that there is a lot of opportunity when I look at just the portfolio we have, and I think that there is so much opportunity that we have to go get, which is why the culture of the go-to-market is so important. That is one piece.
But you have to keep innovating. This is an industry where things move fast. You have to innovate to be ready for the future. That's what excites me. I'm excited about our portfolio today. I'm excited about what you're going to hear about us down the road.