Is There Such a Thing as AI Ethics? Part 1

Tom Stoneman:

Hi, I’m Tom Stoneman, and this is The Intelligent Enterprise, where every two weeks we take a break from the chaos of enterprise life and get inside a big idea by getting outside of it. Each episode, we meet an industry expert who helps cut through the noise from all the updates and rollouts while exploring one of their favorite break time activities. It might be over a kayaking session, a long run, or simply a cup of coffee, something that gives them some head space when they’re deep in a problem.

When most people hear the word ethics in the context of AI, they think of regulation or academic papers. Today’s guest, Ugo Orsi, has a different view. Ugo is the chief customer officer at Digitate, and he spent 30 years implementing new technology inside large enterprises. He believes that the ethics of AI aren’t somewhere out there waiting to be written. They’re already inside your company. And whether you acknowledge them or not, the context being input into your LLM is shaping every decision you make.

Ugo Orsi:

Ethics for me is not a very remote term that belongs to academia or a philosophy class. Ethics is every day. It is the culture that the company has.

Tom Stoneman:

This is part one of our conversation with Ugo where we get into the three risks he believes every AI leader should have in their awareness. We’ll also discuss why the culture of instant results may be the biggest threat to good thinking and what Pope Leo has to say about all of this. Let’s get inside the future of enterprises by stepping outside of them. Discussing AI ethics, inside enterprises, it requires us to move past some of the philosophical debates and that kind of stuff. My view on AI is it’s not really AI, it’s a version of us. And so what we put in is kind of what comes out. And I’d like to understand your personal interest in AI and the ethics intersection.

Ugo Orsi:

So little bit to clarify, what we’re going to talk today is going to be about AI, how I live it every day, and how I live intersection with the ethics that AI must face and touch every day. Why I’m saying so is because if you want to succeed in driving adoption of AI, it’s pretty much my job, you cannot ignore ethics. It is the culture of a company. And if you wish with tribal culture that that company has, you need to understand very clear what matters for the people that are going to be your user of AI. So when you focus on AI adoption, you cannot ignore the impact you have on people. And if you don’t want to ignore the impact of people, you need to focus on what is important for the people, so the ethics. If we look in a broader context, the technology is an important part, but more important, it is the adoption that makes successful the adoption of the user of the technology.

Tom Stoneman:

It’s interesting, Ugo, hearing what you’re saying. On the way over here, actually, I was listening to a Stuart Russell from UC Berkeley, and he was talking about, he used the Wright Brothers, the bird versus the airplane thing. If they had invented a way to reproduce birds, they wouldn’t understand flight. So yes, we could fly, but we wouldn’t understand how flight works. And I thought that was pretty interesting because it puts the situation right now, it puts us at kind of that inflection point where we need to decide, do we understand this enough to be able to implement this widely? And do we know what’s going to be the output from that? And in that case, asking you as a leader, as someone who has to deliver results, what are your thoughts on being able to kind of hold those two things at the same time?

Ugo Orsi:

Very tricky question. First of all, I would like to frame AI and what’s AI represent for me. So if I look to AI, for sure it represents an opportunity to make our society progress. And I don’t want to go as far as say improving human condition, but to progress our society. On the other side, I am aware of the potential risks. So if we use AI as a generic terms, we need to consider a few things. First of all, that we are in a hyper-connected world where everybody are connected and it’s easier to be overwhelmed by information. AI has an embedded capability to generate information to a pace that was never seen before.

So risk number one, we need to be aware that AI has a potentiality to manipulate, if you wish, human being. The other thing is AI has a possibility to replace some of what we call it the white collar jobs and some of the blue collar job, but the high paid job. So AI has a possibility to disrupt the society as we know it. And last but not the least, AI has a risk to make, I’m going to use strong terms, us human more stupid. If you think about a cornerstone of, let’s say, the modern humanity, I think therefore I am. [foreign language 00:05:56]

Well, AI can prey on our lizard brain and it can stop us thinking, can stop our critical thinking. So these three risks when I drive adoption of AI are always very clear in my mind. Then we look to the context of what I do. I implement AI to optimize IT operation. I always deploy AI in a real context. That means a business company. And all the business company have two fundamental things. They have a set of roles or principle that are centering the human being. And the other one, it is a set of tribal culture. If you consider both of them, this is where I find the balance of deploying AI, but thinking also how it can create an impact to the future of that specific company.

Tom Stoneman:

There’s one thing you said that caught my ear. You mentioned that it has a tendency, I think I’m going to paraphrase, but to make us lazy?

Ugo Orsi:

Yes. I mean, I don’t know if you read any book about the concept of a lizard brain. Human being by nature are lazy. If you can find a shortcut, you will take it.

Tom Stoneman:

Yeah. I experience this at work quite often, and I imagine all of us do to some extent. Where it used to be, we would bring things up, we would go into a room, we would talk, we would discuss things, and that was that. Now there seems to be a feeling that AI is, at least ChatGPT or things that we’re asking questions of them, is better than us. And we know that it’s not correct all the time, but there’s more trust, I guess trust. And what I find is the very first thing when you bring something up is, “Well, let’s check Gemini. Let’s check ChatGPT. Let’s take a look at Claude.” And instead of, “Okay, well, let’s discuss this.” And I’m wondering what’s that doing to the thought process, especially inside companies?

Ugo Orsi:

So my thoughts it is, until or as far as the individual use it with what you just said, let’s check. It’s still good because you have critical thinking still in your side. Where you start becoming lazy and you might lose, if you wish, becoming more stupid is then when you give up the possibility of checking or confronting yourself or verifying what the AI solution tells you and you just delegate the whole activity to it. A stupid practical example, you need to provide an answer regarding a business problem. And instead of using AI for providing you additional information that then you can put it together with your critical thinking, you just cut and paste your reply.

Tom Stoneman:

Yeah. It’s funny you see this. I’m seeing it more almost exponentially. And I wonder how much this is starting to change the way people think. What is it? There’s a new thing. It’s a Reddit thing. It’s called Claude Boys. And it was a joke. I think it started out, it was a thing, well, let Claude run your life. And some kids started taking that seriously, and now it’s thousands of kids. They get up in the morning, they ask Claude, “What should I wear? What should I eat? What should I do next? What should I study?” And this and that. And all the way up to bigger decisions. And transferring that back to work, I just see that a lot of these things happening that are transforming the way people think. And so I’m worried about it. I guess my question is, are you worried about this?

Ugo Orsi:

I am worried, but not in the sense you are. Let me be very clear. To me, AI is a technology. What I’m worried is that parents are delegating a technology to grow up their kids. And I don’t want to be judgmental, but if you go back in times when television came in our houses and we are this generation, our parents and our grandparents will be a little bit surprised how much a lot of our education was delegated to television. So what you are describing for me is not a technology problem. It’s a societal problem. So instead of using the technology to make the kids more curious, think about it.

Today we are in a place that we never before. You have your world totally connected. Today, if you want, you can enter in all the library of all the world. And more important, thanks to AI, you can even translate books that before you couldn’t even access if you don’t speak the specific language. That is what the technology we have about. The risk is a technology when it is used for something that you’re just describing, ask what I have to wear. But it goes back to the company and the value system. What’s the value system of those kids? And is really AI or the technology at fault for that?

Tom Stoneman:

I see it as more of a temptation. So especially when you’re young, but any of us really.

Ugo Orsi:

Yeah, I agree with you. And temptation it is, if you think about it, that’s an interesting point because I was talking with some friends. When we were young, the bad temptation or the risks were outside the house. The home was a safe place. Today with this world digitally connected, is no longer. So I agree with you that AI and the connection can create higher number of temptation that they can become deviating forces. But again, then parents has double duty to make certain value even stronger because let me say, they might have some enemy within. It’s not enemy just outside.

Let’s go back to work because I’m not really an expert on family matters in the sense I am an engineer and what I apply AI more to a business, is an example that you said before. There is a culture and expectation of instant result. And that was there before AI. Now these colleagues are just using a technology to get it better. So if you wish, the culture we need to change is not I need it now. I need it correct.

Tom Stoneman:

Yeah, that’s it. That’s really it. Earlier this year, Pope Leo XIV released an encyclical titled Magnifica Humanitas. Magnificent Humanity. It’s a sweeping document about AI, what this technology means for society, and the responsibility of the people building and deploying it. Ugo and I both read this. For me, I kept thinking about the conversations we have on this show about context, adoption, and what happens when we stop thinking for ourselves. I wanted to hear an engineer’s take on this, someone who implements this technology every day with real people. In a world where machines can handle almost anything, where and how do we draw the line?

Ugo Orsi:

I think I said it already a few times. I’m an engineer, so I’m not a philosopher. Most probably you have guessed it from my accent, I am Italian, but I am no ciccio. In any case, let’s give a try. Humanitas translate into human nature, civilization and its kindness. Its creativity. And then humanitas represent we the people. And popular message is for sure to make people aware that this technology can be disruptive. It can change our humanity and especially overall our society. So when we implement it and we are developing, we always have to think about human happiness first, human satisfaction. So let’s make sure we do not put the human as a second choice because our first choice is fulfilling our egos. The fear of humanity to be taken over by a stronger entity and a single individual to be reduced to a single element, a repetitive element with no value add has been around for ages.

If people like music, I would suggest go and listen to a Pink Floyd, “All in all, you are just a brick in the wall.” This is a cry where society was creating so much control and people were becoming like little pieces and with no power. Now, AI has for sure as we discussed before, the capability of manipulating. It’s a manipulating technology, if you use it wrongly, that we have never seen before. You said another thing, it is AI might desire to take over. Desire of control is a human feeling. It’s not a machine. As of today, AI doesn’t have no feeling. So what you can see it is representation through very sophisticated algorithms of how you should behave, but AI doesn’t feel and doesn’t have desire. What I take from your point that is someone behind might have a vision of taking over the world or of a society and they can use AI to do so.

That’s a possibility that in some situation has been already realized as which is a small element. So this is risk. It is there. But if you wish, the desire of a control of a society of a few has been always there. Technology might give them a little bit of a bigger opportunity, but technology can be controlled. So if you go back to my day-to-day example, what I work and I see, you go into any bigger companies talking about North America, Europe. Usually I have my customers. All the company have got a strong value system. And even the people working there, on average, they are always motivated by good principles. So when you apply a technology, because at the end AI is just a technology, follow the principle first. Listen to them. And if you wish, I also saying it in an opportunistic way, because if you do not listen to people, it will be extremely difficult that you succeed with the adoption of AI.

Tom Stoneman:

You intrigued me when you talked about desire. And so something popped into my head here. You’re right. I overlaid a human emotion on a machine that, yeah, it doesn’t have that. But let’s do a what if. So what if, let’s just look at an HR example, let’s say. What if we’re using AI algorithms to determine which people we should hire, which people we should be putting into layoffs, things like this? Big impacts, big impacts on people’s lives. And all of a sudden in the algorithms, as it’s going through these things that we’re inputting, this is what we think it should do.

It makes a decision that, “You know what? It would be more efficient if I, the AI algorithm, takes this over and starts making these decisions. Because from a company perspective, the objective you plugged into me is be more efficient or be more profitable, whatever it is. Okay, to achieve that, then I’m going to go and instead of doing what you told me to do, I think it would be more efficient that we lay off this part of the engineering team and we bring in some different things here and there and making decisions like that.”

That’s I guess what I meant by having the desire to take over. It’s not really a desire, but more of a reaction to what we’ve asked it to do. Does that make sense?

Ugo Orsi:

Well, yes, I wouldn’t call it reaction. It means AI with this algorithm has been designed to optimize. And optimizing is providing a solution that it is a logical consequence of the algorithm that has been created for. But this is quite the point. Yeah, AI can decide that to lay off a certain set of people, but the ultimate decision as of now, it is with leadership. So the leadership can, first of all, check if this decision makes sense. But equally important, if we make it a little bit more generic, is this. If I am using a AI solution in order to select or optimize people or people actions, what is preventing that company to create an HR AI agent that is going to supervise the AI optimizer? And pretty much it is how it works today.

Today, if you think about it, how we do the hiring, the selection of the optimization is set of management that follow certain principle in order to achieve company goals. They make specific decision. And a good HR partner oversees that decision and makes sure they are aligned with the company value. And a good HR is ready, let me use a strong word, to go to a battle if there is any deviation. So my conclusion is yes, there is a possibility of building an AI that is optimizing greatly and not really considering the human consequence, but there is also a possibility to build an AI solution that can control and counter this recommendation.

Tom Stoneman:

Ugo’s take maybe a surprising one coming from an engineer. The solution to a technology problem isn’t actually with the technology. It’s a matter of defining your values and having them be part of your context. In part two, we get into what that looks like inside real companies with real stakes. And I ask him the question a lot of executives are sitting with right now. Should we pause AI deployment until we figure this out?

Ugo Orsi:

Command and control is gone. The challenge of AI adoption is not on the technology. It is to have leaders that are willing to be challenged by the team.

Tom Stoneman:

Thank you for listening to The Intelligent Enterprise, a podcast where we get inside big ideas by getting outside of them. I’ve been your host, Tom Stoneman. Please remember to follow the podcast and leave a comment or review wherever you get your shows. See you next time.