Could HR’s high-tech systems go rogue?

Could HR’s high-tech systems go rogue?

Could HR’s high-tech systems go rogue? Many workers – HR professionals among them – fear AI could take away their jobs. But could those same state-of-the-art systems have the potential to go rogue, and ruin companies?

HRD spoke to Roy Pereira, founder of automated office assistant, about whether humans have anything to fear from the rise of tech, including – only somewhat tongue in cheek – whether AI could become intelligent enough to start a robot war.

Pereira is one of few AI industry insiders to openly admit that some human roles may cease to exist due to automation.

“If they do exist, they’ll probably be working on slightly different things,” he says – and he’s not just referring to lower level administrative roles.

“When you think about what computers are really good for, they’re really good at analyzing data – and especially when you add data analysis functionality to AI, you get some really great benefits.

“When you look at positions that actually do that today, like data analysts within companies – those are not considered low level, and those will probably also be heavily automated. That’s something that we don’t normally talk about that we’re now seeing as well.”

To Pereira, AI’s growth in power and flexibility is both “exciting and frightening”.

“It’s exciting because it is growing so fast. We are definitely going to get to a point where we don’t understand how or why the AI came up with that, but we’re really super happy that it did. That’s both exciting and frightening.”

Does that mean AI systems could go rogue, to the point of destroying a company?

No, is Pereira’s answer: AI can only do what humans program it to do.

He makes an example of Microsoft’s machine learning Twitter bot, Tay, which was programmed to have a “repeat after me” feature – and was quickly coaxed by internet users into tweeting racist, pro-Trump, Holocaust-denying statements.

“The AI that it was using was interesting, but what actually led to its demise wasn’t really its AI, it was just its rule structure – in the end, it was trying to entertain us, and one of its rules was ‘I’m going to repeat what someone tells me to repeat’, which is a pretty silly game. Anything of intelligence is going to understand how not to do that improperly. The problem was that wasn’t taught to the bot,” Pereira says.

“That’s one specific example, that’s really basic and it’s easily fixed … but overall, AI can’t get out of control, because again humans are programming it.”

Yet, he adds, “We may not understand all of the ramifications, all of the tasks that a system can get into, especially when AI is so much more powerful that it’s coming up with these predictions that we never really thought of.”

So, while a AI vs humans war is unlikely, Pereira says we’ll continue to see AI’s capabilities advance, whether that’s analyzing data to predict outcomes, or replacing roles entirely – a difficult issue for HR to consider.

“That’s a harder question because obviously if you’re a unionized shop, you may have some constraints, or you just may have some decisions to make that you’re going to feel uncomfortable with, so you don’t want to go there, or if it’s about cost savings, or if it’s going to be faster, then you’re going to have some real decisions to make. Those are not just business decisions, but they’re also ethical decisions to some people.”

Either way, HR professionals have little choice to embrace AI, as it’ll be a part of every piece of software in years to come.

“They’re not going to have a chance to say yes or no, because the latest update of whatever software that they’re using will probably have some kind of AI in it, and that software will become 10 times better, or 10 times cheaper, or predict 10 times more successfully.

“We’re going to see that in the next several years – just as we saw mobile apps coming in, did you have a choice to say no? No. It’s the way technology runs.”

Roy will be participating in the AI panel on November 14th at HRD's upcoming HR Leaders Summit in Toronto. You can check out the full conference agenda at

Related stories:
Will your company’s next assistant be a robot?
Will this stop your top talent from leaving?