Dr. Rana El Kaliouby Talks About Smart Eye Huge Influence on Automotive Industry

Besides being the Deputy CEO at Smart Eye, Dr. Rana is one of the pioneers of Emotional AI. After all, she is one of the founders of Affectivacreated, a new technological category of Artificial Emotional Intelligence. If you don't understand the term Emotional AI, it refers to technologies that use affective computing and artificial intelligence methods to interact and learn about human emotional life. Meanwhile, Smart Eye is the global leader of Human Insight AI and a leading developer of AI-based Driver Monitoring Systems in the automotive industry.

The interview with her never felled like a basic, boring one, but more like a free lesson from one of the most intelligent people in the engineering industry. A lucky chance for me because she is very busy after the already known AutoSens event and the inaugural InCabin event in Brussels held on September 15, 2022.

What is SmartEye, but in simpler terms?

I tried explaining to my friends what SmartEye is in simpler terms so everybody could understand. Unfortunately, I failed big time, so I asked Dr. Rana to do that for me. "So here at Smart Eye, the main product is driver monitoring, so we are able to understand if a driver is distracted, for example, if a driver is texting and driving or if they are tired and falling asleep. We can detect distraction and drowsiness, which is all done by a car camera focused on the driver," said Rana. However, that is only the core technology because now she and the Smart Eye team are adding more stuff to improve AI.

"We now have a camera that is focused on the entire cabin of the car so that it can see the driver and the other passengers. With the combination of our technologies (Affectiva and Smart Eye), we can do things like detection. For example, we can detect how many people are in the car, if there are any kids, and if anybody has their seatbelts on by using computer vision."

Besides, the system can detect if children and pets left in the car are alright. Unfortunately, some kids got forgotten in the car, and bad things happened. "Now we can detect if kids are left behind in a vehicle, and we can alert an adult. We can see if pets are alright as well", said Dr. Rana el Kaliouby. However, the system is even more advanced than you think.

"There are other applications as well. We can detect the mood and emotion of the passengers and drivers in the vehicle. As a result, we can personalize things like lightning conditions, music, content, and even the smell of the car." Smart Eye unlocks a lot of applications in safety, comfort, and entertainment.

Smart Eye could potentially save your life in so many different ways.

After Dr. Rana told me about these impressive technological achievements in Emotional AI, I wondered if things could go even further. "One of the main trends right now, which I talked about in Brussels, is the intersection of the automotive and health and wellness industries. Think of your car as your next wellness hub. It can measure things like your heart rate, breathing rate, stress, and frustration level."

While this was super impressive, things got even better. "The system can see even things like sudden illness or nausea. And because it can detect these things, it can adapt by changing the environment in the vehicle, getting you help, and intervening to keep you safe." Dr. Rana believes this will be the next wave of innovation we will see in the automotive industry.

Many companies have tried to implement driving monitoring systems, but Smart Eye is by far the most advanced one and the best at what it does. But why is that? What is so different about this company? Dr. Rana gave me an interesting answer.

"Even though at the surface it seems like an easy problem, it's actually really hard because you have to be able to detect all the things I told you about on all types of humans. From young people to older people, males or females with different ethnicities, backgrounds, skin colors, and many more. People will have masks or beards, they will wear sunglasses, and we need to detect everything. Suddenly, it becomes a very challenging problem because the amount of variation is so big. When she put that into perspective, I realized how far we'd come thanks to people like Dr. Rana.

"It's a very hard machine learning problem, so I would say one of our advantages is that we have invested a lot. Now, we have our own data labeling team and over 13 million faces responses from people driving around in their cars, and we collect all this information. Of course, with their consent."

Besides real-life data, Smart Eye collects pieces of information from simulations as well. As a result, the data part represents a massive advantage for them over competitors. Still, more aspects boosted Smart Eye to the top.

"Our machine learning approach is also a competitive advantage because we work on almost any type of camera and processor. Plus, we know how to bring technology and excellent solutions to the automotive world. Building an algorithm that runs on your computer is one thing, but having run in the car while respecting all the regulations and requirements of the industry is not easy. I say these are our advantages over the competitors."

Origins and a short biography

Let's move out and relax for some moments with Dr. Rana el Kaliouby briefly describing her inspiring life story. "I am originally from Egypt, did my Ph.D. at Cambridge University, and basically pioneered the field of Artificial Emotional Intelligence. I was one of the very first people who built algorithms that detect people's facial expressions. And, you know, that was more than 20 years ago when we didn't have smartphones. But still, I was able to create this system."

She joined the MIT Media Lab as a research scientist, and her goal was to become an academic, but she soon changed her plans when she and her colleagues recognized the product's commercial potential. "We started Affectiva in 2009; to put it in words, it was like an emotional roller coaster. Starting the company from scratch, we raised the venture capital and made the product popular. Basically, a lot of work to market it," said Dr. Rana.

As you see, the marketing was quite good because the technology is used in 90 countries worldwide. We definitely have to respect the courage, ambition, and desire she had to leave MIT in order to chase her dream of spreading this fantastic technology. "In 2018, the automotive industry started knocking on our doors and asked me if I could put my technology inside a car. That's when we started doing a lot of work for car manufacturers."

The story of Dr. Rana selling her company to Smart Eye it's quite a funny one, to be honest, because she had to fly from the United States to Sweden when borders between Europe and US were closed due to the Covid-19 pandemic. "In 2020, we met with the Smart Eye team, and while on the surface we believed we were competitors, the two companies were actually complementary. We were kind of the new company, while they had automotive credibility," said Rana.

"Covid hits, and then Smart Eye CEO and I started talking. Up to that point, many people wanted to buy Affectiva, and I always said no, no, no. However, this was the first time I met with a CEO that had the exact vision of the future as me. By early 2021 he asked if I was open to an acquisition, but it was Covid. As a result, the Smart Eye team couldn't visit our team in Boston". Now comes the fun part: Smart Eye CEO Martin Krantz wanted to buy Affectiva, but with one condition.

"We are not going to do it unless I sit across the table from you, look you in the eyes, and we inked the deal." Dr. Rana thought about a more straightforward solution like FaceTime, but Martin disagreed. After sharing a laugh, she continued by saying, "At the time, the borders between Europe and US were still closed, but I showed up at the Boston Logan International Airport, and I begged the counter person to let me get on the plane and sell my company." Fortunately, Dr. Rana was allowed, and she went to Sweden, where the deal was closed.

For 14 months, Affectiva has been part of the Smart Eye company, and everything is going great for them. However, I was curious about Dr. Rana's opinion on how AI will affect our future. She said that while Artificial Intelligence is present almost everywhere in our lives, we need to be careful with it. We should use it only for our benefit. For example, she is an investor in AI-driven companies that prevent diseases, help people with mental issues, etc. "We are living some exciting times, with a lot of positive potential to impact the world in interesting and unique ways. But we have to do it the right way."

And with this quote, I wanted to end everything. We have opportunities all around us, and if we are as brave as Dr. Rana el Kaliouby we can do amazing things, but we should do them the right way.