How is Computer Vision used to detect phishing attacks?

The rise of artificial intelligence platforms such as ChatGPT has seen the technology introduced into the wider community. Whether you love it, hate it or fear it, AI is here to stay. But AI doesn't just represent an intelligent chatbot. It is being used in many creative ways.

One such way is to use AI-powered Computer Vision as a layer of cybersecurity. Let's see how CV helps against phishing attacks through the following article!

How to use computer vision to detect phishing attacks

Phishing attacks are one of the biggest cybersecurity tactics used by scammers. Traditional methods of detecting them are imperfect, while threats are becoming increasingly sophisticated. CV aims to block one of the known vulnerabilities - it's reliance on the blacklist of 'traditional' methods.

The problem here is that updating the blacklist is not a simple matter. Even a few hours from the time a phishing site is launched until it is blacklisted is long enough to do a lot of damage.

CV does not depend on blacklist, nor does it detect embedded malicious code. Instead, it uses several techniques to flag suspicious entries.

1. Images collected from relevant emails, websites or other sources may contain threats. They are then processed using computer vision.
2. The image processing stage examines 4 main factors: Logo/brand detection, object/scene detection, text detection, and visual search.
3. These are checked using a process called 'Risk Factor Aggregation' and as a result suspicious items are flagged.

Let's take a closer look at how CV finds clues in the elements it examines.

Detect logo/trademark

Brand spoofing is a common technique used by scammers. Computer Vision is programmed to detect logos commonly used by scammers, but it can also combine this information with the content and priority of the email.

For example, an email marked as urgent with a bank logo could be flagged as potentially fraudulent. It can also check the authenticity of the logo against the expected results from the CV data store.

Object detection

Scammers will often convert objects like buttons or forms into graphics. This is done using a variety of graphical and coding techniques designed to make things complicated. Additionally, encrypted scripts can be used to perform actions like creating forms, but only after the email or web page has been rendered.

Object detection looks for visual clues after a web page or email is displayed. It can detect objects like buttons or forms even in graphic format. Also, since it checks after the email or web page renders, encrypted elements are checked.

Text detection

Similarly, text can be disguised using a variety of techniques. The preferred tactics used by scammers are:

1. Random letter padding is removed when the page or email is rendered.
2. Disguise words by misspelled words. A common example is Login , which can be easily disguised by turning the L into a capital I - Iogin .
3. Convert text to graphics.

CVs can use text analysis to detect trigger words such as passwords, account details, and logins. Again, because it runs after rendering, all text can be captured and scanned.

Visual search

Although this is part of the anti-phishing CV toolkit, it relies on reference data to work. This leaves it with an Achilles heel like any other blacklist-based system.

It works by keeping a 'template' of KGIs and KBIs in the database. This information can then be used to make comparisons to detect anomalies.

Is Computer Vision a stand-alone anti-phishing system?

The short answer is no. Currently, CV acts as an extra layer of security and is only a viable option for commercial businesses.

However, for these businesses, CV adds a new layer of security that can scan objects in real time without relying on blacklists or detecting encrypted threats. And in the ongoing race between scammers and security experts, this could be a good thing.

Looking ahead, the sudden and rapid proliferation of AI-powered chatbots like ChatGPT shows how unpredictable it can be when discussing any form of AI. But give it a try anyway!

What is the future of Computer Vision as an anti-phishing weapon?

While it's unlikely to be as impactful as AI-powered chatbots, CV's anti-phishing feature has made steady progress.

Not too long ago, this technology was suitable for larger enterprises that had the network infrastructure and bandwidth to run it as a cloud-based solution or on-premises service.

But everything has changed.

More practical subscription services are now opening up to businesses of all sizes. What matters in the age of cloud computing is the ability to protect any device from any location. This is currently an option with many services.

However, if you're looking to add to your home computer, this isn't a realistic option yet. The exponential increase in complexity and availability of AI models will almost certainly bring this functionality to home users.

The only real question is when.

Recently, AI has been in the news a lot, the most prominent of which are platforms like ChatGPT, Bing Chat, and Google Bard. These are disruptive technologies that, over time, will radically change how we access information and what we can do with it.

While these are certainly attention-grabbing tools, less disruptive technologies like CV are quietly making waves in the background. And anything that helps disrupt the growing phishing attacks is a good thing.

Kareem Winters

Update 27 June 2023