Oversight in Sensitive Fields

Phishing and AI: New Tricks and Practical Defenses

How AI can make phishing and scam messages more convincing, and the sensible, practical habits that help individuals and teams stay safe.

On this page
  1. What phishing is
  2. What AI changes
  3. Warning signs that still matter
  4. Practical habits for individuals
  5. Practical habits for organizations
  6. Your own use of AI tools
  7. Staying realistic

What phishing is

Phishing is an attempt to trick someone into doing something harmful by pretending to be a trusted party. The aim might be to steal a password, get a payment sent, install malicious software or collect personal details. Messages often arrive by email, but text messages, phone calls, chat apps and social media are used too.

For years, many phishing messages could be spotted by clumsy writing, odd greetings or generic wording. That made “look for the typos” a common piece of advice. AI changes this somewhat.

What AI changes

General-purpose language tools can produce clear, well-written text in many languages and tones. Reports from security professionals commonly note several ways this can help scammers, though how widespread each one is varies and is hard to measure precisely:

  • Better writing. Fewer spelling and grammar errors, and a more natural, professional tone.
  • Tailoring. Messages can be adapted to a person’s job, company or interests using public information, which makes them feel more relevant.
  • Scale. Many variations can be produced quickly, which can make it harder to filter messages by matching known wording.
  • Impersonation. Writing in the style of a manager, supplier or colleague becomes easier. Synthetic voice and video can add to this, a risk covered in synthetic media.
  • Conversation. Some scams involve back-and-forth replies, which automation can help sustain.

It is worth keeping perspective. The basic structure of most scams has not changed: they create urgency, appeal to trust or fear, and ask for an action. AI mostly makes the surface more convincing, so the defenses that focus on behavior and process remain valuable.

Warning signs that still matter

Because polish is no longer a reliable sign of legitimacy, focus on what the message asks you to do:

  • Pressure to act immediately, or threats if you do not.
  • A request for passwords, codes, payment, gift cards or sensitive data.
  • A change in payment details for a supplier or an unusual request from a senior person.
  • A link or attachment you were not expecting.
  • A sender address or link that looks similar to a real one but is slightly off.
  • A request to keep something secret or bypass normal procedures.

A message with none of these signs can still be malicious, which is why habits matter more than a checklist of clues.

Practical habits for individuals

  • Pause. Urgency is the scammer’s tool. Taking a minute is often enough to notice something is wrong.
  • Verify through a separate channel. If a message asks for money or access, contact the person using a phone number or address you already have, not one given in the message.
  • Do not click through to sign in. Go to the service by typing its address or using a saved bookmark.
  • Use multi-factor authentication where available, and use a password manager, so that a stolen password alone is less useful. Never read out one-time codes to someone who contacts you.
  • Keep software updated.
  • Report suspicious messages to your IT or security team rather than just deleting them.

Practical habits for organizations

Defenses work best in layers, and they should not depend on every employee spotting every trick.

  • Payment and access procedures. Require a second person or a call-back for changes to bank details and for unusual payments. This approach holds up even against a very good impersonation.
  • Technical controls. Email filtering, authentication of sending domains and link and attachment scanning reduce what reaches inboxes. Some of these use machine learning for threat detection, and they miss things too.
  • Clear reporting. Make it easy and blame-free to report a mistake. People who fear punishment tend to hide errors, which gives attackers more time.
  • Regular, realistic training. Short, practical sessions with current examples are usually more useful than a yearly lecture. See staff training for general ideas on making it stick.
  • Limit access. If an account is compromised, restricting what it can reach limits the damage.
  • Prepare a response plan. Know who to call, what to disconnect and what to preserve. A simple escalation path helps.

Your own use of AI tools

Staff can also be tricked through the tools they use, for example by a document or web page that includes hidden instructions meant to manipulate an assistant. Treat content from unknown sources with caution, avoid giving tools access to more accounts than necessary, and check anything an assistant proposes to do on your behalf. Also be careful about what you paste into tools, since it can be useful to attackers if exposed, as discussed in customer data.

Staying realistic

No set of habits eliminates the risk, and attackers keep adapting. The goal is to make attacks harder, limit damage and recover quickly. For a specific organization, a qualified security professional can advise on controls that suit its size and risks. This article is general awareness only.

The complete guide to oversight in sensitive fields

Why Sensitive Fields Need Extra Human Oversight
Read the guide

Keep exploring

Guides from other topics

All topics →