Interpreting the Email Validation risk score algorithm

Auth0 Email Signals enables all users to obtain a score that indicates the risk level of an analyzed Email address. This article intends to explain how the core service gets this score and also the different signals extracted.

Therefore, it applies to either the CLI, or the Slack Bot as they are all direct API endpoint wrappers. This article will detail how the Auth0 Signals API works, but it’s easy to translate the concepts to the different user interfaces. The reader should have some basic knowledge of shell script and the curl command. If not, we recommend installing the Slack Bot.

Get an Auth0 Signals API Key

The Email reputation or verification is a service that any user can use with or without a valid Auth0 Signals API Key. There are very some key differences:

  • With an API Key, the maximum number of hits allowed is 40.000 every 24 hours. Without an API Key (anonymously), the number of “hits” allowed is 100 every 24 hours.
  • The users can configure the API requests with some query string parameters. It’s mandatory to have an API Key.

If you need more information about the quotas of the API and the difference between hits and requests, you should read this sort FAQ.

So we strongly recommend to always sign up in Auth0 Signals and get an API Key. If you have any doubt, you can read this short tutorial detailing all the steps.

How do Email verification and validation work?

Email Verification determines whether or not an email address is entirely valid and deliverable. Overall the process involves an in-depth analysis of each email. In detail, a combination of several validation techniques based on custom-built algorithms scores the legitimacy of an email. After the execution of the process, it has ranked the addresses. Clean and valid email addresses will have a neutral score. Suspicious addresses will have a negative rating: the more negative the score, the more probability of being an address with a low reputation.

Below there is a detailed description of just how our email verification process works:

  • Address: If any of these checks fail, the score of this test is -1.

    1. Address Syntax: This check removes improperly formatted email addresses. It must adhere to IETF standards.
    2. Role-Based Account: Email sent to role-based such as postmaster@, info@, sales@, admin@, etc. can negatively impact the deliverability, and even some ISPs will block them. This process detects and flags such addresses.
  • Domain: The algorithm tests the domain, the MX domains, and NS domains. If any of these checks fail, the score of this test is -1.

    1. Domain blacklisted: The process searches the domain in different blacklists. To be a clean domain, it must not appear in any of them.
    2. MX domain blacklisted: It searches the MX domains in different blacklists too. Clean MX domains must not appear in any of them.
    3. NS domain blacklisted: It searches the NS domains in different blacklists too. Spammers use very well-known suspicious Nameservers, and we also can block them.
  • Free Email Service: Our algorithm can detect emails hosted by Free Email Services like (Google, Yahoo, Hotmail/Microsoft, etc.). Since this is a very aggressive option, the algorithm does not compute the result as negative, but it flags the email to help users to make a decision.

  • Disposable Email Address: If the Disposable Email Address Providers host the Email address. Throwaway/disposable email addresses, or “junk collector” email addresses" are detected and processed appropriately. It is a widespread threat indicator of bad actors using them to bypass signup forms or login forms that require a valid email address. The score for this test is -1.

  • Blacklisted Email Address: If the Email address belongs to some of our Email Abuse blacklists, then the score for this test is -1.

  • SMTP Verification : Performs deep-level extended SMTP verifications on the email address. The process pings the addresses for mailbox existence without sending an actual email to the inbox. It will also check the validity of the MX records and will test if the server implements a catch-all policy. The score for this test is -1 for any of them.

  • Lookup IP in OSINT blacklists: The algorithms check the email addresses and IP addresses against known DNSBLs and RBL to trap spam networks. The score for this test is -1 for any of them.

Invoking the API

For example, let’s get the score for the email test@mailinator.com. Mailinator.com is a very well known Disposable Email Address provider, so it should have a negative rating. To test the API, we will use the command curl. The requests need two parameters: the Auth0 Signals API Key and the Email to analyze. It’s also possible to add parameters in the query string to tweak how the service works.

$ curl -i -H "Accept: application/json" -H "X-Auth-Token: API_KEY" -X GET "https://signals.api.auth0.com/bademail/test@mailinator.com"

or

$ curl -i  -H "Accept: application/json" -X GET "https://signals.api.auth0.com/bademail/test@mailinator.com?token=API_KEY"

the JSON response will be:

HTTP/2 200 
date: Wed, 24 Jun 2020 11:04:06 GMT
content-type: application/json; charset=utf-8
content-length: 876
server: Python/3.6 aiohttp/3.6.2

{
   "response":{
      "email":{
         "blacklist":[
            "STOPFORUMSPAM-90"
         ],
         "score":-1
      },
      "domain":{
         "blacklist":[
            "IVOLO-DED",
            "DEA",
            "LISINGE-DED",
            "MARTENSON-DED"
         ],
         "blacklist_mx":[
            "LISINGE-DED",
            "DEA-MX"
         ],
         "blacklist_ns":[

         ],
         "mx":[
            "mail2.mailinator.com",
            "mail.mailinator.com"
         ],
         "ns":[
            "betty.ns.cloudflare.com",
            "james.ns.cloudflare.com"
         ],
         "score":-1
      },
      "disposable":{
         "is_disposable":true,
         "score":-1
      },
      "freemail":{
         "is_freemail":false,
         "score":0
      },
      "ip":{
         "blacklist":[

         ],
         "is_quarantined":false,
         "address":"104.26.1.114",
         "score":0
      },
      "source_ip":{
         "blacklist":[

         ],
         "is_quarantined":false,
         "address":"AAA.BBB.CCC.DDD",
         "score":0
      },
      "address":{
         "is_role":false,
         "is_well_formed":true,
         "score":0
      },
      "smtp":{
         "exist_mx":true,
         "exist_address":true,
         "exist_catchall":true,
         "graylisted":false,
         "timedout":false,
         "score":0
      },
      "score":-3,
      "email_address":"bitches@mailinator.com"
   },
   "type":"bademail"
}

Using the Slack Signals Bot

The email EMAIL_ADDRESS command will display the same information in a human-readable format ready to be share. For the example given above:

What is the confidence score?

The confidence score is the sum of analyzing different parameters of the Email address. Any user can obtain the following values:

  • 0: Auth0 Signals is neutral about the Email address given. It means the service cannot find the Email address in any given individual service and cannot classify the Email as risky.

  • -1: Auth0 Signals has detected the Email address in one of the checks. This score is the lowest level of risk.

  • lower than -1: Auth0 Signals has detected the Email address in two or more checks. The lower the score, the riskier the Email.

It is already clear that there are values between -3 and 0, and that there are no positive values because the algorithm does not calculate confidence indices, but rather indices of DISTRUST.

Individual testing and scores

Email Blacklist

Auth0 Signals collects email addresses reported in well-known anti-abuse OSINT related to spamming, credential stuffing attacks, phishing, etc. If the scoring algorithm finds the email address in any of these lists, it will score -1.

Domain Blacklist

Auth0 Signals collects domains reported in multiple open sources. The algorithm test the domain, the MX, and NS records of the domain against these datasets. If it finds any of these records in the datasets, it will add -1 to the global score.

IP Blacklist

The algorithm looks up the primary IP address of the domain in the existing IP address datasets (the ones used for IP reputation). If it finds the IP in any of these datasets, it will add -1 to the global score.

Is a Disposable Email Address?

The algorithm looks up the email address in the set of specific domains that store these providers. If it finds the domain of the Email in any of these lists, it will add -1 to the global score.

Is a well-formed email address?

The algorithm also checks if the email address is well-formed according to the IETF standards. It also verifies if the username of the email address is role-based. If any of these tests are True, then it will add a -1 to the global score.

SMTP verification

The SMTP verification performs several tests, but it will add -1 to the global score if any of these conditions occur:

  • The SMTP servers in the MX records don’t respond.
  • The email address was not found in the SMTP server.
    But it will also return the following information:
  • The SMTP servers implement a catch-all policy.
  • The SMTP server implements a graylist policy, and the results of the tests could be invalid.
  • The SMTP server timed out before returning any valid data,

Source IP Blacklist

The algorithm looks up the source IP address of the requester in the existing IP address datasets (the ones used for IP reputation). If it finds the IP in any of these datasets, it will add -1 to the global score. By default, the algorithm uses the client’s source IP address, but it’s possible to use a custom IP address with the source_ip query string parameter.

is a Free Email Provider?

The algorithm looks up the email address in the set of specific domains that store the Free mail providers. The result of this test won’t compute in the global score, but it will return True or False in the local section.

Need more? Read the API specs.

The Auth0 Signals API Docs contains the full list of query string parameters, endpoints, and JSON objects.

1 Like

This is fantastic! Please let us know if you have any questions everyone :pray: