Mail server prerequisites — PTR, SPF, DKIM, DMARC explained

Sending mail from a VPS in 2026 is harder than it used to be — not because the protocols changed, but because every major recipient (Gmail, Outlook, Yahoo, Apple) now rejects or spam-folders mail that doesn't pass a checklist of authentication tests. This article walks through the four tests your mail server needs to pass before it can reliably deliver to a human inbox: reverse DNS, SPF, DKIM, and DMARC.

All four are DNS-based. None require changes to your VPS itself except DKIM (which needs the mail server to sign outgoing messages). Get them all green and your deliverability problems get smaller by an order of magnitude.

1. Reverse DNS (PTR)

The first thing every receiving mail server checks: does your sending IP have a PTR record, and does that PTR resolve to a hostname that resolves back to the same IP? This is called forward-confirmed reverse DNS (FCrDNS), and it's mandatory.

Set up via the LYLIX portal's Reverse DNS tab — see the dedicated article Setting reverse DNS (PTR) for your VPS for the full walkthrough.

Verification:

dig +short -x <your-IP>
# expect: mail.yourdomain.com.

dig +short mail.yourdomain.com
# expect: <your-IP>

Both have to match. If one direction is wrong, FCrDNS fails and you go in the spam folder by default.

2. SPF — Sender Policy Framework

SPF is a DNS TXT record on your domain that lists the IPs allowed to send mail "from" addresses at your domain. Recipient servers check the envelope's MAIL FROM domain, look up that domain's SPF, and reject (or mark suspicious) any mail coming from an IP not on the list.

The minimum SPF record for a VPS that sends its own mail:

# DNS TXT record on yourdomain.com
"v=spf1 ip4:<your-IP> -all"

If you also send mail through a third-party service (Google Workspace, Microsoft 365, SendGrid, Mailgun), include their SPF mechanism too:

"v=spf1 ip4:<your-IP> include:_spf.google.com -all"

The -all at the end means "reject anything not matched above" (a hard fail). Use ~all (soft fail) while you're testing; switch to -all once you're sure the list is complete.

Verify:

dig +short TXT yourdomain.com | grep spf
# Or use a checker:
# https://mxtoolbox.com/spf.aspx

3. DKIM — DomainKeys Identified Mail

DKIM adds a cryptographic signature to every outgoing message that proves it came from a server authorized to sign for your domain. The signature uses a public/private keypair: the private key lives on your mail server; the public key is published in DNS. Receivers verify the signature using the public key.

DKIM requires two steps — one in DNS, one on your mail server.

Generate the keypair

On your VPS, install OpenDKIM (or use your mail server's built-in DKIM module). For Postfix + OpenDKIM on Debian:

apt install -y opendkim opendkim-tools

mkdir -p /etc/opendkim/keys/yourdomain.com
cd /etc/opendkim/keys/yourdomain.com
opendkim-genkey -s mail -d yourdomain.com
# Generates mail.private (keep secret) and mail.txt (the public DNS record).

Publish the public key

The mail.txt file contains the DNS record to publish. It looks something like:

mail._domainkey IN TXT ( "v=DKIM1; h=sha256; k=rsa; "
    "p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA..." )

Add it as a TXT record at the host name mail._domainkey.yourdomain.com with the v=DKIM1; ... content as the value. The "mail" prefix is the selector — you can have multiple selectors for key rotation later.

Configure your mail server to sign

OpenDKIM config (/etc/opendkim.conf):

Domain      yourdomain.com
Selector    mail
KeyFile     /etc/opendkim/keys/yourdomain.com/mail.private
Socket      inet:8891@localhost

Postfix integration (/etc/postfix/main.cf):

milter_default_action = accept
milter_protocol = 6
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

Restart both:

systemctl restart opendkim postfix

Verify by sending a test email to check-auth@verifier.port25.com — it replies with a full DKIM / SPF / DMARC report.

4. DMARC — Domain-based Message Authentication, Reporting & Conformance

DMARC is the policy layer on top of SPF and DKIM. It tells recipients: "If a message claims to be from my domain but fails both SPF and DKIM alignment, here's what to do — quarantine (spam folder) or reject (bounce). And send me a report so I can see who's trying to spoof me."

A starter DMARC record (DNS TXT at _dmarc.yourdomain.com):

"v=DMARC1; p=none; rua=mailto:dmarc-reports@yourdomain.com; pct=100"

The p=none means "don't actually do anything yet, just send me reports." Use this while you confirm SPF and DKIM are working for all your legitimate senders. Once reports show clean, tighten to:

"v=DMARC1; p=quarantine; rua=mailto:dmarc-reports@yourdomain.com; pct=100"

And eventually to p=reject for full enforcement.

The reports themselves are XML attachments emailed to whatever address you put in rua=. Tools like dmarcian.com or postmarkapp.com/dmarc ingest them and show a dashboard if you don't want to read raw XML.

The full checklist

Before you flip the switch on production mail from your VPS:

1. PTR set, returning a hostname that resolves back to your IP (FCrDNS passes both directions).
2. SPF published, listing your IP and any other services that send for your domain. -all at the end once you're confident.
3. DKIM signing on your outgoing mail, with the public key published in DNS and the private key only on the mail server.
4. DMARC published, starting at p=none with reporting on, tightening over time.
5. HELO/EHLO matches PTR — configure your mail server to announce the same hostname that PTR resolves to. Triple alignment: EHLO = PTR = A record.

Testing your setup

Three external tools that do the full check in one shot:

• mail-tester.com — send a test message to the address it gives you; it scores 0-10 and itemizes everything wrong. Best single tool.
• port25 verifier — email check-auth@verifier.port25.com from your server; it replies with a full report.
• mxtoolbox.com — domain-level lookups for SPF, DMARC, MX, blacklists. Good for diagnosing one piece at a time.

Aim for a perfect score on mail-tester. Anything missing will correlate directly with "Gmail thinks my mail is spam" tickets later.

Common mistakes

• SPF includes a hostname instead of an IP — SPF only counts include:, ip4:, ip6:, a, and mx mechanisms. Plain hostnames are ignored.
• Multiple SPF records for the same domain — DNS spec allows one TXT, but multiple SPFs is technically a permerror. Combine them into one.
• DKIM key in the wrong DNS host — the record name must be {selector}._domainkey.yourdomain.com, not just {selector}.yourdomain.com.
• DMARC at the wrong host — must be at _dmarc.yourdomain.com, not at the root.
• Forgot to restart Postfix / OpenDKIM after config changes. SPF and DMARC are pure DNS so DNS-side fixes are instant; DKIM signing happens at the mail server, so config changes there need a restart.

None of this requires LYLIX intervention — it's all on your side (DNS provider, mail server). But if a test result looks wrong and you can't figure out why, open a ticket with the test output and we'll have a look.