smtp_mail_from Always Overwrites from_email in Airflow Email Sending Logic

[opqi]

Apache Airflow version

2.10.4

If "Other Airflow 2 version" selected, which one?

2.10.3

What happened?

Description:

The current implementation of email sending in Airflow does not allow the from_email parameter to function as intended due to the behavior of smtp_mail_from.

The code snippet from send_email_smtp function in airflow/utils/email.py below highlights the issue:

smtp_mail_from = conf.get("smtp", "SMTP_MAIL_FROM")

if smtp_mail_from is not None:
    mail_from = smtp_mail_from
else:
    if from_email is None:
        raise ValueError(
            "You should set from email - either by smtp/smtp_mail_from config or `from_email` parameter"
        )
    mail_from = from_email

Issue:

• smtp_mail_from is always a string. Even if the configuration in airflow.cfg leaves it empty, conf.get("smtp", "SMTP_MAIL_FROM") defaults to an empty string (""), which is not None.
• As a result, the from_email parameter is never used because the if smtp_mail_from is not None condition is always True.
• This behavior renders the from_email parameter in various email-related functionalities effectively useless.

What you think should happen instead?

Expected Behavior:

The from_email parameter should take precedence if explicitly provided, but it currently cannot because smtp_mail_from is never None.

Proposed Solution:

Modify the condition to properly handle empty strings and prioritize from_email when explicitly set:

smtp_mail_from = conf.get("smtp", "SMTP_MAIL_FROM")

if smtp_mail_from:  # Check for non-empty string
    mail_from = smtp_mail_from
else:
    if from_email is None:
        raise ValueError(
            "You should set from email - either by smtp/smtp_mail_from config or `from_email` parameter"
        )
    mail_from = from_email

This change ensures:

1. smtp_mail_from is only used when it is explicitly set to a non-empty value.
2. The from_email parameter is respected when provided.

How to reproduce

Prerequisites:

• An Airflow instance is running.
• Access to the Airflow UI to create an SMTP connection.

---

Steps to Reproduce:

1. Clear SMTP Configuration in airflow.cfg:
   In the airflow.cfg file, ensure all SMTP-related parameter in the [smtp] section are left blank or commented out:

   [smtp]
   smtp_mail_from =

2. Restart Airflow:
   Restart the Airflow services to apply the changes:

   airflow scheduler restart
   airflow webserver restart

3. Create an SMTP Connection in the Airflow UI:

   • Navigate to the Admin -> Connections page.
   • Click + Add a new record.
   • Fill out the connection details:
     • Conn Id: smtp_test
     • Conn Type: SMTP
     • Host: <SMTP server> (e.g., smtp.example.com)
     • Port: 25
     • Email From: [email protected]
   • Leave other fields empty or as defaults.
   • Save the connection.

4. Create a DAG with an EmailOperator:
   Use the following sample DAG to send an email:

   from airflow import DAG
   from airflow.operators.email import EmailOperator
   from datetime import datetime
   
   with DAG(
       dag_id='test_smtp_from_email',
       start_date=datetime(2025, 1, 1),
       schedule_interval=None,
       catchup=False,
   ) as dag:
   
       email_task = EmailOperator(
           task_id='send_email',
           conn_id='smtp_test',
           to='[email protected]',
           subject='Test Email',
           html_content='This is a test email.',
           from_email='[email protected]',  # Explicit from_email
       )

5. Trigger the DAG:

   • Navigate to the Airflow UI.
   • Trigger the test_smtp_from_email DAG.

---

Expected Behavior:

• The email should be sent with [email protected] as the From address when from_email is explicitly provided.

---

Actual Behavior:

• The email is sent using the default smtp_mail_from from airflow.cfg (likely an empty string or a misconfigured value).
• The from_email parameter is ignored completely.

---

This demonstrates that the from_email parameter is not respected due to the logic prioritizing smtp_mail_from, even when it is blank.

Operating System

Debian GNU/Linux

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

curl -LfO 'https://airflow.apache.org/docs/apache-airflow/2.10.3/docker-compose.yaml'

Anything else?

Impact:

This bug impacts all email-sending functionalities, such as email alerts, EmailOperator, and any custom email utilities relying on Airflow's email sending methods.

Environment:

• Airflow version: 2.10.3

Let me know if more details or examples are needed!

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

Can you please propose PR to solve problems. It will be easier to validate the claims by looking at changes in the code that fix it.