SMTP Server Setup with Mailgun.org

[marcc1]

Hi folks,
I’m looking for some assistance getting the eMail Server in Basic Settings working. I’m running Hub 8 on Docker on Ubuntu Server 22.04 with a pretty vanilla setup. Everything seems to be working okay, including HTTPS support and redirection from my internet subdomain.

I’m using mailgun.org as my smtp service, and have gone through all of the setup, validation, and testing with them. I know my Mailgun setup is working fine because I used my smtp credentials to run their provided swak script to send a test message from a) the server’s os, and b) within the nextcloud-app docker container. Here is the test script (redacted):

  ./swaks --auth \
  	--server smtp.mailgun.org:587 \
  	--au <redacted> \
  	--ap <redacted> \
  	--to someone@gmail.com \
  	--h-Subject: "Hello" \
  	--body 'Testing some Mailgun awesomness!'

This sends an email and is sucessfully received.

I then set up the exact same credentials in NextCloud’s Basic Settings / eMail Server page, clicked Save, then clicked “Send email”. Wihout fail, I see this error message:

A problem occurred while sending the email. Please revise your settings. (Error: Email could not be sent. Check your mail server log)

• I looked at my config.php to make sure the smtp settings were exactly those that I successfully used with the swak script, and they are.
• I looked at my mail server (mailgun.org) log, and there is no record of any unsuccessful email submission to it. I suspect that no call to mailgun is being made.
• I looked at the NextCloud log, and there were no log entries (at all) from the time when I attempted to send the email.
• I double-checked connectivity to the smtp server from within the docker container by running telnet smtp.mailgun.org 587 ad got connected.

I’m at my wit’s end, folks. What am I missing?

[marcc1]

So while I was waiting for my post to be approved, I fiddled around and found I had my logging level at the default of 1 - Info. I set it to 0 - Debug, restarted my container, and retried the Send email operation. This time, I saw a Debug entry in the NextCloud log:

Failed to authenticate on SMTP server with username “postmaster@redacted.com” using the following authenticators: “LOGIN”, “PLAIN”. Authenticator “LOGIN” returned “Expected response code “235” but got code “535”, with message “535 Authentication failed”.”. Authenticator “PLAIN” returned “Expected response code “235” but got empty code.”.

Sending mail to "Array ( [admin@redacted.com] => admin ) " with subject “Email setting test” failed

So now I have 2 questions:

1. Why was this ERROR not logged as an ERROR? This is not a debug entry, it’s an error. If it had been logged at the proper level, I would be a couple of days ahead at this point.
2. Sort of a re-post of an earlier fact - I did a successful submission to mailgun using swak with the exact same credentials that I have saved in my config.php file, while this nexcloud submission fails with an authentication error.

I’m lost. Any ideas?

[j-ed]

One main difference in sending email is, that Nextcloud is sending it as the user under which the web server is running, e.g. “wwwrun”. This might limit the possibility to customize the sender address.
It could help to debug the communication process, if you provide detailed information how you’ve set the mail parameters in your config.php file:

  'mail_smtpdebug' => true,
  'mail_smtpmode' => 
  'mail_domain' => 
  'mail_smtphost' => 
  'mail_smtpport' => 
  'mail_smtptimeout' => 
  'mail_smtpsecure' => 
  'mail_smtpauth' => 
  'mail_smtpname' => 
  'mail_smtppassword' => 
  'mail_from_address' => 

[marcc1]

Hi j-ed, thanks for offering to help. Here is the section from my config.php that has the smtp settings:

‘mail_smtpmode’ => ‘smtp’,
‘mail_smtphost’ => ‘smtp.mailgun.org’,
‘mail_smtpport’ => ‘587’,
‘mail_smtpauth’ => true,
‘mail_smtpauthtype’ => ‘login’,
‘mail_smtpname’ => ‘postmaster@mydomain.com’,
‘mail_from_address’ => ‘postmaster@mydomain.com’,
‘mail_domain’ => ‘mydomain.com’,
‘mail_smtppassword’ => ‘redacted’,

In the settings with mydomain.com, mydomain is not what’s in the config. Just redacting it for security.

[jtr]

You mentioned you’re using Docker. Not sure which image, but if it’s the micro-services one multiple config files are used (e.g. to support configuring email via Docker Compose). Please use occ config:list system (appending --private for yourself) to see your merged active config.

The following line is incorrect:

‘mail_from_address’ => ‘postmaster@mydomain.com’,

It should only be part before the @ sign.

To turn on all Email related logging you must follow this approach described in the Admin Manual within the Email chapter.

If still stuck, ping this thread and I’ll check one of my environments (on mobile at the moment). I use Mailgun for all of my Nextcloud deployments.

P.S. Please try to use the preformatted text option in the editor here on the forum when posting log entries or config files. It makes it much easier to read/etc.

[marcc1]

Thank you for your response jtr. I will endeavour to use the preformatted text formatting in the future. In fact, I have an opportunity to use it now, because it’s still not working.

To try and isolate the issue, I went directly to the config.php file and updated it with a new mail_smtpname and mail_smtppassword value. I also fixed up the mail_from_address entry (thanks again for that).

I restarted the nextcloud docker container to get it to pick up the new values from the config, but when I wen into the Basic Settings form, it still showed the old values. Weird. I connected directly to the MariaDB instance and perused the tables. Nothing there showed like it was holding the old data, but I didn’t look extensively. I did, however, update the value for emailTestSuccessful to 1 in the oc_appsettings table.

So my config.php mail section now looks like this:

'mail_smtpdebug' => true,
  'mail_smtpmode' => 'smtp',
  'mail_smtphost' => 'smtp.mailgun.org',
  'mail_smtpport' => '587',
  'mail_smtpauth' => true,
  'mail_smtpauthtype' => 'login',
  'mail_smtpname' => 'smtpuser@worxoft.com',
  'mail_from_address' => 'postmaster',
  'mail_domain' => 'worxoft.com',
  'mail_smtppassword' => 'redacted',

I then “re-sent” the welcome email to my one user and checked the logs.
These were the only log entries for that operation:

Email transport "Symfony\Component\Mailer\Transport\Smtp\SmtpTransport" starting

TransportException
Failed to authenticate on SMTP server with username "postmaster@worxoft.com" using the following authenticators: "LOGIN", "PLAIN". Authenticator "LOGIN" returned "Expected response code "235" but got code "535", with message "535 Authentication failed".". Authenticator "PLAIN" returned "Expected response code "235" but got empty code.".
Sending mail to "Array ( [redacted@gmail.com] => Some Dude ) " with subject "Your Worxoft Cloud account was created" failed

Notice that it attempted to authenticate with mailgun using the smtpname “postmaster@worxoft.com”, when the config.php file entry has that value as “smtpuser@worxoft.com”
Something is being cached or stored somewhere and not getting updated from the config. Any ideas where I can look for this?

[jtr]

@marcc1

Something is being cached or stored somewhere and not getting updated from the config. Any ideas where I can look for this?

Is that config output from your config.php or is it from running occ config:list system in your app container?

Do you have any mail related settings in your Docker Compose file?

[marcc1]

That config output was from my config.php.

(I’m learning lots, so thanks guys!) I didn’t know “occ config:list” was a thing.

I ran it using this command:

docker exec --user www-data -it nextcloud-app-1 php occ config:list

but it redacted all of the output in the mail_section:

      "mail_smtpdebug": true,
        "mail_smtpmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpauth": true,
        "mail_smtpauthtype": "login",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",

So I can’t really tell if what’s comfing out of config:list is the same as I can see in config.php.

But, HO-LEE-COWW! You nailed it dude! I looked at my docker-compose file and there was a bunch of leftover SMATP settings. I completely forgot they were there! I’m going to fix them up and try it again. I have a GOOD feeling about this…!!! I’ll let you know how it goes.

[marcc1]

Okay, major kudos to @jtr for the help on this. My issue is solved. I’ll recap the problem and solution for those who might stumble in the same way I did.

First, I used a docker-compose file from an outside source without really being familar with it. I had some environment settings in that file for the smtp settings for my server. Once I got Nextcloud up and running, I promptly forgot that I had those settings in my docker-compose.yml file.

I faffed about trying different settings, and restarting my docker container while getting educated on the nuances of NextCloud configuration. Every change that I made though, didn’t work. This was because regardless of what I did in the web UI or by editing the config.php file directly, the changes never persisted. Every time the NextCloud container restarted, it read those old values from the docker-compose.yml file overriding what was in config.php.

My solution was to delete those environment variables from the docker-compose.yml file so that whatever was in the config.php was taken as the settings. As soon as I did that, my mail test passed! I sent a welcome email to one of my users to make sure it was working and everything was fine.

I’m sure this is not my last challenge with NextCloud (I’m going to try and attach external storage next, so whatch out!) but I’m happy to have been helped here to solve this problem. Thanks again!

[jtr]

@marcc1

but it redacted all of the output in the mail_section:

FYI: occ config:list system --private will turn off the redaction.