Unable to Send The Test E-Mail in Basic Settings

Support intro

Sorry to hear you’re facing problems. :slightly_frowning_face:

The community help forum (help.nextcloud.com) is for home and non-enterprise users. Support is provided by other community members on a best effort / “as available” basis. All of those responding are volunteering their time to help you.

If you’re using Nextcloud in a business/critical setting, paid and SLA-based support services can be accessed via portal.nextcloud.com where Nextcloud engineers can help ensure your business keeps running smoothly.

Getting help

In order to help you as efficiently (and quickly!) as possible, please fill in as much of the below requested information as you can.

Before clicking submit: Please check if your query is already addressed via the following resources:

(Utilizing these existing resources is typically faster. It also helps reduce the load on our generous volunteers while elevating the signal to noise ratio of the forums otherwise arising from the same queries being posted repeatedly).

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can. :heart:

The Basics

  • Nextcloud Server version (e.g., 29.x.x):
    • 33.0.3 (Winter 2026)
  • Operating system and version (e.g., Ubuntu 24.04):
    • Ubuntu 24.04.4 LTS
  • Web server and version (e.g, Apache 2.4.25):
    • Apache 2.4.6 (via AIO containers)
  • Reverse proxy and version _(e.g. nginx 1.27.2)
    • Docker AIO setup
  • PHP version (e.g, 8.3):
    • 8.3.0
  • Is this the first time you’ve seen this error? (Yes / No):
    • no
  • When did this problem seem to first start?
    • On a fresh NextCloud AIO installation
  • Installation method (e.g. AlO, NCP, Bare Metal/Archive, etc.)
    • NextCloud AIO
  • Are you using CloudfIare, mod_security, or similar? (Yes / No)
    • no

Summary of the issue you are facing:

Whenever I go to send the test e-mail to validate my configuration, no matter what e-mail server I use (whether it’s my private server or SendGrid’s public relay), I get the error that says to check the mail server log and that the email could not be sent.

Steps to replicate it (hint: details matter!):

  1. Open Admin Settings, then Basic Settings and scroll down to the e-mail server section.
  2. Fill in your server details. I am trying to use STARTTLS on port 587 on both servers I’ve tried. The servers require authentication.
  3. Click Send Test E-mail.

Log entries

Nextcloud

Please provide the log entries from your Nextcloud log that are generated during the time of problem (via the Copy raw option from Administration settings->Logging screen or from your nextcloud.log located in your data directory). Feel free to use a pastebin/gist service if necessary.

{"reqId":"Gu9RutCoqMwu7t6nD7DZ","level":3,"time":"2026-04-30T23:30:45+00:00","remoteAddr":"136.32.104.196","user":"admin","app":"settings","method":"POST","url":"/settings/admin/mailtest","scriptName":"/index.php","message":"Failed sending test email: Email could not be sent. Check your mail server log","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0","version":"33.0.3.2","exception":{"Exception":"RuntimeException","Message":"Email could not be sent. Check your mail server log","Code":0,"Trace":[{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":205,"function":"sendTestMail","class":"OCA\\Settings\\Controller\\MailSettingsController","type":"->","args":[]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":118,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Settings\\Controller\\MailSettingsController"},"sendTestMail"]},{"file":"/var/www/html/lib/private/AppFramework/App.php","line":153,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Settings\\Controller\\MailSettingsController"},"sendTestMail"]},{"file":"/var/www/html/lib/private/Route/Router.php","line":321,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\Settings\\Controller\\MailSettingsController","sendTestMail",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"_route":"settings.mailsettings.sendtestmail"}]},{"file":"/var/www/html/lib/base.php","line":1155,"function":"match","class":"OC\\Route\\Router","type":"->","args":["/settings/admin/mailtest"]},{"file":"/var/www/html/index.php","line":25,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/apps/settings/lib/Controller/MailSettingsController.php","Line":148,"message":"Failed sending test email: Email could not be sent. Check your mail server log","exception":"{\"class\":\"RuntimeException\",\"message\":\"Email could not be sent. Check your mail server log\",\"code\":0,\"file\":\"/var/www/html/apps/settings/lib/Controller/MailSettingsController.php:148\",\"trace\":\"#0 /var/www/html/lib/private/AppFramework/Http/Dispatcher.php(205): OCA\\Settings\\Controller\\MailSettingsController->sendTestMail()\\n#1 /var/www/html/lib/private/AppFramework/Http/Dispatcher.php(118): OC\\AppFramework\\Http\\Dispatcher->executeController(Object(OCA\\Settings\\Controller\\MailSettingsController), 'sendTestMail')\\n#2 /var/www/html/lib/private/AppFramework/App.php(153): OC\\AppFramework\\Http\\Dispatcher->dispatch(Object(OCA\\Settings\\Controller\\MailSettingsController), 'sendTestMail')\\n#3 /var/www/html/lib/private/Route/Router.php(321): OC\\AppFramework\\App::main('OCA\\\\Settings\\\\Co...', 'sendTestMail', Object(OC\\AppFramework\\DependencyInjection\\DIContainer), Array)\\n#4 /var/www/html/lib/base.php(1155): OC\\Route\\Router->match('/settings/admin...')\\n#5 /var/www/html/index.php(25): OC::handleRequest()\\n#6 {main}\"}","CustomMessage":"Failed sending test email: Email could not be sent. Check your mail server log"},"id":"69f3e661e0343"}

Web Browser

If the problem is related to the Web interface, open your browser inspector Console and Network tabs while refreshing (reloading) and reproducing the problem. Provide any relevant output/errors here that appear.

Not applicable

Web server / Reverse Proxy

The output of your Apache/nginx/system log in /var/log/____:

Not applicable

Configuration

Nextcloud

The output of occ config:list system or similar is best, but, if not possible, the contents of your config.php file from /path/to/nextcloud is fine (make sure to remove any identifiable information!):

docker exec --user www-data nextcloud-aio-nextcloud php occ config:list system              
{                                                                                                                       
    "system": {                                                                                                         
        "one-click-instance": true,                                                                                     
        "one-click-instance.user-limit": 100,                                                                           
        "memcache.local": "\\OC\\Memcache\\APCu",                                                                       
        "apps_paths": [                                                                                                 
            {                                                                                                           
                "path": "\/var\/www\/html\/apps",                                                                       
                "url": "\/apps",                                                                                        
                "writable": false                                                                                       
            },                                                                                                          
            {                                                                                                           
                "path": "\/var\/www\/html\/custom_apps",                                                                
                "url": "\/custom_apps",                                                                                 
                "writable": true                                                                                        
            }                                                                                                           
        ],                                                                                                              
        "check_data_directory_permissions": false,                                                                      
        "memcache.distributed": "\\OC\\Memcache\\Redis",                                                                
        "memcache.locking": "\\OC\\Memcache\\Redis",                                                                    
        "redis": {                                                                                                      
            "host": "***REMOVED SENSITIVE VALUE***",                                                                    
            "password": "***REMOVED SENSITIVE VALUE***",                                                                
            "port": 6379                                                                                                
        },                                                                                                              
        "overwritehost": "files.district12.xyz",                                                                        
        "overwriteprotocol": "https",                                                                                   
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",                                                                
        "secret": "***REMOVED SENSITIVE VALUE***",                                                                      
        "trusted_domains": [                                                                                            
            "localhost",                                                                                                
            "files.district12.xyz"                                                                                      
        ],                                                                                                              
        "datadirectory": "***REMOVED SENSITIVE VALUE***",                                                               
        "dbtype": "pgsql",                                                                                              
        "version": "33.0.3.2",                                                                                          
        "overwrite.cli.url": "https:\/\/files.district12.xyz\/",                                                        
        "dbname": "***REMOVED SENSITIVE VALUE***",                                                                      
        "dbhost": "***REMOVED SENSITIVE VALUE***",                                                                      
        "dbtableprefix": "oc_",                                                                                         
        "dbuser": "***REMOVED SENSITIVE VALUE***",                                                                      
        "dbpassword": "***REMOVED SENSITIVE VALUE***",                                                                  
        "installed": true,                                                                                              
        "instanceid": "***REMOVED SENSITIVE VALUE***",                                                                  
        "maintenance": false,                                                                                           
        "updatechecker": true,                                                                                          
        "updatedirectory": "\/nc-updater",                                                                              
        "loglevel": 3,                                                                                                  
        "app_install_overwrite": [                                                                                      
            "nextcloud-aio"                                                                                             
        ],                                                                                                              
        "log_type": "file",                                                                                             
        "logfile": "\/var\/www\/html\/data\/nextcloud.log",                                                             
        "log_rotate_size": 10485760,                                                                                    
        "log.condition": {                                                                                              
            "apps": [                                                                                                   
                "admin_audit"                                                                                           
            ]                                                                                                           
        },                                                                                                              
        "preview_max_x": 2048,                                                                                          
        "preview_max_y": 2048,                                                                                          
        "jpeg_quality": 60,                                                                                             
        "enabledPreviewProviders": {                                                                                    
            "1": "OC\\Preview\\Image",                                                                                  
            "2": "OC\\Preview\\MarkDown",                                                                               
            "3": "OC\\Preview\\MP3",                                                                                    
            "4": "OC\\Preview\\TXT",                                                                                    
            "5": "OC\\Preview\\OpenDocument",                                                                           
            "6": "OC\\Preview\\Movie",                                                                                  
            "7": "OC\\Preview\\Krita",                                                                                  
            "0": "OC\\Preview\\Imaginary",                                                                              
            "23": "OC\\Preview\\ImaginaryPDF"                                                                           
        },                                                                                                              
        "enable_previews": true,                                                                                        
        "upgrade.disable-web": true,                                                                                    
        "mail_smtpmode": "smtp",                                                                                        
        "trashbin_retention_obligation": "auto, 30",                                                                    
        "versions_retention_obligation": "auto, 30",                                                                    
        "activity_expire_days": 30,                                                                                     
        "simpleSignUpLink.shown": false,                                                                                
        "share_folder": "\/Shared",                                                                                     
        "one-click-instance.link": "https:\/\/nextcloud.com\/all-in-one\/",                                             
        "upgrade.cli-upgrade-link": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/2726",                   
        "maintenance_window_start": 100,                                                                                
        "allow_local_remote_servers": true,                                                                             
        "davstorage.request_timeout": 3600,                                                                             
        "documentation_url.server_logs": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/5425",              
        "htaccess.RewriteBase": "\/",                                                                                   
        "dbpersistent": false,                                                                                          
        "auth.bruteforce.protection.enabled": true,                                                                     
        "ratelimit.protection.enabled": true,                                                                           
        "files_external_allow_create_new_local": false,                                                                 
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",                                                             
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",                                                       
        "preview_imaginary_key": "***REMOVED SENSITIVE VALUE***",                                                       
        "default_phone_region": "us",                                                                                   
        "config_preset": 2,                                                                                             
        "DOMAIN": "files.district12.xyz",                                                                               
        "AIO_VERSION": "v12.9.2"                                                                                        
    }                                                                                                                   
} 

Apps

The output of occ app:list (if possible).

docker exec --user www-data nextcloud-aio-nextcloud php occ app:list                        
Enabled:                                                                                                                
  - activity: 6.0.0                                                                                                     
  - admin_audit: 1.23.0                                                                                                 
  - app_api: 33.0.0                                                                                                     
  - bruteforcesettings: 6.0.0                                                                                           
  - calendar: 6.2.3                                                                                                     
  - circles: 33.0.0                                                                                                     
  - cloud_federation_api: 1.17.0                                                                                        
  - comments: 1.23.0                                                                                                    
  - contacts: 8.4.5                                                                                                     
  - contactsinteraction: 1.14.1                                                                                         
  - dashboard: 7.13.0                                                                                                   
  - dav: 1.36.0                                                                                                         
  - deck: 1.17.1                                                                                                        
  - encryption: 2.21.0                                                                                                  
  - federatedfilesharing: 1.23.0                                                                                        
  - federation: 1.23.0                                                                                                  
  - files: 2.5.0                                                                                                        
  - files_antivirus: 6.2.0                                                                                              
  - files_downloadlimit: 5.1.0                                                                                          
  - files_external: 1.25.1                                                                                              
  - files_fulltextsearch: 33.0.0                                                                                        
  - files_pdfviewer: 6.0.0                                                                                              
  - files_reminders: 1.6.0                                                                                              
  - files_sharing: 1.25.2                                                                                               
  - files_trashbin: 1.23.0                                                                                              
  - files_versions: 1.26.0                                                                                              
  - firstrunwizard: 6.0.0                                                                                               
  - fulltextsearch: 33.0.0                                                                                              
  - fulltextsearch_elasticsearch: 33.0.0                                                                                
  - guests: 4.7.0                                                                                                       
  - logreader: 6.0.0                                                                                                    
  - lookup_server_connector: 1.21.0                                                                                     
  - nextcloud-aio: 0.8.0                                                                                                
  - nextcloud_announcements: 5.0.0                                                                                      
  - notes: 5.0.0                                                                                                        
  - notifications: 6.0.0                                                                                                
  - notify_push: 1.3.1                                                                                                  
  - oauth2: 1.21.0                                                                                                      
  - password_policy: 5.0.0                                                                                              
  - photos: 6.0.0                                                                                                       
  - privacy: 5.0.0                                                                                                      
  - profile: 1.2.0                                                                                                      
  - provisioning_api: 1.23.0                                                                                            
  - recommendations: 6.0.0                                                                                              
  - related_resources: 4.0.0                                                                                            
  - richdocuments: 10.1.2                                                                                               
  - serverinfo: 5.0.0                                                                                                   
  - settings: 1.16.0                                                                                                    
  - sharebymail: 1.23.0                                                                                                 
  - spreed: 23.0.4                                                                                                      
  - support: 5.0.0                                                                                                      
  - survey_client: 5.0.0                                                                                                
  - systemtags: 1.23.0                                                                                                  
  - tasks: 0.17.1                                                                                                       
  - text: 7.0.1                                                                                                         
  - theming: 2.8.0                                                                                                      
  - twofactor_backupcodes: 1.22.0                                                                                       
  - twofactor_totp: 15.0.0                                                                                              
  - twofactor_webauthn: 2.6.0                                                                                           
  - updatenotification: 1.23.0                                                                                          
  - user_status: 1.13.0                                                                                                 
  - viewer: 6.0.0                                                                                                       
  - weather_status: 1.13.0                                                                                              
  - webhook_listeners: 1.5.0                                                                                            
  - whiteboard: 1.5.7                                                                                                   
  - workflowengine: 2.15.0                                                                                              
Disabled:                                                                                                               
  - suspicious_login: 11.0.0                                                                                            
  - testing: 1.23.0                                                                                                     
  - twofactor_nextcloud_notification: 7.0.0                                                                             
  - user_ldap: 1.24.0 

It is also important to note that I get a second error mentioning the inability to connect to 127.0.0.1:25 at the time I try to send the test e-mail, and I’m not even using sendmail. See below:

{"reqId":"n3iIcHvPA0PCw7HtI8t9","level":3,"time":"2026-04-30T23:46:40+00:00","remoteAddr":"136.32.104.196","user":"admin","app":"core","method":"POST","url":"/settings/admin/mailtest","scriptName":"/index.php","message":"Sending mail to \"Array\n(\n    [tyler@district12.xyz] => Tyler Kavanaugh\n)\n\" with subject \"Email setting test\" failed","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:150.0) Gecko/20100101 Firefox/150.0","version":"33.0.3.2","exception":{"Exception":"Symfony\\Component\\Mailer\\Exception\\TransportException","Message":"Connection could not be established with host \"127.0.0.1:25\": stream_socket_client(): Unable to connect to 127.0.0.1:25 (Connection refused)","Code":0,"Trace":[{"function":"Symfony\\Component\\Mailer\\Transport\\Smtp\\Stream\\{closure}","class":"Symfony\\Component\\Mailer\\Transport\\Smtp\\Stream\\SocketStream","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/symfony/mailer/Transport/Smtp/Stream/SocketStream.php","line":157,"function":"stream_socket_client","args":["127.0.0.1:25",0,"",10,4,null]},{"file":"/var/www/html/3rdparty/symfony/mailer/Transport/Smtp/SmtpTransport.php","line":264,"function":"initialize","class":"Symfony\\Component\\Mailer\\Transport\\Smtp\\Stream\\SocketStream","type":"->","args":[]},{"file":"/var/www/html/3rdparty/symfony/mailer/Transport/Smtp/SmtpTransport.php","line":199,"function":"start","class":"Symfony\\Component\\Mailer\\Transport\\Smtp\\SmtpTransport","type":"->","args":[]},{"file":"/var/www/html/3rdparty/symfony/mailer/Transport/AbstractTransport.php","line":69,"function":"doSend","class":"Symfony\\Component\\Mailer\\Transport\\Smtp\\SmtpTransport","type":"->","args":[{"__class__":"Symfony\\Component\\Mailer\\SentMessage"}]},{"file":"/var/www/html/3rdparty/symfony/mailer/Transport/Smtp/SmtpTransport.php","line":137,"function":"send","class":"Symfony\\Component\\Mailer\\Transport\\AbstractTransport","type":"->","args":[{"__class__":"Symfony\\Component\\Mime\\Email"},{"__class__":"Symfony\\Component\\Mailer\\DelayedEnvelope"}]},{"file":"/var/www/html/3rdparty/symfony/mailer/Mailer.php","line":42,"function":"send","class":"Symfony\\Component\\Mailer\\Transport\\Smtp\\SmtpTransport","type":"->","args":[{"__class__":"Symfony\\Component\\Mime\\Email"},null]},{"file":"/var/www/html/lib/private/Mail/Mailer.php","line":205,"function":"send","class":"Symfony\\Component\\Mailer\\Mailer","type":"->","args":[{"__class__":"Symfony\\Component\\Mime\\Email"}]},{"file":"/var/www/html/apps/settings/lib/Controller/MailSettingsController.php","line":145,"function":"send","class":"OC\\Mail\\Mailer","type":"->","args":[{"__class__":"OC\\Mail\\Message"}]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":205,"function":"sendTestMail","class":"OCA\\Settings\\Controller\\MailSettingsController","type":"->","args":[]},{"file":"/var/www/html/lib/private/AppFramework/Http/Dispatcher.php","line":118,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Settings\\Controller\\MailSettingsController"},"sendTestMail"]},{"file":"/var/www/html/lib/private/AppFramework/App.php","line":153,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Settings\\Controller\\MailSettingsController"},"sendTestMail"]},{"file":"/var/www/html/lib/private/Route/Router.php","line":321,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\Settings\\Controller\\MailSettingsController","sendTestMail",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"_route":"settings.mailsettings.sendtestmail"}]},{"file":"/var/www/html/lib/base.php","line":1155,"function":"match","class":"OC\\Route\\Router","type":"->","args":["/settings/admin/mailtest"]},{"file":"/var/www/html/index.php","line":25,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/html/3rdparty/symfony/mailer/Transport/Smtp/Stream/SocketStream.php","Line":154,"message":"Sending mail to \"Array\n(\n    [tyler@district12.xyz] => Tyler Kavanaugh\n)\n\" with subject \"Email setting test\" failed","exception":"{\"class\":\"Symfony\\Component\\Mailer\\Exception\\TransportException\",\"message\":\"Connection could not be established with host \\\"127.0.0.1:25\\\": stream_socket_client(): Unable to connect to 127.0.0.1:25 (Connection refused)\",\"code\":0,\"file\":\"/var/www/html/3rdparty/symfony/mailer/Transport/Smtp/Stream/SocketStream.php:154\",\"trace\":\"#0 [internal function]: Symfony\\Component\\Mailer\\Transport\\Smtp\\Stream\\SocketStream->Symfony\\Component\\Mailer\\Transport\\Smtp\\Stream\\{closure}(2, 'stream_socket_c...', '/var/www/html/3...', 157)\\n#1 /var/www/html/3rdparty/symfony/mailer/Transport/Smtp/Stream/SocketStream.php(157): stream_socket_client('127.0.0.1:25', 0, '', 10.0, 4, Resource id #26)\\n#2 /var/www/html/3rdparty/symfony/mailer/Transport/Smtp/SmtpTransport.php(264): Symfony\\Component\\Mailer\\Transport\\Smtp\\Stream\\SocketStream->initialize()\\n#3 /var/www/html/3rdparty/symfony/mailer/Transport/Smtp/SmtpTransport.php(199): Symfony\\Component\\Mailer\\Transport\\Smtp\\SmtpTransport->start()\\n#4 /var/www/html/3rdparty/symfony/mailer/Transport/AbstractTransport.php(69): Symfony\\Component\\Mailer\\Transport\\Smtp\\SmtpTransport->doSend(Object(Symfony\\Component\\Mailer\\SentMessage))\\n#5 /var/www/html/3rdparty/symfony/mailer/Transport/Smtp/SmtpTransport.php(137): Symfony\\Component\\Mailer\\Transport\\AbstractTransport->send(Object(Symfony\\Component\\Mime\\Email), Object(Symfony\\Component\\Mailer\\DelayedEnvelope))\\n#6 /var/www/html/3rdparty/symfony/mailer/Mailer.php(42): Symfony\\Component\\Mailer\\Transport\\Smtp\\SmtpTransport->send(Object(Symfony\\Component\\Mime\\Email), NULL)\\n#7 /var/www/html/lib/private/Mail/Mailer.php(205): Symfony\\Component\\Mailer\\Mailer->send(Object(Symfony\\Component\\Mime\\Email))\\n#8 /var/www/html/apps/settings/lib/Controller/MailSettingsController.php(145): OC\\Mail\\Mailer->send(Object(OC\\Mail\\Message))\\n#9 /var/www/html/lib/private/AppFramework/Http/Dispatcher.php(205): OCA\\Settings\\Controller\\MailSettingsController->sendTestMail()\\n#10 /var/www/html/lib/private/AppFramework/Http/Dispatcher.php(118): OC\\AppFramework\\Http\\Dispatcher->executeController(Object(OCA\\Settings\\Controller\\MailSettingsController), 'sendTestMail')\\n#11 /var/www/html/lib/private/AppFramework/App.php(153): OC\\AppFramework\\Http\\Dispatcher->dispatch(Object(OCA\\Settings\\Controller\\MailSettingsController), 'sendTestMail')\\n#12 /var/www/html/lib/private/Route/Router.php(321): OC\\AppFramework\\App::main('OCA\\\\Settings\\\\Co...', 'sendTestMail', Object(OC\\AppFramework\\DependencyInjection\\DIContainer), Array)\\n#13 /var/www/html/lib/base.php(1155): OC\\Route\\Router->match('/settings/admin...')\\n#14 /var/www/html/index.php(25): OC::handleRequest()\\n#15 {main}\"}","CustomMessage":"Sending mail to \"Array\n(\n    [tyler@district12.xyz] => Tyler Kavanaugh\n)\n\" with subject \"Email setting test\" failed"},"id":"69f3e9e6de561"}

So, I decided to save my e-mail server settings to connect to my domain’s mail server on port 587. However, now I see errors in the log about the connection to the mail server timing out despite me being able to access the same mail server from my home PC with no issues. When I check my mail server’s logs, nothing appears, so it appears that NC isn’t even connecting to the mail server in the first instance.

Hi, did you also enter an email address in your administrator profile? An email address is absolutely required in your personal profile as well. If everything is configured correctly, it should work, as long as your address and port information are correct.

So I found out something really strange. When I changed the setup to use my mail server’s private IP address (in the range 10.0.0.0/8) and disabled certificate validation, it worked. But when I try to use the hostname of the server which resolves to its public IP, it times out and can’t send any emails. Yet all other calls to external services work just fine (calls to my S3-compatible storage backend, for example, do not fail). I have my DNS records set up correctly. So why would only e-mail fail when sending to public hostnames but succeed when using private IP addresses?

It sounds like you may have problems with Hairpinning/NAT-loopback not working on your server? You may be able to work around with proper split-brain DNS.

Check on OS-shell of your nextcloud server, whether a ping to your mailserver by hostname really works. In most cases an “Unknown host” may show the access problem.