Can't authenticate with app password Hub 6

  • Nextcloud version: Hub 6/27.1.4

  • Operating system and version:
    Server: Some Linux, probably Debian or Gentoo with kernel 5.15.125 x86_64
    Client: Fedora 39 and Garuda Linux

  • Apache or nginx version: don't know, will have to ask the hoster

  • PHP version: 8.1

  • The issue you are facing:

I actually tried to log in to the WebDAV share with two different Linux devices with KDE Dolphin, and I also tried mounting it in the OS with /etc/fstab on one, and generated several new app passwords.
The error is always something to the effect of “credentials rejected”

My theory is: NC generates me an app password, but doesn’t remember it should let new users log in with that.

Similar problems have been reported for NC 18, but can such a critical bug still be here nine releases later?
Or am I missing something?

  • Is this the first time you’ve seen this error?:
    Yes. This is a pretty fresh install, one of the first things I’ve tried to do is connecting to WebDAV.

  • Steps to replicate it:

  1. Create a new app password
  2. Try to log in any app other than the official NC one with WebDAV
  3. See if the error messages indicate wrong/unknown credentials/password

The output of the Nextcloud log in Admin > Logging:

It's empty

The config.php file in /path/to/nextcloud:

<?php
$CONFIG = array (
  'default_language' => 'de',
  'default_locale' => 'de_DE',
  'default_phone_region' => 'DE',
  'enable_previews' => 'false',
  'skeletondirectory' => '',
  'instanceid' => 'theidofmyinstance',
  'passwordsalt' => 'random string of chars',
  'secret' => 'BigTopSecret',
  'trusted_domains' => 
  array (
    0 => 'my.website.eu',
  ),
  'datadirectory' => '/home/sites/site100011358/web/nextcloud/data',
  'dbtype' => 'mysql',
  'version' => '27.1.4.1',
  'integrity.check.disabled' => true,
  'trashbin_retention_obligation' => 'auto, 30',
  'version_retention_obligation' => 'auto, 30',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'log_type' => 'file',
  'logfile' => '/dev/null',
  'localstorage.umask' => 7,
  'overwrite.cli.url' => 'my.website.eu',
  'dbname' => 'db000000',
  'dbhost' => 'db.host.net',
  'dbport' => '',
  'dbtableprefix' => 'nc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'user',
  'dbpassword' => 'password',
  'installed' => true,
  'twofactor_enforced' => 'true',
  'twofactor_enforced_groups' => 
  array (
    0 => 'admin',
  ),
  'twofactor_enforced_excluded_groups' => 
  array (
  ),
);

The output of your Apache/nginx/system log in /var/log/____:

//I don't have direct access to the web server, it's a shared host, but I found this log file, there's a lot of this:
00.000.000.00 - - [01/Dec/2023:01:20:07 +0100] "PROPFIND /remote.php/dav HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1172 1280
00.000.000.00 - - [01/Dec/2023:01:20:07 +0100] "PROPFIND /remote.php/dav/files HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1178 1280
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1191 1245
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php HTTP/1.1" 404 255 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1168 914
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND / HTTP/1.1" 405 - "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1158 947
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1251 953
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1257 953
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1172 1280
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1178 1280
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1191 1280
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND / HTTP/1.1" 405 - "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1158 947
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1866 6554
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1251 953
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1257 953
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1270 953
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php HTTP/1.1" 404 255 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1168 914
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1178 1280
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1172 1281
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1191 1280
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1257 953
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1251 954
00.000.000.00 - - [01/Dec/2023:01:20:08 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1270 953
// and further up, there's more of this:
00.000.000.00 - - [01/Dec/2023:01:19:43 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1787 6881
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane/ HTTP/1.1" 301 263 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 847 538
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1270 954
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1191 1281
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane/ HTTP/1.1" 301 263 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 846 538
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1270 954
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane/ HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1788 6881
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane/ HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1271 954
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane/ HTTP/1.1" 301 263 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 847 538
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane/ HTTP/1.1" 401 557 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1192 1281
00.000.000.00 - - [01/Dec/2023:01:19:44 +0100] "PROPFIND /remote.php/dav/files/bennypr0fane/ HTTP/1.1" 429 285 "-" "Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 dolphin/23.08.3" 1271 954

There’s also a folder with err.logs:

51.115.151.11 [2023-11-28 19:05:10.072478] [proxy_fcgi:error] [pid 31564] AH01071: Got error 'Primary script unknown'
12.121.121.21 [2023-11-30 17:37:45.254649] [access_compat:error] [pid 1924] AH01797: client denied by server configuration: /home/sites/site100011358/web/nextcloud/data/.ocdata
12.121.121.21 [2023-11-30 17:43:16.113161] [access_compat:error] [pid 6278] AH01797: client denied by server configuration: /home/sites/site100011358/web/nextcloud/data/.ocdata
12.121.121.21 [2023-11-30 17:51:09.521783] [access_compat:error] [pid 14735] AH01797: client denied by server configuration: /home/sites/site100011358/web/nextcloud/data/.ocdata
12.121.121.21 [2023-11-30 18:01:20.994185] [authz_core:error] [pid 24879] AH01630: client denied by server configuration: /home/sites/site100011358/web/nextcloud/apps/theming/img/core
12.121.121.21 [2023-11-30 18:04:54.518081] [authz_core:error] [pid 28483] AH01630: client denied by server configuration: /home/sites/site100011358/web/nextcloud/apps/theming/img/core
21.212.112.13 [2023-11-30 18:19:31.543685] [proxy_fcgi:error] [pid 10597] AH01071: Got error 'Primary script unknown'
13.232.31.31 [2023-11-30 19:45:31.774083] [authz_core:error] [pid 24855] AH01630: client denied by server configuration: /home/sites/site100011358/web/nextcloud/apps/theming/img/core
00.000.000.00 [2023-12-01 00:13:16.473467] [access_compat:error] [pid 7238] AH01797: client denied by server configuration: /home/sites/site100011358/web/nextcloud/data/.ocdata
00.000.000.00 [2023-12-01 00:15:07.703626] [authz_core:error] [pid 10329] AH01630: client denied by server configuration: /home/sites/site100011358/web/nextcloud/apps/theming/img/core
00.000.000.00 [2023-12-01 01:21:31.214076] [authz_core:error] [pid 25513] AH01630: client denied by server configuration: /home/sites/site100011358/web/nextcloud/apps/theming/img/core
00.000.000.00 [2023-12-01 01:21:33.652863] [authz_core:error] [pid 25515] AH01630: client denied by server configuration: /home/sites/site100011358/web/nextcloud/apps/theming/img/core
00.000.000.00 [2023-12-01 01:21:33.706821] [authz_core:error] [pid 25577] AH01630: client denied by server configuration: /home/sites/site100011358/web/nextcloud/apps/theming/img/core
11.111.111.11 [2023-12-01 18:40:13.834542] [access_compat:error] [pid 8911] AH01797: client denied by server configuration: /home/sites/site100011358/web/nextcloud/data/.ocdata
141.414.14.41 [2023-12-01 23:00:34.160597] [access_compat:error] [pid 7937] AH01797: client denied by server configuration: /home/sites/site100011358/web/nextcloud/data/.ocdata
00.000.000.00 [2023-12-01 23:54:33.374485] [access_compat:error] [pid 5201] AH01797: client denied by server configuration: /home/sites/site100011358/web/nextcloud/data/.ocdata

Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors:

Is this not the same as the Nextcloud log in **Admin > Logging**? Which top right menu are we talking about?

Ask your webserver instead:

curl -Is https://yourcloud.tld | grep -i "server"

:wink:

but this :

looks like an apache2 logfile

My advice: ALWAYS doubt yourself first before creating theories based on flaws in the software. This way it is usually easier to find the cause of the error.

You did not yet show how you passed the username and password in detail and how the fstab entry looks like exactly etc.
There are many points where errors can creep in.

So, please provide more information.

Much luck,
ernolf

1 Like
  1. To connect to the share with KDE Dolphin, I did this:

The answer then is this:


Meaning basically “Cannot connect to server. Check your settings and try again”. The reason I’m sure the settings are correct is because the server sends me a login prompt. The reason I think it rejects my password is as follows:

  1. To mount the share in /etc/fstab I pretty much followed the official documentation (the Dolphin part is a bit outdated) exactly:

My fstab entry looks like this:

https://my.nextcloudinstance.eu/remote.php/dav/files/bennypr0fane/ /home/ben/nextcloudinstance.eu-Nextcloud davfs user,rw,auto 0 0

where /home/ben/nextcloudinstance.eu-Nextcloud is the mountpoint.
In the ~/.davfs2/secrets file I used the $PathToMountPoint $USERNAME $PASSWORD syntax, so like /home/ben/nextcloudinstance.eu-Nextcloud bennypr0fane myverystrongpassword

After setting it up, trying to mount it goes like this:

~$ mount /home/ben/nextcloudinstance.eu-Nextcloud
das Einhängen schlug fehl; //failed to mount
Could not authenticate to server: rejected Basic challenge

So, that’s why I believe NC is rejecting my password.
This is a password I generated in Personal Settings > Security > Devices & Sessions by pressing the “Create new app password” button.

As mentioned, I tried it a couple times with fresh passwords on two different devices, same result. Also, I found this very similar issue (unsolved), albeit on an older NC version (20)

What do you think would be the problem?

Where is that file located?
I ask, because you need a .davfs/secrets file and not a davfs/secrets file (the difference is that one single dot)
And it should have 600 mode. What is the echo of

ls -l ~/.davfs/secrets

ernolf

1 Like

Sorry, that was a typo (corrected above now). It is, in fact at /.davfs2/secrets

~$ ls -l ~/.davfs2/secrets
-rw-------. 1 ben ben 2727  2. Dez 03:51 /home/ben/.davfs2/secrets

(or did you mean the echo command:
~$ echo ls -l ~/.davfs2/secrets
ls -l /home/ben/.davfs2/secrets
)

I totally followed this instruction from docs:

  1. Set yourself as the owner and make the permissions read-write owner only:

chown <linux_username>:<linux_username> ~/.davfs2/secrets
chmod 600 ~/.davfs2/secrets

I take it that’s what you meant by

?

EDIT: I tried to mount it directly without using the ~/.davfs2/secrets file, working around potential issues with it:

~$ sudo mount -t davfs https://my.nextcloudinstance.eu/remote.php/dav/files/bennypr0fane ~/nextcloudinstance.eu-Nextcloud/
[sudo] password for ben: 
Please enter the username to authenticate with server
https://my.nextcloudinstance.eu/remote.php/dav/files/bennypr0fane or hit enter for none.
  Username: bennypr0fane
Please enter the password to authenticate user bennypr0fane with server
https://my.nextcloudinstance.eu/remote.php/dav/files/bennypr0fane or hit enter for none.
  Password:  
Mounting failed.
Could not authenticate to server: rejected Basic challenge

P.S:

Thank you for this! Yes, it’s Apache

This will never work:

You have to mount it as user ben and not with sudo as user root.

You passed the credentials inside of ben’s home and you added the option “users” in /etc/fstab file, so you must mount it as user ben and not as root.

Much luck,
ernolf

tried as user too, didn’t work. :frowning_face:

~$ mount -t davfs https://my.nextcloudinstance.eu/remote.php/dav/files/bennypr0fane/ /home/ben/nextcloudinstance.eu-Nextcloud/
different mount options in /etc/fstab
~$ mount /home/ben/nextcloudinstance.eu-Nextcloud/
Mounting failed.
Could not authenticate to server: rejected Basic challenge
```65

My last advise would be, to activate the audit loging. Add this to your config.php:

  'log.condition' => 
  array (
    'apps' => 
    array (
      0 => 'admin_audit',
    ),
  ),

that wil create a logfile “audit.log” in your data directory with log entries for all such things like login attempts etc. Those entries might help you to find the problem.

Much luck,
ernolf

1 Like

thank you. I tried this, and tried logging in again via Dolphin and mounting through the command line.
In my data directory /web/nextcloud/dataI see no file named audit.log, just one nextcloud.log and a flow.log - both are emtpy. Do I maybe need to set a log level for errors to get written in there or sth.? If yes, how?
…or maybe restarting the web server is necessary? Not sure if I can do that on this shared host. Or restarting some other srevice?

You should of course remove this line from your config.php:

Logging to /dev/null means, all of your log output disappears into the void :grinning:

Much luck,
ernolf

1 Like

Brilliant! I finally got a log file:

{"reqId":"ZW5cfVRcVr9IcheUCvg7NAAAAEo","level":2,"time":"2023-12-04T23:10:54+00:00","remoteAddr":"111.101.111.101","user":"--","app":"core","method":"PROPFIND","url":"/remote.php/dav/files/bennypr0fane","message":"Login failed: 'bennypr0fane' (Remote IP: '111.101.111.101')","userAgent":"Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 knetattach/5.27.9","version":"27.1.4.1","data":{"app":"core"}}
{"reqId":"ZW5daXFV6NdimK4E9gw9FgAAACU","level":2,"time":"2023-12-04T23:14:50+00:00","remoteAddr":"111.101.111.101","user":"--","app":"core","method":"PROPFIND","url":"/remote.php/dav/files/bennypr0fane","message":"Login failed: 'ftp200027564' (Remote IP: '111.101.111.101')","userAgent":"Mozilla/5.0 (X11; Linux x86_64) KIO/5.111 knetattach/5.27.9","version":"27.1.4.1","data":{"app":"core"}}
{"reqId":"ZW5de2lUel2wP3GOIfk9UAAAACs","level":3,"time":"2023-12-04T23:15:09+00:00","remoteAddr":"111.101.111.101","user":"--","app":"core","method":"OPTIONS","url":"/remote.php/dav/files/bennypr0fane/","message":"App token login name does not match","userAgent":"davfs2/1.7.0 neon/0.32.5","version":"27.1.4.1","data":{"tokenLoginName":"bennyprofane","sessionLoginName":"bennypr0fane","app":"core","user":"bennyprofane"}}
{"reqId":"ZW5de2lUel2wP3GOIfk9UAAAACs","level":2,"time":"2023-12-04T23:15:09+00:00","remoteAddr":"111.101.111.101","user":"--","app":"core","method":"OPTIONS","url":"/remote.php/dav/files/bennypr0fane/","message":"Login failed: 'bennypr0fane' (Remote IP: '111.101.111.101')","userAgent":"davfs2/1.7.0 neon/0.32.5","version":"27.1.4.1","data":{"app":"core"}}

Can’t see anything helpful… Can you?

Here you have a possible cause.

Beautified (with jq):

{
  "reqId": "ZW5de2lUel2wP3GOIfk9UAAAACs",
  "level": 3,
  "time": "2023-12-04T23:15:09+00:00",
  "remoteAddr": "111.101.111.101",
  "user": "--",
  "app": "core",
  "method": "OPTIONS",
  "url": "/remote.php/dav/files/bennypr0fane/",
  "message": "App token login name does not match",
  "userAgent": "davfs2/1.7.0 neon/0.32.5",
  "version": "27.1.4.1",
  "data": {
    "tokenLoginName": "bennyprofane",
    "sessionLoginName": "bennypr0fane",
    "app": "core",
    "user": "bennyprofane"
  }
}

You have a mismatch with the login name

here you use an ‘o’ (char o)

    "tokenLoginName": "bennyprofane",
    "user": "bennyprofane"

and on other places you use a ‘0’ (digit zero):

"url": "/remote.php/dav/files/bennypr0fane/",
   "sessionLoginName": "bennypr0fane",

Much luck,
ernolf

1 Like

Thanks, I didn’t see that.
The one with the 0 is the correct login.
Just not sure where I typed the o. What exactly is the token loginname?

That must be the name, you want to combine with the login token to get logged in.

You now simply have to retrace the entire path and look for the misspelled username, while upholding the principle that you yourself are responsible for such (human) errors and denying theories that it is the software that “forgets” names :wink:

You can search recursivly in all of your files with

grep -r "bennyprofane"

but I would guess, it is in your secrets file.

Much luck,
ernolf

1 Like

Do you mean the server-side one or the local one?

Do you know a secrets file on your server? :thinking:

You should finally start searching instead of asking me everything, as I’m not sitting in front of your system!

Much luck,
ernolf

Well, it wasn’t the secrets file, I had mistyped when creating the user on NC… I’m embarrassed.

I created a new admin user with the correct spelling, enabled 2FA for it, created a new app password. Then I put that one in the secrets file. Login works, davfs share is locally mounted. (Also deleted the old user with wrong spelling). :+1: :+1: :+1:

Sorry about that. From one stop of the investigation to the next, I run into stretches of lazy.

Lucky that we have good educators in this forum. Am on myself by profession, but off-duty, I sometimes turn into a dumb student.

At least I learned that my hoster’s 1-click installer has logging turned off by default (also, file editing via FTP), finding out what the web server is without login, and some other things

1 Like

This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.