Nextcloud 29.0.3 shows Calendar 2.7.10 update available but will not update?

Support intro

Sorry to hear you’re facing problems :slightly_frowning_face:

help.nextcloud.com is for home/non-enterprise users. If you’re running a business, paid support can be accessed via portal.nextcloud.com where we can ensure your business keeps running smoothly.

In order to help you as quickly as possible, before clicking Create Topic please provide as much of the below as you can. Feel free to use a pastebin service for logs, otherwise either indent short log examples with four spaces:

example

Or for longer, use three backticks above and below the code snippet:

longer
example
here

Some or all of the below information will be requested if it isn’t supplied; for fastest response please provide as much as you can :heart:

Nextcloud version (eg, 29.0.5): 29.0.3
Operating system and version (eg, Ubuntu 24.04): Archlinux : 6.9.7-arch1-1
Apache or nginx version (eg, Apache 2.4.25): Apache : 2.4.60 (Unix)
PHP version (eg, 8.3): 8.2.21

The issue you are facing:

The little bell icon shows an update for calendar 2.7.10 (version 2.7.6 is installed). I click on the icon and then click on the calender app to update. After a 5-10 second wait the screen comes back showing no updates available?

Is this the first time you’ve seen this error? : N

I don’t recall specifics for calendar, but the bell update notification and then not being able to update has been a problem for at least the last 6 months or so.

Steps to replicate it:

  1. Click on Bell Showing update
  2. Click on the Calendar update link provided
  3. 5-10 seconds later the update screen appears showing NO updates

There is a new Apache log error where Nextcloud is failing to find index.php in the document root. The server is installed in /nextcloud and has been there since Ver. 25.

The output of your Nextcloud log in Admin > Logging: The nextcloud log shows nothing bug the same annoying bcmath issues and other non-related nuisances. The curl error may be related:

tail /var/log/nextcloud/nextcloud.log
{"reqId":"y3CJf7yYyPFmewsOiKoz","level":2,"time":"2024-07-01T21:08:41+00:00","remoteAddr":"","user":"--","app":"appstoreFetcher","method":"","url":"--","message":"Could not connect to appstore: cURL error 28: Operation timed out after 60000 milliseconds with 3907584 out of 6577902 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://apps.nextcloud.com/api/v1/apps.json","userAgent":"--","version":"29.0.3.4","data":{"app":"appstoreFetcher"}}
{"reqId":"1hj2q0TJKMtWvlzXF3nf","level":3,"time":"2024-07-02T21:06:46+00:00","remoteAddr":"192.168.6.104","user":"david","app":"PHP","method":"GET","url":"/nextcloud/apps/theming/theme/dark-highcontrast.css?plain=0&v=c973a648","message":"Module \"bcmath\" is already loaded at Unknown#0","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0","version":"29.0.3.4","data":{"app":"PHP"}}
{"reqId":"xL2hxzyXssrMhHWsazOu","level":3,"time":"2024-07-02T21:06:46+00:00","remoteAddr":"192.168.6.104","user":"david","app":"PHP","method":"GET","url":"/nextcloud/apps/theming/theme/dark.css?plain=1&v=c973a648","message":"Module \"bcmath\" is already loaded at Unknown#0","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0","version":"29.0.3.4","data":{"app":"PHP"}}
{"reqId":"0r79nQ4j5jQMX6nOD8MC","level":3,"time":"2024-07-02T21:06:49+00:00","remoteAddr":"192.168.6.104","user":"david","app":"PHP","method":"REPORT","url":"/nextcloud/remote.php/dav/calendars/david/imported-egw_export_calendar-2023-09-14ics/","message":"Module \"bcmath\" is already loaded at Unknown#0","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0","version":"29.0.3.4","data":{"app":"PHP"}}
{"reqId":"yDMLNJNvnMk3jnnrLuVW","level":2,"time":"2024-07-02T21:07:53+00:00","remoteAddr":"","user":"--","app":"appstoreFetcher","method":"","url":"--","message":"Could not connect to appstore: cURL error 28: Operation timed out after 60000 milliseconds with 4939776 out of 6578620 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://apps.nextcloud.com/api/v1/apps.json","userAgent":"--","version":"29.0.3.4","data":{"app":"appstoreFetcher"}}
{"reqId":"5Co9oHg8F10h8W4wtuUX","level":2,"time":"2024-07-03T21:08:11+00:00","remoteAddr":"","user":"--","app":"appstoreFetcher","method":"","url":"--","message":"Could not connect to appstore: cURL error 28: Operation timed out after 60000 milliseconds with 2613248 out of 6581992 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://apps.nextcloud.com/api/v1/apps.json","userAgent":"--","version":"29.0.3.4","data":{"app":"appstoreFetcher"}}
{"reqId":"JjRFqNyRqebe3C6cInCZ","level":2,"time":"2024-07-04T21:09:11+00:00","remoteAddr":"","user":"--","app":"cron","method":"","url":"--","message":"Used memory grew by more than 10 MB when executing job OCA\\UpdateNotification\\BackgroundJob\\UpdateAvailableNotifications (id: 593, arguments: null): 37.3 MB (before: 27.5 MB)","userAgent":"--","version":"29.0.3.4","data":{"app":"cron"}}
{"reqId":"Zl5sBD8iyj2GtyEYBTmt","level":2,"time":"2024-07-05T19:50:57+00:00","remoteAddr":"192.168.6.104","user":"david","app":"appstoreFetcher","method":"GET","url":"/nextcloud/settings/apps/updates","message":"Could not connect to appstore: cURL error 28: Operation timed out after 60000 milliseconds with 1490944 out of 6586995 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://apps.nextcloud.com/api/v1/apps.json","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0","version":"29.0.3.4","data":{"app":"appstoreFetcher"}}
{"reqId":"vHmk4sNPz1jymo6O5KxH","level":3,"time":"2024-07-05T19:53:00+00:00","remoteAddr":"192.168.6.104","user":"david","app":"PHP","method":"GET","url":"/nextcloud/apps/theming/favicon/settings?v=c973a648","message":"Module \"bcmath\" is already loaded at Unknown#0","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0","version":"29.0.3.4","data":{"app":"PHP"}}
{"reqId":"3em5rPqe4JZoZoljEB4L","level":3,"time":"2024-07-05T19:54:03+00:00","remoteAddr":"192.168.6.104","user":"david","app":"PHP","method":"GET","url":"/nextcloud/apps/theming/theme/light-highcontrast.css?plain=0&v=c973a648","message":"Module \"bcmath\" is already loaded at Unknown#0","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0","version":"29.0.3.4","data":{"app":"PHP"}}

The output of your config.php file in /path/to/nextcloud (make sure you remove any identifiable information!):

{
    "system": {
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "logfile": "\/var\/log\/nextcloud\/nextcloud.log",
        "default_locale": "en_US",
        "default_phone_region": "US",
        "knowledgebaseenabled": true,
        "apps_paths": [
            {
                "path": "\/usr\/share\/webapps\/nextcloud\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/lib\/nextcloud\/apps",
                "url": "\/wapps",
                "writable": true
            }
        ],
        "trusted_domains": [
            "localhost",
            "2pi.mydomain.com",
            "192.168.6.111"
        ],
        "overwrite.cli.url": "https:\/\/2pi.mydomain.com\/nextcloud",
        "htaccess.RewriteBase": "\/nextcloud",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "29.0.3.4",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "maintenance": false,
        "maintenance_window_start": 6,
        "app_install_overwrite": [
            "issuetemplate",
            "holiday_calendars"
        ],
        "theme": "",
        "loglevel": 2,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "filelocking.enabled": true,
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "timeout": 0
        }
    }
}

The output of your Apache/nginx/system log in /var/log/____: (see Above before nextcloud log)

Output errors in nextcloud.log in /var/www/ or as admin user in top right menu, filtering for errors. Use a pastebin service if necessary.

    Your web server is not properly set up to resolve `.well-known` URLs, failed on: `/.well-known/webfinger` For more details see the documentation ↗.

Apparently the appstoreFetcher timed out. It looks like your server does not have a good connection to the app store. It could be just a temporary issue.


ernolf

Thanks,

I’ll give it an hour and try again. This has popped up a couple of times during the past 6 months. I have a 300Mb internet connection, so I shouldn’t be timing out unless the update server is busy. I’ll report back either way.

Same issue even though we have tried multiple times over several hours.

I did notice a problem with the URL:

{
  "reqId": "xnOyw3nYZmSvP6TLB7pV",
  "level": 2,
  "time": "2024-07-05T23:33:14+00:00",
  "remoteAddr": "192.168.6.104",
  "user": "david",
  "app": "appstoreFetcher",
  "method": "GET",
  "url": "/nextcloud/settings/apps/updates",
  "message": "Could not connect to appstore: cURL error 28: Operation timed out after 60000 milliseconds with 1807254 out of 6588036 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://apps.nextcloud.com/api/v1/apps.json",
  "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0",
  "version": "29.0.3.4",
  "data": {
    "app": "appstoreFetcher"
  }
}

The URL above is /nextcloud/settings/apps/updates

I have no /nextcloud/settings/apps/, but I do have /nextcloud/apps/settings/

Could this be a URL issue?

[Partially SOLVED]

This was indeed some “temporary” problem. After 8-hours of temporary problems yesterday, this morning the update just worked.

This does bring to the forefront the growing number of temporary problems being faced. As mentioned in the initial post for this thread, I’ve seen this a number of times during the past six months or so. And on several instances, “temporary” seems to last for days at a time.

The only indication we get is a curl error that the appstoreFetcher timed out (it’s a 60-second / 1 minute timeout). So why is the appstoreFetcher timing out? Is it on the Nextcloud install side or the app store side? Is it a bandwidth / load balancing issue with the app store? (it being a victim of it’s own success?) Or is there some breakdown in name resolution or routing related to how/where the app store is hosted? Those I can’t answer on the userbase side.

It would be nice to get to the bottom of what is happening. What I can do, and I’m happy to do it, is run further network diagnostics from this end if you are interested. What I need is the IP or URL of the appstore and I’m happy to collect traceroute data, etc. to see if anything can be identified. Let me know.

Either way, after multiple attempts over two-days the update happened like it was supposed to, so ernolf was 100% correct about it being temporary. Up until six or eight months ago, it always worked without any fuss. Lately it seems there is more fuss than actually working going on.

Thank you for your help and let me know if you want further diagnosis from my end.

Hi @drankinatty,

No.

It is quite possible that your IP address has been throttled. The App Store is equipped with all kinds of mechanisms to counteract misuse or overloading of the server.
That would also explain why it suddenly started working again after one day, because once a day all the restrictions imposed are lifted again.

If your server is runing at your homes network, you could - next time this happens - try to obtain a new IP and look if that solves your issue.


Much and good luck,
ernolf

1 Like

Are you polling app update data via serverinfo by chance?[1]

[1] Upgrade to Nextcloud 28 — Nextcloud latest Administration Manual latest documentation

That was because of a a bug that no one pinpointed for a long time → until I disclosed it ← That’s why it’s now defused. But it led to a ban. It’s entirely possible that the administrators of apps.nextcloud.com installed a throtle mechanism instead of banning the ip for 24 hours. But it doesn’t have to come from serverinfo. I actually think that’s extremely unlikely in this case, since you have to call the info by the serverinfo api and it not hapens while a serverinfo tab is opened as it was before. But it can also happen if an update scan is done a lot - not only using this api - i.e. often in succession. But there is no real pattern discernible. I’ve already tried a lot of things by interacting with the server using the REST API.
The command line app store → nc-apps ← uses → http caching with etag ← , which ensures that only data that has changed is downloaded. This means that many requests can be made to the server within a few seconds without a ban or throttling occurring.
The server code normally uses this mechanism as well, but the bug still occurred…

But, as mentioned, it could also simply be a network bottleneck. There are millions of requests to the app store, so bottlenecks can occur that need to be balanced without any pattern being apparent to individual users.


Much and good luck,
ernolf

I don’t think that is possible. I’m the only user on my test-install and I don’t visit the store. The only thing visiting the store would be the updater??

How can this get me throttled?

Never touched anything related to the skipApps=false setting. In fact, I not sure I even know where to find it. I’m happy to check if you point me to the right place.