App Store content cache

đź’» Development
, ,
1

I just released twofactor_email 2.8.1 (v2) (now NC29…NC32) after I already hat released the rewrite twofactor_email 3.0.3-beta.5 (v3) (≥NC32) marked “as nightly” a couple of days ago:

I wanted to manually test “2.8.1 on NC32” and the “upgrade from v2 to v3 in NC32”. For that I used my test instance that was set to “Beta Channel”. To do the tests, I switched it to “Stable Channel”.

I noticed that it only offered me 3.0.3. So I manually extracted the 2.8.1 release tarball to /apps/ and enabled it. Then, the instance offered to update it to 3.0.3.

With my instance set to stable and no overrides in config.php, the instance should not offer the nightly at all. Or is that intended behavior, e.g. because of the NC is being beta?

I assume that it is a caching issue: The test instance probably still had a cached version of the apps available in the app store (prior to my 2.8.1 release) and with “Beta Channel” set when the cached information was retrieved.

I think that this “app store caching” needs documentation in the Admin and Developer manual. I also think that there should be a mechanism that

  • updates that app store cache upon channel change along with a warning that this is done in the background and may take some time
  • allows an admin to manually clear and repopulate that cache including the possibility to wait for it to finish (for automation)

What are your insights and thoughts on this?

2

OK, a first comment: a beta is no nighly. NC has 3 kinds of releases defined for their apps:

  1. regular releases, e.g. 2.8.1. So, it is [0-9]+\.[0-9]+\.[0-9]+ in regex.
  2. testing (pre) releases, e.g. 3.0.4-beta1. This is a regular release with a non-null appendix (-beta2 in this case).
  3. nighly releases

Please note, that in the appstore, for testing releases the nighly flag must not be set.

The offered updates should be only regular releases for productive setups, regular+testing for beta instances. The daily and git states should show all (also nighly) releases (see here).

The mentioned cache comes from [1] [2] [3] [4] which then reads the appstore/apps.json file from the appdata folder (typically data/appdata_*) [5]. It should be safe to drop that cache file and let NC regenerate it.

Hopefully, productive admins will not got to beta and be surprised by this. Adding it to the admin docs might be feasible IMHO. Feel free to put there a PR on the documentation.

Does this answer your question?

Chris

1 Like
3

Thanks for explaining the cache part. I won’t do a server doc PR. Since this thread is about cache, I marked it “solved”.

As for the beta vs. nightly topic I was clear about release channel ↔ app store, but even after your explanation uncertain about how beta channel corresponds to the app versions and flags in the app store. Thanks for the clearification by phone, which I summarize here:

As an app dev, one may choose to flag an upload as “nightly” in the app store, or not. There is no channel, no “beta”/“pre-release”. But the regex you mentioned above means that

  • Server instances that use the enterprise or stable channel will ONLY see apps that have a three-decimals VERSIONÂą like 12.3.456 AND not marked “as nightly”.
  • Server instances that use the beta channel will use any app that is NOT marked “as nightly”, so e.g. a VERSION 12.3.456. or 12.3.456-beta.1.
  • Only server instances running as nightly themselves will use the apps that are marked “as nightly”.

However, as always, there is one exception: If an app is set to ignore version constraints like in the following example, then the latest² version is installed (or being tried to) no matter what the server channel or app requirement is:

config/config.php:

…
  'app_install_overwrite' => 
  array (
    0 => 'files_frommail',
    1 => 'twofactor_gateway',
    …
  ),
…

Footnotes

    Âą VERSION refers to the `VERSION` line in `config/config.php`.

    ² It probably is documented somewhere, but just as a sidenote here: One must check the comparator function here, like: Is 1.2.3-beta1 a higher or lower version compared to 1.2.3, or does the upload date matter, etc.
1 Like
4

The mentioned cache comes from [1] [2] [3] [4] which then reads the appstore/apps.json file from the appdata folder (typically data/appdata_*) [5]. It should be safe to drop that cache file and let NC regenerate it.

Dropping that file leads to “Internal server error” (500) when calling /apps.

To get rid of the error, I got the latest app list from the app store using wget ``https://apps.nextcloud.com/api/v1/apps.json which results in a 20 MB json. However, in my other instances, the apps.json is about 2,8 MB.

What would be the correct URL and parameters for wget? Are there other ways to make NC re-fetch apps from app-store?