Script "nc-apps" - a helpful and more verbose complement to `occ app:list`

đź“‘ How to
, ,
1

  • occ app:list gives you a list of enabled and disabled apps but you only get the app-ids listed.
    In the app store, however, apps are not sorted, searched and found by app-id, but by its name. These names are sometimes far away from the app-id, which is a pitfall, or at least a point of great confusion especially for newer admins. It’s easy to get one app mixed up with another, or unintentionally install two apps that are actually mutually exclusive.

  • There are a number of shipped apps that are automatically supplied with the server and its update. These apps are in turn divided into those that are always enabled, those that are enabled by default and those that can be enabled by the admin. This is not clearly visible with the provided tools.

  • After an update from e.g. 26 to 27, one like to use the app_install_overwrite function for some apps because you either already know that the app runs without problems or you just want to try it first. Once an app is included in the app_install_overwrite array, many admins simply forget about it and leave it there. However, you can also delete the app_install_overwrite array after the app has been installed and activated. Nevertheless, it is sometimes desirable to get an overview of which apps are actually supported by your server version, which min/max php version is required and which databases are supported etc.

To counter all this I wrote nc-apps

It is a program written in bash that is intended to close this gap(s).

It’s intentionally written in bash so that anyone can read and understand the code, maybe use parts of it for their own routines.

Once you get used to the wealth of information that this tool provides, you will hardly want to miss it.

It also gives you the ability to manipulate the version; If you are running Nextcloud 25, for example, you can tell the tool to simulate version 26 in order to see which of the apps already installed are suitable for 26 without any further updates. (It doesn’t show all updatable apps, which only become visible from the actually installed new version, but it still gives a good preliminary overview.) Because of the appstore integration, this filter is applicable on not installed apps too.

The tool also has the option of “cleaning up” an existing app_install_overwrite array (if present) in config/config.php, i.e. removing those app ids that are meanwhile supported by the current version and even without being listed in the app_install_overwrite array, can be deactivated and reactivated again.

The script can handle a configuration spread over multiple *.config.php files.

But what use a thousand words, it’s best to just try it out. The script can’t do any harm to your system, even if you want to clean up the app_install_overwrite array, it will first list everything that will be changed if you explicitly answer “yes”. Only then changes are applied.

Restriction:

This script is intended for use on conventional “barremetal” linux server installations or dedicated virtual machines. It is not tested with containerized solutions like snap, lxc, docker, aio etc.

Here you can read the code: https://global-social.net/apps/raw/s/nc-apps

and this is how you get the script installed or updated on your server:

sudo wget -O /usr/local/bin/nc-apps https://global-social.net/apps/raw/s/nc-apps
chmod +x /usr/local/bin/nc-apps
nc-apps verify_my_integrity
last updated 2023.08.08 19:00 - to update you only have to run the first line

  • you simply can call it with nc-apps.

  • If your nextcloud path is not recognized, a firstrun wizard will ask you once to sudo root and enter the correct path to your nextcloud installation.

  • If not met dependencies are found, it hints you to install them. (gpg, jq, xmlstarlet)

  • If you call as normal user, it will automatically cal sudo -u $your_ht_user, so not much to remember for the simple call.

  • The help-screen will guide you through your various queries:

  nc-apps - version 2023.08.08 19:00 (latest version)

  Usage:
    nc-apps -h|--help
    nc-apps -H|--filters
    nc-apps appstore [ jq filter code ]
    nc-apps categories [ jq filter code ]
    nc-apps install_overwrite [ -u [ --allow-unstable ]]
    nc-apps shipped [ *'section' ] [ --nopager ]
    nc-apps not_shipped [ *'section' ] [ version=(0-99) ] [ --nopager ]
    nc-apps ns_supported [ *'section' ] [ version=(0-99) ] [ --nopager ]
    nc-apps ns_unsupported [ *'section' ] [ version=(0-99) ] [ --nopager ]
    nc-apps all_apps [ *'section' ] [ version=(0-99) ] [ --nopager ]
    nc-apps id=[pattern]
    nc-apps name=[pattern]
    nc-apps lang=[ISO 639-1]
    nc-apps updatable [ *'section' ] [ version=(0-99) ]
    nc-apps update [ *'section' ] [ --allow-unstable ]
    nc-apps verify_my_integrity

  Options:

    (* 'section' can be eather one of:
                            enabled  - installed and enabled apps (default)
                            disabled - installed but disabled apps
                            local    - enabled and disabled combined
                            store    - not installed, available in appstore
                            all      - all sections

    -h | --help             this page

    -H | --filters          examples of JSON filters to use with 'appstore' and
                            'categories' options

    appstore                get filtered appstore as JSON object.
                            See -H|--filters for examples

    categories              get app-categories as JSON object.
                            See -H|--filters for examples

    install_overwrite [ -u [ --allow-unstable ]]
                            write actual 'app_install_overwrite' array inn
                            config/*config.php file. When invoked with -u, it
                            will first perform an update of al installed apps.
                            This script supports multiple *.config.php files.

    shipped [ *'section' ]  list shipped apps (no store section)

    not_shipped [ *'section' ]
                            list not shipped apps

    ns_supported [ *'section' ]
                            list not shipped, supported apps Âą)

    ns_unsupported [ *'section' ]
                            list not shipped, not suported apps

    all_apps [ *'section' ] list all apps

    name=[pattern]          list apps where name matches "pattern"

    id=[pattern]            list apps where id matches "pattern"

    lang=[ISO 639-1]        language to be used for translations instead of the
                            detected locale on your computer, which is "de"

    updatable [ *'section' ]
                            list updatable apps (no store section)

    update [ *'section' ]   update enabled, disabled or local apps

    --allow-unstable        allow updating to unstable releases (only for up-
                            date and install_overwrite with option -u)

    --nopager               dont use pager (only for long listing functions)

    version=(0-99)          simulates an assumed nextcloud version instead of
                            actual installed real nextcloud version.
                            ('version=' has no effect on install_overwrite,
                            shipped and update)

    verify_my_integrity     verify the integrity of this script with signature

 Âą) In this context, "supported" means that the app in its present version is
    intended for the installed (or simulated) Nextcloud version.

Enjoy learning more about your apps.

New in version:

  • 2023.06.25 03:35:

    • Added the ability to list apps searched by name- or id- pattern
    • Added Code-Signing:
      • Run nc_apps verify_my_integrity to compare the file hash and verify the gpg signature to be sure that you don’t run a script that has been tampered with.

  • 2023.06.26 17:20:

    • Improved codesigning
    • Added colors
    • Minor bug fixes

  • 2023.06.28 14:49:

    • Hardcoded use of less instead of pager to enable colors in pager view
    • Improved color view
    • Minor bug fixes

  • 2023.06.29 20:00

    • Auto-update with auto-backup: The script can now safely update itself. Backups are created that can be restored.

  • 2023.07.02 00:40

    • Improvements and bug fixes

  • 2023.08.07 17:30

    • Added appstore integration
    • json output
    • new options:
      • lang=[ISO 639-1] - to overwrite your systems locale
      • appstore
      • categories
      • -H | --filters - an extra help page with example json-filters for fast and detailed output

  • 2023.08.08 19:99

    • Dramatically increase of speed for large file listings from appstore
    • Bug fixes
6 Likes
2

That is so useful! Thank you for sharing.

I’m always a bit worried about downloading an executable this way. Would you mind providing a hash of the executable so we can verify we downloaded the correct script? I can read bash but it’s long enough that a review is probably not enough to spot something is wrong.

Being able to simulate a version bump is a killer feature.

1 Like
3

I made an update with an integrated integrity check, the code now comes gpg-signed.

1 Like
4

I had no idea you could make it self signed, that’s pretty neat.

1 Like
5

I’ve greatly expanded the script in the meantime. Now it includes an integrated appstore browser/analyzer.

Since the beautified output is very slow with large masses of apps (currently 389 apps are listed in the Appstore), I added a (much faster) interface that outputs pure json.

The apps.json from apps.nextcloud.com, stored and maintained in data/appdata_$instanceid/appstore/ is used as the data source and filtered to the extent that only the translation of your locale (or the one passed as an argument) is displayed and only the latest version of an app.

I have created a help output that provides the most common filters as examples, so that not only command line junkies can get by with them:

  # Query single app "appname" by id:
    appstore '.[] | select(.id == "appname")'

  # Field contains (int)
    appstore '.[] | select(.phpMinIntSize | contains(64))'

  # Field contains ("string")
    appstore '.[] | select(.id | contains("string"))'

  # Field contains ("string") Case insensitive!
    appstore '.[] | select(.name | ascii_downcase | contains("string"))'

  # Field equals "string"
    appstore map(select(.id == "string"))

  # All apps where NCmax is greater than "27"
    appstore 'map(select(.NCmax > "27"))'

  # Apps with NCmax less than "26" and NCmin greater than "20":
    appstore 'map(select(.NCmax < "26" and .NCmin > "20"))'

  # All apps where phpMax is less than 8.2:
    appstore 'map(select(.phpMax < "8.2"))'

  # All apps where phpMax is less than 8.2 and phpMax is not empty:
    appstore 'map(select(.phpMax < "8.2" and (.phpMax | length > 0)))'

  # All apps where phpMin is greater than 7.0 and phpMin is not empty:
    appstore 'map(select(.phpMin > "7.0" and (.phpMin | length > 0)))'

  # Apps with phpMax less than 8.2 and phpMax is not empty, and phpMin greater than 7.0 and phpMin is not empty:
    appstore 'map(select(.phpMax < "8.2" and (.phpMax | length > 0) and .phpMin > "7.0" and (.phpMin | length > 0)))'

  # Apps with phpMax less than 8.2 or phpMin greater than 7.0 and either phpMax or phpMin is not empty:
    appstore 'map(select((.phpMax < "8.2" or .phpMin > "7.0") and ((.phpMax | length > 0) or (.phpMin | length > 0))))'

  # Apps with phpMax less than 8.2 and phpMin greater than 7.0, and both phpMax and phpMin are not empty:
    appstore 'map(select(.phpMax < "8.2" and .phpMin > "7.0" and (.phpMax | length > 0) and (.phpMin | length > 0)))'

  # Apps with a specific pattern in the name or id:
    appstore 'map(select(.name | test("pattern")))'
    appstore 'map(select(.id | test("pattern")))'

  # Apps with specific categories:
    appstore 'map(select(.categories | contains("category1") or contains("category2")))'

  # Apps with NCmax less than 26 and NCmin greater than 20, and phpMax less than 8.2:
    appstore 'map(select(.NCmax < "26" and .NCmin > "20" and .phpMax < "8.2"))'

  # phpExtensions is not empty
    appstore 'map(select(.phpExtensions | length > 0))'

  # Show only selected fields (id, name, and NCmax) for each app:
    appstore 'map({id, name, NCmax})'

  # in Combination
    appstore 'map(select(.NCmax > "26" and .NCmin > "20" and .phpMax > "8.1" and (.phpMax | length > 0) and (.phpExtensions | length > 0))) | map({id, name, NCmax})'

  # Sort by last modified, oldest first
    appstore 'sort_by(.lastModified)'

  # newest first
    appstore 'sort_by(.lastModified) | reverse'

  # in Combination
    appstore 'map(select(.NCmax > "26" and .NCmin > "20" and .phpMax < "8.2")) | sort_by(.lastModified)'

  # Apps created after January 1, 2023:
    appstore 'map(select(.created > "2023-01-01"))'

  # Apps last modified after January 1, 2023:
    appstore 'map(select(.lastModified > "2023-01-01"))'

  # Apps created before January 1, 2023:
    appstore 'map(select(.created < "2023-01-01"))'

  # Apps last modified before January 1, 2023:
    appstore 'map(select(.lastModified < "2023-01-01"))'

  # Apps that haven't been modified after January 1, 2023 (possibly "stale" apps):
    appstore 'map(select(.lastModified < "2023-01-01"))'

  # Apps created before January 1, 2023 and haven't been modified after that date (potentially "stale" apps):
    appstore 'map(select(.created < "2023-01-01" and .lastModified < "2023-01-01"))'

Professionals who want a json object to work with get pure (presorted) validated json with this script.

Besides updating apps and maintaining a custom install_overwrite array in my config.php (fully automatic), searching for single app informations like website or issue-tracker with the options name=“pattern” or id=“pattern” has proven to be extremely helpful in day-to-day use.