Update to 13 fails - core/shipped.json is not available

AlexH · February 11, 2018, 1:56pm

Please help!

2018-02-11T14:12:20+0100 kFO3AJxF7m [info] rollbackChanges(“9”)
2018-02-11T14:12:20+0100 kFO3AJxF7m [info] unlink .step
2018-02-11T14:12:20+0100 kFO3AJxF7m [info] rollbackChanges - step >= 7
2018-02-11T14:12:20+0100 kFO3AJxF7m [info] end of rollbackChanges()
2018-02-11T14:15:08+0100 h0hwSEUwZj [info] request to updater
2018-02-11T14:15:08+0100 h0hwSEUwZj [info] currentStep()
2018-02-11T14:15:08+0100 h0hwSEUwZj [info] POST request for step "9"
2018-02-11T14:15:08+0100 h0hwSEUwZj [info] startStep(“9”)
2018-02-11T14:15:08+0100 h0hwSEUwZj [info] deleteOldFiles()
2018-02-11T14:15:08+0100 h0hwSEUwZj [error] POST request failed with other exception
2018-02-11T14:15:08+0100 h0hwSEUwZj [error] Exception: Exception
Message: core/shipped.json is not available
Code:0
Trace:
#0 /homepages/28/d19879336/htdocs/nextcloud/updater/index.php(1309): Updater->deleteOldFiles()
#1 {main}
File:/homepages/28/d19879336/htdocs/nextcloud/updater/index.php
Line:864

gas85 · February 13, 2018, 9:21am

Got the same Problem, but via Web-UI.
First of all it was “Delete old files Parsing response failed.”
Then “Step 9 is currently in process”
And now this.

Retry Button does not help…

Some Logs:
2018-02-13T10:15:36+0100 rOOaN1U9yf [info] rollbackChanges(“9”)
2018-02-13T10:15:36+0100 rOOaN1U9yf [info] unlink .step
2018-02-13T10:15:36+0100 rOOaN1U9yf [info] rollbackChanges - step >= 7
2018-02-13T10:15:36+0100 rOOaN1U9yf [info] end of rollbackChanges()
2018-02-13T10:20:16+0100 6ptsavTnPN [info] request to updater
2018-02-13T10:20:16+0100 6ptsavTnPN [info] currentStep()
2018-02-13T10:20:16+0100 6ptsavTnPN [info] POST request for step “9”
2018-02-13T10:20:16+0100 6ptsavTnPN [info] startStep(“9”)
2018-02-13T10:20:16+0100 6ptsavTnPN [info] deleteOldFiles()
2018-02-13T10:20:16+0100 6ptsavTnPN [error] POST request failed with other exception
2018-02-13T10:20:16+0100 6ptsavTnPN [error] Exception: Exception
Message: core/shipped.json is not available
Code:0
Trace:
#0 /var/www/nextcloud/updater/index.php(1309): Updater->deleteOldFiles()
#1 {main}
File:/var/www/nextcloud/updater/index.php
Line:864

2018-02-13T10:20:16+0100 6ptsavTnPN [info] rollbackChanges(“9”)
2018-02-13T10:20:16+0100 6ptsavTnPN [info] unlink .step
2018-02-13T10:20:16+0100 6ptsavTnPN [info] rollbackChanges - step >= 7
2018-02-13T10:20:16+0100 6ptsavTnPN [info] end of rollbackChanges()
2018-02-13T10:21:26+0100 xVBxJf6LFb [info] request to updater
2018-02-13T10:21:26+0100 xVBxJf6LFb [info] currentStep()
2018-02-13T10:21:26+0100 xVBxJf6LFb [info] POST request for step “9”
2018-02-13T10:21:26+0100 xVBxJf6LFb [info] startStep(“9”)
2018-02-13T10:21:26+0100 xVBxJf6LFb [info] deleteOldFiles()
2018-02-13T10:21:26+0100 xVBxJf6LFb [error] POST request failed with other exception
2018-02-13T10:21:26+0100 xVBxJf6LFb [error] Exception: Exception
Message: core/shipped.json is not available
Code:0
Trace:
#0 /var/www/nextcloud/updater/index.php(1309): Updater->deleteOldFiles()
#1 {main}
File:/var/www/nextcloud/updater/index.php
Line:864

2018-02-13T10:21:26+0100 xVBxJf6LFb [info] rollbackChanges(“9”)
2018-02-13T10:21:26+0100 xVBxJf6LFb [info] unlink .step
2018-02-13T10:21:26+0100 xVBxJf6LFb [info] rollbackChanges - step >= 7
2018-02-13T10:21:26+0100 xVBxJf6LFb [info] end of rollbackChanges()
2018-02-13T10:23:21+0100 zQWfsQ3eBc [info] request to updater
2018-02-13T10:23:21+0100 zQWfsQ3eBc [info] currentStep()
2018-02-13T10:23:21+0100 zQWfsQ3eBc [info] POST request for step “9”
2018-02-13T10:23:22+0100 zQWfsQ3eBc [info] startStep(“9”)
2018-02-13T10:23:22+0100 zQWfsQ3eBc [info] deleteOldFiles()
2018-02-13T10:23:22+0100 zQWfsQ3eBc [error] POST request failed with other exception
2018-02-13T10:23:22+0100 zQWfsQ3eBc [error] Exception: Exception
Message: core/shipped.json is not available
Code:0
Trace:
#0 /var/www/nextcloud/updater/index.php(1309): Updater->deleteOldFiles()
#1 {main}
File:/var/www/nextcloud/updater/index.php
Line:864

2018-02-13T10:23:22+0100 zQWfsQ3eBc [info] rollbackChanges(“9”)
2018-02-13T10:23:22+0100 zQWfsQ3eBc [info] unlink .step
2018-02-13T10:23:22+0100 zQWfsQ3eBc [info] rollbackChanges - step >= 7
2018-02-13T10:23:22+0100 zQWfsQ3eBc [info] end of rollbackChanges()
2018-02-13T10:23:26+0100 jfISe5DC1K [info] request to updater
2018-02-13T10:23:26+0100 jfISe5DC1K [info] currentStep()
2018-02-13T10:23:26+0100 jfISe5DC1K [info] POST request for step “9”
2018-02-13T10:23:26+0100 jfISe5DC1K [info] startStep(“9”)
2018-02-13T10:23:26+0100 jfISe5DC1K [info] deleteOldFiles()
2018-02-13T10:23:26+0100 jfISe5DC1K [error] POST request failed with other exception
2018-02-13T10:23:26+0100 jfISe5DC1K [error] Exception: Exception
Message: core/shipped.json is not available
Code:0
Trace:
#0 /var/www/nextcloud/updater/index.php(1309): Updater->deleteOldFiles()
#1 {main}
File:/var/www/nextcloud/updater/index.php
Line:864

2018-02-13T10:23:26+0100 jfISe5DC1K [info] rollbackChanges(“9”)
2018-02-13T10:23:26+0100 jfISe5DC1K [info] unlink .step
2018-02-13T10:23:26+0100 jfISe5DC1K [info] rollbackChanges - step >= 7
2018-02-13T10:23:26+0100 jfISe5DC1K [info] end of rollbackChanges()

gas85 · February 13, 2018, 9:34am

Update:
The issue is that all files have being deleted already:

ls -lah
total 44K
drwxr-xr-x  7 www-data www-data 4.0K Feb 13 10:12 .
drwxr-xr-x  5 root     root     4.0K Jan 23 12:49 ..
drwxr-xr-x 20 www-data www-data 4.0K Feb 13 10:00 apps
drwxr-xr-x  2 www-data www-data 4.0K Feb 13 10:00 config
drwxr-x--- 12 www-data www-data 4.0K Feb  5 06:25 data
-rw-r--r--  1 www-data www-data   57 Feb 13 10:00 index.php
-rw-r--r--  1 www-data www-data   57 Feb 13 10:00 public.php
-rw-r--r--  1 www-data www-data   57 Feb 13 10:00 remote.php
-rw-r--r--  1 www-data www-data   57 Feb 13 10:00 status.php
drwxr-xr-x  2 www-data www-data 4.0K Feb 13 10:00 themes
drwxr-xr-x  2 www-data www-data 4.0K Jan 25 14:03 updater

How to jump to next step?

Can I simply move files from …data/updater-xxxxxx/downloads/nextcloud/. do nextcloud root end disable maintenance mode?

gas85 · February 13, 2018, 12:25pm

Looks I solve it:

go to your nextcloud installation path, for me it is /var/www/nextcloud and my data user is www-data

/var/www/nextcloud
if you will see something like this, proceed to step 3:

gas85:

ls -lah
total 44K
drwxr-xr-x  7 www-data www-data 4.0K Feb 13 10:12 .
drwxr-xr-x  5 root     root     4.0K Jan 23 12:49 …
drwxr-xr-x 20 www-data www-data 4.0K Feb 13 10:00 apps
drwxr-xr-x  2 www-data www-data 4.0K Feb 13 10:00 config
drwxr-x— 12 www-data www-data 4.0K Feb  5 06:25 data
-rw-r–r--  1 www-data www-data   57 Feb 13 10:00 index.php
-rw-r–r--  1 www-data www-data   57 Feb 13 10:00 public.php
-rw-r–r--  1 www-data www-data   57 Feb 13 10:00 remote.php
-rw-r–r--  1 www-data www-data   57 Feb 13 10:00 status.php
drwxr-xr-x  2 www-data www-data 4.0K Feb 13 10:00 themes
drwxr-xr-x  2 www-data www-data 4.0K Jan 25 14:03 updater

Copy all files from Nextcloud update downloaded by Web-Updater to actual nextcloud folder

cp -rf data/updater-XXXXXXX/downloads/nextcloud/ /var/www/

(it will be copied into /var/www/nextcloud)
for me it was needed to set correct permissions by running

chown -R www-data:www-data *
then I run to start upgrade:

sudo -u www-data php occ upgrade --no-app-disable --no-interaction

--no-app-disable - Because I have Problem with apps and do not want to remove/disable them - It is ususally not needed.

--no-interaction - Beacuse I do not want to answer questions…

…Checking for update of app workflowengine in appstore
Checked for update of app “workflowengine” in appstore
Starting code integrity check…
Finished code integrity check
Update successful
Maintenance mode is kept active
Reset log level
Disable maintenance mode

sudo -u www-data php occ maintenance:mode --off
Its alive!!!

Or everything in 1 command for /var/www/nextcloud installation and do not forget to replace XXXXXX with your NC identifier:
sudo -u www-data -s -- 'cd /var/www/nextcloud && cp -rf data/updater-XXXXXX/downloads/nextcloud/ /var/www/ && php occ upgrade --no-interaction && php occ maintenance:mode --off'

P.S. I have only 2 errors: one is Cron job was executed long time ago and second one is some files inconsistency:
Results
=======
- core
- INVALID_HASH
- .htaccess
- FILE_MISSING
- .user.ini

But this is easy to solve.

Solution: go to your download folder and copy files to nextcloud root:

cd data/updater-XXXXXXXXX/downloads/nextcloud/
sudo -u www-data cp .htaccess /var/www/nextcloud
sudo -u www-data cp .user.ini /var/www/nextcloud

Here we learned how to rescan htaccess:

sudo -u www-data php occ maintenance:update:htaccess

AlexH · February 14, 2018, 3:34pm

Hi and thank you!
Is there a solution wihout using shell commands?
Can i somehow leave this update mode and let the installer do the rest?

When try to open the UI, i geht “Update in process.”

Thanks

Alex

gas85 · February 15, 2018, 7:50am

I try few different scenario before to end up with this. My goal was not to restore (I did it before already) but proceed with update:

I try to restore my 12 version from the …data/updater-XXXXXXX/backup/nextcloud-12xxxx/ but if you reach it via https you will see the message that upgrade is ongoing. You need to set some occ commands any way to bring it back, but at the end it stucks in the same step 9.
I try to copy data/updater-XXXXXXX/downloads/nextcloud/ to the folder and in the web I can see the same step stuck - deletion. If I proceed I end up with the same empty folder and stuck in this step 9.

Basically post above is only one procedure how to work around for me. I do not thing that if you live it as it is it will brings you some happiness, besides somebody will come and confirm opposite .

Everything was already done by the installer, except copy new version into correct folder. After this you need to run occ upgrade from the terminal, otherwise you will stuck with scenario 2 from above when everything was deleted again.

AlexH · February 15, 2018, 11:04am

Thank you!
I coped the files from the update using my ftp-software.
Then i startet the UI.
The UI went to a specal update mode updating all addins.
After that was redirected to login.

gas85 · February 15, 2018, 12:13pm

Glad for you! Have done the same, but got another error that some apps were incomparable with NC13 and could not be deleted or disabled, this was the reason why I run command occ upgrade --no-app-disable to tell updater to not touch my apps.

Schmu · February 15, 2018, 12:28pm

Especially with NC13 this is a bad idea. There are some issues with older apps from NC12 and these apps should better be disabled.
Furthermore the update will update many apps during the update process. Some apps have a version for NC12 and a version for NC13.

My suggestion: disable most of your apps, run the update, run the app updates and re-enable the apps. In my opinion that is the best way to avoid upgrade issues.

gas85 · February 15, 2018, 12:41pm

I did manual App clean-up after install. Some Apps were not enabled (this is good), but exactly they were incompatible with NC13. I delete them and did manual update of the rest of Apps. Otherwise Web Updater cause an error and do not want to proceed. Do not know why it could not delete or disable some apps, I did not noticed what exactly App was causing the issue, but for me it was easy solution to do like this.

xaos · May 16, 2018, 8:11pm

Hey I made an account just to tell whoever else has this issue how I solved it without using occ, very easily:

Just go to your nextcloud installation path, create the folder core and in that folder create an empty file called shipped.json.
Then go back to the installer web-page and hit retry!

That did it for me so good luck!

Schmu · May 16, 2018, 9:34pm

@xaos Thanks a lot for sharing your solution!

gas85 · July 25, 2018, 7:40am

Update to 13.05… Still the same Problem

I did as you mentioned inside of the nextcloud folder (www-data is my nextcloud user):

sudo -u www-data mkdir core
sudo -u www-data touch core/shipped.json

and hit retry… have the same error after while core folder with json file were successfully deleted… It does not help for me Only post from above works…

fume · July 25, 2018, 9:31am

Ich had the same problems with 13.0.4 to 13.0.5. Update with the built-in updater was not possible. After a few tries I did the update manually. That worked for me.

Michael_F_Peintinger · September 11, 2018, 2:29pm

I had the same problem when updating to version 14.0.0 from 14.0.0 beta 2.
The problem is that the file is already deleted but that is not properly reported.

Here’s what I did on the command line as root:
mkdir core
touch core/shipped.json
chown -R : core

Then hit the reload button.

AGabriel · October 26, 2018, 8:28am

Hi gas85,
did your procedure but at point 8.
PHP Fatal error: require_once(): Failed opening required '/var/www/nextcloud/lib/private/Console/…/…/…/core/register_command.php’

so … I try to figure out how to correct this, any idea ?

gas85 · October 26, 2018, 2:54pm

Hi, to be honest, I did not meet this error. Try to check this topic https://help.nextcloud.com/t/upgrade-to-13-02-completely-failed-trying-to-get-nextcloud-back-working/30994
Issue was made by an error in unpacking NC tar file.

26 октября 2018 г. 10:38:42 GMT+02:00, Gabriel noreply@nextcloud.com пишет:

condoratberlin · November 5, 2018, 9:15pm

I followed your approach and solved the issue. Some directories had some wrong permissions and i had to change the ownership to www of all folders. Then everything worked.

Saeed_Sepehr · January 29, 2019, 7:23pm

thanks a lot. I’ve got my missed files back from the downloaded directory withing the Data directory. I also tried the chrown for www-data with * in the root of nextcloud installation. it works!

Bera · April 12, 2021, 7:02pm

This should be a sticky. Thank you very much very helpful.