Adding a unique user agent string for GuzzleHttp requests in Calendar app (feat. Songkick case study)

Hi folks!

This is my first post in the Nextcloud forums: long-time user, first time contributor here :wave:

I have an issue which I would like fixed and I was hopeful that with some community guidance I might find the best way to go about that. Please let me know if there is a more appropriate place for this too. Here’s the summary:

I’m running a Nextcloud installation (version 14.0.3) with the calendar app installed (v1.6.3) and most things are tickity boo. One of the webcal subscriptions I had was with the Songkick calendar. (For context, Songkick [https://www.songkick.com] is a simple app which tracks bands and artists and provides a calendar of tour dates.)

A few months ago the calendar stopped syncing. I removed the subscription and attempted to re-add it and was greeted with the following message from Nextcloud: “The remote server did not give us access to the calendar (HTTP {403} error)”

nextcloud-calendar-songkick-error

It didn’t make much sense to me that this had suddenly stopped working so I reached out to Songkick to see if anything had changed. A few email exchanges later I received a super detailed email from Sam on their engineering team telling me that Nextcloud had been blocked by the Songkick bot protection software.

I’m Sam I work on the engineering team at Songkick. The high level overview is that it looks like NextCloud talks to us using the user agent “GuzzleHttp/6.2.1 curl/7.47.0 PHP/7.2.10-1+ubuntu16.04.1+deb.sury.org+1” which our firewall and bot protection software has blocked as problematic.

Sam helpfully went on to give a detailed description of the problem: Nextcloud uses the default user agent string when making requests via the GuzzleHttp library, and since this user agent string is prevalent in a lot of bad scripts it has been blocked from making requests. He also suggested a solution, that Nextcloud override this default to identify itself and thus distance itself from those bad scripts using default settings.

Here are the relevant parts of Sam’s email of how to start fixing this:

Unfortunately at the moment we’re not able to update this logic to allow NextCloud without allowing all the bad scripts. If you’re feeling adventurous I believe the code changes would need to be in here https://github.com/nextcloud/calendar/blob/e48958cac396cdffeb1442246eb8372069042829/controller/proxycontroller.php#L88 updating it to use a custom User-Agent header http://docs.guzzlephp.org/en/stable/request-options.html#headers.

I was hoping that someone here could confirm this scenario is the most likely cause of my issue, and also confirm that Sam has suggested a good approach to fixing the problem. I am happy to open a Git issue on the Calendar app repository if I know that I’m at least facing the right direction.

Thanks for any help you can give! :pray:

I’m having the same problem, I’ve opened a PR here https://github.com/nextcloud/calendar/pull/1503 let’s see if it’s acceptable

1 Like

This has been resolved now. Thanks everyone who helped!