# Having a CI setup

**URL:** <https://help.nextcloud.com/t/having-a-ci-setup/223>\
**Category:** 🔮 Meta\
**Created:** [June 3, 2016, 7:58am UTC](https://help.nextcloud.com/t/having-a-ci-setup/223 "2016-06-03T07:58:27Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![MorrisJobke](https://help.nextcloud.com/user_avatar/help.nextcloud.com/morrisjobke/32/19_2.png) [@MorrisJobke](https://help.nextcloud.com/u/MorrisJobke)\
**Post date:** [June 3, 2016, 7:58am UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/1 "2016-06-03T07:58:27Z")

</div>

Continuing the discussion from [[Contribution] Infra Team](https://help.nextcloud.com/t/contribution-infra-team/136/12):

> [@\[Contribution\] Infra Team](https://help.nextcloud.com/t/contribution-infra-team/136/12):
>
> We should not forget about CI. In the past we had a Jenkins with multiple slaves. I think that because of the size we have in this project we should host the CI system ourselves.
> 
> We also should do this as fast as possible, because it is crucial for starting the development. I also volunteer to build a team to bootstrap this. I think @LukasReschke and @jyaworski would very likely help here out too 😉

Should we give a Jenkins 2 a try and set it up? How could we make this lasting? We most likely want some way of automation to bootstrap a new CI slave.

How could we somehow distribute the responsibility of this? We definitely multiple people who could help out fixing the machines.

What else comes to your mind?

cc @stp @elacheche

---

<div class="post-metadata">

**Author:** ![elacheche](https://help.nextcloud.com/user_avatar/help.nextcloud.com/elacheche/32/58_2.png) [@elacheche](https://help.nextcloud.com/u/elacheche)\
**Post date:** [June 3, 2016, 9:03am UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/2 "2016-06-03T09:03:03Z")

</div>

Hey!

Great idea, I agree that it’s a must to setup a CI server asap, as the devs will start contributing asap too…

I’d like to help/contribute…

> How could we somehow distribute the responsibility of this? We definitely multiple people who could help out fixing the machines.

I think that for now, someone should lead the team, he’ll be responsable for pushing things to servers… The rest of the team will be push changes to a git repo with installation instructions (Ansible/Puppet/Chef/Salt???), etc…

---

<div class="post-metadata">

**Author:** ![MorrisJobke](https://help.nextcloud.com/user_avatar/help.nextcloud.com/morrisjobke/32/19_2.png) [@MorrisJobke](https://help.nextcloud.com/u/MorrisJobke)\
**Post date:** [June 3, 2016, 9:04am UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/3 "2016-06-03T09:04:16Z")

</div>

> [@elacheche](#):
>
> (Ansible/Puppet/Chef/Salt???),

We had in the past good experiences with Ansible. I would go for that.

---

<div class="post-metadata">

**Author:** ![elacheche](https://help.nextcloud.com/user_avatar/help.nextcloud.com/elacheche/32/58_2.png) [@elacheche](https://help.nextcloud.com/u/elacheche)\
**Post date:** [June 3, 2016, 4:26pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/4 "2016-06-03T16:26:11Z")

</div>

+1 for Ansible too…

@MorrisJobke wanna lead the team? Or someoen else should do that?

If so, I suggest to start with creating a repo on the infre team on Github and creating some issues?! So I (and others) can contribute when ever there is a new issue.

---

<div class="post-metadata">

**Author:** ![MorrisJobke](https://help.nextcloud.com/user_avatar/help.nextcloud.com/morrisjobke/32/19_2.png) [@MorrisJobke](https://help.nextcloud.com/u/MorrisJobke)\
**Post date:** [June 3, 2016, 9:47pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/5 "2016-06-03T21:47:13Z")

</div>

> [@elacheche](#):
>
> +1 for Ansible too…
> 
> @MorrisJobke wanna lead the team? Or someoen else should do that?
> 
> If so, I suggest to start with creating a repo on the infre team on Github and creating some issues?! So I (and others) can contribute when ever there is a new issue.

I could lead it. Today I played a bit and set up a Jenkins server. I also tried [drone.io](http://drone.io) as a self-hosted setup and it looks quite good. I will share some ansible stuff once I have it 🙈

---

<div class="post-metadata">

**Author:** ![MorrisJobke](https://help.nextcloud.com/user_avatar/help.nextcloud.com/morrisjobke/32/19_2.png) [@MorrisJobke](https://help.nextcloud.com/u/MorrisJobke)\
**Post date:** [June 3, 2016, 9:50pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/6 "2016-06-03T21:50:10Z")

</div>

> [@elacheche](#):
>
> If so, I suggest to start with creating a repo on the infre team on Github and creating some issues?!

[https://github.com/nextcloud/infrastructure](https://github.com/nextcloud/infrastructure)

---

<div class="post-metadata">

**Author:** ![stp](https://help.nextcloud.com/user_avatar/help.nextcloud.com/stp/32/5652_2.png) [@stp](https://help.nextcloud.com/u/stp)\
**Post date:** [June 4, 2016, 1:40am UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/7 "2016-06-04T01:40:25Z")

</div>

As a sidenote I would suggest subdirs of the type: [ci.owncloud.org](http://ci.owncloud.org) or so. That each directory basically (in infrastructure) shows what it is for.

---

<div class="post-metadata">

**Author:** ![elacheche](https://help.nextcloud.com/user_avatar/help.nextcloud.com/elacheche/32/58_2.png) [@elacheche](https://help.nextcloud.com/u/elacheche)\
**Post date:** [June 4, 2016, 9:34pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/8 "2016-06-04T21:34:15Z")

</div>

> [@MorrisJobke](#):
>
> I could lead it.

Great! Let’s rock! 😃

> [@MorrisJobke](#):
>
> Today I played a bit and set up a Jenkins server. I also tried [drone.io](http://drone.io) as a self-hosted setup and it looks quite good.

How can I help on Jenkins?  
[Drone.io](http://Drone.io) isn’t a saas!

> [@MorrisJobke](#):
>
> I will share some ansible stuff once I have it

You can start by creating some issues, and I’ll try to help creating the ansible playbooks, I’m not an expert on that, but I created some before, I’ll be happy to practice more 🙂

---

<div class="post-metadata">

**Author:** ![orion1024](https://help.nextcloud.com/user_avatar/help.nextcloud.com/orion1024/32/69_2.png) [@orion1024](https://help.nextcloud.com/u/orion1024)\
**Post date:** [June 4, 2016, 9:41pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/9 "2016-06-04T21:41:27Z")

</div>

Do you plan on reusing the existing automated tests ?  
Such as smashbox for server/client, perl scripts for client, and so on.

The CI server is critical if you want to merge PR fast and still maintain code reliability.  
I’m not sure for core, but afaik for client no automated tests were run on PR (only on daily builds).  
Running those tests on PR would be ideal.

---

<div class="post-metadata">

**Author:** ![MorrisJobke](https://help.nextcloud.com/user_avatar/help.nextcloud.com/morrisjobke/32/19_2.png) [@MorrisJobke](https://help.nextcloud.com/u/MorrisJobke)\
**Post date:** [June 5, 2016, 8:15pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/10 "2016-06-05T20:15:35Z")

</div>

> [@elacheche](#):
>
> [Drone.io](http://Drone.io) isn’t a saas!

I also noticed this and gave it a try today. Looks promising.

> [@orion1024](#):
>
> Such as smashbox for server/client, perl scripts for client, and so on.

Of course. First of all the `autotest.sh` scripts need to run there. We could then add more.

I would go for drone, because then the configuration is put into the git repo and not inside the CI setup. This would then make it easier to create new task from the developers.

---

<div class="post-metadata">

**Author:** ![MorrisJobke](https://help.nextcloud.com/user_avatar/help.nextcloud.com/morrisjobke/32/19_2.png) [@MorrisJobke](https://help.nextcloud.com/u/MorrisJobke)\
**Post date:** [June 6, 2016, 8:27pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/11 "2016-06-06T20:27:13Z")

</div>

Today we worked on a first setup of a minimal CI instance. The ansible scripts can be found at [https://github.com/nextcloud/infrastructure/tree/master/ci](https://github.com/nextcloud/infrastructure/tree/master/ci)

This instance runs at [https://drone.weasel.rocks/nextcloud/server](https://drone.weasel.rocks/nextcloud/server) (link to the CI setup for the server, because there is no public overview page).

Now the task is to move from our previous setup to a dockerized one. I will soon open a PR in core to run the first tests (`autotest.sh sqlite` in our server repository) and then we can steadily extend this and play around with all the matrix stuff (multiple PHP versions, DBs, tests, …).

A quick preview:

I used this docker file to generate the `php-7-owncloud` image:

```auto
FROM php:7
RUN apt-get update && apt-get install -y \
        libfreetype6-dev \
        libjpeg62-turbo-dev \
        libpng12-dev \
        git \
    && docker-php-ext-configure gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ \
    && docker-php-ext-install gd zip \
    && docker-php-ext-enable gd zip
RUN curl -O -L https://phar.phpunit.de/phpunit.phar \
	&& chmod +x phpunit.phar \
	&& mv phpunit.phar /usr/local/bin/phpunit

```

And following `.drone.yml` that is placed in the server repository:

```auto
build:
  image: php-7-owncloud
  commands:
    - ./occ maintenance:install --database-name oc_autotest --database-user oc_autotest --admin-user admin --admin-pass admin --database sqlite --database-pass=''
    - ./autotest.sh sqlite

```

It currently fails due to a too low memory limit, but it is a nice start. I also opened this on Github as a issue we need to fix as soon as possible in the server repo: [https://github.com/nextcloud/server/issues/14](https://github.com/nextcloud/server/issues/14)

---

<div class="post-metadata">

**Author:** ![orion1024](https://help.nextcloud.com/user_avatar/help.nextcloud.com/orion1024/32/69_2.png) [@orion1024](https://help.nextcloud.com/u/orion1024)\
**Post date:** [June 6, 2016, 9:16pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/12 "2016-06-06T21:16:32Z")

</div>

I quickly looked at what drone does and it seems the feature set is very small compared to Jenkins.  
The following items in particular raise concerns :

- php support is in beta according to the [docs](http://docs.drone.io/newproject.html) (see in the right menu)
- no support for objective C (IOS app)
- no support for Windows builds
- no support for pull requests builds, given that one of the goal is to merge PR faster ; and for this automated builds on PRs are a very important tool

---

<div class="post-metadata">

**Author:** ![MorrisJobke](https://help.nextcloud.com/user_avatar/help.nextcloud.com/morrisjobke/32/19_2.png) [@MorrisJobke](https://help.nextcloud.com/u/MorrisJobke)\
**Post date:** [June 6, 2016, 9:24pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/13 "2016-06-06T21:24:51Z")

</div>

> [@orion1024](#):
>
> php support is in beta according to the docs (see in the right menu)

It works just fine. My local tests just run the tests in a PHP docker image. Clearly not an issue

> [@orion1024](#):
>
> no support for objective C (IOS app)

Not needed for the server component here. We could use something different here, but the vast majority of work is done via server CI - so this needs to be working quite robust.

> [@orion1024](#):
>
> no support for Windows builds

Also not needed.

> [@orion1024](#):
>
> no support for pull requests builds, given that one of the goal is to merge PR faster ; and for this automated builds on PRs are a very important tool

Works definitely. I already tested this and it shows up nicely in the GitHub details for each PR and just runs the tests on my local instance.

---

<div class="post-metadata">

**Author:** ![orion1024](https://help.nextcloud.com/user_avatar/help.nextcloud.com/orion1024/32/69_2.png) [@orion1024](https://help.nextcloud.com/u/orion1024)\
**Post date:** [June 6, 2016, 9:35pm UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/14 "2016-06-06T21:35:45Z")

</div>

If your goal is to only provide CI integration for the server component, then drone may be a good answer.  
But isn’t there value in providing the same CI tool for server and client components ?  
Aside from the obvious advantage of having a single pane of glass for every test, there may be some synergies to be had later down the line.  
I saw that you quoted the ability to put _everything CI_ in a git repo ; do you mean that what is being done with smashbox (which is hosted in a repo) and Jenkins at ownCloud is not enough ?

---

<div class="post-metadata">

**Author:** ![MorrisJobke](https://help.nextcloud.com/user_avatar/help.nextcloud.com/morrisjobke/32/19_2.png) [@MorrisJobke](https://help.nextcloud.com/u/MorrisJobke)\
**Post date:** [June 7, 2016, 9:30am UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/15 "2016-06-07T09:30:06Z")

</div>

> [@orion1024](#):
>
> I saw that you quoted the ability to put everything CI in a git repo ; do you mean that what is being done with smashbox (which is hosted in a repo) and Jenkins at ownCloud is not enough ?

The biggest issue was the maintainability of the Jenkins instance, because there some configuration for a job was placed directly in the CI system.

---

<div class="post-metadata">

**Author:** ![MorrisJobke](https://help.nextcloud.com/user_avatar/help.nextcloud.com/morrisjobke/32/19_2.png) [@MorrisJobke](https://help.nextcloud.com/u/MorrisJobke)\
**Post date:** [June 8, 2016, 9:31am UTC](https://help.nextcloud.com/t/having-a-ci-setup/223/16 "2016-06-08T09:31:33Z")

</div>

I just enabled the travis jobs. Now also some more code checks are run on travis (only those, because we only have limited resources there)

The drone instance now runs the `autotest.sh sqlite` test suite
