Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up builds on travis #384

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open

Clean up builds on travis #384

wants to merge 31 commits into from

Conversation

lucc
Copy link
Contributor

@lucc lucc commented Oct 6, 2018

I cleaned up and commented the travis file. Results:

  • less needless options (docker exec --tty)
  • less installed packages
  • less packages installed outside of the distro package manager
  • less inline shell scripts and less quotes
  • no sudo inside container (together with local bundler installation of requirements)
  • separation of test commands to make failures easier to understand
  • explicit comments why we need locale-gen and su

Further questions about the travis build:

  • Should we also run the tests on more distros? See the example for ubuntu 18.04 on travis
  • Should we run tests on macOS? Travis can do that and I could update the travis file for that.
  • Should we run the ubuntu tests on travis directly, without docker to reduce overhead and speed things up? (This is especially interesting if we also run tests on macOS as these would probably run without docker which would force us to implement different build logics anyways.)

Questions about the tests (in general and on travis):

  • It looks like the features from the ruby tests are all empty. There is no actual testing going on. We should fix that! But I don't know any ruby 😞
  • One of the spec test is flaky and fails randomly on travis. Do you have any idea how we can fix that?

lucc added 8 commits October 5, 2018 11:20
This way travis gives a better indication which command failed.  Also it
makes the lines shorter and easier to read.
The gem is used to interact with travis.ci which is not strictly
necessary to test vdebug (even if the tests run on travis).  This also
makes some dependencies obsolete as they where only needed for the
travis gem.
This makes sudo and the setup of sudo unnecessary.
Apt needs a special environment variable to work but the other commands
do not need a tty.
The ruby specs in spec/startup_spec.rb fail randomly on travis so we
execute the specs separated from the feature tests.  That way rake will
not stop after the failure.
The package installation with bundler does not require a full login
session from su so it can be run with sh.
@BlackIkeEagle
Copy link
Member

Further answers about the travis build:

  • Yes we should run tests on more distro's
  • macOS would be a nice to have
  • at the time of the switch to docker the travis base was using an outdated version of vim compiled against python2, so if this is now improved, it could be switched back to non-docker

Answers to questions about the tests (in general and on travis):

  • I was going to look into that, but did not find sufficient large blocks of time to get into it
  • The flaky test is a very travis + docker specific issue, if docker is remove again from the tests that will fix itself

@lucc
Copy link
Contributor Author

lucc commented Nov 30, 2018

I started to play around with this again. What motivated me where some changes from travis' side: https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration

If we want to run tests on OSX we will not be able to use language: python: travis-ci/travis-ci#2312 . This means we can not use stuff like this to construct a nice build matrix (which we did not use until now anyways):

python:
 - "3.4"
 - "3.5"
 # ...

But which might still be interesting as it should start a different VM with that python version preinstalled so the install step could be faster.

I think we have to decide between two options:

  • use language: python with different python versions only on linux
  • one python version but on linux and on osx with language: minimal

@BlackIkeEagle What do you think? What might be more important to test regarding frequent bug reports?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants