At first, you’ll want to write your tests locally, and test them against as many local browsers as possible. However, to really test out your features, you’ll want to:
- run them against as many real browsers on other operating systems as possible
- have easy access to human- and machine-readable test results and build assets
- integration with development tools like GitHub
Enter Continuous Integration (CI).
Each brings their own syntax, features, and constraints to building and maintaining robust CI workflows.
JupyterLibrarystarted on Travis-CI, but as soon as we wanted to support more platforms and browsers…
Cloud: Azure Pipelines¶
At the risk of putting all your eggs in one (proprietary) basket, Azure Pipelines provides a single-file approach to automating all of your tests against reasonably modern versions of browsers.
If you are working on in-house projects, and/or have the ability to support it, Jenkins is the gold standard for self-hosted continuous integration. It has almost limitless configurability, and commercial support is available.
Approach: It’s Just Scripts¶
No matter how shiny or magical your continuous integration tools appear the long-term well-being of your repo depends on techniques that are: - simple - cross-platform - frequently run outside of your CI
Since this is Jupyter, this boils down to putting as much as possible into platform-independent python (and, when neccessary, nodejs) code.
JupyterLibraryuses a small collection of scripts, not shipped as part of the distribution, which handle the pipeline. In addition, this library uses anaconda-project to manage multiple environment versions, and to combine multiple script invocations with different parameters into small, easy-to-remember (and complete) commands. Unfortunately, some of these approaches don’t quite work in Azure Pipelines, so some duplication of commands and dependencies are present.