Attendees

Daniel Heckenberg (Animal Logic, TAC Chair)
Andrew Grimberg (Linux Foundation)
Jean-Francois Panisset (VES Technology Committee)
Jeff Bradley (Dreamworks)
Jonathan Stone (OpenEXR TSC / Lucasfilm)
Brian Cipriano (OpenCue TSC / Google)
Larry Gritz (Sony Pictures Imageworks)
Michael Dolan (OCIO TSC / Sony Pictures Imageworks)
Dan Bailey (ILM)
Gordon Bradley (Autodesk)
Trevor Thomson (Blue Sky Studios)
Doug Walker (Autodesk)

Agenda & Notes

ASWF CI Goals for Year 1

4 months out from SIGGRAPH, need to focus on tangible outcomes
6 projects:
- Environment configuration?
- CMakeTools?
CI with VFX Reference Platform dependencies
- Commercial components
  - Build
  - Test
Stretch goals:
- Downloadable and installable artefacts (with signing?)
  - Andrew: LF has infrastructure for signing (non-Windows) build artifacts for self hosted CI system.
- Windows, Mac support
- Possibly GPU support

Circle CI, Azure Pipelines, Jenkins

Evaluation criteria:
- Costs ( both hosting and support )
- Capabilities ( OS support, GPUs )
- User experience
Updates
- No updates yet on CircleCI, waiting on someone on vacation
- ASWF CircleCI organization is created, attached OpenVDB to it, didn’t build yet since CircleCI config file not yet committed to master branch. Coordinating with Dan.
- Also set up AppVeyor, not very happy with the way it works for organizations
- Disabled jobs that were failing in Jenkins
- No movement on Azure Pipelines
- Azure Pipelines looks like “best bet” on broad support for all pipelines, including GPU. Good amount of free minutes for open source projects, CircleCI would have more limitations on concurrent jobs.
- Gordon: what are the licensing implications on third party hosted CI environments. Andrew: LF has no expertise so far with commercial software, but has another project which is running into similar issues (Zoe for OpenMainframe project). Gordon: needs to make sure that donated licenses are used only for ASWF CI purposes.
- Andrew: looking at using HashiCorp Vault for CI-platform agnostic storage of secrets / credentials. Just the beginnings of a plan.
- Daniel: asking of Andrew what the specific requirements are. Gordon: trying to consult with internal resources as to what their legal requirements are. What about a company using the CI system for its own projects?
- Daniel: we need to consider incorporation of commercially licensed packages as a primary factor of how we design our CI jobs. We don’t yet have a prototype of all those pieces working together.
- GitLab CI? LF releng may suggest that for very low cost projects, get a bit better platform for CircleCI, they have a documented roadmap. Does offer the option of self hosting. Can execute their runner service on pretty much any system, which can then be incorporated into their build cloud.
- Jeff: what is the progress on CMake Tools? That’s what Dreamworks has been waiting to ingest projects? Daniel: no hard progress yet, but now that OpenEXR is a project, transition to CMake in that project should provide real deliverables.
- Andrew: just created ASWF organization in Azure Pipelines

SonarCloud vs SonarQube

SonarCloud is free to OSS projects operating with a repository presence on GitHub
The AcademySoftwareFoundation org has been created on SonarCloud ( https://sonarcloud.io/organizations/academysoftwarefoundation )
Analysis with SonarCloud happens as a CI job (just as with SonarQube) so jobs will need to be written to handle this. There are a few methodologies for using Sonar that projects LF supports have used. The most common is to just have weekly scans against the HEAD of master and not to scan on every change. This takes care of most scanning needs.
Andrew: did some work on SonarCloud, while it is free for GitHub users, you can hook it up to any CI system, so LF releng is looking at retiring SonarQube support and transition projects to SonarCloud.
Typically projects running Sonar once a week against master branch. One of their projects is looking at gating PRs through Sonar, which puts a lot of overhead on merging contributions, but makes sure you run against smaller datasets (can take a long time to run).
SonarCloud web interface seems to have all the functionality of SonarQube, as well as access to all the plugins.
Dan: OpenVDB is still working on CMake / CI work, so not quite ready to look at code coverage yet, looking at merging CircleCI / AppVeyor configs. Good amount of progress made recently.
Daniel: subtleties related to ownership / permissions can make it tricky to integrate external services. Is it possible to generate a branch that adds SonarCloud scanning? Andrew: SonarCloud needs keys in the job, not sure how that would work with CircleCI yet. SonarCloud has documentation for how to do it in Travis CI (which already has integration), but will need to figure out how to do this in CircleCI. Could be done against a specific branch, can run into issues with how SonarCloud deals with branches, “namespace pollution” in Sonar. Which is why it tends to be simpler to do Sonar analysis against the master branch. Dan: need a dedicated branch for testing purposes during the integration phase. Andrew: one approach would be to ignore the “namespace pollution”, do the integration work in a branch, and when ready to merge into master just “start from scratch”.
Daniel: we see a value in the well presented results presented by Sonar, but our first step will be clang-tidy, but clearly value in both.

Project CI requirements

All
- CII badge static analysis
OpenVDB
- Dan: OpenVDB is still working on CMake / CI work, so not quite ready to look at code coverage yet, looking at merging CircleCI / AppVeyor configs. Good amount of progress made recently.
- Driving CircleCI / AppVeyor from CMake rather than Make. Andrew: have to be on a paid plan to have access to macOS builds with CircleCI (maybe?). Have Linux and Windows builds.
- Dan: took a look at Azure Pipelines, seems to cover all platforms and has flexible time limits.
- How to build jobs is moved into bash scripts rather than CI config files, so transitioning between services is easier (except for AppVeyor which is more Windows specific).
- Andrew: have you looked at Ansible? Dan: building all dependencies at build time, which gives a lot of control, and build time is not prohibitive. Trying to draw a line between “VFX dependencies” and “third party / generic dependencies”, so could make sense to use Ansible / Docker for “generic” dependencies. Also looked at setting up latest / top of tree build of dependencies (boost, tbb…) to watch out for any upcoming issues. Could run once a week to keep an eye on what may be coming down the pipe from these third party dependencies. But don’t want to disrupt pull requests to OpenVDB due to recent changes to these dependencies. Also when building Houdini plugins they build against latest production build, so that can also result in build issues if there are API changes, don’t want pull requests affected by those changes either.
- Currently 4 different CIs running, need to lower that to avoid extra work.
- Once CMake is fully worked out and supported, should provide a strong foundation for further progress.
- Daniel: have you been finding a natural place for Cmake-related “components” for better cross-project support? Dan: using pretty standard CMake approach with “findmodules”, strategy of supporting current and 2 older versions of VFX Reference Platform. Backported some newer CMake packages into older versions to support older builds. Boost is tricky since you need the latest version of CMake for the latest version of Boost. There are some pieces that could be turned into standard components, but don’t want to add an extra dependency.
- Daniel: is there a specific PR to look at in OpenVDB to look at? Dan: yes, but need another week or two, would prefer to point to master branch.
- Dan: want to introduce a CMake linter to OpenVDB project. OpenVDB builds shared and static libraries, CMake 3.12 helps with those kinds of issues.
OpenColorIO
- Michael: no CI-specific updates.
- Exploring SonarCloud integration, really interested for CII badge requirements, will be looking at connecting to it in the coming weeks. Andrew: there’s a token required to make sure the results appear in the right place and a API key. Andrew can generate keys for personal testing (which is where HashiCorp Vault could be useful). How to connect to Travis CI: https://docs.travis-ci.com/user/sonarcloud/ Michael: can prototype in current Travis builds.
OpenEXR
- Jonathan: excited to move forward with CI solutions for OpenEXR. Prerequisite is a robust CMake infrastructure which builds without warnings on all platforms.
- Haven’t had first TSC meeting yet, where the priorities will be discussed.
- Daniel: will need to tackle the discussion of whether OpenEXR should split off some sub components. Jonathan: separate of IlmBase seems the most natural, further separation of components remains to be discussed.
- Lots of experience with Travis and AppVeyor at ILM, but open to other solutions. Would like to treat Windows and Mac as first class citizens so have preference for solution that covers all 3 platforms.
OpenCue
- Brian: OpenCue will have different build and test requirements, mixture of Java and Python, multi-component system.
- Had to rebuild testing infrastructure from ground up to separate from SPI internal infrastructure.
- Have unit tests, no integration tests yet, more of a manual QA process, needs to be improved.
- Jenkins instance connected to GitHub repo, using Jenkins Pipelines.
- Not highly dependent on Jenkins, lightweight wrapper around Docker-based build system.
- GitHub integration running for master, will test branch and merge result for PRs.
- Highest priority is to finish expanding unit tests, and integration test.
- Andrew: can use the ASWF Jenkins instance, should be easy to move build jobs there if that helps.
- Daniel: what is needed to support OpenCue in a similar way to the other projects? There are probably technologies that OpenCue would benefit from for integration testing? Brian: need to do some research.
- Daniel: are there other LF projects that could have relevant expertise? ONAP does orchestration between Docker containers, have some good examples for VM to VM in OpenDaylight including custom network buildout. They have various examples of how to do multi container testing.

VFX Reference Platform Dependencies and Package Management

Questions for DCC Vendors / Build teams (Follow up for Gordon)
1. What build approach / CI systems currently in use?
2. VFX Platform configurations, recipes for Mac & Windows?
3. Approaches for GPU builds and testing?
4. Are there any specific configurations or technologies you could contribute?
Gordon: Autodesk has non M&E products with other requirements. They also have an internal platform for builds and dependency management, could possibly be shared.
Jonathan: would be very interested in any shareable Azure Pipelines recipes.
Andrew: ASWF will be breaking ground with Azure Pipelines. Every CI system uses a different configuration file, so can use multiple CI systems from a single project. Internally a simple proof of concept that uses Azure Pipelines, CircleCI and GitLab CI using Ansible playbooks for the build. Jonathan: would be interested in that example. Andrew: will try to find a way to publish this internal project. Daniel: useful for our members and communities to understand our decision process, especially when we need to seek approval for costs from the board.
CI artifact signing is currently rolling out to LF Jenkins based CI using Sigul ( https://pagure.io/sigul )

Next Steps

Follow up meeting: 1 May 2019
Meeting time / Timezone changes?