Attendees

Daniel Heckenberg (Animal Logic, TAC Chair)
Jean-Francois Panisset (VES Technology Committee)
Andrew Grimberg (Linux Foundation Release Engineering)
Thanh Ha (Linux Foundation Release Engineering Group)
Patrick Hodoul (Autodesk)
Doug Walker (Autodesk)
Aloys Baillet (Animal Logic)
Jeff Bradley (Dreamworks)
Dan Bailey (ILM)
Trevor Thomson (Blue Sky)

Agenda & Notes

ASWF CI Goals for Year 1

6 projects:
- Environment configuration?
- CMakeTools?
CI with VFX Reference Platform dependencies
- Commercial components
  - Build
  - Test
Stretch goals:
- Downloadable and installable artefacts (with signing?)
- Windows, Mac support
- Possibly GPU support

Circle CI vs Jenkins

(Thanh) Update
- Activated CircleCI free for ASWF organization
- Trying to enable open source tier which allows additional resources
- OpenVDB PRs from Dan Bailey to enable CircleCI: this is just for reference, not quite ready to be merged in
- OpenVDB also potentially moving to CMake as part of that effort
- Cost estimates? Was supposed to have update meeting with CircleCI, but that hasn’t been scheduled yet.
Azure Pipelines / DevOps (vs CircleCI from MS)
- Supports all 3 platforms: Windows, Linux, Mac
- Had been discounted early on due to lack of ability to replicate whole stack on-prem
- Has it come up in the context of other LF projects? Early discussions, but decided to go with CircleCI. Can LF releng provide any feedback?
- Do they support GPUs?
- Dan: what are other people’s thoughts on how to structure their CI configurations? Expectations are that we would have a single build on WIndows and Mac rather than having a large number of tests? Can we just use AppVeyor for simple Windows builds? For now OpenVDB focussing more on Linux.
- Patrick: For OpenColorIO the matrix is much more extensive on Linux rather than Windows because AppVeyor is quite slow. We should have extensive builds and unit tests on all platforms, “ideally”.
- Daniel: as a first step full matrix on Linux since we have active users and expertise, especially if we have to use other CI systems on non-Linux platforms. But longer term goal is still to generate downloadable builds on all platforms, so will need to support more extensive build matrices on those platforms as well. But for now we should concentrate on a “MVP” focussed on Linux with simple builds on other platforms.
- Dan: there should be a convention as to how ASWF projects are set up so it’s not arbitrarily different on all platforms.
Are we ready to give up full open source solution / reproducible stack?
- vs SaaS offering
- CircleCI is still using Terraform on Amazon
- Should this be a voted-on decision?
- Dan: What do we need to make a vote, pricing info? Further research on options out there?
- Daniel: costs are important, functionality / capabilities, local reproducibility.
- JF: Is there an expectation from the paying members of a locally reproducible infrastructure.
- Daniel: question to Andrew (LF): is ASWF committed to providing a locally reproducible CI stack, or is that a desirable but not necessary requirement. His understanding is that the original proposal from LF to ASWF member companies would be reproducible in house, which is why they went with Jenkins stack, but also only option that can be done in house. Also part of the original documentation from RelEng team, which got pushed along by ASWF project. If the project pivots to SaaS solutions, the Steering Committee would have to vote on this, since in-house reproducibility was part of the original commitment.
- Daniel: Governing Board and TAC meeting coming up before end of months, so this should be brought up there. Different motivations between projects and studios.
- Jeff (DW): wasn’t initially expecting to be able to bring CI system in-house and integrate with their system, but their short term expectations was more about recipes for dependencies which they could migrate into their internal processes. Being able to bring in the entire CI process for these projects would be valuable but not necessary.
- Andrew: agrees that other studios had similar perspective.
- Jeff: current CI configs are not sufficient since they rely on locally installed libraries.
- Daniel: ties in to dependency management

SonarCloud vs SonarQube

(Thanh) Update
- OpenEXR example on SonarCloud
- Our current SonarQube instance doesn’t have C/C++ plugin so not directly usable, SonarCloud supports C/C++ plugins and is free for Open Source projects, would save on hosting costs since one less server to maintain.
- Cannot be used against private repos for free, not an issue for ASWF repos.

Project CI requirements

All
- CII badge static analysis
OpenVDB
- (Thanh) Houdini for plugin build & test?
  - CLI download tool
  - JF tried it to download 17.5 for all platforms, works well
  - JF: could lead to a common tool for downloading commercial apps
  - Dan: would prefer if it was maintained by SideFX. Thinks it would be preferable if we interfaced with API directly.
  - Thanh: would prefer if vendors would maintain tools themselves. Helps to deal with API changes.
  - Dan: need to document the layer we created
  - Daniel: SideFX rose to the challenge of offering an API, but not from Autodesk or Foundry, we should re-engage them.
  - Nuke is just a URL, same for Maya dev kit. https://www.autodesk.com/developer-network/platform-technologies/maya
OpenColorIO
- Patrick: there are steps to do before considering CI because they have embedded libraries.
- Doug: still working on moving repos to ASWF organization

VFX Reference Platform Dependencies and Package Management

OpenEXR / VFX Platform CI challenge
Aloys: OpenEXR on ASWF Jenkins
Aloys: OpenEXR (VFX Platform 2019) on Circle CI with docker images via conan
- Struggling for IlmBase and OpenEXR CMake builds
- Also PyIlmBase which requires Python and Boost
- CircleCI was straightforward, create YAML file, mostly just works. Seems quite similar to how you would run from Jenkins, but Jenkins takes longer to understand and setup. But should be possible to maintain both CircleCI or Jenkins.
- Started looking at caching system and “workflows” in CircleCI. Took Conan experiments and moved to separate repo, made CircleCI config for that which builds TBB and Boost. Building Docker images and Conan distributions. Building on CentOS 7 with Dev Toolset 6.
- Documented some debugging notes with Conan, how to troubleshoot issues. Still one crash from a numpy test in PyIlmBase.
- Would help if we could agree on a base Docker image for building, and could also be used by current Jenkins solution.
- A very Work In Progress PR against OpenEXR.
- Overall seems CircleCI much simpler to use than current Jenkins CI infrastructure.
Dan: OpenVDB on Circle CI
Dan: OpenEXR on Circle CI
- Just looking at Aloys CircleCI setup, seems to cover a lot of ground. Aloys: most of the “volume” is the Conan stuff, the CircleCI stuff is fairly minimal.
- Questions about how we deal with dependencies between projects, not clear how to trigger jobs from one repo to another in CircleCI (can you have workflows across GitHub repositories?). Should be possible in Jenkins. For instance want to make sure a change in IlmBase doesn’t break OpenVDB.
- Aloys: Version numbers in Conan files are hard coded.
- Aloys: Rez uses package files with hard coded version numbers, could make “ranges of versions” to allow picking up “latest” versions.
- Dan: should we have separate requirements for our CI: kicking off builds and tests when a PR is merged, vs making binary distributions (for instance a single CI toolchain to build a specific version of the VFX Reference Platform).
- Daniel: any feedback on CircleCI / Conan direction? Thanh: this is trailblazing, other LF projects aren’t using those tools. Conan seems similar to Maven for Java projects.
- Patrick: consuming binaries in C++ is challenging because of different compilers, different compiler flags… Need a standard on how to build libraries (VFX Reference Platform solves some of this?). Daniel: one of the goals we’re trying to accomplish, in conjunction with Ref Platform. Trevor: need some clarification as to what “building with C++14” is mentioned. Patrick: needs to cover more than just Linux.
Larry: OIIO on Circle CI

Next Steps

Follow up meeting: 3 April 2019