Attendees

Aloys Baillet (Animal Logic)

Jeff Bradley (Dreamworks)

Mark Elendt (SideFX)

Federico Naum (Animal Logic)

Larry Gritz (Sony Imageworks)

Thanh Ha (Linux Foundation)

Daniel Heckenberg (AL, TAC Chair)

Allan Johns (Rez)

Jean-Francois Panisset (VES Technology Committee)

Trevor Thomson (Blue Sky)

Robert Vinluan (SideFX)

Doug Walker (Autodesk)

Agenda & Notes

ASWF CI Goals for Year 1

6 projects:
- Environment configuration?
  - Rez?
    - Allan Johns to discuss Rez
- CMakeTools?
  - Came up in context of OpenVDB project
CI with VFX Reference Platform dependencies
- Commercial components
  - Build
  - Test
Stretch goals:
- Downloadable and installable artefacts
- Windows, Mac support
- Possibly GPU support (relates to build & test)

Project CI requirements

All
- CII badge static analysis
- Dependency management
OpenVDB
- CMake builds
  - Looking for expertise to help transition from Make to CMake
  - Looking for support to achieve CII badge / static analysis
- Transition from Travis-CI to ASWF / LF CI infrastructure
- Houdini for plugin build & test
  - Thanh: still in talks with SideFX on how to download Houdini API
- GPU for OpenGL tests?
OpenColorIO
- Approved as official incubation project at last week’s TAC meeting
- Starting to work on infrastructure (moving git repo, forming committee)
- No work yet on CI issues
- Nuke plugin build?
- GPU?
  - Related to CI issues
- Windows, Mac?
  - Requirement expressed for full Mac support
OpenEXR
- A major foundational project, will drive dependency management approach

Packer config to build VFX Reference Platform (Aloys)

Conan.io
- Aloys’ branch
- Posted survey of package management systems for C++ to mailing list
- Goals:
  - Distribute binaries securely
- Aloys leaning towards conan.io
- Rez could be extended to support conan.io (to be discussed with Allan)
- Cget is based on CMake, doesn’t support artefact repository, but ASWF already has Nexus repository so may be non-issue.
- Could extend Rez to automatically fetch binaries from repository
- Conda is a large data science distribution, may not allow describing specific compiler settings (such as need for RHEL Development Toolset versions)
- Hunter is CMake based, also doesn’t do artefact management
- Buckaroo seems like a full build system
Rez
- First pass CI setup simply used default dependencies to build projects from source
- Complex dependencies for VFX Platform compliance
- How do you fetch the correct version of upstream projects and build them correctly, need to build an ecosystem of components
- Rez solves most of those problems, some of the facilities in the ASWF already use it.
- Could the package / dependency management system implemented for the ASWF be used by studios / facilities. How do we solve the immediate CI problem while thinking ahead of how this could be used in facilities.
- Rez is different from conan.io, designed as a runtime dependency manager. Initially didn’t support builds, can resolve an environment as quickly as possible, resulting in a set of package versions that don’t conflict with each other.
- Rez is not going to be as good as conan for builds since not designed for that, and not designed for managing public, pre-built artefacts, might be out of scope.
- Integrating conan into Rez could make sense, could be possible to write Rez definition files to pull artefacts from public conan artefact repos.
- There are probably missing components, Allan has ideas of what would be needed to implement and believes it is achievable to have Rez to read Rez package definition files from a public repository.
- Aloys: there could be a public ecosystem of packages, and use Rez to access those (similar to pip for Python).
- Allan: it may make more sense to have a repository of Rez definition files that know how to pull from conan repos.
- Aloys: what happens when packages have similar upstream dependencies
- Allan: could also extend Rez to have the same kind of functionality as conan, but could be too large a project to replicate that functionality.
- Larry: studio is contemplating investment into using Rez, if Rez is extended to support build version management and adopted by other studios in context of ASWF it would be a strong selling point.
- Jeff: has been playing with Rez in last couple of years to help build open source software, defining dependencies in Rez package files, ran into issues with flags that need to be propagated and which are not documented. Packages expect to be “system installed” and auto-discover dependencies, not configured, so have to fight that. Not all libraries respect LD_LIBRARY_PATH to discover where libraries are (CMake project should help as well). May want to pull zlib from a Rez package for instance.
- Allan: Rez doesn’t (yet) manage the boundary between managed packages and packages installed on the system. It should be since Rez is good at runtime resolution.
- Aloys: AL has been using Rez in 4-5 years, currently in “pullback” phase where low-level packages like zlib had been Rez-ified under CentOS 6, but under CentOS 7 easier / safer to use system level packages (using Ansible to make sure correct versions are installed). Abstract the configuration of the system via a single Rez package. They found that this helps with “too many Rez packages” which becomes a maintenance burden.
- Jeff: currently migrating from RHEL 6 to RHEL 7, abstracting system packages to Rez might help with OS transitions.
- Daniel: should the ASWF tackle these type of system-level issues.
- Trevor: how should different DCCs with different VFX platform levels / requirements deal with system level vs managed packages.
- Allan: could grow into “production management”, i.e. what package should be used for what project, since every studio needs to solve this issue.

CII Badge Automation Support (Daniel)

Feedback from CII badge mailing list
Dreamworks and Autodesk have provided some background as to what they have done internally
Not clear how to make practical recommendations / implementations for ASWF projects
C++ project example: curl
- They have a CII badge
- Checks run automatically for PR
- Example PR
  - clang-tidy target in Makefile
    - Some checks disabled to avoid noise
    - CI builds via travis
- lgtm with github integration
  - Commercial tools, free for open source projects
- codacy with github integration
  - Commercial tools, free for open source projects
Identified vulnerabilities need to be addressed in timely manner
In our projects we are more interested in identifying programming issues / “anti patterns”
Larry: have been looking at what curl is doing and clang-tidy, path forward seems clear. Haven’t tried lgtm yet. Seems like a good path towards CII badge requirements.
Thanh: feels like a valid approach for CII requirements, LF has been looking at using free for open source online scanning services. But online tools may be an issue for organizations wanting to pull ASWF projects internally and don’t have Internet access.
Larry: doesn’t feel it’s not a huge issue, as long as these external dependencies are optional. PRs into the main repos would still have to run all the tests.
Vulnerabilities / flaws in ASWF projects are important to commercial vendors adopting those components.
Federico: what about Sonar? Currently part of the ASWF CI infrastructure, but C/C++ components are commercial, there is a community supported plugin that calls out to other tools, we can get what we want without Sonar, but makes sense to use Sonar to integrate clang-tidy, provides a nice aggregation layer for various checking systems.

Next Steps

Change in meeting cadence due to upcoming in-person meeting
- Follow up meeting: 20 February 2019
- Meeting invites will be updated
- TAC meeting next week will also be cancelled