Attendees

Daniel Heckenberg (Animal Logic, TAC Chair)
Jean-Francois Panisset (VES Technology Committee)
Larry Gritz (SPI)
Aloys Baillet (Animal Logic)
Andrew Grimberg (Linux Foundation)
Dan Bailey (ILM)
Jeff Bradley (Dreamworks)
Gordon Bradley (Autodesk)

Agenda & Notes

ASWF CI Goals for Year 1 [0:00-0:05]

Timeframes

CI Platform decision: May
- Implementations ongoing
- Azure Pipelines is the consensus and what we are going forward with
- Some weakness in documentation, can be hard to find
- Andrew: Microsoft suggests that most of your pipeline building should be done in the context-aware editor inside the Azure Pipelines web UI.
- But git commit messages don’t add the DCO / signed-by
- Can also look at existing azure_pipelines.yml files on GitHub
- We need to share project-specifc AP expertise between projects.
Project CII badges: (Security and static analysis) June
- Static analysis via CI
- Security, CVE reporting and remedies
Dependency management: July
- Next phase

CI Updates for Projects [0:05-0:25]

OpenVDB
- Dan: work done trying to use Aloys Docker images is very promising, most issues resolved. Previously using Ubuntu as base image, moving to CentOS 7 caused a few issues related to GLFW, decided to bump up to more recent version.
- Just waiting for official ASWF Docker Images to put in PR, waiting for Andrew to push these out.
- Azure Pipelines integration is just a basic “hello world” fornow, then targetting VFX Platform 2019, 2018 and Houdini builds.
- Haven’t had much time to look at SonarCloud yet, looking at examples from OpenColorIO
- Some outstanding CII-related issues, worked with John Mertic, taking conservative approach to stating compliance for some of the items.
- Probably won’t be done by SIGGRAPH but will have some progress done.
- Big goals for SIGGRAPH for OpenVDB itself.
- Daniel: what has OpenVDB settled on for Houdini downloads, pre-install vs installing at run time? Dan: seems like the simplest approach will be to add Houdini version in a private image that can only be accessed if you sign the EULA. Get one private image with Docker Hub for a personal account, but not for an organization account. Can trim away Houdini components that aren’t needed for build / test, support building against 3 separate Houdini versions. Should we sign up for a paid account? Andrew: Azure Pipelines does have a private Docker registry, not sure if that requires a paid account? Aloys: looks like you need paid account for private image repos (and pay by storage amount). Andrew: we have a Nexus3 system in the ASFW CI account, we could use that to run a private repository (we normally run those public). Daniel: given the EULA requirements for Houdini Apprentice, are we giving up the ability to run Houdini build / tests for private / pre-PR builds? Dan: we need an authentication token anyway, private users can provide their own API key. Aloys: already using his own private key into Docker Hub. Main ASWF pipeline should support builds in context of different organizations for testing.
- OpenVDB still using the Mechanize.python based tool, could look at using the Python tool written by Thanh, but just for houdini might be simpler to use command line tool provided by SideFX. Could document the build process so that individual should provide their own API key for downloads?
- Aloys: we should reach out to Autodesk and SideFX to ask if they have their own Docker images, like a lot of vendors do?
- Aloys: submitted PR for VFX Platform 2020-2019-2018 builds
- Will be moving the minimum supported platform to VFX Platform 2018 at end of year
- Dan: had issues with TDB 4.0, had to use 4.4.6 to get stability
OpenColorIO
- Larry: Mike Dolan on vacation so partial info. He has been working on CI issues and has Azure set up for OCIO builds (at least prototype going). Drew inspiration on OpenVDB. Ready to transition to Azure Pipelines build.
- Windows and Mac builds using the native Azure builds, not clear how far that effort got.
- Dan: saw Windows and Mac on Azure Pipelines in branch, OCIO is using templating approach. OpenVDB will incorporate these ideas back.
- Larry: some of the progress would not have been happening without ASWF
- Daniel: we should have at least one graduated project by SIGGRAPH. Larry: may be possible that OCIO may be able to complete CII requirements, but repo hasn’t been transferred yet, being worked on.
OpenEXR
- Cary (via email):
- Our CII Best Practices Badge progress is at 73%.
- We’ve begun investigating SonarCloud but don’t yet have much progress to report.
- We also haven’t yet looked into the Azure setup.
- OpenEXR has several CVE issues, some already fixed but lacking proper documentation, some with fixes we’re working on integrating, and some we’re still investigating. The issues are primarily buffer overflows.
- Still wondering where to find a security expert.
- We’ve made progress on reducing compiler warnings.
- Python 3?
OpenCue
- No updates

CI Platform [0:25-0:30]

GPU builds
- Small example: https://github.com/jfpanisset/cloud_gpu_build_agent
- Questions:
  - On demand VMs vs Available pool as Azure expects
  - Aloys: Docker images are GPU enabled so could be used as a basis for Linux GPU builder
  - Daniel: is any project ready to take advantage of this work yet?
  - Larry: suspect OpenColorIO would be ready to start integrating
Windows and Mac dependencies?
- OpenColorIO looks like it is close to merge PR with support for Windows / Mac builds? Larry: not enough information to answer definitely, will need info from Michael Dolan.

CII Badges [0:30-0:35]

Static analysis
Security:
- CVE listing
  - OpenEXR has specific issues with existing CVEs, Cary working through those
  - Dan: went through these requirements with John Dolan, requirements are actually fairly loose, so important not interpret too much. OpenVDB needs to make updates to web site: for instance there are requirements on website content that are currently in the GitHub repo. Multiple avenues possible for users to report issues (email, GitHub issues…) so need to figure out what’s the mechanism that will be recommended. Need to add security email address to the website. Main outstanding issue is policy document being worked on that needs to be added to the repo, and static analysis. Previously tried to use code coverage, found that test suite was taking a long time to run, need to differentiate unit tests vs regression tests to satisfy the requirements without adding too large a burden.
  - Daniel: like the security policy document to address the “active awareness of security issues” requirement, as well as a way to share security expertise. Dan: trying to satisfy security requirements without explicitly naming a single “security expert”.
- Reports
- Remedies / Release notes
Other?

Dependency Management [0:35-0:45]

Targets for July
We need clear dependency usage / VFX platform usage
Need to extend this to Windows and macOS, not just Linux
For the purposes of CI, being able to integrate commercial components
Also desire to possibly tackle studio dependency management
Aloys: it’s a vast project, work started to package VFX packages with Conan would take quite a bit of time. Conda has some of these packages already available, Rez could work by combining work from previous studios.
Jeff: are we looking to have completion on this in July? Feel like something we can’t solve in 1 month.
Daniel: agreed, but should have at least started down the path, and define what’s achievable.
Dan: what is liked about the Docker approach is that everything is installed in a “vanilla” way, typical of how our users are likely to have things installed. So this tests a valuable part of what we want to deliver. But don’t want to fall in the middle ground between the hobbyist user and the large studio use case.
Daniel: for SIGGRAPH we are hosting on Open Source day a Rez BOF, so could talk again to the Rez folks as to how it can be leveraged for CI applications.
One set of Docker recipes works well, but no Docker macOS, could work on Windows, but would need to integrate the compiler inside the container image. Having a single solution across platforms is challenging.
We may need to host our own packages, both the dependencies and our project artifacts
Clang takes 2 hours to build, so don’t want to have to rebuild from scratch all the time
Could leverage Azure artifacts? Not clear how much storage is available, but could not use them from private forks (perhaps).
Artifactory Cloud? Andrew: Available to open source projects, extensive form that needs to be filled out. No easy sign up. Supports Conan, Conda, RPM, deb, containers.
Aloys: target for July may be more about design / exploration than actual implementation.
Aloys: in order for Docker images to become part of official ASWF repo, need to do some work on the ASWF docker hub account? Andrew and Aloys to work offline.
Aloys: do we agree that ASWF images are the most stable, “ASWF staging” would be more experimental, “ASWF testing” is where we can all push new images, so alpha state. Does that sound reasonable? Dan: sounds OK, hoping it doesn’t have to change that often.

Action Items

Daniel at TAC: Reach out to Autodesk and SideFX to ask if they have their own Docker images, like a lot of vendors do?
Jeff: no specific CI working group meeting at SIGGRAPH? Could we carve off some time from the overall TAC f2f meeting?
Aloys: document docker image organisation and liaise with Andy about configuring official Azure Pipeline to implement the logic.

Next Steps

Follow up meeting: 10 July 2019