Link Search Menu Expand Document

ASWF CI Working Group

Meeting: 10 July 2019

Attendees

  • Andrew Grimberg (LF Release Engineering)
  • Jean-Francois Panisset (VES Technology Commitee)
  • Jeff Bradley (Dreamworks)
  • Dan Bailey (ILM)
  • Larry Gritz (Sony Pictures Imageworks)
  • Michael Dolan (OCIO, Sony Pictures Imageworks)
  • Brian Cipriano (OpenCue / Google)
  • Gordon Bradley (Autodesk)
  • Trevor Thomson

Agenda & Notes

ASWF CI Goals for Year 1 [0:00-0:05]

Timeframes

  • CI Platform decision: May
    • Azure Pipelines is the consensus and what we are going forward with
    • https://github.com/jfpanisset/cloud_gpu_build_agent/tree/master/gcp
      • Builds on GCP and Azure (some issues with remembering old IP address on Azure)
      • Installs NVIDIA driver, Azure build agent, registers to an Azure instance
    • Brian: OpenCue unblocked the pieces needed to migrate the build system. Instead of migrating Jenkins, will go all in with Azure Pipelines.
    • Andrew: using own Azure Pipelines account to create pull request, can then be hooked up to the official ASWF Azure Pipelines.
    • Andrew: additional variables added to ASWF Azure Pipelines, to push test Docker images. All those images are being built out of the ASWF Docker repo.
    • Andrew: intent is to standardize on those Docker images, based on the VFX Reference Platform.
    • Andrew: best to create a “hello world” azure-pipelines.yml file to get the process started.
  • Project CII badges: (Security and static analysis) June
    • Static analysis via CI
    • Security, CVE reporting and remedies
  • Dependency management: July
    • Next phase

CI Updates for Projects [0:05-0:25]

  • OpenVDB
  • OpenColorIO
  • OpenEXR
    • From Carey via email:
    • Christina has made progress on both SonarCloud and Azure, not fully up and running, but close.
    • First preliminary run of SonarCloud uncovered 48 vulnerabilities, 46 of which were unhandled exceptions, the vast majority in the test suite where no handling is needed, but nevertheless, we’ve “fixed” most of them and the rest should be fixed soon, so we should soon be SonarCloud-vulnerability-free. Test coverage says 78%, although because of build issues some tests weren’t included in this initial run so it’s likely higher.
    • Several of the SonarCloud warnings were in the test suite itself rather than the library itself. The advice is generally good, but no real emergencies identified. The “empty catch” warning were in the test suite, not the code itself (being fixed in a PR). Fixed by putting an other set of assertions in the catch block.
    • Kimball is still working on a revamp of the CMake setup.
    • Still crickets on all fronts looking for advice from security experts, no leads have yet panned out.
    • Should OpenEXR integrate with Google’s OSS-Fuzz? Sounds great, but we fear being beholden to a bot with a short fuse.
    • Peter and Kimball have made good progress on the backlog of outstanding bugs, including ones reported by Google Autofuzz. Several issues have involved buffer overflows due to 32-bit integer overflow with extremely large images or data windows, which we nevertheless feel are important to properly support.
  • OpenCue

CI Platform [0:25-0:30]

  • Commercial components (Houdini, Maya, Nuke)
  • GPU builds
  • Windows and Mac dependencies?

CII Badges [0:30-0:35]

  • Static analysis
  • Security
    • CVE updates
    • Reports
    • Remedies / Release notes
  • Other?

  • OpenVDB / Dan: OCIO 100% on the CII badge, not perfect yet but satisfies all badge criteria. Haven’t satisfied all the optional requirements yet, but have reviewed those optional requirements.
    • Running CI off the Azure platform, transitioned from the Circle CI platform.
    • Until now have had a pretty loose security requirement for downloading Houdini, password now stored in a Azure CI variable instead of hardcoded. What happens if you fork the repo into your own CI? SideFX could add a conditional to avoid those builds if you don’t have the environment variable set. Houdini is not baked into the Docker images, it is downloaded every time.
    • Houdini download is around 2m so not a bottleneck. Could there be a partial Houdini download?
    • There is documentation as to how to set up your own Houdini manual builds, a bit more complicated in a CI. On the list to transition to using the Houdini download API rather than the current mechanize Python module approach.
    • If your own SideFX account has access to beta / pre-release versions it is useful to run builds against that to make sure nothing has broken.
  • Michael / OCIO: currently at 97% on the CII badge, 2 remaining items related to security, talking to Dan / OpenVDB as to their approach. Once security document / policy is established should be able to complete last 2 items.
    • Michael / OCIO: Azure Pipelines is live, including static analysis, badges on the home page. Working on resolving embedded dependencies, and download of upstream dependencies. Want to make it “friendly to build”, exploring multiple options.
    • Andrew: what’s the status of migrating to ASWF repo? Michael: waiting on resolving external dependencies. Waiting on some people on vacation. Andrew: OpenEXR and OpenColorIO not under ASWF. Michael: OpenColorIO Examples repo is a bit more complicated due to copyright / license issues of some of the contents, may stay under Imageworks repo for now.
    • Dan: question to Andrew about Azure Pipelines permissions, which are extensive, could some of those be opened up to certain contributors? Andrew: in the past LF RelEng has kept control on the CI environment, but some of this has to do with Jenkins environment where setting variables gives full control. But very open to discussion to leverage fined grain permission mechanism in Azure Pipelines.
    • Andrew: Linux RelEng helpdesk, if you open an issue you will be added to the ASWF community and will show open issues, also may be able to comment on issues and view resolutions.
    • Andrew: ASWF is still the only LF project using Azure Pipelines, Gerrit/Jenkins and GitHub/Jenkins is still the landscape of other projects. LF CommunityBridge has a component to help people to onboard project onto CI, currently integrating CircleCI but may be targeting Azure Pipelines next. Some other larger projects are considering it.
  • Brian: OpenCue at 40% of the CII badge process, halfway through looking at the initial requirements. Current priority is getting the CI system up and running to avoid PR backlog. By SIGGRAPH should have made real progress on the CII badge.

Dependency Management [0:35-0:45]

  • Targets for July
    • We need clear dependency usage / VFX platform usage
    • Need to extend this to Windows and macOS, not just Linux
    • For the purposes of CI, being able to integrate commercial components
    • Dan: in the process of introducing a dependency to OpenVDB on LLVM (optional). Larry, any advice on version of LLVM (which version is OSL using)? Larry: would recommend staying close to the current version as much as possible. They are currently on version 8, working towards version 9. OSL supports a few different versions, every once in a while LLVM makes a major change that makes support difficult, but recent releases have been incremental. OSL supports LLVM 5, 6, 7 and 8, but doesn’t require a lot of #ifdefs. Stated policy is to support current version and the one right behind it, and then support new versions as long as the OSL version is current. Don’t have great data about what other projects are using, and not included in VFX Reference Platform. If introducing it as a new dependency that you didn’t have before, recommend starting with the latest version. Dan: should this be added to the VFX Reference Platform? Larry: it would be good to have a set of guidelines about common dependencies, and would include LLVM in there. Maybe it does belong in Reference Platform, OpenVDB adding LLVM creates the possibility of a single app having two versions of LLVM included. Traditionally VFX Reference Platform has focussed on dependencies of the major DCC apps. Worth an email to the VFX reference platform / discuss at the SIGGRAPH BoF.

Action Items

Next Steps

  • Follow up meeting: 24 July 2019?