Link Search Menu Expand Document

ASWF CI Working Group

Meeting: 26 May 2021

https://zoom.us/j/757849640?pwd=QzE1K2hrL2FHSFhKK3h5Z3BWTFJsZz09

Attendees

  • Jean-Francois Panisset (VES Technology Committee)
  • Aloys Baillet (Animal Logic)
  • Andrew Grimberg (Linux Foundation RelEng)
  • Ryan Bottriell (ILM)
  • David Aguilar (Disney Animation)
  • Sean Looper (AWS)
  • Larry Gritz (Sony Imageworks)
  • Sergio Rojas (Arena World)
  • Jeff Bradley, Dreamworks

New items

  • VFX 2022 docker images

    • Haven’t started yet, but shouldn’t be too difficult. Python 3.9 plus some small updates here and there.

    • Aloys: Will delete the code that creates the 2018 versions, so won’t remove those images, but they won’t get updated anymore. We should mark them “deprecated” in a way. That’s a proposal, not an official decision. 2018 images haven’t been rebuilt in a while anyway. Maybe rebuild images once a year? Should establish a good set of practices for that: once 3 years have passed, we don’t rebuild them anymore, they won’t break “right away”. Proposing a discussion? Plan is to add 2022 and remove 2018.

    • Larry: no official guidance from VFX Platform about sunsetting versions. Aloys: we should keep something out there with Python 2.7, and 2019 does that (it may have to stay out there longer). Main difference between 2018 and 2019 is the compiler, gcc 4 vs gcc 6. Larry: I don’t think we have an image where gcc 4 is the default? Had to point to system compiler to get gcc 4.8, VFX Platform 2018 is gcc 6. QT 5.12.6 instead of 5.6.1 between CY 2018 to 2019. The images will stay there forever, but will just stop updating 2018 images. Ryan: we probably shouldn’t update the images after the CY they are for. Aloys: have had to rebuild some older images because the OS needed a yum update. Can always handled on off emergencies. Ryan: we can just use tags / branches. Aloys: yes, we use tags. Andrew: policy is to not comment out code, instead remove it and rely on git. Aloys: every docker image is linked to a tag, the pipeline can recreate an image automatically. Trivial to create a new release of an older tag. So should be safe to delete the code to generate 2018 images. 2019 version may have longer “shelf life” because of Python 2.7. Ryan: all we’re talking about is maintenance, since the images themselves will be there “forever”. Aloys: that’s not totally right, Docker Hub deletes images that haven’t been pulled in 6 months? Andrew: since we have purchased a plan on Docker Hub, we are protected against that.

    • Aloys will send an email to the TAC / Slack to discuss this decision going forward.

    • Aloys: Larry started to use OCIO for OSL in Docker images, cyclic dependency between OCIO and OSL.

    • Dave: because of that dependency, build OCIO apps separately from OCIO library, treat a single git repo as two separate sets of artifacts. Larry: how everyone is solving it now. Aloys: need to refactor the packages and how they get built. Larry: at some point OCIO will fix this, they’ve heard this before. Aloys: there’s a diagram in the ASWF Docker repo to show the order of dependencies and builds, https://github.com/AcademySoftwareFoundation/aswf-docker/issues/54 Was reluctant to try solving this in context of aswf-docker “ad hoc” package management.

    • Aloys will be leaving AL in a month, will still be able to work on ASWF projects, will be joining the USD team at NVIDIA.

    • Not yet building MaterialX, should need to be added soon. Haven’t heard from that team yet. Is it mainly Autodesk maintaining the repo? Do they have a GPU test suite yet? Larry: not sure what their testing setup looks like, but they are likely to need to use the GPU setup we have. Aloys: looks like they are using GitHub Actions.

    • Sean: do we have a definitive runbook for the latest agreed upon CI process? AWS is looking at open sourcing an internal project, guidance has been to align with ASWF as much as possible, could yield contributions back to the team.

    • Larry: we all use GitHub Actions, could look at the existing projects, they are mostly similar to each other. Not sure which one is the canonical / best put together project, but can’t really go wrong looking at any of them.

    • OpenColorIO is the one to look at for GPU tests, and it’s likely the one that’s furthest along.

    • Can looking at ASWF Sample Project

    • Sean: also looking for any useful links

    • Larry: can’t go wrong with OpenColorIO, they are really well organized

  • The Pipeline Conference: Jesse Lehrman is doing a DevOps talk and asking for participation from CI WG

Follow Ups

  • Any news from JFrog about OpenSource account for ASWF? (from Aloys)

    • Andrew: SSO has been configured, so can go ahead with creating accounts, but needs to get configured.

    • Got requirements from Aloys

    • One engineer from LF Releng investigating what Xray could bring

    • Will be looking for updates on Jira ticket

  • GitHub Actions Contributor / PR screening:

    • No updates from GitHub apart from public blog posts. They are planning on additional security updates. Some ACLs for controlling write access in PRs.

Action Items

  • Daniel: WG formalization / split into own repo

  • Andrew: Summary of JFrog Xray for ASWF use cases

  • Daniel: Raise QT5, Wayland topics to TAC

Next Steps