Link Search Menu Expand Document

ASWF CI Working Group

Meeting: 12 June 2019

Attendees

  • Andrew Grimberg (Linux Foundation Release Engineering)
  • Michael Dolan (OpenColorIO / SPI)
  • Trevor Thomson (Blue Sky Studios)
  • Jean-Francois Panisset (VES Technology Committee)
  • Aloys Baillet (Animal Logic)
  • Cary Phillips (Open EXR / ILM)
  • Larry Gritz (SPI)

Agenda & Notes

ASWF CI Goals for Year 1 [0:00-0:05]

Timeframes

  • CI Platform decision: May
  • Project CII badges: (Security and static analysis) June
  • Dependency management: July

Project CII badges: June [00:05-0:25]

Project CI requirements [00:25-0:50]

  • General CI discussions / announcements
    • Andrew: had meeting with Microsoft about Azure about upcoming issues:
      • GitHub logins will be coming to Azure
      • ability to create “elastic pool”: for now we would need to create a pool of GPU enabled instances running at all times, but will be possible to dynamically scale pool of resources up and down
      • ability to purchase more parallel processing
    • Aloys: unclear on how a pipeline in a ASWF pipeline can work on a fork of a project? Andrew: when you raise a PR against a ASWF project, you will run the Azure Pipeline under the ASWF Azure Pipelines instance. If you do it on a fork, would have to be your “own”.
    • Aloys: what about if you want to create a PR on the azure_pipelines.yaml? You can’t create a fresh pipeline through a PR, but you can edit an existing pipeline. So a potential for a “denial of service”, could be a way to rate limit the amount of CI. Should there be a list of “recognized contributors” that can fire off Azure builds unrestricted? LF Engineering will be keeping track of Azure resource usage.
    • Aloys: if I create a PR which starts a job, can I requeue the job / manipulate the job? Andrew: this will be easier once we have unified logins, can grant specific rights. Need to ask Microsoft about “comment based triggering”: leaving a comment on an open PR could trigger a job (if you are whitelisted). Note sure if Azure Pipelines currently supports this, this is how LF RelEng does this on their Gerrit based projects.
  • OpenVDB
  • OpenColorIO
    • Michael: outstanding PR to switch CI to Azure, incorporates SonarCloud, everything working well, discussing enabling SonarCloud on all PR builds since Azure builds are fast enough. Their VFX Reference Platform Docker image should be based on CentOS rather than Ubuntu. Aloys: the gotcha with the DTS is the funny tricks it uses to allow C++ compiled apps to run with the “old” runtime, so the binaries are quite different. That may break the assumption that our projects / binaries are as “vanilla” as possible and can be easily modified. Looking into adding standard images to ASWF Docker repo, combining OpenVDB and OpenColorIO scripts. Michael: would be useful to have both the recipe and pre-build Docker images that we can build on top of. Docker images currently being pulled from Michael’s private Docker repo, to be merged into official repo. Michael: overall very pleased with Azure Pipelines.
  • OpenEXR
    • Cary: getting back from being away, no meeting since then. Will be following up on getting Azure setup incorporated. From conversation with Dan Bailey, would be useful to have a basic “hello world” Azure Pipelines setup. Andrew: we had one for OpenVB. Cary: will check into the repo to look at the original version.
    • Hello world pipeline from OpenVDB https://github.com/AcademySoftwareFoundation/openvdb/pull/450
    • Andrew: Microsoft recommends using the built in editor in Azure Pipelines to build azure_pipelines.yml file (based on Visual Studio Code editor)
    • GitHub offering new feature for “template based repositories”: https://github.blog/2019-06-06-generate-new-repositories-with-repository-templates/ Could be used not just for CI, CII requirements, documentation files, an ASWF “skeleton” repository
    • Andrew: will be able to give “manager” rights for Azure Pipelines to repo admins
    • Was working on updating the README files, documentation, need “good” examples of how to do this.
    • Aloys adding a dummy / sample Azure Pipelines YAML file to the Docker repo which can be used as an example. This will probably be Linux only (unclear if it’s worth using Docker images on Windows). Michael: only using Docker on Linux, native on Windows, macOS. Andrew: went ahead and created the dummy Azure Pipelines YAML. Andrew: need additional maintainers on the Docker repo, for now he’s the only one.
    • JF: there should be a “canonical” way to structure multi-platform builds
  • OpenCue

Action Items

Next Steps