Daniel Heckenberg - Chairperson, Animal Logic Pty Ltd
Gordon Bradley, Autodesk
Pilar Molina Lopez, Blue Sky Studios, Inc.
Michael O’Gorman, Cisco Systems Inc.
Henry Vera, Double Negative
Bill Ballew, DreamWorks Animation
Matt Kuhlenschmidt, Epic Games, Inc.
Brian Cipriano, Google & OpenCue Representative
Sean McDuffee, Intel Corporation
Larry Gritz, Sony Pictures Imageworks
Jean-Francois Panisset, VES Technology Committee
Cory Omand, The Walt Disney Studios
Kimball Thurston, Weta Digital Limited
Eric Enderton, NVIDIA
Sean Looper, Amazon Web Services
Michael Min, Netflix
Michael B. Johnson, Apple
Dave Fellows, Microsoft
Ken Museth, OpenVDB Representative
Michael Dolan, OpenColorIO Representative
Cary Phillips, OpenEXR Representative
Joshua Minor, OpenTimelineIO Representative
Other Attendees
Emily Olin, Linux Foundation
John Mertic, Linux Foundation
Jeff Bradley, DreamWorks
Doug Walker, Autodesk / OpenColorIO
Alex Meddick, Rising Sun Pictures
Greg Gewickey, Warner Brothers
Apologies
Agenda
Goals for TAC: Year 2019
GPU enabled CI Proof of concept
VFX Platform 2020 / Python 3
Follow ups
Threading library / best practices
Larry: tricky situation, needs of the projects are varied, OpenEXR on one of the scale where they use internal / home grown thread library, make things easy for applications by adding a public API call that lets calling application provide its own threading library. Other end is OpenVDB that uses Intel TBB so deeply that it would be very difficult to replace it. So seems unlikely to find a solution that would span the range of needs. Some people in isolation express misgivings on using TBB as the solution, but we don’t want to encourage home grown solutions, and encouraging TBB is probably currently the best approach. Maybe we can get some help from Intel to help deal with expressed issues, also room for establishing best practices, for instance how do we deal with an ecosystem of libraries that all want to use multithreading independently of each other.
Gordon: can you elaborate about concerns about TBB? Larry: don’t know that there are real issues, “median” opinion is that TBB is a pretty good solution, we should track down the “outliers” who have concerns. Could also be based on experience from years ago.
Kimball: may have been the one claiming an issue, experience is mostly about mixing versions of TBB, for instance a plugin compiled with a different version of TBB than the main application. Seems to mostly work, but can end up with random crashes at exit.
Sean: have had issues with initialization of MPP, proprietary schedulers and usage of task arenas, when you have multiple libraries who might try to initialize TBB in different orders. But overall think it’s a good idea to encourage use of TBB, with some best practices.
Larry: unless you have a good reason not to, should encourage using TBB. OpenVDB won’t have needs satisfied unless entirety of TBB.
Sean: USD may bring similar needs.
Daniel: Pixar have a wrapper around TBB, but it’s “leaky” (libwork in USD), served the purpose of providing an easy place to consistently restricting the number of threads. That’s apparently less of an issue in current TBB versions. Concerns about mixing multiple clients (maximum consistency) vs being able to control the threading system.
Daniel: can we draw any useful / simplified conclusion? Sounds like a direct opportunity to articulate challenges that people meet, and make it easy for people to use TBB in multiple libraries. Maybe adding some kind of standard layer / shim library, and/or provide feedback to Intel to make things easier. Sean has already suggest that TBB team is eager to get our feedback.
Sean: forwarded Larry email to TBB architect, team is in Russia, they are interested in engaging with us.
GPU CI
Work on costing (JF) -> cannot use K80 instances for OpenGL, need GRID license
Need to look at preemptible instances
Larry: although today’s development of OCIO is primarily interested in testing against OpenGL, going forward is probably going to run CUDA code paths.
Sean: scalability might be an issue, as we expand GPU need into other projects. Also do we want to expose this testing pool to forks of our projects. How widely do we want to make this mechanism available to our developers.
Larry: expanding to expand to PRs seems like a core requirement.
Michael: it’s not like there are dozens of PRs coming in.
John: is concern mostly about cost? Sean: could be premature optimization. John: there are different ways to handle cost issues.
JF: Could also use Software OpenGL in some cases (example of DJV): Michael: OCIO test suite known to fail on software OpenGL implementation.
Eric: are projects interested in a range of GPUs or drivers? Testing on multiple / simultaneous GPUs?
Michael: platform differences can be crucial
Daniel: do we have enough info to generate formal resource request to board or one of our members? One machine is a tractable amount of money, 10 machines would be a lot more. Do we have enough of a concrete use case to “turn things on” now? OCIO project: how urgently do you want this?
Michael: targeting OCIO v2 for next summer, so would be useful to get it going. Doug: don’t get GPU-related PRs every day, a few minor changes every other week. But would like to put in place for the project.
Daniel: will follow up with Dave about support for dynamic instances on Azure. JF: will pick up discussion thread with Azure Pipelines team.
Sean: not so concerned about cost optimization, willing to cover the cost, but dynamic instances could be interesting for scalability concerns (especially if builds start to queue up).
Wave: has been investigating availability of macOS build resources. Shipping of new MacPro later this month may help. JF: will be looking at Orka on MacStadium.
Wave: on Mac Mini hardware, only Intel integrated graphics is available. The restrictions about the number of instances of macOS you can run is a business rather than a technical issue.
Surveys
Daniel: member survey is making the rounds, Emily sent reminder. Emily: trying to finalize the survey, will share results soon.
Python 3
Working group to share studio expertise?
Provide good examples of how to transition different types of code bases
Binding generation (pybind11)
OpenEXR is in process of tackling this issue, Kimball working on CMake changes to allow Python 2.7 and Python 3 to both be built alongside.
Other projects in good shape, but want updates from individual projects.
OpenTimelineIO: differences between Python 2.7 and 3 haven’t been much of an issue, except for some small Unicode issues. More issues about packaging and package management, so could use some help from the wider community.
OCIO: not there yet, in the process of writing new bindings with pybind11 which will give dual version support, 30% of the way done, should be done in a couple of months.
Cary: Python bindings for imath, already building for 2.7 and 3, plan to rewrite bindings with pybind11.
Gordon: Is there a general model that we should suggest how to do this? Struggling for a good model for Maya. Goal is to get a beta out, but depend on a number of libraries, if all the libraries choose “can only build one at a time” vs “can build in parallel” has an effect.
Kimball: assuming that for OpenEXR need to assuming that both versions need to be supported for a while to ensure transition. But OpenEXR is pretty simple. Kimball: can build in same build, but will generate separate DSOs.
Kimball: need more work your CMake files to support building and packaging 2.7 and 3 in the same build, but fairly transparent where stuff ends.
Josh: also using CMake / pybind11, but overall developer instructions have flaws, also issues when pushing to PyPI. Would really like to know “how to reach out to”.
Daniel: should we create a working group on the Python 3 topic? Would Josh like to lead this? Josh: yes, but would have to start next year. Daniel: deep in this process at Animal Logic, keen to share knowledge and learn from others.
Daniel: we should try to collect and gather efforts and resources.
Larry: Gordon, don’t hesitate to communicate with packages that Maya is waiting for movement on Python 3. Gordon: will get the Maya Python 3 lead to join the group. Larry: some library owners may not realize that their library is in the dependency tree of Maya.
Daniel: when you analyze upstream dependencies, sometimes it’s not clear there is a Python 3 equivalent of a library.
Technical Project updates
OpenVDB
OpenColorIO
Already discussed GPU / Python 3
Trying to start up more formal discussion with ACES, trying to clear up licensing on ACES-related code (aces-ocio config is used by many studios)
OpenEXR
Cary: repo was moved to ASWF GitHub organization 3 weeks ago, locked out of submitting PRs due to EasyCLA system. Was reporting that Cary’s company didn’t have a CLA on file, problem was just identified (LucasFilm vs ILM), so problem resolved now. Something to watch out for, need to watch out with dependent company names.
Daniel: had similar issues with some other transitions. What can we do to make things more obvious.
Cary: the ASWF can help with this, should be a more familiar issue if it comes up again.
JF: EasyCLA system works well but is a bit intimidating. John: has heard similar feedback from other contributors.
Larry: important that “CLA Manager” be clearly designated, and close to the project.
OpenCue
Focussed on finishing up Python 3 work, should be done mid next week, every component but 1 has been migrated. Futurize tool did 90% of the work. Not so much in terms of packaging issues though.
OpenTimelineIO
No specific updates
CI updates
Working group
Update on candidate projects
Media player
Dev Analytics preview ( https://lfanalytics.io/projects/academy-software-foundation )
John: goal is to showcase some of the analytics around or projects, more than the basic ones provided by GitHub. Answer higher level questions. Early stages of how do we pull this together. Currently pulling GitHub and Slack. Company Contributor stats currently wrong, being worked on.
Commit growth, number of commits, can break down based on contributors and organizations. Over time will be able to pull in communications info (Slack, mailing lists).
Caveat: “in beta” really should be “in alpha”, asking for feedback. Also “what are the bigger picture questions you want to answer about your project”.
JF: should this be extended to other systems in the ASWF landscape? Or at least some important ones?
Josh: could be interesting to look at stats across projects (are projects siloed vs contributing to multiple projects). John: would be useful to have insight across projects.
Brian: is this pulling in multiple repos for individual projects? John: yes, multiple Slack channels as well.