Skip to content

Add Test pass for build artifacts #1027

@reinecke

Description

@reinecke

Feature Request

With github actions producing wheels to deliver to pypi, it would be helpful to have a separate part of our build process to validate those exact artifacts before they can become candidates for pypi delivery.

Description

At a minimum, smoke tests could be run that did something simple like:

  1. Import otio
  2. create an example timeline
  3. write it to disk
  4. read the timeline back into disk

The testing should be run on target platforms called out as supported.

In a perfect world, the full unittest suite would be run against the wheels that are being delivered.

The main objective of this testing should be to catch packaging issues like:

  • Not building statically linked to libopentimelineio or libopentime or missing .so files for those in the package (whatever the chosen method of including those functions is)
  • Incorrect architecture mismatch in the binary parts of the distribution
  • Missing _otio module from the package distribution

Context

Over the course of improving our build system setup, the above listed issues have been hallmark things that pop up when iterating on our somewhat brittle combo of cmake configs and setup.py. A sanity check that this class of issues hasn't made it into our artifacts will help catch issues close to where they are introduced and increase our confidence when iterating on the build setup.

Some things to consider:

  • In testing, it is important to fail as fast as possible. I think we'd favor relying on the build time unittests a bit more in exchange for faster iteration time on build failures.
  • At the time of writing, there are 17 wheels being built. Running tests to cover every one of these may not be worth the effort. It may be best to choose a representative set of platform/python version combos to maximize coverage while minimizing the variant combos of test machines.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    To do

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions