Transcript miller-Metronome

Improving Software with
the UW Metronome
Becky Gietzel
Todd L Miller
Three Ways to Improve Software



Automated Builds and Tests
Continuous Integration
Platform and Version Testing
Talk Overview



Introduction to the UW Metronome
Use Case: Condor Developers
Getting Started with the UW Metronome
The Metronome at UW



An automated build and test facility at
UW-Madison
Infrastructure (“BaTLab”) maintains
services and fifty distinct platforms.
Software (“Metronome”) reproducibly
and reliably runs builds or tests of your
software.
Metronome Philosophy

Reproducibility.
• The benefits of reproducible testing need not
•

be further elaborated.
Reproducible builds
• don’t have to be stored (for very long).
• don’t require a specific person or machine to build.
• simplify life-cycle management.
Support, rather than restrict, users.
Advantages of Metronome




Not tied to a particular tool or language.
Automatically-generated database of
build and test results, with web interface.
Uses Condor to distribute runs.
The UW Metronome maintains an
infrastructure on your behalf.
Metronome’s Web Interface
Usage Concepts




Platforms
Prerequisites
Input & Output Specifications
Ordered Commands
Ordered Commands
Submit Node
Execute
Node
Execute
ExecuteNode
Node
pre_all
remote_pre_declare
platform_pre
platform_pre
platform_pre
remote_declare
remote_pre
platform_post
platform_post
platform_post
post_all
remote_task
remote_post
Use Case: Condor Developers

Fully automatic builds and tests.
Separate build and test runs.

Used for Condor release binaries.

• Easy to see if build or test failed.
• Easy to fetch a particular build.
• Easy to develop the test suite.
• Enables cross-platform testing.
Condor Developers (2)




Nightly builds and tests.
Hourly runs on a subset of platforms.
• Windows/Linux pairing especially helpful.
Tests patched OSes.
Still want lower-latency testing.
Condor Developers (3)

“Nice” features.

“Big” features.
• Printing URL of run results on submission.
• Streaming user-level output and error.
• Speed improvements on submit nodes.
• Concurrent testing.
• Major web interface additions.
Getting Started

Metronome documentation
• nmi.cs.wisc.edu
Build and Test Lab info
• nmi.cs.wisc.edu/account_request
• nmi.cs.wisc.edu/support
Live Lab status
• nmi-web.cs.wisc.edu/nmi

Any questions?