Universum modes
Along with default mode Universum provides the following useful additional commands:
Initialize Universum
If you have Universum installed via pip3.7 install -U universum
or downloaded it from GitHub and changed
working directory to project root, run python3.7 -m universum init
to create an example configuration file,
that can be updated according to your project needs later.
Running this command will not only create a configuration file, but will also provide you with a command line to execute it with Universum.
Run Universum locally (Non-CI mode)
The python3.7 -m universum run
subcommand uses provided Universum config to execute
the set up scenario locally. This allows to easily perform all required checks before submitting a change
to VCS. This mode can generally be used as a wrapper for a complex build system.
Universum in non-CI mode has the following differences from default mode:
It does not require most of build parameters - especially the VCS-related ones.
It does not report build results to any code review system.
It works with sources ‘in place’, without copying.
It automatically cleans the artifact folder before the build.
usage: python3.7 -m universum run [-h] [--out {console,file}]
[--config CONFIG_PATH]
[--filter STEP_FILTER]
[--project-root PROJECT_ROOT]
[--out-type {tc,term,jenkins,github}]
[--html-log [OUTPUT.HTML_LOG]]
[--artifact-dir ARTIFACT_DIR] [--no-archive]
[--report-build-start]
[--report-build-success]
[--report-only-fails]
[--report-only-fails-short]
[--report-no-vote] [--fail-unsuccessful]
[--server-type {tc,jenkins,local}]
[--tc-server TEAMCITY_SERVER]
[--tc-build-id BUILD_ID]
[--tc-configuration-id CONFIGURATION_ID]
[--tc-auth-user-id TC_USER]
[--tc-auth-passwd TC_PASSWD]
[--jenkins-build-url BUILD_URL]
[--github-server GITHUB_SERVER_URL]
[--github-repo GITHUB_REPOSITORY]
[--github-run-id GITHUB_RUN_ID]
Output
- --out, -o
Possible choices: console, file
Define whether to print build logs to console or file. Log file names are generated based on the names of build steps. By default, logs are printed to console when the build is launched on Jenkins or TeamCity agent
- --out-type, -ot
Possible choices: tc, term, jenkins, github
Type of output to produce (tc - TeamCity, jenkins - Jenkins, term - terminal, github - Github Actions). TeamCity, Jenkins and Github Actions environments are detected automatically when launched on build agent.
- --html-log, -hl
Generate a self-contained user-friendly HTML log. Pass a desired log name as a parameter to this option, or a default ‘universum_log.html’ will be used.
Configuration execution
External command launching and reporting parameters
- --config, -cfg
Path to project configuration file (example: -cfg=my/project/my_conf.py). Default is
.universum.py
Environment variable: $CONFIG_PATH
- --filter, -f
- Allows to filter which steps to execute during launch. String value representing single filter or a set of filters separated by ‘:’. To define exclude pattern use ‘!’ symbol at the beginning of the pattern.A Universum step match specified pattern when ‘filter’ is a substring of step ‘name’. This functionality is similar to ‘boosttest’ and ‘gtest’ filtering, except special characters (like ‘*’, ‘?’, etc.) are ignored.Examples:* -f=’run test’ - run only steps that contain ‘run test’ substring in their names* -f=’!run test’ - run all steps except those containing ‘run test’ substring in their names* -f=’test 1:test 2’ - run all steps with ‘test 1’ OR ‘test 2’ substring in their names* -f=’test 1:!unit test 1’ - run all steps with ‘test 1’ substring in their names except those containing ‘unit test 1’
Source files
Parameters determining the processing of repository files
- --project-root, -pr
Temporary directory to copy sources to. Default is ‘temp’
Environment variable: $PROJECT_ROOT
Artifact collection
Parameters of archiving and collecting of build artifacts
- --artifact-dir, -ad
Directory to collect artifacts to. Default is ‘artifacts’
Environment variable: $ARTIFACT_DIR
- --no-archive
By default all directories noted as artifacts are copied as .zip archives. This option turn archiving off to copy bare directories to artifact directory
Default: False
Result reporting
Build results collecting and publishing parameters
- --report-build-start, -rst
Send additional comment to review system on build started (with link to log)
Default: False
- --report-build-success, -rsu
Send comment to review system on build success (in addition to vote up)
Default: False
- --report-only-fails, -rof
Include only the list of failed steps to reporting comments
Default: False
- --report-only-fails-short, -rofs
Include only the short list of failed steps to reporting comments
Default: False
- --report-no-vote, -rnv
Do not vote up/down review depending on result
Default: False
- --fail-unsuccessful, -rfu
Return non-zero exit code if any step failed
Default: False
Automation server
Automation server options
- --server-type, -st
Possible choices: tc, jenkins, local
Type of environment to refer to (tc - TeamCity, jenkins - Jenkins, local - user local terminal). TeamCity and Jenkins environment is detected automatically when launched on build agent
TeamCity variables
TeamCity-specific parameters
- --tc-server, -ts
TeamCity server URL
Environment variable: $TEAMCITY_SERVER
- --tc-build-id, -tbi
teamcity.build.id
Environment variable: $BUILD_ID
- --tc-configuration-id, -tci
system.teamcity.buildType.id
Environment variable: $CONFIGURATION_ID
- --tc-auth-user-id, -tcu
system.teamcity.auth.userId
Environment variable: $TC_USER
- --tc-auth-passwd, -tcp
system.teamcity.auth.password
Environment variable: $TC_PASSWD
Jenkins variables
Jenkins-specific parameters
- --jenkins-build-url, -jbu
Link to build on Jenkins (automatically set by Jenkins)
Environment variable: $BUILD_URL
GitHub Actions variables
GitHub sets these environment variables and usually do not need to set these arguments. See: https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables
- --github-server
GitHub server URL, used on GitHub Actions for constructing links to builds. For example: https://github.com
Environment variable: $GITHUB_SERVER_URL
- --github-repo
GitHub owner and repository name, used on GitHub Actions for constructing links to builds. For example: octocat/Hello-World
Environment variable: $GITHUB_REPOSITORY
- --github-run-id
GitHub run ID, used on GitHub Actions for constructing links to builds.For example: 1658821493
Environment variable: $GITHUB_RUN_ID
Poll chosen VCS for updates
This mode was created for CI systems that do not have polling feature at all or that are not able to check each change individually.
When launched in poller mode, Universum uses a simple database to remember latest checked commit for every passed VCS source, such as Git or P4 branch. On next launches it consults that database and calculates a list of updates (commits, submits) in that VCS source since last change saved in DB. After that, using an URL defined in parameters, it triggers a build in CI system for each of those changes (instead of testing only the current VCS state).
This mode allows to locate a source of behaviour changes more precisely.
Note
Even being a poller, Universum in this mode does not launch automatically. Please use some outer means (such as cron or any other time-based auto-launcher) for periodical checks.
Here are the parameters for poller mode:
usage: python3.7 -m universum poll [-h] [--file DB_FILE] [--num MAX_NUMBER]
[--out-type {tc,term,jenkins,github}]
[--html-log [OUTPUT.HTML_LOG]]
[--vcs-type VCS_TYPE]
[--project-root PROJECT_ROOT]
[--git-repo GIT_REPO]
[--git-refspec GIT_REFSPEC]
[--p4-project-depot-path P4_PATH]
[--p4-mappings P4_MAPPINGS [P4_MAPPINGS ...]]
[--p4-port P4PORT] [--p4-user P4USER]
[--p4-password P4PASSWD]
[--server-type {tc,jenkins,local}]
[--tc-server TEAMCITY_SERVER]
[--tc-build-id BUILD_ID]
[--tc-configuration-id CONFIGURATION_ID]
[--tc-auth-user-id TC_USER]
[--tc-auth-passwd TC_PASSWD]
[--jenkins-trigger-url URL]
[--github-server GITHUB_SERVER_URL]
[--github-repo GITHUB_REPOSITORY]
[--github-run-id GITHUB_RUN_ID]
Named Arguments
- --file, -f
File to store last known CLs
Environment variable: $DB_FILE
Default: “p4poll.json”
- --num, -n
Maximum number of CLs processed, default is 10
Environment variable: $MAX_NUMBER
Default: 10
Output
Log appearance parameters
- --out-type, -ot
Possible choices: tc, term, jenkins, github
Type of output to produce (tc - TeamCity, jenkins - Jenkins, term - terminal, github - Github Actions). TeamCity, Jenkins and Github Actions environments are detected automatically when launched on build agent.
- --html-log, -hl
Generate a self-contained user-friendly HTML log. Pass a desired log name as a parameter to this option, or a default ‘universum_log.html’ will be used.
Source files
- --vcs-type, -vt
Possible choices: none, p4, git, gerrit, github
Select repository type to download sources from: Perforce (‘p4’), Git (‘git’), Gerrit (‘gerrit’), GitHub (‘github’) or a local directory (‘none’). Gerrit uses Git parameters. Each VCS type has its own settings.
Environment variable: $VCS_TYPE
- --project-root, -pr
Temporary directory to copy sources to. Default is ‘temp’
Environment variable: $PROJECT_ROOT
Git
Git repository settings
- --git-repo, -gr
See your project home page for exact repository identifier, passed to ‘git clone’. If using SSH, ‘–git-repo’ format is ‘ssh://user@server:port/detailed/path’
Environment variable: $GIT_REPO
- --git-refspec, -grs
Any additional refspec to be fetched
Environment variable: $GIT_REFSPEC
Perforce
- --p4-project-depot-path, -p4d
Depot path to get sources from (starts with ‘//’, ends with ‘/…’Only supports one path. Cannot be used with ‘–p4-mappings’ option
Environment variable: $P4_PATH
- --p4-mappings, -p4m
P4 mappings. Cannot be used with ‘–p4-project-depot-path’ option. Use the following format: ‘//depot/path/… /local/path/…’, where the right half is the same as in real P4 mappings, but without client name. Just start from client root with one slash. For more than one add several times or split with ‘,’ character
Environment variable: $P4_MAPPINGS
- --p4-port, -p4p
P4 port (e.g. ‘myhost.net:1666’)
Environment variable: $P4PORT
- --p4-user, -p4u
P4 user name
Environment variable: $P4USER
- --p4-password, -p4P
P4 password (please note, this should be exactly password, not the ticket)
Environment variable: $P4PASSWD
Automation server
Automation server options
- --server-type, -st
Possible choices: tc, jenkins, local
Type of environment to refer to (tc - TeamCity, jenkins - Jenkins, local - user local terminal). TeamCity and Jenkins environment is detected automatically when launched on build agent
TeamCity variables
TeamCity-specific parameters
- --tc-server, -ts
TeamCity server URL
Environment variable: $TEAMCITY_SERVER
- --tc-build-id, -tbi
teamcity.build.id
Environment variable: $BUILD_ID
- --tc-configuration-id, -tci
system.teamcity.buildType.id
Environment variable: $CONFIGURATION_ID
- --tc-auth-user-id, -tcu
system.teamcity.auth.userId
Environment variable: $TC_USER
- --tc-auth-passwd, -tcp
system.teamcity.auth.password
Environment variable: $TC_PASSWD
Jenkins variables
Jenkins-specific parameters
- --jenkins-trigger-url, -jtu
Url to trigger, must include exactly one conversion specifier (%s) to be replaced by CL number, for example: http://localhost/%s
Environment variable: $URL
GitHub Actions variables
GitHub sets these environment variables and usually do not need to set these arguments. See: https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables
- --github-server
GitHub server URL, used on GitHub Actions for constructing links to builds. For example: https://github.com
Environment variable: $GITHUB_SERVER_URL
- --github-repo
GitHub owner and repository name, used on GitHub Actions for constructing links to builds. For example: octocat/Hello-World
Environment variable: $GITHUB_REPOSITORY
- --github-run-id
GitHub run ID, used on GitHub Actions for constructing links to builds.For example: 1658821493
Environment variable: $GITHUB_RUN_ID
Detect changes and submit them automatically
Unlike default mode, Universum in submit mode requires an already prepared local repository. For example:
In case of Git:
the repo should be already cloned
the required branch should be already checked out
In case of P4:
the client should be already created
the directory should be already synced
all the required shelves should be applied
After doing that, any additional changes done to source code (made manually or by script execution) will be detected by Universum submitter and added to VCS with specified description on behalf of specified user.
Here are the parameters for submitter mode:
usage: python3.7 -m universum submit [-h] [--create-review] [--edit-only]
[--commit-message COMMIT_MESSAGE]
[--reconcile-list RECONCILE_LIST]
[--out-type {tc,term,jenkins,github}]
[--html-log [OUTPUT.HTML_LOG]]
[--vcs-type VCS_TYPE]
[--project-root PROJECT_ROOT]
[--git-user GITUSER]
[--git-email GITEMAIL]
[--git-repo GIT_REPO]
[--git-refspec GIT_REFSPEC]
[--p4-client P4CLIENT] [--p4-port P4PORT]
[--p4-user P4USER]
[--p4-password P4PASSWD]
Named Arguments
- --create-review
create deletable review (shelve for P4, temp branch for Git) instead of actual submitting to repo
Default: False
- --edit-only
Only submit existing vcs modifications, no adding or deleting
Default: False
- --commit-message, -cm
Commit message. Enter in command line directly, store in environment variable or pass a file path to read the message from by starting the value string with ‘@’. File path can be either absolute or relative. Please note, that when passing a file, it is expected to be in UTF-8 encoding
Environment variable: $COMMIT_MESSAGE
- --reconcile-list, -rl
Comma-separated or linebreak-separated list of files or directories to be reconciled for commit (relative paths starting at client root are supported). To use command line directly, add quotes if needed (e.g.
-rl 'target1, target2'
). To use a file with reconcile list, start a path to file with ‘@’ (e.g.-rl @/path/to/file
).Please note, that when passing a file, it’s expected to be in UTF-8 encodingEnvironment variable: $RECONCILE_LIST
Output
Log appearance parameters
- --out-type, -ot
Possible choices: tc, term, jenkins, github
Type of output to produce (tc - TeamCity, jenkins - Jenkins, term - terminal, github - Github Actions). TeamCity, Jenkins and Github Actions environments are detected automatically when launched on build agent.
- --html-log, -hl
Generate a self-contained user-friendly HTML log. Pass a desired log name as a parameter to this option, or a default ‘universum_log.html’ will be used.
Source files
- --vcs-type, -vt
Possible choices: none, p4, git, gerrit, github
Select repository type to download sources from: Perforce (‘p4’), Git (‘git’), Gerrit (‘gerrit’), GitHub (‘github’) or a local directory (‘none’). Gerrit uses Git parameters. Each VCS type has its own settings.
Environment variable: $VCS_TYPE
- --project-root, -pr
Temporary directory to copy sources to. Default is ‘temp’
Environment variable: $PROJECT_ROOT
Git
- --git-user, -gu
Git user name for submitting
Environment variable: $GITUSER
- --git-email, -ge
Git user email for submitting
Environment variable: $GITEMAIL
- --git-repo, -gr
See your project home page for exact repository identifier, passed to ‘git clone’. If using SSH, ‘–git-repo’ format is ‘ssh://user@server:port/detailed/path’
Environment variable: $GIT_REPO
- --git-refspec, -grs
Any additional refspec to be fetched
Environment variable: $GIT_REFSPEC
Perforce
- --p4-client, -p4c
Existing P4 client (workspace) name to use for submitting
Environment variable: $P4CLIENT
- --p4-port, -p4p
P4 port (e.g. ‘myhost.net:1666’)
Environment variable: $P4PORT
- --p4-user, -p4u
P4 user name
Environment variable: $P4USER
- --p4-password, -p4P
P4 password (please note, this should be exactly password, not the ticket)
Environment variable: $P4PASSWD
GitHub Handler
GitHub Handler is a Universum mode that serves as GitHub Application, helping to perform and report checks on new commits to a repository. It can create new check runs on GitHub and trigger an already set up automation server to perform these checks. GitHub Handler parses all required params and passes them to the triggered builds.
For GitHub Handler to work, these parameters are mandatory:
--payload
--event
--trigger-url
--github-app-id
--github-private-key
These and other parameters are described below.
usage: python3.7 -m universum github-handler [-h] [--event GITHUB_EVENT]
[--payload GITHUB_PAYLOAD]
[--trigger-url TRIGGER_URL]
[--suite-name SUITE_NAME]
[--target-repo TARGET_REPO]
[--verbose]
[--github-app-id GITHUB_APP_ID]
[--github-private-key GITHUB_PRIVATE_KEY]
[--out-type {tc,term,jenkins,github}]
[--html-log [OUTPUT.HTML_LOG]]
Named Arguments
- --event, -e
Currently parsed from “x-github-event” header of received web-hook
Environment variable: $GITHUB_EVENT
- --payload, -pl
Full contents of web-hook received; pass a JSON file path, starting it with “@” character, or use an environment variable for convenience. Please note, that when passing a file, it is expected to be in UTF-8 encoding
Environment variable: $GITHUB_PAYLOAD
- --trigger-url, -tu
URL for GET request to trigger the CI build and pass all parameters parsed from payload; if any constant parameters (like token) are required, include them in this string as well, e.g.: “http://jenkins.com/job/JobName/build?token=MYTOKEN”
Environment variable: $TRIGGER_URL
- --suite-name, -sn
GitHub check suite name, default is “CI tests”
Environment variable: $SUITE_NAME
Default: “CI tests”
- --target-repo, -tr
Repository name to process; leave blank to process any repo
Environment variable: $TARGET_REPO
- --verbose, -v
Show all params passed in URL (mostly for debug purposes)
Default: False
GitHub
GitHub repository settings
- --github-app-id, -gta
GitHub application ID to use for check run report. Only GitHub App can report a check run result! If you don’t have an App for reporting purposes, please don’t use
--report-to-review
with GitHubEnvironment variable: $GITHUB_APP_ID
- --github-private-key, -gtk
GitHub App private key for obtaining installation authentication token. Pass raw key data via environment variable or pass a file path to read the key from by starting the value string with ‘@’. File path can be either absolute or relative. Please note, that when passing a file, it is expected to be in UTF-8 encoding
Environment variable: $GITHUB_PRIVATE_KEY
Output
Log appearance parameters
- --out-type, -ot
Possible choices: tc, term, jenkins, github
Type of output to produce (tc - TeamCity, jenkins - Jenkins, term - terminal, github - Github Actions). TeamCity, Jenkins and Github Actions environments are detected automatically when launched on build agent.
- --html-log, -hl
Generate a self-contained user-friendly HTML log. Pass a desired log name as a parameter to this option, or a default ‘universum_log.html’ will be used.