Best Practices
Developer Experience
Ideal workflows for using Tusk as a developer
Overview
Tusk generates unit tests for your code in 3 main ways:- PR/MR check, which is triggered when a PR/MR (draft or otherwise) is raised or a new commit is pushed to the PR/MR’s branch. This is a non-blocking check by default, akin to AI code review but with the added benefit of scaling your testing infra. Read more.
-
Manual trigger, which requires you to add the
Tusk - Generate TestsGitHub/GitLab label to the PR/MR. This label is automatically created when you connect Tusk to your GitHub organization or GitLab group/project. - CoverBot, a scheduled pipeline that runs daily or weekly and automatically raises a PR/MR with unit tests that help backfill coverage for untested parts of your repo. This is an async workflow that occurs at a fixed interval. Read more.
For brevity, any mention of “PR” henceforth refers to both GitHub pull requests and GitLab merge requests.
PR Check
⏱️ For Typical PR
1
Create a draft PR
Software engineer creates a draft PR with their code changes.
2
Tusk generates tests
On PR creation, Tusk takes 8-15 mins to generate, run, and iterate on its unit tests (latency varies depending on PR size and complexity).
3
Review Tusk's tests
On completion, engineer clicks on Tusk’s PR comment and reviews the test generation output in the Tusk web app.
4
Commit Tusk's tests
Engineer selects which unit tests they want to commit and clicks Commit tests in the Tusk web app.
Once you commit Tusk’s tests, the agent stops re-running test generation to avoid a loop. If you want Tusk to generate more tests (e.g., after adding new changes), add the Tusk - Generate Tests GitHub/GitLab label to the PR/MR.
5
Fix failing tests
If there are failing tests, engineer checks out the branch from within their IDE and fixes the bug in their code causing the failing test. Tusk labels failing tests with a
[Tusk] FAILING TEST comment in the test file for ease of reference.Engineer re-runs the test file to make sure the failing test now passes, removes the [Tusk] FAILING TEST comment, and pushes the fix.6
Mark PR as ready
Engineer marks the PR as ready for review.
7
Merge the PR
Engineer goes through typical PR review and approval workflow and then merges the PR.
🔥 For Hotfix
1
Create a PR
Software engineer creates a PR ready for review.
2
Merge the PR with the hotfix
On PR creation, Tusk will start its test generation run. Engineer gets PR with the hotfix approved and merged before run completes.
3
Tusk generates tests
Tusk continues generating tests despite the PR being merged. Once completed, Tusk leaves a comment on the merged PR linking to the test generation output.
4
Create separate Tusk PR
Engineer selects which unit tests they want and clicks Create PR in the test generation output page in the Tusk web app. Tusk creates a separate PR containing the selected tests off of the original PR’s base branch.
5
Fix failing tests
If there are failing tests, engineer checks out the branch from within their IDE and fixes the bug in their code causing the failing test. Tusk labels failing tests with a
[Tusk] FAILING TEST comment in the test file for ease of reference.Engineer re-runs the test file to make sure the failing test now passes, removes the [Tusk] FAILING TEST comment, and pushes the fix.6
Approve and merge the Tusk PR
Engineer goes through typical PR review workflow before approving and merging the Tusk PR with the selected tests.
Manual Trigger
🏷️ Human Control
1
Edit preferences
Go to Settings > Seats > Edit preferences in the Tusk web app.
2
Disable automatic test runs
Toggle on “User preferences” and uncheck the “Run on new commits” setting under Test Check Configuration.
3
Create a PR
Write code and create a PR once at a good stopping point.
4
Trigger test generation
Add the
Tusk - Generate Tests GitHub/GitLab label to the PR/MR to generate unit tests for your changes.5
Tusk generates tests
Tusk takes 8-15 mins to generate, run, and iterate on its unit tests (latency varies depending on PR size and complexity). On completion, Tusk leaves a comment with the test summary and removes the label on the PR.
6
Commit Tusk's tests
Engineer selects which unit tests they want to commit and clicks Commit tests in the Tusk web app.
7
Fix failing tests
If there are failing tests, engineer checks out the branch from within their IDE and fixes the bug in their code causing the failing test. Tusk labels failing tests with a
[Tusk] FAILING TEST comment in the test file for ease of reference.Engineer re-runs the test file to make sure the failing test now passes, removes the [Tusk] FAILING TEST comment, and pushes the fix.8
Merge the PR
Engineer goes through typical PR review and approval workflow and then merges the PR.
⏮️ Manual Backfill
1
Find a past PR
Engineer goes to a past merged or closed PR to backfill unit tests for. They make sure the changes are not outdated.
2
Trigger test generation
Engineer adds the
Tusk - Generate Tests GitHub/GitLab label to the PR/MR to generate unit tests for the code changes.3
Tusk generates tests
Tusk takes 8-15 mins to generate, run, and iterate on its unit tests (latency varies depending on PR size and complexity). Once completed, Tusk leaves a comment on the merged/closed PR linking to the test generation output and removes the label.
4
Create separate Tusk PR
From the test generation output page, engineer selects which unit tests they want and clicks Create PR. Tusk creates a separate PR containing the selected tests off of the original PR’s base branch.
5
Fix failing tests
If there are failing tests, engineer checks out the branch from within their IDE and fixes the bug in their code causing the failing test. Tusk labels failing tests with a
[Tusk] FAILING TEST comment in the test file for ease of reference.Engineer re-runs the test file to make sure the failing test now passes, removes the [Tusk] FAILING TEST comment, and pushes the fix.6
Approve and merge the Tusk PR
Engineer goes through typical PR review workflow before approving and merging the Tusk PR with the selected tests.
CoverBot
1
CoverBot runs on schedule
CoverBot runs at its scheduled interval, typically weekly on a Mon or Tue at 9:00am local time.
2
CoverBot creates a PR
CoverBot generates unit tests and creates a PR with passing tests for recently modified files with low coverage. Note that chosen files are batched according to similarity.
3
Review CoverBot's tests
The code owner or default assigned reviewer for CoverBot PRs reviews CoverBot’s unit tests to make sure they are valid.
4
Modify tests (optional)
If modifications are needed, reviewer checks out the branch from within their IDE and modifies or removes unit tests as needed. Reviewer re-runs the test file to make sure it executes properly, then pushes the changes to the same branch.
5
Handle detected bugs (optional)
If any potential bugs are detected, reviewer copies Tusk’s output from the PR description, creates a ticket, and assigns it to the code owner or themself to resolve.
6
Merge the CoverBot PR
Reviewer approves and merges the CoverBot PR to increase project coverage.
CoverBot will not generate a new PR until the previous PR (for the same test execution environment) is either merged or closed, so being timely about merging the PR helps you avoid missing a scheduled run.