FAQs
Answers to frequently asked questions
Onboarding
Tusk will do three things when you sync your repo:
- Embed your repo for semantic understanding
- Set up the Language Server Protocol for retrieving codebase context
- Set up a test environment for the agent to self-lint and run its tests
Tusk may not be able to automatically complete the test environment set-up because of dependencies and environment variables required.
If additional one time setup is required, our team will reach out to you with next steps.
We will resolve the last-mile issues with the test environment. Turnaround time is typically 1-2 days but can be longer for more complex setups.
Contact support@usetusk.ai if you have any questions.
- VP or Director of Engineering who is in charge of quality efforts or developer productivity
- Staff/Principal Software Engineer who is most familar with your testing best practices
- Senior DevOps Engineer who is most familiar with your organization’s CI/CD pipeline
It takes 2 minutes of time to do the initial setup of connecting your version control platform and syncing your repo(s). Most organizations will be fully ready to kick off with a customized Tusk agent after half a day of effort.
Yes. Reach out to your Tusk point of contact or support@usetusk.ai and we’ll get Tusk set up on-prem for you. As to be expected, this will require more implementation time.
Product
Yes. You can choose which symbols you’d like to incorporate tests for within the Tusk web app. We are actively working on the ability to select individual tests to incorporate.
Yes. Tusk will generate new unit tests if the changes are substantial enough to warrant new tests.
Yes. Tusk will look at existing unit tests in the PR/MR and extend them while also looking for edge cases that you have not covered. We make sure to de-dupe tests by comparing all generated tests to existing tests in the PR/MR and repo.
Tusk automatically filters out tests that are “low value” (e.g., testing implementation instead of behavior), duplicates of existing tests, or failing to run properly even after multiple iterations on the test code.
There’s no hard limit on the size of PRs/MRs that Tusk can write tests for, but in general test quality is best when PRs/MRs are less then 50 files modified or 2,500 lines of code added and deleted.
Tusk looks for common refactoring patterns (e.g., high ratio of files modified vs. lines changed per file, high number of string replacements, PR/MR info mentioning words like “refactor” and “reorganize”, etc.) in the PR/MR to determine if it is a large scale refactor.
Tusk will skip test generation for large scale refactors.
Yes. Reach out to your Tusk point of contact or support@usetusk.ai and we will enable a more liberal mode for your Tusk agent. We will soon allow you to toggle this mode on and off from within the Tusk web app.
A seed test is the original happy path test that Tusk uses as an anchor from which to determine edge cases that should be explored.
This scenario happens infrequently but is caused by certain edge cases (listed below). In this scenario, Tusk decides to fail gracefully and create a separate test file for the symbol instead of dropping the tests altogether.
Potential causes:
- Tusk did not find the original test file
- Tusk failed to incorporate tests into the original test file
- Tusk found that the original test file has errors when running
Report this issue to us over Slack or at support@usetusk.ai and we’ll resolve the issue as soon as possible. In the event of #1 and #2, we will investigate to determine if it is a bug and, if so, push a fix.
Once you’ve mentioned UseTusk as a comment inside the symbol definition, you need to create a PR/MR with the comments in the diff.
Tusk will parse the UseTusk comments when reading the PR/MR, and generate a backfill of unit tests for those symbols (along with any other code changes that are relevant to test).
Billing
GitHub or GitLab username. You can activate seats for the usernames that belong to your organization/group in Settings > Seats in the Tusk web app.
Usernames without an active Tusk seat will not have Tusk’s test generation triggered in the PR/MRs that they create.
We have a minimum of 5 seats on our Team Plan. Most companies choose to start with 15 seats for a more comprehensive evaluation of Tusk’s impact on their code coverage.
Firstly, make sure that you’re the billing admin for your Tusk organization. If not, Tusk will prompt you to contact the admin via email to upgrade your plan.
- In Settings > Billing, click “Manage plan” to go to the Stripe customer portal
- Click “Send” to login securely via email (check your inbox)
- Once logged in, click “Update subscription”
- Increase the quantity of seats you currently have
- Click “Continue” and then “Confirm” to add the seats
- In Settings > Seats, click “Sync members” on the top right to fetch usernames for your organization/group
- Toggle on the team member’s seat under Status
If you’re on an Enterprise plan, contact your Tusk representative or support@usetusk.ai, and we will update your subscription.
Was this page helpful?