FAQs
Answers to frequently asked questions
Onboarding
What happens when I sync my repo?
What happens when I sync my repo?
Tusk will do three things when you sync your repo:
- Embed your repo for semantic understanding
- Set up the Language Server Protocol for retrieving codebase context
- Set up a test environment for the agent to self-lint and run its tests
After I sync my repo, it says that additional set-up is required. What happens now?
After I sync my repo, it says that additional set-up is required. What happens now?
Tusk may not be able to automatically complete the test environment set-up because of dependencies and environment variables required.
If additional one time setup is required, our team will reach out to you with next steps.
We will resolve the last-mile issues with the test environment. Turnaround time is typically 1-2 days but can be longer for more complex setups.
Contact support@usetusk.ai if you have any questions.
Who are the key stakeholders involved in implementing Tusk?
Who are the key stakeholders involved in implementing Tusk?
- VP or Director of Engineering who is in charge of quality efforts or developer productivity
- Staff/Principal Software Engineer who is most familar with your testing best practices
- Senior DevOps Engineer who is most familiar with your organization’s CI/CD pipeline
How long does it take to implement Tusk for my organization?
How long does it take to implement Tusk for my organization?
It takes 2 minutes of time to do the initial setup of connecting your version control platform and syncing your repo(s). Most organizations will be fully ready to kick off with a customized Tusk agent after half a day of effort.
Can you do on-prem?
Can you do on-prem?
Yes. Reach out to your Tusk point of contact or support@usetusk.ai and we’ll get Tusk set up on-prem for you. As to be expected, this will require more implementation time.
Product
Can I decide to incorporate only select tests from Tusk's output?
Can I decide to incorporate only select tests from Tusk's output?
Yes. You can choose which symbols you’d like to incorporate tests for within the Tusk web app. We are actively working on the ability to select individual tests to incorporate.
If I push new changes to a PR/MR's branch, will Tusk generate new tests for the change?
If I push new changes to a PR/MR's branch, will Tusk generate new tests for the change?
Yes. Tusk will generate new unit tests if the changes are substantial enough to warrant new tests.
If I have already written unit tests in my PR/MR, will Tusk still generate unit tests?
If I have already written unit tests in my PR/MR, will Tusk still generate unit tests?
Yes. Tusk will look at existing unit tests in the PR/MR and extend them while also looking for edge cases that you have not covered. We make sure to de-dupe tests by comparing all generated tests to existing tests in the PR/MR and repo.
I see in the Tusk web app that there are excluded tests, why is that?
I see in the Tusk web app that there are excluded tests, why is that?
Tusk automatically filters out tests that are “low value” (e.g., testing implementation instead of behavior), duplicates of existing tests, or failing to run properly even after multiple iterations on the test code.
What is the PR/MR size limit for Tusk's test generation?
What is the PR/MR size limit for Tusk's test generation?
There’s no hard limit on the size of PRs/MRs that Tusk can write tests for, but in general test quality is best when PRs/MRs are less then 50 files modified or 2,500 lines of code added and deleted.
How does Tusk determine if a PR/MR involves a large scale refactor?
How does Tusk determine if a PR/MR involves a large scale refactor?
Tusk looks for common refactoring patterns (e.g., high ratio of files modified vs. lines changed per file, high number of string replacements, PR/MR info mentioning words like “refactor” and “reorganize”, etc.) in the PR/MR to determine if it is a large scale refactor.
Tusk will skip test generation for large scale refactors.
Can you loosen Tusk's constraints so I can meet my mandate of always having unit tests for PR/MRs?
Can you loosen Tusk's constraints so I can meet my mandate of always having unit tests for PR/MRs?
Yes. Reach out to your Tusk point of contact or support@usetusk.ai and we will enable a more liberal mode for your Tusk agent. We will soon allow you to toggle this mode on and off from within the Tusk web app.
What is a seed test?
What is a seed test?
A seed test is the original happy path test that Tusk uses as an anchor from which to determine edge cases that should be explored.
Tusk created a different test file than what I was expecting. Why did that happen?
Tusk created a different test file than what I was expecting. Why did that happen?
This scenario happens infrequently but is caused by certain edge cases (listed below). In this scenario, Tusk decides to fail gracefully and create a separate test file for the symbol instead of dropping the tests altogether.
Potential causes:
- Tusk did not find the original test file
- Tusk failed to incorporate tests into the original test file
- Tusk found that the original test file has errors when running
Report this issue to us over Slack or at support@usetusk.ai and we’ll resolve the issue as soon as possible. In the event of #1 and #2, we will investigate to determine if it is a bug and, if so, push a fix.
I added the UseTusk comment in my IDE, how do I trigger Tusk for a backfill?
I added the UseTusk comment in my IDE, how do I trigger Tusk for a backfill?
Once you’ve mentioned UseTusk as a comment inside the symbol definition, you need to create a PR/MR with the comments in the diff.
Tusk will parse the UseTusk comments when reading the PR/MR, and generate a backfill of unit tests for those symbols (along with any other code changes that are relevant to test).
Billing
What does a Tusk seat refer to?
What does a Tusk seat refer to?
GitHub or GitLab username. You can activate seats for the usernames that belong to your organization/group in Settings > Seats in the Tusk web app.
Usernames without an active Tusk seat will not have Tusk’s test generation triggered in the PR/MRs that they create.
How many seats do teams typically start with?
How many seats do teams typically start with?
We have a minimum of 5 seats on our Team Plan. Most companies choose to start with 15 seats for a more comprehensive evaluation of Tusk’s impact on their code coverage.
How do I add more seats to my plan?
How do I add more seats to my plan?
Firstly, make sure that you’re the billing admin for your Tusk organization. If not, Tusk will prompt you to contact the admin via email to upgrade your plan.
- In Settings > Billing, click “Manage plan” to go to the Stripe customer portal
- Click “Send” to login securely via email (check your inbox)
- Once logged in, click “Update subscription”
- Increase the quantity of seats you currently have
- Click “Continue” and then “Confirm” to add the seats
- In Settings > Seats, click “Sync members” on the top right to fetch usernames for your organization/group
- Toggle on the team member’s seat under Status
If you’re on an Enterprise plan, contact your Tusk representative or support@usetusk.ai, and we will update your subscription.