AI-Assisted Application Review

you can now accelerate candidate reviews by using ai to evaluate candidates in application review based on criteria you set you can opt in to use ai features in the opt in features https //app ashbyhq com/admin/organizational settings/early access section of admin the ai assisted application review feature is available on the foundations, legacy plus, plus and enterprise plans foundations legacy plus plus enterprise ✅ ✅ ✅ ✅ overview define your criteria to define your criteria, click into one of your jobs from the jobs > admin https //app ashbyhq com/jobs/all page you can set your criteria in your job’s ai features tab hit set up criteria to start configuring your criteria on the following page, you have two options to add criteria add empty criterion click + add criteria; you’ll then be asked to add in a title and a prompt if you have quite complicated criteria, we’d recommend breaking this down into separate elements there is some additional guidance further below on generating effective criteria title the name of your criterion this title will be displayed in your job settings, on the application review page and will also be the name of the filter generated for this criterion prompt this is what will be used to evaluate applicants the more precise and clear your criteria, the better the ai will perform in evaluating candidates search criteria pool you can click search pool for suggestions generated by ai based on the job description in your posting you can also use this option to add criteria previously used for other roles please note you will only see the criteria previously used for only the other roles you have access to when viewing suggestions or previously used criteria, you can click add in the action column to add the criteria to your role validating criteria once you’ve set up your criteria, click validate your criteria will then be validated for clarity to ensure that the criteria are objective and that is can be evaluated by the ai accurately it will also be checked to ensure there are no eeo violations (either implicit or explicit) if any criteria have issues, this will be raised click review criteria to take a look at any warnings to review specifics, click show details listed underneath any warnings you can then make adjustments if needed and re validate, continue with your original criteria or opt to remove the criterion completely once you’ve made any amendments and your criteria have been validated, click save and evaluate all active candidates to kick off the ai evaluation if you are making adjustments to existing criteria, you will see save and re evaluate all active job considerations ashby’s ai criteria evaluation will then assess candidates (in particular, candidate resumes) and return an evaluation of each candidate against each criterion how do i create good criteria? a general guideline write your criteria as if you were describing them to a pair of junior recruiters you’ll want to phrase the prompt in such a way to make sure that they both understand the same (and correct) version of what you’re asking below you can find some additional guidance for setting up great criteria to ensure you use the tool as effectively as possible stand alone criteria should be able to be understood on its own without further context “can appropriately assimilate feedback” is vague as to whether the feedback is in regards to the applicant’s own performance or something else like product related feedback it could be more effectively rephrased as "effectively incorporates feedback related to personal job performance" singular ensure each criterion focuses on a single skill or type of requirement for example, splitting up “has experience in python, typescript, and aws experience” into 3 separate requirements “has experience building application backends in python”, “has experience building b2b saas applications in typescript” and “has experience building microservices architectures in aws” a good rule to follow is if you find your criteria has lots of ands in it, you may want to split it up into multiple criteria clear avoid vague terms instead of saying "has a technical background," specify the exact type of technical expertise you're looking for specific provide detailed requirements instead of "experienced," mention the number of years of experience or the exact role they should have held unambiguous stay away from general terms like "good communication skills " be more specific about the context, such as "experience in public speaking" or "proficient in technical writing " objectively verifiable from the resume criteria should be objectively verifiable from the candidate’s resume while “has good taste” may generally indicate what you’re looking for, it is unlikely to yield positive results in the same way that “has led design for consumer saas products with millions of users” would application form criteria you can also set criteria assessing whether a candidate answered a specific application form question or not within the application form criteria section you can find this underneath the ai job criteria you’ve configured you’ll have the option to enable the did the application answer the question? option for individual application form questions these will then be added as criteria to be used when candidates are reviewed please note, this criteria only determines whether or not the question was answered and not the quality or accuracy of the answer blank or unrelated answers will be flagged as not meeting the criteria application form criteria can only be enabled for long, unformatted text field questions on your application form reviewing candidate evaluations once your active candidates have been evaluated, you’ll be able to view the full evaluation both in the feedback tab on the candidate’s profile and when viewing the candidate in the bulk application review tool you can click on any of the specified criteria to see all reasoning behind the criteria being labelled as meets, does not meet, or undecided you can click expand all to view all reasoning across all criteria you can also filter candidates by whether or not they meet certain criteria from within the stages of your pipeline click the ai criteria quick filter option to open up the full list of your criteria and select the evaluation status you’d like to filter by for each criterion you can then filter your applicants accordingly and review or take bulk action on the filtered candidates as needed ai criteria can also be combined with other applicant filters that would be normally used automated processing legal notice you can head to admin > data management > data privacy and compliance https //app ashbyhq com/admin/organizational settings/data compliance to enable and configure a legal notice to inform candidates that their data may be processed using ai and give them opt out options is enabled? if this option is disabled, the candidates will not be shown the legal notice rule scope configure filters here if the legal notice should only be shown for specific jobs if filters aren’t added here, the notice will be shown for all roles hide scores if a candidate hasn’t seen this legal notice if you enable this option, then if a candidate has not seen this legal notice, ai informed scores will be hidden this will apply to all jobs in the rule’s scope if hide scores is enabled, a message will display on the candidate's evaluation results that says "the candidate has not seen the legal notice required by this job " this can happen when a candidate is added to ashby through a path other than the ashby hosted job application, such as a direct referral, a manual add, the chrome extension, or possibly indeed apply legal notice text configure the text that will be shown to candidates for jobs that match your filters here how this looks to the candidate when the candidate is on the application tab of a job, a learn more link shows under the submit application button if you have enabled the automated processing legal notice and the job matches the criteria set in the rule scope section (if any was set) the candidate can click this link to then view the configured legal notice and opt out if they’d like to or check their opt out status using their email address if they choose to opt out, a confirmation link will be sent to the entered email address so they can confirm if they check their status, if they’re opted out, they will have the option to opt back in ai opt out filters in your candidate pipeline, you can filter to see all candidates who have opted out of ai processing by following the below steps navigate to the candidate pipeline enable the advanced filters toggle on the right of the screen select + add field to match on the left search for "ai opt out" in the search bar and select the criterion select equals = yes this criteria is available when building reports as well ai credits the ai assisted application review feature is available on all plans and is priced per credit 1 credit = 1 evaluation of a job consideration, with a max of 50 criteria please refer to the table below as to how many credits are included for each plan foundations ai credits legacy plus ai credits plus ai credits enterprise ai credits 1,500/mo or 18,000 per year account wide 1,500/mo or 18,000 per year account wide 2,500/yr, per paid seat 12,500/yr, per paid seat for customers on annual plans, additional credits can be purchased in blocks of 5,000 at $0 10 usd per credit (so $500 usd for 5000 credits ) for monthly foundations customers, additional credits can be purchased in blocks of 1,000 at $0 10 usd per credit (so $100 usd for 1000 credits) for more on purchasing additional credits, check out how do i purchase additional email lookups, ai credits or texting credits in app? docid\ edqz9zv9vdcljyzsxjdhe faq ai assisted application review feature if i edit or amend my criteria, can i re evaluate candidates? yes if you edit a prompt or add or remove criteria, you’ll see a save and re evaluate all active job considerations option click this to save your changes and re evaluate candidates with your new criteria can i flag ai evaluations if they’re not quite right? yes! you can click the three dots in the top right corner of the evaluation to open up the option to flag it as incorrect or misleading can i add the evaluations as a column on the application review page? you can! click add column when viewing candidates on the application review page, then select job consideration>criteria>evaluations please be aware you may need to scroll to the right or remove other columns to view this additional column what permissions are needed to edit criteria on a job? a user would need to have elevated access admin access or elevated access admin private access to the role, or be an organization admin to edit the job and the job criteria the candidate doesn't have a resume but does have education history and work experience listed in their profile in ashby would this be considered and evaluated by the ai assisted application review tool too? it will be considered, yes ai credits i’m a plus plan customer how many included candidates do i get as part of my plan? plus plan customers get 2,500 annual ai candidate evaluations included as part of their plan per seat to give an example, let’s say you have purchased 50 seats in your plan in that case, you would have 50 seats x 2,500 included candidate evaluations per seat = 125,000 candidate evaluations included annually if you needed more, you could purchase additional credits in blocks of 5,000 what if i purchase more seats part of the way into my contract on a seat based plan? purchasing more seats adds the corresponding number of credits on a prorated basis so, if you purchase one seat 6 months into a 1 year plus plan contract, you’ll get 2,500 x 0 5 = 1250 additional ai credits on that contract how many ai credits do i need? customers can better estimate how many ai credits they may need in a given billing cycle by looking at their historical data and target hiring numbers together using ashby's reporting capabilities, customers can quickly look at the number of applicants per role historically and multiply that by the number of planned hires over a given period to get a rough estimate of the number of applications they expect to receive this number should provide an estimate of the number of ai credits they will need over a given period to provide an example, if you have gotten 500 applicants per role historically and plan to add 20 new hires over the next year, you can roughly expect to have 10,000 applicants (and therefore can expect to use roughly 10,000 ai credits) over the next year will you warn me if i’m close to using up all of my credits? we will warn you in the application when you’ve used up 80% of your credits we will also let you know how many credits you have remaining before you create new criteria for a job if i update criteria for a job, will i be billed another credit? how do credits work with multiple criteria applied? you can run up to 50 criteria for any single candidate this means that you can update 1 criterion 50 times or 10 criteria 5 times without getting billed another credit we think this should be plenty for almost any reasonable use case we want to support fair use at a fair cost, and just want to make sure the flexibility doesn’t get abused if i don’t use all of my credits, do they roll over to the next period? credits are tied to a contract period unused credits don’t roll over to the next contract year credits are granted yearly for customers on annual or multi year agreements; credits are granted monthly for customers on monthly agreements how long do i have to use additional purchased credits? you have until the end of the current year of your subscription term additional credit “blocks” are prorated based on your contract term this way you have one contract with ashby with one renewal date, allowing you to budget easier what happens if i run out of credits? we won’t bill you any more than what you’ve already paid we will pause any new ai evaluations and the feature will turn back on automatically once you’ve added additional credits existing customers can contact their customer success manager; new customers can contact their account executive can you give me an example of how add on credits would work? let’s say you are on a plus plan with 10 seats you’d get 2,500 annual credits / seat x 10 seats = 25,000 annual credits to start with if you needed to evaluate 29,000 candidates in the course of the year, you’d add a block of 5,000 credits at $500 at any point into your contract that block of credits would be available to use any time throughout the remainder of your agreement term the following year, you’d have an additional 5,000 credits added to your contract and a $500 line item for those credits that you could adjust up or down, based on your needs for seat based plans, are credits tied to individuals? credits are allocated based on the number of seats for plus and enterprise plans and tied to an organization, not an individual so, if you have 50 seats on a plus plan, you can use 50 x 2,500 credits = 125,000 credits across any number of team members or jobs how do application form criteria work with credits? in the same way as your job criteria you can run up to 50 criteria for any single candidate for one credit this means that you can update 1 criterion 50 times or 10 criteria 5 times without getting billed another credit where can i view my remaining ai credits? you can view your remaining ai credits and purchase additional ai credits by heading to admin > organization setup > billing for more on this, check out how do i purchase additional email lookups, ai credits or texting credits in app? docid\ edqz9zv9vdcljyzsxjdhe will i receive a notification when a candidate opts out? there is no proactive alert sent to customers when a candidate opts out instead, the opt out status is surfaced in product when reviewing an application, you'll see that the job criteria evaluation was skipped, along with the reason for the skip you can also filter to view candidates who have opted out see ai opt out filters docid 0msgwzhfexbtwoagfyrar above for more info