AI-Assisted Application Review

learn how to accelerate candidate reviews by using ai to evaluate applicants based on custom criteria this feature helps your team identify top talent faster by automating the initial resume screening process directly within ashby permissions & access ai assisted application review is available on all ashby plans foundations legacy plus plus enterprise ✅ ✅ ✅ ✅ permissions you must be an org admin to enable the feature to edit job level criteria, you need elevated access admin or elevated access admin private for that specific role step 1 enable ai assisted application review navigate to admin > organization setup > opt in features if enable ai features is not enabled, turn it on it now you will need to accept ashby's terms for ai features to continue then, enable ai assisted application review from the list of available ai feature tiles step 2 define your review criteria criteria that you add to the job defines what the ai assesses you can define resume criteria that the ai looks for within a candidate's resume as well as whether the candidate answered specific application form questions to define your criteria, navigate to jobs > jobs and select a job in the job settings, navigate to the ai features tab and select set up criteria resume criteria in the resume criteria section, you can add criteria in multiple ways add criteria suggested by ai based on the job posting, add previously used criteria from other jobs, or add custom criteria that you create for this role specifically to start adding criteria select + add criteria under the resume criteria section add suggested criteria or previously used criteria select add from # suggestions for suggestions generated by ai based on your job posting you can also reuse criteria from other roles you have permission to access by selecting add previously used to add a suggestion, select add in the action column creating effective custom criteria selecting add custom instead of the suggested or previously used options allows you to create your own criteria you'll need to define the following short title the name of your criterion this appears in job settings, the application review page, and as a filter name prompt the instructions used to evaluate applicants precise and clear prompts result in better ai performance tips on criteria effectiveness write your criteria as if you were instructing a junior recruiter ensure the prompt is phrased so that any reader would reach the same conclusion stand alone criteria should be understandable without extra context instead of "can assimilate feedback," use "effectively incorporates feedback related to personal job performance " singular focus on one skill per requirement instead of "experience in python and aws," create two separate criteria for each clear avoid vague terms like "technical background " specify the exact expertise required specific use measurable requirements, such as years of experience or specific previous job titles objectively verifiable ensure the requirement can be proven by looking at a resume instead of "has good taste," use "has led design for consumer saas products " application form criteria you can evaluate whether a candidate answered long form application questions in the application form criteria section toggle the did the application answer the question? option for individual questions these are added as criteria for the review process this check only determines if the question was answered blank or unrelated answers are flagged as not meeting the criteria this is only available for long, unformatted text fields step 3 validating and reviewing criteria once you have set up your criteria, it must be validated ashby checks your prompts on two dimensions clarity and objectivity to ensure that the criteria is clear and objective which means it can be appropriately evaluated by the ai equal employment opportunity (eeo) violations the ai also scans for implicit or explicit eeo violations and flags them validating criteria to validate your criteria, select validate if all criteria pass validation, you'll see a pop up indicating that hit close and proceed to the next section on saving criteria and evaluating candidates if the system identifies issues, select review criteria on the pop up to view warnings reviewing flagged criteria the flagged criteria will be identified with a warning select show details under any warning to review why the criterion was flagged we recommend to either update the criterion until it is no longer flagged, or remove the criterion all together before proceeding you can proceed with flagged criteria but it is not recommended step 4 saving criteria and evaluating candidates after validation is successful, select save and evaluate all active candidates to begin the initial evaluation depending on the number of candidates, this can take a few seconds to a few minutes you'll see a loading screen with the total number of candidates being evaluated and a progress status bar if you are updating existing criteria, select save and re evaluate all active job considerations once you've made your updates review candidate evaluations ashby's ai assesses candidate resumes against each specific criterion and returns an evaluation status the status may indicate the candidate meets, does not meet, or the ai was undecided on the criterion evaluation results can be seen on the candidate profile, as well as in the application review stage of the candidate pipeline for the role viewing results on the candidate profile view full evaluations in the feedback tab of a candidate profile the feedback tab on a candidate profile displaying the ai evaluation results summary for various resume criteria click on any criterion to see the reasoning for a status of meets, does not meet, or undecided select the expand all icon to see reasoning for all criteria at once evaluation results in application review navigate to the specific job, select the candidate pipeline tab on the left, and click on the application review stage here, there is a column for evaluation results that provides a summary about how the candidate scored hover over the results row to see more details you can also add an ai job criteria met percentage column to see the percentage of ai criteria each candidate met select add column on the application review page and choose ai job criteria met percentage sort by this column to move the best fit candidates (highest percentage) to the top of your review queue and identify the lowest fit candidates (lowest percentage) at a glance filtering by ai criteria you can filter candidates by their evaluation status within your pipeline stages select the ai criteria quick filter option select the criteria and statuses you want to view, such as any candidate who passed or the ai was undecided on a particular criteria perform bulk actions on the filtered list as needed candidate experience & opt out automated processing legal notice you can configure a message to display to candidates who may be impacted by automated ai review processes by following the steps below navigate to admin > data management > data privacy and compliance > privacy & legal to configure a legal notice for candidates locate the automated processing legal notice section and click on the legal notice for all jobs that is displayed configure the settings for when this message will display is enabled? when off, candidates do not see the notice rule scope set criteria here when you want the message to show only based on certain criteria, such as for certain jobs only do not add a scope if you want it to apply to all jobs hide scores if enabled, ai informed scores are hidden from users if the candidate hasn't seen the notice if hide scores is enabled, a message will display on the candidate's evaluation results that says "the candidate has not seen the legal notice required by this job " this can happen when a candidate is added to ashby through a path other than the ashby hosted job application, such as a direct referral, a manual add, the chrome extension, or possibly indeed apply legal notice text the specific text displayed to candidates this can be edited hit save once you've adjusted the settings as needed candidates see a learn more link under the submit application button clicking this allows them to view the notice, opt out of automated processing, or check whether they have already opted out you can view what this page looks like by selecting view in the top right corner when setting up the message ai opt out filters in your candidate pipeline, you can filter to see all candidates who have opted out of ai processing by following the below steps navigate to the candidate pipeline enable the advanced filters toggle on the right of the screen select + add field to match on the left search for "ai opt out" in the search bar and select the criterion select equals = yes ai credits ai assisted application review is priced per credit one credit equals one evaluation of a single candidate (job consideration), supporting up to 50 criteria per evaluation foundations ai credits legacy plus ai credits plus ai credits enterprise ai credits 1,500/mo or 18,000 per year account wide 1,500/mo or 18,000 per year account wide 2,500/yr, per paid seat 12,500/yr, per paid seat additional credits can be purchased in blocks for more information, see our guide on ai credits docid 7q8kj16ibgcp9zsjx86xm faq if i edit my criteria, can i re evaluate candidates? yes if you edit a prompt or change criteria, select the save and re evaluate all active job considerations option to refresh evaluations can i flag ai evaluations if they are incorrect? yes select the three dots in the top right corner of an evaluation to flag it as incorrect or misleading can i add evaluations as a column in application review? yes select add column on the job's application review page and search for the evaluations column for a summary of how each candidate scored against your ai criteria to see a numeric percentage of criteria met, search for ai job criteria met percentage — sort by this column to rank candidates from best fit (highest percentage) to least fit (lowest) what permissions are needed to edit job criteria? you must either have elevated access admin or elevated access admin private for the specific role, or be an organization admin does the ai evaluate education and work history if there is no resume? yes, if the information is listed on the candidate's ashby profile, the ai considers it will i receive a notification when a candidate opts out? there is no proactive alert sent to customers when a candidate opts out, but you can configure alerts to be sent using ashby's reporting capabilities and the ai opt out field see alerts docid\ fckr kkhnmjz6vuu3krvw for more information additionally, the opt out status is surfaced in product when reviewing an application, you'll see that the job criteria evaluation was skipped, along with the reason for the skip you can also filter to view candidates who have opted out see ai opt out filters # above for more info does ai assisted application review affect the average score? no the average score (also referred to as the application review average score) reflects the ratings given by human reviewers during the application review process and is calculated independently from ai criteria evaluations if average score appears empty, no reviewer has manually scored that application yet to measure ai based candidate fit, use the ai job criteria met percentage column, which shows the proportion of ai criteria met for each candidate