Ashby to S3 Data Share

the amazon web services (aws) s3 data share provides a simple and flexible way for organizations on the enterprise plan to access their ashby data directly in amazon s3 this option supports teams who prefer to work with data in their own aws environment, where downstream systems can consume and query that data as needed permissions & access the aws s3 data share is available to organizations on the enterprise plan only foundations legacy plus plus enterprise ❌ ❌ ❌ ✅ overview what you can expect diagram showing how ashby delivers data to your aws s3 bucket once set up, ashby delivers a complete, structured export of your data into a dedicated s3 bucket in your aws account ashby organizes the data in a consistent format so your team can query it using any engine or platform of your choice (e g , aws athena, redshift, or google bigquery) the data refreshes daily, and each cycle delivers a complete copy of your dataset to your s3 bucket this approach removes the need for you to handle deduplication, merging, or change tracking setup what you need to provide to activate the s3 data share, your team creates an s3 bucket and an iam role in your aws account your customer success manager provides the ashby specific values you need, and ashby completes the remaining setup you do not need to grant access to any other aws services or manage aws credentials manually reach out to your customer success manager to start the setup process ashby provides the ashby iam user arn and your unique snowflake external id create or choose an s3 bucket in your aws account for the daily data exports the bucket name can be anything if possible, use the us east 1 (n virginia) region to avoid cross region data egress from ashby’s source environment you can configure s3 lifecycle rules, such as deleting older data after 30 days, according to your data governance needs server side encryption with aws key management service (sse kms) is optional and supported create an iam role for ashby in the aws console, navigate to iam > policies > create policy > json paste the permission policy json from the section below, replacing the bucket name placeholder with the name of your s3 bucket this policy allows access to objects in that bucket only { "version" "2012 10 17", "statement" \[ { "effect" "allow", "action" \["s3\ listbucket", "s3\ getbucketlocation"], "resource" "arn\ aws\ s3 \<your bucket name>" }, { "effect" "allow", "action" \["s3\ putobject", "s3\ deleteobject"], "resource" "arn\ aws\ s3 \<your bucket name>/ " } ] } save the policy and remember its name in the aws console, navigate to iam > roles > create role > custom trust policy paste the trust policy json from the section below this policy allows ashby’s iam user to assume your role using the unique snowflake external id { "version" "2012 10 17", "statement" \[ { "effect" "allow", "principal" { "aws" "arn\ aws\ iam 685689791302\ user/iif00000 s" }, "action" "sts\ assumerole", "condition" { "stringequals" { "sts\ externalid" "\<ashby external id>" } } } ] } select next , select the permission policy you created on the add permissions page, and select next enter a role name, and select create role send your customer success manager the following details your s3 bucket name the iam role arn for the role you created after ashby receives these details, ashby connects your role to your dedicated s3 integration and triggers the first sync to validate access wait for ashby to confirm that setup is complete your first data delivery arrives the following morning, pacific time, and continues daily thereafter what you see in s3 on each sync cycle, your bucket receives a set of parquet files representing the entirety of your ashby data, structured in the following format s3 //\<bucket name>/ashby shares/\<table name>/sync dtm=\<yyyymmddhhmmss>/ parquet ashby organizes these exports for simple downstream use, whether you're loading them into a warehouse or querying directly with an engine like athena with each successful sync, ashby updates a manifest json file in your bucket the file includes the following the sync dtm for the latest successful sync (i e , the most up to date folder for each table in the structure above) the list of tables and the row count for each synced table, with their column names and data types listed the list of included tables and their schema are also available in the ashby data share schema spreadsheet https //docs google com/spreadsheets/d/1jtezzsosdjag9rvxdvvumlriusjeollo4bwwe3 ffhi/edit?gid=1316200980#gid=1316200980 you can find an entity relationship diagram (erd) for the current set of shared tables in the ashby data share documentation https //dbdocs io/data share/ashby data share?view=relationships related data sharing options if your organization uses snowflake, you can also use our enterprise snowflake data share feature, which provides direct access to your ashby data from within your own snowflake instance see ashby to snowflake data share docid\ pg1llokjqbrln5goehn2c for more details faq how often does the s3 data refresh? the s3 data share refreshes data once per day is the data provided as a delta sync or snapshot? each day, ashby delivers a full snapshot of your data, with the snapshot date indicated in the folder structure what data is provided in the data share? the list of included tables and their schema are available in the ashby data share schema spreadsheet https //docs google com/spreadsheets/d/1jtezzsosdjag9rvxdvvumlriusjeollo4bwwe3 ffhi/edit?gid=1316200980#gid=1316200980 or our entity relationship diagram https //dbdocs io/data share/ashby data share (erd) what information does ashby provide before i configure aws? ashby provides an iam user arn and a snowflake external id use both values when you configure the iam role trust policy what information do i send to ashby after i configure aws? send your s3 bucket name and iam role arn to your customer success manager so ashby can complete the setup when do i receive my first data delivery? after ashby confirms that setup is complete, your first data delivery arrives the following morning, pacific time data continues to refresh daily