Connecting BigQuery

Link a Google Cloud project to Neuralift with a dedicated service account, and receive segment-labeled results back as BigQuery tables backed by your own GCS bucket.

Neuralift connects to BigQuery with a Google Cloud service account you provide, scoped to a single project. This page walks through creating that service account, adding the connection, and — when runs are published — provisioning the bucket, dataset, and BigLake connection that bring results back as queryable BigQuery tables (Receiving results in BigQuery).

How this connection works

A service account is Google Cloud’s identity for applications rather than people: you create one in your project, grant it read access to only the BigQuery datasets Neuralift needs, and hand Neuralift its JSON key. IAM on your side stays the source of truth. You can narrow the account’s roles or disable it at any time to sever the connection, and reads are attributable to that account in your audit logs when BigQuery Data Access audit logs are enabled in your project.

Before you begin

You need the admin role in your Neuralift organization; see Roles & permissions.
In Google Cloud you need permission to create service accounts and grant IAM roles in the project that holds your data.
Neuralift does not accept any data with PII. Check Data requirements before connecting tables.

Create a service account

Create a dedicated service account for Neuralift, grant it read access, and export a JSON key:

PROJECT=<my-gcp-project>

# 1. Dedicated service account
gcloud iam service-accounts create neuralift-reader \
  --project=$PROJECT --display-name="Neuralift federation reader"

# 2. Roles: read the data, and use the BigQuery Storage Read API
gcloud projects add-iam-policy-binding $PROJECT \
  --member="serviceAccount:neuralift-reader@$PROJECT.iam.gserviceaccount.com" \
  --role=roles/bigquery.dataViewer
gcloud projects add-iam-policy-binding $PROJECT \
  --member="serviceAccount:neuralift-reader@$PROJECT.iam.gserviceaccount.com" \
  --role=roles/bigquery.readSessionUser

# 3. Export a JSON key to paste into Neuralift
gcloud iam service-accounts keys create neuralift-reader-key.json \
  --iam-account=neuralift-reader@$PROJECT.iam.gserviceaccount.com

A few notes:

Prefer per-dataset scoping. Instead of project-wide roles/bigquery.dataViewer, grant the service account BigQuery Data Viewer on just the datasets Neuralift needs, via each dataset’s sharing/IAM settings. Keep roles/bigquery.readSessionUser at the project level.
Organization policy on key creation. If your organization enforces iam.disableServiceAccountKeyCreation, step 3 is blocked; this integration needs a policy exception for the service account or project before it can proceed.
VPC Service Controls. If your project sits inside a VPC-SC perimeter, the perimeter must allow Neuralift’s access to the BigQuery APIs; work with your Neuralift contact on the exception.
Key handling and rotation. The key is used to provision the connection into Neuralift’s governed compute layer, where it’s encrypted; it is never stored in Neuralift’s own database. To rotate, create a new key and re-save the connection — it rotates in place.

Add the connection

In Neuralift, go to Settings → Data Connections and click Add connection.
Under Warehouse type, select BigQuery.
Enter a Connection name (lowercase letters, digits, and dashes; unique within your organization).
Enter your GCP project ID, for example <my-gcp-project>.
Paste the full contents of the key file into Service account key (JSON): the complete JSON document, starting with {"type": "service_account", ....
Click Save & connect.

The dialog closes and a Setting up … connection… notice appears. After a moment the new connection card appears with an Active badge (or Error with an explanation) and gains Browse tables and Test buttons.

Receiving results in BigQuery

Results are delivered into a GCS bucket you own and registered as queryable BigQuery tables through a BigLake connection, so each published run is a table your analysts query like any other. You create the bucket, dataset, and connection in your project; Neuralift writes the files and creates the tables. The delivery workflow around it is covered in Delivering results to your warehouse.

Two identities are involved — don’t conflate them:

Neuralift’s delivery service account (one per organization, created when you save the delivery configuration; its email is shown on the delivery card) writes the Parquet files, runs BigQuery jobs, and creates the result tables.
The BigLake connection’s Google-managed service agent (created by Google when you make the connection) is what BigQuery uses to read the delivered files at query time.

Each needs its own grants, made below.

1. Fill in the delivery form

Under Settings → Data Connections → Share your results, click Configure delivery and choose BigQuery under Deliver results to. The form collects everything up front:

Storage destination — the GCS URI result files land under, gs://<bucket>/<prefix>.
BigQuery project ID — the project holding your bucket, dataset, and connection.
BigQuery dataset — the dataset result tables are created in, for example neuralift_deliveries.
Region — where all three resources live. Bucket, dataset, and BigLake connection must share this location, and dataset locations are immutable, so pick deliberately. The select currently offers US locations only (the US multi-region and common US regions); if you need another region, contact support@neuralift.ai.
BigLake connection name — the CLOUD_RESOURCE connection BigQuery reads delivered files through, for example neuralift-biglake. Created in step 2 below if it doesn’t exist yet.

2. Create the resources in your project — before saving

The form’s collapsible GCP setup commands block shows the full setup script with your values filled in, updating live as you type. Run steps 1–3 of it before saving (skip anything that already exists):

# 1. Enable the APIs (once per project)
gcloud services enable storage.googleapis.com bigquery.googleapis.com \
  bigqueryconnection.googleapis.com --project=<project>

# 2. Bucket, dataset, and BigLake connection — all in the region you selected
gcloud storage buckets create gs://<bucket> --project=<project> \
  --location=<region> --uniform-bucket-level-access
bq mk --dataset --location=<region> <project>:<dataset>
bq mk --connection --connection_type=CLOUD_RESOURCE --location=<region> \
  --project_id=<project> <connection-name>

# 3. Let BigQuery read delivered files (the connection's service agent)
CONNECTION_SA=$(bq show --connection --format=json <project>.<region>.<connection-name> \
  | jq -r '.cloudResource.serviceAccountId')
gcloud storage buckets add-iam-policy-binding gs://<bucket> \
  --member="serviceAccount:$CONNECTION_SA" --role="roles/storage.objectViewer"

Datasets and connections are managed with the Google Cloud SDK’s bq command; there is no pure-gcloud surface for them.

3. Save, then grant Neuralift’s delivery service account access

Click Configure delivery. Saving validates the storage destination and creates your organization’s dedicated delivery service account; the card then shows Action required — finish GCP setup, with the account’s email filled into step 4 of the setup block:

NEURALIFT_SA=<delivery-service-account-email>   # shown on the delivery card after saving

# Write and manage delivered files
gcloud storage buckets add-iam-policy-binding gs://<bucket> \
  --member="serviceAccount:$NEURALIFT_SA" --role="roles/storage.objectAdmin"

# Run BigQuery jobs in your project
gcloud projects add-iam-policy-binding <project> \
  --member="serviceAccount:$NEURALIFT_SA" --role="roles/bigquery.jobUser"

# Create and update tables in the results dataset: add the account as a
# WRITER via the dataset's policy JSON (dataset-level
# `bq add-iam-policy-binding` is allowlist-gated; the console works too:
# dataset > Sharing > Permissions > BigQuery Data Editor)
bq show --format=prettyjson <project>:<dataset> \
  | jq '.access += [{"role":"WRITER","userByEmail":"'"$NEURALIFT_SA"'"}]' > /tmp/neuralift-dataset.json
bq update --source /tmp/neuralift-dataset.json <project>:<dataset>

One grant is console-only: in the BigQuery console, open the BigLake connection’s sharing/permissions and grant the delivery service account the BigQuery Connection Admin role (it needs bigquery.connections.delegate) — there is no CLI for connection IAM.

Then click Verify & activate. Neuralift verifies every grant and the card flips to Active, labelled BigQuery (GCS push), showing your destination, delivery service account, dataset, and connection. Each published run your Neuralift team delivers appears as a table named sld_<run_id> in the dataset — a BigLake external table backed by Parquet files under your bucket prefix, queryable like any other table.

Gotchas worth knowing up front

Domain-restricted sharing. If your organization enforces iam.allowedPolicyMemberDomains, granting roles to Neuralift’s service account is blocked until you add a policy exception for its domain — this is the most common enterprise blocker.
Region colocation is mandatory (see above). If delivery fails with a DDL error mentioning locations, check that the bucket, dataset, and connection regions match.
If setup lands on Error, the card keeps the full setup script (with your delivery service account’s email preserved) in its collapsible block — complete the missing steps, then click Reset & re-provision to re-verify.

Troubleshooting

The connection lands on Error. The card shows an explanation. Common causes: the pasted key isn’t the complete JSON document, the key has been revoked, or the GCP project ID doesn’t match the project the service account belongs to.
The connection is Active but expected tables are missing. The service account can only expose datasets it can read. Grant it access in Google Cloud, then use Sync tables in the Source tables sheet to re-discover them.
Key creation fails in your project. Your organization policy blocks service-account key export (iam.disableServiceAccountKeyCreation); request an exception from your GCP administrators.
BigQuery isn’t offered as a warehouse type. It isn’t enabled for your organization; contact support@neuralift.ai.

Note. The service account key is used to provision the connection and is handled as described in Data handling & security.

Next steps

Sync source tables from your new connection.
Review data requirements so your tables are ready for segmentation.
When runs are published, see Delivering results to your warehouse for the delivery workflow.