> ## Documentation Index > Fetch the complete documentation index at: https://lightdash-feature-prod-8453-51bb.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # dbt Projects > You can easily make changes in dbt and see them updated in your Lightdash project. Lightdash supports dbt v1.4.0 and above. If you are using an older version of dbt, you will need to upgrade to sync your project to Lightdash. ## Syncing your dbt project to Lightdash There are five ways to sync changes from dbt into Lightdash. They differ in **where the code comes from**, **where compilation happens**, **which `profiles.yml` / target governs the result**, and **whether they respect the project's UI Connection Settings**. For ready-to-use workflow templates, see the [`lightdash/cli-actions`](https://github.com/lightdash/cli-actions) repo. ### Sync mechanisms Compiles the dbt project locally and pushes the resulting `manifest.json` to Lightdash. | Property | Value | | --------------------------------- | ------------------------------------------------------------------------- | | dbt code source | Local working copy (uncommitted changes included) | | Compile location | Laptop | | `profiles.yml` source | Local (`~/.dbt/profiles.yml` by default, or `--profiles-dir`) | | Target | Default in that `profiles.yml`, overridden by `--target` | | UI Connection Settings respected? | No | | Auth | `lightdash login` session or `LIGHTDASH_API_KEY` env var | | Prerequisites | Lightdash CLI installed; dbt installed; warehouse creds reachable locally | | Remote git connection required? | No | **Use cases:** initial project setup, debugging, one-off manual deploys. **Trade-offs:** bypasses PR review; relies on per-developer `profiles.yml`; high footgun risk if a developer's prod target accidentally points at a non-prod warehouse. Same as #1, but runs inside a CI workflow (GitHub Actions, GitLab CI, Azure Pipelines, Bitbucket Pipelines, etc.) with credentials stored as CI secrets. | Property | Value | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | dbt code source | CI runner's checkout of the repo | | Compile location | CI runner | | `profiles.yml` source | Provided by the workflow — either committed as an `env_var()` template, or written to disk from a CI secret at runtime | | Target | Default in the CI `profiles.yml`, overridden by `--target` | | UI Connection Settings respected? | No | | Auth | `LIGHTDASH_API_KEY` from CI secrets | | Prerequisites | Lightdash CLI + dbt installed in the workflow; warehouse creds available to CI; valid `profiles.yml` produced at runtime | | Remote git connection required? | No | | Workflow template | [`deploy.yml`](https://github.com/lightdash/cli-actions) in `lightdash/cli-actions` (GitHub Actions example) | **Use cases:** automated production deploys gated by PR merge; PR preview workflows (`lightdash start-preview` + `lightdash validate`). **Trade-offs:** requires duplicating warehouse credentials into CI on top of what's already in Lightdash; needs dbt available in the runner. Never commit a `profiles.yml` with plaintext credentials. Either use an `env_var()` template committed to the repo, or write the file to disk at runtime from a CI secret. Lightdash fetches the dbt code from the configured remote, compiles it server-side, and updates the project. | Property | Value | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | dbt code source | Lightdash pulls from the remote git source configured in Project Settings (GitHub, GitLab, Azure DevOps, Bitbucket, dbt Cloud) | | Compile location | Lightdash servers | | `profiles.yml` source | Lightdash synthesises one from Project Settings → Connection Settings | | Target | UI Project Settings → Connection Settings → Target name | | UI Connection Settings respected? | Yes | | Auth | Logged-in user in the UI | | Prerequisites | Project must have a remote git source connected | | Remote git connection required? | Yes | **Use cases:** ad-hoc refresh after a merge; quick UI-driven sync without leaving the browser. **Trade-offs:** manual; relies on a human remembering to click it. Calls the same endpoint as the "Refresh dbt" button, but programmatically from CI (GitHub Actions, GitLab CI, Azure Pipelines, Bitbucket Pipelines, etc.). Lightdash does the work server-side using the project's UI Connection Settings. | Property | Value | | --------------------------------- | ------------------------------------------------------------------------------------------------------------- | | dbt code source | Lightdash pulls from the remote git source (configured in Project Settings) | | Compile location | Lightdash servers | | `profiles.yml` source | Synthesised by Lightdash from UI Connection Settings | | Target | UI Project Settings → Connection Settings → Target name | | UI Connection Settings respected? | Yes | | Auth | `LIGHTDASH_API_KEY` from CI secrets | | Prerequisites | Project must have a remote git source connected; Lightdash CLI installed in the workflow | | Remote git connection required? | Yes | | Workflow template | [`refresh.yml`](https://github.com/lightdash/cli-actions) in `lightdash/cli-actions` (GitHub Actions example) | **Use cases:** automated post-merge sync to production when the project is already git-connected. **Trade-offs:** can't push uncommitted changes (only what's in the connected branch); won't catch issues pre-merge — pair with a separate `lightdash validate` PR workflow if you want that. Same as #4, but bypasses the Lightdash CLI entirely — just a `curl` to the API. Works from any CI system (GitHub Actions, GitLab CI, Azure Pipelines, Bitbucket Pipelines, etc.). | Property | Value | | --------------------------------- | --------------------------------------------------------------------------- | | dbt code source | Lightdash pulls from the remote git source (configured in Project Settings) | | Compile location | Lightdash servers | | `profiles.yml` source | Synthesised by Lightdash from UI Connection Settings | | Target | UI Project Settings → Connection Settings → Target name | | UI Connection Settings respected? | Yes | | Auth | `LIGHTDASH_API_KEY` from CI secrets | | Prerequisites | Project must have a remote git source connected | | Remote git connection required? | Yes | | Lightdash CLI required? | No | | Endpoint | `POST /api/v1/projects/{projectUuid}/refresh` | **Use cases:** the simplest possible automated refresh — no dependencies to install. Useful for teams who don't want a Node toolchain in CI. **Trade-offs:** same as #4. The refresh is async (returns a `jobUuid`); for stronger guarantees, poll `GET /api/v1/jobs/{jobUuid}` to confirm completion before exiting the workflow. ### Decision matrix | | #1 `lightdash deploy` — local CLI | #2 `lightdash deploy` — CI | #3 "Refresh dbt" — UI button | #4 `lightdash refresh` — CI | #5 `POST /refresh` API — CI | | -------------------------------------- | --------------------------------- | ----------------------------------------------------- | ---------------------------- | ----------------------------- | --------------------------------- | | Automated? | No | Yes | No | Yes | Yes | | Pushes uncommitted local changes? | Yes | No | No | No | No | | Needs dbt + warehouse creds in CI? | N/A | Yes | No | No | No | | Needs Lightdash CLI installed? | Yes | Yes | No | Yes | No | | Respects UI Target name? | No | No | Yes | Yes | Yes | | Works without a remote git connection? | Yes | Yes | No | No | No | | Catches breaking changes pre-merge? | With `--select` + `validate` | If used in a PR workflow | No | No | No | | Best for | Initial setup, debugging | PR previews; CI deploys with no remote git connection | Ad-hoc UI refresh | Post-merge sync (recommended) | Post-merge sync, minimal-CI shops | ### Recommended setups **Most teams** (Lightdash connected to GitHub, GitLab, Azure DevOps, Bitbucket, or dbt Cloud): * **PR workflow** → `lightdash start-preview` + `lightdash validate` (mechanism #2 with `start-preview`) * **Post-merge to `main`** → `lightdash refresh` (#4) or the refresh API (#5) * **Manual UI refresh** (#3) available for ad-hoc syncs **Teams without a remote git connection** (project loaded via local CLI only): * **Production sync** → `lightdash deploy` in CI (#2) * **Previews** → `lightdash start-preview` in CI **Teams just getting started:** * Local `lightdash deploy` (#1) for first-time setup, then graduate to one of the automated patterns above as soon as the project is stable. Not sure whether to use `lightdash deploy` or `lightdash refresh`? Read about the [differences between the two commands](/references/lightdash-cli#lightdash-deploy-vs-lightdash-refresh) in the CLI reference. ### Things to know **`lightdash deploy` does NOT use the UI "Target name" setting.** It uses the local `profiles.yml` default, or whatever `--target` overrides it with. Only mechanisms #3, #4, and #5 respect the UI Target name. **`lightdash refresh` requires a remote git connection.** It will fail with a `ParameterError` if the project's `dbtConnection.type` is `NONE`. **Refresh is async.** Both #4 and #5 return a `jobUuid` and the work continues server-side. If your CI needs to know the refresh succeeded, poll `GET /api/v1/jobs/{jobUuid}` before exiting the workflow. **Never commit `profiles.yml` with credentials.** For mechanism #2, use either a secret-written-to-file approach or an `env_var()` template — but never check in plaintext credentials. Mechanisms #4 and #5 are underused. Most existing CI examples lead with `lightdash deploy`, which is the heaviest option in CI (requires dbt + warehouse credentials duplicated into the runner). The refresh-based patterns are simpler and often a better fit when the project is git-connected. ### How to trigger each sync #### Click "Refresh dbt" in Lightdash The button can be found on the Query from tables page. screenshot-refresh-dbt *If you're using a git connection (like GitHub, GitLab, or Bitbucket), you'll need to push + merge your changes to the branch that your Lightdash project is connected to before you press `Refresh dbt`.* If you've made any changes to the underlying data (for example, adding a new column in your `model.sql` file or changing the SQL logic of a dimension), then you need to run `dbt run -m yourmodel` before you click `Refresh dbt` in Lightdash. #### Push code from the CLI If you're using the [Lightdash CLI](/guides/cli/how-to-install-the-lightdash-cli), you can use `lightdash deploy` to deploy your changes to Lightdash. [Read more about how to use `lightdash deploy`](/guides/cli/how-to-use-lightdash-deploy). We don't recommend using `lightdash deploy` from your local environment as the primary way you update Lightdash since small mistakes can lead to production issues. #### Set up continuous deployment [Read how to do that and check out our example workflow files](/guides/cli/how-to-use-lightdash-deploy#automatically-deploy-your-changes-to-lightdash-using-a-github-action). ## dbt project settings For more information about dbt connection types (Github, Gitlab, Bitbucket, etc.) and the fields required for each type, [read the dbt project section in our connection guide](/get-started/setup-lightdash/connect-project#2-import-a-dbt-project). Below are details about the universal fields for all connected dbt projects. ### dbt selector This setting is found under Advanced configuration options You can filter out models in your dbt project that you don't want to see in Lightdash. This is useful if you have a large dbt project and you want to speed up the sync process. Unlike [table selection](/get-started/develop-in-lightdash/adding-tables-to-lightdash#limiting-the-tables-in-lightdash-using-dbt-tags), this selector is applied to the dbt models, so it will skip the entire compilation process for the models that you don't want to see in Lightdash. To do this, you can add a **dbt selector** to your project settings. This is a JSON object that contains the models you want to include in Lightdash. For example, if you only want to include the `my_model` and all models with the `lightdash` tag in Lightdash, you can add the following to your dbt project settings: ```console theme={null} my_model tag:lightdash ``` We support all dbt selectors. [Read more about selectors in the dbt docs](https://docs.getdbt.com/reference/node-selection/syntax#combining-state-and-result-selectors). ## dbt targets ### What is a dbt target? A **dbt target** is an environment configuration in your dbt project that defines where and how dbt should run. In dbt Core, targets are defined in your `profiles.yml` file and typically include settings like: * Database connection details * Schema/dataset names * Warehouse specifications * Other environment-specific configurations Most teams have multiple targets for different environments: * `dev` - for development work * `prod` - for production deployments * `staging` - for testing before production Each target can have different connection settings which are stored in your profiles.yml file. The example `profiles.yml` file below shows different connection settings for the dev and prod targets. Note that on line 3 `target: dev` indicates that the dev target is the default target for this project. ```yaml theme={null} # example profiles.yml file jaffle_shop: target: dev outputs: dev: type: postgres host: localhost user: alice password: port: 5432 dbname: jaffle_shop schema: dbt_alice threads: 4 prod: # additional prod target type: postgres host: prod.db.example.com user: alice password: port: 5432 dbname: jaffle_shop schema: analytics threads: 8 ``` ### Configuring your target in Lightdash **Target** contains information about your dbt connection to your warehouse. By default, we set this to be the same value as you have as the default in your `profiles.yml` file when you run `lightdash deploy` (if that's how you created or recently deployed your project). If you want to update this, you can enter the target of your choice in the project settings (for example `prod` or `analytics`). [Read more about dbt targets in the dbt docs.](https://docs.getdbt.com/reference/dbt-jinja-functions/target) ### How targets work with dbt macros When Lightdash connects to your dbt project, it compiles a `profiles.yml` file based on the connection settings you configure in the Lightdash UI or the connection settings from your profiles.yml if you deployed your project from the CLI using `lightdash deploy`. This means that any dbt macros in your project that use target settings will be executed by Lightdash and resolved according to those settings. #### Why use targets in macros? You might want dbt to behave differently depending on the target/environment you're running in. Common use cases include: * Using different schema naming conventions in dev vs. production * Configuring different warehouse sizes (small for dev, large for prod) * Adjusting query performance settings per environment * Loading different amounts of data (sample data in dev, full data in prod) * Applying different data retention policies Let's look at a common example: customizing schema names based on your target. #### Example: Schema naming based on target By default, dbt appends your custom schema names to your target schema. For example, if your target schema is `analytics` and you configure a model with `schema='staging'`, dbt creates it in `analytics_staging`. This default behavior is useful in development because it prevents developers from overwriting each other's work - each developer can have their own target schema like `dbt_jane` or `dbt_john`, resulting in separated schemas like `dbt_jane_staging` and `dbt_john_staging`. However, in production, you typically want cleaner schema names. Instead of `analytics_staging`, you just want `staging`. This is where customizing the `generate_schema_name` macro based on the target becomes helpful. **Default behavior:** ```sql theme={null} {% macro generate_schema_name(custom_schema_name, node) -%} {%- set default_schema = target.schema -%} {%- if custom_schema_name is none -%} {{ default_schema }} {%- else -%} {{ default_schema }}_{{ custom_schema_name | trim }} {%- endif -%} {%- endmacro %} ``` **Modified behavior for production:** ```sql theme={null} {% macro generate_schema_name(custom_schema_name, node) -%} {%- set default_schema = target.schema -%} {%- if custom_schema_name is none -%} {{ default_schema }} {%- elif target.name == 'prod' -%} {{ custom_schema_name | trim }} {%- else -%} {{ default_schema }}_{{ custom_schema_name | trim }} {%- endif -%} {%- endmacro %} ``` This macro says: "If we're in production (`target.name == 'prod'`), use just the custom schema name. Otherwise, use the default behavior of prefixing it." **How this works in Lightdash:** If your Lightdash connection settings specify: * **Target name:** `prod` * **Dataset/Schema:** `analytics` And you have a model configured with: ```sql theme={null} {{ config(schema='staging') }} ``` Then Lightdash will create/query this model in the `staging` schema - not `analytics_staging` - because `target.name == 'prod'`. This gives you clean, professional schema names in production (`staging`, `marts`, `reporting`) while maintaining developer isolation in development environments (`dbt_jane_staging`, `dbt_john_staging`). The `generate_schema_name` macro is a default dbt macro and its behavior can be overridden by adding a macro to dbt with the same name. If this macro is not showing up in your dbt project, it means that your project is using the default macro. It is a good idea to check your project for custom macros which can alter your project's configurations. **Example: Warehouse configuration** Similarly, if you have a macro that dynamically configures settings: ```sql theme={null} {% macro get_warehouse() %} {% if target.name == 'prod' %} {{ return('LARGE_WAREHOUSE') }} {% elif target.name == 'dev' %} {{ return('SMALL_WAREHOUSE') }} {% endif %} {% endmacro %} ``` And use it in your models: ```sql theme={null} {{ config(snowflake_warehouse=get_warehouse()) }} ``` Lightdash will use the configuration that corresponds to the target name you've configured in your Lightdash connection settings. ### Differences between dbt Core and dbt Cloud In dbt Core, teams commonly use targets to manage different environments. In dbt Cloud, targets are replaced by environments, and custom logic such as `if target.name = dev then...` is commonly swapped out using env vars to become 'if env\_var = development then...' or similar. ## dbt environment variables ### What are environment variables? Environment variables allow you to set dynamic values that your dbt macros can reference. This is particularly useful if your dbt project uses environment variables to control behavior (common in dbt Cloud setups). ### Configuring environment variables in Lightdash This setting is found under Advanced configuration options If your dbt project uses [environment variables in your dbt `profiles.yml` file](https://docs.getdbt.com/reference/dbt-jinja-functions/env%5Fvar), you can add these to Lightdash in your project settings. For each environment variable, you'll need to add the `key` + `value` pair for the item. You'll normally find these values in a file called `.env` in your dbt project directory. For example, you might have something like: ```yaml theme={null} profile: target: prod outputs: prod: type: postgres host: 127.0.0.1 user: "{{ env_var('DBT_USER') }}" .... ``` Then a `.env` file like: ```yaml theme={null} export DBT_USER="myspecialuser" ``` So, in Lightdash, you'd add a new environment variable and put `key` as `DBT_USER` and `value` as `myspecialuser`. ### Reserved and blocked variable names When you create or update a project, Lightdash validates each environment variable key and rejects values that could change how dbt or its child processes execute. Names must contain only letters, numbers, and underscores, and cannot start with a number. The following keys are blocked: * `GIT_ASKPASS`, `GIT_SSH`, `GIT_SSH_COMMAND`, and any name starting with `GIT_CONFIG_` * `LD_AUDIT`, `LD_LIBRARY_PATH`, `LD_PRELOAD`, and any name starting with `DYLD_` * `NODE_OPTIONS`, `NODE_PATH` * `PATH`, `SHELL` * `PERL5OPT`, `PYTHONHOME`, `PYTHONPATH`, `RUBYOPT` * `SSH_ASKPASS` * Any name starting with `LIGHTDASH_DBT_PROFILE_VAR_` (reserved for internal use) If you have an existing project that already uses one of these keys, project compilation will continue to work — the check only runs when you save changes to your connection settings. Rename the variable in your dbt project and save the updated settings to clear the validation error. ### How environment variables work with dbt macros When Lightdash compiles your dbt project, any environment variables you configure will be available to your dbt macros through the `env_var()` function. For example, if you have a macro that checks an environment variable: ```sql theme={null} {% macro get_schema() %} {% if env_var('DBT_ENV') == 'production' %} {{ return('prod_analytics') }} {% else %} {{ return('dev_analytics') }} {% endif %} {% endmacro %} ``` You would set `DBT_ENV=production` as an environment variable in your Lightdash connection settings to ensure the macro resolves to `prod_analytics`. **Key points about environment variables:** * They're available to all macros in your dbt project via the `env_var()` function * They're particularly important if your dbt project was built for dbt Cloud, which relies heavily on environment variables * Make sure to set any environment variables your macros depend on in Lightdash to match your production dbt environment ## Key takeaways * Lightdash compiles your dbt project using the target name and other settings you configure in the connection settings * Any macros that use `target.name`, environment variables, or other profile settings will resolve according to your Lightdash configuration * Make sure your Lightdash target name matches the target you use for your production dbt runs to ensure Lightdash behaves consistently with your dbt project * If you have custom macros that depend on target settings or environment variables, ensure those settings are configured correctly in your Lightdash connection settings If you're experiencing unexpected behavior with how Lightdash compiles your dbt project, check that your target name in Lightdash matches what your dbt macros expect, and verify any environment variables your macros depend on are properly set.