Skip to content

Commit 918773a

Browse files
timdenoudentimdenouden
authored
Merge pull request #14 from dbt-msft/setting_up_VScode
add vscode setup docs
2 parents 434da7c + c6dd6d5 commit 918773a

File tree

2 files changed

+248
-0
lines changed

2 files changed

+248
-0
lines changed

‎docs/guides/_category_.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"label": "Getting Started Guides",
3+
"position": 3
4+
}

‎docs/guides/vscode_setup.md‎

Lines changed: 244 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,244 @@
1+
---
2+
title: Using VSCode with dbt
3+
sidebar_position: 2
4+
---
5+
6+
# Using dbt in VSCode
7+
8+
## Intro
9+
10+
When our team first started using the dbt CLI, we started with Claire's well-loved discourse post, [How we set up our computers for working on dbt project](https://discourse.getdbt.com/t/how-we-set-up-our-computers-for-working-on-dbt-projects/243). The post details how the dbt team uses Atom and iTerm 2 on macOS for an improved workflow. Many folks commented on how they acheived similar productivity using VSCode. I thought I'd consolidate some of this into a single article, and expand on it given the recent developments. I'm also going to add things to make it easier for working with Azure databases such as the Azure CLI and Azure Data Studio.
11+
12+
### Goals
13+
14+
Following this guide will enable the following behavior in VSCode (some points lifted from Claire's guide -- linked above)
15+
16+
- a stable, reproducible Python environment for running dbt
17+
- syntax highlighting for SQL files with jinja in them
18+
- graying out the text of files that have compiled code in them, to help prevent you from editing compiled SQL (as opposed to your actual model)
19+
- quick switching b/w a model file and it's `compiled` and `run` counterparts
20+
- allow
21+
22+
23+
### Prerequisite
24+
25+
If you've never used VSCode with Python, I strongly recommend at least the first half of Dan Taylor's [Get Productive with Python in Visual Studio Code](https://www.youtube.com/watch?v=PnOPp4DsY2w) talks. It covers a lot of the basics like installing Python, the Python extension, and the command pallette.
26+
27+
You should also have the following installed:
28+
29+
- Git
30+
- VSCode
31+
- Python 3 (via anaconda, brew or [python.org](https://www.python.org/downloads/) )
32+
33+
In VSCode you'll also need to install the Python extension
34+
35+
### If you already know VSCode
36+
37+
Here's [a gist for an example .vscode directory](https://gist.github.com/swanderz/5cf876d88c7c8d268d8c1e1e5d05bffd) that contains a `settings.json` and an `extensions.json`
38+
39+
### Getting started
40+
41+
To get started, we'll use the [jaffle_shop repo](https://github.com/dbt-labs/jaffle_shop), a self-contained project.
42+
43+
You can use the Git CLI or the VSCode Git extension to Git Clone command in VSCode
44+
```bash
45+
git clone https://github.com/dbt-labs/jaffle_shop.git
46+
```
47+
48+
Then, open the `jaffle_shop/` directory in VSCode.
49+
50+
## Python environment
51+
52+
53+
### Goal
54+
55+
The goal of this section is to ensure that the right version of Python and dbt are always available right away when you open your dbt project in VSCode. Sounds simple, but below is a one-time setup guide on how to make it work. More context is that some folks have bundled this set up process into [bash scripts](https://discourse.getdbt.com/t/setting-up-your-local-dbt-run-environments/2353) and [Docker containers](https://discourse.getdbt.com/t/a-containerized-dbt-environment-for-your-team/2340).
56+
57+
Some folks deem this problem so difficult as to justify having users use Docker containers, but I have yet to be convinced of that yet.
58+
59+
### Walkthrough
60+
61+
Python can be tricky get working in VSCode (and trickier to work on Windows). You OS likely already has a version of python installed, but this can be troublesome because you don't control it's version.
62+
63+
It's better practice to have a dedicated dbt environment. Three popular tools are `venv`'s, `virtualenv`'s and `conda` environments. Our team uses `conda` envs because we have many different projects with different sets of package requirements, but if dbt is 1) your only use case for Python, or 2) your first Python-based use case, you'll likely have a better time with `virtualenvs`. I'm going to only talk about venv because it comes built-in with Python
64+
65+
Open a terminal with `CTRL+`` (which should open within the jaffle_shop directory) and do the following steps:
66+
67+
```bash
68+
# make sure you have Python at least 3.6 and less than 3.10
69+
# Create and activate virtual environment
70+
python3 -m venv .dbtenv
71+
source .dbtenv/bin/activate
72+
73+
# install the dbt package you want
74+
pip install dbt-synapse # or dbt-sqlserver or whatever
75+
76+
# make Git ignore all these newly created files
77+
echo '.dbtenv/' > .gitignore
78+
```
79+
80+
Once you've done this you should now be able to:
81+
1. bring up the command pallette (`CMD+SHIFT+P`)
82+
2. search for "Python: Select Interpreter", and
83+
3. Pick the `.dbtenv` environment (should be the first result)
84+
85+
Those three steps will:
86+
87+
1. activate the Python extension if it hasn't been already
88+
2. ensure that all new terminals opened in VSCode will auto-activate your `.dbtenv` environment
89+
90+
This is huge because now all your terminals in the VSCode will always have your dbt package available. However, this behavior will not persist the next time you open this repo in VSCode. To make this auto-env selection persist, you must do two things:
91+
1. add a `requirements.txt` to you the top level of the repo ([pip docs on `requirements.txt` files](https://pip.pypa.io/en/stable/user_guide/#requirements-files))
92+
2. (optional) add to the `requirements.txt` what packages w/ versions you plan to do in this project (example below)
93+
3. create a new file `.vscode/settings.json` and add the Python path to the `settings.json` (more on VSCode settings later!)
94+
95+
#### `requirements.txt`
96+
```
97+
dbt-synapse==0.19.2
98+
sqlfluff==0.7.1
99+
```
100+
#### `.vscode/settings.json`
101+
```json
102+
{
103+
// change this to your desired path!
104+
"python.pythonPath": "./.dbtenv/bin/python",
105+
}
106+
```
107+
108+
Now that you've done these two things, everytime you open the `jaffle_shop/` dir, in VSCode two things should happen:
109+
1. the Python extension activates right away (do you see the Python version listed alongside your environment name on the bottom info bar?)
110+
2. any terminal you open will auto-activate your `.dbtenv` and each line should begin with `(.dbtenv)`
111+
112+
You should test this by closing VSCode, then opening the `jaffle_shop` repo
113+
114+
## Extensions
115+
116+
In this section, I'll go over some of the extensions that our team uses. Each extension requires that you install it from within VSCode, and most will require adding additional settings to your `.vscode/settings.json`
117+
118+
### vscode-dbt
119+
120+
the [vscode-dbt extension](https://marketplace.visualstudio.com/items?itemName=bastienboutonnet.vscode-dbt) is great because it provides a few things:
121+
1. syntax highlighting for SQL with jinja in it (also for `.md`'s and `.yml`s), and
122+
2. helpful jinja snippets will save you a lot of time
123+
124+
125+
To get this working you should add the following to your `.vscode/settings.json`
126+
127+
There's an optional addition I strongly recommend `"**/target/**": "",`, which will not do any syntax highlighting/colorization to any file in the `target/` folder. This prevents me from making the classic mistake where I start editing a compiled model file, instead of the original model file. Then when I call `dbt run` my changes aren't incorporated, but instead overwritten by the unchanged logic of the model file. with this setting you know something is wrong then the sql has no coloring.
128+
129+
```json
130+
"files.associations": {
131+
// the pattern on the left side can be whatever you want: e.g.
132+
"**/jaffle_shop/**/*.sql": "jinja-sql", // just the .sqlfiles inside of jaffle_shop, or
133+
"*.sql": "jinja-sql", // all .sql files
134+
135+
// optional: don't format models in `target/` dir
136+
"**/target/**": "",
137+
// I don't personally use these, but you can also have jinja work for `yaml` and `md` files
138+
"**/<dbt-project-dir>/**/*.yaml": "jinja-yaml",
139+
"**/<dbt-project-dir>/**/*.yml": "jinja-yaml",
140+
"**/<dbt-project-dir>/**/docs/**/*.md": "jinja-md"
141+
142+
// the vscode-dbt docs say you may need this
143+
"editor.quickSuggestions": {
144+
"strings": true
145+
}
146+
}
147+
```
148+
149+
You'll know it is working when you open a `.sql` model and, in the bottom toolbar on the right it says now says "Jinja SQL" instead of "SQL".
150+
151+
### Find Related
152+
153+
the [find-related extension](https://marketplace.visualstudio.com/items?itemName=amodio.find-related) allows you to use regular expressions to correspond a `.sql` file in your `models/` directory to it's `compiled` and `run` counterparts in the `target/` folder. I find this a huge timesaver compared to manually naviagting the `target/` dir in the explorer sidebar.
154+
155+
After you install the `find-related` extension, you can enable it by adding the following to your `.vscode/settings.json`. There's no dbt or jinja magic going on here, just regex. So you may need to tweak these settings if they're not working for you.
156+
157+
Once it is set up, you can type `Option+R` on any model file to jump to it's compiled version. While on a compiled model file, `Option+R` will take you to it's `target/run` counterpart.
158+
159+
160+
```json
161+
{
162+
// this is so you can easily jump to your compiled SQL files
163+
"findrelated.workspaceRulesets": [
164+
{
165+
"name": "sql",
166+
"rules": [
167+
{
168+
"pattern": "^(.*/)?models/(.*/)?(.+\\.sql)$",
169+
"locators": [
170+
"**/compiled/**/$3"
171+
]
172+
},
173+
{
174+
"pattern": "^(.*/)?compiled/(.*/)?(.+\\.sql)$",
175+
"locators": [
176+
"**/run/**/$3"
177+
]
178+
},
179+
{
180+
"pattern": "^(.*/)?run/(.*/)?(.+\\.sql)$",
181+
"locators": [
182+
"**/models/**/$3"
183+
]
184+
}
185+
]
186+
}
187+
],
188+
"findrelated.applyRulesets": [
189+
"sql"
190+
]
191+
}
192+
```
193+
194+
### Rainbow CSV
195+
196+
the [rainbow-csv extension](https://marketplace.visualstudio.com/items?itemName=mechatroner.rainbow-csv) just highlights csvs where each column is it's own color. It's great to use when you have a csv where character-width varies greatly within a column. You can also hover over a value to see what column it belongs to. Very helpful for seeds!
197+
198+
### SQL Fluff
199+
200+
Our team has recently implemented sqlfluff linting for our dbt projects, especially because versions `0.6.5` and greater now support TSQL. There's also a great VCcode extenstion.
201+
202+
If you already have a `.sqlfluff` and `.sqlfluffignore` configured and working, it is enough to install [vscode-sqlfluff](https://marketplace.visualstudio.com/items?itemName=dorzey.vscode-sqlfluff) and add the following to your `settings.json`
203+
204+
```json
205+
// you get this by calling `where sqlfluff` after calling `pip install sqlfluff`
206+
"sql.linter.executablePath": "<PATH_TO_YOUR_SQLFLUFF_FROM_WHICH_SQLFLUFF_COMMAND",
207+
"sql.linter.run": "onType" // alternatively "onSave" if you'd like it less frequent
208+
```
209+
210+
### dbt Power User
211+
212+
I personally haven't had time to use the [dbt Power User](https://marketplace.visualstudio.com/items?itemName=innoverio.vscode-dbt-power-user) extension, but folks have good things to say. I hope to try it out soon and folks are more than welcome make a Pull Request to this doc as they see fit.
213+
214+
## Settings
215+
216+
### Extra settings
217+
Here's some other settings that I recommend:
218+
219+
220+
```json
221+
// easier to see if there are unsaved changed
222+
"workbench.editor.highlightModifiedTabs": true,
223+
"workbench.editor.labelFormat": "medium",
224+
// make Command Prompt the default shell for Windows instead of Powershell
225+
"terminal.integrated.shell.windows": "C:\\Windows\\System32\\cmd.exe",
226+
227+
// make a vertical line so I don't make lines too long
228+
"editor.rulers": [80],
229+
// show whitespace as dots
230+
// (easier to count out indentation and spot trailing whitesapce)
231+
"editor.renderWhitespace": "all",
232+
```
233+
234+
### Workspace-level settings files
235+
236+
Sometimes it isn't convenient to have a `.vscode/settings.json`, such as when you:
237+
1. have a subset of settings under source control that you'd like all users to be using (it doesn't make sense to source control your specific Python path)
238+
2. you prefer [multi-root workspaces](https://code.visualstudio.com/docs/editor/multi-root-workspaces) a.k.a. more than one repo open at at time (great for when you also want your `.dbt/profiles.yml` close at hand)
239+
240+
A worksapce settings file has the extension `.code-workspace` and encapsulates all the configuration you might find in a `.vscode/` dir into a single file. This file also works as a shortcut that you can double click or navigate to to bring up all your settings.
241+
242+
If someone wants more info on this free free to open an issue. For now I'll leave this as as stub.
243+
244+

0 commit comments

Comments
 (0)