Snowflake

Connecting to Snowflake from Saturn

Connecting to Snowflake

Snowflake

Snowflake is cloud native enterprise data warehouse. They publish native python drivers, the snowflake-connector-python , or if you prefer to use SQLAlchemy, snowflake-sqlalchemy . This article goes through using both packages, as well as building them into your own image.

Snowflake Connector with Pandas

After having spinning up a JupyterLab instance and a Dask cluster connecting to your Snowflake data warehouse using snowflake-connector-python is simple.

import snowflake.connector
import pandas as pd

ctx = snowflake.connector.connect(
    user='YOUR_USER',
    password='YOUR_PASSWORD',
    account='YOUR_ACCOUNT'
)
query = "SELECT * FROM SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.CUSTOMER"
pd.read_sql(query, ctx)

Snowflake also has an optimized routine for loading data into a pandas dataframe

cursor = ctx.cursor()
cursor.execute(query)
df = cursor.fetch_pandas_all()

SqlAlchemy

If you’re used to working with SqlAlchemy you can use it with snowflake as well

from sqlalchemy import create_engine
from snowflake.sqlalchemy import URL
from sqlalchemy import sql
from dask.dataframe.io.sql import read_sql_table

engine = create_engine(URL(
    user='YOUR_USER',
    password='YOUR_PASSWORD',
    account='YOUR_ACCOUNT',
    schema ='TPCH_SF1',
    database = 'SNOWFLAKE_SAMPLE_DATA'
))

engine_url = engine.url
query = "SELECT * FROM SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.CUSTOMER"
pd.read_sql(query, engine)

Packages

The images that come with Saturn come with both Snowflake packages installed. If you are building your own images and want to work with Snowflake, you should install snowflake-connector-python as well as snowflake-sqlalchemy