Create dbt-MariaDB Virtual Environment
About this task
Required only when MariaDB is the CDM target database. The IF02 DAGs
activate ~/mariadb/bin/activate before running dbt for MariaDB.
Procedure
-
Create a virtual environment for MariaDB:
python3 -m venv ~/mariadb -
Upgrade pip and install dbt Core, the MariaDB dbt adapter, and PyMySQL in
the virtual environment:
~/mariadb/bin/pip install --upgrade pip ~/mariadb/bin/pip install --no-cache-dir dbt-core==<version> \ <MariaDB dbt adapter>==<version> pymysqlNote:The IF02 package does not specify the dbt-core version or MariaDB dbt adapter package. Obtain both from HCL before installing, and use the same versions on every Airflow worker. -
Verify the installation:
~/mariadb/bin/dbt --version
What to do next
Airflow Python environment. The
ddl_execution_26.1_IF02 DAG runs the DDL modules with the Python
interpreter of Airflow, not unica_venv. Install the packages
required by the databases you use in that environment. The
module/requirements.txt file lists
oracledb, snowflake-connector-python, and
pyodbc. For MariaDB, install pymysql separately;
it is not included in that file.
pip install -r <AirflowDags>/module/requirements.txt
pip install pymysql # MariaDB only