-
GCP – DataFlow Pipeline Using Flex Template.
GCP – DataFlow Pipeline Using Flex Template. Steps For Creating DataFlow Pipeline: Install Prerequisites: 1 – Python Setup: https://www.python.org/downloads/ 2 – VsCode Setup: 3 – Docker Desktop Setup: https://www.docker.com/products/docker-desktop/ 4 – Google Cloud SDK Setup https://dl.google.com/dl/cloudsdk/channels/rapid/GoogleCloudSDKInstaller.exe 5 – VS Code Project Structure 6 – VS Code Extensions 7 – Authenticate With GCP #1 Login To Your GCP Account gcloud auth login #2 Set application default credentials (important for local development) gcloud auth application-default login #3 Set your project gcloud config set project ccoe-poc-admin #4 Verify Authentication gcloud auth list 8 – Set Up Python Virtual Environment python -m venv log_monitoring_venv
