This topic appears to be something that should have been posted to a Python Discussion Group, not the JMP Discussion Forum
Link to post: (Airflow Cloud Build Operator not executing python function)
by selenanur
https://community.jmp.com/t5/Discussions/Airflow-Cloud-Build-Operator-not-executing-python-function/m-p/321460#M57150
I can’t get the second arg statement to execute: "python", "-c", "from main import foo;print(foo('hello'))" I’m triggering a Cloud Build using the Airflow operator python function. Both the triggering DAG and Cloud Build return successful. The first arg to install the requirements.txt file appears to work based on the Cloud Build returned log message: Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.7/site-packages (from google-auth>=1.2->gcsfs>=0.6.2 My function is using the gcsfs package. The function I have used in the question is just for demonstration. Before using Airflow I had this working in a yaml file. Tried a number of different ways of stating the argument, such as: "args":["-c","pip install -V -r requirements.txt; python -c from main import foo;print(foo('hello'))"] But now going round in circles. I would greatly appreciate any help to get that args statement working. This is the CloudBuildCreateBuildOperator from the DAG: create_build_from_storage_body = { "source": { "repoSource": { "projectId": "dev-6767", "repoName": "bq-pipeline", "branchName": "master", } }, "steps": [{ "name": "docker.io/library/python:3.7", "entrypoint": "/bin/bash", "args": ["-c", "pip install -V -r requirements.txt", "python", "-c", "from main import foo;print(foo('hello'))" ], }], }
This message has 0 replies
