❯ Guillaume Laforge

Cloud-Storage

Upload and use JSON data in your workflow from GCS

Following up the article on writing and reading JSON files in cloud storage buckets (GCS), we saw that we could access the data of the JSON file, and use it in our workflow. Let’s have a look at a concrete use of this. Today, we’ll take advantage of this mechanism to avoid hard-coding the URLs of the APIs we call from our workflow. That way, it makes the workflow more portable across environments. Read more...

Reading in and writing a JSON file to a storage bucket from a workflow

Workflows provides several connectors for interacting with various Google Cloud APIs and services. In the past, I’ve used for example the Document AI connector to parse documents like expense receipts, or the Secret Manager connector to store and access secrets like passwords. Another useful connector I was interested in using today was the Google Cloud Storage connector, to store and read files stored in storage buckets. Those connectors are auto-generated from their API discovery descriptors, but there are some limitations currently that prevent, for example, to download the content of a file. Read more...

Tip: Making a Google Cloud Storage bucket or file public

Google Cloud Storage is the ideal product to store your object files (binary files, pictures, audio/video assets, and more). Until recently, there was an option in the Google cloud console with a checkbox to quickly make a file or bucket public. However, and I would add “unfortunately”, users tended to inadvertently clicking the checkbox, thus making potentail confidential assets public. So this risky, but easy, option, has been removed to avoid any unwanted data leak. Read more...