BigQuery

BigQuery is a Warehouse output connector. Connect to your BigQuery data warehouse to get all your vulnerability data in one place.

IP Allowlists

You might need to add the Monad IP range of 34.210.32.104/32 to your BigQuery allowlist, so Monad can connect to it.

When you connect BigQuery with Monad, you own your own data. You can get all of your security and environment data in one place and run your own queries across all of your security data.

You must create a BigQuery dataset and a Google Cloud IAM service account to grant Monad access to create tables and store data. Below you can find a step-by-step guide on how to integrate monad with your Google Cloud BigQuery datasets.

Setting up Google Cloud and BigQuery for Monad Integration

  1. Create a Google Cloud Project or use an existing project.
    1. You can create a new project by clicking your projects on the top left of your Google Cloud home page. Then select new project. Your poject will be given an ID, this ID is what we will be using in the connector configuration not the project name.
  2. Create an IAM Service Account role for monad to use with BigQuery Job User and BigQuery Data Editor permissions for the project you want to use.
    1. Navigate to IAM and select Service Account in the navigation panel on the left.
    2. Click Create Service Account on the top of the page. Note: Make sure you are in the project you wish to use.
    3. Give the service account a name and select Create and Continue.
    4. Click the Select A Role dropdown and choose BigQuery Job User.
    5. Click Add Another Role and this time choose BigQuery Data Editor.
    6. Select Done
    7. Click the Service Account you just created and select KEYS -> Add Key -> Create Key -> JSON -> Create. This will save a JSON file of the key to your computer. You will need this JSON key to paste into the monad connector UI.
  3. Set up the monad BigQuery Connector.
    1. Select Add Connector and choose BigQuery
    2. Enter in the project id under Project do not use the project name as this is not universally unique and google does not use this to identify your project.
    3. Enter the Dataset name you would like the connector to create for you.
    4. Open your JSON key file. copy the contents and paste it into the Credentials JSON field.
    5. (Optional) - select your models to export
    6. You have 2 boolean field options
      • Denormalize: Determines whether to denormalize the data before writing to BigQuery. When disabled Monad will write data using a fixed JSON column based schema. When enabled Monad will write data using the monad object model Schema. If you choose not to denormalize the vulnerability data will be stored as a single JSON Data column and will be formatted where the keys are the column names.
      • Overwrite: Determines if the target table should be overwritten on load. When enabled Monad will write to a temporary table and then overwrite the target table inside a transaction (so it is safe). When disabled Monad will write to the target table directly (append) This will lead to many duplicate entries but can be used to track the history of your vulnerabilities. Be aware not overwriting the table will result in large quantities of data overtime. Monad handles upserting data already, so you don’t need to worry about missing or duplicate data during an Overwrite operation. You will get all the latest data Monad has for the vulnerabilities.
    7. Click Connect.

Monad then tests the connection to BigQuery, and if successful, begins syncing data from your Monad account into your BigQuery warehouse.

This page was last modified: 10 Oct 2023