In Harmoni, you can use data from various file types as the source for new projects, visualizations, and dashboards. Harmoni automatically maps source variables into Harmoni types.
A data source is the starting point for all you do in Harmoni. A source is a collection of data either in an imported file or a connected data store.
Direct connections allow updated data to flow into Harmoni in real time.
- Achieved using APIs (Application Programming Interface).
- If an API is available in a data collection system, we can potentially develop a direct connection to Harmoni.
Harmoni supports different direct connections, including Google BigQuery.
In this article
1. Connect to BigQuery
In Harmoni, you can choose to create a new project that connects to a BigQuery data source, or you can connect from an existing project.
- To create a new project, type in your project name and click on Create New.
- Choose to Connect.
- Choose Google BigQuery.
- Enter the service account key.
BigQuery Service Account Key
To connect to BigQuery, you need to enter a Service Account Key in JSON format.
There are two types of GCP credentials - user credentials and service account credentials. Below an example with service account credentials. To generate the key:
1. Create a service account
2. Give the service account access to the data set
3. Create a key for the service account
4. Key is saved to downloads folder
Notice that the key can be very long, you need to copy all including the brackets.
3. Select the Dataset
After clicking CONNECT, you are presented with the datasets available in your connection. There are two options available.
- Column-based data (default setting)
- Row-based data
a. Column-based data
The default setting is Column-based.
- Click on a dataset, and you are presented with all the tables and views available in that dataset.
- Click ADD to add the required table(s) to your project.
- Once added, the table will show in the source area.
- Click on the source tile to define the new source, as is required with delimited sources.
- Map the data type of each column.
- Confirm and add the data source to the project.
- Click OK to create the project.
b. Row-based data
Harmoni also has the option for row-based data. Use the switch to change from column-based to row-based.
To make the source selections for row-based data, Harmoni has a three-step process to follow:
1. Select the table/view containing the IDs.
2. Select the table/view with the dictionary.
3. Lastly, select the data.
Once the three-step process is complete, select ADD to add the files to the project.
- Click OK to create the project.
4. Harmoni project
After connecting to your sources, you can load your Harmoni project. Direct connections allow for a real-time flow of survey data.