In Harmoni, you can replace or remove existing data sources from your project; however, it is essential to do this with caution to avoid losing any existing project definitions.
In this article
1. Overwrite
Overwrite allows you to replace a data source with updated data.
Some scenarios include:
- New tracker update, where the project has been created using a cumulative file, and a new wave of data is available.
- When there are changes in the original source, like adding new variables or data cleaning.
Harmoni will only identify an updated source when the name and source type are the same.
- Select view/add sources from the action pane.
- Select ADD/REMOVE from above the source names.
- This opens the Sources area and displays a list of the data sources linked to your project.
- Choose Upload or Connect.
- Navigate to the location of your updated data source(s).
- Harmoni will ask you to overwrite the original file with the new updated one.
- Make sure that all sources you wish to include in your project are selected.
- When ready, click the OK button.
- Prior to loading your project, Harmon may display a warning when there are any original project items that are no longer present in any of the sources and will, therefore, be removed completely from the project. (Preview removed items). If you choose to Continue, the project will load.
2. Remove data sources
You can replace data sources with new data or completely remove specific sources from your project. It is important to do this with caution to avoid losing any existing project definitions that could break your project and stories.
You must add the new data file to your project as a new data source and harmonize it before removing the old source(s).
To conserve project definitions, before removing sources, it is important to add the new sources, load the project, check what is new, harmonize axes and elements as relevant and then remove the sources you no longer wish to use.
a. Add New Sources
For example:
You have created a project with two separate sources for one market (i.e., New Zealand 2017 and 2019), you now have a cumulative file (Brand Monitor_NZ), so you want to use this one instead.
You will need to upload your new sources and load your project, including the sources you wish to remove.
- Select view/add sources from the action pane.
- Select ADD/REMOVE.
- This opens the Sources area and displays a list of the data sources linked to your project.
- Choose Upload or Connect.
- Navigate to the location of your new data source(s).
- Make sure that all sources you wish to include in your project are selected. It is important at this stage to keep the existing sources, even those you wish to remove.
- When ready, click the OK button, and your project will load. The project will include new and old sources; this step is required so the logic that is defined by the old sources can be transferred to the new sources.
b. Harmonize
After loading your project, Harmoni automatically identifies what's new. New items are shown in bold in the project tree. You can also identify new items by using the New icon under the Projet tree filters.
Make sure you apply all harmonization changes required; after this, you can go back and remove the sources you no longer need.
- Identify NEW items (using the Project tree filters) and harmonize as required.
c. Remove Sources
Once you have harmonized axes and elements as needed, you can remove the sources you no longer need in your project.
- Select view/add sources. from the action pane
- Select ADD/REMOVE.
- This opens the Sources area and displays a list of the data sources linked to your project.
- You can now unselect the sources you no longer want to use by unticking directly on the source tile or using the X on the source name in the source diagram.
- Make sure that all sources you wish to include in your project remain selected.
- When ready, click the OK button.
- Before loading your project, Harmoni may display a warning when any original project items are no longer present in any of the sources. If you continue, these items will be removed completely from the project. (Preview removed items). Click Continue to load the project.
- Before loading your project, Harmoni may display a warning when any original project items are no longer present in any of the sources. If you continue, these items will be removed completely from the project. (Preview removed items). Click Continue to load the project.
d. Delete Sources
Once you have removed the sources from your project, you can delete them.
Go back to the Sources area and on the source tile, use the three-dot menu to access the delete option. Note that you will not see the delete option until after the source has been removed from the project.
3. Preview removed items
Before Loading a Project
After overwriting or removing a data source and before you load your project, Harmoni displays a warning message when existing project items are not present in the new data source.
The ‘Preview removed items’ tickbox is enabled by default. When ticked, Harmoni displays a warning when there are original project items no longer present in any of the sources and will, therefore, be removed from the project. At this point, you can choose to continue or cancel the action.
For example:
You have created a project with two separate sources, one for Australia and one for New Zealand. You now have an updated data file for Australia; however, this new file no longer contains the variable "Brand Drink Most Often," which was only present in Australia. By overwriting the data source and updating your project, the variable will be removed from the project and will no longer be available as a reference for any constructions in which it has been used.
- Make sure that all sources you wish to include in your project are selected.
- When ready, click the OK button.
- As we are replacing a source file here, Harmoni asks if you want to Overwrite
- Before loading your project, Harmoni may display a warning when any original project items are no longer present in any of the sources. If you continue, these items will be removed completely from the project. (Preview removed items). Click Continue to load the project.
Harmoni will identify what is new in a project. It also identifies any constructions that need to be redefined due to items being removed from data sources.
After Loading the Project
Invalid Constructions
Harmoni automatically identifies "broken" constructions as a result of a data source being removed or overwritten. Invalid constructions are shown in red in the project tree. You can also identify invalid constructions using the ERR icon under the Project tree filters.
If items break, so too will Stories created using these items.
It is important to discuss with your team and/or researchers using the project before proceeding.
- A source file is overwritten.
- Harmoni warns that items will break.
- You can choose to Continue or Cancel at this point.
- Only continue if you know that removing and breaking items is expected and understood by all users of the project.
- Identify invalid constructions by using the ERR option from the project tree filters.
If an element that is used in construction is removed, Harmoni will list the axis in the construction definition and show the element as missing.
When the whole axis is removed, the definition shows both the axis and element as missing.
Where to from here?
Learn more about data sources
- New Project - Managing data sources
- Add data sources to your project
- Append data sources to your project