Download Jupyter Notebook

Repositories

Repositories are lists of datasources (and other data items).

When a repository is added to the user workspace, all the data items in it are automatically loaded to the project following its creation with the hera-project CLI.

The management of the repositories is performed using the hera-project repository CLI which allows the user to add, list and remove repositories from his workspace.

Each repository is a JSON file that describes datasources, as well as measurements, cache and simulations documents.

Repository management¶

The management of the repository is performed using the

     >> hera-project repository <Action>

Where Action is

• add : Adding a repository to the workspace
• list: List the repositories in the workspace
• remove: Remove a repository from the workspace.
• show : Shows the content of the repository.

Listing repositories¶

Listing the repositories in the workspace is achieved by,

>> hera-project repository list

Adding repositories¶

Adding a repository to the workspace is achieved by stating the full path to the repository JSON file,

>> hera-project repository add {path to file}

Example:

>> hera-project repository add /home/dataSource/repository.json

This adds the repository repository.json to your workspace.

Removing repositories¶

Removing the repository from the workspace is performed,

>> hera-project repository remove {repository name}

Showing repositories¶

Showing the content of the repository was performed by,

>> hera-project repository show {repository name}

Repository definition¶

A repository is a JSON file that lists datasources (e.g external databases that some toolkit use), and other types of documents (e.g measurement, simulation, and cache).

The repository JSON file lists the items that will be loaded for each toolkit.

The definition of the datasource, measurements, simulations and cache items is specified per toolkit. That is,

{
 <toolkit name> : {
    "DataSource" : { 
        ... data items definition ...
    },
    "Measurements" : {
        ... data items definition ...        
    },
    "Simulations" : {
        ... data items definition ...                
    },
    "Cache" : {
        
        
    }    
}

WhereDataSource specifies the data items that will be added as data sources, and the rest as the relevant document type (measurement, simulations, and cache).

The structure of the data item is slightly different for datasource and other data types. Hence, we will delineate each one of them.

The DataSource item structure¶

The data item for the data source is specified as follows

<name> : {
             "isRelativePath" : <true/false>,
             "item" : {
                "resource" : <file name>,
                "dataFormat" : <data type>,
                "desc":{
                        ... user defined fields ...
                }
}

• name: The name of the dataitem is used in the case of the datasource (it is the name of the datasource).
• isRelativePath: If true, change the resource to be relative to the location of the repository JSON. If it is False, then the value of the resource remains as it is.
• item: In this map we will denote the resource, dataFormat and the desc.

The Measurement/Cache/Simulation item structure¶

The data item for the Measurement/Cache/Simulation item data is specified as follows

<stub-name> : {
             "isRelativePath" : <true/false>,
             "item" : {
                "resource" : <file name>,
                "dataFormat" : <data type>,
                 "type" : <type>
                "desc":{
                        ... user defined fields ...
                }
}

• stub-name: A stub. This value is ignored.

• isRelativePath: If true, change the resource to be relative to the location of the repository JSON. If it is False, then the value of the resource remains as it is.

• item: In this map we will denote the resource, dataFormat and the desc.

Example for a repository file¶

Example:

An example to add to the GIS_Demography toolkit 2 data items.

1. A data source called theData. The resource is a JSON file called theData.json that is in the same directory of the repository JSON file.

2. A Measurements data item. The resource is the string my value.

{
	"GIS_Demography" : {
		"DataSource" : {
			"theData" : {
				"isRelativePath" : true,
				"item" : {
					"resource" : "theData.json",
					"dataFormat" : "JSON_dict",
					"desc" : {
						"y" : 1
					}
					
				}
			}
		},
		"Measurements" : {
			"tmp1" : {
				"isRelativePath" : false,
				"item" : {
					"resource" : "my value",
					"dataFormat" : "str",
					"type" : "myType", 
					"desc" : {
						"x" : 1
					}
				}				
			}
		}		
	}	
}