DDB Documentation

# Python - PyDDB

Note for external users - many of the links in the following section refer to internal applications. Please speak to your project team for support.

The pyddb package provides a clear way for non-developers to interact with DDB by simplifying syntax and implementing convenience features.

This guide will cover installation and quick start of the package, as well as a full reference of the package and its features.

# Features

Besides greatly simplifying the interface for interacting with the DDB API endpoints, the library provides other useful features:

Authentication

The client will prompt the user for their ARUP credentials when the client is instantiated and automatically refresh the token as necessary.

Simplified API and Implementation

The library is built for beginner python users who want to create automated workflows.

We've simplified all microservice endpoints into a single library with a consistent interface. This means there's less to learn and it's easy to get started.

You can keep rerunning scripts and we'll handle checking for duplicates and performing any updates for you. Typically scripts can be used on other projects just by changing the project number.

Easily Explore DDB Taxonomy

DDB has it's own taxonomy: you can't just create a new type of asset or parameter.

This library provides ways of searching through our existing taxonomy by name or uuid.

Fully Tested

This library is fully tested against current and future versions of the DDB API, so you can get started with confidence.

Pagination

By default, our endpoints will return the first page of data. This library handles pagination and retrieves all data for you.

IDE Support

The library is fully typed, so you can get autocomplete and type checking in your IDE.

# Installation and Setup

  1. Requires Python 3.10+ (App Store) - Required to run the package.
  2. Requires Git (Service Desk) - Required to install the package.
  3. Requires GitHub account with Arup email - Required to install the package.
  4. Recommended Visual Studio Code (App Store) - Recommended environment for writing code.

Once you have the above installed, open Visual Studio Code and open a folder you would like to store your work in.

Open a Bash terminal and run the following commands:

Create a virtual environment

python -m venv .venv

Activate the virtual environment

source .venv/Scripts/activate

Install the DDB Python SDKs

pip install keyring artifacts-keyring python-dateutil
pip install ddb-user-service \
ddb-reference-data-service \
ddb-qa-service \
ddb-parameter-service \
ddb-parameter-metadata-service \
ddb-environment-context-service \
ddb-comments-service \
ddb-template-service \
-i https://pkgs.dev.azure.com/ovearup/_packaging/ddb/pypi/simple/

Install the PyDDB package

pip install git+https://github.com/arup-group/PyDDB.git

# Quick Start

# Getting Data

Getting data from DDB is as simple as calling the appropriate method on the project object. This will handle pagination and return objects with metadata. Look at the API reference for more information.

Import the parts of the library you need:

from pyddb import DDB, env, Project, ParameterType

Instantiate the client and set the environment:

ddb = DDB(env.sandbox)

Get an existing ddb project:

my_project = Project.get_by_number(project_number="21515700")

Get data from the project:

project_parameters = my_project.get_parameters()
project_assets = my_project.get_assets()
project_sources = my_project.get_sources()

Get data from the taxonomy:

all_parameter_types = ParameterType.get_all()
some_parameter_types = ParameterType.search('area')
my_parameter_type = ParameterType.get("Plot area")

# Posting Data

Import the parts of the library you need:

from pyddb import DDB, env, Project, ParameterType, AssetType, Unit, Source, Parameter, Asset

Instantiate the client and set the environment:

ddb = DDB(env.sandbox)

Create a project or get an existing one:

project = Project.put("21515700")

Define the assets you want to post:

assets = [
    my_site := Asset.create(
        asset_type=AssetType.get("site"),
        name="My DDB Site",
        parent=None,
    ),
    my_building := Asset.create(
        asset_type=AssetType.get("building"),
        name="My DDB Building",
        parent=my_site,
    )
]

Define the parameters you want to post:

parameters = [
    Parameter.create(
        parameter_type=ParameterType.get("Plot area"),
        value=240,
        unit=Unit.get("m²"),
        parent=my_site,
        source=Source.create(
            source_type = SourceType.get("Derived Value"),
            title="My source title",
            reference="My source reference",
        ),
    )
]

Post the data:

project.post_assets(assets)
project.post_parameters(parameters)

Each of these post methods will return what they are posting and do not add duplicate data. Posting new parameters will not override existing parameters if the value, unit, or source has not changed.

This means that these methods can be used to created automated workflows.

# API Reference

These docs are broken down by class and provide a description and example of how they should be used. In this library, we have two main types of classes that serve different purposes:

Data Classes: These represent the values of our data and are what you will be posting and retreiving to interact with your project data. These will typically be tied to the taxonomy classes so we know what kind of data we are working with.

Taxonomy Classes: These classes, such as 'ParameterType' or 'AssetType', are determined by our team. These are ways of categorising our data and giving it context.

In short, the data classes are the data you are working with, and the taxonomy classes are the ways we classify it. You can also explore our glossary for more information on the terms we use.

# Data Classes

The DDB and Project classes are the main entry points for the library. The DDB class is used in every script to instantiate the client and set the environment. The DDB class is used to access data across projects, while the Project class is used to access data on a specific project.

# DDB

The DDB class is the main class for the client. It is used to instantiate the client and set the environment. All scripts should instantiate the client with the environment they are working in.

from pyddb import DDB, env

ddb_client = DDB(env.sandbox)

You can now use this to access data across projects. It's recommended to use very specific queries to reduce the amount of data returned.

all_projects = ddb_client.get_projects()

all_plot_areas = ddb_client.get_parameters(
    parameter_type_id=[
        ParameterType.get("Plot area").id
    ]
)

# Project

The Project class is used to access data on a specific project. It is instantiated by passing the project number to the get_by_number or put methods.

from pyddb import DDB, env, Project

DDB(env.sandbox)

my_project = Project.get_by_number("00000000")
my_project = Project.put("00000000")

The difference between these two methods is that get_by_number will only return the project if it exists in DDB. put will create the project if it does not exist.

# Source

The Source class is used to create and post sources to DDB. A new source is instantiated by passing the source type, title, and reference to the create method.

from pyddb import DDB, env, Project, Source, SourceType

DDB(env.sandbox)

my_source = Source.create(
    title="My source title",
    reference="My source reference",
    source_type=SourceType.get("Derived Value"),
)

Sources can be posted to DDB by passing a list of sources to the post_sources method on a Project.

my_project = Project.put("00000000")

my_project.post_sources([my_source])

Sources on a project can be retreived by calling the get_sources method on a Project.

project_sources = my_project.get_sources()

This means it's very easy to get sources from a past project and reuse them in a new project.

# Asset

The Asset class is used to create and post assets to DDB. A new asset is instantiated by passing the name, asset type, and parent to the create method.

One important thing to note is that in DDB, you cannot have two assets with the same name, type and parent. This means that a single site can't have two buildings with the same name. This library uses this logic to prevent duplicate assets from being created and instead will return the existing assets if scripts are rerun.

from pyddb import DDB, env, Project, Asset, AssetType

DDB(env.sandbox)

my_asset = Asset.create(
    name="My asset name",
    asset_type=AssetType.get("Site"),
)

my_other_asset = Asset.create(
    name="My other asset name",
    asset_type=AssetType.get("Building"),
    parent=my_asset,
)

yet_another_asset = Asset.create(
    asset_sub_type=AssetSubType.get("Cooling"),
    parent=my_other_asset,
)

There are a few rules for creating assets:

  • Assets need either an AssetType, or an AssetSubType so we know what kind of asset it is.
  • Assets without an AssetSubType need a name.
  • Assets need to follow the Asset Type hierarchy. For example, a Building asset needs to have a Site asset as a parent.

You can explore the AssetType class to get an understanding of the hierarchy or use the DDB UI.

Assets can be posted to DDB by passing a list of assets to the post_assets method on a Project.

my_project = Project.put("00000000")
my_project.post_assets([my_asset, my_other_asset, yet_another_asset])

Assets on a project can be retreived by calling the get_assets method on a Project.

project_assets = my_project.get_assets()

This means it's very easy to get assets from a past project and reuse them in a new project.

# Parameter

The Parameter class is used to create and post parameters to DDB. A new parameter is instantiated by passing the parameter type, value, unit, source, and parent to the create method.

You'll want to be familiar with the Source and Asset classes before posting parameters.

from pyddb import DDB, env, Project, Parameter, ParameterType, Source, SourceType, Asset, AssetType

DDB(env.sandbox)

my_source = Source.create(
    title="My source title",
    reference="My source reference",
    source_type=SourceType.get("Derived Value"),
)

my_site = Asset.create(
    name="My site name",
    asset_type=AssetType.get("Site"),
)

my_parameter = Parameter.create(
    parameter_type=ParameterType.get("Height"),
    value=100,
    unit="m",
    source=my_source,
    parent=my_site,
)

Parameters can be posted to DDB by passing a list of parameters to the post_parameters method on a Project.

my_project = Project.put("00000000")

my_project.post_parameters([my_parameter])

A Parameter can be created with just a ParameterType, which when posted, will create a blank parameter without any values. When posted with a value, the library will check if the value, unit, or source has changed and update the parameter if it has.

We can also post comments alongside our parameters with values.

parameter_with_comment = Parameter.create(
    parameter_type=ParameterType.get("Location"),
    value="Glasgow, Scotland",
    source=my_source,
    comment="Includes the city of Glasgow and the surrounding area.",
)

# Taxonomy Classes

These classes represent the different ways we have classified our data.

As an example, we have thousands of Parameter data classes in our projects, and these are all classified by a few ParameterType taxonomy class. This could be hundreds of Parameter objects with the ParameterType of "Area".

These classes are used to provide context to the data, give it consistent structure, and make it easier to work with.

All of these classes have 3 common methods that you will use to interact with them:

get_all - This will return all of the taxonomy objects of that type in DDB.

search - This will return a list of taxonomy objects that match the search term.

get - This will return a single taxonomy object by name or uuid.

# Source Type

DDB has a taxonomy for source types. Examples of these include 'Assumption', 'Derived Value', and 'Industry Guidance'.

from pyddb import DDB, env, SourceType

DDB(env.sandbox)

all_source_types = SourceType.get_all()
my_source_type = SourceType.get("Derived Value")

my_source_type.id
my_source_type.name

# Asset Type

DDB has a taxonomy for asset types. Examples of these include 'Site', 'Building', 'Space', 'Bridge', 'Tunnel', and 'Road'.

from pyddb import DDB, env, AssetType

DDB(env.sandbox)

all_asset_types = AssetType.get_all()
my_asset_type = AssetType.get("Site")

my_asset_type.id
my_asset_type.name

Some asset types have subtypes. You can check their asset_sub_type boolean property to see if they have subtypes.

if my_asset_type.asset_sub_type:
    sub_types = my_asset_type.get_sub_types()

# Asset Subtype

The DDB taxonomy supports further classification of asset types into sub types. When an asset type has sub types, all instances of that type must be classified as one of the sub types.

As an example, the 'building system' AssetType has several sub types, including 'Heating', 'Cooling', and 'Lighting'. All assets of this type need to be one of those sub types.

from pyddb import DDB, env, AssetSubType

DDB(env.sandbox)

all_asset_sub_types = AssetSubType.get_all()
my_asset_sub_type = AssetSubType.get("Heating")

my_asset_sub_type.id
my_asset_sub_type.name
my_asset_sub_type.asset_type_id

# Parameter Type

DDB has a taxonomy for parameter types. Examples of these include 'Summer indoor dry bulb temperature', 'Modulus of elasticity', 'Date of last inspection'.

These are used to classify our Parameters and apply some rules to them, such as the units they can have or the data types they can be.

from pyddb import DDB, env, ParameterType

DDB(env.sandbox)

all_parameter_types = ParameterType.get_all()
my_parameter_type = ParameterType.get("Modulus of elasticity")

my_parameter_type.id
my_parameter_type.name
my_parameter_type.data_type
my_parameter_type.unit_type

# Item Type

An ItemType is a combination of an AssetType, a ParameterType, and potentially an AssetSubType.

These are created by our team and make different ParameterTypes available on different Assets based on their AssetType and AssetSubType. We can use the ItemType class to see which ParameterTypes are available for different AssetTypes.

from pyddb import DDB, env, ItemType

DDB(env.sandbox)

all_item_types = ItemType.get_all()

site_item_types = ItemType.get_by_asset_type(AssetType.get("Site"))

ItemTypes allow us to apply even more specific rules to our data, such as a list of available Options. For example, there could be an ItemType that represents the ParameterType 'Field type' on the AssetType 'Site'. This ItemType could have a list of Options that represent the different types of fields that can be selected, such as 'Brownfield' or 'Greenfield'. This would mean that in DDB, all values of 'Field type' on 'Site' assets would have to be one of those two options.

for item_type in site_item_types:
    if item_type.has_options:
        options = item_type.get_options()

# Option

An Option is a value that can be selected for a Parameter of a certain ItemType. These are preset lists of values that are allowed for different ItemTypes.

You can access their values through the 'value' property:

for option in options:
    print(option.value)

# Unit Type

DDB has a taxonomy for unit types. Examples of these include 'Temperature', 'Length', 'Area', and 'Volume'.

These allow us to group up units that are similar, such as 'Celsius' and 'Fahrenheit' both being in the 'Temperature' unit type. ParameterTypes have a unit_type property that is used to restrict the units that can be used for that ParameterType.

from pyddb import DDB, env, UnitType

DDB(env.sandbox)

all_unit_types = UnitType.get_all()
my_unit_type = UnitType.get("Temperature")

my_unit_type.id
my_unit_type.name

You can also retrieve all units for a given UnitType.

my_unit_type = UnitType.get('Length')
length_units = my_unit_unit.get_units()

# Unit System

DDB has a taxonomy for unit systems. Examples of these include 'SI', 'Imperial', and 'General'.

These allow us to further classify our units.

from pyddb import DDB, env, UnitSystem

DDB(env.sandbox)

all_unit_systems = UnitSystem.get_all()
my_unit_system = UnitSystem.get("General")

my_unit_system.id
my_unit_system.name

You can also retrieve all units for a given UnitSystem.


my_unit_system = UnitSystem.get('General')
general_units = my_unit_unit.get_units()

# Unit

DDB has a taxonomy for units. Examples of these include 'm', 'kg', and 'ft'.

from pyddb import DDB, env, Unit
DDB(env.sandbox)

all_units = Unit.get_all()
my_unit = Unit.get("m")

my_unit.id
my_unit.symbol

# Role Type

Role types define different types of access to the DDB project. A single user can have multiple roles.

The different role types are:

Admin - For modifying user permissions

Reader - For reading project data

Editor - For editing project data

Checker - For updating QA status

Approver - For approving data

RoleType is an Enum object.

from pyddb import DDB, env, Project, RoleType

DDB(env.sandbox)

project = Project.put('00000000')

project.put_user_roles([
    (RoleType.Admin, '[email protected]'),
    (RoleType.Editor, '[email protected]'),
])

# Tag

Tags are used to organise our data and can be useful for filtering and searching. Examples of these include 'Google', 'Scotland', and 'Mechanical'.

from pyddb import DDB, env, Tag, Project, Asset, Parameter

DDB(env.sandbox)

all_tags = Tag.get_all()
my_tag = Tag.get("Google")

project = Project.put('00000000')
some_asset = project.get_assets()[0]
some_parameter = some_asset.get_parameters()[0]

project.post_tags([my_tag])
some_asset.post_tags([my_tag])
some_parameter.post_tags([my_tag])

# Tag Type

Tags are grouped together by their TagTypes. Examples of these include 'Client', 'Region', and 'Workstage'.

from pyddb import DDB, env, TagType

DDB(env.sandbox)

all_tag_types = TagType.get_all()
my_tag_type = TagType.get("Client")

client_tags = my_tag_type.get_tags()
Last Updated: 21/06/2024, 11:34:44

← Postman →