Feature: add Source Digital ocean

README.md

@@ -7,6 +7,7 @@ Available source types:
 * AWS EC2, EBS, and network interfaces, with opt-in S3, Lambda, and EKS inventory (see [AWS source](docs/source_aws.md))
 * oVirt / Red Hat Virtualization (see [oVirt source](docs/source_ovirt.md))
 * [bb-ricardo/check_redfish](https://github.com/bb-Ricardo/check_redfish) inventory files
+* [DigitalOcean](docs/source_digitalocean.md)
 
 **IMPORTANT: READ INSTRUCTIONS CAREFULLY BEFORE RUNNING THIS PROGRAM**
 

docs/source_digitalocean.md

@@ -0,0 +1,193 @@
+# DigitalOcean source
+
+The `digitalocean` source reads Droplets, attached block volumes, and managed
+database clusters from the DigitalOcean API and reconciles them into NetBox.
+API access is read-only; inventory changes are written to NetBox by the normal
+netbox-sync process.
+
+## Installation and access
+
+The source uses the `requests` dependency already installed by netbox-sync; no
+DigitalOcean SDK is required. Create a DigitalOcean API token with read access
+to regions, Droplets, and volumes. If managed database ingestion is enabled, the token
+must also be able to read databases.
+
+Keep the API token out of version control and provide it through a protected
+configuration file or the application's standard source environment variables.
+
+## Configuration
+
+Add a source section to the existing settings file:
+
+```ini
+[source/example-digitalocean]
+type = digitalocean
+api_token = REPLACE_WITH_READ_ONLY_API_TOKEN
+region_mapping = fra1=Germany, nyc1=United States
+sync_volumes = True
+sync_databases = True
+; Optional isolation for overlapping private addresses across teams:
+;private_ip_vrf = DigitalOcean Team-A
+```
+
+`api_token` is required. `sync_volumes` and `sync_databases` default to `True`.
+Set either to `False` when the token should not query that API resource.
+Accounts without managed databases can leave `sync_databases` enabled:
+both `{"databases": []}` and `{"databases": null}` are treated as empty
+database collections. Missing collection fields and other malformed values
+remain errors rather than being treated as empty inventory.
+Generate the complete configuration with:
+
+```sh
+python netbox-sync.py -g
+```
+
+`region_mapping` is optional. It accepts comma-separated, exact
+`DigitalOcean region slug=NetBox Region name` pairs. Quote destination names
+containing commas, for example `nyc1="New York, USA"`. Duplicate source slugs,
+empty entries, and Region names longer than 100 characters are rejected.
+
+On NetBox 4.2+, the source reuses a Region with the destination name, then
+tries its generated slug, and creates it if neither exists. Explicit mappings
+take precedence over Regions matching the DigitalOcean region. Without a
+mapping, the DigitalOcean region name and slug are used (for example
+`Frankfurt 1` and `fra1`). Region slugs (including those returned for databases)
+are resolved through `GET /v2/regions` once per run, so cluster names do not
+depend on whether Droplets exist in the region. If a slug is absent from that
+response, it is used as the name. Existing Regions are not renamed.
+Regions previously created by this source retain their source ownership tags;
+reused, unmanaged Regions are not claimed by the source.
+
+Multiple DigitalOcean regions can map to the same NetBox Region; their
+virtualization clusters remain separate and retain their DigitalOcean names.
+Existing clusters in the source's cluster group are updated in place when
+their scope changes, including previously unscoped clusters. Ambiguous
+duplicate clusters in that group cause an explicit error instead of choosing
+one arbitrarily.
+
+The NetBox token needs permission to add Regions when they do not already
+exist. On NetBox versions before 4.2, Region creation and mapping are not
+applied; clusters remain unscoped.
+
+## NetBox mapping
+
+### Private address isolation
+
+`private_ip_vrf` optionally names a NetBox VRF for this source's private
+network addresses (IPv4 and IPv6). The source reuses a VRF with that exact
+name or creates it if missing; names must be non-empty and at most 100
+characters. The NetBox token needs permission to create VRFs if missing.
+An ambiguous VRF name stops the run rather than selecting one arbitrarily.
+Referenced VRFs inherit source ownership through the normal inventory logic
+and are not pruned.
+
+Use distinct names for isolated teams, for example:
+
+```ini
+[source/team-a]
+type = digitalocean
+api_token = REPLACE_WITH_TEAM_A_READ_ONLY_TOKEN
+private_ip_vrf = DigitalOcean Team-A
+
+[source/team-b]
+type = digitalocean
+api_token = REPLACE_WITH_TEAM_B_READ_ONLY_TOKEN
+private_ip_vrf = DigitalOcean Team-B
+```
+
+Both teams can then have `10.108.0.6` in the same region without sharing an
+IP record. IP matching and per-run conflict checks use the address and VRF;
+prefix-length changes update the same record. Public network addresses remain
+in the global table, regardless of their numerical address range.
+When unset, private addresses also remain global, preserving existing behavior.
+Each run logs the source name and whether private addresses use the global
+table or the configured VRF, making it possible to verify the active settings.
+
+Enabling or changing this option does **not** move existing IPs between VRFs.
+It creates or reuses records in the configured VRF and selects discovered
+primary IPs from the appropriate table, preferring public addresses.
+Existing global records and assignments remain unchanged; review and clean up
+obsolete records manually after confirming ownership in a dry run.
+Previously source-tagged global IPs may become orphaned and eligible for
+normal pruning once their replacements are in a VRF. Disable pruning during
+the transition and review the dry-run changes before enabling writes.
+Turning the option off likewise does not move VRF records back to global.
+This is per-source isolation, not per-VPC isolation: if a team has multiple
+VPCs that reuse addresses, per-VPC isolation would be needed instead of this
+setting.
+
+| DigitalOcean | NetBox |
+| --- | --- |
+| Region | Virtualization cluster scoped to a reused or automatically created NetBox Region on NetBox 4.2+ |
+| Droplet | Virtual machine |
+| Droplet size | VM vCPUs and memory |
+| Droplet image distribution/name | VM platform |
+| Droplet public/private IPv4 and IPv6 | VM interfaces and IP addresses |
+| Droplet root disk | VM disk capacity on older NetBox; virtual disk on NetBox 3.7+ |
+| Attached block volume | Additional virtual disk on NetBox 3.7+ |
+| DigitalOcean resource tags | NetBox tags |
+| Managed database cluster | VM inventory record with `db-` name prefix and metadata custom fields |
+
+DigitalOcean reports Droplet and volume disk sizes in gigabytes. NetBox 4.1+
+receives megabytes using 1024 MB per reported GB; earlier NetBox versions
+receive gigabytes. Database storage, reported in MiB, is converted to whole
+gigabytes on earlier NetBox versions. VM serial numbers are only sent on
+NetBox 4.1+. Database
+clusters are represented as one VM-like record per cluster because this
+project has no native managed-database object type. Database engine, version,
+size, node count, endpoint host, and port are stored in `digitalocean_*` custom
+fields. Connection credentials and connection URIs are never copied. Database
+endpoint hostnames are not resolved into IP addresses.
+
+DigitalOcean private and public network addresses are assigned to interfaces
+named after their network type. The source does not create NetBox prefixes or
+sites.
+
+VMs are matched by `digitalocean_resource_id` before falling back to name and
+cluster for records without a resource ID. Duplicate VM names within a region
+(case-insensitive, including database `db-` prefixes) have ` (<resource id>)`
+appended, with the original name shortened as needed to fit NetBox's limit.
+Resource IDs preserve VM identity across renames and region changes.
+If multiple VMs carry the same resource ID, the source uses the sole
+non-orphaned match when all other matches have the `NetBox-synced: Orphaned`
+tag. It logs the ignored orphaned records and leaves them unchanged for normal
+orphan pruning. A single orphaned match can still be rediscovered.
+If multiple non-orphaned matches, or only multiple orphaned matches, remain,
+the source exits with the matching NetBox IDs, names, and clusters rather than
+choosing arbitrarily.
+Droplets with the same name in different clusters remain distinct. Interfaces
+and virtual disks are matched to their resolved parent VM, not just its name.
+If an earlier run moved child objects between same-named VMs, inspect the next
+dry run for interface, disk, and primary-IP changes before enabling writes.
+
+IP assignments can be repaired automatically when the existing IP belongs to
+the current source, either through this run's ownership or its persisted
+`Source: <source name>` tag. Conflicting or uncertain ownership is logged and
+the assignment is skipped without stopping the run or changing the existing
+assignment. Other addresses, droplets, disks, and databases continue syncing.
+Skipped IPs are not selected as the discovered VM's primary IP. An existing IP
+tagged for this source is retained for the run, preventing the skipped assignment
+from causing orphaning/pruning; ownership tags are preserved.
+Two interfaces claiming the same IP in one source run also produce an error log:
+the first assignment is kept and subsequent conflicting assignments are skipped.
+Source section names must remain stable for
+persisted ownership tags to match. If an earlier run already removed an IP's
+source tag or marked it orphaned, verify its owner and restore the correct tag
+or repair its assignment manually before retrying.
+Ownership-conflict errors include the existing NetBox IP ID, VRF, assigned
+interface, IP source tags, current-run source, and assigned VM tags to help
+identify stale assignments. VM tags alone do not authorize moving an IP.
+
+## Validation
+
+Start with a dry run:
+
+```sh
+python netbox-sync.py -n -c settings.ini
+```
+
+Confirm the discovered Droplet, volume, and database cluster counts and inspect
+the resulting NetBox object changes before enabling writes or pruning.
+API failures stop discovery before resource inventory is modified, close the
+API session, and report the HTTP status and DigitalOcean error message when
+available.

module/config/file_output.py

@@ -19,6 +19,7 @@
 from module.sources.hetzner.config import HetznerConfig
 from module.sources.ovirt.config import OVirtConfig
 from module.sources.aws.config import AWSConfig
+from module.sources.digitalocean.config import DigitalOceanConfig
 from module.common.logging import get_logger
 from module.config import default_config_file_path, source_config_section_name
 from module.config.files import ConfigFile, ConfigFileINI, ConfigFileYAML
@@ -39,6 +40,7 @@ class ConfigFileOutput(DescriptionFormatterMixin):
         CheckRedfishConfig,
         HetznerConfig,
         AWSConfig,
+        DigitalOceanConfig,
         OVirtConfig
     ]
 

module/netbox/inventory.py

@@ -128,6 +128,14 @@ def get_by_data(self, object_type, data=None):
 
         # try to find by primary/secondary key
         elif data.get(object_type.primary_key) is not None:
+            if object_type in (NBVMInterface, NBVirtualDisk) and isinstance(data.get("virtual_machine"), NBVM):
+                # VM names are only unique within a cluster; use the resolved parent object.
+                for this_object in self.get_all_items(object_type):
+                    if this_object.data.get("virtual_machine") is data["virtual_machine"] and \
+                            str(this_object.data.get(object_type.primary_key)).lower() == \
+                            str(data[object_type.primary_key]).lower():
+                        return this_object
+                return None
             object_name_to_find = None
             for this_object in self.get_all_items(object_type):
 

module/netbox/object_classes.py

@@ -1511,7 +1511,6 @@ class NBRegion(NetBoxObject):
     object_type = "dcim.region"
     primary_key = "name"
     prune = False
-    read_only = True
 
     def __init__(self, *args, **kwargs):
         self.data_model = {

module/sources/__init__.py

@@ -12,6 +12,7 @@
 from module.sources.hetzner.connection import HetznerHandler
 from module.sources.ovirt.connection import OVirtHandler
 from module.sources.aws.connection import AWSHandler
+from module.sources.digitalocean.connection import DigitalOceanHandler
 from module.sources.check_redfish.import_inventory import CheckRedfish
 
 from module.common.logging import get_logger
@@ -21,7 +22,7 @@
 from module.config import source_config_section_name
 
 # list of valid sources
-valid_sources = [VMWareHandler, CheckRedfish, HetznerHandler, AWSHandler, OVirtHandler]
+valid_sources = [VMWareHandler, CheckRedfish, HetznerHandler, AWSHandler, DigitalOceanHandler, OVirtHandler]
 
 
 def validate_source(source_class_object=None, state="pre"):

module/sources/digitalocean/__init__.py

@@ -0,0 +1,8 @@
+# -*- coding: utf-8 -*-
+#  Copyright (c) 2020 - 2026 netbox-sync team. All rights reserved.
+#
+#  This work is licensed under the terms of the MIT license.
+#  For a copy, see file LICENSE.txt included in this
+#  repository or visit: <https://opensource.org/licenses/MIT>.
+
+from module.sources.digitalocean.connection import DigitalOceanHandler

module/sources/digitalocean/client.py

@@ -0,0 +1,86 @@
+# -*- coding: utf-8 -*-
+#  Copyright (c) 2020 - 2026 netbox-sync team. All rights reserved.
+#
+#  This work is licensed under the terms of the MIT license.
+#  For a copy, see file LICENSE.txt included in this
+#  repository or visit: <https://opensource.org/licenses/MIT>.
+
+from urllib.parse import urljoin, urlparse
+
+import requests
+
+
+class DigitalOceanClient:
+
+    api_url = "https://api.digitalocean.com/v2/"
+    timeout = 30
+
+    def __init__(self, token, session=None):
+        self.session = session or requests.Session()
+        self.headers = {
+            "Accept": "application/json",
+            "Authorization": f"Bearer {token}",
+            "User-Agent": "netbox-sync",
+        }
+
+    def get_collection(self, path, collection_name):
+        url = urljoin(self.api_url, path.lstrip("/"))
+        params = {"per_page": 200}
+        objects = []
+
+        while url is not None:
+            response = self.session.get(
+                url,
+                headers=self.headers,
+                params=params,
+                timeout=self.timeout,
+            )
+            try:
+                response.raise_for_status()
+            except requests.HTTPError as exc:
+                detail = str(exc)
+                try:
+                    error_payload = response.json()
+                except requests.exceptions.JSONDecodeError:
+                    detail = f"{detail} (error response is not JSON)"
+                else:
+                    if isinstance(error_payload, dict):
+                        detail = error_payload.get("message") or detail
+                raise requests.HTTPError(
+                    f"DigitalOcean API '{path}' returned HTTP {response.status_code}: {detail}",
+                    response=response,
+                ) from exc
+            payload = response.json()
+            if not isinstance(payload, dict):
+                raise ValueError(f"DigitalOcean response for '{path}' is not a JSON object.")
+            collection = payload.get(collection_name)
+            # DigitalOcean can return {"databases": null} for accounts without databases.
+            if collection_name == "databases" and "databases" in payload and collection is None:
+                collection = []
+            if not isinstance(collection, list):
+                raise ValueError(f"DigitalOcean response for '{path}' has no '{collection_name}' list.")
+            objects.extend(collection)
+
+            links = payload.get("links") or {}
+            if not isinstance(links, dict):
+                raise ValueError(f"DigitalOcean response for '{path}' has invalid pagination links.")
+            pages = links.get("pages") or {}
+            if not isinstance(pages, dict):
+                raise ValueError(f"DigitalOcean response for '{path}' has invalid pagination pages.")
+            next_url = pages.get("next")
+            if next_url:
+                if not isinstance(next_url, str):
+                    raise ValueError(f"DigitalOcean response for '{path}' has a non-string pagination link.")
+                url = urljoin(url, next_url)
+                parsed_url = urlparse(url)
+                if parsed_url.scheme != "https" or parsed_url.hostname != "api.digitalocean.com" \
+                        or parsed_url.port not in (None, 443) or parsed_url.username is not None:
+                    raise ValueError("DigitalOcean API returned a pagination link outside its HTTPS API host.")
+            else:
+                url = None
+            params = None
+
+        return objects
+
+    def close(self):
+        self.session.close()