External vocabularies advanced

CESNET Invenio does not natively support controlled vocabularies that live outside the repository (e.g. Catalogue of Life , which is very large and updated daily). Loading such vocabularies as internal fixtures is impractical, as they would need frequent re-imports.

Depending on your use case, integrate an external vocabulary in one of the following ways:

Use case 1: Suggestions in the deposit form UI only

The vocabulary is used only to help users fill in the form; the resulting metadata values are stored as plain data inside the record (no linkage, no validation against the vocabulary afterwards).

Implementation:

1. Implement the lookup in the React form field component of your deposit form: call the external vocabulary’s REST API, parse the results and offer them as autocomplete suggestions. The deposit form components page lists the vocabulary-related building blocks you can start from.
2. Allow the external hostname in the Content Security Policy configuration of your instance — pass a config.CSP instance with connect_src to configure_ui(), as documented in the security headers section of the configuration reference — otherwise browser requests to the external API will be blocked:

# illustrative sketch
config.configure_generic_parameters()
 
config.configure_ui(
    # ... keep the existing arguments ...
    csp=config.CSP(
        # let the deposit form call the external vocabulary API from the browser
        connect_src=["https://api.example.org"],
    ),
)

Use case 2: Validation against a frequently changing vocabulary

The metadata must be validated against the vocabulary, and entries change so often that a local copy would quickly go stale.

Implementation options:

• Marshmallow validator — call the external vocabulary’s API from a Marshmallow  field validator, defined next to your model fields (or passed via field_args for custom fields). The check runs on every form save, and you can use it to enrich the record metadata (e.g. store the canonical label or IRI along with the value).

# illustrative sketch
import requests
from marshmallow import ValidationError, validates
 
 
class MySchema:
    # ... your model fields ...
 
    @validates("species")
    def validate_species(self, value):
        response = requests.get(
            "https://api.example.org/species",
            params={"name": value},
            timeout=5,
        )
        if response.status_code == 404:
            raise ValidationError(f"'{value}' is not a known species")

• Invenio checks — implement the validation as a check that runs before publication. This tolerates slower external APIs, but results are asynchronous — the user sees them only after the check finishes.

Use case 3: Validation against a slowly changing vocabulary

Entries change infrequently (weeks/months), but the full dataset is still too large for a plain fixture import.

Implementation:

1. Load the relevant subset as a local vocabulary (see Initial data).
2. In your Marshmallow validator, look the value up in the local vocabulary first (to avoid overloading the external service).
3. On a cache miss, call the external API and store the confirmed entry into the local vocabulary (caching):

# illustrative sketch
from invenio_access.permissions import system_identity
from invenio_vocabularies.proxies import current_vocabulary_service
from marshmallow import ValidationError, validates
 
 
def resolve_term(vocabulary_type, value):
    """Return the local entry, or resolve it externally and cache it."""
    # 1. local lookup first to avoid overloading the external service
    try:
        return current_vocabulary_service.read(
            system_identity, value, type=vocabulary_type
        )
    except LookupError:
        pass  # not cached yet
 
    # 2. on a cache miss, call the external API with an explicit timeout
    try:
        data = fetch_from_external_api(value, timeout=5)
    except requests.RequestException as e:
        # service down/unreachable — distinguish from "term unknown"
        raise LookupError(
            f"External vocabulary service unavailable: {e}"
        )
        
    if not data:
        raise LookupError(f"'{value}' not found in external vocabulary")
 
    # 3. store the confirmed entry for next time and return it
    return current_vocabulary_service.create(
        system_identity, {"id": value, **data}, type=vocabulary_type
    )
 
 
class MySchema:
    # ... your model fields ...
 
    @validates("species")
    def validate_species(self, value):
        # runs on every form save; unknown values fail validation
        try:
            resolve_term("species", value)
        except LookupError as e:
            raise ValidationError(str(e))

See also

• Initial data for vocabularies — loading local vocabularies from fixtures
• Deposit form components — vocabulary form fields and their props
• Invenio checks — asynchronous validation before publication
• Invenio RDM documentation on vocabularies