Skip to main content
Version: 1.8.x

Hoppr Processing 101

Hoppr has a simple architecture. Prepare a few core input files for the Hippo, run hopctl, and receive outputs. Details below:

Four Input Files

  1. CycloneDX SBOMs (JSON)
  2. Manifest (YAML)
    • A composite of all CycloneDX SBOMs to be processed; may include other Manifests
    • Identifies repositories from which software components are to be retrieved
  3. Transfer (YAML)
    • A list of stages for Hoppr to execute; each stage is a list of Hoppr processing plugins.
  4. Credentials (YAML)
    • (Optional) - specifies authentication credentials for Hoppr network activity

Hoppr Processing

note

See more architectural processing details on our execution sequence diagrams.

Outputs

CycloneDX SBOMs

CycloneDX standard is an SBOM format we're using to define Software Bill of Materials. Components in the SBOMs define "what" gets transferred. SBOM files are referenced by a Manifest YAML file to allow multiple deliverables to be combined into a single product.

Manifests

A Hoppr specific YAML file that specifies product dependencies required for product deployments. Could be provided by product teams for use by end customers.

Hoppr currently supports the following Package URLs (PURLs) to find components. In 1.0.0 the following package types are supported:

PURL TypeDescription
dockercontainer images
gitgit repositories and helm charts
rpmredhat RPMs (yum/dnf)
pypipython packages
mavenmaven packages
genericstand-alone files (e.g. binary) not handled by a package manager
helmhelm charts
---
schemaVersion: v1
kind: manifest

metadata:
name: string
version: string
description: string

sboms:
- url: string
- local: string

includes:
- url: string
- local: string

repositories:
<purl type>:
- url: string
description: string

Credentials

A Hoppr specific YAML file that defines how to find credentials to access Manifest, SBOMs, and the components that will be collected.

---
schemaVersion: v1
kind: authentication

metadata:
name: string
version: string
description: string

credential_required_services:
- url: string

user: string
.. or ...
user_env: string

pass_env: string

Transfers

A Hoppr specific YAML file that defines how to run a series of stages and plugins used to collect, process, and bundle components.

---
schemaVersion: v1
kind: transfer

stages:
Collect:
plugins:
- name: "hoppr.core_plugins.collect_docker_plugin"
- name: "hoppr.core_plugins.collect_git_plugin"
- name: "hoppr.core_plugins.collect_helm_plugin"
- name: "hoppr.core_plugins.collect_maven_plugin"
- name: "hoppr.core_plugins.collect_pypi_plugin"
config:
pip_command: pip3
- name: "hoppr.core_plugins.collect_raw_plugin"
Bundle:
plugins:
- name: "hoppr.core_plugins.bundle_tar"
config:
tarfile_name: ~/tarfile.tar.gz

max_processes: 3

Input File Schemas

Example Hoppr Project

This is a simplified example to illustrate the input files and the relationship between them.

ExampleGraph

In this example, the root manifest references manifests for two other products.

Schemas

All input file schemas are in JSON as YAML can be converted to JSON and validated against the schema. Example schema validation using yq and jsonschema.

$ cat airgapped.yml | yq eval -P -o json > airgap-manifest.json
$ curl "https://gitlab.com/api/v4/projects/34748703/packages/generic/schemas/v1/hoppr-manifest-schema-v1.json" \
-o manifest-schema.json

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3205 100 3205 0 0 436 0 0:00:07 0:00:07 --:--:-- 740

$ jsonschema --instance airgap-manifest.json manifest-schema.json
$ echo $?
0

Example SBOMs & Manifests

  • Product A contains a CycloneDX SBOM on the "as-built" components that need to be transferred and their build dependencies.
  • Product B contains two CycloneDX SBOMs for two different components that need to be transferred and their build dependencies.
  • Both Products have Manifest files to specify what SBOMs are needed for the product to work. Each manifest also specifies a list of repositories to be searched for components specified in their SBOMs.
  • Lastly, the third party has a Manifest that has either local or URL includes of Product A and Product B's Manifests, but does not include any SBOMs directly.

We recommend using a tool like renovate to keep your source projects up-to-date, and include the generation of Manifest and SBOM files in your continuous delivery pipeline(s). In this way, any transfers that are made with Hoppr can be kept current as well.

Example Transfer

The Transfer file will specify the processing steps needed to collect, augment, filter, report, and bundle components.

Example Credentials

The Credentials file specifies any credentials needed to access the Manifest, SBOMs, or Components that are access controlled.