Nexus Bundle - Advanced
This tutorial will build on the "Your First Bundle" project, bundling our test artifacts into a Nexus instance, instead of a tar-ball.
The Nexus bundle offers several advantages over the tar. It can be used immediately for delivery and when a native repository type (e.g. OCI / Docker Container) is available, content will be in that location by default.
As with the previous example, we will not require a credentials file.
Prep
To begin, create a working directory somewhere convenient:
mkdir test-nexus-bundler
cd test-nexus-bundler
We will use this directory to store our input and write Hoppr output files.
Prerequisites
The primary prerequisite for creating a Nexus bundle is to have access to a Nexus repository into which the bundle will be built. There are many ways to bring up a Nexus instance, or an existing instance can be used.
For this instance, you will need to have the following information:
- Userid and password for a user with sufficient privileges to create repositories on the Nexus instance.
- URL for the Nexus UI/API
- The port on which the Nexus instance will listen for Docker requests (defaults to 5000)
- URL to be used for a single Docker repository in the Nexus instance (often the same host as the UI/API URL, but with the Nexus port, above)
Install the plugin
The Nexus Bundler plugin is not a core capability of Hoppr, but a separate plugin that must be installed along with Hoppr.
As with Hoppr, the plugin can be installed simply using pip
(or pip3
):
pip install hoppr-nexus-bundler
or
pip3 install hoppr-nexus-bundler
SBOM
We will re-use the SBOM from our previous example:
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"version": 1,
"components": [
{
"type": "file",
"name": ".bashrc",
"purl": "pkg:generic/~/.bashrc"
},
{
"type": "library",
"name": "HopprHippo-01.png",
"purl": "pkg:generic/img/HopprHippo-01.png"
},
{
"type": "library",
"name": "hello-world",
"version": "latest",
"purl": "pkg:docker/library/hello-world@latest"
},
{
"type": "library",
"name": "hoppr",
"purl": "pkg:git/hoppr/hoppr.git"
}
]
}
Copy the above file and store it in your working directory as my-nexus-sbom.json
.
Manifest
We will also re-use the manifest from the previous example:
---
schemaVersion: v1
kind: manifest
metadata:
name: "My First Manifest"
version: 0.1.0
description: "An introductory manifest"
sboms:
- local: my-nexus-sbom.json
includes: []
repositories:
generic:
- url: "file:"
- url: https://hoppr.dev/
docker:
- url: https://docker.io
git:
- url: https://gitlab.com
Copy the above file and store it in your working directory as my-nexus-manifest.yml
.
Transfer
The transfer file is where we are making a change. We are replacing the plugin in the Bundle stage with the Nexus Bundler:
---
schemaVersion: v1
kind: Transfer
stages:
Collect:
plugins:
- name: "hoppr.core_plugins.collect_docker_plugin"
- name: "hoppr.core_plugins.collect_git_plugin"
- name: "hoppr.core_plugins.collect_raw_plugin"
Bundle:
plugins:
- name: "nexus_bundler.bundler"
By default, the Nexus Bundler is configured based on environment variables. So to run properly, the following environment variables must be set:
NEXUS_IP
: with the IP Address (or host name) of the Nexus instanceNEXUS_PW
: with the password of theadmin
user on the Nexus instance.
The parameters for the plugin default as follows:
url
(the base URL for the UI and API):http://${NEXUS_IP}:8081/
username
:admin
password_env
(An environment variable containing the corresponding password):NEXUS_PW
docker_url
:http://${NEXUS_IP}:5000/
docker_port
: 5000force_http
(Force docker commands to use http even on https addresses):False
If we choose not to accept those defaults, the correct values may be specified in the config
section for the plugin:
---
schemaVersion: v1
kind: Transfer
stages:
Collect:
plugins:
- name: "hoppr.core_plugins.collect_docker_plugin"
- name: "hoppr.core_plugins.collect_git_plugin"
- name: "hoppr.core_plugins.collect_raw_plugin"
Bundle:
plugins:
- name: "nexus_bundler.bundler"
config:
url: "https://my-nexus.mydomain.com/"
docker_url: "https://my-nexus.mydomain.com:5500/"
docker_port: 5500
Fields not overridden in the config section (username
, password_env
, and forse_http
) will default as outlined above.
Please note: The password is not entered into the the transfer config file. The password is specified by an environment variable (password_env
), which is populated with the appropriate password. This prevents the possibility of the password being compromised by being checked into a version control repository.
Copy one of the above files (appropriately populated with information about your nexus instance) and store it in your working directory as my-nexus-transfer.json
.
Nexus
A complete tutorial for configuring and creating a Nexus intance is beyond the scope of this document. That said here are a few tips regarding one way to get started.
- There are docker images for Nexus. For our purposes, we will want a port (5000) open for a docker repository, as well as the UI port (8081), so our command would be:
docker run --detach --name mynexus -p 8081:8081 -p 5000:5000 sonatype/nexus3:3.30.1
The
NEXUS_IP
may be the same as that of the system on which you ran the command, or it may be different. Thedocker inspect mynexus
command will show you if a different IP address is being used.By default, the Nexus admin password is a long complex string. It may be retrieved by the command
docker exec mynexus cat /nexus-data/admin.password
.The default password may be changed by using the rest API, for example:
curl -X PUT "http://$NEXUS_IP:8081/service/rest/v1/security/users/admin/change-password" -H "accept: application/json" -H "Content-Type: text/plain" -d "new_pw_123" -u admin:$DEFAULT_PW
. Assuming that$DEFAULT_PW
contains the long, complex, password retrieved above, the new admin password will benew_pw_123
.If all goes well, you should be able to view the Nexus user interface on port 8081 of the Nexus IP address in any browser. Failure to do so may be due to firewalls or proxies. Consult local experts.
Note that a new Nexus instance is populated, by default, with seven empty repositories (four maven, three nuget).
Run Hoppr
We're now ready to run Hoppr and create our first bundle. Before we proceed, let's check that:
- Hoppr has been installed
- Nexus is up and running. We do this with an API call that tests whether our admin user has write access. In this example, the Nexus instance is running on
localhost
, and the admin password isnew_pw_123
. An HTTP response of 200 indicates success. git
(needed for thecollect_git_plugin
) has been installedskopeo
(needed for thecollect_docker_plugin
) has been installed- The input files described above have been created
$ cd test-nexus-bundler
$ hopctl version
Hoppr Framework Version: 1.6.2
Python Version: 3.10.5
$ curl http://admin:new_pw_123@localhost:8081/service/rest/v1/status/writable -v
* Uses proxy env variable no_proxy == '.localdomain,localhost,127.0.0.1'
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8081 (#0)
* Server auth using Basic with user 'admin'
> GET /service/rest/v1/status/writable HTTP/1.1
> Host: localhost:8081
> Authorization: Basic YWRtaW46YWRtaW4xMjM=
> User-Agent: curl/7.61.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 29 Nov 2022 21:37:58 GMT
< Server: Nexus/3.30.1-01 (OSS)
< X-Content-Type-Options: nosniff
< Content-Length: 0
<
* Connection #0 to host localhost left intact
$ git --version
git version 2.31.1
$ skopeo --version
skopeo version 1.8.0
$ ls -l
total 5
-rwxrwxrwx. 1 vagrant vagrant 331 Nov 17 13:20 my-nexus-manifest.yml
-rwxrwxrwx. 1 vagrant vagrant 699 Nov 17 13:17 my-nexus-sbom.json
-rwxrwxrwx. 1 vagrant vagrant 296 Nov 17 13:18 my-nexus-transfer.yml
Now we are again ready to execute the hopctl bundle
command.
hopctl bundle my-nexus-manifest.yml --transfer my-nexus-transfer.yml
If all goes well, you should see output like this:
Beginning Hoppr Process execution, max_processes=2
Collecting Hoppr Metadata
========== Beginning Stage Collect ==================================================
Beginning method process_component
CollectDockerPlugin SUCCESS for pkg:docker/library/hello-world@latest
CollectGitPlugin SUCCESS for pkg:git/hoppr/hoppr.git
CollectRawPlugin SUCCESS for pkg:generic/~/.bashrc
CollectRawPlugin SUCCESS for pkg:generic/img/HopprHippo-01.png
Beginning method post_stage_process
CollectDockerPlugin SUCCESS
CollectGitPlugin SUCCESS
CollectRawPlugin SUCCESS
========== Beginning Stage Bundle ==================================================
Beginning method pre_stage_process
NexusBundlePlugin SUCCESS
========== Results Summary ==========
Stage: Collect
process_component
4 jobs succeeded, 0 failed
post_stage_process
3 jobs succeeded, 0 failed
Stage: Bundle
pre_stage_process
1 jobs succeeded, 0 failed
GRAND TOTAL: 8 jobs succeeded, 0 failed
Unlike the previous example, the only file that should have been added to your working directory is the log file. There is no equivalent of the bundle.tar.gz
file. What we will see is that several new repositories have been added to the Nexus instance:
- hoppr_metadata
- transfer-docker
- transfer-generic
- transfer-git
These repositories contain all the requested artifacts, along with the the standard Hoppr metadata.