Customize an existing image

Customize the Grafana Docker Image

Let's say that you are going to use, in all your applications, a Grafana dashboard to show metrics to your customers.

Let's suppose that your data source is always an InfluxDB on-premise (it could be anything else, an AWS Timestream database, a remote MySQL database, ...).

You might want to create a version of the public Grafana image, customized for your needs, already bound to the InfluxDB data source, including your custom dashboards and charts.

This walkthrough will create an application template, including customized dashboards, data sources, database, and configuration.

Once you are satisfied with the initial customization, you will be able to deploy the newly created application template for each new customer.

This walkthrough will customize an existing image and we will publish it on Docker Hub (either public or private, depending on your needs).

Once published, it can be used in IOhubTM just like any existing image, with no further customization needed.

This example is oversimplified; in a production container, you might want to make it more configurable. We are keeping the example simple for demo purposes.

Use Case definition

We are creating a customized version of Grafana existing image.
We will include a provisioned ready-made dashboard.
We will include a provisioned data source, pointing to an InfluxDB database.

Environment setup

Let's create a development instance of an InfluxDB as a test database for our custom image.

All the commands listed below are valid for any of the environments defined in our setup walkthrough; Windows 10 with WSL2, Linux, macOS. If you want to use an alternative environment/shell, for example, Powershell, you need to adapt the commands below to your needs.

If you are using Windows 10, please open your WSL2 instance to run the commands.

Create a project folder

mkdir mydashboard && cd mydashboard

Create a folder to store a persistent db

mkdir influxdata

Start locally an InfluxDB container

docker run --name influxdb -v $PWD/influxdata:/var/lib/influxdb -d \
us-central1-docker.pkg.dev/ez-shared/iohub/iohub-influxdb-v1

You can run the above command without the -d option if you want to see all the logs.

In this case, you must run the other commands in another terminal window.

Create an empty database

We can download some sample data to feed into our database.

curl https://raw.githubusercontent.com/wiki/ez-vpn/iohub-documentation-assets/assets/data/NOAA_data.txt -o NOAA_data.txt

docker cp NOAA_data.txt influxdb:/tmp/NOAA_data.txt

docker exec -it influxdb influx -import -path=/tmp/NOAA_data.txt -precision=s -database=mydb

rm NOAA_data.txt

In your terminal window, you should see something like this:

$ curl https://s3.amazonaws.com/noaa.water-database/NOAA_data.txt -o NOAA_data.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4957k  100 4957k    0     0  3944k      0  0:00:01  0:00:01 --:--:-- 3941k
$ docker cp NOAA_data.txt influxdb:/tmp/NOAA_data.txt
$ docker exec -it influxdb influx -import -path=/tmp/NOAA_data.txt -precision=s -database=mydb
2020/10/31 15:20:46 Processed 1 commands
2020/10/31 15:20:46 Processed 76290 inserts
2020/10/31 15:20:46 Failed 0 inserts
$ rm NOAA_data.txt

You now have a running instance of an InfluxDB with some dummy data in the mydb database. We can stop and remove the InfluxDB running instance (your db will not be deleted from the influxdata folder).

docker stop influxdb && docker rm influxdb

Create a customized Grafana image

Let's create a new working folder for our custom image.

Create the provisioning structure for Grafana

mkdir -p provisioning/dashboards
mkdir -p provisioning/datasources
mkdir -p provisioning/notifiers
mkdir -p provisioning/plugins

Create a datasource.yml file in the provisioning/datasources folder, ready to be provisioned to Grafana with this content:

cat >provisioning/datasources/datasource.yml <<EOF
apiVersion: 1

datasources:
    - name: InfluxDB
      type: influxdb
      database: mydb
      url: http://myinfluxdb:8086
      isDefault: true
EOF

This datasource is pointing to a named InfluxDB instance myinfluxdb, on port 8086, to a DB named mydb.

Create a file Dockerfile with the following content

cat >Dockerfile <<EOF
FROM grafana/grafana

COPY provisioning/  /etc/grafana/provisioning
EOF

Create a .dockerignore file to avoid pushing to the Docker engine useless content, with the following content

cat >.dockerignore <<EOF
influxdata
EOF

Let's now build our custom image, tagged as mydashboard.

docker build -t mydashboard .

At the end of the process you should see in your terminal windows something like this:

$ docker build -t mydashboard .
Sending build context to Docker daemon  6.656kB
Step 1/2 : FROM grafana/grafana
latest: Pulling from grafana/grafana
188c0c94c7c5: Pull complete
94db9aec2a58: Pull complete
618563756337: Pull complete
f413b64712e1: Pull complete
18534279664f: Pull complete
4f4fb700ef54: Pull complete
3abba839cfd4: Pull complete
aac362f49dd4: Pull complete
Digest: sha256:9f43d0ac1fdecdd08a47bce4038fa5c9c67cc84fc025df78b87ae7b7b076aee9
Status: Downloaded newer image for grafana/grafana:latest
 ---> 900b03b57e41
Step 2/2 : COPY provisioning/  /etc/grafana/provisioning
 ---> c113a7bc0880
Successfully built c113a7bc0880
Successfully tagged mydashboard:latest

You now have a customized image, automatically pointing to a provisioned InfluxDB instance with a fixed name and port and db name.

We can now crank up a complete application, including InfluxDB and our custom Grafana image, ready to run (we will add a custom dashboard later).

Let’s create a docker-compose.yml file, including both services, to test our new container.

cat >docker-compose.yml <<EOF
version: "3"

services:
    myinfluxdb:
        image: us-central1-docker.pkg.dev/ez-shared/iohub/iohub-influxdb-v1
        volumes:
            - "./influxdata:/var/lib/influxdb"
        expose:
            - "8086"

    mydashboard:
        build: .
        ports:
            - 3000:3000
        depends_on:
            - myinfluxdb
EOF

And start the application.

docker-compose up

You can now browse http://127.0.0.1:3000 and access your Grafana instance.

Fill in the default credentials (admin/admin) to log in and click skip in the following screen.

You can now create a new dashboard, showing the sample data because the default InfluxDB datasource has already been configured.

  • Click on the "Create your first dashboard" panel in the home page.
  • Click on "Add new Panel"

You now have an empty panel that you have to configure to show the NOAA data in your sample database.

  • First of all, let's zoom in on the time range where we have data. On the top right, click the clock and select a time period between August 15, 2019 and September 15 2019.
  • Then click the "select measurement" button, to choose the data you want to show: select h2o_feet.
  • Click the "field(value)" button to select the value to use inside the h2o_feet measurement and choose water_level.
  • In the GROUP BY row, click the fill(null) button and select none to have a continuous line shown.

Your dashboard should look like the image below.

Grafana Dashboard

Now click Save on the top right, to save your work.

The sample data is publicly available data from the National Oceanic and Atmospheric Administration’s (NOAA) Center for Operational Oceanographic Products and Services. The data include 15,258 observations of water levels (ft) collected every six minutes at two stations (Santa Monica, CA (ID 9410840) and Coyote Creek, CA (ID 9414575)) over the period from August 18, 2019, through September 18, 2019.

Note that the measurements average_temperature, h2o_pH, h2o_quality, and h2o_temperature contain fictional data.

Embed the custom Dashboard

Once you are satisfied with your new Grafana dashboard, we can proceed and embed it in the Docker image.

    1. Export your dashboard. (Click the share button on the right of the dashboard name, select Export, and save to file)
    1. Copy the downloaded file in the provisioning/dashboards, rename it to mydashboard.json (feel free to choose any name you like).

You can either copy using file explorer (Windows) / Finder (macOS) / File manager (Linux) or command line.

cp /mnt/c/Users/<home folder>/Downloads/<your downloaded file> provisioning/dashboards/mydashboard.json

If you want to copy the file through Windows File Explorer, please note that the WSL2 share in Windows has name \\$wsl

    1. Create a dashboard.yml file in the provisioning/dashboards
cat >provisioning/dashboards/dashboard.yml <<EOF
apiVersion: 1

providers:
    - name: "InfluxDB"
      orgId: 1
      folder: ""
      type: file
      disableDeletion: true
      editable: false
      allowUiUpdates: false
      options:
          path: /etc/grafana/provisioning/dashboards
EOF
    1. Stop the running docker-compose by pressing ctrl-c or executing docker-compose down in another terminal (within our project directory).
    1. Restart docker-compose rebuilding the image
docker-compose up --build

You have your new dashboard available, embedded in your Grafana custom image, automatically bound to a named InfluxDB instance.