7.5
9.0 7.5
Docs menu
MENU

Managing Windows Containers

This feature is a technical preview. It is not ready for production.

You can use Virtuozzo Server to create and manage Windows containers on Microsoft Windows nodes.

Setting Up the Environment

You will need the following:

  • A node running Virtuozzo Server 7.5 or newer.
  • Three or more Microsoft Windows nodes in a Storage Spaces Direct cluster. The nodes must be running Microsoft Windows Server 2019 v1809 or newer. It is also implied that the nodes meet the Storage Spaces Direct hardware requirements.
  • A network interconnecting the Virtuozzo Server node and Windows nodes. All Microsoft Windows nodes will need Internet access for Docker installation.

Configuring the Virtuozzo Server Node

  1. Install the required packages:

    1
2

    # yum groupinstall "Windows Containers"
# yum install virt-install

  2. Create a temporary directory, e.g, /home/tmp/, and generate certificates in it.

    • The CA certificate:

      1
2

      # /usr/share/winrm-docker-deploy/create-certs.sh -m ca \
-t /home/tmp/ -pw <cert_passwd>

      Where <cert_passwd> is a password for the certificate.

    • The client certificate:

      1
2

      # /usr/share/winrm-docker-deploy/create-certs.sh -m client \
-t /home/tmp/ -us <windows_admin> -pw <cert_passwd>

      Where <windows_admin> is the username of an account that is in the Administrators group on every cluster node.

    • A certificate for every node in the Storage Spaces Direct cluster:

      1
2

      # /usr/share/winrm-docker-deploy/create-certs.sh -m server \
-h <hostname> -hip <hostip> -t /home/tmp/ -pw <cert_passwd>

      Where <hostname> and <hostip> are the host name and IP address of a cluster node, respectively. Run this command for each cluster node.

    For example, if you have three nodes in the Storage Spaces Direct cluster: win1.example.com, win2.example.com, and win3.example.com, you will get these certificates as a result:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

    # ls -1 /home/tmp/
cakey.pem
ca.pem
ca.srl
cert.pem
key.pem
win1.example.com-https.pfx
win1.example.com-server-cert.pem
win1.example.com-server-key.pem
win2.example.com-https.pfx
win2.example.com-server-cert.pem
win2.example.com-server-key.pem
win3.example.com-https.pfx
win3.example.com-server-cert.pem
win3.example.com-server-key.pem
    Non-root users may require additional permissions to read these files.

  3. Copy the CA certificate as well as the client public and private certificates to /etc/pki/libvirt/docker/:

    1

    # cp /home/tmp/{ca,cert,key}.pem /etc/pki/libvirt/docker/

Configuring Microsoft Windows Nodes

Perform the following steps on each Windows Server node in the Storage Spaces Direct cluster:

  1. Install the Containers and Windows Server Backup features. For example, from a PowerShell session run as administrator:

    1

    PS C:\> Install-WindowsFeature -Name Containers, Windows-Server-Backup

  2. Reboot the node.

  3. Download the deployment bundle and unpack it to a temporary directory, e.g., C:\ProgramData\Virtuozzo\docker-bundle\. For example:

    1
2
3

    PS C:\> (New-Object System.Net.WebClient).DownloadFile( `
"https://docs.virtuozzo.com/files/winct/docker-deploy-latest.zip", `
"C:\docker-deploy-latest.zip")

    1
2

    PS C:\> Expand-Archive -LiteralPath C:\docker-deploy-latest.zip `
-DestinationPath C:\ProgramData\Virtuozzo\docker-bundle\

    You will get the following files as a result:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

    PS C:\> ls C:\ProgramData\Virtuozzo\docker-bundle\

Directory: C:\ProgramData\Virtuozzo\docker-bundle\

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       11/26/2020   8:03 AM                bindata
d-----       11/26/2020   8:03 AM                certs
d-----       11/26/2020   8:03 AM                vz-windows-backup
-a----       11/25/2020   3:25 PM          38131 auto-deploy.ps1
-a----       11/25/2020   3:25 PM            140 bundle-version.txt
-a----       11/25/2020   3:25 PM          27905 winct-uninstall.ps1

  4. Copy the CA and node’s certificates from the Virtuozzo Server node to C:\ProgramData\Virtuozzo\docker-bundle\certs. Remove the hostname from file names. For example:

    1
2
3
4

    PS C:\> scp root@10.94.106.235:/etc/pki/libvirt/docker/ca.pem `
C:\ProgramData\Virtuozzo\docker-bundle\certs\
root@10.94.106.235's password:
ca.pem    100% 1781     1.7KB/s   00:00

    1
2
3
4
5
6
7

    PS C:\> scp root@10.94.106.235:/home/tmp/$(`
[System.Net.Dns]::GetHostByName($env:COMPUTERNAME).hostname)* `
C:\ProgramData\Virtuozzo\docker-bundle\certs\
root@10.94.106.235's password:
win1.example.com-https.pfx         100% 4021     3.9KB/s   00:00
win1.example.com-server-cert.pem   100% 1777     1.7KB/s   00:00
win1.example.com-server-key.pem    100% 3243     3.2KB/s   00:00

    1
2

    PS C:\> ls C:\ProgramData\Virtuozzo\docker-bundle\certs\ | Rename-Item -NewName `
{ $_.Name -replace $([System.Net.Dns]::GetHostByName($env:COMPUTERNAME).hostname+"-"),'' }

    You will get the following files as a result:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10

    PS C:\> ls C:\ProgramData\Virtuozzo\docker-bundle\certs\

 Directory: C:\ProgramData\Virtuozzo\docker-bundle\certs\

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-ar---        11/4/2020  11:42 AM           1781 ca.pem
-a----        11/4/2020  11:41 AM           4021 https.pfx
-ar---        11/4/2020  11:41 AM           1777 server-cert.pem
-ar---        11/4/2020  11:41 AM           3243 server-key.pem

  5. Run the deployment script:

    1
2
3

    PS C:\> cd C:\ProgramData\Virtuozzo\docker-bundle\
PS C:\> .\auto-deploy.ps1 -username:<windows_admin> `
-password:<windows_admin_passwd> -pwforpfx:<cert_passwd>

    Where windows_admin and windows_admin_passwd are the username and password of an account that is in the Administrators group on every cluster node and cert_passwd is the certificate password. For simplicity, use the same administrator account that you generated the certificates with.
    The deployment script will download and install Docker and required components, including a base Nano Server 1809 image. It will also replace the original dockerd.exe with a custom one made by Virtuozzo. If Docker is already installed, the script will update it and replace the executable. The script will also add firewall rules and import the certificates. The log will be saved to C:\ProgramData\Virtuozzo\winctfiles\auto-deploy.log.

    If you only need to import or update the certificates, use the deployment script with the -onlycerts option. For more options, run .\auto-deploy.ps1 -help.

After the script finishes, reboot the node and check that the Docker client and server are running. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

PS C:\> docker version
Client: Docker Engine - Enterprise
 Version:           19.03.12
 API version:       1.40
 Go version:        go1.13.13
 Git commit:        4306744
 Built:             08/05/2020 19:27:53
 OS/Arch:           windows/amd64
 Experimental:      false

Server:
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.24)
  Go version:       go1.13.15
  Git commit:       cb7af2e
  Built:            Tue Oct 13 12:15:38 2020
  OS/Arch:          windows/amd64
  Experimental:     false

If an error is shown for the server, try starting it manually:

1

PS C:\> Start-Service docker

Creating Windows Containers

After setting up both the Virtuozzo Server and Microsoft Windows Server nodes, you can create Windows containers using virt-install. Container base images need to be present on Windows Server nodes. The deployment script only installs the Nano Server 1809 image. Any other base images need to be pulled from the Docker Hub first.

You can create a Windows container from the Virtuozzo Server node as follows:

1
2
3
4

# virt-install --connect docker://<hostname> --name <ct_name> --memory 1024 \
--boot init=* --os-variant none --console none --disk bus=virtio,path=<ct_image> \
--network type=network,mac=<ct_MAC>,source=<net_name> \
--filesystem <guest_dir>,<host_dir> --noreboot --noautoconsole

Where:

  • hostname is the hostname of the Windows Server node to create the container on.
  • ct_name is a unique container name.
  • ct_image is a base image to create the container from. For example, mcr.microsoft.com/windows/nanoserver:1809.
  • ct_MAC is a unique container MAC address.
  • net_name is a name of a Docker network to connect the container to. Use default for the default NAT network.
  • guest_dir and host_dir are the container guest and host directories located on a virtual disk in the Storage Spaces Direct cluster. Create these directories if they do not exist.

For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# virt-install --connect docker://win1.example.com --name winct1 --memory 1024 \
--boot init=* --os-variant none --console none \
--disk bus=virtio,path=mcr.microsoft.com/windows/nanoserver:1809 \
--network type=network,mac=00:01:02:03:04:05,source=default \
--filesystem C:/ClusterStorage/disk1/guest_dir/,C:/ClusterStorage/disk1/host_dir/ \
--noreboot --noautoconsole

Starting install...
Domain creation completed.
You can restart your domain by running:
  virsh --connect docker://win1.example.com start winct1

This command will create a Windows container based on a Nano Server image.

As more examples for this technical preview, you can also create Windows containers with DotNetNuke and WordPress from base images published by Virtuozzo at Docker Hub. These base images include the required database management systems, so you can skip to setting up DotNetNuke and WordPress once you run the container. Do the following:

  1. Pull the images from Docker Hub to Windows Server nodes:

    1

    PS C:\> docker pull virtuozzo/dnn:techpreview

    or

    1

    PS C:\> docker pull virtuozzo/wordpress:techpreview

  2. Create the Windows container from the Virtuozzo Server node. For example:

    1
2
3
4
5
6
7

    # virt-install --connect docker://win1.example.com --name winct1dnn --memory 2048 \
--boot init=* --os-variant none --console none \
--disk bus=virtio,path=virtuozzo/dnn:techpreview \
--network type=network,mac=00:01:02:03:04:06,source=default \
--filesystem C:/ClusterStorage/disk1/guest_dir/,C:/ClusterStorage/disk1/host_dir/ \
--noreboot --noautoconsole
# virsh --connect docker://win1.example.com start winct1dnn

    Provide A DotNetNuke container with at least 2GB of memory for the setup to complete.

    or

    1
2
3
4
5
6
7

    # virt-install --connect docker://win1.example.com --name winct1wp --memory 1024 \
--boot init=* --os-variant none --console none \
--disk bus=virtio,path=virtuozzo/wordpress:techpreview \
--network type=network,mac=00:01:02:03:04:07,source=default \
--filesystem C:/ClusterStorage/disk1/guest_dir/,C:/ClusterStorage/disk1/host_dir/ \
--noreboot --noautoconsole
# virsh --connect docker://win1.example.com start winct1wp

  3. Find out the container’s IP address. For example:

    1
2

    # virsh --connect docker://win1.example.com dumpxml winct1dnn | grep "ip address"
<ip address='192.168.175.204' family='ipv4' prefix='24'/>

    or

    1
2

    # virsh --connect docker://win1.example.com dumpxml winct1wp | grep "ip address"
<ip address='192.168.175.114' family='ipv4' prefix='24'/>

  4. As the Windows containers use a NAT network by default, they are only accessible from the Windows Server node the container is running on. From that node, visit the container’s IP address (and optionally port) in a web browser to proceed with setup.

To make Windows containers publicly accessible, let them use a virtual switch bound to a node’s physical network adapter connected to a public network. To create a virtual switch, do the following:

  1. Open the Hyper-V Manager. E.g., in Server Manager, click Tools > Hyper-V Manager or run virtmgmt.msc from the Command Prompt or PowerShell.

  2. In the Hyper-V Manager, click Virtual Switch Manager in the Actions menu.

    image

  3. In the Virtual Switch Manager, choose New virtual network switch > External. Click Create Virtual Switch.

    image

  4. In Virtual Switch Properties, enter a name for the switch, e.g., vSwitch1, and select a public physical network adapter in External network. Click OK.

    image

  5. Restart Docker:

    1

    PS C:\> Restart-Service docker

  6. Check that the switch is listed in Docker networks. For example:

    1
2
3
4
5

    PS C:\> docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
ad2c3ef4de3e        nat                 nat                 local
ea6c5aa76c42        none                null                local
7ae7bd5c638e        vSwitch1            transparent         local

Now you can specify the public network, e.g., --network ... source=vSwitch1 instead of --network ... source=default, when creating Windows containers with virt-install.

Starting, Listing, and Stopping Windows Containers

After creating a Windows container, start it as indicated by virt-install. For example:

1
2

# virsh --connect docker://win1.example.com start winct1
Domain winct1 started

Using virsh, you can also list and shut down Windows containers on the node. For example:

1
2
3
4

# virsh --connect docker://win1.example.com list --all
 Id      Name      State
-----------------------------
 39370   winct1    running

1

# virsh --connect docker://win1.example.com shutdown winct1

Running Commands in Windows Containers

To attach to a Windows container in this technical preview, use the docker attach command on the Windows Server node the container is running on.

To run a command inside a Windows container, use the docker exec command as follows:

1

PS C:\> docker exec <ct> <shell> /c "<command>"

Where:

  • ct is the container name or ID.
  • shell is the name of the shell bundled with the container image, e.g., cmd is Command Prompt and pwsh is PowerShell.
  • command is the command to execute in the shell.

For example:

1

PS C:\> docker exec winct1 cmd /c "echo Test > C:\ClusterStorage\disk1\host_dir\test.txt"

1
2

PS C:\> docker exec winct1 cmd /c "type C:\ClusterStorage\disk1\host_dir\test.txt"
Test

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

PS C:\> docker exec winct1 cmd /c "dir C:\ClusterStorage\disk1\host_dir\"
 Volume in drive C has no label.
 Volume Serial Number is 34FA-4299

 Directory of C:\ClusterStorage\disk1\host_dir

11/05/2020  03:04 AM    <DIR>          .
11/05/2020  03:04 AM    <DIR>          ..
11/05/2020  03:01 AM                 9 test.txt
               1 File(s)              9 bytes
               2 Dir(s)  544,967,794,688 bytes free

Configuring Windows Container Parameters

In this technical preview, redefining container configuration (editing its XML scheme) is not supported. You can, however, update container resource limits as follows:

  • Change available memory using virsh setmem. For example:

    1
2
3
4

    # virsh -c docker://win1.example.com setmem winct1 512M
# virsh -c docker://win1.example.com dumpxml winct1 | grep -i memory
<memory unit='KiB'>524288</memory>
<currentMemory unit='KiB'>524288</currentMemory>

  • Edit the CPU shares or quotas for the container using virsh schedinfo. For example:

    1
2
3
4
5

    # virsh -c docker://win1.example.com schedinfo winct1 cpu_shares=1000
Scheduler      : posix
cpu_shares     : 1000
vcpu_period    : 0
vcpu_quota     : 0

    or

    1
2
3
4
5

    # virsh -c docker://win1.example.com schedinfo winct1 vcpu_period=4000 vcpu_quota=2000
Scheduler      : posix
cpu_shares     : 0
vcpu_period    : 4000
vcpu_quota     : 2000

    You cannot set both cpu_shares and vcpu_quota at the same time.

    Set cpu_shares or vcpu_quota or to -1 to reset it:

    1

    # virsh -c docker://win1.example.com schedinfo winct1 cpu_shares=-1

    or

    1

    # virsh -c docker://win1.example.com schedinfo winct1 vcpu_quota=-1

  • Throttle container disk with virsh blkdeviotune by changing the total throughput limit, in bytes per second, as well as the total I/O operations limit per second. For example:

    1
2

    # virsh -c docker://win1.example.com blkdeviotune winct1 vda \
--total-bytes-sec=104857600 --total-iops-sec=1000

    Use virsh domblklist to find out container’s device names.

Migrating Windows Containers

Take note of the following:

  • The source container must be stopped.
  • The name and UUID of the migrated container must be unique on the destination node.
  • The migrated container keeps the source’s UUID.
  • The MAC address of the migrated container must be unique within the destination node’s network. For example, if all nodes belong to the same network, MAC addresses must be unique across all nodes.
  • The data on container’s drive C: will be lost after migration. The data on persistent volumes will remain.

To migrate a Windows container, use the migrate subcommand of virsh. For example:

  • Migrate a container and delete the source:

    1
2

    # virsh --connect docker://node3.winctpreview.com migrate winct1 --desturi \
docker://node1.winctpreview.com --offline --persistent --undefinesource

  • Migrate a container under a new name and delete the source:

    1
2
3

    # virsh --connect docker://node3.winctpreview.com migrate winct1 --desturi \
docker://node1.winctpreview.com --offline --persistent --undefinesource \
--dname winct1_migrated

  • Migrate a container under a new name and keep the source:

    1
2

    # virsh --connect docker://node3.winctpreview.com migrate winct1 --desturi \
docker://node1.winctpreview.com --dname winct1_migrated --offline --persistent
    In this case, the container can only be migrated once. On the second attempt, the UUIDs of the source container and the one migrated earlier will be the same, which will cause a conflict. You can, however, create a container with a new UUID from the source’s configuration file and migrate it.

Managing Windows Container Backups

It is recommended to use the Windows Server Backup feature for container backups. You can use the provided PowerShell scripts to partially automate backup-related tasks.

Run the script on each node to find the containers with persistent CSVs and add those volumes to the backup schedule:

1

PS> Config-Docker-Backup.ps1 -Times '<1st> <2nd> <3rd>'

The script will configure the Windows Server Backup feature to create incremental backups daily at <1st>, <2nd>, and <3rd>. If you omit the -Times parameter, backups will be created at 9:00, 12:00, and 15:00.

Container backups will be listed in the Windows Server Backup snap-in. To open it, run wbadmin.msc.

To restore a container backup, run

1

PS> Restore-Docker.ps1 <ct_name>

Where <ct_name> is container name.

Restoring will overwrite the backed up container volume(s).

Troubleshooting Windows Containers

In case of issues with Windows containers, you can create a problem report and send it to the technical support team:

1
2
3
4

usage: vz-problem-report [-h] [-v] [--proxy PROXY] [--name NAME]
                         [--email EMAIL] [--description DESC]
                         [--reportfile FILE] [--version]
                         address

Where:

  • address is the hostname or IP address of the Microsoft Windows Server host to collect the report from
  • name and email are your name and email address, respectively
  • desc is the description of the problem
  • reportfile is the name of the report file to save locally

If the node is not connected to the Internet, you can create a local report file with the --reportfile option and manually email it to the support team.

Uninstalling the Windows Containers Feature

To remove the ability to create and manage Windows Containers on Microsoft Windows Server nodes from a Virtuozzo Server node, run the uninstallation script:

1

PS C:> C:\Program Files\Virtuozzo\winctfiles\winct-uninstall.ps1

The script will attempt to undo changes made by the deployment script:

  • Uninstall Docker if it was not installed before the deployment. If Docker was already installed, leave it with the custom dockerd.exe but restore its service’s original startup type (e.g., Automatic or Manual). Stop the service if the original startup type is Manual.
  • Uninstall DockerMsftProvider.
  • Delete joblib, Prometheus exporter, and backup scripts.
  • Remove the path to the bundle from PATH.
  • Restore the original startup type of the WinRM service. Stop the service if the original startup type is Manual.
  • Delete the firewall rules for Docker and WinRM. If rules existed and were changed, restore them.
  • Delete the imported certificate files from ProgramDatadockercerts.d. Delete the certificates from the store. Kill the HTTP Listener in WinRM and remove authentication by the imported certificate.