Braindump - Install Service Mesh 0.9 on OpenShift 3.11

Maistra (upstream project for the RH product) 0.9 got released recently, and shortly after the productized bits of Red Hat (Tech Preview 9, “TP9” in short). Here’s how I installed it on my OCP 3.11 - with a few notes. the official docs can be found here: https://docs.openshift.com/container-platform/3.11/servicemesh-install/servicemesh-install.html

First of all, the prerequisites need to be fulfilled as described here https://docs.openshift.com/container-platform/3.11/servicemesh-install/servicemesh-install.html#updating-node-configuration. This can be easily automated using Ansible:

- name: prereqs service mesh
  hosts: all # for non-playgrounds, you might want to limit the hosts to the relevant ones.
  gather_facts: false
  become: true

  tasks:
    - name: ensure kernel config on nodes is updated for ES
      copy:
        src: files/99-elasticsearch.conf
        dest: /etc/sysctl.d/99-elasticsearch.conf

    - name: ensure sysctl config is updated for the above entry
      sysctl:
        name: vm.max_map_count
        value: "{{ 262144 | int }}"
        state: present

In order to install Istio on OCP 3.11, you need to be cluster-admin since it requires you to use custom resources.

The to be executed steps are the following:

Create the istio-system namespace
Create an app there from an operator template
Deploy the control plane using a custom resource

The concrete steps are:

oc new-project istio-operator
oc create -f istio_product_operator_template.yaml -n istio-operator

Wait for the deployment to finish. You should see something like this in the logs (oc logs -n istio-operator $(oc -n istio-operator get pods -l name=istio-operator --output=jsonpath={.items..metadata.name}))

time="2018-08-31T17:42:39Z" level=info msg="Go Version: go1.9.4"
time="2018-08-31T17:42:39Z" level=info msg="Go OS/Arch: linux/amd64"
time="2018-08-31T17:42:39Z" level=info msg="operator-sdk Version: 0.0.5+git"
time="2018-08-31T17:42:39Z" level=info msg="Metrics service istio-operator created"
time="2018-08-31T17:42:39Z" level=info msg="Watching resource istio.openshift.com/v1alpha1, kind Installation, namespace istio-operator, resyncPeriod 0"

Create the custom resource: oc create -f istio-installatio.yaml -n istio-operator

A few things are to be considered:

the custom resource has to be named istio-installation
if you need policy enforcement in Mixer (in non-playground environment this is very likely), you need to set disablePolicyChecks to false in the istio-system config map. The default has changed with TP9.

If you don’t need the launcher and the 3Scale adapter, the custom resource is fairly simple:

apiVersion: "istio.openshift.com/v1alpha1"
kind: "Installation"
metadata:
  name: "istio-installation"
spec:
  deployment_type: openshift
  istio:
    authentication: true
    community: false
    prefix: openshift-istio-tech-preview/
    version: 0.9.1
  jaeger:
    prefix: distributed-tracing-tech-preview/
    version: 1.11.0
    elasticsearch_memory: 1Gi
  kiali:
    username: kiali
    password: supersecret
    prefix: openshift-istio-tech-preview/
    version: 0.15.0

If you want automatic sidecar injection, you need to enable a mutating webhook. For that the master config has to be adapted. I automated that for my home lab (not pretty, but does the job) with the following tasks in a playbook:

    - name: collect minimal facts
      setup:
        gather_subset:
          - min

    - name: take backup of master-config.yaml
      copy:
        src: /etc/origin/master/master-config.yaml
        dest: /etc/origin/master/master-config-before-istio-{{ ansible_date_time.date }}-{{ ansible_date_time.time }}
        mode: 0644
        owner: root
        group: root
        remote_src: yes

    - name: configure Istio admissionwebhook
      blockinfile:
        path: /etc/origin/master/master-config.yaml
        marker_begin: "Adding Istio MutatingAdmissionWebhook - Start - DO NOT REMOVE"
        marker_end: "Adding Istio MutatingAdmissionWebhook - End - DO NOT REMOVE"
        insertafter: "  pluginConfig:"
        block: |2
              MutatingAdmissionWebhook:
                configuration:
                  apiVersion: apiserver.config.k8s.io/v1alpha1
                  kubeConfigFile: /dev/null
                  kind: WebhookAdmission
              ValidatingAdmissionWebhook:
                configuration:
                  apiVersion: apiserver.config.k8s.io/v1alpha1
                  kubeConfigFile: /dev/null
                  kind: WebhookAdmission

    - name: Restart API
      shell: /usr/local/bin/master-restart api

    - name: Restart controller
      shell: /usr/local/bin/master-restart controllers