## Release notes for kOps 1.22 series

# Significant changes

## Instance metadata service version 2

 On AWS, kOps will enable [Instance Metadata Service Version 2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html) and require tokens on new clusters with Kubernetes 1.22. In addition, the following max hop limits will be set by default:

 * worker and API server Nodes, and bastions, will have a limit of 1 hop.
 * control plane nodes will have a limit of 3 hops to accommodate for controller Pods without host networking that need to assume roles.

This will increase security by default, but may break some types of workloads. In order to revert to old behavior, add the following to the InstanceGroup:

```
spec:
  instanceMetadata:
    httpTokens: optional
```

## External ServiceAccountPermissions

Many of kOps addons can now make direct use of external permissions.
This can be enabled by adding the following to the Cluster spec:

```
spec:
  iam:
    useServiceAccountExternalPermissions: true
```

Currently this is only available using the AWS cloud provider.

## Managed nvidia instances

kOps can now provision instances with nvidia GPUs and configure it for container workloads without the need of hooks and operators. See [GPU support](https://kops.sigs.k8s.io/gpu/)

## Breaking change in NodeLocalDNS

Since 1.22.0 Cluster `spec.kubeDNS.nodeLocalDNS.forwardToKubeDNS` default behaviour changes from `true` to `false`.

## Other significant changes

* New clusters on AWS will no longer provision an SSH public key by default. To provision
  an SSH public key on a new cluster, use the `--ssh-public-key` flag to `kops create cluster`.

* The kOps Terraform support now renders managed files through the Terraform configuration instead
  of writing them to S3 directly. This defers changes to these files until the time of `terraform apply`.
  This feature may be temporarily disabled by turning off the `TerraformManagedFiles` feature flag
  using `export KOPS_FEATURE_FLAGS="-TerraformManagedFiles"`.

* kOps now implements graceful rotation of its Certificate Authorities and the service
  account signing key. See the documentation on [How to rotate all secrets / credentials](../operations/rotate-secrets.md)

* New clusters running Kubernetes 1.22 will have AWS EBS CSI driver enabled by default.

* kOps now supports Debian 11 (Bullseye).

* kOps can now use [external-dns](https://github.com/kubernetes-sigs/external-dns/) as a drop-in replacement for dns-controller.

# Breaking changes

## Control plane pods no longer mount /srv/kubernetes

 For security reasons, `/srv/kubernetes` is no longer mounted in the kube-apiserver and kube-controller-manager Pods. This also means the files in the default file assets path will be unavailable. If you have file assets or other files needed by kube-apiserver, you must put these into `/srv/kubernetes/kube-apiserver/` or `/srv/kubernetes/kube-controller-manager`, respectively.

For file assets, it means adding an explicit path as shown below:

```yaml
  fileAssets:
  - name: audit-policy-config
    path: /srv/kubernetes/kube-apiserver/audit-policy-config.yaml # make sure you add the path
    roles:
    - Master
    content: |
      apiVersion: audit.k8s.io/v1
      kind: Policy
      rules:
      - level: Metadata
```

## Other breaking changes

* Support for Kubernetes versions 1.15 and 1.16 has been removed.

* The legacy addons from `https://github.com/kubernetes/kops/tree/master/addons` have been deprecated and will not be available in Kubernetes 1.23+. Use [managed addons](https://kops.sigs.k8s.io/addons) instead.

* The legacy location for downloads `s3://https://kubeupv2.s3.amazonaws.com/kops/` has been deprecated and will not be used for new releases. The new canonical downloads location is `https://artifacts.k8s.io/binaries/kops/`.

* The `assets` phase of `kops update cluster` has been removed. It is replaced by the new `kops get assets --copy` command.

* Support for importing and converting kubeup clusters has been removed.

* Support for Cilium and RHEL 8 has been removed. Cilium users will need to migrate to a distribution with a newer Linux kernel.

# Required actions

* Amazon Linux 2 users are encouraged to use the AMIs based on the 5.10 Linux kernel. See [the documentation](../operations/images.md#amazon-linux-2) for more information.

* Terraform support now requires Terraform >=0.15.0.
  Users on older versions must follow Terraform's recommended upgrade path of applying one minor version at a time prior to running `kops update cluster --target terraform`.

* The kOps Terraform support now renders managed files through the Terraform configuration instead
  of writing them to S3 directly. If, after upgrading kOps and applying a new Terraform plan,
  you subsequently downgrade to an earlier version of kOps, the generated plan will delete these
  files, breaking the cluster. Prior to applying the plan, you will need to orphan all the
  `aws_s3_bucket_object` objects the plan wants to destroy. Use `terraform state rm` on each of them.
  Then re-run `terraform plan` until there are no such objects in the plan.

  If you applied the plan without first orphaning all of these objects, fix the cluster by re-running
  `kops update cluster --target terraform`.

* Terraform users of clusters with names beginning with digits will need to move resources prior to upgrading to kOps 1.22. Some of the following commands will need to be run depending on the particular cluster configuration. Confirm the Terraform plan doesn't destroy any of these resources before running `terraform apply`.
  ```bash
  # View the existing terraform resource names for the exact value to use
  HYPHENATED_CLUSTER_NAME=123-cluster-example-com
  terraform state mv "aws_iam_openid_connect_provider.${HYPHENATED_CLUSTER_NAME}" "aws_iam_openid_connect_provider.prefix_${HYPHENATED_CLUSTER_NAME}"
  terraform state mv "aws_internet_gateway.${HYPHENATED_CLUSTER_NAME}" "aws_internet_gateway.prefix_${HYPHENATED_CLUSTER_NAME}"
  terraform state mv "aws_route_table.${HYPHENATED_CLUSTER_NAME}" "aws_route_table.prefix_${HYPHENATED_CLUSTER_NAME}"
  terraform state mv "aws_vpc.${HYPHENATED_CLUSTER_NAME}" "aws_vpc.prefix_${HYPHENATED_CLUSTER_NAME}"
  terraform state mv "aws_vpc_dhcp_options.${HYPHENATED_CLUSTER_NAME}" "aws_vpc_dhcp_options.prefix_${HYPHENATED_CLUSTER_NAME}"
  terraform state mv "aws_vpc_dhcp_options_association.${HYPHENATED_CLUSTER_NAME}" "aws_vpc_dhcp_options_association.prefix_${HYPHENATED_CLUSTER_NAME}"
  ```

# Deprecations

* Support for Kubernetes version 1.17 is deprecated and will be removed in kOps 1.23.

* Support for Kubernetes version 1.18 is deprecated and will be removed in kOps 1.24.

* Support for the Lyft CNI is deprecated and will be removed in kOps 1.23.

* Support for CentOS 7 is deprecated and will be removed in future versions of kOps.

* Support for CentOS 8 is deprecated and will be removed in future versions of kOps.

* Support for Debian 9 (Stretch) is deprecated and will be removed in future versions of kOps.

* Support for RHEL 7 is deprecated and will be removed in future versions of kOps.

* Support for Ubuntu 18.04 (Bionic) is deprecated and will be removed in future versions of kOps.

* All legacy addons are deprecated in favor of managed addons, including the [metrics server addon](https://github.com/kubernetes/kops/tree/master/addons/metrics-server) and the [autoscaler addon](https://github.com/kubernetes/kops/tree/master/addons/cluster-autoscaler).

* The `node-role.kubernetes.io/master` and `kubernetes.io/role` labels are deprecated and might be removed from control plane nodes in kOps 1.23.

* The `TerraformJSON` feature flag is deprecated and will be removed in kOps 1.23. Only native HCL2 Terraform output will be supported.

* Due to lack of maintainers, the Aliyun/Alibaba Cloud support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this cloud provider.

* Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

# Other changes of note

* Support for shell completion has been substantially improved. kOps has added support for shell completion in `fish` and `PowerShell`.

* It is no longer necessary to set `AWS_SDK_LOAD_CONFIG=1` in the environment when using AWS assumed roles with the `kops` CLI.

* There is a new command `kops get assets` for listing image and file assets used by a cluster.
  It also includes a `--copy` flag to copy the assets to local repositories.
  See the documentation on [Using local asset repositories](../operations/asset-repository.md) for more information.

* kOps now provisions TLS server certificates signed by the Kubernetes general CA to kube-controller-manager and kube-scheduler.
  The previous behavior of using self-signed certs may be restored by setting `kubeControllerManager.tlsCertFile` and/or
  `kubeScheduler.tlsCertFile` to `""` in the cluster spec.

* kOps has changed the CN of the Kubernetes general CA from `kubernetes` to `kubernetes-ca`. This change is backward-compatible, but any tools using the CN will need to be updated.

* Cilium now supports the wireguard protocol for transparent encryption.


# Full change list since 1.21.0 release

## 1.22.0-alpha.1 to 1.22.0-alpha.2

* e2e upgrade-ab: fix a few errors [@justinsb](https://github.com/justinsb) [#11409](https://github.com/kubernetes/kops/pull/11409)
* Verify all versions are set correctly [@johngmyers](https://github.com/johngmyers) [#11413](https://github.com/kubernetes/kops/pull/11413)
* Use etcd-manager built from etcdadm repo [@justinsb](https://github.com/justinsb),[@hakman](https://github.com/hakman) [#11098](https://github.com/kubernetes/kops/pull/11098)
* Remove code for no-longer-supported k8s versions  [@johngmyers](https://github.com/johngmyers) [#11412](https://github.com/kubernetes/kops/pull/11412)
* Update the release process documentation [@johngmyers](https://github.com/johngmyers) [#11419](https://github.com/kubernetes/kops/pull/11419)
* [addons/awscsidriver] Bump to GA release [@dntosas](https://github.com/dntosas) [#11418](https://github.com/kubernetes/kops/pull/11418)
* [Digital Ocean] Add an e2e job for DO [@srikiz](https://github.com/srikiz) [#10963](https://github.com/kubernetes/kops/pull/10963)
* Fix references to v1.20 in v1.21 release notes [@hakman](https://github.com/hakman) [#11427](https://github.com/kubernetes/kops/pull/11427)
* Release notes for 1.21.0-beta.1 [@johngmyers](https://github.com/johngmyers) [#11426](https://github.com/kubernetes/kops/pull/11426)
* e2e: only get ExternalIPRange if we need it [@justinsb](https://github.com/justinsb) [#11431](https://github.com/kubernetes/kops/pull/11431)
* e2e upgrade-ab: a few more fixes and notes on how to run locally [@justinsb](https://github.com/justinsb) [#11432](https://github.com/kubernetes/kops/pull/11432)
* Create new clusters without forcing a container runtime [@hakman](https://github.com/hakman) [#11428](https://github.com/kubernetes/kops/pull/11428)
* Update verify-terraform to use 0.15.3 [@rifelpet](https://github.com/rifelpet) [#11433](https://github.com/kubernetes/kops/pull/11433)
* Carry forward 1.20 deprecations to 1.21 release notes [@johngmyers](https://github.com/johngmyers) [#11438](https://github.com/kubernetes/kops/pull/11438)
* Start release notes for 1.22 [@johngmyers](https://github.com/johngmyers) [#11439](https://github.com/kubernetes/kops/pull/11439)
* Sort --extra-tags of ebs-csi-driver [@codablock](https://github.com/codablock) [#11444](https://github.com/kubernetes/kops/pull/11444)
* Fix typo in 1.22 release notes [@johngmyers](https://github.com/johngmyers) [#11448](https://github.com/kubernetes/kops/pull/11448)
* Add test scenario for aws ebs csi driver [@olemarkus](https://github.com/olemarkus) [#11449](https://github.com/kubernetes/kops/pull/11449)
* Always install the latest plugin versions for Terraform tests [@hakman](https://github.com/hakman) [#11447](https://github.com/kubernetes/kops/pull/11447)
* Set the output base for fitask [@hakman](https://github.com/hakman) [#11411](https://github.com/kubernetes/kops/pull/11411)
* Simplify buildLaunchTemplateTask() part one [@johngmyers](https://github.com/johngmyers) [#11452](https://github.com/kubernetes/kops/pull/11452)
* Add missing carryover items from 1.21 release notes [@johngmyers](https://github.com/johngmyers) [#11451](https://github.com/kubernetes/kops/pull/11451)
* Add support for CAS 1.21.0 [@olemarkus](https://github.com/olemarkus) [#11462](https://github.com/kubernetes/kops/pull/11462)
* Allow AWS instance types with multiple architectures [@hakman](https://github.com/hakman) [#11463](https://github.com/kubernetes/kops/pull/11463)
* Fix KCM livenessProbe to use secure port [@rifelpet](https://github.com/rifelpet) [#11454](https://github.com/kubernetes/kops/pull/11454)
* Simplify buildLaunchTemplateTask() part two [@johngmyers](https://github.com/johngmyers) [#11461](https://github.com/kubernetes/kops/pull/11461)
* Use kubernetes.default for OIDC discovery in gossip clusters [@rifelpet](https://github.com/rifelpet) [#11470](https://github.com/kubernetes/kops/pull/11470)
* Add instructions for updating the k8s versions periodic jobs [@rifelpet](https://github.com/rifelpet) [#11473](https://github.com/kubernetes/kops/pull/11473)
* Release notes for 1.20.1 [@justinsb](https://github.com/justinsb) [#11475](https://github.com/kubernetes/kops/pull/11475)
* Release notes for 1.19.3 [@justinsb](https://github.com/justinsb) [#11474](https://github.com/kubernetes/kops/pull/11474)
* Update alpha channel with K8s releases from May-12 2021 [@MoShitrit](https://github.com/MoShitrit) [#11476](https://github.com/kubernetes/kops/pull/11476)
* upup: gcetasks: fix diffs in instance template and router [@nicktrav](https://github.com/nicktrav) [#11460](https://github.com/kubernetes/kops/pull/11460)
* Discover what zone the cluster is in for the aws-ebs-csi driver tests [@olemarkus](https://github.com/olemarkus) [#11472](https://github.com/kubernetes/kops/pull/11472)
* Use ginkgo to run the tests so we can run things in parallel [@olemarkus](https://github.com/olemarkus) [#11479](https://github.com/kubernetes/kops/pull/11479)
* Kubetest2 - Increase validation time for DO jobs [@rifelpet](https://github.com/rifelpet) [#11481](https://github.com/kubernetes/kops/pull/11481)
* upup: gcetasks: force send AutoCreateSubnetworks field when set to false [@nicktrav](https://github.com/nicktrav) [#11457](https://github.com/kubernetes/kops/pull/11457)
* Add kOps and k8s 1.21 to alpha channel [@MoShitrit](https://github.com/MoShitrit) [#11482](https://github.com/kubernetes/kops/pull/11482)
* Reduce kOps supported version range [@johngmyers](https://github.com/johngmyers) [#11485](https://github.com/kubernetes/kops/pull/11485)
* More release process documentation improvements [@johngmyers](https://github.com/johngmyers) [#11434](https://github.com/kubernetes/kops/pull/11434)
* Set the test cluster-tag [@olemarkus](https://github.com/olemarkus) [#11487](https://github.com/kubernetes/kops/pull/11487)
* Set canonical location for downloads to artifacts.k8s.io [@hakman](https://github.com/hakman) [#11486](https://github.com/kubernetes/kops/pull/11486)
* [AWS CCM] Permission to create SA token [@nckturner](https://github.com/nckturner) [#11368](https://github.com/kubernetes/kops/pull/11368)
* Add link to release notes on first beta release [@johngmyers](https://github.com/johngmyers) [#11488](https://github.com/kubernetes/kops/pull/11488)
* Remove etcd-manager certificate expiration advisory [@hakman](https://github.com/hakman) [#11480](https://github.com/kubernetes/kops/pull/11480)
* Adjust deprecation announcements [@johngmyers](https://github.com/johngmyers) [#11489](https://github.com/kubernetes/kops/pull/11489)
* Update cert-manager [@olemarkus](https://github.com/olemarkus) [#11493](https://github.com/kubernetes/kops/pull/11493)
* Set priorityClassName on critical addons [@olemarkus](https://github.com/olemarkus) [#11495](https://github.com/kubernetes/kops/pull/11495)
* fix(coredns/rbac): add permission to list and watch endpointslices [@nettoclaudio](https://github.com/nettoclaudio) [#11459](https://github.com/kubernetes/kops/pull/11459)
* bump aws lb controller to 2.2.0 [@olemarkus](https://github.com/olemarkus) [#11502](https://github.com/kubernetes/kops/pull/11502)
* Aws lb scenario fix flags [@olemarkus](https://github.com/olemarkus) [#11506](https://github.com/kubernetes/kops/pull/11506)
* AWS LB controller requires multiple subnets to work [@olemarkus](https://github.com/olemarkus) [#11507](https://github.com/kubernetes/kops/pull/11507)
* Cleanup some of the scenario scripts [@rifelpet](https://github.com/rifelpet) [#11508](https://github.com/kubernetes/kops/pull/11508)
* Include new pipeline job in the release branch process [@rifelpet](https://github.com/rifelpet) [#11509](https://github.com/kubernetes/kops/pull/11509)
* Spotinst: Update spotinst/ocean-controller to v1.0.75 [@liranp](https://github.com/liranp) [#11512](https://github.com/kubernetes/kops/pull/11512)
* Subsume StatusStore into fi.Cloud [@johngmyers](https://github.com/johngmyers) [#11498](https://github.com/kubernetes/kops/pull/11498)
* Split genkgo in two [@olemarkus](https://github.com/olemarkus) [#11519](https://github.com/kubernetes/kops/pull/11519)
* [DigitalOcean] [WIP] Increase droplet size for e2e tests [@srikiz](https://github.com/srikiz) [#11520](https://github.com/kubernetes/kops/pull/11520)
* Add initial support for configuring IPv6 with AWS [@hakman](https://github.com/hakman) [#11442](https://github.com/kubernetes/kops/pull/11442)
* Add default tags to LB controller and cilium eni resources [@olemarkus](https://github.com/olemarkus) [#11517](https://github.com/kubernetes/kops/pull/11517)
* Remove dead code in bootstrap script [@johngmyers](https://github.com/johngmyers) [#11521](https://github.com/kubernetes/kops/pull/11521)
* Set default fstype for ebs volumes to ext4 [@olemarkus](https://github.com/olemarkus) [#11525](https://github.com/kubernetes/kops/pull/11525)
* Skip feature tests for ebs csi e2e [@olemarkus](https://github.com/olemarkus) [#11530](https://github.com/kubernetes/kops/pull/11530)
* Update etcd_backup_restore_encryption.md [@aberenshtein](https://github.com/aberenshtein) [#11533](https://github.com/kubernetes/kops/pull/11533)
* Don't download nodeup if already in the AMI [@johngmyers](https://github.com/johngmyers) [#11524](https://github.com/kubernetes/kops/pull/11524)
* [addons/networking.cilium.io] enable prometheus scraping [@ulfox](https://github.com/ulfox) [#11514](https://github.com/kubernetes/kops/pull/11514)
* feat(openstack): enable configuration of servergroup affinities [@mitch000001](https://github.com/mitch000001) [#11531](https://github.com/kubernetes/kops/pull/11531)
* Update containerd to v1.4.6 [@hakman](https://github.com/hakman) [#11535](https://github.com/kubernetes/kops/pull/11535)
* Cleanup orphaned IAM service account roles in direct render [@johngmyers](https://github.com/johngmyers) [#11497](https://github.com/kubernetes/kops/pull/11497)
* Support terraform 0.12+'s filebase64() in json output [@rifelpet](https://github.com/rifelpet) [#11540](https://github.com/kubernetes/kops/pull/11540)
* Release images bundle instead of separate images [@hakman](https://github.com/hakman) [#11522](https://github.com/kubernetes/kops/pull/11522)
* Bump CoreDNS manifests to latest stable version 1.8.3 [@dntosas](https://github.com/dntosas) [#11500](https://github.com/kubernetes/kops/pull/11500)
* Run the tests requiring snapshotcontroller again [@olemarkus](https://github.com/olemarkus) [#11544](https://github.com/kubernetes/kops/pull/11544)
* Update CAS manifest [@olemarkus](https://github.com/olemarkus) [#11491](https://github.com/kubernetes/kops/pull/11491)
* Make events etcd cluster optional [@codablock](https://github.com/codablock) [#11330](https://github.com/kubernetes/kops/pull/11330)
* Add support for arbitrary terraform functions [@rifelpet](https://github.com/rifelpet) [#11542](https://github.com/kubernetes/kops/pull/11542)
* Add snapshot-controller [@olemarkus](https://github.com/olemarkus) [#10730](https://github.com/kubernetes/kops/pull/10730)
* Add etcd-server related tests [@hakman](https://github.com/hakman) [#11552](https://github.com/kubernetes/kops/pull/11552)
* Bump default cilium to 1.9.7 [@olemarkus](https://github.com/olemarkus) [#11554](https://github.com/kubernetes/kops/pull/11554)
* Document updating conformance is first stable minor release only [@johngmyers](https://github.com/johngmyers) [#11556](https://github.com/kubernetes/kops/pull/11556)
* Add hubble documentation [@olemarkus](https://github.com/olemarkus) [#11557](https://github.com/kubernetes/kops/pull/11557)
* Allow using insecure TLS for metrics-server with Kubernetes 1.19+ [@hakman](https://github.com/hakman) [#11559](https://github.com/kubernetes/kops/pull/11559)
* Add snapshot-controller [@olemarkus](https://github.com/olemarkus) [#11561](https://github.com/kubernetes/kops/pull/11561)
* Fix deletion of IAM roles and policies [@johngmyers](https://github.com/johngmyers) [#11558](https://github.com/kubernetes/kops/pull/11558)
* Allow Spotinst to use comma separated instance types [@hakman](https://github.com/hakman) [#11560](https://github.com/kubernetes/kops/pull/11560)
* Release notes for 1.21.0-beta.2 [@johngmyers](https://github.com/johngmyers) [#11570](https://github.com/kubernetes/kops/pull/11570)
* Set flags on AWS CCM mimicking KCM [@olemarkus](https://github.com/olemarkus) [#11566](https://github.com/kubernetes/kops/pull/11566)
* Enable cert-manager in the ebs csi e2e test [@olemarkus](https://github.com/olemarkus) [#11569](https://github.com/kubernetes/kops/pull/11569)
* Only allow deletion of snapshots owned by the cluster [@olemarkus](https://github.com/olemarkus) [#11571](https://github.com/kubernetes/kops/pull/11571)
* Avoid error when first creating VPC with IPv6 [@justinsb](https://github.com/justinsb) [#11575](https://github.com/kubernetes/kops/pull/11575)
* Improve some small issues with the release process [@hakman](https://github.com/hakman) [#11572](https://github.com/kubernetes/kops/pull/11572)
* Cleanup InstanceProfile only that have ownership tags in delete cluster [@h3poteto](https://github.com/h3poteto) [#11568](https://github.com/kubernetes/kops/pull/11568)
* Don't set the master address for aws ccm [@olemarkus](https://github.com/olemarkus) [#11582](https://github.com/kubernetes/kops/pull/11582)
* Enable reading shared config when possibly from CLI [@johngmyers](https://github.com/johngmyers) [#11387](https://github.com/kubernetes/kops/pull/11387)
* Only update kubeconfig user when we have user info [@justinsb](https://github.com/justinsb) [#11584](https://github.com/kubernetes/kops/pull/11584)
* Add release note for AWS shared config [@johngmyers](https://github.com/johngmyers) [#11585](https://github.com/kubernetes/kops/pull/11585)
* Use latest CI build instead of building in the test [@olemarkus](https://github.com/olemarkus) [#11588](https://github.com/kubernetes/kops/pull/11588)
* Remove unused files [@johngmyers](https://github.com/johngmyers) [#11591](https://github.com/kubernetes/kops/pull/11591)
* Use the downloaded kops version for awslbc test [@olemarkus](https://github.com/olemarkus) [#11593](https://github.com/kubernetes/kops/pull/11593)
* Use the OnDelete updateStrategy for AWS VPC CNI DaemonSet [@johngmyers](https://github.com/johngmyers) [#11590](https://github.com/kubernetes/kops/pull/11590)
* Update Calico to v3.19.1 [@hakman](https://github.com/hakman) [#11594](https://github.com/kubernetes/kops/pull/11594)
* Cleanup Docs [@hakman](https://github.com/hakman) [#11595](https://github.com/kubernetes/kops/pull/11595)
* First addon operator integration: CoreDNS [@justinsb](https://github.com/justinsb) [#9374](https://github.com/kubernetes/kops/pull/9374)
* Add a note about NTH Queue Process mode [@olemarkus](https://github.com/olemarkus) [#11600](https://github.com/kubernetes/kops/pull/11600)
* Enable AWS EBS CSI driver by default [@olemarkus](https://github.com/olemarkus) [#11605](https://github.com/kubernetes/kops/pull/11605)
* Add documentation about snapshot-controller [@olemarkus](https://github.com/olemarkus) [#11606](https://github.com/kubernetes/kops/pull/11606)
* Convert all indents to spaces in node bootstrap script [@hakman](https://github.com/hakman) [#11611](https://github.com/kubernetes/kops/pull/11611)
* Use version marker for kops upgrade scenario [@olemarkus](https://github.com/olemarkus) [#11612](https://github.com/kubernetes/kops/pull/11612)
* Add init image field for Amazon VPC CNI [@ryan-dyer](https://github.com/ryan-dyer) [#11602](https://github.com/kubernetes/kops/pull/11602)
* Add to release process documentation [@johngmyers](https://github.com/johngmyers) [#11581](https://github.com/kubernetes/kops/pull/11581)
* Change toolbox template flag for consistency [@johngmyers](https://github.com/johngmyers) [#11616](https://github.com/kubernetes/kops/pull/11616)
* Fix duplicate CopyFile tasks [@johngmyers](https://github.com/johngmyers) [#11619](https://github.com/kubernetes/kops/pull/11619)
* Don't stage kops as file assets [@johngmyers](https://github.com/johngmyers) [#11620](https://github.com/kubernetes/kops/pull/11620)
* Dump all CP node logs to artifacts [@olemarkus](https://github.com/olemarkus) [#11615](https://github.com/kubernetes/kops/pull/11615)
* Simplify release steps [@johngmyers](https://github.com/johngmyers) [#11624](https://github.com/kubernetes/kops/pull/11624)
* Remove debug code and copy kops to PATH [@olemarkus](https://github.com/olemarkus) [#11625](https://github.com/kubernetes/kops/pull/11625)
* Update Go to v1.16.4 [@hakman](https://github.com/hakman) [#11626](https://github.com/kubernetes/kops/pull/11626)
* Add "kops get assets" command [@johngmyers](https://github.com/johngmyers) [#11617](https://github.com/kubernetes/kops/pull/11617)
* Set lifecycle on WarmPool task [@johngmyers](https://github.com/johngmyers) [#11618](https://github.com/kubernetes/kops/pull/11618)
* Label issue types in issue templates [@johngmyers](https://github.com/johngmyers) [#11637](https://github.com/kubernetes/kops/pull/11637)
* Remove fallback support for legacy IAM [@johngmyers](https://github.com/johngmyers) [#11641](https://github.com/kubernetes/kops/pull/11641)
* Rename CopyDockerImage to CopyImage [@johngmyers](https://github.com/johngmyers) [#11640](https://github.com/kubernetes/kops/pull/11640)
* Update the service account issuer discovery documentation [@olemarkus](https://github.com/olemarkus) [#11642](https://github.com/kubernetes/kops/pull/11642)
* Require all HasLifecycle tasks to have lifecycle set  [@johngmyers](https://github.com/johngmyers) [#11650](https://github.com/kubernetes/kops/pull/11650)
* Consolidate CSI livenessprobe images for multi-arch support [@rifelpet](https://github.com/rifelpet) [#11652](https://github.com/kubernetes/kops/pull/11652)
* Protokube needs dns-controller IAM permissions [@johngmyers](https://github.com/johngmyers) [#11645](https://github.com/kubernetes/kops/pull/11645)
* Remove docs on static addons [@olemarkus](https://github.com/olemarkus) [#11653](https://github.com/kubernetes/kops/pull/11653)
* Skip some steps if not doing cluster lifecycle [@johngmyers](https://github.com/johngmyers) [#11657](https://github.com/kubernetes/kops/pull/11657)
* Fix detection of virtual-hosted-style S3 urls in us-east-1 [@johngmyers](https://github.com/johngmyers) [#11655](https://github.com/kubernetes/kops/pull/11655)
* Promote channel alpha to stable [@johngmyers](https://github.com/johngmyers) [#11658](https://github.com/kubernetes/kops/pull/11658)
* Bump default cilium to 1.10 [@olemarkus](https://github.com/olemarkus) [#11659](https://github.com/kubernetes/kops/pull/11659)
* [Digital Ocean] Code cleanup with no functional modifications [@srikiz](https://github.com/srikiz) [#11592](https://github.com/kubernetes/kops/pull/11592)
* Fix jwks object path in S3 for IRSA [@h3poteto](https://github.com/h3poteto) [#11649](https://github.com/kubernetes/kops/pull/11649)
* Use version marker for kops ab scenario [@olemarkus](https://github.com/olemarkus) [#11648](https://github.com/kubernetes/kops/pull/11648)
* Kubetest2 scenario script cleanup [@rifelpet](https://github.com/rifelpet) [#11664](https://github.com/kubernetes/kops/pull/11664)
* Add more lifecycles to HasLifecycle tasks [@rifelpet](https://github.com/rifelpet) [#11666](https://github.com/kubernetes/kops/pull/11666)
* Set lifecycle on Droplet task [@johngmyers](https://github.com/johngmyers) [#11665](https://github.com/kubernetes/kops/pull/11665)
* Don't describe CloudLabels as being AWS-specific [@johngmyers](https://github.com/johngmyers) [#11667](https://github.com/kubernetes/kops/pull/11667)
* Move common stuff in e2e scenarios to common.sh [@olemarkus](https://github.com/olemarkus) [#11668](https://github.com/kubernetes/kops/pull/11668)
* Fix kubetest2 upgrade scripts [@rifelpet](https://github.com/rifelpet) [#11670](https://github.com/kubernetes/kops/pull/11670)
* Clean up straggling autogenerated code [@johngmyers](https://github.com/johngmyers) [#11671](https://github.com/kubernetes/kops/pull/11671)
* Remove dead code [@johngmyers](https://github.com/johngmyers) [#11672](https://github.com/kubernetes/kops/pull/11672)
* Explicitly set kubeconfig flag where we want to use it [@olemarkus](https://github.com/olemarkus) [#11676](https://github.com/kubernetes/kops/pull/11676)
* Add support for Docker v20.10.7 [@hakman](https://github.com/hakman) [#11674](https://github.com/kubernetes/kops/pull/11674)
* Use release markers instead of releases [@olemarkus](https://github.com/olemarkus) [#11679](https://github.com/kubernetes/kops/pull/11679)
* Drop trailing slash from oidc issuer [@olemarkus](https://github.com/olemarkus) [#11682](https://github.com/kubernetes/kops/pull/11682)
* Make Lifecycle field non-pointer [@johngmyers](https://github.com/johngmyers) [#11673](https://github.com/kubernetes/kops/pull/11673)
* Update Go to v1.16.5 [@hakman](https://github.com/hakman) [#11686](https://github.com/kubernetes/kops/pull/11686)
* Fix set-version leaving backup files with "-e" suffix [@johngmyers](https://github.com/johngmyers) [#11691](https://github.com/kubernetes/kops/pull/11691)
* Release notes for 1.21.0-beta.3 [@johngmyers](https://github.com/johngmyers) [#11694](https://github.com/kubernetes/kops/pull/11694)
* Update release process documentation [@johngmyers](https://github.com/johngmyers) [#11695](https://github.com/kubernetes/kops/pull/11695)
* Set IMDSv2 on by default for nodes and apiservers [@olemarkus](https://github.com/olemarkus) [#11329](https://github.com/kubernetes/kops/pull/11329)
* Deprecate old OS versions [@johngmyers](https://github.com/johngmyers) [#11696](https://github.com/kubernetes/kops/pull/11696)
* Fix panic in dryrun report [@johngmyers](https://github.com/johngmyers) [#11698](https://github.com/kubernetes/kops/pull/11698)
* Add options for configuring IPv4 and IPv6 support with Calico [@hakman](https://github.com/hakman) [#11688](https://github.com/kubernetes/kops/pull/11688)
* add e2e scenario script for testing cilium connectivity [@olemarkus](https://github.com/olemarkus) [#11697](https://github.com/kubernetes/kops/pull/11697)
* Fix copying of images from docker.io [@johngmyers](https://github.com/johngmyers) [#11656](https://github.com/kubernetes/kops/pull/11656)
* Fix the CSI EBS DS CRB. [@olemarkus](https://github.com/olemarkus) [#11701](https://github.com/kubernetes/kops/pull/11701)
* Use v1 certificate for LB controller [@olemarkus](https://github.com/olemarkus) [#11703](https://github.com/kubernetes/kops/pull/11703)
* Move asset copying out of apply_cluster [@johngmyers](https://github.com/johngmyers) [#11700](https://github.com/kubernetes/kops/pull/11700)
* Remove documentation of legacy IAM permissions [@johngmyers](https://github.com/johngmyers) [#11706](https://github.com/kubernetes/kops/pull/11706)
* Add some tests around channel adding needs-update annotation [@olemarkus](https://github.com/olemarkus) [#11598](https://github.com/kubernetes/kops/pull/11598)
* Update kube-router to v1.2.3 [@hakman](https://github.com/hakman) [#11124](https://github.com/kubernetes/kops/pull/11124)
* Create document on asset repositories [@johngmyers](https://github.com/johngmyers) [#11654](https://github.com/kubernetes/kops/pull/11654)
* Make relnotes match the new max hop limit IMDS behaviour [@olemarkus](https://github.com/olemarkus) [#11702](https://github.com/kubernetes/kops/pull/11702)
* Add proxy envs to calico to make possible usage of AWS source destination check [@DOboznyi](https://github.com/DOboznyi) [#11709](https://github.com/kubernetes/kops/pull/11709)
* Update controller-runtime to v0.9.0 [@hakman](https://github.com/hakman) [#11713](https://github.com/kubernetes/kops/pull/11713)
* Generate AWSEBSCSIDriver model only when using AWS [@hakman](https://github.com/hakman) [#11716](https://github.com/kubernetes/kops/pull/11716)
* Make AWS EBS CSI Driver default as of k8s 1.22 [@olemarkus](https://github.com/olemarkus) [#11721](https://github.com/kubernetes/kops/pull/11721)
* Use quay images for cilium [@olemarkus](https://github.com/olemarkus) [#11722](https://github.com/kubernetes/kops/pull/11722)
* Allow master to touch volumes tagged with kubernetes.io/cluster/<clusterName>:owned [@wongma7](https://github.com/wongma7) [#11729](https://github.com/kubernetes/kops/pull/11729)
* Update release branch docs with kubetest2 presubmit job [@rifelpet](https://github.com/rifelpet) [#11732](https://github.com/kubernetes/kops/pull/11732)
* Perform ClusterCIDR and ServiceClusterIPRange assignments for IPv6 [@johngmyers](https://github.com/johngmyers) [#11724](https://github.com/kubernetes/kops/pull/11724)
* Spotinst: Support for API Load Balancer with AWS/NLB [@liranp](https://github.com/liranp) [#11604](https://github.com/kubernetes/kops/pull/11604)
* Add support for setting latest k8s in ab scenario [@olemarkus](https://github.com/olemarkus) [#11735](https://github.com/kubernetes/kops/pull/11735)
* Deprecate CloudFormation support [@johngmyers](https://github.com/johngmyers) [#11630](https://github.com/kubernetes/kops/pull/11630)
* Calculate IPv6 subnet CIDR based on cluster CIDR [@hakman](https://github.com/hakman) [#11523](https://github.com/kubernetes/kops/pull/11523)
* Only warm-pull images used by the CSI DS [@olemarkus](https://github.com/olemarkus) [#11734](https://github.com/kubernetes/kops/pull/11734)
* Remove k8s-upgrade script as upgrade-ab is now used instead [@olemarkus](https://github.com/olemarkus) [#11738](https://github.com/kubernetes/kops/pull/11738)
* Add small note about rotating cluster after backup restore [@olemarkus](https://github.com/olemarkus) [#11733](https://github.com/kubernetes/kops/pull/11733)
* Make forwardToKubeDNS work in the NodeLocal DNSCache template [@ederst](https://github.com/ederst) [#11743](https://github.com/kubernetes/kops/pull/11743)
* Add test scenario for if channels is able to delete dangling resources [@olemarkus](https://github.com/olemarkus) [#11739](https://github.com/kubernetes/kops/pull/11739)
* Remove InstanceGroup from NodeupModelContext [@johngmyers](https://github.com/johngmyers) [#9294](https://github.com/kubernetes/kops/pull/9294)
* Refactor keypair code in preparation for secret rotation [@johngmyers](https://github.com/johngmyers) [#11219](https://github.com/kubernetes/kops/pull/11219)
* Remove unused field [@johngmyers](https://github.com/johngmyers) [#11749](https://github.com/kubernetes/kops/pull/11749)
* Hyperlink Sprig reference [@OutdatedVersion](https://github.com/OutdatedVersion) [#11730](https://github.com/kubernetes/kops/pull/11730)
* Compare OpenStack security groups deterministically [@ederst](https://github.com/ederst) [#11741](https://github.com/kubernetes/kops/pull/11741)
* Don't set Subnet dependency on AmazonIPv6CIDR for shared VPCs [@hakman](https://github.com/hakman) [#11752](https://github.com/kubernetes/kops/pull/11752)
* Set BindAddress appropriately when in IPv6-only mode [@johngmyers](https://github.com/johngmyers) [#11737](https://github.com/kubernetes/kops/pull/11737)
* Add --ipv6 experimental cli flag [@hakman](https://github.com/hakman) [#11629](https://github.com/kubernetes/kops/pull/11629)
* Don't restrict nodeup download to IPv4 [@johngmyers](https://github.com/johngmyers) [#11755](https://github.com/kubernetes/kops/pull/11755)
* Cilium: disable masquerade by default when in ENI IPAM mode [@johngmyers](https://github.com/johngmyers) [#11753](https://github.com/kubernetes/kops/pull/11753)
* Set default ClusterCIDR through the PodCIDR [@johngmyers](https://github.com/johngmyers) [#11756](https://github.com/kubernetes/kops/pull/11756)
* Enable IPv6 support for Cilium [@johngmyers](https://github.com/johngmyers) [#11754](https://github.com/kubernetes/kops/pull/11754)
* Allow unsetting fields from the command line [@johngmyers](https://github.com/johngmyers) [#11745](https://github.com/kubernetes/kops/pull/11745)
* Adjustments to SpecOverride [@johngmyers](https://github.com/johngmyers) [#11761](https://github.com/kubernetes/kops/pull/11761)
* Make the AdminAccess default inclusive of IPv6 [@johngmyers](https://github.com/johngmyers) [#11763](https://github.com/kubernetes/kops/pull/11763)
* Default the NodeCIDRMaskSize appropriately for IPv6 [@johngmyers](https://github.com/johngmyers) [#11762](https://github.com/kubernetes/kops/pull/11762)
* Simplify Calico IPv6 configuration [@johngmyers](https://github.com/johngmyers) [#11725](https://github.com/kubernetes/kops/pull/11725)
* Fix typo in IRSA docs [@yurrriq](https://github.com/yurrriq) [#11770](https://github.com/kubernetes/kops/pull/11770)
* Fix typo in populate_instancegroup_spec.go [@yurrriq](https://github.com/yurrriq) [#11769](https://github.com/kubernetes/kops/pull/11769)
* fix enable default SC when EBS driver is not installed [@olemarkus](https://github.com/olemarkus) [#11771](https://github.com/kubernetes/kops/pull/11771)
* Set containerd config on nodeup.Config instead of clusterspec [@olemarkus](https://github.com/olemarkus) [#11750](https://github.com/kubernetes/kops/pull/11750)
* Make it easy to run scenarios with irsa enabled [@olemarkus](https://github.com/olemarkus) [#11758](https://github.com/kubernetes/kops/pull/11758)
* Trim unnecessary paths from worker node IAM [@johngmyers](https://github.com/johngmyers) [#11775](https://github.com/kubernetes/kops/pull/11775)
* Allocate smaller IPv6 PodCIDRs by default [@johngmyers](https://github.com/johngmyers) [#11772](https://github.com/kubernetes/kops/pull/11772)
* Update github.com/spf13/viper to v1.8.0 [@hakman](https://github.com/hakman) [#11777](https://github.com/kubernetes/kops/pull/11777)
* [cni/cilium] Add support for additional config options [@dntosas](https://github.com/dntosas) [#11678](https://github.com/kubernetes/kops/pull/11678)
* Bump the cas addon version. [@olemarkus](https://github.com/olemarkus) [#11780](https://github.com/kubernetes/kops/pull/11780)
* Also set haveUserInfo=true in case --user was provided in "kops export kubecfg" [@codablock](https://github.com/codablock) [#11778](https://github.com/kubernetes/kops/pull/11778)
* Don't try to build etcd-manager secrets for cilium twice [@olemarkus](https://github.com/olemarkus) [#11764](https://github.com/kubernetes/kops/pull/11764)
* [addons] Introduce NodeProblemDetector [@dntosas](https://github.com/dntosas) [#11381](https://github.com/kubernetes/kops/pull/11381)
* Enable ability to use IRSA for cluster autoscaler [@olemarkus](https://github.com/olemarkus) [#11748](https://github.com/kubernetes/kops/pull/11748)
* Allow using IRSA for EBS CSI Driver [@olemarkus](https://github.com/olemarkus) [#11747](https://github.com/kubernetes/kops/pull/11747)
* Delete all files in the provided discoveryStore on cluster deletion [@olemarkus](https://github.com/olemarkus) [#11791](https://github.com/kubernetes/kops/pull/11791)
* Release notes for 1.20.2 [@justinsb](https://github.com/justinsb) [#11804](https://github.com/kubernetes/kops/pull/11804)
* Update alpha channel k8s versions and ec2 ami base image [@MoShitrit](https://github.com/MoShitrit) [#11803](https://github.com/kubernetes/kops/pull/11803)
* Seed the random number generator on AWS  [@johngmyers](https://github.com/johngmyers) [#11789](https://github.com/kubernetes/kops/pull/11789)
* Upgrade AWS CNI to latest release 1.8.0 [@MoShitrit](https://github.com/MoShitrit) [#11805](https://github.com/kubernetes/kops/pull/11805)
* bump the version of gophercloud [@cardoe](https://github.com/cardoe) [#11788](https://github.com/kubernetes/kops/pull/11788)
* Allow "kops create keypair" to stage next CA cert [@johngmyers](https://github.com/johngmyers) [#11252](https://github.com/kubernetes/kops/pull/11252)
* Reduce policy size [@olemarkus](https://github.com/olemarkus) [#11814](https://github.com/kubernetes/kops/pull/11814)
* Fix lbc permissions [@olemarkus](https://github.com/olemarkus) [#11815](https://github.com/kubernetes/kops/pull/11815)
* doc: remove `brew switch` ref and simplify version bump [@chenrui333](https://github.com/chenrui333) [#11817](https://github.com/kubernetes/kops/pull/11817)
* brew: remove kops.rb [@chenrui333](https://github.com/chenrui333) [#11819](https://github.com/kubernetes/kops/pull/11819)
* Split out get, describe, and delete keypairs commands [@johngmyers](https://github.com/johngmyers) [#11820](https://github.com/kubernetes/kops/pull/11820)
* Include multiple cluster CAs in trust stores [@johngmyers](https://github.com/johngmyers) [#11809](https://github.com/kubernetes/kops/pull/11809)
* Fix validating presence of AWS EBS CSI [@olemarkus](https://github.com/olemarkus) [#11795](https://github.com/kubernetes/kops/pull/11795)
* Pre-pull all container images used by components and addons [@hakman](https://github.com/hakman) [#11717](https://github.com/kubernetes/kops/pull/11717)
* skip flaking ebs csi flakes [@olemarkus](https://github.com/olemarkus) [#11821](https://github.com/kubernetes/kops/pull/11821)
* Set EnableExternalCloudController to true by default [@hakman](https://github.com/hakman) [#11825](https://github.com/kubernetes/kops/pull/11825)
* Put versioned API of cluster into state store [@johngmyers](https://github.com/johngmyers) [#9229](https://github.com/kubernetes/kops/pull/9229)
* Support creating new service-account keypairs [@johngmyers](https://github.com/johngmyers) [#11822](https://github.com/kubernetes/kops/pull/11822)
* Add support for logging-format option (text/json) [@dntosas](https://github.com/dntosas) [#11583](https://github.com/kubernetes/kops/pull/11583)
* Add back createvolume to master + bump ebs driver [@olemarkus](https://github.com/olemarkus) [#11811](https://github.com/kubernetes/kops/pull/11811)
* Improve the output of 'kops get keypairs'  [@johngmyers](https://github.com/johngmyers) [#11823](https://github.com/kubernetes/kops/pull/11823)
* Fix kOps version for managed flag on cert-manager [@djablonski-moia](https://github.com/djablonski-moia) [#11828](https://github.com/kubernetes/kops/pull/11828)
* Run scenarios as presubmit tests [@olemarkus](https://github.com/olemarkus) [#11801](https://github.com/kubernetes/kops/pull/11801)
* Include multiple CA certs in exported kubeconfigs [@johngmyers](https://github.com/johngmyers) [#11831](https://github.com/kubernetes/kops/pull/11831)
* Remove support for importing and converting kubeup clusters [@johngmyers](https://github.com/johngmyers) [#11824](https://github.com/kubernetes/kops/pull/11824)
* Ignore failing tests in upgrade scenario [@rifelpet](https://github.com/rifelpet) [#11832](https://github.com/kubernetes/kops/pull/11832)
* Set priority class for AWS CCM addon [@hakman](https://github.com/hakman) [#11834](https://github.com/kubernetes/kops/pull/11834)
* Limit concurrency of asset copy tasks  [@johngmyers](https://github.com/johngmyers) [#11708](https://github.com/kubernetes/kops/pull/11708)
* Add 'kops promote keypair' command [@johngmyers](https://github.com/johngmyers) [#11835](https://github.com/kubernetes/kops/pull/11835)
* Kubetest2 fix periodic end to end tests [@olemarkus](https://github.com/olemarkus) [#11838](https://github.com/kubernetes/kops/pull/11838)
* Kubetest2 - Add --skip-regex logic [@rifelpet](https://github.com/rifelpet) [#11841](https://github.com/kubernetes/kops/pull/11841)
* Fix skip regex for ebs csi test [@olemarkus](https://github.com/olemarkus) [#11840](https://github.com/kubernetes/kops/pull/11840)
* Mark nodes NeedsUpdate when keys they use change  [@johngmyers](https://github.com/johngmyers) [#11833](https://github.com/kubernetes/kops/pull/11833)
* Completely remove EnableExternalCloudController feature flag [@hakman](https://github.com/hakman) [#11839](https://github.com/kubernetes/kops/pull/11839)
* Only set default --skip-regex if it hasn't been set [@rifelpet](https://github.com/rifelpet) [#11842](https://github.com/kubernetes/kops/pull/11842)
* Clarify the limitations of Azure DNS support [@kenji-cloudnatix](https://github.com/kenji-cloudnatix) [#11844](https://github.com/kubernetes/kops/pull/11844)
* Refactor kube-controller-manager secrets [@johngmyers](https://github.com/johngmyers) [#11847](https://github.com/kubernetes/kops/pull/11847)
* Escape --skip-regex pattern [@rifelpet](https://github.com/rifelpet) [#11851](https://github.com/kubernetes/kops/pull/11851)
* Make aws-cni config more flexible and generalized [@MoShitrit](https://github.com/MoShitrit) [#11816](https://github.com/kubernetes/kops/pull/11816)
* Weaken some interfaces [@johngmyers](https://github.com/johngmyers) [#11837](https://github.com/kubernetes/kops/pull/11837)
* Handle containerExec hooks when using containerd [@hakman](https://github.com/hakman) [#11852](https://github.com/kubernetes/kops/pull/11852)
* Improve image copying [@johngmyers](https://github.com/johngmyers) [#11854](https://github.com/kubernetes/kops/pull/11854)
* Update helm to v3.6.1 [@olemarkus](https://github.com/olemarkus) [#11860](https://github.com/kubernetes/kops/pull/11860)
* Update CNI plugins to v0.9.1 [@hakman](https://github.com/hakman) [#11846](https://github.com/kubernetes/kops/pull/11846)
* Don't include irrelevant bootstrap addons [@johngmyers](https://github.com/johngmyers) [#11861](https://github.com/kubernetes/kops/pull/11861)
* Remove obsolete Spotinst manifest [@johngmyers](https://github.com/johngmyers) [#11862](https://github.com/kubernetes/kops/pull/11862)
* Enable cross-subnet mode with Calico by default [@hakman](https://github.com/hakman) [#11810](https://github.com/kubernetes/kops/pull/11810)
* Fix dryrun cluster creation [@johngmyers](https://github.com/johngmyers) [#11863](https://github.com/kubernetes/kops/pull/11863)
* Push alpha channel to stable [@MoShitrit](https://github.com/MoShitrit) [#11864](https://github.com/kubernetes/kops/pull/11864)
* Add a note about running update-expected when updating base AMI [@MoShitrit](https://github.com/MoShitrit) [#11865](https://github.com/kubernetes/kops/pull/11865)
* Make it simpler to spot missing files in integration tests [@olemarkus](https://github.com/olemarkus) [#11866](https://github.com/kubernetes/kops/pull/11866)
* fix: broken link [@choeffer](https://github.com/choeffer) [#11793](https://github.com/kubernetes/kops/pull/11793)
* Decrease default values for net.ipv4.tcp_rmem and net.ipv4.tcp_wmem [@hakman](https://github.com/hakman) [#11868](https://github.com/kubernetes/kops/pull/11868)
* Remove version from addons [@hakman](https://github.com/hakman) [#11867](https://github.com/kubernetes/kops/pull/11867)
* Move most nodeup.Config data to config store [@johngmyers](https://github.com/johngmyers) [#11869](https://github.com/kubernetes/kops/pull/11869)
* Don't reconcile roles and policies if a profile is provided [@olemarkus](https://github.com/olemarkus) [#11836](https://github.com/kubernetes/kops/pull/11836)
* Use DualStack API NLB for IPv6 [@hakman](https://github.com/hakman) [#11870](https://github.com/kubernetes/kops/pull/11870)
* Simplify config server protocol [@johngmyers](https://github.com/johngmyers) [#11871](https://github.com/kubernetes/kops/pull/11871)
* Refactor etcd-client-cilium secrets [@johngmyers](https://github.com/johngmyers) [#11848](https://github.com/kubernetes/kops/pull/11848)
* Retain deleted keypairs [@johngmyers](https://github.com/johngmyers) [#11845](https://github.com/kubernetes/kops/pull/11845)
* Write config as ManagedFile [@johngmyers](https://github.com/johngmyers) [#11796](https://github.com/kubernetes/kops/pull/11796)
* Improve "kops distrust keypair" command [@johngmyers](https://github.com/johngmyers) [#11876](https://github.com/kubernetes/kops/pull/11876)
* Avoid spurious changes for ASG InstanceProtection and LT InstanceMonitoring [@hakman](https://github.com/hakman) [#11873](https://github.com/kubernetes/kops/pull/11873)
* Kubetest2 - set node-os-arch flag instead of skipping kubectl test on arm64 [@rifelpet](https://github.com/rifelpet) [#11879](https://github.com/kubernetes/kops/pull/11879)
* Improve completion for kops root command [@johngmyers](https://github.com/johngmyers) [#11880](https://github.com/kubernetes/kops/pull/11880)
* Spotinst: Update `spotinst/ocean-controller` to v1.0.76 [@liranp](https://github.com/liranp) [#11885](https://github.com/kubernetes/kops/pull/11885)
* support large/slow downloads [@aojea](https://github.com/aojea) [#11884](https://github.com/kubernetes/kops/pull/11884)
* Add support for darwin/arm64 on the client-side [@hakman](https://github.com/hakman) [#11883](https://github.com/kubernetes/kops/pull/11883)
* Refactor nodeup APIServer builder, part one  [@johngmyers](https://github.com/johngmyers) [#11872](https://github.com/kubernetes/kops/pull/11872)
* Allow rotation of etcd-clients-ca-cilium [@johngmyers](https://github.com/johngmyers) [#11877](https://github.com/kubernetes/kops/pull/11877)
* [DigitalOcean] Increase droplet size for e2e tests [@srikiz](https://github.com/srikiz) [#11887](https://github.com/kubernetes/kops/pull/11887)
* Set download timeout to 3 minutes [@hakman](https://github.com/hakman) [#11886](https://github.com/kubernetes/kops/pull/11886)
* Implement completion for "kops create keypair" [@johngmyers](https://github.com/johngmyers) [#11888](https://github.com/kubernetes/kops/pull/11888)
* Render managed files with Terraform [@johngmyers](https://github.com/johngmyers) [#9621](https://github.com/kubernetes/kops/pull/9621)
* Implement completion for "kops promote keypair" [@johngmyers](https://github.com/johngmyers) [#11892](https://github.com/kubernetes/kops/pull/11892)
* Fix nil-pointer dereference on dryrun [@johngmyers](https://github.com/johngmyers) [#11894](https://github.com/kubernetes/kops/pull/11894)
* Implement completion for "kops distrust keypair" [@johngmyers](https://github.com/johngmyers) [#11899](https://github.com/kubernetes/kops/pull/11899)
* Refactor etcd-clients-ca keyset for api-server [@johngmyers](https://github.com/johngmyers) [#11897](https://github.com/kubernetes/kops/pull/11897)
* Allow overriding the ServiceAccountIssuer for IRSA [@johngmyers](https://github.com/johngmyers) [#11853](https://github.com/kubernetes/kops/pull/11853)
* Remove unnecessary parameters from terraform finish methods [@rifelpet](https://github.com/rifelpet) [#11900](https://github.com/kubernetes/kops/pull/11900)
* Include GCP Project in terraform HCL2 output [@rifelpet](https://github.com/rifelpet) [#11901](https://github.com/kubernetes/kops/pull/11901)
* Use Cobra's built-in completion command  [@johngmyers](https://github.com/johngmyers) [#11905](https://github.com/kubernetes/kops/pull/11905)
* Refactor apiserver-aggregator-ca [@johngmyers](https://github.com/johngmyers) [#11906](https://github.com/kubernetes/kops/pull/11906)
* Add support for IPv6 addresses to dns-controller [@hakman](https://github.com/hakman) [#11907](https://github.com/kubernetes/kops/pull/11907)
* Improve "kops get keypairs" [@johngmyers](https://github.com/johngmyers) [#11904](https://github.com/kubernetes/kops/pull/11904)
* Release notes for 1.21.0 [@justinsb](https://github.com/justinsb) [#11910](https://github.com/kubernetes/kops/pull/11910)
* Update pause image to 3.5 [@rifelpet](https://github.com/rifelpet) [#11909](https://github.com/kubernetes/kops/pull/11909)
* Upgrade Cobra to 1.2.1 [@johngmyers](https://github.com/johngmyers) [#11912](https://github.com/kubernetes/kops/pull/11912)
* Capture logs from the containerd service [@hakman](https://github.com/hakman) [#11914](https://github.com/kubernetes/kops/pull/11914)
* Do not set both CIDR and IPv6CIDR on sg rules [@olemarkus](https://github.com/olemarkus) [#11915](https://github.com/kubernetes/kops/pull/11915)
* Remove unused test files from legacy IAM [@rifelpet](https://github.com/rifelpet) [#11918](https://github.com/kubernetes/kops/pull/11918)
* Reduce policy size further [@olemarkus](https://github.com/olemarkus) [#11843](https://github.com/kubernetes/kops/pull/11843)
* Set KOPS_RUN_TOO_NEW_VERSION in scenario scripts [@rifelpet](https://github.com/rifelpet) [#11923](https://github.com/kubernetes/kops/pull/11923)
* Update version support matrix for 1.21 [@johngmyers](https://github.com/johngmyers) [#11922](https://github.com/kubernetes/kops/pull/11922)
* Rename the "ca" keyset to "kubernetes-ca" [@johngmyers](https://github.com/johngmyers) [#11921](https://github.com/kubernetes/kops/pull/11921)
* Allow fsstore to be used for mock s3 rules [@olemarkus](https://github.com/olemarkus) [#11916](https://github.com/kubernetes/kops/pull/11916)
* Implement completion for "kops rolling-update cluster" [@johngmyers](https://github.com/johngmyers) [#11924](https://github.com/kubernetes/kops/pull/11924)
* Implement completion for "kops update cluster" [@johngmyers](https://github.com/johngmyers) [#11926](https://github.com/kubernetes/kops/pull/11926)
* Update the status of cloud providers [@johngmyers](https://github.com/johngmyers) [#11930](https://github.com/kubernetes/kops/pull/11930)
* Remove obsolete files [@johngmyers](https://github.com/johngmyers) [#11932](https://github.com/kubernetes/kops/pull/11932)
* Implement completion for validate and upgrade [@johngmyers](https://github.com/johngmyers) [#11927](https://github.com/kubernetes/kops/pull/11927)
* Continue if a single addon fails to be applied [@olemarkus](https://github.com/olemarkus) [#11933](https://github.com/kubernetes/kops/pull/11933)
* Remove unused golden files from manyaddons test [@olemarkus](https://github.com/olemarkus) [#11935](https://github.com/kubernetes/kops/pull/11935)
* Schedule certmanager webhook on control plane [@olemarkus](https://github.com/olemarkus) [#11934](https://github.com/kubernetes/kops/pull/11934)
* [Digital Ocean] Remove PrivateNetworking option in droplet since it's deprecated [@srikiz](https://github.com/srikiz) [#11936](https://github.com/kubernetes/kops/pull/11936)
* Run cert-manager cainjector on CP nodes as well [@olemarkus](https://github.com/olemarkus) [#11938](https://github.com/kubernetes/kops/pull/11938)
* Fix various CCM issues [@olemarkus](https://github.com/olemarkus) [#11939](https://github.com/kubernetes/kops/pull/11939)
* Add podPidsLimit / --pod-max-pids support [@uthark](https://github.com/uthark) [#11898](https://github.com/kubernetes/kops/pull/11898)
* Add log rotation for etcd-cilium.log [@hakman](https://github.com/hakman) [#11943](https://github.com/kubernetes/kops/pull/11943)
* [Digital Ocean] Modify error message when multiple zones are specified [@srikiz](https://github.com/srikiz) [#11944](https://github.com/kubernetes/kops/pull/11944)
* Fix bullet point rendering in state doc [@rothgar](https://github.com/rothgar) [#11948](https://github.com/kubernetes/kops/pull/11948)
* Implement some completion for "kops create cluster" [@johngmyers](https://github.com/johngmyers) [#11940](https://github.com/kubernetes/kops/pull/11940)
* check if the instance is under an asg [@olivierpilotte](https://github.com/olivierpilotte) [#11958](https://github.com/kubernetes/kops/pull/11958)
* Use etcd v3.5.0 for Kubernetes 1.22+ [@hakman](https://github.com/hakman) [#11941](https://github.com/kubernetes/kops/pull/11941)
* Unconditionally reenable KMS and Volume Limit tests [@rifelpet](https://github.com/rifelpet) [#11966](https://github.com/kubernetes/kops/pull/11966)
* Suppress usage for errors returned from RunE [@johngmyers](https://github.com/johngmyers) [#11969](https://github.com/kubernetes/kops/pull/11969)
* Implement completion for "kops create instancegroup"  [@johngmyers](https://github.com/johngmyers) [#11957](https://github.com/kubernetes/kops/pull/11957)
* Refactor keysets for etcd-manager [@johngmyers](https://github.com/johngmyers) [#11964](https://github.com/kubernetes/kops/pull/11964)
* Cilium etcd fixes [@olemarkus](https://github.com/olemarkus) [#11961](https://github.com/kubernetes/kops/pull/11961)
* Refactor service-account signing key [@johngmyers](https://github.com/johngmyers) [#11974](https://github.com/kubernetes/kops/pull/11974)
* Add "all" variants of key rotation commands [@johngmyers](https://github.com/johngmyers) [#11971](https://github.com/kubernetes/kops/pull/11971)
* Add documentation for keypair rotation [@johngmyers](https://github.com/johngmyers) [#11972](https://github.com/kubernetes/kops/pull/11972)
* Implement completion for delete commands [@johngmyers](https://github.com/johngmyers) [#11970](https://github.com/kubernetes/kops/pull/11970)
* Issue certs using CA KeypairID in NodeupConfig [@johngmyers](https://github.com/johngmyers) [#11975](https://github.com/kubernetes/kops/pull/11975)
* Stop writing the certificate-only keyset.yaml [@johngmyers](https://github.com/johngmyers) [#11977](https://github.com/kubernetes/kops/pull/11977)
* Provide more information on rotating secrets [@johngmyers](https://github.com/johngmyers) [#11978](https://github.com/kubernetes/kops/pull/11978)
* Spotinst: Update `spotinst/ocean-controller` to v1.0.77 [@liranp](https://github.com/liranp) [#11981](https://github.com/kubernetes/kops/pull/11981)
* [Digital Ocean] Fix sporadic volume detach error when volume is already detached [@srikiz](https://github.com/srikiz) [#11963](https://github.com/kubernetes/kops/pull/11963)
* Fix broken link to contributing [@MoShitrit](https://github.com/MoShitrit) [#11979](https://github.com/kubernetes/kops/pull/11979)
* Add "kops trust keypair" command [@johngmyers](https://github.com/johngmyers) [#11973](https://github.com/kubernetes/kops/pull/11973)
* Implement completion for "kops edit" commands [@johngmyers](https://github.com/johngmyers) [#11980](https://github.com/kubernetes/kops/pull/11980)
* Add missing IAM permissions to the NTH docs [@olemarkus](https://github.com/olemarkus) [#11984](https://github.com/kubernetes/kops/pull/11984)
* Implement completion for "kops export kubeconfig" [@johngmyers](https://github.com/johngmyers) [#11983](https://github.com/kubernetes/kops/pull/11983)
* Cobra cleanups [@johngmyers](https://github.com/johngmyers) [#11985](https://github.com/kubernetes/kops/pull/11985)
* Fix "kops export kubeconfig" [@johngmyers](https://github.com/johngmyers) [#11988](https://github.com/kubernetes/kops/pull/11988)
* Add region to aws lbc [@olemarkus](https://github.com/olemarkus) [#11990](https://github.com/kubernetes/kops/pull/11990)
* Move containerd config from cloudup to nodeup [@olemarkus](https://github.com/olemarkus) [#11986](https://github.com/kubernetes/kops/pull/11986)
* Change set and unset commands into flags on "kops edit cluster" [@johngmyers](https://github.com/johngmyers) [#11987](https://github.com/kubernetes/kops/pull/11987)
* Remove dead code [@johngmyers](https://github.com/johngmyers) [#11993](https://github.com/kubernetes/kops/pull/11993)
* Implement completion for "kops toolbox", part one [@johngmyers](https://github.com/johngmyers) [#11992](https://github.com/kubernetes/kops/pull/11992)
* hack/upload: avoid ACLs for GCS buckets with UBLA enabled [@spiffxp](https://github.com/spiffxp) [#11994](https://github.com/kubernetes/kops/pull/11994)
* Azure - support VMSS availability zones [@rifelpet](https://github.com/rifelpet) [#11962](https://github.com/kubernetes/kops/pull/11962)
* Upgrade aws-sdk-go [@rifelpet](https://github.com/rifelpet) [#11996](https://github.com/kubernetes/kops/pull/11996)
* remove references to kubernetes-release-dev [@spiffxp](https://github.com/spiffxp) [#11997](https://github.com/kubernetes/kops/pull/11997)
* Clean up extra spaces [@jayonlau](https://github.com/jayonlau) [#11989](https://github.com/kubernetes/kops/pull/11989)
* Verify CA keypair IDs for kops-controller-issued certs [@johngmyers](https://github.com/johngmyers) [#11982](https://github.com/kubernetes/kops/pull/11982)
* Use keypair IDs for non-kops-controller-issued worker node certs [@johngmyers](https://github.com/johngmyers) [#11998](https://github.com/kubernetes/kops/pull/11998)
* Update alpha channel with July k8s releases and bump Ubuntu EC2 AMI version [@MoShitrit](https://github.com/MoShitrit) [#12000](https://github.com/kubernetes/kops/pull/12000)
* Dedicated function for ccm permissons [@olemarkus](https://github.com/olemarkus) [#11991](https://github.com/kubernetes/kops/pull/11991)
* Add keypair rotation test scenario [@rifelpet](https://github.com/rifelpet) [#12001](https://github.com/kubernetes/kops/pull/12001)
* Fix file permissions on new keypair rotation test scenario [@rifelpet](https://github.com/rifelpet) [#12005](https://github.com/kubernetes/kops/pull/12005)
* Implement completion for "kops toolbox", part two [@johngmyers](https://github.com/johngmyers) [#11999](https://github.com/kubernetes/kops/pull/11999)
* Issue kubelet cert on apiserver nodes for k8s before 1.19 [@johngmyers](https://github.com/johngmyers) [#12002](https://github.com/kubernetes/kops/pull/12002)
* Refactor more kube-apiserver credentials [@johngmyers](https://github.com/johngmyers) [#12003](https://github.com/kubernetes/kops/pull/12003)
* Accommodate older destination kops versions in upgrade-ab scenario [@johngmyers](https://github.com/johngmyers) [#12008](https://github.com/kubernetes/kops/pull/12008)
* Fix kops binary references in keypair rotation scenario [@rifelpet](https://github.com/rifelpet) [#12009](https://github.com/kubernetes/kops/pull/12009)
* Don't provision SSH key by default on AWS [@johngmyers](https://github.com/johngmyers) [#12011](https://github.com/kubernetes/kops/pull/12011)
* Deprecate the Lyft CNI [@johngmyers](https://github.com/johngmyers) [#12010](https://github.com/kubernetes/kops/pull/12010)
* Remove apiserver's access to controller-manager secrets [@johngmyers](https://github.com/johngmyers) [#12006](https://github.com/kubernetes/kops/pull/12006)

## 1.22.0-alpha.2 to 1.22.0-beta.1

* Fix keypair rotation scenario kops binary [@rifelpet](https://github.com/rifelpet) [#12013](https://github.com/kubernetes/kops/pull/12013)
* Keypair rotation scenario - create report directory [@rifelpet](https://github.com/rifelpet) [#12015](https://github.com/kubernetes/kops/pull/12015)
* Release notes for 1.22.0-alpha.2 [@hakman](https://github.com/hakman) [#12014](https://github.com/kubernetes/kops/pull/12014)
* Clarify how cloud labels are used in getting started docs [@rifelpet](https://github.com/rifelpet) [#12017](https://github.com/kubernetes/kops/pull/12017)
* Add azure support for specifying a shared vpc [@rifelpet](https://github.com/rifelpet) [#12018](https://github.com/kubernetes/kops/pull/12018)
* Report unknown feature flags as such [@johngmyers](https://github.com/johngmyers) [#12020](https://github.com/kubernetes/kops/pull/12020)
* Promote AWS VPC CNI to stable [@johngmyers](https://github.com/johngmyers) [#12021](https://github.com/kubernetes/kops/pull/12021)
* keypair rotation scenario - fix base64 decoding [@rifelpet](https://github.com/rifelpet) [#12022](https://github.com/kubernetes/kops/pull/12022)
* Set vpc-id on aws lbc [@olemarkus](https://github.com/olemarkus) [#12023](https://github.com/kubernetes/kops/pull/12023)
* Add irsa support for node termination handler [@olemarkus](https://github.com/olemarkus) [#12024](https://github.com/kubernetes/kops/pull/12024)
* Return a clearer error when terraform is used on an unsupported provider [@rifelpet](https://github.com/rifelpet) [#11953](https://github.com/kubernetes/kops/pull/11953)
* Remove redundant call to addSnapshotPermissions [@olemarkus](https://github.com/olemarkus) [#12025](https://github.com/kubernetes/kops/pull/12025)
* Remove addons that no longer works [@olemarkus](https://github.com/olemarkus) [#12027](https://github.com/kubernetes/kops/pull/12027)
* Fix certificate bootstrap for non-kops-controller-bootstrap cloud providers [@johngmyers](https://github.com/johngmyers) [#12019](https://github.com/kubernetes/kops/pull/12019)
* Assert the correct number of kubeconfig CAs during keypair rotation [@rifelpet](https://github.com/rifelpet) [#12029](https://github.com/kubernetes/kops/pull/12029)
* keypair rotation - export new credentials between promote and distruts [@rifelpet](https://github.com/rifelpet) [#12032](https://github.com/kubernetes/kops/pull/12032)
* Update mkdocs dependencies to latest versions [@rifelpet](https://github.com/rifelpet) [#12031](https://github.com/kubernetes/kops/pull/12031)
* Implement completion for "kops get", part one [@johngmyers](https://github.com/johngmyers) [#12028](https://github.com/kubernetes/kops/pull/12028)
* Use kubeconfig for authentication and authorization as well [@johngmyers](https://github.com/johngmyers) [#12036](https://github.com/kubernetes/kops/pull/12036)
* Implement completion for "kops get", part two [@johngmyers](https://github.com/johngmyers) [#12039](https://github.com/kubernetes/kops/pull/12039)
* Remove unnecessary IAM permission [@johngmyers](https://github.com/johngmyers) [#12044](https://github.com/kubernetes/kops/pull/12044)
* Replace "kops describe keypair" with "kops get keypair -oyaml" [@johngmyers](https://github.com/johngmyers) [#12040](https://github.com/kubernetes/kops/pull/12040)
* Provision TLS server certs for controller-manager and scheduler [@johngmyers](https://github.com/johngmyers) [#12030](https://github.com/kubernetes/kops/pull/12030)
* Recommend kops 1.21.0 in alpha channel [@johngmyers](https://github.com/johngmyers) [#12051](https://github.com/kubernetes/kops/pull/12051)
* Promote channel alpha to stable [@johngmyers](https://github.com/johngmyers) [#12050](https://github.com/kubernetes/kops/pull/12050)
* Pull sshpubkey subcommands out of secrets [@johngmyers](https://github.com/johngmyers) [#12045](https://github.com/kubernetes/kops/pull/12045)
* Clean up "create secret" subcommands [@johngmyers](https://github.com/johngmyers) [#12055](https://github.com/kubernetes/kops/pull/12055)
* Clean up remaining secrets subcommands [@johngmyers](https://github.com/johngmyers) [#12056](https://github.com/kubernetes/kops/pull/12056)
* Use regional STS endpoint [@johngmyers](https://github.com/johngmyers) [#12043](https://github.com/kubernetes/kops/pull/12043)
* Use static pattern rules to reduce duplication [@johngmyers](https://github.com/johngmyers) [#12046](https://github.com/kubernetes/kops/pull/12046)
* Update aws-sdk-go to v1.40.10 [@hakman](https://github.com/hakman) [#12060](https://github.com/kubernetes/kops/pull/12060)
* Update controller-runtime to v0.9.5 [@hakman](https://github.com/hakman) [#12061](https://github.com/kubernetes/kops/pull/12061)
* Update containerd to v1.4.8 [@hakman](https://github.com/hakman) [#12059](https://github.com/kubernetes/kops/pull/12059)
* Cobra cleanups [@johngmyers](https://github.com/johngmyers) [#12063](https://github.com/kubernetes/kops/pull/12063)
* Update AWS CNI to v1.9.0 [@MoShitrit](https://github.com/MoShitrit) [#12065](https://github.com/kubernetes/kops/pull/12065)
* Update core-dns to v1.8.4 [@hakman](https://github.com/hakman) [#12062](https://github.com/kubernetes/kops/pull/12062)
* Cleanup various references to LaunchConfigurations [@rifelpet](https://github.com/rifelpet) [#12072](https://github.com/kubernetes/kops/pull/12072)
* Update containerd to v1.4.9 [@hakman](https://github.com/hakman) [#12073](https://github.com/kubernetes/kops/pull/12073)
* Fix cluster list action [@justinsb](https://github.com/justinsb) [#12075](https://github.com/kubernetes/kops/pull/12075)
* GCE: TargetPool should ignore Lifecycle field [@justinsb](https://github.com/justinsb) [#12079](https://github.com/kubernetes/kops/pull/12079)
* Update Ubuntu images to v20210720 [@hakman](https://github.com/hakman) [#12080](https://github.com/kubernetes/kops/pull/12080)
* Use SHA-256 for manifest hashes [@johngmyers](https://github.com/johngmyers) [#12087](https://github.com/kubernetes/kops/pull/12087)
* Update Calico to v3.20.0 [@hakman](https://github.com/hakman) [#12088](https://github.com/kubernetes/kops/pull/12088)
* Temporarily skip MetricsGrabber test [@rifelpet](https://github.com/rifelpet) [#12090](https://github.com/kubernetes/kops/pull/12090)
* Kubetest2 scenarios - provide absolute path to kops binary [@rifelpet](https://github.com/rifelpet) [#12091](https://github.com/kubernetes/kops/pull/12091)
* Add nth rebalance recommendation configs [@CheyiLin](https://github.com/CheyiLin) [#12083](https://github.com/kubernetes/kops/pull/12083)
* Also skip MetricsGrabber tests on 1.23 version marker [@rifelpet](https://github.com/rifelpet) [#12093](https://github.com/kubernetes/kops/pull/12093)
* Update Docker to v20.10.8 [@hakman](https://github.com/hakman) [#12096](https://github.com/kubernetes/kops/pull/12096)
* Update README.md [@youvegotmoxie](https://github.com/youvegotmoxie) [#12097](https://github.com/kubernetes/kops/pull/12097)
* Launch Openstack instances with config drive [@ederst](https://github.com/ederst) [#11784](https://github.com/kubernetes/kops/pull/11784)
* Add Option to Specify OpenStack Octavia Provider [@iGene](https://github.com/iGene) [#12049](https://github.com/kubernetes/kops/pull/12049)
* Update Kubernetes dependencies to 1.22.0 [@johngmyers](https://github.com/johngmyers) [#12101](https://github.com/kubernetes/kops/pull/12101)
* Skip GlusterFS tests [@hakman](https://github.com/hakman) [#12102](https://github.com/kubernetes/kops/pull/12102)
* Revert most of #12023 and keep awslbc on CP nodes [@olemarkus](https://github.com/olemarkus) [#12107](https://github.com/kubernetes/kops/pull/12107)
* Stop skipping MetricsGrabber tests on CI k8s versions [@rifelpet](https://github.com/rifelpet) [#12106](https://github.com/kubernetes/kops/pull/12106)
* Support Debian 11 Bullseye [@ReillyBrogan](https://github.com/ReillyBrogan) [#12108](https://github.com/kubernetes/kops/pull/12108)
* Enable IRSA for CCM [@olemarkus](https://github.com/olemarkus) [#11818](https://github.com/kubernetes/kops/pull/11818)
* In-line openstack loadbalancer feature detection [@rifelpet](https://github.com/rifelpet) [#12111](https://github.com/kubernetes/kops/pull/12111)
* Disallow the use of legacy addons [@hakman](https://github.com/hakman) [#12064](https://github.com/kubernetes/kops/pull/12064)
* Add release note for Debian 11 [@johngmyers](https://github.com/johngmyers) [#12115](https://github.com/kubernetes/kops/pull/12115)
* Remove UseServiceAccountIAM feature flag and rename feature to UseServiceAccountExternalPermissions [@olemarkus](https://github.com/olemarkus) [#12109](https://github.com/kubernetes/kops/pull/12109)
* Make metrics-server insecure if insecure is true [@olemarkus](https://github.com/olemarkus) [#12114](https://github.com/kubernetes/kops/pull/12114)
* Pass AZURE_ENVIRONMENT to nodeup [@rifelpet](https://github.com/rifelpet) [#12118](https://github.com/kubernetes/kops/pull/12118)
* Update addon manifests referencing RBAC v1beta1 [@rifelpet](https://github.com/rifelpet) [#12120](https://github.com/kubernetes/kops/pull/12120)
* Update dns-controller to use networking.k8s.io/v1 Ingress API [@rifelpet](https://github.com/rifelpet) [#12119](https://github.com/kubernetes/kops/pull/12119)
* Fix cases when the VPC doesn't exist yet [@mikesplain](https://github.com/mikesplain) [#12124](https://github.com/kubernetes/kops/pull/12124)
* Fix disabling unattended upgrades  [@olemarkus](https://github.com/olemarkus) [#12123](https://github.com/kubernetes/kops/pull/12123)
* Openstack: add port metadata [@mitch000001](https://github.com/mitch000001) [#12122](https://github.com/kubernetes/kops/pull/12122)
* Debian 11: python-apt is not available [@ReillyBrogan](https://github.com/ReillyBrogan) [#12130](https://github.com/kubernetes/kops/pull/12130)
* Bump CAS for 1.22 [@olemarkus](https://github.com/olemarkus) [#12133](https://github.com/kubernetes/kops/pull/12133)
* Set IRSA flag on scenarios where needed [@olemarkus](https://github.com/olemarkus) [#12135](https://github.com/kubernetes/kops/pull/12135)
* Enable ipv6 forwarding and router announcements [@olemarkus](https://github.com/olemarkus) [#12134](https://github.com/kubernetes/kops/pull/12134)
*  Bump Cert Manager for 1.22 [@olemarkus](https://github.com/olemarkus) [#12137](https://github.com/kubernetes/kops/pull/12137)
* Add e2e scenario for metrics-server [@olemarkus](https://github.com/olemarkus) [#12139](https://github.com/kubernetes/kops/pull/12139)
*  Bump cilium to 1.10.3 [@olemarkus](https://github.com/olemarkus) [#12141](https://github.com/kubernetes/kops/pull/12141)
* Bump alpha channel's k8s versions with August releases and add kOps version 1.22 along with k8s 1.22 [@MoShitrit](https://github.com/MoShitrit) [#12145](https://github.com/kubernetes/kops/pull/12145)
* Bump AWS CSI Driver to 1.2.0 [@olemarkus](https://github.com/olemarkus) [#12129](https://github.com/kubernetes/kops/pull/12129)
* Update Debian 11 first supported kOps version [@hakman](https://github.com/hakman) [#12144](https://github.com/kubernetes/kops/pull/12144)
* Correct list of beta cloud providers [@johngmyers](https://github.com/johngmyers) [#12116](https://github.com/kubernetes/kops/pull/12116)
* leverage proxy env variables [@aojea](https://github.com/aojea) [#12150](https://github.com/kubernetes/kops/pull/12150)
* Reconcile if managedFile is public or not [@olemarkus](https://github.com/olemarkus) [#12148](https://github.com/kubernetes/kops/pull/12148)
* Update Go to v1.16.7 [@hakman](https://github.com/hakman) [#12153](https://github.com/kubernetes/kops/pull/12153)
* Add API validation for subcomponents using API versions removed in k8s 1.22 [@johngmyers](https://github.com/johngmyers) [#12155](https://github.com/kubernetes/kops/pull/12155)
* Update kopeio test to test kopeio networking also [@justinsb](https://github.com/justinsb) [#12157](https://github.com/kubernetes/kops/pull/12157)
* Debian 11: Release AMIs use same AWS Owner ID as Buster [@ReillyBrogan](https://github.com/ReillyBrogan) [#12161](https://github.com/kubernetes/kops/pull/12161)
* Make metrics-server scenario executable [@rifelpet](https://github.com/rifelpet) [#12163](https://github.com/kubernetes/kops/pull/12163)
* Fix unbound variable in metrics server scenario [@rifelpet](https://github.com/rifelpet) [#12166](https://github.com/kubernetes/kops/pull/12166)
* Cilium wireguard support [@dntosas](https://github.com/dntosas),[@olemarkus](https://github.com/olemarkus) [#12158](https://github.com/kubernetes/kops/pull/12158)
* Fix scaling of metrics-server deployment in e2e scenario [@rifelpet](https://github.com/rifelpet) [#12168](https://github.com/kubernetes/kops/pull/12168)
* Fix parse channels error [@mikesplain](https://github.com/mikesplain) [#12171](https://github.com/kubernetes/kops/pull/12171)
* Incorrect url to the swift container [@yadneshk](https://github.com/yadneshk) [#12159](https://github.com/kubernetes/kops/pull/12159)
* Bump cert-manager to 1.5.1 [@olemarkus](https://github.com/olemarkus) [#12174](https://github.com/kubernetes/kops/pull/12174)
* Bump snapshot-controller [@olemarkus](https://github.com/olemarkus) [#12176](https://github.com/kubernetes/kops/pull/12176)
* [DigitalOcean] Code refactor to use the existing dnsprovider package [@srikiz](https://github.com/srikiz) [#12173](https://github.com/kubernetes/kops/pull/12173)
* Hardcode Flatcar containerd exec command [@hakman](https://github.com/hakman) [#12177](https://github.com/kubernetes/kops/pull/12177)
* Fix incorrect validation error message [@johngmyers](https://github.com/johngmyers) [#12181](https://github.com/kubernetes/kops/pull/12181)
* Fix MetricsGrabber skip regex condition [@rifelpet](https://github.com/rifelpet) [#12165](https://github.com/kubernetes/kops/pull/12165)
* Cleanup unused TerraformFmt feature flag [@rifelpet](https://github.com/rifelpet) [#12185](https://github.com/kubernetes/kops/pull/12185)
* Add option in Cluster Autoscaler AddOn for AWS EC2 Static instance list [@amitpd](https://github.com/amitpd) [#12187](https://github.com/kubernetes/kops/pull/12187)
* Bump cert-manager to 1.5.2 [@olemarkus](https://github.com/olemarkus) [#12189](https://github.com/kubernetes/kops/pull/12189)
* Consolidate statements in control plane IAM role [@rifelpet](https://github.com/rifelpet) [#12183](https://github.com/kubernetes/kops/pull/12183)
* Initial IPv6 support for GCE [@justinsb](https://github.com/justinsb) [#12170](https://github.com/kubernetes/kops/pull/12170)
* Release notes for 1.21.1 [@justinsb](https://github.com/justinsb) [#12195](https://github.com/kubernetes/kops/pull/12195)
* Warn if KOPS_ARCH is not a recognized value [@justinsb](https://github.com/justinsb) [#12194](https://github.com/kubernetes/kops/pull/12194)
* Add terraform outputs for OIDC provider ARN and issuer [@rifelpet](https://github.com/rifelpet) [#12197](https://github.com/kubernetes/kops/pull/12197)
* Script for iterating development on GCE [@justinsb](https://github.com/justinsb) [#12169](https://github.com/kubernetes/kops/pull/12169)
* Push alpha to stable [@MoShitrit](https://github.com/MoShitrit) [#12204](https://github.com/kubernetes/kops/pull/12204)
* Update terraform to 1.0.5 [@rifelpet](https://github.com/rifelpet) [#12203](https://github.com/kubernetes/kops/pull/12203)
* Support AWS LB access log configuration in cluster spec  [@h3poteto](https://github.com/h3poteto) [#12067](https://github.com/kubernetes/kops/pull/12067)
* Added the docs related to CPU/Memory resources override on weave [@diegocn](https://github.com/diegocn) [#12201](https://github.com/kubernetes/kops/pull/12201)
* Add ip addresses into kubelet certs [@olemarkus](https://github.com/olemarkus) [#12188](https://github.com/kubernetes/kops/pull/12188)
* Use ExternalDNS as drop-in for dns-controller [@olemarkus](https://github.com/olemarkus) [#11859](https://github.com/kubernetes/kops/pull/11859)
* Update all e2e scenario tests to use k8s 1.22 [@rifelpet](https://github.com/rifelpet) [#12209](https://github.com/kubernetes/kops/pull/12209)
* Fix terraform output for cluster names that begin with a digit [@rifelpet](https://github.com/rifelpet) [#12202](https://github.com/kubernetes/kops/pull/12202)

## 1.22.0-beta.1 to 1.22.0-beta.2

* Release 1.22.0-beta.1 [@johngmyers](https://github.com/johngmyers) [#12212](https://github.com/kubernetes/kops/pull/12212)
* [addons/metrics-server] Bump minor version [@dntosas](https://github.com/dntosas) [#12198](https://github.com/kubernetes/kops/pull/12198)
* fix parse semver [@zetaab](https://github.com/zetaab) [#12227](https://github.com/kubernetes/kops/pull/12227)
* Bump cert-manager to 1.5.3 [@olemarkus](https://github.com/olemarkus) [#12229](https://github.com/kubernetes/kops/pull/12229)
* check does iface has associations [@zetaab](https://github.com/zetaab) [#12232](https://github.com/kubernetes/kops/pull/12232)
* Bump node termination handler to 1.13.3 [@olemarkus](https://github.com/olemarkus) [#12237](https://github.com/kubernetes/kops/pull/12237)
* [addons/node-local-dns] Bump version and make image field configurable [@dntosas](https://github.com/dntosas) [#12219](https://github.com/kubernetes/kops/pull/12219)
* Set kube-apiserver as default logs container [@olemarkus](https://github.com/olemarkus) [#12246](https://github.com/kubernetes/kops/pull/12246)
* Enable protect-kernel-defaults by default and set the correct sysctls in nodeup [@olemarkus](https://github.com/olemarkus) [#12193](https://github.com/kubernetes/kops/pull/12193)
* update openstack CSI [@zetaab](https://github.com/zetaab) [#12251](https://github.com/kubernetes/kops/pull/12251)
* Use node internal IP for metrics-server [@olemarkus](https://github.com/olemarkus) [#12260](https://github.com/kubernetes/kops/pull/12260)
* Set ipv6 nameservers on aws [@olemarkus](https://github.com/olemarkus) [#12259](https://github.com/kubernetes/kops/pull/12259)
* Disable masquerade means disable masquerade if ipv6 too [@olemarkus](https://github.com/olemarkus) [#12265](https://github.com/kubernetes/kops/pull/12265)
* Update controller-runtime to v0.9.7 [@hakman](https://github.com/hakman) [#12264](https://github.com/kubernetes/kops/pull/12264)
* Try to bootstrap when at least one IP is available [@hakman](https://github.com/hakman) [#12270](https://github.com/kubernetes/kops/pull/12270)
* Fix kernel parameter for IPv6 forwarding [@hakman](https://github.com/hakman) [#12271](https://github.com/kubernetes/kops/pull/12271)
* Set explicit fsType to be able to mount volumes [@pa1op](https://github.com/pa1op) [#12280](https://github.com/kubernetes/kops/pull/12280)
* use ipip Always by default in OpenStack [@zetaab](https://github.com/zetaab) [#12286](https://github.com/kubernetes/kops/pull/12286)
* Enable IMDS IPv6 endpoint [@rifelpet](https://github.com/rifelpet) [#12290](https://github.com/kubernetes/kops/pull/12290)
* Recognize Ubuntu 21.10 (Impish Indri) [@hakman](https://github.com/hakman) [#12300](https://github.com/kubernetes/kops/pull/12300)
* Include kops- prefix in external-dns TXT record [@rifelpet](https://github.com/rifelpet),[@olemarkus](https://github.com/olemarkus) [#12289](https://github.com/kubernetes/kops/pull/12289)
* Bump snapshot-controller to 4.2.1 [@olemarkus](https://github.com/olemarkus) [#12303](https://github.com/kubernetes/kops/pull/12303)
* Bump aws ebs csi driver to 1.2.1 [@olemarkus](https://github.com/olemarkus) [#12310](https://github.com/kubernetes/kops/pull/12310)
* Update Go to v1.16.8 [@hakman](https://github.com/hakman) [#12306](https://github.com/kubernetes/kops/pull/12306)
* Do not precreate dns record for api lbs [@olemarkus](https://github.com/olemarkus) [#12308](https://github.com/kubernetes/kops/pull/12308)
* Update AWS CCM tags [@rifelpet](https://github.com/rifelpet),[@hakman](https://github.com/hakman) [#12104](https://github.com/kubernetes/kops/pull/12104)
* Set NodeIPFamilies in ipv6 mode [@olemarkus](https://github.com/olemarkus),[@hakman](https://github.com/hakman) [#12258](https://github.com/kubernetes/kops/pull/12258)
* Pre-install nvidia container runtime + drivers on GPU instances [@olemarkus](https://github.com/olemarkus) [#11628](https://github.com/kubernetes/kops/pull/11628)
* Add IMDS IPv6 endpoint in terraform [@rifelpet](https://github.com/rifelpet) [#12298](https://github.com/kubernetes/kops/pull/12298)
* Allow arbitrary length terraform literals [@rifelpet](https://github.com/rifelpet) [#12316](https://github.com/kubernetes/kops/pull/12316)
* Deprecate TerraformJSON feature flag [@rifelpet](https://github.com/rifelpet) [#12341](https://github.com/kubernetes/kops/pull/12341)
* Recognize pending EC2 instances as needed deletion [@justinsb](https://github.com/justinsb) [#12357](https://github.com/kubernetes/kops/pull/12357)
* Create a second Terraform provider for managed files [@rifelpet](https://github.com/rifelpet) [#12322](https://github.com/kubernetes/kops/pull/12322)
* Use MasterInternalName for gossip cluster SA issuer [@rifelpet](https://github.com/rifelpet) [#12297](https://github.com/kubernetes/kops/pull/12297)
* Use EC2 and Metadata IPv6 endpoints in IPv6 mode for EBS CSI Driver [@hakman](https://github.com/hakman) [#12349](https://github.com/kubernetes/kops/pull/12349)
* Update Calico to v3.20.1 [@hakman](https://github.com/hakman) [#12360](https://github.com/kubernetes/kops/pull/12360)
* Remove unneeded network related sysctls [@hakman](https://github.com/hakman) [#12385](https://github.com/kubernetes/kops/pull/12385)
* Upgrade terraform to 1.0.7 [@rifelpet](https://github.com/rifelpet),[@hakman](https://github.com/hakman) [#12403](https://github.com/kubernetes/kops/pull/12403)
* Run verify-cloudformation in host network [@rifelpet](https://github.com/rifelpet),[@hakman](https://github.com/hakman) [#12410](https://github.com/kubernetes/kops/pull/12410)
* Add specific taints to dns-controller. [@olemarkus](https://github.com/olemarkus) [#12389](https://github.com/kubernetes/kops/pull/12389)
* Add more tolerations to kops-controller and CCM . [@olemarkus](https://github.com/olemarkus) [#12400](https://github.com/kubernetes/kops/pull/12400)
* Revert "Remove unneeded network related sysctls" [@olemarkus](https://github.com/olemarkus) [#12415](https://github.com/kubernetes/kops/pull/12415)
* feat: add support for wildcard in roles generated for IRSA [@eddycharly](https://github.com/eddycharly) [#12342](https://github.com/kubernetes/kops/pull/12342)
* Allow aws-iam-authenticator to be scheduled onto dedicated apiserver nodes [@rifelpet](https://github.com/rifelpet) [#12426](https://github.com/kubernetes/kops/pull/12426)
* Truncate cluster name in NTH EventBridgeRules [@rifelpet](https://github.com/rifelpet) [#12439](https://github.com/kubernetes/kops/pull/12439)
* Mount cgroupv2 for cilium at a custom location [@olemarkus](https://github.com/olemarkus) [#12431](https://github.com/kubernetes/kops/pull/12431)
* Use separate cloud.config file for in-tree vs out-of-tree components [@rifelpet](https://github.com/rifelpet) [#12435](https://github.com/kubernetes/kops/pull/12435)
* feat: add support for custom audience in aws oidc provider [@eddycharly](https://github.com/eddycharly) [#12419](https://github.com/kubernetes/kops/pull/12419)
* Fix AWS IAM Authenticator support for k8s 1.22 [@rifelpet](https://github.com/rifelpet) [#12425](https://github.com/kubernetes/kops/pull/12425)
* Add ability to provide custom CoreDNS tolerations and affinity [@hierynomus](https://github.com/hierynomus) [#12234](https://github.com/kubernetes/kops/pull/12234)
* Add support for YAML/JSON output to 'kops get instances' [@hierynomus](https://github.com/hierynomus) [#12442](https://github.com/kubernetes/kops/pull/12442)
* Update containerd to v1.4.10 [@hakman](https://github.com/hakman) [#12459](https://github.com/kubernetes/kops/pull/12459)
* Add fixed version to all addons [@olemarkus](https://github.com/olemarkus) [#12416](https://github.com/kubernetes/kops/pull/12416)
* Remove unnecessary sysctl "net.ipv6.conf.all.accept_ra=2" [@hakman](https://github.com/hakman) [#12461](https://github.com/kubernetes/kops/pull/12461)

## 1.22.0-beta.2 to 1.22.0

* Release 1.22.0-beta.2 [@johngmyers](https://github.com/johngmyers) [#12467](https://github.com/kubernetes/kops/pull/12467)
* Bump Cluster Autoscaler images [@olemarkus](https://github.com/olemarkus) [#12463](https://github.com/kubernetes/kops/pull/12463)
* Update k8s dependencies to v1.22.2 for kOps 1.22 [@hakman](https://github.com/hakman) [#12478](https://github.com/kubernetes/kops/pull/12478)
* Update containerd to v1.4.11 [@hakman](https://github.com/hakman) [#12479](https://github.com/kubernetes/kops/pull/12479)
* Update Bazel to v4.2.1 [@hakman](https://github.com/hakman) [#12311](https://github.com/kubernetes/kops/pull/12311)
* Add kubescheduler.config.k8s.io/v1beta2 for k8s 1.22+ [@hakman](https://github.com/hakman) [#12486](https://github.com/kubernetes/kops/pull/12486)
* Update remaining k8s dependencies to v1.22.2 for kOps 1.22 [@hakman](https://github.com/hakman) [#12484](https://github.com/kubernetes/kops/pull/12484)
* protokube: don't try to connect to apiserver if not control-plane [@justinsb](https://github.com/justinsb) [#12424](https://github.com/kubernetes/kops/pull/12424)
* Update Docker to v20.10.9 [@hakman](https://github.com/hakman) [#12481](https://github.com/kubernetes/kops/pull/12481)
* Allow adding more subnets to an NLB [@olemarkus](https://github.com/olemarkus) [#12412](https://github.com/kubernetes/kops/pull/12412)
* Update Calico to v3.20.2 [@hakman](https://github.com/hakman) [#12491](https://github.com/kubernetes/kops/pull/12491)
* Update Bazel rules_go to v0.29.0 [@hakman](https://github.com/hakman) [#12496](https://github.com/kubernetes/kops/pull/12496)
* Update etcd-manager to 3.0.20211007 [@hakman](https://github.com/hakman) [#12497](https://github.com/kubernetes/kops/pull/12497)
* Enable ingress hostname feature for OpenStack [@zetaab](https://github.com/zetaab) [#12525](https://github.com/kubernetes/kops/pull/12525)
* Update Bazel rules_docker to v0.20.0 [@hakman](https://github.com/hakman) [#12523](https://github.com/kubernetes/kops/pull/12523)

## 1.22.0 to 1.22.1

* Release 1.22.0 [@johngmyers](https://github.com/johngmyers) [#12532](https://github.com/kubernetes/kops/pull/12532)
* Don't hard-code the SQS Queue ARN partition [@rifelpet](https://github.com/rifelpet) [#12540](https://github.com/kubernetes/kops/pull/12540)
* Fix nil pointer error where containerd is not in use [@justinsb](https://github.com/justinsb) [#12374](https://github.com/kubernetes/kops/pull/12374)
* Re-add ec2:DescribeLaunchTemplateVersions to CA IAM policy [@rifelpet](https://github.com/rifelpet) [#12568](https://github.com/kubernetes/kops/pull/12568)
* Upgrade AWS VPC CNI to 1.9.3 w/ k8s 1.22 support [@rifelpet](https://github.com/rifelpet) [#12573](https://github.com/kubernetes/kops/pull/12573)

## 1.22.1 to 1.22.2

* Release 1.22.1 [@johngmyers](https://github.com/johngmyers) [#12577](https://github.com/kubernetes/kops/pull/12577)
* Upgrade Canal to v3.20 with k8s 1.22 support [@rifelpet](https://github.com/rifelpet) [#12584](https://github.com/kubernetes/kops/pull/12584)
* Allow AWS LBC to attach certificates [@olemarkus](https://github.com/olemarkus) [#12309](https://github.com/kubernetes/kops/pull/12309)
* Add calico-kube-controllers for Canal [@hakman](https://github.com/hakman) [#12593](https://github.com/kubernetes/kops/pull/12593)
* Handle keypair items without certificates [@johngmyers](https://github.com/johngmyers) [#12601](https://github.com/kubernetes/kops/pull/12601)
* Respect any MaxPods value the user sets explicitly [@hakman](https://github.com/hakman) [#12603](https://github.com/kubernetes/kops/pull/12603)
* Add permissions needed for KCM to provision NLBs [@olemarkus](https://github.com/olemarkus) [#12611](https://github.com/kubernetes/kops/pull/12611)
* Make it possible to set CAS max-node-provision-time [@olemarkus](https://github.com/olemarkus) [#12437](https://github.com/kubernetes/kops/pull/12437)
* Use InternalIP as preferred kubelet address only in ivp6 mode [@olemarkus](https://github.com/olemarkus) [#12626](https://github.com/kubernetes/kops/pull/12626)
* Fix hardcoded ARN partitions [@rifelpet](https://github.com/rifelpet) [#12638](https://github.com/kubernetes/kops/pull/12638)
* Fix cluster name used in IAM policies [@rifelpet](https://github.com/rifelpet) [#12672](https://github.com/kubernetes/kops/pull/12672)
* Remove tag conditions on certain AWS IAM actions [@rifelpet](https://github.com/rifelpet) [#12674](https://github.com/kubernetes/kops/pull/12674)
* Increase upup http response header timeout [@AlexLast](https://github.com/AlexLast) [#12694](https://github.com/kubernetes/kops/pull/12694)
* Ignore white space when validating IAM policy size limits [@rifelpet](https://github.com/rifelpet) [#12700](https://github.com/kubernetes/kops/pull/12700)
* GCE: use chrony on Ubuntu + GCE [@justinsb](https://github.com/justinsb),[@hakman](https://github.com/hakman) [#12587](https://github.com/kubernetes/kops/pull/12587)
* set calico-node readiness/liveness timeout to 10s [@estahn](https://github.com/estahn) [#12713](https://github.com/kubernetes/kops/pull/12713)
* Add missing status fields to IAMIdentityMapping v1 CRD [@rifelpet](https://github.com/rifelpet) [#12716](https://github.com/kubernetes/kops/pull/12716)
* Don't fail validation if Nvidia and containerRuntime defaults [@johngmyers](https://github.com/johngmyers) [#12729](https://github.com/kubernetes/kops/pull/12729)
* Support setting empty maps and structs [@johngmyers](https://github.com/johngmyers) [#12728](https://github.com/kubernetes/kops/pull/12728)
* Fix out of bounds error when instance detach fails [@johngmyers](https://github.com/johngmyers) [#12698](https://github.com/kubernetes/kops/pull/12698)
* Upgrade external-dns to 0.10.1 for Kubernetes >= 1.19 [@johngmyers](https://github.com/johngmyers) [#12724](https://github.com/kubernetes/kops/pull/12724)
* Do not return error when there is no error checking for cgroupfs [@olemarkus](https://github.com/olemarkus) [#12744](https://github.com/kubernetes/kops/pull/12744)
* Fix render template cilium AgentPrometheusPort into a UNICODE char error [@zhengtianbao](https://github.com/zhengtianbao) [#12721](https://github.com/kubernetes/kops/pull/12721)
* Update Go to v1.16.10 for kOps 1.22 [@hakman](https://github.com/hakman) [#12754](https://github.com/kubernetes/kops/pull/12754)
* Partial backport of #12564 [@olemarkus](https://github.com/olemarkus) [#12760](https://github.com/kubernetes/kops/pull/12760)
* Fix that states AWS IAM Instance Profile blocks IAM Role [@angeloskaltsikis](https://github.com/angeloskaltsikis) [#12677](https://github.com/kubernetes/kops/pull/12677)
* Add hashes for latest containerd and Docker versions [@hakman](https://github.com/hakman) [#12767](https://github.com/kubernetes/kops/pull/12767)
* Shorten filenames in the asset store [@johngmyers](https://github.com/johngmyers) [#12765](https://github.com/kubernetes/kops/pull/12765)
* Update containerd to v1.4.12 [@hakman](https://github.com/hakman) [#12772](https://github.com/kubernetes/kops/pull/12772)
* Add create cluster flag for enabling IRSA [@johngmyers](https://github.com/johngmyers) [#12741](https://github.com/kubernetes/kops/pull/12741)
* Fix volume ratio comparisons [@olemarkus](https://github.com/olemarkus) [#12791](https://github.com/kubernetes/kops/pull/12791)
* Bump etcd manager to 20211117 [@justinsb](https://github.com/justinsb) [#12763](https://github.com/kubernetes/kops/pull/12763)

## 1.22.2 to 1.22.3

* Release 1.22.2 [@johngmyers](https://github.com/johngmyers) [#12797](https://github.com/kubernetes/kops/pull/12797)
* Add support for etcd v3.5.1 [@hakman](https://github.com/hakman) [#12826](https://github.com/kubernetes/kops/pull/12826)
* Update Calico and Canal to v3.20.3 for kOps 1.22 [@hakman](https://github.com/hakman) [#12836](https://github.com/kubernetes/kops/pull/12836)
* Add support for --dns flag in Docker config [@jwolski2](https://github.com/jwolski2) [#12789](https://github.com/kubernetes/kops/pull/12789)
* Reissue client keypairs on issuer change [@johngmyers](https://github.com/johngmyers) [#12846](https://github.com/kubernetes/kops/pull/12846)
* Add gofumpt scripts [@hakman](https://github.com/hakman) [#12867](https://github.com/kubernetes/kops/pull/12867)
* Update Go to v1.16.11 [@hakman](https://github.com/hakman) [#12897](https://github.com/kubernetes/kops/pull/12897)
* Fix external-dns service name [@johngmyers](https://github.com/johngmyers) [#12893](https://github.com/kubernetes/kops/pull/12893)
* pkg/apis/kops: Allow configuring dockerd --max-* upload and download concurrency and retry options. [@anthonyrisinger](https://github.com/anthonyrisinger) [#12320](https://github.com/kubernetes/kops/pull/12320)
* Update Go to v1.16.12 [@hakman](https://github.com/hakman) [#12956](https://github.com/kubernetes/kops/pull/12956)
* Prevent creation of unsupported etcd clusters [@olemarkus](https://github.com/olemarkus) [#13011](https://github.com/kubernetes/kops/pull/13011)
* force update deps [@zetaab](https://github.com/zetaab) [#13057](https://github.com/kubernetes/kops/pull/13057)
* Add action for automatically tagging releases [@johngmyers](https://github.com/johngmyers) [#12805](https://github.com/kubernetes/kops/pull/12805)
* Release 1.22.3 [@johngmyers](https://github.com/johngmyers) [#13083](https://github.com/kubernetes/kops/pull/13083)

## 1.22.3 to 1.22.4

* Don't try to add node name to instances without node object [@olemarkus](https://github.com/olemarkus) [#13106](https://github.com/kubernetes/kops/pull/13106)
* Bump CCM images [@olemarkus](https://github.com/olemarkus) [#13143](https://github.com/kubernetes/kops/pull/13143)
* Bump etcd-manager to v3.0.20220128 [@olemarkus](https://github.com/olemarkus) [#13158](https://github.com/kubernetes/kops/pull/13158)
* JWKS / IRSA: Expose public ACLs to terraform [@justinsb](https://github.com/justinsb) [#13166](https://github.com/kubernetes/kops/pull/13166)
* Remove tag condition on listeners [@olemarkus](https://github.com/olemarkus) [#13123](https://github.com/kubernetes/kops/pull/13123)
* Remove snapshot controller dependency on ebs csi driver [@olemarkus](https://github.com/olemarkus) [#13213](https://github.com/kubernetes/kops/pull/13213)
* upgrade cluster: support comma separated list for machineType [1.22] [@MeirP-3](https://github.com/MeirP-3) [#13211](https://github.com/kubernetes/kops/pull/13211)
* Use v1 certificate for snapshot-validation-service [@olemarkus](https://github.com/olemarkus) [#12874](https://github.com/kubernetes/kops/pull/12874)
* Fix CSI migration feature gates [@olemarkus](https://github.com/olemarkus) [#13203](https://github.com/kubernetes/kops/pull/13203)
* Fix irsa for k8s < 1.20 [@olemarkus](https://github.com/olemarkus) [#13212](https://github.com/kubernetes/kops/pull/13212)
* Disable some flags in kube-apiserver when logging-format is not text [@h3poteto](https://github.com/h3poteto) [#13264](https://github.com/kubernetes/kops/pull/13264)
* Update LBC to 2.4.0 [@olemarkus](https://github.com/olemarkus) [#13267](https://github.com/kubernetes/kops/pull/13267)
* Simplify Flatcar containerd exec command [@pothos](https://github.com/pothos) [#12900](https://github.com/kubernetes/kops/pull/12900)
* Validate taints in IG spec [@olemarkus](https://github.com/olemarkus) [#13266](https://github.com/kubernetes/kops/pull/13266)
* Add missing permissions to aws lbc for irsa [@olemarkus](https://github.com/olemarkus) [#13280](https://github.com/kubernetes/kops/pull/13280)
* Do not create a cert-manager namespace [@olemarkus](https://github.com/olemarkus) [#13284](https://github.com/kubernetes/kops/pull/13284)
* Update to etcd-manager v3.0.20220203 [@justinsb](https://github.com/justinsb) [#13196](https://github.com/kubernetes/kops/pull/13196)
* Release 1.22.4 [@heybronson](https://github.com/heybronson) [#13279](https://github.com/kubernetes/kops/pull/13279)

## 1.22.4 to 1.22.5

* Add support for ed25519 keys in AWS [@aclevername](https://github.com/aclevername) [#13304](https://github.com/kubernetes/kops/pull/13304)
* If kubetest2 fails cluster validation, we run down before exiting [@olemarkus](https://github.com/olemarkus) [#13373](https://github.com/kubernetes/kops/pull/13373)
* Fix long role names [@olemarkus](https://github.com/olemarkus) [#13364](https://github.com/kubernetes/kops/pull/13364)
* Correctly detect GovCloud regions [@mixja](https://github.com/mixja) [#13410](https://github.com/kubernetes/kops/pull/13410)
* Pick the right OS server group when creating cloud groups [@ederst](https://github.com/ederst) [#13461](https://github.com/kubernetes/kops/pull/13461)
* Only delete node object on GCE [@olemarkus](https://github.com/olemarkus) [#13289](https://github.com/kubernetes/kops/pull/13289)
* Update to etcd-manager 3.0.20220417 [@justinsb](https://github.com/justinsb) [#13499](https://github.com/kubernetes/kops/pull/13499)
* Bump CCM 1.22 and 1.23 images to stable versions [@olemarkus](https://github.com/olemarkus) [#13506](https://github.com/kubernetes/kops/pull/13506)
* etcd 3 5 3 [@justinsb](https://github.com/justinsb) [#13501](https://github.com/kubernetes/kops/pull/13501)
* Update Calico and Canal to v3.20.5 for kOps 1.22 [@hakman](https://github.com/hakman) [#13518](https://github.com/kubernetes/kops/pull/13518)

## 1.22.5 to 1.22.6

* Add hashes for containerd and Docker in order to fix CVE-2022-23648 [@drequena](https://github.com/drequena) [#13606](https://github.com/kubernetes/kops/pull/13606)
* Avoid "/etc/resolv.conf" file loopback for Flatcar Container Linux distribution [@seh](https://github.com/seh) [#13617](https://github.com/kubernetes/kops/pull/13617)
* Update etcd-manager to v3.0.20220717 [@hakman](https://github.com/hakman) [#13990](https://github.com/kubernetes/kops/pull/13990)
* Update Go to v1.16.15 for kOps 1.22 [@hakman](https://github.com/hakman) [#13998](https://github.com/kubernetes/kops/pull/13998)
* Switch to latest MacOS version for CI [@hakman](https://github.com/hakman) [#14015](https://github.com/kubernetes/kops/pull/14015)
*  Check keyset existence before attempting to distrust [@yurrriq](https://github.com/yurrriq) [#14041](https://github.com/kubernetes/kops/pull/14041)
