Migrate a ThirdPartyResource to CustomResourceDefinition
This page shows how to migrate data stored in a ThirdPartyResource (TPR) to a CustomResourceDefinition (CRD).
Kubernetes does not automatically migrate existing TPRs. This is due to API changes introduced as part of graduating to beta under a new name and API group. Instead, both TPR and CRD are available and operate independently in Kubernetes 1.7. Users must migrate each TPR one by one to preserve their data before upgrading to Kubernetes 1.8.
The simplest way to migrate is to stop all clients that use a given TPR, then delete the TPR and start from scratch with a CRD. This page describes an optional process that eases the transition by migrating existing TPR data for you on a best-effort basis.
Before you begin
You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube, or you can use one of these Kubernetes playgrounds:
To check the version, enter kubectl version.
- Make sure your Kubernetes cluster has a master version of exactly 1.7.x (any patch release), as this is the only version that supports both TPR and CRD.
- If you use a TPR-based custom controller, check with the author of the controller first. Some or all of these steps may be unnecessary if the custom controller handles the migration for you.
- Be familiar with the concept of custom resources, which were known as third-party resources until Kubernetes 1.7.
- Be familiar with CustomResourceDefinitions, which are a simple way to implement custom resources.
- Before performing a migration on real data, conduct a dry run by going through these steps in a test cluster.
Migrate TPR data
-
Rewrite the TPR definition
Clients that access the REST API for your custom resource should not need any changes. However, you will need to rewrite your TPR definition as a CRD.
Make sure you specify values for the CRD fields that match what the server used to fill in for you with TPR.
For example, if your ThirdPartyResource looks like this:
apiVersion: extensions/v1beta1 kind: ThirdPartyResource metadata: name: cron-tab.stable.example.com description: "A specification of a Pod to run on a cron style schedule" versions: - name: v1A matching CustomResourceDefinition could look like this:
apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: crontabs.stable.example.com spec: scope: Namespaced group: stable.example.com version: v1 names: kind: CronTab plural: crontabs singular: crontab -
Install the CustomResourceDefinition
While the source TPR is still active, install the matching CRD with
kubectl create. Existing TPR data remains accessible because TPRs take precedence over CRDs when both try to serve the same resource.After you create the CRD, make sure the Established condition goes to True. You can check it with a command like this:
kubectl get crd -o 'custom-columns=NAME:{.metadata.name},ESTABLISHED:{.status.conditions[?(@.type=="Established")].status}'The output should look like this:
NAME ESTABLISHED crontabs.stable.example.com True -
Stop all clients that use the TPR
The API server attempts to prevent TPR data for the resource from changing while it copies objects to the CRD, but it can’t guarantee consistency in all cases, such as with multiple masters. Stopping clients, such as TPR-based custom controllers, helps to avoid inconsistencies in the copied data.
In addition, clients that watch TPR data do not receive any more events once the migration begins. You must restart them after the migration completes so they start watching CRD data instead.
-
Back up TPR data
In case the data migration fails, save a copy of existing data for the resource:
kubectl get crontabs --all-namespaces -o yaml > crontabs.yamlYou should also save a copy of the TPR definition if you don’t have one already:
kubectl get thirdpartyresource cron-tab.stable.example.com -o yaml --export > tpr.yaml -
Delete the TPR definition
Normally, when you delete a TPR definition, the API server tries to clean up any objects stored in that resource. Because a matching CRD exists, the server copies objects to the CRD instead of deleting them.
kubectl delete thirdpartyresource cron-tab.stable.example.com -
Verify the new CRD data
It can take up to 10 seconds for the TPR controller to notice when you delete the TPR definition and to initiate the migration. The TPR data remains accessible during this time.
Once the migration completes, the resource begins serving through the CRD. Check that all your objects were correctly copied:
kubectl get crontabs --all-namespaces -o yamlIf the copy failed, you can quickly revert to the set of objects that existed just before the migration by recreating the TPR definition:
kubectl create -f tpr.yaml -
Restart clients
After verifying the CRD data, restart any clients you stopped before the migration, such as custom controllers and other watchers. These clients now access CRD data when they make requests on the same API endpoints that the TPR previously served.
What’s next
- Learn more about custom resources.
- Learn more about using CustomResourceDefinitions.
- See CustomResourceDefinition.