WEBVTT 00:00.900 --> 00:03.770 Hello and welcome to this lecture. In this lecture 00:03.780 --> 00:08.070 we look at the various Backup and Restore methodologies. 00:08.070 --> 00:14.160 Let’s start by looking at what you should consider backing up in a Kubernetes cluster. So far in 00:14.160 --> 00:14.710 this course, 00:14.720 --> 00:20.310 we have deployed a number of different applications on our Kubernetes cluster using deployment, pod 00:20.580 --> 00:22.840 and Service definition files. 00:22.890 --> 00:28.060 We know that the ETCD cluster is where all cluster related information is stored. 00:28.360 --> 00:33.930 And if your applications are configured with persistent storage then that is another candidate for backups. 00:35.510 --> 00:38.330 With respect to resources that we created in the cluster, 00:38.330 --> 00:44.390 at times we used the imperative way of creating an object by executing a command. Such as while creating 00:44.390 --> 00:52.290 a namespace or a secret or config map or at times for exposing applications. And at times we used 00:52.380 --> 00:57.540 the declarative approach by first creating a definition file and then running the kubectl apply 00:57.540 --> 00:59.630 command on that file. 00:59.670 --> 01:05.700 This is the preferred approach if you want to save your configuration. Because now you have all the objects 01:05.700 --> 01:11.960 required for a single application in the form of object definition files in a single folder. 01:11.970 --> 01:18.820 This can easily be reused at a later time or shared with others of course you must have a copy of these 01:18.820 --> 01:20.790 files saved at all times. 01:21.670 --> 01:25.810 A good practice is to store these on source code repositories. 01:25.810 --> 01:32.090 That way it can be maintained by a team the source code repository should be configured with the right 01:32.150 --> 01:37.320 backup solutions. With managed/public source code repositories like Github 01:37.340 --> 01:38.680 You don't have to worry about this. 01:40.000 --> 01:46.000 With that even when you lose your entire cluster you can redeploy your application on the cluster by 01:46.000 --> 01:53.240 simply applying this configuration files on them. While the declarative approach is the preferred approach 01:53.390 --> 01:58.430 it is not necessary that all of your team members stick to those standards. 01:58.430 --> 02:03.740 What if someone created an object the imperative way without documenting that information anywhere. 02:03.740 --> 02:11.690 So a better approach to backing up resource configuration is to use query the kube-api server. . Query the kube-api 02:11.690 --> 02:18.110 server using kubectl or by accessing the API server directly and save all resource configurations 02:18.110 --> 02:21.910 for all objects created on the cluster has a copy. 02:22.340 --> 02:27.860 For example one of the commands that can be used in a backup script is to get all pods, deployments 02:27.860 --> 02:34.910 and services in all namespaces using the kubectl utility’s get all command and extract the output 02:34.970 --> 02:41.220 in a YAML format. Then save that file. And that’s just for a few resources groups. 02:41.240 --> 02:44.810 There are many other resource groups that must be considered. 02:44.960 --> 02:48.650 Of course you don't have to develop that solutions yourself. 02:48.680 --> 02:55.760 There are tools like ARK or now called Velero by Heptio that can do this for you. 02:55.880 --> 03:03.320 It can help in taking backups of your Kubernetes cluster using the Kubernetes API. Let us now move 03:03.320 --> 03:05.630 on to ETCD. 03:05.660 --> 03:13.280 The ETCD cluster stores information about the state of our cluster. So information about the cluster itself, 03:13.280 --> 03:18.190 The nodes and every other resource as created within the cluster are stored here. 03:18.260 --> 03:26.080 So instead of backing up resource as before, you may chose to backup the ETCD server itself. As we have 03:26.080 --> 03:33.430 seen the ETCD cluster is hosted on the master nodes. While configuring ETCD we specified a location 03:33.460 --> 03:35.620 where all the data would be stored. 03:35.650 --> 03:44.570 The data directory. That is the directory that can be configured to be backup by your backup tool. ETCD 03:44.610 --> 03:48.570 also comes with a built in snapshot solution. 03:48.570 --> 03:55.230 You can take a snapshot of the ETCD database by using the etdctl utilities snapshot save command. 03:55.950 --> 04:04.090 Give the snapshot a name snapshot.db. A snapshot file is created by the name in the current directory. 04:04.260 --> 04:09.020 If you wanted to be created in another location specified the full path. 04:09.090 --> 04:16.630 You can view the status of the backup using the snapshot status command. To restore the cluster from 04:16.630 --> 04:18.960 this backup at a later point in time. 04:18.970 --> 04:25.210 first stop the kube-api server service, as the restore process will require you to restart the ETCD 04:25.210 --> 04:32.620 cluster and the kube-api server depends on it. Then run the etcdctl’s snapshot restore command 04:33.010 --> 04:40.990 with the path set to the path of the backup file which is the snapshot.db file. When ETCD restores 04:40.990 --> 04:48.460 from a backup, it initializes a new cluster configuration and configures the members of ETCD as new 04:48.460 --> 04:50.770 members to a new cluster. 04:50.770 --> 04:55.870 This is to prevent a new member from accidentally joining an existing cluster. 04:55.870 --> 05:02.660 Say for example, you use this backup snapshot to provision a new etcd-cluster for testing purposes. 05:02.680 --> 05:08.660 You don’t want the members in the new test cluster to accidentally join the production cluster. 05:08.660 --> 05:15.550 So during a restore you must specify a new cluster token and the same initial cluster configuration 05:15.610 --> 05:23.170 options specified in the original configuration file. On running this command a new data directory is 05:23.170 --> 05:23.960 created. 05:24.160 --> 05:29.420 in this example at location /var/lib/etcd-from-backup. 05:29.680 --> 05:35.980 We then configure the ETCD configuration file to use the new cluster-token and data directory. 05:44.110 --> 05:48.370 The reload the service daemon and restart etcd service. 05:48.370 --> 05:56.970 Finally start the kube-apiserver service. Your cluster should now be back in the original state. A quick 05:56.970 --> 06:00.560 note before I let you go With all the ETCD commands 06:00.600 --> 06:04.230 remember to specify the certificate files for authentication. 06:04.230 --> 06:10.830 Specify the endpoint to the ETCD cluster and the ca certificate, the etcd-server certificate and 06:10.830 --> 06:19.330 the key, So we have seen two options and backup using ETCD and backup by querying the kube-apiserver . 06:19.370 --> 06:21.680 Both of these have their pros and cons. 06:22.160 --> 06:29.540 Well if you're using a managed Kubernetes environment then at times you may not even have access to 06:29.540 --> 06:31.940 the ETCD cluster in that case 06:31.950 --> 06:35.570 Backup by querying the kube-apiserver probably the better way. 06:36.350 --> 06:38.410 Well that's it for this demo. 06:38.420 --> 06:44.630 Head over to the practice test and practice backing up ETCD in a cluster and then restoring from backups.