Difference between revisions of "Freezer-docs"
Memo Garcia (talk | contribs) |
Memo Garcia (talk | contribs) |
||
Line 45: | Line 45: | ||
export OS_REGION_NAME=RegionOne | export OS_REGION_NAME=RegionOne | ||
export OS_IDENTITY_API_VERSION=3 | export OS_IDENTITY_API_VERSION=3 | ||
+ | |||
+ | for windows replace export by set | ||
freezer-scheduler start | freezer-scheduler start | ||
Line 122: | Line 124: | ||
* for newton onwards: | * for newton onwards: | ||
freezer job-get 07999ea33a494ccf84590191d6fe850c -C node12 | freezer job-get 07999ea33a494ccf84590191d6fe850c -C node12 | ||
+ | |||
+ | |||
+ | = Freezer Agent = | ||
+ | |||
+ | Keystone v2.0 and v3 authentication is supported for freezer | ||
+ | |||
+ | v2.0: | ||
+ | |||
+ | export OS_BACKUP_URL=http://192.168.33.10:9090 | ||
+ | export OS_AUTH_URL=http://192.168.33.10:5000/v2.0 | ||
+ | export OS_USERNAME=admin | ||
+ | export OS_PASSWORD=quiet | ||
+ | export OS_TENANT_NAME=admin | ||
+ | export OS_REGION_NAME=RegionOne | ||
+ | export OS_IDENTITY_API_VERSION=2 | ||
+ | |||
+ | v3: | ||
+ | |||
+ | export OS_BACKUP_URL=http://192.168.33.10:9090 | ||
+ | export OS_AUTH_URL=http://192.168.33.10:5000/v3 | ||
+ | export OS_USERNAME=admin | ||
+ | export OS_PASSWORD=quiet | ||
+ | export OS_PROJECT_DOMAIN_NAME=admin | ||
+ | export OS_USER_DOMAIN_NAME=admin | ||
+ | export OS_PROJECT_NAME=admin | ||
+ | export OS_REGION_NAME=RegionOne | ||
+ | export OS_IDENTITY_API_VERSION=3 | ||
+ | |||
+ | for windows replace export by set | ||
+ | |||
+ | == Advanced features == | ||
+ | |||
+ | === Bandwidth limitation === | ||
+ | |||
+ | Trickle for bandwidth limiting ( How it works ): We have 3 cases to handle | ||
+ | |||
+ | 1: User used –upload-limit or –download-limit from the cli We need to remove these limits from the cli arguments and then run trickle using subprocess: | ||
+ | |||
+ | freezer-agent --action backup -F /etc/ -C freezer --upload-limit = 1k | ||
+ | |||
+ | this will be translated to: | ||
+ | |||
+ | trickle -u 1024 -d -1 freezer-agent --action backup -F /etc/ -C freezer | ||
+ | |||
+ | 2: User used config files to execute an action | ||
+ | |||
+ | We need to create a new config file without the limits so we will get the all the arguments provided and remove limits then run trickle using subprocess: | ||
+ | |||
+ | [default] | ||
+ | action = backup | ||
+ | storage = ssh | ||
+ | ssh_host = 127.0.0.1 | ||
+ | ssh_username = saad | ||
+ | ssh_key = /home/saad/.ssh/saad | ||
+ | container = /home/saad/backups_freezers | ||
+ | backup_name = freezer_jobs | ||
+ | path_to_backup = /etc | ||
+ | upload_limit=2k | ||
+ | download_limit=1k | ||
+ | |||
+ | and we are going to execute this job as follow: | ||
+ | |||
+ | freezer-agent --config /home/user/job1.ini | ||
+ | |||
+ | this will be translated to: | ||
+ | |||
+ | trickle -u 2048 -d 1024 freezer-agent --config /tmp/freezer_job_x21aj29 | ||
+ | |||
+ | 3: Hybrid using config file and cli options we will use a mix of both procedures: - remove limits (cli or config ) - reproduce the same command again with trickle: | ||
+ | |||
+ | freezer-agent --config /home/user/job2.ini --upload-limit 1k | ||
+ | |||
+ | === How incremental works === | ||
+ | |||
+ | The incremental backups is one of the most crucial features. The following basic logic happens when Freezer executes: | ||
+ | |||
+ | * Freezer starts the execution and checks if the provided backup name for the current node already exists in Swift. | ||
+ | * If the backup exists, then the Manifest file is retrieved. This is important as the Manifest file contains the information of the previous Freezer execution. |
Revision as of 10:48, 27 June 2016
Contents
Freezer Scheduler
Client Registration
Make the agent/scheduler available to the api/cli/ui:
freezer-scheduler register
Verify the client is registered correctly:
- for kilo, liberty and mitaka:
freezer-scheduler client-list
- for newton onwards:
freezer client-list
Scheduler Start/Stop
Start the scheduler
In order to poll the api to get new jobs, actions, etc. we need to start the scheduler
make sure to use the credentials for keystone v2.0 or v3 and you are root or Administrator on Windows
Keystone v2.0 and v3 authentication is supported for freezer
v2.0:
export OS_BACKUP_URL=http://192.168.33.10:9090 export OS_AUTH_URL=http://192.168.33.10:5000/v2.0 export OS_USERNAME=admin export OS_PASSWORD=quiet export OS_TENANT_NAME=admin export OS_REGION_NAME=RegionOne export OS_IDENTITY_API_VERSION=2
v3:
export OS_BACKUP_URL=http://192.168.33.10:9090 export OS_AUTH_URL=http://192.168.33.10:5000/v3 export OS_USERNAME=admin export OS_PASSWORD=quiet export OS_PROJECT_DOMAIN_NAME=admin export OS_USER_DOMAIN_NAME=admin export OS_PROJECT_NAME=admin export OS_REGION_NAME=RegionOne export OS_IDENTITY_API_VERSION=3
for windows replace export by set
freezer-scheduler start
Verify the scheduler is running:
freezer-scheduler status
Stop the scheduler
freezer-scheduler stop
It can be also be started as a foreground process using the –no-daemon flag:
freezer-scheduler --no-daemon start
This can be useful for testing purposes, when launched in a Docker container, or by a babysitting process such as systemd.
Using the Scheduler
The cli-tool version is used to manage the jobs in the API. A “job” is basically a container; a document which contains one or more “actions”. An action contains the instructions for the freezer-agent. They are the same parameters that would be passed to the agent on the command line. For example: “backup_name”, “path_to_backup”, “max_level”
To sum it up, a job is a sequence of parameters that the scheduler pulls from the API and passes to a newly spawned freezer-agent process at the right time.
The scheduler understands the “scheduling” part of the job document, which it uses to actually schedule the job, while the rest of the parameters are substantially opaque.
It may also be useful to use the “-c” parameter to specify the client-id that the scheduler will use when interacting with the API.
The purpose of the client-id is to associate a job with the scheduler instance which is supposed to execute that job.
A single openstack user could manage different resources on different nodes (and actually may even have different freezer-scheduler instances running on the same node with different local privileges, for example), and the client-id allows him to associate the specific scheduler instance with its specific jobs.
When not provided with a custom client-id, the scheduler falls back to the default which is composed from the tenant-id and the hostname of the machine on which it is running.
Create a job
The first step to use the scheduler is creating a document with the job:
{ "job_actions": [{ "freezer_action": { "action": "backup", "mode": "fs", "backup_name": "backup1", "path_to_backup": "/home/me/datadir", "container": "schedule_backups", "log_file": "/home/me/.freezer/freezer.log" }, "max_retries": 3, "max_retries_interval": 60 }], "job_schedule": { "schedule_interval": "4 hours", "schedule_start_date": "2015-08-16T17:58:00" }, "description": "schedule_backups 6" }
Then upload that job into the API
- for kilo, liberty and mitaka:
freezer-scheduler -c node12 job-create --file test_job.json
- for newton onwards:
freezer job-create --file test_job.json -C node12
List jobs
- for kilo, liberty and mitaka:
freezer-scheduler -c node12 job-list
- for newton onwards:
freezer job-list -C node12
Get the content of a job
- for kilo, liberty and mitaka:
freezer-scheduler -c node12 job-get -j 07999ea33a494ccf84590191d6fe850c
- for newton onwards:
freezer job-get 07999ea33a494ccf84590191d6fe850c -C node12
Freezer Agent
Keystone v2.0 and v3 authentication is supported for freezer
v2.0:
export OS_BACKUP_URL=http://192.168.33.10:9090 export OS_AUTH_URL=http://192.168.33.10:5000/v2.0 export OS_USERNAME=admin export OS_PASSWORD=quiet export OS_TENANT_NAME=admin export OS_REGION_NAME=RegionOne export OS_IDENTITY_API_VERSION=2
v3:
export OS_BACKUP_URL=http://192.168.33.10:9090 export OS_AUTH_URL=http://192.168.33.10:5000/v3 export OS_USERNAME=admin export OS_PASSWORD=quiet export OS_PROJECT_DOMAIN_NAME=admin export OS_USER_DOMAIN_NAME=admin export OS_PROJECT_NAME=admin export OS_REGION_NAME=RegionOne export OS_IDENTITY_API_VERSION=3
for windows replace export by set
Advanced features
Bandwidth limitation
Trickle for bandwidth limiting ( How it works ): We have 3 cases to handle
1: User used –upload-limit or –download-limit from the cli We need to remove these limits from the cli arguments and then run trickle using subprocess:
freezer-agent --action backup -F /etc/ -C freezer --upload-limit = 1k
this will be translated to:
trickle -u 1024 -d -1 freezer-agent --action backup -F /etc/ -C freezer
2: User used config files to execute an action
We need to create a new config file without the limits so we will get the all the arguments provided and remove limits then run trickle using subprocess:
[default] action = backup storage = ssh ssh_host = 127.0.0.1 ssh_username = saad ssh_key = /home/saad/.ssh/saad container = /home/saad/backups_freezers backup_name = freezer_jobs path_to_backup = /etc upload_limit=2k download_limit=1k
and we are going to execute this job as follow:
freezer-agent --config /home/user/job1.ini
this will be translated to:
trickle -u 2048 -d 1024 freezer-agent --config /tmp/freezer_job_x21aj29
3: Hybrid using config file and cli options we will use a mix of both procedures: - remove limits (cli or config ) - reproduce the same command again with trickle:
freezer-agent --config /home/user/job2.ini --upload-limit 1k
How incremental works
The incremental backups is one of the most crucial features. The following basic logic happens when Freezer executes:
- Freezer starts the execution and checks if the provided backup name for the current node already exists in Swift.
- If the backup exists, then the Manifest file is retrieved. This is important as the Manifest file contains the information of the previous Freezer execution.