Jump to: navigation, search

Difference between revisions of "Freezer-docs"

Line 45: Line 45:
 
     export OS_REGION_NAME=RegionOne
 
     export OS_REGION_NAME=RegionOne
 
     export OS_IDENTITY_API_VERSION=3
 
     export OS_IDENTITY_API_VERSION=3
 +
 +
for windows replace export by set
  
 
     freezer-scheduler start
 
     freezer-scheduler start
Line 122: Line 124:
 
* for newton onwards:
 
* for newton onwards:
 
   freezer job-get 07999ea33a494ccf84590191d6fe850c -C node12
 
   freezer job-get 07999ea33a494ccf84590191d6fe850c -C node12
 +
 +
 +
= Freezer Agent =
 +
 +
Keystone v2.0 and v3 authentication is supported for freezer
 +
 +
v2.0:
 +
 +
    export OS_BACKUP_URL=http://192.168.33.10:9090
 +
    export OS_AUTH_URL=http://192.168.33.10:5000/v2.0
 +
    export OS_USERNAME=admin
 +
    export OS_PASSWORD=quiet
 +
    export OS_TENANT_NAME=admin
 +
    export OS_REGION_NAME=RegionOne
 +
    export OS_IDENTITY_API_VERSION=2
 +
 +
v3:
 +
 +
    export OS_BACKUP_URL=http://192.168.33.10:9090
 +
    export OS_AUTH_URL=http://192.168.33.10:5000/v3
 +
    export OS_USERNAME=admin
 +
    export OS_PASSWORD=quiet
 +
    export OS_PROJECT_DOMAIN_NAME=admin
 +
    export OS_USER_DOMAIN_NAME=admin
 +
    export OS_PROJECT_NAME=admin
 +
    export OS_REGION_NAME=RegionOne
 +
    export OS_IDENTITY_API_VERSION=3
 +
 +
for windows replace export by set
 +
 +
== Advanced features ==
 +
 +
=== Bandwidth limitation ===
 +
 +
Trickle for bandwidth limiting ( How it works ): We have 3 cases to handle
 +
 +
1: User used –upload-limit or –download-limit from the cli We need to remove these limits from the cli arguments and then run trickle using subprocess:
 +
 +
    freezer-agent --action backup -F /etc/ -C freezer --upload-limit = 1k
 +
 +
this will be translated to:
 +
 +
    trickle -u 1024 -d -1 freezer-agent --action backup -F /etc/ -C freezer
 +
 +
2: User used config files to execute an action
 +
 +
We need to create a new config file without the limits so we will get the all the arguments provided and remove limits then run trickle using subprocess:
 +
 +
    [default]
 +
    action = backup
 +
    storage = ssh
 +
    ssh_host = 127.0.0.1
 +
    ssh_username = saad
 +
    ssh_key = /home/saad/.ssh/saad
 +
    container = /home/saad/backups_freezers
 +
    backup_name = freezer_jobs
 +
    path_to_backup = /etc
 +
    upload_limit=2k
 +
    download_limit=1k
 +
 +
and we are going to execute this job as follow:
 +
 +
    freezer-agent --config /home/user/job1.ini
 +
 +
this will be translated to:
 +
 +
    trickle -u 2048 -d 1024 freezer-agent --config /tmp/freezer_job_x21aj29
 +
 +
3: Hybrid using config file and cli options we will use a mix of both procedures: - remove limits (cli or config ) - reproduce the same command again with trickle:
 +
 +
    freezer-agent --config /home/user/job2.ini --upload-limit 1k
 +
 +
=== How incremental works ===
 +
 +
The incremental backups is one of the most crucial features. The following basic logic happens when Freezer executes:
 +
 +
* Freezer starts the execution and checks if the provided backup name for the current node already exists in Swift.
 +
* If the backup exists, then the Manifest file is retrieved. This is important as the Manifest file contains the information of the previous Freezer execution.

Revision as of 10:48, 27 June 2016

Freezer Scheduler

Client Registration

Make the agent/scheduler available to the api/cli/ui:

   freezer-scheduler register

Verify the client is registered correctly:

  • for kilo, liberty and mitaka:
   freezer-scheduler client-list
  • for newton onwards:
   freezer client-list

Scheduler Start/Stop

Start the scheduler

In order to poll the api to get new jobs, actions, etc. we need to start the scheduler

make sure to use the credentials for keystone v2.0 or v3 and you are root or Administrator on Windows

Keystone v2.0 and v3 authentication is supported for freezer

v2.0:

   export OS_BACKUP_URL=http://192.168.33.10:9090
   export OS_AUTH_URL=http://192.168.33.10:5000/v2.0
   export OS_USERNAME=admin
   export OS_PASSWORD=quiet
   export OS_TENANT_NAME=admin
   export OS_REGION_NAME=RegionOne
   export OS_IDENTITY_API_VERSION=2

v3:

   export OS_BACKUP_URL=http://192.168.33.10:9090
   export OS_AUTH_URL=http://192.168.33.10:5000/v3
   export OS_USERNAME=admin
   export OS_PASSWORD=quiet
   export OS_PROJECT_DOMAIN_NAME=admin
   export OS_USER_DOMAIN_NAME=admin
   export OS_PROJECT_NAME=admin
   export OS_REGION_NAME=RegionOne
   export OS_IDENTITY_API_VERSION=3

for windows replace export by set

   freezer-scheduler start

Verify the scheduler is running:

   freezer-scheduler status

Stop the scheduler

   freezer-scheduler stop

It can be also be started as a foreground process using the –no-daemon flag:

  freezer-scheduler --no-daemon start

This can be useful for testing purposes, when launched in a Docker container, or by a babysitting process such as systemd.

Using the Scheduler

The cli-tool version is used to manage the jobs in the API. A “job” is basically a container; a document which contains one or more “actions”. An action contains the instructions for the freezer-agent. They are the same parameters that would be passed to the agent on the command line. For example: “backup_name”, “path_to_backup”, “max_level”

To sum it up, a job is a sequence of parameters that the scheduler pulls from the API and passes to a newly spawned freezer-agent process at the right time.

The scheduler understands the “scheduling” part of the job document, which it uses to actually schedule the job, while the rest of the parameters are substantially opaque.

It may also be useful to use the “-c” parameter to specify the client-id that the scheduler will use when interacting with the API.

The purpose of the client-id is to associate a job with the scheduler instance which is supposed to execute that job.

A single openstack user could manage different resources on different nodes (and actually may even have different freezer-scheduler instances running on the same node with different local privileges, for example), and the client-id allows him to associate the specific scheduler instance with its specific jobs.

When not provided with a custom client-id, the scheduler falls back to the default which is composed from the tenant-id and the hostname of the machine on which it is running.

Create a job

The first step to use the scheduler is creating a document with the job:

   {
       "job_actions": [{
            "freezer_action": {
                 "action": "backup",
                      "mode": "fs",
                      "backup_name": "backup1",
                       "path_to_backup": "/home/me/datadir",
                       "container": "schedule_backups",
                       "log_file": "/home/me/.freezer/freezer.log"
                },
               "max_retries": 3,
               "max_retries_interval": 60
         }],
         "job_schedule": {
              "schedule_interval": "4 hours",
              "schedule_start_date": "2015-08-16T17:58:00"
        },
        "description": "schedule_backups 6"
   }


Then upload that job into the API

  • for kilo, liberty and mitaka:
  freezer-scheduler -c node12 job-create --file test_job.json
  • for newton onwards:
  freezer job-create --file test_job.json  -C node12


List jobs

  • for kilo, liberty and mitaka:
  freezer-scheduler -c node12 job-list
  • for newton onwards:
 freezer job-list -C node12

Get the content of a job

  • for kilo, liberty and mitaka:
  freezer-scheduler -c node12 job-get -j 07999ea33a494ccf84590191d6fe850c
  • for newton onwards:
 freezer job-get 07999ea33a494ccf84590191d6fe850c -C node12


Freezer Agent

Keystone v2.0 and v3 authentication is supported for freezer

v2.0:

   export OS_BACKUP_URL=http://192.168.33.10:9090
   export OS_AUTH_URL=http://192.168.33.10:5000/v2.0
   export OS_USERNAME=admin
   export OS_PASSWORD=quiet
   export OS_TENANT_NAME=admin
   export OS_REGION_NAME=RegionOne
   export OS_IDENTITY_API_VERSION=2

v3:

   export OS_BACKUP_URL=http://192.168.33.10:9090
   export OS_AUTH_URL=http://192.168.33.10:5000/v3
   export OS_USERNAME=admin
   export OS_PASSWORD=quiet
   export OS_PROJECT_DOMAIN_NAME=admin
   export OS_USER_DOMAIN_NAME=admin
   export OS_PROJECT_NAME=admin
   export OS_REGION_NAME=RegionOne
   export OS_IDENTITY_API_VERSION=3

for windows replace export by set

Advanced features

Bandwidth limitation

Trickle for bandwidth limiting ( How it works ): We have 3 cases to handle

1: User used –upload-limit or –download-limit from the cli We need to remove these limits from the cli arguments and then run trickle using subprocess:

   freezer-agent --action backup -F /etc/ -C freezer --upload-limit = 1k

this will be translated to:

   trickle -u 1024 -d -1 freezer-agent --action backup -F /etc/ -C freezer

2: User used config files to execute an action

We need to create a new config file without the limits so we will get the all the arguments provided and remove limits then run trickle using subprocess:

   [default]
   action = backup
   storage = ssh
   ssh_host = 127.0.0.1
   ssh_username = saad
   ssh_key = /home/saad/.ssh/saad
   container = /home/saad/backups_freezers
   backup_name = freezer_jobs
   path_to_backup = /etc
   upload_limit=2k
   download_limit=1k

and we are going to execute this job as follow:

   freezer-agent --config /home/user/job1.ini

this will be translated to:

   trickle -u 2048 -d 1024 freezer-agent --config /tmp/freezer_job_x21aj29

3: Hybrid using config file and cli options we will use a mix of both procedures: - remove limits (cli or config ) - reproduce the same command again with trickle:

   freezer-agent --config /home/user/job2.ini --upload-limit 1k

How incremental works

The incremental backups is one of the most crucial features. The following basic logic happens when Freezer executes:

  • Freezer starts the execution and checks if the provided backup name for the current node already exists in Swift.
  • If the backup exists, then the Manifest file is retrieved. This is important as the Manifest file contains the information of the previous Freezer execution.