Fabric自动化部署的实践

2019-05-13  本文已影响0人  _Cappuccino_

介绍

Fabric是基于Python(2.7,3.4+以上版本)实现的SSH命令行工具,简化了SSH的应用程序部署及系统管理任务,它提供了系统基础的操作组件,可以实现本地或远程shell命令,包括命令执行、文件上传、下载及完整执行日志输出等功能。Fabric在paramiko的基础上做了更高一层的封装,操作起来会更加简单。

安装

参数介绍

fab作为Fabric程序的命令行入口,提供了丰富的参数调用,命令格式如下:
fab [options] <command>[:arg1,arg2-val2,host=foo,host='h1;h2'...]...
常用参数介绍:

全部参数如下:

Options:
  -h, --help            show this help message and exit
  -d NAME, --display=NAME
                        print detailed info about command NAME
  -F FORMAT, --list-format=FORMAT
                        formats --list, choices: short, normal, nested
  -I, --initial-password-prompt
                        Force password prompt up-front
  --initial-sudo-password-prompt
                        Force sudo password prompt up-front
  -l, --list            print list of possible commands and exit
  --set=KEY=VALUE,...   comma separated KEY=VALUE pairs to set Fab env vars
  --shortlist           alias for -F short --list
  -V, --version         show program's version number and exit
  -a, --no_agent        don't use the running SSH agent
  -A, --forward-agent   forward local agent to remote end
  --abort-on-prompts    abort instead of prompting (for password, host, etc)
  -c PATH, --config=PATH
                        specify location of config file to use
  --colorize-errors     Color error output
  -D, --disable-known-hosts
                        do not load user known_hosts file
  -e, --eagerly-disconnect
                        disconnect from hosts as soon as possible
  -f PATH, --fabfile=PATH
                        python module file to import, e.g. '../other.py'
  -g HOST, --gateway=HOST
                        gateway host to connect through
  --gss-auth            Use GSS-API authentication
  --gss-deleg           Delegate GSS-API client credentials or not
  --gss-kex             Perform GSS-API Key Exchange and user authentication
  --hide=LEVELS         comma-separated list of output levels to hide
  -H HOSTS, --hosts=HOSTS
                        comma-separated list of hosts to operate on
  -i PATH               path to SSH private key file. May be repeated.
  -k, --no-keys         don't load private key files from ~/.ssh/
  --keepalive=N         enables a keepalive every N seconds
  --linewise            print line-by-line instead of byte-by-byte
  -n M, --connection-attempts=M
                        make M attempts to connect before giving up
  --no-pty              do not use pseudo-terminal in run/sudo
  -p PASSWORD, --password=PASSWORD
                        password for use with authentication and/or sudo
  -P, --parallel        default to parallel execution method
  --port=PORT           SSH connection port
  -r, --reject-unknown-hosts
                        reject unknown hosts
  --sudo-password=SUDO_PASSWORD
                        password for use with sudo only
  --system-known-hosts=SYSTEM_KNOWN_HOSTS
                        load system known_hosts file before reading user
                        known_hosts
  -R ROLES, --roles=ROLES
                        comma-separated list of roles to operate on
  -s SHELL, --shell=SHELL
                        specify a new shell, defaults to '/bin/bash -l -c'
  --show=LEVELS         comma-separated list of output levels to show
  --skip-bad-hosts      skip over hosts that can't be reached
  --skip-unknown-tasks  skip over unknown tasks
  --ssh-config-path=PATH
                        Path to SSH config file
  -t N, --timeout=N     set connection timeout to N seconds
  -T N, --command-timeout=N
                        set remote command timeout to N seconds
  -u USER, --user=USER  username to use when connecting to remote hosts
  -w, --warn-only       warn, instead of abort, when commands fail
  -x HOSTS, --exclude-hosts=HOSTS
                        comma-separated list of hosts to exclude
  -z INT, --pool-size=INT
                        number of concurrent processes to use in parallel mode

编写方式

全局属性

env对象的作用是定义fabfile的全局设定,支持多个属性,包含目标主机、用户名、密码、等角色,各属性说明如下:

env.passwords = {
    'root@192.168.56.131:22':'1234567',
    'root@192.168.56.132:22':'1234567',
    'root@192.168.56.133:22':'1234567',
    'root@192.168.56.134:22':'1234567'
}
env.roledefs = {
    'webservers':['192.168.56.131','192.168.56.132','192.168.56.133'],
    'dbserver':['192.168.56.134','192.168.56.135']
}
不同角色执行不同的任务函数

引用时使用Python修饰符的形式进行,角色修饰符下面的任务函数为其作用域,下面来看一个示例:

@roles('webservers')
def webtask():
    run('/etc/init.d/nginx start')

@roles('dbservers')
def dbtask():
    run('/etc/init.d/mysql start')

@roles('webservers','dbservers')
def pubclitasj():
    run('uptime')

def deploy():
    execute(webtask)
    execute(dbtask)
    execute(pubclitask)

常用API

Fabric提供了一组简单但功能强大的fabric.api命令集,简单地调用这些API就能完成大部分应用场景需求。Fabric常用方法及说明如下:

简单实例

查看远程主机信息

本示例调用local()方法执行本地(主控端)命令,添加"@runs_once"修饰符保证该任务函数只执行一次。调用run()方法执行远程命令。

from fabric.api import *

env.user = 'devops'
env.hosts = ['localhost']
env.password = '1234567'

@runs_once              # 查看本地系统信息,当有多台主机时只运行一次
def local_task():       # 本地任务函数
    local("uname -a")
[localhost] Executing task 'local_task'
[localhost] local: uname -a
Linux devops-virtual-machine 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Done.
动态获取远程目录列表

本示例使用"@task'修复符标志入口函数go()对外部可见,配合"@runs_once"修饰符接受用户输入,最后调用worktask()任务函数实现远程命令执行。

from fabric.api import *

env.user = 'root'
env.hosts = ['192.168.56.11','192.168.56.12']
env.password = '1234567'

@runs_once           # 主机遍历过程中,只有第一台触发此函数
def input_raw():
    return prompt("Please input directory name:",default="/home")


def worktask(dirname):
    run("ls -l "+dirname)

@task           # 限定只有go函数对fab命令可见
def go():
    getdirname = input_raw()
    worktask(getdirname)

该示例实现了一个动态输入远程目录名称,再获取目录列表的功能,由于我们只要求输入一次,在显示所有主机上该目录的列表信息,调用一个子函数input_raw()同时配置@runs_once修复符来达到此目的。

[192.168.56.11] Executing task 'go'
Please input directory name: [/home] /root
[192.168.56.11] run: ls -l /root
[192.168.56.11] out: total 4
[192.168.56.11] out: -rw-------. 1 root root 1273 May 29 11:47 anaconda-ks.cfg
[192.168.56.11] out:

[192.168.56.12] Executing task 'go'
[192.168.56.12] run: ls -l /root
[192.168.56.12] out: total 4
[192.168.56.12] out: -rw-------. 1 root root 1273 May 29 11:59 anaconda-ks.cfg
[192.168.56.12] out:


Done.
Disconnecting from 192.168.56.11... done.
Disconnecting from 192.168.56.12... done.
网关模式文件上传与执行

本示例通过Fabric的env对象定义网关模式,即俗称的中转、堡垒机环境。定义格式为"env.gateway='192.168.56.11'",其中IP“192.168.56.11”为堡垒机IP,再结合任务韩素实现目标主机文件上传与执行的操作。

from fabric.api import *
from fabric.context_managers import *
from fabric.contrib.console import confirm


env.user = 'root'
env.gateway = '192.168.56.11'                           #定义堡垒机IP,作为文件上传、执行的中转设备
env.hosts = ['192.168.56.12','192.168.56.13']

env.passwords = {
    'root@192.168.56.11:22':'1234567',                  #堡垒机账号信息
    'root@192.168.56.12:22':'1234567',
    'root@192.168.56.13:22':'1234567'
}

l_pack_path = "/home/install/nginx-1.6.3.tar.gz"        #本地安装包路径
r_pack_path = "/tmp/install"                            #远程安装包路径


@task
def put_task():
    run("mkdir -p /tmp/install")
    with settings(warn_only=True):
        result = put(l_pack_path,r_pack_path)          #上传安装包
    if result.failed and not confirm("put file failed, Continue[Y/N]?"):
        abort("Aborint file put task!")


@task
def run_task():                    #执行远程命令,安装nginx
    with cd(r_pack_path):
        run("tar -xvf nginx-1.6.3.tar.gz")
        with cd("nginx-1.6.3/"):                     #使用with继续继承/tmp/install目录位置状态
            run("./nginx_install.sh")

@task
def go():       #上传、安装
    put_task()
    run_task()
devops@devops-virtual-machine:~/devops$ fab -f simple4.py go
[192.168.56.12] Executing task 'go'
[192.168.56.12] run: mkdir -p /tmp/install
[192.168.56.12] put: /home/install/nginx-1.6.3.tar.gz -> /tmp/install/nginx-1.6.3.tar.gz
[192.168.56.12] run: tar -xvf nginx-1.6.3.tar.gz
.....
.....
.....
[192.168.56.12] out: cp conf/nginx.conf '/usr/local/nginx/conf/nginx.conf.default'
[192.168.56.12] out: test -d '/usr/local/nginx/logs'         || mkdir -p '/usr/local/nginx/logs'
[192.168.56.12] out: test -d '/usr/local/nginx/logs' ||         mkdir -p '/usr/local/nginx/logs'
[192.168.56.12] out: test -d '/usr/local/nginx/html'         || cp -R html '/usr/local/nginx'
[192.168.56.12] out: test -d '/usr/local/nginx/logs' ||         mkdir -p '/usr/local/nginx/logs'
[192.168.56.12] out: make[1]: Leaving directory `/tmp/install/nginx-1.6.3'
[192.168.56.12] out: nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
[192.168.56.12] out: nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
[192.168.56.12] out:

[192.168.56.13] Executing task 'go'
[192.168.56.13] run: mkdir -p /tmp/install
[192.168.56.13] put: /home/install/nginx-1.6.3.tar.gz -> /tmp/install/nginx-1.6.3.tar.gz
[192.168.56.13] run: tar -xvf nginx-1.6.3.tar.gz
....
....
....
[192.168.56.13] out: cp conf/nginx.conf '/usr/local/nginx/conf/nginx.conf.default'
[192.168.56.13] out: test -d '/usr/local/nginx/logs'         || mkdir -p '/usr/local/nginx/logs'
[192.168.56.13] out: test -d '/usr/local/nginx/logs' ||         mkdir -p '/usr/local/nginx/logs'
[192.168.56.13] out: test -d '/usr/local/nginx/html'         || cp -R html '/usr/local/nginx'
[192.168.56.13] out: test -d '/usr/local/nginx/logs' ||         mkdir -p '/usr/local/nginx/logs'
[192.168.56.13] out: make[1]: Leaving directory `/tmp/install/nginx-1.6.3'
[192.168.56.13] out: nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
[192.168.56.13] out: nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
[192.168.56.13] out:


Done.
Disconnecting from 192.168.56.11... done.
Disconnecting from 192.168.56.12... done.
Disconnecting from 192.168.56.13... done.

fab -Pf simple4.py go
运行结果如下:

devops@devops-virtual-machine:~/devops$ fab -Pf simple4.py go
[192.168.56.12] Executing task 'go'
[192.168.56.13] Executing task 'go'
[192.168.56.12] run: mkdir -p /tmp/install
[192.168.56.13] run: mkdir -p /tmp/install
[192.168.56.12] put: /home/install/nginx-1.6.3.tar.gz -> /tmp/install/nginx-1.6.3.tar.gz
[192.168.56.13] put: /home/install/nginx-1.6.3.tar.gz -> /tmp/install/nginx-1.6.3.tar.gz
[192.168.56.12] run: tar -xvf nginx-1.6.3.tar.gz
....
....
....
[192.168.56.12] out: nginx-1.6.3/html/index.html
[192.168.56.12] out: nginx-1.6.3/README
[192.168.56.12] out: nginx-1.6.3/nginx_install.sh
[192.168.56.12] out: nginx-1.6.3/configure
[192.168.56.12] out:

[192.168.56.12] run: ./nginx_install.sh
[192.168.56.13] run: tar -xvf nginx-1.6.3.tar.gz
[192.168.56.13] out: nginx-1.6.3/
[192.168.56.13] out: nginx-1.6.3/src/
....
....
....
[192.168.56.12] out: make[1]: Leaving directory `/tmp/install/nginx-1.6.3'
[192.168.56.12] out: nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
[192.168.56.12] out: nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
[192.168.56.12] out:
....
....
...
[192.168.56.13] out: make[1]: Leaving directory `/tmp/install/nginx-1.6.3'
[192.168.56.13] out: nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
[192.168.56.13] out: nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
[192.168.56.13] out:

文件打包、上传与校验

我们时常做一些文件包分发的工作,实施步骤一般是先压缩打包,在批量上传至目标服务器,最后做一致性校验。本示例通过put()方法实现文件的上传,通过对比本地与远程主机文件的md5,最终实现文件一致性校验。

from fabric.api import *
from fabric.context_managers import *
from fabric.contrib.console import confirm

env.user = 'root'
env.hosts = ['192.168.56.12','192.168.56.13']
env.passwords = {
    'root@192.168.56.12:22':'1234567',
    'root@192.168.56.13:22':'1234567',
}

@runs_once
def tar_task():             #本地打包任务函数,只执行一次
    with lcd('/home/devops/devops'):
        local("tar -zcf devops.tar.gz  *")

@task
def put_task():                 #上传文件任务函数
    run("mkdir -p /root/devops")
    with cd("/root/devops"):
        with settings(warn_only=True):                  #put(上传)出现异常时继续执行,非终止
            result = put("/home/devops/devops/devops.tar.gz","/root/devops/devops.tar.gz")
        if result.failed and not confirm("put file failed.Continue[Y/N]?"):
            abort("Aborting file put task!")                        #出现异常时,确认用户是否继续,(Y继续)


@task
def check_task():               #校验文件任务函数
    with settings(warn_only=True):
        #本地local命令需要配置capture=True才能捕获返回值
        lmd5 = local("md5sum /home/devops/devops/devops.tar.gz",capture=True).split(' ')[0]
        rmd5 = run("md5sum /root/devops/devops.tar.gz").split(' ')[0]
    if lmd5 == rmd5:                #对比本地及远程文件md5信息
        prompt("OK")
    else:
        prompt("ERROR")

@task
def go():
    tar_task()
    put_task()
    check_task()
devops@devops-virtual-machine:~/devops$ fab -f simple5.py go
[192.168.56.12] Executing task 'go'
[localhost] local: tar -zcf devops.tar.gz  *
[192.168.56.12] run: mkdir -p /root/devops
[192.168.56.12] put: /home/devops/devops/devops.tar.gz -> /root/devops/devops.tar.gz
[localhost] local: md5sum /home/devops/devops/devops.tar.gz
[192.168.56.12] run: md5sum /root/devops/devops.tar.gz
[192.168.56.12] out: a1cf2be82647cbed0d41514bd80373de  /root/devops/devops.tar.gz
[192.168.56.12] out:

OK
[192.168.56.13] Executing task 'go'
[192.168.56.13] run: mkdir -p /root/devops
[192.168.56.13] put: /home/devops/devops/devops.tar.gz -> /root/devops/devops.tar.gz
[localhost] local: md5sum /home/devops/devops/devops.tar.gz
[192.168.56.13] run: md5sum /root/devops/devops.tar.gz
[192.168.56.13] out: a1cf2be82647cbed0d41514bd80373de  /root/devops/devops.tar.gz
[192.168.56.13] out:

OK

Done.
Disconnecting from 192.168.56.12... done.
Disconnecting from 192.168.56.13... done.

部署LNMP业务服务环境

本示例通过env.roledefs定义不同主机角色,在使用"@roles('webservers')"修复符绑定到对应的任务函数,实现不同角色主机的部署差异。

from fabric.colors import *
from fabric.api import *

env.user = 'root'
env.roledefs = {
    'webservers':['192.168.56.11','192.168.56.12'],
    'dbservers':['192.168.56.13']
}

env.passwords = {
    'root@192.168.56.11:22':'1234567',
    'root@192.168.56.12:22':'1234567',
    'root@192.168.56.13:22':'1234567',
}

@roles('webservers')                      #使用webtask任务函数引用'webservers'角色修复符
def webtask():
    print(yellow('Install nginx php php-fpm...'))
    with settings(warn_only=True):
        run("yum -y install nginx")
        run("yum -y install php-fpm php-mysql php-mbstring php-xml php-mcrypt php-gd")
        run("chkconfig --levels 235 php-fpm on")
        run("chkconfig --levels 235 nginx on")

@roles('dbservers')                       #dbtask任务函数引用'dbservers'角色修复符
def dbtask():
    print(yellow("Install Mysql..."))
    with settings(warn_only=True):
        run("yum -y install mysql mysql-server")
        run("chkconfig --levels 235 mysqld on")

@roles('webservers','dbservers')           #publictask任务函数同时引用两个角色修复符
def publictask():                          #部署公共类环境,如epel、ntp等
    print(yellow("Install epel ntp...."))
    with settings(warn_only=True):
        run("wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo")
        run("yum -y install ntp")

def deploy():
    execute(publictask)
    execute(webtask)
    execute(dbtask)
devops@devops-virtual-machine:~/devops$ fab -Pf simple6.py deploy
[192.168.56.11] Executing task 'publictask'
[192.168.56.12] Executing task 'publictask'
[192.168.56.13] Executing task 'publictask'
Install epel ntp....
[192.168.56.13] run: wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
Install epel ntp....
[192.168.56.12] run: wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
Install epel ntp....
[192.168.56.11] run: wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
[192.168.56.12] out: --2018-06-23 20:32:30--  http://mirrors.aliyun.com/repo/epel-7.repo
[192.168.56.11] out: --2018-06-23 20:32:30--  http://mirrors.aliyun.com/repo/epel-7.repo
[192.168.56.13] out: --2018-06-23 20:32:30--  http://mirrors.aliyun.com/repo/epel-7.repo
....
[192.168.56.13] run: yum -y install ntp
[192.168.56.12] run: yum -y install ntp
[192.168.56.11] run: yum -y install ntp
....
....
....
[192.168.56.11] Executing task 'webtask'
[192.168.56.12] Executing task 'webtask'
Install nginx php php-fpm...
[192.168.56.11] run: yum -y install nginx
Install nginx php php-fpm...
[192.168.56.12] run: yum -y install nginx
....
....
....
[192.168.56.13] Executing task 'dbtask'
Install Mysql...
[192.168.56.13] run: rpm -ivh http://dev.mysql.com/get/mysql-community-release-el6-5.noarch.rpm
.....
.....
.....
[192.168.56.13] run: chkconfig --levels 235 mysqld on

Done.

生产环境代码包发布管理

程序生产环境的发布是业务上线的最后一个环境,要求具备源码打包、发布、切换、回滚、版本管理等功能。本示例实现了这一套流程功能,其中版本切换与回滚使用了Linux下的软链接实现。

from fabric.api import *
from fabric.colors import *
from fabric.context_managers import *
from fabric.contrib.console import confirm
import time

env.user = 'root'
env.host = ['192.168.56.12','192.168.56.13']
env.passwords = {
    'root@192.168.56.12:22':'1234567',
    'root@192.168.56.13:22':'1234567',
}

env.project_dev_source = '/data/dev/Lwebadmin/'              #开发服务器项目主目录
env.project_tar_source = '/data/dev/releases/'               #开发服务器项目压缩包存储目录
env.project_pack_name = 'release'                            #项目压缩包前缀,文件名为release.tar.gz

env.deploy_project_root = '/data/www/Lwebadmin/'            #项目生产环境主目录
env.deploy_release_dir = 'releases'                         #项目发布目录,位于主目录下面
env.deploy_current_dir = 'current'                          #对外服务的当前版本软链接
env.deploy_version = time.strftime("%Y%m%d")+"v2"           #版本号

@runs_once
def input_versionid():                                      #获得用户输入的版本号,以便做版本回滚操作
    return prompt("Please input project rollback version ID:",default="")

@task
@runs_once
def tar_source():                                           #打包本地项目主目录,并将压缩包存储到本地压缩包目录
    prompt(yellow("Creating source package...."))
    with lcd(env.project_dev_source):
        local("tar -zcf %s.tar.gz ." %(env.project_tar_source + env.project_pack_name))
    prompt(green("Creating source package success!"))

@task
def put_package():                                          #上传任务函数
    prompt(yellow("Start put package...."))
    with settings(warn_only=True):
        with cd(env.deploy_project_root + env.deploy_release_dir):
            run("mkdir %s" %(env.deploy_version))           #创建版本目录
    env.deploy_full_path = env.deploy_project_root + env.deploy_release_dir + "/" + env.deploy_version
    with settings(warn_only=True):                          #上传项目压缩包至此目录
        result = put(env.project_tar_source + env.project_pack_name + ".tar.gz",env.deploy_full_path)
    if result.failed and not ("put file failed,Continue[Y/N]?"):
        abort("Aborting file put task!")

    with cd(env.deploy_full_path):                          #成功解压后删除压缩包
        run("tar -zxvf %s.tar.gz" %(env.project_pack_name))
        run("rm -rf %s.tar.gz" %(env.project_pack_name))

    print(green("Put & untar package success!"))

@task
def make_symlink():                                         #为当前版本目录做软链接
    print(yellow("update current symlink"))
    env.deploy_full_path = env.deploy_project_root + env.deploy_release_dir + "/" + env.deploy_version
    with settings(warn_only=True):                           #删除软链接,重新创建并指定软链接源目录,新版本生效
        run("rm -rf %s" %(env.deploy_project_root + env.deploy_current_dir))
        run("ln -s %s %s" %(env.deploy_full_path,env.deploy_project_root + env.deploy_current_dir))
    print(green("make symlink success!"))

@task
def rollback():                                             #版本回滚任务函数
    print(yellow("rollback project version"))
    versionid = input_versionid()                           #获取用户输入的回滚版本号
    if versionid == '':
        abort("Project version ID error,abort!")

    env.deploy_full_path = env.deploy_project_root + env.deploy_release_dir + "/" + versionid
    run("rm -r %s" %(env.deploy_project_root + env.deploy_current_dir))
    run("ln -s %s %s" %(env.deploy_full_path,env.deploy_project_root + env.deploy_current_dir))     #删除软链接,重新创建并指定软链接源目录,新版本生效
    print(green("rollback sucess!"))


@task
def go():               #自动化程序版本发布入口函数
    tar_source()
    put_package()
    make_symlink()

在生产环境中将站点的根目录指向"/data/www/Lwebadmin/current",由于使用Linux软链接做切换,管理员的版本发布、回滚操作用户无感知。


转载自:https://www.cnblogs.com/hwlong/p/9216492.html

上一篇下一篇

猜你喜欢

热点阅读