程序员瑞士军刀之 Fabric

2016-09-07 本文已影响747人老瓦在霸都

Fabric是一个Python库, 也是一个命令行工具, 通过 SSH 来做应用程序的部署和系统管理任务

它可以执行本地的远程的系统命令, 上传下载文件, 以及其他能用Python编程完成的任务

其实它一个工具框架, 执行一个默认的 python 文件 fabfile.py

简单写个小例子

$vi fabfile

    from fabric.api import *

    env.hosts = ['10.224.64.106']
    env.user   = "root"
    env.password = "pass"
    
    def freedisk(param='-h'):
        cmd = 'df ' + param
        run(cmd)
    
    def listfile(folder='~'):
        cmd = 'ls -l ' + folder
        run(cmd)
        
    def pullcodes(folder='/workspace/cpp/snippets'):
        with cd(folder):
            run("git pull origin master")

# 察看远程服务器上的磁盘剩余空间
$ fab listfile:folder=/home/walter

为安全起见, 不用在文件中存放密码, 在命令行提示输入

$ fab -u root -I -H 10.224.64.106 freedisk
更好的做法是把本机私钥预先拷贝到目标服务器上, 这样就不用输入密码了

    1. 在本机上生成公钥  ~/.ssh/id_rsa.pub
    ssh-keygen -t rsa
    
    2. 拷贝此公钥到目标服务器 10.224.64.106 上
    scp id_rs.pub root@10.224.64.106:/root
    
    3. 目标服务器 10.224.64.106 上
    cat id_rsa.pub >> ~/.ssh/authorized_keys
    chmod 700 ~/.ssh/authorized_keys

常用方法

run (fabric.operations.run)
sudo (fabric.operations.sudo)
local (fabric.operations.local)
get (fabric.operations.get)
put (fabric.operations.put)
prompt (fabric.operations.prompt)
reboot (fabric.operations.reboot)

常用函数

cd (fabric.context_managers.cd)
lcd (fabric.context_managers.lcd)
path (fabric.context_managers.path)
settings (fabric.context_managers.settings)
prefix (fabric.context_managers.prefix)

例子:批量上传下载文件

from fabric.api import *
from fabric.context_managers import *
from fabric.contrib.console import confirm 

env.user='root'
env.hosts=['10.224.64.106'] 
env.passwords = { 
    'root@10.224.64.106:22': 'password'
  } 

local_dir='/workspace/cpp/codelab'
remote_dir = '/home/walter/cpp/codelab'
file_list = [
    'src/FileUtils.cpp',
    'src/FileUtils.h',
    'src/Makefile.am',
    'src/StringUtils.cpp'
]

@task
def hostinfo():
    run('uname -s')
          
@task
def upload(): #upload file task 
    with cd(remote_dir) :
        for filename in file_list:
            local_file  = local_dir  + "/" + filename
            remote_file = remote_dir + "/" + filename
            #print local_file, " to ", remote_file
            with settings(warn_only=True):    #when upload error,continue 
                result = put(local_file, remote_file) 
            if result.failed and not confirm("put file failed,Continue[Y/N]?"): 
                abort("Aborting file put task!")


@task
def download(): #upload file task 
    with cd(remote_dir) :
        for filename in file_list:
            local_file  = local_dir  + "/" + filename
            remote_file = remote_dir + "/" + filename
            #print local_file, " to ", remote_file
            with settings(warn_only=True):    #when upload error,continue 
                result = get(remote_file,local_file)
            if result.failed and not confirm("put file failed,Continue[Y/N]?"): 
                abort("Aborting file put task!")

高阶用法

设置角色role来指定远程的服务器范围
或者直接用字典由输入参数指定, 例如:

# usage:  
# fab localpull:rtc
# fab checkfiles:hf2
from fabric.api import *
from fabric.context_managers import *
from fabric.contrib.console import confirm 


env.user = 'root'
env.roledefs = {
    'qa': ['root@10.224.57.202:22'],
    'dev': ['root@10.224.64.106:22']
}

env.passwords = { 
    'root@10.224.57.202:22': 'pass',
    'root@10.224.64.106:22': 'pass',
    'root@10.224.64.107:22': 'pass'
  } 

@roles('dev')
@task
def localpull(app='web'):
    if app == 'web':
        code_dir = '/workspace/walter/hfweb'
        with lcd(code_dir):
            local("git pull origin master")
    elif app == 'rtc':
        code_dir = '/workspace/walter/hfrtc'
        with lcd(code_dir):
            local("git pull origin master")
            local("git branch -l")

test_servers = {'hf1':['root@10.224.64.46:22'],
    'hf2':['root@10.224.64.106:22'],
    'hf3':['root@10.224.64.107:22']}

@task
def listfiles():
    run("ls -l")

@task
def checkfiles(target_env='hf2'):
    execute("listfiles", hosts=test_servers[target_env])

FAQ

问题: 切换环境

写一个字典对象,在参数里传入环境类型, 再组成所需的环境变量


environments = {
    "integration": {
         "ApiServiceUrl" :"https://checklist-int.walterfan.com/checklist/api/v1", 
         "environment":"integration"
    },
    "production":{
         "ApiServiceUrl" :"https://checklist.walterfan.com/checklist/api/v1", 
         "environment":"production"
    },
    "lab":{
         "ApiServiceUrl" :"https://checklist-lab.walterfan.com/checklist/api/v1", 
         "environment":"lab"
    },
    "local":{
         "ApiServiceUrl" :"https://localhost:2008/checklist/api/v1", 
         "environment":"lab"
    }

}

def get_env_vars(env_type):
    defined_vars = " "
    for key, value in environments[env_type].iteritems():
        defined_vars = defined_vars + " -D%s=%s" %(key, value)
    return defined_vars;

读取配置文件

以 json file 为例

class ProvisionConfig:
    def __init__(self, json_file):

        self.read_config(json_file)

        self.base_path = self.config_data['basePath']
        self.username = self.config_data['username']
        self.locale = self.config_data['locale']

    def read_config(self, json_file):
        json_data=open(json_file)
        self.config_data = json.load(json_data)

问题: 如何获取命令行的输出结果

比如想获得 docker 容器的id, 可以用如下命令

docker ps -aqf name=jenkins

用 subprocess.check_output 方法就可以获取输出，以下面这个函数为例

def get_container_id(container_name):
    str_filter = "-aqf name=%s" % container_name;
    arr_cmd = ["docker", "ps", str_filter]
    container_id = subprocess.check_output(arr_cmd).strip()
    return container_id

问题: fab put error: paramiko.ssh_exception.SSHException: Channel closed

解决方法:

编辑 /etc/ssh/sshd_config:
<pre>
vi /etc/ssh/sshd_config
</pre>
加上一行 Subsystem sftp internal-sftp
<pre>
Port 22
Protocol 2
LogLevel INFO
X11Forwarding no
MaxAuthTries 4
IgnoreRhosts yes
HostbasedAuthentication no
PermitRootLogin yes
PermitEmptyPasswords no
PermitUserEnvironment no
Ciphers aes128-ctr,aes192-ctr,aes256-ctr
ClientAliveInterval 600
Banner /etc/issue
Subsystem sftp internal-sftp
</pre>
保存并重启 SSH server:

    service sshd restart