Python 拉取日志文件paramiko

标签：python mongodb

paramiko的另一篇博文:http://467754239.blog.51cto.com/4878013/1619166

场景：

在游戏行业的集群中，日志分析或许是必不可少的，那么为了更方便的管理日志，就是统一存放日志，然后入库数据库

#!/usr/bin/env python
#coding:utf8

from multiprocessing import Process
from datetime import *
import paramiko
import string
import sys
import os

Hostinfo = {
	"192.168.1.100" : {
				"s200" : "/home/platform/work/business/s200/deploy/container/statistics/log/",
			    	"s202": "/home/platform/work/business/s202/deploy/container/statistics/log/"
				},
	"192.168.1.102" : {
				"s201" : "/home/platform/work/business/s201/deploy/container/statistics/log/",
				"s203" : "/home/platform/work/business/s203/deploy/container/statistics/log/"
				},
	"192.168.1.103" :  {
				"s000" : "/root/project/business/deploy/container/statistics/log/",
			    	"s100" : "/home/platform/work/business/s100/deploy/container/statistics/log/",
				"s101" : "/home/platform/work/business/s101/deploy/container/statistics/log/"
				},
	"192.168.1.104" : {
				"s001" : "/home/game/business_test/deploy/container/statistics/log/"
			},
					
	}

def sftpGet(ip,subdir,logdir,remotefile):
    Username, Port, Password = ‘root‘, 22, ‘password‘
    cmd = ‘cd %s && rm -rf *.tar.gz && ls %s && [ $? -eq 0 ] && tar zcf %s.tar.gz %s || exit 1‘ % (logdir, remotefile, remotefile, remotefile)

    s = paramiko.SSHClient()
    s.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    s.connect(hostname=ip, port=Port, username=Username, password=Password)
    s.exec_command(cmd)
    
    s = paramiko.Transport((ip,Port))
    s.connect(username=Username,password=Password)
    sftp = paramiko.SFTPClient.from_transport(s)

    formatfile = string.join([ip,subdir,remotefile],sep="_")
    localfile = formatfile + ‘.tar.gz‘

    localDir = os.path.abspath(os.path.join(os.path.dirname(__file__), "collect"))
    if not os.path.exists(localDir):
        os.makedirs(localDir)
   
    local_abspath_file = os.path.join(localDir,localfile)
    try:
	remote_tarfile = remotefile + ‘.tar.gz‘
        remote_abspath_file = os.path.join(logdir,remote_tarfile)
        sftp.get(remote_abspath_file,local_abspath_file)
        s.close()
    except:
	print ‘‘, 
    

def ipProcess():
    yesterday = datetime.now() + timedelta(days=-1)
    now = yesterday.strftime("%Y%m%d")
    remotefile = string.join(["action",now,"log"],sep=".")

    for key, value in Hostinfo.items():
	ip = key
	for subdir, logdir in value.items():
	    sftpGet(ip,subdir,logdir,remotefile)
            #p = Process(target=sftpGet,args=(ip,subdir,logdir,remotefile))
	    #p.start()
	    #p.join()

if __name__ == "__main__":
    ipProcess()

本文出自 “郑彦生” 博客，请务必保留此出处http://467754239.blog.51cto.com/4878013/1692088

Python 拉取日志文件paramiko

标签：python mongodb

原文地址：http://467754239.blog.51cto.com/4878013/1692088