码迷,mamicode.com
首页 > Web开发 > 详细

(转载)基于Bash命令行的百度云上传下载工具

时间:2014-11-23 20:15:47      阅读:298      评论:0      收藏:0      [点我收藏+]

标签:des   style   blog   http   io   ar   color   os   使用   

原文链接:http://hi.baidu.com/meoow/item/aef5814bbd5be3e1bcf451e9

 

这是我根据百度云PCS的API写的一个基于bash的命令行工具,

使用了curl, awk, sed, xxd, bash,没有使用任何其他的语言解释器做辅助

目前,上传下载,文件列表,文件信息,创建文件夹,删除文件都没问题

复制和移动总是返回参数错误,实在找不出来哪里跟API不一致了,暂时放弃。

另外离线下载相关的API也都添加了,但是因为现在百度云不能申请,我是用的是bypy的client_id,

不知道它的离线下载相关权限没有抑或其他问题,添加任务返回500。

不过总的来说基本上对于一般使用百毒云是够用了

#!/bin/bash

# Baidu Yun Command Line Interface

# Depends: bash, curl, grep, awk, sed, xxd
# (Thay are basicly builtin tools of any *nix system.)
# Additionally, fastupload depends: head, wc, md5sum or md5, cksum
# (Which are also builtin tools)

# issues:
# move, copy and all offline-download related commnads do not work
# the latters are probably prohibited by baidu
# or the current bypy client has no permission for them.

#### Variables ####
# This three to be changed to your own code #
TOKEN=00.00000000000000000000000000000000.0000000.0000000000.000000000000000000

# baidu has prohibited registering new client id
# the client_id is borrowed from bypy (Another great baidu yun cli tool written in python)
CLIENT_ID=q8WE4EpCsau1oS0MplgMKNBn

# you should at least login pan.baidu.com from browser once,
# then you can obtain the BDUSS code in the Cookie.
BDUSS=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA;
########################################

pathprefix=/apps/bypy/
ACCEPT=text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
ACCPET_LANGUAGE=zh-cn,zh;q=0.8,en-us;q=0.5,en;q=0.3
ACCPET_ENCODING=identity
CACHE_CONTROL=no-cache
CONNECTION=keep-alive
DNT=1
HOST=pcs.baidu.com
ORIGIN=http://pan.baidu.com
PRAGMA=no-cache
REFERER=http://pan.baidu.com/disk/home
UA=Mozilla/5.0 (X11; Linux; rv:5.0) Gecko/5.0 Firefox/5.0
CURL_DEFAULT_ARGS=(
    -H "Accept: $ACCEPT"
    -H "Accept-Language: $ACCPET_LANGUAGE"
    -H "Cache-Control: $CACHE_CONTROL"
    -H "Connection: $CONNECTION"
    -H "DNT: $DNT"
    -H "Host: $HOST"
    -H "Origin: $ORIGIN"
    -H "Pragma: $PRAGMA"
    -H "Referer: $REFERER"
    -H "Accept-Encoding: $ACCPET_ENCODING"
    -A "$UA"
)
####################

#### Defining functions ####

# encoding character to %xx format
urlencode() {
    local length="${#1}"
    local c
    for (( i = 0; i < length; i++ )); do
    c="${1:i:1}"
    case $c in
    testa-zA-Z0-9.~_-) printf "%s" "$c" ;;
    *) printf "%s" "$c" | xxd -p -c1 | while read x;do printf "%%%s" "$x";done
    esac
done
}

# for *basic* regexp syntax surrounded by double quotes
regexescape() {
local len="${#1}"
local c
for(( i=0;i<len;i++ ));do
c="${1:i:1}"
case "$c" in
test*\\/^$|]|[|-)
    printf "\\\\%s" "$c"
;;
*)
    printf "%s" "$c"
    esac
done
}

# baidu yun authorize
auth() {
local code
echo "http://openapi.baidu.com/oauth/2.0/authorize?response_type=code&client_id=$CLIENT_ID&redirect_uri=oob&scope=basic%20netdisk&"
code=$(curl -Ls "http://openapi.baidu.com/oauth/2.0/authorize?response_type=code&client_id=$CLIENT_ID&redirect_uri=oob&scope=basic%20netdisk&" -H "Cookie: BDUSS=$BDUSS"|grep -o readonly value="[^"]*"|grep -o [[:xdigit:]]\{32\})
curl -Ls "https://bypy-tianze.rhcloud.com/auth?code=$code&redirect_uri=oob&"|
grep -o "[^"]\{1,\}":"[^"]\{1,\}"|sed -n /access_token/s/^.*:"\([[:xdigit:].-]\{1,\}\)"$/\1/p
}

# print help
usage() {
cat <<_EOF_
    $0 <command > [arguments...]
    Command:
    auth
    upload    <FILE > [PATH] [overwrite|newcopy]
    fastupload    <FILE > [PATH] [overwrite|newcopy]
    download  <PATH > [SAVEDIR] [SAVENAME]
    list      [PATH]
    mkdir     <PATH >
    meta      <PATH >
    move      <FROM > <TO>
    copy      <FROM > <TO>
    delete    <PATH >
    search    <PATH > <WORD>
    add       <URI > [PATH]
    query     <TASKID >
    tasks
    cancel    <TASKID >
    trash
    retore    <FILEID >
    _EOF_
}

# JSON pretty printer
tokenize () {
    local GREP
    local ESCAPE
    local CHAR
    GREP=egrep -ao
    ESCAPE=(\\[^u[:cntrl:]]|\\u[0-9a-fA-F]{4})
    CHAR=[^[:cntrl:]"\\]

    local STRING="\"$CHAR*($ESCAPE$CHAR*)*\""
    local NUMBER=-?(0|[1-9][0-9]*)([.][0-9]*)?([eE][+-]?[0-9]*)?
    local KEYWORD=null|false|true
    local SPACE=[[:space:]]+

    $GREP "$STRING|$NUMBER|$KEYWORD|$SPACE|." | egrep -v "^$SPACE$"
}

printw() {
    local n="$1"
    if test[ "$n" =~ [1-9][0-9]* ];then
        :
    else
        return 0
    fi
    for((i=0;i<n;i++));do
    printf " "
done
}

pprint() {
local indent=0
local value=0
while read x; do
    case "$x" in
    {|[)
        if ((value));then
            printf " "
            value=0
        else
            printw $indent
        fi
        ((indent++))
        printf "%s\n" "$x"
    ;;
    }|])
        value=0
        ((indent > 0 && indent--))
        echo
        printw $indent
        printf "%s" "$x"
    ;;
    ,)
        value=0
        echo
    ;;
    :)
        value=1
        printf "%s" "$x"
    ;;
    *)
        if ((value));then
            value=0
            printf " "
        else
            printw $indent
        fi
        printf "%s" "$x"
    ;;
    esac
done
}

# compact json output (parse output from pprint only)
compact() {
# arguments: -f -t -b -m -s
# -f no fs_id
# -t no ctime and mtime
# -b no brackets
# -m no md5
# -s no size

#default is to remove double quotes
local sedcmd=s/"\([^"]*\)"/\1/g;
while getopts :ftbms arg;do
    case "$arg" in
    f)
        sedcmd="$sedcmd"‘/^ *fs_id/d;
    ;;
    t)
        sedcmd="$sedcmd"‘/^ *ctime/d;/^ *mtime/d;
    ;;
    m)
        sedcmd="$sedcmd"‘/^ *md5:/d;
    ;;
    s)
        sedcmd="$sedcmd"‘/^ *size:/d;
    ;;
    b) sedcmd="$sedcmd"‘/^ *{/d;s/^ *}$//;/^ *\[$/d;/^ *\]/d;s/: \[$/:/;
    ;;
    esac
done
sed "$sedcmd"
}

autounit() {
awk -v G=$((1024**3)) -v M=$((1024**2)) -v K=1024 {
    if($0~/^ *"?size"?: [[:digit:]]{1,}/){
        if($2> =G){
            size=sprintf("%0.1fG",$2/G)
        } else if($2> =M){
            size=sprintf("%0.1fM",$2/M)
        } else if($2> =K){
            size=sprintf("%0.1fK",$2/K)
        } else {
            size=$2
        }
        sub($2,size)
    }
    print
}
}

crc32() {
cksum -o3 < "$1" |
while read x y;do
    echo $x
    break
done
}
if type -t md5sum > /dev/null ;then
MD5=md5sum -b
else
MD5=md5
fi
md5() {
MD5 < "$1" |
while read x y;do
    echo "$x"
    break
done
}
md5_256k() {
head -c262144 "$1" | MD5 |
while read x y;do
    echo "$x"
    break
done
}
size() {
wc -c "$1"|
while read x y;do
    echo "$x"
    break
done
}

########################################

#### Parse command line arguments ####
case "$1" in
auth) auth
exit 0
;;
u|upload)
method=upload
shift
if test[ -z "$1" ];then
    echo "$0 upload <FILE > [PATH] [overwrite|newcopy]" >&2
    exit 1
fi
filepath="$1"
filename="$(basename "$filepath")"
shift
path="${1:-/}/$filename"
shift
ondup="${1:-overwrite}"
;;
f|fastupload)
method=rapidupload
shift
if test[ -z "$1" ];then
    echo "$0 upload <FILE > [PATH] [overwrite|newcopy]" >&2
    exit 1
fi
filepath="$1"
filename="$(basename "$filepath")"
shift
path="${1:-/}/$filename"
shift
ondup="${1:-overwrite}"
;;
d|download)
method=download
shift
if test[ -z "$1" ];then
    echo "$0 download <PATH > [SAVEDIR] [SAVENAME]" >&2
    exit 1
fi
path="$1"
shift
savedir="${1:-.}"
shift
savename="${1:-$(basename "$path")}"
savefile="$savedir/$savename"
;;
mkdir)
req=POST
method=mkdir
shift
if test[ -z "$1" ];then
    echo "$0 mkdir <PATH > " >&2
    exit 1
fi
path="$1"
;;
meta)
req=GET
method=meta
shift
if test[ -z "$1" ];then
    echo "$0 meta <PATH > " >&2
    exit 1
fi
path="$1"
;;
l|list)
method=list
shift
path="${1:-/}"
;;
m|move)
method=move
shift
from="$1"
shift
to="$1"
if test[ -z "$from" || -z "$to" ];then
    echo "$0 move <FROM > <TO> " >&2
    exit 1
fi
;;
c|copy)
method=copy
shift
from="$1"
shift
to="$1"
if test[ -z "$from" || -z "$to" ];then
    echo "$0 copy <FROM > <TO> " >&2
    exit 1
fi
;;
s|search)
method=search
shift
path="$1"
shift
wd="$1"
if test[ -z "$path" || -z "$wd" ];then
    echo "$0 search <PATH > <WORD> " >&2
    exit 1
fi
;;
del|delete)
method=delete
if test[ -z "$1" ];then
    echo "$0 delete <PATH > " >&2
    exit 1
fi
shift
path="$1"
;;
add)
method=add_task
shift
if test[ -z "$1" ];then
    echo "$0 add <LINK > " >&2
    exit 1
fi
link="$1"
shift
path="$1"
;;
query)
method=query_task
shift
if test[ -z "$1" ];then
    echo "$0 query <TASKID > " >&2
    exit 1
fi
taskid="$1"
;;
tasks)
method=list_task
;;
cancel)
method=cancel_task
shift
if test[ -z "$1" ];then
    echo "$0 cancel <TASKID > " >&2
    exit 1
fi
taskid="$1"
;;
trash)
method=listrecycle
;;
restore)
method=restore
shift
if test[ -z "$1" ];then
    echo "$0 restore <FILEID > " >&2
    exit 1
fi
fsid="$1"
;;
h|help)
usage
exit 0
;;
*)
usage
exit 0
;;
esac

path="$pathprefix""$path"
from="$pathprefix""$from"
to="$pathprefix""$to"

#### determining request type ####
case $method in
info|download|meta|list|search|generate|diff|streaming|listrecycle)
req=GET
;;
upload|delete|createsuperfile|move|copy|rapidupload|add_task|list_task|query_task|cancel_task|restore|mkdir)
req=POST
;;
esac
##################################

#### determining URI #############
case $method in
upload) uriprefix=https://c.pcs.baidu.com/rest/2.0/pcs/file
;;
download) uriprefix=https://d.pcs.baidu.com/rest/2.0/pcs/file
;;
list_task|query_task|cancel_task)
uriprefix=https://pcs.baidu.com/rest/2.0/pcs/services/cloud_dl
;;
*) uriprefix=https://pcs.baidu.com/rest/2.0/pcs/file
;;
esac
##################################


#### Building URI depends on specific method ####
uriprefix="$uriprefix?access_token=$TOKEN&method=$method"
case $method in
upload)
uri="$uriprefix&path=$(urlencode "$path")&ondup=$ondup"
;;
download|mkdir|meta|delete)
uri="$uriprefix&path=$(urlencode "$path")"
;;
list)
by=name #time name size
order=asc #asc desc
uri="$uriprefix&path=$(urlencode "$path")&by=$by&order=$order"
;;
move|copy)
uri="$uriprefix"
;;
search)
re=1
uri="$uriprefix&path=$(urlencode "$path")&wd=$(urlencode "$wd")&re=$re"
;;
rapidupload)
filelength=$(size "$filepath")
filemd5=$(md5 "$filepath")
fileslicemd5=$(md5_256k "$filepath")
filecrc32=$(crc32 "$filepath")
uri="$uriprefix&path=$(urlencode "$path")&content-length=$filelength&content-md5=$filemd5&slice-md5=$fileslicemd5&content-crc32=$filecrc32&ondup=$ondup"
;;
add_task)
uri="$uriprefix&save_path=$(urlencode "$path")&source_url=$(urlencode "$link")"
;;
query_task)
optype=1
uri="$uriprefix&task_ids=$taskid&op_type=$optype"
;;
list_task)
start=0
limit=20
asc=0 #0 1
task_info=1 #1 0
uri="$uriprefix&start=$start&limit=$limit&asc=$asc&need_task_info=$task_info"
;;
cancel_task)
uri="$uriprefix&task_id=$taskid"
;;
listrecycle)
uri="$uriprefix&start=0"
;;
restore)
uri="$uriprefix&fs_id=$fsid"
;;
esac
######################################

## Building arguments for CURL ##
curlopts=(
-X$req
"${CURL_DEFAULT_ARGS[@]}"
)
case $method in
upload)
curlopts=("${curlopts[@]}" -F file=@"$filepath" "$uri")
;;
download)
curlopts=("${curlopts[@]}" -o "$savefile" "$uri")
;;
delete|add_task|list_task|query_task|cancel_task|restore|mkdir|rapidupload)
curlopts=(
    "${curlopts[@]}"
    -H "Content-Length: 0"
    -Ls "$uri"
)
;;
move|copy)
curlopts=(
    "${curlopts[@]}"
    -H "Content-Type: application/json"
    --data param="{\"list\":[{\"from\":\"$from\",\"to\":\"$to\"}]}"
    -Ls "$uri"
)

;;
*)
curlopts=("${curlopts[@]}" -Ls "$uri")
;;
esac
#################################

### Start the operation ###
# echo "${curlopts[@]}"; exit
case $method in
download)
curl "${curlopts[@]}"
;;
*)
curl "${curlopts[@]}"|
tokenize|pprint|
sed /^ *"path": /s/^\( *"path": "\)‘"$(regexescape "$pathprefix")"‘\(.*"\)$/\1\2/|
compact -ftbm | autounit
;;
esac
##########################

 

使用,假设保存成文件名bdcurl的执行文件:
如果想不用浏览器,而是直接在命令行授权给bypy来使用的话,需要在浏览器里登录 过百毒云,然后在cookie里找到BDUSS(可以用firebug,或者使用sqlite打开浏览器的数据库也能找到),把内容复制到脚本开头的 BDUSS变量里(这个有效期比较长,所以基本只复制一次就可以了,以后token过期后重新认证时可以省略这步),
bdcurl auth
等一小会就会输出token id,把这个段字符串复制到脚本开头的TOKEN变量里,然后就可以使用了。

如果不方便获取BDUSS,就需要把终端输出的网址粘贴到浏览器里获取一个code,然后把code替换
https://bypy-tianze.rhcloud.com/auth?code=$code&redirect_uri=oob&
里面$code的部分,获得最后的access_token放到脚本开始的TOKEN变量里。
然后命令行运行:
bdcurl h 看帮助


bdcurl u file1 down #上传文件到down文件夹
bdcurl f file1 down #快速上传文件(只向服务器查询校验信息,如果百度云上有相同的文件,则直接在建立该文件,不需上传,如果返回error_code,则百毒云上没有相同文件,需要使用bdcurl u上传文件)
bdcurl d down/file1 # 下载指定路径的文件
bdcurl del down/file1 # 删除指定路径的文件
bdcurl l down # 列出 down目录下的所有文件

bubuko.com,布布扣

bubuko.com,布布扣

暂时不支持批量上传下载,毕竟百毒本身没有这样的API,所谓批量得在客户端实现,但是bash来实现相对会麻烦很多,因为没有很好的解析json的方法,如果使用python,perl之类的辅助解析,那就不如干脆直接用python或perl写了。

 

另外输出结果使用一个compact函数对其做一定程度的简化,其简化程度可通过参数指定,脚本里默认(compact -ftbm, 倒数第四行)使用最大程度简化(如,去除花括号和方括号,去除fs_id,md5校验码,size文件大小,mtime和ctime创建/修改时间 等项目,如果传递参数里去掉-s则会保留size项),可以自行参看脚本里定义compact函数接收参数的说明,选择你合适的输出简化程度

(转载)基于Bash命令行的百度云上传下载工具

标签:des   style   blog   http   io   ar   color   os   使用   

原文地址:http://www.cnblogs.com/lichmama/p/4117225.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!