JSON+YAML初步学习+ciscoconfparse

Git

• git clone

• 在github.com右上角点击加号创建新的repository
• 在Linux或Mac命令行下，找到你想存放这个repository的目录，然后git clone 某个repository的https链接
• git clone 除了上面那种方法还可以通过SSH的方式，但是我还没试过，那种是clone一个 .git为结尾的链接好像
• cd 到.git 目录下，ll是看不出来的，只能ls -al，看到objects目录下有很多文件，其实每个文件都对应着一个“版本”，git的版本控制就在本地会把能保存的版本都保存下来。

• 推送本地更新到repository

1) 创建新文件到master

• 每当创建好一个新文件或者更新了一个文件，git add xxx.py
• git diff查看自己改动了什么
• 然后 git commit -m “some message for the commit”
• 然后 git push origin master
• origin master 的意思就是推到master
• git log 查看每次推送
• git status查看当前状态

2) 创建一个branch

• git branch test命令创建一个叫做test的branch
• git branch 直接查看现在在哪个branch
• git checkout test切到test 这个branch
• git status
• 在branch test创建或修改完的文件，git push的时候，git push origin test
• 当一个branch的文件足够“成熟”了，你想把它merge到master对吧，那就先切到master, 然后git merge test，当然这是merge整个test这个branch，如果想只merge某些文件，那就用下面这段(截取自这个链接http://jasonrudolph.com/blog/2009/02/25/git-tip-how-to-merge-specific-files-from-another-branch/)

$ git branch
* master
  twitter_integration
$ git checkout twitter_integration app/models/avatar.rb db/migrate/20090223104419_create_avatars.rb test/unit/models/avatar_test.rb test/functional/models/avatar_test.rb
$ git status
# On branch master
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# new file: app/models/avatar.rb
# new file: db/migrate/20090223104419_create_avatars.rb
# new file: test/functional/models/avatar_test.rb
# new file: test/unit/models/avatar_test.rb
#
$ git commit -m "‘Merge‘ avatar code from ‘twitter_integration‘ branch"
[master]: created 4d3e37b: "‘Merge‘ avatar code from ‘twitter_integration‘ branch"
4 files changed, 72 insertions(+), 0 deletions(-)
create mode 100644 app/models/avatar.rb
create mode 100644 db/migrate/20090223104419_create_avatars.rb
create mode 100644 test/functional/models/avatar_test.rb
create mode 100644 test/unit/models/avatar_test.rb

• 关于git的基本概念的一个很好的介绍的http://gitolite.com/gcs.html#(15)

3) fork
fork的repository，在被你自己更新了之后，不会影响原owner的内容，除非原owner启动pull request

YAML+JSON

• 关于为什么我们需要YAML和JSON就不说了，Kirk首先介绍了为什么用YAML，主要是因为他们能处理比较复杂的data structure

• 在介绍YAML的时候，Kirk首先使用了list，并且展示了list的几个features，比如下面这个把一个dictionary加到列表里去 my_list.append({}), 千万注意不要把{}的两边加上quote，如果加上quote就成了一个string了

>>> import yaml
>>> my_list = range(8)
>>> my_list
[0, 1, 2, 3, 4, 5, 6, 7]
>>> my_list.append(‘whatever‘)
>>> my_list.append(‘hello‘)
>>> my_list.delete(‘{}‘) # this is not a dictionary
>>> my_list.append({})
>>> my_list[-1][‘ip_addr‘] = ‘10.10.10.239‘ # -1 means the last item of list, it‘s a dictionary, and we are here define that dictionary
>>> my_list
[0, 1, 2, 3, 4, 5, 6, 7, ‘whatever‘, ‘hello‘, ‘{}‘, {‘ip_addr‘: ‘10.10.10.239‘}]
>>> my_list[-1][‘attribs‘] = range(7)
>>> my_list
[0, 1, 2, 3, 4, 5, 6, 7, ‘whatever‘, ‘hello‘, ‘{}‘, {‘attribs‘: [0, 1, 2, 3, 4, 5, 6], ‘ip_addr‘: ‘10.10.10.239‘}]
#这个地方要注意的是，[-1]一直是那个dictionary，刚刚把range(7)放进list仅仅是放进dictionary，而不是再站一个[-1]的位置，我第一次敲这个地方#的时候理解错了
>>> type(my_list[-1]) #使用type功能看是什么数据类型
<type ‘dict‘>

• 把list转换成YAML格式

>>> yaml.dump(my_list)
"- 0\n- 1\n- 2\n- 3\n- 4\n- 5\n- 6\n- 7\n- whatever\n- hello\n- ‘{}‘\n- attribs: [0, 1, 2, 3, 4, 5, 6]\n ctest: the_second_key\n ip_addr: 10.10.10.239\n"
>>>
>>> yaml.dump(my_list, default_flow_style = True)
"[0, 1, 2, 3, 4, 5, 6, 7, whatever, hello, ‘{}‘, {attribs: [0, 1, 2, 3, 4, 5, 6], ctest: the_second_key,\n ip_addr: 10.10.10.239}]\n"
>>># 注意YAML里string两端没有quote
>>> print yaml.dump(my_list, default_flow_style = True)
[0, 1, 2, 3, 4, 5, 6, 7, whatever, hello, ‘{}‘, {attribs: [0, 1, 2, 3, 4, 5, 6], ctest: the_second_key,
  ip_addr: 10.10.10.239}]

>>> print yaml.dump(my_list, default_flow_style = False) #这种更适合人类阅读
- 0
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- whatever
- hello
- ‘{}‘
- attribs:
  - 0
  - 1
  - 2
  - 3
  - 4
  - 5
  - 6
  ctest: the_second_key
  ip_addr: 10.10.10.239

>>>

• 把YAML写进某YAML文件

#接着上面的来
>>> with open("some_file.yml", "w") as stream:
# 我的理解就是open一个“莫须有”的文件some_file.yml，反正他会自动创建，如果当前目录没有，用write的方式打开它，as sth，我喜欢命名成stream，因为我的理解是它就像一个stream，一串内存流一样的东西
... stream.write(yaml.dump(my_list, default_flow_style = False)) #调用这串stream的写功能，就能写进这个some_file.yml
...
>>>

• 读一个YAML文件

>>> with open("some_file.yml", ) as stream: #读刚刚那个yml文件
... new_list = yaml.load(stream)
...
>>> new_list
[0, 1, 2, 3, 4, 5, 6, 7, ‘whatever‘, ‘hello‘, ‘{}‘, {‘attribs‘: [0, 1, 2, 3, 4, 5, 6], ‘ip_addr‘: ‘10.10.10.239‘, ‘ctest‘: ‘the_second_key‘}]
>>> my_list
[0, 1, 2, 3, 4, 5, 6, 7, ‘whatever‘, ‘hello‘, ‘{}‘, {‘attribs‘: [0, 1, 2, 3, 4, 5, 6], ‘ip_addr‘: ‘10.10.10.239‘, ‘ctest‘: ‘the_second_key‘}]
>>> new_list == my_list
True #新的这个list跟老的这个完全是一样的
>>>

• JSON部分

其实和YAML很像，但是需要注意的是，json.dumps是用来读的，这里有个s，也是以string的形式读出的意思，json.dump没有s的这个是用来写的，json不接受末尾有逗号的list，哪怕是其中某个dictionary的末尾item带着一个多余的comma也不可以，虽然Python本身对于末尾有逗号的list是无所谓的

? ~ python
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> my_list = range(8)
>>> my_list.append(‘whatever‘)
>>> my_list.append(‘hello‘)
>>> my_list.append({})
>>> my_list.[-1][‘ip_addr‘] = ‘10.19.25.1‘ #注意这里的dot是不应该有的
  File "<stdin>", line 1
  my_list.[-1][‘ip_addr‘] = ‘10.19.25.1‘
  ^
SyntaxError: invalid syntax
>>> my_list[-1][‘ip_addr‘] = ‘10.19.25.1‘
>>> my_list[-1][‘attribs‘] = range(5)
>>>
>>> my_list
[0, 1, 2, 3, 4, 5, 6, 7, ‘whatever‘, ‘hello‘, {‘attribs‘: [0, 1, 2, 3, 4], ‘ip_addr‘: ‘10.19.25.1‘}]
>>>
>>> json.dumps(my_list) #注意这里是dumps，也注意下面的输出，两头有quotes
‘[0, 1, 2, 3, 4, 5, 6, 7, "whatever", "hello", {"attribs": [0, 1, 2, 3, 4], "ip_addr": "10.19.25.1"}]‘
>>>
>>> string_or_not = json.dumps(my_list) #看这个地方，说明了dumps的确是以string的形式dump出来
>>> type(string_or_not)
<type ‘str‘>
>>>
>>>

• 写和读JSON file，注意这里的dump没有s

>>> with open("my_file.json", "w") as stream:
... json.dump(my_list, stream)
...
>>>
>>> with open("my_file.json") as stream:
... new_list = json.load(stream)
...
>>> new_list
[0, 1, 2, 3, 4, 5, 6, 7, u‘whatever‘, u‘hello‘, {u‘attribs‘: [0, 1, 2, 3, 4], u‘ip_addr‘: u‘10.19.25.1‘}]
>>>

• pprint介绍

>>> from pprint import pprint as pp #这个地方也是在炫技
>>> pp(new_list)
[0,
 1,
 2,
 3,
 4,
 5,
 6,
 7,
 u‘whatever‘,
 u‘hello‘,
 {u‘attribs‘: [0, 1, 2, 3, 4], u‘ip_addr‘: u‘10.19.25.1‘}]
>>>

ciscoconfparse

• pip install 即可安装，可以处理IOS和类IOS风格的配置文本文件，它有个parent和children的概念，用于表示配置的段落和缩进

• .find_objects(r”^something”) “”里是正则，所以这里有^做开头

? Desktop python
Python 2.7.5 (default, Mar 9 2014, 22:15:05) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ciscoconfparse import CiscoConfParse
>>> cisco_cfg = CiscoConfParse("test_file.txt")
>>> cisco_cfg
<CiscoConfParse: 718 lines / syntax: ios / comment delimiter: ‘!‘ / factory: False>
>>> 
>>> intface = cisco_cfg.find_object(r"^interface")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: ‘CiscoConfParse‘ object has no attribute ‘find_object‘
>>> intface = cisco_cfg.find_objects(r"^interface")  # 注意是find_objects，这里有个s
>>> intface
[<IOSCfgLine # 13 ‘interface Ethernet0/0‘>, <IOSCfgLine # 20 ‘interface Ethernet0/1‘>, <IOSCfgLine # 27 ‘interface Ethernet0/1.829‘>, <IOSCfgLine # 33 ‘interface Ethernet0/1.837‘>, <IOSCfgLine # 39 ‘interface Ethernet0/2‘>, <IOSCfgLine # 42 ‘interface Ethernet0/3‘>, <IOSCfgLine # 47 ‘interface Management0/0‘>]
>>> type(intface)     # type(x)真的好好用，所以每一个被抓出来的interface都是一个string
<type ‘list‘>
>>> 
>>> 
>>> for i in intface:
... print i.text   # 必须print i.text才能打印出人看的格式，这里要说的是，i必须是这个list中的某个string，直接list是不能调用.text的，所以这里写了循环去调用
... 
interface Ethernet0/0
interface Ethernet0/1
interface Ethernet0/1.829
interface Ethernet0/1.837
interface Ethernet0/2
interface Ethernet0/3
interface Management0/0
>>>

• 注意上面被抓出来的interface形成的list，他们的每个interface又是parent，所以每个parent的children又是list，如下

>>> fa3 = intface[5]    #我需要先赋值，把刚刚抓出来的list的第6??个string, 也就是[5] 赋值给fa3，fa3是e0/3口
>>> fa3
<IOSCfgLine # 42 ‘interface Ethernet0/3‘>
>>> 
>>> 
>>> fa3.children  # e0/3的children
[<IOSCfgLine # 43 ‘ no nameif‘ (parent is # 42)>, <IOSCfgLine # 44 ‘ security-level 100‘ (parent is # 42)>, <IOSCfgLine # 45 ‘ no ip address‘ (parent is # 42)>]
>>> fa3.children.text  # 看这里，children也是list，不能直接调用.text，
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: ‘list‘ object has no attribute ‘text‘
>>> 
>>> 
>>> type(fa3.children)
<type ‘list‘>
>>> for child in fa3.children:  #依然需要借助循环才可以
... print child.text
... 
 no nameif
 security-level 100
 no ip address
>>>

• string_somthing.all_children 注意打印出来的也是list，我们找一个有孙子辈儿缩进的配置段落，就用global-policy吧

>>> global_policy = cisco_cfg.find_objects(r"^policy-map global_policy")
>>> global_policy
[<IOSCfgLine # 703 ‘policy-map global_policy‘>]
>>> str_global_policy = global_policy[0]
>>> str_global_policy.text
‘policy-map global_policy‘
>>> str_global_policy.all_children
[<IOSCfgLine # 704 ‘ class by-pass‘ (parent is # 703)>, <IOSCfgLine # 705 ‘ set connection advanced-options tcp-state-bypass‘ (parent is # 704)>, <IOSCfgLine # 706 ‘ class inspection_default‘ (parent is # 703)>, <IOSCfgLine # 707 ‘ inspect ftp ‘ (parent is # 706)>, <IOSCfgLine # 708 ‘ inspect icmp ‘ (parent is # 706)>, <IOSCfgLine # 709 ‘ inspect icmp error ‘ (parent is # 706)>]
>>>
>>> for child in str_global_policy.all_children:
... print child.text
...
 class by-pass # 儿子辈儿
  set connection advanced-options tcp-state-bypass  #孙子辈儿
 class inspection_default
  inspect ftp
  inspect icmp
  inspect icmp error
>>>

• str_somthing.is_child or .is_parent 这个内置的功能是方便检查一个string是不是child或者parent
  比如下面的这个class by-pass既是parent也是child

>>> from pprint import pprint as pp  #这里用了pprint，还是很好用的，起码给分个行
>>> pp (str_global_policy.all_children)
[<IOSCfgLine # 704 ‘ class by-pass‘ (parent is # 703)>,
 <IOSCfgLine # 705 ‘ set connection advanced-options tcp-state-bypass‘ (parent is # 704)>,
 <IOSCfgLine # 706 ‘ class inspection_default‘ (parent is # 703)>,
 <IOSCfgLine # 707 ‘ inspect ftp ‘ (parent is # 706)>,
 <IOSCfgLine # 708 ‘ inspect icmp ‘ (parent is # 706)>,
 <IOSCfgLine # 709 ‘ inspect icmp error ‘ (parent is # 706)>]
>>>
>>> str_test = str_global_policy.all_children[0]
>>> pp (str_test)
<IOSCfgLine # 704 ‘ class by-pass‘ (parent is # 703)>
>>> str_test.is_child
True
>>> str_test.is_parent
True
>>>

• .all_parents 就是把父亲辈和爷爷辈儿的都打印出来

>>> str_test.all_parents
[<IOSCfgLine # 703 ‘policy-map global_policy‘>]
>>> 
>>> 
>>> str_test2 = str_global_policy.all_children[-1]
>>> str_test2.all_parents
[<IOSCfgLine # 703 ‘policy-map global_policy‘>, <IOSCfgLine # 706 ‘ class inspection_default‘ (parent is # 703)>]
>>>
>>> pp (str_test2.all_parents)
[<IOSCfgLine # 703 ‘policy-map global_policy‘>,
 <IOSCfgLine # 706 ‘ class inspection_default‘ (parent is # 703)>]
>>>

• .find_objects_w_child(parentspec=r”^something”, childspec=r”something”) 只搜索特定的parent和child

>>> cisco_cfg.find_objects_w_child(parentspec=r"policy-map", childspec=r"inspection_default")
[<IOSCfgLine # 703 ‘policy-map global_policy‘>]
>>>

• 当然ciscoconfparse这货也可以处理cisco风格的其他类型的输出，比如下面这段show interface

>>> show_int = CiscoConfParse("hkg_vpn_router_show_int.txt")
>>> show_int
<CiscoConfParse: 309 lines / syntax: ios / comment delimiter: ‘!‘ / factory: False>
>>>

>>> int_status = show_int.find_objects(r"^line protocol is")  # 这里不应该用^作为regex的开头，因为这里是正则
>>> pp (int_status)
[]
>>> int_status = show_int.find_objects(r"line protocol is")  # 不带^才能找到
>>> pp (int_status)
[<IOSCfgLine # 1 ‘GigabitEthernet0/0 is up, line protocol is up ‘>,
 <IOSCfgLine # 30 ‘GigabitEthernet0/1 is up, line protocol is up ‘>,
 <IOSCfgLine # 59 ‘GigabitEthernet0/2 is down, line protocol is down ‘>,
 <IOSCfgLine # 86 ‘Loopback0 is up, line protocol is up ‘>,
 <IOSCfgLine # 106 ‘Loopback1 is up, line protocol is up ‘>,
 <IOSCfgLine # 126 ‘Tunnel17 is up, line protocol is down ‘>,
 <IOSCfgLine # 157 ‘Tunnel38 is up, line protocol is down ‘>,
 <IOSCfgLine # 188 ‘Tunnel102 is up, line protocol is up ‘>,
 <IOSCfgLine # 219 ‘Tunnel103 is up, line protocol is up ‘>,
 <IOSCfgLine # 250 ‘Tunnel200 is up, line protocol is up ‘>,
 <IOSCfgLine # 279 ‘Tunnel201 is up, line protocol is up ‘>]
>>> for int in int_status:
... print int.text
... 
GigabitEthernet0/0 is up, line protocol is up 
GigabitEthernet0/1 is up, line protocol is up 
GigabitEthernet0/2 is down, line protocol is down 
Loopback0 is up, line protocol is up 
Loopback1 is up, line protocol is up 
Tunnel17 is up, line protocol is down 
Tunnel38 is up, line protocol is down 
Tunnel102 is up, line protocol is up 
Tunnel103 is up, line protocol is up 
Tunnel200 is up, line protocol is up 
Tunnel201 is up, line protocol is up 
>>> int_status[2]
>>> for i in int_status[2].children:
... print i.text
... 
  Hardware is BCM1250 Internal MAC, address is 0017.0f40.7819 (bia 0017.0f40.7819)
  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec, 
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Auto-duplex, Auto Speed, media type is RJ45
  output flow-control is XON, input flow-control is XON
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output never, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
>>># quite useful
>>>