memcached基础

时间：2014-07-08 23:20:06 阅读：287 评论：0 收藏：0 [点我收藏+]

1.1、memcached是什么？

memcached 是以LiveJournal 旗下Danga Interactive 公司的Brad Fitzpatric 为首开发的一款软件。现在已成为 mixi、 hatena、 Facebook、 Vox、LiveJournal等众多服务中提高Web应用扩展性的重要因素。

许多Web应用都将数据保存到RDBMS中，应用服务器从中读取数据并在浏览器中显示。但随着数据量的增大、访问的集中，就会出现RDBMS的负担加重、数据库响应恶化、网站显示延迟等重大影响。

这时就该memcached大显身手了。memcached是高性能的分布式内存缓存服务器。一般的使用目的是，通过缓存数据库查询结果，减少数据库访问次数，以提高动态Web应用的速度、提高可扩展性。

1.2、memcached特性

1.2.1 协议简单
1.2.2 基于libevent的事件处理
1.2.3 内置内存存储方式
1.2.4 memcached不互相通信的分布式

1.2.1、协议简单

memcached的服务器客户端通信并不使用复杂的XML等格式，而使用简单的基于文本行的协议。因此，通过telnet 也能在memcached上保存数据、取得数据。下面是例子。

$ telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost.
Escape character is ‘^]‘.
get foo
VALUE foo 0 2
hi
END
stats
STAT pid 8861
(etc)

协议文档位于memcached的源代码内，也可以参考以下的URL。

http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt

1.2.2、基于libevent的事件处理

libevent是个程序库，它将Linux的epoll、BSD类操作系统的kqueue等事件处理功能封装成统一的接口。即使对服务器的连接数增加，也能发挥O(1)的性能。 memcached使用这个libevent库，因此能在Linux、BSD、Solaris等操作系统上发挥其高性能。

关于事件处理这里就不再详细介绍，可以参考Dan Kegel的The C10K Problem。

1.2.3、内置内存存储方式

为了提高性能，memcached中保存的数据都存储在memcached内置的内存存储空间中。由于数据仅存在于内存中，因此重启memcached、重启操作系统会导致全部数据消失。另外，内容容量达到指定值之后，就基于LRU(Least Recently Used)算法自动删除不使用的缓存。 memcached本身是为缓存而设计的服务器，因此并没有过多考虑数据的永久性问题。

1.2.4、memcached不互相通信的分布式

memcached尽管是“分布式”缓存服务器，但服务器端并没有分布式功能。

各个memcached不会互相通信以共享信息。那么，怎样进行分布式呢？这完全取决于客户端的实现。

1.3、安装memcached

1.3.1、安装memcached

运行memcached前需要本文开头介绍的libevent库。安装libevent如下：

[root@localhost ~]# tar -zxvf libevent-2.0.21-stable.tar.gz
...
[root@localhost ~]# cd libevent-2.0.21-stable/
[root@localhost libevent-2.0.21-stable]# ./configure
...
[root@localhost libevent-2.0.21-stable]# make
...
[root@localhost libevent-2.0.21-stable]# make install
...

安装 memcached，下载memcached，安装如下：

[root@localhost ~]# tar -xzvf memcached-1.4.20.tar.gz 
...
[root@localhost ~]# cd memcached-1.4.20/
[root@localhost memcached-1.4.20]# ./configure
...
[root@localhost memcached-1.4.20]# make
...
[root@localhost memcached-1.4.20]# make install
...

测试安装是否成功

[root@localhost ~]# /usr/local/bin/memcached -h
/usr/local/bin/memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory
[root@localhost ~]# ldd /usr/local/bin/memcached 
        linux-vdso.so.1 =>  (0x00007fffd5b69000)
        libevent-2.0.so.5 => not found
        librt.so.1 => /lib64/librt.so.1 (0x0000003a44600000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003a43e00000)
        libc.so.6 => /lib64/libc.so.6 (0x0000003a43a00000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003a43200000)
[root@localhost ~]# LD_DEBUG=libs /usr/local/bin/memcached -v
      7715:     find library=libevent-2.0.so.5 [0]; searching
      7715:      search cache=/etc/ld.so.cache
      7715:      search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64                (system search path)
      7715:       trying file=/lib64/tls/x86_64/libevent-2.0.so.5
      7715:       trying file=/lib64/tls/libevent-2.0.so.5
      7715:       trying file=/lib64/x86_64/libevent-2.0.so.5
      7715:       trying file=/lib64/libevent-2.0.so.5
      7715:       trying file=/usr/lib64/tls/x86_64/libevent-2.0.so.5
      7715:       trying file=/usr/lib64/tls/libevent-2.0.so.5
      7715:       trying file=/usr/lib64/x86_64/libevent-2.0.so.5
      7715:       trying file=/usr/lib64/libevent-2.0.so.5
      7715:     /usr/local/bin/memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory
[root@localhost ~]# whereis libevent-2.0.so.5
libevent-2.0.so: /usr/local/lib/libevent-2.0.so.5
[root@localhost ~]# /usr/local/bin/memcached -h
[root@localhost ~]# /usr/local/bin/memcached -h
memcached 1.4.20
-p <num>      TCP port number to listen on (default: 11211)
-U <num>      UDP port number to listen on (default: 11211, 0 is off)
-s <file>     UNIX socket path to listen on (disables network support)
-A            enable ascii "shutdown" command
-a <mask>     access mask for UNIX socket, in octal (default: 0700)
-l <addr>     interface to listen on (default: INADDR_ANY, all addresses)
              <addr> may be specified as host:port. If you don‘t specify
              a port number, the value you specified with -p or -U is
              used. You may specify multiple addresses separated by comma
              or by using -l multiple times
-d            run as a daemon
-r            maximize core file limit
-u <username> assume identity of <username> (only when run as root)
-m <num>      max memory to use for items in megabytes (default: 64 MB)
-M            return error on memory exhausted (rather than removing items)
-c <num>      max simultaneous connections (default: 1024)
-k            lock down all paged memory.  Note that there is a
              limit on how much memory you may lock.  Trying to
              allocate more than that would fail, so be sure you
              set the limit correctly for the user you started
              the daemon with (not for -u <username> user;
              under sh this is done with ‘ulimit -S -l NUM_KB‘).
-v            verbose (print errors/warnings while in event loop)
-vv           very verbose (also print client commands/reponses)
-vvv          extremely verbose (also print internal state transitions)
-h            print this help and exit
-i            print memcached and libevent license
-P <file>     save PID in <file>, only used with -d option
-f <factor>   chunk size growth factor (default: 1.25)
-n <bytes>    minimum space allocated for key+value+flags (default: 48)
-L            Try to use large memory pages (if available). Increasing
              the memory page size could reduce the number of TLB misses
              and improve the performance. In order to get large pages
              from the OS, memcached will allocate the total item-cache
              in one large chunk.
-D <char>     Use <char> as the delimiter between key prefixes and IDs.
              This is used for per-prefix stats reporting. The default is
              ":" (colon). If this option is specified, stats collection
              is turned on automatically; if not, then it may be turned on
              by sending the "stats detail on" command to the server.
-t <num>      number of threads to use (default: 4)
-R            Maximum number of requests per event, limits the number of
              requests process for a given connection to prevent 
              starvation (default: 20)
-C            Disable use of CAS
-b            Set the backlog queue limit (default: 1024)
-B            Binding protocol - one of ascii, binary, or auto (default)
-I            Override the size of each slab page. Adjusts max item size
              (default: 1mb, min: 1k, max: 128m)
-F            Disable flush_all command
-o            Comma separated list of extended or experimental options
              - (EXPERIMENTAL) maxconns_fast: immediately close new
                connections if over maxconns limit
              - hashpower: An integer multiplier for how large the hash
                table should be. Can be grown at runtime if not big enough.
                Set this based on "STAT hash_power_level" before a 
                restart.
              - tail_repair_time: Time in seconds that indicates how long to wait before
                forcefully taking over the LRU tail item whose refcount has leaked.
                The default is 3 hours.
              - hash_algorithm: The hash table algorithm
                default is jenkins hash. options: jenkins, murmur3
              - lru_crawler: Enable LRU Crawler background thread
              - lru_crawler_sleep: Microseconds to sleep between items
                default is 100.
              - lru_crawler_tocrawl: Max items to crawl per slab per run
                default is 0 (unlimited)

1.3.2、启动memcached

从终端输入以下命令，启动memcached

[root@localhost ~]# /usr/local/bin/memcached -p 11211 -m 64m -u root -vv
slab class   1: chunk size        96 perslab   10922
slab class   2: chunk size       120 perslab    8738
slab class   3: chunk size       152 perslab    6898
slab class   4: chunk size       192 perslab    5461
slab class   5: chunk size       240 perslab    4369
slab class   6: chunk size       304 perslab    3449
slab class   7: chunk size       384 perslab    2730
slab class   8: chunk size       480 perslab    2184
slab class   9: chunk size       600 perslab    1747
slab class  10: chunk size       752 perslab    1394
slab class  11: chunk size       944 perslab    1110
slab class  12: chunk size      1184 perslab     885
slab class  13: chunk size      1480 perslab     708
slab class  14: chunk size      1856 perslab     564
slab class  15: chunk size      2320 perslab     451
slab class  16: chunk size      2904 perslab     361
slab class  17: chunk size      3632 perslab     288
slab class  18: chunk size      4544 perslab     230
slab class  19: chunk size      5680 perslab     184
slab class  20: chunk size      7104 perslab     147
slab class  21: chunk size      8880 perslab     118
slab class  22: chunk size     11104 perslab      94
slab class  23: chunk size     13880 perslab      75
slab class  24: chunk size     17352 perslab      60
slab class  25: chunk size     21696 perslab      48
slab class  26: chunk size     27120 perslab      38
slab class  27: chunk size     33904 perslab      30
slab class  28: chunk size     42384 perslab      24
slab class  29: chunk size     52984 perslab      19
slab class  30: chunk size     66232 perslab      15
slab class  31: chunk size     82792 perslab      12
slab class  32: chunk size    103496 perslab      10
slab class  33: chunk size    129376 perslab       8
slab class  34: chunk size    161720 perslab       6
slab class  35: chunk size    202152 perslab       5
slab class  36: chunk size    252696 perslab       4
slab class  37: chunk size    315872 perslab       3
slab class  38: chunk size    394840 perslab       2
slab class  39: chunk size    493552 perslab       2
slab class  40: chunk size    616944 perslab       1
slab class  41: chunk size    771184 perslab       1
slab class  42: chunk size   1048576 perslab       1
<26 server listening (auto-negotiate)
<27 server listening (auto-negotiate)
<28 send buffer was 262144, now 268435456
<28 server listening (udp)
<29 server listening (udp)
<32 send buffer was 262144, now 268435456
<32 server listening (udp)
<30 server listening (udp)
<33 server listening (udp)
<34 server listening (udp)
<31 server listening (udp)
<35 server listening (udp)

这里显示了调试信息。这样就在前台启动了memcached，监听TCP端口11211 最大内存使用量为64M。调试信息的内容大部分是关于存储的信息。

作为daemon后台启动时，只需

#/usr/local/bin/memcached -p 11211 -m 64m -d -u root

1.4、客户端连接

许多语言都实现了连接memcached的客户端，其中以Perl、PHP为主。仅仅memcached网站上列出的语言就有Perl、PHP、Python、Ruby、C#、C/C++、Lua等等。下面介绍通过Java语言操作memcached。

1.5、使用java-memcached

1.5.1、下载Memcached-Java-Client

下载 Memcached-Java-Client 最新版本，解压缩，获取里面的4个jar文件，加入工程classpath。

1.5.2、编码测试

 1         String[] addr = { "10.108.1.97:11211" };
 2         Integer[] weights = { 3 };
 3         SockIOPool pool = SockIOPool.getInstance();
 4         pool.setServers(addr);
 5         pool.setWeights(weights);
 6         pool.setInitConn(5);
 7         pool.setMinConn(5);
 8         pool.setMaxConn(200);
 9         pool.setMaxIdle(1000 * 30 * 30);
10         pool.setMaintSleep(30);
11         pool.setNagle(false);
12         pool.setSocketTO(30);
13         pool.setSocketConnectTO(0);
14         pool.initialize();
15 
16         MemCachedClient client = new MemCachedClient();
17 
18         client.set("user", "Jobs");
19 
20         // 将数据放入缓存,并设置失效时间
21         // Date date = new Date(2000000);
22         // client.set("test1", "test1", date);
23 
24         // 删除缓存数据
25         // client.delete("test1");
26         // client.delete("test2");
27         // client.flushAll();
28 
29         // 获取缓存数据
30         System.out.println((String) client.get("user"));