码迷,mamicode.com
首页 > 其他好文 > 详细

GroupVarint

时间:2018-07-24 13:56:42      阅读:123      评论:0      收藏:0      [点我收藏+]

标签:UNC   post   class   min   func   value   can   support   http   

folly/GroupVarint.h

folly/GroupVarint.h is an implementation of variable-length encoding for 32- and 64-bit integers using the Group Varint encoding scheme as described in Jeff Dean‘s WSDM 2009 talk and in Information Retrieval: Implementing and Evaluating Search Engines.

Briefly, a group of four 32-bit integers is encoded as a sequence of variable length, between 5 and 17 bytes; the first byte encodes the length (in bytes) of each integer in the group. A group of five 64-bit integers is encoded as a sequence of variable length, between 7 and 42 bytes; the first two bytes encode the length (in bytes) of each integer in the group.

GroupVarint.h defines a few classes:

  • GroupVarint<T>, where T is uint32_t or uint64_t:

    Basic encoding / decoding interface, mainly aimed at encoding / decoding one group at a time.

  • GroupVarintEncoder<T, Output>, where T is uint32_t or uint64_t, and Output is a functor that accepts StringPieceobjects as arguments:

    Streaming encoder: add values one at a time, and they will be flushed to the output one group at a time. Handles the case where the last group is incomplete (the number of integers to encode isn‘t a multiple of the group size)

  • GroupVarintDecoder<T>, where T is uint32_t or uint64_t:

    Streaming decoder: extract values one at a time. Handles the case where the last group is incomplete.

The 32-bit implementation is significantly faster than the 64-bit implementation; on platforms supporting the SSSE3 instruction set, we use the PSHUFB instruction to speed up lookup, as described in SIMD-Based Decoding of Posting Lists(CIKM 2011).

For more details, see the header file folly/GroupVarint.h and the associated test file folly/test/GroupVarintTest.cpp.

GroupVarint

标签:UNC   post   class   min   func   value   can   support   http   

原文地址:https://www.cnblogs.com/lenmom/p/9359357.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!