十三：Transparent Encryption in HDFS(转)

• 应用程序级加密。这是最安全、最灵活的方法。应用程序最终控制是什么加密,可以准确地反映用户的需求。然而,编写应用程序这样做是很难的。这也不是一个选择为客户现有的应用程序不支持加密。

• 数据库级加密。类似的应用程序级加密的属性。大多数数据库厂商提供某种形式的加密。然而,可能有性能问题。一个例子是,索引不能加密。

• 文件系统级进行加密。该选项提供了高性能、应用程序透明,通常容易部署。但是,它无法模型应用程序级别的一些政策。例如,多租户应用程序基于最终用户可能想要加密。一个数据库可能需要不同的加密设置每一列存储在单个文件中。

• 磁盘级别加密。容易部署和高性能,但也很不灵活。

 <!-- KMS Cache -->
  <property>
    <name>hadoop.kms.cache.enable</name>
    <value>true</value>
    <description>
      Whether the KMS will act as a cache for the backing KeyProvider.
      When the cache is enabled, operations like getKeyVersion, getMetadata,
      and getCurrentKey will sometimes return cached data without consulting
      the backing KeyProvider. Cached values are flushed when keys are deleted
      or modified.
    </description>
  </property>

  <property>
    <name>hadoop.kms.cache.timeout.ms</name>
    <value>600000</value>
    <description>
      Expiry time for the KMS key version and key metadata cache, in
      milliseconds. This affects getKeyVersion and getMetadata.
    </description>
  </property>

  <property>
    <name>hadoop.kms.current.key.cache.timeout.ms</name>
    <value>30000</value>
    <description>
      Expiry time for the KMS current key cache, in milliseconds. This
      affects getCurrentKey operations.
    </description>
  </property>

Audit logs are aggregated for API accesses to the GET_KEY_VERSION, GET_CURRENT_KEY, DECRYPT_EEK, GENERATE_EEK operations.

<!-- KMS Audit -->
  <property>
    <name>hadoop.kms.audit.aggregation.window.ms</name>
    <value>10000</value>
    <description>
      Duplicate audit log events within the aggregation window (specified in
      ms) are quashed to reduce log traffic. A single message for aggregated
      events is printed at the end of the window, along with a count of the
      number of aggregated eventsfu.
    </description>
  </property>

  <!-- KMS Security -->

  <property>
    <name>hadoop.kms.authentication.type</name>
    <value>simple</value>
    <description>
      Authentication type for the KMS. Can be either &quot;simple&quot;
      or &quot;kerberos&quot;.
    </description>
  </property>

• KMS_HTTP_PORT
• KMS_ADMIN_PORT
• KMS_MAX_THREADS
• KMS_LOG

 export KMS_LOG=${KMS_HOME}/logs/kms

 export KMS_HTTP_PORT=16000

 export KMS_ADMIN_PORT=16001

/etc/hadoop/conf/hdfs-site.xml

<property>
     <name>dfs.encryption.key.provider.uri</name>
     <value>kms://http@localhost:16000/kms</value>
</property>

/etc/hadoop/confcore-site.xml

<property>
      <name>hadoop.security.key.provider.path</name>
      <value>kms://http@localhost:16000/kms</value>
</property>

# su - hdfs
# hadoop key create key1    
# hadoop key list -metadata

# hdfs dfs -mkdir /secureweblogs
# hdfs crypto -createZone -keyName key1 -path /secureweblogs
# hdfs crypto -listZones

生成秘钥 keytool -delete -alias ‘kms.keystore‘; 删除别名为kms.keystore的秘钥
keytool -genkey -alias ‘kms.keystore‘;  生成 别名为kms.keystore的秘钥
使用生成秘钥的命令后，会在用户根目录下生成相应的文件

hdfs dfs -cat /.reserved/raw/zone1/localfile.dat     zone1 是加密区  localfile.dat 是加密文件 ，查看该文件 显示的是被加密的文件（乱码）；