当前位置：首页 > article >正文

CDP集群安全指南-动态数据加密

article 2025/1/6 3:49:37

[〇]关于本文

集群的动态数据加密主要指的是加密通过网络协议传输的数据，防止数据在传输的过程中被窃取。由于大数据涉及的主机及服务众多。你需要更具集群的实际环境来评估需要为哪些环节实施动态加密。

这里介绍一种通过Cloudera Manager 的Auto-TLS功能来为整个Cloudera Manager层面开启动态加密的步骤。Auto-TLS 功能可以自动完成在集群级别启用 TLS 加密所需的所有步骤。通过使用 Auto-TLS，您可以选择让 Cloudera 管理集群中所有证书的证书颁发机构 (CA)，或者使用公司现有的 CA。

在大多数情况下，所有必要的步骤都可以通过 Cloudera Manager 的 UI 界面轻松完成。

开启后将会发生以下变化

对 Admin Console 使用 TLS 加密：启用用户和 Cloudera Manager Admin Console 之间的 TLS 加密 (HTTPS)。检查时会使用 HTTPS 端口
为Cloudera Manager Agent使用 TLS 加密：在服务器和agent之间启用 TLS 加密。
使用代理到服务器的 TLS 身份验证：启用代理到服务器的 TLS 身份验证。
Cloudera Management Service所有服务启用TLS/SSL

【重要提醒】我这里只为Cloudera Manager开启TLS 加密，并不打算为CDP的服务启用TLS/SSL，因为开启后所有服务的使用方式都会发生改变。这是一个非常大的变更。所以我这里再次提醒您，请谨慎评估您是否需要为整个CDP的服务启用TLS/SSL

[一]开启Auto-TLS

1-生成CA证书

[root@cdp73-1 ~]# mkdir -p /etc/tls/ca
[root@cdp73-1 ~]# cd /etc/tls/ca
[root@cdp73-1 ca]# openssl genrsa -out ca.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
...............................................................................................................................................................................................+++++
.................................+++++
e is 65537 (0x010001)
[root@cdp73-1 ca]# openssl rsa -check -in ca.key
RSA key ok
writing RSA key
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAoI77S2bmyDU+2J/K+DdFsPSz7+NIfmoFdXvR+jrmBdUoAkXJ
2iYI1+2RuxTIySj6tQYI8njlfcpXXUe7qLA6K1NAYSuCrK4602YgfSlNuozF5v7Y
oPXsFjhUd8ifrKpQXcockaQTRIVfkqszo+le7HkUwnM75udI99KZtNZy07g8oqs8
aPYeZLCw6qiBVs+1bBkRaEPx5ZMpOnEPl3z61d/3yAJEMxlhEr6qFQOStYtYtXAG
tckDH3I77Wz1LbwyWGV5Pg2YOm9Yyf1S+xxNybKMHnkXrGpZ7gH36uaFoYVufW60
B4Q5GUisScTCb1axcC4OR/Lnm5feCxkyvCCjwwIDAQABAoIBAAVeIlqX+xkwZoR5
eyCnQGY1GBrp/09ynfIajJ+P/oatZKQGz0PCx8LoR1n4zOmkvBT3Oa9ZiVqWPCt7
LXPrSFaQdvOIr9q0DHVq0aU9j0KwWCFr3bQr5JOtmG1UwLnWC8/G5QOdd6Nvzg0q
OhS9xZWkSqRhk9wZWWAno0nfbYFUdutat8CJQnSrKy3CgHybGNiQoEuOl/C1TNNw
oP3HEs+HxRGyKZ+w903uMGsj3FleBCfGLlxSGwB1LUSi6rrWxq1dzRvSiTFJ8vdS
qjVzB/WOvmd/IbBktIFsbinihqqLYgxvu4KtK/prGmXJfHbJEIVyvHzGUW4nQRsK
3dylTuECgYEAzzDJtAjKvcHgdfJrAz5O7muaV5vLdY1f/d17hdp+EYJM1FUQ04bE
lhCdJafkReKrIqbZ5GGmIck/xezYCMivU3aXLBzDUDoMRnRpZQg5NEIj9P5OIL3v
lW8ekveqnlvlOtvxxRXcdK60SbXJOEy31llvDckwqjFoPXtLcH+du5MCgYEAxmHq
hF+VeHrlvz4mGG5QPDvLn7i8x6wQZ68LsAPLUMcQNK/oSgQCp2I74fpZ5b4m9MRa
S6HIDn0VohLJk4G79Lb8ZfHG8xP/9csL+wWNwfJEolB6R0HbwgbmPyxn42hwWBtD
OEXRhSLUUaYbSoqqJ1OXKp2CxV0FEQuouyp/dRECgYAXiJ8gh+8fZqosO4DUOXuV
sTsywEt36rsAhuvE5HB1ZKt9YrwqiqBBu1leMZfIKFrv8KvHOSA5rjZEMQbI2KKx
hELfi9TThARo7EgcZba5rNmQtmIBbhGMk7aRUvhaTG3ZJapsjHMh/cYUqUVV08D9
4+KtWjDg5APHF/4VpSkxaQKBgBNEXT9//QdXgErDoXWL+TTwZcVcbtFBr9IyGQN+
StfMjZFgaEIQA6X4D3LSGrsKbcQl8dMYolJt6ZT1GCjAV93bi8Xm5nijP5/CmaZG
ks78VZgiEs4q4koE24XVLT3T3d1gwHWNqlyw1kgbxtjFgOMS5kKYS6QZda2DIV8U
MI7RAoGAV7aQMIzQ5V/FBtN8T0e3+zTXmg9d9c5vjHRCe737AOBNLSeRHvWk/Nt/
6113PhacXqzE/ZRX0XM/oNjX0jil13wte8z1yXdLVdNfPUr8zV/0FV0NqpivyOqT
sujPUay17tD9gdg03tz6TGJIMLu7jo8rx7SgTdeNAPjjN5hfp0w=
-----END RSA PRIVATE KEY-----
[root@cdp73-1 ca]# openssl rsa -text -in ca.key -noout
RSA Private-Key: (2048 bit, 2 primes)
modulus:
    00:a0:8e:fb:4b:66:e6:c8:35:3e:d8:9f:ca:f8:37:
    45:b0:f4:b3:ef:e3:48:7e:6a:05:75:7b:d1:fa:3a:
    e6:05:d5:28:02:45:c9:da:26:08:d7:ed:91:bb:14:
    c8:c9:28:fa:b5:06:08:f2:78:e5:7d:ca:57:5d:47:
    bb:a8:b0:3a:2b:53:40:61:2b:82:ac:ae:3a:d3:66:
    20:7d:29:4d:ba:8c:c5:e6:fe:d8:a0:f5:ec:16:38:
    54:77:c8:9f:ac:aa:50:5d:ca:1c:91:a4:13:44:85:
    5f:92:ab:33:a3:e9:5e:ec:79:14:c2:73:3b:e6:e7:
    48:f7:d2:99:b4:d6:72:d3:b8:3c:a2:ab:3c:68:f6:
    1e:64:b0:b0:ea:a8:81:56:cf:b5:6c:19:11:68:43:
    f1:e5:93:29:3a:71:0f:97:7c:fa:d5:df:f7:c8:02:
    44:33:19:61:12:be:aa:15:03:92:b5:8b:58:b5:70:
    06:b5:c9:03:1f:72:3b:ed:6c:f5:2d:bc:32:58:65:
    79:3e:0d:98:3a:6f:58:c9:fd:52:fb:1c:4d:c9:b2:
    8c:1e:79:17:ac:6a:59:ee:01:f7:ea:e6:85:a1:85:
    6e:7d:6e:b4:07:84:39:19:48:ac:49:c4:c2:6f:56:
    b1:70:2e:0e:47:f2:e7:9b:97:de:0b:19:32:bc:20:
    a3:c3
publicExponent: 65537 (0x10001)
privateExponent:
    05:5e:22:5a:97:fb:19:30:66:84:79:7b:20:a7:40:
    66:35:18:1a:e9:ff:4f:72:9d:f2:1a:8c:9f:8f:fe:
    86:ad:64:a4:06:cf:43:c2:c7:c2:e8:47:59:f8:cc:
    e9:a4:bc:14:f7:39:af:59:89:5a:96:3c:2b:7b:2d:
    73:eb:48:56:90:76:f3:88:af:da:b4:0c:75:6a:d1:
    a5:3d:8f:42:b0:58:21:6b:dd:b4:2b:e4:93:ad:98:
    6d:54:c0:b9:d6:0b:cf:c6:e5:03:9d:77:a3:6f:ce:
    0d:2a:3a:14:bd:c5:95:a4:4a:a4:61:93:dc:19:59:
    60:27:a3:49:df:6d:81:54:76:eb:5a:b7:c0:89:42:
    74:ab:2b:2d:c2:80:7c:9b:18:d8:90:a0:4b:8e:97:
    f0:b5:4c:d3:70:a0:fd:c7:12:cf:87:c5:11:b2:29:
    9f:b0:f7:4d:ee:30:6b:23:dc:59:5e:04:27:c6:2e:
    5c:52:1b:00:75:2d:44:a2:ea:ba:d6:c6:ad:5d:cd:
    1b:d2:89:31:49:f2:f7:52:aa:35:73:07:f5:8e:be:
    67:7f:21:b0:64:b4:81:6c:6e:29:e2:86:aa:8b:62:
    0c:6f:bb:82:ad:2b:fa:6b:1a:65:c9:7c:76:c9:10:
    85:72:bc:7c:c6:51:6e:27:41:1b:0a:dd:dc:a5:4e:
    e1
prime1:
    00:cf:30:c9:b4:08:ca:bd:c1:e0:75:f2:6b:03:3e:
    4e:ee:6b:9a:57:9b:cb:75:8d:5f:fd:dd:7b:85:da:
    7e:11:82:4c:d4:55:10:d3:86:c4:96:10:9d:25:a7:
    e4:45:e2:ab:22:a6:d9:e4:61:a6:21:c9:3f:c5:ec:
    d8:08:c8:af:53:76:97:2c:1c:c3:50:3a:0c:46:74:
    69:65:08:39:34:42:23:f4:fe:4e:20:bd:ef:95:6f:
    1e:92:f7:aa:9e:5b:e5:3a:db:f1:c5:15:dc:74:ae:
    b4:49:b5:c9:38:4c:b7:d6:59:6f:0d:c9:30:aa:31:
    68:3d:7b:4b:70:7f:9d:bb:93
prime2:
    00:c6:61:ea:84:5f:95:78:7a:e5:bf:3e:26:18:6e:
    50:3c:3b:cb:9f:b8:bc:c7:ac:10:67:af:0b:b0:03:
    cb:50:c7:10:34:af:e8:4a:04:02:a7:62:3b:e1:fa:
    59:e5:be:26:f4:c4:5a:4b:a1:c8:0e:7d:15:a2:12:
    c9:93:81:bb:f4:b6:fc:65:f1:c6:f3:13:ff:f5:cb:
    0b:fb:05:8d:c1:f2:44:a2:50:7a:47:41:db:c2:06:
    e6:3f:2c:67:e3:68:70:58:1b:43:38:45:d1:85:22:
    d4:51:a6:1b:4a:8a:aa:27:53:97:2a:9d:82:c5:5d:
    05:11:0b:a8:bb:2a:7f:75:11
exponent1:
    17:88:9f:20:87:ef:1f:66:aa:2c:3b:80:d4:39:7b:
    95:b1:3b:32:c0:4b:77:ea:bb:00:86:eb:c4:e4:70:
    75:64:ab:7d:62:bc:2a:8a:a0:41:bb:59:5e:31:97:
    c8:28:5a:ef:f0:ab:c7:39:20:39:ae:36:44:31:06:
    c8:d8:a2:b1:84:42:df:8b:d4:d3:84:04:68:ec:48:
    1c:65:b6:b9:ac:d9:90:b6:62:01:6e:11:8c:93:b6:
    91:52:f8:5a:4c:6d:d9:25:aa:6c:8c:73:21:fd:c6:
    14:a9:45:55:d3:c0:fd:e3:e2:ad:5a:30:e0:e4:03:
    c7:17:fe:15:a5:29:31:69
exponent2:
    13:44:5d:3f:7f:fd:07:57:80:4a:c3:a1:75:8b:f9:
    34:f0:65:c5:5c:6e:d1:41:af:d2:32:19:03:7e:4a:
    d7:cc:8d:91:60:68:42:10:03:a5:f8:0f:72:d2:1a:
    bb:0a:6d:c4:25:f1:d3:18:a2:52:6d:e9:94:f5:18:
    28:c0:57:dd:db:8b:c5:e6:e6:78:a3:3f:9f:c2:99:
    a6:46:92:ce:fc:55:98:22:12:ce:2a:e2:4a:04:db:
    85:d5:2d:3d:d3:dd:dd:60:c0:75:8d:aa:5c:b0:d6:
    48:1b:c6:d8:c5:80:e3:12:e6:42:98:4b:a4:19:75:
    ad:83:21:5f:14:30:8e:d1
coefficient:
    57:b6:90:30:8c:d0:e5:5f:c5:06:d3:7c:4f:47:b7:
    fb:34:d7:9a:0f:5d:f5:ce:6f:8c:74:42:7b:bd:fb:
    00:e0:4d:2d:27:91:1e:f5:a4:fc:db:7f:eb:5d:77:
    3e:16:9c:5e:ac:c4:fd:94:57:d1:73:3f:a0:d8:d7:
    d2:38:a5:d7:7c:2d:7b:cc:f5:c9:77:4b:55:d3:5f:
    3d:4a:fc:cd:5f:f4:15:5d:0d:aa:98:af:c8:ea:93:
    b2:e8:cf:51:ac:b5:ee:d0:fd:81:d8:34:de:dc:fa:
    4c:62:48:30:bb:bb:8e:8f:2b:c7:b4:a0:4d:d7:8d:
    00:f8:e3:37:98:5f:a7:4c
[root@cdp73-1 ca]# openssl req -x509 -new -key ca.key -out ca.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:lh
State or Province Name (full name) []:lh
Locality Name (eg, city) [Default City]:lh
Organization Name (eg, company) [Default Company Ltd]:lh
Organizational Unit Name (eg, section) []:lh
Common Name (eg, your name or your server's hostname) []:lh
Email Address []:lh
[root@cdp73-1 ca]#

2-开启Auto-TLS

进入管理->安全，点击Enable Aoto-TLS
填入信息
汇总

重启Cloudera-scm-server

[root@cdp73-1 ca]# systemctl restart cloudera-scm-server
[root@cdp73-1 ca]#

登录到Cloudera Manger web界面，此时http://192.168.0.171:7180变为https://192.168.0.171:7183
重启Cloudera Management Service

[三]回退Auto-TLS

1-数据库中配置

[root@cdp73-1 ~]# mysql -uroot -p
Enter password:

mysql> use scm;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> update CONFIGS set value = 'false' where attr = 'web_tls';
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0  Changed: 0  Warnings: 0

mysql> update CONFIGS set value = 'false' where attr = 'agent_tls';
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0  Changed: 0  Warnings: 0

mysql>

2-修改/etc/default/cloudera-scm-server

export CMF_JAVA_OPTS="-Xmx8G -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp -Dcom.sun.management.jmxremote.ssl.enabled.protocols=TLSv1.2 -Dorg.apache.avro.specific.use_custom_coders=true"

改为

export CMF_JAVA_OPTS="-Xmx8G -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp  -Dorg.apache.avro.specific.use_custom_coders