当前位置：首页 > article >正文

828华为云征文 | 云服务器Flexus X实例，Docker集成搭建搭建Flink

article 2024/11/15 13:51:51

828华为云征文 | 云服务器Flexus X实例，Docker集成搭建搭建Flink

Apache Flink是一个分布式大数据计算引擎，专为处理无界和有界数据流上的有状态计算而设计，以其高吞吐量、低延迟和高性能在实时流处理和批量计算领域脱颖而出，Flink支持批流一体化，即能够同时处理实时数据流和静态数据集，使得数据处理更加灵活高效

华为云端口放行

服务器放行对应端口8082

Docker安装并配置镜像加速

1、购买华为云 Flexus X 实例

Flexus云服务器X实例-华为云 (huaweicloud.com)

2、docker安装
yum install -y docker-ce

3、验证 Docker 是否安装成功，启动docker服务并设置开机自启动，且查看docker状态
docker --version
systemctl enable docker
systemctl start docker
systemctl status docker

4、docker-ce下载

wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.huaweicloud.com/docker-ce/linux/centos/docker-ce.repo

在这里插入图片描述

5、路径/etc/yum.repos.d修改配置文件

[docker-ce-stable]
name=Docker CE Stable - $basearch
baseurl=https://repo.huaweicloud.com/docker-ce/linux/centos/9.9/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/docker-ce/linux/centos/gpg

[docker-ce-stable-debuginfo]
name=Docker CE Stable - Debuginfo $basearch
baseurl=https://repo.huaweicloud.com/docker-ce/linux/centos/9.9/debug-$basearch/stable
enabled=0
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/docker-ce/linux/centos/gpg

[docker-ce-stable-source]
name=Docker CE Stable - Sources
baseurl=https://repo.huaweicloud.com/docker-ce/linux/centos/9.9/source/stable
enabled=0
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/docker-ce/linux/centos/gpg

[docker-ce-test]
name=Docker CE Test - $basearch
baseurl=https://repo.huaweicloud.com/docker-ce/linux/centos/9.9/$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/docker-ce/linux/centos/gpg

[docker-ce-test-debuginfo]
name=Docker CE Test - Debuginfo $basearch
baseurl=https://repo.huaweicloud.com/docker-ce/linux/centos/9.9/debug-$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/docker-ce/linux/centos/gpg

[docker-ce-test-source]
name=Docker CE Test - Sources
baseurl=https://repo.huaweicloud.com/docker-ce/linux/centos/9.9/source/test
enabled=0
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/docker-ce/linux/centos/gpg

[docker-ce-nightly]
name=Docker CE Nightly - $basearch
baseurl=https://repo.huaweicloud.com/docker-ce/linux/centos/9.9/$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/docker-ce/linux/centos/gpg

[docker-ce-nightly-debuginfo]
name=Docker CE Nightly - Debuginfo $basearch
baseurl=https://repo.huaweicloud.com/docker-ce/linux/centos/9.9/debug-$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/docker-ce/linux/centos/gpg

[docker-ce-nightly-source]
name=Docker CE Nightly - Sources
baseurl=https://repo.huaweicloud.com/docker-ce/linux/centos/9.9/source/nightly
enabled=0
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/docker-ce/linux/centos/gpg

在这里插入图片描述

6、清除和构建缓存
yum clean all
yum makecache

7、docker配置镜像加速（在华为云镜像中心可获取自己的镜像加速配置）

8、创建镜像加速配置文件

vim /etc/docker/daemon.json

{
"registry-mirrors": [ "https://fd0cfec0ac0d446db7fd9d27e92d4a8e.mirror.swr.myhuaweicloud.com" ]
}

在这里插入图片描述

9、重新加载配置文件，重新启动docker，查看是否配置成功
systemctl daemon-reload
systemctl restart docker.service

10、docker info查看配置信息，是否配置成功镜像加速
docker info

搭建Flink

1、创建文件夹
mkdir /flink

2、编写docker-compose.yml文件

vim docker-compose.yml

version: "2.2"
services:
  jobmanager:
    image: flink:1.14.4-scala_2.11
    ports:
      - "8082:8082"
    command: jobmanager
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager        

  taskmanager:
    image: flink:1.14.4-scala_2.11
    depends_on:
      - jobmanager
    command: taskmanager
    scale: 1
    environment:
      - |
        FLINK_PROPERTIES=
        jobmanager.rpc.address: jobmanager
        taskmanager.numberOfTaskSlots: 2

在这里插入图片描述