当前位置： 首页 > article >正文

ais_server 学习笔记

article 2025/2/21 3:53:28

ais_server 学习笔记

  • 一前序
  • 二、ais init
    • 1、时序图如下
    • 2. 初始化一共分为以下几个重要步骤：
      • 2.1.1、在ais_server中启动main函数，然后创建AisEngine，接着初始化AisEngine
      • 2.1.2、解析/var/camera_config.xml 文件，获取相关配置参数。这个camera_config.xml 后面会细聊
      • 2.1.3、启动EventHandler线程，会启动四个EventHandler，来承接外面调回来的通知，比如FRAME_DONE、FATAL_ERROR等等, 至于为什么是四个EventHandler，我猜测可能是一个CSIPHY 对应一个 Thread handler吧。
      • 2.1.4、"加载" sensor so、CSIPHY 、IFE
      • 2.1.5、接着会根据前面解析的camera_config.xml 里面的配置来给input、ife、csiphy 里面的参数进行配置，在csiphy里面会进行mipi协议的一些配置, 配置好mipiconfig 后就会去 start mipi
      • 2.1.6、启动平台图像处理等功能
  • 三、amera_client -> qcarcam_initialize
  • 四、camera_client -> qcarcam_open
  • 五、camera_client -> qcarcam_s_param
  • 六、camera_client -> qcarcam_start
  • 七、camera_client -> qcarcam_get_frame

一前序

ais server 的主要功能，简单理解就是高通平台用于使能相机模块，处理camera 数据，并在camera client连接的时候将camera数据顺利给到client。

我们还是从ais的init，然后client调用接口，init、open、s_param、start、get_frame、release_frame、stop、close、deinit聊起, 每个调用都会有对应的时序图。

二、ais init

1、时序图如下

在这里插入图片描述

2. 初始化一共分为以下几个重要步骤：

2.1.1、在ais_server中启动main函数，然后创建AisEngine，接着初始化AisEngine

2.1.2、解析/var/camera_config.xml 文件，获取相关配置参数。这个camera_config.xml 后面会细聊

<?xml version="1.0" encoding="UTF-8"?>

<!--
  Copyright (c) 2020 Qualcomm Technologies, Inc.
  All Rights Reserved.
  Confidential and Proprietary - Qualcomm Technologies, Inc.
 -->

<!-- THIS IS A SAMPLE CAMERA CONFIG XML TO ILLUSTRATE HOW IT CAN BE USED -->

<!--
                             SA8155 SAMPLE
=========================================================================== */
/*              CSI     lanes   IFEs   I2C Port    Sensors
 * max96716    2        4      2    /dev/cci1    2 x 96717 YUV SENSORS
*/
-->

<CameraConfig version="0x403">
    <board name="SA8155_ADP" boardType="CAMERA_HW_BOARD_ADPAIR_V2_PL195" multiSocEnable="0">
        <i2cDevs>
            <i2cDev name="cci0">
                <properties i2ctype = "CAMERA_I2C_TYPE_CCI" device_id = "0" port_id = "0"/>
                <sda_pin gpio = "17" function = "1"/>
                <scl_pin gpio = "18" function = "1"/>
            </i2cDev>
            <i2cDev name="cci1">
                <properties i2ctype = "CAMERA_I2C_TYPE_CCI" device_id = "0" port_id = "1"/>
                <sda_pin gpio = "19" function = "1"/>
                <scl_pin gpio = "20" function = "1"/>
            </i2cDev>
            <i2cDev name="cci2">
                <properties i2ctype = "CAMERA_I2C_TYPE_CCI" device_id = "1" port_id = "0"/>
                <sda_pin gpio = "31" function = "1"/>
                <scl_pin gpio = "32" function = "1"/>
            </i2cDev>
        </i2cDevs>
        <inputDevs>
            <inputDev devId = "0">
                <properties>
                    <!-- 2 AR0231 Bayer Sensor -->
                    <config subdevId = "0" type="1" numSensors="1" pmasterSocId="0" socMap="1" opMode="0" />
                    <sensor link="0" type="1" />
                </properties>
                <driverInfo devCategory = "CAMERA_DEVICE_CATEGORY_SENSOR" libName="ais_nio_max96716f" openFcn="CameraSensorDevice_Open_max96716"/>
                <i2cPort soc_id = "0" i2ctype = "CAMERA_I2C_TYPE_CCI" device_id = "0" port_id = "1" />
                <csiInfo csiId = "0" numLanes = "2" laneAssign = "0x10" numIfeMap="1" ifeMap="0x0" />
                <gpio id = "CAMERA_GPIO_RESET" num = "22" config = "0x31"/>
                <intr id = "CAMERA_GPIO_INTR" pin_id = "106" intr_type = "CAMERA_GPIO_INTR_TLMM" trigger = "CAMERA_GPIO_TRIGGER_FALLING" gpio_cfg = "0x30"/>
            </inputDev>
            <inputDev devId = "1">
                <properties>
                    <!-- 2 AR0231 Bayer Sensor -->
                    <config subdevId = "1" type="1" numSensors="4" pmasterSocId="0" socMap="1" opMode="0" />
                    <sensor link="0" type="1" />
                    <sensor link="1" type="1" />
                    <sensor link="2" type="1" />
                    <sensor link="3" type="1" />
                </properties>
                <driverInfo devCategory = "CAMERA_DEVICE_CATEGORY_SENSOR" libName="ais_nio_max96722" openFcn="CameraSensorDevice_Open_max96722"/>
                <i2cPort soc_id = "0" i2ctype = "CAMERA_I2C_TYPE_CCI" device_id = "1" port_id = "0" />
                <csiInfo csiId = "3" numLanes = "4" laneAssign = "0x3210" ifeMap="0x32" numIfeMap="2"/>
                <gpio id = "CAMERA_GPIO_RESET" num = "25" config = "0x31"/>
                <intr id = "CAMERA_GPIO_INTR" pin_id = "13" intr_type = "CAMERA_GPIO_INTR_TLMM" trigger = "CAMERA_GPIO_TRIGGER_FALLING" gpio_cfg = "0x30"/>
                <intr id = "CAMERA_GPIO_LOCK" pin_id = "15" intr_type = "CAMERA_GPIO_INTR_TLMM" trigger = "CAMERA_GPIO_TRIGGER_FALLING" gpio_cfg = "0x30"/>
                <intr id = "CAMERA_GPIO_PASS" pin_id = "90" intr_type = "CAMERA_GPIO_INTR_TLMM" trigger = "CAMERA_GPIO_TRIGGER_EDGE" gpio_cfg = "0x10"/>
                <intr id = "CAMERA_GPIO_CUSTOM1" pin_id = "149" intr_type = "CAMERA_GPIO_INTR_TLMM" trigger = "CAMERA_GPIO_TRIGGER_FALLING" gpio_cfg = "0x30"/>
                <intr id = "CAMERA_GPIO_CUSTOM2" pin_id = "150" intr_type = "CAMERA_GPIO_INTR_TLMM" trigger = "CAMERA_GPIO_TRIGGER_FALLING" gpio_cfg = "0x30"/>
                <intr id = "CAMERA_GPIO_CUSTOM_REG1" pin_id = "151" intr_type = "CAMERA_GPIO_INTR_TLMM" trigger = "CAMERA_GPIO_TRIGGER_FALLING" gpio_cfg = "0x30"/>
                <intr id = "CAMERA_GPIO_CUSTOM_REG2" pin_id = "152" intr_type = "CAMERA_GPIO_INTR_TLMM" trigger = "CAMERA_GPIO_TRIGGER_FALLING" gpio_cfg = "0x30"/>
            </inputDev>
            <inputDev devId = "2">
                <properties>
                    <!-- 2 AR0231 Bayer Sensor -->
                    <config subdevId = "0" type="1" numSensors="3" pmasterSocId="0" socMap="1" opMode="0" />
                    <sensor link="0" type="1" />
                    <sensor link="1" type="1" />
                    <sensor link="2" type="1" />
                </properties>
                <driverInfo devCategory = "CAMERA_DEVICE_CATEGORY_SENSOR" libName="ais_nio_svc_fy" openFcn="CameraSensorDevice_Open_max96712"/>
                <i2cPort soc_id = "0" i2ctype = "CAMERA_I2C_TYPE_CCI" device_id = "0" port_id = "0" />
                <csiInfo csiId = "2" numLanes = "4" laneAssign = "0x3210" ifeMap="0x1" numIfeMap="1"/>
            </inputDev>
        </inputDevs>
        <engineSettings numBufMax="12" powerManagementPolicy="CAMERA_PM_POLICY_LPM_EVENT_ALL" latencyMeasurementMode="CAMERA_LM_MODE_DISABLE"
            recoveryRestartDelay="33"  recoveryTimeoutAfterUsrCtxtRestart="500" recoveryRetryDelay="300" recoveryMaxNumAttempts="1">
            <inputFatalError type="MATCH_INPUT_DEVICE" severity="0"/>
            <csidFatalError type="MATCH_IFE_DEVICE" severity="0"/>
            <ifeOutputError type="MATCH_IFE_DEVICE" severity="0"/>
        </engineSettings>
    </board>

    <inputMapping>
        <inputMap qcarcamId = "1" opMode = "QCARCAM_OPMODE_RAW_DUMP">
            <inputSrc devId = "0" srcId = "0"/>
        </inputMap>
        <inputMap qcarcamId = "2" opMode = "QCARCAM_OPMODE_RAW_DUMP">
            <inputSrc devId = "2" srcId = "2"/>
        </inputMap>
        <inputMap qcarcamId = "3" opMode = "QCARCAM_OPMODE_RAW_DUMP">
            <inputSrc devId = "2" srcId = "0"/>
        </inputMap>
        <inputMap qcarcamId = "4" opMode = "QCARCAM_OPMODE_RAW_DUMP">
            <inputSrc devId = "2" srcId = "1"/>
        </inputMap>
        <inputMap qcarcamId = "5" opMode = "QCARCAM_OPMODE_RAW_DUMP">
            <inputSrc devId = "1" srcId = "0"/>
        </inputMap>
        <inputMap qcarcamId = "6" opMode = "QCARCAM_OPMODE_RAW_DUMP">
            <inputSrc devId = "1" srcId = "1"/>
        </inputMap>
        <inputMap qcarcamId = "7" opMode = "QCARCAM_OPMODE_RAW_DUMP">
            <inputSrc devId = "1" srcId = "2"/>
        </inputMap>
        <inputMap qcarcamId = "8" opMode = "QCARCAM_OPMODE_RAW_DUMP">
            <inputSrc devId = "1" srcId = "3"/>
        </inputMap>
    </inputMapping>
</CameraConfig>

2.1.3、启动EventHandler线程，会启动四个EventHandler，来承接外面调回来的通知，比如FRAME_DONE、FATAL_ERROR等等, 至于为什么是四个EventHandler，我猜测可能是一个CSIPHY 对应一个 Thread handler吧。

在这里插入图片描述在这里插入图片描述

2.1.4、“加载” sensor so、CSIPHY 、IFE

分别会调用RegisterDeviceFromLib()、RegisterDevice()将libais_nio_*.so 和 IFE 、CSIPHY open

static CameraDeviceManagerRegisteredType sStaticDevicesSA8155[] =
{
#ifndef AIS_WITH_HAL_CAMERA
    {{CAMERADEVICEID_IFE_0, CAMERA_DEVICE_CATEGORY_IFE}, ife_device_open},
    {{CAMERADEVICEID_IFELITE_1, CAMERA_DEVICE_CATEGORY_IFE}, ifelite_device_open},
#endif
    {{CAMERADEVICEID_IFE_1, CAMERA_DEVICE_CATEGORY_IFE}, ife_device_open},
    {{CAMERADEVICEID_IFELITE_0, CAMERA_DEVICE_CATEGORY_IFE}, ifelite_device_open},
    {{CAMERADEVICEID_CSIPHY_0, CAMERA_DEVICE_CATEGORY_CSIPHY}, csiphy_device_open},
    {{CAMERADEVICEID_CSIPHY_1, CAMERA_DEVICE_CATEGORY_CSIPHY}, csiphy_device_open},
    {{CAMERADEVICEID_CSIPHY_2, CAMERA_DEVICE_CATEGORY_CSIPHY}, csiphy_device_open},
    {{CAMERADEVICEID_CSIPHY_3, CAMERA_DEVICE_CATEGORY_CSIPHY}, csiphy_device_open},
};

这里继续调用到RegisterDevice，将
libais_nio_max96722.so -> CameraSensorDevice_Open_max96722
libais_nio_svc_fy.so -> CameraSensorDevice_Open_max96712
libais_nio_max96716f.so -> CameraSensorDevice_Open_max96716
保存在registeredDevices中

CameraSensorDevice_Open_max96722 -> max96722_sensor_open_lib()
CameraSensorDevice_Open_max96712 -> max96712_sensor_open_lib()
CameraSensorDevice_Open_max96716 -> max96716_sensor_open_lib()

创建AisInputConfigurer、AisIFEConfigurer、AisCSIConfigurer，然后init他们三个，在init中分别调用DeviceOpen方法，最终会调用到CameraSensorDevice_Open_max967*、 ife_device_open、csiphy_device_open。
ife_device_open 和 ifelite_device_open 分别会被调用两次，每次都会创建一个VFE/CSID(IFE持有VFE和CSID) 对象，ife_device_open 和 ifelite_device_open的取别主要在于RDI的个数，前者会有四个RDI通道，后者会有三个RDI通道。
在这里插入图片描述
CSID和IFE分别在创建的时候对应一个线程，分别open四次就会有四个线程，这个四个线程用于监听中断和其他消息通知，后面会用到
在这里插入图片描述

2.1.5、接着会根据前面解析的camera_config.xml 里面的配置来给input、ife、csiphy 里面的参数进行配置，在csiphy里面会进行mipi协议的一些配置, 配置好mipiconfig 后就会去 start mipi

在这里插入图片描述

2.1.6、启动平台图像处理等功能

在这里插入图片描述当ais完全启动后，后续将从client端使用的角度来分析，ais_server对应的每一步处理

三、amera_client -> qcarcam_initialize

Camera client和ais_server是通过libais_client进行通信的，如果是client是在qnx 则通过socket，如果client是在

android侧，则通过hab（也是基于共享内存封装的类socket机制）
Camera client 开始使用的时候首先要调用接口qcarcam_initialize, 首先会调用到ais_client中，会为这个camera分配一个s_ais_client变量，然后再建立该camera client和ais_server(ais_client 是camera client和ais_server之间的桥梁)之间的心跳机制

这里注意sgs_ais_client数组最大值是64，意味着最多同时可以连接64个client，但是这里有个奇怪的点，就是我们可以open的camera个数其实和rdi有关系？

–> 所以答案是:
### 三级目录
每个camera client qcarcam_initialize之后都会有个心跳线程
在这里插入图片描述

四、camera_client -> qcarcam_open

camera_client 调用qcarcam_open到AisEngine 中这里并没有对IFE、INPUT、CSI做什么操作，只是为这个open的camera分配了一个属于，camera_client和ais_server 识别这个camera的唯一标识句柄qcarcam_hndl
在这里插入图片描述

五、camera_client -> qcarcam_s_param

这里client会为每个camera申请5个buffer（MIN_USR_NUMBER_OF_BUFFERS 3 <buffer < QCARCAM_MAX_NUM_BUFFERS 12)

在这里插入图片描述
将client传过来的buffer地址和ais_server中的buffer做内存映射

smmu_map_pt_v2 函数可能用于将物理地址（DA, Device Address）映射到虚拟地址空间，具体是通过系统的 SMMU（System Memory Management Unit，系统内存管理单元）进行的，这里就简单理解camera_client buffer和ais_server的buffer做了映射，然后ais_server的buffer又绑定了一块专门用于camera data传输的内存。
在这里插入图片描述
在这里插入图片描述
如上log分析，handle 就是camera_client buffer的地址，DA就是做完映射后，ais_server使用的buffer地址，我们打开了四个camera，每个camera申请了5个buffer。

根据环视的camera_config.xml 配置，当有camera_client连接ais_server的时候，会优先给该camera分配IFE2，然后当IFE2的四个RDI都占用后，再连接的时候就分配IFE3的RDI

在这里插入图片描述如下图，CSID2 对应的 IFE就可以理解为IFE2、CSID3 对应的IFE就可以理解为IFE3，他们IFE都是Lite模式四路数据传输线。

在这里插入图片描述
这个后面camera data传递的时候会深入讨论

六、camera_client -> qcarcam_start

在这里插入图片描述
如上时序图，camera_client的start操作，其实主要针对的就是ais_input_configurer, CISPHY已经在ais_server初始化的时候已经start过了，然后根据先后顺序会start IFE 中的VFE/CISD，然后会start camera sensor, 这里我们用的摄像头加串器是96716/解串器是96722。

在96722sensor中 ，max96722_sensor_open_lib的时候，

    memcpy(&pCtxt->sensor_lib, &sensor_lib_ptr, sizeof(pCtxt->sensor_lib));
    memcpy(&pCtxt->max96722_sensors, max96722_sensors_init_table, sizeof(pCtxt->max96722_sensors));

static max96722_sensor_info_t max96722_sensors_init_table[] =
{
    {
      .state = SENSOR_STATE_INVALID,
      .serializer_addr = MSM_DES_0_ADDR_CAM_SER_0,
      .sensor_addr = MSM_DES_0_ADDR_CAM_SNSR_0,
    },
    {
      .state = SENSOR_STATE_INVALID,
      .serializer_addr = MSM_DES_0_ADDR_CAM_SER_1,
      .sensor_addr = MSM_DES_0_ADDR_CAM_SNSR_1,
    },
    {
      .state = SENSOR_STATE_INVALID,
      .serializer_addr = MSM_DES_0_ADDR_CAM_SER_2,
      .sensor_addr = MSM_DES_0_ADDR_CAM_SNSR_2,
    },
    {
      .state = SENSOR_STATE_INVALID,
      .serializer_addr = MSM_DES_0_ADDR_CAM_SER_3,
      .sensor_addr = MSM_DES_0_ADDR_CAM_SNSR_3,
    },
};


static sensor_lib_t sensor_lib_ptr =
{
    .sensor_slave_info =
    {
       .sensor_name = SENSOR_MODEL,
       .slave_addr = MSM_DES_0_SLAVEADDR,
       .i2c_freq_mode = SENSOR_I2C_MODE_CUSTOM,
       .addr_type = CAMERA_I2C_WORD_ADDR,
       .data_type = CAMERA_I2C_BYTE_DATA,
       .sensor_id_info =
       {
         .sensor_id_reg_addr = 0x00,
         .sensor_id = MSM_DES_0_SLAVEADDR,
         .sensor_id_mask = 0xff00,
       },
      .power_setting_array =
      {
        .power_up_setting_a =
        {
          {
            .seq_type = CAMERA_POW_SEQ_VREG,
            .seq_val = CAMERA_VDIG,
            .config_val = 0,
            .delay = 1,
          },
          {
            .seq_type = CAMERA_POW_SEQ_GPIO,
            .seq_val = CAMERA_GPIO_RESET,
            .config_val = GPIO_OUT_LOW,
            .delay = 10,
          },
          {
            .seq_type = CAMERA_POW_SEQ_GPIO,
            .seq_val = CAMERA_GPIO_RESET,
            .config_val = GPIO_OUT_HIGH,
            .delay = 100,
          },
        },
        .size_up = 3,
        .power_down_setting_a =
        {
          {
            .seq_type = CAMERA_POW_SEQ_GPIO,
            .seq_val = CAMERA_GPIO_RESET,
            .config_val = GPIO_OUT_LOW,
            .delay = 0,
          },
        },
        .size_down = 1,
      },
      .is_init_params_valid = 1,
    },
    .csi_params =
    {
      {
        .lane_cnt = 4, /*4 lane mode is selected */
        .settle_cnt = 0xE,
        .lane_mask = 0x1F,
        .combo_mode = 0,
        .is_csi_3phase = 0,
      }
    },
    .sensor_close_lib = max96722_sensor_close_lib,
    .sensor_capability = 0,
    /*custom functions that were defined in the driver */
    .sensor_custom_func =
    {
      .sensor_set_platform_func_table = &max96722_sensor_set_platform_func_table,
      .sensor_power_suspend = &max96722_sensor_power_suspend,
      .sensor_power_resume = &max96722_sensor_power_resume,
      .sensor_detect_device = &max96722_sensor_detect_device,
      .sensor_detect_device_channels = &max96722_sensor_detect_device_channels,
      .sensor_init_setting = &max96722_sensor_init_setting,
      .sensor_set_channel_mode = &max96722_sensor_set_channel_mode,
      .sensor_start_stream = &max96722_sensor_start_stream,
      .sensor_stop_stream = &max96722_sensor_stop_stream,
      .sensor_s_param = &max96722_des_sensor_s_param,
      .sensor_g_param = &max96722_des_sensor_g_param,
      //.sensor_power_on = &max96722_sensor_power_on,
    },
    .use_sensor_custom_func = TRUE,
};

如上代码所示，在open的时候就将sensor_lib_ptr和max96722_sensors_init_table copy给了sensor_lib和max96722_sensors，后面对于camera寄存器的读写都操作sensor_lib_ptr，对于加/解串操作max96722_sensors，
max96722_sensor_start_stream() -> ser96717_start_link() -> i2c_slave_write_array(max96722_sensors->serializer_addr, cam_ser_reg_setting), 这里96722/96716对应的解串器都是96717，这里做了统一化处理，包括来自J5的摄像头也是定义了一个96717的解串器名字
在这里插入图片描述

七、camera_client -> qcarcam_get_frame

当ais_server 准备好camera data后就会通知camera_client去使用camera data，大致时序如下：
在这里插入图片描述
在这里插入图片描述
如上时序图，VFE中的camera data流转过程：
1、在VEF模块初始化的时候，会注册函数ifeIST()到CameraPlatformISTInit中，在CameraPlatformISTInit中会启动一个线程CameraIST，会专门用于监听中断, 然后回调ifeIST().
2、ifeIST() -> ProcessIST() -> CameraSetSignal(m_ifeIstProcThreadContext.evIfeIstNotification) -> ISTWorkerThread(), 回调到 ifeIST() 之后会间接的唤醒等待的函数ISTWorkerThread ->
CameraWaitOnSignal(pIfeCtx->m_ifeIstProcThreadContext.evIfeIstNotification), 然后会紧接着调用
ProcessISTQueueItem(), 将填充好sQueueItem的根据中断类型执行相应的操作：
ProcessResetIRQ(): 触发重启VFE中断
ProcessSofIRQ(): camera data从硬件缓冲区到内存，将client申请的buffer映射出来的软件缓冲区提交到硬件缓冲区，去填充camera data
ProcessWrBusIRQ(): 软件缓冲区数据填满后就会触发中断，通知去消费数据
ProcessOverflowIRQ(): 将IFE、MIPI相关的error, waring 通知到camera client端。

如上都是基于学习的精神做的笔记，若有错误的地方请大佬们多多指正。


http://www.kler.cn/a/415086.html

相关文章：

  • 列表上移下移功能实现
  • Qt Graphics View 绘图实例
  • C0034.在Ubuntu中安装的Qt路径
  • 构建现代Web应用：FastAPI、SQLModel、Vue 3与Axios的结合使用
  • linux系统中常用文件日常使用命令记录
  • 架构第十四章：zabbix-4
  • C++条件编译指令
  • Unity ShaderLab 实现交互地毯
  • 【zookeeper04】消息队列与微服务之zookeeper客户端访问
  • Linux基础项目包含（DNS，FTP，Samba）
  • 华为IPD流程学习之——深入解读123页华为IPD流程体系设计方法论PPT
  • STM32-- 看门狗--介绍、使用场景、失效场景
  • 2024健康大数据与智能医疗（ICHIH 2024）
  • LLM*：路径规划的大型语言模型增强增量启发式搜索
  • 第一篇：Admin.Net前端项目启动
  • LSTM卫星轨道预测(一)
  • 【HarmonyOS开发模板/组件分享 – 用户隐私政策弹窗】
  • 贪心算法-Huffman树 不等式 推公式
  • iscsi服务器
  • [问题记录] Android裁剪Provision应用后无法打开开发者选项