【WebRTC】视频编码链路中各个类的简单分析——VideoStreamEncoder

1.视频流编码器（VideoStreamEncoder）

VideoStreamEncoder是编码流程当中至关重要的一个类，它实现的功能包括：
（1）开始编码器，暂停编码器，停止编码器
（2）设置编码器（根据config）
（3）调控编码参数，如码控参数，帧率等等
（4）调控前向纠错器
（5）视频帧的传输和接收
（6）根据网络环境，调控丢帧
（7）根据编码情况，调控屯帧
（8）根据情况，调控编码质量（因为网络环境，编码器情况，可能会调整编码策略，影响编码质量）
（9）对已编码图像进行后处理
（10）统计编码信息

可以说，在视频编码流程中，VideoStreamEncoder是获取了视频帧之后，第一个会使用到的类。这个类十分综合，大多数的编码过程都与这个类相关，所以其声明比较长，这里做简单记录，深入到每个模块之后再分析。

// VideoStreamEncoder represent a video encoder that accepts raw video frames as
// input and produces an encoded bit stream.
// Usage:
//  Instantiate.
//  Call SetSink.
//  Call SetSource.
//  Call ConfigureEncoder with the codec settings.
//  Call Stop() when done.
/*
	VideoStreamEncoder表示了一个视频编码器，它接收原始视频帧作为输入并且产生编码后的比特流
	用法:
	(1) 创建实例
	(2) SetSink，设置接收器
	(3) SetSource，设置资源
	(4) ConfigureEncoder，配置编码器参数
	(5) Stop()，结束流程
*/
class VideoStreamEncoder : public VideoStreamEncoderInterface,
                           private EncodedImageCallback,
                           public VideoSourceRestrictionsListener {
 public:
  // TODO(bugs.webrtc.org/12000): Reporting of VideoBitrateAllocation is being
  // deprecated. Instead VideoLayersAllocation should be reported.
  enum class BitrateAllocationCallbackType {
    kVideoBitrateAllocation,
    kVideoBitrateAllocationWhenScreenSharing,
    kVideoLayersAllocation
  };
  VideoStreamEncoder(
      const Environment& env,
      uint32_t number_of_cores,
      VideoStreamEncoderObserver* encoder_stats_observer,
      const VideoStreamEncoderSettings& settings,
      std::unique_ptr<OveruseFrameDetector> overuse_detector,
      std::unique_ptr<FrameCadenceAdapterInterface> frame_cadence_adapter,
      std::unique_ptr<webrtc::TaskQueueBase, webrtc::TaskQueueDeleter>
          encoder_queue,
      BitrateAllocationCallbackType allocation_cb_type,
      webrtc::VideoEncoderFactory::EncoderSelectorInterface* encoder_selector =
          nullptr);
  ~VideoStreamEncoder() override;

  VideoStreamEncoder(const VideoStreamEncoder&) = delete;
  VideoStreamEncoder& operator=(const VideoStreamEncoder&) = delete;
  // 添加自适应资源
  void AddAdaptationResource(rtc::scoped_refptr<Resource> resource) override;
  // 获取自适应资源
  std::vector<rtc::scoped_refptr<Resource>> GetAdaptationResources() override;
  // 设置资源
  void SetSource(rtc::VideoSourceInterface<VideoFrame>* source,
                 const DegradationPreference& degradation_preference) override;
  // 设置接收器
  void SetSink(EncoderSink* sink, bool rotation_applied) override;

  // TODO(perkj): Can we remove VideoCodec.startBitrate ?
  // 设置起始码率
  void SetStartBitrate(int start_bitrate_bps) override;
  // 设置前向纠错控制器
  void SetFecControllerOverride(
      FecControllerOverride* fec_controller_override) override;
  // 配置编码器
  void ConfigureEncoder(VideoEncoderConfig config,
                        size_t max_data_payload_length) override;
  void ConfigureEncoder(VideoEncoderConfig config,
                        size_t max_data_payload_length,
                        SetParametersCallback callback) override;

  // Permanently stop encoding. After this method has returned, it is
  // guaranteed that no encoded frames will be delivered to the sink.
  // 永久性停止编码，此方法返回后，保证不会有编码后的帧传递给接收器
  void Stop() override;
  // 传递关键帧
  void SendKeyFrame(const std::vector<VideoFrameType>& layers = {}) override;
  // 检测到丢包，需要调整编码策略，例如降低分辨率或帧率
  void OnLossNotification(
      const VideoEncoder::LossNotification& loss_notification) override;
  // 检测到码率变化，需要调整编码参数
  void OnBitrateUpdated(DataRate target_bitrate,
                        DataRate stable_target_bitrate,
                        DataRate target_headroom,
                        uint8_t fraction_lost,
                        int64_t round_trip_time_ms,
                        double cwnd_reduce_ratio) override;
  // 根据拥塞窗口减少比例，来调整目标比特率，避免丢包和网络拥塞
  DataRate UpdateTargetBitrate(DataRate target_bitrate,
                               double cwnd_reduce_ratio);

 protected:
  friend class VideoStreamEncoderFrameCadenceRestrictionTest;

  // Used for testing. For example the `ScalingObserverInterface` methods must
  // be called on `encoder_queue_`.
  TaskQueueBase* encoder_queue() { return encoder_queue_.get(); }
  // 检测到视频限制性资源发生变化
  void OnVideoSourceRestrictionsUpdated(
      VideoSourceRestrictions restrictions,
      const VideoAdaptationCounters& adaptation_counters,
      rtc::scoped_refptr<Resource> reason,
      const VideoSourceRestrictions& unfiltered_restrictions) override;

  // Used for injected test resources.
  // TODO(eshr): Move all adaptation tests out of VideoStreamEncoder tests.
  // 注入自适应资源，这里与AddAdaptationResource不同，还需要入参reason，即更加
  // 精准的调控
  void InjectAdaptationResource(rtc::scoped_refptr<Resource> resource,
                                VideoAdaptationReason reason);
  // 注入自适应限制
  void InjectAdaptationConstraint(AdaptationConstraint* adaptation_constraint);

  void AddRestrictionsListenerForTesting(
      VideoSourceRestrictionsListener* restrictions_listener);
  void RemoveRestrictionsListenerForTesting(
      VideoSourceRestrictionsListener* restrictions_listener);

 private:
  class CadenceCallback : public FrameCadenceAdapterInterface::Callback {
   public:
    explicit CadenceCallback(VideoStreamEncoder& video_stream_encoder)
        : video_stream_encoder_(video_stream_encoder) {}
    // FrameCadenceAdapterInterface::Callback overrides.
    // 下面是对FrameCadenceAdapterInterface::Callback当中函数的重载
    void OnFrame(Timestamp post_time,
                 bool queue_overload,
                 const VideoFrame& frame) override {
      // 使用VideoStreamEncoder当中的OnFrame()函数重载
      video_stream_encoder_.OnFrame(post_time, queue_overload, frame);
    }
    void OnDiscardedFrame() override {
      video_stream_encoder_.OnDiscardedFrame();
    }
    void RequestRefreshFrame() override {
      video_stream_encoder_.RequestRefreshFrame();
    }

   private:
    VideoStreamEncoder& video_stream_encoder_;
  };
  // VideoFrameInfo记录长宽，是否是纹理等信息
  class VideoFrameInfo {
   public:
    VideoFrameInfo(int width, int height, bool is_texture)
        : width(width), height(height), is_texture(is_texture) {}
    int width;
    int height;
    bool is_texture;
    int pixel_count() const { return width * height; }
  };
  // 编码速率设定
  struct EncoderRateSettings {
    EncoderRateSettings();
    EncoderRateSettings(const VideoBitrateAllocation& bitrate,
                        double framerate_fps,
                        DataRate bandwidth_allocation,
                        DataRate encoder_target,
                        DataRate stable_encoder_target);
    bool operator==(const EncoderRateSettings& rhs) const;
    bool operator!=(const EncoderRateSettings& rhs) const;
    // 码率控制参数
    VideoEncoder::RateControlParameters rate_control;
    // This is the scalar target bitrate before the VideoBitrateAllocator, i.e.
    // the `target_bitrate` argument of the OnBitrateUpdated() method. This is
    // needed because the bitrate allocator may truncate the total bitrate and a
    // later call to the same allocator instance, e.g.
    // |using last_encoder_rate_setings_->bitrate.get_sum_bps()|, may trick it
    // into thinking the available bitrate has decreased since the last call.
    /*
		这是`VideoBitrateAllocator`之前的标量目标比特率，即`OnBitrateUpdated()`方法
		的`target_bitrate`参数。这是因为比特率分配器可能会截断总比特率，而在稍后对同一
		分配器实例的调用中，例如`|using last_encoder_rate_setings_->bitrate.get_sum_bps()|`，
		可能会误导它认为自上次调用以来可用的比特率已经减少。
	*/
    DataRate encoder_target;
    DataRate stable_encoder_target;
  };

  class DegradationPreferenceManager;
  // 重新配置编码器
  void ReconfigureEncoder() RTC_RUN_ON(encoder_queue_);
  // 检测到编码器配置发生变化
  void OnEncoderSettingsChanged() RTC_RUN_ON(encoder_queue_);
  // 检测到帧
  void OnFrame(Timestamp post_time,
               bool queue_overload,
               const VideoFrame& video_frame);
  // 检测到丢弃帧
  void OnDiscardedFrame();
  // 需要刷新帧
  void RequestRefreshFrame();
  // 可能会编码视频帧
  void MaybeEncodeVideoFrame(const VideoFrame& frame,
                             int64_t time_when_posted_in_ms);
  // 编码视频帧
  void EncodeVideoFrame(const VideoFrame& frame,
                        int64_t time_when_posted_in_ms);
  // Indicates whether frame should be dropped because the pixel count is too
  // large for the current bitrate configuration.
  // 在当前码率配置下，根据pixle count来检查当前帧是否需要被丢弃
  bool DropDueToSize(uint32_t pixel_count) const RTC_RUN_ON(encoder_queue_);

  // Implements EncodedImageCallback.
  // 接收编码器编码之后的图像
  EncodedImageCallback::Result OnEncodedImage(
      const EncodedImage& encoded_image,
      const CodecSpecificInfo* codec_specific_info) override;
  // 丢帧的回调函数，需要给出reason
  void OnDroppedFrame(EncodedImageCallback::DropReason reason) override;
  // 暂停编码器
  bool EncoderPaused() const;
  // 标记丢帧开始
  void TraceFrameDropStart();
  // 标记丢帧结束
  void TraceFrameDropEnd();

  // Returns a copy of `rate_settings` with the `bitrate` field updated using
  // the current VideoBitrateAllocator.
  // 返回一个`rate_settings`的副本，其中`bitrate`字段已使用当前的`VideoBitrateAllocator`更新。
  EncoderRateSettings UpdateBitrateAllocation(
      const EncoderRateSettings& rate_settings) RTC_RUN_ON(encoder_queue_);
  // 获取输入帧率Fps
  uint32_t GetInputFramerateFps() RTC_RUN_ON(encoder_queue_);
  // 设置编码器速率
  void SetEncoderRates(const EncoderRateSettings& rate_settings)
      RTC_RUN_ON(encoder_queue_);
  // 进行编码后处理
  void RunPostEncode(const EncodedImage& encoded_image,
                     int64_t time_sent_us,
                     int temporal_index,
                     DataSize frame_size);
  // 释放编码器
  void ReleaseEncoder() RTC_RUN_ON(encoder_queue_);
  // After calling this function `resource_adaptation_processor_` will be null.
  // 关闭资源自适应队列，resource_adaptation_processor会设置为null
  void ShutdownResourceAdaptationQueue();
  // 申请更换编码器
  void RequestEncoderSwitch() RTC_RUN_ON(encoder_queue_);

  // Augments an EncodedImage received from an encoder with parsable
  // information.
  // 修改或增强已编码图像
  EncodedImage AugmentEncodedImage(
      const EncodedImage& encoded_image,
      const CodecSpecificInfo* codec_specific_info);
  // 处理已丢弃的帧
  void ProcessDroppedFrame(const VideoFrame& frame,
                           VideoStreamEncoderObserver::DropReason reason)
      RTC_RUN_ON(encoder_queue_);

  const Environment env_;
  TaskQueueBase* const worker_queue_;

  const int number_of_cores_;
  // sink是接收器
  EncoderSink* sink_ = nullptr;
  // VideoStreamEncoder配置
  const VideoStreamEncoderSettings settings_;
  // 比特分配回调的类型
  const BitrateAllocationCallbackType allocation_cb_type_;
  // 码率控制配置
  const RateControlSettings rate_control_settings_;

  webrtc::VideoEncoderFactory::EncoderSelectorInterface* const
      encoder_selector_from_constructor_;
  std::unique_ptr<VideoEncoderFactory::EncoderSelectorInterface> const
      encoder_selector_from_factory_;
  // Pointing to either encoder_selector_from_constructor_ or
  // encoder_selector_from_factory_ but can be nullptr.
  // 编码器选择器
  VideoEncoderFactory::EncoderSelectorInterface* const encoder_selector_;
  // 视频流编码器观察者的行为
  VideoStreamEncoderObserver* const encoder_stats_observer_;
  // Adapter that avoids public inheritance of the cadence adapter's callback
  // interface.
  // 适配器，避免了公开继承节拍适配器的回调接口。
  CadenceCallback cadence_callback_{*this};
  // Frame cadence encoder adapter. Frames enter this adapter first, and it then
  // forwards them to our OnFrame method.
  // 帧节拍编码器适配器。帧首先进入这个适配器，然后它将帧转发到我们的OnFrame方法。
  std::unique_ptr<FrameCadenceAdapterInterface> frame_cadence_adapter_
      RTC_GUARDED_BY(encoder_queue_) RTC_PT_GUARDED_BY(encoder_queue_);
  // 编码器配置项
  VideoEncoderConfig encoder_config_ RTC_GUARDED_BY(encoder_queue_);
  // 编码器
  std::unique_ptr<VideoEncoder> encoder_ RTC_GUARDED_BY(encoder_queue_)
      RTC_PT_GUARDED_BY(encoder_queue_);
  bool encoder_initialized_ = false;
  // 码率分配器
  std::unique_ptr<VideoBitrateAllocator> rate_allocator_
      RTC_GUARDED_BY(encoder_queue_) RTC_PT_GUARDED_BY(encoder_queue_);
  int max_framerate_ RTC_GUARDED_BY(encoder_queue_) = -1;

  // Set when ConfigureEncoder has been called in order to lazy reconfigure the
  // encoder on the next frame.
  // 当调用了ConfigureEncoder时设置，以便在下一帧上延迟重新配置编码器。
  bool pending_encoder_reconfiguration_ RTC_GUARDED_BY(encoder_queue_) = false;
  // Set when configuration must create a new encoder object, e.g.,
  // because of a codec change.
  // 当配置必须创建一个新的编码器对象时设置，例如，由于编解码器变更。
  bool pending_encoder_creation_ RTC_GUARDED_BY(encoder_queue_) = false;
  // 编码器配置的回调函数
  absl::InlinedVector<SetParametersCallback, 2> encoder_configuration_callbacks_
      RTC_GUARDED_BY(encoder_queue_);
  // 上一帧的信息
  std::optional<VideoFrameInfo> last_frame_info_ RTC_GUARDED_BY(encoder_queue_);
  // 边界裁剪的width和height
  int crop_width_ RTC_GUARDED_BY(encoder_queue_) = 0;
  int crop_height_ RTC_GUARDED_BY(encoder_queue_) = 0;
  // 编码器目标比特率
  std::optional<uint32_t> encoder_target_bitrate_bps_
      RTC_GUARDED_BY(encoder_queue_);
  // 最大数据负载长度
  size_t max_data_payload_length_ RTC_GUARDED_BY(encoder_queue_) = 0;
  // 上一个编码器的码率设定
  std::optional<EncoderRateSettings> last_encoder_rate_settings_
      RTC_GUARDED_BY(encoder_queue_);
  // 编码器是否停止并丢帧
  bool encoder_paused_and_dropped_frame_ RTC_GUARDED_BY(encoder_queue_) = false;

  // Set to true if at least one frame was sent to encoder since last encoder
  // initialization.
  // 如果自上次编码器初始化以来至少有一帧已发送到编码器，则设置为true
  bool was_encode_called_since_last_initialization_
      RTC_GUARDED_BY(encoder_queue_) = false;
  // 编码器是否失败
  bool encoder_failed_ RTC_GUARDED_BY(encoder_queue_) = false;

  // Used to make sure incoming time stamp is increasing for every frame.
  // 用于确保每个帧的到达时间戳都在增加。
  int64_t last_captured_timestamp_ RTC_GUARDED_BY(encoder_queue_) = 0;
  // Delta used for translating between NTP and internal timestamps.
  // 用于在NTP时间戳和内部时间戳之间转换的差值。
  const int64_t delta_ntp_internal_ms_ RTC_GUARDED_BY(encoder_queue_);
  // 最后一次记录帧日志的时间戳
  int64_t last_frame_log_ms_ RTC_GUARDED_BY(encoder_queue_);
  // 已捕捉帧的数量
  int captured_frame_count_ RTC_GUARDED_BY(encoder_queue_) = 0;
  // 由于网络阻塞丢弃的帧数量
  int dropped_frame_cwnd_pushback_count_ RTC_GUARDED_BY(encoder_queue_) = 0;
  // 由于编码器阻塞丢弃的帧数量
  int dropped_frame_encoder_block_count_ RTC_GUARDED_BY(encoder_queue_) = 0;
  // 屯帧的数量（待编码帧数量）
  std::optional<VideoFrame> pending_frame_ RTC_GUARDED_BY(encoder_queue_);
  // 屯帧的时间
  int64_t pending_frame_post_time_us_ RTC_GUARDED_BY(encoder_queue_) = 0;
  // 存储了视频帧中自上次更新以来发生变化的区域
  VideoFrame::UpdateRect accumulated_update_rect_
      RTC_GUARDED_BY(encoder_queue_);
  // 指示accumulated_update_rect_是否包含有效的更新区域信息
  bool accumulated_update_rect_is_valid_ RTC_GUARDED_BY(encoder_queue_) = true;
  // 前向纠错控制
  FecControllerOverride* fec_controller_override_
      RTC_GUARDED_BY(encoder_queue_) = nullptr;
  // 记录最近一次参数更新的时间
  std::optional<int64_t> last_parameters_update_ms_
      RTC_GUARDED_BY(encoder_queue_);
  // 记录最近一次编码操作的时间戳
  std::optional<int64_t> last_encode_info_ms_ RTC_GUARDED_BY(encoder_queue_);
  // 编码器信息
  VideoEncoder::EncoderInfo encoder_info_ RTC_GUARDED_BY(encoder_queue_);
  // 编码器，存储视频编解码器的相关配置信息，包括编解码器类型、编码设置、分辨率、帧率、比特率等参数
  VideoCodec send_codec_ RTC_GUARDED_BY(encoder_queue_);
  // 帧丢弃器
  FrameDropper frame_dropper_ RTC_GUARDED_BY(encoder_queue_);
  // If frame dropper is not force disabled, frame dropping might still be
  // disabled if VideoEncoder::GetEncoderInfo() indicates that the encoder has a
  // trusted rate controller. This is determined on a per-frame basis, as the
  // encoder behavior might dynamically change.
  /*
	如果帧丢弃器没有被强制禁用，那么即使 VideoEncoder::GetEncoderInfo() 表示编码器
	拥有一个可信的速率控制器，帧丢弃可能仍然被禁用。这是逐帧决定的，因为编码器的行为可能会动态变化。
  */
  // 强制禁用frameDropper
  bool force_disable_frame_dropper_ RTC_GUARDED_BY(encoder_queue_) = false;
  // Incremented on worker thread whenever `frame_dropper_` determines that a
  // frame should be dropped. Decremented on whichever thread runs
  // OnEncodedImage(), which is only called by one thread but not necessarily
  // the worker thread.
  /*
	每当 `frame_dropper_` 确定应该丢弃一帧时，在工作线程上增加。在运行 `OnEncodedImage()` 
	的线程上减少，这个函数只由一个线程调用，但不一定是工作线程。
  */
  std::atomic<int> pending_frame_drops_{0};

  // Congestion window frame drop ratio (drop 1 in every
  // cwnd_frame_drop_interval_ frames).
  // 拥塞窗口帧丢弃比例（每 cwnd_frame_drop_interval_ 帧丢弃 1 帧）。
  std::optional<int> cwnd_frame_drop_interval_ RTC_GUARDED_BY(encoder_queue_);
  // Frame counter for congestion window frame drop.
  // 用于拥塞窗口帧丢弃的帧计数器
  int cwnd_frame_counter_ RTC_GUARDED_BY(encoder_queue_) = 0;
  // 码率调节器
  std::unique_ptr<EncoderBitrateAdjuster> bitrate_adjuster_
      RTC_GUARDED_BY(encoder_queue_);

  // TODO(sprang): Change actually support keyframe per simulcast stream, or
  // turn this into a simple bool `pending_keyframe_request_`.
  // 下一帧的类型
  std::vector<VideoFrameType> next_frame_types_ RTC_GUARDED_BY(encoder_queue_);
  // 用于处理和写入与视频帧编码相关的元数据
  FrameEncodeMetadataWriter frame_encode_metadata_writer_{this};

  // Provides video stream input states: current resolution and frame rate.
  // 处理视频流输入状态
  VideoStreamInputStateProvider input_state_provider_;
  // 视频流适应器，用于适配不同的视频流需求，包括分辨率、帧率和比特率
  const std::unique_ptr<VideoStreamAdapter> video_stream_adapter_
      RTC_GUARDED_BY(encoder_queue_);
  // Responsible for adapting input resolution or frame rate to ensure resources
  // (e.g. CPU or bandwidth) are not overused. Adding resources can occur on any
  // thread.
  // 负责调整输入分辨率或帧率，以确保资源（如CPU或带宽）不被过度使用。增加资源可以在任何线程上进行。
  // 资源自适应处理器
  std::unique_ptr<ResourceAdaptationProcessorInterface>
      resource_adaptation_processor_ RTC_GUARDED_BY(encoder_queue_);
  // 管理质量下降的偏好设置，因为在调控编码策略时，很有可能会带来视频质量的下降，例如快速算法等
  std::unique_ptr<DegradationPreferenceManager> degradation_preference_manager_
      RTC_GUARDED_BY(encoder_queue_);
  // 自适应限制
  std::vector<AdaptationConstraint*> adaptation_constraints_
      RTC_GUARDED_BY(encoder_queue_);
  // Handles input, output and stats reporting related to VideoStreamEncoder
  // specific resources, such as "encode usage percent" measurements and "QP
  // scaling". Also involved with various mitigations such as initial frame
  // dropping.
  // The manager primarily operates on the `encoder_queue_` but its lifetime is
  // tied to the VideoStreamEncoder (which is destroyed off the encoder queue)
  // and its resource list is accessible from any thread.
  /*
	处理与VideoStreamEncoder相关的输入、输出和统计报告，特别是“编码使用百分比”测量和
	“QP缩放”等资源。还涉及各种缓解措施，如初始帧丢弃。该管理器主要在`encoder_queue_`
	上操作，但其生命周期与VideoStreamEncoder（在编码器队列外销毁）相关联，其资源列表
	可以从任何线程访问。
  */
  VideoStreamEncoderResourceManager stream_resource_manager_
      RTC_GUARDED_BY(encoder_queue_);
  // 额外的资源
  std::vector<rtc::scoped_refptr<Resource>> additional_resources_
      RTC_GUARDED_BY(encoder_queue_);
  // Carries out the VideoSourceRestrictions provided by the
  // ResourceAdaptationProcessor, i.e. reconfigures the source of video frames
  // to provide us with different resolution or frame rate.
  // This class is thread-safe.
  /*
    执行由ResourceAdaptationProcessor提供的VideoSourceRestrictions，
    即重新配置视频帧的源，为我们提供不同的分辨率或帧率。这个类是线程安全的。
  */
  VideoSourceSinkController video_source_sink_controller_
      RTC_GUARDED_BY(worker_queue_);

  // Default bitrate limits in EncoderInfoSettings allowed.
  // 默认的比特率限制
  const bool default_limits_allowed_;

  // QP parser is used to extract QP value from encoded frame when that is not
  // provided by encoder.
  // QP解析器用于在编码器未提供QP值时，从编码帧中提取QP值。
  QpParser qp_parser_;
  const bool qp_parsing_allowed_;

  // The quality convergence controller is used to determine if a codec has
  // reached its target quality. This is used for screenshare to determine when
  // there's no need to continue encoding the same repeated frame.
  /*
    质量收敛控制器用于确定一个编解码器是否已经达到了其目标质量。
    这在屏幕共享中用于判断何时不需要继续编码相同的重复帧。
  */
  QualityConvergenceController quality_convergence_controller_
      RTC_GUARDED_BY(encoder_queue_);

  // Enables encoder switching on initialization failures.
  // 在初始化失败时启用编码器切换。
  bool switch_encoder_on_init_failures_;

  const std::optional<int> vp9_low_tier_core_threshold_;
  const std::optional<int> experimental_encoder_thread_limit_;

  // This is a copy of restrictions (glorified max_pixel_count) set by
  // OnVideoSourceRestrictionsUpdated. It is used to scale down encoding
  // resolution if needed when using requested_resolution.
  //
  // TODO(webrtc:14451) Split video_source_sink_controller_
  // so that ownership on restrictions/wants is kept on &encoder_queue_, that
  // these extra copies would not be needed.
  /*
	这是由OnVideoSourceRestrictionsUpdated设置的限制（实际上是最大像素计数）的副本。
	当使用requested_resolution时，如果需要，它被用来降低编码分辨率。
	
	TODO(webrtc:14451) 分离video_source_sink_controller_，以便将限制/需求的所有权
	保留在&encoder_queue_上，这样就不需要这些额外的副本了。
  */
  std::optional<VideoSourceRestrictions> latest_restrictions_
      RTC_GUARDED_BY(encoder_queue_);

  // Used to cancel any potentially pending tasks to the worker thread.
  // Refrenced by tasks running on `encoder_queue_` so need to be destroyed
  // after stopping that queue. Must be created and destroyed on
  // `worker_queue_`.
  /*
	用于取消任何可能挂起的工作任务线程任务。被在`encoder_queue_`上运行的任务引用，
	因此需要在停止该队列后销毁。必须在`worker_queue_`上创建和销毁。
  */
  ScopedTaskSafety task_safety_;
  // 编码队列
  std::unique_ptr<TaskQueueBase, TaskQueueDeleter> encoder_queue_;

  //  Required for automatic corruption detection.
  // 自动检测数据损坏
  std::unique_ptr<FrameInstrumentationGenerator>
      frame_instrumentation_generator_;
};

1.1 视频流编码器接口（VideoStreamEncoderInterface）

VideoStreamEncoder的父类是VideoStreamEncoderInterface，这里定义了一些基础的函数。类的声明位于video/video_stream_encoder_interface.h中

// This interface represents a class responsible for creating and driving the
// encoder(s) for a single video stream. It is also responsible for adaptation
// decisions related to video quality, requesting reduced frame rate or
// resolution from the VideoSource when needed.
// TODO(bugs.webrtc.org/8830): This interface is under development. Changes
// under consideration include:
//
// 1. Taking out responsibility for adaptation decisions, instead only reporting
//    per-frame measurements to the decision maker.
//
// 2. Moving responsibility for simulcast and for software fallback into this
//    class.
/*
  这个接口代表一个负责创建和驱动单个视频流的编码器（encoder）的类。
  它还负责与视频质量相关的适应性决策，例如在需要时从 VideoSource 请求降低帧率或分辨率。
  
  TODO(bugs.webrtc.org/8830): 此接口正在开发中。正在考虑的变更包括：
  1. 移除适应性决策的责任，改为仅向决策者报告每帧的测量数据。
  2. 将simulcast（同时向多个接收者发送不同质量的视频流）和软件回退的责任转移到这个类中。
*/
class VideoStreamEncoderInterface {
 public:
  // Interface for receiving encoded video frames and notifications about
  // configuration changes.
  // 接收编码后的视频帧以及关于配置变更通知的接口
  class EncoderSink : public EncodedImageCallback {
   public:
    // 检测到编码器配置发生变化
    virtual void OnEncoderConfigurationChanged(
        std::vector<VideoStream> streams,
        bool is_svc,
        VideoEncoderConfig::ContentType content_type,
        int min_transmit_bitrate_bps) = 0;
    // 检测到码率分配发生变化
    virtual void OnBitrateAllocationUpdated(
        const VideoBitrateAllocation& allocation) = 0;
    // 检测到视频层级分配发生变化
    virtual void OnVideoLayersAllocationUpdated(
        VideoLayersAllocation allocation) = 0;
  };

  virtual ~VideoStreamEncoderInterface() = default;

  // If the resource is overusing, the VideoStreamEncoder will try to reduce
  // resolution or frame rate until no resource is overusing.
  // TODO(https://crbug.com/webrtc/11565): When the ResourceAdaptationProcessor
  // is moved to Call this method could be deleted altogether in favor of
  // Call-level APIs only
  
  // 如果资源使用过量，VideoStreamEncoder 将尝试降低分辨率或帧率，直到没有资源使用过量为止。
  // TODO(https://crbug.com/webrtc/11565): 当 ResourceAdaptationProcessor 被移动到 
  // Call 时，这个方法可能会被完全删除，转而只使用 Call 级别的 API。
  virtual void AddAdaptationResource(rtc::scoped_refptr<Resource> resource) = 0;
  virtual std::vector<rtc::scoped_refptr<Resource>>
  GetAdaptationResources() = 0;

  // Sets the source that will provide video frames to the VideoStreamEncoder's
  // OnFrame method. `degradation_preference` control whether or not resolution
  // or frame rate may be reduced. The VideoStreamEncoder registers itself with
  // `source`, and signals adaptation decisions to the source in the form of
  // VideoSinkWants.
  // TODO(bugs.webrtc.org/14246): When adaptation logic is extracted from this
  // class, it no longer needs to know the source.
  /*
	设置提供视频帧给 VideoStreamEncoder 的 OnFrame 方法的源。`degradation_preference` 
	控制是否允许降低分辨率或帧率。VideoStreamEncoder 在 `source` 中注册自己，并以 
	VideoSinkWants 的形式向源发出适应性决策信号。
    
    TODO(bugs.webrtc.org/14246): 当适应性逻辑从这个类中提取出来时，它就不再需要知道源。
  */
  virtual void SetSource(
      rtc::VideoSourceInterface<VideoFrame>* source,
      const DegradationPreference& degradation_preference) = 0;

  // Sets the `sink` that gets the encoded frames. `rotation_applied` means
  // that the source must support rotation. Only set `rotation_applied` if the
  // remote side does not support the rotation extension.
  /*
	设置接收编码帧的 `sink`。`rotation_applied` 表示源必须支持旋转。
	只有在远端不支持旋转扩展时才设置 `rotation_applied`。
  */
  virtual void SetSink(EncoderSink* sink, bool rotation_applied) = 0;

  // Sets an initial bitrate, later overriden by OnBitrateUpdated. Mainly
  // affects the resolution of the initial key frame: If incoming frames are
  // larger than reasonable for the start bitrate, and scaling is enabled,
  // VideoStreamEncoder asks the source to scale down and drops a few initial
  // frames.
  // TODO(nisse): This is a poor interface, and mixes bandwidth estimation and
  // codec configuration in an undesired way. For the actual send bandwidth, we
  // should always be somewhat conservative, but we may nevertheless want to let
  // the application configure a more optimistic quality for the initial
  // resolution. Should be replaced by a construction time setting.
  /*
	设置初始比特率，后续会被 OnBitrateUpdated 方法覆盖。主要影响初始关键帧的分辨率：
	如果传入的帧比初始比特率合理的大小要大，并且缩放被启用，VideoStreamEncoder 
	会请求源缩小尺寸并丢弃一些初始帧。
	
	TODO(nisse): 这是一个设计不佳的接口，它以不期望的方式混合了带宽估计和编解码器配置。
	对于实际的发送带宽，我们应该始终保持一定的保守性，但我们可能仍然希望允许应用程序为
	初始分辨率配置一个更乐观的质量。应该被构造时的设置所取代。
  */
  virtual void SetStartBitrate(int start_bitrate_bps) = 0;

  // Request a key frame. Used for signalling from the remote receiver with
  // no arguments and for RTCRtpSender.generateKeyFrame with a list of
  // rids/layers.
  /*
	请求一个关键帧。用于从远程接收方发出信号，无需参数，
	以及用于 RTCRtpSender.generateKeyFrame 方法，该方法带有一系列 rids/layers。
  */
  virtual void SendKeyFrame(const std::vector<VideoFrameType>& layers = {}) = 0;

  // Inform the encoder that a loss has occurred.
  // 通知编码器发生了丢包
  virtual void OnLossNotification(
      const VideoEncoder::LossNotification& loss_notification) = 0;

  // Set the currently estimated network properties. A `target_bitrate`
  // of zero pauses the encoder.
  // `stable_target_bitrate` is a filtered version of `target_bitrate`. It  is
  // always less or equal to it. It can be used to avoid rapid changes of
  // expensive encoding settings, such as resolution.
  // `link_allocation` is the bandwidth available for this video stream on the
  // network link. It is always at least `target_bitrate` but may be higher
  // if we are not network constrained.
  /*
	设置当前估计的网络属性。`target_bitrate` 为零时会暂停编码器。 `stable_target_bitrate` 
	是 `target_bitrate` 的一个过滤版本。它总是小于或等于 `target_bitrate`。它可以用来避免
	昂贵编码设置（如分辨率）的快速变化。`link_allocation` 是网络链路上可用于此视频流的带宽。
	它总是至少为 `target_bitrate`，但如果我们不受网络限制，可能会更高。
  */
  virtual void OnBitrateUpdated(DataRate target_bitrate,
                                DataRate stable_target_bitrate,
                                DataRate link_allocation,
                                uint8_t fraction_lost,
                                int64_t round_trip_time_ms,
                                double cwnd_reduce_ratio) = 0;

  // Set a FecControllerOverride, through which the encoder may override
  // decisions made by FecController.
  /*
	设置一个 FecControllerOverride，通过它编码器可以覆盖 FecController 所做的决策
  */
  virtual void SetFecControllerOverride(
      FecControllerOverride* fec_controller_override) = 0;

  // Creates and configures an encoder with the given `config`. The
  // `max_data_payload_length` is used to support single NAL unit
  // packetization for H.264.
  /*
	使用给定的 `config` 创建并配置一个编码器。`max_data_payload_length` 用于支持 H.264 
	的单个 NAL 单元打包。
  */ 
  virtual void ConfigureEncoder(VideoEncoderConfig config,
                                size_t max_data_payload_length) = 0;
  virtual void ConfigureEncoder(VideoEncoderConfig config,
                                size_t max_data_payload_length,
                                SetParametersCallback callback) = 0;

  // Permanently stop encoding. After this method has returned, it is
  // guaranteed that no encoded frames will be delivered to the sink.
  virtual void Stop() = 0;
};

1.2 资源类（Resource）

上面关于Resource类的声明位于api/adaptation/resource.h中，这里的可以表示帧率，码率等编码信息，这个类存在的意义是对这些信息进行监控，如果某个资源超出了预期范围，需要将这个资源进行调控。

// A Resource monitors an implementation-specific resource. It may report
// kOveruse or kUnderuse when resource usage is high or low enough that we
// should perform some sort of mitigation to fulfil the resource's constraints.
//
// The methods on this interface are invoked on the adaptation task queue.
// Resource usage measurements may be performed on an any task queue.
//
// The Resource is reference counted to prevent use-after-free when posting
// between task queues. As such, the implementation MUST NOT make any
// assumptions about which task queue Resource is destructed on.
/*
  1.一个资源监控特定实现的资源。当资源使用量足够高或足够低，以至于我们应该执行某种缓解
  措施以满足资源的约束时，它可能会报告kOveruse（过载）或kUnderuse（欠载）。

  2.这个接口上的方法在适应性任务队列上被调用。资源使用量的测量可以在任何任务队列上执行。

  3.资源被引用计数，以防止在任务队列之间发布时出现使用后释放的问题。因此，
  实现必须不对资源被销毁的任务队列做任何假设。
*/
class RTC_EXPORT Resource : public RefCountInterface {
 public:
  Resource();
  // Destruction may happen on any task queue.
  ~Resource() override;

  virtual std::string Name() const = 0;
  // The `listener` may be informed of resource usage measurements on any task
  // queue, but not after this method is invoked with the null argument.
  virtual void SetResourceListener(ResourceListener* listener) = 0;
};

// ResourceListenter和ResourceUsageState的声明如下
enum class ResourceUsageState {
  // Action is needed to minimze the load on this resource.
  // 过载，需要降低这个资源
  kOveruse,
  // Increasing the load on this resource is desired, if possible.
  // 欠载，需要增加这个资源
  kUnderuse,
};

RTC_EXPORT const char* ResourceUsageStateToString(
    ResourceUsageState usage_state);

class RTC_EXPORT ResourceListener {
 public:
  virtual ~ResourceListener();
  // 评估当前资源的状态
  virtual void OnResourceUsageStateMeasured(
      rtc::scoped_refptr<Resource> resource,
      ResourceUsageState usage_state) = 0;
};

1.3 已编码图像的回调（EncodedImageCallback）

EncodedImageCallback用于图像已经编码之后的回调，声明位于api/video_codecs/video_encoder.h中

class RTC_EXPORT EncodedImageCallback {
 public:
  virtual ~EncodedImageCallback() {}

  struct Result {
    enum Error {
      OK,

      // Failed to send the packet.
      ERROR_SEND_FAILED,
    };

    explicit Result(Error error) : error(error) {}
    Result(Error error, uint32_t frame_id) : error(error), frame_id(frame_id) {}

    Error error;

    // Frame ID assigned to the frame. The frame ID should be the same as the ID
    // seen by the receiver for this frame. RTP timestamp of the frame is used
    // as frame ID when RTP is used to send video. Must be used only when
    // error=OK.
    /*
		分配给帧的帧ID。该帧ID应与接收方看到的该帧的ID相同。当使用RTP发送视频时，
		帧的RTP时间戳被用作帧ID。仅当错误为OK时必须使用。
	*/
    uint32_t frame_id = 0;

    // Tells the encoder that the next frame is should be dropped.
    // 告诉编码器下一个帧应该被丢弃。
    bool drop_next_frame = false;
  };

  // Used to signal the encoder about reason a frame is dropped.
  // kDroppedByMediaOptimizations - dropped by MediaOptimizations (for rate
  // limiting purposes).
  // kDroppedByEncoder - dropped by encoder's internal rate limiter.
  // TODO(bugs.webrtc.org/10164): Delete this enum? It duplicates the more
  // general VideoStreamEncoderObserver::DropReason. Also,
  // kDroppedByMediaOptimizations is not produced by any encoder, but by
  // VideoStreamEncoder.
  /*
	用于向编码器发出信号，说明帧被丢弃的原因。
    kDroppedByMediaOptimizations: 由 MediaOptimizations 丢弃（出于速率限制的目的）。
    kDroppedByEncoder: 由编码器的内部速率限制器丢弃。
    
    TODO(bugs.webrtc.org/10164): 删除这个枚举？它与更通用的 VideoStreamEncoderObserver::DropReason 重复。此外，
  	kDroppedByMediaOptimizations 不是由任何编码器产生的，而是由 VideoStreamEncoder 产生的。
  */
  enum class DropReason : uint8_t {
    kDroppedByMediaOptimizations,
    kDroppedByEncoder
  };

  // Callback function which is called when an image has been encoded.
  // 在一帧被编码之后会被调用
  virtual Result OnEncodedImage(
      const EncodedImage& encoded_image,
      const CodecSpecificInfo* codec_specific_info) = 0;
  // 丢弃一帧
  virtual void OnDroppedFrame(DropReason /* reason */) {}
};

1.4 视频资源监听器（VideoSourceRestrictionsListener）

// The listener is responsible for carrying out the reconfiguration of the video
// source such that the VideoSourceRestrictions are fulfilled.
// 监听器负责执行视频源的重新配置，以满足 VideoSourceRestrictions 的要求。
class VideoSourceRestrictionsListener {
 public:
  virtual ~VideoSourceRestrictionsListener();

  // The `restrictions` are filtered by degradation preference but not the
  // `adaptation_counters`, which are currently only reported for legacy stats
  // calculation purposes.
  /*
	`restrictions` 会根据降级偏好进行过滤，但 `adaptation_counters` 不会，
	它们目前仅用于传统统计计算目的的报告。
  */
  virtual void OnVideoSourceRestrictionsUpdated(
      VideoSourceRestrictions restrictions,
      const VideoAdaptationCounters& adaptation_counters,
      rtc::scoped_refptr<Resource> reason,
      const VideoSourceRestrictions& unfiltered_restrictions) = 0;
};

总体来讲，VideoStreamEncoder完成了进行视频编码的第一道工序，包括获取视频帧，根据编码情况进行码率调整，根据网络情况丢帧等等，由于涵盖内容太广泛，一些其他很重要的类没有分析，例如DegradationPreferenceManager、QpParser等，后续单独记录