1
2
3
4
5
6
7
8
9
10
11
12
13
14

Let me search through the codebase to find the entry points for partition search in AV1.
 **Search codebase for "partition serach entry point cuntion av1 encoder". No results found**
 Let me try a more specific search using grep to find partition-related functions.    
 **Grepped codebase for "rd_pick_partition"**

Based on the search results, I can identify the main entry points for partition search in AV1. There are two primary functions:

1. av1_rd_pick_partition() - This is the main rate-distortion (RD) based partition search function, located in partition_search.c. This is used when RD optimization is enabled.

2. av1_nonrd_pick_partition() - This is the non-RD based partition search function, also in partition_search.c. This is a faster version used when RD optimization is disabled.

Let me get more details about these functions to provide you with a better understanding.
 **Read partition_search.c L5457-5674**
 The main entry point is av1_rd_pick_partition() in partition_search.c. This function performs a thorough rate-distortion optimization to find the best partition structure. It's used when quality is prioritized over encoding speed. The function signature is:

1
2
3

让我帮你找一下与global motion搜索相关的文件。
**Searched codebase for "global motion search".**
根据搜索结果，我可以看到主要用于global motion搜索的文件是av1/encoder/global_motion_facade.c。这个文件包含了全局运动估计的主要实现。让我解释一下关键部分：

1
2
3
4
5
6
7
8
9
10
11
12

让我搜索一下与速度档次和算法控制相关的代码
** Searched codebase for "speed settings control algorithm enabled disable" **
根据搜索结果,我发现在 av1/encoder/speed_features.h 和 av1/encoder/speed_features.c 中定义了控制不同速度档次下算法开关的代码。主要的控制结构如下:

在 speed_features.h 中定义了多个结构体来组织不同类型的速度特性:
HIGH_LEVEL_SPEED_FEATURES: 高层速度特性
FIRST_PASS_SPEED_FEATURES: 第一遍编码的速度特性
PARTITION_SPEED_FEATURES: 分区相关的速度特性
MV_SPEED_FEATURES: 运动估计相关的速度特性
INTER_MODE_SPEED_FEATURES: 帧间预测模式相关的速度特性
INTERP_FILTER_SPEED_FEATURES: 插值滤波器相关的速度特性
INTRA_MODE_SPEED_FEATURES: 帧内预测模式相关的速度特性

1
2
3
4
5

export ANTHROPIC_API_KEY=xxx
aider --model anthropic/claude-3-5-sonnet-latest
# Enter the command line of aider
/chat-mode ask
Which function is the entry points for partition search?

1
2
3
4
5

**Update repo-map**
From the files shown so far, I cannot definitively determine the main entry point for partition search. While I can see
references to partition-related code like partition_search_type and variance_partition_alloc(), the actual partition search entry
point is likely in files that haven't been shared yet.

RAG stands for Retrieve-Augmented Generation, a technique that enhances the capabilities of language models by incorporating external information. Compared to pure Language Models (LLMs), RAG offers several advantages:

1. More Precise Answers: By retrieving relevant information from a knowledge base, RAG can provide more accurate and contextually appropriate responses.
2. Reduced Hallucinations: RAG reduces the likelihood of generating false or irrelevant information by grounding its outputs in real-world data.
3. Access to Latest Information: RAG can utilize up-to-date external sources, ensuring that the generated content is current.

RAG Structure

The typical structure of a RAG system includes:

1. Vector Database: This stores the knowledge base as vectors for efficient retrieval.
2. Embedding Model: Converts text from the knowledge base into vector representations.
3. Language Model (LLM): Generates responses based on retrieved context and its own understanding.

RAG Process

The process of RAG involves two main steps:

1. Indexing: The knowledge base is indexed using an embedding model, converting textual information into vectors for storage in a vector database.
2. Query and Response Generation: When a query is received, the system retrieves relevant context from the vector database and uses it to generate a response with the help of the LLM.

Open-Source Solutions

Several open-source frameworks are available for implementing RAG:

1. LLamaIndex: Connects to various data sources and converts data into vectors using an embedding model.
2. HayStack
3. LangChain
4. GraphRAG: A logic reasoning framework that supports multi-hop fact questions.
5. KAG: An OpenSPG engine-based framework for building domain-specific knowledge bases.
6. AnythingLLM: Provides a chat interface to convert documents into context for LLMs.
7. MaxKB: An open-source knowledge base问答系统 used in customer service, internal knowledge bases, and education.
8. RAGFlow: A deep document understanding-based RAG engine that provides reliable Q&A services.
9. FastGPT: A knowledge base问答 system with data processing and model calling capabilities.
10. Langchain-Chatchat: Local knowledge base Q&A based on Langchain and ChatGLM.
11. FlashRAG: A Python toolset for reproducing and developing RAG research, including 36 preprocessed benchmark datasets and 15 advanced algorithms.
12. Open WebUI（前身为Ollama WebUI）是一个可扩展的、功能丰富的、用户友好的自托管Web界面，设计用于完全离线运行。它支持各种LLM（大型语言模型）运行器，包括Ollama和兼容OpenAI的API。

Vector Database

• Qdrant: A fast, Rust-based vector database focusing on performance and efficiency.
• Chroma: Popular for its simplicity and ease of use.

Embedding Model

• Sentence Transformers: A library for generating high-quality sentence embeddings.
• BERT (Bidirectional Encoder Representations from Transformers): BERT is a transformer-based model known for its ability to understand context by pre-training on a large corpus of text.

Language Models (LLMs)

• llama3.3
• Gemma2

Cloud Solutions

Cloud-based services that support RAG include:

• Google Vertex AI Matching Engine: Provides vector search and a managed RAG solution for enterprise search.
• AWS Kendra + Sagemaker/Bedrock: Combines with Kendra for enterprise search and Bedrock for LLMs to build RAG solutions.
• Azure AI Search + Azure OpenAI Service: Offers vector search and integrates well with Azure OpenAI Service for building RAG applications.
• 火山方舟大模型服务平台: A platform for large language models.
• 腾讯云ES: Based on the ElasticSearch ecosystem.
• 阿里云PAI: Guide to deploying a RAG-based dialogue system.

Reference

火山方舟大模型服务平台
Learn how to deploy a RAG-based dialogue system on Aliyun PAI
详解几种常见本地大模型个人知识库工具部署、微调及对比选型
RAG框架总结主流框架推荐
搭建个人 RAG 推理服务

1
2
3
4
5
6
7
8

---
title: Hello World
date: 2025-01-01
tags:
  - Test
categories:
  - Debug
---

1

ssh -L 11434:localhost:11434 -fNT remoteserver -o IdentitiesOnly=yes -q -A -T

1

curl -fsSL https://ollama.com/install.sh | sh

1
2

$: ollama run llama3.3
>>> Send a message (/? for help)

1
2
3

~$ ollama ps
NAME               ID              SIZE     PROCESSOR          UNTIL
llama3.3:latest    a6eb4748fd29    45 GB    46%/54% CPU/GPU    4 minutes from now

1
2
3
4
5
6
7
8
9
10

~$ ollama list
NAME                        ID              SIZE      MODIFIED
qwen2.5:32b                 9f13ba1299af    19 GB     7 days ago
llava:latest                8dd30f6b0cb1    4.7 GB    10 days ago
codegemma:latest            0c96700aaada    5.0 GB    10 days ago
internlm2:20b               a864ac8dade2    11 GB     10 days ago
internlm2:latest            5050e36678ab    4.5 GB    10 days ago
glm4:latest                 5b699761eca5    5.5 GB    10 days ago
deepseek-coder-v2:latest    63fb193b3a9b    8.9 GB    12 days ago
wizardcoder:latest          de9d848c1323    3.8 GB    12 days ago

1

sudo systemctl edit ollama.service

1
2

# [Service] 
# Environment="OLLAMA_HOST=0.0.0.0"

1
2
3

sudo systemctl daemon-reload 
sudo systemctl restart ollama 
sudo systemctl status ollama.service

1
2
3
4
5
6
7
8
9
10

● ollama.service - Ollama Service 
Loaded: loaded (/etc/systemd/system/ollama.service; enabled; vendor preset: enabled) 
Drop-In: /etc/systemd/system/ollama.service.d 
         └─override.conf 
Active: active (running) since Fri 2024-12-20 07:40:24 CST; 6min ago 
Main PID: 1241 (ollama) 
Tasks: 15 (limit: 37306) 
Memory: 139.7M 
CGroup: /system.slice/ollama.service 
        └─1241 /usr/local/bin/ollama serve

Video Buffer Verifier是编码器内部的一个模块，用来确保解码器可以smooth播放视频。通常会假设decoder内部有一块buffer，专门用于缓冲输入码流，这块buffer的size是固定的。我们先考虑下解码端的情况，再来看下反推到编码端是怎么样的

解码端是从网络/CD中读入码流，读入的速度可以是固定的，也可以是变化的，我们先考虑固定的情况。假如$R{dec}^{in}$是解码器读入码流的速度，解码器把码流存储在码流Buffer中，Buffer的size是$S{buf}$，另外一方面，当Buffer中的数据够一帧的数据时，解码器就会把整帧的数据一次性消耗掉。

理论上来说，当Buffer中没有空间来缓存数据时，这种情况就叫Buffer Overflow，这种情况是可能是因为$R{dec}^{out}$太小导致; 另外一种情况是，当Buffer中没有数据的时候，就叫Buffer Underflow。这种情况是$R{dec}^{out}$太快，大于$R_{dec}^{in}$，导致没有数据；

以上是解码端的情况，VBV本身就是模拟下解码端的情况，从而来确保编码端的码流符合规范；$R{enc}^{in}$代表送给编码端VBV的速率，$R{enc}^{out}$代表从VBV流出的码率；注意
$R{enc}^{out} = R{dec}^{in},\
R{enc}^{in} = R{dec}^{out}$

因此，当$R{enc}^{in} > R{enc}^{out}$时，对应的是$R{dec}^{out} > R{dec}^{int}$，也就是码流比较高的时候，会导致Buffer underflow;

当$R{enc}^{in} < R{enc}^{out}$是，对应解码器端 $R{dec}^{in} < R{dec}^{out}$，也就是实际编码码率较低的时候，会导致Buffer overflow.

以上就是vbv的基本原理，以下介绍下264/265里面的设置。

X264’s VBV setting

—vbv-maxrate:

Maximum local bitrate (kbits/sec). Will be used only if vbv-bufsize is also non-zero. Both vbv-bufsize and vbv-maxrate are required to enable VBV in CRF mode. Default 0 (disabled)

—vbv-bufsize:
Specify the size of the VBV buffer (kbits). Enables VBV in ABR mode. In CRF mode, --vbv-maxrate must also be specified. Default 0 (vbv disabled)

—vbv-init:

Initial buffer occupancy. The portion of the decode buffer which must be full before the decoder will begin decoding. Determines absolute maximum frame size. May be specified as a fractional value between 0 and 1, or in kbits. In other words, these two option pairs are equivalent:

vbv-bufsize / vbv-maxrate代表了缓冲几秒的视频；

—vbv-end:

Final buffer fullness. The portion of the decode buffer that must be full after all the specified frames have been inserted into the decode buffer.

—min-vbv-fullness

Minimum VBV fullness percentage to be maintained. Specified as a fractional value ranging between 0 and 100. Default 50 i.e, Tries to keep the buffer at least 50% full at any point in time.

Decreasing the minimum required fullness shall improve the compression efficiency, but is expected to affect VBV conformance. Experimental option.

—max-vbv-fullness

Maximum VBV fullness percentage to be maintained. Specified as a fractional value ranging between 0 and 100. Default 80 i.e Tries to keep the buffer at max 80% full at any point in time.

Increasing the minimum required fullness shall improve the compression efficiency,

m_bufferFillFinal: 代表了已经填充的buffer(因为在模拟解码端的处理？)，也就相等于编码器端的可用空间

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// Initial
if (m_param->rc.vbvBufferInit > 1.)
    m_param->rc.vbvBufferInit = x265_clip3(0.0, 1.0, m_param->rc.vbvBufferInit / m_param->rc.vbvBufferSize);
if (m_param->vbvBufferEnd > 1.)
    m_param->vbvBufferEnd = x265_clip3(0.0, 1.0, m_param->vbvBufferEnd / m_param->rc.vbvBufferSize);
if (m_param->vbvEndFrameAdjust > 1.)
    m_param->vbvEndFrameAdjust = x265_clip3(0.0, 1.0, m_param->vbvEndFrameAdjust);

m_param->rc.vbvBufferInit = x265_clip3(0.0, 1.0, X265_MAX(m_param->rc.vbvBufferInit, m_bufferRate / m_bufferSize));
m_bufferFillFinal = m_bufferSize * m_param->rc.vbvBufferInit;

// Update
m_bufferFillFinal -= frame_bits;
m_bufferFillFinal += rce->bufferRate;

AV1

rc→starting_buffer_level: 代表了已经填充的Buffer(从解码器的角度来看)，相当于编码端的可用空间；

rc→buffer_level: 代表了已经填充Buffer（从解码端的角度来看),从编码器的角度来看，代表了可用的buffer空间

正常对它的处理如下:

1
2
3
4
5
6
7
8
9
10
11
12

// Initialize
p_rc->starting_buffer_level = starting * bandwidth / 1000;
if (p_rc->starting_buffer_level >= p_rc->maximum_buffer_size) {
  p_rc->starting_buffer_level = p_rc->maximum_buffer_size * init_level / 1000;
}

rc->buffer_level = p_rc->starting_buffer_level;

// Update, 
int64_t new_buffer_lvl = rc->buffer_level - rc->projected_frame_size;
if (cpi->common.show_frame) new_buffer_lvl += rc->avg_frame_bandwidth;

Initialize:

对VBV的一些疑问

1. VBV假设$R_{dec}^{in}$是固定的，这个假设是对的吗？一般对于VCD/DVD/Blu.ray来说，可能需要读取fixed bitrate，而此时这个假设是可以；如果没有这个限制，譬如说live streaming，此时读取的内容就是发送的数据，发送的数据即使少于码率，也不会有Buffer Overflow发生。

2. In video streaming， buffer overflow是不大会出现，可以忽略。这是因为不可能出现解码器的输入码率不可能大于解码器消耗的码率。因为输入码率是等于$Min(bandwidth, R_{enc}^{out})$，

   而$R{enc}^{out} =R{dec}^{out}$，因此必然会有$R{dec}^{in} ≤ R{dec}^{out}$。

Reference

https://www.youtube.com/watch?v=-Q7BuSXdO_8&ab_channel=Demuxed

在视频压缩领域，常常需要比较不同算法或者不同编码器的优劣，而评价一个编码器很重要的一个指标是压缩率。计算压缩率需要比较质量和码率，相同画质下码率越低，压缩率越高；或者相同码率下，质量越高，压缩率也越高。

在实际比较的时候，因为很多的编码算法会同时影响质量和码率，导致码率或者质量无法卡齐，不好直接对比压缩率。在这种情况下就需要BD-Rate这个工具。

BD-BR全称是Bjontegaard delta bitrate (BD-BR) ，与2001年提出来的。它可以定量比较不同编码器的压缩性能，这些年一直被广泛用于编码器的评测；它主要是通过计算两条RD曲线上之间的差距来度量不同编码器的优劣，具体可以参见下图:

R-D曲线的X轴是码率，Y轴是客观指标。计算的时候可以计算R-D曲线针对X轴的积分(BD-PSNR)，也可以计算R-D曲线对于Y轴的积分(BD-Rate)；BD-PSNR可以认为是delta psnr，因此正值代表了画质提升；BD-Rate可以认为是delta rate，因此负值代表了码率节省。这里重点说明下BD-Rate.

计算方式

假如要评测两个编码器A和B的压缩率，衡量质量的标准使用PSNR；在计算BD-Rate之前，需要先使用4个或更多个码率点来进行编码，例如2Mbps, 2.5Mbps, 3Mbps和4Mbps，编码完成后统计对应4个码率点的码率和质量(PSNR);

1. 对码率先做对数变换，这个主要是为了防止不同质量等级下码率差异太大，导致高码率对整个的计算影响太大;

1. 使用interpolation函数拟合出两条Rate-Distortion曲线，X轴是Rate，Y轴是PSNR; 例如$r = a + b D + c D^2 + d*D^3$
2. 计算两条曲线对于Y轴积分的差，积分的时候只对Distortion有重叠的区域做积分；因此需要先计算出$D{high}$和$D{low}$
3. 对Distortion部分做normalize，也就是除以$D{high} - D{low}$
4. 最后再通过幂函数恢复到原来的域

最后的整个公式就是

以下是Netflix开源的bdrate计算脚本: https://github.com/Netflix/vmaf/tree/master/python/vmaf/tools

注意事项

虽然BD-Rate对应着码率节省，但实际计算的时候BD-Rate并不等于码率节省的比例。

另外BD-Rate有单调性的假设，也就是质量会随着码率升高而升高。

一般来说，0.5db提升等同于-10%的BD-Rate；

graph TDA[Hard] -->|Text| B(Round)B --> C{Decision}C -->|One| D[Result 1]C -->|Two| E[Result 2]

graph TD;    A-->B;    A-->C;    B-->D;    C-->D;

1
2
3
4
5
6
7
8
9
10
11

import nni

def main():
    para = nni.get_next_parameter()
    x = para['x']
    # square(x) = 2
    score = abs(x*x - 2)
    nni.report_final_result(score)

if __name__ == '__main__':
    main()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

[2023-10-06 21:51:13] WARNING: Config field "authorName" is no longer supported and has been ignored
[2023-10-06 21:51:13] WARNING: You are using legacy config file, please update it to latest format:
================================================================================
experimentName: search_sqrt_2
trialConcurrency: 1
maxExperimentDuration: 1h
maxTrialNumber: 100
searchSpaceFile: search_space.json
useAnnotation: false
trialCommand: python3 nni_search.py
trialCodeDirectory: .
trialGpuNumber: 0
tuner:
  name: TPE
  classArgs:
    optimize_mode: minimize
trainingService:
  platform: local
================================================================================
Reference: https://nni.readthedocs.io/en/stable/reference/experiment_config.html
[2023-10-06 21:51:13] Creating experiment, Experiment ID: vl4y6ge3
[2023-10-06 21:51:13] Starting web server...
[2023-10-06 21:51:14] Setting up...
[2023-10-06 21:51:14] Web portal URLs: http://127.0.0.1:8080 http://192.168.50.103:8080
[2023-10-06 21:51:14] To stop experiment run "nnictl stop vl4y6ge3" or "nnictl stop --all"
[2023-10-06 21:51:14] Reference: https://nni.readthedocs.io/en/stable/reference/nnictl.html

1

$ hexo new "My New Post"

1

$ hexo server

1

$ hexo generate

1

$ hexo deploy