Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/guide/17-txsql_architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,19 @@ InnoDB的底层结构主要由两部分组成:内存结构和物理文件。

### 物理文件
TXSQL的物理文件主要有如下集中类型:



- 数据文件(data file)是存储引擎存储数据的主要文件,它是由一个或多个固定大小的数据页(data page)组成的文件,每个数据页的大小默认为16KB。数据文件可以是单个文件(ibdata1)或多个文件(ibdata1, ibdata2, …),也可以是按表分割的文件(table_name.ibd)。数据文件中存储了表的数据、索引、元数据等信息。



- 日志文件(log file)是存储日志的主要文件,它是由两个或多个固定大小的日志段(log segment)组成的循环文件,每个日志段的大小默认为48MB。日志文件可以是单个文件(ib_logfile0)或多个文件(ib_logfile0, ib_logfile1, …)。日志文件中存储了事务的修改记录,用于保证事务的持久性和原子性。



- 索引文件(index file)是存储索引的辅助文件,它是由一个或多个固定大小的索引页(index page)组成的文件,每个索引页的大小默认为16KB。索引文件可以是单个文件(table_name.MYI)或多个文件(table_name.MYI, table_name.MYI.1, …)。索引文件中存储了表的辅助索引,用于加速数据查询和排序等操作。



- 临时文件(temp file)是存储临时数据的辅助文件,它是由一个或多个固定大小的临时页(temp page)组成的文件,每个临时页的大小默认为16KB。临时文件可以是单个文件(ibtmp1)或多个文件(ibtmp1, ibtmp2, …)。临时文件中存储了查询过程中产生的临时表、排序结果、分组结果等信息。
36 changes: 36 additions & 0 deletions docs/release/v5-0.en.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# V5.0-release

The main changes in V5.0-release focus on the following areas:

## 1. Distributed Architecture and Performance Optimization

- Upgraded the transaction mechanism by eliminating the ProcArray performance bottleneck and replacing clog with csnlog, improving performance by more than 50% in high-concurrency TP scenarios.
- Upgraded the distributed execution architecture with a CN Coordination + DOP + Pipeline execution model, reducing redundant processes and blocking across data shards during distributed query execution and improving parallel query execution efficiency.
- Optimized the data forwarding layer through mechanisms such as control flow optimization and data flow reuse, reducing data forwarding overhead in large-scale clusters and improving query performance under HTAP mixed workloads.

## 2. Enhanced Oracle Compatibility

- Enhanced Oracle database compatibility by implementing isolation between PG and Oracle modes at the syntax, metadata, and view layers.
- Enhanced compatibility with Oracle syntax, data types, system views, functions, PLSQL, and other features.
- Supported the creation of both PG-mode and Oracle-mode databases within the same cluster while reusing the underlying distributed storage and computing framework, reducing the cost of Oracle application migration and adaptation.

## 3. Enhanced Resource Management

- Added SQL firewall capabilities to identify and block abnormal and high-resource-consumption SQL statements.
- Enhanced real-time database resource monitoring and management, with support for managing resources such as concurrency, memory, and CPU.
- Added a circuit-breaking mechanism for high-resource-consumption SQL statements, reducing the impact of abnormal SQL on overall cluster workload and stability.

## 4. Enhanced AI and Vector Database Capabilities

- Added distributed adaptation for pgvector, enhancing vector data storage, query, and indexing capabilities in distributed environments.
- Supported parallel creation of vector indexes, improving indexing efficiency for embedding data.
- Supported building RAG knowledge bases based on vector retrieval, enabling unified storage and querying of relational and vector data.
- Added the opentenbase_ai extension, enabling direct invocation of large language model capabilities through SQL.
- Supported large language models including OpenAI, DeepSeek, Tencent Hunyuan, and Alibaba Tongyi Qianwen, as well as integration with other model services through custom interfaces.
- Supported AI capabilities including text generation, text summarization, translation, sentiment analysis, Q&A extraction, embedding generation, and image analysis.

## 5. Enhanced Multimodal Data Processing Capabilities

- Enhanced unified processing capabilities for structured, semi-structured, and unstructured data.
- Supported the storage and analysis of text, images, vectors, and relational data within the same database.
- Supported joint analysis of relational and multimodal data through SQL, extending OpenTenBase capabilities for scenarios such as intelligent retrieval, knowledge bases, and multimodal data analytics.
36 changes: 36 additions & 0 deletions docs/release/v5-0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# V5.0-release

V5.0-release 版本主要修改集中在:

## 1. 分布式架构及性能优化

- 事务机制升级,去除 ProcArray 性能瓶颈,使用 csnlog 替代 clog,高并发 TP 场景性能提升 50% 以上。
- 分布式执行架构升级,采用 CN 协调 + DOP + Pipeline 执行模式,减少分布式查询执行过程中的冗余进程及分片阻塞,提升并行查询执行效率。
- 数据转发层优化,通过控制流优化、数据流复用等机制,降低大规模集群下的数据转发开销,提升 HTAP 混合负载场景下的查询性能。

## 2. Oracle兼容性增强

- 增强 Oracle 数据库兼容能力,在语法层、元数据层及视图层实现 PG/Oracle 模式隔离。
- 增强 Oracle 语法、数据类型、系统视图、函数及 PLSQL 等兼容能力。
- 支持在同一集群中创建 PG 模式和 Oracle 模式数据库,并复用底层分布式存储及计算框架,降低 Oracle 应用迁移及改造成本。

## 3. 资源管控能力增强

- 增加 SQL 防火墙能力,支持对异常及高资源消耗 SQL 进行识别和拦截。
- 增强数据库资源实时监控及管控能力,支持对并发数、内存、CPU 等资源进行管理。
- 增加高消耗 SQL 熔断机制,降低异常 SQL 对集群整体负载及稳定性的影响。

## 4. AI及向量数据库能力增强

- 支持 pgvector 分布式适配,增强分布式环境下的向量数据存储、查询及索引能力。
- 支持向量索引并行创建,提升 embedding 数据的索引构建效率。
- 支持基于向量检索构建 RAG 知识库,实现关系数据与向量数据的统一存储和查询。
- 新增 opentenbase_ai 插件,支持直接通过 SQL 调用大语言模型能力。
- 支持 OpenAI、DeepSeek、腾讯混元、阿里通义千问等大模型,并支持通过自定义接口接入其他模型服务。
- 支持文本生成、文本摘要、翻译、情感分析、问答提取、Embedding 生成及图像分析等 AI 能力。

## 5. 多模态数据处理能力增强

- 增强结构化、半结构化及非结构化数据的统一处理能力。
- 支持文本、图像、向量及关系数据在同一数据库中进行存储和分析。
- 支持通过 SQL 对关系数据与多模态数据进行关联分析,扩展 OpenTenBase 在智能检索、知识库及多模态数据分析等场景下的应用能力。
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ nav:
- 使用docker构建一个简单集群: guide/15-docker-deploy.md
- 版本说明:
- OpenTenBase:
- v5.0: release/v5-0.md
- v2.6.0: release/v2-6-0.md
- v2.5.0: release/v2-5-0.md
- v2.4.0: release/v2-4-0.md
Expand Down