中文
首页/案例/Apache ShardingSphere & iQiyi: What's Database Mesh?

Apache ShardingSphere & iQiyi: What's Database Mesh?

Also known as the Netflix of Asia, iQiyi is an ad-supported television and movie portal providing fully-licensed, high-definition media content.
2022/09/27by Yacine Si Tayeb

To further understand application scenarios, enterprises’ needs, and improve dev teams’ understanding of Apache ShardingSphere, SphereEx launched the “Enterprise Visits” series. The second stop took us to the leading online streaming platform in Asia, iQiyi.

In November 2021, a team from the ShardingSphere community visited the innovation center of iQiyi for in-depth interactions and discussions with their counterparts from Beijing and Shanghai. During the meetup, Zhang Liang, Apache ShardingSphere PMC Chair and SphereEx Founder provided details on the latest Apache ShardingSphere community initiatives, its future development, and Database Mesh.

During the meetup, iQiyi was especially interested in the capabilities and future plans of ShardingSphere and Database Mesh. here are some key takeaway questions from the visit:

Q: How does ShardingSphere manage its encryption keys?

Encryption key management is a capability subordinate to the encryption algorithm. When ShardingSphere needs an encryption key in its decryption, the key could be passed through properties configured by the encryption algorithm. In principle, encryption key management is not a capability of ShardingSphere but the job of the encryption algorithm.

Q:Is there a built-in cryptographic algorithm in ShardingSphere? Or is it done through outside cooperation?

Strictly speaking, ShardingSphere is not equipped with built-in capability. That’s because the pluggable architecture of ShardingSphere is divided into multiple layers, and the encryption module itself is only a capability of L2 of the pluggable architecture.

The encryption module only defines the top-layer encryption connector and the detailed encryption algorithm achieves the pluggable part of the connector. Currently, ShardingSphere supports several of the most commonly used open source algorithms, but it does not support any national cryptographic algorithm in particular.

Integrating an encryption algorithm is easy for developers: instead of changing source codes, all they need is a ShardingSphere encryption algorithm connector. ShardingSphere’s data encryption process is completely transparent to users and very easy to use.

What’s the future plan for Database Mesh?

Database Mesh is another ShardingSphere initiative. We are now planning to integrate Database Mesh through Sidecar. Sidecar will be used to manage traffic, while ShardingSphere will be used to manage to compute, and the underlying databases’ nodes to manage storage.

Sidecar’s and ShardingSphere functions overlap in part, but not completely. ShardingSphere has more than 190 modules, therefore it’s not possible to completely replace ShardingSphere with Sidecar. That’s why ShardingSphere empowers some lightweight capabilities to Sidecar, and Sidecar is equipped with the capabilities to buffer some traffic and requests.

The database computing capability is still processed by ShardingSphere, as it would consume too many application resources to run heavyweight computation via Sidecar for the distributed transaction, query and optimization of heterogeneous databases.

In the future, other Sidecar capabilities, such as management, SQL auditing and authorities, will be developed. In this way, Sidecar and ShardingSphere will have their own focuses, providing a better solution.

Sidecar is equivalent to a data plane. Will you combine the control plane with Istio or will you build the control plane separately?

ShardingSphere uses DistSQL (Distributed SQL) to create rules for sharding, encryption and so on. ShardingSphere is already equipped with the control plane capability of Sidecar. Any capability achieved on Sidecar can be controlled by ShardingSphere’s DistSQL. In the future, we will consider integrating Sidecar into the Istio ecosystem.

So far, Apache ShardingSphere with the release of a version of 5.0 has officially repositioned itself in a new field with the Database Plus concept at its core. In addition to the powerful incremental capability at the upper layer of databases, the Database Plus architecture also opens a highly extensible database ecosystem to developers and users, charting a new future development course for Apache ShardingSphere.

Apache ShardingSphere Open Source Project Links:

[ShardingSphere Github] (https://github.com/apache/shardingsphere/issues?page=1&q=is%3Aopen+is%3Aissue+label%3A%22project%3A+OpenForce+2022%22)

ShardingSphere Twitter

Contributor Guideline

分享文章
wechat qrcode

扫码关注
微信公众号

Apache ShardingSphere & Sohu
Apache ShardingSphere & JD Baitiao: Story of an Implementation Journey

推荐阅读

Apache ShardingSphere & Energy Monster

NASDAQ-listed Energy Monster is the largest provider of mobile device charging services in Asia. This article illustrates how Energy Monster uses ShardingSphere-JDBC.

Apache ShardingSphere & China Unicom Digital Technology: Ensuring Stability, Efficiency, and Replicability at Scale

China Unicom Digital Technology, a global telco operator with 310 million subscribers, chose ShardingSphere to manage its gov and enterprise hotlines data infrastructure.

Heterogeneous migration: reducing Dangdang’s customer system RTO 60x and increasing speed by 20%

E-commerce giant DangDang rebuilt its architecture & migrated 350M users with ShardingSphere & MySQL. RTO improved 60x, response speed by 20%, and introduced cross-data source double write, R/W split, and intelligent gateway innovations.

Apache ShardingSphere & Bilibili

The ShardingSphere community visited one of its enterprise users: the major Chinese video-on-demand streaming platform - Bilibili. Originally known for anime, comics & games, Bilibili has since expanded - as did its total users.

Apache ShardingSphere & Huimin: Huimin’s Transaction Middle Platform Architecture

HuiMin does order fulfillment for over 1 million supermarkets. With ShardingSphere HuiMin shifted to microservices & built a new "middle platform" architecture with packaged business capability for order transaction processing.
即刻免费体验新一代数据库增强引擎
400-900-2818 GitHub
合作伙伴:中国信通院重庆大学ShardingSphere
Privacy PolicyTerms Of UseDisclaimerCookie PolicyDo Not Sell My Personal Information
wechat qrcode

扫码关注
微信公众号