https://vitess.io/blog/Recent content in The Vitess blog on VitessHugo -- gohugo.iohttps://vitess.io/blog/2026-04-30-announcing-vitess-24/Thu, 30 Apr 2026 00:00:00 +0000https://vitess.io/blog/2026-04-30-announcing-vitess-24/releaseVitessv24MySQLkubernetesoperatorvreplicationtracingobservabilitybackupAnnouncing Vitess 24 # The Vitess maintainers are happy to announce the release of version 24.0.0, along with version 2.17.0 of the Vitess Kubernetes Operator. Version 24.0.0 expands query serving capabilities for sharded keyspaces, modernizes Vitess's observability stack, and introduces faster replica provisioning through native MySQL CLONE support. The companion v2.17.0 operator release brings significant improvements to scheduled backups, with new cluster- and keyspace-level schedules that make production backup management much easier to configure at scale.https://vitess.io/blog/2025-11-04-announcing-vitess-23/Tue, 04 Nov 2025 00:00:00 +0000https://vitess.io/blog/2025-11-04-announcing-vitess-23/releaseVitessv23MySQLkubernetesoperatorvreplicationmulti-tenancyUsabilityOnline DDLAnnouncing Vitess 23.0.0 # We’re excited to release Vitess 23.0.0 — the latest major version of Vitess — bringing new defaults, better operational tooling, and refined metrics. This release builds on the strong foundation of version 22 and is designed to make deployment and observability smoother, while continuing to scale MySQL workloads horizontally with confidence. ✅ Why This Release Matters # For production users of Vitess, this release is meaningful in several ways:https://vitess.io/blog/2025-04-29-announcing-vitess-22/Tue, 29 Apr 2025 00:00:00 +0000https://vitess.io/blog/2025-04-29-announcing-vitess-22/releaseVitessv22MySQLkubernetesoperatorvreplicationmulti-tenancyUsabilityOnline DDLAnnouncing Vitess 22 # The Vitess maintainers are happy to announce the release of version 22.0.0, along with version 2.15.0 of the Vitess Kubernetes Operator. This release is the first to benefit from a 6-month-long development cycle, after our recent change to the release cadence. Version 22.0.0 comes with significant enhancements to query serving and cluster management. These changes have allowed Vitess to be more performant and easier to operate compared to version 21.https://vitess.io/blog/2024-11-12-grouping-and-aggregations-on-vitess/Tue, 12 Nov 2024 00:00:00 +0000https://vitess.io/blog/2024-11-12-grouping-and-aggregations-on-vitess/VitessPlanetScaleMySQLQuery ServingVindexplanexecution planexplainoptimizerI love my job. One of the best feelings is when I find an interesting paper and use it to solve a real problem. It feels like I found a cheat code. Instead of having to do a lot of hard thinking, I can just stand on the shoulders of really big people and take a shortcut. Here, I want to share a recent project that I could solve using a public paper.https://vitess.io/blog/2024-11-05-optimizing-query-planning-in-vitess-a-step-by-step-approach/Tue, 05 Nov 2024 00:00:00 +0000https://vitess.io/blog/2024-11-05-optimizing-query-planning-in-vitess-a-step-by-step-approach/VitessPlanetScaleMySQLQuery ServingVindexplanexecution planexplainoptimizeraggregategroup byorder byIntroduction # In this blog post, we will discuss an example of a change to the Vitess query planner and how it enhances the optimization process. The new model focuses on making every step in the optimization pipeline a runnable plan. This approach offers several benefits, including simpler understanding and reasoning, ease of testing, and the ability to use arbitrary expressions in ordering, grouping, and aggregations. Vitess distributed query planner # VTGate is the proxy component of Vitess.https://vitess.io/blog/2024-10-29-announcing-vitess-21/Tue, 29 Oct 2024 00:00:00 +0000https://vitess.io/blog/2024-10-29-announcing-vitess-21/releaseVitessv21MySQLkubernetesoperatorvreplicationmulti-tenancyUsabilityOnline DDLAnnouncing Vitess 21 # We're delighted to announce the release of Vitess 21 along with version 2.14.0 of the Vitess Kubernetes Operator. Version 21 focuses on enhancing query compatibility, improving cluster management, and expanding VReplication capabilities, with experimental support for atomic distributed transactions and recursive CTEs. Key features include reference table materialization, multi-metric throttler support, and enhanced Online DDL functionality. Backup and restore processes benefit from a new mysqlshell engine, while vexplain now offers detailed execution traces and schema analysis.https://vitess.io/blog/2024-09-06-lfx-mentorship-journey/Fri, 06 Sep 2024 00:00:00 +0000https://vitess.io/blog/2024-09-06-lfx-mentorship-journey/VitessArewefastyetmentorshipIntroduction # This summer, I had the incredible opportunity to participate in the LFX Mentorship program, where I contributed to Vitess, a CNCF graduated project. Vitess is a cloud-native database designed for horizontal scaling of MySQL. My specific focus during this mentorship was on the "Arewefastyet" subproject, an essential benchmarking platform for Vitess. "Arewefastyet" plays a crucial role in ensuring that each new version of Vitess maintains or improves performance, making it a vital component of the Vitess ecosystem.https://vitess.io/blog/2024-08-27-vitess-user-survey/Tue, 27 Aug 2024 00:00:00 +0000https://vitess.io/blog/2024-08-27-vitess-user-survey/VitessShardingMySQLWe recently conducted a survey of how Vitess is being used by the community. This blog post summarizes what we learned. "Vitess solves an existential threat for services which outgrow a single MySQL database." "Horizontal sharding helps us scale quickly, and the new generation execution plan, Gen4, enables us to support more SQL queries." "Vitess has allowed us to scale and step away from our dev-ops role, allowing us to focus on higher level and higher impact tooling and automation.https://vitess.io/blog/2024-08-23-recursive-cte/Fri, 23 Aug 2024 00:00:00 +0000https://vitess.io/blog/2024-08-23-recursive-cte/VitessPlanetScaleMySQLCompatibilityCTEWe are excited to announce that Vitess now supports recursive Common Table Expressions (CTEs), marking another significant step in our journey to fully align with MySQL’s capabilities. Recursive CTEs, often a critical feature for complex query handling, allow for the execution of recursive queries within a single CTE. This addition brings more flexibility and power to developers using Vitess, especially those working with distributed databases. One of the key challenges in implementing recursive CTEs within a sharded environment is managing the distribution of data across multiple shards.https://vitess.io/blog/2024-07-29-cdc-vstream/Mon, 29 Jul 2024 00:00:00 +0000https://vitess.io/blog/2024-07-29-cdc-vstream/VitessPlanetScaleMySQLVReplicationCDCDebeziumVitess is a popular CNCF project that is used to scale some of the largest MySQL installations in the world — by companies like Slack, Square, Shopify, and GitHub. It provides sharding, connection pooling, and many other features that make it easy to scale MySQL horizontally. Vitess and MySQL are ideally suited for use as an Online Transaction Processing (OLTP) system — where the end-user interacts directly with the system and fast response times are essential as they get product and service information, generating critical business records such as orders, user profiles, and more.https://vitess.io/blog/2024-07-22-an-interesting-optimization/Mon, 22 Jul 2024 00:00:00 +0000https://vitess.io/blog/2024-07-22-an-interesting-optimization/VitessPlanetScaleMySQLQuery ServingVindexplanexecution planexplainoptimizerIntroduction # I recently encountered an intriguing bug. A user reported that their query was causing vtgate to fetch a large amount of data, sometimes resulting in an Out Of Memory (OOM) error. For a deeper understanding of grouping and aggregations on Vitess, I recommend reading this prior blog post. The Query # The problematic query was: selectsum(user.type)fromuserjoinuser_extraonuser.team_id=user_extra.idgroupbyuser_extra.idorderbyuser_extra.id;The planner was unable to delegate aggregation to MySQL, leading to the fetching of a significant amount of data for aggregation.https://vitess.io/blog/2024-06-27-announcing-vitess-20/Thu, 27 Jun 2024 00:00:00 +0000https://vitess.io/blog/2024-06-27-announcing-vitess-20/releaseVitessv20MySQLkubernetesoperatorvreplicationmulti-tenancyUsabilityOnline DDLWe're delighted to announce the release of Vitess 20 along with version 2.13.0 of the Vitess Kubernetes Operator. Version 20 focuses on usability and maturity of existing features, and continues to build on the solid foundation of scalability and performance established in previous versions. Our commitment remains steadfast in providing a powerful, scalable, and reliable solution for your database scaling needs. What's New in Vitess 20 # Query Compatibility: enhanced DML support including improved query compatibility, Vindex hints, and extended support for various sharded update and delete operations.https://vitess.io/blog/2024-04-29-consistent-lookup-vindex/Mon, 29 Apr 2024 00:00:00 +0000https://vitess.io/blog/2024-04-29-consistent-lookup-vindex/VitessPlanetScaleMySQLQuery ServingVindexVindex # Vitess uses Vindexes (short for Vitess Index) to associate rows in a table with a designated address known as Keyspace ID. This allows Vitess to direct a row to its intended destination, typically a shard within the cluster. Vindexes play a dual role: enabling data sharding through Primary Vindexes and facilitating global indexing via Secondary Vindexes. Through this mechanism, Vindexes serve as an indispensable tool for routing queries in a sharded database, ensuring optimal performance and scalability.https://vitess.io/blog/2024-04-08-vitess-fuzzing-summer-2023-internship/Mon, 08 Apr 2024 00:00:00 +0000https://vitess.io/blog/2024-04-08-vitess-fuzzing-summer-2023-internship/VitessPlanetScaleGolangMySQLQuery ServingMy name is Arvind Murty, and from May to July of 2023, I worked on Vitess via an internship with PlanetScale. I was first introduced to Vitess when I was in high school as a potential open-source project for me to work on. I had been interested in working on one because they’re a relatively easy way to get some real-world experience in large-scale software development. Vitess seemed like an good place to start, so I started contributing, mostly on internal cleanup.https://vitess.io/blog/2024-03-06-announcing-vitess-19/Wed, 06 Mar 2024 00:00:00 +0000https://vitess.io/blog/2024-03-06-announcing-vitess-19/releaseVitessMySQLkubernetesoperatorshardingOrchestrationFailoverHigh-AvailabilityAnnouncing Vitess 19 # We're thrilled to announce the release of Vitess 19, our latest version packed with enhancements aimed at improving scalability, performance, and usability of your database systems. With this release, we continue our commitment to providing a powerful, scalable, and reliable database clustering solution for MySQL. What's New in Vitess 19 # Dropping Support for MySQL 5.7: As Oracle has marked MySQL 5.7 end of life in October 2023, we're also moving forward by dropping support for MySQL 5.https://vitess.io/blog/2023-11-07-announcing-vitess-18/Tue, 07 Nov 2023 00:00:00 +0000https://vitess.io/blog/2023-11-07-announcing-vitess-18/releaseVitessMySQLkubernetesoperatorshardingOrchestrationFailoverHigh-AvailabilityVitess 18 is now Generally Available, with a number of new enhancements designed to improve usability, performance and MySQL compatibility. MySQL Compatibility Improvements # Foreign Keys # In the past, foreign keys had to be managed outside Vitess. This was a significant blocker for adoption. We are now able to support Vitess-managed foreign keys within the same shard. This includes the ability to import data into Vitess from an existing MySQL database with foreign keys.https://vitess.io/blog/2023-06-27-announcing-vitess-17/Tue, 27 Jun 2023 00:00:00 +0000https://vitess.io/blog/2023-06-27-announcing-vitess-17/releaseVitessMySQLkubernetesoperatorshardingOrchestrationFailoverHigh-AvailabilityWe are pleased to announce the general availability of Vitess 17! Major Themes in Vitess 17 # In this release of Vitess, several significant enhancements have been introduced to improve the compatibility, performance, and usability of the system. GA Announcements # The VTTablet settings connection pool feature, introduced in v15, is now enabled by default in this release. This feature simplifies the management and configuration of system settings, providing users with a more streamlined and convenient experience.https://vitess.io/blog/2023-06-05-vitess-security-audit/Mon, 05 Jun 2023 00:00:00 +0000https://vitess.io/blog/2023-06-05-vitess-security-audit/securityVitessMySQLThe Vitess Maintainer team is pleased to announce the results of a recent third-party security audit of the Vitess code base. Vitess had previously been audited in 2019. Given the amount of time that has passed, and the magnitude of change during that time, the maintainer team decided to request a fresh audit. Starting in March 2023, an independent team from Ada Logics performed a full security audit of Vitess with a special focus on VTAdmin, which is a relatively new addition to Vitess.https://vitess.io/blog/2023-05-08-backup-and-restore-performance/Mon, 08 May 2023 00:00:00 +0000https://vitess.io/blog/2023-05-08-backup-and-restore-performance/VitessbackupCluster ManagementrestoreThe performance of backups and restores is a business requirement for Vitess users and an ongoing concern for Vitess maintainers. For sufficiently large databases, if we can't take backups fast enough, we risk missing daily SLAs in a production context. In the event we need to perform an emergency restore, it is paramount that we can do so as fast as possible. The performance of backups and restores is driven by a number of factors:https://vitess.io/blog/2023-04-24-schemadiff/Mon, 24 Apr 2023 00:00:00 +0000https://vitess.io/blog/2023-04-24-schemadiff/VitessMySQLDDLschemaIntroducing schemadiff, an internal library in Vitess that has been one of its best-kept secrets until now. At its core, schemadiff is a declarative, programmatic library that can produce a diff in SQL format of two entities: tables, views, or full blown database schemas. But it then goes beyond that to normalize, validate, export, and even apply schema changes, all declaratively and without having to use a MySQL server. Let's dive in to understand its functionality and capabilities.https://vitess.io/blog/2023-04-17-vtctldserver-api/Mon, 17 Apr 2023 00:00:00 +0000https://vitess.io/blog/2023-04-17-vtctldserver-api/VitessvtctldCluster ManagementWe are thrilled to discuss VtctldServer, the new gRPC API for performing cluster management operations with vtctld components. This is the (near-) culmination of long, steady migration that began back in Vitess v9 (!!!) and went GA in Vitess v15, so we'd like to talk a bit about the motivation behind the move, the design of the new API, and where we go from here. Why? # Vitess users may have found themselves invoking various cluster management operations (think CreateKeyspace, EmergencyReparentShard, Backup, and so on) via the vtctlclient program.https://vitess.io/blog/2023-03-27-connection-pooling-in-vitess/Mon, 27 Mar 2023 00:00:00 +0000https://vitess.io/blog/2023-03-27-connection-pooling-in-vitess/vitessmysqlconnection poolingmysql system settingsconnection poolNote: The content was originally published at PlanetScale Connection pooling is a commonly used technique in modern applications to manage database connections efficiently. It involves creating a cache of database connections that the application can use as needed. Instead of creating a new connection for each request to the database, the application retrieves a connection from the pool. After the application finishes using the connection, it is returned to the pool to be reused later, rather than being closed outright.https://vitess.io/blog/2023-02-28-announcing-vitess-16/Tue, 28 Feb 2023 00:00:00 +0000https://vitess.io/blog/2023-02-28-announcing-vitess-16/releaseVitessMySQLkubernetesoperatorcloudGKEshardingOrchestrationFailoverHigh-AvailabilityBackupWe are pleased to announce the general availability of Vitess 16! Documentation improvements # In this release the maintainer team has decided to put an emphasis on reviewing, editing, and rewriting the website documentation to be current with the code. With help from CNCF, we have also improved the search experience. We welcome feedback on the current incarnation of the docs. GA announcements # We are marking VDiff v2 as Generally Available or production-ready in v16.https://vitess.io/blog/2023-02-13-fosdem-2023-recap/Mon, 13 Feb 2023 00:00:00 +0000https://vitess.io/blog/2023-02-13-fosdem-2023-recap/VitessMySQLFOSDEMMySQLDayconferenceseventsSome of the Vitess maintainers traveled to Brussels, Belgium to attend and speak at pre-FOSDEM MySQL Days and at FOSDEM. pre-FOSDEM MySQL Days # When: Feb 2-3 Who: Deepthi Sigireddi, Harshit Gangal, Shlomi Noach What: VTOrc - How Vitess achieves consensus using MySQL Replication by Deepthi Highlights # There were a lot of questions about how Vitess handles various failure scenarios. We had good conversations with both the MySQL team and MySQL/Vitess community users.https://vitess.io/blog/2022-12-05-vtadmin-intro/Mon, 05 Dec 2022 00:00:00 +0000https://vitess.io/blog/2022-12-05-vtadmin-intro/VitessVTAdminvtctldvtctld2VTAdmin is now generally available for use! VTAdmin provides both a web client and API for managing multiple Vitess clusters, and is the successor to the now-deprecated UI for vtctld. What is VTAdmin? # VTAdmin is made up of two components: VTAdmin API: An HTTP(S) and gRPC API server VTAdmin Web: A React + Typescript web client built with Create React App What can you do with VTAdmin? # VTAdmin can do everything the old vtctld2 UI could do, now that the vtctld2 parity project has been completed.https://vitess.io/blog/2022-11-22-vdiff-v2/Tue, 22 Nov 2022 00:00:00 +0000https://vitess.io/blog/2022-11-22-vdiff-v2/Vitess'MySQLshardingreplicationVitess is a solution that allows you to infinitely scale MySQL while providing clients and apps with a single logical view of the fleet of MySQL instances comprising any number of Keyspaces and Shards. Vitess also provides the cluster and data management tools that make it possible to manage a massive cluster and perform complex workflows using VReplication, such as: Moving tables into Vitess or between keyspaces Resharding to adjust to changes in data size and load Materialized views and rollups for data analytics and data locality Online schema changes that are trackable, cancellable, revertible, and retryable Why a Diff Tool?https://vitess.io/blog/2022-10-26-announcing-vitess-15/Wed, 26 Oct 2022 00:00:00 +0000https://vitess.io/blog/2022-10-26-announcing-vitess-15/releaseVitessMySQLkubernetesoperatorcloudGKEshardingVitess 15 is now generally available, with a number of new enhancements designed to make Vitess easier to use, more resilient, and easier to scale! VTOrc release # VTOrc, a Vitess-native cluster monitoring and recovery component, is now GA. VTOrc monitors and repairs Vitess clusters, eliminating paging and manual intervention and automating recovery. This makes Vitess fully self-healing and resilient to MySQL server failures. It also obsoletes the third-party integration with Orchestrator that users have traditionally relied on to recover from MySQL server failures.https://vitess.io/blog/2022-09-21-vtorc-vitess-native-orchestrator/Wed, 21 Sep 2022 00:00:00 +0000https://vitess.io/blog/2022-09-21-vtorc-vitess-native-orchestrator/VitessOrchestratorVTOrcMySQLCluster ManagementFailoverFault toleranceDistributed SystemsCloud NativeThere was an idea. An idea to make Vitess self-reliant. An idea to get rid of the friction between Vitess and external fault-detection-and-repair tools. An idea that gave birth to VTOrc… Both VTOrc and Orchestrator are tools for managing MySQL instances. If I were to describe these tools using a metaphor, I would say that they are kinda like the monitor of a class of students. They are responsible for keeping the MySQL instances in check and fixing them up in case they misbehave, just like how a monitor ensures that no mischief happens in the classroom.https://vitess.io/blog/2022-09-12-vitess-group-replication/Mon, 12 Sep 2022 00:00:00 +0000https://vitess.io/blog/2022-09-12-vitess-group-replication/VitessMySQLGroup ReplicationUpdate June 12, 2023: This component has been deprecated in v17 and will be removed in v18! We recommend that you instead use VTOrc with the semi_sync durability policy. Introduction # MySQL group replication is a new replication mechanism that was released in 2016. Group replication involves establishing a group of nodes that are coordinated automatically via Group Communication System (GCS) protocols, an implementation of Paxos. For a transaction to commit, a majority of the group has to agree on the order of a given transaction in the global sequence of transactions.https://vitess.io/blog/2022-06-28-announcing-vitess-14/Tue, 28 Jun 2022 00:00:00 +0000https://vitess.io/blog/2022-06-28-announcing-vitess-14/releaseVitessMySQLkubernetesoperatorcloudGKEshardingWe are pleased to announce the general availability of Vitess 14. Major Themes # In this new release, major improvements have been made in several areas of Vitess, including usability and reliability. Online DDL is now GA. Gen4 planner is the new default planner. VTAdmin and VTOrc are officially in beta with Vitess 14. Usability # Command-Line Syntax Deprecation # This release marks the beginning of Vitess standardizing its command-line and flags syntax.https://vitess.io/blog/2022-06-10-kubecon-eu-recap/Fri, 10 Jun 2022 00:00:00 +0000https://vitess.io/blog/2022-06-10-kubecon-eu-recap/VitessMySQLKubeconCloudNativeConconferenceseventsKubecon+CloudNativeCon EU 2022 # When: May 16-20 Who: Deepthi Sigireddi, Harshit Gangal, Manan Gupta What: Vitess project booth, maintainer talk, office hours Event Details # May 16-17 were co-located events. We were there for the main conference which was May 18-20. There were 7000+ attendees. The Vitess team were attending in their capacity as project maintainers to staff a booth in the Project Pavilion, hold office hours and to give a talk.https://vitess.io/blog/2022-06-01-lfx-experience/Wed, 01 Jun 2022 00:00:00 +0000https://vitess.io/blog/2022-06-01-lfx-experience/VitessCNCFLFXparserinternshipgolangsqlyaccMy name is Kushal Kumar and I was selected as a LFX mentee for the spring term 2022 under the organization CNCF:Vitess. It was a great learning experience. The 3 month journey was an exciting one where I got to learn about parsers, queries in sql and golang. How did I Land up at Vitess # I had heard about LFX mentorship program from one of my friends and applied last year in the fall term.https://vitess.io/blog/2022-04-06-online-ddl-vitess-cut-over/Wed, 06 Apr 2022 00:00:00 +0000https://vitess.io/blog/2022-04-06-online-ddl-vitess-cut-over/VitessMySQLDDLschema migrationsoperationsVitess supports managed, non-blocking schema migrations based on VReplication, aptly named vitess migrations. Vitess migrations are powerful, revertible, and failure agnostic. They take an asynchronous approach, which is more lightweight on the database server. The asynchronous approach comes with an implementation challenge: how to cut-over with minimal impact to the user/app and risk free of data loss. In this post we take a deep dive into the cut-over logic used in vitess migrations.https://vitess.io/blog/2022-02-22-announcing-vitess-13/Tue, 22 Feb 2022 00:00:00 +0000https://vitess.io/blog/2022-02-22-announcing-vitess-13/releaseVitessMySQLkubernetesoperatorcloudGKEshardingThe Vitess maintainers are pleased to announce the general availability of Vitess 13. Major Themes # In this release, Vitess maintainers have made significant progress in several areas, including query serving and cluster management. Compatibility # This release comes with major compatibility improvements. We added support for a large number of character sets and improved our evaluation engine to perform more evaluations at the VTGate level. Gen4 planner is no longer experimental and we have used it to add support for a number of previously unsupported complex queries.https://vitess.io/blog/2022-01-11-schema-tracking/Tue, 11 Jan 2022 00:00:00 +0000https://vitess.io/blog/2022-01-11-schema-tracking/VitessCNCFSchemaTrackingPlannerQueryServingMySQLWhat is Schema Tracking? # In a distributed relational database system, like Vitess, a central component is responsible for serving queries across multiple shards. For Vitess, it is VTGate. One of the challenges this component faces is being aware of the underlying SQL schema being used. This awareness facilitates query planning. Table schemas are stored in MySQL’s information_schema, meaning that they are located in a VTTablet’s MySQL instance and not in VTGate.https://vitess.io/blog/2021-12-17-log4j-cve-patches/Fri, 17 Dec 2021 00:00:00 +0000https://vitess.io/blog/2021-12-17-log4j-cve-patches/VitessMySQLCNCFlog4jlog4shelljavacvepatchreleaseBackground # A critical vulnerability CVE-2021-44228 in the Apache Log4j logging library was disclosed on Dec 9. The project provided release 2.15.0 with a patch that mitigates the impact of this CVE. It was quickly found that the initial patch was insufficient, and an additional CVE CVE-2021-45046 followed. This has been fixed in release 2.16.0. Who is affected? # The bulk of vitess code is in golang, and is unaffected by these vulnerabilities.https://vitess.io/blog/2021-12-16-rails-that-scales/Thu, 16 Dec 2021 00:00:00 +0000https://vitess.io/blog/2021-12-16-rails-that-scales/VitessMySQLCNCFRailsscalingwebframeworkSQLPast - Frameworks without scale # Over the past couple of decades, there has been a steady rise in the complexity of the development stacks that the developers across the globe have been using. The web has advanced from being just HTML files, to also include CSS and JavaScript with their own multitudes of frameworks like Redwood, Next.js, and Angular, among many others. The number of library dependencies that each project has has also shot up, leading to package managers like npm gaining popularity.https://vitess.io/blog/2021-12-10-lfx-experience/Fri, 10 Dec 2021 00:00:00 +0000https://vitess.io/blog/2021-12-10-lfx-experience/VitessCNCFLFXparserinternshipI was a part of the LFX Fall Intern program working on Vitess for 2021 and it was a great learning experience for me. I got introduced to Kubernetes in my previous internship through which I came to know about CNCF and then about the LFX program (CNCF is part of the LinuxFoundation). Participating in LFX Intern program helped me develop my interest towards the internals of databases, query parsing and distributed systems, topics which I hope to explore in my final year.https://vitess.io/blog/2021-12-06-connecteam-story/Mon, 06 Dec 2021 00:00:00 +0000https://vitess.io/blog/2021-12-06-connecteam-story/VitessMySQLusecasecasestudystoryjourneyproductionshardingConnecteam is a SaaS company that provides an employee management solution for deskless teams. Over the last couple of years, we’ve been growing tremendously and we recently started to face one of the hardest technical challenges: horizontal scaling. In the era of cloud computing, provisioning new resources is a breeze, but handling those resources in an efficient manner and providing a five nines uptime while allowing a fast-paced development environment is not an easy undertaking.https://vitess.io/blog/2021-11-02-why-write-new-planner/Tue, 02 Nov 2021 00:00:00 +0000https://vitess.io/blog/2021-11-02-why-write-new-planner/VitessMySQLDDLqueryplanexamineexplainoptimizerQuery planning is hard # Have you ever wondered what goes on behind the scenes when you execute a SQL query? What steps are taken to access your data? In this article, I'll talk about the history of Vitess's V3 query planner, why we created a new query planner, and the development of the new Gen4 query planner. Vitess is a horizontally scalable database solution which means that a single table can be spread out across multiple database instances.https://vitess.io/blog/2021-10-26-announcing-vitess-12/Tue, 26 Oct 2021 00:00:00 +0000https://vitess.io/blog/2021-10-26-announcing-vitess-12/releaseVitessMySQLkubernetesoperatorcloudGKEawsshardingOn behalf of the Vitess maintainers, I am pleased to announce the general availability of Vitess 12. Major Themes # In this release, Vitess Maintainers have made significant progress in several areas, including Gen4 planner, VTAdmin, and other improvements. Please take a moment to review the Release Notes. Please read them carefully and report any issues via GitHub. Gen4 Planner # The newest version of the query planner, Gen4, becomes an experimental feature as part of this release.https://vitess.io/blog/2021-09-07-examine-query-plan/Tue, 07 Sep 2021 00:00:00 +0000https://vitess.io/blog/2021-09-07-examine-query-plan/VitessMySQLDDLqueryplanexamineexplainoptimizerOriginally posted at Andres's blog. Traditional query optimizing is mostly about two things: first, in which order and from where to access data, and then how to then combine it. You have probably seen the tree shapes execution plans that are produced from query planning. I’ll use an example from the MySQL docs, using FORMAT=TREE which was introduced in MySQL 8.0: mysql>EXPLAINFORMAT=TREE->SELECT*->FROMt1->JOINt2->ON(t1.c1=t2.c1ANDt1.c2<t2.c2)->JOINt3->ON(t2.c1=t3.c1)\G***************************1.row***************************EXPLAIN:->Innerhashjoin(t3.c1=t1.c1)(cost=1.05rows=1)->Tablescanont3(cost=0.35rows=1)->Hash->Filter:(t1.c2<t2.c2)(cost=0.70rows=1)->Innerhashjoin(t2.c1=t1.c1)(cost=0.70rows=1)->Tablescanont2(cost=0.35rows=1)->Hash->Tablescanont1(cost=0.35rows=1)Here we can see that the MySQL optimizer thinks the best plan is to start reading from t1 using a table scan.https://vitess.io/blog/2021-07-27-announcing-vitess-11/Tue, 27 Jul 2021 00:00:00 +0000https://vitess.io/blog/2021-07-27-announcing-vitess-11/releaseVitessMySQLkubernetesoperatorcloudGKEshardingOn behalf of the Vitess maintainers, I am pleased to announce the general availability of Vitess 11. Major Themes # In this release, Vitess Maintainers have made significant progress in several areas, including Benchmarking, VTAdmin, Schema Tracking, Online DDL, and Performance improvements. While Schema Tracking is experimental, we’re very excited to have Gen4 planner evolving as well. Please take a moment to review the Release Notes. Please read them carefully and report any issues via GitHub.https://vitess.io/blog/2021-07-08-announcing-vitess-arewefastyet/Thu, 08 Jul 2021 00:00:00 +0000https://vitess.io/blog/2021-07-08-announcing-vitess-arewefastyet/VitessMySQLbenchmarkarewefastyetBenchmarking is a critical technique for delivering high performance software. The basic idea behind benchmarking is measuring and comparing the performance of one software version against another. Over the years, many benchmarking techniques have emerged, but we can broadly separate them into two categories: micro and macro benchmarks. Micro-benchmarks measure a small part of the codebase, usually by isolating a single function call and calling it repeatedly, whereas macro-benchmarks measure the performance of the codebase as a whole and run in an environment similar to what end-users experience.https://vitess.io/blog/2021-06-15-online-ddl-why-no-fk/Tue, 15 Jun 2021 00:00:00 +0000https://vitess.io/blog/2021-06-15-online-ddl-why-no-fk/VitessMySQLDDLschema migrationsoperationsforeign keysThis post explains the inherent problem of running online schema changes in MySQL, on tables participating in a foreign key relationship. We'll lay some ground rules and facts, sketch a simplified schema, and dive into an online schema change operation. Our discussion applies to gh-ost, pt-online-schema-change, and VReplication based migrations, or any other online schema change tool that works with a shadow/ghost table like the Facebook tools. Why Online DDL? # Online schema change tools come as workarounds to an old problem: schema migrations in MySQL were blocking, uninterruptible, aggressive in resources, replication unfriendly.https://vitess.io/blog/2021-06-03-a-new-protobuf-generator-for-go/Thu, 03 Jun 2021 09:07:21 -0800https://vitess.io/blog/2021-06-03-a-new-protobuf-generator-for-go/VitessMySQLcode generationGoGolangProtocol BuffersProtoBufGRPCAlthough the main interface between applications and a Vitess database is through the MySQL protocol, Vitess is a large and complex distributed system, and all the communication between the different services in a Vitess cluster is performed through GRPC. Because of this, all service boundaries and messages between Vitess' systems are specified using Protocol Buffers. The history of Vitess' integration with Protocol Buffers is rather involved: We have been using and keeping up to date with the Go Protocol Buffers package since its earliest releases, up until May last year, when Google released a new Go API for Protocol Buffers, which is not backwards compatible with the previous Go package.https://vitess.io/blog/2021-04-27-announcing-vitess-10/Tue, 27 Apr 2021 00:00:00 +0000https://vitess.io/blog/2021-04-27-announcing-vitess-10/releaseVitessMySQLkubernetesoperatorcloudGKEshardingOn behalf of the Vitess maintainers, I am pleased to announce the general availability of Vitess 10. Major Themes # In this release, Vitess Maintainers have continued to focus on compatibility. It is still the most critical component of Vitess being part of the MySQL ecosystem. We have also started working on benchmarking and performance optimizations. These improvements have given us a clear vision of which areas of Vitess can be improved in terms of performance.https://vitess.io/blog/2021-03-24-code-generation-vitess/Wed, 24 Mar 2021 00:00:00 +0000https://vitess.io/blog/2021-03-24-code-generation-vitess/VitessMySQLDDLcode generationGoGolangcloudkubernetesCross posting link Golang is a wonderful language. It's simple, and most of the time not confusing or surprising. This makes it easy to jump into library code and start reading and quickly understand what's going on. On the other hand, coming from other languages, there are a few features that would make our lives easier. We are building Vitess using mostly golang, and most of us are happy with this choice.https://vitess.io/blog/2021-02-19-online-ddl-in-vitess/Fri, 19 Feb 2021 00:00:00 +0000https://vitess.io/blog/2021-02-19-online-ddl-in-vitess/VitessMySQLDDLschema migrationsoperationsgh-ostpercona toolkitpt-online-schema-changeVitess introduces a new way to run schema migrations: non-blocking, asynchronous, scheduled online DDL. With online DDL Vitess simplifies the schema migration process by taking ownership of the operational overhead, and providing the user a simple, familiar interface: the standard ALTER TABLE statement. Let’s first give some background and explain why schema migrations are such an issue in the databases world, and then dive into implementation details The relational model and the operational overhead # The relational model is one of the longest surviving models in the software world, introduced decades ago and widely used until today.https://vitess.io/blog/2020-01-26-announcing-vitess-9/Tue, 26 Jan 2021 00:00:00 +0000https://vitess.io/blog/2020-01-26-announcing-vitess-9/releaseVitessMySQLkubernetesoperatorcloudGKEshardingOn behalf of the Vitess maintainers team, I am pleased to announce the general availability of Vitess 9. Major Themes # In this release, we have focused on making Vitess more stable after the successful release of Version 8. There have been no major issues reported. So there were no patches released for Version 8. This has allowed us to push further on compatibility and adoption of common frameworks as priorities.https://vitess.io/blog/2020-12-15-vitess-at-kubecon-na/Tue, 15 Dec 2020 00:00:00 +0000https://vitess.io/blog/2020-12-15-vitess-at-kubecon-na/VitessMySQLconferenceThe Vitess team had a successful presence at Kubecon + CloudNativeCon North America 2020. This year's event was in a virtual format. It ran from Nov 17-20. We had 2 talks, a project booth and 3 office hours sessions. In addition, we had mentions in 2 major keynote talks. Day 1 # Day 1 was a non-keynote day. It opened relatively quietly for us. We had 25+ attendees at our office hours session hosted by maintainers Alkin Tezuysal, Deepthi Sigireddi and Derek Perkins.https://vitess.io/blog/2020-11-30-how-to-deploy-django/Mon, 30 Nov 2020 00:00:00 +0000https://vitess.io/blog/2020-11-30-how-to-deploy-django/VitessMySQLkubernetesoperatorcloudGKEshardingDjango is a popular framework for Python application developers. It includes packages which make tasks like authorization and content administration easier. Django supports a number of databases including MySQL which makes it possible to run a Django application over Vitess without having to change the application code. Let’s take a look at how to combine the strengths of these two open source frameworks. We built this example using Vitess operator. You can see the details of the implementation in the blog post Vitess Operator for Kubernetes.https://vitess.io/blog/2020-11-09-vitess-operator-for-kubernetes/Mon, 09 Nov 2020 00:00:00 +0000https://vitess.io/blog/2020-11-09-vitess-operator-for-kubernetes/VitessMySQLkubernetesoperatorcloudGKEshardingIntroduction # In this blog, I would like to explore Vitess Operator for Kubernetes. This post demonstrates the sample implementation of Vitess in Kubernetes topology. I also explore common DBA tasks by demonstrating how they are handled in the Vitess ecosystem. Vitess, out of the box, comes with a lot of tools and utilities that one has to either incorporate or develop to manage MySQL topology. Let’s take a look at the capabilities of Vitess in these areas and demonstrate how they are performed under the operator realm.https://vitess.io/blog/2020-11-03-streaming-vitess-at-bolt/Tue, 03 Nov 2020 00:00:00 +0000https://vitess.io/blog/2020-11-03-streaming-vitess-at-bolt/VitessMySQLDebeziumKafkaCDCChange Data CaptureApachePreviously posted on link at Nov 3, 2020. Traditionally, MySQL has been used to power most of the backend services at Bolt. We've designed our schemas in a way that they're sharded into different MySQL clusters. Each MySQL cluster contains a subset of data and consists of one primary and multiple replication nodes. Once data is persisted to the database, we use the Debezium MySQL Connector to capture data change events and send them to Kafka.https://vitess.io/blog/2020-10-27-announcing-vitess-8/Tue, 27 Oct 2020 00:00:00 +0000https://vitess.io/blog/2020-10-27-announcing-vitess-8/releaseOn behalf of the Vitess maintainers team, I am pleased to announce the general availability of Vitess 8. Major Themes # In this release, we have continued to make important improvements to the Vitess project with over 200 PRs in several areas. Some of the major bug fixes and changes in behaviors are documented in the Release Notes. Please read them carefully and report any issues via GitHub. We would like to highlight the following themes for this release.https://vitess.io/blog/2020-07-28-announcing-vitess-7/Tue, 28 Jul 2020 00:00:00 +0000https://vitess.io/blog/2020-07-28-announcing-vitess-7/releaseOn behalf of the Vitess maintainers team, I am pleased to announce the general availability of Vitess 7. Major Themes # Improved SQL Support # We continued to progress towards (almost) full MySQL compatibility. The highlights in Vitess 7 are replica transactions, savepoint support, and ability to set system variables per session. We expect to continue down this path for Vitess 8. Stability # Vitess had significant technical debt because of functionality that has been added organically.https://vitess.io/blog/2020-05-26-vitess-cross-cell/Tue, 26 May 2020 11:20:00 +0000https://vitess.io/blog/2020-05-26-vitess-cross-cell/vitessvitess clustervitess cross-cellvitess getting startedThis post explains about how VTGate handles cross-cell operations and how to configure CellAlias for cross-cell read operations. If you are new to Vitess, it is recommended to read this blog post to get more familiar with the various components and their configuration in Vitess. To understand CellAlias, first let's get familiar with what a cell means in Vitess. A cell is a group of servers and network infrastructure collocated in an area, and isolated from failures in other cells.https://vitess.io/blog/2020-04-29-announcing-vitess-6/Wed, 29 Apr 2020 08:00:21 -0700https://vitess.io/blog/2020-04-29-announcing-vitess-6/GuidesI am excited to announce the general availability of Vitess 6, the second release to follow our new accelerated release schedule. While only 12 weeks have elapsed since the previous release, it feels like a few key investments have started to pay dividends all at once. To provide some personal highlights: Improved SQL Support # Vitess now understands much more of MySQL’s syntax. We have taken the approach of studying the queries issued by common applications and frameworks, and baking them right into the end-to-end test suite.https://vitess.io/blog/2020-04-27-life-of-a-cluster/Mon, 27 Apr 2020 20:30:00 +0000https://vitess.io/blog/2020-04-27-life-of-a-cluster/vitessvitess clustervitess troubleshootingvitess getting startedThis post goes into the details of what goes on behind the scenes when a cluster is brought up, for example using helm or the local installation guide. This can be used both as a learning tool and troubleshooting guide. We assume that you have downloaded and installed all the necessary binaries before proceeding further. Vitess is a very flexible system that is capable of running in a variety of environments.https://vitess.io/blog/2020-02-04-announcing-the-release-of-vitess-5.0/Tue, 04 Feb 2020 08:00:21 -0700https://vitess.io/blog/2020-02-04-announcing-the-release-of-vitess-5.0/GuidesOn behalf of the Vitess maintainers team, I am pleased to announce the immediate release of Vitess 5.0. Vitess 5.0 is largely a clean up release, where we have both deprecated and removed some minor features, and improved both the installation experience and health of the Vitess code-base. When upgrading from a previous release of Vitess, we recommend reading the Incompatible Changes highlighted in the release notes for both 5.0 as well as Beta 0 and Beta 1.https://vitess.io/blog/2020-01-14-vitess-graduation-retrospective/Tue, 14 Jan 2020 09:09:21 -0800https://vitess.io/blog/2020-01-14-vitess-graduation-retrospective/graduationvitess graduationLast November, Vitess became the eighth CNCF project to reach graduation, joining a host of amazing projects such as Kubernetes, Prometheus, Envoy, CoreDNS, containerd, Fluentd, and Jaeger. To contextualize this milestone, I picked some tidbits from the brain of Vitess co-creator, Sugu Sougoumarane, allowing him to share internal perspective about how we got here, the hurdles we faced, and where we’re headed. Incubation # Considering that it was only the February of 2018 when the CNCF agreed to host Vitess as an incubating project, Vitess has made efficient strides toward becoming the de-facto standard for horizontally scaling MySQL.https://vitess.io/blog/2019-11-05-vitess-4.0-has-been-released/Tue, 05 Nov 2019 07:50:21 -0700https://vitess.io/blog/2019-11-05-vitess-4.0-has-been-released/GuidesHead to the release notes for an overview of functionality added or changed, as well as important changes for those upgrading from earlier releases. We wanted to use this post as an opportunity to reflect on three aspects of Vitess’ development, as it has evolved from an internal project at YouTube to a CNCF project with almost 200 contributors. Improved SQL Query Support # Vitess 4.0 takes a big leap forward in improving the coverage of MySQL syntax that is supported.https://vitess.io/blog/2019-09-03-why-250gb-shards/Tue, 03 Sep 2019 09:07:21 -0700https://vitess.io/blog/2019-09-03-why-250gb-shards/GuidesVitess has an opinionated approach to database scalability. Some of those opinions have minimal controversy such as how durability should be provided via replication, but the one I find interesting is the 250GB per MySQL server recommendation. Is this a physical MySQL Limit? # In short: no. By “physical limit” I mean is there a file format restriction that says databases can not be greater than 250GB? The physical limit for InnoDB is 64TB per tablespace, and in the default configuration each table is its own tablespace.https://vitess.io/blog/2019-06-17-unsharded-vitess-benefits/Mon, 17 Jun 2019 09:07:21 -0800https://vitess.io/blog/2019-06-17-unsharded-vitess-benefits/GuidesUnshardedBenefitsFor many large companies seeking help with horizontal scaling, Vitess' value proposition is easily understood; running stateful workloads at astronomical scale is a hard problem that Vitess has boldly solved in the past. However, for businesses that aren't hitting the performance limitations of standard MySQL, it may seem difficult to justify placing seemingly complex middleware in your data path with no immediate reward. I'm here to show you why unsharded Vitess is not just a pre-optimization for future horizontal scaling - it provides many upgrades to the MySQL experience.https://vitess.io/blog/2019-04-29-mysql-community-contributor-award/Mon, 29 Apr 2019 07:15:00 -0700https://vitess.io/blog/2019-04-29-mysql-community-contributor-award/mysql, oracle, vitessBuilding the Vitess community has been our pride and joy. Being able to contribute to the MySQL community, even more so. Vitess' Sugu Sougoumarane has been nominated by the MySQL group for Oracle's 2019 MySQL Community Contributor Award Program, where he joins folks like Shlomi Noach, Peter Zaitsev, Gabriela D'Ávila Ferrara, Giuseppe Maxia and many other active MySQL community members recognised for their contributions to MySQL. Criteria for the nominations included: most active code contributor, bug report,most active MySQL blogger, people who play an active role in translating or documenting MySQL articles, people who provide feedback on DMR releases, Labs release, or change proposal, as well as anyone in the community who did really useful work that ought to be thanked publicly.https://vitess.io/blog/2019-04-01-vitess-at-gosv/Mon, 01 Apr 2019 07:15:00 -0700https://vitess.io/blog/2019-04-01-vitess-at-gosv/eventsWhen Vitess started at YouTube back in 2010, it was designed to work in Google’s Borg. As a result of that interesting bit of history (and the Google origin story), Vitess wound up being one of the earliest adopters of (a) Go and (b) Kubernetes. At last week’s GoSV meetup (organized by the indefatigable Aarti and Conrad; join their meetup to get the latest updates on other Go events in Silicon Valley), Vitess co-creator Sugu Sougoumarane shared background stories about Vitess’ origin story, why Go was selected, and how the teams working on Vitess and Go overlapped during their time at Google.https://vitess.io/blog/2019-03-12-vitess-security-audit/Tue, 12 Mar 2019 07:15:00 -0700https://vitess.io/blog/2019-03-12-vitess-security-audit/SecurityWe are happy to announce that Vitess recently underwent a security assessment, funded by CNCF / The Linux Foundation. In February 2019, the team from Cure53 performed tests in the following areas: system complexity cloud infrastructure source code auditing operating system interaction low-level protocol analysis multi-angled penetration testing This independent security audit was performed on locally installed systems, as well as with a Kubernetes-based cluster. The auditors performed: (1) manual code auditing and (2) code-assisted penetration testing.https://vitess.io/blog/2019-02-07-choosing-a-vindex/Thu, 07 Feb 2019 09:07:21 -0800https://vitess.io/blog/2019-02-07-choosing-a-vindex/GuidesVindexesDocumentationWhen we talk to Vitess, it appears as if we are talking to a single unit, so what sets up this seemingly magical feature? This is achieved through a VSchema, the beautiful mind that sends all of your queries to the correct shard. When Vitess shards your database, it assigns each row an identifier called a keyspace ID. We then break up the domain of all keyspace ID values into ranges and assign ranges to each shard.https://vitess.io/blog/2018-12-10-introducing-vitess-3.0/Mon, 10 Dec 2018 09:07:21 -0800https://vitess.io/blog/2018-12-10-introducing-vitess-3.0/Major Versions, AnnouncementsWe are pleased to announce Vitess 3.0, a major upgrade from Vitess 2.2. Every major release of Vitess brings new features and improvements for our community, and also sometimes introduces new ways of doing things. Vitess 3.0 is built using “pure” go 1.11 (we do not need cgo now) and supports MySQL 8.0 and MariaDB 10.3. The key features of Vitess 3.0 are: Usability Simplified db parameters for vttablet Formal support for externally managed mysql More tutorials including how to run Vitess with Minikube Prometheus monitoring Performance improvements Snappier InitShardMaster and PlannedReparentShard Improved coordination with Orchestrator vtbench for benchmarking MySQL protocol performance improvements Faster reparents 4X faster parallelized backups Many more performance improvements VReplication Improvements to resharding We first showcased VReplication at the first ever Vitess meetup held at Slack in July 2018.https://vitess.io/blog/2018-11-12-vitess-weekly-digest/Mon, 12 Nov 2018 07:15:00 -0700https://vitess.io/blog/2018-11-12-vitess-weekly-digest/Weekly DigestWe continue the digest from the Slack discussions for Sep 22 2018 to Oct 5 2018. We've fallen slightly behind on this, but will catch up again soon. Enable VtGateExecute in vtctld # Arsalan [Sep 22nd] Hi, I want to query vitess on vtgate but I have below error. How can i fix this problem? vitess@vtctldv3-hrl74:/$ vtctlclient -server 10.5.61.20:16999 VtGateExecute -server 10.5.61.21:16991 "show tables" E0923 05:14:55.169771 1102 main.go:61] Remote error: rpc error: code = Unknown desc = query commands are disabled (set the -enable_queries flag to enable)https://vitess.io/blog/2018-09-24-vitess-weekly-digest/Mon, 24 Sep 2018 07:15:00 -0700https://vitess.io/blog/2018-09-24-vitess-weekly-digest/Weekly DigestThis week, we continue the digest from the Slack discussions for Sep 1 2018 to Sep 21 2018. As of this post, we are fully caught up on our backlog. Tracking progress of resharding # jk [Sep 5th] In the SplitClone Stage of resharding, how can i get the percent of process ? Or Could I estimate the time left ? sougou I don't know of a formal way, but you can look at the database size.https://vitess.io/blog/2018-09-10-vitess-weekly-digest/Mon, 10 Sep 2018 07:15:00 -0700https://vitess.io/blog/2018-09-10-vitess-weekly-digest/Weekly DigestThis week, we continue the digest from the Slack discussions for Aug 3 2018 to Aug 31 2018. Secondary Vindexes # raj.veerappan [Aug 3 9:27 AM] how do secondary vindexes work? would they result in further sharding? weitzman [9:32 AM] If you have an authors table and a books table and you shard by author ID, a secondary vindex is a performance tool to help respond to the query, select * from books where book_id = :idhttps://vitess.io/blog/2018-08-24-vitess-weekly-digest/Sun, 05 Aug 2018 07:15:00 -0700https://vitess.io/blog/2018-08-24-vitess-weekly-digest/Weekly DigestThis week, we continue the digest from the Slack discussions. Update stream # Jian [Jul 25th at 1:27 PM] hi there, I'm new to Vitess, now I'm following the user-guide from vitess.io to explore vitess, in update stream section, I notice they have change log, where could I see these change logs so I can have a better understanding of the update stream? sougou That's the only documentation we have about the update stream, but we'll be fixing docs for all vitess very soon.https://vitess.io/blog/2018-08-05-vitess-weekly-digest/Sun, 05 Aug 2018 07:15:00 -0700https://vitess.io/blog/2018-08-05-vitess-weekly-digest/Weekly DigestThis week, we kick off our new weekly blog updates — bringing you the best of Vitess questions and topics on our Slack discussions. The goal is to show the most interesting topics and requests so those of you just getting started can see highlights of what has been covered. Since this is our first ever digest, we’re going to go back in time and publish a little more than what happened last week.https://vitess.io/blog/2017-09-18-custom-sharding-with-vitess/Mon, 18 Sep 2017 07:15:00 -0700https://vitess.io/blog/2017-09-18-custom-sharding-with-vitess/Vitess supports a variety of predefined sharding algorithms that can suit different needs. This is achieved by associating a Vindex with your main sharding column. A Vindex essentially provides a mapping function that converts your column value to a keyspace_id. This keyspace_id is then used to decide the target shard. A full description of VSchema and Vindexes can be found here. However, such predefined vindexes will work only if you intend to shard your system using Vitess.https://vitess.io/blog/2017-04-25-vitess-releases-version-2-1/Tue, 25 Apr 2017 15:59:00 -0700https://vitess.io/blog/2017-04-25-vitess-releases-version-2-1/The Vitess project is proud to announce the release of version 2.1. This version comes packed with new features that improve usability, availability and resilience of the overall system. The release coincides with the Percona Live 2017 Conference, where project co-founder Sugu Sougoumarane will give the talk "Vitess beyond YouTube". He is joined by Robert Navarro from Stitch Labs who is going to describe how Stitch Labs uses Vitess in production.https://vitess.io/blog/2016-06-07-distributed-transactions-in-vitess/Tue, 07 Jun 2016 17:05:00 -0700https://vitess.io/blog/2016-06-07-distributed-transactions-in-vitess/distributed transactionssharding2PCWith Vitess introducing sharding and allowing you to create cross-shard indexes, distributed transactions become unavoidable for certain workloads. Currently, Vitess only supports best-effort distributed transactions. So, it’s possible that a distributed commit only completes partially, leaving data in an inconsistent state. At this point, 2 Phase Commit (2PC) is the only known protocol that allows you to give atomic guarantees for distributed transactions. For this protocol to work, a database must be able to support the ‘Prepare’ contract.https://vitess.io/blog/2016-04-26-vitess-v2-now-with-more-v3/Tue, 26 Apr 2016 14:05:00 -0700https://vitess.io/blog/2016-04-26-vitess-v2-now-with-more-v3/conferencereleaseshardingStarting with Vitess v2.0.0-beta.2, the VTGate V3 API can route complex single-shard queries (containing joins, subqueries, aggregation, sorting, and any combination thereof) as well as perform cross-shard joins. That means you no longer need to tell VTGate the keyspace ID that a query targets, as you did with the VTGate V2 API. The fact that keyspace IDs are now hidden from the application has enabled drop-in Vitess libraries for standard database interfaces like JDBC (written by Flipkart), PDO (written by Pixel Federation), PEP 249, and database/sql.https://vitess.io/blog/2016-03-10-percona-live-featured-talk-with-sugu-sougoumarane-vitess-the-complete-story/Thu, 10 Mar 2016 11:13:00 -0800https://vitess.io/blog/2016-03-10-percona-live-featured-talk-with-sugu-sougoumarane-vitess-the-complete-story/conferenceCross-posted from Percona Blog. Welcome to the next installment of our talks with Percona Live Data Performance Conference 2016 speakers! In this series of blogs, we’ll highlight some of the speakers that will be at this year’s conference, as well as discuss the technologies and outlooks of the speakers themselves. Make sure to read to the end to get a special Percona Live registration bonus! In this installment, our Percona Live featured talk with Sugu Sougoumarane, Infrastructure & Storage Engineer at YouTube is about Vitess: The Complete Story.https://vitess.io/blog/2016-03-03-vitess-2-0-is-now-beta/Thu, 03 Mar 2016 14:07:00 -0800https://vitess.io/blog/2016-03-03-vitess-2-0-is-now-beta/releaseThat means we've accomplished all our planned overhauls of client APIs and backward-incompatible protocol changes. See the release notes for what's new. We're now working closely with several users who are evaluating Vitess and providing feedback on the use cases that are important for their particular applications and production environments. If you're at the same stage, we welcome you to join the discussion by posting on the mailing list. We're also trying out Slack for more conversational topics.https://vitess.io/blog/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes/Tue, 06 Oct 2015 13:22:00 -0700https://vitess.io/blog/2015-10-06-cloud-native-mysql-sharding-with-vitess-and-kubernetes/kubernetescloudshardingCross-posted on Google Cloud Platform Blog. Cloud native technologies like Kubernetes help you compose scalable services out of a sea of small logical units. In our last post, we introduced Vitess(an open-source project that powers YouTube's main database) as a way of turning MySQL into a scalable Kubernetes application. Our goal was to make scaling your persistent datastore in Kubernetes as simple as scaling stateless app servers - just run a single command to launch more pods.https://vitess.io/blog/2015-03-20-scaling-mysql-in-the-cloud-with-vitess-and-kubernetes/Fri, 20 Mar 2015 13:53:00 -0700https://vitess.io/blog/2015-03-20-scaling-mysql-in-the-cloud-with-vitess-and-kubernetes/kubernetescloudCross-posted on Google Cloud Platform Blog. Your new website is growing exponentially. After a few rounds of high fives, you start scaling to meet this unexpected demand. While you can always add more front-end servers, eventually your database becomes a bottleneck, which leads you to... Add more replicas for better read throughput and data durability Introduce sharding to scale your write throughput and let your data set grow beyond a single machine Create separate replica pools for batch jobs and backups, to isolate them from live traffic Clone the whole deployment into multiple datacenters worldwide for disaster recovery and lower latency At YouTube, we went on thatjourney as we scaled our MySQL deployment, which today handles the metadata for billions of daily video views and 300 hours of new video uploads per minute.