vtgate

VTGate is a stateless proxy responsible for accepting requests from applications and routing them to the appropriate tablet server(s) for query execution. It speaks both the MySQL Protocol and a gRPC protocol.

Example Usage #

Start a vtgate proxy:

export TOPOLOGY_FLAGS="-topo_implementation etcd2 -topo_global_server_address localhost:2379 -topo_global_root /vitess/global"
export VTDATAROOT="/tmp"

vtgate \
  $TOPOLOGY_FLAGS \
  -log_dir $VTDATAROOT/tmp \
  -port 15001 \
  -grpc_port 15991 \
  -mysql_server_port 15306 \
  -cell test \
  -cells_to_watch test \
  -tablet_types_to_wait PRIMARY,REPLICA \
  -gateway_implementation tabletgateway \
  -service_map 'grpc-vtgateservice' \
  -pid_file $VTDATAROOT/tmp/vtgate.pid \
  -mysql_auth_server_impl none

Options #

The following global options apply to vtgate:

NameTypeDefinition
-allowed_tablet_typesvalueSpecifies the tablet types this vtgate is allowed to route queries to
-alsologtostderrbooleanLog to standard error as well as files
-buffer_drain_concurrencyintMaximum number of requests retried simultaneously. More concurrency will increase the load on the PRIMARY vttablet when draining the buffer (default 1)
-buffer_implementationstringThe algorithm used for managing request buffering during cluster availability events (allowed values: "keyspace_events" (default), "healthcheck" (legacy value))
-buffer_keyspace_shardsstringIf not empty, limit buffering to these entries (comma separated). Entry format: keyspace or keyspace/shard. Requires --enable_buffer=true
-buffer_max_failover_durationdurationStop buffering completely if a failover takes longer than this duration (default 20s)
-buffer_min_time_between_failoversdurationMinimum time between the end of a failover and the start of the next one (tracked per shard). Faster consecutive failovers will not trigger buffering (default 1m0s)
-buffer_sizeintMaximum number of buffered requests in flight (across all ongoing failovers) (default 1000)
-buffer_windowdurationDuration for at most how long a request should be buffered (default 10s)
-catch-sigpipebooleanCatch and ignore SIGPIPE on stdout and stderr if specified
-cellstringCell to use (default "test_nj")
-cells_to_watchstringComma-separated list of cells for watching tablets
-consul_auth_static_filestringJSON File to read the topos/tokens from
-cpu_profilestringDeprecated: use '-pprof=cpu' instead
-datadog-agent-hoststringHost to send spans to; if empty, no tracing will be done
-datadog-agent-portstringPort to send spans to; if empty, no tracing will be done
-dbddl_pluginstringControls how to handle CREATE/DROP DATABASE. Add it if you are using your own database provisioning service (default "fail")
-ddl_strategystringSet default strategy for DDL statements. Override with @@ddl_strategy session variable (default "direct")
-default_tablet_typevalueThe default tablet type to set for queries, when one is not explicitly selected (default PRIMARY)
-disable_local_gatewaybooleanDeprecated: if specified, this process will not route any queries to local tablets in the local cell
-discovery_high_replication_lag_minimum_servingdurationThe replication lag considered too high when applying the min_number_serving_vttablets threshold (default 2h0m0s)
-discovery_low_replication_lagdurationThe replication lag considered low enough to be healthy (default 30s)
-emit_statsbooleanIf set, emit stats to push-based monitoring and stats backends
-enable_bufferbooleanEnable buffering (stalling) of primary traffic during failovers
-enable_buffer_dry_runbooleanDetect and log failover events, but do not actually buffer requests
-enable_direct_ddlbooleanAllow users to submit direct DDL statements (default true)
-enable_online_ddlbooleanAllow users to submit, review and control Online DDL (default true)
-enable_system_settingsbooleanThis will enable the system settings to be changed per session at the database connection level (default true)
-foreign_key_modestringThis allows users to provide a method to handle a foreign key constraint in create/alter table. Valid values are: allow, disallow (default "allow")
-gate_query_cache_lfubooleanGate server cache algorithm. When set to true, a new cache algorithm based on a TinyLFU admission policy will be used to improve cache behavior and prevent pollution from sparse queries (default true)
-gate_query_cache_memoryintGate server query cache size in bytes, maximum amount of memory to be cached. vtgate analyzes every incoming query and generate a query plan, these plans are being cached in a lru cache. This config controls the capacity of the lru cache (default 33554432)
-gate_query_cache_sizeintGate server query cache size, maximum number of queries to be cached. vtgate analyzes every incoming query and generate a query plan, these plans are being cached in a cache. This config controls the expected amount of unique entries in the cache (default 5000)
-gateway_implementationstringDeprecated: Only tabletgateway is now supported, discoverygateway is no longer available
-gateway_initial_tablet_timeoutdurationAt startup, the gateway will wait up to that duration to get one tablet per keyspace/shard/tablettype (default 30s)
-grpc_auth_modestringWhich auth plugin implementation to use (eg: static)
-grpc_auth_mtls_allowed_substringsstringList of substrings of at least one of the client certificate names (separated by colon)
-grpc_auth_static_client_credsstringWhen using grpc_static_auth in the server, this file provides the credentials to use to authenticate with the server
-grpc_auth_static_password_filestringJSON File to read the users/passwords from
-grpc_castringServer CA to use for gRPC connections, requires TLS, and enforces client certificate check
-grpc_certstringServer certificate to use for gRPC connections, requires grpc_key, enables TLS
-grpc_compressionstringWhich protocol to use for compressing gRPC. (Default: nothing) Supported: snappy
-grpc_crlstringPath to a certificate revocation list in PEM format, client certificates will be further verified against this file during TLS handshake
-grpc_enable_optional_tlsbooleanEnable optional TLS mode when a server accepts both TLS and plain-text connections on the same port
-grpc_enable_tracingbooleanEnable GRPC tracing
-grpc_initial_conn_window_sizeintgRPC initial connection window size
-grpc_initial_window_sizeintgRPC initial window size
-grpc_keepalive_timedurationAfter a duration of this time, if the client doesn't see any activity, it pings the server to see if the transport is still alive (default 10s)
-grpc_keepalive_timeoutdurationAfter having pinged for keepalive check, the client waits for a duration of Timeout and if no activity is seen even after that the connection is closed. (default 10s)
-grpc_keystringServer private key to use for gRPC connections, requires grpc_cert, enables TLS
-grpc_max_connection_agedurationMaximum age of a client connection before GoAway is sent (default 2562047h47m16.854775807s)
-grpc_max_connection_age_gracedurationAdditional grace period after grpc_max_connection_age, after which connections are forcibly closed (default 2562047h47m16.854775807s)
-grpc_max_message_sizeintMaximum allowed RPC message size. Larger messages will be rejected by gRPC with the error 'exceeding the max size' (default 16777216)
-grpc_portintPort to listen on for gRPC calls
-grpc_prometheusbooleanEnable gRPC monitoring with Prometheus
-grpc_server_castringPath to server CA in PEM format, which will be combine with server cert, return full certificate chain to clients
-grpc_server_initial_conn_window_sizeintgRPC server initial connection window size
-grpc_server_initial_window_sizeintgRPC server initial window size
-grpc_server_keepalive_enforcement_policy_min_timedurationgRPC server minimum keepalive time (default 10s)
-grpc_server_keepalive_enforcement_policy_permit_without_streambooleangRPC server permit client keepalive pings even when there are no active streams (RPCs)
-grpc_use_effective_calleridbooleanIf set, and SSL is not used, will set the immediate caller id from the effective caller id's principal
-healthcheck_retry_delaydurationHealth check retry delay (default 2ms)
-healthcheck_timeoutdurationHealth check timeout period (default 1m0s)
-jaeger-agent-hoststringHost and port to send spans to; if empty, no tracing will be done
-keep_logsdurationKeep logs for this long (using ctime) (zero to keep forever)
-keep_logs_by_mtimedurationKeep logs for this long (using mtime) (zero to keep forever)
-keyspaces_to_watchvalueSpecifies which keyspaces this vtgate should have access to while routing queries or accessing the vschema
-lameduck-perioddurationKeep running at least this long after SIGTERM before stopping (default 50ms)
-legacy_replication_lag_algorithmbooleanUse the legacy algorithm when selecting the vttablets for serving (default true)
-lock_heartbeat_timedurationIf there is lock function used, this will keep the lock connection active by using this heartbeat (default 5s)
-log_backtrace_atvalueWhen logging hits line file:N, emit a stack trace
-log_dirstringIf non-empty, write log files in this directory
-log_err_stacksbooleanLog stack traces for errors
-log_queries_to_filestringEnable query logging to the specified file
-log_rotate_max_sizeuintSize in bytes at which logs are rotated (glog.MaxSize) (default 1887436800)
-logtostderrbooleanLog to standard error instead of files
-max_memory_rowsintMaximum number of rows that will be held in memory for intermediate results, as well as the final result (default 300000)
-max_payload_sizeintThe threshold for query payloads in bytes. A payload greater than this threshold will result in a failure to handle the query
-mem-profile-rateintDeprecated: use '-pprof=mem' instead (default 524288)
-message_stream_grace_perioddurationThe amount of time for a vttablet to resume if it ends a message stream, usually because of a reparent (default 30s)
-min_number_serving_vttabletsintThe minimum number of vttablets for each replicating tablet_type (e.g. replica, rdonly) that will be continue to be used, even with replication lag above discovery_low_replication_lag, but still below discovery_high_replication_lag_minimum_serving (default 2)
-mutex-profile-fractionintDeprecated: use '-pprof=mutex' instead
-mysql_allow_clear_text_without_tlsbooleanIf set, the server will allow the use of a clear text password over non-SSL connections
-mysql_auth_server_implstringWhich auth server implementation to use. Options: none, ldap, clientcert, static, vault (default "static")
-mysql_auth_server_static_filestringJSON File to read the users/passwords from
-mysql_auth_server_static_stringstringJSON representation of the users/passwords config
-mysql_auth_static_reload_intervaldurationTicker to reload credentials
-mysql_auth_vault_addrstringURL to Vault server
-mysql_auth_vault_pathstringVault path to vtgate credentials JSON blob, e.g.: secret/data/prod/vtgatecreds
-mysql_auth_vault_role_mountpointstringVault AppRole mountpoint; can also be passed using VAULT_MOUNTPOINT environment variable (default "approle")
-mysql_auth_vault_role_secretidfilestringPath to file containing Vault AppRole secret_id; can also be passed using VAULT_SECRETID environment variable
-mysql_auth_vault_roleidstringVault AppRole id; can also be passed using VAULT_ROLEID environment variable
-mysql_auth_vault_timeoutdurationTimeout for vault API operations (default 10s)
-mysql_auth_vault_tls_castringPath to CA PEM for validating Vault server certificate
-mysql_auth_vault_tokenfilestringPath to file containing Vault auth token; token can also be passed using VAULT_TOKEN environment variable
-mysql_auth_vault_ttldurationHow long to cache vtgate credentials from the Vault server (default 30m0s)
-mysql_clientcert_auth_methodstringClient-side authentication method to use. Supported values: mysql_clear_password, dialog (default "mysql_clear_password")
-mysql_default_workloadstringDefault session workload (OLTP, OLAP, DBA) (default "OLTP")
-mysql_ldap_auth_config_filestringJSON File from which to read LDAP server config
-mysql_ldap_auth_config_stringstringJSON representation of LDAP server config
-mysql_ldap_auth_methodstringClient-side authentication method to use. Supported values: mysql_clear_password, dialog (default "mysql_clear_password")
-mysql_server_bind_addressstringBinds on this address when listening to MySQL binary protocol. Useful to restrict listening to 'localhost' only for instance
-mysql_server_flush_delaydurationDelay after which buffered response will be flushed to the client (default 100ms)
-mysql_server_portintIf set, also listen for MySQL binary protocol connections on this port (default -1)
-mysql_server_query_timeoutdurationMySQL query timeout
-mysql_server_read_timeoutdurationConnection read timeout
-mysql_server_require_secure_transportbooleanReject insecure connections, but only if mysql_server_ssl_cert and mysql_server_ssl_key are provided
-mysql_server_socket_pathstringThis option specifies the Unix socket file to use when listening for local connections. By default it will be empty and it won't listen to a unix socket
-mysql_server_ssl_castringPath to SSL CA for mysql server plugin SSL. If specified, server will require and validate client certs.
-mysql_server_ssl_certstringPath to the SSL cert for mysql server plugin SSL
-mysql_server_ssl_crlstringPath to SSL CRL for mysql server plugin SSL
-mysql_server_ssl_keystringPath to SSL key for mysql server plugin SSL
-mysql_server_ssl_server_castringPath to server CA in PEM format, which will be combine with server cert, and return full certificate chain to clients
-mysql_server_tls_min_versionstringConfigures the minimal TLS version negotiated when SSL is enabled. Options: TLSv1.0, TLSv1.1, TLSv1.2, TLSv1.3 (Defaults to TLSv1.2)
-mysql_server_versionstringMySQL server version to advertise.
-mysql_server_write_timeoutdurationConnection write timeout
-mysql_slow_connect_warn_thresholddurationWarn if it takes more than the given threshold for a MySQL connection to establish
-mysql_tcp_versionstringSelect tcp, tcp4, or tcp6 to control the socket type (default "tcp")
-no_scatterbooleanWhen set to true, the planner will fail instead of producing a plan that includes scatter queries
-normalize_queriesbooleanRewrite queries with bind vars. Turn this off if the app itself sends normalized queries with bind vars (default true)
-onclose_timeoutdurationWait no more than this time for OnClose handlers before stopping (default 1ns)
-onterm_timeoutdurationWait no more than this time for OnTermSync handlers before stopping (default 10s)
-opentsdb_uristringURI of opentsdb /api/put method
-pid_filestringIf set, the process will write its pid to the named file, and delete it on graceful shutdown
-planner_versionstringSets the default planner to use when the session has not changed it. Valid values are: V3, Gen4, Gen4Greedy and Gen4Fallback. Gen4Fallback tries the new gen4 planner and falls back to the V3 planner if the gen4 fails. All Gen4 versions should be considered experimental! (default "v3")
-portintPort for the server
-pprofstringEnable profiling
-proxy_protocolbooleanEnable HAProxy PROXY protocol on MySQL listener socket
-purge_logs_intervaldurationHow often try to remove old logs (default 1h0m0s)
-querylog-filter-tagstringString that must be present in the query for it to be logged; if using a value as the tag, you need to disable query normalization
-querylog-formatstringFormat for query logs ("text" or "json") (default "text")
-querylog-row-thresholduintNumber of rows a query has to return or affect before being logged; not useful for streaming queries. 0 means all queries will be logged
-redact-debug-ui-queriesbooleanRedact full queries and bind variables from debug UI
-remote_operation_timeoutdurationTime to wait for a remote operation (default 30s)
-retry-countintRetry count (default 2)
-schema_change_signalbooleanEnable the schema tracker; requires queryserver-config-schema-change-signal to be enabled on the underlying vttablets
-schema_change_signal_userstringUser to be used to send down query to vttablet to retrieve schema changes
-security_policystringThe name of a registered security policy to use for controlling access to URLs - empty means allow all for anyone (built-in policies: deny-all, read-only)
-service_mapvalueComma separated list of services to enable (or disable if prefixed with '-') Example: grpc-vtworker
-sql-max-length-errorsintTruncate queries in error logs to the given length (default unlimited)
-sql-max-length-uiintTruncate queries in debug UIs to the given length (default 512)
-srv_topo_cache_refreshdurationHow frequently to refresh the topology for cached entries (default 1s)
-srv_topo_cache_ttldurationHow long to use cached entries for topology (default 1s)
-srv_topo_timeoutdurationTopo server timeout (default 5s)
-stats_backendstringThe name of the registered push-based monitoring/stats backend to use
-stats_combine_dimensionsstringList of dimensions to be combined into a single "all" value in exported stats vars
-stats_common_tagsstringComma-separated list of common tags for the stats backend. It provides both label and values. Example: label1:value1,label2:value2
-stats_drop_variablesstringVariables to be dropped from the list of exported variables
-stats_emit_perioddurationInterval between emitting stats to all registered backends (default 1m0s)
-statsd_addressstringAddress for statsd client
-statsd_sample_ratefloat(default 1)
-stderrthresholdvalueLogs at or above this threshold go to stderr (default 1)
-stream_buffer_sizeintThe number of bytes sent from vtgate for each stream call. It's recommended to keep this value in sync with vttablet's query-server-config-stream-buffer-size (default 32768)
-tablet_filtersvalueSpecifies a comma-separated list of 'keyspace
-tablet_grpc_castringThe server CA to use to validate servers when connecting
-tablet_grpc_certstringThe cert to use to connect
-tablet_grpc_crlstringThe server crl to use to validate server certificates when connecting
-tablet_grpc_keystringThe key to use to connect
-tablet_grpc_server_namestringThe server name to use to validate server certificate
-tablet_manager_protocolstringThe protocol to use to talk to vttablet (default "grpc")
-tablet_protocolstringHow to talk to the vttablets (default "grpc")
-tablet_refresh_intervaldurationTablet refresh interval (default 1m0s)
-tablet_refresh_known_tabletsbooleanTablet refresh reloads the tablet address/port map from topo in case it changes (default true)
-tablet_types_to_waitstringWait till connected for specified tablet types during Gateway initialization
-tablet_url_templatestringFormat string describing debug tablet url formatting. See the Go code for getTabletDebugURL() how to customize this. (default "http://{{.GetTabletHostPort}}")
-topo_consul_lock_delaydurationLockDelay for consul session (default 15s)
-topo_consul_lock_session_checksstringList of checks for consul session (default "serfHealth")
-topo_consul_lock_session_ttlstringTTL for consul session.
-topo_consul_watch_poll_durationdurationTime of the long poll for watch queries (default 30s)
-topo_etcd_lease_ttlintLease TTL for locks and leader election. The client will use KeepAlive to keep the lease going (default 30)
-topo_etcd_tls_castringPath to the CA to use to validate the server cert when connecting to the etcd topo server
-topo_etcd_tls_certstringPath to the client cert to use to connect to the etcd topo server, requires topo_etcd_tls_key, enables TLS
-topo_etcd_tls_keystringPath to the client key to use to connect to the etcd topo server, enables TLS
-topo_global_rootstringPath of the global topology data in the global topology server
-topo_global_server_addressstringAddress of the global topology server
-topo_implementationstringTopology implementation to use
-topo_k8s_contextstringThe kubeconfig context to use, overrides the 'current-context' from the config
-topo_k8s_kubeconfigstringPath to a valid kubeconfig file. When running as a k8s pod inside the same cluster you wish to use as the topo, you may omit this and the below arguments, and Vitess is capable of auto-discovering the correct values. https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/#accessing-the-api-from-a-pod
-topo_k8s_namespacestringThe kubernetes namespace to use for all objects. Default comes from the context or in-cluster config
-topo_read_concurrencyintConcurrent topo reads (default 32)
-topo_zk_auth_filestringAuth to use when connecting to the zk topo server, file contents should be :, e.g., digest:user:pass
-topo_zk_base_timeoutdurationzk base timeout (see zk.Connect) (default 30s)
-topo_zk_max_concurrencyintMaximum number of pending requests to send to a Zookeeper server (default 64)
-topo_zk_tls_castringThe server CA to use to validate servers when connecting to the zk topo server
-topo_zk_tls_certstringThe cert to use to connect to the zk topo server, requires topo_zk_tls_key, enables TLS
-topo_zk_tls_keystringThe key to use to connect to the zk topo server, enables TLS
-tracerstringTracing service to use (default "noop")
-tracing-enable-loggingbooleanWhether to enable logging in the tracing service
-tracing-sampling-ratevalueSampling rate for the probabilistic jaeger sampler (default 0.1)
-tracing-sampling-typevalueSampling strategy to use for jaeger. possible values are 'const', 'probabilistic', 'rateLimiting', or 'remote' (default const)
-transaction_modestringSINGLE: disallow multi-db transactions, MULTI: allow multi-db transactions with best effort commit, TWOPC: allow multi-db transactions with 2pc commit (default "MULTI")
-vvalueLog level for V logs
-versionbooleanPrint binary version
-vmodulevalueComma-separated list of pattern=N settings for file-filtered logging
-vschema_ddl_authorized_usersstringList of users authorized to execute vschema ddl operations, or '%' to allow all users.
-vtctld_addrstringAddress of a vtctld instance
-vtgate-config-terse-errorsbooleanPrevent bind vars from escaping in returned errors
-warn_memory_rowsintWarning threshold for in-memory results. A row count higher than this amount will cause the VtGateWarnings.ResultsExceeded counter to be incremented (default 30000)
-warn_payload_sizeintThe warning threshold for query payloads in bytes. A payload greater than this threshold will cause the VtGateWarnings.WarnPayloadSizeExceeded counter to be incremented.
-warn_sharded_onlybooleanIf any features that are only available in unsharded mode are used, query execution warnings will be added to the session

Key Options #

  • -srv_topo_cache_ttl: There may be instances where you will need to increase the cached TTL from the default of 1 second to a higher number:
    • You may want to increase this option if you see that your topo leader goes down and keeps your queries waiting for a few seconds