vtgate

VTGate is a stateless proxy responsible for accepting requests from applications and routing them to the appropriate tablet server(s) for query execution. It speaks both the MySQL Protocol and a gRPC protocol.

Example Usage #

Start a vtgate proxy:

export TOPOLOGY_FLAGS="--topo_implementation etcd2 --topo_global_server_address localhost:2379 --topo_global_root /vitess/global"
export VTDATAROOT="/tmp"

vtgate \
  $TOPOLOGY_FLAGS \
  --log_dir $VTDATAROOT/tmp \
  --port 15001 \
  --grpc_port 15991 \
  --mysql_server_port 15306 \
  --cell test \
  --cells_to_watch test \
  --tablet_types_to_wait PRIMARY,REPLICA \
  --service_map 'grpc-vtgateservice' \
  --pid_file $VTDATAROOT/tmp/vtgate.pid \
  --mysql_auth_server_impl none

Options #

The following global options apply to vtgate:

NameTypeDefinition
--allowed_tablet_typesvalueSpecifies the tablet types this vtgate is allowed to route queries to
--alsologtostderrbooleanlog to standard error as well as files
--buffer_drain_concurrencyintMaximum number of requests retried simultaneously. More concurrency will increase the load on the PRIMARY vttablet when draining the buffer. (default 1)
--buffer_implementationstringThe algorithm used for managing request buffering during cluster availability events (allowed values: "keyspace_events" (default), "healthcheck" (legacy value))
--buffer_keyspace_shardsstringIf not empty, limit buffering to these entries (comma separated). Entry format: keyspace or keyspace/shard. Requires --enable_buffer=true.
--buffer_max_failover_durationdurationStop buffering completely if a failover takes longer than this duration. (default 20s)
--buffer_min_time_between_failoversdurationMinimum time between the end of a failover and the start of the next one (tracked per shard). Faster consecutive failovers will not trigger buffering. (default 1m0s)
--buffer_sizeintMaximum number of buffered requests in flight (across all ongoing failovers). (default 1000)
--buffer_windowdurationDuration for how long a request should be buffered at most. (default 10s)
--cellstringcell to use (default "test_nj")
--cells_to_watchstringcomma-separated list of cells for watching tablets
--consul_auth_static_filestringJSON File to read the topos/tokens from.
--cpu_profilestringwrite cpu profile to file
--datadog-agent-hoststringhost to send spans to. if empty, no tracing will be done
--datadog-agent-portstringport to send spans to. if empty, no tracing will be done
--ddl_strategystringSet default strategy for DDL statements. Override with @@ddl_strategy session variable.
--default_tablet_typevalueThe default tablet type to set for queries, when one is not explicitly selected (default PRIMARY)
--discovery_high_replication_lag_minimum_servingdurationthe replication lag that is considered too high when selecting the minimum num vttablets for serving (default 2h0m0s)
--discovery_low_replication_lagdurationthe replication lag that is considered low enough to be healthy (default 30s)
--emit_statsbooleantrue iff we should emit stats to push-based monitoring/stats backends
--enable_bufferbooleanEnable buffering (stalling) of primary traffic during failovers.
--enable_buffer_dry_runbooleanDetect and log failover events, but do not actually buffer requests.
--enable_system_settingsbooleanEnables the system settings to be changed per session at the database connection level. Override with @@enable_system_settings session variable.
--enable_set_varbooleanThis will enable the use of MySQL's SET_VAR query hint for certain system variables instead of using reserved connections.
--gate_query_cache_sizeintgate server query cache size, maximum number of queries to be cached. vtgate analyzes every incoming query and generate a query plan, these plans are being cached in a lru cache. This config controls the capacity of the lru cache. (default 10000)
--gateway_initial_tablet_timeoutdurationAt startup, the gateway will wait up to that duration to get one tablet per keyspace/shard/tablettype (default 30s)
--grpc_auth_modestringWhich auth plugin implementation to use (eg: static)
--grpc_auth_mtls_allowed_substringsstringList of substrings of at least one of the client certificate names (separated by colon).
--grpc_auth_static_client_credsstringwhen using grpc_static_auth in the server, this file provides the credentials to use to authenticate with server
--grpc_auth_static_password_filestringJSON File to read the users/passwords from.
--grpc_castringca to use, requires TLS, and enforces client cert check
--grpc_certstringcertificate to use, requires grpc_key, enables TLS
--grpc_compressionstringhow to compress gRPC, default: nothing, supported: snappy
--grpc_enable_tracingbooleanEnable GRPC tracing
--grpc_initial_conn_window_sizeintgrpc initial connection window size
--grpc_initial_window_sizeintgrpc initial window size
--grpc_keepalive_timedurationAfter a duration of this time if the client doesn't see any activity it pings the server to see if the transport is still alive. (default 10s)
--grpc_keepalive_timeoutdurationAfter having pinged for keepalive check, the client waits for a duration of Timeout and if no activity is seen even after that the connection is closed. (default 10s)
--grpc_keystringkey to use, requires grpc_cert, enables TLS
--grpc_max_connection_agedurationMaximum age of a client connection before GoAway is sent. (default 2562047h47m16.854775807s)
--grpc_max_connection_age_gracedurationAdditional grace period after grpc_max_connection_age, after which connections are forcibly closed. (default 2562047h47m16.854775807s)
--grpc_max_message_sizeintMaximum allowed RPC message size. Larger messages will be rejected by gRPC with the error 'exceeding the max size'. (default 16777216)
--grpc_portintPort to listen on for gRPC calls
--grpc_prometheusbooleanEnable gRPC monitoring with Prometheus
--grpc_server_initial_conn_window_sizeintgrpc server initial connection window size
--grpc_server_initial_window_sizeintgrpc server initial window size
--grpc_server_keepalive_enforcement_policy_min_timedurationgrpc server minimum keepalive time (default 5m0s)
--grpc_server_keepalive_enforcement_policy_permit_without_streambooleangrpc server permit client keepalive pings even when there are no active streams (RPCs)
--grpc_use_effective_calleridbooleanIf set, and SSL is not used, will set the immediate caller id from the effective caller id's principal.
--healthcheck_retry_delaydurationhealth check retry delay (default 2ms)
--healthcheck_timeoutdurationthe health check timeout period (default 1m0s)
--jaeger-agent-hoststringhost and port to send spans to. if empty, no tracing will be done
--keep_logsdurationkeep logs for this long (using ctime) (zero to keep forever)
--keep_logs_by_mtimedurationkeep logs for this long (using mtime) (zero to keep forever)
--keyspaces_to_watchvalueSpecifies which keyspaces this vtgate should have access to while routing queries or accessing the vschema
--lameduck-perioddurationkeep running at least this long after SIGTERM before stopping (default 50ms)
--legacy_replication_lag_algorithmbooleanuse the legacy algorithm when selecting the vttablets for serving (default true)
--log_backtrace_atvaluewhen logging hits line file:N, emit a stack trace
--lock_heartbeat_timedurationIf there is lock function used. This will keep the lock connection active by using this heartbeat. (default 5 seconds)
--log_dirstringIf non-empty, write log files in this directory
--log_err_stacksbooleanlog stack traces for errors
--log_queries_to_filestringEnable query logging to the specified file
--log_rotate_max_sizeuintsize in bytes at which logs are rotated (glog.MaxSize) (default 1887436800)
--logtostderrbooleanlog to standard error instead of files
--max_memory_rowsintMaximum number of rows that will be held in memory for intermediate results as well as the final result. (default 300000)
--max_payload_sizeintThe threshold for query payloads in bytes. A payload greater than this threshold will result in a failure to handle the query.
--mem-profile-rateintprofile every n bytes allocated (default 524288)
--message_stream_grace_perioddurationthe amount of time to give for a vttablet to resume if it ends a message stream, usually because of a reparent. (default 30s)
--min_number_serving_vttabletsintthe minimum number of vttablets that will be continue to be used even with low replication lag (default 2)
--mutex-profile-fractionintprofile every n mutex contention events (see runtime.SetMutexProfileFraction)
--mysql_allow_clear_text_without_tlsbooleanIf set, the server will allow the use of a clear text password over non-SSL connections.
--mysql_auth_server_implstringWhich auth server implementation to use. (default "static")
--mysql_auth_server_static_filestringJSON File to read the users/passwords from.
--mysql_auth_server_static_stringstringJSON representation of the users/passwords config.
--mysql_auth_static_reload_intervaldurationTicker to reload credentials
--mysql_clientcert_auth_methodstringclient-side authentication method to use. Supported values: mysql_clear_password, dialog. (default "mysql_clear_password")
--mysql_default_workloadstringDefault session workload (OLTP, OLAP, DBA) (default "UNSPECIFIED")
--mysql_ldap_auth_config_filestringJSON File from which to read LDAP server config.
--mysql_ldap_auth_config_stringstringJSON representation of LDAP server config.
--mysql_ldap_auth_methodstringclient-side authentication method to use. Supported values: mysql_clear_password, dialog. (default "mysql_clear_password")
--mysql_server_bind_addressstringBinds on this address when listening to MySQL binary protocol. Useful to restrict listening to 'localhost' only for instance.
--mysql_server_flush_delaydurationDelay after which buffered response will flushed to client. (default 100ms)
--mysql_server_portintIf set, also listen for MySQL binary protocol connections on this port. (default -1)
--mysql_server_query_timeoutdurationmysql query timeout
--mysql_server_read_timeoutdurationconnection read timeout
--mysql_server_require_secure_transportbooleanReject insecure connections but only if --mysql_server_ssl_cert and --mysql_server_ssl_key are provided
--mysql_server_socket_pathstringThis option specifies the Unix socket file to use when listening for local connections. By default it will be empty and it won't listen to a unix socket
--mysql_server_ssl_castringPath to ssl CA for mysql server plugin SSL. If specified, server will require and validate client certs.
--mysql_server_ssl_certstringPath to the ssl cert for mysql server plugin SSL
--mysql_server_ssl_keystringPath to ssl key for mysql server plugin SSL
--mysql_server_versionstringMySQL server version to advertise.
--mysql_server_write_timeoutdurationconnection write timeout
--mysql_slow_connect_warn_thresholddurationWarn if it takes more than the given threshold for a mysql connection to establish
--mysql_tcp_versionstringSelect tcp, tcp4, or tcp6 to control the socket type. (default "tcp")
--normalize_queriesbooleanRewrite queries with bind vars. Turn this off if the app itself sends normalized queries with bind vars. (default true)
--onterm_timeoutdurationwait no more than this for OnTermSync handlers before stopping (default 10s)
--opentsdb_uristringURI of opentsdb /api/put method
--planner-versionstringSets the default planner to use when the session has not changed it. Valid values are: V3, Gen4, Gen4Greedy and Gen4Fallback. Gen4Fallback tries the gen4 planner and falls back to the V3 planner if the gen4 fails.
--pid_filestringIf set, the process will write its pid to the named file, and delete it on graceful shutdown.
--portintport for the server
--proxy_protocolbooleanEnable HAProxy PROXY protocol on MySQL listener socket
--purge_logs_intervaldurationhow often try to remove old logs (default 1h0m0s)
--querylog-filter-tagstringstring that must be present in the query as a comment for the query to be logged, works for both vtgate and vttablet
--querylog-formatstringformat for query logs ("text" or "json") (default "text")
--redact-debug-ui-queriesbooleanredact full queries and bind variables from debug UI
--remote_operation_timeoutdurationtime to wait for a remote operation (default 30s)
--retry-countintretry count (default 2)
--schema_change_signalbooleanenable schema tracking
--security_policystringthe name of a registered security policy to use for controlling access to URLs - empty means allow all for anyone (built-in policies: deny-all, read-only)
--service_mapvaluecomma separated list of services to enable (or disable if prefixed with '-') Example: grpc-vtworker
--sql-max-length-errorsinttruncate queries in error logs to the given length (default unlimited)
--sql-max-length-uiinttruncate queries in debug UIs to the given length (default 512) (default 512)
--srv_topo_cache_refreshdurationhow frequently to refresh the topology for cached entries (default 1s)
--srv_topo_cache_ttldurationhow long to use cached entries for topology (default 1s)
--stats_backendstringThe name of the registered push-based monitoring/stats backend to use
--stats_combine_dimensionsstringList of dimensions to be combined into a single "all" value in exported stats vars
--stats_drop_variablesstringVariables to be dropped from the list of exported variables.
--stats_emit_perioddurationInterval between emitting stats to all registered backends (default 1m0s)
--stderrthresholdvaluelogs at or above this threshold go to stderr (default 1)
--stream_buffer_sizeintthe number of bytes sent from vtgate for each stream call. It's recommended to keep this value in sync with vttablet's --query-server-config-stream-buffer-size. (default 32768)
--tablet_filtersvalueSpecifies a comma-separated list of 'keyspace
--tablet_grpc_castringthe server ca to use to validate servers when connecting
--tablet_grpc_certstringthe cert to use to connect
--tablet_grpc_keystringthe key to use to connect
--tablet_grpc_server_namestringthe server name to use to validate server certificate
--tablet_protocolstringhow to talk to the vttablets (default "grpc")
--tablet_refresh_intervaldurationtablet refresh interval (default 1m0s)
--tablet_refresh_known_tabletsbooleantablet refresh reloads the tablet address/port map from topo in case it changes (default true)
--tablet_types_to_waitstringwait till connected for specified tablet types during Gateway initialization
--tablet_url_templatestringformat string describing debug tablet url formatting. See the Go code for getTabletDebugURL() how to customize this. (default "http://{{.GetTabletHostPort}}")
--topo_consul_watch_poll_durationdurationtime of the long poll for watch queries. (default 30s)
--topo_etcd_lease_ttlintLease TTL for locks and leader election. The client will use KeepAlive to keep the lease going. (default 30)
--topo_etcd_tls_castringpath to the ca to use to validate the server cert when connecting to the etcd topo server
--topo_etcd_tls_certstringpath to the client cert to use to connect to the etcd topo server, requires topo_etcd_tls_key, enables TLS
--topo_etcd_tls_keystringpath to the client key to use to connect to the etcd topo server, enables TLS
--topo_global_rootstringthe path of the global topology data in the global topology server
--topo_global_server_addressstringthe address of the global topology server
--topo_implementationstringthe topology implementation to use
--topo_k8s_contextstringThe kubeconfig context to use, overrides the 'current-context' from the config
--topo_k8s_kubeconfigstringPath to a valid kubeconfig file.
--topo_k8s_namespacestringThe kubernetes namespace to use for all objects. Default comes from the context or in-cluster config
--topo_read_concurrencyintconcurrent topo reads (default 32)
--topo_zk_auth_filestringauth to use when connecting to the zk topo server, file contents should be :, e.g., digest:user:pass
--topo_zk_base_timeoutdurationzk base timeout (see zk.Connect) (default 30s)
--topo_zk_max_concurrencyintmaximum number of pending requests to send to a Zookeeper server. (default 64)
--topo_zk_tls_castringthe server ca to use to validate servers when connecting to the zk topo server
--topo_zk_tls_certstringthe cert to use to connect to the zk topo server, requires topo_zk_tls_key, enables TLS
--topo_zk_tls_keystringthe key to use to connect to the zk topo server, enables TLS
--tracerstringtracing service to use (default "noop")
--tracing-sampling-ratefloatsampling rate for the probabilistic jaeger sampler (default 0.1)
--transaction_modestringthe default transaction mode -- SINGLE: disallow multi-db transactions, MULTI: allow multi-db transactions with best effort commit, TWOPC: allow multi-db transactions with 2pc commit (default "MULTI"); this can be overridden at the session level when needed using SET transaction_mode="<mode>";
-vvaluelog level for V logs
--versionbooleanprint binary version
--vmodulevaluecomma-separated list of pattern=N settings for file-filtered logging
--vschema_ddl_authorized_usersstringList of users authorized to execute vschema ddl operations, or '%' to allow all users.
--vtctld_addrstringaddress of a vtctld instance
--vtgate-config-terse-errorsbooleanprevent bind vars from escaping in returned errors
--warn_memory_rowsintWarning threshold for in-memory results. A row count higher than this amount will cause the VtGateWarnings.ResultsExceeded counter to be incremented. (default 30000)
--warn_payload_sizeintThe warning threshold for query payloads in bytes. A payload greater than this threshold will cause the VtGateWarnings.WarnPayloadSizeExceeded counter to be incremented.

Key Options #

  • --srv_topo_cache_ttl: There may be instances where you will need to increase the cached TTL from the default of 1 second to a higher number:
    • You may want to increase this option if you see that your topo leader goes down and keeps your queries waiting for a few seconds