vtgate
vtgate #
VTGate is a stateless proxy responsible for accepting requests from applications and routing them to the appropriate tablet server(s) for query execution. It speaks both the MySQL Protocol and a gRPC protocol.
Synopsis #
VTGate is a stateless proxy responsible for accepting requests from applications and routing them to the appropriate tablet server(s) for query execution. It speaks both the MySQL Protocol and a gRPC protocol.
Key Options #
--srv_topo_cache_ttl: There may be instances where you will need to increase the cached TTL from the default of 1 second to a higher number:- You may want to increase this option if you see that your topo leader goes down and keeps your queries waiting for a few seconds.
vtgate [flags]
Examples #
vtgate \
--topo_implementation etcd2 \
--topo_global_server_address localhost:2379 \
--topo_global_root /vitess/global \
--log_dir $VTDATAROOT/tmp \
--port 15001 \
--grpc_port 15991 \
--mysql_server_port 15306 \
--cell test \
--cells_to_watch test \
--tablet_types_to_wait PRIMARY,REPLICA \
--service_map 'grpc-vtgateservice' \
--pid_file $VTDATAROOT/tmp/vtgate.pid \
--mysql_auth_server_impl none
Options #
--allow-kill-statement Allows the execution of kill statement
--allowed_tablet_types strings Specifies the tablet types this vtgate is allowed to route queries to. Should be provided as a comma-separated set of tablet types.
--alsologtostderr log to standard error as well as files
--balancer-keyspaces strings When in balanced mode, a comma-separated list of keyspaces for which to use the balancer (optional)
--balancer-vtgate-cells strings When in balanced mode, a comma-separated list of cells that contain vtgates (required)
--bind-address string Bind address for the server. If empty, the server will listen on all available unicast and anycast IP addresses of the local system.
--buffer_drain_concurrency int Maximum number of requests retried simultaneously. More concurrency will increase the load on the PRIMARY vttablet when draining the buffer. (default 1)
--buffer_keyspace_shards string If not empty, limit buffering to these entries (comma separated). Entry format: keyspace or keyspace/shard. Requires --enable_buffer=true.
--buffer_max_failover_duration duration Stop buffering completely if a failover takes longer than this duration. (default 20s)
--buffer_min_time_between_failovers duration Minimum time between the end of a failover and the start of the next one (tracked per shard). Faster consecutive failovers will not trigger buffering. (default 1m0s)
--buffer_size int Maximum number of buffered requests in flight (across all ongoing failovers). (default 1000)
--buffer_window duration Duration for how long a request should be buffered at most. (default 10s)
--catch-sigpipe catch and ignore SIGPIPE on stdout and stderr if specified
--cell string cell to use
--cells_to_watch string comma-separated list of cells for watching tablets
--config-file string Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
--config-file-not-found-handling ConfigFileNotFoundHandling Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
--config-name string Name of the config file (without extension) to search for. (default "vtconfig")
--config-path strings Paths to search for config files in. (default [<WORKDIR>])
--config-persistence-min-interval duration minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
--config-type string Config file type (omit to infer config type from file extension).
--consul_auth_static_file string JSON File to read the topos/tokens from.
--datadog-agent-host string host to send spans to. if empty, no tracing will be done
--datadog-agent-port string port to send spans to. if empty, no tracing will be done
--dbddl_plugin string controls how to handle CREATE/DROP DATABASE. use it if you are using your own database provisioning service (default "fail")
--ddl_strategy string Set default strategy for DDL statements. Override with @@ddl_strategy session variable (default "direct")
--default_tablet_type topodatapb.TabletType The default tablet type to set for queries, when one is not explicitly selected. (default PRIMARY)
-d, --dir string output directory to write documentation (default "doc")
--discovery_high_replication_lag_minimum_serving duration Threshold above which replication lag is considered too high when applying the min_number_serving_vttablets flag. (default 2h0m0s)
--discovery_low_replication_lag duration Threshold below which replication lag is considered low enough to be healthy. (default 30s)
--emit_stats If set, emit stats to push-based monitoring and stats backends
--enable-balancer Enable the tablet balancer to evenly spread query load for a given tablet type
--enable-partial-keyspace-migration (Experimental) Follow shard routing rules: enable only while migrating a keyspace shard by shard. See documentation on Partial MoveTables for more. (default false)
--enable-views Enable views support in vtgate.
--enable_buffer Enable buffering (stalling) of primary traffic during failovers.
--enable_buffer_dry_run Detect and log failover events, but do not actually buffer requests.
--enable_direct_ddl Allow users to submit direct DDL statements (default true)
--enable_online_ddl Allow users to submit, review and control Online DDL (default true)
--enable_set_var This will enable the use of MySQL's SET_VAR query hint for certain system variables instead of using reserved connections (default true)
--enable_system_settings This will enable the system settings to be changed per session at the database connection level (default true)
--foreign_key_mode string This is to provide how to handle foreign key constraint in create/alter table. Valid values are: allow, disallow (default "allow")
--gate_query_cache_memory int gate server query cache size in bytes, maximum amount of memory to be cached. vtgate analyzes every incoming query and generate a query plan, these plans are being cached in a lru cache. This config controls the capacity of the lru cache. (default 33554432)
--gateway_initial_tablet_timeout duration At startup, the tabletGateway will wait up to this duration to get at least one tablet per keyspace/shard/tablet type (default 30s)
--grpc-dial-concurrency-limit int Maximum concurrency of grpc dial operations. This should be less than the golang max thread limit of 10000. (default 1024)
--grpc-send-session-in-streaming If set, will send the session as last packet in streaming api to support transactions in streaming
--grpc-use-effective-groups If set, and SSL is not used, will set the immediate caller's security groups from the effective caller id's groups.
--grpc-use-static-authentication-callerid If set, will set the immediate caller id to the username authenticated by the static auth plugin.
--grpc_auth_mode string Which auth plugin implementation to use (eg: static)
--grpc_auth_mtls_allowed_substrings string List of substrings of at least one of the client certificate names (separated by colon).
--grpc_auth_static_client_creds string When using grpc_static_auth in the server, this file provides the credentials to use to authenticate with server.
--grpc_auth_static_password_file string JSON File to read the users/passwords from.
--grpc_bind_address string Bind address for gRPC calls. If empty, listen on all addresses.
--grpc_ca string server CA to use for gRPC connections, requires TLS, and enforces client certificate check
--grpc_cert string server certificate to use for gRPC connections, requires grpc_key, enables TLS
--grpc_compression string Which protocol to use for compressing gRPC. Default: nothing. Supported: snappy
--grpc_crl string path to a certificate revocation list in PEM format, client certificates will be further verified against this file during TLS handshake
--grpc_enable_optional_tls enable optional TLS mode when a server accepts both TLS and plain-text connections on the same port
--grpc_enable_tracing Enable gRPC tracing.
--grpc_initial_conn_window_size int gRPC initial connection window size
--grpc_initial_window_size int gRPC initial window size
--grpc_keepalive_time duration After a duration of this time, if the client doesn't see any activity, it pings the server to see if the transport is still alive. (default 10s)
--grpc_keepalive_timeout duration After having pinged for keepalive check, the client waits for a duration of Timeout and if no activity is seen even after that the connection is closed. (default 10s)
--grpc_key string server private key to use for gRPC connections, requires grpc_cert, enables TLS
--grpc_max_connection_age duration Maximum age of a client connection before GoAway is sent. (default 2562047h47m16.854775807s)
--grpc_max_connection_age_grace duration Additional grace period after grpc_max_connection_age, after which connections are forcibly closed. (default 2562047h47m16.854775807s)
--grpc_max_message_size int Maximum allowed RPC message size. Larger messages will be rejected by gRPC with the error 'exceeding the max size'. (default 16777216)
--grpc_port int Port to listen on for gRPC calls. If zero, do not listen.
--grpc_prometheus Enable gRPC monitoring with Prometheus.
--grpc_server_ca string path to server CA in PEM format, which will be combine with server cert, return full certificate chain to clients
--grpc_server_initial_conn_window_size int gRPC server initial connection window size
--grpc_server_initial_window_size int gRPC server initial window size
--grpc_server_keepalive_enforcement_policy_min_time duration gRPC server minimum keepalive time (default 10s)
--grpc_server_keepalive_enforcement_policy_permit_without_stream gRPC server permit client keepalive pings even when there are no active streams (RPCs)
--grpc_server_keepalive_time duration After a duration of this time, if the server doesn't see any activity, it pings the client to see if the transport is still alive. (default 10s)
--grpc_server_keepalive_timeout duration After having pinged for keepalive check, the server waits for a duration of Timeout and if no activity is seen even after that the connection is closed. (default 10s)
--grpc_use_effective_callerid If set, and SSL is not used, will set the immediate caller id from the effective caller id's principal.
--healthcheck_retry_delay duration health check retry delay (default 2ms)
--healthcheck_timeout duration the health check timeout period (default 1m0s)
-h, --help help for docgen
--jaeger-agent-host string host and port to send spans to. if empty, no tracing will be done
--keep_logs duration keep logs for this long (using ctime) (zero to keep forever)
--keep_logs_by_mtime duration keep logs for this long (using mtime) (zero to keep forever)
--keyspaces_to_watch strings Specifies which keyspaces this vtgate should have access to while routing queries or accessing the vschema.
--lameduck-period duration keep running at least this long after SIGTERM before stopping (default 50ms)
--legacy_replication_lag_algorithm Use the legacy algorithm when selecting vttablets for serving. (default true)
--lock-timeout duration Maximum time to wait when attempting to acquire a lock from the topo server (default 45s)
--lock_heartbeat_time duration If there is lock function used. This will keep the lock connection active by using this heartbeat (default 5s)
--log_backtrace_at traceLocations when logging hits line file:N, emit a stack trace
--log_dir string If non-empty, write log files in this directory
--log_err_stacks log stack traces for errors
--log_queries_to_file string Enable query logging to the specified file
--log_rotate_max_size uint size in bytes at which logs are rotated (glog.MaxSize) (default 1887436800)
--logtostderr log to standard error instead of files
--max-stack-size int configure the maximum stack size in bytes (default 67108864)
--max_memory_rows int Maximum number of rows that will be held in memory for intermediate results as well as the final result. (default 300000)
--max_payload_size int The threshold for query payloads in bytes. A payload greater than this threshold will result in a failure to handle the query.
--message_stream_grace_period duration the amount of time to give for a vttablet to resume if it ends a message stream, usually because of a reparent. (default 30s)
--min_number_serving_vttablets int The minimum number of vttablets for each replicating tablet_type (e.g. replica, rdonly) that will be continue to be used even with replication lag above discovery_low_replication_lag, but still below discovery_high_replication_lag_minimum_serving. (default 2)
--mysql-server-drain-onterm If set, the server waits for --onterm_timeout for already connected clients to complete their in flight work
--mysql-server-keepalive-period duration TCP period between keep-alives
--mysql-server-pool-conn-read-buffers If set, the server will pool incoming connection read buffers
--mysql_allow_clear_text_without_tls If set, the server will allow the use of a clear text password over non-SSL connections.
--mysql_auth_server_impl string Which auth server implementation to use. Options: none, ldap, clientcert, static, vault. (default "static")
--mysql_auth_server_static_file string JSON File to read the users/passwords from.
--mysql_auth_server_static_string string JSON representation of the users/passwords config.
--mysql_auth_static_reload_interval duration Ticker to reload credentials
--mysql_auth_vault_addr string URL to Vault server
--mysql_auth_vault_path string Vault path to vtgate credentials JSON blob, e.g.: secret/data/prod/vtgatecreds
--mysql_auth_vault_role_mountpoint string Vault AppRole mountpoint; can also be passed using VAULT_MOUNTPOINT environment variable (default "approle")
--mysql_auth_vault_role_secretidfile string Path to file containing Vault AppRole secret_id; can also be passed using VAULT_SECRETID environment variable
--mysql_auth_vault_roleid string Vault AppRole id; can also be passed using VAULT_ROLEID environment variable
--mysql_auth_vault_timeout duration Timeout for vault API operations (default 10s)
--mysql_auth_vault_tls_ca string Path to CA PEM for validating Vault server certificate
--mysql_auth_vault_tokenfile string Path to file containing Vault auth token; token can also be passed using VAULT_TOKEN environment variable
--mysql_auth_vault_ttl duration How long to cache vtgate credentials from the Vault server (default 30m0s)
--mysql_clientcert_auth_method string client-side authentication method to use. Supported values: mysql_clear_password, dialog. (default "mysql_clear_password")
--mysql_default_workload string Default session workload (OLTP, OLAP, DBA) (default "OLTP")
--mysql_ldap_auth_config_file string JSON File from which to read LDAP server config.
--mysql_ldap_auth_config_string string JSON representation of LDAP server config.
--mysql_ldap_auth_method string client-side authentication method to use. Supported values: mysql_clear_password, dialog. (default "mysql_clear_password")
--mysql_server_bind_address string Binds on this address when listening to MySQL binary protocol. Useful to restrict listening to 'localhost' only for instance.
--mysql_server_flush_delay duration Delay after which buffered response will be flushed to the client. (default 100ms)
--mysql_server_port int If set, also listen for MySQL binary protocol connections on this port. (default -1)
--mysql_server_query_timeout duration mysql query timeout
--mysql_server_read_timeout duration connection read timeout
--mysql_server_require_secure_transport Reject insecure connections but only if mysql_server_ssl_cert and mysql_server_ssl_key are provided
--mysql_server_socket_path string This option specifies the Unix socket file to use when listening for local connections. By default it will be empty and it won't listen to a unix socket
--mysql_server_ssl_ca string Path to ssl CA for mysql server plugin SSL. If specified, server will require and validate client certs.
--mysql_server_ssl_cert string Path to the ssl cert for mysql server plugin SSL
--mysql_server_ssl_crl string Path to ssl CRL for mysql server plugin SSL
--mysql_server_ssl_key string Path to ssl key for mysql server plugin SSL
--mysql_server_ssl_server_ca string path to server CA in PEM format, which will be combine with server cert, return full certificate chain to clients
--mysql_server_tls_min_version string Configures the minimal TLS version negotiated when SSL is enabled. Defaults to TLSv1.2. Options: TLSv1.0, TLSv1.1, TLSv1.2, TLSv1.3.
--mysql_server_version string MySQL server version to advertise. (default "8.0.30-Vitess")
--mysql_server_write_timeout duration connection write timeout
--mysql_slow_connect_warn_threshold duration Warn if it takes more than the given threshold for a mysql connection to establish
--mysql_tcp_version string Select tcp, tcp4, or tcp6 to control the socket type. (default "tcp")
--no_scatter when set to true, the planner will fail instead of producing a plan that includes scatter queries
--normalize_queries Rewrite queries with bind vars. Turn this off if the app itself sends normalized queries with bind vars. (default true)
--onclose_timeout duration wait no more than this for OnClose handlers before stopping (default 10s)
--onterm_timeout duration wait no more than this for OnTermSync handlers before stopping (default 10s)
--opentsdb_uri string URI of opentsdb /api/put method
--pid_file string If set, the process will write its pid to the named file, and delete it on graceful shutdown.
--planner-version string Sets the default planner to use when the session has not changed it. Valid values are: Gen4, Gen4Greedy, Gen4Left2Right
--port int port for the server
--pprof strings enable profiling
--pprof-http enable pprof http endpoints
--proxy_protocol Enable HAProxy PROXY protocol on MySQL listener socket
--purge_logs_interval duration how often try to remove old logs (default 1h0m0s)
--query-timeout int Sets the default query timeout (in ms). Can be overridden by session variable (query_timeout) or comment directive (QUERY_TIMEOUT_MS)
--querylog-buffer-size int Maximum number of buffered query logs before throttling log output (default 10)
--querylog-filter-tag string string that must be present in the query for it to be logged; if using a value as the tag, you need to disable query normalization
--querylog-format string format for query logs ("text" or "json") (default "text")
--querylog-row-threshold uint Number of rows a query has to return or affect before being logged; not useful for streaming queries. 0 means all queries will be logged.
--querylog-sample-rate float Sample rate for logging queries. Value must be between 0.0 (no logging) and 1.0 (all queries)
--redact-debug-ui-queries redact full queries and bind variables from debug UI
--remote_operation_timeout duration time to wait for a remote operation (default 15s)
--retry-count int retry count (default 2)
--schema_change_signal Enable the schema tracker; requires queryserver-config-schema-change-signal to be enabled on the underlying vttablets for this to work (default true)
--security_policy string the name of a registered security policy to use for controlling access to URLs - empty means allow all for anyone (built-in policies: deny-all, read-only)
--service_map strings comma separated list of services to enable (or disable if prefixed with '-') Example: grpc-queryservice
--sql-max-length-errors int truncate queries in error logs to the given length (default unlimited)
--sql-max-length-ui int truncate queries in debug UIs to the given length (default 512) (default 512)
--srv_topo_cache_refresh duration how frequently to refresh the topology for cached entries (default 1s)
--srv_topo_cache_ttl duration how long to use cached entries for topology (default 1s)
--srv_topo_timeout duration topo server timeout (default 5s)
--stats_backend string The name of the registered push-based monitoring/stats backend to use
--stats_combine_dimensions string List of dimensions to be combined into a single "all" value in exported stats vars
--stats_common_tags strings Comma-separated list of common tags for the stats backend. It provides both label and values. Example: label1:value1,label2:value2
--stats_drop_variables string Variables to be dropped from the list of exported variables.
--stats_emit_period duration Interval between emitting stats to all registered backends (default 1m0s)
--statsd_address string Address for statsd client
--statsd_sample_rate float Sample rate for statsd metrics (default 1)
--stderrthreshold severityFlag logs at or above this threshold go to stderr (default 1)
--stream_buffer_size int the number of bytes sent from vtgate for each stream call. It's recommended to keep this value in sync with vttablet's query-server-config-stream-buffer-size. (default 32768)
--table-refresh-interval int interval in milliseconds to refresh tables in status page with refreshRequired class
--tablet-filter-tags StringMap Specifies a comma-separated list of tablet tags (as key:value pairs) to filter the tablets to watch.
--tablet_filters strings Specifies a comma-separated list of 'keyspace|shard_name or keyrange' values to filter the tablets to watch.
--tablet_grpc_ca string the server ca to use to validate servers when connecting
--tablet_grpc_cert string the cert to use to connect
--tablet_grpc_crl string the server crl to use to validate server certificates when connecting
--tablet_grpc_key string the key to use to connect
--tablet_grpc_server_name string the server name to use to validate server certificate
--tablet_protocol string Protocol to use to make queryservice RPCs to vttablets. (default "grpc")
--tablet_refresh_interval duration Tablet refresh interval. (default 1m0s)
--tablet_refresh_known_tablets Whether to reload the tablet's address/port map from topo in case they change. (default true)
--tablet_types_to_wait strings Wait till connected for specified tablet types during Gateway initialization. Should be provided as a comma-separated set of tablet types.
--tablet_url_template string Format string describing debug tablet url formatting. See getTabletDebugURL() for how to customize this. (default "http://{{.GetTabletHostPort}}")
--topo_consul_lock_delay duration LockDelay for consul session. (default 15s)
--topo_consul_lock_session_checks string List of checks for consul session. (default "serfHealth")
--topo_consul_lock_session_ttl string TTL for consul session.
--topo_consul_watch_poll_duration duration time of the long poll for watch queries. (default 30s)
--topo_etcd_lease_ttl int Lease TTL for locks and leader election. The client will use KeepAlive to keep the lease going. (default 30)
--topo_etcd_tls_ca string path to the ca to use to validate the server cert when connecting to the etcd topo server
--topo_etcd_tls_cert string path to the client cert to use to connect to the etcd topo server, requires topo_etcd_tls_key, enables TLS
--topo_etcd_tls_key string path to the client key to use to connect to the etcd topo server, enables TLS
--topo_global_root string the path of the global topology data in the global topology server
--topo_global_server_address string the address of the global topology server
--topo_implementation string the topology implementation to use
--topo_read_concurrency int Concurrency of topo reads. (default 32)
--topo_zk_auth_file string auth to use when connecting to the zk topo server, file contents should be <scheme>:<auth>, e.g., digest:user:pass
--topo_zk_base_timeout duration zk base timeout (see zk.Connect) (default 30s)
--topo_zk_max_concurrency int maximum number of pending requests to send to a Zookeeper server. (default 64)
--topo_zk_tls_ca string the server ca to use to validate servers when connecting to the zk topo server
--topo_zk_tls_cert string the cert to use to connect to the zk topo server, requires topo_zk_tls_key, enables TLS
--topo_zk_tls_key string the key to use to connect to the zk topo server, enables TLS
--tracer string tracing service to use (default "noop")
--tracing-enable-logging whether to enable logging in the tracing service
--tracing-sampling-rate float sampling rate for the probabilistic jaeger sampler (default 0.1)
--tracing-sampling-type string sampling strategy to use for jaeger. possible values are 'const', 'probabilistic', 'rateLimiting', or 'remote' (default "const")
--track-udfs Track UDFs in vtgate.
--transaction_mode string SINGLE: disallow multi-db transactions, MULTI: allow multi-db transactions with best effort commit, TWOPC: allow multi-db transactions with 2pc commit (default "MULTI")
--truncate-error-len int truncate errors sent to client if they are longer than this value (0 means do not truncate)
--v Level log level for V logs
-v, --version print binary version
--vmodule vModuleFlag comma-separated list of pattern=N settings for file-filtered logging
--vschema_ddl_authorized_users string List of users authorized to execute vschema ddl operations, or '%' to allow all users.
--vtgate-config-terse-errors prevent bind vars from escaping in returned errors
--warming-reads-concurrency int Number of concurrent warming reads allowed (default 500)
--warming-reads-percent int Percentage of reads on the primary to forward to replicas. Useful for keeping buffer pools warm
--warming-reads-query-timeout duration Timeout of warming read queries (default 5s)
--warn_memory_rows int Warning threshold for in-memory results. A row count higher than this amount will cause the VtGateWarnings.ResultsExceeded counter to be incremented. (default 30000)
--warn_payload_size int The warning threshold for query payloads in bytes. A payload greater than this threshold will cause the VtGateWarnings.WarnPayloadSizeExceeded counter to be incremented.
--warn_sharded_only If any features that are only available in unsharded mode are used, query execution warnings will be added to the session