Introduction
This page covers the Tyk Pump types that write traffic logs and aggregated analytics into Tyk Dashboard’s persistent storage (MongoDB or SQL): the pumps that ultimately feed Tyk Dashboard’s Log Browser and Traffic Analytics screens. How you deploy these pumps depends on your deployment topology:- In a combined control and data plane, such as the
tyk-stackchart, you configure them directly: Tyk Pump already has access to both Redis and the persistent storage. - In a distributed deployment, with separate control and data planes connected via Tyk MDCB, these same pump types are instead configured as the Control Plane Pump, running on the control plane, which receives data forwarded by the Hybrid Pump on each data plane. See Data Plane Pump for that mechanism.
filters and timeout, see Common Pump Settings.
Choosing a Pump Type
Tyk Pump implements separate pump types for MongoDB and SQL because of how differently the two store data.- MongoDB groups records into collections, so Tyk Pump has a separate pump type for each way of storing them: one collection holding every Organisation’s traffic logs together, one collection per Organisation splitting traffic logs apart, and one collection holding aggregated analytics.
- SQL databases store records as rows in a single table, so there’s no equivalent to Mongo’s “one collection per Organisation” pump type: every Organisation’s rows live in the same table, distinguished by an indexed
org_idcolumn.
For Tyk Dashboard’s Log Browser:
For your own downstream querying:
These pumps write the data into a MongoDB collection or SQL table, where it’s queryable by your own tooling. For REST, that’s the same
mongo/mongo-pump-selective/sql pump you’re already running for the Log Browser; GraphQL and MCP need a dedicated pump instead, since the plain pumps either pass GraphQL through without the extra structured detail, or exclude MCP entirely. Tyk Dashboard itself doesn’t read any of this back out, so you’ll need your own tooling, such as a BI tool or a scheduled export job, connected directly to that database. This is different from the External Data Sink Pumps, which actively forward traffic logs to a separate third-party system such as Splunk or Kafka.
MongoDB
Tyk Pump offers MongoDB pump types for REST, MCP proxy and GraphQL traffic. All share the same connection settings. The pumps are declared and configured as described in the Tyk Pump configuration guide. Each pump to be deployed has its own entry in that configuration and has both common and specific configuration options. The specific config is held in themeta object - some of this is common, some is dependent on the specific pump as outlined in the following sections.
Common Mongo Meta
Every Mongo pump type on this page accepts these fields inmeta, in addition to whatever is listed in its own section below. Each pump’s own snippet only shows its additional fields; combine them with these to build a complete meta block.
Standard Mongo Pump
Themongo pump stores every individual traffic log as a separate document in one collection. It’s the source for Tyk Dashboard’s Log Browser in a single-Organisation deployment.
In addition to the common pump settings and common meta fields, the pump has the following settings:
Because this collection grows with every request, especially with detailed recording enabled, it should be capped.
Tyk Dashboard Setting: set
use_sharded_analytics: false in the Tyk Dashboard configuration so that it queries the tyk_analytics collections to populate the Log Browser.
Per-Organisation Mongo Pump
Themongo-pump-selective pump stores every individual traffic log as a separate document, in a collection called z_tyk_analyticz_{ORG_ID} (i.e. one collection per Organisation). It’s the source for Tyk Dashboard’s Log Browser in a multi-Organisation deployment, isolating each Organisation’s data from the others.
In addition to the common pump settings and common meta fields, the pump has the following settings:
There’s no
collection_name field: the collection name is always computed per-Organisation. There’s also no collection_cap_* support, so, unlike the Standard Mongo Pump, Tyk Pump can’t cap these collections for you. Given the volume of individual documents, cap each Organisation’s collection manually instead, using MongoDB’s convertToCapped command; see Analytics Storage Management for the exact command.
Tyk Dashboard Setting: set use_sharded_analytics: true in the Tyk Dashboard configuration so that it queries z_tyk_analyticz_{ORG_ID} collections to populate the Log Browser.
Mongo Aggregate Pump
Themongo-pump-aggregate pump computes analytics from traffic logs, aggregating hourly or per minute, and stores them in a collection called z_tyk_analyticz_aggregate_{ORG_ID} (i.e. one collection per Organisation). Tyk Dashboard’s Traffic Analytics graphs are built on this aggregated analytics.
In addition to the common pump settings and common meta fields, the pump has the following settings:
Because it stores aggregated analytics, this collection needs minimal capping. If an API definition tags unique per-request headers, such as a
request_id, aggregation creates one document per unique value and the collection can grow rapidly; avoid tagging unique headers where possible, or add them to ignore_aggregations.
Tyk Dashboard Setting: this pump supplies the API Usage Data screens. Set enable_aggregate_lookups: true in the Tyk Dashboard configuration so it reads this pre-computed data, rather than computing it live from traffic logs on every request; see Pre-Computed vs Live Aggregation for the trade-off.
Tyk Dashboard’s
use_sharded_analytics controls which collection(s) it queries. If use_sharded_analytics: false, you must set use_mixed_collection: true so the pump populates the shared collection tyk_analytics_aggregates, which Dashboard always reads in this mode. If use_sharded_analytics: true, ordinary per-Organisation queries read the per-Organisation collections, which the pump populates regardless of use_mixed_collection, but a superuser session not attached to a single Organisation still falls back to the shared collection, so set use_mixed_collection: true too if you need those cross-org views to show data.Self-Healing
MongoDB, DocumentDB, and CosmosDB all cap individual document size (16MB for standard MongoDB). The Mongo Aggregate Pump writes one document peraggregation_time period, and if that document grows past the limit before the period ends, the database rejects further writes for that period.
The pump’s self-healing feature solves this problem by automatically adjusting the aggregation window for each document, resulting in more, smaller documents for a given time period. The performance trade-off is that this can increase the query complexity and load for Tyk Dashboard when it reads and consolidates this data to render the Dashboard Analytics graphs.
Set enable_aggregate_self_healing: true to trigger self-healing: when a write fails because the document is too large, Tyk Pump creates a new document immediately and halves aggregation_time for subsequent periods, reducing the chance of hitting the limit again. This can repeat, halving aggregation_time further each time the limit is hit, down to a minimum of 1 minute. For example, with aggregation_time: 50, if the document reaches 16MB, Tyk Pump starts a new document and reduces aggregation_time to 25.
store_analytics_per_minute takes precedence over aggregation_time. If store_analytics_per_minute is true, aggregation_time is fixed at 1 and self-healing has no effect.Mongo GraphQL Pump
Themongo-graph pump parses GraphQL-specific detail out of the raw request and response of every GraphQL request: types requested, fields requested per type, operation type, variables, root fields, and any errors.
The resultant MongoDB collection exists for your own downstream querying, such as with a BI tool or data warehouse; Tyk Dashboard doesn’t read this data back out anywhere.
Enable detailed recording first, so that GraphQL information can be parsed from the request body and response; this applies globally across all APIs on the Gateway.
In addition to the common pump settings and common meta fields, the pump has the following settings:
collection_name, max_insert_batch_size_bytes, max_document_size_bytes, and collection_cap_* fields as the Standard Mongo Pump; collection_name is required, with no default.
The Standard Mongo Pump doesn’t filter out traffic logs for GraphQL requests, so they already reach
tyk_analytics as ordinary, undifferentiated entries. If you point mongo-graph’s collection_name at the same collection as your Standard Mongo Pump, you’ll get duplicate entries in the Log Browser for every GraphQL request, one plain copy and one GraphQL-enriched copy, without gaining anything: Log Browser can’t display the extra fields anyway. Always use a separate, dedicated collection.- Records can grow quickly, since detailed recording is required.
- Subgraph requests in a federation setup are not recorded, only supergraph requests.
- Universal Data Graph requests are recorded, but subsequent requests to data sources are ignored.
Mongo MCP Pump
Themongo-mcp pump stores MCP (Model Context Protocol) tool-call traffic logs in their own collection, mirroring the Standard Mongo Pump. Unlike GraphQL records, MCP records are excluded from the Standard Mongo Pump, so this is the only way to get MCP traffic logs into MongoDB.
As with the Mongo GraphQL Pump, Tyk Dashboard doesn’t read this data anywhere, including the Activity by MCP screen; it exists for your own downstream querying.
In addition to the common pump settings and common meta fields, the pump has the following settings:
collection_name, max_insert_batch_size_bytes, max_document_size_bytes, and collection_cap_* fields as the Standard Mongo Pump; collection_name is required, with no default.
Mongo MCP Aggregate Pump
Themongo-mcp-aggregate pump computes aggregated MCP analytics, mirroring the Mongo Aggregate Pump, and stores them in a collection called z_tyk_mcp_analyticz_aggregate_{ORG_ID}. Tyk Dashboard’s MCP Traffic Analytics graphs are built on this aggregated analytics.
In addition to the common pump settings and common meta fields, the pump has the following settings:
use_mixed_collection, track_all_paths, ignore_tag_prefix_list, threshold_len_tag_list, store_analytics_per_minute, aggregation_time, enable_aggregate_self_healing, and ignore_aggregations fields as the Mongo Aggregate Pump above. use_mixed_collection: true writes to the org-less tyk_mcp_analytics_aggregate collection instead of tyk_analytics_aggregates.
Tyk Dashboard Setting: this pump supplies the Activity by MCP screen. enable_aggregate_lookups: true is required in the Tyk Dashboard configuration, not just recommended: unlike the REST case, there’s no working live fallback if it’s left false, since that fallback has no MCP-specific logic and queries the plain REST traffic log collection, returning nothing useful. See MCP Analytics for the Dashboard side of this feature.
Tyk Dashboard’s
use_sharded_analytics controls which collection(s) it queries. If use_sharded_analytics: false, you must set use_mixed_collection: true so the pump populates the shared collection tyk_mcp_analytics_aggregate, which Dashboard always reads in this mode. If use_sharded_analytics: true, ordinary per-Organisation queries read the per-Organisation collections, which the pump populates regardless of use_mixed_collection, but a superuser session not attached to a single Organisation still falls back to the shared collection, so set use_mixed_collection: true too if you need those cross-org views to show data.SQL
Tyk Pump offers SQL pump types for REST, MCP proxy and GraphQL traffic. All share the same connection settings. The pumps are declared and configured as described in the Tyk Pump configuration guide. Each pump to be deployed has its own entry in that configuration and has both common and specific configuration options. The specific config is held in themeta object - some of this is common, some is dependent on the specific pump as outlined in the following sections.
Tyk no longer supports SQLite as of Tyk 5.7.0. Transition to PostgreSQL, MongoDB, or a listed compatible alternative.
Common SQL Meta
Every SQL pump type on this page accepts these fields inmeta, in addition to whatever is listed in its own section below. Each pump’s own snippet only shows its additional fields; combine them with these to build a complete meta block.
Standard SQL Pump
Thesql pump stores every individual traffic log as a separate row in one table. It’s the source for Tyk Dashboard’s Log Browser, for both single- and multi-Organisation deployments: unlike MongoDB, there’s no separate per-Organisation pump type, since every Organisation’s rows already live in the same table, distinguished by an indexed org_id column.
It has no pump-specific fields beyond Common SQL Meta and the common pump settings: set type to postgres or mysql and configure those.
By default, the pump stores all records in a single table, tyk_analytics. With table_sharding: true, records are instead stored in per-day tables named tyk_analytics_YYYYMMDD.
Tyk Dashboard Setting: the API Usage Data > Log Browser screen shows requests recorded by the sql pump. Unlike MongoDB, there’s no Dashboard setting to configure: the same table is always queried, regardless of Organisation.
SQL Aggregate Pump
Thesql_aggregate pump computes analytics from traffic logs, aggregating hourly or per minute, and stores them in a table called tyk_aggregated by default.
In addition to the common pump settings and common meta fields, the pump has the following settings:
With
table_sharding: true, records are stored in a tyk_aggregated_YYYYMMDD table per day instead of a single tyk_aggregated table.
Tyk Dashboard Setting: supplies Activity by API, Activity by Key, and Errors. Set enable_aggregate_lookups: true in the Tyk Dashboard configuration so it reads this pre-computed data, rather than computing it live from traffic logs on every request, and configure matching SQL connection settings, as above. See Pre-Computed vs Live Aggregation for the trade-off.
Unlike MongoDB, there’s no
use_mixed_collection equivalent here, and use_sharded_analytics has no effect on this pump: Tyk Dashboard always reads the single tyk_aggregated table, regardless of Organisation, since SQL storage never splits into per-Organisation tables the way MongoDB can.SQL GraphQL Pump
Thesql-graph pump parses GraphQL-specific detail out of the raw request and response of every GraphQL request, storing it in a dedicated table (tyk_analytics_graph by default): types requested, fields requested per type, operation type, variables, root fields, and any errors.
The resultant table exists for your own downstream querying, such as with a BI tool or data warehouse; Tyk Dashboard doesn’t read this data back out anywhere.
Enable detailed recording first, so that GraphQL information can be parsed from the request body and response; this applies globally across all APIs on the Gateway.
In addition to the common pump settings and common meta fields, the pump has the following settings:
The Standard SQL Pump doesn’t filter out traffic logs for GraphQL requests, so they already reach
tyk_analytics as ordinary, undifferentiated rows. If you point sql-graph’s table_name at the same table as your Standard SQL Pump, you’ll get duplicate rows in the Log Browser for every GraphQL request, one plain copy and one GraphQL-enriched copy, without gaining anything: Log Browser can’t display the extra fields anyway. Always use a separate, dedicated table.SQL GraphQL Aggregate Pump
Thesql-graph-aggregate pump computes aggregated GraphQL-specific analytics, mirroring the SQL Aggregate Pump, and stores them in a fixed table, tyk_graph_aggregated. Unlike the GraphQL pumps that store traffic logs, this one does feed Tyk Dashboard. Unlike Mongo, there’s no mongo-graph-aggregate equivalent.
In addition to the common pump settings and common meta fields, the pump has the following settings:
track_all_paths, ignore_tag_prefix_list, store_analytics_per_minute, and omit_index_creation fields as the SQL Aggregate Pump above. The table name is fixed at tyk_graph_aggregated; there’s no table_name override, though table sharding still applies, storing per-day tables such as tyk_graph_aggregated_20230327.
Tyk Dashboard Setting: powers the Activity by Graph screen (Popularity by Graph API, Errors by Graph API, All Graph APIs). Tyk Dashboard’s Postgres storage driver queries this pump’s table directly; there’s no separate Dashboard config flag to enable it, and it’s PostgreSQL-only.
SQL MCP Pump
Thesql-mcp pump stores MCP tool-call traffic logs in a dedicated table, tyk_analytics_mcp by default, mirroring the Standard SQL Pump. Unlike GraphQL records, MCP records are excluded from the Standard SQL Pump, so this is the only way to get MCP traffic logs into SQL.
As with the SQL GraphQL Pump, Tyk Dashboard doesn’t read this data anywhere, including the Activity by MCP screen; it exists for your own downstream querying.
In addition to the common pump settings and common meta fields, the pump has the following settings:
SQL MCP Aggregate Pump
Thesql-mcp-aggregate pump computes aggregated MCP analytics, mirroring the SQL Aggregate Pump, and stores them in a fixed table, tyk_mcp_aggregated; unlike the SQL MCP Pump, this table name isn’t configurable.
In addition to the common pump settings and common meta fields, the pump has the following settings:
track_all_paths, ignore_tag_prefix_list, store_analytics_per_minute, and omit_index_creation fields as the SQL Aggregate Pump above. The table name is fixed at tyk_mcp_aggregated; there’s no table_name override, though table sharding still applies, storing per-day tables such as tyk_mcp_aggregated_20230327.
Tyk Dashboard Setting: this pump supplies the Activity by MCP screen. enable_aggregate_lookups: true is required in the Tyk Dashboard configuration, not just recommended: unlike the REST case, there’s no working live fallback if it’s left false, since that fallback has no MCP-specific logic and queries the plain REST traffic log table, returning nothing useful. See MCP Analytics for the Dashboard side of this feature.