Pump v1.8
Last updated: 4 minutes read.
1.8.3
Changelog
Fixed
- Corrected configuration for pumps.kafka.meta.timeout to be interpreted as the number of seconds (Type: int) instead of a duration requiring a unit (Type: Duration).
- Fixed an issue where Graph SQL Pump couldn’t restart correctly when analytics storage table name was changed in Pump config. Some relations were not torn down and migrated correctly.
1.8.2
Changelog
Fixed
- Resolved performance issue where SQL Aggregate analytics failed to load on the Dashboard during heavy traffic by introducing a new index on the sql_aggregate Pump called idx_dimension.
- Fixed Prometheus Pump crashes on non UTF-8 URLs by updating to prometheus-client v1.16.
- Fixed MongoDB connection string issues related to certain characters ("?" and “@”), recommending URL-encoded values in usernames and passwords. This ensures compatibility with both mgo and mongo-go drivers.
- Fixed security vulnerabilities: CVE-2022-36640, CVE-2022-21698, GO-2022-0322 and GHSA-cg3q-j54f-5p7p.
Added
- Add
track_all_paths
configuration for Prometheus Pump. If enabled, all APIs will have path in thetyk_http_status_per_path
metric. Otherwise, only endpoint that have “track” plugin set with have path shown in the metric. Endpoints without “track” plugin set will have “unknown” path shown in the metric.
Updated
- Improved security by obfuscating Mongo Pump credentials in log outputs.
1.8.1
Notes on MongoDB v5 and v6 compatibility
For MongoDB v5 and v6 users, please set mongo driver type to mongo-go
.
From pump v1.8.1, the default MongoDB driver it uses is mgo. This is to align with the default MongoDB driver of other Tyk components. This driver supports MongoDB versions up to v4. If you are using a later version of MongoDB v5 or MongoDB v6, please follow this guide to update the driver type to mongo-go.
Changelog
Fixed
- GraphQL analytics records were being excluded from the tyk_analytics collection for Mongo Pump. This has been fixed so that GraphQL analytic records are now included as expected.
- Fixed MongoDB connection issue when using a password with URL escape characters (with mongo-go driver)
- Fixed an issue in Prometheus pump when filtering fields , e.g. API Name, that contain
--
in their value. For example,test--name
. Prometheus Pump filtered the field as two separate instances, e.g.test
&name
, instead of the expectedtest--name
. - When <code>omit_configfile</code> is set to
true
, Pump will not try to load the config file and spit out error logs
Updated
- Updated the default Hybrid Pump RPC pool size from 20 to 5 connections in order to reduce default CPU and memory footprint. See Pump configurations
- Import and use latest storage library v1.0.5
- Updated default MongoDB driver to
mgo
. Follow this guide to update the driver type - Pump name is now case-insensitive. It will override two or more pumps with the same name but in different cases (e.g. Mongo / mongo)
1.8
Release date: 2023-05-04
Major features
Pump 1.8 introduces two new pumps: The GraphQL SQL Aggregate Pump - which allows you to transfer GraphQL transaction logs to SQL; and Resurface Pump - which allows you to transfer data to Resurface.io for context based security analysis.
We have changed the default MongoDB driver from mgo to mongo-go. The new driver supports MongoDB versions greater or equal to v4. If you are using older version of MongoDB v3.x, please follow this guide to update the driver type.
We have also added a config option that allow you to decode the raw requests and responses for all pumps so you don’t need to worry about processing them in your data pipeline. For demo mode, there is now an option to generate future data for your convenience.
In this release, we are using a new Tyk storage library to connect to Mongo DB. This would allow us to switch to use the official Mongo Driver very easily in the future.
Notes on MongoDB v3.x compatibility
In 1.8.0, the default MongoDB driver it use is mongo-go. This driver supports MongoDB versions greater or equal to v4. If you are using older version of MongoDB v3.x, please follow this guide to update the driver type.
Changelog
Added
- Added GraphQL SQL Aggregate Pump.
- Added Resurface Pump - Resurface can provide context-based security analysis for attack and failure triage, root cause, threat and risk identification based on detailed API logs sent from Tyk Pump.
- Add config option raw_request_decoded and raw_response_decoded for decoding from base64 the raw requests/responses fields before writing to Pump. This is useful if you want to search for specific values in the raw request/response. Both are disabled by default. This setting is not available for Mongo and SQL pumps, since the dashboard will decode the raw request/response.
- Add the ability to generate future data in demo mode using –demo-future-data flag.
- Remove critical CVE go.uuid vulnerability
- Use the latest Tyk storage library to connect to Mongo
- Hybrid Pump refactoring - we now have better RPC connection control, testability, and documentation
Fixed
- Std pump does not log accurate time when set to json format
- GraphPump doesn’t include names of queries/mutation and subscriptions called
- Mongo Pump’s connection hangs forever if misconfigured