Percona Monitoring and Management 2.38.0¶
|Release date:||Jul 3, 2023|
|Installation:||Installing Percona Monitoring and Management|
Percona Monitoring and Management (PMM) is an open-source database monitoring, management, and observability solution for MySQL, PostgreSQL, and MongoDB.
Important security updates¶
Oracle Linux 9 as new base OS for PMM
As security is paramount to us, PMM 2.38.0 adds a solid and modernized enterprise-grade foundation to PMM. This release migrates our base operating system on which we build our PMM image from CentOS 7 to Oracle Linux 9. Thus, starting with PMM 2.38.0, PMM is publicly available on Oracle Linux 9.
This transition brings many benefits:
- a large number of CVEs automatically fixed
- improved upstream response time to security-related issues
- better compatibility with new technologies
- modern libraries for our team to build on
Since not all users will be able to seamlessly make the move, we will be rolling this out in phases:
Automated migration paths
The following users will be able to take advantage of this immediately:
New installations of PMM:
Docker/Podman users who do full container replacement upgrades to newer versions of PMM. Your existing
docker runcommands will use the new image.
Manual upgrade options
If you are upgrading through the UI’s Upgrade Now button, you’ll have to wait a little longer for an automated migration path. This includes:
- Docker users that use UI upgrade
- AMI users
- OVF users
If you want to upgrade manually in the meantime, check out the instructions in the PMM is getting a modernized enterprise-grade foundation blog post.
PMM now uses Grafana to 9.2.20, which fixes some critical vulnerabilities. For more information, see Grafana’s release blog.
Updated the mysqld_exporter, node_exporter, postgres_exporter, and mongo_exporter to fix the CVEs.
Due to this migration, we have also updated our experimental Environment Overview Dashboard.
Though we have removed the datasource from PMM, we are still reusing the same name (ClickHouse) to keep the changes minimal. Thus, if you are using the old datasource in your dashboards, ensure that you upgrade the underlying queries that will pull the data from the new datasource.
The following example shows the difference in the query before and after migration.
// Altinity ClickHouse datasource SELECT $timeSeries as t, sum(m_query_time_sum)/sum(m_query_time_cnt) as QLatency, environment FROM $table WHERE $timeFilter and environment != '' GROUP BY t,environment ORDER BY t
// Grafana ClickHouse datasource SELECT period_start as t, sum(m_query_time_sum)/sum(m_query_time_cnt) as QLatency, environment FROM pmm."metrics" WHERE $__timeFilter(period_start) AND environment != '' GROUP BY t, environment ORDER BY t
Altinity data sources used macros like
$table, which are no longer present in the Grafana datasource. As a result, the query needs to be rewritten to use the new macros instead of the old ones.
You will notice that in the example above, the macro
$timeFilter has changed to
$__timeFilter, and now it requires a parameter, which should be a column of type datetime.
Depending on which macros you have used, your migration effort will vary. However, we recommend checking the plugin documentation to apply the new syntax correctly, including macros and template variables.
Improved nodes view on the Inventory page¶
In our continuing endeavor to enhance user experience, we further improved the Inventory page by improving the Nodes tab. With this change, you can get more context on your inventory objects, such as nodes, and more actionable information for exploring and resolving possible issues.
Here is the change as part of PMM 2.38.0:
Check the number of agents running on any particular node. When you click on any node, the UI navigates to the view of agents, which is filtered to display only agents related to that specific node.
To see the details of the agents running:
On the Nodes tab, under the Monitoring column, click OK or Failed, depending on the status of the node that you have selected. A page that provides the user with crucial information regarding the total number of agents deployed on that node is displayed.
View comprehensive information about each agent, including key attributes such as node type, IP address, and associated services. This gives you a complete overview of the nodes at a glance.
Check the health status of the agents to see if they are running or have failed.
Better folder management for enhanced PBM compatibility¶
We’ve added a new Folder field to the Create Backup pages. Use this to set a specific target directory within the selected local or S3-compatible location for the backup.
Organizing backups in folders makes grouping PBM backups for clusters easier, and it improves PMM-PBM (Percona Backup for MongoDB) integration workflows.
The Folder field is automatically populated with the value of the cluster label. You can change this default folder from PMM’s Advanced Settings, but make sure you understand how your custom folder will impact PBM integration workflows.
Sharded cluster MongoDB backup support¶
This release introduces support for working with backups for sharded clusters. PMM handles the backup process end-to-end, but restoring such artifacts is currently possible only via the CLI, using Percona Backup for MongoDB.
For information on restoring sharded backups, check the PBM documentation.
SQL comments support in QAN¶
This feature is still in Technical Preview and is subject to change. We recommend that early adopters use this feature for testing purposes only.
You can now collect labels from the comments placed in SQL queries. You can tag the query by key value in the query comment. This helps in query management and optimization. This feature is currently supported for MySQL and PostgreSQL.
SELECT * /* cluster=’east’ */ FROM city;
- CLI - Pass the parameter
UI - It is enabled by default. To disable, uncheck the Disable comments parsing checkbox in the Additional Options section. This is supported for PostgreSQL and MySQL.
New PMM Agent Down alert template¶
We’ve added a new default alert template to Percona Alerting.
The new PMM Agent down alert template monitors your Node status to notify if the Agent is down. Before using this template:
- ensure the PMM Agent is healthy and present on the Database Node.
- check that PMM Client is on version 2.38 version or later.
PMM now uses Grafana v9.2.20, which includes fixes for some critical vulnerabilities. For more information, see Grafana’s release blog.
- PMM-2852 - [QAN]: Starting with PMM 2.38.0, MySQL and PostgreSQL supports collection of labels from the comments placed in the SQL queries.
- PMM-11961 - [Inventory]: Starting with PMM 2.38.0, we have enhanced the Inventory page by improving the Nodes tab. With this change, you can get more context on your inventory objects, such as node, and more actionable information for exploring and resolving possible issues.
- PMM-11962 - [Inventory]: Starting with PMM 2.38.0, you can filter the nodes and services on the Inventory page using the parameters such as ID, names, status, etc., instead of scrolling through an exhaustive list of nodes or services.
- PMM-12087 - [Inventory]: Added tooltips to explain the meaning of the different service statuses in the Inventory view.
- PMM-9367 - [Backups]: Added support for working with backups for sharded clusters.
- PMM-11250 - [Backup]: Added option to specify target directory for the selected backup location to improve PMM-PBM integration workflows.
- PMM-12038 - [Backup]: Extended support for MongoDB backups to cover all authentication mechanisms available for MongoDB (including LDAP, x509 certs).
- PMM-9544 - [Alerting]: Added new default alert template to Node status and notify if the Agent is down.
- PMM-12114 - Upgraded Grafana to 9.2.18 to fix to CVEs.
- PMM-12136 - We have migrated our base operating system on which we build our base image from CentOS 7 to Oracle Linux 9 to fix the CVEs.
- PMM-11578 - [Alerting]: The Last notified column under Alerting > Fired Alerts has been renamed to Last triggered. This better illustrates the fact that triggered alerts may not always generate a notification. This can happen when the contact points are not correctly configured. In addition, the Suppressed status in the State column has been renamed to Silenced to keep consistency with how this status is displayed on other Alerting pages.
- PMM-12067 - [QAN]: Fixed vulnerability in Vitess package.
- PMM-12105 - [QAN]: Fixed an issue where QAN was broken due to wrong pgsm_query_ids.
- PMM-10063 - [QAN]: The QAN filter section could display the available list of services/nodes. But the load time was too much, with the UI being unresponsive. This issue has been resolved now. The entire list is loaded with the count displayed on the upper right side.
- PMM-11386 - [QAN]: MongoDB QAN query time metrics was showing the wrong unit. It should be ms. The issue has been resolved now.
- PMM-12024 - [QAN]: Fixed an issue where a QAN agent couldn’t start due to an expected return value type.
- PMM-12091 - [QAN]: The tooltip for QAN > MongoDB > Query Details was displayed in seconds instead of decimals. The issue has been fixed now.
- PMM-9844 - [QAN]: Data for a large number of partitions or possible indexes was shown in the QAN Details tab in one scrolling column making it practically impossible to read. The issue has been resolved now.
- PMM-11938 - [QAN]: Fixed the problem when PMM tried to create
pg_stat_monitor_settingsview all over again.
- PMM-11645 - Fixed mysql service detection when this is not loaded in systemd.
- PMM-9541 - PostgreSQL exporter now works as expected with
- PMM-10799 - Now in the EXPLAIN/TABLE tab you can get results even for queries with table alias.
- PMM-11692 - On the MySQL Instances Overview dashboard, the
Top MySQL Active Client Threadspanel color was misleading. The issue has been resolved now.
- PMM-11715 - CPU Utilization Graph for RDS instances was not matching what Cloudwatch reported. The issue has been resolved now.
- PMM-11950 - Non-admin roles are calling restricted endpoints after login or page refresh.
- PMM-12085 - Fixed an issue where SSH access to AMI and OVF images for Oracle Linux 9-based images did not work. The issue has been resolved now.
- PMM-12118 - Addressed the Grafana ClickHouse DS plugin CVEs.
- PMM-12119 - When a user successfully adds an RDS service to PMM, the RDS exporter’s status does not change from “UNKNOWN” (the initial status of all exporters) to “RUNNING.” This issue has been resolved.
- PMM-9224 - Data from Arbiter nodes is now correctly displayed on the MongoDB ReplSet Summary dashboard.