MySQL Cluster 8.0.21 has been released
Posted by: Jocelyn Ramilison
Date: July 13, 2020 06:43AM
Date: July 13, 2020 06:43AM
Dear MySQL Users,
MySQL Cluster is the distributed, shared-nothing variant of MySQL.
This storage engine provides:
- In-Memory storage - Real-time performance (with optional
checkpointing to disk)
- Transparent Auto-Sharding - Read & write scalability
- Active-Active/Multi-Master geographic replication
- 99.9999% High Availability with no single point of failure
and on-line maintenance
- NoSQL and SQL APIs (including C++, Java, http, Memcached
and JavaScript/Node.js)
MySQL Cluster 8.0.21 has been released and can be downloaded from
http://www.mysql.com/downloads/cluster/
where you will also find Quick Start guides to help you get your
first MySQL Cluster database up and running.
The release notes are available from
http://dev.mysql.com/doc/relnotes/mysql-cluster/8.0/en/index.html
MySQL Cluster enables users to meet the database challenges of next
generation web, cloud, and communications services with uncompromising
scalability, uptime and agility.
More details can be found at
http://www.mysql.com/products/cluster/
Enjoy !
Changes in MySQL NDB Cluster 8.0.21 (2020-07-13, General Availability)
MySQL NDB Cluster 8.0.21 is a new release of NDB 8.0, based
on MySQL Server 8.0 and including features in version 8.0 of
the NDB storage engine, as well as fixing recently discovered
bugs in previous NDB Cluster releases.
Obtaining NDB Cluster 8.0. NDB Cluster 8.0 source code and
binaries can be obtained from
https://dev.mysql.com/downloads/cluster/.
For an overview of changes made in NDB Cluster 8.0, see What
is New in NDB Cluster
(https://dev.mysql.com/doc/refman/8.0/en/mysql-cluster-what-is-new.html).
This release also incorporates all bug fixes and changes made
in previous NDB Cluster releases, as well as all bug fixes
and feature changes which were added in mainline MySQL 8.0
through MySQL 8.0.21 (see Changes in MySQL 8.0.21 (2020-07-13,
General Availability)
(https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-21.html)).
Packaging Notes
* For Windows, MSI installer packages for MySQL Cluster now
include a check for the required Visual Studio
redistributable package, and produce a message asking the
user to install it if it is missing. (Bug #30541398)
Functionality Added or Changed
* NDB Disk Data: An initial restart of the cluster now
causes the removal of all NDB tablespaces and log file
groups from the NDB dictionary and the MySQL data
dictionary. This includes the removal of all data files
and undo log files associated with these objects. (Bug
#30435378)
References: See also: Bug #29894166.
* The status variable Ndb_metadata_blacklist_size is now
deprecated, and is replaced in NDB 8.0.22 by
Ndb_metadata_excluded_count. (Bug #31465469)
* It now possible to consolidate data from separate
instances of NDB Cluster into a single target NDB Cluster
when the original datasets all use the same schema. This
is supported when using backups created using START
BACKUP in ndb_mgm and restoring them with ndb_restore,
using the --remap-column option implemented in this
release (along with --restore-data and possibly other
options). --remap-column can be employed to handle cases
of overlapping primary, unique, or both sorts of key
values between source clusters, and you need to make sure
that they do not overlap in the target cluster. This can
also be done to preserve other relationships between
tables.
When used together with --restore-data, the new option
applies a function to the value of the indicated column.
The value set for this option is a string of the format
db.tbl.col:fn:args, whose components are listed here:
+ db: Database name, after performing any renames.
+ tbl: Table name.
+ col: Name of the column to be updated. This column's
type must be one of INT or BIGINT, and can
optionally be UNSIGNED.
+ fn: Function name; currently, the only supported
name is offset.
+ args: The size of the offset to be added to the
column value by offset. The range of the argument is
that of the signed variant of the column's type;
thus, negative offsets are supported.
You can use --remap-column for updating multiple columns
of the same table and different columns of different
tables, as well as combinations of multiple tables and
columns. Different offset values can be employed for
different columns of the same table.
As part of this work, two new options are also added to
ndb_desc in this release:
+ --auto-inc (short form -a): Includes the the next
auto-increment value in the output, if the table has
an AUTO_INCREMENT column.
+ --context (short form -x): Provides extra
information about the table, including the schema,
database name, table name, and internal ID.
These options may be useful for obtaining information
about NDB tables when planning a merge, particularly in
situations where the mysql client may not be readily
available.
For more information, see the descriptions for
--remap-column, --auto-inc, and --context. (Bug
#30383950)
* Detailed real-time information about the state of
automatic metadata mismatch detection and synchronization
can now be obtained from tables in the MySQL Performance
Schema. These two tables are listed here:
+ ndb_sync_pending_objects: Contains information about
NDB database objects for which mismatches have been
detected between the NDB dictionary and the MySQL
data dictionary. It does not include objects which
have been excluded from mismatch detection due to
permanent errors raised when attempting to
synchronize them.
+ ndb_sync_excluded_objects: Contains information
about NDB database objects which have been excluded
because they cannot be synchronized between the NDB
dictionary and the MySQL data dictionary, and thus
require manual intervention. These objects are no
longer subject to mismatch detection until such
intervention has been performed.
In each of these tables, each row corresponds to a
database object, and contains the database object's
parent schema (if any), the object's name, and the
object's type. Types of objects include schemas,
tablespaces, log file groups, and tables. The
ndb_sync_excluded_objects table shows in addition to this
information the reason for which the object has been
excluded.
Performance Schema NDB Cluster Tables
(https://dev.mysql.com/doc/refman/8.0/en/performance-schema-ndb-cluster-tables.html)
, provides further information about these Performance
Schema tables. (Bug #30107543)
* ndb_restore now supports different primary key
definitions for source and target tables when restoring
from an NDB native backup, using the --allow-pk-changes
option introduced in this release. Both increasing and
decreasing the number of columns making up the original
primary key are supported. This may be useful when it is
necessary to accommodate schema version changes while
restoring data, or when doing so is more efficient or
less time-consuming than performing ALTER TABLE
statements involving primary key changes on a great many
tables following the restore operation.
When extending a primary key with additional columns, any
columns added must not be nullable, and any values stored
in any such columns must not change while the backup is
being taken. Changes in the values of any such column
while trying to add it to the table's primary key causes
the restore operation to fail. Due to the fact that some
applications set the values of all columns when updating
a row even if the values of one or more of the columns
does not change, it is possible to override this behavior
by using the --ignore-extended-pk-updates option which is
also added in this release. If you do this, care must be
taken to insure that such column values do not actually
change.
When removing columns from the table's primary key, it is
not necessary that the columns dropped from the primary
key remain part of the table afterwards.
For more information, see the description of the
--allow-pk-changes option in the documentation for
ndb_restore. (Bug #26435136, Bug #30383947, Bug
#30634010)
* Added the --ndb-log-fail-terminate option for mysqld.
When used, this causes the SQL node to terminate if it is
unable to log all row events. (Bug #21911930)
References: See also: Bug #30383919.
* When a scalar subquery has no outer references to the
table to which the embedding condition is attached, the
subquery may be evaluated independent of that table; that
is, the subquery is not dependent. NDB now attempts to
identify and evaluate such a subquery before trying to
retrieve any rows from the table to which it is attached,
and to use the value thus obtained in a pushed condition,
rather than using the subquery which provided the value.
* In MySQL 8.0.17 and later, the MySQL Optimizer transforms
NOT EXISTS and NOT IN queries into antijoins. NDB can now
push these down to the data nodes.
This can be done when there is no unpushed condition on
the table, and the query fulfills any other conditions
which must be met for an outer join to be pushed down.
Bugs Fixed
* Important Change; NDB Disk Data: An online change of
tablespace is not supported for NDB tables. Now, for an
NDB table, the statement ALTER TABLE ndb_table ...
ALGORITHM=INPLACE, TABLESPACE=new_tablespace is
specifically disallowed.
As part of this fix, the output of the ndb_desc utility
is improved to include the tablespace name and ID for an
NDB table which is using one. (Bug #31180526)
* The wrong index was used in the array of indexes while
dropping an index. For a table with 64 indexes this
caused uninitialized memory to be released. This problem
also caused a memory leak when a new index was created at
any later time following the drop. (Bug #31408095)
* Removed an unnecessary dependency of ndb_restore on the
NDBCLUSTER plugin. (Bug #31347684)
* Objects for which auto-synchronization fails due to
temporary errors, such as failed acquisitions of metadata
locks, are simply removed from the list of detected
objects, making such objects eligible for detection in
later cycles in which the synchronization is retried and
hopefully succeeds. This best-effort approach is suitable
for the default auto-synchronization behaviour but is not
ideal when the using the ndb_metadata_sync system
variable, which triggers synchronization of all metadata,
and when synchronization is complete, is automatically
set to false to indicate that this has been done.
What happened, when a temporary error persisted for a
sizable length of time, was that metadata synchronization
could take much longer than expected and, in extreme
cases, could hang indefinitely, pending user action. One
such case occurred when using ndb_restore with the
--disable-indexes option to restore metadata, when the
synchronization process entered a vicious cycle of
detection and failed synchronization attempts due to the
missing indexes until the indexes were rebuilt using
ndb_restore --rebuild-indexes.
The fix for this issue is, whenever ndb_metadata_sync is
set to true, to exclude an object after synchronization
of it fails 10 times with temporary errors by promoting
these errors to a permanent error, in order to prevent
stalling. This is done by maintaining a list of such
objects, this list including a count of the number of
times each such object has been retried. Validation of
this list is performed during change detection in a
similar manner to validation of the exclusion list. (Bug
#31341888)
* 32-bit platforms are not supported by NDB 8.0. Beginning
with this release, the build process checks the system
architecture and aborts if it is not 64-bit. (Bug
#31340969)
* Page-oriented allocations on the data nodes are divided
into nine resource groups, some having pages dedicated to
themselves, and some having pages dedicated to shared
global memory which can be allocated by any resource
group. To prevent the query memory resource group from
depriving other, more important resource groups such as
transaction memory of resources, allocations for query
memory are performed with low priority and are not
allowed to use the last 10% of shared global memory. This
change was introduced by poolification work done in NDB
8.0.15.
Subsequently, it was observed that the calculation for
the number of pages of shared global memory kept
inaccessible to query memory was correct only when no
pages were in use, which is the case when the LateAlloc
data node parameter is disabled (0).
This fix corrects that calculation as performed when
LateAlloc is enabled. (Bug #31328947)
References: See also: Bug #31231286.
* Multi-threaded restore is able to drive greater cluster
load than the previous single-threaded restore,
especially while restoring of the data file. To avoid
load-related issues, the insert operation parallelism
specified for an ndb_restore instance is divided equally
among the part threads, so that a multithreaded instance
has a similar level of parallelism for transactions and
operations to a single-threaded instance.
An error in division caused some part threads to have
lower insert operation parallelism than they should have,
leading to an slower restore than expected. This fix
ensures all part threads in a multi-threaded ndb_restore
instance get an equal share for parallelism. (Bug
#31256989)
* DUMP 1001 (DumpPageMemoryOnFail) now prints out
information about the internal state of the data node
page memory manager when allocation of pages fails due to
resource constraints. (Bug #31231286)
* Statistics generated by NDB for use in tracking internal
objects allocated and deciding when to release them were
not calculated correctly, with the result that the
threshold for resource usage was 50% higher than
intended. This fix corrects the issue, and should allow
for reduced memory usage. (Bug #31127237)
* An excessive number of entries were written to the backup
log when performing an online backup, using ndbmtd on the
data nodes. This happened because row changes occurring
to fragments managed by LDM instance 1 were always
recorded in the log, even when they were for primary
fragments other than primary fragments. This wasted
resources when restoring from the backup, and could cause
other problems when doing so, such as when using staging
tables for schema transforms while executing ndb_restore.
(Bug #31034270)
* The Dojo toolkit included with NDB Cluster and used by
the Auto-Installer was upgraded to version 1.15.3. (Bug
#31029110)
* A packed version 1 configuration file returned by
ndb_mgmd could contain duplicate entries following an
upgrade to NDB 8.0, which made the file incompatible with
clients using version 1. This occurs due to the fact that
the code for handling backwards compatibility assumed
that the entries in each section were already sorted when
merging it with the default section. To fix this, we now
make sure that this sort is performed prior to merging.
(Bug #31020183)
* When executing any of the SHUTDOWN, ALL STOP, or ALL
RESTART management commands, it is possible for different
nodes to attempt to stop on different global checkpoint
index (CGI) boundaries. If they succeed in doing so, then
a subsequent system restart is slower than normal because
any nodes having an earlier stop GCI must undergo
takeover as part of the process. When nodes failing on
the first GCI boundary cause surviving nodes to be
nonviable, surviving nodes suffer an arbitration failure;
this has the positive effect of causing such nodes to
halt at the correct GCI, but can give rise to spurious
errors or similar.
To avoid such issues, extra synchronization is now
performed during a planned shutdown to reduce the
likelihood that different data nodes attempt to shut down
at different GCIs as well as the use of unnecessary node
takeovers during system restarts. (Bug #31008713)
* During an upgrade, a client could connect to an NDB 8.0
data node without specifying a multiple transporter
instance ID, so that this ID defaulted to -1. Due to an
assumption that this would occur only in the Node
starting state with a single transporter, the node could
hang during the restart. (Bug #30899046)
* When an NDB cluster was upgraded from a version that does
not support the data dictionary to one that does, any DDL
executed on a newer SQL node was not properly distributed
to older ones. In addition, newer SDI generated during
DDL execution was ignored by any data nodes that had not
yet been upgraded. These two issues led to schema states
that were not consistent between nodes of different NDB
software versions.
We fix this problem by blocking any DDL affecting NDB
data objects while an upgrade from a previous NDB version
to a version with data dictionary support is ongoing.
(Bug #30877440)
References: See also: Bug #30184658.
* The mysql.ndb_schema table, used internally for schema
distribution among SQL nodes, has been modified in NDB
8.0. When a cluster is being upgraded from a older
version of NDB, the first SQL node to be upgraded updates
the definition of this table to match that used by NDB
8.0 GA releases. (For this purpose, NDB now uses 8.0.21
as the cutoff version.) This is done by dropping the
existing table and re-creating it using the newer
definition. SQL nodes which have not yet been upgraded
receive this ndb_schema table drop event and enter
read-only mode, becoming writable again only after they
are upgraded.
To keep SQL nodes running older versions of NDB from
going into read-only mode, we change the upgrade behavior
of mysqld such that the ndb_schema table definition is
updated only if all SQL nodes connected to the cluster
are running an 8.0 GA version of NDB and thus having the
updated ndb_schema table definition. This means that,
during an upgrade to the current or any later version, no
MySQL Server that is being upgraded updates the
ndb_schema table if there is at least one SQL node with
an older version connected to the cluster. Any SQL node
running an older version of NDB remains writable
throughout the upgrade process. (Bug #30876990, Bug
#31016905)
* When mysqld was upgraded to a version that used a new SDI
version, all NDB tables become inaccessible. This was
because, during an upgrade, synchronization of NDB tables
relies on deserializing the SDI packed into the NDB
Dictionary; if the SDI format was of an version older
than that used prior to the upgrade, deserialization
could not take place if the format was not the same as
that of the new version, which made it impossible to
create a table object in the MySQL data dictionary.
This is fixed by making it possible for NDB to bypass the
SDI version check in the MySQL server when necessary to
perform deserialization as part of an upgrade. (Bug
#30789293, Bug #30825260)
* When responding to a SCANTABREQ, an API node can provide
a distribution key if it knows that the scan should work
on only one fragment, in which case the distribution key
should be the fragment ID, but in some cases a hash of
the partition key was used instead, leading to failures
in DBTC. (Bug #30774226)
* Several memory leaks found in ndb_import have been
removed. (Bug #30756434, Bug #30727956)
* The master node in a backup shut down unexpectedly on
receiving duplicate replies to a DEFINE_BACKUP_REQ
signal. These occurred when a data node other than the
master errored out during the backup, and the backup
master handled the situation by sending itself a
DEFINE_BACKUP_REF signal on behalf of the missing node,
which resulted in two replies being received from the
same node (a CONF signal from the problem node prior to
shutting down and the REF signal from the master on
behalf of this node), even though the master expected
only one reply per node. This scenario was also
encountered for START_BACKUP_REQ and STOP_BACKUP_REQ
signals.
This is fixed in such cases by allowing duplicate replies
when the error is the result of an unplanned node
shutdown. (Bug #30589827)
* When processing a CSV file, ndb_import did not accept
trailing field terminators at the ends of lines that were
accepted by mysqlimport. (Bug #30434663)
* When updating NDB_TABLE comment options using ALTER
TABLE, other options which has been set to non-default
values when the table was created but which were not
specified in the ALTER TABLE statement could be reset to
their defaults.
See Setting NDB_TABLE Options
(https://dev.mysql.com/doc/refman/8.0/en/create-table-ndb-table-comment-options.html)
, for more information. (Bug #30428829)
* Removed a memory leak found in the ndb_import utility.
(Bug #29820879)
* Incorrect handling of operations on replicas during node
restarts could result in a forced shutdown, or in content
diverging between replicas, when primary keys with
nonbinary (case-sensitive) equality conditions were used.
(Bug #98526, Bug #30884622)
On Behalf of Oracle/MySQL Release Engineering Team,
Hery Ramilison
Subject
Views
Written By
Posted
Sorry, you can't reply to this topic. It has been closed.
Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.