MySQL Shell 8.0.16 for MySQL Server 8.0 and 5.7 has been released

[Nawaz Nazeer ahamed]

Dear MySQL users,

MySQL Shell 8.0.16 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under
Oracle's dual-license.

MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and
5.7. Please upgrade to MySQL Shell 8.0.16.

MySQL Shell is an interactive JavaScript, Python and SQL console
interface, supporting development and administration for the MySQL
Server. It provides APIs implemented in JavaScript and Python that
enable you to work with MySQL InnoDB cluster and use MySQL as a document
store.

The AdminAPI enables you to work with MySQL InnoDB cluster, providing an
integrated solution for high availability and scalability using InnoDB
based MySQL databases, without requiring advanced MySQL expertise. For
more information about how to configure and work with MySQL InnoDB
cluster see

  https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-userguide.html

The X DevAPI enables you to create "schema-less" JSON document
collections and perform Create, Update, Read, Delete (CRUD) operations
on those collections from your favorite scripting language.  For more
information about how to use MySQL Shell and the MySQL Document Store
support see

  https://dev.mysql.com/doc/refman/en/document-store.html

For more information about the X DevAPI see

  https://dev.mysql.com/doc/x-devapi-userguide/en/

If you want to write applications that use the the CRUD based X DevAPI
you can also use the latest MySQL Connectors for your language of
choice. For more information about Connectors see

  https://dev.mysql.com/doc/index-connectors.html

For more information on the APIs provided with MySQL Shell see

  https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/

and

  https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/

Using MySQL Shell's SQL mode you can communicate with servers using the
legacy MySQL protocol. Additionally, MySQL Shell provides partial
compatibility with the mysql client by supporting many of the same
command line options.

For full documentation on MySQL Server, MySQL Shell and related topics,
see

  https://dev.mysql.com/doc/mysql-shell/8.0/en/

For more information about how to download MySQL Shell 8.0.16, see the
"Generally Available (GA) Releases" tab at

  http://dev.mysql.com/downloads/shell/

We welcome and appreciate your feedback and bug reports, see

  http://bugs.mysql.com/

Enjoy and thanks for the support!

Changes in MySQL Shell 8.0.16 (2019-04-25, General Availability)

     * Functionality Added or Changed

     * Bugs Fixed

Functionality Added or Changed


     * Important Change: Attempting to connect to an X Protocol
       port, 33060 by default, using the classic MySQL protocol resulted
       in the following error: ERROR 2013 (HY000): Lost connection to
       MySQL server at 'reading initial communication packet', system
       error: 0 This was because of differences in X Protocol and
       classic MySQL protocol clients expectations on how connections
       were initialized. Now, in such a situation the generated error
       message is ERROR 2007 (HY000): Protocol mismatch; server version
       = 11, client version = 10. If you encounter this error then you
       are probably trying to use the wrong port for the protocol your
       client is using.  As part of this improvement the
       mysqlx_enable_hello_notice system variable has been added, which
       controls messages sent to classic MySQL protocol clients that try
       to connect over X Protocol.  When enabled, clients which do not
       support X Protocol that attempt to connect to the server X
       Protocol port receive an error explaining they are using the
       wrong protocol. Set mysqlx_enable_hello_notice to false to permit
       clients which do not recognize the hello message to still
       connect.

     * MySQL Shell's upgrade checker utility can now check the
       configuration file (my.cnf or my.ini) for the server instance.
       The utility checks for any system variables that are defined in
       the configuration file but have been removed in the target MySQL
       Server release, and also for any system variables that are not
       defined in the configuration file and will have a different
       default value in the target MySQL Server release. For these
       checks, when you invoke checkForServerUpgrade(), you must provide
       the file path to the configuration file. If you omit the file
       path and the upgrade checker utility needs to run a check that
       requires the configuration file, that check fails with a message
       informing you that you must specify the file path. (Bug
       #27801824, Bug #29222179)

     * MySQL InnoDB cluster automatically and transparently
       manages the communication protocol versions of its members,
       whenever the cluster topology is changed using AdminAPI
       operations. An InnoDB cluster always uses the most recent
       communication protocol version that is supported by all instances
       that are part of the cluster or joining it.

          + When an instance is added to, removed from, or
            rejoins the cluster, or a rescan or reboot operation
            is carried out on the cluster, the communication
            protocol version is automatically set to a version
            supported by the instance that is now at the
            earliest MySQL Server version.

          + When you carry out a rolling upgrade by removing
            instances from the cluster, upgrading them, and
            adding them back into the cluster, the communication
            protocol version is automatically upgraded when the
            last remaining instance at the old MySQL Server
            version is removed from the cluster prior to its
            upgrade.
       To see the communication protocol version in use in an
       InnoDB cluster, use the Cluster.status() function with
       the 'extended' option enabled. The communication protocol
       version is returned in the 'GRProtocolVersion' field,
       provided that the cluster has quorum and no cluster
       members are unreachable.

     * MySQL Shell now has a framework and commands that you can
       use to set up and run reports to display live information from a
       MySQL server, such as status and performance information. Reports
       can be run once using the MySQL Shell \show command, or run then
       refreshed continuously in a MySQL Shell session using the \watch
       command. They can also be accessed as API functions in the
       shell.reports object.  The reporting facility supports both
       built-in reports and user-defined reports. User-defined reports
       can be created in the supported scripting languages JavaScript
       and Python, and can be run in any MySQL Shell mode (JavaScript,
       Python, or SQL), regardless of the language that the report was
       written in. Reports can be saved in a folder in the MySQL Shell
       configuration path and automatically loaded at startup. You can
       also create a report directly in the MySQL Shell prompt. You
       register a report to MySQL Shell using the shell.registerReport
       method to provide information about the report and the options
       and arguments that it supports.  For more information, see
       Reporting with MySQL Shell
 (http://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-shell-reporting.html).

     * When running MySQL Shell in interactive mode, you can now
       execute an SQL statement without switching to SQL mode and back
       again afterwards. This function enables you to conveniently issue
       some SQL statements in the context of a longer AdminAPI workflow
       in JavaScript or Python mode.  Use the \sql command immediately
       followed by the SQL statement, for example:
\sql select * from sakila.actor limit 3;
       The SQL statement does not need any additional quoting, and the
       statement delimiter is optional. With this format, MySQL Shell
       does not switch mode as it would if you entered the \sql command.
       After the SQL statement has been executed, MySQL Shell remains in
       JavaScript or Python mode.
       You cannot use multiple line mode when you use the \sql command
       with a query to execute single SQL statements while another
       language is active. The command only accepts a single SQL query
       on a single line.

     * MySQL Shell history is now split per active language
       which the command was issued under. This means that your history
       now matches the active language, for example when you are running
       in JavaScript mode having issued \js, the history contains the
       previous JavaScript statements you issued, and when you issue
       \sql to change to SQL mode your history contains the previous SQL
       statements you issued. Similarly, now any history related
       commands such as \history clear or \history delete are performed
       on the history of the current active language. When you install
       this version, any existing MySQL Shell history files are
       duplicated to ensure that existing history is not lost.
       Subsequent operations are then added to the language specific
       history file.

     * The new autoRejoinTries option enables you to configure
       how many times an instance tries to rejoin a group after being
       expelled. In scenarios where network glitches happen but recover
       quickly, setting this option prevents you from having to manually
       add the expelled instance back to the group. The autoRejoinTries
       option accepts positive integer values between 0 and 2016 and the
       default value is 0, which means that instances do not try to
       automatically rejoin. Set the value to a valid integer to
       configure the number of attempts expelled instances should make
       to rejoin the group. You can pass the autoRejoinTries option to
       these AdminAPI operations:

          + dba.createCluster()

          + Cluster.addInstance()

          + Cluster.setOption()

          + Cluster.setInstanceOption()
       When you configure the autoRejoinTries option, it sets
       the group_replication_autorejoin_tries system variable.
       Passing the option to dba.createCluster(),
       Cluster.addInstance() or Cluster.setInstanceOption()
       configures the automatic rejoin for specific cluster
       instances. Passing the option to Cluster.setOption()
       configures the automatic rejoin for all cluster
       instances.
       For more information, see Responses to Failure Detection
       and Network Partitioning
 (http://dev.mysql.com/doc/refman/8.0/en/group-replication-responses-failure.html).

     * When resultFormat was set to json or json/raw, every
       result was being returned as a JSON document. This behavior was
       expected when JSON wrapping is off (in other words the --json
       command option was not used when starting MySQL Shell). Now, for
       consistency reasons when JSON wrapping is off and resultFormat is
       set to json or json/raw, every record is printed in a separate
       document and statistics and warnings are printed in plain text.
       For example if MySQL Shell is started without --json and
       resultFormat=json/raw:
mysqlsh-sql> SHOW DATABASES;
{"Database":"information_schema"}
{"Database":"mysql"}
{"Database":"performance_schema"}
{"Database":"sys"}
4 rows in set (0.0035 sec)

       If MySQL Shell is started with --json and with
       resultFormat=json/raw:
mysqlsh-sql> SHOW DATABASES;
{
    "hasData": true,
    "rows": [
        {
            "Database": "information_schema"
        },
        {
            "Database": "mysql"
        },
        {
            "Database": "performance_schema"
        },
        {
            "Database": "sys"
        }
    ],
    "executionTime": "0.0018 sec",
    "affectedRowCount": 0,
    "affectedItemsCount": 0,
    "warningCount": 0,
    "warningsCount": 0,
    "warnings": [],
    "info": "",
    "autoIncrementValue": 0
}


     * AdminAPI now reports information about the version of
       MySQL running on instances. This information is available from
       the following operations:

          + Cluster.status()

          + Cluster.describe()

          + Cluster.rescan()

       See Checking the MySQL Version on Instances
 (http://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-cluster-working-
 with-cluster.html#checking-version-on-instances) for
       more information.

Bugs Fixed


     * Removing an instance from a cluster when the instance to
       be removed had no user defined for the group_replication_recovery
       channel resulted in dropping users on the remaining instances of
       the cluster. (Bug #29617572)

     * MySQL Shell could be installed in an environment where
       Python was not present, but the application has a dependency on
       many standard Python modules, resulting in error messages at
       startup. The RPM and Debian packages for MySQL Shell now
       explicitly specify the dependency on Python. (Bug #29469201)

     * The MSI file that is used by Windows Installer to install
       MySQL Shell now adds the path to the application binary (mysqlsh)
       to the Windows PATH environment variable, so that the application
       can be started from a command prompt. (Bug #29457639)

     * In the instructions to build MySQL Shell from source (the
       INSTALL document), the required version of the optional V8
       dependency has been updated from 3.28.71.19 to 6.7.288.46. Thanks
       to Evgeniy Patlan for spotting this.  (Bug #29430049, Bug #94529)

     * The failoverConsistency option has been deprecated and a
       new option named consistency has been added, to make it more
       consistent with the target Group Replication
       group_replication_consistency system variable name. The MySQL
       Shell online documentation now also correctly describes all of
       the values you can assign to the consistency option. (Bug
       #29356599)

     * The dba.configureLocalInstance() operation would remove
       any section that did not start with mysqld from the provided
       option file. This could remove sections such as the client
       section from the option file. (Bug #29349014)

     * MySQL Shell's upgrade checker utility
       checkForServerUpgrade() could incorrectly report a schema
       inconsistency error for a table whose name included a special
       character such as a hyphen. (Bug #29346836, Bug #94303)

     * When an instance with X Plugin disabled was added to an
       InnoDB cluster, if the instance was later removed from the
       cluster using Cluster.removeInstance() the operation failed with
       LogicError "get_string(7): field is NULL".  This was a regression
       introduced by the fix for Bug#27677227. (Bug #29304183)

     * There was an inconsistency between the behavior of
       dba.checkInstanceConfiguration() and the commands to add
       instances to the cluster (dba.createCluster() and
       Cluster.addInstance()) regarding the localhost and loopback
       address validation. In particular, a simple error was printed by
       dba.checkInstanceConfiguration() but the execution of the
       operation continued showing that everything was correct at the
       end of the operation, while an error was issued and the execution
       stopped for dba.createCluster() and Cluster.addInstance().  As
       part of fixing this issue, it was decided that the existing
       localhost and loopback address validations are no longer needed
       and should be removed. In particular, whatever address is
       specified for report_host, even if it is localhost or the
       loopback address (127.0.0.1), should be allowed, because it was
       explicitly specified by the user to use it. (Bug #29279941)

     * The dba.rebootClusterFromCompleteOutage() operation was
       not preserving the existing Group Replication configurations
       previously set for the instances. In particular, the Group
       Replication local address and exit state action values were being
       changed. Now all settings are read at the time of rebooting the
       cluster. (Bug #29265869)

     * On Windows, MySQL Shell's upgrade checker utility
       checkForServerUpgrade() incorrectly reported a schema
       inconsistency error for partitioned tables. (Bug #29256562)

     * Using either Cluster.setOption() or
       Cluster.setInstanceOption() to set an option which only exists in
       MySQL 8.0 on an instance running MySQL 5.7 was not being caught
       correctly. (Bug #29246657)

     * On Debian-based platforms (such as Ubuntu), if the
       hostname resolved to 127.0.1.1 - which is the default on these
       platforms - it was not possible to create a cluster using the
       default settings. Now, in such situations a proper validation of
       the instance is performed before creating a cluster and adding
       instances to it. (Bug #29246110)

     * MySQL Shell stopped unexpectedly if Python code was
       running in interactive mode and threw exceptions from C++
       libraries. These exceptions are now caught and translated to
       Python's built-in RuntimeError exceptions. (Bug #29057116)

     * The dba.checkInstanceConfiguration() operation did not
       validate host restrictions for the account provided for cluster
       administration, for example if the account could actually connect
       to all of the instances in the cluster.  In particular, now an
       error is issued if the provided user account is only able to
       connect through localhost.  (Bug #29018457)

     * When a connection is specified using key-value pairs in
       MySQL Shell's shell.connect() method, the host name cannot be an
       empty string. MySQL Shell now handles this situation consistently
       and returns an error if the supplied host name is an empty
       string. (Bug #28899522)

     * InnoDB cluster configured auto_increment_increment and
       auto_increment_offset on instances for clusters running in
       multi-primary mode and consisting of up to 7 instances based on
       the logic described at InnoDB cluster and Auto-increment
 (http://dev.mysql.com/doc/refman/8.0/en/mysql-innodb-cluster-working-with-
 cluster.html#mysql-innodb-cluster-auto-increment). But Group
       Replication permits groups to contain up to 9 members, and
       Cluster.addInstance() and Cluster.removeInstance() were not
       following the logic used for other operations. Now, InnoDB
       cluster uses the same logic for auto increment regardless of the
       operation used and correctly handles multi-primary clusters with
       more than 7 instances. (Bug #28812763)

     * MySQL Shell's JSON import utility can now accept input
       from FIFO special files (named pipes) when you invoke the utility
       using the util.importJSON function, so you can carry out large
       imports by this method without needing to put the data into a
       file. (Bug #28785527)

     * When you use the MySQL Shell command \help (or \h, or \?)
       with a search pattern to search for help on a specific subject,
       multiple help topic titles can match the pattern and be returned
       as a list, to be selected by entering the command again with an
       extended search pattern. With this system, it was possible for
       help topics with a single-word title to be inaccessible from such
       a list because there was nothing further to add to the search
       pattern. To avoid this situation, the handling of multiple
       matches has now been improved. If a topic title is found that
       matches the given search pattern exactly (case-sensitive in the
       event of multiple topic matches, and case-insensitive in the
       event of no case-sensitive matches), the topic is identified as
       the exact match and its help data is printed. The rest of the
       topics with pattern matches in their titles are listed in a "see
       also" section and can be selected by further pattern matching.
       (Bug #28393119)

     * MySQL Shell uses the host value of the provided
       connection parameters as the target hostname used for AdminAPI
       operations, namely to register the instance in the metadata (for
       the dba.createCluster() and cluster.addInstance() operations).
       However, the host used for the connection parameters might not
       match the hostname that is used or reported by Group Replication,
       which uses the value of the report_host system variable when it
       is defined (in other words it is not NULL), otherwise the value
       of hostname is used. Therefore, AdminAPI now follows the same
       logic to register the target instance in the metadata and as the
       default value for the group_replication_local_address variable on
       instances, instead of using the host value from the instance
       connection parameters. During this fix it was detected that when
       the report_host variable was set to empty, Group Replication uses
       an empty value for the host but AdminAPI (for example in commands
       such as dba.checkInstanceConfiguration(),
       dba.configureInstance(), dba.createCluster()) reports the
       hostname as the value used which is inconsistent with the value
       reported by Group Replication. An error is now issued by AdminAPI
       if an empty value is set for the report_host system variable.
       (Bug #28285389)

     * In the event that dba.createCluster() failed and a
       rollback was performed to remove the created replication
       (recovery) users, the account created at localhost and any of the
       ipWhitelist addresses were not being removed.  The fix ensures
       that the replication accounts are removed whenever a rollback
       related to dba.createCluster() is performed. This work was based
       on a code contribution from Bin Hong. (Bug #94182, Bug #29308037)

On Behalf of Oracle/MySQL Release Engineering Team,
Nawaz Nazeer Ahamed