Possible MySQL server UUID duplication for server

    在mysql enterprise monitor监控过程中出现这样的event事件，Topic: Possible MySQL server UUID duplication for server 事件，从该提示的描述来看貌似是存在重复的uuid，而实际上主从关系并不存在重复的uuid。主从关系是通过xtrabackup来构建的。那到底是哪里的问题呢？下文是描述基于xtrabackup复制时导致监控出现重复uuid的问题。

1、故障现象
Topic: Possible MySQL server UUID duplication for server afd6bca4-6636-11e3-9d60-74867ae1c47c: NOTICE
Categories:Monitoring and Support Services
Current State:Open
Auto-Closes by Default:Yes
Advisor:Duplicate MySQL Server UUID
Current Status:Notice
Last Checked:May 4, 2015 2:18:02 PM

###提示uuid改变了N多次，在MySQL Instances监控见面会出现这2个主机配置在不停的切换中
MySQL server afd6bca4-6636-11e3-9d60-74867ae1c47c changed its hostname 181 times between the following hostnames:
DBSRV-TXT01
DBSRV-SLAVE02

MySQL server afd6bca4-6636-11e3-9d60-74867ae1c47c changed its connection TCP properties 96 times between the following TCP properties:

127.0.0.1:3306
127.0.0.1:3307

Advice
Check that you are not monitoring more than one instance with the following server UUID: afd6bca4-6636-11e3-9d60-74867ae1c47c.
This can be caused by server or host cloning. If this is expected (example: HA scenarios), then please ignore this notice.

2、校验uuid
###经google文章描述，mysql.inventory保存了被mysql enterpirse moitor监控用到的uuid信息
###查看主库上的uuid及mysql.inventory表
[root@DBSRV-TXT01 ~]# mysql -uroot -p -e "
show variables like 'server_uuid';
select * from mysql.inventory;" -S /tmp/mysql3307.sock
Enter password:
+---------------+--------------------------------------+
| Variable_name | Value                                |
+---------------+--------------------------------------+
| server_uuid   | 1ed85852-dd27-11e4-aa4a-44a8420ba7a5 |
+---------------+--------------------------------------+
+--------+-------------------------------------------------------+
| name   | value                                                 |
+--------+-------------------------------------------------------+
| uuid   | afd6bca4-6636-11e3-9d60-74867ae1c47c                  |
| hostid | ssh:{8a:c7:a9:42:3a:6b:06:ad:fa:ed:04:ac:a5:fa:f0:b5} |
+--------+-------------------------------------------------------+

###查看从库上的uuid及mysql.inventory表
[root@DBSRV-SLAVE02 ~]# mysql -uroot -p -e "    ---Author : Leshami
> show variables like 'server_uuid';            ---Blog   : http://blog.csdn.net/leshami
> select * from mysql.inventory;"
Enter password:
+---------------+--------------------------------------+
| Variable_name | Value                                |
+---------------+--------------------------------------+
| server_uuid   | f7e00194-2f59-11e4-bcf6-b82a72d46b21 |
+---------------+--------------------------------------+
+--------+-------------------------------------------------------+
| name   | value                                                 |
+--------+-------------------------------------------------------+
| uuid   | afd6bca4-6636-11e3-9d60-74867ae1c47c                   |
| hostid | ssh:{8a:c7:a9:42:3a:6b:06:ad:fa:ed:04:ac:a5:fa:f0:b5} |
+--------+-------------------------------------------------------+

###从上面的查询结果得知，mysql.inventory表里边确实保存了相同的uuid
###这个相同的uuid是由于使用了实例级别的热备，所以2个实例具有相同的uuid

###清空mysql.inventory，然后重启监控agent(略)，问题解决
[root@DBSRV-SLAVE02 ~]# mysql -uroot -p -e "truncate table mysql.inventory" -S /tmp/mysql3307.sock
Enter password:

3、关于MySQL MEM UUID duplication

MySQL Enterprise Monitor uses a number of unique values known as UUIDs to identify the different components, including the MySQL instance being monitored. UUID values related to the MySQL instance and the host on which it runs are stored in a table mysql.inventory within the instance. MySQL Enterprise Monitor creates this table if it does not exist already.

These UUIDs are used in combination to register and collate information, and to determine the location and source of an issue.

Because each host must be unique, be careful when restoring from a backup so you do not have hosts with duplicated SSH keys or UUIDs.

http://dev.mysql.com/doc/mysql-monitor/2.3/en/mem-introduction-mysql-server.html

The MySQL Enterprise Monitor Agent and MySQL Enterprise Service Manager use the unique host ID, stored within the mysql.inventory table on the monitored MySQL Server, to determine whether the instance being monitored is a clone. The host ID of the current server is checked against the stored value when the agent starts. If the generated host ID and stored host ID do not match, you get an error similar to the following in the agent log file:

%s: [%s] the hostid from mysql.inventory doesn't match our agent's host-id (%s != %s)
We assume that this is a cloned host and shutdown now.
Please TRUNCATE TABLE mysql.inventory on this mysql-instance and restart the agent.
If this is a master for replication, please also run SET SQL_LOG_BIN = 0; first.

To fix the problem, connect to the MySQL server using the credentials configured when you installed the agent, and then truncate the mysql.inventory table:

mysql> TRUNCATE mysql.inventory;

Now restart the agent, which recreates the mysql.inventory table with the updated instance UUID and hostid information.

http://dev.mysql.com/doc/mysql-monitor/2.3/en/mem-troubleshooting-agent-start.html