UnicodeEncodeError occurs when attempting to migrate MSSQL data to MariaDB with MySQL Workbench

Question

I'm trying to move data from SQL Server to MariaDB using MySQL Workbench.

I installed it as 8.0.27 version at first, and after the problem occurred, I downgraded it to 8.0.26 version, but the problem was not solved.

I'm confused whether it's a problem with the data you're trying to move or something that's happening in MySQL Workbench.

I tried googling, but there was a similar phenomenon, but I couldn't find an answer.

https://bugs.mysql.com/bug.php?id=105772

Can you give me some advice? thanks you.

------------------------------------------------------------------ Starting... Create Script File...

Traceback (most recent call last): File "C:\\Program Files\\MySQL\\MySQL Workbench 8.0 CE\\workbench\\wizard_progress_page_widget.py", line 71, in run self.func() File "C:\\Program Files\\MySQL\\MySQL Workbench 8.0 CE\\modules\\migration_schema_creation.py", line 188, in _create_script_task self.main.plan.createTargetScript(self.main.plan.state.objectCreationParams["CreateSQLFile"]) File "C:\\Program Files\\MySQL\\MySQL Workbench 8.0 CE\\modules\\migration.py", line 527, in createTargetScript grt.modules.DbMySQLFE.createScriptForCatalogObjects(path, self.migrationTarget.catalog, self.state.objectCreationParams) SystemError: UnicodeEncodeError("'charmap' codec can't encode characters in position 99-104: character maps to "): error calling Python module function DbMySQLFE.createScriptForCatalogObjects ERROR: Error during Create Script File: UnicodeEncodeError("'charmap' codec can't encode characters in position 99-104: character maps to "): error calling Python module function DbMySQLFE.createScriptForCatalogObjects ERROR: Exception in task 'Create Script File': SystemError('UnicodeEncodeError("'charmap' codec can't encode characters in position 99-104: character maps to "): error calling Python module function DbMySQLFE.createScriptForCatalogObjects')

Failed

enter image description here

Answer 1

If you are seeing \� , that is the REPLACEMENT CHARACTER - -

It means that the data was already corrupted beyond repair. Go back a step in the process of the data to see what first led to it.

See also "black diamond" in Trouble with UTF-8 characters; what I see is not what I stored to find some ways that it can occur purely in MySQL. However, it sounds like SQL Server created it; I think the problem is back there.

Fetch the hex for any characters involved, noting that \￝ (UTF-8 hex: EFBFBD ) is after things are already corrupted.

In UTF-8 (MySQL's utf8 or utf8mb4), Hex EAB0zz through ED9Ezz is used for Hangul (Korean). For example, 박상우 is hex EBB095 EC8381 EC9AB0 (minus the spaces).

Answer 2

I cannot put the production db here. I just would like it to be possible to filter out the bad chars while importing