Java switch from MS SQL to MySQL - Character sets

Question

I am developing an application connecting to a database in Java. The customer has SQL Server and I tried the SQLExpress version von Microsoft as long as we don't want to buy a licence for the developement time. Sadly, the SQL Server Express does not allow network access so we can't work over network on the same database and have to install the SQL Server Express server on every developer client.

Today morning I decided to switch to MySQL during the developement process. I created a MySQL database with UTF8 charset and exported the data into CSV files which I also converted to UTF8 and imported them.

I connect with Java with the JDBC driver and now got weird behavior during execution. The results completetly differ from the client connecting to SQL Server. I have written the SQL to Java glue code myselft and am not using a framework like Hibernate or JPA.

I guess that the problem has to do with character encoding. The source code files are - dont't hit me - encoded with CP1252, because I just started developing in Eclipse on Windows and trusted the default settings. I query the database getting a ResultSet object and then read out the data with the getter methods provided by the ResultSet. I don't do any character conversion during the data fetching.

The problem is now that I don't get cryptic not well encoded output but instead NullPointerExceptions and weird data handling. For example: I have written a method which compares Jobs (an object representing a Job with a name, id, cargo and all that stuff). The results during the runtime differ. Some Jobs are equal on SQL Server and some on MySQL but the SQL Server result is the correct one.

I just viewed the database properties and saw that the character encoding of the SQL Server is in ISO-Latin-1.

Thanks for your help and regards from Cologne, Marco

Answer 1

I know little about MS SQL, but if the MS SQL data is latin1 encoded, importing it into a UTF-8 database on the mySQL end must fail and result in broken data.

I would try to get the data fixed first. Can you retry the process without converting the dump file, and a latin1 database in mySQL?

Plus, there seem to be notable differences in data types between mySQL and MS SQL:

Be careful when planning this: you want to match data types by capacity and not necessarily by name. For example: a MySQL VARCHAR can hold up to 255 characters, whereas a SQL Server VARCHAR can hold up to 4000 characters. In this case you would need to use a MySQL TEXT column type instead of VARCHAR.